2026-02-25T11:34:27.487 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-25T11:34:27.505 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-25T11:34:27.587 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481 branch: wip-vshankar-testing-20260224.100235 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/fs/test_o_trunc}} email: vshankar@redhat.com first_in_suite: false flavor: default job_id: '69481' kernel: sha1: 712f4a43c1258070d29a784bd5f0c212504b234b last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: trial name: vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20260224.100235 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd - disabled max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: client_fscrypt_dummy_encryption: true mds: mds_bal_replicate_threshold: 1 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - is offline because no MDS - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - BLUESTORE_SLOW_OP_ALERT - slow operation indications in BlueStore - experiencing slow operations in BlueStore - MGR_MODULE_ERROR - OSD_DOWN - osd.* is down - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - osds down - osd.* is down - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - pg .* is stuck inactive - pg .* is .*degraded - slow metadata IO - SLOW_OPS - slow request sha1: 92715cd3c42198664811edeb975bd585312154aa subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 92715cd3c42198664811edeb975bd585312154aa kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20260225.073815 sha1: 7c9c9f709b02b16b3e2005e450ecf519f2bb54de owner: scheduled_vshankar@soko04.front.sepia.ceph.com priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3412 sha1: 92715cd3c42198664811edeb975bd585312154aa sleep_before_teardown: 0 subset: 813/1024 suite: fs suite_branch: wip-vshankar-testing-20260225.073815 suite_path: /home/teuthworker/src/github.com_vshankar_ceph_7c9c9f709b02b16b3e2005e450ecf519f2bb54de/qa suite_relpath: qa suite_repo: http://github.com/vshankar/ceph.git suite_sha1: 7c9c9f709b02b16b3e2005e450ecf519f2bb54de syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: trial028.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/DEY/qvKVPngUu8miP/2MgeJe8x3ujbk4sIKKsdlJxvP1SCSTXKNtCvfD2qJ4f/Xc18ytNJG7jqQAtfAEmHC4= trial119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI2zXI8yiq5UKKOdJDuOzGbKZjzPnJDehNWQ3RolQSF9/1aPvwiwUmxXijfXoWRrB7N/Sn9F6d4AzOD0ZtYk6FE= trial167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDV0Qc4bNQXK48u6ex/tZjNzNyuwo/rwtgKKJwUUi8i1gDaI+NgwpY6036ctTdiHPUt+52V2fDI7i/YOLRF5hzA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - "local kernel = py_attrgetter(yaml).get('kernel')\nif kernel ~= nil then\n local\ \ branch = py_attrgetter(kernel).get('branch')\n if branch and not kernel.branch:find\ \ \"-all$\" then\n log.debug(\"removing default kernel specification: %s\"\ , kernel)\n py_attrgetter(kernel).pop('branch', nil)\n py_attrgetter(kernel).pop('deb',\ \ nil)\n py_attrgetter(kernel).pop('flavor', nil)\n py_attrgetter(kernel).pop('kdb',\ \ nil)\n py_attrgetter(kernel).pop('koji', nil)\n py_attrgetter(kernel).pop('koji_task',\ \ nil)\n py_attrgetter(kernel).pop('rpm', nil)\n py_attrgetter(kernel).pop('sha1',\ \ nil)\n py_attrgetter(kernel).pop('tag', nil)\n end\nend\n" - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-25_09:28:04 tube: trial user: vshankar verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-25T11:34:27.587 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_vshankar_ceph_7c9c9f709b02b16b3e2005e450ecf519f2bb54de/qa; will attempt to use it 2026-02-25T11:34:27.587 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_vshankar_ceph_7c9c9f709b02b16b3e2005e450ecf519f2bb54de/qa/tasks 2026-02-25T11:34:27.587 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-25T11:34:27.589 INFO:teuthology.task.internal:Checking packages... 2026-02-25T11:34:27.589 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '92715cd3c42198664811edeb975bd585312154aa' 2026-02-25T11:34:27.589 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-25T11:34:27.589 INFO:teuthology.packaging:ref: None 2026-02-25T11:34:27.589 INFO:teuthology.packaging:tag: None 2026-02-25T11:34:27.589 INFO:teuthology.packaging:branch: wip-vshankar-testing-20260224.100235 2026-02-25T11:34:27.589 INFO:teuthology.packaging:sha1: 92715cd3c42198664811edeb975bd585312154aa 2026-02-25T11:34:27.589 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20260224.100235 2026-02-25T11:34:27.708 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5549.g92715cd3 2026-02-25T11:34:27.709 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-25T11:34:27.720 INFO:teuthology.task.internal:no buildpackages task found 2026-02-25T11:34:27.720 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-25T11:34:27.732 INFO:teuthology.task.internal:Saving configuration 2026-02-25T11:34:27.748 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-25T11:34:27.760 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-25T11:34:27.848 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial028.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 11:30:58.505366', 'locked_by': 'scheduled_vshankar@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE/DEY/qvKVPngUu8miP/2MgeJe8x3ujbk4sIKKsdlJxvP1SCSTXKNtCvfD2qJ4f/Xc18ytNJG7jqQAtfAEmHC4='} 2026-02-25T11:34:27.916 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial119.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 11:30:58.504683', 'locked_by': 'scheduled_vshankar@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI2zXI8yiq5UKKOdJDuOzGbKZjzPnJDehNWQ3RolQSF9/1aPvwiwUmxXijfXoWRrB7N/Sn9F6d4AzOD0ZtYk6FE='} 2026-02-25T11:34:27.972 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial167.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 11:30:58.505804', 'locked_by': 'scheduled_vshankar@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDV0Qc4bNQXK48u6ex/tZjNzNyuwo/rwtgKKJwUUi8i1gDaI+NgwpY6036ctTdiHPUt+52V2fDI7i/YOLRF5hzA='} 2026-02-25T11:34:27.972 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-25T11:34:27.984 INFO:teuthology.task.internal:roles: ubuntu@trial028.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2026-02-25T11:34:27.984 INFO:teuthology.task.internal:roles: ubuntu@trial119.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2026-02-25T11:34:27.984 INFO:teuthology.task.internal:roles: ubuntu@trial167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2026-02-25T11:34:27.984 INFO:teuthology.run_tasks:Running task console_log... 2026-02-25T11:34:28.171 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7d37ba03b920>, signals=[15]) 2026-02-25T11:34:28.171 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-25T11:34:28.182 INFO:teuthology.task.internal:Opening connections... 2026-02-25T11:34:28.182 DEBUG:teuthology.task.internal:connecting to ubuntu@trial028.front.sepia.ceph.com 2026-02-25T11:34:28.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T11:34:28.244 DEBUG:teuthology.task.internal:connecting to ubuntu@trial119.front.sepia.ceph.com 2026-02-25T11:34:28.244 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T11:34:28.305 DEBUG:teuthology.task.internal:connecting to ubuntu@trial167.front.sepia.ceph.com 2026-02-25T11:34:28.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T11:34:28.366 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-25T11:34:28.378 DEBUG:teuthology.orchestra.run.trial028:> uname -m 2026-02-25T11:34:28.393 INFO:teuthology.orchestra.run.trial028.stdout:x86_64 2026-02-25T11:34:28.393 DEBUG:teuthology.orchestra.run.trial028:> cat /etc/os-release 2026-02-25T11:34:28.449 INFO:teuthology.orchestra.run.trial028.stdout:NAME="CentOS Stream" 2026-02-25T11:34:28.449 INFO:teuthology.orchestra.run.trial028.stdout:VERSION="9" 2026-02-25T11:34:28.449 INFO:teuthology.orchestra.run.trial028.stdout:ID="centos" 2026-02-25T11:34:28.449 INFO:teuthology.orchestra.run.trial028.stdout:ID_LIKE="rhel fedora" 2026-02-25T11:34:28.449 INFO:teuthology.orchestra.run.trial028.stdout:VERSION_ID="9" 2026-02-25T11:34:28.449 INFO:teuthology.orchestra.run.trial028.stdout:PLATFORM_ID="platform:el9" 2026-02-25T11:34:28.449 INFO:teuthology.orchestra.run.trial028.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T11:34:28.449 INFO:teuthology.orchestra.run.trial028.stdout:ANSI_COLOR="0;31" 2026-02-25T11:34:28.450 INFO:teuthology.orchestra.run.trial028.stdout:LOGO="fedora-logo-icon" 2026-02-25T11:34:28.450 INFO:teuthology.orchestra.run.trial028.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T11:34:28.450 INFO:teuthology.orchestra.run.trial028.stdout:HOME_URL="https://centos.org/" 2026-02-25T11:34:28.450 INFO:teuthology.orchestra.run.trial028.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T11:34:28.450 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T11:34:28.450 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T11:34:28.450 INFO:teuthology.lock.ops:Updating trial028.front.sepia.ceph.com on lock server 2026-02-25T11:34:28.519 DEBUG:teuthology.orchestra.run.trial119:> uname -m 2026-02-25T11:34:28.534 INFO:teuthology.orchestra.run.trial119.stdout:x86_64 2026-02-25T11:34:28.535 DEBUG:teuthology.orchestra.run.trial119:> cat /etc/os-release 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:NAME="CentOS Stream" 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:VERSION="9" 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:ID="centos" 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:ID_LIKE="rhel fedora" 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:VERSION_ID="9" 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:PLATFORM_ID="platform:el9" 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:ANSI_COLOR="0;31" 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:LOGO="fedora-logo-icon" 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:HOME_URL="https://centos.org/" 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T11:34:28.596 INFO:teuthology.orchestra.run.trial119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T11:34:28.597 INFO:teuthology.lock.ops:Updating trial119.front.sepia.ceph.com on lock server 2026-02-25T11:34:28.680 DEBUG:teuthology.orchestra.run.trial167:> uname -m 2026-02-25T11:34:28.695 INFO:teuthology.orchestra.run.trial167.stdout:x86_64 2026-02-25T11:34:28.695 DEBUG:teuthology.orchestra.run.trial167:> cat /etc/os-release 2026-02-25T11:34:28.751 INFO:teuthology.orchestra.run.trial167.stdout:NAME="CentOS Stream" 2026-02-25T11:34:28.751 INFO:teuthology.orchestra.run.trial167.stdout:VERSION="9" 2026-02-25T11:34:28.751 INFO:teuthology.orchestra.run.trial167.stdout:ID="centos" 2026-02-25T11:34:28.751 INFO:teuthology.orchestra.run.trial167.stdout:ID_LIKE="rhel fedora" 2026-02-25T11:34:28.751 INFO:teuthology.orchestra.run.trial167.stdout:VERSION_ID="9" 2026-02-25T11:34:28.751 INFO:teuthology.orchestra.run.trial167.stdout:PLATFORM_ID="platform:el9" 2026-02-25T11:34:28.751 INFO:teuthology.orchestra.run.trial167.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T11:34:28.751 INFO:teuthology.orchestra.run.trial167.stdout:ANSI_COLOR="0;31" 2026-02-25T11:34:28.751 INFO:teuthology.orchestra.run.trial167.stdout:LOGO="fedora-logo-icon" 2026-02-25T11:34:28.751 INFO:teuthology.orchestra.run.trial167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T11:34:28.751 INFO:teuthology.orchestra.run.trial167.stdout:HOME_URL="https://centos.org/" 2026-02-25T11:34:28.751 INFO:teuthology.orchestra.run.trial167.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T11:34:28.751 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T11:34:28.752 INFO:teuthology.orchestra.run.trial167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T11:34:28.752 INFO:teuthology.lock.ops:Updating trial167.front.sepia.ceph.com on lock server 2026-02-25T11:34:28.834 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-25T11:34:28.850 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-25T11:34:28.860 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-25T11:34:28.861 DEBUG:teuthology.orchestra.run.trial028:> test '!' -e /home/ubuntu/cephtest 2026-02-25T11:34:28.862 DEBUG:teuthology.orchestra.run.trial119:> test '!' -e /home/ubuntu/cephtest 2026-02-25T11:34:28.864 DEBUG:teuthology.orchestra.run.trial167:> test '!' -e /home/ubuntu/cephtest 2026-02-25T11:34:28.879 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-25T11:34:28.891 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-25T11:34:28.891 DEBUG:teuthology.orchestra.run.trial028:> test -z $(ls -A /var/lib/ceph) 2026-02-25T11:34:28.918 DEBUG:teuthology.orchestra.run.trial119:> test -z $(ls -A /var/lib/ceph) 2026-02-25T11:34:28.920 DEBUG:teuthology.orchestra.run.trial167:> test -z $(ls -A /var/lib/ceph) 2026-02-25T11:34:28.937 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-25T11:34:29.157 INFO:teuthology.run_tasks:Running task kernel... 2026-02-25T11:34:29.176 INFO:teuthology.task.kernel:normalize config orig: {'sha1': '712f4a43c1258070d29a784bd5f0c212504b234b'} 2026-02-25T11:34:29.176 INFO:teuthology.task.kernel:config {'mon.a': {'sha1': '712f4a43c1258070d29a784bd5f0c212504b234b'}, 'mon.b': {'sha1': '712f4a43c1258070d29a784bd5f0c212504b234b'}, 'mon.c': {'sha1': '712f4a43c1258070d29a784bd5f0c212504b234b'}}, timeout 300 2026-02-25T11:34:29.176 DEBUG:teuthology.orchestra.run.trial028:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T11:34:29.176 DEBUG:teuthology.orchestra.run.trial119:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T11:34:29.176 DEBUG:teuthology.orchestra.run.trial167:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T11:34:29.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T11:34:29.191 DEBUG:teuthology.task.kernel:sha1 for mon.b is 712f4a43c1258070d29a784bd5f0c212504b234b 2026-02-25T11:34:29.191 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "712f4a43c1258070d29a784bd5f0c212504b234b"... 2026-02-25T11:34:29.191 DEBUG:teuthology.orchestra.run.trial119:> uname -r 2026-02-25T11:34:29.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T11:34:29.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T11:34:29.192 DEBUG:teuthology.task.kernel:sha1 for mon.c is 712f4a43c1258070d29a784bd5f0c212504b234b 2026-02-25T11:34:29.192 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "712f4a43c1258070d29a784bd5f0c212504b234b"... 2026-02-25T11:34:29.192 DEBUG:teuthology.orchestra.run.trial167:> uname -r 2026-02-25T11:34:29.192 DEBUG:teuthology.task.kernel:sha1 for mon.a is 712f4a43c1258070d29a784bd5f0c212504b234b 2026-02-25T11:34:29.192 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "712f4a43c1258070d29a784bd5f0c212504b234b"... 2026-02-25T11:34:29.192 DEBUG:teuthology.orchestra.run.trial028:> uname -r 2026-02-25T11:34:29.247 INFO:teuthology.orchestra.run.trial119.stdout:5.14.0-681.el9.x86_64 2026-02-25T11:34:29.247 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-681.el9.x86_64 vs 712f4a43c1258070d29a784bd5f0c212504b234b 2026-02-25T11:34:29.247 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2026-02-25T11:34:29.247 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=712f4a43c1258070d29a784bd5f0c212504b234b 2026-02-25T11:34:29.250 INFO:teuthology.orchestra.run.trial167.stdout:5.14.0-681.el9.x86_64 2026-02-25T11:34:29.250 INFO:teuthology.orchestra.run.trial028.stdout:5.14.0-681.el9.x86_64 2026-02-25T11:34:29.250 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-681.el9.x86_64 vs 712f4a43c1258070d29a784bd5f0c212504b234b 2026-02-25T11:34:29.250 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2026-02-25T11:34:29.250 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=712f4a43c1258070d29a784bd5f0c212504b234b 2026-02-25T11:34:29.251 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-681.el9.x86_64 vs 712f4a43c1258070d29a784bd5f0c212504b234b 2026-02-25T11:34:29.251 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2026-02-25T11:34:29.251 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=712f4a43c1258070d29a784bd5f0c212504b234b 2026-02-25T11:34:29.449 DEBUG:teuthology.orchestra.run.trial119:> sudo yum upgrade -y linux-firmware 2026-02-25T11:34:29.450 DEBUG:teuthology.orchestra.run.trial028:> sudo yum upgrade -y linux-firmware 2026-02-25T11:34:29.458 DEBUG:teuthology.orchestra.run.trial167:> sudo yum upgrade -y linux-firmware 2026-02-25T11:34:30.031 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - BaseOS 37 MB/s | 8.9 MB 00:00 2026-02-25T11:34:30.560 INFO:teuthology.orchestra.run.trial167.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.9 MB 00:00 2026-02-25T11:34:32.660 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - AppStream 13 MB/s | 27 MB 00:02 2026-02-25T11:34:33.721 INFO:teuthology.orchestra.run.trial167.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-25T11:34:36.340 INFO:teuthology.orchestra.run.trial119.stdout:CentOS Stream 9 - BaseOS 1.4 MB/s | 8.9 MB 00:06 2026-02-25T11:34:36.937 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 8.0 MB 00:01 2026-02-25T11:34:37.051 INFO:teuthology.orchestra.run.trial167.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 8.0 MB 00:00 2026-02-25T11:34:37.865 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - Extras packages 110 kB/s | 20 kB 00:00 2026-02-25T11:34:38.083 INFO:teuthology.orchestra.run.trial167.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 20 kB 00:00 2026-02-25T11:34:38.587 INFO:teuthology.orchestra.run.trial028.stdout:Extra Packages for Enterprise Linux 31 MB/s | 20 MB 00:00 2026-02-25T11:34:38.855 INFO:teuthology.orchestra.run.trial167.stdout:Extra Packages for Enterprise Linux 29 MB/s | 20 MB 00:00 2026-02-25T11:34:39.420 INFO:teuthology.orchestra.run.trial119.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-25T11:34:41.834 INFO:teuthology.orchestra.run.trial028.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-25T11:34:42.062 INFO:teuthology.orchestra.run.trial167.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-25T11:34:42.830 INFO:teuthology.orchestra.run.trial119.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 8.0 MB 00:01 2026-02-25T11:34:42.860 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-25T11:34:42.860 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-25T11:34:42.860 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-25T11:34:42.884 INFO:teuthology.task.kernel:Downloading kernel 712f4a43c1258070d29a784bd5f0c212504b234b on mon.a... 2026-02-25T11:34:42.884 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=712f4a43c1258070d29a784bd5f0c212504b234b 2026-02-25T11:34:43.056 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/712f4a43c1258070d29a784bd5f0c212504b234b/centos/9/flavors/default/x86_64/ 2026-02-25T11:34:43.056 DEBUG:teuthology.orchestra.run.trial028:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.19.0_g712f4a43c125-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/712f4a43c1258070d29a784bd5f0c212504b234b/centos/9/flavors/default/x86_64/ --input-file=- 2026-02-25T11:34:43.058 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@trial028.front.sepia.ceph.com to complete... 2026-02-25T11:34:43.091 INFO:teuthology.orchestra.run.trial167.stdout:Dependencies resolved. 2026-02-25T11:34:43.092 INFO:teuthology.orchestra.run.trial167.stdout:Nothing to do. 2026-02-25T11:34:43.092 INFO:teuthology.orchestra.run.trial167.stdout:Complete! 2026-02-25T11:34:43.115 INFO:teuthology.task.kernel:Downloading kernel 712f4a43c1258070d29a784bd5f0c212504b234b on mon.c... 2026-02-25T11:34:43.115 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=712f4a43c1258070d29a784bd5f0c212504b234b 2026-02-25T11:34:43.280 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/712f4a43c1258070d29a784bd5f0c212504b234b/centos/9/flavors/default/x86_64/ 2026-02-25T11:34:43.280 DEBUG:teuthology.orchestra.run.trial167:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.19.0_g712f4a43c125-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/712f4a43c1258070d29a784bd5f0c212504b234b/centos/9/flavors/default/x86_64/ --input-file=- 2026-02-25T11:34:43.282 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@trial167.front.sepia.ceph.com to complete... 2026-02-25T11:34:43.696 INFO:teuthology.orchestra.run.trial119.stdout:CentOS Stream 9 - Extras packages 170 kB/s | 20 kB 00:00 2026-02-25T11:34:45.718 INFO:teuthology.orchestra.run.trial119.stdout:Extra Packages for Enterprise Linux 10 MB/s | 20 MB 00:01 2026-02-25T11:34:48.942 INFO:teuthology.orchestra.run.trial119.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-25T11:34:49.981 INFO:teuthology.orchestra.run.trial119.stdout:Dependencies resolved. 2026-02-25T11:34:49.982 INFO:teuthology.orchestra.run.trial119.stdout:Nothing to do. 2026-02-25T11:34:49.982 INFO:teuthology.orchestra.run.trial119.stdout:Complete! 2026-02-25T11:34:50.004 INFO:teuthology.task.kernel:Downloading kernel 712f4a43c1258070d29a784bd5f0c212504b234b on mon.b... 2026-02-25T11:34:50.004 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=712f4a43c1258070d29a784bd5f0c212504b234b 2026-02-25T11:34:50.167 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/712f4a43c1258070d29a784bd5f0c212504b234b/centos/9/flavors/default/x86_64/ 2026-02-25T11:34:50.167 DEBUG:teuthology.orchestra.run.trial119:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.19.0_g712f4a43c125-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/712f4a43c1258070d29a784bd5f0c212504b234b/centos/9/flavors/default/x86_64/ --input-file=- 2026-02-25T11:34:50.169 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@trial119.front.sepia.ceph.com to complete... 2026-02-25T11:35:05.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25 11:35:05 URL:https://1.chacra.ceph.com/r/kernel/testing/712f4a43c1258070d29a784bd5f0c212504b234b/centos/9/flavors/default/x86_64/kernel-6.19.0_g712f4a43c125-1.x86_64.rpm [388118810/388118810] -> "/tmp/kernel.x86_64.rpm" [1] 2026-02-25T11:35:05.055 INFO:teuthology.orchestra.run.trial028.stderr:FINISHED --2026-02-25 11:35:05-- 2026-02-25T11:35:05.055 INFO:teuthology.orchestra.run.trial028.stderr:Total wall clock time: 22s 2026-02-25T11:35:05.055 INFO:teuthology.orchestra.run.trial028.stderr:Downloaded: 1 files, 370M in 22s (17.1 MB/s) 2026-02-25T11:35:05.060 INFO:teuthology.task.kernel:Installing kernel 712f4a43c1258070d29a784bd5f0c212504b234b on mon.a... 2026-02-25T11:35:05.060 DEBUG:teuthology.orchestra.run.trial028:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2026-02-25T11:35:06.188 INFO:teuthology.orchestra.run.trial028.stdout:Verifying... ######################################## 2026-02-25T11:35:06.239 INFO:teuthology.orchestra.run.trial028.stdout:Preparing... ######################################## 2026-02-25T11:35:06.256 INFO:teuthology.orchestra.run.trial028.stdout:Updating / installing... 2026-02-25T11:35:07.118 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25 11:35:07 URL:https://1.chacra.ceph.com/r/kernel/testing/712f4a43c1258070d29a784bd5f0c212504b234b/centos/9/flavors/default/x86_64/kernel-6.19.0_g712f4a43c125-1.x86_64.rpm [388118810/388118810] -> "/tmp/kernel.x86_64.rpm" [1] 2026-02-25T11:35:07.119 INFO:teuthology.orchestra.run.trial119.stderr:FINISHED --2026-02-25 11:35:07-- 2026-02-25T11:35:07.119 INFO:teuthology.orchestra.run.trial119.stderr:Total wall clock time: 17s 2026-02-25T11:35:07.119 INFO:teuthology.orchestra.run.trial119.stderr:Downloaded: 1 files, 370M in 17s (22.4 MB/s) 2026-02-25T11:35:07.123 INFO:teuthology.task.kernel:Installing kernel 712f4a43c1258070d29a784bd5f0c212504b234b on mon.b... 2026-02-25T11:35:07.123 DEBUG:teuthology.orchestra.run.trial119:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2026-02-25T11:35:08.251 INFO:teuthology.orchestra.run.trial119.stdout:Verifying... ######################################## 2026-02-25T11:35:08.302 INFO:teuthology.orchestra.run.trial119.stdout:Preparing... ######################################## 2026-02-25T11:35:08.320 INFO:teuthology.orchestra.run.trial119.stdout:Updating / installing... 2026-02-25T11:35:09.429 INFO:teuthology.orchestra.run.trial028.stdout:kernel-6.19.0_g712f4a43c125-1 ######################################## 2026-02-25T11:35:11.531 INFO:teuthology.orchestra.run.trial119.stdout:kernel-6.19.0_g712f4a43c125-1 ######################################## 2026-02-25T11:35:11.868 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25 11:35:11 URL:https://1.chacra.ceph.com/r/kernel/testing/712f4a43c1258070d29a784bd5f0c212504b234b/centos/9/flavors/default/x86_64/kernel-6.19.0_g712f4a43c125-1.x86_64.rpm [388118810/388118810] -> "/tmp/kernel.x86_64.rpm" [1] 2026-02-25T11:35:11.868 INFO:teuthology.orchestra.run.trial167.stderr:FINISHED --2026-02-25 11:35:11-- 2026-02-25T11:35:11.868 INFO:teuthology.orchestra.run.trial167.stderr:Total wall clock time: 29s 2026-02-25T11:35:11.868 INFO:teuthology.orchestra.run.trial167.stderr:Downloaded: 1 files, 370M in 28s (13.1 MB/s) 2026-02-25T11:35:11.872 INFO:teuthology.task.kernel:Installing kernel 712f4a43c1258070d29a784bd5f0c212504b234b on mon.c... 2026-02-25T11:35:11.873 DEBUG:teuthology.orchestra.run.trial167:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2026-02-25T11:35:12.999 INFO:teuthology.orchestra.run.trial167.stdout:Verifying... ######################################## 2026-02-25T11:35:13.049 INFO:teuthology.orchestra.run.trial167.stdout:Preparing... ######################################## 2026-02-25T11:35:13.066 INFO:teuthology.orchestra.run.trial167.stdout:Updating / installing... 2026-02-25T11:35:16.229 INFO:teuthology.orchestra.run.trial167.stdout:kernel-6.19.0_g712f4a43c125-1 ######################################## 2026-02-25T11:35:26.419 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial028.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2026-02-25T11:35:26.419 DEBUG:teuthology.orchestra.run.trial028:> rpm -qlp /tmp/kernel.x86_64.rpm 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/boot/System.map-6.19.0-g712f4a43c125 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/boot/config-6.19.0-g712f4a43c125 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/boot/vmlinuz-6.19.0-g712f4a43c125 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/System.map 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/config 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/blowfish-x86_64.ko 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/camellia-x86_64.ko 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2026-02-25T11:35:26.450 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/serpent-avx2.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/twofish-x86_64.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kernel 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kernel/cpu 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kernel/cpu/mce 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kvm 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kvm/kvm-amd.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kvm/kvm-intel.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kvm/kvm.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/anubis.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/arc4.ko 2026-02-25T11:35:26.451 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx 2026-02-25T11:35:26.452 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_memcpy.ko 2026-02-25T11:35:26.452 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_pq.ko 2026-02-25T11:35:26.452 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_raid6_recov.ko 2026-02-25T11:35:26.452 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_tx.ko 2026-02-25T11:35:26.452 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_xor.ko 2026-02-25T11:35:26.452 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/raid6test.ko 2026-02-25T11:35:26.452 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/authenc.ko 2026-02-25T11:35:26.453 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/authencesn.ko 2026-02-25T11:35:26.453 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/blake2b.ko 2026-02-25T11:35:26.453 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/blowfish_common.ko 2026-02-25T11:35:26.453 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/blowfish_generic.ko 2026-02-25T11:35:26.453 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/camellia_generic.ko 2026-02-25T11:35:26.453 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/cast5_generic.ko 2026-02-25T11:35:26.454 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/cast6_generic.ko 2026-02-25T11:35:26.454 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/cast_common.ko 2026-02-25T11:35:26.454 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/ccm.ko 2026-02-25T11:35:26.454 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/cmac.ko 2026-02-25T11:35:26.454 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/crc32-cryptoapi.ko 2026-02-25T11:35:26.454 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/crypto_engine.ko 2026-02-25T11:35:26.454 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/crypto_user.ko 2026-02-25T11:35:26.455 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/des_generic.ko 2026-02-25T11:35:26.455 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/dh_generic.ko 2026-02-25T11:35:26.455 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/echainiv.ko 2026-02-25T11:35:26.455 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/essiv.ko 2026-02-25T11:35:26.455 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/fcrypt.ko 2026-02-25T11:35:26.455 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/khazad.ko 2026-02-25T11:35:26.455 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/krb5 2026-02-25T11:35:26.455 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/krb5/krb5.ko 2026-02-25T11:35:26.455 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/krb5enc.ko 2026-02-25T11:35:26.455 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/lz4.ko 2026-02-25T11:35:26.455 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/lz4hc.ko 2026-02-25T11:35:26.456 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/md4.ko 2026-02-25T11:35:26.456 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/michael_mic.ko 2026-02-25T11:35:26.456 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/pcbc.ko 2026-02-25T11:35:26.456 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/pcrypt.ko 2026-02-25T11:35:26.456 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/rmd160.ko 2026-02-25T11:35:26.456 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/seed.ko 2026-02-25T11:35:26.456 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/serpent_generic.ko 2026-02-25T11:35:26.456 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/tea.ko 2026-02-25T11:35:26.456 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/twofish_common.ko 2026-02-25T11:35:26.456 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/twofish_generic.ko 2026-02-25T11:35:26.457 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/wp512.ko 2026-02-25T11:35:26.457 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/xcbc.ko 2026-02-25T11:35:26.457 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/xor.ko 2026-02-25T11:35:26.457 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/xxhash_generic.ko 2026-02-25T11:35:26.457 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers 2026-02-25T11:35:26.457 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi 2026-02-25T11:35:26.457 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/acpi_ipmi.ko 2026-02-25T11:35:26.458 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/acpi_pad.ko 2026-02-25T11:35:26.458 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/ec_sys.ko 2026-02-25T11:35:26.458 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/platform_profile.ko 2026-02-25T11:35:26.458 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/sbs.ko 2026-02-25T11:35:26.458 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/sbshc.ko 2026-02-25T11:35:26.458 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/video.ko 2026-02-25T11:35:26.458 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata 2026-02-25T11:35:26.458 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/acard-ahci.ko 2026-02-25T11:35:26.458 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/ahci_platform.ko 2026-02-25T11:35:26.458 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/ata_generic.ko 2026-02-25T11:35:26.458 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/libahci_platform.ko 2026-02-25T11:35:26.459 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_acpi.ko 2026-02-25T11:35:26.459 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_ali.ko 2026-02-25T11:35:26.459 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_amd.ko 2026-02-25T11:35:26.459 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_artop.ko 2026-02-25T11:35:26.459 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_atiixp.ko 2026-02-25T11:35:26.459 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_atp867x.ko 2026-02-25T11:35:26.459 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_cmd640.ko 2026-02-25T11:35:26.459 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_cmd64x.ko 2026-02-25T11:35:26.459 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_cypress.ko 2026-02-25T11:35:26.459 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_efar.ko 2026-02-25T11:35:26.459 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_hpt366.ko 2026-02-25T11:35:26.459 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_hpt37x.ko 2026-02-25T11:35:26.459 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_hpt3x2n.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_hpt3x3.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_it8213.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_it821x.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_jmicron.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_marvell.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_mpiix.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_netcell.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_ninja32.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_ns87410.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_ns87415.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_oldpiix.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_opti.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_optidma.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_pcmcia.ko 2026-02-25T11:35:26.460 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_pdc2027x.ko 2026-02-25T11:35:26.461 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_pdc202xx_old.ko 2026-02-25T11:35:26.461 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_piccolo.ko 2026-02-25T11:35:26.461 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_rdc.ko 2026-02-25T11:35:26.461 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_sch.ko 2026-02-25T11:35:26.461 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_serverworks.ko 2026-02-25T11:35:26.461 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_sil680.ko 2026-02-25T11:35:26.461 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_sis.ko 2026-02-25T11:35:26.461 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_sl82c105.ko 2026-02-25T11:35:26.461 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_triflex.ko 2026-02-25T11:35:26.461 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_via.ko 2026-02-25T11:35:26.461 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pdc_adma.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_inic162x.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_mv.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_nv.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_promise.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_qstor.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_sil.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_sil24.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_sis.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_svw.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_sx4.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_uli.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_via.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_vsc.ko 2026-02-25T11:35:26.462 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/auxdisplay 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/auxdisplay/ks0108.ko 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/base 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/base/regmap 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/base/regmap/regmap-i2c.ko 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/base/regmap/regmap-mmio.ko 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/bcma 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/bcma/bcma.ko 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/aoe 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/aoe/aoe.ko 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/brd.ko 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/drbd 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/drbd/drbd.ko 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/floppy.ko 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/loop.ko 2026-02-25T11:35:26.463 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/mtip32xx 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/mtip32xx/mtip32xx.ko 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/nbd.ko 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/null_blk 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/null_blk/null_blk.ko 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/rbd.ko 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/virtio_blk.ko 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/xen-blkback 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/xen-blkback/xen-blkback.ko 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/xen-blkfront.ko 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/zram 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/zram/zram.ko 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hangcheck-timer.ko 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/amd-rng.ko 2026-02-25T11:35:26.464 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/intel-rng.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/timeriomem-rng.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/via-rng.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/virtio-rng.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_devintf.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_si.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_ssif.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/lp.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/mwave 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/mwave/mwave.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ppdev.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tlclk.ko 2026-02-25T11:35:26.465 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_atmel.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_infineon.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_nsc.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_tis.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_tis_core.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/uv_mmtimer.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/virtio_console.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/acpi-cpufreq.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/p4-clockmod.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/pcc-cpufreq.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/powernow-k8.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/speedstep-lib.ko 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/ccp 2026-02-25T11:35:26.466 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/ccp/ccp-crypto.ko 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/ccp/ccp.ko 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat/qat_common 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/padlock-aes.ko 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/padlock-sha.ko 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/virtio 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/virtio/virtio_crypto.ko 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dca 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dca/dca.ko 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/devfreq 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/devfreq/governor_simpleondemand.ko 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma/dw 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma/dw/dw_dmac.ko 2026-02-25T11:35:26.467 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma/ioat 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma/ioat/ioatdma.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/amd64_edac.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/e752x_edac.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/edac_mce_amd.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i3000_edac.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i3200_edac.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i5100_edac.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i5400_edac.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i7300_edac.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i7core_edac.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i82975x_edac.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/ie31200_edac.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/sb_edac.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/x38_edac.ko 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/firmware 2026-02-25T11:35:26.468 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/firmware/edd.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/firmware/iscsi_ibft.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/gpio 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/gpio/gpio-ich.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/gpio/gpio-viperboard.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-appleir.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-aureal.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-axff.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-dr.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-elecom.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-elo.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-emsff.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-gaff.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-gt683r.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-gyration.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-holtek-kbd.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-holtek-mouse.ko 2026-02-25T11:35:26.469 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-holtekff.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-icade.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-keytouch.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-kye.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-lcpower.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-led.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-lenovo.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-multitouch.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-ortek.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-penmount.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-petalynx.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-picolcd.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-pl.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-plantronics.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-primax.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-rmi.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-arvo.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-common.ko 2026-02-25T11:35:26.470 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-isku.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-kone.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-koneplus.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-konepure.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-kovaplus.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-lua.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-pyra.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-ryos.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-savu.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-saitek.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-samsung.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-sensor-hub.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-sjoy.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-sony.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-speedlink.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-steelseries.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-sunplus.ko 2026-02-25T11:35:26.471 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-thrustmaster.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-tivo.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-tmff.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-topseed.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-twinhan.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-uclogic.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-waltop.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-wiimote.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-xinmo.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-zpff.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-zydacron.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/uhid.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/wacom.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/abituguru.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/abituguru3.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/acpi_power_meter.ko 2026-02-25T11:35:26.472 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ad7414.ko 2026-02-25T11:35:26.473 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ad7418.ko 2026-02-25T11:35:26.473 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adc128d818.ko 2026-02-25T11:35:26.473 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm1025.ko 2026-02-25T11:35:26.473 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm1026.ko 2026-02-25T11:35:26.473 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm1029.ko 2026-02-25T11:35:26.473 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm1031.ko 2026-02-25T11:35:26.473 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm9240.ko 2026-02-25T11:35:26.473 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ads7828.ko 2026-02-25T11:35:26.473 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7410.ko 2026-02-25T11:35:26.473 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7411.ko 2026-02-25T11:35:26.473 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7462.ko 2026-02-25T11:35:26.473 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7470.ko 2026-02-25T11:35:26.473 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7475.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7x10.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/amc6821.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/applesmc.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/asb100.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/asc7621.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/asus_atk0110.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/atxp1.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/coretemp.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/dell-smm-hwmon.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/dme1737.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ds1621.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ds620.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/emc1403.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/emc6w201.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/f71805f.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/f71882fg.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/f75375s.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/fam15h_power.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/fschmd.ko 2026-02-25T11:35:26.474 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/g760a.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/g762.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/gl518sm.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/gl520sm.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/hwmon-vid.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/i5500_temp.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/i5k_amb.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ibmaem.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ibmpex.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ina209.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ina2xx.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/it87.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/k10temp.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/k8temp.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lineage-pem.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm63.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm73.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm75.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm77.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm78.ko 2026-02-25T11:35:26.475 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm80.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm83.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm85.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm87.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm90.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm92.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm93.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm95234.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm95241.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm95245.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc2945.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4151.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4215.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4222.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4245.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4260.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4261.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max16065.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max1619.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max1668.ko 2026-02-25T11:35:26.476 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max197.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max6639.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max6650.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max6697.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/mcp3021.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/nct6683.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/nct6775-core.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/nct6775.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/nct7802.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pc87360.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pc87427.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pcf8591.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/adm1275.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/lm25066.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/ltc2978.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/max16064.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/max34440.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/max8688.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/pmbus.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/tps40422.ko 2026-02-25T11:35:26.477 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/ucd9000.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/ucd9200.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/zl6100.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/powr1220.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sch5627.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sch5636.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sch56xx-common.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sht15.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sht21.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/shtc1.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sis5595.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/smsc47b397.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/smsc47m1.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/smsc47m192.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/thmc50.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/tmp102.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/tmp103.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/tmp401.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/tmp421.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/via-cputemp.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/via686a.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/vt1211.ko 2026-02-25T11:35:26.478 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/vt8231.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83627ehf.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83627hf.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83781d.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83791d.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83792d.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83793.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83795.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83l785ts.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83l786ng.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/algos 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-amd756.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-amd8111.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-designware-core.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2026-02-25T11:35:26.479 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-i801.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-isch.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-ismt.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-nforce2.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-parport.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-piix4.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-scmi.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-simtec.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-sis96x.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-via.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-viapro.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-viperboard.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-dev.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-mux.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-slave-eeprom.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-smbus.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-stub.ko 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core 2026-02-25T11:35:26.480 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/ib_cm.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/ib_core.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/ib_umad.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/ib_uverbs.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/iw_cm.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/rdma_cm.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/rdma_ucm.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/cxgb4 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mlx4 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mlx5 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mthca 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/ipoib 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/iser 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/isert 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/srp 2026-02-25T11:35:26.481 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/srpt 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/gameport 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/gameport/emu10k1-gp.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/gameport/fm801-gp.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/gameport/gameport.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/joydev.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/keyboard 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/keyboard/gpio_keys.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/appletouch.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/bcm5974.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/cyapatp.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/elan_i2c.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/sermouse.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/synaptics_i2c.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/synaptics_usb.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/vsxxxaa.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/rmi4 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/rmi4/rmi_core.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/serio 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/serio/altera_ps2.ko 2026-02-25T11:35:26.482 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/serio/arc_ps2.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/serio/serio_raw.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/sparse-keymap.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/leds-blinkm.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/leds-lm3530.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/leds-lp3944.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/leds-ss4200.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-camera.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-timer.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-transient.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/bcache 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/bcache/bcache.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-bio-prison.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-cache-smq.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-cache.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-crypt.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-delay.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-flakey.ko 2026-02-25T11:35:26.483 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-log-userspace.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-multipath.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-queue-length.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-raid.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-round-robin.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-service-time.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-switch.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-thin-pool.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-verity.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/linear.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/persistent-data 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/raid0.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/raid1.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/raid10.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/raid456.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptbase.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptctl.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptfc.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptlan.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptsas.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptscsih.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptspi.ko 2026-02-25T11:35:26.484 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/lpc_ich.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/lpc_sch.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/mfd-core.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/sm501.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/viperboard.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/vx855.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/altera-stapl 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/apds9802als.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/apds990x.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/bh1770glc.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/cb710 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/cb710/cb710.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/eeprom 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/eeprom/at24.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/eeprom/max6875.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/enclosure.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/hpilo.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/isl29003.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/isl29020.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/lis3lv02d 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/mei 2026-02-25T11:35:26.485 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/mei/mei-me.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/mei/mei-txe.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/mei/mei.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-gru 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-gru/gru.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-xp 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-xp/xp.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-xp/xpc.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-xp/xpnet.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/tifm_7xx1.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/tifm_core.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/tsl2550.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/vmw_balloon.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/vmw_vmci 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/bonding 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/bonding/bonding.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/b53 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/b53/b53_common.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/bcm-sf2.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/mv88e6060.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/mv88e6xxx 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dummy.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/eql.ko 2026-02-25T11:35:26.486 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com/3c59x.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com/typhoon.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390/8390.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/adaptec 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/adaptec/starfire.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/agere 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/agere/et131x.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/alteon 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/alteon/acenic.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/altera 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/altera/altera_tse.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/amd 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/amd/amd8111e.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/amd/pcnet32.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/alx 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atl1c 2026-02-25T11:35:26.487 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atl1e 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atlx 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/b44.ko 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/bnx2x 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/cnic.ko 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/genet 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/tg3.ko 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/brocade 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/brocade/bna 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2026-02-25T11:35:26.488 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb3 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb4 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/libcxgb 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/cisco 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/cisco/enic 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dlink 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dlink/dl2k.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dlink/sundance.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dnet.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/emulex 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/emulex/benet 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2026-02-25T11:35:26.489 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/ethoc.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/fealnx.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e100.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e1000 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e1000e 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/fm10k 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/i40e 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/iavf 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ice 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ice/ice.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/igb 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/igb/igb.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/igbvf 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ixgbe 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ixgbevf 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libeth 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libeth/libeth_xdp.ko 2026-02-25T11:35:26.490 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libie 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libie/libie.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libie/libie_adminq.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libie/libie_fwlog.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/jme.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/marvell 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/marvell/skge.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/marvell/sky2.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx4 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx5 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx5/core 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/micrel 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/myricom 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/myricom/myri10ge 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/natsemi 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/neterion 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/neterion/s2io.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/nvidia 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/packetengines 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2026-02-25T11:35:26.491 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/netxen 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/qlcnic 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/rdc 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/rdc/r6040.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek/8139cp.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek/8139too.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek/atp.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek/r8169.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/rocker 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/rocker/rocker.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sfc 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sfc/sfc.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/silan 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/silan/sc92031.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sis 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sis/sis190.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sis/sis900.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc/epic100.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/stmicro 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/stmicro/stmmac 2026-02-25T11:35:26.492 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun/cassini.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun/niu.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun/sungem.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun/sunhme.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/tehuti 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/ti 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/ti/tlan.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/via 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/via/via-rhine.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/via/via-velocity.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/wiznet 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/wiznet/w5100.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/wiznet/w5300.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/xircom 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/geneve.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ieee802154 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ieee802154/fakelb.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ifb.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ipvlan 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ipvlan/ipvlan.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/macvlan.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/macvtap.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2026-02-25T11:35:26.493 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio/mdio-bitbang.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio/mdio-regmap.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mii.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/net_failover.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/netconsole.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/nlmon.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/pcs 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/pcs/pcs-lynx.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/pcs/pcs_xpcs.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/amd.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/bcm-phy-lib.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/bcm-phy-ptp.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/bcm7xxx.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/bcm87xx.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/broadcom.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/cicada.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/davicom.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/dp83640.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/et1011c.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/icplus.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/lxt.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/marvell.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/micrel.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/national.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/phy_package.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/phylink.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/qsemi.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/realtek 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/realtek/realtek.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/smsc.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/ste10Xp.ko 2026-02-25T11:35:26.494 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/vitesse.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/bsd_comp.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_async.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_deflate.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_generic.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_mppe.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_synctty.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/pppoe.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/pppox.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/pptp.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/slip 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/slip/slhc.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/slip/slip.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/sungem_phy.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/tap.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_activebackup.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_broadcast.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_loadbalance.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_random.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_roundrobin.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/tun.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/veth.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/virtio_net.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/vmxnet3 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/vmxnet3/vmxnet3.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/vxlan 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/vxlan/vxlan.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/xen-netback 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/xen-netback/xen-netback.ko 2026-02-25T11:35:26.495 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/xen-netfront.ko 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ntb 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ntb/ntb.ko 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/nvme 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/nvme/host 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/nvme/host/nvme-core.ko 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/nvme/host/nvme.ko 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport/parport.ko 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport/parport_cs.ko 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport/parport_pc.ko 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport/parport_serial.ko 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/hotplug 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2026-02-25T11:35:26.496 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/pcie 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/pcie/aer_inject.ko 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/xen-pcifront.ko 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pcmcia 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pcmcia/i82092.ko 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pcmcia/pd6729.ko 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pcmcia/yenta_socket.ko 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pinctrl 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pinctrl/intel 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/wmi 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/wmi/wmi.ko 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/acer-wmi.ko 2026-02-25T11:35:26.497 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/acerhdf.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/apple-gmux.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/asus-laptop.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/asus-nb-wmi.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/asus-wmi.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/classmate-laptop.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/eeepc-laptop.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/eeepc-wmi.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/fujitsu-laptop.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/fujitsu-tablet.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/hdaps.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/intel 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/intel/intel-rst.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/intel_ips.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/msi-wmi.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/mxm-wmi.ko 2026-02-25T11:35:26.498 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/panasonic-laptop.ko 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/samsung-laptop.ko 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/samsung-q10.ko 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/topstar-laptop.ko 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/toshiba_haps.ko 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/wmi-bmof.ko 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/powercap 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/powercap/intel_rapl_common.ko 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/powercap/intel_rapl_msr.ko 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/clients 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/clients/pps-gpio.ko 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/clients/pps-ldisc.ko 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/clients/pps_parport.ko 2026-02-25T11:35:26.499 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/pps_core.ko 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ptp 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ptp/ptp.ko 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ptp/ptp_kvm.ko 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ptp/ptp_vmclock.ko 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras/amd 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras/amd/atl 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras/amd/atl/amd_atl.ko 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras/amd/fmpm.ko 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-bq32k.ko 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1286.ko 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1307.ko 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1374.ko 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1511.ko 2026-02-25T11:35:26.500 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1553.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1672.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1742.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds2404.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds3232.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-em3027.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-fm3130.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-isl12022.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-isl1208.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-m41t80.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-m48t35.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-m48t59.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-max6900.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-msm6242.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf2127.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf85063.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf8523.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf8563.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf8583.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rp5c01.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rs5c372.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rv3029c2.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rx8025.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rx8581.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-stk17ta8.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-x1205.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/3w-9xxx.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/3w-sas.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/3w-xxxx.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/BusLogic.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/a100u2w.ko 2026-02-25T11:35:26.501 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aacraid 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aacraid/aacraid.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/advansys.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aic7xxx 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/am53c974.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/arcmsr 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/arcmsr/arcmsr.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/atp870u.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/be2iscsi 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bfa 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bfa/bfa.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bnx2fc 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bnx2i 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bnx2i/bnx2i.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/ch.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/csiostor 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/csiostor/csiostor.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/cxgb3i 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/cxgb4i 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/dc395x.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/dmx3191d.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/esas2r 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/esas2r/esas2r.ko 2026-02-25T11:35:26.502 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/esp_scsi.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fcoe 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fcoe/fcoe.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fcoe/libfcoe.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fnic 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fnic/fnic.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/hpsa.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/hptiop.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/initio.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/ipr.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/ips.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/isci 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/isci/isci.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/iscsi_tcp.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libfc 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libfc/libfc.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libiscsi.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libiscsi_tcp.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libsas 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libsas/libsas.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/lpfc 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/lpfc/lpfc.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mpt3sas 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mvsas 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mvsas/mvsas.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mvumi.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/pm8001 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/pm8001/pm80xx.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/pmcraid.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla1280.ko 2026-02-25T11:35:26.503 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla2xxx 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla4xxx 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/raid_class.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_debug.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_fc.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_iscsi.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_sas.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_spi.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_srp.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/ses.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/st.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/stex.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/sym53c8xx_2 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/virtio_scsi.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/vmw_pvscsi.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/wd719x.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ssb 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ssb/ssb.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/iscsi 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/loopback 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/loopback/tcm_loop.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_file.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_iblock.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_mod.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_pscsi.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_user.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/tcm_fc 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/tcm_fc/tcm_fc.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2026-02-25T11:35:26.504 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/platform_temperature_control.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_soc_slider.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/ipwireless 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/ipwireless/ipwireless.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/n_gsm.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/n_hdlc.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/nozomi.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/8250 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/8250/serial_cs.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/arc_uart.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/jsm 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/jsm/jsm.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs/core 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs/core/ufshcd-core.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs/host 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs/host/ufshcd-pci.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio_aec.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio_cif.ko 2026-02-25T11:35:26.505 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio_pci_generic.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio_sercos3.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class/cdc-acm.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class/cdc-wdm.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class/usblp.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class/usbtmc.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/host 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/host/sl811-hcd.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/image 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/image/mdc800.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/image/microtek.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/adutux.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/appledisplay.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/emi26.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/emi62.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/ezusb.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/idmouse.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/iowarrior.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/isight_firmware.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/ldusb.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/legousbtower.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/sisusbvga 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/trancevibrator.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/usb3503.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/usblcd.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/usbsevseg.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/uss720.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/yurex.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/phy 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/phy/phy-generic.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/aircable.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ark3116.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/belkin_sa.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ch341.ko 2026-02-25T11:35:26.506 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/cp210x.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/cyberjack.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/cypress_m8.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/digi_acceleport.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/empeg.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ftdi_sio.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/garmin_gps.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/io_edgeport.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/io_ti.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ipaq.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ipw.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ir-usb.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/iuu_phoenix.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/keyspan.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/keyspan_pda.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/kl5kusb105.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/kobil_sct.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/mct_u232.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/mos7720.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/mos7840.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/navman.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/omninet.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/opticon.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/option.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/oti6858.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/pl2303.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/qcaux.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/qcserial.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/quatech2.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/safe_serial.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/sierra.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/spcp8x5.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ssu100.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/symbolserial.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/usb-serial-simple.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/usb_debug.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/usb_wwan.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/visor.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/whiteheat.ko 2026-02-25T11:35:26.507 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/xsens_mt.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/uas.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-alauda.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-cypress.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-datafab.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-eneub6250.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-freecom.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-isd200.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-jumpshot.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-karma.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-onetouch.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-realtek.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-sddr09.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-sddr55.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-usbat.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/usb-storage.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/usbip 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/usbip/usbip-core.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/usbip/usbip-host.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/usbip/vhci-hcd.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/pci 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/pci/vfio-pci-core.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/pci/vfio-pci.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/vfio.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/vfio_iommu_type1.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost/vhost.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost/vhost_iotlb.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost/vhost_net.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost/vhost_scsi.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video/backlight 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video/backlight/apple_bl.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video/backlight/lcd.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video/backlight/platform_lcd.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_balloon.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_mmio.ko 2026-02-25T11:35:26.508 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_pci.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_ring.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/masters 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/masters/ds2482.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/masters/ds2490.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2406.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2408.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2413.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2423.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2431.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2433.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2780.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2781.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds28e04.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_smem.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_therm.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/wire.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/alim1535_wdt.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/alim7101_wdt.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/f71808e_wdt.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/hpwdt.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/i6300esb.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/iTCO_vendor_support.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/iTCO_wdt.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/ib700wdt.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/ibmasr.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/ie6xx_wdt.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/it8712f_wdt.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/it87_wdt.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/machzwd.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/nv_tco.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/pcwd_pci.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/pcwd_usb.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/sch311x_wdt.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/softdog.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/sp5100_tco.ko 2026-02-25T11:35:26.509 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/via_wdt.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/w83627hf_wdt.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/w83877f_wdt.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/w83977f_wdt.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/wdt_pci.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/xen_wdt.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-acpi-processor.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-evtchn.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-gntalloc.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-gntdev.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-pciback 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-privcmd.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xenfs 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xenfs/xenfs.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/9p 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/9p/9p.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/affs 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/affs/affs.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/befs 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/befs/befs.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/binfmt_misc.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/btrfs 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/btrfs/btrfs.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/cachefiles 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/cachefiles/cachefiles.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ceph 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ceph/ceph.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/coda 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/coda/coda.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/cramfs 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/cramfs/cramfs.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/dlm 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/dlm/dlm.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ecryptfs 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ecryptfs/ecryptfs.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/f2fs 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/f2fs/f2fs.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fat 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fat/fat.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fat/msdos.ko 2026-02-25T11:35:26.510 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fat/vfat.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fuse 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fuse/cuse.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fuse/fuse.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/gfs2 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/gfs2/gfs2.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/hfs 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/hfs/hfs.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/hfsplus 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/hfsplus/hfsplus.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/isofs 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/isofs/isofs.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/jfs 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/jfs/jfs.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/lockd 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/lockd/lockd.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/minix 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/minix/minix.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/netfs 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/netfs/netfs.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/blocklayout 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/filelayout 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/flexfilelayout 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/nfs.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/nfsv3.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/nfsv4.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs_common 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs_common/grace.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs_common/nfs_acl.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfsd 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfsd/nfsd.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nilfs2 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nilfs2/nilfs2.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-celtic.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-centeuro.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-croatian.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-cyrillic.ko 2026-02-25T11:35:26.511 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-gaelic.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-greek.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-iceland.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-inuit.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-roman.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-romanian.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-turkish.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp1250.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp1251.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp1255.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp737.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp775.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp850.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp852.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp855.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp857.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp860.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp861.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp862.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp863.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp864.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp865.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp866.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp869.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp874.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp932.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp936.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp949.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp950.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_euc-jp.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-1.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-13.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-14.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-15.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-2.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-3.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-4.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-5.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-6.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-7.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-9.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_koi8-r.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_koi8-ru.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_koi8-u.ko 2026-02-25T11:35:26.512 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_ucs2_utils.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_utf8.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/cluster 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/dlm 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/dlmfs 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/ocfs2.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/ocfs2_stack_user.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/ocfs2_stackglue.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/overlayfs 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/overlayfs/overlay.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/pstore 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/pstore/ramoops.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/romfs 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/romfs/romfs.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb/client 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb/client/cifs.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb/common 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb/common/cifs_md4.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/squashfs 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/squashfs/squashfs.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/udf 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/udf/udf.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ufs 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ufs/ufs.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/xfs 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/xfs/xfs.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/rcu 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/rcu/rcutorture.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/torture.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/trace 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/trace/ring_buffer_benchmark.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/asn1_encoder.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crc 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crc/crc-itu-t.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crc/crc8.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crypto 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crypto/libarc4.ko 2026-02-25T11:35:26.513 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crypto/libblake2b.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crypto/libdes.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/lru_cache.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/lz4 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/lz4/lz4_compress.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/lz4/lz4hc_compress.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/math 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/math/cordic.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/raid6 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/raid6/raid6_pq.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/reed_solomon 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/reed_solomon/reed_solomon.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/ts_bm.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/ts_fsm.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/ts_kmp.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/zstd 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/zstd/zstd_compress.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/mm 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/mm/hwpoison-inject.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/6lowpan.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_dest.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_fragment.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_hop.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_ipv6.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_mobility.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_routing.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_udp.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802/garp.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802/mrp.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802/psnap.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802/stp.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/8021q 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/8021q/8021q.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p/9pnet.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p/9pnet_fd.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p/9pnet_rdma.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p/9pnet_virtio.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/batman-adv 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/batman-adv/batman-adv.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/br_netfilter.ko 2026-02-25T11:35:26.514 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/bridge.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_802_3.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_among.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_arp.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_arpreply.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_dnat.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_ip.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_ip6.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_limit.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_log.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_mark.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_mark_m.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_nflog.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_pkttype.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_redirect.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_snat.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_stp.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_vlan.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ceph 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ceph/libceph.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/core 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/core/failover.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/core/pktgen.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dns_resolver 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dns_resolver/dns_resolver.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/dsa_core.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/tag_brcm.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/tag_dsa.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/tag_none.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/tag_trailer.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154/6lowpan 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154/ieee802154.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154/ieee802154_socket.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ah4.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/esp4.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/fou.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/gre.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/inet_diag.ko 2026-02-25T11:35:26.515 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ip_gre.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ip_tunnel.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ip_vti.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ipcomp.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ipip.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/arpt_mangle.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_ECN.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_ah.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_bic.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_dctcp.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_diag.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_highspeed.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_htcp.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_hybla.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_illinois.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_lp.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_scalable.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_vegas.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_veno.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_westwood.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_yeah.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tunnel4.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/udp_diag.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/udp_tunnel.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/xfrm4_tunnel.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ah6.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/esp6.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/fou6.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ip6_tunnel.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ip6_udp_tunnel.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ip6_vti.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ipcomp6.ko 2026-02-25T11:35:26.516 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_ah.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_frag.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_mh.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_rt.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/sit.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/tunnel6.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/xfrm6_tunnel.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/key 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/key/af_key.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_core.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_debugfs.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_eth.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_ip.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_ip6.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_netlink.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_ppp.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/llc 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/llc/llc.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/mac802154 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/mac802154/mac802154.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/mpls 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/mpls/mpls_gso.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2026-02-25T11:35:26.517 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_list_set.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conncount.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_amanda.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_broadcast.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_ftp.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_h323.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_irc.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_netlink.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_pptp.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_sane.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_sip.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_snmp.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_tftp.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_dup_netdev.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_flow_table.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_flow_table_inet.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_log_syslog.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_amanda.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_ftp.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_irc.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_sip.ko 2026-02-25T11:35:26.518 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_tftp.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_synproxy_core.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_tables.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink_acct.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink_log.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink_osf.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink_queue.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_chain_nat.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_compat.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_ct.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_dup_netdev.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_fib.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_fwd_netdev.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_hash.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_limit.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_log.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_masq.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_nat.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_numgen.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_queue.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_quota.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_redir.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_reject.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_reject_inet.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_socket.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_tunnel.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_AUDIT.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_CHECKSUM.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_CLASSIFY.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_CONNSECMARK.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_CT.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_DSCP.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_HL.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_HMARK.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_IDLETIMER.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_LED.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_LOG.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_MASQUERADE.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_NETMAP.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_NFLOG.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_NFQUEUE.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_RATEEST.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_REDIRECT.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_SECMARK.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_TCPMSS.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_TEE.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_TPROXY.ko 2026-02-25T11:35:26.519 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_addrtype.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_bpf.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_cgroup.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_cluster.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_comment.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_connbytes.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_connlabel.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_connlimit.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_connmark.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_conntrack.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_cpu.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_dccp.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_devgroup.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_dscp.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_ecn.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_esp.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_hashlimit.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_helper.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_hl.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_ipcomp.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_iprange.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_ipvs.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_l2tp.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_length.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_limit.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_mac.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_mark.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_multiport.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_nat.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_nfacct.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_osf.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_owner.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_physdev.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_pkttype.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_policy.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_quota.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_rateest.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_realm.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_recent.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_sctp.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_set.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_socket.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_state.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_statistic.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_string.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_tcpmss.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_time.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_u32.ko 2026-02-25T11:35:26.520 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netlink 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netlink/netlink_diag.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/nsh 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/nsh/nsh.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch/openvswitch.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch/vport-geneve.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch/vport-gre.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch/vport-vxlan.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/packet 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/packet/af_packet_diag.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/rds 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/rds/rds.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/rds/rds_rdma.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/rds/rds_tcp.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_csum.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_gact.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_mirred.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_nat.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_pedit.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_police.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_simple.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_skbedit.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_vlan.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_basic.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_bpf.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_flow.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_fw.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_route.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_u32.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_cmp.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_ipset.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_meta.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_nbyte.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_text.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_u32.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_choke.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_codel.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_drr.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_fq.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_gred.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_hfsc.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_hhf.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_htb.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_ingress.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_mqprio.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_mqprio_lib.ko 2026-02-25T11:35:26.521 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_multiq.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_netem.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_pie.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_plug.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_prio.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_qfq.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_red.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_sfb.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_sfq.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_tbf.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_teql.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sctp 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sctp/sctp.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sctp/sctp_diag.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/auth_gss 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/sunrpc.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/xprtrdma 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/tipc 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/tipc/tipc.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/tipc/tipc_diag.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/unix 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/unix/unix_diag.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vsock.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vsock_diag.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vsock_loopback.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/xfrm 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/xfrm/xfrm_ipcomp.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys/encrypted-keys 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys/trusted-keys 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys/trusted-keys/trusted.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/virt 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/virt/lib 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/virt/lib/irqbypass.ko 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.alias 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.alias.bin 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.builtin 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.builtin.alias.bin 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.builtin.bin 2026-02-25T11:35:26.522 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.builtin.modinfo 2026-02-25T11:35:26.523 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.dep 2026-02-25T11:35:26.523 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.dep.bin 2026-02-25T11:35:26.523 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.devname 2026-02-25T11:35:26.523 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.order 2026-02-25T11:35:26.523 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.softdep 2026-02-25T11:35:26.523 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.symbols 2026-02-25T11:35:26.523 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.symbols.bin 2026-02-25T11:35:26.523 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.weakdep 2026-02-25T11:35:26.523 INFO:teuthology.orchestra.run.trial028.stdout:/lib/modules/6.19.0-g712f4a43c125/vmlinuz 2026-02-25T11:35:26.525 DEBUG:teuthology.task.kernel:get_image_version: 6.19.0-g712f4a43c125 2026-02-25T11:35:26.525 DEBUG:teuthology.orchestra.run.trial028:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout:postinstall scriptlet (using /bin/sh): 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout:if [ -x /usr/bin/kernel-install ]; then 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout: /usr/bin/kernel-install add 6.19.0-g712f4a43c125 /lib/modules/6.19.0-g712f4a43c125/vmlinuz 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout:fi 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout:for file in vmlinuz System.map config; do 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout: if ! cmp --silent "/lib/modules/6.19.0-g712f4a43c125/${file}" "/boot/${file}-6.19.0-g712f4a43c125"; then 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout: cp "/lib/modules/6.19.0-g712f4a43c125/${file}" "/boot/${file}-6.19.0-g712f4a43c125" 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout: fi 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout:done 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout:if [ -d "/lib/modules/6.19.0-g712f4a43c125/dtb" ] && \ 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout: ! diff -rq "/lib/modules/6.19.0-g712f4a43c125/dtb" "/boot/dtb-6.19.0-g712f4a43c125" >/dev/null 2>&1; then 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout: rm -rf "/boot/dtb-6.19.0-g712f4a43c125" 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout: cp -r "/lib/modules/6.19.0-g712f4a43c125/dtb" "/boot/dtb-6.19.0-g712f4a43c125" 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout:fi 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout:if [ ! -e "/lib/modules/6.19.0-g712f4a43c125/modules.dep" ]; then 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout: /usr/sbin/depmod 6.19.0-g712f4a43c125 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout:fi 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout:preuninstall scriptlet (using /bin/sh): 2026-02-25T11:35:26.545 INFO:teuthology.orchestra.run.trial028.stdout:if [ -x /usr/bin/kernel-install ]; then 2026-02-25T11:35:26.546 INFO:teuthology.orchestra.run.trial028.stdout:kernel-install remove 6.19.0-g712f4a43c125 2026-02-25T11:35:26.546 INFO:teuthology.orchestra.run.trial028.stdout:fi 2026-02-25T11:35:26.546 DEBUG:teuthology.orchestra.run.trial028:> sudo rpm -qi grub2-tools 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Name : grub2-tools 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Epoch : 1 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Version : 2.06 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Release : 120.el9 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Architecture: x86_64 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Group : Unspecified 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Size : 9108611 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:License : GPLv3+ 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Packager : builder@centos.org 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Vendor : CentOS 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Summary : Support tools for GRUB. 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:Description : 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T11:35:26.616 INFO:teuthology.orchestra.run.trial028.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T11:35:26.617 INFO:teuthology.orchestra.run.trial028.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T11:35:26.617 INFO:teuthology.orchestra.run.trial028.stdout:hardware devices. 2026-02-25T11:35:26.617 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:35:26.617 INFO:teuthology.orchestra.run.trial028.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T11:35:26.618 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T11:35:26.618 INFO:teuthology.task.kernel:Updating grub on trial028 to boot 6.19.0-g712f4a43c125 2026-02-25T11:35:26.618 DEBUG:teuthology.orchestra.run.trial028:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T11:35:26.682 INFO:teuthology.orchestra.run.trial028.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T11:35:26.684 DEBUG:teuthology.orchestra.run.trial028:> sudo command -v grubby && echo yes || echo no 2026-02-25T11:35:26.747 INFO:teuthology.orchestra.run.trial028.stdout:/sbin/grubby 2026-02-25T11:35:26.749 INFO:teuthology.orchestra.run.trial028.stdout:yes 2026-02-25T11:35:26.749 DEBUG:teuthology.orchestra.run.trial028:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "6.19.0-g712f4a43c125" | head -n 1 2026-02-25T11:35:26.813 INFO:teuthology.orchestra.run.trial028.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-6.19.0-g712f4a43c125.conf 2026-02-25T11:35:26.815 DEBUG:teuthology.orchestra.run.trial028:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-6.19.0-g712f4a43c125 2026-02-25T11:35:26.888 DEBUG:teuthology.orchestra.run.trial028:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T11:35:27.092 INFO:teuthology.orchestra.run.trial028.stderr:Generating grub configuration file ... 2026-02-25T11:35:27.160 INFO:teuthology.orchestra.run.trial028.stderr:Found linux image: /boot/vmlinuz-6.19.0-g712f4a43c125 2026-02-25T11:35:27.167 INFO:teuthology.orchestra.run.trial028.stderr:Found initrd image: /boot/initramfs-6.19.0-g712f4a43c125.img 2026-02-25T11:35:27.419 INFO:teuthology.orchestra.run.trial028.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T11:35:27.426 INFO:teuthology.orchestra.run.trial028.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T11:35:27.460 INFO:teuthology.orchestra.run.trial028.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T11:35:27.466 INFO:teuthology.orchestra.run.trial028.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T11:35:27.497 INFO:teuthology.orchestra.run.trial028.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T11:35:27.503 INFO:teuthology.orchestra.run.trial028.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T11:35:27.526 INFO:teuthology.orchestra.run.trial028.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T11:35:27.532 INFO:teuthology.orchestra.run.trial028.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T11:35:27.617 INFO:teuthology.orchestra.run.trial028.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T11:35:27.626 INFO:teuthology.orchestra.run.trial028.stderr:done 2026-02-25T11:35:27.628 DEBUG:teuthology.orchestra.run.trial028:> test -d /sys/firmware/efi 2026-02-25T11:35:27.645 DEBUG:teuthology.orchestra.run.trial028:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T11:35:27.712 INFO:teuthology.orchestra.run.trial028.stdout:/boot/efi/EFI/BOOT 2026-02-25T11:35:27.712 INFO:teuthology.orchestra.run.trial028.stdout:/boot/efi/EFI/centos 2026-02-25T11:35:27.714 DEBUG:teuthology.orchestra.run.trial028:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T11:35:27.779 INFO:teuthology.orchestra.run.trial028.stdout:no 2026-02-25T11:35:27.780 DEBUG:teuthology.orchestra.run.trial028:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T11:35:27.845 INFO:teuthology.orchestra.run.trial028.stdout:yes 2026-02-25T11:35:27.845 DEBUG:teuthology.orchestra.run.trial028:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T11:35:27.908 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial028 2026-02-25T11:35:27.908 DEBUG:teuthology.orchestra.run.trial028:> sudo bash -c ' 2026-02-25T11:35:27.908 DEBUG:teuthology.orchestra.run.trial028:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T11:35:27.908 DEBUG:teuthology.orchestra.run.trial028:> ' 2026-02-25T11:35:28.506 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial119.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2026-02-25T11:35:28.507 DEBUG:teuthology.orchestra.run.trial119:> rpm -qlp /tmp/kernel.x86_64.rpm 2026-02-25T11:35:28.563 INFO:teuthology.orchestra.run.trial119.stdout:/boot/System.map-6.19.0-g712f4a43c125 2026-02-25T11:35:28.563 INFO:teuthology.orchestra.run.trial119.stdout:/boot/config-6.19.0-g712f4a43c125 2026-02-25T11:35:28.563 INFO:teuthology.orchestra.run.trial119.stdout:/boot/vmlinuz-6.19.0-g712f4a43c125 2026-02-25T11:35:28.563 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125 2026-02-25T11:35:28.563 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/System.map 2026-02-25T11:35:28.564 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/config 2026-02-25T11:35:28.564 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel 2026-02-25T11:35:28.564 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch 2026-02-25T11:35:28.564 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86 2026-02-25T11:35:28.564 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto 2026-02-25T11:35:28.564 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/blowfish-x86_64.ko 2026-02-25T11:35:28.564 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2026-02-25T11:35:28.564 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2026-02-25T11:35:28.564 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/camellia-x86_64.ko 2026-02-25T11:35:28.564 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2026-02-25T11:35:28.564 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2026-02-25T11:35:28.565 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2026-02-25T11:35:28.565 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2026-02-25T11:35:28.565 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2026-02-25T11:35:28.565 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/serpent-avx2.ko 2026-02-25T11:35:28.565 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2026-02-25T11:35:28.565 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2026-02-25T11:35:28.565 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2026-02-25T11:35:28.565 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/twofish-x86_64.ko 2026-02-25T11:35:28.565 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kernel 2026-02-25T11:35:28.565 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kernel/cpu 2026-02-25T11:35:28.565 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kernel/cpu/mce 2026-02-25T11:35:28.565 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2026-02-25T11:35:28.565 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kvm 2026-02-25T11:35:28.566 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kvm/kvm-amd.ko 2026-02-25T11:35:28.566 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kvm/kvm-intel.ko 2026-02-25T11:35:28.566 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kvm/kvm.ko 2026-02-25T11:35:28.566 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto 2026-02-25T11:35:28.566 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/anubis.ko 2026-02-25T11:35:28.566 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/arc4.ko 2026-02-25T11:35:28.566 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx 2026-02-25T11:35:28.566 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_memcpy.ko 2026-02-25T11:35:28.566 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_pq.ko 2026-02-25T11:35:28.566 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_raid6_recov.ko 2026-02-25T11:35:28.566 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_tx.ko 2026-02-25T11:35:28.566 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_xor.ko 2026-02-25T11:35:28.566 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/raid6test.ko 2026-02-25T11:35:28.567 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/authenc.ko 2026-02-25T11:35:28.567 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/authencesn.ko 2026-02-25T11:35:28.567 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/blake2b.ko 2026-02-25T11:35:28.567 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/blowfish_common.ko 2026-02-25T11:35:28.567 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/blowfish_generic.ko 2026-02-25T11:35:28.567 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/camellia_generic.ko 2026-02-25T11:35:28.567 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/cast5_generic.ko 2026-02-25T11:35:28.568 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/cast6_generic.ko 2026-02-25T11:35:28.568 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/cast_common.ko 2026-02-25T11:35:28.568 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/ccm.ko 2026-02-25T11:35:28.568 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/cmac.ko 2026-02-25T11:35:28.568 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/crc32-cryptoapi.ko 2026-02-25T11:35:28.568 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/crypto_engine.ko 2026-02-25T11:35:28.568 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/crypto_user.ko 2026-02-25T11:35:28.568 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/des_generic.ko 2026-02-25T11:35:28.569 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/dh_generic.ko 2026-02-25T11:35:28.569 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/echainiv.ko 2026-02-25T11:35:28.569 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/essiv.ko 2026-02-25T11:35:28.569 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/fcrypt.ko 2026-02-25T11:35:28.569 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/khazad.ko 2026-02-25T11:35:28.569 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/krb5 2026-02-25T11:35:28.569 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/krb5/krb5.ko 2026-02-25T11:35:28.569 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/krb5enc.ko 2026-02-25T11:35:28.570 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/lz4.ko 2026-02-25T11:35:28.570 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/lz4hc.ko 2026-02-25T11:35:28.570 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/md4.ko 2026-02-25T11:35:28.570 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/michael_mic.ko 2026-02-25T11:35:28.570 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/pcbc.ko 2026-02-25T11:35:28.570 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/pcrypt.ko 2026-02-25T11:35:28.570 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/rmd160.ko 2026-02-25T11:35:28.570 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/seed.ko 2026-02-25T11:35:28.570 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/serpent_generic.ko 2026-02-25T11:35:28.571 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/tea.ko 2026-02-25T11:35:28.571 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/twofish_common.ko 2026-02-25T11:35:28.571 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/twofish_generic.ko 2026-02-25T11:35:28.571 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/wp512.ko 2026-02-25T11:35:28.571 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/xcbc.ko 2026-02-25T11:35:28.571 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/xor.ko 2026-02-25T11:35:28.571 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/xxhash_generic.ko 2026-02-25T11:35:28.571 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers 2026-02-25T11:35:28.571 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi 2026-02-25T11:35:28.572 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/acpi_ipmi.ko 2026-02-25T11:35:28.572 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/acpi_pad.ko 2026-02-25T11:35:28.572 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/ec_sys.ko 2026-02-25T11:35:28.572 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/platform_profile.ko 2026-02-25T11:35:28.572 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/sbs.ko 2026-02-25T11:35:28.572 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/sbshc.ko 2026-02-25T11:35:28.572 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/video.ko 2026-02-25T11:35:28.572 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata 2026-02-25T11:35:28.573 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/acard-ahci.ko 2026-02-25T11:35:28.573 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/ahci_platform.ko 2026-02-25T11:35:28.573 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/ata_generic.ko 2026-02-25T11:35:28.573 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/libahci_platform.ko 2026-02-25T11:35:28.573 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_acpi.ko 2026-02-25T11:35:28.573 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_ali.ko 2026-02-25T11:35:28.573 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_amd.ko 2026-02-25T11:35:28.573 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_artop.ko 2026-02-25T11:35:28.573 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_atiixp.ko 2026-02-25T11:35:28.574 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_atp867x.ko 2026-02-25T11:35:28.574 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_cmd640.ko 2026-02-25T11:35:28.574 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_cmd64x.ko 2026-02-25T11:35:28.574 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_cypress.ko 2026-02-25T11:35:28.574 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_efar.ko 2026-02-25T11:35:28.574 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_hpt366.ko 2026-02-25T11:35:28.574 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_hpt37x.ko 2026-02-25T11:35:28.574 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_hpt3x2n.ko 2026-02-25T11:35:28.575 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_hpt3x3.ko 2026-02-25T11:35:28.575 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_it8213.ko 2026-02-25T11:35:28.575 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_it821x.ko 2026-02-25T11:35:28.575 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_jmicron.ko 2026-02-25T11:35:28.575 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_marvell.ko 2026-02-25T11:35:28.575 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_mpiix.ko 2026-02-25T11:35:28.575 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_netcell.ko 2026-02-25T11:35:28.575 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_ninja32.ko 2026-02-25T11:35:28.575 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_ns87410.ko 2026-02-25T11:35:28.575 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_ns87415.ko 2026-02-25T11:35:28.576 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_oldpiix.ko 2026-02-25T11:35:28.576 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_opti.ko 2026-02-25T11:35:28.576 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_optidma.ko 2026-02-25T11:35:28.576 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_pcmcia.ko 2026-02-25T11:35:28.576 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_pdc2027x.ko 2026-02-25T11:35:28.576 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_pdc202xx_old.ko 2026-02-25T11:35:28.576 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_piccolo.ko 2026-02-25T11:35:28.576 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_rdc.ko 2026-02-25T11:35:28.576 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_sch.ko 2026-02-25T11:35:28.576 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_serverworks.ko 2026-02-25T11:35:28.577 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_sil680.ko 2026-02-25T11:35:28.577 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_sis.ko 2026-02-25T11:35:28.577 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_sl82c105.ko 2026-02-25T11:35:28.577 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_triflex.ko 2026-02-25T11:35:28.577 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_via.ko 2026-02-25T11:35:28.577 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pdc_adma.ko 2026-02-25T11:35:28.577 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_inic162x.ko 2026-02-25T11:35:28.577 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_mv.ko 2026-02-25T11:35:28.577 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_nv.ko 2026-02-25T11:35:28.578 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_promise.ko 2026-02-25T11:35:28.578 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_qstor.ko 2026-02-25T11:35:28.578 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_sil.ko 2026-02-25T11:35:28.578 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_sil24.ko 2026-02-25T11:35:28.578 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_sis.ko 2026-02-25T11:35:28.578 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_svw.ko 2026-02-25T11:35:28.578 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_sx4.ko 2026-02-25T11:35:28.578 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_uli.ko 2026-02-25T11:35:28.578 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_via.ko 2026-02-25T11:35:28.578 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_vsc.ko 2026-02-25T11:35:28.579 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/auxdisplay 2026-02-25T11:35:28.579 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/auxdisplay/ks0108.ko 2026-02-25T11:35:28.579 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/base 2026-02-25T11:35:28.579 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/base/regmap 2026-02-25T11:35:28.579 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/base/regmap/regmap-i2c.ko 2026-02-25T11:35:28.579 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/base/regmap/regmap-mmio.ko 2026-02-25T11:35:28.579 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/bcma 2026-02-25T11:35:28.579 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/bcma/bcma.ko 2026-02-25T11:35:28.579 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block 2026-02-25T11:35:28.579 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/aoe 2026-02-25T11:35:28.579 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/aoe/aoe.ko 2026-02-25T11:35:28.580 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/brd.ko 2026-02-25T11:35:28.580 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/drbd 2026-02-25T11:35:28.580 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/drbd/drbd.ko 2026-02-25T11:35:28.580 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/floppy.ko 2026-02-25T11:35:28.580 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/loop.ko 2026-02-25T11:35:28.580 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/mtip32xx 2026-02-25T11:35:28.580 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/mtip32xx/mtip32xx.ko 2026-02-25T11:35:28.580 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/nbd.ko 2026-02-25T11:35:28.580 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/null_blk 2026-02-25T11:35:28.581 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/null_blk/null_blk.ko 2026-02-25T11:35:28.581 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/rbd.ko 2026-02-25T11:35:28.581 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/virtio_blk.ko 2026-02-25T11:35:28.581 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/xen-blkback 2026-02-25T11:35:28.581 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/xen-blkback/xen-blkback.ko 2026-02-25T11:35:28.581 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/xen-blkfront.ko 2026-02-25T11:35:28.581 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/zram 2026-02-25T11:35:28.581 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/zram/zram.ko 2026-02-25T11:35:28.581 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char 2026-02-25T11:35:28.581 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hangcheck-timer.ko 2026-02-25T11:35:28.582 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random 2026-02-25T11:35:28.582 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/amd-rng.ko 2026-02-25T11:35:28.582 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/intel-rng.ko 2026-02-25T11:35:28.582 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/timeriomem-rng.ko 2026-02-25T11:35:28.582 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/via-rng.ko 2026-02-25T11:35:28.582 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/virtio-rng.ko 2026-02-25T11:35:28.582 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi 2026-02-25T11:35:28.582 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_devintf.ko 2026-02-25T11:35:28.582 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2026-02-25T11:35:28.582 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2026-02-25T11:35:28.582 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_si.ko 2026-02-25T11:35:28.583 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_ssif.ko 2026-02-25T11:35:28.583 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2026-02-25T11:35:28.583 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/lp.ko 2026-02-25T11:35:28.583 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/mwave 2026-02-25T11:35:28.583 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/mwave/mwave.ko 2026-02-25T11:35:28.583 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ppdev.ko 2026-02-25T11:35:28.583 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tlclk.ko 2026-02-25T11:35:28.583 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm 2026-02-25T11:35:28.583 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm.ko 2026-02-25T11:35:28.583 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_atmel.ko 2026-02-25T11:35:28.583 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_infineon.ko 2026-02-25T11:35:28.583 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_nsc.ko 2026-02-25T11:35:28.584 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_tis.ko 2026-02-25T11:35:28.584 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_tis_core.ko 2026-02-25T11:35:28.584 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/uv_mmtimer.ko 2026-02-25T11:35:28.584 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/virtio_console.ko 2026-02-25T11:35:28.584 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq 2026-02-25T11:35:28.584 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/acpi-cpufreq.ko 2026-02-25T11:35:28.584 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2026-02-25T11:35:28.584 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/p4-clockmod.ko 2026-02-25T11:35:28.584 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/pcc-cpufreq.ko 2026-02-25T11:35:28.584 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/powernow-k8.ko 2026-02-25T11:35:28.585 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/speedstep-lib.ko 2026-02-25T11:35:28.585 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto 2026-02-25T11:35:28.585 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/ccp 2026-02-25T11:35:28.585 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/ccp/ccp-crypto.ko 2026-02-25T11:35:28.585 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/ccp/ccp.ko 2026-02-25T11:35:28.585 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel 2026-02-25T11:35:28.585 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat 2026-02-25T11:35:28.585 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat/qat_common 2026-02-25T11:35:28.585 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2026-02-25T11:35:28.585 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2026-02-25T11:35:28.585 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2026-02-25T11:35:28.585 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/padlock-aes.ko 2026-02-25T11:35:28.586 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/padlock-sha.ko 2026-02-25T11:35:28.586 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/virtio 2026-02-25T11:35:28.586 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/virtio/virtio_crypto.ko 2026-02-25T11:35:28.586 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dca 2026-02-25T11:35:28.586 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dca/dca.ko 2026-02-25T11:35:28.586 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/devfreq 2026-02-25T11:35:28.586 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/devfreq/governor_simpleondemand.ko 2026-02-25T11:35:28.586 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma 2026-02-25T11:35:28.586 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma/dw 2026-02-25T11:35:28.586 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma/dw/dw_dmac.ko 2026-02-25T11:35:28.586 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma/ioat 2026-02-25T11:35:28.586 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma/ioat/ioatdma.ko 2026-02-25T11:35:28.587 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac 2026-02-25T11:35:28.587 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/amd64_edac.ko 2026-02-25T11:35:28.587 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/e752x_edac.ko 2026-02-25T11:35:28.587 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/edac_mce_amd.ko 2026-02-25T11:35:28.587 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i3000_edac.ko 2026-02-25T11:35:28.587 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i3200_edac.ko 2026-02-25T11:35:28.587 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i5100_edac.ko 2026-02-25T11:35:28.587 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i5400_edac.ko 2026-02-25T11:35:28.587 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i7300_edac.ko 2026-02-25T11:35:28.587 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i7core_edac.ko 2026-02-25T11:35:28.587 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i82975x_edac.ko 2026-02-25T11:35:28.587 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/ie31200_edac.ko 2026-02-25T11:35:28.587 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/sb_edac.ko 2026-02-25T11:35:28.588 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/x38_edac.ko 2026-02-25T11:35:28.588 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/firmware 2026-02-25T11:35:28.588 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/firmware/edd.ko 2026-02-25T11:35:28.588 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/firmware/iscsi_ibft.ko 2026-02-25T11:35:28.588 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/gpio 2026-02-25T11:35:28.588 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/gpio/gpio-ich.ko 2026-02-25T11:35:28.588 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/gpio/gpio-viperboard.ko 2026-02-25T11:35:28.588 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid 2026-02-25T11:35:28.588 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-appleir.ko 2026-02-25T11:35:28.588 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-aureal.ko 2026-02-25T11:35:28.588 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-axff.ko 2026-02-25T11:35:28.588 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-dr.ko 2026-02-25T11:35:28.588 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-elecom.ko 2026-02-25T11:35:28.589 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-elo.ko 2026-02-25T11:35:28.589 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-emsff.ko 2026-02-25T11:35:28.589 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-gaff.ko 2026-02-25T11:35:28.589 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-gt683r.ko 2026-02-25T11:35:28.589 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-gyration.ko 2026-02-25T11:35:28.589 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-holtek-kbd.ko 2026-02-25T11:35:28.589 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-holtek-mouse.ko 2026-02-25T11:35:28.589 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-holtekff.ko 2026-02-25T11:35:28.589 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-icade.ko 2026-02-25T11:35:28.589 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-keytouch.ko 2026-02-25T11:35:28.589 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-kye.ko 2026-02-25T11:35:28.589 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-lcpower.ko 2026-02-25T11:35:28.589 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-led.ko 2026-02-25T11:35:28.590 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-lenovo.ko 2026-02-25T11:35:28.590 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-multitouch.ko 2026-02-25T11:35:28.590 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-ortek.ko 2026-02-25T11:35:28.590 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-penmount.ko 2026-02-25T11:35:28.590 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-petalynx.ko 2026-02-25T11:35:28.590 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-picolcd.ko 2026-02-25T11:35:28.590 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-pl.ko 2026-02-25T11:35:28.590 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-plantronics.ko 2026-02-25T11:35:28.590 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-primax.ko 2026-02-25T11:35:28.590 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-rmi.ko 2026-02-25T11:35:28.590 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-arvo.ko 2026-02-25T11:35:28.590 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-common.ko 2026-02-25T11:35:28.591 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-isku.ko 2026-02-25T11:35:28.591 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-kone.ko 2026-02-25T11:35:28.591 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-koneplus.ko 2026-02-25T11:35:28.591 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-konepure.ko 2026-02-25T11:35:28.591 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-kovaplus.ko 2026-02-25T11:35:28.591 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-lua.ko 2026-02-25T11:35:28.591 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-pyra.ko 2026-02-25T11:35:28.591 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-ryos.ko 2026-02-25T11:35:28.591 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-savu.ko 2026-02-25T11:35:28.591 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat.ko 2026-02-25T11:35:28.591 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-saitek.ko 2026-02-25T11:35:28.591 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-samsung.ko 2026-02-25T11:35:28.591 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-sensor-hub.ko 2026-02-25T11:35:28.592 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-sjoy.ko 2026-02-25T11:35:28.592 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-sony.ko 2026-02-25T11:35:28.592 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-speedlink.ko 2026-02-25T11:35:28.592 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-steelseries.ko 2026-02-25T11:35:28.592 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-sunplus.ko 2026-02-25T11:35:28.592 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-thrustmaster.ko 2026-02-25T11:35:28.592 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-tivo.ko 2026-02-25T11:35:28.592 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-tmff.ko 2026-02-25T11:35:28.592 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-topseed.ko 2026-02-25T11:35:28.592 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-twinhan.ko 2026-02-25T11:35:28.592 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-uclogic.ko 2026-02-25T11:35:28.592 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-waltop.ko 2026-02-25T11:35:28.592 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-wiimote.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-xinmo.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-zpff.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-zydacron.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/uhid.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/wacom.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/abituguru.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/abituguru3.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/acpi_power_meter.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ad7414.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ad7418.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adc128d818.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm1025.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm1026.ko 2026-02-25T11:35:28.593 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm1029.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm1031.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm9240.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ads7828.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7410.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7411.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7462.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7470.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7475.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7x10.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/amc6821.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/applesmc.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/asb100.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/asc7621.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/asus_atk0110.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/atxp1.ko 2026-02-25T11:35:28.594 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/coretemp.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/dell-smm-hwmon.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/dme1737.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ds1621.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ds620.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/emc1403.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/emc6w201.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/f71805f.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/f71882fg.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/f75375s.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/fam15h_power.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/fschmd.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/g760a.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/g762.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/gl518sm.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/gl520sm.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/hwmon-vid.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/i5500_temp.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/i5k_amb.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ibmaem.ko 2026-02-25T11:35:28.595 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ibmpex.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ina209.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ina2xx.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/it87.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/k10temp.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/k8temp.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lineage-pem.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm63.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm73.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm75.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm77.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm78.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm80.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm83.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm85.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm87.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm90.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm92.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm93.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm95234.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm95241.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm95245.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc2945.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4151.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4215.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4222.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4245.ko 2026-02-25T11:35:28.596 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4260.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4261.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max16065.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max1619.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max1668.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max197.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max6639.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max6650.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max6697.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/mcp3021.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/nct6683.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/nct6775-core.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/nct6775.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/nct7802.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pc87360.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pc87427.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pcf8591.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/adm1275.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/lm25066.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/ltc2978.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/max16064.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/max34440.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/max8688.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/pmbus.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/tps40422.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/ucd9000.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/ucd9200.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/zl6100.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/powr1220.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sch5627.ko 2026-02-25T11:35:28.597 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sch5636.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sch56xx-common.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sht15.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sht21.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/shtc1.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sis5595.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/smsc47b397.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/smsc47m1.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/smsc47m192.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/thmc50.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/tmp102.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/tmp103.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/tmp401.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/tmp421.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/via-cputemp.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/via686a.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/vt1211.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/vt8231.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83627ehf.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83627hf.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83781d.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83791d.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83792d.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83793.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83795.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83l785ts.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83l786ng.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/algos 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2026-02-25T11:35:28.598 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-amd756.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-amd8111.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-designware-core.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-i801.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-isch.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-ismt.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-nforce2.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-parport.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-piix4.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-scmi.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-simtec.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-sis96x.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-via.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-viapro.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-viperboard.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-dev.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-mux.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-slave-eeprom.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-smbus.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-stub.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/ib_cm.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/ib_core.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/ib_umad.ko 2026-02-25T11:35:28.599 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/ib_uverbs.ko 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/iw_cm.ko 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/rdma_cm.ko 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/rdma_ucm.ko 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/cxgb4 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mlx4 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mlx5 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mthca 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/ipoib 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/iser 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/isert 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/srp 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/srpt 2026-02-25T11:35:28.600 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/gameport 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/gameport/emu10k1-gp.ko 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/gameport/fm801-gp.ko 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/gameport/gameport.ko 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/joydev.ko 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/keyboard 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/keyboard/gpio_keys.ko 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/appletouch.ko 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/bcm5974.ko 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/cyapatp.ko 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/elan_i2c.ko 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/sermouse.ko 2026-02-25T11:35:28.601 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/synaptics_i2c.ko 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/synaptics_usb.ko 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/vsxxxaa.ko 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/rmi4 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/rmi4/rmi_core.ko 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/serio 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/serio/altera_ps2.ko 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/serio/arc_ps2.ko 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/serio/serio_raw.ko 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/sparse-keymap.ko 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/leds-blinkm.ko 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/leds-lm3530.ko 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/leds-lp3944.ko 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/leds-ss4200.ko 2026-02-25T11:35:28.602 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-camera.ko 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-timer.ko 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-transient.ko 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/bcache 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/bcache/bcache.ko 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-bio-prison.ko 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-cache-smq.ko 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-cache.ko 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-crypt.ko 2026-02-25T11:35:28.603 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-delay.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-flakey.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-log-userspace.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-multipath.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-queue-length.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-raid.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-round-robin.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-service-time.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-switch.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-thin-pool.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-verity.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/linear.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/persistent-data 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/raid0.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/raid1.ko 2026-02-25T11:35:28.604 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/raid10.ko 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/raid456.ko 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptbase.ko 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptctl.ko 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptfc.ko 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptlan.ko 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptsas.ko 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptscsih.ko 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptspi.ko 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/lpc_ich.ko 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/lpc_sch.ko 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/mfd-core.ko 2026-02-25T11:35:28.605 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/sm501.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/viperboard.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/vx855.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/altera-stapl 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/apds9802als.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/apds990x.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/bh1770glc.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/cb710 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/cb710/cb710.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/eeprom 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/eeprom/at24.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/eeprom/max6875.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/enclosure.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/hpilo.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/isl29003.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/isl29020.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/lis3lv02d 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/mei 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/mei/mei-me.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/mei/mei-txe.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/mei/mei.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-gru 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-gru/gru.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-xp 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-xp/xp.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-xp/xpc.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-xp/xpnet.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/tifm_7xx1.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/tifm_core.ko 2026-02-25T11:35:28.606 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/tsl2550.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/vmw_balloon.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/vmw_vmci 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/bonding 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/bonding/bonding.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/b53 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/b53/b53_common.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/bcm-sf2.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/mv88e6060.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/mv88e6xxx 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dummy.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/eql.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com/3c59x.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com/typhoon.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390/8390.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/adaptec 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/adaptec/starfire.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/agere 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/agere/et131x.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/alteon 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/alteon/acenic.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/altera 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/altera/altera_tse.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/amd 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/amd/amd8111e.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2026-02-25T11:35:28.607 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/amd/pcnet32.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/alx 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atl1c 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atl1e 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atlx 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/b44.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/bnx2x 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/cnic.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/genet 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/tg3.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/brocade 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/brocade/bna 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb3 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb4 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/libcxgb 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/cisco 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/cisco/enic 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip 2026-02-25T11:35:28.608 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dlink 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dlink/dl2k.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dlink/sundance.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dnet.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/emulex 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/emulex/benet 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/ethoc.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/fealnx.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e100.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e1000 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e1000e 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/fm10k 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/i40e 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/iavf 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ice 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ice/ice.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/igb 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/igb/igb.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/igbvf 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ixgbe 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ixgbevf 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2026-02-25T11:35:28.609 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libeth 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libeth/libeth_xdp.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libie 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libie/libie.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libie/libie_adminq.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libie/libie_fwlog.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/jme.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/marvell 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/marvell/skge.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/marvell/sky2.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx4 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx5 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx5/core 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/micrel 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/myricom 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/myricom/myri10ge 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/natsemi 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/neterion 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/neterion/s2io.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/nvidia 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/packetengines 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/netxen 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/qlcnic 2026-02-25T11:35:28.610 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/rdc 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/rdc/r6040.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek/8139cp.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek/8139too.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek/atp.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek/r8169.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/rocker 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/rocker/rocker.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sfc 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sfc/sfc.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/silan 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/silan/sc92031.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sis 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sis/sis190.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sis/sis900.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc/epic100.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/stmicro 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/stmicro/stmmac 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun/cassini.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun/niu.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun/sungem.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun/sunhme.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/tehuti 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/ti 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/ti/tlan.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/via 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/via/via-rhine.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/via/via-velocity.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/wiznet 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/wiznet/w5100.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/wiznet/w5300.ko 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/xircom 2026-02-25T11:35:28.611 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/geneve.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ieee802154 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ieee802154/fakelb.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ifb.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ipvlan 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ipvlan/ipvlan.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/macvlan.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/macvtap.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio/mdio-bitbang.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio/mdio-regmap.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mii.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/net_failover.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/netconsole.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/nlmon.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/pcs 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/pcs/pcs-lynx.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/pcs/pcs_xpcs.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/amd.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/bcm-phy-lib.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/bcm-phy-ptp.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/bcm7xxx.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/bcm87xx.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/broadcom.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/cicada.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/davicom.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/dp83640.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/et1011c.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/icplus.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/lxt.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/marvell.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/micrel.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/national.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/phy_package.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/phylink.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/qsemi.ko 2026-02-25T11:35:28.612 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/realtek 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/realtek/realtek.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/smsc.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/ste10Xp.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/vitesse.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/bsd_comp.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_async.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_deflate.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_generic.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_mppe.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_synctty.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/pppoe.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/pppox.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/pptp.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/slip 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/slip/slhc.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/slip/slip.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/sungem_phy.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/tap.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_activebackup.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_broadcast.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_loadbalance.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_random.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_roundrobin.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/tun.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/veth.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/virtio_net.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/vmxnet3 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/vmxnet3/vmxnet3.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/vxlan 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/vxlan/vxlan.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/xen-netback 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/xen-netback/xen-netback.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/xen-netfront.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ntb 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ntb/ntb.ko 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/nvme 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/nvme/host 2026-02-25T11:35:28.613 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/nvme/host/nvme-core.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/nvme/host/nvme.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport/parport.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport/parport_cs.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport/parport_pc.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport/parport_serial.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/hotplug 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/pcie 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/pcie/aer_inject.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/xen-pcifront.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pcmcia 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pcmcia/i82092.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pcmcia/pd6729.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pcmcia/yenta_socket.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pinctrl 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pinctrl/intel 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/wmi 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/wmi/wmi.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/acer-wmi.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/acerhdf.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/apple-gmux.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/asus-laptop.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/asus-nb-wmi.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/asus-wmi.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/classmate-laptop.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/eeepc-laptop.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/eeepc-wmi.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/fujitsu-laptop.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/fujitsu-tablet.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/hdaps.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/intel 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/intel/intel-rst.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/intel_ips.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/msi-wmi.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/mxm-wmi.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/panasonic-laptop.ko 2026-02-25T11:35:28.614 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/samsung-laptop.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/samsung-q10.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/topstar-laptop.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/toshiba_haps.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/wmi-bmof.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/powercap 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/powercap/intel_rapl_common.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/powercap/intel_rapl_msr.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/clients 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/clients/pps-gpio.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/clients/pps-ldisc.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/clients/pps_parport.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/pps_core.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ptp 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ptp/ptp.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ptp/ptp_kvm.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ptp/ptp_vmclock.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras/amd 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras/amd/atl 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras/amd/atl/amd_atl.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras/amd/fmpm.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-bq32k.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1286.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1307.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1374.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1511.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1553.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1672.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1742.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds2404.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds3232.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-em3027.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-fm3130.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-isl12022.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-isl1208.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-m41t80.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-m48t35.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-m48t59.ko 2026-02-25T11:35:28.615 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-max6900.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-msm6242.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf2127.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf85063.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf8523.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf8563.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf8583.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rp5c01.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rs5c372.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rv3029c2.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rx8025.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rx8581.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-stk17ta8.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-x1205.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/3w-9xxx.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/3w-sas.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/3w-xxxx.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/BusLogic.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/a100u2w.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aacraid 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aacraid/aacraid.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/advansys.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aic7xxx 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/am53c974.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/arcmsr 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/arcmsr/arcmsr.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/atp870u.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/be2iscsi 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bfa 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bfa/bfa.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bnx2fc 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bnx2i 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bnx2i/bnx2i.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/ch.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/csiostor 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/csiostor/csiostor.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/cxgb3i 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2026-02-25T11:35:28.616 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/cxgb4i 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/dc395x.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/dmx3191d.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/esas2r 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/esas2r/esas2r.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/esp_scsi.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fcoe 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fcoe/fcoe.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fcoe/libfcoe.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fnic 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fnic/fnic.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/hpsa.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/hptiop.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/initio.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/ipr.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/ips.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/isci 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/isci/isci.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/iscsi_tcp.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libfc 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libfc/libfc.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libiscsi.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libiscsi_tcp.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libsas 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libsas/libsas.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/lpfc 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/lpfc/lpfc.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mpt3sas 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mvsas 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mvsas/mvsas.ko 2026-02-25T11:35:28.617 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mvumi.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/pm8001 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/pm8001/pm80xx.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/pmcraid.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla1280.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla2xxx 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla4xxx 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/raid_class.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_debug.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_fc.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_iscsi.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_sas.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_spi.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_srp.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/ses.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/st.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/stex.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/sym53c8xx_2 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/virtio_scsi.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/vmw_pvscsi.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/wd719x.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ssb 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ssb/ssb.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/iscsi 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/loopback 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/loopback/tcm_loop.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_file.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_iblock.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_mod.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_pscsi.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_user.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/tcm_fc 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/tcm_fc/tcm_fc.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2026-02-25T11:35:28.618 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/platform_temperature_control.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_soc_slider.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/ipwireless 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/ipwireless/ipwireless.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/n_gsm.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/n_hdlc.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/nozomi.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/8250 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/8250/serial_cs.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/arc_uart.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/jsm 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/jsm/jsm.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs/core 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs/core/ufshcd-core.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs/host 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs/host/ufshcd-pci.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio_aec.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio_cif.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio_pci_generic.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio_sercos3.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class/cdc-acm.ko 2026-02-25T11:35:28.619 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class/cdc-wdm.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class/usblp.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class/usbtmc.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/host 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/host/sl811-hcd.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/image 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/image/mdc800.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/image/microtek.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/adutux.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/appledisplay.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/emi26.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/emi62.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/ezusb.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/idmouse.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/iowarrior.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/isight_firmware.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/ldusb.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/legousbtower.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/sisusbvga 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/trancevibrator.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/usb3503.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/usblcd.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/usbsevseg.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/uss720.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/yurex.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/phy 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/phy/phy-generic.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/aircable.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ark3116.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/belkin_sa.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ch341.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/cp210x.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/cyberjack.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/cypress_m8.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/digi_acceleport.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/empeg.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ftdi_sio.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/garmin_gps.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/io_edgeport.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/io_ti.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ipaq.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ipw.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ir-usb.ko 2026-02-25T11:35:28.620 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/iuu_phoenix.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/keyspan.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/keyspan_pda.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/kl5kusb105.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/kobil_sct.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/mct_u232.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/mos7720.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/mos7840.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/navman.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/omninet.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/opticon.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/option.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/oti6858.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/pl2303.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/qcaux.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/qcserial.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/quatech2.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/safe_serial.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/sierra.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/spcp8x5.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ssu100.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/symbolserial.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/usb-serial-simple.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/usb_debug.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/usb_wwan.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/visor.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/whiteheat.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/xsens_mt.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/uas.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-alauda.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-cypress.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-datafab.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-eneub6250.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-freecom.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-isd200.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-jumpshot.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-karma.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-onetouch.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-realtek.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-sddr09.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-sddr55.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-usbat.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/usb-storage.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/usbip 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/usbip/usbip-core.ko 2026-02-25T11:35:28.621 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/usbip/usbip-host.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/usbip/vhci-hcd.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/pci 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/pci/vfio-pci-core.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/pci/vfio-pci.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/vfio.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/vfio_iommu_type1.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost/vhost.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost/vhost_iotlb.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost/vhost_net.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost/vhost_scsi.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video/backlight 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video/backlight/apple_bl.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video/backlight/lcd.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video/backlight/platform_lcd.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_balloon.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_mmio.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_pci.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_ring.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/masters 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/masters/ds2482.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/masters/ds2490.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2406.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2408.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2413.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2423.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2431.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2433.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2780.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2781.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds28e04.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_smem.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_therm.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/wire.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/alim1535_wdt.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/alim7101_wdt.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/f71808e_wdt.ko 2026-02-25T11:35:28.622 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/hpwdt.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/i6300esb.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/iTCO_vendor_support.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/iTCO_wdt.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/ib700wdt.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/ibmasr.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/ie6xx_wdt.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/it8712f_wdt.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/it87_wdt.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/machzwd.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/nv_tco.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/pcwd_pci.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/pcwd_usb.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/sch311x_wdt.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/softdog.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/sp5100_tco.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/via_wdt.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/w83627hf_wdt.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/w83877f_wdt.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/w83977f_wdt.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/wdt_pci.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/xen_wdt.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-acpi-processor.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-evtchn.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-gntalloc.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-gntdev.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-pciback 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-privcmd.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xenfs 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xenfs/xenfs.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/9p 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/9p/9p.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/affs 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/affs/affs.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/befs 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/befs/befs.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/binfmt_misc.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/btrfs 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/btrfs/btrfs.ko 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/cachefiles 2026-02-25T11:35:28.623 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/cachefiles/cachefiles.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ceph 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ceph/ceph.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/coda 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/coda/coda.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/cramfs 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/cramfs/cramfs.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/dlm 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/dlm/dlm.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ecryptfs 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ecryptfs/ecryptfs.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/f2fs 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/f2fs/f2fs.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fat 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fat/fat.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fat/msdos.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fat/vfat.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fuse 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fuse/cuse.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fuse/fuse.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/gfs2 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/gfs2/gfs2.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/hfs 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/hfs/hfs.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/hfsplus 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/hfsplus/hfsplus.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/isofs 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/isofs/isofs.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/jfs 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/jfs/jfs.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/lockd 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/lockd/lockd.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/minix 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/minix/minix.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/netfs 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/netfs/netfs.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/blocklayout 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/filelayout 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/flexfilelayout 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/nfs.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/nfsv3.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/nfsv4.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs_common 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs_common/grace.ko 2026-02-25T11:35:28.624 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs_common/nfs_acl.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfsd 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfsd/nfsd.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nilfs2 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nilfs2/nilfs2.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-celtic.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-centeuro.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-croatian.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-cyrillic.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-gaelic.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-greek.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-iceland.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-inuit.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-roman.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-romanian.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-turkish.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp1250.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp1251.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp1255.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp737.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp775.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp850.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp852.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp855.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp857.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp860.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp861.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp862.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp863.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp864.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp865.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp866.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp869.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp874.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp932.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp936.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp949.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp950.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_euc-jp.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-1.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-13.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-14.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-15.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-2.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-3.ko 2026-02-25T11:35:28.625 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-4.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-5.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-6.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-7.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-9.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_koi8-r.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_koi8-ru.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_koi8-u.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_ucs2_utils.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_utf8.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/cluster 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/dlm 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/dlmfs 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/ocfs2.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/ocfs2_stack_user.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/ocfs2_stackglue.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/overlayfs 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/overlayfs/overlay.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/pstore 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/pstore/ramoops.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/romfs 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/romfs/romfs.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb/client 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb/client/cifs.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb/common 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb/common/cifs_md4.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/squashfs 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/squashfs/squashfs.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/udf 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/udf/udf.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ufs 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ufs/ufs.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/xfs 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/xfs/xfs.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/rcu 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/rcu/rcutorture.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/torture.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/trace 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/trace/ring_buffer_benchmark.ko 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib 2026-02-25T11:35:28.626 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/asn1_encoder.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crc 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crc/crc-itu-t.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crc/crc8.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crypto 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crypto/libarc4.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crypto/libblake2b.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crypto/libdes.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/lru_cache.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/lz4 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/lz4/lz4_compress.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/lz4/lz4hc_compress.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/math 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/math/cordic.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/raid6 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/raid6/raid6_pq.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/reed_solomon 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/reed_solomon/reed_solomon.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/ts_bm.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/ts_fsm.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/ts_kmp.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/zstd 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/zstd/zstd_compress.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/mm 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/mm/hwpoison-inject.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/6lowpan.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_dest.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_fragment.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_hop.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_ipv6.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_mobility.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_routing.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_udp.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802/garp.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802/mrp.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802/psnap.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802/stp.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/8021q 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/8021q/8021q.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p/9pnet.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p/9pnet_fd.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p/9pnet_rdma.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p/9pnet_virtio.ko 2026-02-25T11:35:28.627 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/batman-adv 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/batman-adv/batman-adv.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/br_netfilter.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/bridge.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_802_3.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_among.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_arp.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_arpreply.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_dnat.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_ip.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_ip6.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_limit.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_log.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_mark.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_mark_m.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_nflog.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_pkttype.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_redirect.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_snat.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_stp.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_vlan.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ceph 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ceph/libceph.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/core 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/core/failover.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/core/pktgen.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dns_resolver 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dns_resolver/dns_resolver.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/dsa_core.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/tag_brcm.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/tag_dsa.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/tag_none.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/tag_trailer.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154/6lowpan 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154/ieee802154.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154/ieee802154_socket.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ah4.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/esp4.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/fou.ko 2026-02-25T11:35:28.628 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/gre.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/inet_diag.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ip_gre.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ip_tunnel.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ip_vti.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ipcomp.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ipip.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/arpt_mangle.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_ECN.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_ah.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_bic.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_dctcp.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_diag.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_highspeed.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_htcp.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_hybla.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_illinois.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_lp.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_scalable.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_vegas.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_veno.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_westwood.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_yeah.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tunnel4.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/udp_diag.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/udp_tunnel.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/xfrm4_tunnel.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ah6.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/esp6.ko 2026-02-25T11:35:28.629 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/fou6.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ip6_tunnel.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ip6_udp_tunnel.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ip6_vti.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ipcomp6.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_ah.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_frag.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_mh.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_rt.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/sit.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/tunnel6.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/xfrm6_tunnel.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/key 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/key/af_key.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_core.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_debugfs.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_eth.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_ip.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_ip6.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_netlink.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_ppp.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/llc 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/llc/llc.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/mac802154 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/mac802154/mac802154.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/mpls 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/mpls/mpls_gso.ko 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset 2026-02-25T11:35:28.630 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_list_set.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2026-02-25T11:35:28.631 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conncount.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_amanda.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_broadcast.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_ftp.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_h323.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_irc.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_netlink.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_pptp.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_sane.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_sip.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_snmp.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_tftp.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_dup_netdev.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_flow_table.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_flow_table_inet.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_log_syslog.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_amanda.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_ftp.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_irc.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_sip.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_tftp.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_synproxy_core.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_tables.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink_acct.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink_log.ko 2026-02-25T11:35:28.632 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink_osf.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink_queue.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_chain_nat.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_compat.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_ct.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_dup_netdev.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_fib.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_fwd_netdev.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_hash.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_limit.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_log.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_masq.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_nat.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_numgen.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_queue.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_quota.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_redir.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_reject.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_reject_inet.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_socket.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_tunnel.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_AUDIT.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_CHECKSUM.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_CLASSIFY.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_CONNSECMARK.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_CT.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_DSCP.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_HL.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_HMARK.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_IDLETIMER.ko 2026-02-25T11:35:28.633 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_LED.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_LOG.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_MASQUERADE.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_NETMAP.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_NFLOG.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_NFQUEUE.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_RATEEST.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_REDIRECT.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_SECMARK.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_TCPMSS.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_TEE.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_TPROXY.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_addrtype.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_bpf.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_cgroup.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_cluster.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_comment.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_connbytes.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_connlabel.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_connlimit.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_connmark.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_conntrack.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_cpu.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_dccp.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_devgroup.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_dscp.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_ecn.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_esp.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_hashlimit.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_helper.ko 2026-02-25T11:35:28.634 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_hl.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_ipcomp.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_iprange.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_ipvs.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_l2tp.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_length.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_limit.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_mac.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_mark.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_multiport.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_nat.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_nfacct.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_osf.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_owner.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_physdev.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_pkttype.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_policy.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_quota.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_rateest.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_realm.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_recent.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_sctp.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_set.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_socket.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_state.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_statistic.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_string.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_tcpmss.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_time.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_u32.ko 2026-02-25T11:35:28.635 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netlink 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netlink/netlink_diag.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/nsh 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/nsh/nsh.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch/openvswitch.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch/vport-geneve.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch/vport-gre.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch/vport-vxlan.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/packet 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/packet/af_packet_diag.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/rds 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/rds/rds.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/rds/rds_rdma.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/rds/rds_tcp.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_csum.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_gact.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_mirred.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_nat.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_pedit.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_police.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_simple.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_skbedit.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_vlan.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_basic.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_bpf.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_flow.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_fw.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_route.ko 2026-02-25T11:35:28.636 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_u32.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_cmp.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_ipset.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_meta.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_nbyte.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_text.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_u32.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_choke.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_codel.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_drr.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_fq.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_gred.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_hfsc.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_hhf.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_htb.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_ingress.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_mqprio.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_mqprio_lib.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_multiq.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_netem.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_pie.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_plug.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_prio.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_qfq.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_red.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_sfb.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_sfq.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_tbf.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_teql.ko 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sctp 2026-02-25T11:35:28.637 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sctp/sctp.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sctp/sctp_diag.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/auth_gss 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/sunrpc.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/xprtrdma 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/tipc 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/tipc/tipc.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/tipc/tipc_diag.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/unix 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/unix/unix_diag.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vsock.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vsock_diag.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vsock_loopback.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/xfrm 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/xfrm/xfrm_ipcomp.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys/encrypted-keys 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys/trusted-keys 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys/trusted-keys/trusted.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/virt 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/virt/lib 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/virt/lib/irqbypass.ko 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.alias 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.alias.bin 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.builtin 2026-02-25T11:35:28.638 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.builtin.alias.bin 2026-02-25T11:35:28.639 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.builtin.bin 2026-02-25T11:35:28.639 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.builtin.modinfo 2026-02-25T11:35:28.639 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.dep 2026-02-25T11:35:28.639 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.dep.bin 2026-02-25T11:35:28.639 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.devname 2026-02-25T11:35:28.639 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.order 2026-02-25T11:35:28.639 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.softdep 2026-02-25T11:35:28.639 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.symbols 2026-02-25T11:35:28.639 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.symbols.bin 2026-02-25T11:35:28.639 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.weakdep 2026-02-25T11:35:28.639 INFO:teuthology.orchestra.run.trial119.stdout:/lib/modules/6.19.0-g712f4a43c125/vmlinuz 2026-02-25T11:35:28.641 DEBUG:teuthology.task.kernel:get_image_version: 6.19.0-g712f4a43c125 2026-02-25T11:35:28.641 DEBUG:teuthology.orchestra.run.trial119:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout:postinstall scriptlet (using /bin/sh): 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout:if [ -x /usr/bin/kernel-install ]; then 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout: /usr/bin/kernel-install add 6.19.0-g712f4a43c125 /lib/modules/6.19.0-g712f4a43c125/vmlinuz 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout:fi 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout:for file in vmlinuz System.map config; do 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout: if ! cmp --silent "/lib/modules/6.19.0-g712f4a43c125/${file}" "/boot/${file}-6.19.0-g712f4a43c125"; then 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout: cp "/lib/modules/6.19.0-g712f4a43c125/${file}" "/boot/${file}-6.19.0-g712f4a43c125" 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout: fi 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout:done 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout:if [ -d "/lib/modules/6.19.0-g712f4a43c125/dtb" ] && \ 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout: ! diff -rq "/lib/modules/6.19.0-g712f4a43c125/dtb" "/boot/dtb-6.19.0-g712f4a43c125" >/dev/null 2>&1; then 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout: rm -rf "/boot/dtb-6.19.0-g712f4a43c125" 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout: cp -r "/lib/modules/6.19.0-g712f4a43c125/dtb" "/boot/dtb-6.19.0-g712f4a43c125" 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout:fi 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout:if [ ! -e "/lib/modules/6.19.0-g712f4a43c125/modules.dep" ]; then 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout: /usr/sbin/depmod 6.19.0-g712f4a43c125 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout:fi 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout:preuninstall scriptlet (using /bin/sh): 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout:if [ -x /usr/bin/kernel-install ]; then 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout:kernel-install remove 6.19.0-g712f4a43c125 2026-02-25T11:35:28.661 INFO:teuthology.orchestra.run.trial119.stdout:fi 2026-02-25T11:35:28.662 DEBUG:teuthology.orchestra.run.trial119:> sudo rpm -qi grub2-tools 2026-02-25T11:35:28.730 INFO:teuthology.orchestra.run.trial119.stdout:Name : grub2-tools 2026-02-25T11:35:28.730 INFO:teuthology.orchestra.run.trial119.stdout:Epoch : 1 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Version : 2.06 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Release : 120.el9 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Architecture: x86_64 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Group : Unspecified 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Size : 9108611 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:License : GPLv3+ 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Packager : builder@centos.org 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Vendor : CentOS 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Summary : Support tools for GRUB. 2026-02-25T11:35:28.731 INFO:teuthology.orchestra.run.trial119.stdout:Description : 2026-02-25T11:35:28.732 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:35:28.732 INFO:teuthology.orchestra.run.trial119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T11:35:28.732 INFO:teuthology.orchestra.run.trial119.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T11:35:28.732 INFO:teuthology.orchestra.run.trial119.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T11:35:28.732 INFO:teuthology.orchestra.run.trial119.stdout:hardware devices. 2026-02-25T11:35:28.732 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:35:28.732 INFO:teuthology.orchestra.run.trial119.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T11:35:28.733 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T11:35:28.733 INFO:teuthology.task.kernel:Updating grub on trial119 to boot 6.19.0-g712f4a43c125 2026-02-25T11:35:28.733 DEBUG:teuthology.orchestra.run.trial119:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T11:35:28.797 INFO:teuthology.orchestra.run.trial119.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T11:35:28.798 DEBUG:teuthology.orchestra.run.trial119:> sudo command -v grubby && echo yes || echo no 2026-02-25T11:35:28.862 INFO:teuthology.orchestra.run.trial119.stdout:/sbin/grubby 2026-02-25T11:35:28.864 INFO:teuthology.orchestra.run.trial119.stdout:yes 2026-02-25T11:35:28.864 DEBUG:teuthology.orchestra.run.trial119:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "6.19.0-g712f4a43c125" | head -n 1 2026-02-25T11:35:28.929 INFO:teuthology.orchestra.run.trial119.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-6.19.0-g712f4a43c125.conf 2026-02-25T11:35:28.930 DEBUG:teuthology.orchestra.run.trial119:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-6.19.0-g712f4a43c125 2026-02-25T11:35:29.003 DEBUG:teuthology.orchestra.run.trial119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T11:35:29.211 INFO:teuthology.orchestra.run.trial119.stderr:Generating grub configuration file ... 2026-02-25T11:35:29.280 INFO:teuthology.orchestra.run.trial119.stderr:Found linux image: /boot/vmlinuz-6.19.0-g712f4a43c125 2026-02-25T11:35:29.287 INFO:teuthology.orchestra.run.trial119.stderr:Found initrd image: /boot/initramfs-6.19.0-g712f4a43c125.img 2026-02-25T11:35:29.544 INFO:teuthology.orchestra.run.trial119.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T11:35:29.551 INFO:teuthology.orchestra.run.trial119.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T11:35:29.585 INFO:teuthology.orchestra.run.trial119.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T11:35:29.591 INFO:teuthology.orchestra.run.trial119.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T11:35:29.622 INFO:teuthology.orchestra.run.trial119.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T11:35:29.629 INFO:teuthology.orchestra.run.trial119.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T11:35:29.652 INFO:teuthology.orchestra.run.trial119.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T11:35:29.658 INFO:teuthology.orchestra.run.trial119.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T11:35:29.745 INFO:teuthology.orchestra.run.trial119.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T11:35:29.753 INFO:teuthology.orchestra.run.trial119.stderr:done 2026-02-25T11:35:29.755 DEBUG:teuthology.orchestra.run.trial119:> test -d /sys/firmware/efi 2026-02-25T11:35:29.771 DEBUG:teuthology.orchestra.run.trial119:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T11:35:29.837 INFO:teuthology.orchestra.run.trial119.stdout:/boot/efi/EFI/BOOT 2026-02-25T11:35:29.837 INFO:teuthology.orchestra.run.trial119.stdout:/boot/efi/EFI/centos 2026-02-25T11:35:29.839 DEBUG:teuthology.orchestra.run.trial119:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T11:35:29.903 INFO:teuthology.orchestra.run.trial119.stdout:no 2026-02-25T11:35:29.904 DEBUG:teuthology.orchestra.run.trial119:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T11:35:29.968 INFO:teuthology.orchestra.run.trial119.stdout:yes 2026-02-25T11:35:29.969 DEBUG:teuthology.orchestra.run.trial119:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T11:35:30.034 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial119 2026-02-25T11:35:30.034 DEBUG:teuthology.orchestra.run.trial119:> sudo bash -c ' 2026-02-25T11:35:30.034 DEBUG:teuthology.orchestra.run.trial119:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T11:35:30.034 DEBUG:teuthology.orchestra.run.trial119:> ' 2026-02-25T11:35:32.971 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial167.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2026-02-25T11:35:32.971 DEBUG:teuthology.orchestra.run.trial167:> rpm -qlp /tmp/kernel.x86_64.rpm 2026-02-25T11:35:33.016 INFO:teuthology.orchestra.run.trial167.stdout:/boot/System.map-6.19.0-g712f4a43c125 2026-02-25T11:35:33.017 INFO:teuthology.orchestra.run.trial167.stdout:/boot/config-6.19.0-g712f4a43c125 2026-02-25T11:35:33.017 INFO:teuthology.orchestra.run.trial167.stdout:/boot/vmlinuz-6.19.0-g712f4a43c125 2026-02-25T11:35:33.017 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125 2026-02-25T11:35:33.017 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/System.map 2026-02-25T11:35:33.017 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/config 2026-02-25T11:35:33.017 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel 2026-02-25T11:35:33.017 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch 2026-02-25T11:35:33.017 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86 2026-02-25T11:35:33.017 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto 2026-02-25T11:35:33.018 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/blowfish-x86_64.ko 2026-02-25T11:35:33.018 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2026-02-25T11:35:33.018 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2026-02-25T11:35:33.018 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/camellia-x86_64.ko 2026-02-25T11:35:33.018 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2026-02-25T11:35:33.018 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2026-02-25T11:35:33.018 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2026-02-25T11:35:33.018 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2026-02-25T11:35:33.018 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2026-02-25T11:35:33.018 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/serpent-avx2.ko 2026-02-25T11:35:33.018 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2026-02-25T11:35:33.019 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2026-02-25T11:35:33.019 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2026-02-25T11:35:33.019 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/crypto/twofish-x86_64.ko 2026-02-25T11:35:33.019 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kernel 2026-02-25T11:35:33.019 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kernel/cpu 2026-02-25T11:35:33.019 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kernel/cpu/mce 2026-02-25T11:35:33.019 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2026-02-25T11:35:33.019 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kvm 2026-02-25T11:35:33.019 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kvm/kvm-amd.ko 2026-02-25T11:35:33.019 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kvm/kvm-intel.ko 2026-02-25T11:35:33.019 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/arch/x86/kvm/kvm.ko 2026-02-25T11:35:33.019 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto 2026-02-25T11:35:33.020 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/anubis.ko 2026-02-25T11:35:33.020 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/arc4.ko 2026-02-25T11:35:33.020 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx 2026-02-25T11:35:33.020 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_memcpy.ko 2026-02-25T11:35:33.020 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_pq.ko 2026-02-25T11:35:33.020 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_raid6_recov.ko 2026-02-25T11:35:33.020 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_tx.ko 2026-02-25T11:35:33.020 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/async_xor.ko 2026-02-25T11:35:33.020 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/async_tx/raid6test.ko 2026-02-25T11:35:33.020 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/authenc.ko 2026-02-25T11:35:33.020 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/authencesn.ko 2026-02-25T11:35:33.020 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/blake2b.ko 2026-02-25T11:35:33.021 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/blowfish_common.ko 2026-02-25T11:35:33.021 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/blowfish_generic.ko 2026-02-25T11:35:33.021 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/camellia_generic.ko 2026-02-25T11:35:33.021 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/cast5_generic.ko 2026-02-25T11:35:33.021 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/cast6_generic.ko 2026-02-25T11:35:33.021 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/cast_common.ko 2026-02-25T11:35:33.021 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/ccm.ko 2026-02-25T11:35:33.021 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/cmac.ko 2026-02-25T11:35:33.021 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/crc32-cryptoapi.ko 2026-02-25T11:35:33.021 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/crypto_engine.ko 2026-02-25T11:35:33.021 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/crypto_user.ko 2026-02-25T11:35:33.021 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/des_generic.ko 2026-02-25T11:35:33.021 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/dh_generic.ko 2026-02-25T11:35:33.022 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/echainiv.ko 2026-02-25T11:35:33.022 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/essiv.ko 2026-02-25T11:35:33.022 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/fcrypt.ko 2026-02-25T11:35:33.022 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/khazad.ko 2026-02-25T11:35:33.022 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/krb5 2026-02-25T11:35:33.022 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/krb5/krb5.ko 2026-02-25T11:35:33.022 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/krb5enc.ko 2026-02-25T11:35:33.022 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/lz4.ko 2026-02-25T11:35:33.022 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/lz4hc.ko 2026-02-25T11:35:33.022 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/md4.ko 2026-02-25T11:35:33.022 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/michael_mic.ko 2026-02-25T11:35:33.022 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/pcbc.ko 2026-02-25T11:35:33.023 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/pcrypt.ko 2026-02-25T11:35:33.023 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/rmd160.ko 2026-02-25T11:35:33.023 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/seed.ko 2026-02-25T11:35:33.023 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/serpent_generic.ko 2026-02-25T11:35:33.023 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/tea.ko 2026-02-25T11:35:33.023 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/twofish_common.ko 2026-02-25T11:35:33.023 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/twofish_generic.ko 2026-02-25T11:35:33.023 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/wp512.ko 2026-02-25T11:35:33.023 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/xcbc.ko 2026-02-25T11:35:33.023 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/xor.ko 2026-02-25T11:35:33.023 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/crypto/xxhash_generic.ko 2026-02-25T11:35:33.023 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers 2026-02-25T11:35:33.023 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi 2026-02-25T11:35:33.024 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/acpi_ipmi.ko 2026-02-25T11:35:33.024 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/acpi_pad.ko 2026-02-25T11:35:33.024 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/ec_sys.ko 2026-02-25T11:35:33.024 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/platform_profile.ko 2026-02-25T11:35:33.024 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/sbs.ko 2026-02-25T11:35:33.024 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/sbshc.ko 2026-02-25T11:35:33.024 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/acpi/video.ko 2026-02-25T11:35:33.024 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata 2026-02-25T11:35:33.024 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/acard-ahci.ko 2026-02-25T11:35:33.024 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/ahci_platform.ko 2026-02-25T11:35:33.024 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/ata_generic.ko 2026-02-25T11:35:33.024 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/libahci_platform.ko 2026-02-25T11:35:33.025 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_acpi.ko 2026-02-25T11:35:33.025 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_ali.ko 2026-02-25T11:35:33.025 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_amd.ko 2026-02-25T11:35:33.025 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_artop.ko 2026-02-25T11:35:33.025 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_atiixp.ko 2026-02-25T11:35:33.025 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_atp867x.ko 2026-02-25T11:35:33.025 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_cmd640.ko 2026-02-25T11:35:33.025 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_cmd64x.ko 2026-02-25T11:35:33.025 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_cypress.ko 2026-02-25T11:35:33.025 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_efar.ko 2026-02-25T11:35:33.025 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_hpt366.ko 2026-02-25T11:35:33.025 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_hpt37x.ko 2026-02-25T11:35:33.025 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_hpt3x2n.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_hpt3x3.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_it8213.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_it821x.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_jmicron.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_marvell.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_mpiix.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_netcell.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_ninja32.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_ns87410.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_ns87415.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_oldpiix.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_opti.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_optidma.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_pcmcia.ko 2026-02-25T11:35:33.026 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_pdc2027x.ko 2026-02-25T11:35:33.027 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_pdc202xx_old.ko 2026-02-25T11:35:33.027 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_piccolo.ko 2026-02-25T11:35:33.027 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_rdc.ko 2026-02-25T11:35:33.027 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_sch.ko 2026-02-25T11:35:33.027 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_serverworks.ko 2026-02-25T11:35:33.027 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_sil680.ko 2026-02-25T11:35:33.027 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_sis.ko 2026-02-25T11:35:33.027 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_sl82c105.ko 2026-02-25T11:35:33.027 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_triflex.ko 2026-02-25T11:35:33.027 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pata_via.ko 2026-02-25T11:35:33.027 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/pdc_adma.ko 2026-02-25T11:35:33.027 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_inic162x.ko 2026-02-25T11:35:33.027 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_mv.ko 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_nv.ko 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_promise.ko 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_qstor.ko 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_sil.ko 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_sil24.ko 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_sis.ko 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_svw.ko 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_sx4.ko 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_uli.ko 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_via.ko 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ata/sata_vsc.ko 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/auxdisplay 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/auxdisplay/ks0108.ko 2026-02-25T11:35:33.028 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/base 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/base/regmap 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/base/regmap/regmap-i2c.ko 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/base/regmap/regmap-mmio.ko 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/bcma 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/bcma/bcma.ko 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/aoe 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/aoe/aoe.ko 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/brd.ko 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/drbd 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/drbd/drbd.ko 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/floppy.ko 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/loop.ko 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/mtip32xx 2026-02-25T11:35:33.029 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/mtip32xx/mtip32xx.ko 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/nbd.ko 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/null_blk 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/null_blk/null_blk.ko 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/rbd.ko 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/virtio_blk.ko 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/xen-blkback 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/xen-blkback/xen-blkback.ko 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/xen-blkfront.ko 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/zram 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/block/zram/zram.ko 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hangcheck-timer.ko 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random 2026-02-25T11:35:33.030 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/amd-rng.ko 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/intel-rng.ko 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/timeriomem-rng.ko 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/via-rng.ko 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/hw_random/virtio-rng.ko 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_devintf.ko 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_si.ko 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_ssif.ko 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/lp.ko 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/mwave 2026-02-25T11:35:33.031 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/mwave/mwave.ko 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/ppdev.ko 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tlclk.ko 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm.ko 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_atmel.ko 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_infineon.ko 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_nsc.ko 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_tis.ko 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/tpm/tpm_tis_core.ko 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/uv_mmtimer.ko 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/char/virtio_console.ko 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/acpi-cpufreq.ko 2026-02-25T11:35:33.032 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/p4-clockmod.ko 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/pcc-cpufreq.ko 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/powernow-k8.ko 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/cpufreq/speedstep-lib.ko 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/ccp 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/ccp/ccp-crypto.ko 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/ccp/ccp.ko 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat/qat_common 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/padlock-aes.ko 2026-02-25T11:35:33.033 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/padlock-sha.ko 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/virtio 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/crypto/virtio/virtio_crypto.ko 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dca 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dca/dca.ko 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/devfreq 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/devfreq/governor_simpleondemand.ko 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma/dw 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma/dw/dw_dmac.ko 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma/ioat 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/dma/ioat/ioatdma.ko 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/amd64_edac.ko 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/e752x_edac.ko 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/edac_mce_amd.ko 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i3000_edac.ko 2026-02-25T11:35:33.034 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i3200_edac.ko 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i5100_edac.ko 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i5400_edac.ko 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i7300_edac.ko 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i7core_edac.ko 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/i82975x_edac.ko 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/ie31200_edac.ko 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/sb_edac.ko 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/edac/x38_edac.ko 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/firmware 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/firmware/edd.ko 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/firmware/iscsi_ibft.ko 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/gpio 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/gpio/gpio-ich.ko 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/gpio/gpio-viperboard.ko 2026-02-25T11:35:33.035 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-appleir.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-aureal.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-axff.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-dr.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-elecom.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-elo.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-emsff.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-gaff.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-gt683r.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-gyration.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-holtek-kbd.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-holtek-mouse.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-holtekff.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-icade.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-keytouch.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-kye.ko 2026-02-25T11:35:33.036 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-lcpower.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-led.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-lenovo.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-multitouch.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-ortek.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-penmount.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-petalynx.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-picolcd.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-pl.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-plantronics.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-primax.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-rmi.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-arvo.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-common.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-isku.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-kone.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-koneplus.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-konepure.ko 2026-02-25T11:35:33.037 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-kovaplus.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-lua.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-pyra.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-ryos.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat-savu.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-roccat.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-saitek.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-samsung.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-sensor-hub.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-sjoy.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-sony.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-speedlink.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-steelseries.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-sunplus.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-thrustmaster.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-tivo.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-tmff.ko 2026-02-25T11:35:33.038 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-topseed.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-twinhan.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-uclogic.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-waltop.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-wiimote.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-xinmo.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-zpff.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/hid-zydacron.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/uhid.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hid/wacom.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/abituguru.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/abituguru3.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/acpi_power_meter.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ad7414.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ad7418.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adc128d818.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm1025.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm1026.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm1029.ko 2026-02-25T11:35:33.039 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm1031.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adm9240.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ads7828.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7410.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7411.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7462.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7470.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7475.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/adt7x10.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/amc6821.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/applesmc.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/asb100.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/asc7621.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/asus_atk0110.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/atxp1.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/coretemp.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/dell-smm-hwmon.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/dme1737.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ds1621.ko 2026-02-25T11:35:33.040 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ds620.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/emc1403.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/emc6w201.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/f71805f.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/f71882fg.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/f75375s.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/fam15h_power.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/fschmd.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/g760a.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/g762.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/gl518sm.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/gl520sm.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/hwmon-vid.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/i5500_temp.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/i5k_amb.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ibmaem.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ibmpex.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ina209.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ina2xx.ko 2026-02-25T11:35:33.041 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/it87.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/k10temp.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/k8temp.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lineage-pem.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm63.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm73.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm75.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm77.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm78.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm80.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm83.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm85.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm87.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm90.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm92.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm93.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm95234.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm95241.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/lm95245.ko 2026-02-25T11:35:33.042 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc2945.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4151.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4215.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4222.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4245.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4260.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/ltc4261.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max16065.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max1619.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max1668.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max197.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max6639.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max6650.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/max6697.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/mcp3021.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/nct6683.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/nct6775-core.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/nct6775.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/nct7802.ko 2026-02-25T11:35:33.043 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pc87360.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pc87427.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pcf8591.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/adm1275.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/lm25066.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/ltc2978.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/max16064.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/max34440.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/max8688.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/pmbus.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/tps40422.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/ucd9000.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/ucd9200.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/pmbus/zl6100.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/powr1220.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sch5627.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sch5636.ko 2026-02-25T11:35:33.044 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sch56xx-common.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sht15.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sht21.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/shtc1.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/sis5595.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/smsc47b397.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/smsc47m1.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/smsc47m192.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/thmc50.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/tmp102.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/tmp103.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/tmp401.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/tmp421.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/via-cputemp.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/via686a.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/vt1211.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/vt8231.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83627ehf.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83627hf.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83781d.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83791d.ko 2026-02-25T11:35:33.045 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83792d.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83793.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83795.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83l785ts.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/hwmon/w83l786ng.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/algos 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-amd756.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-amd8111.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-designware-core.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-i801.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-isch.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-ismt.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-nforce2.ko 2026-02-25T11:35:33.046 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-parport.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-piix4.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-scmi.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-simtec.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-sis96x.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-via.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-viapro.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/busses/i2c-viperboard.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-dev.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-mux.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-slave-eeprom.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-smbus.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/i2c/i2c-stub.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/ib_cm.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/ib_core.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/ib_umad.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/ib_uverbs.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/iw_cm.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/rdma_cm.ko 2026-02-25T11:35:33.047 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/core/rdma_ucm.ko 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/cxgb4 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mlx4 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mlx5 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mthca 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/ipoib 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/iser 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/isert 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/srp 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/srpt 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input 2026-02-25T11:35:33.048 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/gameport 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/gameport/emu10k1-gp.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/gameport/fm801-gp.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/gameport/gameport.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/joydev.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/keyboard 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/keyboard/gpio_keys.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/appletouch.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/bcm5974.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/cyapatp.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/elan_i2c.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/sermouse.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/synaptics_i2c.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/synaptics_usb.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/mouse/vsxxxaa.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/rmi4 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/rmi4/rmi_core.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/serio 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/serio/altera_ps2.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/serio/arc_ps2.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/serio/serio_raw.ko 2026-02-25T11:35:33.049 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/input/sparse-keymap.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/leds-blinkm.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/leds-lm3530.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/leds-lp3944.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/leds-ss4200.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-camera.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-timer.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/leds/trigger/ledtrig-transient.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/bcache 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/bcache/bcache.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-bio-prison.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-cache-smq.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-cache.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-crypt.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-delay.ko 2026-02-25T11:35:33.050 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-flakey.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-log-userspace.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-multipath.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-queue-length.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-raid.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-round-robin.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-service-time.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-switch.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-thin-pool.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/dm-verity.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/linear.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/persistent-data 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/raid0.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/raid1.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/raid10.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/md/raid456.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptbase.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptctl.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptfc.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptlan.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptsas.ko 2026-02-25T11:35:33.051 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptscsih.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/message/fusion/mptspi.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/lpc_ich.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/lpc_sch.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/mfd-core.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/sm501.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/viperboard.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/mfd/vx855.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/altera-stapl 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/apds9802als.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/apds990x.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/bh1770glc.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/cb710 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/cb710/cb710.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/eeprom 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/eeprom/at24.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/eeprom/max6875.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/enclosure.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/hpilo.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/isl29003.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/isl29020.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/lis3lv02d 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2026-02-25T11:35:33.052 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/mei 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/mei/mei-me.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/mei/mei-txe.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/mei/mei.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-gru 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-gru/gru.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-xp 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-xp/xp.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-xp/xpc.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/sgi-xp/xpnet.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/tifm_7xx1.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/tifm_core.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/tsl2550.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/vmw_balloon.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/vmw_vmci 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/bonding 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/bonding/bonding.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/b53 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/b53/b53_common.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/bcm-sf2.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/mv88e6060.ko 2026-02-25T11:35:33.053 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/mv88e6xxx 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/dummy.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/eql.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com/3c59x.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/3com/typhoon.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390/8390.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/adaptec 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/adaptec/starfire.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/agere 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/agere/et131x.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/alteon 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/alteon/acenic.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/altera 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/altera/altera_tse.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/amd 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/amd/amd8111e.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2026-02-25T11:35:33.054 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/amd/pcnet32.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/alx 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atl1c 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atl1e 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atlx 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/b44.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/bnx2x 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/cnic.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/genet 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/broadcom/tg3.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/brocade 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/brocade/bna 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2026-02-25T11:35:33.055 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb3 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb4 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/libcxgb 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/cisco 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/cisco/enic 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dlink 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dlink/dl2k.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dlink/sundance.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/dnet.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/emulex 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/emulex/benet 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/ethoc.ko 2026-02-25T11:35:33.056 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/fealnx.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e100.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e1000 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e1000e 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/fm10k 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/i40e 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/iavf 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ice 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ice/ice.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/igb 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/igb/igb.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/igbvf 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ixgbe 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ixgbevf 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libeth 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libeth/libeth_xdp.ko 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libie 2026-02-25T11:35:33.057 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libie/libie.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libie/libie_adminq.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/intel/libie/libie_fwlog.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/jme.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/marvell 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/marvell/skge.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/marvell/sky2.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx4 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx5 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx5/core 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/micrel 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/myricom 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/myricom/myri10ge 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/natsemi 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/neterion 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/neterion/s2io.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/nvidia 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/packetengines 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2026-02-25T11:35:33.058 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/netxen 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/qlcnic 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/rdc 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/rdc/r6040.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek/8139cp.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek/8139too.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek/atp.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/realtek/r8169.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/rocker 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/rocker/rocker.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sfc 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sfc/sfc.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/silan 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/silan/sc92031.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sis 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sis/sis190.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sis/sis900.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc/epic100.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/stmicro 2026-02-25T11:35:33.059 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/stmicro/stmmac 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun/cassini.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun/niu.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun/sungem.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/sun/sunhme.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/tehuti 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/ti 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/ti/tlan.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/via 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/via/via-rhine.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/via/via-velocity.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/wiznet 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/wiznet/w5100.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/wiznet/w5300.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/xircom 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/geneve.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ieee802154 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ieee802154/fakelb.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ifb.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ipvlan 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ipvlan/ipvlan.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/macvlan.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/macvtap.ko 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio 2026-02-25T11:35:33.060 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio/mdio-bitbang.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mdio/mdio-regmap.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/mii.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/net_failover.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/netconsole.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/nlmon.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/pcs 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/pcs/pcs-lynx.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/pcs/pcs_xpcs.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/amd.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/bcm-phy-lib.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/bcm-phy-ptp.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/bcm7xxx.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/bcm87xx.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/broadcom.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/cicada.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/davicom.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/dp83640.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/et1011c.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/icplus.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/lxt.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/marvell.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/micrel.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/national.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/phy_package.ko 2026-02-25T11:35:33.061 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/phylink.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/qsemi.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/realtek 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/realtek/realtek.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/smsc.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/ste10Xp.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/phy/vitesse.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/bsd_comp.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_async.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_deflate.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_generic.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_mppe.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/ppp_synctty.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/pppoe.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/pppox.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/ppp/pptp.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/slip 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/slip/slhc.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/slip/slip.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/sungem_phy.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/tap.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_activebackup.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_broadcast.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_loadbalance.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_random.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/team/team_mode_roundrobin.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/tun.ko 2026-02-25T11:35:33.062 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/veth.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/virtio_net.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/vmxnet3 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/vmxnet3/vmxnet3.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/vxlan 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/vxlan/vxlan.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/xen-netback 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/xen-netback/xen-netback.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/net/xen-netfront.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ntb 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ntb/ntb.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/nvme 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/nvme/host 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/nvme/host/nvme-core.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/nvme/host/nvme.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport/parport.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport/parport_cs.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport/parport_pc.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/parport/parport_serial.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/hotplug 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/pcie 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/pcie/aer_inject.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pci/xen-pcifront.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pcmcia 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pcmcia/i82092.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pcmcia/pd6729.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pcmcia/yenta_socket.ko 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pinctrl 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pinctrl/intel 2026-02-25T11:35:33.063 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/wmi 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/wmi/wmi.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/acer-wmi.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/acerhdf.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/apple-gmux.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/asus-laptop.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/asus-nb-wmi.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/asus-wmi.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/classmate-laptop.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/eeepc-laptop.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/eeepc-wmi.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/fujitsu-laptop.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/fujitsu-tablet.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/hdaps.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/intel 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/intel/intel-rst.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/intel_ips.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/msi-wmi.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/mxm-wmi.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/panasonic-laptop.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/samsung-laptop.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/samsung-q10.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/topstar-laptop.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/toshiba_haps.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/platform/x86/wmi-bmof.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/powercap 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/powercap/intel_rapl_common.ko 2026-02-25T11:35:33.064 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/powercap/intel_rapl_msr.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/clients 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/clients/pps-gpio.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/clients/pps-ldisc.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/clients/pps_parport.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/pps/pps_core.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ptp 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ptp/ptp.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ptp/ptp_kvm.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ptp/ptp_vmclock.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras/amd 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras/amd/atl 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras/amd/atl/amd_atl.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ras/amd/fmpm.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-bq32k.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1286.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1307.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1374.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1511.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1553.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1672.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds1742.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds2404.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-ds3232.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-em3027.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-fm3130.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-isl12022.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-isl1208.ko 2026-02-25T11:35:33.065 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-m41t80.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-m48t35.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-m48t59.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-max6900.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-msm6242.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf2127.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf85063.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf8523.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf8563.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-pcf8583.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rp5c01.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rs5c372.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rv3029c2.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rx8025.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-rx8581.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-stk17ta8.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/rtc/rtc-x1205.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/3w-9xxx.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/3w-sas.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/3w-xxxx.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/BusLogic.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/a100u2w.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aacraid 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aacraid/aacraid.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/advansys.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aic7xxx 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/am53c974.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/arcmsr 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/arcmsr/arcmsr.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/atp870u.ko 2026-02-25T11:35:33.066 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/be2iscsi 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bfa 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bfa/bfa.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bnx2fc 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bnx2i 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/bnx2i/bnx2i.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/ch.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/csiostor 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/csiostor/csiostor.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/cxgb3i 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/cxgb4i 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/dc395x.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/dmx3191d.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/esas2r 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/esas2r/esas2r.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/esp_scsi.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fcoe 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fcoe/fcoe.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fcoe/libfcoe.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fnic 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/fnic/fnic.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/hpsa.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/hptiop.ko 2026-02-25T11:35:33.067 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/initio.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/ipr.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/ips.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/isci 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/isci/isci.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/iscsi_tcp.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libfc 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libfc/libfc.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libiscsi.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libiscsi_tcp.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libsas 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/libsas/libsas.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/lpfc 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/lpfc/lpfc.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mpt3sas 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mvsas 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mvsas/mvsas.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/mvumi.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/pm8001 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/pm8001/pm80xx.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/pmcraid.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla1280.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla2xxx 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla4xxx 2026-02-25T11:35:33.068 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/raid_class.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_debug.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_fc.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_iscsi.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_sas.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_spi.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/scsi_transport_srp.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/ses.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/st.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/stex.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/sym53c8xx_2 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/virtio_scsi.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/vmw_pvscsi.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/scsi/wd719x.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ssb 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ssb/ssb.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/iscsi 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/loopback 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/loopback/tcm_loop.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_file.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_iblock.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_mod.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_pscsi.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/target_core_user.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/tcm_fc 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/target/tcm_fc/tcm_fc.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2026-02-25T11:35:33.069 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/platform_temperature_control.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_soc_slider.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/ipwireless 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/ipwireless/ipwireless.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/n_gsm.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/n_hdlc.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/nozomi.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/8250 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/8250/serial_cs.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/arc_uart.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/jsm 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/tty/serial/jsm/jsm.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs/core 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs/core/ufshcd-core.ko 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs/host 2026-02-25T11:35:33.070 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/ufs/host/ufshcd-pci.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio_aec.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio_cif.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio_pci_generic.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/uio/uio_sercos3.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class/cdc-acm.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class/cdc-wdm.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class/usblp.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/class/usbtmc.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/host 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/host/sl811-hcd.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/image 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/image/mdc800.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/image/microtek.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/adutux.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/appledisplay.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/emi26.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/emi62.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/ezusb.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/idmouse.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/iowarrior.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/isight_firmware.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/ldusb.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/legousbtower.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/sisusbvga 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/trancevibrator.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/usb3503.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/usblcd.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/usbsevseg.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/uss720.ko 2026-02-25T11:35:33.071 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/misc/yurex.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/phy 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/phy/phy-generic.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/aircable.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ark3116.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/belkin_sa.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ch341.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/cp210x.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/cyberjack.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/cypress_m8.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/digi_acceleport.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/empeg.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ftdi_sio.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/garmin_gps.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/io_edgeport.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/io_ti.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ipaq.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ipw.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ir-usb.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/iuu_phoenix.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/keyspan.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/keyspan_pda.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/kl5kusb105.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/kobil_sct.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/mct_u232.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/mos7720.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/mos7840.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/navman.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/omninet.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/opticon.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/option.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/oti6858.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/pl2303.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/qcaux.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/qcserial.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/quatech2.ko 2026-02-25T11:35:33.072 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/safe_serial.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/sierra.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/spcp8x5.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ssu100.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/symbolserial.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/usb-serial-simple.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/usb_debug.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/usb_wwan.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/visor.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/whiteheat.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/serial/xsens_mt.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/uas.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-alauda.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-cypress.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-datafab.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-eneub6250.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-freecom.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-isd200.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-jumpshot.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-karma.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-onetouch.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-realtek.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-sddr09.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-sddr55.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/ums-usbat.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/storage/usb-storage.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/usbip 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/usbip/usbip-core.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/usbip/usbip-host.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/usb/usbip/vhci-hcd.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/pci 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/pci/vfio-pci-core.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/pci/vfio-pci.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/vfio.ko 2026-02-25T11:35:33.073 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vfio/vfio_iommu_type1.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost/vhost.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost/vhost_iotlb.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost/vhost_net.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/vhost/vhost_scsi.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video/backlight 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video/backlight/apple_bl.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video/backlight/lcd.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/video/backlight/platform_lcd.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_balloon.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_mmio.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_pci.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/virtio/virtio_ring.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/masters 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/masters/ds2482.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/masters/ds2490.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2406.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2408.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2413.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2423.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2431.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2433.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2780.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds2781.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_ds28e04.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_smem.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/slaves/w1_therm.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/w1/wire.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/alim1535_wdt.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/alim7101_wdt.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/f71808e_wdt.ko 2026-02-25T11:35:33.074 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/hpwdt.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/i6300esb.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/iTCO_vendor_support.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/iTCO_wdt.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/ib700wdt.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/ibmasr.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/ie6xx_wdt.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/it8712f_wdt.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/it87_wdt.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/machzwd.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/nv_tco.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/pcwd_pci.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/pcwd_usb.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/sch311x_wdt.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/softdog.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/sp5100_tco.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/via_wdt.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/w83627hf_wdt.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/w83877f_wdt.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/w83977f_wdt.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/wdt_pci.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/watchdog/xen_wdt.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-acpi-processor.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-evtchn.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-gntalloc.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-gntdev.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-pciback 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xen-privcmd.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xenfs 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/drivers/xen/xenfs/xenfs.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/9p 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/9p/9p.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/affs 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/affs/affs.ko 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/befs 2026-02-25T11:35:33.075 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/befs/befs.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/binfmt_misc.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/btrfs 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/btrfs/btrfs.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/cachefiles 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/cachefiles/cachefiles.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ceph 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ceph/ceph.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/coda 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/coda/coda.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/cramfs 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/cramfs/cramfs.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/dlm 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/dlm/dlm.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ecryptfs 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ecryptfs/ecryptfs.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/f2fs 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/f2fs/f2fs.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fat 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fat/fat.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fat/msdos.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fat/vfat.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fuse 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fuse/cuse.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/fuse/fuse.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/gfs2 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/gfs2/gfs2.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/hfs 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/hfs/hfs.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/hfsplus 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/hfsplus/hfsplus.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/isofs 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/isofs/isofs.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/jfs 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/jfs/jfs.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/lockd 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/lockd/lockd.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/minix 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/minix/minix.ko 2026-02-25T11:35:33.076 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/netfs 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/netfs/netfs.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/blocklayout 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/filelayout 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/flexfilelayout 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/nfs.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/nfsv3.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs/nfsv4.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs_common 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs_common/grace.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfs_common/nfs_acl.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfsd 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nfsd/nfsd.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nilfs2 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nilfs2/nilfs2.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-celtic.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-centeuro.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-croatian.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-cyrillic.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-gaelic.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-greek.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-iceland.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-inuit.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-roman.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-romanian.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/mac-turkish.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp1250.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp1251.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp1255.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp737.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp775.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp850.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp852.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp855.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp857.ko 2026-02-25T11:35:33.077 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp860.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp861.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp862.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp863.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp864.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp865.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp866.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp869.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp874.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp932.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp936.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp949.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_cp950.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_euc-jp.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-1.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-13.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-14.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-15.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-2.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-3.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-4.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-5.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-6.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-7.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_iso8859-9.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_koi8-r.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_koi8-ru.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_koi8-u.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_ucs2_utils.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/nls/nls_utf8.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/cluster 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/dlm 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/dlmfs 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/ocfs2.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/ocfs2_stack_user.ko 2026-02-25T11:35:33.078 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ocfs2/ocfs2_stackglue.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/overlayfs 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/overlayfs/overlay.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/pstore 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/pstore/ramoops.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/romfs 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/romfs/romfs.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb/client 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb/client/cifs.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb/common 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/smb/common/cifs_md4.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/squashfs 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/squashfs/squashfs.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/udf 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/udf/udf.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ufs 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/ufs/ufs.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/xfs 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/fs/xfs/xfs.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/rcu 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/rcu/rcutorture.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/torture.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/trace 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/kernel/trace/ring_buffer_benchmark.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/asn1_encoder.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crc 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crc/crc-itu-t.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crc/crc8.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crypto 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crypto/libarc4.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crypto/libblake2b.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/crypto/libdes.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/lru_cache.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/lz4 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/lz4/lz4_compress.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/lz4/lz4hc_compress.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/math 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/math/cordic.ko 2026-02-25T11:35:33.079 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/raid6 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/raid6/raid6_pq.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/reed_solomon 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/reed_solomon/reed_solomon.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/ts_bm.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/ts_fsm.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/ts_kmp.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/zstd 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/lib/zstd/zstd_compress.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/mm 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/mm/hwpoison-inject.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/6lowpan.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_dest.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_fragment.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_hop.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_ipv6.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_mobility.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_routing.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/6lowpan/nhc_udp.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802/garp.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802/mrp.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802/psnap.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/802/stp.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/8021q 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/8021q/8021q.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p/9pnet.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p/9pnet_fd.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p/9pnet_rdma.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/9p/9pnet_virtio.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/batman-adv 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/batman-adv/batman-adv.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/br_netfilter.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/bridge.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_802_3.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_among.ko 2026-02-25T11:35:33.080 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_arp.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_arpreply.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_dnat.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_ip.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_ip6.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_limit.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_log.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_mark.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_mark_m.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_nflog.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_pkttype.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_redirect.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_snat.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_stp.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/ebt_vlan.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ceph 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ceph/libceph.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/core 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/core/failover.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/core/pktgen.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dns_resolver 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dns_resolver/dns_resolver.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/dsa_core.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/tag_brcm.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/tag_dsa.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/tag_none.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/dsa/tag_trailer.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154/6lowpan 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154/ieee802154.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ieee802154/ieee802154_socket.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ah4.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/esp4.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/fou.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/gre.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/inet_diag.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ip_gre.ko 2026-02-25T11:35:33.081 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ip_tunnel.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ip_vti.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ipcomp.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/ipip.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/arpt_mangle.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_ECN.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_ah.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_bic.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_dctcp.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_diag.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_highspeed.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_htcp.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_hybla.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_illinois.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_lp.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_scalable.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_vegas.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_veno.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_westwood.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tcp_yeah.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/tunnel4.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/udp_diag.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/udp_tunnel.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv4/xfrm4_tunnel.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ah6.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/esp6.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/fou6.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ip6_tunnel.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ip6_udp_tunnel.ko 2026-02-25T11:35:33.082 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ip6_vti.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/ipcomp6.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_ah.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_frag.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_mh.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/ip6t_rt.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/sit.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/tunnel6.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/ipv6/xfrm6_tunnel.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/key 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/key/af_key.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_core.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_debugfs.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_eth.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_ip.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_ip6.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_netlink.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/l2tp/l2tp_ppp.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/llc 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/llc/llc.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/mac802154 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/mac802154/mac802154.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/mpls 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/mpls/mpls_gso.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2026-02-25T11:35:33.083 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipset/ip_set_list_set.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conncount.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_amanda.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_broadcast.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_ftp.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_h323.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_irc.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_netlink.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_pptp.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_sane.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_sip.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_snmp.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_conntrack_tftp.ko 2026-02-25T11:35:33.084 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_dup_netdev.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_flow_table.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_flow_table_inet.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_log_syslog.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_amanda.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_ftp.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_irc.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_sip.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_nat_tftp.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_synproxy_core.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nf_tables.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink_acct.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink_log.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink_osf.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nfnetlink_queue.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_chain_nat.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_compat.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_ct.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_dup_netdev.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_fib.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_fwd_netdev.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_hash.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_limit.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_log.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_masq.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_nat.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_numgen.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_queue.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_quota.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_redir.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_reject.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_reject_inet.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_socket.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/nft_tunnel.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_AUDIT.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_CHECKSUM.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_CLASSIFY.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_CONNSECMARK.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_CT.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_DSCP.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_HL.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_HMARK.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_IDLETIMER.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_LED.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_LOG.ko 2026-02-25T11:35:33.085 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_MASQUERADE.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_NETMAP.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_NFLOG.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_NFQUEUE.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_RATEEST.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_REDIRECT.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_SECMARK.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_TCPMSS.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_TEE.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_TPROXY.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_addrtype.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_bpf.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_cgroup.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_cluster.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_comment.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_connbytes.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_connlabel.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_connlimit.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_connmark.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_conntrack.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_cpu.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_dccp.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_devgroup.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_dscp.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_ecn.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_esp.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_hashlimit.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_helper.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_hl.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_ipcomp.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_iprange.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_ipvs.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_l2tp.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_length.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_limit.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_mac.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_mark.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_multiport.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_nat.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_nfacct.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_osf.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_owner.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_physdev.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_pkttype.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_policy.ko 2026-02-25T11:35:33.086 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_quota.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_rateest.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_realm.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_recent.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_sctp.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_set.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_socket.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_state.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_statistic.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_string.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_tcpmss.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_time.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netfilter/xt_u32.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netlink 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/netlink/netlink_diag.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/nsh 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/nsh/nsh.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch/openvswitch.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch/vport-geneve.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch/vport-gre.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/openvswitch/vport-vxlan.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/packet 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/packet/af_packet_diag.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/rds 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/rds/rds.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/rds/rds_rdma.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/rds/rds_tcp.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_csum.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_gact.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_mirred.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_nat.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_pedit.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_police.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_simple.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_skbedit.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/act_vlan.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_basic.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_bpf.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_flow.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_fw.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_route.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/cls_u32.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_cmp.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_ipset.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_meta.ko 2026-02-25T11:35:33.087 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_nbyte.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_text.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/em_u32.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_choke.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_codel.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_drr.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_fq.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_gred.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_hfsc.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_hhf.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_htb.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_ingress.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_mqprio.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_mqprio_lib.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_multiq.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_netem.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_pie.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_plug.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_prio.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_qfq.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_red.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_sfb.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_sfq.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_tbf.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sched/sch_teql.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sctp 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sctp/sctp.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sctp/sctp_diag.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/auth_gss 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/sunrpc.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/xprtrdma 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/tipc 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/tipc/tipc.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/tipc/tipc_diag.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/unix 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/unix/unix_diag.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vsock.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vsock_diag.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/vmw_vsock/vsock_loopback.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/xfrm 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/net/xfrm/xfrm_ipcomp.ko 2026-02-25T11:35:33.088 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys/encrypted-keys 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys/trusted-keys 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/security/keys/trusted-keys/trusted.ko 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/virt 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/virt/lib 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/kernel/virt/lib/irqbypass.ko 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.alias 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.alias.bin 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.builtin 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.builtin.alias.bin 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.builtin.bin 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.builtin.modinfo 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.dep 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.dep.bin 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.devname 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.order 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.softdep 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.symbols 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.symbols.bin 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/modules.weakdep 2026-02-25T11:35:33.089 INFO:teuthology.orchestra.run.trial167.stdout:/lib/modules/6.19.0-g712f4a43c125/vmlinuz 2026-02-25T11:35:33.091 DEBUG:teuthology.task.kernel:get_image_version: 6.19.0-g712f4a43c125 2026-02-25T11:35:33.091 DEBUG:teuthology.orchestra.run.trial167:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2026-02-25T11:35:33.111 INFO:teuthology.orchestra.run.trial167.stdout:postinstall scriptlet (using /bin/sh): 2026-02-25T11:35:33.111 INFO:teuthology.orchestra.run.trial167.stdout:if [ -x /usr/bin/kernel-install ]; then 2026-02-25T11:35:33.111 INFO:teuthology.orchestra.run.trial167.stdout: /usr/bin/kernel-install add 6.19.0-g712f4a43c125 /lib/modules/6.19.0-g712f4a43c125/vmlinuz 2026-02-25T11:35:33.111 INFO:teuthology.orchestra.run.trial167.stdout:fi 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout:for file in vmlinuz System.map config; do 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout: if ! cmp --silent "/lib/modules/6.19.0-g712f4a43c125/${file}" "/boot/${file}-6.19.0-g712f4a43c125"; then 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout: cp "/lib/modules/6.19.0-g712f4a43c125/${file}" "/boot/${file}-6.19.0-g712f4a43c125" 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout: fi 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout:done 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout:if [ -d "/lib/modules/6.19.0-g712f4a43c125/dtb" ] && \ 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout: ! diff -rq "/lib/modules/6.19.0-g712f4a43c125/dtb" "/boot/dtb-6.19.0-g712f4a43c125" >/dev/null 2>&1; then 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout: rm -rf "/boot/dtb-6.19.0-g712f4a43c125" 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout: cp -r "/lib/modules/6.19.0-g712f4a43c125/dtb" "/boot/dtb-6.19.0-g712f4a43c125" 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout:fi 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout:if [ ! -e "/lib/modules/6.19.0-g712f4a43c125/modules.dep" ]; then 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout: /usr/sbin/depmod 6.19.0-g712f4a43c125 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout:fi 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout:preuninstall scriptlet (using /bin/sh): 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout:if [ -x /usr/bin/kernel-install ]; then 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout:kernel-install remove 6.19.0-g712f4a43c125 2026-02-25T11:35:33.112 INFO:teuthology.orchestra.run.trial167.stdout:fi 2026-02-25T11:35:33.112 DEBUG:teuthology.orchestra.run.trial167:> sudo rpm -qi grub2-tools 2026-02-25T11:35:33.182 INFO:teuthology.orchestra.run.trial167.stdout:Name : grub2-tools 2026-02-25T11:35:33.182 INFO:teuthology.orchestra.run.trial167.stdout:Epoch : 1 2026-02-25T11:35:33.182 INFO:teuthology.orchestra.run.trial167.stdout:Version : 2.06 2026-02-25T11:35:33.182 INFO:teuthology.orchestra.run.trial167.stdout:Release : 120.el9 2026-02-25T11:35:33.182 INFO:teuthology.orchestra.run.trial167.stdout:Architecture: x86_64 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:Group : Unspecified 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:Size : 9108611 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:License : GPLv3+ 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:Packager : builder@centos.org 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:Vendor : CentOS 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:Summary : Support tools for GRUB. 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:Description : 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout:hardware devices. 2026-02-25T11:35:33.183 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:35:33.184 INFO:teuthology.orchestra.run.trial167.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T11:35:33.184 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T11:35:33.184 INFO:teuthology.task.kernel:Updating grub on trial167 to boot 6.19.0-g712f4a43c125 2026-02-25T11:35:33.184 DEBUG:teuthology.orchestra.run.trial167:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T11:35:33.249 INFO:teuthology.orchestra.run.trial167.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T11:35:33.251 DEBUG:teuthology.orchestra.run.trial167:> sudo command -v grubby && echo yes || echo no 2026-02-25T11:35:33.316 INFO:teuthology.orchestra.run.trial167.stdout:/sbin/grubby 2026-02-25T11:35:33.317 INFO:teuthology.orchestra.run.trial167.stdout:yes 2026-02-25T11:35:33.317 DEBUG:teuthology.orchestra.run.trial167:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "6.19.0-g712f4a43c125" | head -n 1 2026-02-25T11:35:33.383 INFO:teuthology.orchestra.run.trial167.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-6.19.0-g712f4a43c125.conf 2026-02-25T11:35:33.384 DEBUG:teuthology.orchestra.run.trial167:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-6.19.0-g712f4a43c125 2026-02-25T11:35:33.458 DEBUG:teuthology.orchestra.run.trial167:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T11:35:33.666 INFO:teuthology.orchestra.run.trial167.stderr:Generating grub configuration file ... 2026-02-25T11:35:33.734 INFO:teuthology.orchestra.run.trial167.stderr:Found linux image: /boot/vmlinuz-6.19.0-g712f4a43c125 2026-02-25T11:35:33.741 INFO:teuthology.orchestra.run.trial167.stderr:Found initrd image: /boot/initramfs-6.19.0-g712f4a43c125.img 2026-02-25T11:35:33.994 INFO:teuthology.orchestra.run.trial167.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T11:35:34.001 INFO:teuthology.orchestra.run.trial167.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T11:35:34.035 INFO:teuthology.orchestra.run.trial167.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T11:35:34.041 INFO:teuthology.orchestra.run.trial167.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T11:35:34.072 INFO:teuthology.orchestra.run.trial167.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T11:35:34.078 INFO:teuthology.orchestra.run.trial167.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T11:35:34.101 INFO:teuthology.orchestra.run.trial167.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T11:35:34.107 INFO:teuthology.orchestra.run.trial167.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T11:35:34.192 INFO:teuthology.orchestra.run.trial167.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T11:35:34.200 INFO:teuthology.orchestra.run.trial167.stderr:done 2026-02-25T11:35:34.202 DEBUG:teuthology.orchestra.run.trial167:> test -d /sys/firmware/efi 2026-02-25T11:35:34.219 DEBUG:teuthology.orchestra.run.trial167:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T11:35:34.283 INFO:teuthology.orchestra.run.trial167.stdout:/boot/efi/EFI/BOOT 2026-02-25T11:35:34.283 INFO:teuthology.orchestra.run.trial167.stdout:/boot/efi/EFI/centos 2026-02-25T11:35:34.285 DEBUG:teuthology.orchestra.run.trial167:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T11:35:34.353 INFO:teuthology.orchestra.run.trial167.stdout:no 2026-02-25T11:35:34.353 DEBUG:teuthology.orchestra.run.trial167:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T11:35:34.418 INFO:teuthology.orchestra.run.trial167.stdout:yes 2026-02-25T11:35:34.418 DEBUG:teuthology.orchestra.run.trial167:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T11:35:34.482 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial167 2026-02-25T11:35:34.483 DEBUG:teuthology.orchestra.run.trial167:> sudo bash -c ' 2026-02-25T11:35:34.483 DEBUG:teuthology.orchestra.run.trial167:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T11:35:34.483 DEBUG:teuthology.orchestra.run.trial167:> ' 2026-02-25T11:35:57.952 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-25T11:35:57.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-25T11:35:57.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T11:36:00.077 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2026-02-25T11:36:00.078 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial119.front.sepia.ceph.com' 2026-02-25T11:36:00.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T11:36:04.525 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2026-02-25T11:36:04.526 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial167.front.sepia.ceph.com' 2026-02-25T11:36:04.526 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T11:36:12.224 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.28 2026-02-25T11:36:14.336 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.119 2026-02-25T11:36:18.816 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.167 2026-02-25T11:36:21.227 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-25T11:36:21.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T11:36:23.339 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial119.front.sepia.ceph.com' 2026-02-25T11:36:23.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T11:36:27.820 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial167.front.sepia.ceph.com' 2026-02-25T11:36:27.821 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T11:36:57.109 DEBUG:teuthology.orchestra.run.trial028:> true 2026-02-25T11:36:57.355 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-25T11:36:57.355 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.19.0-g712f4a43c125"... 2026-02-25T11:36:57.355 DEBUG:teuthology.orchestra.run.trial028:> uname -r 2026-02-25T11:36:57.433 INFO:teuthology.orchestra.run.trial028.stdout:6.19.0-g712f4a43c125 2026-02-25T11:36:57.433 DEBUG:teuthology.task.kernel:current kernel version is 6.19.0-g712f4a43c125 vs 6.19.0-g712f4a43c125 2026-02-25T11:36:57.434 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T11:36:57.434 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T11:36:58.434 DEBUG:teuthology.parallel:result is None 2026-02-25T11:36:59.164 DEBUG:teuthology.orchestra.run.trial119:> true 2026-02-25T11:36:59.421 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial119.front.sepia.ceph.com' 2026-02-25T11:36:59.421 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.19.0-g712f4a43c125"... 2026-02-25T11:36:59.421 DEBUG:teuthology.orchestra.run.trial119:> uname -r 2026-02-25T11:36:59.499 INFO:teuthology.orchestra.run.trial119.stdout:6.19.0-g712f4a43c125 2026-02-25T11:36:59.499 DEBUG:teuthology.task.kernel:current kernel version is 6.19.0-g712f4a43c125 vs 6.19.0-g712f4a43c125 2026-02-25T11:36:59.499 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T11:36:59.499 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T11:37:00.501 DEBUG:teuthology.parallel:result is None 2026-02-25T11:37:03.761 DEBUG:teuthology.orchestra.run.trial167:> true 2026-02-25T11:37:04.030 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial167.front.sepia.ceph.com' 2026-02-25T11:37:04.030 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.19.0-g712f4a43c125"... 2026-02-25T11:37:04.030 DEBUG:teuthology.orchestra.run.trial167:> uname -r 2026-02-25T11:37:04.105 INFO:teuthology.orchestra.run.trial167.stdout:6.19.0-g712f4a43c125 2026-02-25T11:37:04.106 DEBUG:teuthology.task.kernel:current kernel version is 6.19.0-g712f4a43c125 vs 6.19.0-g712f4a43c125 2026-02-25T11:37:04.106 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T11:37:04.106 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T11:37:05.107 DEBUG:teuthology.parallel:result is None 2026-02-25T11:37:05.107 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-25T11:37:05.119 INFO:teuthology.task.internal:Creating test directory... 2026-02-25T11:37:05.120 DEBUG:teuthology.orchestra.run.trial028:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T11:37:05.124 DEBUG:teuthology.orchestra.run.trial119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T11:37:05.129 DEBUG:teuthology.orchestra.run.trial167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T11:37:05.179 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-25T11:37:05.194 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-25T11:37:05.209 INFO:teuthology.task.internal:Creating archive directory... 2026-02-25T11:37:05.209 DEBUG:teuthology.orchestra.run.trial028:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T11:37:05.213 DEBUG:teuthology.orchestra.run.trial119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T11:37:05.216 DEBUG:teuthology.orchestra.run.trial167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T11:37:05.271 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-25T11:37:05.283 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-25T11:37:05.283 DEBUG:teuthology.orchestra.run.trial028:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T11:37:05.291 DEBUG:teuthology.orchestra.run.trial119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T11:37:05.295 DEBUG:teuthology.orchestra.run.trial167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T11:37:05.350 INFO:teuthology.orchestra.run.trial028.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T11:37:05.356 INFO:teuthology.orchestra.run.trial119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T11:37:05.368 INFO:teuthology.orchestra.run.trial167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T11:37:05.370 INFO:teuthology.orchestra.run.trial028.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T11:37:05.377 INFO:teuthology.orchestra.run.trial119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T11:37:05.389 INFO:teuthology.orchestra.run.trial167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T11:37:05.392 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-25T11:37:05.404 INFO:teuthology.task.internal:Configuring sudo... 2026-02-25T11:37:05.404 DEBUG:teuthology.orchestra.run.trial028:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T11:37:05.408 DEBUG:teuthology.orchestra.run.trial119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T11:37:05.411 DEBUG:teuthology.orchestra.run.trial167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T11:37:05.472 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-25T11:37:05.488 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-25T11:37:05.488 DEBUG:teuthology.orchestra.run.trial028:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T11:37:05.492 DEBUG:teuthology.orchestra.run.trial119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T11:37:05.496 DEBUG:teuthology.orchestra.run.trial167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T11:37:05.534 DEBUG:teuthology.orchestra.run.trial028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T11:37:05.711 DEBUG:teuthology.orchestra.run.trial028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T11:37:05.767 DEBUG:teuthology.orchestra.run.trial028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T11:37:05.908 DEBUG:teuthology.orchestra.run.trial028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T11:37:05.963 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:37:05.963 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T11:37:06.018 DEBUG:teuthology.orchestra.run.trial119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T11:37:06.160 DEBUG:teuthology.orchestra.run.trial119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T11:37:06.215 DEBUG:teuthology.orchestra.run.trial119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T11:37:06.350 DEBUG:teuthology.orchestra.run.trial119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T11:37:06.403 DEBUG:teuthology.orchestra.run.trial119:> set -ex 2026-02-25T11:37:06.403 DEBUG:teuthology.orchestra.run.trial119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T11:37:06.460 DEBUG:teuthology.orchestra.run.trial167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T11:37:06.605 DEBUG:teuthology.orchestra.run.trial167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T11:37:06.663 DEBUG:teuthology.orchestra.run.trial167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T11:37:06.799 DEBUG:teuthology.orchestra.run.trial167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T11:37:06.856 DEBUG:teuthology.orchestra.run.trial167:> set -ex 2026-02-25T11:37:06.856 DEBUG:teuthology.orchestra.run.trial167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T11:37:06.912 DEBUG:teuthology.orchestra.run.trial028:> sudo service rsyslog restart 2026-02-25T11:37:06.915 DEBUG:teuthology.orchestra.run.trial119:> sudo service rsyslog restart 2026-02-25T11:37:06.919 DEBUG:teuthology.orchestra.run.trial167:> sudo service rsyslog restart 2026-02-25T11:37:06.977 INFO:teuthology.orchestra.run.trial028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T11:37:06.980 INFO:teuthology.orchestra.run.trial119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T11:37:07.017 INFO:teuthology.orchestra.run.trial167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T11:37:07.512 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-25T11:37:07.527 INFO:teuthology.task.internal:Starting timer... 2026-02-25T11:37:07.527 INFO:teuthology.run_tasks:Running task pcp... 2026-02-25T11:37:07.551 INFO:teuthology.run_tasks:Running task selinux... 2026-02-25T11:37:07.569 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2026-02-25T11:37:07.569 DEBUG:teuthology.orchestra.run.trial028:> sudo service auditd rotate 2026-02-25T11:37:07.637 INFO:teuthology.orchestra.run.trial028.stdout:Rotating logs: 2026-02-25T11:37:07.639 DEBUG:teuthology.orchestra.run.trial119:> sudo service auditd rotate 2026-02-25T11:37:07.706 INFO:teuthology.orchestra.run.trial119.stdout:Rotating logs: 2026-02-25T11:37:07.709 DEBUG:teuthology.orchestra.run.trial167:> sudo service auditd rotate 2026-02-25T11:37:07.776 INFO:teuthology.orchestra.run.trial167.stdout:Rotating logs: 2026-02-25T11:37:07.779 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-25T11:37:07.779 DEBUG:teuthology.orchestra.run.trial028:> /usr/sbin/getenforce 2026-02-25T11:37:07.818 INFO:teuthology.orchestra.run.trial028.stdout:Permissive 2026-02-25T11:37:07.818 DEBUG:teuthology.orchestra.run.trial119:> /usr/sbin/getenforce 2026-02-25T11:37:07.862 INFO:teuthology.orchestra.run.trial119.stdout:Permissive 2026-02-25T11:37:07.862 DEBUG:teuthology.orchestra.run.trial167:> /usr/sbin/getenforce 2026-02-25T11:37:07.902 INFO:teuthology.orchestra.run.trial167.stdout:Permissive 2026-02-25T11:37:07.902 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial028.front.sepia.ceph.com': 'permissive', 'ubuntu@trial119.front.sepia.ceph.com': 'permissive', 'ubuntu@trial167.front.sepia.ceph.com': 'permissive'} 2026-02-25T11:37:07.903 DEBUG:teuthology.orchestra.run.trial028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2026-02-25T11:37:07.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T11:37:07.962 DEBUG:teuthology.orchestra.run.trial119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2026-02-25T11:37:08.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T11:37:08.019 DEBUG:teuthology.orchestra.run.trial167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2026-02-25T11:37:08.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T11:37:08.078 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-25T11:37:08.079 DEBUG:teuthology.orchestra.run.trial028:> sudo /usr/sbin/setenforce permissive 2026-02-25T11:37:08.134 DEBUG:teuthology.orchestra.run.trial119:> sudo /usr/sbin/setenforce permissive 2026-02-25T11:37:08.191 DEBUG:teuthology.orchestra.run.trial167:> sudo /usr/sbin/setenforce permissive 2026-02-25T11:37:08.247 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-25T11:37:08.262 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-25T11:37:08.273 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-25T11:37:08.274 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial028.front.sepia.ceph.com,trial119.front.sepia.ceph.com,trial167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-25T11:41:15.462 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial028.front.sepia.ceph.com'), Remote(name='ubuntu@trial119.front.sepia.ceph.com'), Remote(name='ubuntu@trial167.front.sepia.ceph.com')] 2026-02-25T11:41:15.468 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-25T11:41:15.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T11:41:15.539 DEBUG:teuthology.orchestra.run.trial028:> true 2026-02-25T11:41:15.639 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-25T11:41:15.639 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial119.front.sepia.ceph.com' 2026-02-25T11:41:15.640 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T11:41:15.738 DEBUG:teuthology.orchestra.run.trial119:> true 2026-02-25T11:41:15.844 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial119.front.sepia.ceph.com' 2026-02-25T11:41:15.844 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial167.front.sepia.ceph.com' 2026-02-25T11:41:15.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T11:41:15.962 DEBUG:teuthology.orchestra.run.trial167:> true 2026-02-25T11:41:16.065 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial167.front.sepia.ceph.com' 2026-02-25T11:41:16.067 INFO:teuthology.run_tasks:Running task clock... 2026-02-25T11:41:16.081 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-25T11:41:16.081 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T11:41:16.082 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T11:41:16.085 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T11:41:16.086 DEBUG:teuthology.orchestra.run.trial119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T11:41:16.089 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T11:41:16.090 DEBUG:teuthology.orchestra.run.trial167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T11:41:16.141 INFO:teuthology.orchestra.run.trial028.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T11:41:16.146 INFO:teuthology.orchestra.run.trial119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T11:41:16.166 INFO:teuthology.orchestra.run.trial028.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T11:41:16.167 INFO:teuthology.orchestra.run.trial167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T11:41:16.171 INFO:teuthology.orchestra.run.trial119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T11:41:16.193 INFO:teuthology.orchestra.run.trial167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T11:41:16.209 INFO:teuthology.orchestra.run.trial028.stderr:sudo: ntpd: command not found 2026-02-25T11:41:16.214 INFO:teuthology.orchestra.run.trial119.stderr:sudo: ntpd: command not found 2026-02-25T11:41:16.230 INFO:teuthology.orchestra.run.trial028.stdout:506 Cannot talk to daemon 2026-02-25T11:41:16.236 INFO:teuthology.orchestra.run.trial119.stdout:506 Cannot talk to daemon 2026-02-25T11:41:16.237 INFO:teuthology.orchestra.run.trial167.stderr:sudo: ntpd: command not found 2026-02-25T11:41:16.256 INFO:teuthology.orchestra.run.trial028.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T11:41:16.259 INFO:teuthology.orchestra.run.trial167.stdout:506 Cannot talk to daemon 2026-02-25T11:41:16.261 INFO:teuthology.orchestra.run.trial119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T11:41:16.280 INFO:teuthology.orchestra.run.trial028.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T11:41:16.283 INFO:teuthology.orchestra.run.trial167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T11:41:16.285 INFO:teuthology.orchestra.run.trial119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T11:41:16.308 INFO:teuthology.orchestra.run.trial167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T11:41:16.358 INFO:teuthology.orchestra.run.trial028.stderr:bash: line 1: ntpq: command not found 2026-02-25T11:41:16.360 INFO:teuthology.orchestra.run.trial119.stderr:bash: line 1: ntpq: command not found 2026-02-25T11:41:16.383 INFO:teuthology.orchestra.run.trial028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T11:41:16.383 INFO:teuthology.orchestra.run.trial028.stdout:=============================================================================== 2026-02-25T11:41:16.383 INFO:teuthology.orchestra.run.trial028.stdout:^? 23.186.168.130 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T11:41:16.383 INFO:teuthology.orchestra.run.trial028.stdout:^? 155.248.196.28 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T11:41:16.383 INFO:teuthology.orchestra.run.trial028.stdout:^? 172-233-155-39.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T11:41:16.383 INFO:teuthology.orchestra.run.trial028.stdout:^? lax1.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T11:41:16.383 INFO:teuthology.orchestra.run.trial119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T11:41:16.383 INFO:teuthology.orchestra.run.trial119.stdout:=============================================================================== 2026-02-25T11:41:16.383 INFO:teuthology.orchestra.run.trial119.stdout:^? 23.186.168.130 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T11:41:16.383 INFO:teuthology.orchestra.run.trial119.stdout:^? 155.248.196.28 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T11:41:16.383 INFO:teuthology.orchestra.run.trial119.stdout:^? 172-233-155-39.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T11:41:16.384 INFO:teuthology.orchestra.run.trial119.stdout:^? lax1.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T11:41:16.388 INFO:teuthology.orchestra.run.trial167.stderr:bash: line 1: ntpq: command not found 2026-02-25T11:41:16.393 INFO:teuthology.orchestra.run.trial167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T11:41:16.393 INFO:teuthology.orchestra.run.trial167.stdout:=============================================================================== 2026-02-25T11:41:16.393 INFO:teuthology.orchestra.run.trial167.stdout:^? 23.186.168.130 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T11:41:16.393 INFO:teuthology.orchestra.run.trial167.stdout:^? 155.248.196.28 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T11:41:16.393 INFO:teuthology.orchestra.run.trial167.stdout:^? 172-233-155-39.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T11:41:16.393 INFO:teuthology.orchestra.run.trial167.stdout:^? lax1.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T11:41:16.394 INFO:teuthology.run_tasks:Running task install... 2026-02-25T11:41:16.406 DEBUG:teuthology.task.install:project ceph 2026-02-25T11:41:16.406 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '92715cd3c42198664811edeb975bd585312154aa'}} 2026-02-25T11:41:16.406 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '92715cd3c42198664811edeb975bd585312154aa'} 2026-02-25T11:41:16.406 INFO:teuthology.task.install:Using flavor: default 2026-02-25T11:41:16.411 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-25T11:41:16.411 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2026-02-25T11:41:16.412 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '92715cd3c42198664811edeb975bd585312154aa', 'tag': None, 'wait_for_package': False} 2026-02-25T11:41:16.412 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=92715cd3c42198664811edeb975bd585312154aa 2026-02-25T11:41:16.413 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '92715cd3c42198664811edeb975bd585312154aa', 'tag': None, 'wait_for_package': False} 2026-02-25T11:41:16.414 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=92715cd3c42198664811edeb975bd585312154aa 2026-02-25T11:41:16.414 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '92715cd3c42198664811edeb975bd585312154aa', 'tag': None, 'wait_for_package': False} 2026-02-25T11:41:16.415 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=92715cd3c42198664811edeb975bd585312154aa 2026-02-25T11:41:16.633 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/ 2026-02-25T11:41:16.633 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5549.g92715cd3 2026-02-25T11:41:16.635 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/ 2026-02-25T11:41:16.635 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5549.g92715cd3 2026-02-25T11:41:16.641 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/ 2026-02-25T11:41:16.641 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5549.g92715cd3 2026-02-25T11:41:16.825 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T11:41:16.826 DEBUG:teuthology.orchestra.run.trial167:> set -ex 2026-02-25T11:41:16.826 DEBUG:teuthology.orchestra.run.trial167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T11:41:16.826 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T11:41:16.826 DEBUG:teuthology.orchestra.run.trial119:> set -ex 2026-02-25T11:41:16.826 DEBUG:teuthology.orchestra.run.trial119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T11:41:16.835 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T11:41:16.835 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:41:16.835 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T11:41:16.881 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2026-02-25T11:41:16.881 DEBUG:teuthology.orchestra.run.trial167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/92715cd3c42198664811edeb975bd585312154aa/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T11:41:16.882 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2026-02-25T11:41:16.882 DEBUG:teuthology.orchestra.run.trial119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/92715cd3c42198664811edeb975bd585312154aa/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T11:41:16.892 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2026-02-25T11:41:16.892 DEBUG:teuthology.orchestra.run.trial028:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/92715cd3c42198664811edeb975bd585312154aa/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T11:41:16.978 DEBUG:teuthology.orchestra.run.trial167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T11:41:16.979 DEBUG:teuthology.orchestra.run.trial119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T11:41:16.990 DEBUG:teuthology.orchestra.run.trial028:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T11:41:17.054 DEBUG:teuthology.orchestra.run.trial119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T11:41:17.056 DEBUG:teuthology.orchestra.run.trial167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T11:41:17.064 DEBUG:teuthology.orchestra.run.trial028:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T11:41:17.110 INFO:teuthology.orchestra.run.trial119.stdout:check_obsoletes = 1 2026-02-25T11:41:17.111 INFO:teuthology.orchestra.run.trial167.stdout:check_obsoletes = 1 2026-02-25T11:41:17.113 DEBUG:teuthology.orchestra.run.trial119:> sudo yum clean all 2026-02-25T11:41:17.114 DEBUG:teuthology.orchestra.run.trial167:> sudo yum clean all 2026-02-25T11:41:17.120 INFO:teuthology.orchestra.run.trial028.stdout:check_obsoletes = 1 2026-02-25T11:41:17.122 DEBUG:teuthology.orchestra.run.trial028:> sudo yum clean all 2026-02-25T11:41:17.281 INFO:teuthology.orchestra.run.trial119.stdout:41 files removed 2026-02-25T11:41:17.283 INFO:teuthology.orchestra.run.trial167.stdout:41 files removed 2026-02-25T11:41:17.294 INFO:teuthology.orchestra.run.trial028.stdout:41 files removed 2026-02-25T11:41:17.299 DEBUG:teuthology.orchestra.run.trial119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2026-02-25T11:41:17.301 DEBUG:teuthology.orchestra.run.trial167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2026-02-25T11:41:17.313 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2026-02-25T11:41:17.671 INFO:teuthology.orchestra.run.trial119.stdout:ceph packages for x86_64 684 kB/s | 91 kB 00:00 2026-02-25T11:41:17.672 INFO:teuthology.orchestra.run.trial167.stdout:ceph packages for x86_64 697 kB/s | 91 kB 00:00 2026-02-25T11:41:17.679 INFO:teuthology.orchestra.run.trial028.stdout:ceph packages for x86_64 716 kB/s | 91 kB 00:00 2026-02-25T11:41:17.813 INFO:teuthology.orchestra.run.trial119.stdout:ceph noarch packages 208 kB/s | 24 kB 00:00 2026-02-25T11:41:17.814 INFO:teuthology.orchestra.run.trial167.stdout:ceph noarch packages 206 kB/s | 24 kB 00:00 2026-02-25T11:41:17.820 INFO:teuthology.orchestra.run.trial028.stdout:ceph noarch packages 209 kB/s | 24 kB 00:00 2026-02-25T11:41:17.940 INFO:teuthology.orchestra.run.trial119.stdout:ceph source packages 20 kB/s | 2.1 kB 00:00 2026-02-25T11:41:17.942 INFO:teuthology.orchestra.run.trial167.stdout:ceph source packages 20 kB/s | 2.1 kB 00:00 2026-02-25T11:41:17.947 INFO:teuthology.orchestra.run.trial028.stdout:ceph source packages 20 kB/s | 2.1 kB 00:00 2026-02-25T11:41:19.336 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.9 MB 00:01 2026-02-25T11:41:19.405 INFO:teuthology.orchestra.run.trial167.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.9 MB 00:01 2026-02-25T11:41:20.338 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - AppStream 62 MB/s | 27 MB 00:00 2026-02-25T11:41:22.612 INFO:teuthology.orchestra.run.trial167.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-25T11:41:23.305 INFO:teuthology.orchestra.run.trial119.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.9 MB 00:05 2026-02-25T11:41:24.485 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - CRB 5.4 MB/s | 8.0 MB 00:01 2026-02-25T11:41:25.668 INFO:teuthology.orchestra.run.trial167.stdout:CentOS Stream 9 - CRB 24 MB/s | 8.0 MB 00:00 2026-02-25T11:41:26.436 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 20 kB 00:01 2026-02-25T11:41:26.474 INFO:teuthology.orchestra.run.trial119.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-25T11:41:26.893 INFO:teuthology.orchestra.run.trial028.stdout:Extra Packages for Enterprise Linux 52 MB/s | 20 MB 00:00 2026-02-25T11:41:27.392 INFO:teuthology.orchestra.run.trial167.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 20 kB 00:00 2026-02-25T11:41:28.415 INFO:teuthology.orchestra.run.trial167.stdout:Extra Packages for Enterprise Linux 21 MB/s | 20 MB 00:00 2026-02-25T11:41:30.064 INFO:teuthology.orchestra.run.trial119.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 8.0 MB 00:00 2026-02-25T11:41:31.062 INFO:teuthology.orchestra.run.trial119.stdout:CentOS Stream 9 - Extras packages 123 kB/s | 20 kB 00:00 2026-02-25T11:41:31.098 INFO:teuthology.orchestra.run.trial028.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-25T11:41:31.583 INFO:teuthology.orchestra.run.trial119.stdout:Extra Packages for Enterprise Linux 45 MB/s | 20 MB 00:00 2026-02-25T11:41:32.366 INFO:teuthology.orchestra.run.trial028.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-25T11:41:32.367 INFO:teuthology.orchestra.run.trial028.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-25T11:41:32.373 INFO:teuthology.orchestra.run.trial028.stdout:Package NetworkManager-1:1.54.3-2.el9.x86_64 is already installed. 2026-02-25T11:41:32.374 INFO:teuthology.orchestra.run.trial028.stdout:Package iproute-6.17.0-2.el9.x86_64 is already installed. 2026-02-25T11:41:32.374 INFO:teuthology.orchestra.run.trial028.stdout:Package util-linux-2.37.4-21.el9.x86_64 is already installed. 2026-02-25T11:41:32.374 INFO:teuthology.orchestra.run.trial028.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2026-02-25T11:41:32.375 INFO:teuthology.orchestra.run.trial028.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2026-02-25T11:41:32.376 INFO:teuthology.orchestra.run.trial028.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2026-02-25T11:41:32.377 INFO:teuthology.orchestra.run.trial028.stdout:Package xfsprogs-6.4.0-7.el9.x86_64 is already installed. 2026-02-25T11:41:32.407 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-25T11:41:32.412 INFO:teuthology.orchestra.run.trial028.stdout:======================================================================================= 2026-02-25T11:41:32.412 INFO:teuthology.orchestra.run.trial028.stdout: Package Arch Version Repository Size 2026-02-25T11:41:32.412 INFO:teuthology.orchestra.run.trial028.stdout:======================================================================================= 2026-02-25T11:41:32.412 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-25T11:41:32.413 INFO:teuthology.orchestra.run.trial028.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2026-02-25T11:41:32.413 INFO:teuthology.orchestra.run.trial028.stdout: ceph x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 6.5 k 2026-02-25T11:41:32.413 INFO:teuthology.orchestra.run.trial028.stdout: ceph-base x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 5.2 M 2026-02-25T11:41:32.413 INFO:teuthology.orchestra.run.trial028.stdout: ceph-fuse x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 987 k 2026-02-25T11:41:32.413 INFO:teuthology.orchestra.run.trial028.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 147 k 2026-02-25T11:41:32.413 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 951 k 2026-02-25T11:41:32.413 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 184 k 2026-02-25T11:41:32.413 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 15 M 2026-02-25T11:41:32.413 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 7.4 M 2026-02-25T11:41:32.413 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-rook noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 50 k 2026-02-25T11:41:32.413 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 167 k 2026-02-25T11:41:32.413 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-classic x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 16 M 2026-02-25T11:41:32.414 INFO:teuthology.orchestra.run.trial028.stdout: ceph-radosgw x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 13 M 2026-02-25T11:41:32.414 INFO:teuthology.orchestra.run.trial028.stdout: ceph-test x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 76 M 2026-02-25T11:41:32.414 INFO:teuthology.orchestra.run.trial028.stdout: ceph-volume noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 301 k 2026-02-25T11:41:32.414 INFO:teuthology.orchestra.run.trial028.stdout: cephadm noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 1.0 M 2026-02-25T11:41:32.414 INFO:teuthology.orchestra.run.trial028.stdout: cephfs-mirror x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 257 k 2026-02-25T11:41:32.414 INFO:teuthology.orchestra.run.trial028.stdout: cephfs-shell x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 27 k 2026-02-25T11:41:32.414 INFO:teuthology.orchestra.run.trial028.stdout: cephfs-top noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 18 k 2026-02-25T11:41:32.414 INFO:teuthology.orchestra.run.trial028.stdout: elfutils-libelf-devel x86_64 0.194-1.el9 appstream 79 k 2026-02-25T11:41:32.414 INFO:teuthology.orchestra.run.trial028.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2026-02-25T11:41:32.414 INFO:teuthology.orchestra.run.trial028.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2026-02-25T11:41:32.414 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-devel x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 37 k 2026-02-25T11:41:32.414 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs2 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 908 k 2026-02-25T11:41:32.415 INFO:teuthology.orchestra.run.trial028.stdout: librados-devel x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 123 k 2026-02-25T11:41:32.415 INFO:teuthology.orchestra.run.trial028.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2026-02-25T11:41:32.415 INFO:teuthology.orchestra.run.trial028.stdout: libuuid-devel x86_64 2.37.4-25.el9 appstream 23 k 2026-02-25T11:41:32.415 INFO:teuthology.orchestra.run.trial028.stdout: openssl-devel x86_64 1:3.5.5-1.el9 appstream 4.8 M 2026-02-25T11:41:32.415 INFO:teuthology.orchestra.run.trial028.stdout: postgresql x86_64 13.23-1.el9 appstream 1.6 M 2026-02-25T11:41:32.415 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-contrib x86_64 13.23-1.el9 appstream 903 k 2026-02-25T11:41:32.415 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-server x86_64 13.23-1.el9 appstream 5.8 M 2026-02-25T11:41:32.415 INFO:teuthology.orchestra.run.trial028.stdout: python3-cephfs x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 204 k 2026-02-25T11:41:32.415 INFO:teuthology.orchestra.run.trial028.stdout: python3-rados x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 324 k 2026-02-25T11:41:32.415 INFO:teuthology.orchestra.run.trial028.stdout: python3-rbd x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 305 k 2026-02-25T11:41:32.416 INFO:teuthology.orchestra.run.trial028.stdout: python3-rgw x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 100 k 2026-02-25T11:41:32.416 INFO:teuthology.orchestra.run.trial028.stdout: rbd-fuse x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 84 k 2026-02-25T11:41:32.416 INFO:teuthology.orchestra.run.trial028.stdout: rbd-mirror x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 2.9 M 2026-02-25T11:41:32.416 INFO:teuthology.orchestra.run.trial028.stdout: rbd-nbd x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 172 k 2026-02-25T11:41:32.416 INFO:teuthology.orchestra.run.trial028.stdout: xfsprogs-devel x86_64 6.4.0-7.el9 appstream 96 k 2026-02-25T11:41:32.416 INFO:teuthology.orchestra.run.trial028.stdout:Upgrading: 2026-02-25T11:41:32.416 INFO:teuthology.orchestra.run.trial028.stdout: libblkid x86_64 2.37.4-25.el9 baseos 105 k 2026-02-25T11:41:32.416 INFO:teuthology.orchestra.run.trial028.stdout: libfdisk x86_64 2.37.4-25.el9 baseos 152 k 2026-02-25T11:41:32.416 INFO:teuthology.orchestra.run.trial028.stdout: libmount x86_64 2.37.4-25.el9 baseos 133 k 2026-02-25T11:41:32.416 INFO:teuthology.orchestra.run.trial028.stdout: librados2 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 3.5 M 2026-02-25T11:41:32.417 INFO:teuthology.orchestra.run.trial028.stdout: librbd1 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 2.8 M 2026-02-25T11:41:32.417 INFO:teuthology.orchestra.run.trial028.stdout: libsmartcols x86_64 2.37.4-25.el9 baseos 61 k 2026-02-25T11:41:32.417 INFO:teuthology.orchestra.run.trial028.stdout: libuuid x86_64 2.37.4-25.el9 baseos 26 k 2026-02-25T11:41:32.417 INFO:teuthology.orchestra.run.trial028.stdout: util-linux x86_64 2.37.4-25.el9 baseos 2.3 M 2026-02-25T11:41:32.417 INFO:teuthology.orchestra.run.trial028.stdout: util-linux-core x86_64 2.37.4-25.el9 baseos 458 k 2026-02-25T11:41:32.417 INFO:teuthology.orchestra.run.trial028.stdout:Installing dependencies: 2026-02-25T11:41:32.417 INFO:teuthology.orchestra.run.trial028.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T11:41:32.418 INFO:teuthology.orchestra.run.trial028.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2026-02-25T11:41:32.418 INFO:teuthology.orchestra.run.trial028.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T11:41:32.418 INFO:teuthology.orchestra.run.trial028.stdout: ceph-common x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 24 M 2026-02-25T11:41:32.418 INFO:teuthology.orchestra.run.trial028.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 57 k 2026-02-25T11:41:32.418 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mds x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 2.4 M 2026-02-25T11:41:32.418 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 298 k 2026-02-25T11:41:32.419 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mon x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 5.0 M 2026-02-25T11:41:32.419 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-crimson x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 19 M 2026-02-25T11:41:32.419 INFO:teuthology.orchestra.run.trial028.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 17 k 2026-02-25T11:41:32.419 INFO:teuthology.orchestra.run.trial028.stdout: ceph-selinux x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 25 k 2026-02-25T11:41:32.419 INFO:teuthology.orchestra.run.trial028.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T11:41:32.419 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T11:41:32.419 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T11:41:32.420 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T11:41:32.420 INFO:teuthology.orchestra.run.trial028.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T11:41:32.420 INFO:teuthology.orchestra.run.trial028.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T11:41:32.420 INFO:teuthology.orchestra.run.trial028.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-25T11:41:32.420 INFO:teuthology.orchestra.run.trial028.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T11:41:32.420 INFO:teuthology.orchestra.run.trial028.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T11:41:32.420 INFO:teuthology.orchestra.run.trial028.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T11:41:32.421 INFO:teuthology.orchestra.run.trial028.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-25T11:41:32.421 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 26 k 2026-02-25T11:41:32.421 INFO:teuthology.orchestra.run.trial028.stdout: libcephsqlite x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 159 k 2026-02-25T11:41:32.421 INFO:teuthology.orchestra.run.trial028.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T11:41:32.421 INFO:teuthology.orchestra.run.trial028.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T11:41:32.421 INFO:teuthology.orchestra.run.trial028.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T11:41:32.421 INFO:teuthology.orchestra.run.trial028.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T11:41:32.421 INFO:teuthology.orchestra.run.trial028.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T11:41:32.422 INFO:teuthology.orchestra.run.trial028.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T11:41:32.422 INFO:teuthology.orchestra.run.trial028.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T11:41:32.422 INFO:teuthology.orchestra.run.trial028.stdout: libradosstriper1 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 248 k 2026-02-25T11:41:32.422 INFO:teuthology.orchestra.run.trial028.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T11:41:32.422 INFO:teuthology.orchestra.run.trial028.stdout: librgw2 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 6.5 M 2026-02-25T11:41:32.422 INFO:teuthology.orchestra.run.trial028.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T11:41:32.422 INFO:teuthology.orchestra.run.trial028.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T11:41:32.422 INFO:teuthology.orchestra.run.trial028.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T11:41:32.423 INFO:teuthology.orchestra.run.trial028.stdout: libzstd-devel x86_64 1.5.5-1.el9 appstream 51 k 2026-02-25T11:41:32.423 INFO:teuthology.orchestra.run.trial028.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-25T11:41:32.423 INFO:teuthology.orchestra.run.trial028.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T11:41:32.423 INFO:teuthology.orchestra.run.trial028.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T11:41:32.423 INFO:teuthology.orchestra.run.trial028.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T11:41:32.423 INFO:teuthology.orchestra.run.trial028.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T11:41:32.424 INFO:teuthology.orchestra.run.trial028.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T11:41:32.424 INFO:teuthology.orchestra.run.trial028.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T11:41:32.424 INFO:teuthology.orchestra.run.trial028.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T11:41:32.424 INFO:teuthology.orchestra.run.trial028.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T11:41:32.424 INFO:teuthology.orchestra.run.trial028.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T11:41:32.424 INFO:teuthology.orchestra.run.trial028.stdout: perl-Thread-Queue noarch 3.14-460.el9 appstream 22 k 2026-02-25T11:41:32.425 INFO:teuthology.orchestra.run.trial028.stdout: perl-threads x86_64 1:2.25-460.el9 appstream 58 k 2026-02-25T11:41:32.425 INFO:teuthology.orchestra.run.trial028.stdout: perl-threads-shared x86_64 1.61-460.el9 appstream 45 k 2026-02-25T11:41:32.425 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-private-libs x86_64 13.23-1.el9 appstream 138 k 2026-02-25T11:41:32.425 INFO:teuthology.orchestra.run.trial028.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T11:41:32.425 INFO:teuthology.orchestra.run.trial028.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T11:41:32.425 INFO:teuthology.orchestra.run.trial028.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T11:41:32.425 INFO:teuthology.orchestra.run.trial028.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T11:41:32.425 INFO:teuthology.orchestra.run.trial028.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T11:41:32.425 INFO:teuthology.orchestra.run.trial028.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T11:41:32.426 INFO:teuthology.orchestra.run.trial028.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T11:41:32.426 INFO:teuthology.orchestra.run.trial028.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T11:41:32.426 INFO:teuthology.orchestra.run.trial028.stdout: python3-ceph-argparse x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 45 k 2026-02-25T11:41:32.426 INFO:teuthology.orchestra.run.trial028.stdout: python3-ceph-common x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 186 k 2026-02-25T11:41:32.426 INFO:teuthology.orchestra.run.trial028.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T11:41:32.426 INFO:teuthology.orchestra.run.trial028.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T11:41:32.426 INFO:teuthology.orchestra.run.trial028.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T11:41:32.426 INFO:teuthology.orchestra.run.trial028.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T11:41:32.426 INFO:teuthology.orchestra.run.trial028.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2026-02-25T11:41:32.426 INFO:teuthology.orchestra.run.trial028.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2026-02-25T11:41:32.426 INFO:teuthology.orchestra.run.trial028.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T11:41:32.426 INFO:teuthology.orchestra.run.trial028.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T11:41:32.427 INFO:teuthology.orchestra.run.trial028.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T11:41:32.427 INFO:teuthology.orchestra.run.trial028.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T11:41:32.427 INFO:teuthology.orchestra.run.trial028.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T11:41:32.427 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T11:41:32.427 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T11:41:32.427 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-25T11:41:32.427 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T11:41:32.427 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T11:41:32.427 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T11:41:32.427 INFO:teuthology.orchestra.run.trial028.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T11:41:32.427 INFO:teuthology.orchestra.run.trial028.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T11:41:32.427 INFO:teuthology.orchestra.run.trial028.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T11:41:32.427 INFO:teuthology.orchestra.run.trial028.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T11:41:32.428 INFO:teuthology.orchestra.run.trial028.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T11:41:32.428 INFO:teuthology.orchestra.run.trial028.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T11:41:32.428 INFO:teuthology.orchestra.run.trial028.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T11:41:32.428 INFO:teuthology.orchestra.run.trial028.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T11:41:32.428 INFO:teuthology.orchestra.run.trial028.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T11:41:32.428 INFO:teuthology.orchestra.run.trial028.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T11:41:32.428 INFO:teuthology.orchestra.run.trial028.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T11:41:32.428 INFO:teuthology.orchestra.run.trial028.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T11:41:32.428 INFO:teuthology.orchestra.run.trial028.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T11:41:32.428 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T11:41:32.428 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T11:41:32.428 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T11:41:32.428 INFO:teuthology.orchestra.run.trial028.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T11:41:32.429 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T11:41:32.429 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2026-02-25T11:41:32.429 INFO:teuthology.orchestra.run.trial028.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T11:41:32.429 INFO:teuthology.orchestra.run.trial028.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T11:41:32.429 INFO:teuthology.orchestra.run.trial028.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T11:41:32.429 INFO:teuthology.orchestra.run.trial028.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T11:41:32.429 INFO:teuthology.orchestra.run.trial028.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T11:41:32.429 INFO:teuthology.orchestra.run.trial028.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T11:41:32.429 INFO:teuthology.orchestra.run.trial028.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T11:41:32.429 INFO:teuthology.orchestra.run.trial028.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T11:41:32.429 INFO:teuthology.orchestra.run.trial028.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T11:41:32.429 INFO:teuthology.orchestra.run.trial028.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T11:41:32.429 INFO:teuthology.orchestra.run.trial028.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2026-02-25T11:41:32.430 INFO:teuthology.orchestra.run.trial028.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T11:41:32.430 INFO:teuthology.orchestra.run.trial028.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T11:41:32.430 INFO:teuthology.orchestra.run.trial028.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T11:41:32.430 INFO:teuthology.orchestra.run.trial028.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T11:41:32.430 INFO:teuthology.orchestra.run.trial028.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T11:41:32.430 INFO:teuthology.orchestra.run.trial028.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T11:41:32.430 INFO:teuthology.orchestra.run.trial028.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T11:41:32.430 INFO:teuthology.orchestra.run.trial028.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T11:41:32.430 INFO:teuthology.orchestra.run.trial028.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T11:41:32.430 INFO:teuthology.orchestra.run.trial028.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2026-02-25T11:41:32.430 INFO:teuthology.orchestra.run.trial028.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T11:41:32.430 INFO:teuthology.orchestra.run.trial028.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-25T11:41:32.430 INFO:teuthology.orchestra.run.trial028.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T11:41:32.431 INFO:teuthology.orchestra.run.trial028.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2026-02-25T11:41:32.431 INFO:teuthology.orchestra.run.trial028.stdout:Installing weak dependencies: 2026-02-25T11:41:32.431 INFO:teuthology.orchestra.run.trial028.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T11:41:32.431 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:41:32.431 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-25T11:41:32.431 INFO:teuthology.orchestra.run.trial028.stdout:======================================================================================= 2026-02-25T11:41:32.431 INFO:teuthology.orchestra.run.trial028.stdout:Install 163 Packages 2026-02-25T11:41:32.431 INFO:teuthology.orchestra.run.trial028.stdout:Upgrade 9 Packages 2026-02-25T11:41:32.431 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:41:32.432 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 293 M 2026-02-25T11:41:32.432 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-25T11:41:32.706 INFO:teuthology.orchestra.run.trial167.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-25T11:41:32.820 INFO:teuthology.orchestra.run.trial028.stdout:(1/172): ceph-20.3.0-5549.g92715cd3.el9.x86_64. 56 kB/s | 6.5 kB 00:00 2026-02-25T11:41:33.464 INFO:teuthology.orchestra.run.trial028.stdout:(2/172): ceph-fuse-20.3.0-5549.g92715cd3.el9.x8 1.5 MB/s | 987 kB 00:00 2026-02-25T11:41:33.559 INFO:teuthology.orchestra.run.trial028.stdout:(3/172): ceph-immutable-object-cache-20.3.0-554 1.5 MB/s | 147 kB 00:00 2026-02-25T11:41:33.977 INFO:teuthology.orchestra.run.trial167.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-25T11:41:33.978 INFO:teuthology.orchestra.run.trial167.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-25T11:41:33.984 INFO:teuthology.orchestra.run.trial167.stdout:Package NetworkManager-1:1.54.3-2.el9.x86_64 is already installed. 2026-02-25T11:41:33.984 INFO:teuthology.orchestra.run.trial167.stdout:Package iproute-6.17.0-2.el9.x86_64 is already installed. 2026-02-25T11:41:33.985 INFO:teuthology.orchestra.run.trial167.stdout:Package util-linux-2.37.4-21.el9.x86_64 is already installed. 2026-02-25T11:41:33.985 INFO:teuthology.orchestra.run.trial167.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2026-02-25T11:41:33.986 INFO:teuthology.orchestra.run.trial167.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2026-02-25T11:41:33.987 INFO:teuthology.orchestra.run.trial167.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2026-02-25T11:41:33.988 INFO:teuthology.orchestra.run.trial167.stdout:Package xfsprogs-6.4.0-7.el9.x86_64 is already installed. 2026-02-25T11:41:34.018 INFO:teuthology.orchestra.run.trial167.stdout:Dependencies resolved. 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout:======================================================================================= 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: Package Arch Version Repository Size 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout:======================================================================================= 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout:Installing: 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 6.5 k 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph-base x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 5.2 M 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph-fuse x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 987 k 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 147 k 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mgr x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 951 k 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 184 k 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 15 M 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 7.4 M 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mgr-rook noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 50 k 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph-osd x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 167 k 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph-osd-classic x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 16 M 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph-radosgw x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 13 M 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph-test x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 76 M 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: ceph-volume noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 301 k 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: cephadm noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 1.0 M 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: cephfs-mirror x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 257 k 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: cephfs-shell x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 27 k 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: cephfs-top noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 18 k 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: elfutils-libelf-devel x86_64 0.194-1.el9 appstream 79 k 2026-02-25T11:41:34.023 INFO:teuthology.orchestra.run.trial167.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: libcephfs-devel x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 37 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: libcephfs2 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 908 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: librados-devel x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 123 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: libuuid-devel x86_64 2.37.4-25.el9 appstream 23 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: openssl-devel x86_64 1:3.5.5-1.el9 appstream 4.8 M 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: postgresql x86_64 13.23-1.el9 appstream 1.6 M 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: postgresql-contrib x86_64 13.23-1.el9 appstream 903 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: postgresql-server x86_64 13.23-1.el9 appstream 5.8 M 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: python3-cephfs x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 204 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: python3-rados x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 324 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: python3-rbd x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 305 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: python3-rgw x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 100 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: rbd-fuse x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 84 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: rbd-mirror x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 2.9 M 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: rbd-nbd x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 172 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: xfsprogs-devel x86_64 6.4.0-7.el9 appstream 96 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout:Upgrading: 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: libblkid x86_64 2.37.4-25.el9 baseos 105 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: libfdisk x86_64 2.37.4-25.el9 baseos 152 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: libmount x86_64 2.37.4-25.el9 baseos 133 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: librados2 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 3.5 M 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: librbd1 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 2.8 M 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: libsmartcols x86_64 2.37.4-25.el9 baseos 61 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: libuuid x86_64 2.37.4-25.el9 baseos 26 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: util-linux x86_64 2.37.4-25.el9 baseos 2.3 M 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: util-linux-core x86_64 2.37.4-25.el9 baseos 458 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout:Installing dependencies: 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: ceph-common x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 24 M 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 57 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mds x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 2.4 M 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 298 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mon x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 5.0 M 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: ceph-osd-crimson x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 19 M 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 17 k 2026-02-25T11:41:34.024 INFO:teuthology.orchestra.run.trial167.stdout: ceph-selinux x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 25 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 26 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libcephsqlite x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 159 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libradosstriper1 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 248 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: librgw2 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 6.5 M 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: libzstd-devel x86_64 1.5.5-1.el9 appstream 51 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: perl-Thread-Queue noarch 3.14-460.el9 appstream 22 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: perl-threads x86_64 1:2.25-460.el9 appstream 58 k 2026-02-25T11:41:34.025 INFO:teuthology.orchestra.run.trial167.stdout: perl-threads-shared x86_64 1.61-460.el9 appstream 45 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: postgresql-private-libs x86_64 13.23-1.el9 appstream 138 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-ceph-argparse x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 45 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-ceph-common x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 186 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T11:41:34.026 INFO:teuthology.orchestra.run.trial167.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout:Installing weak dependencies: 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout:Transaction Summary 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout:======================================================================================= 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout:Install 163 Packages 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout:Upgrade 9 Packages 2026-02-25T11:41:34.027 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:41:34.028 INFO:teuthology.orchestra.run.trial167.stdout:Total download size: 293 M 2026-02-25T11:41:34.028 INFO:teuthology.orchestra.run.trial167.stdout:Downloading Packages: 2026-02-25T11:41:34.197 INFO:teuthology.orchestra.run.trial028.stdout:(4/172): ceph-mds-20.3.0-5549.g92715cd3.el9.x86 3.7 MB/s | 2.4 MB 00:00 2026-02-25T11:41:34.313 INFO:teuthology.orchestra.run.trial028.stdout:(5/172): ceph-base-20.3.0-5549.g92715cd3.el9.x8 3.2 MB/s | 5.2 MB 00:01 2026-02-25T11:41:34.377 INFO:teuthology.orchestra.run.trial028.stdout:(6/172): ceph-mgr-20.3.0-5549.g92715cd3.el9.x86 5.2 MB/s | 951 kB 00:00 2026-02-25T11:41:34.422 INFO:teuthology.orchestra.run.trial028.stdout:(7/172): ceph-osd-20.3.0-5549.g92715cd3.el9.x86 3.6 MB/s | 167 kB 00:00 2026-02-25T11:41:34.716 INFO:teuthology.orchestra.run.trial167.stdout:(1/172): ceph-20.3.0-5549.g92715cd3.el9.x86_64. 32 kB/s | 6.5 kB 00:00 2026-02-25T11:41:35.223 INFO:teuthology.orchestra.run.trial028.stdout:(8/172): ceph-mon-20.3.0-5549.g92715cd3.el9.x86 5.5 MB/s | 5.0 MB 00:00 2026-02-25T11:41:35.450 INFO:teuthology.orchestra.run.trial167.stdout:(2/172): ceph-fuse-20.3.0-5549.g92715cd3.el9.x8 1.3 MB/s | 987 kB 00:00 2026-02-25T11:41:35.512 INFO:teuthology.orchestra.run.trial167.stdout:(3/172): ceph-immutable-object-cache-20.3.0-554 2.3 MB/s | 147 kB 00:00 2026-02-25T11:41:35.798 INFO:teuthology.orchestra.run.trial119.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-25T11:41:36.251 INFO:teuthology.orchestra.run.trial167.stdout:(4/172): ceph-mds-20.3.0-5549.g92715cd3.el9.x86 3.2 MB/s | 2.4 MB 00:00 2026-02-25T11:41:36.451 INFO:teuthology.orchestra.run.trial167.stdout:(5/172): ceph-mgr-20.3.0-5549.g92715cd3.el9.x86 4.6 MB/s | 951 kB 00:00 2026-02-25T11:41:37.073 INFO:teuthology.orchestra.run.trial119.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-25T11:41:37.074 INFO:teuthology.orchestra.run.trial119.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-25T11:41:37.079 INFO:teuthology.orchestra.run.trial119.stdout:Package NetworkManager-1:1.54.3-2.el9.x86_64 is already installed. 2026-02-25T11:41:37.080 INFO:teuthology.orchestra.run.trial119.stdout:Package iproute-6.17.0-2.el9.x86_64 is already installed. 2026-02-25T11:41:37.080 INFO:teuthology.orchestra.run.trial119.stdout:Package util-linux-2.37.4-21.el9.x86_64 is already installed. 2026-02-25T11:41:37.080 INFO:teuthology.orchestra.run.trial119.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2026-02-25T11:41:37.081 INFO:teuthology.orchestra.run.trial119.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2026-02-25T11:41:37.083 INFO:teuthology.orchestra.run.trial119.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2026-02-25T11:41:37.083 INFO:teuthology.orchestra.run.trial119.stdout:Package xfsprogs-6.4.0-7.el9.x86_64 is already installed. 2026-02-25T11:41:37.113 INFO:teuthology.orchestra.run.trial119.stdout:Dependencies resolved. 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout:======================================================================================= 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: Package Arch Version Repository Size 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout:======================================================================================= 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout:Installing: 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 6.5 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph-base x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 5.2 M 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph-fuse x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 987 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 147 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mgr x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 951 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 184 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 15 M 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 7.4 M 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mgr-rook noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 50 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph-osd x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 167 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph-osd-classic x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 16 M 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph-radosgw x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 13 M 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph-test x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 76 M 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: ceph-volume noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 301 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: cephadm noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 1.0 M 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: cephfs-mirror x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 257 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: cephfs-shell x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 27 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: cephfs-top noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 18 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: elfutils-libelf-devel x86_64 0.194-1.el9 appstream 79 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2026-02-25T11:41:37.119 INFO:teuthology.orchestra.run.trial119.stdout: libcephfs-devel x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 37 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: libcephfs2 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 908 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: librados-devel x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 123 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: libuuid-devel x86_64 2.37.4-25.el9 appstream 23 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: openssl-devel x86_64 1:3.5.5-1.el9 appstream 4.8 M 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: postgresql x86_64 13.23-1.el9 appstream 1.6 M 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: postgresql-contrib x86_64 13.23-1.el9 appstream 903 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: postgresql-server x86_64 13.23-1.el9 appstream 5.8 M 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: python3-cephfs x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 204 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: python3-rados x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 324 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: python3-rbd x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 305 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: python3-rgw x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 100 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: rbd-fuse x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 84 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: rbd-mirror x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 2.9 M 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: rbd-nbd x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 172 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: xfsprogs-devel x86_64 6.4.0-7.el9 appstream 96 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout:Upgrading: 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: libblkid x86_64 2.37.4-25.el9 baseos 105 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: libfdisk x86_64 2.37.4-25.el9 baseos 152 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: libmount x86_64 2.37.4-25.el9 baseos 133 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: librados2 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 3.5 M 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: librbd1 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 2.8 M 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: libsmartcols x86_64 2.37.4-25.el9 baseos 61 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: libuuid x86_64 2.37.4-25.el9 baseos 26 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: util-linux x86_64 2.37.4-25.el9 baseos 2.3 M 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: util-linux-core x86_64 2.37.4-25.el9 baseos 458 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout:Installing dependencies: 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: ceph-common x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 24 M 2026-02-25T11:41:37.120 INFO:teuthology.orchestra.run.trial119.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 57 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mds x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 2.4 M 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 298 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mon x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 5.0 M 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: ceph-osd-crimson x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 19 M 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5549.g92715cd3.el9 ceph-noarch 17 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: ceph-selinux x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 25 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 26 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libcephsqlite x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 159 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libradosstriper1 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 248 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: librgw2 x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 6.5 M 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T11:41:37.121 INFO:teuthology.orchestra.run.trial119.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: libzstd-devel x86_64 1.5.5-1.el9 appstream 51 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: perl-Thread-Queue noarch 3.14-460.el9 appstream 22 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: perl-threads x86_64 1:2.25-460.el9 appstream 58 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: perl-threads-shared x86_64 1.61-460.el9 appstream 45 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: postgresql-private-libs x86_64 13.23-1.el9 appstream 138 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-ceph-argparse x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 45 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-ceph-common x86_64 2:20.3.0-5549.g92715cd3.el9 ceph 186 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2026-02-25T11:41:37.122 INFO:teuthology.orchestra.run.trial119.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2026-02-25T11:41:37.123 INFO:teuthology.orchestra.run.trial119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout:Installing weak dependencies: 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout:Transaction Summary 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout:======================================================================================= 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout:Install 163 Packages 2026-02-25T11:41:37.124 INFO:teuthology.orchestra.run.trial119.stdout:Upgrade 9 Packages 2026-02-25T11:41:37.125 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:41:37.125 INFO:teuthology.orchestra.run.trial119.stdout:Total download size: 293 M 2026-02-25T11:41:37.125 INFO:teuthology.orchestra.run.trial119.stdout:Downloading Packages: 2026-02-25T11:41:37.464 INFO:teuthology.orchestra.run.trial167.stdout:(6/172): ceph-mon-20.3.0-5549.g92715cd3.el9.x86 4.9 MB/s | 5.0 MB 00:01 2026-02-25T11:41:37.500 INFO:teuthology.orchestra.run.trial167.stdout:(7/172): ceph-base-20.3.0-5549.g92715cd3.el9.x8 1.7 MB/s | 5.2 MB 00:02 2026-02-25T11:41:37.508 INFO:teuthology.orchestra.run.trial167.stdout:(8/172): ceph-osd-20.3.0-5549.g92715cd3.el9.x86 3.8 MB/s | 167 kB 00:00 2026-02-25T11:41:37.697 INFO:teuthology.orchestra.run.trial119.stdout:(1/172): ceph-20.3.0-5549.g92715cd3.el9.x86_64. 32 kB/s | 6.5 kB 00:00 2026-02-25T11:41:37.777 INFO:teuthology.orchestra.run.trial028.stdout:(9/172): ceph-osd-crimson-20.3.0-5549.g92715cd3 7.5 MB/s | 19 MB 00:02 2026-02-25T11:41:37.842 INFO:teuthology.orchestra.run.trial167.stdout:(9/172): ceph-common-20.3.0-5549.g92715cd3.el9. 7.3 MB/s | 24 MB 00:03 2026-02-25T11:41:38.015 INFO:teuthology.orchestra.run.trial119.stdout:(2/172): ceph-fuse-20.3.0-5549.g92715cd3.el9.x8 3.0 MB/s | 987 kB 00:00 2026-02-25T11:41:38.076 INFO:teuthology.orchestra.run.trial119.stdout:(3/172): ceph-immutable-object-cache-20.3.0-554 2.4 MB/s | 147 kB 00:00 2026-02-25T11:41:38.231 INFO:teuthology.orchestra.run.trial119.stdout:(4/172): ceph-base-20.3.0-5549.g92715cd3.el9.x8 7.1 MB/s | 5.2 MB 00:00 2026-02-25T11:41:38.357 INFO:teuthology.orchestra.run.trial119.stdout:(5/172): ceph-mgr-20.3.0-5549.g92715cd3.el9.x86 7.3 MB/s | 951 kB 00:00 2026-02-25T11:41:38.394 INFO:teuthology.orchestra.run.trial028.stdout:(10/172): ceph-osd-classic-20.3.0-5549.g92715cd 3.9 MB/s | 16 MB 00:03 2026-02-25T11:41:38.453 INFO:teuthology.orchestra.run.trial028.stdout:(11/172): ceph-selinux-20.3.0-5549.g92715cd3.el 427 kB/s | 25 kB 00:00 2026-02-25T11:41:38.458 INFO:teuthology.orchestra.run.trial119.stdout:(6/172): ceph-mds-20.3.0-5549.g92715cd3.el9.x86 6.2 MB/s | 2.4 MB 00:00 2026-02-25T11:41:38.530 INFO:teuthology.orchestra.run.trial119.stdout:(7/172): ceph-osd-20.3.0-5549.g92715cd3.el9.x86 2.3 MB/s | 167 kB 00:00 2026-02-25T11:41:38.759 INFO:teuthology.orchestra.run.trial119.stdout:(8/172): ceph-mon-20.3.0-5549.g92715cd3.el9.x86 12 MB/s | 5.0 MB 00:00 2026-02-25T11:41:39.562 INFO:teuthology.orchestra.run.trial119.stdout:(9/172): ceph-common-20.3.0-5549.g92715cd3.el9. 12 MB/s | 24 MB 00:02 2026-02-25T11:41:39.789 INFO:teuthology.orchestra.run.trial028.stdout:(12/172): ceph-radosgw-20.3.0-5549.g92715cd3.el 6.6 MB/s | 13 MB 00:02 2026-02-25T11:41:39.851 INFO:teuthology.orchestra.run.trial028.stdout:(13/172): cephfs-mirror-20.3.0-5549.g92715cd3.e 4.0 MB/s | 257 kB 00:00 2026-02-25T11:41:39.908 INFO:teuthology.orchestra.run.trial028.stdout:(14/172): cephfs-shell-20.3.0-5549.g92715cd3.el 473 kB/s | 27 kB 00:00 2026-02-25T11:41:39.966 INFO:teuthology.orchestra.run.trial028.stdout:(15/172): libcephfs-devel-20.3.0-5549.g92715cd3 647 kB/s | 37 kB 00:00 2026-02-25T11:41:40.024 INFO:teuthology.orchestra.run.trial028.stdout:(16/172): libcephfs-proxy2-20.3.0-5549.g92715cd 455 kB/s | 26 kB 00:00 2026-02-25T11:41:40.059 INFO:teuthology.orchestra.run.trial167.stdout:(10/172): ceph-radosgw-20.3.0-5549.g92715cd3.el 6.0 MB/s | 13 MB 00:02 2026-02-25T11:41:40.118 INFO:teuthology.orchestra.run.trial167.stdout:(11/172): ceph-selinux-20.3.0-5549.g92715cd3.el 424 kB/s | 25 kB 00:00 2026-02-25T11:41:40.203 INFO:teuthology.orchestra.run.trial028.stdout:(17/172): libcephfs2-20.3.0-5549.g92715cd3.el9. 5.0 MB/s | 908 kB 00:00 2026-02-25T11:41:40.265 INFO:teuthology.orchestra.run.trial028.stdout:(18/172): libcephsqlite-20.3.0-5549.g92715cd3.e 2.5 MB/s | 159 kB 00:00 2026-02-25T11:41:40.327 INFO:teuthology.orchestra.run.trial028.stdout:(19/172): librados-devel-20.3.0-5549.g92715cd3. 1.9 MB/s | 123 kB 00:00 2026-02-25T11:41:40.392 INFO:teuthology.orchestra.run.trial028.stdout:(20/172): libradosstriper1-20.3.0-5549.g92715cd 3.7 MB/s | 248 kB 00:00 2026-02-25T11:41:40.476 INFO:teuthology.orchestra.run.trial119.stdout:(10/172): ceph-radosgw-20.3.0-5549.g92715cd3.el 15 MB/s | 13 MB 00:00 2026-02-25T11:41:40.530 INFO:teuthology.orchestra.run.trial119.stdout:(11/172): ceph-selinux-20.3.0-5549.g92715cd3.el 475 kB/s | 25 kB 00:00 2026-02-25T11:41:40.739 INFO:teuthology.orchestra.run.trial119.stdout:(12/172): ceph-osd-classic-20.3.0-5549.g92715cd 7.1 MB/s | 16 MB 00:02 2026-02-25T11:41:40.810 INFO:teuthology.orchestra.run.trial119.stdout:(13/172): cephfs-mirror-20.3.0-5549.g92715cd3.e 3.6 MB/s | 257 kB 00:00 2026-02-25T11:41:40.873 INFO:teuthology.orchestra.run.trial119.stdout:(14/172): cephfs-shell-20.3.0-5549.g92715cd3.el 428 kB/s | 27 kB 00:00 2026-02-25T11:41:40.939 INFO:teuthology.orchestra.run.trial119.stdout:(15/172): libcephfs-devel-20.3.0-5549.g92715cd3 565 kB/s | 37 kB 00:00 2026-02-25T11:41:41.002 INFO:teuthology.orchestra.run.trial119.stdout:(16/172): libcephfs-proxy2-20.3.0-5549.g92715cd 423 kB/s | 26 kB 00:00 2026-02-25T11:41:41.133 INFO:teuthology.orchestra.run.trial119.stdout:(17/172): ceph-osd-crimson-20.3.0-5549.g92715cd 8.0 MB/s | 19 MB 00:02 2026-02-25T11:41:41.138 INFO:teuthology.orchestra.run.trial119.stdout:(18/172): libcephfs2-20.3.0-5549.g92715cd3.el9. 6.5 MB/s | 908 kB 00:00 2026-02-25T11:41:41.188 INFO:teuthology.orchestra.run.trial119.stdout:(19/172): libcephsqlite-20.3.0-5549.g92715cd3.e 2.9 MB/s | 159 kB 00:00 2026-02-25T11:41:41.198 INFO:teuthology.orchestra.run.trial119.stdout:(20/172): librados-devel-20.3.0-5549.g92715cd3. 2.0 MB/s | 123 kB 00:00 2026-02-25T11:41:41.248 INFO:teuthology.orchestra.run.trial119.stdout:(21/172): libradosstriper1-20.3.0-5549.g92715cd 4.0 MB/s | 248 kB 00:00 2026-02-25T11:41:41.308 INFO:teuthology.orchestra.run.trial119.stdout:(22/172): python3-ceph-argparse-20.3.0-5549.g92 752 kB/s | 45 kB 00:00 2026-02-25T11:41:41.373 INFO:teuthology.orchestra.run.trial119.stdout:(23/172): python3-ceph-common-20.3.0-5549.g9271 2.8 MB/s | 186 kB 00:00 2026-02-25T11:41:41.415 INFO:teuthology.orchestra.run.trial028.stdout:(21/172): librgw2-20.3.0-5549.g92715cd3.el9.x86 6.4 MB/s | 6.5 MB 00:01 2026-02-25T11:41:41.434 INFO:teuthology.orchestra.run.trial119.stdout:(24/172): python3-cephfs-20.3.0-5549.g92715cd3. 3.2 MB/s | 204 kB 00:00 2026-02-25T11:41:41.472 INFO:teuthology.orchestra.run.trial028.stdout:(22/172): python3-ceph-argparse-20.3.0-5549.g92 786 kB/s | 45 kB 00:00 2026-02-25T11:41:41.496 INFO:teuthology.orchestra.run.trial119.stdout:(25/172): python3-rados-20.3.0-5549.g92715cd3.e 5.2 MB/s | 324 kB 00:00 2026-02-25T11:41:41.536 INFO:teuthology.orchestra.run.trial028.stdout:(23/172): python3-ceph-common-20.3.0-5549.g9271 2.9 MB/s | 186 kB 00:00 2026-02-25T11:41:41.560 INFO:teuthology.orchestra.run.trial119.stdout:(26/172): python3-rbd-20.3.0-5549.g92715cd3.el9 4.7 MB/s | 305 kB 00:00 2026-02-25T11:41:41.600 INFO:teuthology.orchestra.run.trial028.stdout:(24/172): python3-cephfs-20.3.0-5549.g92715cd3. 3.1 MB/s | 204 kB 00:00 2026-02-25T11:41:41.619 INFO:teuthology.orchestra.run.trial119.stdout:(27/172): python3-rgw-20.3.0-5549.g92715cd3.el9 1.7 MB/s | 100 kB 00:00 2026-02-25T11:41:41.662 INFO:teuthology.orchestra.run.trial028.stdout:(25/172): python3-rados-20.3.0-5549.g92715cd3.e 5.1 MB/s | 324 kB 00:00 2026-02-25T11:41:41.680 INFO:teuthology.orchestra.run.trial119.stdout:(28/172): rbd-fuse-20.3.0-5549.g92715cd3.el9.x8 1.3 MB/s | 84 kB 00:00 2026-02-25T11:41:41.724 INFO:teuthology.orchestra.run.trial028.stdout:(26/172): python3-rbd-20.3.0-5549.g92715cd3.el9 4.8 MB/s | 305 kB 00:00 2026-02-25T11:41:41.792 INFO:teuthology.orchestra.run.trial028.stdout:(27/172): python3-rgw-20.3.0-5549.g92715cd3.el9 1.5 MB/s | 100 kB 00:00 2026-02-25T11:41:41.861 INFO:teuthology.orchestra.run.trial028.stdout:(28/172): rbd-fuse-20.3.0-5549.g92715cd3.el9.x8 1.2 MB/s | 84 kB 00:00 2026-02-25T11:41:42.026 INFO:teuthology.orchestra.run.trial119.stdout:(29/172): rbd-mirror-20.3.0-5549.g92715cd3.el9. 8.3 MB/s | 2.9 MB 00:00 2026-02-25T11:41:42.076 INFO:teuthology.orchestra.run.trial119.stdout:(30/172): librgw2-20.3.0-5549.g92715cd3.el9.x86 7.4 MB/s | 6.5 MB 00:00 2026-02-25T11:41:42.078 INFO:teuthology.orchestra.run.trial119.stdout:(31/172): rbd-nbd-20.3.0-5549.g92715cd3.el9.x86 3.2 MB/s | 172 kB 00:00 2026-02-25T11:41:42.125 INFO:teuthology.orchestra.run.trial119.stdout:(32/172): ceph-grafana-dashboards-20.3.0-5549.g 1.2 MB/s | 57 kB 00:00 2026-02-25T11:41:42.130 INFO:teuthology.orchestra.run.trial119.stdout:(33/172): ceph-mgr-cephadm-20.3.0-5549.g92715cd 3.5 MB/s | 184 kB 00:00 2026-02-25T11:41:42.257 INFO:teuthology.orchestra.run.trial167.stdout:(12/172): ceph-osd-crimson-20.3.0-5549.g92715cd 4.0 MB/s | 19 MB 00:04 2026-02-25T11:41:42.380 INFO:teuthology.orchestra.run.trial167.stdout:(13/172): cephfs-mirror-20.3.0-5549.g92715cd3.e 2.0 MB/s | 257 kB 00:00 2026-02-25T11:41:42.440 INFO:teuthology.orchestra.run.trial167.stdout:(14/172): cephfs-shell-20.3.0-5549.g92715cd3.el 449 kB/s | 27 kB 00:00 2026-02-25T11:41:42.471 INFO:teuthology.orchestra.run.trial028.stdout:(29/172): rbd-mirror-20.3.0-5549.g92715cd3.el9. 4.7 MB/s | 2.9 MB 00:00 2026-02-25T11:41:42.502 INFO:teuthology.orchestra.run.trial167.stdout:(15/172): libcephfs-devel-20.3.0-5549.g92715cd3 609 kB/s | 37 kB 00:00 2026-02-25T11:41:42.531 INFO:teuthology.orchestra.run.trial028.stdout:(30/172): rbd-nbd-20.3.0-5549.g92715cd3.el9.x86 2.8 MB/s | 172 kB 00:00 2026-02-25T11:41:42.561 INFO:teuthology.orchestra.run.trial167.stdout:(16/172): libcephfs-proxy2-20.3.0-5549.g92715cd 447 kB/s | 26 kB 00:00 2026-02-25T11:41:42.588 INFO:teuthology.orchestra.run.trial028.stdout:(31/172): ceph-grafana-dashboards-20.3.0-5549.g 998 kB/s | 57 kB 00:00 2026-02-25T11:41:42.652 INFO:teuthology.orchestra.run.trial028.stdout:(32/172): ceph-mgr-cephadm-20.3.0-5549.g92715cd 2.9 MB/s | 184 kB 00:00 2026-02-25T11:41:42.877 INFO:teuthology.orchestra.run.trial167.stdout:(17/172): libcephfs2-20.3.0-5549.g92715cd3.el9. 2.8 MB/s | 908 kB 00:00 2026-02-25T11:41:42.945 INFO:teuthology.orchestra.run.trial167.stdout:(18/172): libcephsqlite-20.3.0-5549.g92715cd3.e 2.3 MB/s | 159 kB 00:00 2026-02-25T11:41:43.010 INFO:teuthology.orchestra.run.trial167.stdout:(19/172): librados-devel-20.3.0-5549.g92715cd3. 1.8 MB/s | 123 kB 00:00 2026-02-25T11:41:43.061 INFO:teuthology.orchestra.run.trial119.stdout:(34/172): ceph-mgr-diskprediction-local-20.3.0- 7.9 MB/s | 7.4 MB 00:00 2026-02-25T11:41:43.125 INFO:teuthology.orchestra.run.trial119.stdout:(35/172): ceph-mgr-modules-core-20.3.0-5549.g92 4.6 MB/s | 298 kB 00:00 2026-02-25T11:41:43.131 INFO:teuthology.orchestra.run.trial167.stdout:(20/172): libradosstriper1-20.3.0-5549.g92715cd 2.0 MB/s | 248 kB 00:00 2026-02-25T11:41:43.182 INFO:teuthology.orchestra.run.trial119.stdout:(36/172): ceph-mgr-rook-20.3.0-5549.g92715cd3.e 897 kB/s | 50 kB 00:00 2026-02-25T11:41:43.240 INFO:teuthology.orchestra.run.trial119.stdout:(37/172): ceph-prometheus-alerts-20.3.0-5549.g9 300 kB/s | 17 kB 00:00 2026-02-25T11:41:43.304 INFO:teuthology.orchestra.run.trial119.stdout:(38/172): ceph-volume-20.3.0-5549.g92715cd3.el9 4.6 MB/s | 301 kB 00:00 2026-02-25T11:41:43.441 INFO:teuthology.orchestra.run.trial119.stdout:(39/172): cephadm-20.3.0-5549.g92715cd3.el9.noa 7.5 MB/s | 1.0 MB 00:00 2026-02-25T11:41:43.501 INFO:teuthology.orchestra.run.trial119.stdout:(40/172): cephfs-top-20.3.0-5549.g92715cd3.el9. 305 kB/s | 18 kB 00:00 2026-02-25T11:41:44.305 INFO:teuthology.orchestra.run.trial119.stdout:(41/172): cryptsetup-2.8.1-3.el9.x86_64.rpm 436 kB/s | 351 kB 00:00 2026-02-25T11:41:44.312 INFO:teuthology.orchestra.run.trial167.stdout:(21/172): ceph-osd-classic-20.3.0-5549.g92715cd 2.3 MB/s | 16 MB 00:06 2026-02-25T11:41:44.373 INFO:teuthology.orchestra.run.trial167.stdout:(22/172): python3-ceph-argparse-20.3.0-5549.g92 737 kB/s | 45 kB 00:00 2026-02-25T11:41:44.449 INFO:teuthology.orchestra.run.trial119.stdout:(42/172): ceph-mgr-dashboard-20.3.0-5549.g92715 6.5 MB/s | 15 MB 00:02 2026-02-25T11:41:44.476 INFO:teuthology.orchestra.run.trial167.stdout:(23/172): python3-ceph-common-20.3.0-5549.g9271 1.8 MB/s | 186 kB 00:00 2026-02-25T11:41:44.587 INFO:teuthology.orchestra.run.trial167.stdout:(24/172): python3-cephfs-20.3.0-5549.g92715cd3. 1.8 MB/s | 204 kB 00:00 2026-02-25T11:41:44.755 INFO:teuthology.orchestra.run.trial167.stdout:(25/172): python3-rados-20.3.0-5549.g92715cd3.e 1.9 MB/s | 324 kB 00:00 2026-02-25T11:41:44.864 INFO:teuthology.orchestra.run.trial167.stdout:(26/172): python3-rbd-20.3.0-5549.g92715cd3.el9 2.7 MB/s | 305 kB 00:00 2026-02-25T11:41:44.889 INFO:teuthology.orchestra.run.trial167.stdout:(27/172): librgw2-20.3.0-5549.g92715cd3.el9.x86 3.7 MB/s | 6.5 MB 00:01 2026-02-25T11:41:44.901 INFO:teuthology.orchestra.run.trial167.stdout:(28/172): python3-rgw-20.3.0-5549.g92715cd3.el9 2.7 MB/s | 100 kB 00:00 2026-02-25T11:41:44.925 INFO:teuthology.orchestra.run.trial167.stdout:(29/172): rbd-fuse-20.3.0-5549.g92715cd3.el9.x8 2.3 MB/s | 84 kB 00:00 2026-02-25T11:41:44.963 INFO:teuthology.orchestra.run.trial167.stdout:(30/172): rbd-nbd-20.3.0-5549.g92715cd3.el9.x86 4.4 MB/s | 172 kB 00:00 2026-02-25T11:41:44.980 INFO:teuthology.orchestra.run.trial119.stdout:(43/172): ceph-test-20.3.0-5549.g92715cd3.el9.x 17 MB/s | 76 MB 00:04 2026-02-25T11:41:45.000 INFO:teuthology.orchestra.run.trial167.stdout:(31/172): ceph-grafana-dashboards-20.3.0-5549.g 1.5 MB/s | 57 kB 00:00 2026-02-25T11:41:45.039 INFO:teuthology.orchestra.run.trial167.stdout:(32/172): ceph-mgr-cephadm-20.3.0-5549.g92715cd 4.6 MB/s | 184 kB 00:00 2026-02-25T11:41:45.053 INFO:teuthology.orchestra.run.trial119.stdout:(44/172): ledmon-libs-1.1.0-3.el9.x86_64.rpm 67 kB/s | 40 kB 00:00 2026-02-25T11:41:45.138 INFO:teuthology.orchestra.run.trial028.stdout:(33/172): ceph-mgr-dashboard-20.3.0-5549.g92715 6.1 MB/s | 15 MB 00:02 2026-02-25T11:41:45.445 INFO:teuthology.orchestra.run.trial119.stdout:(45/172): hwloc-libs-2.4.1-6.el9.x86_64.rpm 1.8 MB/s | 2.1 MB 00:01 2026-02-25T11:41:45.456 INFO:teuthology.orchestra.run.trial028.stdout:(34/172): ceph-common-20.3.0-5549.g92715cd3.el9 1.9 MB/s | 24 MB 00:12 2026-02-25T11:41:45.524 INFO:teuthology.orchestra.run.trial119.stdout:(46/172): libatomic-11.5.0-14.el9.x86_64.rpm 42 kB/s | 23 kB 00:00 2026-02-25T11:41:45.529 INFO:teuthology.orchestra.run.trial028.stdout:(35/172): ceph-mgr-modules-core-20.3.0-5549.g92 4.0 MB/s | 298 kB 00:00 2026-02-25T11:41:45.565 INFO:teuthology.orchestra.run.trial028.stdout:(36/172): ceph-mgr-rook-20.3.0-5549.g92715cd3.e 1.4 MB/s | 50 kB 00:00 2026-02-25T11:41:45.569 INFO:teuthology.orchestra.run.trial167.stdout:(33/172): rbd-mirror-20.3.0-5549.g92715cd3.el9. 4.3 MB/s | 2.9 MB 00:00 2026-02-25T11:41:45.602 INFO:teuthology.orchestra.run.trial028.stdout:(37/172): ceph-prometheus-alerts-20.3.0-5549.g9 473 kB/s | 17 kB 00:00 2026-02-25T11:41:45.682 INFO:teuthology.orchestra.run.trial119.stdout:(47/172): libconfig-1.7.2-9.el9.x86_64.rpm 115 kB/s | 72 kB 00:00 2026-02-25T11:41:45.688 INFO:teuthology.orchestra.run.trial028.stdout:(38/172): ceph-volume-20.3.0-5549.g92715cd3.el9 3.4 MB/s | 301 kB 00:00 2026-02-25T11:41:45.969 INFO:teuthology.orchestra.run.trial028.stdout:(39/172): ceph-mgr-diskprediction-local-20.3.0- 8.9 MB/s | 7.4 MB 00:00 2026-02-25T11:41:45.980 INFO:teuthology.orchestra.run.trial028.stdout:(40/172): cephadm-20.3.0-5549.g92715cd3.el9.noa 3.5 MB/s | 1.0 MB 00:00 2026-02-25T11:41:46.004 INFO:teuthology.orchestra.run.trial028.stdout:(41/172): cephfs-top-20.3.0-5549.g92715cd3.el9. 524 kB/s | 18 kB 00:00 2026-02-25T11:41:46.153 INFO:teuthology.orchestra.run.trial028.stdout:(42/172): cryptsetup-2.8.1-3.el9.x86_64.rpm 2.0 MB/s | 351 kB 00:00 2026-02-25T11:41:46.180 INFO:teuthology.orchestra.run.trial028.stdout:(43/172): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.5 MB/s | 40 kB 00:00 2026-02-25T11:41:46.205 INFO:teuthology.orchestra.run.trial028.stdout:(44/172): libatomic-11.5.0-14.el9.x86_64.rpm 907 kB/s | 23 kB 00:00 2026-02-25T11:41:46.233 INFO:teuthology.orchestra.run.trial028.stdout:(45/172): libconfig-1.7.2-9.el9.x86_64.rpm 2.6 MB/s | 72 kB 00:00 2026-02-25T11:41:46.238 INFO:teuthology.orchestra.run.trial028.stdout:(46/172): hwloc-libs-2.4.1-6.el9.x86_64.rpm 9.0 MB/s | 2.1 MB 00:00 2026-02-25T11:41:46.260 INFO:teuthology.orchestra.run.trial119.stdout:(48/172): libquadmath-11.5.0-14.el9.x86_64.rpm 251 kB/s | 184 kB 00:00 2026-02-25T11:41:46.266 INFO:teuthology.orchestra.run.trial028.stdout:(47/172): libquadmath-11.5.0-14.el9.x86_64.rpm 6.6 MB/s | 184 kB 00:00 2026-02-25T11:41:46.293 INFO:teuthology.orchestra.run.trial028.stdout:(48/172): lksctp-tools-1.0.19-2.el9.x86_64.rpm 3.5 MB/s | 94 kB 00:00 2026-02-25T11:41:46.296 INFO:teuthology.orchestra.run.trial028.stdout:(49/172): libgfortran-11.5.0-14.el9.x86_64.rpm 12 MB/s | 794 kB 00:00 2026-02-25T11:41:46.318 INFO:teuthology.orchestra.run.trial028.stdout:(50/172): mailcap-2.1.49-5.el9.noarch.rpm 1.3 MB/s | 33 kB 00:00 2026-02-25T11:41:46.323 INFO:teuthology.orchestra.run.trial028.stdout:(51/172): pciutils-3.7.0-7.el9.x86_64.rpm 3.4 MB/s | 93 kB 00:00 2026-02-25T11:41:46.337 INFO:teuthology.orchestra.run.trial119.stdout:(49/172): lksctp-tools-1.0.19-2.el9.x86_64.rpm 144 kB/s | 94 kB 00:00 2026-02-25T11:41:46.342 INFO:teuthology.orchestra.run.trial119.stdout:(50/172): libgfortran-11.5.0-14.el9.x86_64.rpm 885 kB/s | 794 kB 00:00 2026-02-25T11:41:46.346 INFO:teuthology.orchestra.run.trial028.stdout:(52/172): python3-cffi-1.14.5-5.el9.x86_64.rpm 9.1 MB/s | 253 kB 00:00 2026-02-25T11:41:46.373 INFO:teuthology.orchestra.run.trial028.stdout:(53/172): python3-ply-3.11-14.el9.noarch.rpm 3.8 MB/s | 106 kB 00:00 2026-02-25T11:41:46.377 INFO:teuthology.orchestra.run.trial028.stdout:(54/172): python3-cryptography-36.0.1-5.el9.x86 23 MB/s | 1.2 MB 00:00 2026-02-25T11:41:46.400 INFO:teuthology.orchestra.run.trial028.stdout:(55/172): python3-pycparser-2.20-6.el9.noarch.r 4.9 MB/s | 135 kB 00:00 2026-02-25T11:41:46.405 INFO:teuthology.orchestra.run.trial028.stdout:(56/172): python3-pyparsing-2.4.7-9.el9.noarch. 5.3 MB/s | 150 kB 00:00 2026-02-25T11:41:46.428 INFO:teuthology.orchestra.run.trial028.stdout:(57/172): python3-requests-2.25.1-10.el9.noarch 4.5 MB/s | 126 kB 00:00 2026-02-25T11:41:46.433 INFO:teuthology.orchestra.run.trial028.stdout:(58/172): python3-urllib3-1.26.5-7.el9.noarch.r 7.5 MB/s | 218 kB 00:00 2026-02-25T11:41:46.454 INFO:teuthology.orchestra.run.trial028.stdout:(59/172): unzip-6.0-59.el9.x86_64.rpm 6.8 MB/s | 182 kB 00:00 2026-02-25T11:41:46.461 INFO:teuthology.orchestra.run.trial028.stdout:(60/172): zip-3.0-35.el9.x86_64.rpm 9.3 MB/s | 266 kB 00:00 2026-02-25T11:41:46.586 INFO:teuthology.orchestra.run.trial028.stdout:(61/172): automake-1.16.2-8.el9.noarch.rpm 5.1 MB/s | 689 kB 00:00 2026-02-25T11:41:46.590 INFO:teuthology.orchestra.run.trial028.stdout:(62/172): bison-3.7.4-5.el9.x86_64.rpm 7.2 MB/s | 943 kB 00:00 2026-02-25T11:41:46.609 INFO:teuthology.orchestra.run.trial028.stdout:(63/172): boost-program-options-1.75.0-13.el9.x 4.4 MB/s | 104 kB 00:00 2026-02-25T11:41:46.613 INFO:teuthology.orchestra.run.trial028.stdout:(64/172): elfutils-libelf-devel-0.194-1.el9.x86 3.4 MB/s | 79 kB 00:00 2026-02-25T11:41:46.633 INFO:teuthology.orchestra.run.trial028.stdout:(65/172): flex-2.6.4-9.el9.x86_64.rpm 13 MB/s | 313 kB 00:00 2026-02-25T11:41:46.635 INFO:teuthology.orchestra.run.trial028.stdout:(66/172): flexiblas-3.0.4-9.el9.x86_64.rpm 1.3 MB/s | 30 kB 00:00 2026-02-25T11:41:46.657 INFO:teuthology.orchestra.run.trial028.stdout:(67/172): flexiblas-openblas-openmp-3.0.4-9.el9 676 kB/s | 15 kB 00:00 2026-02-25T11:41:46.678 INFO:teuthology.orchestra.run.trial028.stdout:(68/172): libaio-devel-0.3.111-13.el9.x86_64.rp 519 kB/s | 11 kB 00:00 2026-02-25T11:41:46.701 INFO:teuthology.orchestra.run.trial028.stdout:(69/172): libnbd-1.20.3-4.el9.x86_64.rpm 7.2 MB/s | 164 kB 00:00 2026-02-25T11:41:46.725 INFO:teuthology.orchestra.run.trial028.stdout:(70/172): libpmemobj-1.12.1-1.el9.x86_64.rpm 6.6 MB/s | 160 kB 00:00 2026-02-25T11:41:46.747 INFO:teuthology.orchestra.run.trial028.stdout:(71/172): librabbitmq-0.11.0-7.el9.x86_64.rpm 2.0 MB/s | 45 kB 00:00 2026-02-25T11:41:46.765 INFO:teuthology.orchestra.run.trial028.stdout:(72/172): flexiblas-netlib-3.0.4-9.el9.x86_64.r 23 MB/s | 3.0 MB 00:00 2026-02-25T11:41:46.790 INFO:teuthology.orchestra.run.trial028.stdout:(73/172): libstoragemgmt-1.10.1-1.el9.x86_64.rp 9.9 MB/s | 246 kB 00:00 2026-02-25T11:41:46.792 INFO:teuthology.orchestra.run.trial028.stdout:(74/172): librdkafka-1.6.1-102.el9.x86_64.rpm 14 MB/s | 662 kB 00:00 2026-02-25T11:41:46.814 INFO:teuthology.orchestra.run.trial028.stdout:(75/172): libuuid-devel-2.37.4-25.el9.x86_64.rp 1.0 MB/s | 23 kB 00:00 2026-02-25T11:41:46.823 INFO:teuthology.orchestra.run.trial119.stdout:(51/172): mailcap-2.1.49-5.el9.noarch.rpm 59 kB/s | 33 kB 00:00 2026-02-25T11:41:46.823 INFO:teuthology.orchestra.run.trial028.stdout:(76/172): libtool-2.4.6-46.el9.x86_64.rpm 17 MB/s | 581 kB 00:00 2026-02-25T11:41:46.838 INFO:teuthology.orchestra.run.trial028.stdout:(77/172): libxslt-1.1.34-12.el9.x86_64.rpm 9.5 MB/s | 233 kB 00:00 2026-02-25T11:41:46.845 INFO:teuthology.orchestra.run.trial028.stdout:(78/172): libzstd-devel-1.5.5-1.el9.x86_64.rpm 2.2 MB/s | 51 kB 00:00 2026-02-25T11:41:46.863 INFO:teuthology.orchestra.run.trial028.stdout:(79/172): lttng-ust-2.12.0-6.el9.x86_64.rpm 11 MB/s | 292 kB 00:00 2026-02-25T11:41:46.870 INFO:teuthology.orchestra.run.trial028.stdout:(80/172): lua-5.4.4-4.el9.x86_64.rpm 7.7 MB/s | 188 kB 00:00 2026-02-25T11:41:46.886 INFO:teuthology.orchestra.run.trial028.stdout:(81/172): openblas-0.3.29-1.el9.x86_64.rpm 1.9 MB/s | 42 kB 00:00 2026-02-25T11:41:46.995 INFO:teuthology.orchestra.run.trial119.stdout:(52/172): pciutils-3.7.0-7.el9.x86_64.rpm 142 kB/s | 93 kB 00:00 2026-02-25T11:41:47.114 INFO:teuthology.orchestra.run.trial119.stdout:(53/172): python3-cffi-1.14.5-5.el9.x86_64.rpm 328 kB/s | 253 kB 00:00 2026-02-25T11:41:47.128 INFO:teuthology.orchestra.run.trial028.stdout:(82/172): openssl-devel-3.5.5-1.el9.x86_64.rpm 20 MB/s | 4.8 MB 00:00 2026-02-25T11:41:47.145 INFO:teuthology.orchestra.run.trial028.stdout:(83/172): openblas-openmp-0.3.29-1.el9.x86_64.r 19 MB/s | 5.3 MB 00:00 2026-02-25T11:41:47.150 INFO:teuthology.orchestra.run.trial028.stdout:(84/172): perl-Thread-Queue-3.14-460.el9.noarch 1.0 MB/s | 22 kB 00:00 2026-02-25T11:41:47.167 INFO:teuthology.orchestra.run.trial028.stdout:(85/172): perl-threads-2.25-460.el9.x86_64.rpm 2.6 MB/s | 58 kB 00:00 2026-02-25T11:41:47.172 INFO:teuthology.orchestra.run.trial028.stdout:(86/172): perl-threads-shared-1.61-460.el9.x86_ 2.0 MB/s | 45 kB 00:00 2026-02-25T11:41:47.174 INFO:teuthology.orchestra.run.trial167.stdout:(34/172): ceph-mgr-diskprediction-local-20.3.0- 4.6 MB/s | 7.4 MB 00:01 2026-02-25T11:41:47.220 INFO:teuthology.orchestra.run.trial028.stdout:(87/172): postgresql-contrib-13.23-1.el9.x86_64 19 MB/s | 903 kB 00:00 2026-02-25T11:41:47.238 INFO:teuthology.orchestra.run.trial028.stdout:(88/172): postgresql-13.23-1.el9.x86_64.rpm 23 MB/s | 1.6 MB 00:00 2026-02-25T11:41:47.244 INFO:teuthology.orchestra.run.trial028.stdout:(89/172): postgresql-private-libs-13.23-1.el9.x 5.7 MB/s | 138 kB 00:00 2026-02-25T11:41:47.261 INFO:teuthology.orchestra.run.trial167.stdout:(35/172): ceph-mgr-modules-core-20.3.0-5549.g92 3.4 MB/s | 298 kB 00:00 2026-02-25T11:41:47.290 INFO:teuthology.orchestra.run.trial028.stdout:(90/172): protobuf-3.14.0-17.el9.x86_64.rpm 22 MB/s | 1.0 MB 00:00 2026-02-25T11:41:47.302 INFO:teuthology.orchestra.run.trial167.stdout:(36/172): ceph-mgr-rook-20.3.0-5549.g92715cd3.e 1.2 MB/s | 50 kB 00:00 2026-02-25T11:41:47.355 INFO:teuthology.orchestra.run.trial167.stdout:(37/172): ceph-mgr-dashboard-20.3.0-5549.g92715 6.5 MB/s | 15 MB 00:02 2026-02-25T11:41:47.355 INFO:teuthology.orchestra.run.trial167.stdout:(38/172): ceph-prometheus-alerts-20.3.0-5549.g9 324 kB/s | 17 kB 00:00 2026-02-25T11:41:47.439 INFO:teuthology.orchestra.run.trial167.stdout:(39/172): ceph-volume-20.3.0-5549.g92715cd3.el9 3.5 MB/s | 301 kB 00:00 2026-02-25T11:41:47.458 INFO:teuthology.orchestra.run.trial028.stdout:(91/172): postgresql-server-13.23-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2026-02-25T11:41:47.482 INFO:teuthology.orchestra.run.trial167.stdout:(40/172): cephfs-top-20.3.0-5549.g92715cd3.el9. 433 kB/s | 18 kB 00:00 2026-02-25T11:41:47.482 INFO:teuthology.orchestra.run.trial028.stdout:(92/172): python3-colorama-0.4.6-3.el9.noarch.r 2.6 MB/s | 61 kB 00:00 2026-02-25T11:41:47.514 INFO:teuthology.orchestra.run.trial028.stdout:(93/172): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2026-02-25T11:41:47.517 INFO:teuthology.orchestra.run.trial028.stdout:(94/172): python3-devel-3.9.25-3.el9.x86_64.rpm 6.9 MB/s | 244 kB 00:00 2026-02-25T11:41:47.539 INFO:teuthology.orchestra.run.trial028.stdout:(95/172): python3-jmespath-1.0.1-1.el9.noarch.r 2.1 MB/s | 48 kB 00:00 2026-02-25T11:41:47.541 INFO:teuthology.orchestra.run.trial028.stdout:(96/172): python3-jinja2-2.11.3-8.el9.noarch.rp 9.1 MB/s | 249 kB 00:00 2026-02-25T11:41:47.564 INFO:teuthology.orchestra.run.trial028.stdout:(97/172): python3-libstoragemgmt-1.10.1-1.el9.x 7.2 MB/s | 177 kB 00:00 2026-02-25T11:41:47.564 INFO:teuthology.orchestra.run.trial028.stdout:(98/172): python3-markupsafe-1.1.1-12.el9.x86_6 1.5 MB/s | 35 kB 00:00 2026-02-25T11:41:47.591 INFO:teuthology.orchestra.run.trial028.stdout:(99/172): python3-numpy-f2py-1.23.5-2.el9.x86_6 16 MB/s | 442 kB 00:00 2026-02-25T11:41:47.609 INFO:teuthology.orchestra.run.trial167.stdout:(41/172): cephadm-20.3.0-5549.g92715cd3.el9.noa 4.0 MB/s | 1.0 MB 00:00 2026-02-25T11:41:47.615 INFO:teuthology.orchestra.run.trial028.stdout:(100/172): python3-packaging-20.9-5.el9.noarch. 3.2 MB/s | 77 kB 00:00 2026-02-25T11:41:47.639 INFO:teuthology.orchestra.run.trial028.stdout:(101/172): python3-protobuf-3.14.0-17.el9.noarc 11 MB/s | 267 kB 00:00 2026-02-25T11:41:47.665 INFO:teuthology.orchestra.run.trial028.stdout:(102/172): python3-pyasn1-0.4.8-7.el9.noarch.rp 6.1 MB/s | 157 kB 00:00 2026-02-25T11:41:47.666 INFO:teuthology.orchestra.run.trial119.stdout:(54/172): python3-ply-3.11-14.el9.noarch.rpm 159 kB/s | 106 kB 00:00 2026-02-25T11:41:47.691 INFO:teuthology.orchestra.run.trial028.stdout:(103/172): python3-pyasn1-modules-0.4.8-7.el9.n 10 MB/s | 277 kB 00:00 2026-02-25T11:41:47.715 INFO:teuthology.orchestra.run.trial028.stdout:(104/172): python3-requests-oauthlib-1.3.0-12.e 2.3 MB/s | 54 kB 00:00 2026-02-25T11:41:47.774 INFO:teuthology.orchestra.run.trial119.stdout:(55/172): python3-cryptography-36.0.1-5.el9.x86 1.3 MB/s | 1.2 MB 00:00 2026-02-25T11:41:47.789 INFO:teuthology.orchestra.run.trial028.stdout:(105/172): python3-numpy-1.23.5-2.el9.x86_64.rp 27 MB/s | 6.1 MB 00:00 2026-02-25T11:41:47.812 INFO:teuthology.orchestra.run.trial028.stdout:(106/172): python3-toml-0.10.2-6.el9.noarch.rpm 1.8 MB/s | 42 kB 00:00 2026-02-25T11:41:47.820 INFO:teuthology.orchestra.run.trial119.stdout:(56/172): python3-pycparser-2.20-6.el9.noarch.r 191 kB/s | 135 kB 00:00 2026-02-25T11:41:47.834 INFO:teuthology.orchestra.run.trial028.stdout:(107/172): python3-wcwidth-0.2.5-8.el9.noarch.r 1.9 MB/s | 44 kB 00:00 2026-02-25T11:41:47.859 INFO:teuthology.orchestra.run.trial028.stdout:(108/172): qatlib-25.08.0-2.el9.x86_64.rpm 9.8 MB/s | 240 kB 00:00 2026-02-25T11:41:47.881 INFO:teuthology.orchestra.run.trial028.stdout:(109/172): qatlib-service-25.08.0-2.el9.x86_64. 1.6 MB/s | 37 kB 00:00 2026-02-25T11:41:47.904 INFO:teuthology.orchestra.run.trial028.stdout:(110/172): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.9 MB/s | 66 kB 00:00 2026-02-25T11:41:47.928 INFO:teuthology.orchestra.run.trial028.stdout:(111/172): socat-1.7.4.1-8.el9.x86_64.rpm 12 MB/s | 303 kB 00:00 2026-02-25T11:41:47.951 INFO:teuthology.orchestra.run.trial028.stdout:(112/172): uuid-1.6.2-55.el9.x86_64.rpm 2.5 MB/s | 58 kB 00:00 2026-02-25T11:41:47.974 INFO:teuthology.orchestra.run.trial028.stdout:(113/172): xfsprogs-devel-6.4.0-7.el9.x86_64.rp 4.1 MB/s | 96 kB 00:00 2026-02-25T11:41:47.997 INFO:teuthology.orchestra.run.trial028.stdout:(114/172): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.8 MB/s | 64 kB 00:00 2026-02-25T11:41:48.019 INFO:teuthology.orchestra.run.trial028.stdout:(115/172): zlib-devel-1.2.11-41.el9.x86_64.rpm 2.0 MB/s | 45 kB 00:00 2026-02-25T11:41:48.381 INFO:teuthology.orchestra.run.trial119.stdout:(57/172): python3-pyparsing-2.4.7-9.el9.noarch. 211 kB/s | 150 kB 00:00 2026-02-25T11:41:48.458 INFO:teuthology.orchestra.run.trial028.stdout:(116/172): lua-devel-5.4.4-4.el9.x86_64.rpm 51 kB/s | 22 kB 00:00 2026-02-25T11:41:48.472 INFO:teuthology.orchestra.run.trial119.stdout:(58/172): python3-requests-2.25.1-10.el9.noarch 181 kB/s | 126 kB 00:00 2026-02-25T11:41:48.565 INFO:teuthology.orchestra.run.trial028.stdout:(117/172): python3-scipy-1.9.3-2.el9.x86_64.rpm 23 MB/s | 19 MB 00:00 2026-02-25T11:41:48.574 INFO:teuthology.orchestra.run.trial119.stdout:(59/172): python3-urllib3-1.26.5-7.el9.noarch.r 289 kB/s | 218 kB 00:00 2026-02-25T11:41:48.638 INFO:teuthology.orchestra.run.trial028.stdout:(118/172): abseil-cpp-20211102.0-4.el9.x86_64.r 7.4 MB/s | 551 kB 00:00 2026-02-25T11:41:48.649 INFO:teuthology.orchestra.run.trial119.stdout:(60/172): automake-1.16.2-8.el9.noarch.rpm 9.0 MB/s | 689 kB 00:00 2026-02-25T11:41:48.661 INFO:teuthology.orchestra.run.trial028.stdout:(119/172): gperftools-libs-2.9.1-3.el9.x86_64.r 13 MB/s | 308 kB 00:00 2026-02-25T11:41:48.669 INFO:teuthology.orchestra.run.trial028.stdout:(120/172): grpc-data-1.46.7-10.el9.noarch.rpm 2.5 MB/s | 19 kB 00:00 2026-02-25T11:41:48.686 INFO:teuthology.orchestra.run.trial119.stdout:(61/172): bison-3.7.4-5.el9.x86_64.rpm 25 MB/s | 943 kB 00:00 2026-02-25T11:41:48.716 INFO:teuthology.orchestra.run.trial119.stdout:(62/172): boost-program-options-1.75.0-13.el9.x 3.4 MB/s | 104 kB 00:00 2026-02-25T11:41:48.741 INFO:teuthology.orchestra.run.trial119.stdout:(63/172): elfutils-libelf-devel-0.194-1.el9.x86 3.1 MB/s | 79 kB 00:00 2026-02-25T11:41:48.769 INFO:teuthology.orchestra.run.trial119.stdout:(64/172): flex-2.6.4-9.el9.x86_64.rpm 11 MB/s | 313 kB 00:00 2026-02-25T11:41:48.793 INFO:teuthology.orchestra.run.trial119.stdout:(65/172): flexiblas-3.0.4-9.el9.x86_64.rpm 1.2 MB/s | 30 kB 00:00 2026-02-25T11:41:48.834 INFO:teuthology.orchestra.run.trial028.stdout:(121/172): libarrow-9.0.0-15.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2026-02-25T11:41:48.838 INFO:teuthology.orchestra.run.trial119.stdout:(66/172): flexiblas-netlib-3.0.4-9.el9.x86_64.r 67 MB/s | 3.0 MB 00:00 2026-02-25T11:41:48.845 INFO:teuthology.orchestra.run.trial028.stdout:(122/172): libarrow-doc-9.0.0-15.el9.noarch.rpm 2.2 MB/s | 25 kB 00:00 2026-02-25T11:41:48.869 INFO:teuthology.orchestra.run.trial119.stdout:(67/172): flexiblas-openblas-openmp-3.0.4-9.el9 474 kB/s | 15 kB 00:00 2026-02-25T11:41:48.892 INFO:teuthology.orchestra.run.trial119.stdout:(68/172): libaio-devel-0.3.111-13.el9.x86_64.rp 483 kB/s | 11 kB 00:00 2026-02-25T11:41:48.902 INFO:teuthology.orchestra.run.trial028.stdout:(123/172): liboath-2.6.12-1.el9.x86_64.rpm 857 kB/s | 49 kB 00:00 2026-02-25T11:41:48.912 INFO:teuthology.orchestra.run.trial028.stdout:(124/172): libunwind-1.6.2-1.el9.x86_64.rpm 6.9 MB/s | 67 kB 00:00 2026-02-25T11:41:48.921 INFO:teuthology.orchestra.run.trial119.stdout:(69/172): libnbd-1.20.3-4.el9.x86_64.rpm 5.7 MB/s | 164 kB 00:00 2026-02-25T11:41:48.921 INFO:teuthology.orchestra.run.trial028.stdout:(125/172): luarocks-3.9.2-5.el9.noarch.rpm 15 MB/s | 151 kB 00:00 2026-02-25T11:41:48.946 INFO:teuthology.orchestra.run.trial119.stdout:(70/172): libpmemobj-1.12.1-1.el9.x86_64.rpm 6.3 MB/s | 160 kB 00:00 2026-02-25T11:41:48.968 INFO:teuthology.orchestra.run.trial119.stdout:(71/172): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.9 MB/s | 45 kB 00:00 2026-02-25T11:41:48.993 INFO:teuthology.orchestra.run.trial119.stdout:(72/172): librdkafka-1.6.1-102.el9.x86_64.rpm 26 MB/s | 662 kB 00:00 2026-02-25T11:41:49.019 INFO:teuthology.orchestra.run.trial119.stdout:(73/172): libstoragemgmt-1.10.1-1.el9.x86_64.rp 9.4 MB/s | 246 kB 00:00 2026-02-25T11:41:49.022 INFO:teuthology.orchestra.run.trial167.stdout:(42/172): cryptsetup-2.8.1-3.el9.x86_64.rpm 228 kB/s | 351 kB 00:01 2026-02-25T11:41:49.049 INFO:teuthology.orchestra.run.trial119.stdout:(74/172): libtool-2.4.6-46.el9.x86_64.rpm 19 MB/s | 581 kB 00:00 2026-02-25T11:41:49.073 INFO:teuthology.orchestra.run.trial119.stdout:(75/172): libuuid-devel-2.37.4-25.el9.x86_64.rp 941 kB/s | 23 kB 00:00 2026-02-25T11:41:49.098 INFO:teuthology.orchestra.run.trial119.stdout:(76/172): libxslt-1.1.34-12.el9.x86_64.rpm 9.4 MB/s | 233 kB 00:00 2026-02-25T11:41:49.118 INFO:teuthology.orchestra.run.trial119.stdout:(77/172): unzip-6.0-59.el9.x86_64.rpm 247 kB/s | 182 kB 00:00 2026-02-25T11:41:49.123 INFO:teuthology.orchestra.run.trial028.stdout:(126/172): ceph-test-20.3.0-5549.g92715cd3.el9. 7.1 MB/s | 76 MB 00:10 2026-02-25T11:41:49.124 INFO:teuthology.orchestra.run.trial119.stdout:(78/172): libzstd-devel-1.5.5-1.el9.x86_64.rpm 1.9 MB/s | 51 kB 00:00 2026-02-25T11:41:49.125 INFO:teuthology.orchestra.run.trial028.stdout:(127/172): parquet-libs-9.0.0-15.el9.x86_64.rpm 4.0 MB/s | 838 kB 00:00 2026-02-25T11:41:49.141 INFO:teuthology.orchestra.run.trial028.stdout:(128/172): python3-autocommand-2.2.2-8.el9.noar 1.9 MB/s | 29 kB 00:00 2026-02-25T11:41:49.145 INFO:teuthology.orchestra.run.trial119.stdout:(79/172): lttng-ust-2.12.0-6.el9.x86_64.rpm 10 MB/s | 292 kB 00:00 2026-02-25T11:41:49.149 INFO:teuthology.orchestra.run.trial119.stdout:(80/172): lua-5.4.4-4.el9.x86_64.rpm 7.4 MB/s | 188 kB 00:00 2026-02-25T11:41:49.150 INFO:teuthology.orchestra.run.trial028.stdout:(129/172): python3-backports-tarfile-1.2.0-1.el 6.4 MB/s | 60 kB 00:00 2026-02-25T11:41:49.159 INFO:teuthology.orchestra.run.trial028.stdout:(130/172): python3-bcrypt-3.2.2-1.el9.x86_64.rp 4.9 MB/s | 43 kB 00:00 2026-02-25T11:41:49.167 INFO:teuthology.orchestra.run.trial028.stdout:(131/172): python3-cachetools-4.2.4-1.el9.noarc 3.7 MB/s | 32 kB 00:00 2026-02-25T11:41:49.171 INFO:teuthology.orchestra.run.trial119.stdout:(81/172): openblas-0.3.29-1.el9.x86_64.rpm 1.6 MB/s | 42 kB 00:00 2026-02-25T11:41:49.175 INFO:teuthology.orchestra.run.trial028.stdout:(132/172): python3-certifi-2023.05.07-4.el9.noa 1.7 MB/s | 14 kB 00:00 2026-02-25T11:41:49.187 INFO:teuthology.orchestra.run.trial028.stdout:(133/172): python3-cheroot-10.0.1-4.el9.noarch. 15 MB/s | 173 kB 00:00 2026-02-25T11:41:49.200 INFO:teuthology.orchestra.run.trial167.stdout:(43/172): ledmon-libs-1.1.0-3.el9.x86_64.rpm 227 kB/s | 40 kB 00:00 2026-02-25T11:41:49.200 INFO:teuthology.orchestra.run.trial028.stdout:(134/172): python3-asyncssh-2.13.2-5.el9.noarch 6.9 MB/s | 548 kB 00:00 2026-02-25T11:41:49.205 INFO:teuthology.orchestra.run.trial119.stdout:(82/172): openblas-openmp-0.3.29-1.el9.x86_64.r 94 MB/s | 5.3 MB 00:00 2026-02-25T11:41:49.218 INFO:teuthology.orchestra.run.trial028.stdout:(135/172): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2026-02-25T11:41:49.229 INFO:teuthology.orchestra.run.trial028.stdout:(136/172): python3-cmd2-2.4.2-2.el9.noarch.rpm 10 MB/s | 303 kB 00:00 2026-02-25T11:41:49.231 INFO:teuthology.orchestra.run.trial119.stdout:(83/172): openssl-devel-3.5.5-1.el9.x86_64.rpm 81 MB/s | 4.8 MB 00:00 2026-02-25T11:41:49.236 INFO:teuthology.orchestra.run.trial028.stdout:(137/172): python3-google-auth-2.45.0-1.el9.noa 14 MB/s | 254 kB 00:00 2026-02-25T11:41:49.238 INFO:teuthology.orchestra.run.trial119.stdout:(84/172): perl-Thread-Queue-3.14-460.el9.noarch 749 kB/s | 22 kB 00:00 2026-02-25T11:41:49.248 INFO:teuthology.orchestra.run.trial028.stdout:(138/172): python3-grpcio-tools-1.46.7-10.el9.x 12 MB/s | 144 kB 00:00 2026-02-25T11:41:49.250 INFO:teuthology.orchestra.run.trial119.stdout:(85/172): zip-3.0-35.el9.x86_64.rpm 342 kB/s | 266 kB 00:00 2026-02-25T11:41:49.256 INFO:teuthology.orchestra.run.trial028.stdout:(139/172): python3-jaraco-8.2.1-3.el9.noarch.rp 1.3 MB/s | 11 kB 00:00 2026-02-25T11:41:49.257 INFO:teuthology.orchestra.run.trial119.stdout:(86/172): perl-threads-2.25-460.el9.x86_64.rpm 2.2 MB/s | 58 kB 00:00 2026-02-25T11:41:49.265 INFO:teuthology.orchestra.run.trial028.stdout:(140/172): python3-jaraco-classes-3.2.1-5.el9.n 2.1 MB/s | 18 kB 00:00 2026-02-25T11:41:49.273 INFO:teuthology.orchestra.run.trial028.stdout:(141/172): python3-jaraco-collections-3.0.0-8.e 2.8 MB/s | 23 kB 00:00 2026-02-25T11:41:49.282 INFO:teuthology.orchestra.run.trial028.stdout:(142/172): python3-jaraco-context-6.0.1-3.el9.n 2.3 MB/s | 20 kB 00:00 2026-02-25T11:41:49.285 INFO:teuthology.orchestra.run.trial119.stdout:(87/172): postgresql-13.23-1.el9.x86_64.rpm 47 MB/s | 1.6 MB 00:00 2026-02-25T11:41:49.289 INFO:teuthology.orchestra.run.trial119.stdout:(88/172): perl-threads-shared-1.61-460.el9.x86_ 887 kB/s | 45 kB 00:00 2026-02-25T11:41:49.290 INFO:teuthology.orchestra.run.trial028.stdout:(143/172): python3-jaraco-functools-3.5.0-2.el9 2.3 MB/s | 19 kB 00:00 2026-02-25T11:41:49.291 INFO:teuthology.orchestra.run.trial119.stdout:(89/172): postgresql-contrib-13.23-1.el9.x86_64 26 MB/s | 903 kB 00:00 2026-02-25T11:41:49.295 INFO:teuthology.orchestra.run.trial028.stdout:(144/172): python3-grpcio-1.46.7-10.el9.x86_64. 31 MB/s | 2.0 MB 00:00 2026-02-25T11:41:49.298 INFO:teuthology.orchestra.run.trial028.stdout:(145/172): python3-jaraco-text-4.0.0-2.el9.noar 3.1 MB/s | 26 kB 00:00 2026-02-25T11:41:49.300 INFO:teuthology.orchestra.run.trial167.stdout:(44/172): libatomic-11.5.0-14.el9.x86_64.rpm 229 kB/s | 23 kB 00:00 2026-02-25T11:41:49.312 INFO:teuthology.orchestra.run.trial119.stdout:(90/172): postgresql-private-libs-13.23-1.el9.x 5.6 MB/s | 138 kB 00:00 2026-02-25T11:41:49.313 INFO:teuthology.orchestra.run.trial028.stdout:(146/172): python3-more-itertools-8.12.0-2.el9. 5.3 MB/s | 79 kB 00:00 2026-02-25T11:41:49.323 INFO:teuthology.orchestra.run.trial028.stdout:(147/172): python3-natsort-7.1.1-5.el9.noarch.r 5.7 MB/s | 58 kB 00:00 2026-02-25T11:41:49.331 INFO:teuthology.orchestra.run.trial119.stdout:(91/172): protobuf-3.14.0-17.el9.x86_64.rpm 25 MB/s | 1.0 MB 00:00 2026-02-25T11:41:49.336 INFO:teuthology.orchestra.run.trial028.stdout:(148/172): python3-portend-3.1.0-2.el9.noarch.r 1.2 MB/s | 16 kB 00:00 2026-02-25T11:41:49.344 INFO:teuthology.orchestra.run.trial028.stdout:(149/172): python3-kubernetes-26.1.0-3.el9.noar 21 MB/s | 1.0 MB 00:00 2026-02-25T11:41:49.347 INFO:teuthology.orchestra.run.trial028.stdout:(150/172): python3-pyOpenSSL-21.0.0-1.el9.noarc 8.8 MB/s | 90 kB 00:00 2026-02-25T11:41:49.354 INFO:teuthology.orchestra.run.trial028.stdout:(151/172): python3-pyperclip-1.8.2-2.el9.noarch 2.7 MB/s | 28 kB 00:00 2026-02-25T11:41:49.355 INFO:teuthology.orchestra.run.trial028.stdout:(152/172): python3-repoze-lru-0.7-16.el9.noarch 3.6 MB/s | 31 kB 00:00 2026-02-25T11:41:49.357 INFO:teuthology.orchestra.run.trial119.stdout:(92/172): postgresql-server-13.23-1.el9.x86_64. 87 MB/s | 5.8 MB 00:00 2026-02-25T11:41:49.365 INFO:teuthology.orchestra.run.trial028.stdout:(153/172): python3-routes-2.5.1-5.el9.noarch.rp 17 MB/s | 188 kB 00:00 2026-02-25T11:41:49.366 INFO:teuthology.orchestra.run.trial028.stdout:(154/172): python3-rsa-4.9-2.el9.noarch.rpm 5.2 MB/s | 59 kB 00:00 2026-02-25T11:41:49.371 INFO:teuthology.orchestra.run.trial119.stdout:(93/172): python3-colorama-0.4.6-3.el9.noarch.r 1.6 MB/s | 61 kB 00:00 2026-02-25T11:41:49.375 INFO:teuthology.orchestra.run.trial028.stdout:(155/172): python3-tempora-5.0.0-2.el9.noarch.r 3.7 MB/s | 36 kB 00:00 2026-02-25T11:41:49.378 INFO:teuthology.orchestra.run.trial028.stdout:(156/172): python3-typing-extensions-4.15.0-1.e 7.5 MB/s | 86 kB 00:00 2026-02-25T11:41:49.385 INFO:teuthology.orchestra.run.trial028.stdout:(157/172): python3-websocket-client-1.2.3-2.el9 9.1 MB/s | 90 kB 00:00 2026-02-25T11:41:49.387 INFO:teuthology.orchestra.run.trial119.stdout:(94/172): python3-babel-2.9.1-2.el9.noarch.rpm 80 MB/s | 6.0 MB 00:00 2026-02-25T11:41:49.389 INFO:teuthology.orchestra.run.trial119.stdout:(95/172): python3-devel-3.9.25-3.el9.x86_64.rpm 8.5 MB/s | 244 kB 00:00 2026-02-25T11:41:49.391 INFO:teuthology.orchestra.run.trial028.stdout:(158/172): python3-xmltodict-0.12.0-15.el9.noar 1.7 MB/s | 22 kB 00:00 2026-02-25T11:41:49.393 INFO:teuthology.orchestra.run.trial028.stdout:(159/172): python3-zc-lockfile-2.0-10.el9.noarc 2.3 MB/s | 20 kB 00:00 2026-02-25T11:41:49.400 INFO:teuthology.orchestra.run.trial119.stdout:(96/172): python3-jinja2-2.11.3-8.el9.noarch.rp 9.6 MB/s | 249 kB 00:00 2026-02-25T11:41:49.411 INFO:teuthology.orchestra.run.trial119.stdout:(97/172): python3-jmespath-1.0.1-1.el9.noarch.r 1.9 MB/s | 48 kB 00:00 2026-02-25T11:41:49.420 INFO:teuthology.orchestra.run.trial119.stdout:(98/172): python3-libstoragemgmt-1.10.1-1.el9.x 5.6 MB/s | 177 kB 00:00 2026-02-25T11:41:49.427 INFO:teuthology.orchestra.run.trial119.stdout:(99/172): python3-markupsafe-1.1.1-12.el9.x86_6 1.3 MB/s | 35 kB 00:00 2026-02-25T11:41:49.438 INFO:teuthology.orchestra.run.trial028.stdout:(160/172): thrift-0.15.0-4.el9.x86_64.rpm 35 MB/s | 1.6 MB 00:00 2026-02-25T11:41:49.447 INFO:teuthology.orchestra.run.trial119.stdout:(100/172): python3-numpy-f2py-1.23.5-2.el9.x86_ 16 MB/s | 442 kB 00:00 2026-02-25T11:41:49.450 INFO:teuthology.orchestra.run.trial119.stdout:(101/172): python3-packaging-20.9-5.el9.noarch. 3.2 MB/s | 77 kB 00:00 2026-02-25T11:41:49.453 INFO:teuthology.orchestra.run.trial028.stdout:(161/172): re2-20211101-20.el9.x86_64.rpm 3.0 MB/s | 191 kB 00:00 2026-02-25T11:41:49.454 INFO:teuthology.orchestra.run.trial028.stdout:(162/172): yaml-cpp-0.6.3-5.el9.x86_64.rpm 7.6 MB/s | 122 kB 00:00 2026-02-25T11:41:49.477 INFO:teuthology.orchestra.run.trial119.stdout:(102/172): python3-protobuf-3.14.0-17.el9.noarc 9.3 MB/s | 267 kB 00:00 2026-02-25T11:41:49.478 INFO:teuthology.orchestra.run.trial119.stdout:(103/172): python3-pyasn1-0.4.8-7.el9.noarch.rp 5.5 MB/s | 157 kB 00:00 2026-02-25T11:41:49.495 INFO:teuthology.orchestra.run.trial119.stdout:(104/172): python3-numpy-1.23.5-2.el9.x86_64.rp 73 MB/s | 6.1 MB 00:00 2026-02-25T11:41:49.503 INFO:teuthology.orchestra.run.trial119.stdout:(105/172): python3-pyasn1-modules-0.4.8-7.el9.n 11 MB/s | 277 kB 00:00 2026-02-25T11:41:49.505 INFO:teuthology.orchestra.run.trial119.stdout:(106/172): python3-requests-oauthlib-1.3.0-12.e 2.2 MB/s | 54 kB 00:00 2026-02-25T11:41:49.527 INFO:teuthology.orchestra.run.trial119.stdout:(107/172): python3-toml-0.10.2-6.el9.noarch.rpm 1.7 MB/s | 42 kB 00:00 2026-02-25T11:41:49.528 INFO:teuthology.orchestra.run.trial119.stdout:(108/172): python3-wcwidth-0.2.5-8.el9.noarch.r 1.8 MB/s | 44 kB 00:00 2026-02-25T11:41:49.556 INFO:teuthology.orchestra.run.trial119.stdout:(109/172): qatlib-service-25.08.0-2.el9.x86_64. 1.3 MB/s | 37 kB 00:00 2026-02-25T11:41:49.556 INFO:teuthology.orchestra.run.trial167.stdout:(45/172): libconfig-1.7.2-9.el9.x86_64.rpm 280 kB/s | 72 kB 00:00 2026-02-25T11:41:49.559 INFO:teuthology.orchestra.run.trial119.stdout:(110/172): qatlib-25.08.0-2.el9.x86_64.rpm 7.3 MB/s | 240 kB 00:00 2026-02-25T11:41:49.576 INFO:teuthology.orchestra.run.trial028.stdout:(163/172): protobuf-compiler-3.14.0-17.el9.x86_ 770 kB/s | 862 kB 00:01 2026-02-25T11:41:49.586 INFO:teuthology.orchestra.run.trial119.stdout:(111/172): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.2 MB/s | 66 kB 00:00 2026-02-25T11:41:49.591 INFO:teuthology.orchestra.run.trial119.stdout:(112/172): socat-1.7.4.1-8.el9.x86_64.rpm 11 MB/s | 303 kB 00:00 2026-02-25T11:41:49.603 INFO:teuthology.orchestra.run.trial028.stdout:(164/172): libblkid-2.37.4-25.el9.x86_64.rpm 3.8 MB/s | 105 kB 00:00 2026-02-25T11:41:49.617 INFO:teuthology.orchestra.run.trial119.stdout:(113/172): uuid-1.6.2-55.el9.x86_64.rpm 2.0 MB/s | 58 kB 00:00 2026-02-25T11:41:49.619 INFO:teuthology.orchestra.run.trial119.stdout:(114/172): xfsprogs-devel-6.4.0-7.el9.x86_64.rp 3.7 MB/s | 96 kB 00:00 2026-02-25T11:41:49.630 INFO:teuthology.orchestra.run.trial028.stdout:(165/172): libfdisk-2.37.4-25.el9.x86_64.rpm 5.6 MB/s | 152 kB 00:00 2026-02-25T11:41:49.657 INFO:teuthology.orchestra.run.trial028.stdout:(166/172): libmount-2.37.4-25.el9.x86_64.rpm 4.8 MB/s | 133 kB 00:00 2026-02-25T11:41:49.662 INFO:teuthology.orchestra.run.trial119.stdout:(115/172): python3-scipy-1.9.3-2.el9.x86_64.rpm 116 MB/s | 19 MB 00:00 2026-02-25T11:41:49.663 INFO:teuthology.orchestra.run.trial119.stdout:(116/172): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.4 MB/s | 64 kB 00:00 2026-02-25T11:41:49.663 INFO:teuthology.orchestra.run.trial119.stdout:(117/172): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2026-02-25T11:41:49.688 INFO:teuthology.orchestra.run.trial028.stdout:(167/172): librados2-20.3.0-5549.g92715cd3.el9. 15 MB/s | 3.5 MB 00:00 2026-02-25T11:41:49.688 INFO:teuthology.orchestra.run.trial028.stdout:(168/172): libsmartcols-2.37.4-25.el9.x86_64.rp 1.9 MB/s | 61 kB 00:00 2026-02-25T11:41:49.713 INFO:teuthology.orchestra.run.trial028.stdout:(169/172): libuuid-2.37.4-25.el9.x86_64.rpm 1.0 MB/s | 26 kB 00:00 2026-02-25T11:41:49.734 INFO:teuthology.orchestra.run.trial028.stdout:(170/172): util-linux-2.37.4-25.el9.x86_64.rpm 50 MB/s | 2.3 MB 00:00 2026-02-25T11:41:49.743 INFO:teuthology.orchestra.run.trial028.stdout:(171/172): util-linux-core-2.37.4-25.el9.x86_64 15 MB/s | 458 kB 00:00 2026-02-25T11:41:49.898 INFO:teuthology.orchestra.run.trial028.stdout:(172/172): librbd1-20.3.0-5549.g92715cd3.el9.x8 6.4 MB/s | 2.8 MB 00:00 2026-02-25T11:41:49.900 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-25T11:41:49.900 INFO:teuthology.orchestra.run.trial028.stdout:Total 17 MB/s | 293 MB 00:17 2026-02-25T11:41:49.995 INFO:teuthology.orchestra.run.trial119.stdout:(118/172): abseil-cpp-20211102.0-4.el9.x86_64.r 1.6 MB/s | 551 kB 00:00 2026-02-25T11:41:50.027 INFO:teuthology.orchestra.run.trial167.stdout:(46/172): ceph-test-20.3.0-5549.g92715cd3.el9.x 7.7 MB/s | 76 MB 00:09 2026-02-25T11:41:50.047 INFO:teuthology.orchestra.run.trial119.stdout:(119/172): gperftools-libs-2.9.1-3.el9.x86_64.r 5.8 MB/s | 308 kB 00:00 2026-02-25T11:41:50.085 INFO:teuthology.orchestra.run.trial119.stdout:(120/172): grpc-data-1.46.7-10.el9.noarch.rpm 507 kB/s | 19 kB 00:00 2026-02-25T11:41:50.105 INFO:teuthology.orchestra.run.trial119.stdout:(121/172): lua-devel-5.4.4-4.el9.x86_64.rpm 50 kB/s | 22 kB 00:00 2026-02-25T11:41:50.251 INFO:teuthology.orchestra.run.trial119.stdout:(122/172): libarrow-9.0.0-15.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2026-02-25T11:41:50.266 INFO:teuthology.orchestra.run.trial119.stdout:(123/172): libarrow-doc-9.0.0-15.el9.noarch.rpm 154 kB/s | 25 kB 00:00 2026-02-25T11:41:50.289 INFO:teuthology.orchestra.run.trial119.stdout:(124/172): liboath-2.6.12-1.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2026-02-25T11:41:50.329 INFO:teuthology.orchestra.run.trial119.stdout:(125/172): luarocks-3.9.2-5.el9.noarch.rpm 3.8 MB/s | 151 kB 00:00 2026-02-25T11:41:50.355 INFO:teuthology.orchestra.run.trial119.stdout:(126/172): libunwind-1.6.2-1.el9.x86_64.rpm 763 kB/s | 67 kB 00:00 2026-02-25T11:41:50.373 INFO:teuthology.orchestra.run.trial119.stdout:(127/172): parquet-libs-9.0.0-15.el9.x86_64.rpm 19 MB/s | 838 kB 00:00 2026-02-25T11:41:50.411 INFO:teuthology.orchestra.run.trial119.stdout:(128/172): python3-autocommand-2.2.2-8.el9.noar 775 kB/s | 29 kB 00:00 2026-02-25T11:41:50.450 INFO:teuthology.orchestra.run.trial119.stdout:(129/172): python3-backports-tarfile-1.2.0-1.el 1.5 MB/s | 60 kB 00:00 2026-02-25T11:41:50.506 INFO:teuthology.orchestra.run.trial119.stdout:(130/172): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2026-02-25T11:41:50.605 INFO:teuthology.orchestra.run.trial119.stdout:(131/172): python3-bcrypt-3.2.2-1.el9.x86_64.rp 281 kB/s | 43 kB 00:00 2026-02-25T11:41:50.606 INFO:teuthology.orchestra.run.trial119.stdout:(132/172): python3-cachetools-4.2.4-1.el9.noarc 323 kB/s | 32 kB 00:00 2026-02-25T11:41:50.843 INFO:teuthology.orchestra.run.trial119.stdout:(133/172): python3-certifi-2023.05.07-4.el9.noa 59 kB/s | 14 kB 00:00 2026-02-25T11:41:50.850 INFO:teuthology.orchestra.run.trial119.stdout:(134/172): python3-cheroot-10.0.1-4.el9.noarch. 710 kB/s | 173 kB 00:00 2026-02-25T11:41:50.901 INFO:teuthology.orchestra.run.trial119.stdout:(135/172): python3-cherrypy-18.6.1-2.el9.noarch 6.1 MB/s | 358 kB 00:00 2026-02-25T11:41:50.911 INFO:teuthology.orchestra.run.trial119.stdout:(136/172): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.9 MB/s | 303 kB 00:00 2026-02-25T11:41:50.961 INFO:teuthology.orchestra.run.trial119.stdout:(137/172): python3-google-auth-2.45.0-1.el9.noa 4.1 MB/s | 254 kB 00:00 2026-02-25T11:41:50.973 INFO:teuthology.orchestra.run.trial167.stdout:(47/172): libgfortran-11.5.0-14.el9.x86_64.rpm 561 kB/s | 794 kB 00:01 2026-02-25T11:41:51.000 INFO:teuthology.orchestra.run.trial119.stdout:(138/172): python3-grpcio-tools-1.46.7-10.el9.x 3.6 MB/s | 144 kB 00:00 2026-02-25T11:41:51.041 INFO:teuthology.orchestra.run.trial119.stdout:(139/172): python3-grpcio-1.46.7-10.el9.x86_64. 16 MB/s | 2.0 MB 00:00 2026-02-25T11:41:51.042 INFO:teuthology.orchestra.run.trial119.stdout:(140/172): python3-jaraco-8.2.1-3.el9.noarch.rp 256 kB/s | 11 kB 00:00 2026-02-25T11:41:51.062 INFO:teuthology.orchestra.run.trial167.stdout:(48/172): libquadmath-11.5.0-14.el9.x86_64.rpm 178 kB/s | 184 kB 00:01 2026-02-25T11:41:51.079 INFO:teuthology.orchestra.run.trial119.stdout:(141/172): python3-jaraco-classes-3.2.1-5.el9.n 465 kB/s | 18 kB 00:00 2026-02-25T11:41:51.080 INFO:teuthology.orchestra.run.trial119.stdout:(142/172): python3-jaraco-collections-3.0.0-8.e 607 kB/s | 23 kB 00:00 2026-02-25T11:41:51.117 INFO:teuthology.orchestra.run.trial119.stdout:(143/172): python3-jaraco-context-6.0.1-3.el9.n 517 kB/s | 20 kB 00:00 2026-02-25T11:41:51.118 INFO:teuthology.orchestra.run.trial119.stdout:(144/172): python3-jaraco-functools-3.5.0-2.el9 511 kB/s | 19 kB 00:00 2026-02-25T11:41:51.152 INFO:teuthology.orchestra.run.trial167.stdout:(49/172): lksctp-tools-1.0.19-2.el9.x86_64.rpm 527 kB/s | 94 kB 00:00 2026-02-25T11:41:51.156 INFO:teuthology.orchestra.run.trial119.stdout:(145/172): python3-jaraco-text-4.0.0-2.el9.noar 691 kB/s | 26 kB 00:00 2026-02-25T11:41:51.170 INFO:teuthology.orchestra.run.trial119.stdout:(146/172): python3-kubernetes-26.1.0-3.el9.noar 20 MB/s | 1.0 MB 00:00 2026-02-25T11:41:51.195 INFO:teuthology.orchestra.run.trial119.stdout:(147/172): python3-more-itertools-8.12.0-2.el9. 2.0 MB/s | 79 kB 00:00 2026-02-25T11:41:51.202 INFO:teuthology.orchestra.run.trial119.stdout:(148/172): protobuf-compiler-3.14.0-17.el9.x86_ 560 kB/s | 862 kB 00:01 2026-02-25T11:41:51.209 INFO:teuthology.orchestra.run.trial119.stdout:(149/172): python3-natsort-7.1.1-5.el9.noarch.r 1.5 MB/s | 58 kB 00:00 2026-02-25T11:41:51.237 INFO:teuthology.orchestra.run.trial167.stdout:(50/172): mailcap-2.1.49-5.el9.noarch.rpm 189 kB/s | 33 kB 00:00 2026-02-25T11:41:51.245 INFO:teuthology.orchestra.run.trial119.stdout:(150/172): python3-portend-3.1.0-2.el9.noarch.r 325 kB/s | 16 kB 00:00 2026-02-25T11:41:51.247 INFO:teuthology.orchestra.run.trial119.stdout:(151/172): python3-pyperclip-1.8.2-2.el9.noarch 742 kB/s | 28 kB 00:00 2026-02-25T11:41:51.283 INFO:teuthology.orchestra.run.trial119.stdout:(152/172): python3-repoze-lru-0.7-16.el9.noarch 811 kB/s | 31 kB 00:00 2026-02-25T11:41:51.287 INFO:teuthology.orchestra.run.trial119.stdout:(153/172): python3-routes-2.5.1-5.el9.noarch.rp 4.6 MB/s | 188 kB 00:00 2026-02-25T11:41:51.322 INFO:teuthology.orchestra.run.trial119.stdout:(154/172): python3-rsa-4.9-2.el9.noarch.rpm 1.5 MB/s | 59 kB 00:00 2026-02-25T11:41:51.326 INFO:teuthology.orchestra.run.trial119.stdout:(155/172): python3-tempora-5.0.0-2.el9.noarch.r 936 kB/s | 36 kB 00:00 2026-02-25T11:41:51.330 INFO:teuthology.orchestra.run.trial167.stdout:(51/172): pciutils-3.7.0-7.el9.x86_64.rpm 523 kB/s | 93 kB 00:00 2026-02-25T11:41:51.360 INFO:teuthology.orchestra.run.trial119.stdout:(156/172): python3-typing-extensions-4.15.0-1.e 2.2 MB/s | 86 kB 00:00 2026-02-25T11:41:51.364 INFO:teuthology.orchestra.run.trial119.stdout:(157/172): python3-websocket-client-1.2.3-2.el9 2.3 MB/s | 90 kB 00:00 2026-02-25T11:41:51.399 INFO:teuthology.orchestra.run.trial119.stdout:(158/172): python3-xmltodict-0.12.0-15.el9.noar 582 kB/s | 22 kB 00:00 2026-02-25T11:41:51.403 INFO:teuthology.orchestra.run.trial119.stdout:(159/172): python3-zc-lockfile-2.0-10.el9.noarc 526 kB/s | 20 kB 00:00 2026-02-25T11:41:51.406 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-25T11:41:51.421 INFO:teuthology.orchestra.run.trial119.stdout:(160/172): python3-pyOpenSSL-21.0.0-1.el9.noarc 411 kB/s | 90 kB 00:00 2026-02-25T11:41:51.438 INFO:teuthology.orchestra.run.trial119.stdout:(161/172): re2-20211101-20.el9.x86_64.rpm 4.8 MB/s | 191 kB 00:00 2026-02-25T11:41:51.462 INFO:teuthology.orchestra.run.trial119.stdout:(162/172): thrift-0.15.0-4.el9.x86_64.rpm 26 MB/s | 1.6 MB 00:00 2026-02-25T11:41:51.499 INFO:teuthology.orchestra.run.trial119.stdout:(163/172): yaml-cpp-0.6.3-5.el9.x86_64.rpm 1.5 MB/s | 122 kB 00:00 2026-02-25T11:41:51.507 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-25T11:41:51.507 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-25T11:41:51.601 INFO:teuthology.orchestra.run.trial119.stdout:(164/172): librbd1-20.3.0-5549.g92715cd3.el9.x8 20 MB/s | 2.8 MB 00:00 2026-02-25T11:41:51.609 INFO:teuthology.orchestra.run.trial119.stdout:(165/172): librados2-20.3.0-5549.g92715cd3.el9. 20 MB/s | 3.5 MB 00:00 2026-02-25T11:41:51.761 INFO:teuthology.orchestra.run.trial167.stdout:(52/172): hwloc-libs-2.4.1-6.el9.x86_64.rpm 517 kB/s | 2.1 MB 00:04 2026-02-25T11:41:51.940 INFO:teuthology.orchestra.run.trial167.stdout:(53/172): python3-ply-3.11-14.el9.noarch.rpm 596 kB/s | 106 kB 00:00 2026-02-25T11:41:51.957 INFO:teuthology.orchestra.run.trial167.stdout:(54/172): python3-cffi-1.14.5-5.el9.x86_64.rpm 352 kB/s | 253 kB 00:00 2026-02-25T11:41:52.119 INFO:teuthology.orchestra.run.trial167.stdout:(55/172): python3-pycparser-2.20-6.el9.noarch.r 756 kB/s | 135 kB 00:00 2026-02-25T11:41:52.168 INFO:teuthology.orchestra.run.trial119.stdout:(166/172): libblkid-2.37.4-25.el9.x86_64.rpm 157 kB/s | 105 kB 00:00 2026-02-25T11:41:52.298 INFO:teuthology.orchestra.run.trial167.stdout:(56/172): python3-requests-2.25.1-10.el9.noarch 707 kB/s | 126 kB 00:00 2026-02-25T11:41:52.308 INFO:teuthology.orchestra.run.trial167.stdout:(57/172): python3-pyparsing-2.4.7-9.el9.noarch. 428 kB/s | 150 kB 00:00 2026-02-25T11:41:52.314 INFO:teuthology.orchestra.run.trial119.stdout:(167/172): libfdisk-2.37.4-25.el9.x86_64.rpm 213 kB/s | 152 kB 00:00 2026-02-25T11:41:52.315 INFO:teuthology.orchestra.run.trial119.stdout:(168/172): libmount-2.37.4-25.el9.x86_64.rpm 188 kB/s | 133 kB 00:00 2026-02-25T11:41:52.532 INFO:teuthology.orchestra.run.trial167.stdout:(58/172): python3-cryptography-36.0.1-5.el9.x86 1.0 MB/s | 1.2 MB 00:01 2026-02-25T11:41:52.555 INFO:teuthology.orchestra.run.trial167.stdout:(59/172): python3-urllib3-1.26.5-7.el9.noarch.r 847 kB/s | 218 kB 00:00 2026-02-25T11:41:52.716 INFO:teuthology.orchestra.run.trial167.stdout:(60/172): unzip-6.0-59.el9.x86_64.rpm 446 kB/s | 182 kB 00:00 2026-02-25T11:41:52.750 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-25T11:41:52.750 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-25T11:41:52.783 INFO:teuthology.orchestra.run.trial119.stdout:(169/172): libsmartcols-2.37.4-25.el9.x86_64.rp 99 kB/s | 61 kB 00:00 2026-02-25T11:41:52.789 INFO:teuthology.orchestra.run.trial167.stdout:(61/172): zip-3.0-35.el9.x86_64.rpm 1.0 MB/s | 266 kB 00:00 2026-02-25T11:41:52.864 INFO:teuthology.orchestra.run.trial119.stdout:(170/172): libuuid-2.37.4-25.el9.x86_64.rpm 47 kB/s | 26 kB 00:00 2026-02-25T11:41:52.874 INFO:teuthology.orchestra.run.trial167.stdout:(62/172): boost-program-options-1.75.0-13.el9.x 1.2 MB/s | 104 kB 00:00 2026-02-25T11:41:52.958 INFO:teuthology.orchestra.run.trial167.stdout:(63/172): elfutils-libelf-devel-0.194-1.el9.x86 945 kB/s | 79 kB 00:00 2026-02-25T11:41:53.041 INFO:teuthology.orchestra.run.trial167.stdout:(64/172): automake-1.16.2-8.el9.noarch.rpm 1.4 MB/s | 689 kB 00:00 2026-02-25T11:41:53.125 INFO:teuthology.orchestra.run.trial167.stdout:(65/172): flexiblas-3.0.4-9.el9.x86_64.rpm 355 kB/s | 30 kB 00:00 2026-02-25T11:41:53.200 INFO:teuthology.orchestra.run.trial167.stdout:(66/172): flex-2.6.4-9.el9.x86_64.rpm 1.3 MB/s | 313 kB 00:00 2026-02-25T11:41:53.284 INFO:teuthology.orchestra.run.trial167.stdout:(67/172): flexiblas-openblas-openmp-3.0.4-9.el9 178 kB/s | 15 kB 00:00 2026-02-25T11:41:53.332 INFO:teuthology.orchestra.run.trial119.stdout:(171/172): util-linux-2.37.4-25.el9.x86_64.rpm 2.2 MB/s | 2.3 MB 00:01 2026-02-25T11:41:53.367 INFO:teuthology.orchestra.run.trial167.stdout:(68/172): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2026-02-25T11:41:53.531 INFO:teuthology.orchestra.run.trial167.stdout:(69/172): libnbd-1.20.3-4.el9.x86_64.rpm 1.0 MB/s | 164 kB 00:00 2026-02-25T11:41:53.623 INFO:teuthology.orchestra.run.trial119.stdout:(172/172): util-linux-core-2.37.4-25.el9.x86_64 546 kB/s | 458 kB 00:00 2026-02-25T11:41:53.628 INFO:teuthology.orchestra.run.trial119.stdout:-------------------------------------------------------------------------------- 2026-02-25T11:41:53.628 INFO:teuthology.orchestra.run.trial119.stdout:Total 18 MB/s | 293 MB 00:16 2026-02-25T11:41:53.694 INFO:teuthology.orchestra.run.trial167.stdout:(70/172): libpmemobj-1.12.1-1.el9.x86_64.rpm 986 kB/s | 160 kB 00:00 2026-02-25T11:41:53.777 INFO:teuthology.orchestra.run.trial167.stdout:(71/172): librabbitmq-0.11.0-7.el9.x86_64.rpm 542 kB/s | 45 kB 00:00 2026-02-25T11:41:53.886 INFO:teuthology.orchestra.run.trial167.stdout:(72/172): bison-3.7.4-5.el9.x86_64.rpm 807 kB/s | 943 kB 00:01 2026-02-25T11:41:54.060 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-25T11:41:54.064 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : libuuid-2.37.4-25.el9.x86_64 1/181 2026-02-25T11:41:54.066 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : libblkid-2.37.4-25.el9.x86_64 2/181 2026-02-25T11:41:54.090 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libblkid-2.37.4-25.el9.x86_64 2/181 2026-02-25T11:41:54.095 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 3/181 2026-02-25T11:41:54.109 INFO:teuthology.orchestra.run.trial028.stdout: Installing : liboath-2.6.12-1.el9.x86_64 4/181 2026-02-25T11:41:54.124 INFO:teuthology.orchestra.run.trial167.stdout:(73/172): libstoragemgmt-1.10.1-1.el9.x86_64.rp 1.0 MB/s | 246 kB 00:00 2026-02-25T11:41:54.126 INFO:teuthology.orchestra.run.trial028.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 5/181 2026-02-25T11:41:54.176 INFO:teuthology.orchestra.run.trial167.stdout:(74/172): librdkafka-1.6.1-102.el9.x86_64.rpm 1.6 MB/s | 662 kB 00:00 2026-02-25T11:41:54.260 INFO:teuthology.orchestra.run.trial167.stdout:(75/172): libuuid-devel-2.37.4-25.el9.x86_64.rp 271 kB/s | 23 kB 00:00 2026-02-25T11:41:54.294 INFO:teuthology.orchestra.run.trial028.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 6/181 2026-02-25T11:41:54.297 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : librados2-2:20.3.0-5549.g92715cd3.el9.x86_64 7/181 2026-02-25T11:41:54.341 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librados2-2:20.3.0-5549.g92715cd3.el9.x86_64 7/181 2026-02-25T11:41:54.354 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-rados-2:20.3.0-5549.g92715cd3.el9.x86_64 8/181 2026-02-25T11:41:54.358 INFO:teuthology.orchestra.run.trial028.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/181 2026-02-25T11:41:54.364 INFO:teuthology.orchestra.run.trial028.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 10/181 2026-02-25T11:41:54.368 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 11/181 2026-02-25T11:41:54.371 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 12/181 2026-02-25T11:41:54.379 INFO:teuthology.orchestra.run.trial028.stdout: Installing : postgresql-private-libs-13.23-1.el9.x86_64 13/181 2026-02-25T11:41:54.422 INFO:teuthology.orchestra.run.trial167.stdout:(76/172): libxslt-1.1.34-12.el9.x86_64.rpm 1.4 MB/s | 233 kB 00:00 2026-02-25T11:41:54.471 INFO:teuthology.orchestra.run.trial167.stdout:(77/172): flexiblas-netlib-3.0.4-9.el9.x86_64.r 2.2 MB/s | 3.0 MB 00:01 2026-02-25T11:41:54.506 INFO:teuthology.orchestra.run.trial167.stdout:(78/172): libzstd-devel-1.5.5-1.el9.x86_64.rpm 610 kB/s | 51 kB 00:00 2026-02-25T11:41:54.514 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 14/181 2026-02-25T11:41:54.517 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : librbd1-2:20.3.0-5549.g92715cd3.el9.x86_64 15/181 2026-02-25T11:41:54.595 INFO:teuthology.orchestra.run.trial167.stdout:(79/172): libtool-2.4.6-46.el9.x86_64.rpm 1.2 MB/s | 581 kB 00:00 2026-02-25T11:41:54.631 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librbd1-2:20.3.0-5549.g92715cd3.el9.x86_64 15/181 2026-02-25T11:41:54.633 INFO:teuthology.orchestra.run.trial167.stdout:(80/172): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2026-02-25T11:41:54.646 INFO:teuthology.orchestra.run.trial028.stdout: Installing : postgresql-13.23-1.el9.x86_64 16/181 2026-02-25T11:41:54.647 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libcephsqlite-2:20.3.0-5549.g92715cd3.el9.x86_64 17/181 2026-02-25T11:41:54.668 INFO:teuthology.orchestra.run.trial167.stdout:(81/172): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2026-02-25T11:41:54.677 INFO:teuthology.orchestra.run.trial167.stdout:(82/172): openblas-0.3.29-1.el9.x86_64.rpm 511 kB/s | 42 kB 00:00 2026-02-25T11:41:54.683 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5549.g92715cd3.el9.x86_64 17/181 2026-02-25T11:41:54.685 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libradosstriper1-2:20.3.0-5549.g92715cd3.el9.x86 18/181 2026-02-25T11:41:54.711 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5549.g92715cd3.el9.x86 18/181 2026-02-25T11:41:54.716 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : libmount-2.37.4-25.el9.x86_64 19/181 2026-02-25T11:41:54.721 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : libsmartcols-2.37.4-25.el9.x86_64 20/181 2026-02-25T11:41:54.726 INFO:teuthology.orchestra.run.trial028.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 21/181 2026-02-25T11:41:54.760 INFO:teuthology.orchestra.run.trial167.stdout:(83/172): perl-Thread-Queue-3.14-460.el9.noarch 269 kB/s | 22 kB 00:00 2026-02-25T11:41:54.767 INFO:teuthology.orchestra.run.trial028.stdout: Installing : re2-1:20211101-20.el9.x86_64 22/181 2026-02-25T11:41:54.801 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 23/181 2026-02-25T11:41:54.809 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 24/181 2026-02-25T11:41:54.819 INFO:teuthology.orchestra.run.trial028.stdout: Installing : perl-threads-1:2.25-460.el9.x86_64 25/181 2026-02-25T11:41:54.827 INFO:teuthology.orchestra.run.trial028.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/181 2026-02-25T11:41:54.833 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 27/181 2026-02-25T11:41:54.841 INFO:teuthology.orchestra.run.trial028.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 28/181 2026-02-25T11:41:54.842 INFO:teuthology.orchestra.run.trial167.stdout:(84/172): perl-threads-2.25-460.el9.x86_64.rpm 706 kB/s | 58 kB 00:00 2026-02-25T11:41:54.877 INFO:teuthology.orchestra.run.trial028.stdout: Installing : unzip-6.0-59.el9.x86_64 29/181 2026-02-25T11:41:54.897 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 30/181 2026-02-25T11:41:54.907 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 31/181 2026-02-25T11:41:54.913 INFO:teuthology.orchestra.run.trial028.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 32/181 2026-02-25T11:41:54.921 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 33/181 2026-02-25T11:41:54.924 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 34/181 2026-02-25T11:41:54.925 INFO:teuthology.orchestra.run.trial167.stdout:(85/172): perl-threads-shared-1.61-460.el9.x86_ 539 kB/s | 45 kB 00:00 2026-02-25T11:41:54.927 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 35/181 2026-02-25T11:41:54.942 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 36/181 2026-02-25T11:41:54.991 INFO:teuthology.orchestra.run.trial028.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 37/181 2026-02-25T11:41:54.998 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-ceph-common-2:20.3.0-5549.g92715cd3.el9. 38/181 2026-02-25T11:41:55.001 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-ceph-argparse-2:20.3.0-5549.g92715cd3.el 39/181 2026-02-25T11:41:55.002 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libcephfs-proxy2-2:20.3.0-5549.g92715cd3.el9.x86 40/181 2026-02-25T11:41:55.068 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5549.g92715cd3.el9.x86 40/181 2026-02-25T11:41:55.071 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libcephfs2-2:20.3.0-5549.g92715cd3.el9.x86_64 41/181 2026-02-25T11:41:55.106 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libcephfs2-2:20.3.0-5549.g92715cd3.el9.x86_64 41/181 2026-02-25T11:41:55.125 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cephfs-2:20.3.0-5549.g92715cd3.el9.x86_6 42/181 2026-02-25T11:41:55.126 INFO:teuthology.orchestra.run.trial119.stdout:Running transaction check 2026-02-25T11:41:55.135 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 43/181 2026-02-25T11:41:55.174 INFO:teuthology.orchestra.run.trial028.stdout: Installing : zip-3.0-35.el9.x86_64 44/181 2026-02-25T11:41:55.183 INFO:teuthology.orchestra.run.trial028.stdout: Installing : luarocks-3.9.2-5.el9.noarch 45/181 2026-02-25T11:41:55.188 INFO:teuthology.orchestra.run.trial028.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 46/181 2026-02-25T11:41:55.193 INFO:teuthology.orchestra.run.trial028.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 47/181 2026-02-25T11:41:55.198 INFO:teuthology.orchestra.run.trial028.stdout: Installing : perl-threads-shared-1.61-460.el9.x86_64 48/181 2026-02-25T11:41:55.227 INFO:teuthology.orchestra.run.trial119.stdout:Transaction check succeeded. 2026-02-25T11:41:55.227 INFO:teuthology.orchestra.run.trial119.stdout:Running transaction test 2026-02-25T11:41:55.239 INFO:teuthology.orchestra.run.trial028.stdout: Installing : perl-Thread-Queue-3.14-460.el9.noarch 49/181 2026-02-25T11:41:55.315 INFO:teuthology.orchestra.run.trial028.stdout: Installing : automake-1.16.2-8.el9.noarch 50/181 2026-02-25T11:41:55.335 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 51/181 2026-02-25T11:41:55.379 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-rsa-4.9-2.el9.noarch 52/181 2026-02-25T11:41:55.385 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : util-linux-core-2.37.4-25.el9.x86_64 53/181 2026-02-25T11:41:55.418 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: util-linux-core-2.37.4-25.el9.x86_64 53/181 2026-02-25T11:41:55.427 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-rbd-2:20.3.0-5549.g92715cd3.el9.x86_64 54/181 2026-02-25T11:41:55.438 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 55/181 2026-02-25T11:41:55.450 INFO:teuthology.orchestra.run.trial028.stdout: Installing : librados-devel-2:20.3.0-5549.g92715cd3.el9.x86_6 56/181 2026-02-25T11:41:55.482 INFO:teuthology.orchestra.run.trial028.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 57/181 2026-02-25T11:41:55.488 INFO:teuthology.orchestra.run.trial028.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 58/181 2026-02-25T11:41:55.633 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : libfdisk-2.37.4-25.el9.x86_64 59/181 2026-02-25T11:41:55.651 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : util-linux-2.37.4-25.el9.x86_64 60/181 2026-02-25T11:41:55.666 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libuuid-devel-2.37.4-25.el9.x86_64 61/181 2026-02-25T11:41:55.675 INFO:teuthology.orchestra.run.trial028.stdout: Installing : thrift-0.15.0-4.el9.x86_64 62/181 2026-02-25T11:41:55.682 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 63/181 2026-02-25T11:41:55.704 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 64/181 2026-02-25T11:41:55.712 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 65/181 2026-02-25T11:41:55.720 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 66/181 2026-02-25T11:41:55.739 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 67/181 2026-02-25T11:41:55.748 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 68/181 2026-02-25T11:41:55.762 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 69/181 2026-02-25T11:41:55.770 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 70/181 2026-02-25T11:41:55.783 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 71/181 2026-02-25T11:41:55.786 INFO:teuthology.orchestra.run.trial167.stdout:(86/172): postgresql-13.23-1.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2026-02-25T11:41:55.825 INFO:teuthology.orchestra.run.trial167.stdout:(87/172): openblas-openmp-0.3.29-1.el9.x86_64.r 4.4 MB/s | 5.3 MB 00:01 2026-02-25T11:41:55.844 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 72/181 2026-02-25T11:41:55.909 INFO:teuthology.orchestra.run.trial167.stdout:(88/172): postgresql-private-libs-13.23-1.el9.x 1.6 MB/s | 138 kB 00:00 2026-02-25T11:41:55.940 INFO:teuthology.orchestra.run.trial167.stdout:(89/172): openssl-devel-3.5.5-1.el9.x86_64.rpm 3.8 MB/s | 4.8 MB 00:01 2026-02-25T11:41:56.105 INFO:teuthology.orchestra.run.trial167.stdout:(90/172): postgresql-contrib-13.23-1.el9.x86_64 2.8 MB/s | 903 kB 00:00 2026-02-25T11:41:56.110 INFO:teuthology.orchestra.run.trial167.stdout:(91/172): protobuf-3.14.0-17.el9.x86_64.rpm 5.9 MB/s | 1.0 MB 00:00 2026-02-25T11:41:56.194 INFO:teuthology.orchestra.run.trial167.stdout:(92/172): python3-colorama-0.4.6-3.el9.noarch.r 738 kB/s | 61 kB 00:00 2026-02-25T11:41:56.310 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 73/181 2026-02-25T11:41:56.329 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 74/181 2026-02-25T11:41:56.337 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 75/181 2026-02-25T11:41:56.346 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 76/181 2026-02-25T11:41:56.352 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 77/181 2026-02-25T11:41:56.354 INFO:teuthology.orchestra.run.trial167.stdout:(93/172): python3-devel-3.9.25-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2026-02-25T11:41:56.362 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 78/181 2026-02-25T11:41:56.367 INFO:teuthology.orchestra.run.trial028.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 79/181 2026-02-25T11:41:56.369 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 80/181 2026-02-25T11:41:56.414 INFO:teuthology.orchestra.run.trial028.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 81/181 2026-02-25T11:41:56.468 INFO:teuthology.orchestra.run.trial119.stdout:Transaction test succeeded. 2026-02-25T11:41:56.468 INFO:teuthology.orchestra.run.trial119.stdout:Running transaction 2026-02-25T11:41:56.481 INFO:teuthology.orchestra.run.trial028.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 82/181 2026-02-25T11:41:56.490 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 83/181 2026-02-25T11:41:56.496 INFO:teuthology.orchestra.run.trial028.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 84/181 2026-02-25T11:41:56.509 INFO:teuthology.orchestra.run.trial028.stdout: Installing : uuid-1.6.2-55.el9.x86_64 85/181 2026-02-25T11:41:56.515 INFO:teuthology.orchestra.run.trial167.stdout:(94/172): python3-jinja2-2.11.3-8.el9.noarch.rp 1.5 MB/s | 249 kB 00:00 2026-02-25T11:41:56.520 INFO:teuthology.orchestra.run.trial028.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 86/181 2026-02-25T11:41:56.562 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 87/181 2026-02-25T11:41:56.573 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/181 2026-02-25T11:41:56.580 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 89/181 2026-02-25T11:41:56.589 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 90/181 2026-02-25T11:41:56.596 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 91/181 2026-02-25T11:41:56.598 INFO:teuthology.orchestra.run.trial167.stdout:(95/172): python3-jmespath-1.0.1-1.el9.noarch.r 573 kB/s | 48 kB 00:00 2026-02-25T11:41:56.608 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 92/181 2026-02-25T11:41:56.615 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 93/181 2026-02-25T11:41:56.656 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 94/181 2026-02-25T11:41:56.673 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 95/181 2026-02-25T11:41:56.683 INFO:teuthology.orchestra.run.trial167.stdout:(96/172): python3-libstoragemgmt-1.10.1-1.el9.x 2.0 MB/s | 177 kB 00:00 2026-02-25T11:41:56.684 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 96/181 2026-02-25T11:41:56.695 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 97/181 2026-02-25T11:41:56.712 INFO:teuthology.orchestra.run.trial167.stdout:(97/172): postgresql-server-13.23-1.el9.x86_64. 7.2 MB/s | 5.8 MB 00:00 2026-02-25T11:41:56.746 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 98/181 2026-02-25T11:41:56.766 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 99/181 2026-02-25T11:41:56.766 INFO:teuthology.orchestra.run.trial167.stdout:(98/172): python3-markupsafe-1.1.1-12.el9.x86_6 418 kB/s | 35 kB 00:00 2026-02-25T11:41:56.910 INFO:teuthology.orchestra.run.trial167.stdout:(99/172): python3-babel-2.9.1-2.el9.noarch.rpm 7.4 MB/s | 6.0 MB 00:00 2026-02-25T11:41:56.928 INFO:teuthology.orchestra.run.trial167.stdout:(100/172): python3-numpy-f2py-1.23.5-2.el9.x86_ 2.7 MB/s | 442 kB 00:00 2026-02-25T11:41:56.994 INFO:teuthology.orchestra.run.trial167.stdout:(101/172): python3-packaging-20.9-5.el9.noarch. 915 kB/s | 77 kB 00:00 2026-02-25T11:41:57.013 INFO:teuthology.orchestra.run.trial167.stdout:(102/172): python3-protobuf-3.14.0-17.el9.noarc 3.1 MB/s | 267 kB 00:00 2026-02-25T11:41:57.069 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 100/181 2026-02-25T11:41:57.080 INFO:teuthology.orchestra.run.trial167.stdout:(103/172): python3-pyasn1-0.4.8-7.el9.noarch.rp 1.8 MB/s | 157 kB 00:00 2026-02-25T11:41:57.109 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 101/181 2026-02-25T11:41:57.116 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 102/181 2026-02-25T11:41:57.163 INFO:teuthology.orchestra.run.trial167.stdout:(104/172): python3-requests-oauthlib-1.3.0-12.e 640 kB/s | 54 kB 00:00 2026-02-25T11:41:57.174 INFO:teuthology.orchestra.run.trial167.stdout:(105/172): python3-pyasn1-modules-0.4.8-7.el9.n 1.7 MB/s | 277 kB 00:00 2026-02-25T11:41:57.189 INFO:teuthology.orchestra.run.trial028.stdout: Installing : openblas-0.3.29-1.el9.x86_64 103/181 2026-02-25T11:41:57.194 INFO:teuthology.orchestra.run.trial028.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 104/181 2026-02-25T11:41:57.222 INFO:teuthology.orchestra.run.trial028.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 105/181 2026-02-25T11:41:57.283 INFO:teuthology.orchestra.run.trial167.stdout:(106/172): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2026-02-25T11:41:57.360 INFO:teuthology.orchestra.run.trial167.stdout:(107/172): python3-numpy-1.23.5-2.el9.x86_64.rp 9.5 MB/s | 6.1 MB 00:00 2026-02-25T11:41:57.373 INFO:teuthology.orchestra.run.trial167.stdout:(108/172): python3-wcwidth-0.2.5-8.el9.noarch.r 491 kB/s | 44 kB 00:00 2026-02-25T11:41:57.449 INFO:teuthology.orchestra.run.trial167.stdout:(109/172): qatlib-25.08.0-2.el9.x86_64.rpm 2.7 MB/s | 240 kB 00:00 2026-02-25T11:41:57.461 INFO:teuthology.orchestra.run.trial167.stdout:(110/172): qatlib-service-25.08.0-2.el9.x86_64. 423 kB/s | 37 kB 00:00 2026-02-25T11:41:57.532 INFO:teuthology.orchestra.run.trial167.stdout:(111/172): qatzip-libs-1.3.1-1.el9.x86_64.rpm 794 kB/s | 66 kB 00:00 2026-02-25T11:41:57.616 INFO:teuthology.orchestra.run.trial167.stdout:(112/172): uuid-1.6.2-55.el9.x86_64.rpm 693 kB/s | 58 kB 00:00 2026-02-25T11:41:57.624 INFO:teuthology.orchestra.run.trial167.stdout:(113/172): socat-1.7.4.1-8.el9.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2026-02-25T11:41:57.690 INFO:teuthology.orchestra.run.trial028.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 106/181 2026-02-25T11:41:57.699 INFO:teuthology.orchestra.run.trial167.stdout:(114/172): xfsprogs-devel-6.4.0-7.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2026-02-25T11:41:57.714 INFO:teuthology.orchestra.run.trial167.stdout:(115/172): xmlstarlet-1.6.1-20.el9.x86_64.rpm 712 kB/s | 64 kB 00:00 2026-02-25T11:41:57.778 INFO:teuthology.orchestra.run.trial167.stdout:(116/172): lua-devel-5.4.4-4.el9.x86_64.rpm 350 kB/s | 22 kB 00:00 2026-02-25T11:41:57.779 INFO:teuthology.orchestra.run.trial119.stdout: Preparing : 1/1 2026-02-25T11:41:57.783 INFO:teuthology.orchestra.run.trial119.stdout: Upgrading : libuuid-2.37.4-25.el9.x86_64 1/181 2026-02-25T11:41:57.783 INFO:teuthology.orchestra.run.trial167.stdout:(117/172): zlib-devel-1.2.11-41.el9.x86_64.rpm 533 kB/s | 45 kB 00:00 2026-02-25T11:41:57.785 INFO:teuthology.orchestra.run.trial119.stdout: Upgrading : libblkid-2.37.4-25.el9.x86_64 2/181 2026-02-25T11:41:57.806 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 107/181 2026-02-25T11:41:57.809 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: libblkid-2.37.4-25.el9.x86_64 2/181 2026-02-25T11:41:57.814 INFO:teuthology.orchestra.run.trial167.stdout:(118/172): protobuf-compiler-3.14.0-17.el9.x86_ 23 MB/s | 862 kB 00:00 2026-02-25T11:41:57.815 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 3/181 2026-02-25T11:41:57.828 INFO:teuthology.orchestra.run.trial119.stdout: Installing : liboath-2.6.12-1.el9.x86_64 4/181 2026-02-25T11:41:57.846 INFO:teuthology.orchestra.run.trial119.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 5/181 2026-02-25T11:41:58.023 INFO:teuthology.orchestra.run.trial119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 6/181 2026-02-25T11:41:58.026 INFO:teuthology.orchestra.run.trial119.stdout: Upgrading : librados2-2:20.3.0-5549.g92715cd3.el9.x86_64 7/181 2026-02-25T11:41:58.046 INFO:teuthology.orchestra.run.trial167.stdout:(119/172): gperftools-libs-2.9.1-3.el9.x86_64.r 1.3 MB/s | 308 kB 00:00 2026-02-25T11:41:58.070 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: librados2-2:20.3.0-5549.g92715cd3.el9.x86_64 7/181 2026-02-25T11:41:58.082 INFO:teuthology.orchestra.run.trial167.stdout:(120/172): grpc-data-1.46.7-10.el9.noarch.rpm 549 kB/s | 19 kB 00:00 2026-02-25T11:41:58.082 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-rados-2:20.3.0-5549.g92715cd3.el9.x86_64 8/181 2026-02-25T11:41:58.091 INFO:teuthology.orchestra.run.trial119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/181 2026-02-25T11:41:58.093 INFO:teuthology.orchestra.run.trial167.stdout:(121/172): abseil-cpp-20211102.0-4.el9.x86_64.r 1.7 MB/s | 551 kB 00:00 2026-02-25T11:41:58.097 INFO:teuthology.orchestra.run.trial119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 10/181 2026-02-25T11:41:58.101 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 11/181 2026-02-25T11:41:58.104 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 12/181 2026-02-25T11:41:58.112 INFO:teuthology.orchestra.run.trial119.stdout: Installing : postgresql-private-libs-13.23-1.el9.x86_64 13/181 2026-02-25T11:41:58.133 INFO:teuthology.orchestra.run.trial167.stdout:(122/172): libarrow-doc-9.0.0-15.el9.noarch.rpm 633 kB/s | 25 kB 00:00 2026-02-25T11:41:58.177 INFO:teuthology.orchestra.run.trial167.stdout:(123/172): liboath-2.6.12-1.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2026-02-25T11:41:58.216 INFO:teuthology.orchestra.run.trial167.stdout:(124/172): libunwind-1.6.2-1.el9.x86_64.rpm 1.7 MB/s | 67 kB 00:00 2026-02-25T11:41:58.246 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 14/181 2026-02-25T11:41:58.249 INFO:teuthology.orchestra.run.trial119.stdout: Upgrading : librbd1-2:20.3.0-5549.g92715cd3.el9.x86_64 15/181 2026-02-25T11:41:58.258 INFO:teuthology.orchestra.run.trial167.stdout:(125/172): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2026-02-25T11:41:58.268 INFO:teuthology.orchestra.run.trial167.stdout:(126/172): libarrow-9.0.0-15.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2026-02-25T11:41:58.315 INFO:teuthology.orchestra.run.trial167.stdout:(127/172): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2026-02-25T11:41:58.337 INFO:teuthology.orchestra.run.trial167.stdout:(128/172): parquet-libs-9.0.0-15.el9.x86_64.rpm 10 MB/s | 838 kB 00:00 2026-02-25T11:41:58.351 INFO:teuthology.orchestra.run.trial167.stdout:(129/172): python3-autocommand-2.2.2-8.el9.noar 818 kB/s | 29 kB 00:00 2026-02-25T11:41:58.363 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: librbd1-2:20.3.0-5549.g92715cd3.el9.x86_64 15/181 2026-02-25T11:41:58.374 INFO:teuthology.orchestra.run.trial167.stdout:(130/172): python3-backports-tarfile-1.2.0-1.el 1.6 MB/s | 60 kB 00:00 2026-02-25T11:41:58.379 INFO:teuthology.orchestra.run.trial119.stdout: Installing : postgresql-13.23-1.el9.x86_64 16/181 2026-02-25T11:41:58.380 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libcephsqlite-2:20.3.0-5549.g92715cd3.el9.x86_64 17/181 2026-02-25T11:41:58.388 INFO:teuthology.orchestra.run.trial167.stdout:(131/172): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.2 MB/s | 43 kB 00:00 2026-02-25T11:41:58.410 INFO:teuthology.orchestra.run.trial167.stdout:(132/172): python3-cachetools-4.2.4-1.el9.noarc 883 kB/s | 32 kB 00:00 2026-02-25T11:41:58.415 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5549.g92715cd3.el9.x86_64 17/181 2026-02-25T11:41:58.417 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libradosstriper1-2:20.3.0-5549.g92715cd3.el9.x86 18/181 2026-02-25T11:41:58.424 INFO:teuthology.orchestra.run.trial167.stdout:(133/172): python3-certifi-2023.05.07-4.el9.noa 393 kB/s | 14 kB 00:00 2026-02-25T11:41:58.443 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5549.g92715cd3.el9.x86 18/181 2026-02-25T11:41:58.448 INFO:teuthology.orchestra.run.trial119.stdout: Upgrading : libmount-2.37.4-25.el9.x86_64 19/181 2026-02-25T11:41:58.450 INFO:teuthology.orchestra.run.trial167.stdout:(134/172): python3-cheroot-10.0.1-4.el9.noarch. 4.3 MB/s | 173 kB 00:00 2026-02-25T11:41:58.453 INFO:teuthology.orchestra.run.trial119.stdout: Upgrading : libsmartcols-2.37.4-25.el9.x86_64 20/181 2026-02-25T11:41:58.459 INFO:teuthology.orchestra.run.trial119.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 21/181 2026-02-25T11:41:58.463 INFO:teuthology.orchestra.run.trial167.stdout:(135/172): python3-cherrypy-18.6.1-2.el9.noarch 8.9 MB/s | 358 kB 00:00 2026-02-25T11:41:58.491 INFO:teuthology.orchestra.run.trial167.stdout:(136/172): python3-cmd2-2.4.2-2.el9.noarch.rpm 7.2 MB/s | 303 kB 00:00 2026-02-25T11:41:58.500 INFO:teuthology.orchestra.run.trial119.stdout: Installing : re2-1:20211101-20.el9.x86_64 22/181 2026-02-25T11:41:58.502 INFO:teuthology.orchestra.run.trial167.stdout:(137/172): python3-google-auth-2.45.0-1.el9.noa 6.4 MB/s | 254 kB 00:00 2026-02-25T11:41:58.534 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 23/181 2026-02-25T11:41:58.541 INFO:teuthology.orchestra.run.trial167.stdout:(138/172): python3-grpcio-tools-1.46.7-10.el9.x 3.6 MB/s | 144 kB 00:00 2026-02-25T11:41:58.543 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 24/181 2026-02-25T11:41:58.552 INFO:teuthology.orchestra.run.trial119.stdout: Installing : perl-threads-1:2.25-460.el9.x86_64 25/181 2026-02-25T11:41:58.561 INFO:teuthology.orchestra.run.trial119.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/181 2026-02-25T11:41:58.566 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 27/181 2026-02-25T11:41:58.572 INFO:teuthology.orchestra.run.trial167.stdout:(139/172): python3-grpcio-1.46.7-10.el9.x86_64. 25 MB/s | 2.0 MB 00:00 2026-02-25T11:41:58.574 INFO:teuthology.orchestra.run.trial119.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 28/181 2026-02-25T11:41:58.577 INFO:teuthology.orchestra.run.trial167.stdout:(140/172): python3-jaraco-8.2.1-3.el9.noarch.rp 299 kB/s | 11 kB 00:00 2026-02-25T11:41:58.609 INFO:teuthology.orchestra.run.trial167.stdout:(141/172): python3-jaraco-classes-3.2.1-5.el9.n 483 kB/s | 18 kB 00:00 2026-02-25T11:41:58.611 INFO:teuthology.orchestra.run.trial119.stdout: Installing : unzip-6.0-59.el9.x86_64 29/181 2026-02-25T11:41:58.613 INFO:teuthology.orchestra.run.trial167.stdout:(142/172): python3-jaraco-collections-3.0.0-8.e 643 kB/s | 23 kB 00:00 2026-02-25T11:41:58.632 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 30/181 2026-02-25T11:41:58.642 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 31/181 2026-02-25T11:41:58.645 INFO:teuthology.orchestra.run.trial167.stdout:(143/172): python3-jaraco-context-6.0.1-3.el9.n 538 kB/s | 20 kB 00:00 2026-02-25T11:41:58.648 INFO:teuthology.orchestra.run.trial119.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 32/181 2026-02-25T11:41:58.649 INFO:teuthology.orchestra.run.trial167.stdout:(144/172): python3-jaraco-functools-3.5.0-2.el9 543 kB/s | 19 kB 00:00 2026-02-25T11:41:58.656 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 33/181 2026-02-25T11:41:58.659 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 34/181 2026-02-25T11:41:58.662 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 35/181 2026-02-25T11:41:58.677 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 36/181 2026-02-25T11:41:58.682 INFO:teuthology.orchestra.run.trial167.stdout:(145/172): python3-jaraco-text-4.0.0-2.el9.noar 718 kB/s | 26 kB 00:00 2026-02-25T11:41:58.692 INFO:teuthology.orchestra.run.trial167.stdout:(146/172): python3-kubernetes-26.1.0-3.el9.noar 24 MB/s | 1.0 MB 00:00 2026-02-25T11:41:58.720 INFO:teuthology.orchestra.run.trial167.stdout:(147/172): python3-more-itertools-8.12.0-2.el9. 2.1 MB/s | 79 kB 00:00 2026-02-25T11:41:58.728 INFO:teuthology.orchestra.run.trial119.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 37/181 2026-02-25T11:41:58.729 INFO:teuthology.orchestra.run.trial167.stdout:(148/172): python3-natsort-7.1.1-5.el9.noarch.r 1.5 MB/s | 58 kB 00:00 2026-02-25T11:41:58.735 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-ceph-common-2:20.3.0-5549.g92715cd3.el9. 38/181 2026-02-25T11:41:58.738 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-ceph-argparse-2:20.3.0-5549.g92715cd3.el 39/181 2026-02-25T11:41:58.739 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libcephfs-proxy2-2:20.3.0-5549.g92715cd3.el9.x86 40/181 2026-02-25T11:41:58.757 INFO:teuthology.orchestra.run.trial167.stdout:(149/172): python3-portend-3.1.0-2.el9.noarch.r 449 kB/s | 16 kB 00:00 2026-02-25T11:41:58.766 INFO:teuthology.orchestra.run.trial167.stdout:(150/172): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.4 MB/s | 90 kB 00:00 2026-02-25T11:41:58.776 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 108/181 2026-02-25T11:41:58.794 INFO:teuthology.orchestra.run.trial167.stdout:(151/172): python3-pyperclip-1.8.2-2.el9.noarch 769 kB/s | 28 kB 00:00 2026-02-25T11:41:58.802 INFO:teuthology.orchestra.run.trial167.stdout:(152/172): python3-repoze-lru-0.7-16.el9.noarch 853 kB/s | 31 kB 00:00 2026-02-25T11:41:58.804 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5549.g92715cd3.el9.x86 40/181 2026-02-25T11:41:58.807 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libcephfs2-2:20.3.0-5549.g92715cd3.el9.x86_64 41/181 2026-02-25T11:41:58.808 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 109/181 2026-02-25T11:41:58.812 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libzstd-devel-1.5.5-1.el9.x86_64 110/181 2026-02-25T11:41:58.820 INFO:teuthology.orchestra.run.trial028.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 111/181 2026-02-25T11:41:58.842 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: libcephfs2-2:20.3.0-5549.g92715cd3.el9.x86_64 41/181 2026-02-25T11:41:58.857 INFO:teuthology.orchestra.run.trial167.stdout:(153/172): python3-scipy-1.9.3-2.el9.x86_64.rpm 11 MB/s | 19 MB 00:01 2026-02-25T11:41:58.858 INFO:teuthology.orchestra.run.trial167.stdout:(154/172): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2026-02-25T11:41:58.859 INFO:teuthology.orchestra.run.trial167.stdout:(155/172): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2026-02-25T11:41:58.860 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-cephfs-2:20.3.0-5549.g92715cd3.el9.x86_6 42/181 2026-02-25T11:41:58.871 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 43/181 2026-02-25T11:41:58.895 INFO:teuthology.orchestra.run.trial167.stdout:(156/172): python3-typing-extensions-4.15.0-1.e 2.3 MB/s | 86 kB 00:00 2026-02-25T11:41:58.897 INFO:teuthology.orchestra.run.trial167.stdout:(157/172): python3-websocket-client-1.2.3-2.el9 2.3 MB/s | 90 kB 00:00 2026-02-25T11:41:58.910 INFO:teuthology.orchestra.run.trial119.stdout: Installing : zip-3.0-35.el9.x86_64 44/181 2026-02-25T11:41:58.919 INFO:teuthology.orchestra.run.trial119.stdout: Installing : luarocks-3.9.2-5.el9.noarch 45/181 2026-02-25T11:41:58.923 INFO:teuthology.orchestra.run.trial119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 46/181 2026-02-25T11:41:58.928 INFO:teuthology.orchestra.run.trial119.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 47/181 2026-02-25T11:41:58.931 INFO:teuthology.orchestra.run.trial167.stdout:(158/172): python3-xmltodict-0.12.0-15.el9.noar 613 kB/s | 22 kB 00:00 2026-02-25T11:41:58.933 INFO:teuthology.orchestra.run.trial119.stdout: Installing : perl-threads-shared-1.61-460.el9.x86_64 48/181 2026-02-25T11:41:58.934 INFO:teuthology.orchestra.run.trial167.stdout:(159/172): python3-zc-lockfile-2.0-10.el9.noarc 549 kB/s | 20 kB 00:00 2026-02-25T11:41:58.969 INFO:teuthology.orchestra.run.trial167.stdout:(160/172): re2-20211101-20.el9.x86_64.rpm 5.0 MB/s | 191 kB 00:00 2026-02-25T11:41:58.974 INFO:teuthology.orchestra.run.trial119.stdout: Installing : perl-Thread-Queue-3.14-460.el9.noarch 49/181 2026-02-25T11:41:58.977 INFO:teuthology.orchestra.run.trial167.stdout:(161/172): python3-tempora-5.0.0-2.el9.noarch.r 296 kB/s | 36 kB 00:00 2026-02-25T11:41:58.983 INFO:teuthology.orchestra.run.trial167.stdout:(162/172): thrift-0.15.0-4.el9.x86_64.rpm 32 MB/s | 1.6 MB 00:00 2026-02-25T11:41:59.006 INFO:teuthology.orchestra.run.trial167.stdout:(163/172): yaml-cpp-0.6.3-5.el9.x86_64.rpm 3.2 MB/s | 122 kB 00:00 2026-02-25T11:41:59.050 INFO:teuthology.orchestra.run.trial119.stdout: Installing : automake-1.16.2-8.el9.noarch 50/181 2026-02-25T11:41:59.070 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 51/181 2026-02-25T11:41:59.114 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 52/181 2026-02-25T11:41:59.121 INFO:teuthology.orchestra.run.trial119.stdout: Upgrading : util-linux-core-2.37.4-25.el9.x86_64 53/181 2026-02-25T11:41:59.124 INFO:teuthology.orchestra.run.trial028.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 112/181 2026-02-25T11:41:59.126 INFO:teuthology.orchestra.run.trial167.stdout:(164/172): libblkid-2.37.4-25.el9.x86_64.rpm 880 kB/s | 105 kB 00:00 2026-02-25T11:41:59.127 INFO:teuthology.orchestra.run.trial028.stdout: Installing : librgw2-2:20.3.0-5549.g92715cd3.el9.x86_64 113/181 2026-02-25T11:41:59.154 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: util-linux-core-2.37.4-25.el9.x86_64 53/181 2026-02-25T11:41:59.163 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-rbd-2:20.3.0-5549.g92715cd3.el9.x86_64 54/181 2026-02-25T11:41:59.164 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librgw2-2:20.3.0-5549.g92715cd3.el9.x86_64 113/181 2026-02-25T11:41:59.167 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-rgw-2:20.3.0-5549.g92715cd3.el9.x86_64 114/181 2026-02-25T11:41:59.173 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 55/181 2026-02-25T11:41:59.186 INFO:teuthology.orchestra.run.trial119.stdout: Installing : librados-devel-2:20.3.0-5549.g92715cd3.el9.x86_6 56/181 2026-02-25T11:41:59.205 INFO:teuthology.orchestra.run.trial167.stdout:(165/172): librados2-20.3.0-5549.g92715cd3.el9. 15 MB/s | 3.5 MB 00:00 2026-02-25T11:41:59.219 INFO:teuthology.orchestra.run.trial119.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 57/181 2026-02-25T11:41:59.223 INFO:teuthology.orchestra.run.trial167.stdout:(166/172): librbd1-20.3.0-5549.g92715cd3.el9.x8 12 MB/s | 2.8 MB 00:00 2026-02-25T11:41:59.225 INFO:teuthology.orchestra.run.trial119.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 58/181 2026-02-25T11:41:59.227 INFO:teuthology.orchestra.run.trial167.stdout:(167/172): libfdisk-2.37.4-25.el9.x86_64.rpm 1.5 MB/s | 152 kB 00:00 2026-02-25T11:41:59.327 INFO:teuthology.orchestra.run.trial167.stdout:(168/172): libuuid-2.37.4-25.el9.x86_64.rpm 258 kB/s | 26 kB 00:00 2026-02-25T11:41:59.370 INFO:teuthology.orchestra.run.trial119.stdout: Upgrading : libfdisk-2.37.4-25.el9.x86_64 59/181 2026-02-25T11:41:59.388 INFO:teuthology.orchestra.run.trial119.stdout: Upgrading : util-linux-2.37.4-25.el9.x86_64 60/181 2026-02-25T11:41:59.403 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libuuid-devel-2.37.4-25.el9.x86_64 61/181 2026-02-25T11:41:59.412 INFO:teuthology.orchestra.run.trial119.stdout: Installing : thrift-0.15.0-4.el9.x86_64 62/181 2026-02-25T11:41:59.418 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 63/181 2026-02-25T11:41:59.440 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 64/181 2026-02-25T11:41:59.448 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 65/181 2026-02-25T11:41:59.456 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 66/181 2026-02-25T11:41:59.462 INFO:teuthology.orchestra.run.trial167.stdout:(169/172): libmount-2.37.4-25.el9.x86_64.rpm 516 kB/s | 133 kB 00:00 2026-02-25T11:41:59.474 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 67/181 2026-02-25T11:41:59.484 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 68/181 2026-02-25T11:41:59.498 INFO:teuthology.orchestra.run.trial167.stdout:(170/172): libsmartcols-2.37.4-25.el9.x86_64.rp 221 kB/s | 61 kB 00:00 2026-02-25T11:41:59.498 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 69/181 2026-02-25T11:41:59.506 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 70/181 2026-02-25T11:41:59.518 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 71/181 2026-02-25T11:41:59.579 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 72/181 2026-02-25T11:41:59.592 INFO:teuthology.orchestra.run.trial167.stdout:(171/172): util-linux-2.37.4-25.el9.x86_64.rpm 8.5 MB/s | 2.3 MB 00:00 2026-02-25T11:41:59.645 INFO:teuthology.orchestra.run.trial167.stdout:(172/172): util-linux-core-2.37.4-25.el9.x86_64 2.5 MB/s | 458 kB 00:00 2026-02-25T11:41:59.650 INFO:teuthology.orchestra.run.trial167.stdout:-------------------------------------------------------------------------------- 2026-02-25T11:41:59.650 INFO:teuthology.orchestra.run.trial167.stdout:Total 11 MB/s | 293 MB 00:25 2026-02-25T11:42:00.042 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 73/181 2026-02-25T11:42:00.061 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 74/181 2026-02-25T11:42:00.069 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 75/181 2026-02-25T11:42:00.079 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 76/181 2026-02-25T11:42:00.085 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 77/181 2026-02-25T11:42:00.094 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 78/181 2026-02-25T11:42:00.099 INFO:teuthology.orchestra.run.trial119.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 79/181 2026-02-25T11:42:00.102 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 80/181 2026-02-25T11:42:00.146 INFO:teuthology.orchestra.run.trial119.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 81/181 2026-02-25T11:42:00.214 INFO:teuthology.orchestra.run.trial119.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 82/181 2026-02-25T11:42:00.222 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 83/181 2026-02-25T11:42:00.229 INFO:teuthology.orchestra.run.trial119.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 84/181 2026-02-25T11:42:00.242 INFO:teuthology.orchestra.run.trial119.stdout: Installing : uuid-1.6.2-55.el9.x86_64 85/181 2026-02-25T11:42:00.253 INFO:teuthology.orchestra.run.trial119.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 86/181 2026-02-25T11:42:00.295 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 87/181 2026-02-25T11:42:00.307 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/181 2026-02-25T11:42:00.313 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 89/181 2026-02-25T11:42:00.323 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 90/181 2026-02-25T11:42:00.330 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 91/181 2026-02-25T11:42:00.341 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 92/181 2026-02-25T11:42:00.348 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 93/181 2026-02-25T11:42:00.390 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 94/181 2026-02-25T11:42:00.391 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 115/181 2026-02-25T11:42:00.398 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 115/181 2026-02-25T11:42:00.406 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 95/181 2026-02-25T11:42:00.418 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 96/181 2026-02-25T11:42:00.429 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 97/181 2026-02-25T11:42:00.433 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 115/181 2026-02-25T11:42:00.448 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 116/181 2026-02-25T11:42:00.460 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-packaging-20.9-5.el9.noarch 117/181 2026-02-25T11:42:00.480 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/181 2026-02-25T11:42:00.481 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 98/181 2026-02-25T11:42:00.499 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 99/181 2026-02-25T11:42:00.505 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/181 2026-02-25T11:42:00.622 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/181 2026-02-25T11:42:00.639 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 121/181 2026-02-25T11:42:00.674 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/181 2026-02-25T11:42:00.721 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 123/181 2026-02-25T11:42:00.794 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/181 2026-02-25T11:42:00.804 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 100/181 2026-02-25T11:42:00.806 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/181 2026-02-25T11:42:00.813 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/181 2026-02-25T11:42:00.820 INFO:teuthology.orchestra.run.trial028.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 127/181 2026-02-25T11:42:00.825 INFO:teuthology.orchestra.run.trial028.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 128/181 2026-02-25T11:42:00.827 INFO:teuthology.orchestra.run.trial028.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 129/181 2026-02-25T11:42:00.843 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 101/181 2026-02-25T11:42:00.850 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 102/181 2026-02-25T11:42:00.864 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 129/181 2026-02-25T11:42:00.923 INFO:teuthology.orchestra.run.trial119.stdout: Installing : openblas-0.3.29-1.el9.x86_64 103/181 2026-02-25T11:42:00.928 INFO:teuthology.orchestra.run.trial119.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 104/181 2026-02-25T11:42:00.956 INFO:teuthology.orchestra.run.trial119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 105/181 2026-02-25T11:42:01.138 INFO:teuthology.orchestra.run.trial167.stdout:Running transaction check 2026-02-25T11:42:01.151 INFO:teuthology.orchestra.run.trial028.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 130/181 2026-02-25T11:42:01.159 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-base-2:20.3.0-5549.g92715cd3.el9.x86_64 131/181 2026-02-25T11:42:01.231 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-base-2:20.3.0-5549.g92715cd3.el9.x86_64 131/181 2026-02-25T11:42:01.231 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T11:42:01.231 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T11:42:01.231 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:01.237 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-selinux-2:20.3.0-5549.g92715cd3.el9.x86_64 132/181 2026-02-25T11:42:01.238 INFO:teuthology.orchestra.run.trial167.stdout:Transaction check succeeded. 2026-02-25T11:42:01.238 INFO:teuthology.orchestra.run.trial167.stdout:Running transaction test 2026-02-25T11:42:01.423 INFO:teuthology.orchestra.run.trial119.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 106/181 2026-02-25T11:42:01.539 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 107/181 2026-02-25T11:42:02.480 INFO:teuthology.orchestra.run.trial167.stdout:Transaction test succeeded. 2026-02-25T11:42:02.480 INFO:teuthology.orchestra.run.trial167.stdout:Running transaction 2026-02-25T11:42:02.506 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 108/181 2026-02-25T11:42:02.539 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 109/181 2026-02-25T11:42:02.542 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libzstd-devel-1.5.5-1.el9.x86_64 110/181 2026-02-25T11:42:02.550 INFO:teuthology.orchestra.run.trial119.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 111/181 2026-02-25T11:42:02.852 INFO:teuthology.orchestra.run.trial119.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 112/181 2026-02-25T11:42:02.855 INFO:teuthology.orchestra.run.trial119.stdout: Installing : librgw2-2:20.3.0-5549.g92715cd3.el9.x86_64 113/181 2026-02-25T11:42:02.892 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: librgw2-2:20.3.0-5549.g92715cd3.el9.x86_64 113/181 2026-02-25T11:42:02.894 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-rgw-2:20.3.0-5549.g92715cd3.el9.x86_64 114/181 2026-02-25T11:42:03.807 INFO:teuthology.orchestra.run.trial167.stdout: Preparing : 1/1 2026-02-25T11:42:03.812 INFO:teuthology.orchestra.run.trial167.stdout: Upgrading : libuuid-2.37.4-25.el9.x86_64 1/181 2026-02-25T11:42:03.813 INFO:teuthology.orchestra.run.trial167.stdout: Upgrading : libblkid-2.37.4-25.el9.x86_64 2/181 2026-02-25T11:42:03.838 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: libblkid-2.37.4-25.el9.x86_64 2/181 2026-02-25T11:42:03.843 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 3/181 2026-02-25T11:42:03.857 INFO:teuthology.orchestra.run.trial167.stdout: Installing : liboath-2.6.12-1.el9.x86_64 4/181 2026-02-25T11:42:03.874 INFO:teuthology.orchestra.run.trial167.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 5/181 2026-02-25T11:42:04.057 INFO:teuthology.orchestra.run.trial167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 6/181 2026-02-25T11:42:04.060 INFO:teuthology.orchestra.run.trial167.stdout: Upgrading : librados2-2:20.3.0-5549.g92715cd3.el9.x86_64 7/181 2026-02-25T11:42:04.093 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 115/181 2026-02-25T11:42:04.100 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 115/181 2026-02-25T11:42:04.103 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: librados2-2:20.3.0-5549.g92715cd3.el9.x86_64 7/181 2026-02-25T11:42:04.115 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-rados-2:20.3.0-5549.g92715cd3.el9.x86_64 8/181 2026-02-25T11:42:04.129 INFO:teuthology.orchestra.run.trial167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/181 2026-02-25T11:42:04.136 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 115/181 2026-02-25T11:42:04.152 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 116/181 2026-02-25T11:42:04.153 INFO:teuthology.orchestra.run.trial167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 10/181 2026-02-25T11:42:04.157 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 11/181 2026-02-25T11:42:04.161 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 12/181 2026-02-25T11:42:04.163 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-packaging-20.9-5.el9.noarch 117/181 2026-02-25T11:42:04.171 INFO:teuthology.orchestra.run.trial167.stdout: Installing : postgresql-private-libs-13.23-1.el9.x86_64 13/181 2026-02-25T11:42:04.184 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/181 2026-02-25T11:42:04.209 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/181 2026-02-25T11:42:04.309 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 14/181 2026-02-25T11:42:04.312 INFO:teuthology.orchestra.run.trial167.stdout: Upgrading : librbd1-2:20.3.0-5549.g92715cd3.el9.x86_64 15/181 2026-02-25T11:42:04.325 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/181 2026-02-25T11:42:04.342 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 121/181 2026-02-25T11:42:04.377 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/181 2026-02-25T11:42:04.423 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 123/181 2026-02-25T11:42:04.426 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: librbd1-2:20.3.0-5549.g92715cd3.el9.x86_64 15/181 2026-02-25T11:42:04.441 INFO:teuthology.orchestra.run.trial167.stdout: Installing : postgresql-13.23-1.el9.x86_64 16/181 2026-02-25T11:42:04.442 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libcephsqlite-2:20.3.0-5549.g92715cd3.el9.x86_64 17/181 2026-02-25T11:42:04.477 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5549.g92715cd3.el9.x86_64 17/181 2026-02-25T11:42:04.479 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libradosstriper1-2:20.3.0-5549.g92715cd3.el9.x86 18/181 2026-02-25T11:42:04.497 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/181 2026-02-25T11:42:04.504 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5549.g92715cd3.el9.x86 18/181 2026-02-25T11:42:04.509 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/181 2026-02-25T11:42:04.509 INFO:teuthology.orchestra.run.trial167.stdout: Upgrading : libmount-2.37.4-25.el9.x86_64 19/181 2026-02-25T11:42:04.514 INFO:teuthology.orchestra.run.trial167.stdout: Upgrading : libsmartcols-2.37.4-25.el9.x86_64 20/181 2026-02-25T11:42:04.516 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/181 2026-02-25T11:42:04.520 INFO:teuthology.orchestra.run.trial167.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 21/181 2026-02-25T11:42:04.523 INFO:teuthology.orchestra.run.trial119.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 127/181 2026-02-25T11:42:04.527 INFO:teuthology.orchestra.run.trial119.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 128/181 2026-02-25T11:42:04.529 INFO:teuthology.orchestra.run.trial119.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 129/181 2026-02-25T11:42:04.562 INFO:teuthology.orchestra.run.trial167.stdout: Installing : re2-1:20211101-20.el9.x86_64 22/181 2026-02-25T11:42:04.566 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 129/181 2026-02-25T11:42:04.596 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 23/181 2026-02-25T11:42:04.604 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 24/181 2026-02-25T11:42:04.613 INFO:teuthology.orchestra.run.trial167.stdout: Installing : perl-threads-1:2.25-460.el9.x86_64 25/181 2026-02-25T11:42:04.622 INFO:teuthology.orchestra.run.trial167.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/181 2026-02-25T11:42:04.627 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 27/181 2026-02-25T11:42:04.635 INFO:teuthology.orchestra.run.trial167.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 28/181 2026-02-25T11:42:04.672 INFO:teuthology.orchestra.run.trial167.stdout: Installing : unzip-6.0-59.el9.x86_64 29/181 2026-02-25T11:42:04.693 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 30/181 2026-02-25T11:42:04.704 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 31/181 2026-02-25T11:42:04.709 INFO:teuthology.orchestra.run.trial167.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 32/181 2026-02-25T11:42:04.717 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 33/181 2026-02-25T11:42:04.721 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 34/181 2026-02-25T11:42:04.723 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 35/181 2026-02-25T11:42:04.739 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 36/181 2026-02-25T11:42:04.788 INFO:teuthology.orchestra.run.trial167.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 37/181 2026-02-25T11:42:04.795 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-ceph-common-2:20.3.0-5549.g92715cd3.el9. 38/181 2026-02-25T11:42:04.798 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-ceph-argparse-2:20.3.0-5549.g92715cd3.el 39/181 2026-02-25T11:42:04.799 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libcephfs-proxy2-2:20.3.0-5549.g92715cd3.el9.x86 40/181 2026-02-25T11:42:04.852 INFO:teuthology.orchestra.run.trial119.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 130/181 2026-02-25T11:42:04.860 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-base-2:20.3.0-5549.g92715cd3.el9.x86_64 131/181 2026-02-25T11:42:04.864 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5549.g92715cd3.el9.x86 40/181 2026-02-25T11:42:04.866 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libcephfs2-2:20.3.0-5549.g92715cd3.el9.x86_64 41/181 2026-02-25T11:42:04.902 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: libcephfs2-2:20.3.0-5549.g92715cd3.el9.x86_64 41/181 2026-02-25T11:42:04.919 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-cephfs-2:20.3.0-5549.g92715cd3.el9.x86_6 42/181 2026-02-25T11:42:04.928 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-base-2:20.3.0-5549.g92715cd3.el9.x86_64 131/181 2026-02-25T11:42:04.928 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T11:42:04.928 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T11:42:04.929 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:04.930 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 43/181 2026-02-25T11:42:04.934 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-selinux-2:20.3.0-5549.g92715cd3.el9.x86_64 132/181 2026-02-25T11:42:04.968 INFO:teuthology.orchestra.run.trial167.stdout: Installing : zip-3.0-35.el9.x86_64 44/181 2026-02-25T11:42:04.978 INFO:teuthology.orchestra.run.trial167.stdout: Installing : luarocks-3.9.2-5.el9.noarch 45/181 2026-02-25T11:42:04.982 INFO:teuthology.orchestra.run.trial167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 46/181 2026-02-25T11:42:04.988 INFO:teuthology.orchestra.run.trial167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 47/181 2026-02-25T11:42:04.992 INFO:teuthology.orchestra.run.trial167.stdout: Installing : perl-threads-shared-1.61-460.el9.x86_64 48/181 2026-02-25T11:42:05.033 INFO:teuthology.orchestra.run.trial167.stdout: Installing : perl-Thread-Queue-3.14-460.el9.noarch 49/181 2026-02-25T11:42:05.108 INFO:teuthology.orchestra.run.trial167.stdout: Installing : automake-1.16.2-8.el9.noarch 50/181 2026-02-25T11:42:05.128 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 51/181 2026-02-25T11:42:05.171 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 52/181 2026-02-25T11:42:05.178 INFO:teuthology.orchestra.run.trial167.stdout: Upgrading : util-linux-core-2.37.4-25.el9.x86_64 53/181 2026-02-25T11:42:05.210 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: util-linux-core-2.37.4-25.el9.x86_64 53/181 2026-02-25T11:42:05.220 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-rbd-2:20.3.0-5549.g92715cd3.el9.x86_64 54/181 2026-02-25T11:42:05.230 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 55/181 2026-02-25T11:42:05.241 INFO:teuthology.orchestra.run.trial167.stdout: Installing : librados-devel-2:20.3.0-5549.g92715cd3.el9.x86_6 56/181 2026-02-25T11:42:05.274 INFO:teuthology.orchestra.run.trial167.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 57/181 2026-02-25T11:42:05.280 INFO:teuthology.orchestra.run.trial167.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 58/181 2026-02-25T11:42:05.425 INFO:teuthology.orchestra.run.trial167.stdout: Upgrading : libfdisk-2.37.4-25.el9.x86_64 59/181 2026-02-25T11:42:05.443 INFO:teuthology.orchestra.run.trial167.stdout: Upgrading : util-linux-2.37.4-25.el9.x86_64 60/181 2026-02-25T11:42:05.458 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libuuid-devel-2.37.4-25.el9.x86_64 61/181 2026-02-25T11:42:05.467 INFO:teuthology.orchestra.run.trial167.stdout: Installing : thrift-0.15.0-4.el9.x86_64 62/181 2026-02-25T11:42:05.473 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 63/181 2026-02-25T11:42:05.495 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 64/181 2026-02-25T11:42:05.503 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 65/181 2026-02-25T11:42:05.511 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 66/181 2026-02-25T11:42:05.529 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 67/181 2026-02-25T11:42:05.537 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 68/181 2026-02-25T11:42:05.552 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 69/181 2026-02-25T11:42:05.560 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 70/181 2026-02-25T11:42:05.573 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 71/181 2026-02-25T11:42:05.633 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 72/181 2026-02-25T11:42:06.096 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 73/181 2026-02-25T11:42:06.115 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 74/181 2026-02-25T11:42:06.122 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 75/181 2026-02-25T11:42:06.132 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 76/181 2026-02-25T11:42:06.138 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 77/181 2026-02-25T11:42:06.147 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 78/181 2026-02-25T11:42:06.152 INFO:teuthology.orchestra.run.trial167.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 79/181 2026-02-25T11:42:06.154 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 80/181 2026-02-25T11:42:06.199 INFO:teuthology.orchestra.run.trial167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 81/181 2026-02-25T11:42:06.267 INFO:teuthology.orchestra.run.trial167.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 82/181 2026-02-25T11:42:06.275 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 83/181 2026-02-25T11:42:06.281 INFO:teuthology.orchestra.run.trial167.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 84/181 2026-02-25T11:42:06.294 INFO:teuthology.orchestra.run.trial167.stdout: Installing : uuid-1.6.2-55.el9.x86_64 85/181 2026-02-25T11:42:06.305 INFO:teuthology.orchestra.run.trial167.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 86/181 2026-02-25T11:42:06.347 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 87/181 2026-02-25T11:42:06.359 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/181 2026-02-25T11:42:06.365 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 89/181 2026-02-25T11:42:06.375 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 90/181 2026-02-25T11:42:06.381 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 91/181 2026-02-25T11:42:06.393 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 92/181 2026-02-25T11:42:06.399 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 93/181 2026-02-25T11:42:06.440 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 94/181 2026-02-25T11:42:06.457 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 95/181 2026-02-25T11:42:06.467 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 96/181 2026-02-25T11:42:06.479 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 97/181 2026-02-25T11:42:06.530 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 98/181 2026-02-25T11:42:06.550 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 99/181 2026-02-25T11:42:06.853 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 100/181 2026-02-25T11:42:06.892 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 101/181 2026-02-25T11:42:06.899 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 102/181 2026-02-25T11:42:06.973 INFO:teuthology.orchestra.run.trial167.stdout: Installing : openblas-0.3.29-1.el9.x86_64 103/181 2026-02-25T11:42:06.977 INFO:teuthology.orchestra.run.trial167.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 104/181 2026-02-25T11:42:07.005 INFO:teuthology.orchestra.run.trial167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 105/181 2026-02-25T11:42:07.291 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5549.g92715cd3.el9.x86_64 132/181 2026-02-25T11:42:07.291 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /sys 2026-02-25T11:42:07.292 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /proc 2026-02-25T11:42:07.292 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /mnt 2026-02-25T11:42:07.292 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /var/tmp 2026-02-25T11:42:07.292 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /home 2026-02-25T11:42:07.292 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /root 2026-02-25T11:42:07.292 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /tmp 2026-02-25T11:42:07.292 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:07.406 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mds-2:20.3.0-5549.g92715cd3.el9.x86_64 133/181 2026-02-25T11:42:07.455 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mds-2:20.3.0-5549.g92715cd3.el9.x86_64 133/181 2026-02-25T11:42:07.455 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:07.455 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T11:42:07.455 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T11:42:07.455 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T11:42:07.456 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:07.472 INFO:teuthology.orchestra.run.trial167.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 106/181 2026-02-25T11:42:07.587 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 107/181 2026-02-25T11:42:07.690 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mon-2:20.3.0-5549.g92715cd3.el9.x86_64 134/181 2026-02-25T11:42:07.738 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mon-2:20.3.0-5549.g92715cd3.el9.x86_64 134/181 2026-02-25T11:42:07.738 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:07.738 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T11:42:07.738 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T11:42:07.738 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T11:42:07.739 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:07.749 INFO:teuthology.orchestra.run.trial028.stdout: Installing : mailcap-2.1.49-5.el9.noarch 135/181 2026-02-25T11:42:07.751 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 136/181 2026-02-25T11:42:07.790 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 137/181 2026-02-25T11:42:07.790 INFO:teuthology.orchestra.run.trial028.stdout:Creating group 'qat' with GID 992. 2026-02-25T11:42:07.790 INFO:teuthology.orchestra.run.trial028.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T11:42:07.790 INFO:teuthology.orchestra.run.trial028.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T11:42:07.790 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:07.804 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 137/181 2026-02-25T11:42:07.858 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 137/181 2026-02-25T11:42:07.858 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T11:42:07.858 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:08.537 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 138/181 2026-02-25T11:42:08.560 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 108/181 2026-02-25T11:42:08.594 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 109/181 2026-02-25T11:42:08.597 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libzstd-devel-1.5.5-1.el9.x86_64 110/181 2026-02-25T11:42:08.606 INFO:teuthology.orchestra.run.trial167.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 111/181 2026-02-25T11:42:08.918 INFO:teuthology.orchestra.run.trial167.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 112/181 2026-02-25T11:42:08.921 INFO:teuthology.orchestra.run.trial167.stdout: Installing : librgw2-2:20.3.0-5549.g92715cd3.el9.x86_64 113/181 2026-02-25T11:42:08.957 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: librgw2-2:20.3.0-5549.g92715cd3.el9.x86_64 113/181 2026-02-25T11:42:08.960 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-rgw-2:20.3.0-5549.g92715cd3.el9.x86_64 114/181 2026-02-25T11:42:09.338 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-osd-classic-2:20.3.0-5549.g92715cd3.el9.x86 139/181 2026-02-25T11:42:09.428 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-osd-crimson-2:20.3.0-5549.g92715cd3.el9.x86 140/181 2026-02-25T11:42:09.434 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-volume-2:20.3.0-5549.g92715cd3.el9.noarch 141/181 2026-02-25T11:42:09.468 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-volume-2:20.3.0-5549.g92715cd3.el9.noarch 141/181 2026-02-25T11:42:09.468 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:09.468 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T11:42:09.468 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:09.483 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-osd-2:20.3.0-5549.g92715cd3.el9.x86_64 142/181 2026-02-25T11:42:09.537 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-osd-2:20.3.0-5549.g92715cd3.el9.x86_64 142/181 2026-02-25T11:42:09.537 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:09.537 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T11:42:09.538 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T11:42:09.538 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T11:42:09.538 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:09.651 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 143/181 2026-02-25T11:42:09.655 INFO:teuthology.orchestra.run.trial028.stdout: Installing : cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 143/181 2026-02-25T11:42:09.668 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5549.g92715cd3.e 144/181 2026-02-25T11:42:09.701 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5549.g92715cd3. 145/181 2026-02-25T11:42:09.704 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5549.g92715cd3.el9.noa 146/181 2026-02-25T11:42:10.196 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 115/181 2026-02-25T11:42:10.202 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 115/181 2026-02-25T11:42:10.240 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 115/181 2026-02-25T11:42:10.256 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 116/181 2026-02-25T11:42:10.268 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 117/181 2026-02-25T11:42:10.289 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/181 2026-02-25T11:42:10.313 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/181 2026-02-25T11:42:10.430 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/181 2026-02-25T11:42:10.447 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 121/181 2026-02-25T11:42:10.482 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/181 2026-02-25T11:42:10.528 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 123/181 2026-02-25T11:42:10.602 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/181 2026-02-25T11:42:10.613 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/181 2026-02-25T11:42:10.620 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/181 2026-02-25T11:42:10.627 INFO:teuthology.orchestra.run.trial167.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 127/181 2026-02-25T11:42:10.632 INFO:teuthology.orchestra.run.trial167.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 128/181 2026-02-25T11:42:10.634 INFO:teuthology.orchestra.run.trial167.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 129/181 2026-02-25T11:42:10.671 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 129/181 2026-02-25T11:42:10.961 INFO:teuthology.orchestra.run.trial167.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 130/181 2026-02-25T11:42:10.970 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-base-2:20.3.0-5549.g92715cd3.el9.x86_64 131/181 2026-02-25T11:42:10.995 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5549.g92715cd3.el9.x86_64 132/181 2026-02-25T11:42:10.995 INFO:teuthology.orchestra.run.trial119.stdout:skipping the directory /sys 2026-02-25T11:42:10.995 INFO:teuthology.orchestra.run.trial119.stdout:skipping the directory /proc 2026-02-25T11:42:10.995 INFO:teuthology.orchestra.run.trial119.stdout:skipping the directory /mnt 2026-02-25T11:42:10.995 INFO:teuthology.orchestra.run.trial119.stdout:skipping the directory /var/tmp 2026-02-25T11:42:10.995 INFO:teuthology.orchestra.run.trial119.stdout:skipping the directory /home 2026-02-25T11:42:10.995 INFO:teuthology.orchestra.run.trial119.stdout:skipping the directory /root 2026-02-25T11:42:10.995 INFO:teuthology.orchestra.run.trial119.stdout:skipping the directory /tmp 2026-02-25T11:42:10.996 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:11.039 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-base-2:20.3.0-5549.g92715cd3.el9.x86_64 131/181 2026-02-25T11:42:11.039 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T11:42:11.039 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T11:42:11.040 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:11.045 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-selinux-2:20.3.0-5549.g92715cd3.el9.x86_64 132/181 2026-02-25T11:42:11.108 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-mds-2:20.3.0-5549.g92715cd3.el9.x86_64 133/181 2026-02-25T11:42:11.157 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-mds-2:20.3.0-5549.g92715cd3.el9.x86_64 133/181 2026-02-25T11:42:11.158 INFO:teuthology.orchestra.run.trial119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:11.158 INFO:teuthology.orchestra.run.trial119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T11:42:11.158 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T11:42:11.158 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T11:42:11.158 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:11.389 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-mon-2:20.3.0-5549.g92715cd3.el9.x86_64 134/181 2026-02-25T11:42:11.437 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-mon-2:20.3.0-5549.g92715cd3.el9.x86_64 134/181 2026-02-25T11:42:11.438 INFO:teuthology.orchestra.run.trial119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:11.438 INFO:teuthology.orchestra.run.trial119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T11:42:11.438 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T11:42:11.438 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T11:42:11.438 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:11.448 INFO:teuthology.orchestra.run.trial119.stdout: Installing : mailcap-2.1.49-5.el9.noarch 135/181 2026-02-25T11:42:11.451 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 136/181 2026-02-25T11:42:11.488 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 137/181 2026-02-25T11:42:11.489 INFO:teuthology.orchestra.run.trial119.stdout:Creating group 'qat' with GID 992. 2026-02-25T11:42:11.489 INFO:teuthology.orchestra.run.trial119.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T11:42:11.489 INFO:teuthology.orchestra.run.trial119.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T11:42:11.489 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:11.503 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 137/181 2026-02-25T11:42:11.555 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 137/181 2026-02-25T11:42:11.556 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T11:42:11.556 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:11.594 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5549.g92715cd3.el9.noa 146/181 2026-02-25T11:42:11.609 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5549.g92715cd3.el9.n 147/181 2026-02-25T11:42:12.117 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5549.g92715cd3.el9.n 147/181 2026-02-25T11:42:12.120 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5549.g927 148/181 2026-02-25T11:42:12.216 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5549.g927 148/181 2026-02-25T11:42:12.224 INFO:teuthology.orchestra.run.trial119.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 138/181 2026-02-25T11:42:12.265 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5549.g92715cd3.el 149/181 2026-02-25T11:42:12.268 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-2:20.3.0-5549.g92715cd3.el9.x86_64 150/181 2026-02-25T11:42:12.314 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5549.g92715cd3.el9.x86_64 150/181 2026-02-25T11:42:12.315 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:12.315 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T11:42:12.315 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T11:42:12.315 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T11:42:12.315 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:12.332 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-rook-2:20.3.0-5549.g92715cd3.el9.noarch 151/181 2026-02-25T11:42:12.354 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5549.g92715cd3.el9.noarch 151/181 2026-02-25T11:42:12.966 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-2:20.3.0-5549.g92715cd3.el9.x86_64 152/181 2026-02-25T11:42:12.970 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-radosgw-2:20.3.0-5549.g92715cd3.el9.x86_64 153/181 2026-02-25T11:42:13.007 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-osd-classic-2:20.3.0-5549.g92715cd3.el9.x86 139/181 2026-02-25T11:42:13.021 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5549.g92715cd3.el9.x86_64 153/181 2026-02-25T11:42:13.021 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:13.021 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T11:42:13.021 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T11:42:13.021 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T11:42:13.021 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:13.033 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5549.g92715 154/181 2026-02-25T11:42:13.079 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5549.g92715 154/181 2026-02-25T11:42:13.079 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:13.080 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T11:42:13.080 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:13.096 INFO:teuthology.orchestra.run.trial028.stdout: Installing : cephfs-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 155/181 2026-02-25T11:42:13.097 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-osd-crimson-2:20.3.0-5549.g92715cd3.el9.x86 140/181 2026-02-25T11:42:13.103 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-volume-2:20.3.0-5549.g92715cd3.el9.noarch 141/181 2026-02-25T11:42:13.137 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-volume-2:20.3.0-5549.g92715cd3.el9.noarch 141/181 2026-02-25T11:42:13.138 INFO:teuthology.orchestra.run.trial119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:13.138 INFO:teuthology.orchestra.run.trial119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T11:42:13.138 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:13.143 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: cephfs-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 155/181 2026-02-25T11:42:13.143 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:13.143 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2026-02-25T11:42:13.143 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2026-02-25T11:42:13.144 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2026-02-25T11:42:13.144 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:13.152 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-osd-2:20.3.0-5549.g92715cd3.el9.x86_64 142/181 2026-02-25T11:42:13.206 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-osd-2:20.3.0-5549.g92715cd3.el9.x86_64 142/181 2026-02-25T11:42:13.206 INFO:teuthology.orchestra.run.trial119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:13.206 INFO:teuthology.orchestra.run.trial119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T11:42:13.206 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T11:42:13.206 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T11:42:13.206 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:13.282 INFO:teuthology.orchestra.run.trial028.stdout: Installing : rbd-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 156/181 2026-02-25T11:42:13.323 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 143/181 2026-02-25T11:42:13.327 INFO:teuthology.orchestra.run.trial119.stdout: Installing : cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 143/181 2026-02-25T11:42:13.329 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 156/181 2026-02-25T11:42:13.329 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:13.329 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T11:42:13.329 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T11:42:13.330 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T11:42:13.330 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:13.339 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5549.g92715cd3.e 144/181 2026-02-25T11:42:13.372 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5549.g92715cd3. 145/181 2026-02-25T11:42:13.376 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5549.g92715cd3.el9.noa 146/181 2026-02-25T11:42:15.249 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5549.g92715cd3.el9.noa 146/181 2026-02-25T11:42:15.263 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5549.g92715cd3.el9.n 147/181 2026-02-25T11:42:15.764 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5549.g92715cd3.el9.n 147/181 2026-02-25T11:42:15.766 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5549.g927 148/181 2026-02-25T11:42:15.862 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5549.g927 148/181 2026-02-25T11:42:15.910 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5549.g92715cd3.el 149/181 2026-02-25T11:42:15.913 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-mgr-2:20.3.0-5549.g92715cd3.el9.x86_64 150/181 2026-02-25T11:42:15.960 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5549.g92715cd3.el9.x86_64 150/181 2026-02-25T11:42:15.961 INFO:teuthology.orchestra.run.trial119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:15.961 INFO:teuthology.orchestra.run.trial119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T11:42:15.961 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T11:42:15.961 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T11:42:15.961 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:15.978 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-mgr-rook-2:20.3.0-5549.g92715cd3.el9.noarch 151/181 2026-02-25T11:42:15.999 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5549.g92715cd3.el9.noarch 151/181 2026-02-25T11:42:16.600 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-2:20.3.0-5549.g92715cd3.el9.x86_64 152/181 2026-02-25T11:42:16.605 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-radosgw-2:20.3.0-5549.g92715cd3.el9.x86_64 153/181 2026-02-25T11:42:16.653 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5549.g92715cd3.el9.x86_64 153/181 2026-02-25T11:42:16.653 INFO:teuthology.orchestra.run.trial119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:16.653 INFO:teuthology.orchestra.run.trial119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T11:42:16.653 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T11:42:16.653 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T11:42:16.654 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:16.666 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5549.g92715 154/181 2026-02-25T11:42:16.710 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5549.g92715 154/181 2026-02-25T11:42:16.711 INFO:teuthology.orchestra.run.trial119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:16.711 INFO:teuthology.orchestra.run.trial119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T11:42:16.711 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:16.728 INFO:teuthology.orchestra.run.trial119.stdout: Installing : cephfs-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 155/181 2026-02-25T11:42:16.773 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: cephfs-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 155/181 2026-02-25T11:42:16.774 INFO:teuthology.orchestra.run.trial119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:16.774 INFO:teuthology.orchestra.run.trial119.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2026-02-25T11:42:16.774 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2026-02-25T11:42:16.774 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2026-02-25T11:42:16.774 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:16.908 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-test-2:20.3.0-5549.g92715cd3.el9.x86_64 157/181 2026-02-25T11:42:16.908 INFO:teuthology.orchestra.run.trial119.stdout: Installing : rbd-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 156/181 2026-02-25T11:42:16.913 INFO:teuthology.orchestra.run.trial028.stdout: Installing : elfutils-libelf-devel-0.194-1.el9.x86_64 158/181 2026-02-25T11:42:16.955 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 156/181 2026-02-25T11:42:16.955 INFO:teuthology.orchestra.run.trial119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:16.955 INFO:teuthology.orchestra.run.trial119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T11:42:16.955 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T11:42:16.955 INFO:teuthology.orchestra.run.trial119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T11:42:16.955 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:17.023 INFO:teuthology.orchestra.run.trial028.stdout: Installing : cephfs-shell-2:20.3.0-5549.g92715cd3.el9.x86_64 159/181 2026-02-25T11:42:17.042 INFO:teuthology.orchestra.run.trial028.stdout: Installing : postgresql-contrib-13.23-1.el9.x86_64 160/181 2026-02-25T11:42:17.049 INFO:teuthology.orchestra.run.trial028.stdout: Installing : xfsprogs-devel-6.4.0-7.el9.x86_64 161/181 2026-02-25T11:42:17.063 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libcephfs-devel-2:20.3.0-5549.g92715cd3.el9.x86_ 162/181 2026-02-25T11:42:17.066 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libtool-2.4.6-46.el9.x86_64 163/181 2026-02-25T11:42:17.130 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5549.g92715cd3.el9.x86_64 132/181 2026-02-25T11:42:17.130 INFO:teuthology.orchestra.run.trial167.stdout:skipping the directory /sys 2026-02-25T11:42:17.130 INFO:teuthology.orchestra.run.trial167.stdout:skipping the directory /proc 2026-02-25T11:42:17.130 INFO:teuthology.orchestra.run.trial167.stdout:skipping the directory /mnt 2026-02-25T11:42:17.130 INFO:teuthology.orchestra.run.trial167.stdout:skipping the directory /var/tmp 2026-02-25T11:42:17.130 INFO:teuthology.orchestra.run.trial167.stdout:skipping the directory /home 2026-02-25T11:42:17.131 INFO:teuthology.orchestra.run.trial167.stdout:skipping the directory /root 2026-02-25T11:42:17.131 INFO:teuthology.orchestra.run.trial167.stdout:skipping the directory /tmp 2026-02-25T11:42:17.131 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:17.249 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-mds-2:20.3.0-5549.g92715cd3.el9.x86_64 133/181 2026-02-25T11:42:17.276 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: postgresql-server-13.23-1.el9.x86_64 164/181 2026-02-25T11:42:17.286 INFO:teuthology.orchestra.run.trial028.stdout: Installing : postgresql-server-13.23-1.el9.x86_64 164/181 2026-02-25T11:42:17.298 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-mds-2:20.3.0-5549.g92715cd3.el9.x86_64 133/181 2026-02-25T11:42:17.298 INFO:teuthology.orchestra.run.trial167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:17.298 INFO:teuthology.orchestra.run.trial167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T11:42:17.298 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T11:42:17.298 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T11:42:17.299 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:17.326 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: postgresql-server-13.23-1.el9.x86_64 164/181 2026-02-25T11:42:17.338 INFO:teuthology.orchestra.run.trial028.stdout: Installing : rbd-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 165/181 2026-02-25T11:42:17.344 INFO:teuthology.orchestra.run.trial028.stdout: Installing : rbd-nbd-2:20.3.0-5549.g92715cd3.el9.x86_64 166/181 2026-02-25T11:42:17.390 INFO:teuthology.orchestra.run.trial028.stdout: Installing : cephfs-top-2:20.3.0-5549.g92715cd3.el9.noarch 167/181 2026-02-25T11:42:17.534 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-mon-2:20.3.0-5549.g92715cd3.el9.x86_64 134/181 2026-02-25T11:42:17.582 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-mon-2:20.3.0-5549.g92715cd3.el9.x86_64 134/181 2026-02-25T11:42:17.582 INFO:teuthology.orchestra.run.trial167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:17.582 INFO:teuthology.orchestra.run.trial167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T11:42:17.582 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T11:42:17.582 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T11:42:17.582 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:17.593 INFO:teuthology.orchestra.run.trial167.stdout: Installing : mailcap-2.1.49-5.el9.noarch 135/181 2026-02-25T11:42:17.595 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 136/181 2026-02-25T11:42:17.633 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 137/181 2026-02-25T11:42:17.633 INFO:teuthology.orchestra.run.trial167.stdout:Creating group 'qat' with GID 992. 2026-02-25T11:42:17.633 INFO:teuthology.orchestra.run.trial167.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T11:42:17.633 INFO:teuthology.orchestra.run.trial167.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T11:42:17.633 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:17.647 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 137/181 2026-02-25T11:42:17.701 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 137/181 2026-02-25T11:42:17.701 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T11:42:17.701 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:18.264 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 168/181 2026-02-25T11:42:18.300 INFO:teuthology.orchestra.run.trial028.stdout: Installing : openssl-devel-1:3.5.5-1.el9.x86_64 169/181 2026-02-25T11:42:18.315 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 170/181 2026-02-25T11:42:18.358 INFO:teuthology.orchestra.run.trial028.stdout: Installing : flex-2.6.4-9.el9.x86_64 171/181 2026-02-25T11:42:18.371 INFO:teuthology.orchestra.run.trial028.stdout: Installing : bison-3.7.4-5.el9.x86_64 172/181 2026-02-25T11:42:18.386 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : util-linux-2.37.4-21.el9.x86_64 173/181 2026-02-25T11:42:18.392 INFO:teuthology.orchestra.run.trial167.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 138/181 2026-02-25T11:42:18.392 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : util-linux-core-2.37.4-21.el9.x86_64 174/181 2026-02-25T11:42:18.395 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : libfdisk-2.37.4-21.el9.x86_64 175/181 2026-02-25T11:42:18.399 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : libmount-2.37.4-21.el9.x86_64 176/181 2026-02-25T11:42:18.399 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 177/181 2026-02-25T11:42:18.429 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 177/181 2026-02-25T11:42:18.429 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 178/181 2026-02-25T11:42:18.455 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 178/181 2026-02-25T11:42:18.458 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : libblkid-2.37.4-21.el9.x86_64 179/181 2026-02-25T11:42:18.460 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : libuuid-2.37.4-21.el9.x86_64 180/181 2026-02-25T11:42:18.462 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : libsmartcols-2.37.4-21.el9.x86_64 181/181 2026-02-25T11:42:18.477 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5549.g92715cd3.el9.x86 181/181 2026-02-25T11:42:18.511 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5549.g92715cd3.el9.x86 181/181 2026-02-25T11:42:18.675 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libsmartcols-2.37.4-21.el9.x86_64 181/181 2026-02-25T11:42:18.675 INFO:teuthology.orchestra.run.trial028.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2026-02-25T11:42:18.675 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:19.216 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-osd-classic-2:20.3.0-5549.g92715cd3.el9.x86 139/181 2026-02-25T11:42:19.305 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-osd-crimson-2:20.3.0-5549.g92715cd3.el9.x86 140/181 2026-02-25T11:42:19.311 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-volume-2:20.3.0-5549.g92715cd3.el9.noarch 141/181 2026-02-25T11:42:19.344 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-volume-2:20.3.0-5549.g92715cd3.el9.noarch 141/181 2026-02-25T11:42:19.344 INFO:teuthology.orchestra.run.trial167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:19.344 INFO:teuthology.orchestra.run.trial167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T11:42:19.344 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:19.359 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-osd-2:20.3.0-5549.g92715cd3.el9.x86_64 142/181 2026-02-25T11:42:19.412 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-osd-2:20.3.0-5549.g92715cd3.el9.x86_64 142/181 2026-02-25T11:42:19.413 INFO:teuthology.orchestra.run.trial167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:19.413 INFO:teuthology.orchestra.run.trial167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T11:42:19.413 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T11:42:19.413 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T11:42:19.413 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:19.528 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 143/181 2026-02-25T11:42:19.532 INFO:teuthology.orchestra.run.trial167.stdout: Installing : cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 143/181 2026-02-25T11:42:19.544 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5549.g92715cd3.e 144/181 2026-02-25T11:42:19.577 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5549.g92715cd3. 145/181 2026-02-25T11:42:19.581 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5549.g92715cd3.el9.noa 146/181 2026-02-25T11:42:19.739 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-2:20.3.0-5549.g92715cd3.el9.x86_64 1/181 2026-02-25T11:42:19.740 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-base-2:20.3.0-5549.g92715cd3.el9.x86_64 2/181 2026-02-25T11:42:19.740 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 3/181 2026-02-25T11:42:19.740 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 4/181 2026-02-25T11:42:19.740 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5549.g92715 5/181 2026-02-25T11:42:19.740 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mds-2:20.3.0-5549.g92715cd3.el9.x86_64 6/181 2026-02-25T11:42:19.740 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-2:20.3.0-5549.g92715cd3.el9.x86_64 7/181 2026-02-25T11:42:19.740 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mon-2:20.3.0-5549.g92715cd3.el9.x86_64 8/181 2026-02-25T11:42:19.740 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-osd-2:20.3.0-5549.g92715cd3.el9.x86_64 9/181 2026-02-25T11:42:19.741 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-osd-classic-2:20.3.0-5549.g92715cd3.el9.x86 10/181 2026-02-25T11:42:19.741 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5549.g92715cd3.el9.x86 11/181 2026-02-25T11:42:19.741 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-radosgw-2:20.3.0-5549.g92715cd3.el9.x86_64 12/181 2026-02-25T11:42:19.741 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-selinux-2:20.3.0-5549.g92715cd3.el9.x86_64 13/181 2026-02-25T11:42:19.741 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-test-2:20.3.0-5549.g92715cd3.el9.x86_64 14/181 2026-02-25T11:42:19.741 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : cephfs-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 15/181 2026-02-25T11:42:19.741 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : cephfs-shell-2:20.3.0-5549.g92715cd3.el9.x86_64 16/181 2026-02-25T11:42:19.741 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libcephfs-devel-2:20.3.0-5549.g92715cd3.el9.x86_ 17/181 2026-02-25T11:42:19.741 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5549.g92715cd3.el9.x86 18/181 2026-02-25T11:42:19.741 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libcephfs2-2:20.3.0-5549.g92715cd3.el9.x86_64 19/181 2026-02-25T11:42:19.742 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libcephsqlite-2:20.3.0-5549.g92715cd3.el9.x86_64 20/181 2026-02-25T11:42:19.742 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librados-devel-2:20.3.0-5549.g92715cd3.el9.x86_6 21/181 2026-02-25T11:42:19.742 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libradosstriper1-2:20.3.0-5549.g92715cd3.el9.x86 22/181 2026-02-25T11:42:19.742 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librgw2-2:20.3.0-5549.g92715cd3.el9.x86_64 23/181 2026-02-25T11:42:19.742 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5549.g92715cd3.el 24/181 2026-02-25T11:42:19.742 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-ceph-common-2:20.3.0-5549.g92715cd3.el9. 25/181 2026-02-25T11:42:19.742 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cephfs-2:20.3.0-5549.g92715cd3.el9.x86_6 26/181 2026-02-25T11:42:19.742 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-rados-2:20.3.0-5549.g92715cd3.el9.x86_64 27/181 2026-02-25T11:42:19.742 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-rbd-2:20.3.0-5549.g92715cd3.el9.x86_64 28/181 2026-02-25T11:42:19.742 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-rgw-2:20.3.0-5549.g92715cd3.el9.x86_64 29/181 2026-02-25T11:42:19.742 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : rbd-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 30/181 2026-02-25T11:42:19.743 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : rbd-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 31/181 2026-02-25T11:42:19.743 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : rbd-nbd-2:20.3.0-5549.g92715cd3.el9.x86_64 32/181 2026-02-25T11:42:19.743 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5549.g92715cd3. 33/181 2026-02-25T11:42:19.743 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5549.g92715cd3.el9.noa 34/181 2026-02-25T11:42:19.743 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5549.g92715cd3.el9.n 35/181 2026-02-25T11:42:19.743 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5549.g927 36/181 2026-02-25T11:42:19.743 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5549.g92715cd3.el 37/181 2026-02-25T11:42:19.743 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5549.g92715cd3.el9.noarch 38/181 2026-02-25T11:42:19.743 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5549.g92715cd3.e 39/181 2026-02-25T11:42:19.743 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-volume-2:20.3.0-5549.g92715cd3.el9.noarch 40/181 2026-02-25T11:42:19.743 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 41/181 2026-02-25T11:42:19.744 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : cephfs-top-2:20.3.0-5549.g92715cd3.el9.noarch 42/181 2026-02-25T11:42:19.744 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 43/181 2026-02-25T11:42:19.744 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 44/181 2026-02-25T11:42:19.744 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 45/181 2026-02-25T11:42:19.744 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 46/181 2026-02-25T11:42:19.744 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 47/181 2026-02-25T11:42:19.744 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 48/181 2026-02-25T11:42:19.744 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 49/181 2026-02-25T11:42:19.744 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 50/181 2026-02-25T11:42:19.744 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 51/181 2026-02-25T11:42:19.744 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 52/181 2026-02-25T11:42:19.745 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 53/181 2026-02-25T11:42:19.745 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 54/181 2026-02-25T11:42:19.745 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-ply-3.11-14.el9.noarch 55/181 2026-02-25T11:42:19.745 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 56/181 2026-02-25T11:42:19.745 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 57/181 2026-02-25T11:42:19.745 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 58/181 2026-02-25T11:42:19.745 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 59/181 2026-02-25T11:42:19.745 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : unzip-6.0-59.el9.x86_64 60/181 2026-02-25T11:42:19.745 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : zip-3.0-35.el9.x86_64 61/181 2026-02-25T11:42:19.745 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : automake-1.16.2-8.el9.noarch 62/181 2026-02-25T11:42:19.746 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : bison-3.7.4-5.el9.x86_64 63/181 2026-02-25T11:42:19.746 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 64/181 2026-02-25T11:42:19.746 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : elfutils-libelf-devel-0.194-1.el9.x86_64 65/181 2026-02-25T11:42:19.746 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : flex-2.6.4-9.el9.x86_64 66/181 2026-02-25T11:42:19.746 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 67/181 2026-02-25T11:42:19.746 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 68/181 2026-02-25T11:42:19.746 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 69/181 2026-02-25T11:42:19.746 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 70/181 2026-02-25T11:42:19.746 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 71/181 2026-02-25T11:42:19.746 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 72/181 2026-02-25T11:42:19.746 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 73/181 2026-02-25T11:42:19.746 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 74/181 2026-02-25T11:42:19.747 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 75/181 2026-02-25T11:42:19.747 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 76/181 2026-02-25T11:42:19.747 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libuuid-devel-2.37.4-25.el9.x86_64 77/181 2026-02-25T11:42:19.747 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 78/181 2026-02-25T11:42:19.747 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libzstd-devel-1.5.5-1.el9.x86_64 79/181 2026-02-25T11:42:19.747 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 80/181 2026-02-25T11:42:19.747 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : lua-5.4.4-4.el9.x86_64 81/181 2026-02-25T11:42:19.747 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 82/181 2026-02-25T11:42:19.747 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 83/181 2026-02-25T11:42:19.747 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openssl-devel-1:3.5.5-1.el9.x86_64 84/181 2026-02-25T11:42:19.747 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : perl-Thread-Queue-3.14-460.el9.noarch 85/181 2026-02-25T11:42:19.747 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : perl-threads-1:2.25-460.el9.x86_64 86/181 2026-02-25T11:42:19.748 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : perl-threads-shared-1.61-460.el9.x86_64 87/181 2026-02-25T11:42:19.748 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : postgresql-13.23-1.el9.x86_64 88/181 2026-02-25T11:42:19.748 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : postgresql-contrib-13.23-1.el9.x86_64 89/181 2026-02-25T11:42:19.748 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : postgresql-private-libs-13.23-1.el9.x86_64 90/181 2026-02-25T11:42:19.748 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : postgresql-server-13.23-1.el9.x86_64 91/181 2026-02-25T11:42:19.748 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 92/181 2026-02-25T11:42:19.748 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 93/181 2026-02-25T11:42:19.748 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 94/181 2026-02-25T11:42:19.748 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 95/181 2026-02-25T11:42:19.748 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 96/181 2026-02-25T11:42:19.748 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 97/181 2026-02-25T11:42:19.748 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 98/181 2026-02-25T11:42:19.749 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 99/181 2026-02-25T11:42:19.749 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 100/181 2026-02-25T11:42:19.749 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 101/181 2026-02-25T11:42:19.749 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 102/181 2026-02-25T11:42:19.749 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 103/181 2026-02-25T11:42:19.749 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 104/181 2026-02-25T11:42:19.749 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 105/181 2026-02-25T11:42:19.749 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 106/181 2026-02-25T11:42:19.749 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 107/181 2026-02-25T11:42:19.749 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 108/181 2026-02-25T11:42:19.749 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 109/181 2026-02-25T11:42:19.749 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 110/181 2026-02-25T11:42:19.750 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 111/181 2026-02-25T11:42:19.750 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 112/181 2026-02-25T11:42:19.750 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 113/181 2026-02-25T11:42:19.750 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 114/181 2026-02-25T11:42:19.750 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : xfsprogs-devel-6.4.0-7.el9.x86_64 115/181 2026-02-25T11:42:19.750 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 116/181 2026-02-25T11:42:19.750 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 117/181 2026-02-25T11:42:19.750 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 118/181 2026-02-25T11:42:19.750 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 119/181 2026-02-25T11:42:19.750 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 120/181 2026-02-25T11:42:19.750 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 121/181 2026-02-25T11:42:19.750 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 122/181 2026-02-25T11:42:19.751 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 123/181 2026-02-25T11:42:19.751 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 124/181 2026-02-25T11:42:19.751 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 125/181 2026-02-25T11:42:19.751 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 126/181 2026-02-25T11:42:19.751 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 127/181 2026-02-25T11:42:19.751 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 128/181 2026-02-25T11:42:19.751 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 129/181 2026-02-25T11:42:19.751 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 130/181 2026-02-25T11:42:19.751 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 131/181 2026-02-25T11:42:19.751 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 132/181 2026-02-25T11:42:19.751 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 133/181 2026-02-25T11:42:19.751 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 134/181 2026-02-25T11:42:19.751 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 135/181 2026-02-25T11:42:19.752 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 136/181 2026-02-25T11:42:19.752 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 137/181 2026-02-25T11:42:19.752 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 138/181 2026-02-25T11:42:19.752 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 139/181 2026-02-25T11:42:19.752 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 140/181 2026-02-25T11:42:19.752 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 141/181 2026-02-25T11:42:19.752 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 142/181 2026-02-25T11:42:19.752 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 143/181 2026-02-25T11:42:19.752 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 144/181 2026-02-25T11:42:19.752 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 145/181 2026-02-25T11:42:19.752 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 146/181 2026-02-25T11:42:19.752 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 147/181 2026-02-25T11:42:19.752 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 148/181 2026-02-25T11:42:19.753 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 149/181 2026-02-25T11:42:19.753 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 150/181 2026-02-25T11:42:19.753 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 151/181 2026-02-25T11:42:19.753 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 152/181 2026-02-25T11:42:19.753 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 153/181 2026-02-25T11:42:19.753 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 154/181 2026-02-25T11:42:19.753 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 155/181 2026-02-25T11:42:19.753 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 156/181 2026-02-25T11:42:19.753 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 157/181 2026-02-25T11:42:19.753 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 158/181 2026-02-25T11:42:19.753 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 159/181 2026-02-25T11:42:19.753 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 160/181 2026-02-25T11:42:19.754 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : re2-1:20211101-20.el9.x86_64 161/181 2026-02-25T11:42:19.754 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 162/181 2026-02-25T11:42:19.754 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 163/181 2026-02-25T11:42:19.754 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librados2-2:20.3.0-5549.g92715cd3.el9.x86_64 164/181 2026-02-25T11:42:19.754 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 165/181 2026-02-25T11:42:19.754 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librbd1-2:20.3.0-5549.g92715cd3.el9.x86_64 166/181 2026-02-25T11:42:19.754 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 167/181 2026-02-25T11:42:19.754 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libblkid-2.37.4-25.el9.x86_64 168/181 2026-02-25T11:42:19.754 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libblkid-2.37.4-21.el9.x86_64 169/181 2026-02-25T11:42:19.754 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libfdisk-2.37.4-25.el9.x86_64 170/181 2026-02-25T11:42:19.755 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libfdisk-2.37.4-21.el9.x86_64 171/181 2026-02-25T11:42:19.755 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libmount-2.37.4-25.el9.x86_64 172/181 2026-02-25T11:42:19.755 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libmount-2.37.4-21.el9.x86_64 173/181 2026-02-25T11:42:19.755 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libsmartcols-2.37.4-25.el9.x86_64 174/181 2026-02-25T11:42:19.755 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libsmartcols-2.37.4-21.el9.x86_64 175/181 2026-02-25T11:42:19.755 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libuuid-2.37.4-25.el9.x86_64 176/181 2026-02-25T11:42:19.755 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libuuid-2.37.4-21.el9.x86_64 177/181 2026-02-25T11:42:19.755 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : util-linux-2.37.4-25.el9.x86_64 178/181 2026-02-25T11:42:19.755 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : util-linux-2.37.4-21.el9.x86_64 179/181 2026-02-25T11:42:19.755 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : util-linux-core-2.37.4-25.el9.x86_64 180/181 2026-02-25T11:42:19.835 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : util-linux-core-2.37.4-21.el9.x86_64 181/181 2026-02-25T11:42:19.835 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:19.836 INFO:teuthology.orchestra.run.trial028.stdout:Upgraded: 2026-02-25T11:42:19.836 INFO:teuthology.orchestra.run.trial028.stdout: libblkid-2.37.4-25.el9.x86_64 2026-02-25T11:42:19.836 INFO:teuthology.orchestra.run.trial028.stdout: libfdisk-2.37.4-25.el9.x86_64 2026-02-25T11:42:19.836 INFO:teuthology.orchestra.run.trial028.stdout: libmount-2.37.4-25.el9.x86_64 2026-02-25T11:42:19.836 INFO:teuthology.orchestra.run.trial028.stdout: librados2-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.836 INFO:teuthology.orchestra.run.trial028.stdout: librbd1-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.836 INFO:teuthology.orchestra.run.trial028.stdout: libsmartcols-2.37.4-25.el9.x86_64 2026-02-25T11:42:19.836 INFO:teuthology.orchestra.run.trial028.stdout: libuuid-2.37.4-25.el9.x86_64 2026-02-25T11:42:19.836 INFO:teuthology.orchestra.run.trial028.stdout: util-linux-2.37.4-25.el9.x86_64 2026-02-25T11:42:19.836 INFO:teuthology.orchestra.run.trial028.stdout: util-linux-core-2.37.4-25.el9.x86_64 2026-02-25T11:42:19.836 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-25T11:42:19.837 INFO:teuthology.orchestra.run.trial028.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T11:42:19.837 INFO:teuthology.orchestra.run.trial028.stdout: automake-1.16.2-8.el9.noarch 2026-02-25T11:42:19.837 INFO:teuthology.orchestra.run.trial028.stdout: bison-3.7.4-5.el9.x86_64 2026-02-25T11:42:19.837 INFO:teuthology.orchestra.run.trial028.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T11:42:19.837 INFO:teuthology.orchestra.run.trial028.stdout: ceph-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.837 INFO:teuthology.orchestra.run.trial028.stdout: ceph-base-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.837 INFO:teuthology.orchestra.run.trial028.stdout: ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.837 INFO:teuthology.orchestra.run.trial028.stdout: ceph-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.837 INFO:teuthology.orchestra.run.trial028.stdout: ceph-grafana-dashboards-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:19.837 INFO:teuthology.orchestra.run.trial028.stdout: ceph-immutable-object-cache-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.837 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mds-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.838 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.838 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:19.838 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-dashboard-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:19.838 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:19.838 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-modules-core-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:19.838 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-rook-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:19.838 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mon-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.838 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.838 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-classic-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.838 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-crimson-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.838 INFO:teuthology.orchestra.run.trial028.stdout: ceph-prometheus-alerts-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:19.838 INFO:teuthology.orchestra.run.trial028.stdout: ceph-radosgw-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.839 INFO:teuthology.orchestra.run.trial028.stdout: ceph-selinux-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.839 INFO:teuthology.orchestra.run.trial028.stdout: ceph-test-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.839 INFO:teuthology.orchestra.run.trial028.stdout: ceph-volume-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:19.839 INFO:teuthology.orchestra.run.trial028.stdout: cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:19.839 INFO:teuthology.orchestra.run.trial028.stdout: cephfs-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.839 INFO:teuthology.orchestra.run.trial028.stdout: cephfs-shell-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.839 INFO:teuthology.orchestra.run.trial028.stdout: cephfs-top-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:19.839 INFO:teuthology.orchestra.run.trial028.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T11:42:19.839 INFO:teuthology.orchestra.run.trial028.stdout: elfutils-libelf-devel-0.194-1.el9.x86_64 2026-02-25T11:42:19.839 INFO:teuthology.orchestra.run.trial028.stdout: flex-2.6.4-9.el9.x86_64 2026-02-25T11:42:19.839 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T11:42:19.839 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T11:42:19.840 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T11:42:19.840 INFO:teuthology.orchestra.run.trial028.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T11:42:19.840 INFO:teuthology.orchestra.run.trial028.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T11:42:19.840 INFO:teuthology.orchestra.run.trial028.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-25T11:42:19.840 INFO:teuthology.orchestra.run.trial028.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T11:42:19.840 INFO:teuthology.orchestra.run.trial028.stdout: libaio-devel-0.3.111-13.el9.x86_64 2026-02-25T11:42:19.840 INFO:teuthology.orchestra.run.trial028.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T11:42:19.840 INFO:teuthology.orchestra.run.trial028.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T11:42:19.840 INFO:teuthology.orchestra.run.trial028.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-25T11:42:19.840 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-devel-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.840 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-proxy2-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.840 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs2-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.841 INFO:teuthology.orchestra.run.trial028.stdout: libcephsqlite-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.841 INFO:teuthology.orchestra.run.trial028.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T11:42:19.841 INFO:teuthology.orchestra.run.trial028.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T11:42:19.841 INFO:teuthology.orchestra.run.trial028.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T11:42:19.841 INFO:teuthology.orchestra.run.trial028.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T11:42:19.841 INFO:teuthology.orchestra.run.trial028.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T11:42:19.841 INFO:teuthology.orchestra.run.trial028.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T11:42:19.841 INFO:teuthology.orchestra.run.trial028.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T11:42:19.841 INFO:teuthology.orchestra.run.trial028.stdout: librados-devel-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.841 INFO:teuthology.orchestra.run.trial028.stdout: libradosstriper1-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.841 INFO:teuthology.orchestra.run.trial028.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T11:42:19.841 INFO:teuthology.orchestra.run.trial028.stdout: librgw2-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.842 INFO:teuthology.orchestra.run.trial028.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T11:42:19.842 INFO:teuthology.orchestra.run.trial028.stdout: libtool-2.4.6-46.el9.x86_64 2026-02-25T11:42:19.842 INFO:teuthology.orchestra.run.trial028.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T11:42:19.842 INFO:teuthology.orchestra.run.trial028.stdout: libuuid-devel-2.37.4-25.el9.x86_64 2026-02-25T11:42:19.842 INFO:teuthology.orchestra.run.trial028.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T11:42:19.842 INFO:teuthology.orchestra.run.trial028.stdout: libzstd-devel-1.5.5-1.el9.x86_64 2026-02-25T11:42:19.842 INFO:teuthology.orchestra.run.trial028.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-25T11:42:19.842 INFO:teuthology.orchestra.run.trial028.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T11:42:19.842 INFO:teuthology.orchestra.run.trial028.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T11:42:19.842 INFO:teuthology.orchestra.run.trial028.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T11:42:19.842 INFO:teuthology.orchestra.run.trial028.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T11:42:19.842 INFO:teuthology.orchestra.run.trial028.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T11:42:19.842 INFO:teuthology.orchestra.run.trial028.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T11:42:19.843 INFO:teuthology.orchestra.run.trial028.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T11:42:19.843 INFO:teuthology.orchestra.run.trial028.stdout: openssl-devel-1:3.5.5-1.el9.x86_64 2026-02-25T11:42:19.843 INFO:teuthology.orchestra.run.trial028.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T11:42:19.843 INFO:teuthology.orchestra.run.trial028.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T11:42:19.843 INFO:teuthology.orchestra.run.trial028.stdout: perl-Thread-Queue-3.14-460.el9.noarch 2026-02-25T11:42:19.843 INFO:teuthology.orchestra.run.trial028.stdout: perl-threads-1:2.25-460.el9.x86_64 2026-02-25T11:42:19.843 INFO:teuthology.orchestra.run.trial028.stdout: perl-threads-shared-1.61-460.el9.x86_64 2026-02-25T11:42:19.843 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-13.23-1.el9.x86_64 2026-02-25T11:42:19.843 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-contrib-13.23-1.el9.x86_64 2026-02-25T11:42:19.843 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-private-libs-13.23-1.el9.x86_64 2026-02-25T11:42:19.843 INFO:teuthology.orchestra.run.trial028.stdout: postgresql-server-13.23-1.el9.x86_64 2026-02-25T11:42:19.844 INFO:teuthology.orchestra.run.trial028.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T11:42:19.844 INFO:teuthology.orchestra.run.trial028.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T11:42:19.844 INFO:teuthology.orchestra.run.trial028.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T11:42:19.844 INFO:teuthology.orchestra.run.trial028.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T11:42:19.844 INFO:teuthology.orchestra.run.trial028.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T11:42:19.844 INFO:teuthology.orchestra.run.trial028.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T11:42:19.844 INFO:teuthology.orchestra.run.trial028.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T11:42:19.844 INFO:teuthology.orchestra.run.trial028.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T11:42:19.844 INFO:teuthology.orchestra.run.trial028.stdout: python3-ceph-argparse-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.844 INFO:teuthology.orchestra.run.trial028.stdout: python3-ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.844 INFO:teuthology.orchestra.run.trial028.stdout: python3-cephfs-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.844 INFO:teuthology.orchestra.run.trial028.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T11:42:19.845 INFO:teuthology.orchestra.run.trial028.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T11:42:19.845 INFO:teuthology.orchestra.run.trial028.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T11:42:19.845 INFO:teuthology.orchestra.run.trial028.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T11:42:19.845 INFO:teuthology.orchestra.run.trial028.stdout: python3-cmd2-2.4.2-2.el9.noarch 2026-02-25T11:42:19.845 INFO:teuthology.orchestra.run.trial028.stdout: python3-colorama-0.4.6-3.el9.noarch 2026-02-25T11:42:19.845 INFO:teuthology.orchestra.run.trial028.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T11:42:19.845 INFO:teuthology.orchestra.run.trial028.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T11:42:19.845 INFO:teuthology.orchestra.run.trial028.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T11:42:19.845 INFO:teuthology.orchestra.run.trial028.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T11:42:19.845 INFO:teuthology.orchestra.run.trial028.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T11:42:19.845 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T11:42:19.845 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T11:42:19.846 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T11:42:19.846 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T11:42:19.846 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T11:42:19.846 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T11:42:19.846 INFO:teuthology.orchestra.run.trial028.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T11:42:19.846 INFO:teuthology.orchestra.run.trial028.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T11:42:19.846 INFO:teuthology.orchestra.run.trial028.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T11:42:19.846 INFO:teuthology.orchestra.run.trial028.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T11:42:19.846 INFO:teuthology.orchestra.run.trial028.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T11:42:19.846 INFO:teuthology.orchestra.run.trial028.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T11:42:19.846 INFO:teuthology.orchestra.run.trial028.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T11:42:19.846 INFO:teuthology.orchestra.run.trial028.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T11:42:19.846 INFO:teuthology.orchestra.run.trial028.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T11:42:19.847 INFO:teuthology.orchestra.run.trial028.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T11:42:19.847 INFO:teuthology.orchestra.run.trial028.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T11:42:19.847 INFO:teuthology.orchestra.run.trial028.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T11:42:19.847 INFO:teuthology.orchestra.run.trial028.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T11:42:19.847 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T11:42:19.847 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T11:42:19.847 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T11:42:19.847 INFO:teuthology.orchestra.run.trial028.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T11:42:19.847 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T11:42:19.847 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2026-02-25T11:42:19.847 INFO:teuthology.orchestra.run.trial028.stdout: python3-rados-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.847 INFO:teuthology.orchestra.run.trial028.stdout: python3-rbd-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.848 INFO:teuthology.orchestra.run.trial028.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T11:42:19.848 INFO:teuthology.orchestra.run.trial028.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T11:42:19.848 INFO:teuthology.orchestra.run.trial028.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T11:42:19.848 INFO:teuthology.orchestra.run.trial028.stdout: python3-rgw-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.848 INFO:teuthology.orchestra.run.trial028.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T11:42:19.848 INFO:teuthology.orchestra.run.trial028.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T11:42:19.848 INFO:teuthology.orchestra.run.trial028.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T11:42:19.848 INFO:teuthology.orchestra.run.trial028.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T11:42:19.848 INFO:teuthology.orchestra.run.trial028.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T11:42:19.848 INFO:teuthology.orchestra.run.trial028.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T11:42:19.848 INFO:teuthology.orchestra.run.trial028.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T11:42:19.848 INFO:teuthology.orchestra.run.trial028.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2026-02-25T11:42:19.849 INFO:teuthology.orchestra.run.trial028.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T11:42:19.849 INFO:teuthology.orchestra.run.trial028.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T11:42:19.849 INFO:teuthology.orchestra.run.trial028.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T11:42:19.849 INFO:teuthology.orchestra.run.trial028.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T11:42:19.849 INFO:teuthology.orchestra.run.trial028.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T11:42:19.849 INFO:teuthology.orchestra.run.trial028.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T11:42:19.849 INFO:teuthology.orchestra.run.trial028.stdout: rbd-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.849 INFO:teuthology.orchestra.run.trial028.stdout: rbd-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.849 INFO:teuthology.orchestra.run.trial028.stdout: rbd-nbd-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:19.849 INFO:teuthology.orchestra.run.trial028.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T11:42:19.849 INFO:teuthology.orchestra.run.trial028.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T11:42:19.849 INFO:teuthology.orchestra.run.trial028.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T11:42:19.849 INFO:teuthology.orchestra.run.trial028.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T11:42:19.850 INFO:teuthology.orchestra.run.trial028.stdout: uuid-1.6.2-55.el9.x86_64 2026-02-25T11:42:19.850 INFO:teuthology.orchestra.run.trial028.stdout: xfsprogs-devel-6.4.0-7.el9.x86_64 2026-02-25T11:42:19.850 INFO:teuthology.orchestra.run.trial028.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T11:42:19.850 INFO:teuthology.orchestra.run.trial028.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-25T11:42:19.850 INFO:teuthology.orchestra.run.trial028.stdout: zip-3.0-35.el9.x86_64 2026-02-25T11:42:19.850 INFO:teuthology.orchestra.run.trial028.stdout: zlib-devel-1.2.11-41.el9.x86_64 2026-02-25T11:42:19.850 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:19.850 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-25T11:42:20.081 DEBUG:teuthology.parallel:result is None 2026-02-25T11:42:20.464 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-test-2:20.3.0-5549.g92715cd3.el9.x86_64 157/181 2026-02-25T11:42:20.470 INFO:teuthology.orchestra.run.trial119.stdout: Installing : elfutils-libelf-devel-0.194-1.el9.x86_64 158/181 2026-02-25T11:42:20.580 INFO:teuthology.orchestra.run.trial119.stdout: Installing : cephfs-shell-2:20.3.0-5549.g92715cd3.el9.x86_64 159/181 2026-02-25T11:42:20.600 INFO:teuthology.orchestra.run.trial119.stdout: Installing : postgresql-contrib-13.23-1.el9.x86_64 160/181 2026-02-25T11:42:20.607 INFO:teuthology.orchestra.run.trial119.stdout: Installing : xfsprogs-devel-6.4.0-7.el9.x86_64 161/181 2026-02-25T11:42:20.621 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libcephfs-devel-2:20.3.0-5549.g92715cd3.el9.x86_ 162/181 2026-02-25T11:42:20.624 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libtool-2.4.6-46.el9.x86_64 163/181 2026-02-25T11:42:20.832 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: postgresql-server-13.23-1.el9.x86_64 164/181 2026-02-25T11:42:20.842 INFO:teuthology.orchestra.run.trial119.stdout: Installing : postgresql-server-13.23-1.el9.x86_64 164/181 2026-02-25T11:42:20.880 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: postgresql-server-13.23-1.el9.x86_64 164/181 2026-02-25T11:42:20.892 INFO:teuthology.orchestra.run.trial119.stdout: Installing : rbd-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 165/181 2026-02-25T11:42:20.898 INFO:teuthology.orchestra.run.trial119.stdout: Installing : rbd-nbd-2:20.3.0-5549.g92715cd3.el9.x86_64 166/181 2026-02-25T11:42:20.943 INFO:teuthology.orchestra.run.trial119.stdout: Installing : cephfs-top-2:20.3.0-5549.g92715cd3.el9.noarch 167/181 2026-02-25T11:42:21.456 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5549.g92715cd3.el9.noa 146/181 2026-02-25T11:42:21.471 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5549.g92715cd3.el9.n 147/181 2026-02-25T11:42:21.815 INFO:teuthology.orchestra.run.trial119.stdout: Installing : ceph-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 168/181 2026-02-25T11:42:21.850 INFO:teuthology.orchestra.run.trial119.stdout: Installing : openssl-devel-1:3.5.5-1.el9.x86_64 169/181 2026-02-25T11:42:21.866 INFO:teuthology.orchestra.run.trial119.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 170/181 2026-02-25T11:42:21.908 INFO:teuthology.orchestra.run.trial119.stdout: Installing : flex-2.6.4-9.el9.x86_64 171/181 2026-02-25T11:42:21.921 INFO:teuthology.orchestra.run.trial119.stdout: Installing : bison-3.7.4-5.el9.x86_64 172/181 2026-02-25T11:42:21.935 INFO:teuthology.orchestra.run.trial119.stdout: Cleanup : util-linux-2.37.4-21.el9.x86_64 173/181 2026-02-25T11:42:21.941 INFO:teuthology.orchestra.run.trial119.stdout: Cleanup : util-linux-core-2.37.4-21.el9.x86_64 174/181 2026-02-25T11:42:21.944 INFO:teuthology.orchestra.run.trial119.stdout: Cleanup : libfdisk-2.37.4-21.el9.x86_64 175/181 2026-02-25T11:42:21.948 INFO:teuthology.orchestra.run.trial119.stdout: Cleanup : libmount-2.37.4-21.el9.x86_64 176/181 2026-02-25T11:42:21.948 INFO:teuthology.orchestra.run.trial119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 177/181 2026-02-25T11:42:21.979 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 177/181 2026-02-25T11:42:21.979 INFO:teuthology.orchestra.run.trial119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 178/181 2026-02-25T11:42:21.986 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5549.g92715cd3.el9.n 147/181 2026-02-25T11:42:21.989 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5549.g927 148/181 2026-02-25T11:42:22.006 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 178/181 2026-02-25T11:42:22.008 INFO:teuthology.orchestra.run.trial119.stdout: Cleanup : libblkid-2.37.4-21.el9.x86_64 179/181 2026-02-25T11:42:22.010 INFO:teuthology.orchestra.run.trial119.stdout: Cleanup : libuuid-2.37.4-21.el9.x86_64 180/181 2026-02-25T11:42:22.012 INFO:teuthology.orchestra.run.trial119.stdout: Cleanup : libsmartcols-2.37.4-21.el9.x86_64 181/181 2026-02-25T11:42:22.027 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5549.g92715cd3.el9.x86 181/181 2026-02-25T11:42:22.061 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5549.g92715cd3.el9.x86 181/181 2026-02-25T11:42:22.084 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5549.g927 148/181 2026-02-25T11:42:22.133 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5549.g92715cd3.el 149/181 2026-02-25T11:42:22.136 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-mgr-2:20.3.0-5549.g92715cd3.el9.x86_64 150/181 2026-02-25T11:42:22.183 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5549.g92715cd3.el9.x86_64 150/181 2026-02-25T11:42:22.184 INFO:teuthology.orchestra.run.trial167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:22.184 INFO:teuthology.orchestra.run.trial167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T11:42:22.184 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T11:42:22.184 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T11:42:22.184 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:22.201 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-mgr-rook-2:20.3.0-5549.g92715cd3.el9.noarch 151/181 2026-02-25T11:42:22.222 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5549.g92715cd3.el9.noarch 151/181 2026-02-25T11:42:22.224 INFO:teuthology.orchestra.run.trial119.stdout: Running scriptlet: libsmartcols-2.37.4-21.el9.x86_64 181/181 2026-02-25T11:42:22.225 INFO:teuthology.orchestra.run.trial119.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2026-02-25T11:42:22.225 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:22.839 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-2:20.3.0-5549.g92715cd3.el9.x86_64 152/181 2026-02-25T11:42:22.844 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-radosgw-2:20.3.0-5549.g92715cd3.el9.x86_64 153/181 2026-02-25T11:42:22.893 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5549.g92715cd3.el9.x86_64 153/181 2026-02-25T11:42:22.893 INFO:teuthology.orchestra.run.trial167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:22.893 INFO:teuthology.orchestra.run.trial167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T11:42:22.893 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T11:42:22.893 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T11:42:22.894 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:22.906 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5549.g92715 154/181 2026-02-25T11:42:22.950 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5549.g92715 154/181 2026-02-25T11:42:22.951 INFO:teuthology.orchestra.run.trial167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:22.951 INFO:teuthology.orchestra.run.trial167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T11:42:22.951 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:22.967 INFO:teuthology.orchestra.run.trial167.stdout: Installing : cephfs-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 155/181 2026-02-25T11:42:23.014 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: cephfs-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 155/181 2026-02-25T11:42:23.014 INFO:teuthology.orchestra.run.trial167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:23.014 INFO:teuthology.orchestra.run.trial167.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2026-02-25T11:42:23.014 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2026-02-25T11:42:23.015 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2026-02-25T11:42:23.015 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:23.157 INFO:teuthology.orchestra.run.trial167.stdout: Installing : rbd-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 156/181 2026-02-25T11:42:23.204 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 156/181 2026-02-25T11:42:23.204 INFO:teuthology.orchestra.run.trial167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T11:42:23.204 INFO:teuthology.orchestra.run.trial167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T11:42:23.204 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T11:42:23.204 INFO:teuthology.orchestra.run.trial167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T11:42:23.205 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:23.246 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-2:20.3.0-5549.g92715cd3.el9.x86_64 1/181 2026-02-25T11:42:23.246 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-base-2:20.3.0-5549.g92715cd3.el9.x86_64 2/181 2026-02-25T11:42:23.246 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 3/181 2026-02-25T11:42:23.246 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 4/181 2026-02-25T11:42:23.246 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5549.g92715 5/181 2026-02-25T11:42:23.247 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-mds-2:20.3.0-5549.g92715cd3.el9.x86_64 6/181 2026-02-25T11:42:23.247 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-mgr-2:20.3.0-5549.g92715cd3.el9.x86_64 7/181 2026-02-25T11:42:23.247 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-mon-2:20.3.0-5549.g92715cd3.el9.x86_64 8/181 2026-02-25T11:42:23.247 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-osd-2:20.3.0-5549.g92715cd3.el9.x86_64 9/181 2026-02-25T11:42:23.247 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-osd-classic-2:20.3.0-5549.g92715cd3.el9.x86 10/181 2026-02-25T11:42:23.247 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5549.g92715cd3.el9.x86 11/181 2026-02-25T11:42:23.247 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-radosgw-2:20.3.0-5549.g92715cd3.el9.x86_64 12/181 2026-02-25T11:42:23.247 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-selinux-2:20.3.0-5549.g92715cd3.el9.x86_64 13/181 2026-02-25T11:42:23.247 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-test-2:20.3.0-5549.g92715cd3.el9.x86_64 14/181 2026-02-25T11:42:23.248 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : cephfs-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 15/181 2026-02-25T11:42:23.248 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : cephfs-shell-2:20.3.0-5549.g92715cd3.el9.x86_64 16/181 2026-02-25T11:42:23.248 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libcephfs-devel-2:20.3.0-5549.g92715cd3.el9.x86_ 17/181 2026-02-25T11:42:23.248 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5549.g92715cd3.el9.x86 18/181 2026-02-25T11:42:23.248 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libcephfs2-2:20.3.0-5549.g92715cd3.el9.x86_64 19/181 2026-02-25T11:42:23.248 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libcephsqlite-2:20.3.0-5549.g92715cd3.el9.x86_64 20/181 2026-02-25T11:42:23.248 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : librados-devel-2:20.3.0-5549.g92715cd3.el9.x86_6 21/181 2026-02-25T11:42:23.248 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libradosstriper1-2:20.3.0-5549.g92715cd3.el9.x86 22/181 2026-02-25T11:42:23.248 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : librgw2-2:20.3.0-5549.g92715cd3.el9.x86_64 23/181 2026-02-25T11:42:23.248 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5549.g92715cd3.el 24/181 2026-02-25T11:42:23.248 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-ceph-common-2:20.3.0-5549.g92715cd3.el9. 25/181 2026-02-25T11:42:23.248 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-cephfs-2:20.3.0-5549.g92715cd3.el9.x86_6 26/181 2026-02-25T11:42:23.248 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-rados-2:20.3.0-5549.g92715cd3.el9.x86_64 27/181 2026-02-25T11:42:23.249 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-rbd-2:20.3.0-5549.g92715cd3.el9.x86_64 28/181 2026-02-25T11:42:23.249 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-rgw-2:20.3.0-5549.g92715cd3.el9.x86_64 29/181 2026-02-25T11:42:23.249 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : rbd-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 30/181 2026-02-25T11:42:23.249 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : rbd-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 31/181 2026-02-25T11:42:23.249 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : rbd-nbd-2:20.3.0-5549.g92715cd3.el9.x86_64 32/181 2026-02-25T11:42:23.249 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5549.g92715cd3. 33/181 2026-02-25T11:42:23.249 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5549.g92715cd3.el9.noa 34/181 2026-02-25T11:42:23.249 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5549.g92715cd3.el9.n 35/181 2026-02-25T11:42:23.249 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5549.g927 36/181 2026-02-25T11:42:23.249 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5549.g92715cd3.el 37/181 2026-02-25T11:42:23.249 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5549.g92715cd3.el9.noarch 38/181 2026-02-25T11:42:23.249 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5549.g92715cd3.e 39/181 2026-02-25T11:42:23.249 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ceph-volume-2:20.3.0-5549.g92715cd3.el9.noarch 40/181 2026-02-25T11:42:23.250 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 41/181 2026-02-25T11:42:23.250 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : cephfs-top-2:20.3.0-5549.g92715cd3.el9.noarch 42/181 2026-02-25T11:42:23.250 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 43/181 2026-02-25T11:42:23.250 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 44/181 2026-02-25T11:42:23.250 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 45/181 2026-02-25T11:42:23.250 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 46/181 2026-02-25T11:42:23.250 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 47/181 2026-02-25T11:42:23.250 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 48/181 2026-02-25T11:42:23.250 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 49/181 2026-02-25T11:42:23.250 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 50/181 2026-02-25T11:42:23.250 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 51/181 2026-02-25T11:42:23.250 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 52/181 2026-02-25T11:42:23.251 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 53/181 2026-02-25T11:42:23.251 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 54/181 2026-02-25T11:42:23.251 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 55/181 2026-02-25T11:42:23.251 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 56/181 2026-02-25T11:42:23.251 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 57/181 2026-02-25T11:42:23.251 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 58/181 2026-02-25T11:42:23.251 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 59/181 2026-02-25T11:42:23.251 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : unzip-6.0-59.el9.x86_64 60/181 2026-02-25T11:42:23.251 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : zip-3.0-35.el9.x86_64 61/181 2026-02-25T11:42:23.251 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : automake-1.16.2-8.el9.noarch 62/181 2026-02-25T11:42:23.251 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : bison-3.7.4-5.el9.x86_64 63/181 2026-02-25T11:42:23.251 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 64/181 2026-02-25T11:42:23.251 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : elfutils-libelf-devel-0.194-1.el9.x86_64 65/181 2026-02-25T11:42:23.252 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : flex-2.6.4-9.el9.x86_64 66/181 2026-02-25T11:42:23.252 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 67/181 2026-02-25T11:42:23.252 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 68/181 2026-02-25T11:42:23.252 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 69/181 2026-02-25T11:42:23.252 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 70/181 2026-02-25T11:42:23.252 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 71/181 2026-02-25T11:42:23.252 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 72/181 2026-02-25T11:42:23.252 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 73/181 2026-02-25T11:42:23.252 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 74/181 2026-02-25T11:42:23.252 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 75/181 2026-02-25T11:42:23.252 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 76/181 2026-02-25T11:42:23.252 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libuuid-devel-2.37.4-25.el9.x86_64 77/181 2026-02-25T11:42:23.252 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 78/181 2026-02-25T11:42:23.253 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libzstd-devel-1.5.5-1.el9.x86_64 79/181 2026-02-25T11:42:23.253 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 80/181 2026-02-25T11:42:23.253 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : lua-5.4.4-4.el9.x86_64 81/181 2026-02-25T11:42:23.253 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 82/181 2026-02-25T11:42:23.253 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 83/181 2026-02-25T11:42:23.253 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : openssl-devel-1:3.5.5-1.el9.x86_64 84/181 2026-02-25T11:42:23.253 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : perl-Thread-Queue-3.14-460.el9.noarch 85/181 2026-02-25T11:42:23.253 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : perl-threads-1:2.25-460.el9.x86_64 86/181 2026-02-25T11:42:23.253 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : perl-threads-shared-1.61-460.el9.x86_64 87/181 2026-02-25T11:42:23.253 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : postgresql-13.23-1.el9.x86_64 88/181 2026-02-25T11:42:23.253 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : postgresql-contrib-13.23-1.el9.x86_64 89/181 2026-02-25T11:42:23.253 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : postgresql-private-libs-13.23-1.el9.x86_64 90/181 2026-02-25T11:42:23.253 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : postgresql-server-13.23-1.el9.x86_64 91/181 2026-02-25T11:42:23.254 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 92/181 2026-02-25T11:42:23.254 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 93/181 2026-02-25T11:42:23.254 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 94/181 2026-02-25T11:42:23.254 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 95/181 2026-02-25T11:42:23.254 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 96/181 2026-02-25T11:42:23.254 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 97/181 2026-02-25T11:42:23.254 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 98/181 2026-02-25T11:42:23.254 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 99/181 2026-02-25T11:42:23.254 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 100/181 2026-02-25T11:42:23.254 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 101/181 2026-02-25T11:42:23.254 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 102/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 103/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 104/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 105/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 106/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 107/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 108/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 109/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 110/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 111/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 112/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 113/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 114/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : xfsprogs-devel-6.4.0-7.el9.x86_64 115/181 2026-02-25T11:42:23.255 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 116/181 2026-02-25T11:42:23.256 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 117/181 2026-02-25T11:42:23.256 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 118/181 2026-02-25T11:42:23.256 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 119/181 2026-02-25T11:42:23.256 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 120/181 2026-02-25T11:42:23.256 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 121/181 2026-02-25T11:42:23.256 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 122/181 2026-02-25T11:42:23.256 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 123/181 2026-02-25T11:42:23.256 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 124/181 2026-02-25T11:42:23.256 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 125/181 2026-02-25T11:42:23.256 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 126/181 2026-02-25T11:42:23.256 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 127/181 2026-02-25T11:42:23.256 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 128/181 2026-02-25T11:42:23.256 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 129/181 2026-02-25T11:42:23.257 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 130/181 2026-02-25T11:42:23.257 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 131/181 2026-02-25T11:42:23.257 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 132/181 2026-02-25T11:42:23.257 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 133/181 2026-02-25T11:42:23.257 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 134/181 2026-02-25T11:42:23.257 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 135/181 2026-02-25T11:42:23.257 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 136/181 2026-02-25T11:42:23.257 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 137/181 2026-02-25T11:42:23.257 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 138/181 2026-02-25T11:42:23.257 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 139/181 2026-02-25T11:42:23.258 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 140/181 2026-02-25T11:42:23.258 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 141/181 2026-02-25T11:42:23.258 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 142/181 2026-02-25T11:42:23.258 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 143/181 2026-02-25T11:42:23.258 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 144/181 2026-02-25T11:42:23.258 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 145/181 2026-02-25T11:42:23.258 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 146/181 2026-02-25T11:42:23.258 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 147/181 2026-02-25T11:42:23.258 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 148/181 2026-02-25T11:42:23.258 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 149/181 2026-02-25T11:42:23.258 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 150/181 2026-02-25T11:42:23.258 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 151/181 2026-02-25T11:42:23.259 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 152/181 2026-02-25T11:42:23.259 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 153/181 2026-02-25T11:42:23.259 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 154/181 2026-02-25T11:42:23.259 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 155/181 2026-02-25T11:42:23.259 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 156/181 2026-02-25T11:42:23.259 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 157/181 2026-02-25T11:42:23.259 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 158/181 2026-02-25T11:42:23.259 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 159/181 2026-02-25T11:42:23.259 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 160/181 2026-02-25T11:42:23.259 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : re2-1:20211101-20.el9.x86_64 161/181 2026-02-25T11:42:23.259 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 162/181 2026-02-25T11:42:23.259 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 163/181 2026-02-25T11:42:23.259 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : librados2-2:20.3.0-5549.g92715cd3.el9.x86_64 164/181 2026-02-25T11:42:23.260 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 165/181 2026-02-25T11:42:23.260 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : librbd1-2:20.3.0-5549.g92715cd3.el9.x86_64 166/181 2026-02-25T11:42:23.260 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 167/181 2026-02-25T11:42:23.260 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libblkid-2.37.4-25.el9.x86_64 168/181 2026-02-25T11:42:23.260 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libblkid-2.37.4-21.el9.x86_64 169/181 2026-02-25T11:42:23.260 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libfdisk-2.37.4-25.el9.x86_64 170/181 2026-02-25T11:42:23.260 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libfdisk-2.37.4-21.el9.x86_64 171/181 2026-02-25T11:42:23.260 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libmount-2.37.4-25.el9.x86_64 172/181 2026-02-25T11:42:23.260 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libmount-2.37.4-21.el9.x86_64 173/181 2026-02-25T11:42:23.260 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libsmartcols-2.37.4-25.el9.x86_64 174/181 2026-02-25T11:42:23.260 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libsmartcols-2.37.4-21.el9.x86_64 175/181 2026-02-25T11:42:23.260 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libuuid-2.37.4-25.el9.x86_64 176/181 2026-02-25T11:42:23.260 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : libuuid-2.37.4-21.el9.x86_64 177/181 2026-02-25T11:42:23.261 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : util-linux-2.37.4-25.el9.x86_64 178/181 2026-02-25T11:42:23.261 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : util-linux-2.37.4-21.el9.x86_64 179/181 2026-02-25T11:42:23.261 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : util-linux-core-2.37.4-25.el9.x86_64 180/181 2026-02-25T11:42:23.348 INFO:teuthology.orchestra.run.trial119.stdout: Verifying : util-linux-core-2.37.4-21.el9.x86_64 181/181 2026-02-25T11:42:23.348 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:23.348 INFO:teuthology.orchestra.run.trial119.stdout:Upgraded: 2026-02-25T11:42:23.348 INFO:teuthology.orchestra.run.trial119.stdout: libblkid-2.37.4-25.el9.x86_64 2026-02-25T11:42:23.348 INFO:teuthology.orchestra.run.trial119.stdout: libfdisk-2.37.4-25.el9.x86_64 2026-02-25T11:42:23.348 INFO:teuthology.orchestra.run.trial119.stdout: libmount-2.37.4-25.el9.x86_64 2026-02-25T11:42:23.349 INFO:teuthology.orchestra.run.trial119.stdout: librados2-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.349 INFO:teuthology.orchestra.run.trial119.stdout: librbd1-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.349 INFO:teuthology.orchestra.run.trial119.stdout: libsmartcols-2.37.4-25.el9.x86_64 2026-02-25T11:42:23.349 INFO:teuthology.orchestra.run.trial119.stdout: libuuid-2.37.4-25.el9.x86_64 2026-02-25T11:42:23.349 INFO:teuthology.orchestra.run.trial119.stdout: util-linux-2.37.4-25.el9.x86_64 2026-02-25T11:42:23.349 INFO:teuthology.orchestra.run.trial119.stdout: util-linux-core-2.37.4-25.el9.x86_64 2026-02-25T11:42:23.349 INFO:teuthology.orchestra.run.trial119.stdout:Installed: 2026-02-25T11:42:23.349 INFO:teuthology.orchestra.run.trial119.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T11:42:23.349 INFO:teuthology.orchestra.run.trial119.stdout: automake-1.16.2-8.el9.noarch 2026-02-25T11:42:23.349 INFO:teuthology.orchestra.run.trial119.stdout: bison-3.7.4-5.el9.x86_64 2026-02-25T11:42:23.350 INFO:teuthology.orchestra.run.trial119.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T11:42:23.350 INFO:teuthology.orchestra.run.trial119.stdout: ceph-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.350 INFO:teuthology.orchestra.run.trial119.stdout: ceph-base-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.350 INFO:teuthology.orchestra.run.trial119.stdout: ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.350 INFO:teuthology.orchestra.run.trial119.stdout: ceph-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.350 INFO:teuthology.orchestra.run.trial119.stdout: ceph-grafana-dashboards-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:23.350 INFO:teuthology.orchestra.run.trial119.stdout: ceph-immutable-object-cache-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.350 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mds-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.350 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mgr-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.350 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mgr-cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:23.351 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mgr-dashboard-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:23.351 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:23.351 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mgr-modules-core-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:23.351 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mgr-rook-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:23.351 INFO:teuthology.orchestra.run.trial119.stdout: ceph-mon-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.351 INFO:teuthology.orchestra.run.trial119.stdout: ceph-osd-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.351 INFO:teuthology.orchestra.run.trial119.stdout: ceph-osd-classic-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.351 INFO:teuthology.orchestra.run.trial119.stdout: ceph-osd-crimson-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.351 INFO:teuthology.orchestra.run.trial119.stdout: ceph-prometheus-alerts-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:23.352 INFO:teuthology.orchestra.run.trial119.stdout: ceph-radosgw-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.352 INFO:teuthology.orchestra.run.trial119.stdout: ceph-selinux-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.352 INFO:teuthology.orchestra.run.trial119.stdout: ceph-test-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.352 INFO:teuthology.orchestra.run.trial119.stdout: ceph-volume-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:23.352 INFO:teuthology.orchestra.run.trial119.stdout: cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:23.352 INFO:teuthology.orchestra.run.trial119.stdout: cephfs-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.352 INFO:teuthology.orchestra.run.trial119.stdout: cephfs-shell-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.352 INFO:teuthology.orchestra.run.trial119.stdout: cephfs-top-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:23.352 INFO:teuthology.orchestra.run.trial119.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T11:42:23.352 INFO:teuthology.orchestra.run.trial119.stdout: elfutils-libelf-devel-0.194-1.el9.x86_64 2026-02-25T11:42:23.352 INFO:teuthology.orchestra.run.trial119.stdout: flex-2.6.4-9.el9.x86_64 2026-02-25T11:42:23.352 INFO:teuthology.orchestra.run.trial119.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T11:42:23.352 INFO:teuthology.orchestra.run.trial119.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T11:42:23.353 INFO:teuthology.orchestra.run.trial119.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T11:42:23.353 INFO:teuthology.orchestra.run.trial119.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T11:42:23.353 INFO:teuthology.orchestra.run.trial119.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T11:42:23.353 INFO:teuthology.orchestra.run.trial119.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-25T11:42:23.353 INFO:teuthology.orchestra.run.trial119.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T11:42:23.353 INFO:teuthology.orchestra.run.trial119.stdout: libaio-devel-0.3.111-13.el9.x86_64 2026-02-25T11:42:23.353 INFO:teuthology.orchestra.run.trial119.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T11:42:23.353 INFO:teuthology.orchestra.run.trial119.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T11:42:23.353 INFO:teuthology.orchestra.run.trial119.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-25T11:42:23.353 INFO:teuthology.orchestra.run.trial119.stdout: libcephfs-devel-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.353 INFO:teuthology.orchestra.run.trial119.stdout: libcephfs-proxy2-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.353 INFO:teuthology.orchestra.run.trial119.stdout: libcephfs2-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.354 INFO:teuthology.orchestra.run.trial119.stdout: libcephsqlite-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.354 INFO:teuthology.orchestra.run.trial119.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T11:42:23.354 INFO:teuthology.orchestra.run.trial119.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T11:42:23.354 INFO:teuthology.orchestra.run.trial119.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T11:42:23.354 INFO:teuthology.orchestra.run.trial119.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T11:42:23.354 INFO:teuthology.orchestra.run.trial119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T11:42:23.354 INFO:teuthology.orchestra.run.trial119.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T11:42:23.354 INFO:teuthology.orchestra.run.trial119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T11:42:23.354 INFO:teuthology.orchestra.run.trial119.stdout: librados-devel-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.354 INFO:teuthology.orchestra.run.trial119.stdout: libradosstriper1-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.354 INFO:teuthology.orchestra.run.trial119.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T11:42:23.354 INFO:teuthology.orchestra.run.trial119.stdout: librgw2-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.354 INFO:teuthology.orchestra.run.trial119.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T11:42:23.355 INFO:teuthology.orchestra.run.trial119.stdout: libtool-2.4.6-46.el9.x86_64 2026-02-25T11:42:23.355 INFO:teuthology.orchestra.run.trial119.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T11:42:23.355 INFO:teuthology.orchestra.run.trial119.stdout: libuuid-devel-2.37.4-25.el9.x86_64 2026-02-25T11:42:23.355 INFO:teuthology.orchestra.run.trial119.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T11:42:23.355 INFO:teuthology.orchestra.run.trial119.stdout: libzstd-devel-1.5.5-1.el9.x86_64 2026-02-25T11:42:23.355 INFO:teuthology.orchestra.run.trial119.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-25T11:42:23.355 INFO:teuthology.orchestra.run.trial119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T11:42:23.355 INFO:teuthology.orchestra.run.trial119.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T11:42:23.355 INFO:teuthology.orchestra.run.trial119.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T11:42:23.355 INFO:teuthology.orchestra.run.trial119.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T11:42:23.355 INFO:teuthology.orchestra.run.trial119.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T11:42:23.355 INFO:teuthology.orchestra.run.trial119.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T11:42:23.355 INFO:teuthology.orchestra.run.trial119.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T11:42:23.356 INFO:teuthology.orchestra.run.trial119.stdout: openssl-devel-1:3.5.5-1.el9.x86_64 2026-02-25T11:42:23.356 INFO:teuthology.orchestra.run.trial119.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T11:42:23.356 INFO:teuthology.orchestra.run.trial119.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T11:42:23.356 INFO:teuthology.orchestra.run.trial119.stdout: perl-Thread-Queue-3.14-460.el9.noarch 2026-02-25T11:42:23.356 INFO:teuthology.orchestra.run.trial119.stdout: perl-threads-1:2.25-460.el9.x86_64 2026-02-25T11:42:23.356 INFO:teuthology.orchestra.run.trial119.stdout: perl-threads-shared-1.61-460.el9.x86_64 2026-02-25T11:42:23.356 INFO:teuthology.orchestra.run.trial119.stdout: postgresql-13.23-1.el9.x86_64 2026-02-25T11:42:23.356 INFO:teuthology.orchestra.run.trial119.stdout: postgresql-contrib-13.23-1.el9.x86_64 2026-02-25T11:42:23.356 INFO:teuthology.orchestra.run.trial119.stdout: postgresql-private-libs-13.23-1.el9.x86_64 2026-02-25T11:42:23.356 INFO:teuthology.orchestra.run.trial119.stdout: postgresql-server-13.23-1.el9.x86_64 2026-02-25T11:42:23.356 INFO:teuthology.orchestra.run.trial119.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T11:42:23.356 INFO:teuthology.orchestra.run.trial119.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T11:42:23.356 INFO:teuthology.orchestra.run.trial119.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T11:42:23.357 INFO:teuthology.orchestra.run.trial119.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T11:42:23.357 INFO:teuthology.orchestra.run.trial119.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T11:42:23.357 INFO:teuthology.orchestra.run.trial119.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T11:42:23.357 INFO:teuthology.orchestra.run.trial119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T11:42:23.357 INFO:teuthology.orchestra.run.trial119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T11:42:23.357 INFO:teuthology.orchestra.run.trial119.stdout: python3-ceph-argparse-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.357 INFO:teuthology.orchestra.run.trial119.stdout: python3-ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.357 INFO:teuthology.orchestra.run.trial119.stdout: python3-cephfs-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.357 INFO:teuthology.orchestra.run.trial119.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T11:42:23.357 INFO:teuthology.orchestra.run.trial119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T11:42:23.357 INFO:teuthology.orchestra.run.trial119.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T11:42:23.357 INFO:teuthology.orchestra.run.trial119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T11:42:23.358 INFO:teuthology.orchestra.run.trial119.stdout: python3-cmd2-2.4.2-2.el9.noarch 2026-02-25T11:42:23.358 INFO:teuthology.orchestra.run.trial119.stdout: python3-colorama-0.4.6-3.el9.noarch 2026-02-25T11:42:23.358 INFO:teuthology.orchestra.run.trial119.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T11:42:23.358 INFO:teuthology.orchestra.run.trial119.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T11:42:23.358 INFO:teuthology.orchestra.run.trial119.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T11:42:23.358 INFO:teuthology.orchestra.run.trial119.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T11:42:23.358 INFO:teuthology.orchestra.run.trial119.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T11:42:23.358 INFO:teuthology.orchestra.run.trial119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T11:42:23.358 INFO:teuthology.orchestra.run.trial119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T11:42:23.358 INFO:teuthology.orchestra.run.trial119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T11:42:23.358 INFO:teuthology.orchestra.run.trial119.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T11:42:23.358 INFO:teuthology.orchestra.run.trial119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T11:42:23.359 INFO:teuthology.orchestra.run.trial119.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T11:42:23.359 INFO:teuthology.orchestra.run.trial119.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T11:42:23.359 INFO:teuthology.orchestra.run.trial119.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T11:42:23.359 INFO:teuthology.orchestra.run.trial119.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T11:42:23.359 INFO:teuthology.orchestra.run.trial119.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T11:42:23.359 INFO:teuthology.orchestra.run.trial119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T11:42:23.359 INFO:teuthology.orchestra.run.trial119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T11:42:23.359 INFO:teuthology.orchestra.run.trial119.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T11:42:23.359 INFO:teuthology.orchestra.run.trial119.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T11:42:23.359 INFO:teuthology.orchestra.run.trial119.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T11:42:23.359 INFO:teuthology.orchestra.run.trial119.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T11:42:23.359 INFO:teuthology.orchestra.run.trial119.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T11:42:23.359 INFO:teuthology.orchestra.run.trial119.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T11:42:23.360 INFO:teuthology.orchestra.run.trial119.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T11:42:23.360 INFO:teuthology.orchestra.run.trial119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T11:42:23.360 INFO:teuthology.orchestra.run.trial119.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T11:42:23.360 INFO:teuthology.orchestra.run.trial119.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T11:42:23.360 INFO:teuthology.orchestra.run.trial119.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T11:42:23.360 INFO:teuthology.orchestra.run.trial119.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T11:42:23.360 INFO:teuthology.orchestra.run.trial119.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2026-02-25T11:42:23.360 INFO:teuthology.orchestra.run.trial119.stdout: python3-rados-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.360 INFO:teuthology.orchestra.run.trial119.stdout: python3-rbd-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.360 INFO:teuthology.orchestra.run.trial119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T11:42:23.361 INFO:teuthology.orchestra.run.trial119.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T11:42:23.361 INFO:teuthology.orchestra.run.trial119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T11:42:23.361 INFO:teuthology.orchestra.run.trial119.stdout: python3-rgw-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.361 INFO:teuthology.orchestra.run.trial119.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T11:42:23.361 INFO:teuthology.orchestra.run.trial119.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T11:42:23.361 INFO:teuthology.orchestra.run.trial119.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T11:42:23.361 INFO:teuthology.orchestra.run.trial119.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T11:42:23.361 INFO:teuthology.orchestra.run.trial119.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T11:42:23.361 INFO:teuthology.orchestra.run.trial119.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T11:42:23.362 INFO:teuthology.orchestra.run.trial119.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T11:42:23.362 INFO:teuthology.orchestra.run.trial119.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2026-02-25T11:42:23.362 INFO:teuthology.orchestra.run.trial119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T11:42:23.362 INFO:teuthology.orchestra.run.trial119.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T11:42:23.362 INFO:teuthology.orchestra.run.trial119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T11:42:23.362 INFO:teuthology.orchestra.run.trial119.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T11:42:23.362 INFO:teuthology.orchestra.run.trial119.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T11:42:23.362 INFO:teuthology.orchestra.run.trial119.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T11:42:23.362 INFO:teuthology.orchestra.run.trial119.stdout: rbd-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.362 INFO:teuthology.orchestra.run.trial119.stdout: rbd-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.362 INFO:teuthology.orchestra.run.trial119.stdout: rbd-nbd-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:23.362 INFO:teuthology.orchestra.run.trial119.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T11:42:23.363 INFO:teuthology.orchestra.run.trial119.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T11:42:23.363 INFO:teuthology.orchestra.run.trial119.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T11:42:23.363 INFO:teuthology.orchestra.run.trial119.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T11:42:23.363 INFO:teuthology.orchestra.run.trial119.stdout: uuid-1.6.2-55.el9.x86_64 2026-02-25T11:42:23.363 INFO:teuthology.orchestra.run.trial119.stdout: xfsprogs-devel-6.4.0-7.el9.x86_64 2026-02-25T11:42:23.363 INFO:teuthology.orchestra.run.trial119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T11:42:23.363 INFO:teuthology.orchestra.run.trial119.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-25T11:42:23.363 INFO:teuthology.orchestra.run.trial119.stdout: zip-3.0-35.el9.x86_64 2026-02-25T11:42:23.363 INFO:teuthology.orchestra.run.trial119.stdout: zlib-devel-1.2.11-41.el9.x86_64 2026-02-25T11:42:23.363 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:42:23.363 INFO:teuthology.orchestra.run.trial119.stdout:Complete! 2026-02-25T11:42:23.590 DEBUG:teuthology.parallel:result is None 2026-02-25T11:42:26.855 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-test-2:20.3.0-5549.g92715cd3.el9.x86_64 157/181 2026-02-25T11:42:26.861 INFO:teuthology.orchestra.run.trial167.stdout: Installing : elfutils-libelf-devel-0.194-1.el9.x86_64 158/181 2026-02-25T11:42:26.969 INFO:teuthology.orchestra.run.trial167.stdout: Installing : cephfs-shell-2:20.3.0-5549.g92715cd3.el9.x86_64 159/181 2026-02-25T11:42:26.989 INFO:teuthology.orchestra.run.trial167.stdout: Installing : postgresql-contrib-13.23-1.el9.x86_64 160/181 2026-02-25T11:42:26.995 INFO:teuthology.orchestra.run.trial167.stdout: Installing : xfsprogs-devel-6.4.0-7.el9.x86_64 161/181 2026-02-25T11:42:27.010 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libcephfs-devel-2:20.3.0-5549.g92715cd3.el9.x86_ 162/181 2026-02-25T11:42:27.012 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libtool-2.4.6-46.el9.x86_64 163/181 2026-02-25T11:42:27.223 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: postgresql-server-13.23-1.el9.x86_64 164/181 2026-02-25T11:42:27.233 INFO:teuthology.orchestra.run.trial167.stdout: Installing : postgresql-server-13.23-1.el9.x86_64 164/181 2026-02-25T11:42:27.272 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: postgresql-server-13.23-1.el9.x86_64 164/181 2026-02-25T11:42:27.284 INFO:teuthology.orchestra.run.trial167.stdout: Installing : rbd-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 165/181 2026-02-25T11:42:27.290 INFO:teuthology.orchestra.run.trial167.stdout: Installing : rbd-nbd-2:20.3.0-5549.g92715cd3.el9.x86_64 166/181 2026-02-25T11:42:27.337 INFO:teuthology.orchestra.run.trial167.stdout: Installing : cephfs-top-2:20.3.0-5549.g92715cd3.el9.noarch 167/181 2026-02-25T11:42:28.205 INFO:teuthology.orchestra.run.trial167.stdout: Installing : ceph-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 168/181 2026-02-25T11:42:28.241 INFO:teuthology.orchestra.run.trial167.stdout: Installing : openssl-devel-1:3.5.5-1.el9.x86_64 169/181 2026-02-25T11:42:28.256 INFO:teuthology.orchestra.run.trial167.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 170/181 2026-02-25T11:42:28.299 INFO:teuthology.orchestra.run.trial167.stdout: Installing : flex-2.6.4-9.el9.x86_64 171/181 2026-02-25T11:42:28.312 INFO:teuthology.orchestra.run.trial167.stdout: Installing : bison-3.7.4-5.el9.x86_64 172/181 2026-02-25T11:42:28.327 INFO:teuthology.orchestra.run.trial167.stdout: Cleanup : util-linux-2.37.4-21.el9.x86_64 173/181 2026-02-25T11:42:28.333 INFO:teuthology.orchestra.run.trial167.stdout: Cleanup : util-linux-core-2.37.4-21.el9.x86_64 174/181 2026-02-25T11:42:28.336 INFO:teuthology.orchestra.run.trial167.stdout: Cleanup : libfdisk-2.37.4-21.el9.x86_64 175/181 2026-02-25T11:42:28.340 INFO:teuthology.orchestra.run.trial167.stdout: Cleanup : libmount-2.37.4-21.el9.x86_64 176/181 2026-02-25T11:42:28.340 INFO:teuthology.orchestra.run.trial167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 177/181 2026-02-25T11:42:28.371 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 177/181 2026-02-25T11:42:28.371 INFO:teuthology.orchestra.run.trial167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 178/181 2026-02-25T11:42:28.398 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 178/181 2026-02-25T11:42:28.401 INFO:teuthology.orchestra.run.trial167.stdout: Cleanup : libblkid-2.37.4-21.el9.x86_64 179/181 2026-02-25T11:42:28.403 INFO:teuthology.orchestra.run.trial167.stdout: Cleanup : libuuid-2.37.4-21.el9.x86_64 180/181 2026-02-25T11:42:28.405 INFO:teuthology.orchestra.run.trial167.stdout: Cleanup : libsmartcols-2.37.4-21.el9.x86_64 181/181 2026-02-25T11:42:28.420 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5549.g92715cd3.el9.x86 181/181 2026-02-25T11:42:28.455 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5549.g92715cd3.el9.x86 181/181 2026-02-25T11:42:28.617 INFO:teuthology.orchestra.run.trial167.stdout: Running scriptlet: libsmartcols-2.37.4-21.el9.x86_64 181/181 2026-02-25T11:42:28.617 INFO:teuthology.orchestra.run.trial167.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2026-02-25T11:42:28.617 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-2:20.3.0-5549.g92715cd3.el9.x86_64 1/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-base-2:20.3.0-5549.g92715cd3.el9.x86_64 2/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 3/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 4/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5549.g92715 5/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-mds-2:20.3.0-5549.g92715cd3.el9.x86_64 6/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-mgr-2:20.3.0-5549.g92715cd3.el9.x86_64 7/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-mon-2:20.3.0-5549.g92715cd3.el9.x86_64 8/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-osd-2:20.3.0-5549.g92715cd3.el9.x86_64 9/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-osd-classic-2:20.3.0-5549.g92715cd3.el9.x86 10/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5549.g92715cd3.el9.x86 11/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-radosgw-2:20.3.0-5549.g92715cd3.el9.x86_64 12/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-selinux-2:20.3.0-5549.g92715cd3.el9.x86_64 13/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-test-2:20.3.0-5549.g92715cd3.el9.x86_64 14/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : cephfs-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 15/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : cephfs-shell-2:20.3.0-5549.g92715cd3.el9.x86_64 16/181 2026-02-25T11:42:29.653 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libcephfs-devel-2:20.3.0-5549.g92715cd3.el9.x86_ 17/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5549.g92715cd3.el9.x86 18/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libcephfs2-2:20.3.0-5549.g92715cd3.el9.x86_64 19/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libcephsqlite-2:20.3.0-5549.g92715cd3.el9.x86_64 20/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : librados-devel-2:20.3.0-5549.g92715cd3.el9.x86_6 21/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libradosstriper1-2:20.3.0-5549.g92715cd3.el9.x86 22/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : librgw2-2:20.3.0-5549.g92715cd3.el9.x86_64 23/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5549.g92715cd3.el 24/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-ceph-common-2:20.3.0-5549.g92715cd3.el9. 25/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-cephfs-2:20.3.0-5549.g92715cd3.el9.x86_6 26/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-rados-2:20.3.0-5549.g92715cd3.el9.x86_64 27/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-rbd-2:20.3.0-5549.g92715cd3.el9.x86_64 28/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-rgw-2:20.3.0-5549.g92715cd3.el9.x86_64 29/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : rbd-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 30/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : rbd-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 31/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : rbd-nbd-2:20.3.0-5549.g92715cd3.el9.x86_64 32/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5549.g92715cd3. 33/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5549.g92715cd3.el9.noa 34/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5549.g92715cd3.el9.n 35/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5549.g927 36/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5549.g92715cd3.el 37/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5549.g92715cd3.el9.noarch 38/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5549.g92715cd3.e 39/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ceph-volume-2:20.3.0-5549.g92715cd3.el9.noarch 40/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 41/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : cephfs-top-2:20.3.0-5549.g92715cd3.el9.noarch 42/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 43/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 44/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 45/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 46/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 47/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 48/181 2026-02-25T11:42:29.654 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 49/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 50/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 51/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 52/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 53/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 54/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 55/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 56/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 57/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 58/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 59/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : unzip-6.0-59.el9.x86_64 60/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : zip-3.0-35.el9.x86_64 61/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : automake-1.16.2-8.el9.noarch 62/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : bison-3.7.4-5.el9.x86_64 63/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 64/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : elfutils-libelf-devel-0.194-1.el9.x86_64 65/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : flex-2.6.4-9.el9.x86_64 66/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 67/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 68/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 69/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 70/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 71/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 72/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 73/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 74/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 75/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 76/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libuuid-devel-2.37.4-25.el9.x86_64 77/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 78/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libzstd-devel-1.5.5-1.el9.x86_64 79/181 2026-02-25T11:42:29.655 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 80/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 81/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 82/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 83/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : openssl-devel-1:3.5.5-1.el9.x86_64 84/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : perl-Thread-Queue-3.14-460.el9.noarch 85/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : perl-threads-1:2.25-460.el9.x86_64 86/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : perl-threads-shared-1.61-460.el9.x86_64 87/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : postgresql-13.23-1.el9.x86_64 88/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : postgresql-contrib-13.23-1.el9.x86_64 89/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : postgresql-private-libs-13.23-1.el9.x86_64 90/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : postgresql-server-13.23-1.el9.x86_64 91/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 92/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 93/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 94/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 95/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 96/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 97/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 98/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 99/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 100/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 101/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 102/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 103/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 104/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 105/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 106/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 107/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 108/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 109/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 110/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 111/181 2026-02-25T11:42:29.656 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 112/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 113/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 114/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : xfsprogs-devel-6.4.0-7.el9.x86_64 115/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 116/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 117/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 118/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 119/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 120/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 121/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 122/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 123/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 124/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 125/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 126/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 127/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 128/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 129/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 130/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 131/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 132/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 133/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 134/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 135/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 136/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 137/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 138/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 139/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 140/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 141/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 142/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 143/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 144/181 2026-02-25T11:42:29.657 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 145/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 146/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 147/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 148/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 149/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 150/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 151/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 152/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 153/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 154/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 155/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 156/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 157/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 158/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 159/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 160/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : re2-1:20211101-20.el9.x86_64 161/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 162/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 163/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : librados2-2:20.3.0-5549.g92715cd3.el9.x86_64 164/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 165/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : librbd1-2:20.3.0-5549.g92715cd3.el9.x86_64 166/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 167/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libblkid-2.37.4-25.el9.x86_64 168/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libblkid-2.37.4-21.el9.x86_64 169/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libfdisk-2.37.4-25.el9.x86_64 170/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libfdisk-2.37.4-21.el9.x86_64 171/181 2026-02-25T11:42:29.658 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libmount-2.37.4-25.el9.x86_64 172/181 2026-02-25T11:42:29.659 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libmount-2.37.4-21.el9.x86_64 173/181 2026-02-25T11:42:29.659 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libsmartcols-2.37.4-25.el9.x86_64 174/181 2026-02-25T11:42:29.659 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libsmartcols-2.37.4-21.el9.x86_64 175/181 2026-02-25T11:42:29.659 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libuuid-2.37.4-25.el9.x86_64 176/181 2026-02-25T11:42:29.659 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : libuuid-2.37.4-21.el9.x86_64 177/181 2026-02-25T11:42:29.659 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : util-linux-2.37.4-25.el9.x86_64 178/181 2026-02-25T11:42:29.659 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : util-linux-2.37.4-21.el9.x86_64 179/181 2026-02-25T11:42:29.659 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : util-linux-core-2.37.4-25.el9.x86_64 180/181 2026-02-25T11:42:29.759 INFO:teuthology.orchestra.run.trial167.stdout: Verifying : util-linux-core-2.37.4-21.el9.x86_64 181/181 2026-02-25T11:42:29.760 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:29.760 INFO:teuthology.orchestra.run.trial167.stdout:Upgraded: 2026-02-25T11:42:29.760 INFO:teuthology.orchestra.run.trial167.stdout: libblkid-2.37.4-25.el9.x86_64 2026-02-25T11:42:29.760 INFO:teuthology.orchestra.run.trial167.stdout: libfdisk-2.37.4-25.el9.x86_64 2026-02-25T11:42:29.760 INFO:teuthology.orchestra.run.trial167.stdout: libmount-2.37.4-25.el9.x86_64 2026-02-25T11:42:29.760 INFO:teuthology.orchestra.run.trial167.stdout: librados2-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.760 INFO:teuthology.orchestra.run.trial167.stdout: librbd1-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.760 INFO:teuthology.orchestra.run.trial167.stdout: libsmartcols-2.37.4-25.el9.x86_64 2026-02-25T11:42:29.760 INFO:teuthology.orchestra.run.trial167.stdout: libuuid-2.37.4-25.el9.x86_64 2026-02-25T11:42:29.761 INFO:teuthology.orchestra.run.trial167.stdout: util-linux-2.37.4-25.el9.x86_64 2026-02-25T11:42:29.761 INFO:teuthology.orchestra.run.trial167.stdout: util-linux-core-2.37.4-25.el9.x86_64 2026-02-25T11:42:29.761 INFO:teuthology.orchestra.run.trial167.stdout:Installed: 2026-02-25T11:42:29.761 INFO:teuthology.orchestra.run.trial167.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T11:42:29.761 INFO:teuthology.orchestra.run.trial167.stdout: automake-1.16.2-8.el9.noarch 2026-02-25T11:42:29.761 INFO:teuthology.orchestra.run.trial167.stdout: bison-3.7.4-5.el9.x86_64 2026-02-25T11:42:29.761 INFO:teuthology.orchestra.run.trial167.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T11:42:29.761 INFO:teuthology.orchestra.run.trial167.stdout: ceph-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.761 INFO:teuthology.orchestra.run.trial167.stdout: ceph-base-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.762 INFO:teuthology.orchestra.run.trial167.stdout: ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.762 INFO:teuthology.orchestra.run.trial167.stdout: ceph-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.762 INFO:teuthology.orchestra.run.trial167.stdout: ceph-grafana-dashboards-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:29.762 INFO:teuthology.orchestra.run.trial167.stdout: ceph-immutable-object-cache-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.762 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mds-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.762 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mgr-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.762 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mgr-cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:29.762 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mgr-dashboard-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:29.762 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:29.762 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mgr-modules-core-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:29.762 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mgr-rook-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:29.763 INFO:teuthology.orchestra.run.trial167.stdout: ceph-mon-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.763 INFO:teuthology.orchestra.run.trial167.stdout: ceph-osd-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.763 INFO:teuthology.orchestra.run.trial167.stdout: ceph-osd-classic-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.763 INFO:teuthology.orchestra.run.trial167.stdout: ceph-osd-crimson-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.763 INFO:teuthology.orchestra.run.trial167.stdout: ceph-prometheus-alerts-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:29.763 INFO:teuthology.orchestra.run.trial167.stdout: ceph-radosgw-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.763 INFO:teuthology.orchestra.run.trial167.stdout: ceph-selinux-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.763 INFO:teuthology.orchestra.run.trial167.stdout: ceph-test-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.763 INFO:teuthology.orchestra.run.trial167.stdout: ceph-volume-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:29.763 INFO:teuthology.orchestra.run.trial167.stdout: cephadm-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:29.763 INFO:teuthology.orchestra.run.trial167.stdout: cephfs-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.763 INFO:teuthology.orchestra.run.trial167.stdout: cephfs-shell-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.763 INFO:teuthology.orchestra.run.trial167.stdout: cephfs-top-2:20.3.0-5549.g92715cd3.el9.noarch 2026-02-25T11:42:29.764 INFO:teuthology.orchestra.run.trial167.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T11:42:29.764 INFO:teuthology.orchestra.run.trial167.stdout: elfutils-libelf-devel-0.194-1.el9.x86_64 2026-02-25T11:42:29.764 INFO:teuthology.orchestra.run.trial167.stdout: flex-2.6.4-9.el9.x86_64 2026-02-25T11:42:29.764 INFO:teuthology.orchestra.run.trial167.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T11:42:29.764 INFO:teuthology.orchestra.run.trial167.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T11:42:29.764 INFO:teuthology.orchestra.run.trial167.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T11:42:29.764 INFO:teuthology.orchestra.run.trial167.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T11:42:29.764 INFO:teuthology.orchestra.run.trial167.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T11:42:29.764 INFO:teuthology.orchestra.run.trial167.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-25T11:42:29.764 INFO:teuthology.orchestra.run.trial167.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T11:42:29.764 INFO:teuthology.orchestra.run.trial167.stdout: libaio-devel-0.3.111-13.el9.x86_64 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: libcephfs-devel-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: libcephfs-proxy2-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: libcephfs2-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: libcephsqlite-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T11:42:29.765 INFO:teuthology.orchestra.run.trial167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T11:42:29.766 INFO:teuthology.orchestra.run.trial167.stdout: librados-devel-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.766 INFO:teuthology.orchestra.run.trial167.stdout: libradosstriper1-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.766 INFO:teuthology.orchestra.run.trial167.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T11:42:29.766 INFO:teuthology.orchestra.run.trial167.stdout: librgw2-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.766 INFO:teuthology.orchestra.run.trial167.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T11:42:29.766 INFO:teuthology.orchestra.run.trial167.stdout: libtool-2.4.6-46.el9.x86_64 2026-02-25T11:42:29.766 INFO:teuthology.orchestra.run.trial167.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T11:42:29.766 INFO:teuthology.orchestra.run.trial167.stdout: libuuid-devel-2.37.4-25.el9.x86_64 2026-02-25T11:42:29.766 INFO:teuthology.orchestra.run.trial167.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T11:42:29.766 INFO:teuthology.orchestra.run.trial167.stdout: libzstd-devel-1.5.5-1.el9.x86_64 2026-02-25T11:42:29.766 INFO:teuthology.orchestra.run.trial167.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-25T11:42:29.766 INFO:teuthology.orchestra.run.trial167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T11:42:29.767 INFO:teuthology.orchestra.run.trial167.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T11:42:29.767 INFO:teuthology.orchestra.run.trial167.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T11:42:29.767 INFO:teuthology.orchestra.run.trial167.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T11:42:29.767 INFO:teuthology.orchestra.run.trial167.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T11:42:29.767 INFO:teuthology.orchestra.run.trial167.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T11:42:29.767 INFO:teuthology.orchestra.run.trial167.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T11:42:29.767 INFO:teuthology.orchestra.run.trial167.stdout: openssl-devel-1:3.5.5-1.el9.x86_64 2026-02-25T11:42:29.767 INFO:teuthology.orchestra.run.trial167.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T11:42:29.767 INFO:teuthology.orchestra.run.trial167.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T11:42:29.767 INFO:teuthology.orchestra.run.trial167.stdout: perl-Thread-Queue-3.14-460.el9.noarch 2026-02-25T11:42:29.767 INFO:teuthology.orchestra.run.trial167.stdout: perl-threads-1:2.25-460.el9.x86_64 2026-02-25T11:42:29.767 INFO:teuthology.orchestra.run.trial167.stdout: perl-threads-shared-1.61-460.el9.x86_64 2026-02-25T11:42:29.767 INFO:teuthology.orchestra.run.trial167.stdout: postgresql-13.23-1.el9.x86_64 2026-02-25T11:42:29.768 INFO:teuthology.orchestra.run.trial167.stdout: postgresql-contrib-13.23-1.el9.x86_64 2026-02-25T11:42:29.768 INFO:teuthology.orchestra.run.trial167.stdout: postgresql-private-libs-13.23-1.el9.x86_64 2026-02-25T11:42:29.768 INFO:teuthology.orchestra.run.trial167.stdout: postgresql-server-13.23-1.el9.x86_64 2026-02-25T11:42:29.768 INFO:teuthology.orchestra.run.trial167.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T11:42:29.768 INFO:teuthology.orchestra.run.trial167.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T11:42:29.768 INFO:teuthology.orchestra.run.trial167.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T11:42:29.768 INFO:teuthology.orchestra.run.trial167.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T11:42:29.768 INFO:teuthology.orchestra.run.trial167.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T11:42:29.768 INFO:teuthology.orchestra.run.trial167.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T11:42:29.768 INFO:teuthology.orchestra.run.trial167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T11:42:29.768 INFO:teuthology.orchestra.run.trial167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T11:42:29.768 INFO:teuthology.orchestra.run.trial167.stdout: python3-ceph-argparse-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.769 INFO:teuthology.orchestra.run.trial167.stdout: python3-ceph-common-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.769 INFO:teuthology.orchestra.run.trial167.stdout: python3-cephfs-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.769 INFO:teuthology.orchestra.run.trial167.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T11:42:29.769 INFO:teuthology.orchestra.run.trial167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T11:42:29.769 INFO:teuthology.orchestra.run.trial167.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T11:42:29.769 INFO:teuthology.orchestra.run.trial167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T11:42:29.769 INFO:teuthology.orchestra.run.trial167.stdout: python3-cmd2-2.4.2-2.el9.noarch 2026-02-25T11:42:29.769 INFO:teuthology.orchestra.run.trial167.stdout: python3-colorama-0.4.6-3.el9.noarch 2026-02-25T11:42:29.769 INFO:teuthology.orchestra.run.trial167.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T11:42:29.769 INFO:teuthology.orchestra.run.trial167.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T11:42:29.769 INFO:teuthology.orchestra.run.trial167.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T11:42:29.769 INFO:teuthology.orchestra.run.trial167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T11:42:29.770 INFO:teuthology.orchestra.run.trial167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T11:42:29.770 INFO:teuthology.orchestra.run.trial167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T11:42:29.770 INFO:teuthology.orchestra.run.trial167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T11:42:29.770 INFO:teuthology.orchestra.run.trial167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T11:42:29.770 INFO:teuthology.orchestra.run.trial167.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T11:42:29.770 INFO:teuthology.orchestra.run.trial167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T11:42:29.770 INFO:teuthology.orchestra.run.trial167.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T11:42:29.770 INFO:teuthology.orchestra.run.trial167.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T11:42:29.770 INFO:teuthology.orchestra.run.trial167.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T11:42:29.770 INFO:teuthology.orchestra.run.trial167.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T11:42:29.770 INFO:teuthology.orchestra.run.trial167.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T11:42:29.770 INFO:teuthology.orchestra.run.trial167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T11:42:29.771 INFO:teuthology.orchestra.run.trial167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T11:42:29.771 INFO:teuthology.orchestra.run.trial167.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T11:42:29.771 INFO:teuthology.orchestra.run.trial167.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T11:42:29.771 INFO:teuthology.orchestra.run.trial167.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T11:42:29.771 INFO:teuthology.orchestra.run.trial167.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T11:42:29.771 INFO:teuthology.orchestra.run.trial167.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T11:42:29.771 INFO:teuthology.orchestra.run.trial167.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T11:42:29.771 INFO:teuthology.orchestra.run.trial167.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T11:42:29.771 INFO:teuthology.orchestra.run.trial167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T11:42:29.771 INFO:teuthology.orchestra.run.trial167.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T11:42:29.771 INFO:teuthology.orchestra.run.trial167.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T11:42:29.772 INFO:teuthology.orchestra.run.trial167.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T11:42:29.772 INFO:teuthology.orchestra.run.trial167.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T11:42:29.772 INFO:teuthology.orchestra.run.trial167.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2026-02-25T11:42:29.772 INFO:teuthology.orchestra.run.trial167.stdout: python3-rados-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.772 INFO:teuthology.orchestra.run.trial167.stdout: python3-rbd-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.772 INFO:teuthology.orchestra.run.trial167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T11:42:29.772 INFO:teuthology.orchestra.run.trial167.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T11:42:29.772 INFO:teuthology.orchestra.run.trial167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T11:42:29.772 INFO:teuthology.orchestra.run.trial167.stdout: python3-rgw-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.772 INFO:teuthology.orchestra.run.trial167.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T11:42:29.772 INFO:teuthology.orchestra.run.trial167.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T11:42:29.773 INFO:teuthology.orchestra.run.trial167.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T11:42:29.773 INFO:teuthology.orchestra.run.trial167.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T11:42:29.773 INFO:teuthology.orchestra.run.trial167.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T11:42:29.773 INFO:teuthology.orchestra.run.trial167.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T11:42:29.773 INFO:teuthology.orchestra.run.trial167.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T11:42:29.773 INFO:teuthology.orchestra.run.trial167.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2026-02-25T11:42:29.773 INFO:teuthology.orchestra.run.trial167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T11:42:29.773 INFO:teuthology.orchestra.run.trial167.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T11:42:29.773 INFO:teuthology.orchestra.run.trial167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T11:42:29.773 INFO:teuthology.orchestra.run.trial167.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T11:42:29.773 INFO:teuthology.orchestra.run.trial167.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T11:42:29.773 INFO:teuthology.orchestra.run.trial167.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T11:42:29.773 INFO:teuthology.orchestra.run.trial167.stdout: rbd-fuse-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.774 INFO:teuthology.orchestra.run.trial167.stdout: rbd-mirror-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.774 INFO:teuthology.orchestra.run.trial167.stdout: rbd-nbd-2:20.3.0-5549.g92715cd3.el9.x86_64 2026-02-25T11:42:29.774 INFO:teuthology.orchestra.run.trial167.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T11:42:29.774 INFO:teuthology.orchestra.run.trial167.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T11:42:29.774 INFO:teuthology.orchestra.run.trial167.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T11:42:29.774 INFO:teuthology.orchestra.run.trial167.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T11:42:29.774 INFO:teuthology.orchestra.run.trial167.stdout: uuid-1.6.2-55.el9.x86_64 2026-02-25T11:42:29.774 INFO:teuthology.orchestra.run.trial167.stdout: xfsprogs-devel-6.4.0-7.el9.x86_64 2026-02-25T11:42:29.774 INFO:teuthology.orchestra.run.trial167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T11:42:29.774 INFO:teuthology.orchestra.run.trial167.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-25T11:42:29.774 INFO:teuthology.orchestra.run.trial167.stdout: zip-3.0-35.el9.x86_64 2026-02-25T11:42:29.774 INFO:teuthology.orchestra.run.trial167.stdout: zlib-devel-1.2.11-41.el9.x86_64 2026-02-25T11:42:29.774 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:42:29.775 INFO:teuthology.orchestra.run.trial167.stdout:Complete! 2026-02-25T11:42:30.015 DEBUG:teuthology.parallel:result is None 2026-02-25T11:42:30.016 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=92715cd3c42198664811edeb975bd585312154aa 2026-02-25T11:42:30.196 DEBUG:teuthology.orchestra.run.trial028:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T11:42:30.242 INFO:teuthology.orchestra.run.trial028.stdout:20.3.0-5549.g92715cd3.el9 2026-02-25T11:42:30.243 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5549.g92715cd3.el9 2026-02-25T11:42:30.243 INFO:teuthology.task.install:The correct ceph version 20.3.0-5549.g92715cd3 is installed. 2026-02-25T11:42:30.247 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=92715cd3c42198664811edeb975bd585312154aa 2026-02-25T11:42:30.413 DEBUG:teuthology.orchestra.run.trial119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T11:42:30.459 INFO:teuthology.orchestra.run.trial119.stdout:20.3.0-5549.g92715cd3.el9 2026-02-25T11:42:30.460 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5549.g92715cd3.el9 2026-02-25T11:42:30.460 INFO:teuthology.task.install:The correct ceph version 20.3.0-5549.g92715cd3 is installed. 2026-02-25T11:42:30.463 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=92715cd3c42198664811edeb975bd585312154aa 2026-02-25T11:42:30.625 DEBUG:teuthology.orchestra.run.trial167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T11:42:30.671 INFO:teuthology.orchestra.run.trial167.stdout:20.3.0-5549.g92715cd3.el9 2026-02-25T11:42:30.671 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5549.g92715cd3.el9 2026-02-25T11:42:30.671 INFO:teuthology.task.install:The correct ceph version 20.3.0-5549.g92715cd3 is installed. 2026-02-25T11:42:30.675 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-25T11:42:30.675 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:42:30.675 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T11:42:30.734 DEBUG:teuthology.orchestra.run.trial119:> set -ex 2026-02-25T11:42:30.734 DEBUG:teuthology.orchestra.run.trial119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T11:42:30.790 DEBUG:teuthology.orchestra.run.trial167:> set -ex 2026-02-25T11:42:30.791 DEBUG:teuthology.orchestra.run.trial167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T11:42:30.848 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-25T11:42:30.849 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:42:30.849 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T11:42:30.904 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T11:42:31.001 DEBUG:teuthology.orchestra.run.trial119:> set -ex 2026-02-25T11:42:31.001 DEBUG:teuthology.orchestra.run.trial119:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T11:42:31.056 DEBUG:teuthology.orchestra.run.trial119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T11:42:31.152 DEBUG:teuthology.orchestra.run.trial167:> set -ex 2026-02-25T11:42:31.153 DEBUG:teuthology.orchestra.run.trial167:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T11:42:31.209 DEBUG:teuthology.orchestra.run.trial167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T11:42:31.304 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-25T11:42:31.304 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:42:31.305 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T11:42:31.359 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T11:42:31.454 DEBUG:teuthology.orchestra.run.trial119:> set -ex 2026-02-25T11:42:31.454 DEBUG:teuthology.orchestra.run.trial119:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T11:42:31.509 DEBUG:teuthology.orchestra.run.trial119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T11:42:31.603 DEBUG:teuthology.orchestra.run.trial167:> set -ex 2026-02-25T11:42:31.603 DEBUG:teuthology.orchestra.run.trial167:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T11:42:31.657 DEBUG:teuthology.orchestra.run.trial167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T11:42:31.753 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-25T11:42:31.754 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:42:31.754 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T11:42:31.808 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T11:42:31.905 DEBUG:teuthology.orchestra.run.trial119:> set -ex 2026-02-25T11:42:31.905 DEBUG:teuthology.orchestra.run.trial119:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T11:42:31.961 DEBUG:teuthology.orchestra.run.trial119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T11:42:32.055 DEBUG:teuthology.orchestra.run.trial167:> set -ex 2026-02-25T11:42:32.055 DEBUG:teuthology.orchestra.run.trial167:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T11:42:32.111 DEBUG:teuthology.orchestra.run.trial167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T11:42:32.207 INFO:teuthology.run_tasks:Running task cephadm... 2026-02-25T11:42:32.359 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'client_fscrypt_dummy_encryption': True}, 'mds': {'mds_bal_replicate_threshold': 1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'is offline because no MDS', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'BLUESTORE_SLOW_OP_ALERT', 'slow operation indications in BlueStore', 'experiencing slow operations in BlueStore', 'MGR_MODULE_ERROR', 'OSD_DOWN', 'osd.* is down', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'osds down', 'osd.* is down', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'pg .* is stuck inactive', 'pg .* is .*degraded', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '92715cd3c42198664811edeb975bd585312154aa', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2026-02-25T11:42:32.359 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa 2026-02-25T11:42:32.361 INFO:tasks.cephadm:Cluster fsid is 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:42:32.361 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2026-02-25T11:42:32.361 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '10.20.193.28', 'mon.b': '10.20.193.119', 'mon.c': '10.20.193.167'} 2026-02-25T11:42:32.361 INFO:tasks.cephadm:First mon is mon.a on trial028 2026-02-25T11:42:32.361 INFO:tasks.cephadm:First mgr is x 2026-02-25T11:42:32.361 INFO:tasks.cephadm:Normalizing hostnames... 2026-02-25T11:42:32.361 DEBUG:teuthology.orchestra.run.trial028:> sudo hostname $(hostname -s) 2026-02-25T11:42:32.417 DEBUG:teuthology.orchestra.run.trial119:> sudo hostname $(hostname -s) 2026-02-25T11:42:32.472 DEBUG:teuthology.orchestra.run.trial167:> sudo hostname $(hostname -s) 2026-02-25T11:42:32.531 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2026-02-25T11:42:32.531 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=92715cd3c42198664811edeb975bd585312154aa 2026-02-25T11:42:32.699 INFO:tasks.cephadm:builder_project result: [{'url': 'https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/flavors/default/', 'ref': 'wip-vshankar-testing-20260224.100235', 'sha1': '92715cd3c42198664811edeb975bd585312154aa', 'distro': 'centos', 'distro_version': '9', 'distro_codename': None, 'modified': '2026-02-24 16:46:15.859196', 'status': 'ready', 'flavor': 'default', 'project': 'ceph', 'archs': ['source', 'x86_64'], 'extra': {'version': '20.3.0-5549-g92715cd3', 'package_manager_version': '20.3.0-5549.g92715cd3', 'build_url': 'https://jenkins.ceph.com/job/ceph-dev-pipeline/3239/', 'root_build_cause': '', 'node_name': '10.20.192.98+toko08', 'job_name': 'ceph-dev-pipeline'}}] 2026-02-25T11:42:32.845 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-vshankar-testing-20260224.100235, sha1 92715cd3c42198664811edeb975bd585312154aa from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=92715cd3c42198664811edeb975bd585312154aa 2026-02-25T11:42:32.847 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/x86_64/flavors/default/cephadm 2026-02-25T11:42:32.847 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/x86_64/flavors/default/cephadm 2026-02-25T11:42:32.847 DEBUG:teuthology.orchestra.run.trial028:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2026-02-25T11:42:33.218 INFO:teuthology.orchestra.run.trial028.stdout:-rw-r--r--. 1 ubuntu ubuntu 1076263 Feb 25 11:42 /home/ubuntu/cephtest/cephadm 2026-02-25T11:42:33.219 DEBUG:teuthology.orchestra.run.trial119:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2026-02-25T11:42:33.568 INFO:teuthology.orchestra.run.trial119.stdout:-rw-r--r--. 1 ubuntu ubuntu 1076263 Feb 25 11:42 /home/ubuntu/cephtest/cephadm 2026-02-25T11:42:33.568 DEBUG:teuthology.orchestra.run.trial167:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20260224.100235/92715cd3c42198664811edeb975bd585312154aa/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2026-02-25T11:42:33.933 INFO:teuthology.orchestra.run.trial167.stdout:-rw-r--r--. 1 ubuntu ubuntu 1076263 Feb 25 11:42 /home/ubuntu/cephtest/cephadm 2026-02-25T11:42:33.934 DEBUG:teuthology.orchestra.run.trial028:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2026-02-25T11:42:33.975 DEBUG:teuthology.orchestra.run.trial119:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2026-02-25T11:42:34.015 DEBUG:teuthology.orchestra.run.trial167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2026-02-25T11:42:34.086 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa on all hosts... 2026-02-25T11:42:34.086 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa pull 2026-02-25T11:42:34.090 DEBUG:teuthology.orchestra.run.trial119:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa pull 2026-02-25T11:42:34.094 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa pull 2026-02-25T11:42:34.439 INFO:teuthology.orchestra.run.trial028.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa... 2026-02-25T11:42:34.448 INFO:teuthology.orchestra.run.trial119.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa... 2026-02-25T11:42:34.456 INFO:teuthology.orchestra.run.trial167.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa... 2026-02-25T11:42:47.287 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:42:47.288 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version": "ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev)", 2026-02-25T11:42:47.288 INFO:teuthology.orchestra.run.trial028.stdout: "image_id": "8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a", 2026-02-25T11:42:47.288 INFO:teuthology.orchestra.run.trial028.stdout: "repo_digests": [ 2026-02-25T11:42:47.288 INFO:teuthology.orchestra.run.trial028.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33" 2026-02-25T11:42:47.288 INFO:teuthology.orchestra.run.trial028.stdout: ] 2026-02-25T11:42:47.288 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:42:47.292 INFO:teuthology.orchestra.run.trial167.stdout:{ 2026-02-25T11:42:47.293 INFO:teuthology.orchestra.run.trial167.stdout: "ceph_version": "ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev)", 2026-02-25T11:42:47.293 INFO:teuthology.orchestra.run.trial167.stdout: "image_id": "8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a", 2026-02-25T11:42:47.293 INFO:teuthology.orchestra.run.trial167.stdout: "repo_digests": [ 2026-02-25T11:42:47.293 INFO:teuthology.orchestra.run.trial167.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33" 2026-02-25T11:42:47.293 INFO:teuthology.orchestra.run.trial167.stdout: ] 2026-02-25T11:42:47.293 INFO:teuthology.orchestra.run.trial167.stdout:} 2026-02-25T11:42:47.336 INFO:teuthology.orchestra.run.trial119.stdout:{ 2026-02-25T11:42:47.337 INFO:teuthology.orchestra.run.trial119.stdout: "ceph_version": "ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev)", 2026-02-25T11:42:47.337 INFO:teuthology.orchestra.run.trial119.stdout: "image_id": "8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a", 2026-02-25T11:42:47.337 INFO:teuthology.orchestra.run.trial119.stdout: "repo_digests": [ 2026-02-25T11:42:47.337 INFO:teuthology.orchestra.run.trial119.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33" 2026-02-25T11:42:47.337 INFO:teuthology.orchestra.run.trial119.stdout: ] 2026-02-25T11:42:47.337 INFO:teuthology.orchestra.run.trial119.stdout:} 2026-02-25T11:42:47.363 DEBUG:teuthology.orchestra.run.trial028:> sudo mkdir -p /etc/ceph 2026-02-25T11:42:47.562 DEBUG:teuthology.orchestra.run.trial119:> sudo mkdir -p /etc/ceph 2026-02-25T11:42:47.622 DEBUG:teuthology.orchestra.run.trial167:> sudo mkdir -p /etc/ceph 2026-02-25T11:42:47.682 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod 777 /etc/ceph 2026-02-25T11:42:47.738 DEBUG:teuthology.orchestra.run.trial119:> sudo chmod 777 /etc/ceph 2026-02-25T11:42:47.794 DEBUG:teuthology.orchestra.run.trial167:> sudo chmod 777 /etc/ceph 2026-02-25T11:42:47.850 INFO:tasks.cephadm:Writing seed config... 2026-02-25T11:42:47.851 INFO:tasks.cephadm: override: [client] client_fscrypt_dummy_encryption = True 2026-02-25T11:42:47.852 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2026-02-25T11:42:47.852 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2026-02-25T11:42:47.852 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2026-02-25T11:42:47.852 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2026-02-25T11:42:47.852 INFO:tasks.cephadm: override: [mon] debug mon = 20 2026-02-25T11:42:47.852 INFO:tasks.cephadm: override: [mon] debug ms = 1 2026-02-25T11:42:47.852 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2026-02-25T11:42:47.852 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2026-02-25T11:42:47.852 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2026-02-25T11:42:47.852 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2026-02-25T11:42:47.853 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2026-02-25T11:42:47.853 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2026-02-25T11:42:47.853 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2026-02-25T11:42:47.853 INFO:tasks.cephadm: override: [osd] debug ms = 1 2026-02-25T11:42:47.853 INFO:tasks.cephadm: override: [osd] debug osd = 20 2026-02-25T11:42:47.853 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2026-02-25T11:42:47.853 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2026-02-25T11:42:47.853 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2026-02-25T11:42:47.853 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2026-02-25T11:42:47.853 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2026-02-25T11:42:47.853 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2026-02-25T11:42:47.853 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2026-02-25T11:42:47.854 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2026-02-25T11:42:47.855 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:42:47.855 DEBUG:teuthology.orchestra.run.trial028:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2026-02-25T11:42:47.894 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 123fa7bf-123f-11f1-80da-d404e6e7d460 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd mclock skip benchmark = true osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client_fscrypt_dummy_encryption = True [mds] mds_bal_replicate_threshold = 1 2026-02-25T11:42:47.895 DEBUG:teuthology.orchestra.run.trial028:mon.a> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.a.service 2026-02-25T11:42:47.938 DEBUG:teuthology.orchestra.run.trial028:mgr.x> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mgr.x.service 2026-02-25T11:42:47.982 INFO:tasks.cephadm:Bootstrapping... 2026-02-25T11:42:47.983 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa -v bootstrap --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 10.20.193.28 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2026-02-25T11:42:48.172 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-25T11:42:48.172 INFO:teuthology.orchestra.run.trial028.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', '-v', 'bootstrap', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '10.20.193.28', '--skip-admin-label'] 2026-02-25T11:42:48.173 INFO:teuthology.orchestra.run.trial028.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2026-02-25T11:42:48.173 INFO:teuthology.orchestra.run.trial028.stdout:Verifying podman|docker is present... 2026-02-25T11:42:48.206 INFO:teuthology.orchestra.run.trial028.stdout:/bin/podman: stdout 5.8.0 2026-02-25T11:42:48.206 INFO:teuthology.orchestra.run.trial028.stdout:Verifying lvm2 is present... 2026-02-25T11:42:48.207 INFO:teuthology.orchestra.run.trial028.stdout:Verifying time synchronization is in place... 2026-02-25T11:42:48.221 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2026-02-25T11:42:48.221 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2026-02-25T11:42:48.231 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2026-02-25T11:42:48.231 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout inactive 2026-02-25T11:42:48.242 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout enabled 2026-02-25T11:42:48.253 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout active 2026-02-25T11:42:48.253 INFO:teuthology.orchestra.run.trial028.stdout:Unit chronyd.service is enabled and running 2026-02-25T11:42:48.253 INFO:teuthology.orchestra.run.trial028.stdout:Repeating the final host check... 2026-02-25T11:42:48.286 INFO:teuthology.orchestra.run.trial028.stdout:/bin/podman: stdout 5.8.0 2026-02-25T11:42:48.287 INFO:teuthology.orchestra.run.trial028.stdout:podman (/bin/podman) version 5.8.0 is present 2026-02-25T11:42:48.287 INFO:teuthology.orchestra.run.trial028.stdout:systemctl is present 2026-02-25T11:42:48.287 INFO:teuthology.orchestra.run.trial028.stdout:lvcreate is present 2026-02-25T11:42:48.298 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2026-02-25T11:42:48.298 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2026-02-25T11:42:48.309 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2026-02-25T11:42:48.309 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout inactive 2026-02-25T11:42:48.320 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout enabled 2026-02-25T11:42:48.331 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout active 2026-02-25T11:42:48.331 INFO:teuthology.orchestra.run.trial028.stdout:Unit chronyd.service is enabled and running 2026-02-25T11:42:48.331 INFO:teuthology.orchestra.run.trial028.stdout:Host looks OK 2026-02-25T11:42:48.331 INFO:teuthology.orchestra.run.trial028.stdout:Cluster fsid: 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:42:48.331 INFO:teuthology.orchestra.run.trial028.stdout:Acquiring lock 139924559831344 on /run/cephadm/123fa7bf-123f-11f1-80da-d404e6e7d460.lock 2026-02-25T11:42:48.332 INFO:teuthology.orchestra.run.trial028.stdout:Lock 139924559831344 acquired on /run/cephadm/123fa7bf-123f-11f1-80da-d404e6e7d460.lock 2026-02-25T11:42:48.332 INFO:teuthology.orchestra.run.trial028.stdout:Verifying IP 10.20.193.28 port 3300 ... 2026-02-25T11:42:48.332 INFO:teuthology.orchestra.run.trial028.stdout:Verifying IP 10.20.193.28 port 6789 ... 2026-02-25T11:42:48.333 INFO:teuthology.orchestra.run.trial028.stdout:Base mon IP(s) is [10.20.193.28:3300, 10.20.193.28:6789], mon addrv is [v2:10.20.193.28:3300,v1:10.20.193.28:6789] 2026-02-25T11:42:48.337 INFO:teuthology.orchestra.run.trial028.stdout:/sbin/ip: stdout default via 10.20.192.1 dev enp1s0f0np0 proto dhcp src 10.20.193.28 metric 100 2026-02-25T11:42:48.337 INFO:teuthology.orchestra.run.trial028.stdout:/sbin/ip: stdout 10.20.192.0/20 dev enp1s0f0np0 proto kernel scope link src 10.20.193.28 metric 100 2026-02-25T11:42:48.343 INFO:teuthology.orchestra.run.trial028.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2026-02-25T11:42:48.343 INFO:teuthology.orchestra.run.trial028.stdout:/sbin/ip: stdout fe80::/64 dev enp1s0f0np0 proto kernel metric 256 pref medium 2026-02-25T11:42:48.349 INFO:teuthology.orchestra.run.trial028.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2026-02-25T11:42:48.349 INFO:teuthology.orchestra.run.trial028.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2026-02-25T11:42:48.349 INFO:teuthology.orchestra.run.trial028.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2026-02-25T11:42:48.349 INFO:teuthology.orchestra.run.trial028.stdout:/sbin/ip: stdout 2: enp1s0f0np0: mtu 1500 state UP qlen 1000 2026-02-25T11:42:48.349 INFO:teuthology.orchestra.run.trial028.stdout:/sbin/ip: stdout inet6 fe80::925a:8ff:fe77:630c/64 scope link proto kernel_ll 2026-02-25T11:42:48.349 INFO:teuthology.orchestra.run.trial028.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2026-02-25T11:42:48.349 INFO:teuthology.orchestra.run.trial028.stdout:Mon IP `10.20.193.28` is in CIDR network `10.20.192.0/20` 2026-02-25T11:42:48.349 INFO:teuthology.orchestra.run.trial028.stdout:Mon IP `10.20.193.28` is in CIDR network `10.20.192.0/20` 2026-02-25T11:42:48.350 INFO:teuthology.orchestra.run.trial028.stdout:Inferred mon public CIDR from local network configuration ['10.20.192.0/20', '10.20.192.0/20'] 2026-02-25T11:42:48.350 INFO:teuthology.orchestra.run.trial028.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2026-02-25T11:42:48.350 INFO:teuthology.orchestra.run.trial028.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa... 2026-02-25T11:42:48.609 INFO:teuthology.orchestra.run.trial028.stdout:/bin/podman: stdout 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a 2026-02-25T11:42:48.609 INFO:teuthology.orchestra.run.trial028.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa... 2026-02-25T11:42:48.609 INFO:teuthology.orchestra.run.trial028.stdout:/bin/podman: stderr Getting image source signatures 2026-02-25T11:42:48.609 INFO:teuthology.orchestra.run.trial028.stdout:/bin/podman: stderr Copying blob sha256:b4aa6d1f1c32e33c02831d3a7a5bdaf8aaf22da6e8166666057f9d88149a3abf 2026-02-25T11:42:48.610 INFO:teuthology.orchestra.run.trial028.stdout:/bin/podman: stderr Copying blob sha256:8e380faede39ebd4286247457b408d979ab568aafd8389c42ec304b8cfba4e92 2026-02-25T11:42:48.610 INFO:teuthology.orchestra.run.trial028.stdout:/bin/podman: stderr Copying config sha256:8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a 2026-02-25T11:42:48.610 INFO:teuthology.orchestra.run.trial028.stdout:/bin/podman: stderr Writing manifest to image destination 2026-02-25T11:42:48.845 INFO:teuthology.orchestra.run.trial028.stdout:ceph: stdout ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev) 2026-02-25T11:42:48.845 INFO:teuthology.orchestra.run.trial028.stdout:Ceph version: ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev) 2026-02-25T11:42:48.845 INFO:teuthology.orchestra.run.trial028.stdout:Extracting ceph user uid/gid from container image... 2026-02-25T11:42:49.192 INFO:teuthology.orchestra.run.trial028.stdout:stat: stdout 167 167 2026-02-25T11:42:49.192 INFO:teuthology.orchestra.run.trial028.stdout:Creating initial keys... 2026-02-25T11:42:49.669 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph-authtool: stdout AQA54J5poDAoIhAAcLyV+MeumP6bKym7MOd0Tg== 2026-02-25T11:42:49.898 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph-authtool: stdout AQA54J5pnrWwLxAAmJJyYUzTpgenR3ZLGQWYAg== 2026-02-25T11:42:50.087 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph-authtool: stdout AQA64J5prSVgARAAMDMPjvjjKQ9WcbEolLMv2A== 2026-02-25T11:42:50.088 INFO:teuthology.orchestra.run.trial028.stdout:Creating initial monmap... 2026-02-25T11:42:50.332 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2026-02-25T11:42:50.332 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = tentacle 2026-02-25T11:42:50.332 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:42:50.332 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2026-02-25T11:42:50.333 INFO:teuthology.orchestra.run.trial028.stdout:monmaptool for a [v2:10.20.193.28:3300,v1:10.20.193.28:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2026-02-25T11:42:50.333 INFO:teuthology.orchestra.run.trial028.stdout:setting min_mon_release = tentacle 2026-02-25T11:42:50.333 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/monmaptool: set fsid to 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:42:50.333 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2026-02-25T11:42:50.333 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:42:50.333 INFO:teuthology.orchestra.run.trial028.stdout:Creating mon... 2026-02-25T11:42:50.611 INFO:teuthology.orchestra.run.trial028.stdout:create mon.a on 2026-02-25T11:42:50.866 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2026-02-25T11:42:51.042 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2026-02-25T11:42:51.227 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-123fa7bf-123f-11f1-80da-d404e6e7d460.target → /etc/systemd/system/ceph-123fa7bf-123f-11f1-80da-d404e6e7d460.target. 2026-02-25T11:42:51.227 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-123fa7bf-123f-11f1-80da-d404e6e7d460.target → /etc/systemd/system/ceph-123fa7bf-123f-11f1-80da-d404e6e7d460.target. 2026-02-25T11:42:51.441 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.a 2026-02-25T11:42:51.441 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stderr Failed to reset failed state of unit ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.a.service: Unit ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.a.service not loaded. 2026-02-25T11:42:51.666 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-123fa7bf-123f-11f1-80da-d404e6e7d460.target.wants/ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.a.service → /etc/systemd/system/ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@.service. 2026-02-25T11:42:51.944 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-25T11:42:51.944 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout disabled 2026-02-25T11:42:51.955 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-25T11:42:51.955 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout inactive 2026-02-25T11:42:51.955 INFO:teuthology.orchestra.run.trial028.stdout:firewalld.service is not enabled 2026-02-25T11:42:51.955 INFO:teuthology.orchestra.run.trial028.stdout:Not possible to enable service . firewalld.service is not available 2026-02-25T11:42:51.956 INFO:teuthology.orchestra.run.trial028.stdout:Waiting for mon to start... 2026-02-25T11:42:51.956 INFO:teuthology.orchestra.run.trial028.stdout:Waiting for mon... 2026-02-25T11:42:51.965 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:51 trial028 systemd[1]: Started Ceph mon.a for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:42:52.228 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:51 trial028 ceph-mon[38867]: mkfs 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:42:52.229 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:51 trial028 ceph-mon[38867]: mon.a is new leader, mons a in quorum (ranks 0) 2026-02-25T11:42:52.268 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout cluster: 2026-02-25T11:42:52.269 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout id: 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:42:52.269 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2026-02-25T11:42:52.269 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 2026-02-25T11:42:52.269 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout services: 2026-02-25T11:42:52.269 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.198285s) [leader: a] 2026-02-25T11:42:52.269 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2026-02-25T11:42:52.269 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2026-02-25T11:42:52.269 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 2026-02-25T11:42:52.269 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout data: 2026-02-25T11:42:52.269 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2026-02-25T11:42:52.269 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2026-02-25T11:42:52.270 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2026-02-25T11:42:52.270 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout pgs: 2026-02-25T11:42:52.270 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 2026-02-25T11:42:52.270 INFO:teuthology.orchestra.run.trial028.stdout:mon is available 2026-02-25T11:42:52.270 INFO:teuthology.orchestra.run.trial028.stdout:Assimilating anything we can from ceph.conf... 2026-02-25T11:42:52.544 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 2026-02-25T11:42:52.544 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout [global] 2026-02-25T11:42:52.545 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout fsid = 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:42:52.545 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2026-02-25T11:42:52.545 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_host = [v2:10.20.193.28:3300,v1:10.20.193.28:6789] 2026-02-25T11:42:52.545 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2026-02-25T11:42:52.545 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2026-02-25T11:42:52.545 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2026-02-25T11:42:52.545 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2026-02-25T11:42:52.545 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 2026-02-25T11:42:52.545 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout [mgr] 2026-02-25T11:42:52.545 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2026-02-25T11:42:52.545 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 2026-02-25T11:42:52.546 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout [osd] 2026-02-25T11:42:52.546 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2026-02-25T11:42:52.546 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2026-02-25T11:42:52.546 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2026-02-25T11:42:52.546 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2026-02-25T11:42:52.546 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2026-02-25T11:42:52.546 INFO:teuthology.orchestra.run.trial028.stdout:Generating new minimal ceph.conf... 2026-02-25T11:42:52.846 INFO:teuthology.orchestra.run.trial028.stdout:Restarting the monitor... 2026-02-25T11:42:53.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 systemd[1]: Stopping Ceph mon.a for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:42:53.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: mon.a is new leader, mons a in quorum (ranks 0) 2026-02-25T11:42:53.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: monmap epoch 1 2026-02-25T11:42:53.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:42:53.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: last_changed 2026-02-25T11:42:50.233340+0000 2026-02-25T11:42:53.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: created 2026-02-25T11:42:50.233340+0000 2026-02-25T11:42:53.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: min_mon_release 20 (tentacle) 2026-02-25T11:42:53.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: election_strategy: 1 2026-02-25T11:42:53.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: 0: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.a 2026-02-25T11:42:53.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: fsmap 2026-02-25T11:42:53.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: osdmap e1: 0 total, 0 up, 0 in 2026-02-25T11:42:53.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: mgrmap e1: no daemons active 2026-02-25T11:42:53.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: from='client.? 10.20.193.28:0/68170403' entity='client.admin' cmd={"prefix": "status"} : dispatch 2026-02-25T11:42:53.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: from='client.? 10.20.193.28:0/1173800642' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2026-02-25T11:42:53.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: from='client.? 10.20.193.28:0/1173800642' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2026-02-25T11:42:53.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-mon[38867]: from='client.? 10.20.193.28:0/3475089742' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:42:53.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-a[38863]: 2026-02-25T11:42:52.979+0000 7fb7bde32640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T11:42:53.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:52 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-a[38863]: 2026-02-25T11:42:52.979+0000 7fb7bde32640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2026-02-25T11:42:53.421 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 podman[39108]: 2026-02-25 11:42:53.092405192 +0000 UTC m=+0.134992405 container died 87fa09106ed7d9790ba8903882b0bbedbc14b380437f0f7eb319c9fd08dd2fb1 (image=quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-a, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-vshankar-testing-20260224.100235, io.buildah.version=1.41.3, org.label-schema.build-date=20260223, ceph=True, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.schema-version=1.0, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T11:42:53.421 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 podman[39108]: 2026-02-25 11:42:53.149413877 +0000 UTC m=+0.192001080 container remove 87fa09106ed7d9790ba8903882b0bbedbc14b380437f0f7eb319c9fd08dd2fb1 (image=quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-a, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.vendor=CentOS, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, ceph=True, FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.41.3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2026-02-25T11:42:53.421 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 bash[39108]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-a 2026-02-25T11:42:53.422 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 systemd[1]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.a.service: Deactivated successfully. 2026-02-25T11:42:53.422 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 systemd[1]: Stopped Ceph mon.a for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:42:53.422 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 systemd[1]: Starting Ceph mon.a for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:42:53.475 INFO:teuthology.orchestra.run.trial028.stdout:Setting public_network to 10.20.192.0/20 in global config section 2026-02-25T11:42:53.706 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 podman[39204]: 2026-02-25 11:42:53.419632389 +0000 UTC m=+0.027437624 container create 42d9b9b6f58f247e368277dca0c2b8413ae75a54edb177f02bac9d753c8bfa47 (image=quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-a, org.label-schema.build-date=20260223, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, io.buildah.version=1.41.3, org.label-schema.license=GPLv2, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.schema-version=1.0, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-25T11:42:53.706 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 podman[39204]: 2026-02-25 11:42:53.461597306 +0000 UTC m=+0.069402561 container init 42d9b9b6f58f247e368277dca0c2b8413ae75a54edb177f02bac9d753c8bfa47 (image=quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, ceph=True, org.label-schema.license=GPLv2, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.schema-version=1.0, org.label-schema.build-date=20260223, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, io.buildah.version=1.41.3) 2026-02-25T11:42:53.706 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 podman[39204]: 2026-02-25 11:42:53.466852911 +0000 UTC m=+0.074658156 container start 42d9b9b6f58f247e368277dca0c2b8413ae75a54edb177f02bac9d753c8bfa47 (image=quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-a, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, org.label-schema.build-date=20260223, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, ceph=True, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.vendor=CentOS, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, io.buildah.version=1.41.3, org.label-schema.license=GPLv2, FROM_IMAGE=quay.io/centos/centos:stream9) 2026-02-25T11:42:53.706 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 bash[39204]: 42d9b9b6f58f247e368277dca0c2b8413ae75a54edb177f02bac9d753c8bfa47 2026-02-25T11:42:53.706 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 podman[39204]: 2026-02-25 11:42:53.409809732 +0000 UTC m=+0.017614977 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa 2026-02-25T11:42:53.706 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 systemd[1]: Started Ceph mon.a for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:42:53.706 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: set uid:gid to 167:167 (ceph:ceph) 2026-02-25T11:42:53.707 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev - RelWithDebInfo), process ceph-mon, pid 2 2026-02-25T11:42:53.707 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: pidfile_write: ignore empty --pid-file 2026-02-25T11:42:53.707 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: load: jerasure load: lrc 2026-02-25T11:42:53.707 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: RocksDB version: 7.9.2 2026-02-25T11:42:53.707 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Git sha 0 2026-02-25T11:42:53.707 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Compile date 2026-02-24 16:18:01 2026-02-25T11:42:53.707 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: DB SUMMARY 2026-02-25T11:42:53.707 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: DB Session ID: Y8TTODCYWZP3ISFQR1KH 2026-02-25T11:42:53.707 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: CURRENT file: CURRENT 2026-02-25T11:42:53.707 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: IDENTITY file: IDENTITY 2026-02-25T11:42:53.707 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2026-02-25T11:42:53.708 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2026-02-25T11:42:53.708 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95679 ; 2026-02-25T11:42:53.708 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.error_if_exists: 0 2026-02-25T11:42:53.708 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.create_if_missing: 0 2026-02-25T11:42:53.708 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.paranoid_checks: 1 2026-02-25T11:42:53.708 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.flush_verify_memtable_count: 1 2026-02-25T11:42:53.708 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-25T11:42:53.708 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-25T11:42:53.708 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.env: 0xed6d60 2026-02-25T11:42:53.708 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.fs: PosixFileSystem 2026-02-25T11:42:53.708 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.info_log: 0xd11a240 2026-02-25T11:42:53.708 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_file_opening_threads: 16 2026-02-25T11:42:53.708 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.statistics: (nil) 2026-02-25T11:42:53.709 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.use_fsync: 0 2026-02-25T11:42:53.709 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_log_file_size: 0 2026-02-25T11:42:53.709 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-25T11:42:53.709 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.log_file_time_to_roll: 0 2026-02-25T11:42:53.709 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.keep_log_file_num: 1000 2026-02-25T11:42:53.709 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.recycle_log_file_num: 0 2026-02-25T11:42:53.709 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.allow_fallocate: 1 2026-02-25T11:42:53.709 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.allow_mmap_reads: 0 2026-02-25T11:42:53.709 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.allow_mmap_writes: 0 2026-02-25T11:42:53.709 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.use_direct_reads: 0 2026-02-25T11:42:53.709 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-25T11:42:53.709 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.create_missing_column_families: 0 2026-02-25T11:42:53.710 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.db_log_dir: 2026-02-25T11:42:53.710 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.wal_dir: 2026-02-25T11:42:53.710 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.table_cache_numshardbits: 6 2026-02-25T11:42:53.710 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.WAL_ttl_seconds: 0 2026-02-25T11:42:53.710 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.WAL_size_limit_MB: 0 2026-02-25T11:42:53.710 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-25T11:42:53.710 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-25T11:42:53.710 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.is_fd_close_on_exec: 1 2026-02-25T11:42:53.710 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.advise_random_on_open: 1 2026-02-25T11:42:53.710 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.db_write_buffer_size: 0 2026-02-25T11:42:53.710 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.write_buffer_manager: 0xd116000 2026-02-25T11:42:53.710 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-25T11:42:53.711 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-25T11:42:53.711 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.use_adaptive_mutex: 0 2026-02-25T11:42:53.711 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.rate_limiter: (nil) 2026-02-25T11:42:53.711 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-25T11:42:53.711 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.wal_recovery_mode: 2 2026-02-25T11:42:53.711 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.enable_thread_tracking: 0 2026-02-25T11:42:53.711 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.enable_pipelined_write: 0 2026-02-25T11:42:53.711 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.unordered_write: 0 2026-02-25T11:42:53.711 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-25T11:42:53.711 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-25T11:42:53.711 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-25T11:42:53.711 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-25T11:42:53.711 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.row_cache: None 2026-02-25T11:42:53.712 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.wal_filter: None 2026-02-25T11:42:53.712 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-25T11:42:53.712 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.allow_ingest_behind: 0 2026-02-25T11:42:53.712 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.two_write_queues: 0 2026-02-25T11:42:53.712 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.manual_wal_flush: 0 2026-02-25T11:42:53.712 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.wal_compression: 0 2026-02-25T11:42:53.712 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.atomic_flush: 0 2026-02-25T11:42:53.712 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-25T11:42:53.712 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.persist_stats_to_disk: 0 2026-02-25T11:42:53.712 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.write_dbid_to_manifest: 0 2026-02-25T11:42:53.712 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.log_readahead_size: 0 2026-02-25T11:42:53.712 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-25T11:42:53.712 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.best_efforts_recovery: 0 2026-02-25T11:42:53.713 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-25T11:42:53.713 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-25T11:42:53.713 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.allow_data_in_errors: 0 2026-02-25T11:42:53.713 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.db_host_id: __hostname__ 2026-02-25T11:42:53.713 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.enforce_single_del_contracts: true 2026-02-25T11:42:53.713 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_background_jobs: 2 2026-02-25T11:42:53.713 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_background_compactions: -1 2026-02-25T11:42:53.713 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_subcompactions: 1 2026-02-25T11:42:53.713 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-25T11:42:53.713 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2026-02-25T11:42:53.713 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.delayed_write_rate : 16777216 2026-02-25T11:42:53.713 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_total_wal_size: 0 2026-02-25T11:42:53.714 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-25T11:42:53.714 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.stats_dump_period_sec: 600 2026-02-25T11:42:53.714 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.stats_persist_period_sec: 600 2026-02-25T11:42:53.714 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-25T11:42:53.715 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_open_files: -1 2026-02-25T11:42:53.715 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.bytes_per_sync: 0 2026-02-25T11:42:53.715 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.wal_bytes_per_sync: 0 2026-02-25T11:42:53.715 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.strict_bytes_per_sync: 0 2026-02-25T11:42:53.715 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compaction_readahead_size: 0 2026-02-25T11:42:53.715 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_background_flushes: -1 2026-02-25T11:42:53.715 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Compression algorithms supported: 2026-02-25T11:42:53.715 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: kZSTD supported: 0 2026-02-25T11:42:53.715 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: kXpressCompression supported: 0 2026-02-25T11:42:53.715 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: kBZip2Compression supported: 0 2026-02-25T11:42:53.715 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-25T11:42:53.715 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: kLZ4Compression supported: 1 2026-02-25T11:42:53.716 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: kZlibCompression supported: 1 2026-02-25T11:42:53.716 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: kLZ4HCCompression supported: 1 2026-02-25T11:42:53.716 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: kSnappyCompression supported: 1 2026-02-25T11:42:53.716 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Fast CRC32 supported: Supported on x86 2026-02-25T11:42:53.716 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: DMutex implementation: pthread_mutex_t 2026-02-25T11:42:53.716 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2026-02-25T11:42:53.716 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-25T11:42:53.716 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-25T11:42:53.716 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.merge_operator: 2026-02-25T11:42:53.716 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compaction_filter: None 2026-02-25T11:42:53.716 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compaction_filter_factory: None 2026-02-25T11:42:53.717 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.sst_partitioner_factory: None 2026-02-25T11:42:53.717 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.memtable_factory: SkipListFactory 2026-02-25T11:42:53.717 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.table_factory: BlockBasedTable 2026-02-25T11:42:53.717 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0xd0c8ac0) 2026-02-25T11:42:53.717 INFO:journalctl@ceph.mon.a.trial028.stdout: cache_index_and_filter_blocks: 1 2026-02-25T11:42:53.717 INFO:journalctl@ceph.mon.a.trial028.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2026-02-25T11:42:53.717 INFO:journalctl@ceph.mon.a.trial028.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2026-02-25T11:42:53.717 INFO:journalctl@ceph.mon.a.trial028.stdout: pin_top_level_index_and_filter: 1 2026-02-25T11:42:53.717 INFO:journalctl@ceph.mon.a.trial028.stdout: index_type: 0 2026-02-25T11:42:53.717 INFO:journalctl@ceph.mon.a.trial028.stdout: data_block_index_type: 0 2026-02-25T11:42:53.718 INFO:journalctl@ceph.mon.a.trial028.stdout: index_shortening: 1 2026-02-25T11:42:53.718 INFO:journalctl@ceph.mon.a.trial028.stdout: data_block_hash_table_util_ratio: 0.750000 2026-02-25T11:42:53.718 INFO:journalctl@ceph.mon.a.trial028.stdout: checksum: 4 2026-02-25T11:42:53.718 INFO:journalctl@ceph.mon.a.trial028.stdout: no_block_cache: 0 2026-02-25T11:42:53.718 INFO:journalctl@ceph.mon.a.trial028.stdout: block_cache: 0xd0f5a50 2026-02-25T11:42:53.718 INFO:journalctl@ceph.mon.a.trial028.stdout: block_cache_name: BinnedLRUCache 2026-02-25T11:42:53.718 INFO:journalctl@ceph.mon.a.trial028.stdout: block_cache_options: 2026-02-25T11:42:53.718 INFO:journalctl@ceph.mon.a.trial028.stdout: capacity : 536870912 2026-02-25T11:42:53.718 INFO:journalctl@ceph.mon.a.trial028.stdout: num_shard_bits : 4 2026-02-25T11:42:53.718 INFO:journalctl@ceph.mon.a.trial028.stdout: strict_capacity_limit : 0 2026-02-25T11:42:53.718 INFO:journalctl@ceph.mon.a.trial028.stdout: high_pri_pool_ratio: 0.000 2026-02-25T11:42:53.719 INFO:journalctl@ceph.mon.a.trial028.stdout: block_cache_compressed: (nil) 2026-02-25T11:42:53.719 INFO:journalctl@ceph.mon.a.trial028.stdout: persistent_cache: (nil) 2026-02-25T11:42:53.719 INFO:journalctl@ceph.mon.a.trial028.stdout: block_size: 4096 2026-02-25T11:42:53.719 INFO:journalctl@ceph.mon.a.trial028.stdout: block_size_deviation: 10 2026-02-25T11:42:53.719 INFO:journalctl@ceph.mon.a.trial028.stdout: block_restart_interval: 16 2026-02-25T11:42:53.719 INFO:journalctl@ceph.mon.a.trial028.stdout: index_block_restart_interval: 1 2026-02-25T11:42:53.719 INFO:journalctl@ceph.mon.a.trial028.stdout: metadata_block_size: 4096 2026-02-25T11:42:53.719 INFO:journalctl@ceph.mon.a.trial028.stdout: partition_filters: 0 2026-02-25T11:42:53.719 INFO:journalctl@ceph.mon.a.trial028.stdout: use_delta_encoding: 1 2026-02-25T11:42:53.719 INFO:journalctl@ceph.mon.a.trial028.stdout: filter_policy: bloomfilter 2026-02-25T11:42:53.719 INFO:journalctl@ceph.mon.a.trial028.stdout: whole_key_filtering: 1 2026-02-25T11:42:53.719 INFO:journalctl@ceph.mon.a.trial028.stdout: verify_compression: 0 2026-02-25T11:42:53.720 INFO:journalctl@ceph.mon.a.trial028.stdout: read_amp_bytes_per_bit: 0 2026-02-25T11:42:53.720 INFO:journalctl@ceph.mon.a.trial028.stdout: format_version: 5 2026-02-25T11:42:53.720 INFO:journalctl@ceph.mon.a.trial028.stdout: enable_index_compression: 1 2026-02-25T11:42:53.720 INFO:journalctl@ceph.mon.a.trial028.stdout: block_align: 0 2026-02-25T11:42:53.720 INFO:journalctl@ceph.mon.a.trial028.stdout: max_auto_readahead_size: 262144 2026-02-25T11:42:53.720 INFO:journalctl@ceph.mon.a.trial028.stdout: prepopulate_block_cache: 0 2026-02-25T11:42:53.720 INFO:journalctl@ceph.mon.a.trial028.stdout: initial_auto_readahead_size: 8192 2026-02-25T11:42:53.720 INFO:journalctl@ceph.mon.a.trial028.stdout: num_file_reads_for_auto_readahead: 2 2026-02-25T11:42:53.720 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.write_buffer_size: 33554432 2026-02-25T11:42:53.720 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_write_buffer_number: 2 2026-02-25T11:42:53.720 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compression: NoCompression 2026-02-25T11:42:53.720 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.bottommost_compression: Disabled 2026-02-25T11:42:53.720 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.prefix_extractor: nullptr 2026-02-25T11:42:53.721 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-25T11:42:53.721 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.num_levels: 7 2026-02-25T11:42:53.721 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2026-02-25T11:42:53.721 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-25T11:42:53.721 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-25T11:42:53.721 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-25T11:42:53.721 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-25T11:42:53.721 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-25T11:42:53.721 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-25T11:42:53.721 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-25T11:42:53.721 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-25T11:42:53.721 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-25T11:42:53.721 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-25T11:42:53.722 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-25T11:42:53.722 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compression_opts.window_bits: -14 2026-02-25T11:42:53.722 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compression_opts.level: 32767 2026-02-25T11:42:53.722 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compression_opts.strategy: 0 2026-02-25T11:42:53.722 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-25T11:42:53.722 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-25T11:42:53.722 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-25T11:42:53.722 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-25T11:42:53.722 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compression_opts.enabled: false 2026-02-25T11:42:53.722 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-25T11:42:53.722 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2026-02-25T11:42:53.722 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-25T11:42:53.723 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-25T11:42:53.723 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.target_file_size_base: 67108864 2026-02-25T11:42:53.723 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.target_file_size_multiplier: 1 2026-02-25T11:42:53.723 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_bytes_for_level_base: 268435456 2026-02-25T11:42:53.723 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2026-02-25T11:42:53.723 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2026-02-25T11:42:53.723 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-25T11:42:53.723 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-25T11:42:53.723 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-25T11:42:53.723 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-25T11:42:53.723 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-25T11:42:53.723 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-25T11:42:53.724 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-25T11:42:53.724 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-25T11:42:53.724 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-25T11:42:53.724 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-25T11:42:53.724 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.arena_block_size: 1048576 2026-02-25T11:42:53.724 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-25T11:42:53.724 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-25T11:42:53.724 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.disable_auto_compactions: 0 2026-02-25T11:42:53.724 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-25T11:42:53.724 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-25T11:42:53.724 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-25T11:42:53.724 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-25T11:42:53.724 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-25T11:42:53.725 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-25T11:42:53.725 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-25T11:42:53.725 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-25T11:42:53.725 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-25T11:42:53.725 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-25T11:42:53.725 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-25T11:42:53.725 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.inplace_update_support: 0 2026-02-25T11:42:53.725 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.inplace_update_num_locks: 10000 2026-02-25T11:42:53.725 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-25T11:42:53.725 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-25T11:42:53.725 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.memtable_huge_page_size: 0 2026-02-25T11:42:53.725 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.bloom_locality: 0 2026-02-25T11:42:53.726 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.max_successive_merges: 0 2026-02-25T11:42:53.726 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.optimize_filters_for_hits: 0 2026-02-25T11:42:53.726 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.paranoid_file_checks: 0 2026-02-25T11:42:53.726 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.force_consistency_checks: 1 2026-02-25T11:42:53.726 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.report_bg_io_stats: 0 2026-02-25T11:42:53.726 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.ttl: 2592000 2026-02-25T11:42:53.726 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.periodic_compaction_seconds: 0 2026-02-25T11:42:53.726 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-25T11:42:53.726 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-25T11:42:53.726 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.enable_blob_files: false 2026-02-25T11:42:53.726 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.min_blob_size: 0 2026-02-25T11:42:53.726 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.blob_file_size: 268435456 2026-02-25T11:42:53.726 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.blob_compression_type: NoCompression 2026-02-25T11:42:53.727 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.enable_blob_garbage_collection: false 2026-02-25T11:42:53.727 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-25T11:42:53.727 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-25T11:42:53.727 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-25T11:42:53.727 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.blob_file_starting_level: 0 2026-02-25T11:42:53.727 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-25T11:42:53.727 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2026-02-25T11:42:53.727 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2026-02-25T11:42:53.727 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0ae0cae3-0b6c-4cca-82c3-10839c9c5e23 2026-02-25T11:42:53.727 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: EVENT_LOG_v1 {"time_micros": 1772019773497857, "job": 1, "event": "recovery_started", "wal_files": [9]} 2026-02-25T11:42:53.727 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2026-02-25T11:42:53.727 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: EVENT_LOG_v1 {"time_micros": 1772019773499162, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92171, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 274, "table_properties": {"data_size": 90213, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11703, "raw_average_key_size": 48, "raw_value_size": 83576, "raw_average_value_size": 346, "num_data_blocks": 12, "num_entries": 241, "num_filter_entries": 241, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1772019773, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0ae0cae3-0b6c-4cca-82c3-10839c9c5e23", "db_session_id": "Y8TTODCYWZP3ISFQR1KH", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2026-02-25T11:42:53.728 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: EVENT_LOG_v1 {"time_micros": 1772019773499275, "job": 1, "event": "recovery_finished"} 2026-02-25T11:42:53.728 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2026-02-25T11:42:53.728 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2026-02-25T11:42:53.728 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0xd138e00 2026-02-25T11:42:53.728 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: DB pointer 0xd258000 2026-02-25T11:42:53.728 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2026-02-25T11:42:53.728 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: rocksdb: [db/db_impl/db_impl.cc:1111] 2026-02-25T11:42:53.728 INFO:journalctl@ceph.mon.a.trial028.stdout: ** DB Stats ** 2026-02-25T11:42:53.728 INFO:journalctl@ceph.mon.a.trial028.stdout: Uptime(secs): 0.0 total, 0.0 interval 2026-02-25T11:42:53.728 INFO:journalctl@ceph.mon.a.trial028.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2026-02-25T11:42:53.728 INFO:journalctl@ceph.mon.a.trial028.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2026-02-25T11:42:53.728 INFO:journalctl@ceph.mon.a.trial028.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2026-02-25T11:42:53.728 INFO:journalctl@ceph.mon.a.trial028.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2026-02-25T11:42:53.729 INFO:journalctl@ceph.mon.a.trial028.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2026-02-25T11:42:53.729 INFO:journalctl@ceph.mon.a.trial028.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2026-02-25T11:42:53.729 INFO:journalctl@ceph.mon.a.trial028.stdout: 2026-02-25T11:42:53.729 INFO:journalctl@ceph.mon.a.trial028.stdout: ** Compaction Stats [default] ** 2026-02-25T11:42:53.729 INFO:journalctl@ceph.mon.a.trial028.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2026-02-25T11:42:53.729 INFO:journalctl@ceph.mon.a.trial028.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2026-02-25T11:42:53.729 INFO:journalctl@ceph.mon.a.trial028.stdout: L0 2/0 91.88 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 107.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-25T11:42:53.729 INFO:journalctl@ceph.mon.a.trial028.stdout: Sum 2/0 91.88 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 107.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-25T11:42:53.729 INFO:journalctl@ceph.mon.a.trial028.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 107.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-25T11:42:53.729 INFO:journalctl@ceph.mon.a.trial028.stdout: 2026-02-25T11:42:53.729 INFO:journalctl@ceph.mon.a.trial028.stdout: ** Compaction Stats [default] ** 2026-02-25T11:42:53.729 INFO:journalctl@ceph.mon.a.trial028.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2026-02-25T11:42:53.730 INFO:journalctl@ceph.mon.a.trial028.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2026-02-25T11:42:53.730 INFO:journalctl@ceph.mon.a.trial028.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 107.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-25T11:42:53.730 INFO:journalctl@ceph.mon.a.trial028.stdout: 2026-02-25T11:42:53.730 INFO:journalctl@ceph.mon.a.trial028.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2026-02-25T11:42:53.730 INFO:journalctl@ceph.mon.a.trial028.stdout: 2026-02-25T11:42:53.730 INFO:journalctl@ceph.mon.a.trial028.stdout: Uptime(secs): 0.0 total, 0.0 interval 2026-02-25T11:42:53.730 INFO:journalctl@ceph.mon.a.trial028.stdout: Flush(GB): cumulative 0.000, interval 0.000 2026-02-25T11:42:53.731 INFO:journalctl@ceph.mon.a.trial028.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2026-02-25T11:42:53.731 INFO:journalctl@ceph.mon.a.trial028.stdout: AddFile(Total Files): cumulative 0, interval 0 2026-02-25T11:42:53.731 INFO:journalctl@ceph.mon.a.trial028.stdout: AddFile(L0 Files): cumulative 0, interval 0 2026-02-25T11:42:53.731 INFO:journalctl@ceph.mon.a.trial028.stdout: AddFile(Keys): cumulative 0, interval 0 2026-02-25T11:42:53.731 INFO:journalctl@ceph.mon.a.trial028.stdout: Cumulative compaction: 0.00 GB write, 16.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2026-02-25T11:42:53.731 INFO:journalctl@ceph.mon.a.trial028.stdout: Interval compaction: 0.00 GB write, 16.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2026-02-25T11:42:53.732 INFO:journalctl@ceph.mon.a.trial028.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2026-02-25T11:42:53.732 INFO:journalctl@ceph.mon.a.trial028.stdout: Block cache BinnedLRUCache@0xd0f5a50#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2026-02-25T11:42:53.732 INFO:journalctl@ceph.mon.a.trial028.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2026-02-25T11:42:53.732 INFO:journalctl@ceph.mon.a.trial028.stdout: 2026-02-25T11:42:53.732 INFO:journalctl@ceph.mon.a.trial028.stdout: ** File Read Latency Histogram By Level [default] ** 2026-02-25T11:42:53.732 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: starting mon.a rank 0 at public addrs [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] at bind addrs [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:42:53.733 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: mon.a@-1(???) e1 preinit fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:42:53.733 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: mon.a@-1(???).mds e1 new map 2026-02-25T11:42:53.733 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: mon.a@-1(???).mds e1 print_map 2026-02-25T11:42:53.733 INFO:journalctl@ceph.mon.a.trial028.stdout: e1 2026-02-25T11:42:53.733 INFO:journalctl@ceph.mon.a.trial028.stdout: btime 2026-02-25T11:42:51:962673+0000 2026-02-25T11:42:53.734 INFO:journalctl@ceph.mon.a.trial028.stdout: enable_multiple, ever_enabled_multiple: 1,1 2026-02-25T11:42:53.734 INFO:journalctl@ceph.mon.a.trial028.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2026-02-25T11:42:53.734 INFO:journalctl@ceph.mon.a.trial028.stdout: legacy client fscid: -1 2026-02-25T11:42:53.734 INFO:journalctl@ceph.mon.a.trial028.stdout: 2026-02-25T11:42:53.734 INFO:journalctl@ceph.mon.a.trial028.stdout: No filesystems configured 2026-02-25T11:42:53.734 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2026-02-25T11:42:53.734 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2026-02-25T11:42:53.735 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2026-02-25T11:42:53.735 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2026-02-25T11:42:53.735 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2026-02-25T11:42:53.735 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: mon.a is new leader, mons a in quorum (ranks 0) 2026-02-25T11:42:53.735 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: monmap epoch 1 2026-02-25T11:42:53.735 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:42:53.735 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: last_changed 2026-02-25T11:42:50.233340+0000 2026-02-25T11:42:53.735 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: created 2026-02-25T11:42:50.233340+0000 2026-02-25T11:42:53.735 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: min_mon_release 20 (tentacle) 2026-02-25T11:42:53.735 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: election_strategy: 1 2026-02-25T11:42:53.736 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: 0: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.a 2026-02-25T11:42:53.736 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: fsmap 2026-02-25T11:42:53.736 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: osdmap e1: 0 total, 0 up, 0 in 2026-02-25T11:42:53.736 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:53 trial028 ceph-mon[39224]: mgrmap e1: no daemons active 2026-02-25T11:42:53.768 INFO:teuthology.orchestra.run.trial028.stdout:Wrote config to /etc/ceph/ceph.conf 2026-02-25T11:42:53.769 INFO:teuthology.orchestra.run.trial028.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2026-02-25T11:42:53.769 INFO:teuthology.orchestra.run.trial028.stdout:Creating mgr... 2026-02-25T11:42:53.770 INFO:teuthology.orchestra.run.trial028.stdout:Verifying port 0.0.0.0:9283 ... 2026-02-25T11:42:53.770 INFO:teuthology.orchestra.run.trial028.stdout:Verifying port 0.0.0.0:8765 ... 2026-02-25T11:42:53.996 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mgr.x 2026-02-25T11:42:53.996 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stderr Failed to reset failed state of unit ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mgr.x.service: Unit ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mgr.x.service not loaded. 2026-02-25T11:42:54.186 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-123fa7bf-123f-11f1-80da-d404e6e7d460.target.wants/ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mgr.x.service → /etc/systemd/system/ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@.service. 2026-02-25T11:42:54.310 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:42:54 trial028 systemd[1]: Starting Ceph mgr.x for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:42:54.454 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-25T11:42:54.454 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout disabled 2026-02-25T11:42:54.466 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-25T11:42:54.466 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout inactive 2026-02-25T11:42:54.466 INFO:teuthology.orchestra.run.trial028.stdout:firewalld.service is not enabled 2026-02-25T11:42:54.466 INFO:teuthology.orchestra.run.trial028.stdout:Not possible to enable service . firewalld.service is not available 2026-02-25T11:42:54.480 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-25T11:42:54.480 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout disabled 2026-02-25T11:42:54.491 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-25T11:42:54.492 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout inactive 2026-02-25T11:42:54.492 INFO:teuthology.orchestra.run.trial028.stdout:firewalld.service is not enabled 2026-02-25T11:42:54.492 INFO:teuthology.orchestra.run.trial028.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2026-02-25T11:42:54.492 INFO:teuthology.orchestra.run.trial028.stdout:Waiting for mgr to start... 2026-02-25T11:42:54.492 INFO:teuthology.orchestra.run.trial028.stdout:Waiting for mgr... 2026-02-25T11:42:54.697 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:42:54 trial028 podman[39441]: 2026-02-25 11:42:54.383511155 +0000 UTC m=+0.027933522 container create d6b9f684dc01ab9efd29ab55b945d8738eb1da04c56ebda86ee94ddfb347ee48 (image=quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.41.3, org.label-schema.build-date=20260223, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.license=GPLv2) 2026-02-25T11:42:54.697 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:42:54 trial028 podman[39441]: 2026-02-25 11:42:54.424732055 +0000 UTC m=+0.069154422 container init d6b9f684dc01ab9efd29ab55b945d8738eb1da04c56ebda86ee94ddfb347ee48 (image=quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x, CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.41.3, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-25T11:42:54.697 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:42:54 trial028 podman[39441]: 2026-02-25 11:42:54.429438895 +0000 UTC m=+0.073861272 container start d6b9f684dc01ab9efd29ab55b945d8738eb1da04c56ebda86ee94ddfb347ee48 (image=quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.schema-version=1.0, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223, ceph=True, org.label-schema.vendor=CentOS, org.opencontainers.image.authors=Ceph Release Team , io.buildah.version=1.41.3, org.label-schema.license=GPLv2, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T11:42:54.697 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:42:54 trial028 bash[39441]: d6b9f684dc01ab9efd29ab55b945d8738eb1da04c56ebda86ee94ddfb347ee48 2026-02-25T11:42:54.697 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:42:54 trial028 podman[39441]: 2026-02-25 11:42:54.372882547 +0000 UTC m=+0.017304924 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa 2026-02-25T11:42:54.697 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:42:54 trial028 systemd[1]: Started Ceph mgr.x for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:42:54.790 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 2026-02-25T11:42:54.791 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout { 2026-02-25T11:42:54.791 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "fsid": "123fa7bf-123f-11f1-80da-d404e6e7d460", 2026-02-25T11:42:54.791 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "health": { 2026-02-25T11:42:54.791 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2026-02-25T11:42:54.791 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "checks": {}, 2026-02-25T11:42:54.791 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "mutes": [] 2026-02-25T11:42:54.791 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:54.791 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2026-02-25T11:42:54.792 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "quorum": [ 2026-02-25T11:42:54.792 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 0 2026-02-25T11:42:54.792 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout ], 2026-02-25T11:42:54.792 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2026-02-25T11:42:54.792 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "a" 2026-02-25T11:42:54.792 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout ], 2026-02-25T11:42:54.792 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "quorum_age": 1, 2026-02-25T11:42:54.792 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "monmap": { 2026-02-25T11:42:54.792 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-25T11:42:54.792 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "tentacle", 2026-02-25T11:42:54.792 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_mons": 1 2026-02-25T11:42:54.792 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:54.793 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "osdmap": { 2026-02-25T11:42:54.793 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-25T11:42:54.793 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2026-02-25T11:42:54.793 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2026-02-25T11:42:54.793 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2026-02-25T11:42:54.793 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2026-02-25T11:42:54.793 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2026-02-25T11:42:54.793 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2026-02-25T11:42:54.793 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:54.793 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "pgmap": { 2026-02-25T11:42:54.793 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2026-02-25T11:42:54.793 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2026-02-25T11:42:54.794 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2026-02-25T11:42:54.794 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2026-02-25T11:42:54.794 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2026-02-25T11:42:54.794 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2026-02-25T11:42:54.794 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2026-02-25T11:42:54.794 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2026-02-25T11:42:54.794 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:54.794 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "fsmap": { 2026-02-25T11:42:54.794 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-25T11:42:54.794 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "btime": "2026-02-25T11:42:51:962673+0000", 2026-02-25T11:42:54.794 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "by_rank": [], 2026-02-25T11:42:54.794 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "up:standby": 0 2026-02-25T11:42:54.794 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:54.795 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "mgrmap": { 2026-02-25T11:42:54.795 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "available": false, 2026-02-25T11:42:54.795 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2026-02-25T11:42:54.795 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "modules": [ 2026-02-25T11:42:54.795 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "iostat", 2026-02-25T11:42:54.795 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "nfs" 2026-02-25T11:42:54.795 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout ], 2026-02-25T11:42:54.795 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-25T11:42:54.795 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:54.795 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "servicemap": { 2026-02-25T11:42:54.795 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-25T11:42:54.795 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "modified": "2026-02-25T11:42:51.963119+0000", 2026-02-25T11:42:54.796 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-25T11:42:54.796 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:54.796 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "progress_events": {} 2026-02-25T11:42:54.796 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout } 2026-02-25T11:42:54.796 INFO:teuthology.orchestra.run.trial028.stdout:mgr not available, waiting (1/15)... 2026-02-25T11:42:55.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:54 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/175157633' entity='client.admin' 2026-02-25T11:42:56.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:55 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/4115412768' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2026-02-25T11:42:56.060 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:42:55 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: /lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-25T11:42:56.060 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:42:55 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-25T11:42:56.060 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:42:55 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: from numpy import show_config as show_numpy_config 2026-02-25T11:42:57.100 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:57 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2716198742' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2026-02-25T11:42:57.112 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 2026-02-25T11:42:57.113 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout { 2026-02-25T11:42:57.113 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "fsid": "123fa7bf-123f-11f1-80da-d404e6e7d460", 2026-02-25T11:42:57.113 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "health": { 2026-02-25T11:42:57.113 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2026-02-25T11:42:57.113 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "checks": {}, 2026-02-25T11:42:57.113 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "mutes": [] 2026-02-25T11:42:57.113 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:57.113 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2026-02-25T11:42:57.113 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "quorum": [ 2026-02-25T11:42:57.114 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 0 2026-02-25T11:42:57.114 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout ], 2026-02-25T11:42:57.114 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2026-02-25T11:42:57.114 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "a" 2026-02-25T11:42:57.114 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout ], 2026-02-25T11:42:57.114 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2026-02-25T11:42:57.114 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "monmap": { 2026-02-25T11:42:57.114 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-25T11:42:57.114 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "tentacle", 2026-02-25T11:42:57.114 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_mons": 1 2026-02-25T11:42:57.114 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:57.114 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "osdmap": { 2026-02-25T11:42:57.114 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-25T11:42:57.115 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2026-02-25T11:42:57.115 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2026-02-25T11:42:57.115 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2026-02-25T11:42:57.115 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2026-02-25T11:42:57.115 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2026-02-25T11:42:57.115 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2026-02-25T11:42:57.115 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:57.115 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "pgmap": { 2026-02-25T11:42:57.115 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2026-02-25T11:42:57.115 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2026-02-25T11:42:57.115 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2026-02-25T11:42:57.115 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2026-02-25T11:42:57.116 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2026-02-25T11:42:57.116 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2026-02-25T11:42:57.116 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2026-02-25T11:42:57.116 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2026-02-25T11:42:57.116 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:57.116 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "fsmap": { 2026-02-25T11:42:57.116 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-25T11:42:57.116 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "btime": "2026-02-25T11:42:51:962673+0000", 2026-02-25T11:42:57.116 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "by_rank": [], 2026-02-25T11:42:57.116 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "up:standby": 0 2026-02-25T11:42:57.116 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:57.116 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "mgrmap": { 2026-02-25T11:42:57.116 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "available": false, 2026-02-25T11:42:57.117 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2026-02-25T11:42:57.117 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "modules": [ 2026-02-25T11:42:57.117 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "iostat", 2026-02-25T11:42:57.117 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "nfs" 2026-02-25T11:42:57.117 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout ], 2026-02-25T11:42:57.117 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-25T11:42:57.117 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:57.117 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "servicemap": { 2026-02-25T11:42:57.117 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-25T11:42:57.117 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "modified": "2026-02-25T11:42:51.963119+0000", 2026-02-25T11:42:57.117 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-25T11:42:57.117 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:57.117 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "progress_events": {} 2026-02-25T11:42:57.118 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout } 2026-02-25T11:42:57.118 INFO:teuthology.orchestra.run.trial028.stdout:mgr not available, waiting (2/15)... 2026-02-25T11:42:58.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:58 trial028 ceph-mon[39224]: Activating manager daemon x 2026-02-25T11:42:58.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:58 trial028 ceph-mon[39224]: mgrmap e2: x(active, starting, since 0.00411603s) 2026-02-25T11:42:58.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:58 trial028 ceph-mon[39224]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2026-02-25T11:42:58.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:58 trial028 ceph-mon[39224]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2026-02-25T11:42:58.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:58 trial028 ceph-mon[39224]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2026-02-25T11:42:58.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:58 trial028 ceph-mon[39224]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:42:58.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:58 trial028 ceph-mon[39224]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2026-02-25T11:42:58.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:58 trial028 ceph-mon[39224]: Manager daemon x is now available 2026-02-25T11:42:58.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:58 trial028 ceph-mon[39224]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2026-02-25T11:42:58.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:58 trial028 ceph-mon[39224]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' 2026-02-25T11:42:58.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:58 trial028 ceph-mon[39224]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2026-02-25T11:42:58.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:58 trial028 ceph-mon[39224]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' 2026-02-25T11:42:58.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:42:58 trial028 ceph-mon[39224]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' 2026-02-25T11:42:59.516 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 2026-02-25T11:42:59.517 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout { 2026-02-25T11:42:59.517 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "fsid": "123fa7bf-123f-11f1-80da-d404e6e7d460", 2026-02-25T11:42:59.517 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "health": { 2026-02-25T11:42:59.517 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2026-02-25T11:42:59.517 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "checks": {}, 2026-02-25T11:42:59.517 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "mutes": [] 2026-02-25T11:42:59.517 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:59.518 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2026-02-25T11:42:59.518 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "quorum": [ 2026-02-25T11:42:59.518 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 0 2026-02-25T11:42:59.518 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout ], 2026-02-25T11:42:59.518 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2026-02-25T11:42:59.518 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "a" 2026-02-25T11:42:59.518 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout ], 2026-02-25T11:42:59.518 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2026-02-25T11:42:59.518 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "monmap": { 2026-02-25T11:42:59.518 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-25T11:42:59.518 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "tentacle", 2026-02-25T11:42:59.518 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_mons": 1 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "osdmap": { 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "pgmap": { 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2026-02-25T11:42:59.519 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2026-02-25T11:42:59.520 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2026-02-25T11:42:59.520 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2026-02-25T11:42:59.520 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2026-02-25T11:42:59.520 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2026-02-25T11:42:59.520 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2026-02-25T11:42:59.520 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:59.520 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "fsmap": { 2026-02-25T11:42:59.520 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-25T11:42:59.520 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "btime": "2026-02-25T11:42:51:962673+0000", 2026-02-25T11:42:59.520 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "by_rank": [], 2026-02-25T11:42:59.520 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "up:standby": 0 2026-02-25T11:42:59.520 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:59.520 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "mgrmap": { 2026-02-25T11:42:59.521 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "available": true, 2026-02-25T11:42:59.521 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2026-02-25T11:42:59.521 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "modules": [ 2026-02-25T11:42:59.521 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "iostat", 2026-02-25T11:42:59.521 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "nfs" 2026-02-25T11:42:59.521 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout ], 2026-02-25T11:42:59.521 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-25T11:42:59.521 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:59.521 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "servicemap": { 2026-02-25T11:42:59.521 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 1, 2026-02-25T11:42:59.521 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "modified": "2026-02-25T11:42:51.963119+0000", 2026-02-25T11:42:59.521 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "services": {} 2026-02-25T11:42:59.522 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout }, 2026-02-25T11:42:59.522 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "progress_events": {} 2026-02-25T11:42:59.522 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout } 2026-02-25T11:42:59.522 INFO:teuthology.orchestra.run.trial028.stdout:mgr is available 2026-02-25T11:42:59.883 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 2026-02-25T11:42:59.883 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout [global] 2026-02-25T11:42:59.883 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout fsid = 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:42:59.883 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2026-02-25T11:42:59.883 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_host = [v2:10.20.193.28:3300,v1:10.20.193.28:6789] 2026-02-25T11:42:59.883 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout [mgr] 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout [osd] 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2026-02-25T11:42:59.884 INFO:teuthology.orchestra.run.trial028.stdout:Enabling cephadm module... 2026-02-25T11:43:00.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:00 trial028 ceph-mon[39224]: mgrmap e3: x(active, since 1.00779s) 2026-02-25T11:43:00.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:00 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2886477994' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2026-02-25T11:43:00.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:00 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1007823634' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2026-02-25T11:43:00.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:00 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3904241517' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2026-02-25T11:43:00.561 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:00 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: ignoring --setuser ceph since I am not root 2026-02-25T11:43:00.561 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:00 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: ignoring --setgroup ceph since I am not root 2026-02-25T11:43:00.735 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout { 2026-02-25T11:43:00.735 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 4, 2026-02-25T11:43:00.735 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "available": true, 2026-02-25T11:43:00.735 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "active_name": "x", 2026-02-25T11:43:00.735 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_standby": 0 2026-02-25T11:43:00.735 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout } 2026-02-25T11:43:00.736 INFO:teuthology.orchestra.run.trial028.stdout:Waiting for the mgr to restart... 2026-02-25T11:43:00.736 INFO:teuthology.orchestra.run.trial028.stdout:Waiting for mgr epoch 4... 2026-02-25T11:43:01.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:01 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3904241517' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2026-02-25T11:43:01.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:01 trial028 ceph-mon[39224]: mgrmap e4: x(active, since 2s) 2026-02-25T11:43:01.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:01 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/526933894' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2026-02-25T11:43:02.060 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:01 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: /lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-25T11:43:02.060 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:01 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-25T11:43:02.060 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:01 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: from numpy import show_config as show_numpy_config 2026-02-25T11:43:04.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:04 trial028 ceph-mon[39224]: Active manager daemon x restarted 2026-02-25T11:43:04.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:04 trial028 ceph-mon[39224]: Activating manager daemon x 2026-02-25T11:43:04.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:04 trial028 ceph-mon[39224]: osdmap e2: 0 total, 0 up, 0 in 2026-02-25T11:43:04.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:04 trial028 ceph-mon[39224]: mgrmap e5: x(active, starting, since 0.00293698s) 2026-02-25T11:43:04.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:04 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:43:04.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:04 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2026-02-25T11:43:04.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:04 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2026-02-25T11:43:04.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:04 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2026-02-25T11:43:04.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:04 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2026-02-25T11:43:04.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:04 trial028 ceph-mon[39224]: Manager daemon x is now available 2026-02-25T11:43:05.187 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout { 2026-02-25T11:43:05.188 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 6, 2026-02-25T11:43:05.188 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "initialized": true 2026-02-25T11:43:05.188 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout } 2026-02-25T11:43:05.188 INFO:teuthology.orchestra.run.trial028.stdout:mgr epoch 4 is available 2026-02-25T11:43:05.188 INFO:teuthology.orchestra.run.trial028.stdout:Verifying orchestrator module is enabled... 2026-02-25T11:43:06.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:06.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: Found migration_current of "None". Setting to last migration. 2026-02-25T11:43:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2026-02-25T11:43:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2026-02-25T11:43:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:06.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:06.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: mgrmap e6: x(active, since 1.00559s) 2026-02-25T11:43:06.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2026-02-25T11:43:06.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2026-02-25T11:43:06.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: [25/Feb/2026:11:43:05] ENGINE Bus STARTING 2026-02-25T11:43:06.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: [25/Feb/2026:11:43:05] ENGINE Serving on https://10.20.193.28:7150 2026-02-25T11:43:06.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: [25/Feb/2026:11:43:05] ENGINE Client ('10.20.193.28', 47276) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)') 2026-02-25T11:43:06.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:06.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:05 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2068688925' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2026-02-25T11:43:06.186 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2026-02-25T11:43:06.186 INFO:teuthology.orchestra.run.trial028.stdout:Setting orchestrator backend to cephadm... 2026-02-25T11:43:07.300 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout value unchanged 2026-02-25T11:43:07.300 INFO:teuthology.orchestra.run.trial028.stdout:Generating ssh key... 2026-02-25T11:43:07.379 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:07 trial028 ceph-mon[39224]: [25/Feb/2026:11:43:05] ENGINE Serving on http://10.20.193.28:8765 2026-02-25T11:43:07.379 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:07 trial028 ceph-mon[39224]: [25/Feb/2026:11:43:05] ENGINE Bus STARTED 2026-02-25T11:43:07.379 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:07 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2068688925' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2026-02-25T11:43:07.379 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:07 trial028 ceph-mon[39224]: mgrmap e7: x(active, since 2s) 2026-02-25T11:43:07.379 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:07 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:07.379 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:07 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:07.886 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: Generating public/private ed25519 key pair. 2026-02-25T11:43:07.886 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: Your identification has been saved in /tmp/tmpx1cehd61/key 2026-02-25T11:43:07.886 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: Your public key has been saved in /tmp/tmpx1cehd61/key.pub 2026-02-25T11:43:07.886 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: The key fingerprint is: 2026-02-25T11:43:07.886 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: SHA256:x82C1zIdBBX09run49DtsHy2GthnWmGPKet5IiyTWSA ceph-123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:07.886 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: The key's randomart image is: 2026-02-25T11:43:07.887 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: +--[ED25519 256]--+ 2026-02-25T11:43:07.887 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: | .+=. | 2026-02-25T11:43:07.887 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: | . . | 2026-02-25T11:43:07.887 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: | . o | 2026-02-25T11:43:07.887 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: | E .o = o . | 2026-02-25T11:43:07.887 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: | .S.B = o.| 2026-02-25T11:43:07.887 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: | o.+o..++| 2026-02-25T11:43:07.887 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: | = .oo==o| 2026-02-25T11:43:07.887 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: | = o .*O+=| 2026-02-25T11:43:07.887 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: | o o+**B+| 2026-02-25T11:43:07.887 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: +----[SHA256]-----+ 2026-02-25T11:43:08.144 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:08 trial028 ceph-mon[39224]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:08.145 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:08 trial028 ceph-mon[39224]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:08.145 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:08 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:08.145 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:08 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:08.149 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIITx7ZWMLLJChya+dmOdkhXMY4wEg91fOmZDgDrWLkzz ceph-123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:08.149 INFO:teuthology.orchestra.run.trial028.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2026-02-25T11:43:08.150 INFO:teuthology.orchestra.run.trial028.stdout:Adding key to root@localhost authorized_keys... 2026-02-25T11:43:08.150 INFO:teuthology.orchestra.run.trial028.stdout:Adding host trial028... 2026-02-25T11:43:09.304 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:09 trial028 ceph-mon[39224]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:09.305 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:09 trial028 ceph-mon[39224]: Generating ssh key... 2026-02-25T11:43:09.305 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:09 trial028 ceph-mon[39224]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:10.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:10 trial028 ceph-mon[39224]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial028", "addr": "10.20.193.28", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:10.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:10 trial028 ceph-mon[39224]: Deploying cephadm binary to trial028 2026-02-25T11:43:10.562 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout Added host 'trial028' with addr '10.20.193.28' 2026-02-25T11:43:10.563 INFO:teuthology.orchestra.run.trial028.stdout:Deploying unmanaged mon service... 2026-02-25T11:43:10.970 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2026-02-25T11:43:10.970 INFO:teuthology.orchestra.run.trial028.stdout:Deploying unmanaged mgr service... 2026-02-25T11:43:11.355 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2026-02-25T11:43:11.650 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:11 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:11.650 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:11 trial028 ceph-mon[39224]: Added host trial028 2026-02-25T11:43:11.650 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:11 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:11.650 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:11 trial028 ceph-mon[39224]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:11.650 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:11 trial028 ceph-mon[39224]: Saving service mon spec with placement count:5 2026-02-25T11:43:11.650 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:11 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:11.650 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:11 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:11.650 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:11 trial028 ceph-mon[39224]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:11.650 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:11 trial028 ceph-mon[39224]: Saving service mgr spec with placement count:2 2026-02-25T11:43:11.650 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:11 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:12.085 INFO:teuthology.orchestra.run.trial028.stdout:Enabling the dashboard module... 2026-02-25T11:43:12.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:12 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:12.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:12 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2255955359' entity='client.admin' 2026-02-25T11:43:12.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:12 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1308784641' entity='client.admin' 2026-02-25T11:43:12.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:12 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:12.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:12 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2796583430' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2026-02-25T11:43:13.448 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:13 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: ignoring --setuser ceph since I am not root 2026-02-25T11:43:13.448 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:13 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: ignoring --setgroup ceph since I am not root 2026-02-25T11:43:13.704 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout { 2026-02-25T11:43:13.704 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "epoch": 8, 2026-02-25T11:43:13.704 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "available": true, 2026-02-25T11:43:13.704 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "active_name": "x", 2026-02-25T11:43:13.704 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "num_standby": 0 2026-02-25T11:43:13.705 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout } 2026-02-25T11:43:13.705 INFO:teuthology.orchestra.run.trial028.stdout:Waiting for the mgr to restart... 2026-02-25T11:43:13.705 INFO:teuthology.orchestra.run.trial028.stdout:Waiting for mgr epoch 8... 2026-02-25T11:43:14.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:14 trial028 ceph-mon[39224]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:14.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:14 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2796583430' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2026-02-25T11:43:14.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:14 trial028 ceph-mon[39224]: mgrmap e8: x(active, since 9s) 2026-02-25T11:43:14.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:14 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/924690553' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2026-02-25T11:43:15.060 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:14 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: /lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-25T11:43:15.060 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:14 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-25T11:43:15.060 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:14 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: from numpy import show_config as show_numpy_config 2026-02-25T11:43:17.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:17 trial028 ceph-mon[39224]: Active manager daemon x restarted 2026-02-25T11:43:17.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:17 trial028 ceph-mon[39224]: Activating manager daemon x 2026-02-25T11:43:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:17 trial028 ceph-mon[39224]: osdmap e3: 0 total, 0 up, 0 in 2026-02-25T11:43:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:17 trial028 ceph-mon[39224]: mgrmap e9: x(active, starting, since 0.0028656s) 2026-02-25T11:43:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:43:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2026-02-25T11:43:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2026-02-25T11:43:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2026-02-25T11:43:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2026-02-25T11:43:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:17 trial028 ceph-mon[39224]: Manager daemon x is now available 2026-02-25T11:43:18.125 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout { 2026-02-25T11:43:18.125 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 10, 2026-02-25T11:43:18.125 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout "initialized": true 2026-02-25T11:43:18.125 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout } 2026-02-25T11:43:18.125 INFO:teuthology.orchestra.run.trial028.stdout:mgr epoch 8 is available 2026-02-25T11:43:18.125 INFO:teuthology.orchestra.run.trial028.stdout:Using certmgr to generate dashboard self-signed certificate... 2026-02-25T11:43:18.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:18.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2026-02-25T11:43:18.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2026-02-25T11:43:18.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:18.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:18 trial028 ceph-mon[39224]: mgrmap e10: x(active, since 1.00624s) 2026-02-25T11:43:18.991 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIFHzCCAwegAwIBAgIUA9XfubCk5l8U9HZPSVp74NmLgaEwDQYJKoZIhvcNAQEL\nBQAwPDE6MDgGA1UEAwwxY2VwaGFkbS1yb290LTEyM2ZhN2JmLTEyM2YtMTFmMS04\nMGRhLWQ0MDRlNmU3ZDQ2MDAeFw0yNjAyMjUxMTQzMThaFw0yOTAyMjQxMTQzMTha\nMBcxFTATBgNVBAMMDDEwLjIwLjE5My4yODCCAiIwDQYJKoZIhvcNAQEBBQADggIP\nADCCAgoCggIBAN/9ZbTLVS43At1gkkTvNzKo1231RAQLqd+vmOnFEJ342C/wEe1h\nTZkVPZG3miQQr8wboiQ4uNeD+PoEY5tFuS3QMHuxZXcvYWzWlw+pJQc8twpiMkRj\nNIMuwLdy+aB71XVkcDJcwANQFj66Aa3XED1bjWZNXdDjFwFSi1awgI8NoTWv2vUy\netHUDJ1zLc+ewekb4Dp1gH90fGjMCUSGGh4m1ew4UBN+6g50yAhUXQBiDNNZybzM\nDn4MUXrYRD2Q5H31SqzrPCTWBJ6GnsO69RDdmrtQ00M3uNlpr3R0t7iW6FCQtLJY\n03y8MNz0ywPZXV4vhw50t3cEZY4XzLwt879JAar6o21+gO20wrhAneD2LM+w8zs9\nBIBTE7nEkjR1u9sLkp4swKFExWrStVdZGrzxBlysSKS69RqJwo9l779BrG6hmEbP\n4t/aqoRUBEWJnfbwa2x82uZGPya3DZOhMufzNuJcbF0jgTqcJGe8p++B/0K2GC+I\nmj2PeNccbjOTpUwvWB21sIwvGvIY7tGdTs+PJmQtn/ee9pa2RVb9lLf4tK9NMECI\nm/COVKri4xsVoOSK7Z+U58eCnzem9re1+1gqlCc94qNMNaLvJBfB/SsRzeE1C2zx\n2SX4e+v4g4pkMucoQkIz8cBObmY34HnUHk+jeBwbTaEOb+39siuJX4tBAgMBAAGj\nPjA8MCwGA1UdEQQlMCOCCHRyaWFsMDI4ghFkYXNoYm9hcmRfc2VydmVyc4cEChTB\nHDAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQB1N/wLD/7YVoK6Fkt9\nBOBmJiJmXBmhCOaOu4SaemYa5niZeu1sCbBbMr9bBWLP5piDMkpD34I3luo0TEax\nX/xjz24S08yTOxVFZsml/AL1mn7eDThxM/zwk394bVLXjenLo69kHQ3xKh6mD8c0\n6CeupuAjzr1eOUv8EUkeJWMG0u5e3Oh+Fbe5oFHbP/LOdDH0SN0OiAkzKs+JuKbP\n+L5eHfVuPueHTwvnCZulYKbI1BueyCFPfToTt3NXdP0ORPCyri8m+TizFUI3oSt0\nXPSJb2ntgIvV/ZTKJFJGoyj4wF5rQkRbCkwLFu5uhAySebS9E7FHAbzP8wspgJWD\nJ0qR32r9tKKYxqlkwer3gFAP+QAts3mqLR6LRCFdyZbSM5PHqNUIPZMAFIivrv41\nmGSU+BYnXmTcPTzeOg63huszOkfpl0qCJdpYxoWq2yOYEQ9e8TdTvTU1O/cQ9Dtp\ns8BcMWUgkN3AJwhzrN5Af5nWm9mwK4/4EQYdcalZv3pvl0IKMDyxNO41uxC6nh4F\nCCJEm42N03tp1uvTJ8B7f1hZBgCIr8AMn8CL/MebGkQFA1m6lFcGehlS8k0YNCbl\n99Sxln2om85t2ZkF5DXV31QUNJPgpP/2JoTKGh0YR++mPw2unzgwRJRG6iWfr9W4\nI5eg2sSkblSjzsFOgwDt300H5Q==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEA3/1ltMtVLjcC3WCSRO83MqjXbfVEBAup36+Y6cUQnfjYL/AR\n7WFNmRU9kbeaJBCvzBuiJDi414P4+gRjm0W5LdAwe7Fldy9hbNaXD6klBzy3CmIy\nRGM0gy7At3L5oHvVdWRwMlzAA1AWProBrdcQPVuNZk1d0OMXAVKLVrCAjw2hNa/a\n9TJ60dQMnXMtz57B6RvgOnWAf3R8aMwJRIYaHibV7DhQE37qDnTICFRdAGIM01nJ\nvMwOfgxRethEPZDkffVKrOs8JNYEnoaew7r1EN2au1DTQze42WmvdHS3uJboUJC0\nsljTfLww3PTLA9ldXi+HDnS3dwRljhfMvC3zv0kBqvqjbX6A7bTCuECd4PYsz7Dz\nOz0EgFMTucSSNHW72wuSnizAoUTFatK1V1kavPEGXKxIpLr1GonCj2Xvv0GsbqGY\nRs/i39qqhFQERYmd9vBrbHza5kY/JrcNk6Ey5/M24lxsXSOBOpwkZ7yn74H/QrYY\nL4iaPY941xxuM5OlTC9YHbWwjC8a8hju0Z1Oz48mZC2f9572lrZFVv2Ut/i0r00w\nQIib8I5UquLjGxWg5Irtn5Tnx4KfN6b2t7X7WCqUJz3io0w1ou8kF8H9KxHN4TUL\nbPHZJfh76/iDimQy5yhCQjPxwE5uZjfgedQeT6N4HBtNoQ5v7f2yK4lfi0ECAwEA\nAQKCAgAA6frd/5pYBMraZz62DOSGmJjfUbHftwooDc50wd5lmV+VFZ6GW55FXSBf\nDJTqo7n6qhBDs+q8zATEVQe4tT6ETmMoAk4xW+tWrkcLzgQ29fRqkKI7OE0ZUd/I\n2Nl2AjrZpKOiwDk7KzrBwgy0LFB4WmAEuVUgJ1DRe0Qq4gfGg6kNOfPVmkV9HyuP\ncfobZMLGdSf1d5u++2VHgIGDPN2I8u5GkVfR7kupXCiJvInEDN5f33fv0RSoHuvn\nQZgg0N9GkH1n4HJqhWA7uw2NU5E85r1XIU0xIwpWQmawLcqvEFgzndVLrPI1GhvM\nP0RHvmGVswma33L1I5KoM2KRMBIgh0D0xz0j70IJC4ZByevEbYU6ONaxiOW3yliW\nOPdmJUGQhURzCEH8efQDNuTYhZ8xAD8tfD1m2uGrUPL3eoB72scfqi7fWvgUgAH5\nnLh+1J9IRm66eiyv8AWMHVgKNj5vAyNNiws2JFdODJcQ+Dwctpjfd2RHANvW7FcI\nLw/wYYI57ibZo83P7Q4JY/XzOxdipOykTPweHqOT9RVaApNxw24IbrF44fXlA4iP\n4Pn1hrLnZnCGkx5IhQdssZAY5IN5RrhjrxujRoDel/FxzkBhtfbBUTcQ6aZK60j/\nYlH4+gP28UXAZCPlFi8uwXoNdmI6dRCwPwT4nLhrfB/3GzQzgQKCAQEA83Mt2bV7\nVASzb7IzCaW1PgyEW+iMWqH/aDzS56enW6mDMJ8DYMN4bakBrHR9oDGxx60HylPg\nx6jDmPsyBIdruuPBDtTJiqWoX9IdO2SDjQL8FX/v4YSOeus6lMSei+JAEB4ukUnV\nEfiJWuPepLwIbARqc/FNChngErxNsI13o18P1bhQsVgZ+/1YLBT/aWiLApv6BI/o\nUOFjgm1cXilaWtG0R5mAP8cXqKyWREjQvH7DgE0BOh0qcNzRr7KdSugMiUEuu/jX\ncikqLT/ET5JEtxn9nS6d4bE0vMdF7WIVhPTNtU61wpAjZvtz0mMI6qwrxYxhEywn\ncxdl2Z3AB8O5gQKCAQEA64lnB24G2FJioMt5qV8iZPpDo/qx6Jbx4EMk+dj9C8Aj\n7wYLeAs8IkD4px2WCkFPjBPuKGFMCHR9t5gX17Lc6o6jMrrWQW1KTWIMPFCNiZAV\nlpM0ELtdO7gRZJBfHZT590lpG3AiKBxt7SeHagU7FP/5Z85n81Yb81VpN5aJ8L16\nBHxyLmhKXDHzfndtBPxfvqEVmnHq1Pnes7CkAzcq8ObtOUi4H0hvPtI9iWnPbLhQ\nTtSZAvYoZmXxAlcoLYxDvcO5KhaKosqFO18N9eU28b9kPhBZpgteCRBmvNhUJCYa\nTIyDaEwkE0V5IcA12GftOtUwxhRGrBUpGiaYuc8xwQKCAQEAs0Cojve4oil/8YtV\nVxIjPFe9EWtOBOHpDFhUtmtUqAJNbXx34USsJJpfeBqh8VUF4M8/wDM4e+qAcuDq\neDk6EnbmTMRSzXqiWkN8cKHW1vpOoIn+F6AhxNH05VtMnhTuNK/0e9fQQhYtFLZZ\n1hP5Y4dw1oPixMA+o6zwy0GuyKUTJLs89we2wjtNM9AWLAfk6vvsAHXxibQbMArU\nCyUJuXtU8a9Npigy/GhvkEHJKDFyQ8LvV1noSQfegS7T8DaiHGHZRmC4Sf+TZZPh\nOPGrFOLXQWf0qhxvh1x22Y3x6F7UJF6bKHI3Dvmw00VGaHxzElcFlzdYqL6eqLgl\nj9IGgQKCAQAu60URztIgP3EUNNQ8cA4K7WuR+7t66K6wDNAN0TK/J3xA/gRE2oAV\nFroxmAwuaMouzj2P36nkuUfdzbdaKXAmbVmHHn7Vs66scY6caTN4prIWN6aUeq5c\n2hLk5Hl+DBIrN31vkMvND7S6HDi2XuphCYimLBC7AXk55Na70woRYb0vrZaVNgMy\nEGU/2QxnhNqRv8bs6CpuWIQLaXGSjmfIa0fwNFMxMB2w9vJ5MBG4x5J/0YvVcjGk\nDESRQcloIVvRLm7xOmJNgWg+5nWCeTGTwJt4XYtuLy+BQLomfjVsqgZtH0SB/0Dk\ncd9qYL6ZKEKyiItZuepGF0UDx5J+2Q5BAoIBABpkMUAOsxwNi4To4kiKNZI5Yx4V\nC+GRowbAJqsuuFwZ+pjXIqP3OncWxbJyRckLjj/s9JaEztV3F5vz+J8jVBJIq9qD\nk9czDCzSUNkfJms6DrVVBwUgpr+N1tPN3zaf+3tazWXRIJBEDkdFRS50lJjgETVV\n8VDdoe1HYn3bJlqp5/UpJGxOggW7dySnqYIs4v8IGdkweu69HwFEDgUMhVmPBmDr\nv37iASkz3f4RcEurtnr57nCuYi45NB8W2iPG1Q3OqEXhGX3FTF4+3+3PLhsJdUO6\n1NAhF1j+RetzRTxlzffb1mTdxNWUsxdgam9InJPmUR9WRYjmTg6Kh88p0IM=\n-----END RSA PRIVATE KEY-----\n"} 2026-02-25T11:43:19.174 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:19 trial028 ceph-mon[39224]: [25/Feb/2026:11:43:17] ENGINE Bus STARTING 2026-02-25T11:43:19.174 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:19 trial028 ceph-mon[39224]: [25/Feb/2026:11:43:17] ENGINE Serving on https://10.20.193.28:7150 2026-02-25T11:43:19.174 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:19 trial028 ceph-mon[39224]: [25/Feb/2026:11:43:17] ENGINE Client ('10.20.193.28', 55036) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)') 2026-02-25T11:43:19.175 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:19 trial028 ceph-mon[39224]: [25/Feb/2026:11:43:17] ENGINE Serving on http://10.20.193.28:8765 2026-02-25T11:43:19.175 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:19 trial028 ceph-mon[39224]: [25/Feb/2026:11:43:17] ENGINE Bus STARTED 2026-02-25T11:43:19.175 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:19 trial028 ceph-mon[39224]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2026-02-25T11:43:19.175 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:19 trial028 ceph-mon[39224]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2026-02-25T11:43:19.175 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:19 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:19.357 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout SSL certificate updated 2026-02-25T11:43:19.731 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2026-02-25T11:43:19.731 INFO:teuthology.orchestra.run.trial028.stdout:Creating initial admin user... 2026-02-25T11:43:20.341 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$8p9gSJGHhsuLBXoD5dpnUuq53yDU0fmqI6fSs6kDGDwg0S4ZFeVLC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1772019800, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2026-02-25T11:43:20.341 INFO:teuthology.orchestra.run.trial028.stdout:Fetching dashboard port number... 2026-02-25T11:43:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:20 trial028 ceph-mon[39224]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:20 trial028 ceph-mon[39224]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:20.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:20.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:20 trial028 ceph-mon[39224]: mgrmap e11: x(active, since 2s) 2026-02-25T11:43:20.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:20.697 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stdout 8443 2026-02-25T11:43:20.713 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2026-02-25T11:43:20.714 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout disabled 2026-02-25T11:43:20.725 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2026-02-25T11:43:20.725 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout inactive 2026-02-25T11:43:20.725 INFO:teuthology.orchestra.run.trial028.stdout:firewalld.service is not enabled 2026-02-25T11:43:20.725 INFO:teuthology.orchestra.run.trial028.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2026-02-25T11:43:20.726 INFO:teuthology.orchestra.run.trial028.stdout:Ceph Dashboard is now available at: 2026-02-25T11:43:20.726 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:43:20.726 INFO:teuthology.orchestra.run.trial028.stdout: URL: https://trial028:8443/ 2026-02-25T11:43:20.727 INFO:teuthology.orchestra.run.trial028.stdout: User: admin 2026-02-25T11:43:20.727 INFO:teuthology.orchestra.run.trial028.stdout: Password: fkguwlgc35 2026-02-25T11:43:20.727 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:43:20.727 INFO:teuthology.orchestra.run.trial028.stdout:Saving cluster configuration to /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config directory 2026-02-25T11:43:21.143 INFO:teuthology.orchestra.run.trial028.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2026-02-25T11:43:21.143 INFO:teuthology.orchestra.run.trial028.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2026-02-25T11:43:21.143 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:43:21.143 INFO:teuthology.orchestra.run.trial028.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2026-02-25T11:43:21.143 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:43:21.144 INFO:teuthology.orchestra.run.trial028.stdout:Or, if you are only running a single cluster on this host: 2026-02-25T11:43:21.144 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:43:21.144 INFO:teuthology.orchestra.run.trial028.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2026-02-25T11:43:21.144 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:43:21.144 INFO:teuthology.orchestra.run.trial028.stdout:Please consider enabling telemetry to help improve Ceph: 2026-02-25T11:43:21.144 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:43:21.144 INFO:teuthology.orchestra.run.trial028.stdout: ceph telemetry on 2026-02-25T11:43:21.144 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:43:21.144 INFO:teuthology.orchestra.run.trial028.stdout:For more information see: 2026-02-25T11:43:21.144 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:43:21.144 INFO:teuthology.orchestra.run.trial028.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2026-02-25T11:43:21.145 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:43:21.145 INFO:teuthology.orchestra.run.trial028.stdout:Bootstrap complete. 2026-02-25T11:43:21.160 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout static 2026-02-25T11:43:21.170 INFO:teuthology.orchestra.run.trial028.stdout:Non-zero exit code 3 from systemctl is-active logrotate 2026-02-25T11:43:21.171 INFO:teuthology.orchestra.run.trial028.stdout:systemctl: stdout inactive 2026-02-25T11:43:21.171 INFO:teuthology.orchestra.run.trial028.stdout:Enabling the logrotate.timer service to perform daily log rotation. 2026-02-25T11:43:21.392 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:21 trial028 ceph-mon[39224]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:21.392 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:21 trial028 ceph-mon[39224]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:21.392 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:21 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/217289784' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2026-02-25T11:43:21.393 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:21 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2068748540' entity='client.admin' 2026-02-25T11:43:21.416 INFO:tasks.cephadm:Fetching config... 2026-02-25T11:43:21.416 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:43:21.416 DEBUG:teuthology.orchestra.run.trial028:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2026-02-25T11:43:21.456 INFO:tasks.cephadm:Fetching client.admin keyring... 2026-02-25T11:43:21.456 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:43:21.456 DEBUG:teuthology.orchestra.run.trial028:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2026-02-25T11:43:21.533 INFO:tasks.cephadm:Fetching mon keyring... 2026-02-25T11:43:21.533 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:43:21.533 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/keyring of=/dev/stdout 2026-02-25T11:43:21.638 INFO:tasks.cephadm:Fetching pub ssh key... 2026-02-25T11:43:21.638 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:43:21.638 DEBUG:teuthology.orchestra.run.trial028:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2026-02-25T11:43:21.716 INFO:tasks.cephadm:Installing pub ssh key for root users... 2026-02-25T11:43:21.717 DEBUG:teuthology.orchestra.run.trial028:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIITx7ZWMLLJChya+dmOdkhXMY4wEg91fOmZDgDrWLkzz ceph-123fa7bf-123f-11f1-80da-d404e6e7d460' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2026-02-25T11:43:21.835 INFO:teuthology.orchestra.run.trial028.stdout:ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIITx7ZWMLLJChya+dmOdkhXMY4wEg91fOmZDgDrWLkzz ceph-123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:21.858 DEBUG:teuthology.orchestra.run.trial119:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIITx7ZWMLLJChya+dmOdkhXMY4wEg91fOmZDgDrWLkzz ceph-123fa7bf-123f-11f1-80da-d404e6e7d460' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2026-02-25T11:43:21.935 INFO:teuthology.orchestra.run.trial119.stdout:ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIITx7ZWMLLJChya+dmOdkhXMY4wEg91fOmZDgDrWLkzz ceph-123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:21.956 DEBUG:teuthology.orchestra.run.trial167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIITx7ZWMLLJChya+dmOdkhXMY4wEg91fOmZDgDrWLkzz ceph-123fa7bf-123f-11f1-80da-d404e6e7d460' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2026-02-25T11:43:22.034 INFO:teuthology.orchestra.run.trial167.stdout:ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIITx7ZWMLLJChya+dmOdkhXMY4wEg91fOmZDgDrWLkzz ceph-123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:22.056 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2026-02-25T11:43:22.240 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:22.669 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2026-02-25T11:43:22.670 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch client-keyring set client.admin '*' --mode 0755 2026-02-25T11:43:22.861 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:23.254 INFO:tasks.cephadm:Writing (initial) conf and keyring to trial119 2026-02-25T11:43:23.254 DEBUG:teuthology.orchestra.run.trial119:> set -ex 2026-02-25T11:43:23.254 DEBUG:teuthology.orchestra.run.trial119:> dd of=/etc/ceph/ceph.conf 2026-02-25T11:43:23.294 DEBUG:teuthology.orchestra.run.trial119:> set -ex 2026-02-25T11:43:23.294 DEBUG:teuthology.orchestra.run.trial119:> dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-25T11:43:23.370 INFO:tasks.cephadm:Adding host trial119 to orchestrator... 2026-02-25T11:43:23.371 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch host add trial119 2026-02-25T11:43:23.557 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:23.690 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:23 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3763411681' entity='client.admin' 2026-02-25T11:43:23.690 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:23 trial028 ceph-mon[39224]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:23.690 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:23 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:25.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:24 trial028 ceph-mon[39224]: mgrmap e12: x(active, since 6s) 2026-02-25T11:43:25.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:24 trial028 ceph-mon[39224]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial119", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:25.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:25.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:25.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:trial028", "name": "osd_memory_target"} : dispatch 2026-02-25T11:43:25.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:25.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:25.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:24 trial028 ceph-mon[39224]: Updating trial028:/etc/ceph/ceph.conf 2026-02-25T11:43:25.876 INFO:teuthology.orchestra.run.trial028.stdout:Added host 'trial119' with addr '10.20.193.119' 2026-02-25T11:43:25.876 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:25 trial028 ceph-mon[39224]: Updating trial028:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:25.876 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:25 trial028 ceph-mon[39224]: Deploying cephadm binary to trial119 2026-02-25T11:43:25.876 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:25 trial028 ceph-mon[39224]: Updating trial028:/etc/ceph/ceph.client.admin.keyring 2026-02-25T11:43:25.876 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:25 trial028 ceph-mon[39224]: Updating trial028:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.client.admin.keyring 2026-02-25T11:43:25.979 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch host ls --format=json 2026-02-25T11:43:26.165 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:26.471 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:43:26.472 INFO:teuthology.orchestra.run.trial028.stdout:[{"addr": "10.20.193.28", "hostname": "trial028", "labels": [], "status": ""}, {"addr": "10.20.193.119", "hostname": "trial119", "labels": [], "status": ""}] 2026-02-25T11:43:26.573 INFO:tasks.cephadm:Writing (initial) conf and keyring to trial167 2026-02-25T11:43:26.573 DEBUG:teuthology.orchestra.run.trial167:> set -ex 2026-02-25T11:43:26.573 DEBUG:teuthology.orchestra.run.trial167:> dd of=/etc/ceph/ceph.conf 2026-02-25T11:43:26.614 DEBUG:teuthology.orchestra.run.trial167:> set -ex 2026-02-25T11:43:26.615 DEBUG:teuthology.orchestra.run.trial167:> dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-25T11:43:26.699 INFO:tasks.cephadm:Adding host trial167 to orchestrator... 2026-02-25T11:43:26.699 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch host add trial167 2026-02-25T11:43:26.750 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:26 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:26.750 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:26 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:26.750 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:26 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:26.750 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:26 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:26.750 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:26 trial028 ceph-mon[39224]: Added host trial119 2026-02-25T11:43:26.750 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:26 trial028 ceph-mon[39224]: Certificate for "cephadm-signed_agent_cert (trial028)" is still valid for 1824 days. 2026-02-25T11:43:26.751 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:26 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:26.751 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:26 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:26.751 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:26 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:26.882 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:28.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:27 trial028 ceph-mon[39224]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:43:28.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:27 trial028 ceph-mon[39224]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial167", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:28.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:27 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:29.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:28 trial028 ceph-mon[39224]: Deploying cephadm binary to trial167 2026-02-25T11:43:29.455 INFO:teuthology.orchestra.run.trial028.stdout:Added host 'trial167' with addr '10.20.193.167' 2026-02-25T11:43:29.562 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch host ls --format=json 2026-02-25T11:43:29.748 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:30.043 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:43:30.043 INFO:teuthology.orchestra.run.trial028.stdout:[{"addr": "10.20.193.28", "hostname": "trial028", "labels": [], "status": ""}, {"addr": "10.20.193.119", "hostname": "trial119", "labels": [], "status": ""}, {"addr": "10.20.193.167", "hostname": "trial167", "labels": [], "status": ""}] 2026-02-25T11:43:30.144 INFO:tasks.cephadm:Setting crush tunables to default 2026-02-25T11:43:30.144 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd crush tunables default 2026-02-25T11:43:30.328 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:30.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:30.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:30.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:trial119", "name": "osd_memory_target"} : dispatch 2026-02-25T11:43:30.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:30.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:30.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:31.424 INFO:teuthology.orchestra.run.trial028.stderr:adjusted tunables profile to default 2026-02-25T11:43:31.528 INFO:tasks.cephadm:Adding mon.a on trial028 2026-02-25T11:43:31.528 INFO:tasks.cephadm:Adding mon.b on trial119 2026-02-25T11:43:31.529 INFO:tasks.cephadm:Adding mon.c on trial167 2026-02-25T11:43:31.529 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch apply mon '3;trial028:10.20.193.28=a;trial119:10.20.193.119=b;trial167:10.20.193.167=c' 2026-02-25T11:43:31.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:31 trial028 ceph-mon[39224]: Updating trial119:/etc/ceph/ceph.conf 2026-02-25T11:43:31.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:31 trial028 ceph-mon[39224]: Added host trial167 2026-02-25T11:43:31.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:31 trial028 ceph-mon[39224]: Updating trial119:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:31.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:31 trial028 ceph-mon[39224]: Updating trial119:/etc/ceph/ceph.client.admin.keyring 2026-02-25T11:43:31.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:31 trial028 ceph-mon[39224]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:43:31.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:31 trial028 ceph-mon[39224]: Updating trial119:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.client.admin.keyring 2026-02-25T11:43:31.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:31.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:31.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:31.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:31.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:31 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3825654374' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2026-02-25T11:43:31.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:31.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:31.713 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-25T11:43:31.714 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-25T11:43:32.029 INFO:teuthology.orchestra.run.trial167.stdout:Scheduled mon update... 2026-02-25T11:43:32.172 DEBUG:teuthology.orchestra.run.trial119:mon.b> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.b.service 2026-02-25T11:43:32.177 DEBUG:teuthology.orchestra.run.trial167:mon.c> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.c.service 2026-02-25T11:43:32.181 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2026-02-25T11:43:32.181 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph mon dump -f json 2026-02-25T11:43:32.405 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-25T11:43:32.405 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-25T11:43:32.744 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:43:32.744 INFO:teuthology.orchestra.run.trial167.stdout:{"epoch":1,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","modified":"2026-02-25T11:42:50.233340Z","created":"2026-02-25T11:42:50.233340Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:3300","nonce":0},{"type":"v1","addr":"10.20.193.28:6789","nonce":0}]},"addr":"10.20.193.28:6789/0","public_addr":"10.20.193.28:6789/0","priority":0,"weight":0,"time_added":"2026-02-25T11:42:50:233366+0000","crush_location":[]}],"quorum":[0]} 2026-02-25T11:43:32.744 INFO:teuthology.orchestra.run.trial167.stderr:dumped monmap epoch 1 2026-02-25T11:43:32.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:32 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3825654374' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2026-02-25T11:43:32.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:32 trial028 ceph-mon[39224]: osdmap e4: 0 total, 0 up, 0 in 2026-02-25T11:43:32.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:32 trial028 ceph-mon[39224]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;trial028:10.20.193.28=a;trial119:10.20.193.119=b;trial167:10.20.193.167=c", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:32.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:32 trial028 ceph-mon[39224]: Saving service mon spec with placement trial028:10.20.193.28=a;trial119:10.20.193.119=b;trial167:10.20.193.167=c;count:3 2026-02-25T11:43:32.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:32 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:33.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:33 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:33.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:33 trial028 ceph-mon[39224]: from='client.? 10.20.193.167:0/4106283337' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-25T11:43:33.887 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2026-02-25T11:43:33.887 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph mon dump -f json 2026-02-25T11:43:34.078 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-25T11:43:34.078 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /etc/ceph/ceph.conf 2026-02-25T11:43:34.417 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:43:34.417 INFO:teuthology.orchestra.run.trial167.stdout:{"epoch":1,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","modified":"2026-02-25T11:42:50.233340Z","created":"2026-02-25T11:42:50.233340Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:3300","nonce":0},{"type":"v1","addr":"10.20.193.28:6789","nonce":0}]},"addr":"10.20.193.28:6789/0","public_addr":"10.20.193.28:6789/0","priority":0,"weight":0,"time_added":"2026-02-25T11:42:50:233366+0000","crush_location":[]}],"quorum":[0]} 2026-02-25T11:43:34.417 INFO:teuthology.orchestra.run.trial167.stderr:dumped monmap epoch 1 2026-02-25T11:43:35.524 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2026-02-25T11:43:35.524 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph mon dump -f json 2026-02-25T11:43:35.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:35.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:35.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:35.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:35.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:trial167", "name": "osd_memory_target"} : dispatch 2026-02-25T11:43:35.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:35.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:35.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:35 trial028 ceph-mon[39224]: Updating trial167:/etc/ceph/ceph.conf 2026-02-25T11:43:35.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:35 trial028 ceph-mon[39224]: from='client.? 10.20.193.167:0/2932256792' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-25T11:43:35.712 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:36.055 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:43:36.056 INFO:teuthology.orchestra.run.trial167.stdout:{"epoch":1,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","modified":"2026-02-25T11:42:50.233340Z","created":"2026-02-25T11:42:50.233340Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:3300","nonce":0},{"type":"v1","addr":"10.20.193.28:6789","nonce":0}]},"addr":"10.20.193.28:6789/0","public_addr":"10.20.193.28:6789/0","priority":0,"weight":0,"time_added":"2026-02-25T11:42:50:233366+0000","crush_location":[]}],"quorum":[0]} 2026-02-25T11:43:36.056 INFO:teuthology.orchestra.run.trial167.stderr:dumped monmap epoch 1 2026-02-25T11:43:36.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:36 trial028 ceph-mon[39224]: Updating trial167:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:36.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:36 trial028 ceph-mon[39224]: Updating trial167:/etc/ceph/ceph.client.admin.keyring 2026-02-25T11:43:36.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:36 trial028 ceph-mon[39224]: Updating trial167:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.client.admin.keyring 2026-02-25T11:43:36.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:36.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:36.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:36.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-25T11:43:36.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:36.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:36 trial028 ceph-mon[39224]: from='client.? 10.20.193.167:0/219839841' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-25T11:43:37.210 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2026-02-25T11:43:37.210 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph mon dump -f json 2026-02-25T11:43:37.440 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.c/config 2026-02-25T11:43:37.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:37 trial028 ceph-mon[39224]: Deploying daemon mon.c on trial167 2026-02-25T11:43:38.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:37 trial167 ceph-mon[43392]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2026-02-25T11:43:39.743 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 systemd[1]: Started Ceph mon.b for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:43:40.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: set uid:gid to 167:167 (ceph:ceph) 2026-02-25T11:43:40.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev - RelWithDebInfo), process ceph-mon, pid 2 2026-02-25T11:43:40.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: pidfile_write: ignore empty --pid-file 2026-02-25T11:43:40.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: load: jerasure load: lrc 2026-02-25T11:43:40.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: RocksDB version: 7.9.2 2026-02-25T11:43:40.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Git sha 0 2026-02-25T11:43:40.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Compile date 2026-02-24 16:18:01 2026-02-25T11:43:40.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: DB SUMMARY 2026-02-25T11:43:40.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: DB Session ID: I1103WFJL2XI8L1AQQPY 2026-02-25T11:43:40.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: CURRENT file: CURRENT 2026-02-25T11:43:40.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: IDENTITY file: IDENTITY 2026-02-25T11:43:40.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2026-02-25T11:43:40.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2026-02-25T11:43:40.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 519 ; 2026-02-25T11:43:40.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.error_if_exists: 0 2026-02-25T11:43:40.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.create_if_missing: 0 2026-02-25T11:43:40.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.paranoid_checks: 1 2026-02-25T11:43:40.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.flush_verify_memtable_count: 1 2026-02-25T11:43:40.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2026-02-25T11:43:40.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2026-02-25T11:43:40.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.env: 0xed6d60 2026-02-25T11:43:40.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.fs: PosixFileSystem 2026-02-25T11:43:40.013 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.info_log: 0x19858240 2026-02-25T11:43:40.013 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_file_opening_threads: 16 2026-02-25T11:43:40.013 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.statistics: (nil) 2026-02-25T11:43:40.013 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.use_fsync: 0 2026-02-25T11:43:40.013 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_log_file_size: 0 2026-02-25T11:43:40.013 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_manifest_file_size: 1073741824 2026-02-25T11:43:40.013 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.log_file_time_to_roll: 0 2026-02-25T11:43:40.013 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.keep_log_file_num: 1000 2026-02-25T11:43:40.013 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.recycle_log_file_num: 0 2026-02-25T11:43:40.013 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.allow_fallocate: 1 2026-02-25T11:43:40.013 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.allow_mmap_reads: 0 2026-02-25T11:43:40.013 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.allow_mmap_writes: 0 2026-02-25T11:43:40.013 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.use_direct_reads: 0 2026-02-25T11:43:40.014 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2026-02-25T11:43:40.014 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.create_missing_column_families: 0 2026-02-25T11:43:40.014 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.db_log_dir: 2026-02-25T11:43:40.014 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.wal_dir: 2026-02-25T11:43:40.014 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.table_cache_numshardbits: 6 2026-02-25T11:43:40.014 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.WAL_ttl_seconds: 0 2026-02-25T11:43:40.014 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.WAL_size_limit_MB: 0 2026-02-25T11:43:40.014 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2026-02-25T11:43:40.014 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.manifest_preallocation_size: 4194304 2026-02-25T11:43:40.014 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.is_fd_close_on_exec: 1 2026-02-25T11:43:40.014 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.advise_random_on_open: 1 2026-02-25T11:43:40.014 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.db_write_buffer_size: 0 2026-02-25T11:43:40.014 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.write_buffer_manager: 0x19854000 2026-02-25T11:43:40.015 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.access_hint_on_compaction_start: 1 2026-02-25T11:43:40.015 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.random_access_max_buffer_size: 1048576 2026-02-25T11:43:40.015 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.use_adaptive_mutex: 0 2026-02-25T11:43:40.015 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.rate_limiter: (nil) 2026-02-25T11:43:40.015 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2026-02-25T11:43:40.015 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.wal_recovery_mode: 2 2026-02-25T11:43:40.015 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.enable_thread_tracking: 0 2026-02-25T11:43:40.015 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.enable_pipelined_write: 0 2026-02-25T11:43:40.015 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.unordered_write: 0 2026-02-25T11:43:40.015 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.allow_concurrent_memtable_write: 1 2026-02-25T11:43:40.015 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2026-02-25T11:43:40.015 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.write_thread_max_yield_usec: 100 2026-02-25T11:43:40.015 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.write_thread_slow_yield_usec: 3 2026-02-25T11:43:40.016 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.row_cache: None 2026-02-25T11:43:40.016 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.wal_filter: None 2026-02-25T11:43:40.016 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.avoid_flush_during_recovery: 0 2026-02-25T11:43:40.016 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.allow_ingest_behind: 0 2026-02-25T11:43:40.016 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.two_write_queues: 0 2026-02-25T11:43:40.016 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.manual_wal_flush: 0 2026-02-25T11:43:40.016 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.wal_compression: 0 2026-02-25T11:43:40.016 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.atomic_flush: 0 2026-02-25T11:43:40.016 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2026-02-25T11:43:40.016 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.persist_stats_to_disk: 0 2026-02-25T11:43:40.016 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.write_dbid_to_manifest: 0 2026-02-25T11:43:40.016 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.log_readahead_size: 0 2026-02-25T11:43:40.017 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.file_checksum_gen_factory: Unknown 2026-02-25T11:43:40.017 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.best_efforts_recovery: 0 2026-02-25T11:43:40.017 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2026-02-25T11:43:40.017 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2026-02-25T11:43:40.017 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.allow_data_in_errors: 0 2026-02-25T11:43:40.017 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.db_host_id: __hostname__ 2026-02-25T11:43:40.017 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.enforce_single_del_contracts: true 2026-02-25T11:43:40.017 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_background_jobs: 2 2026-02-25T11:43:40.017 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_background_compactions: -1 2026-02-25T11:43:40.017 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_subcompactions: 1 2026-02-25T11:43:40.017 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.avoid_flush_during_shutdown: 0 2026-02-25T11:43:40.017 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2026-02-25T11:43:40.018 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.delayed_write_rate : 16777216 2026-02-25T11:43:40.018 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_total_wal_size: 0 2026-02-25T11:43:40.018 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2026-02-25T11:43:40.018 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.stats_dump_period_sec: 600 2026-02-25T11:43:40.018 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.stats_persist_period_sec: 600 2026-02-25T11:43:40.018 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.stats_history_buffer_size: 1048576 2026-02-25T11:43:40.018 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_open_files: -1 2026-02-25T11:43:40.018 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.bytes_per_sync: 0 2026-02-25T11:43:40.018 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.wal_bytes_per_sync: 0 2026-02-25T11:43:40.018 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.strict_bytes_per_sync: 0 2026-02-25T11:43:40.018 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compaction_readahead_size: 0 2026-02-25T11:43:40.018 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_background_flushes: -1 2026-02-25T11:43:40.019 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Compression algorithms supported: 2026-02-25T11:43:40.019 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: kZSTD supported: 0 2026-02-25T11:43:40.019 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: kXpressCompression supported: 0 2026-02-25T11:43:40.019 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: kBZip2Compression supported: 0 2026-02-25T11:43:40.019 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: kZSTDNotFinalCompression supported: 0 2026-02-25T11:43:40.019 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: kLZ4Compression supported: 1 2026-02-25T11:43:40.019 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: kZlibCompression supported: 1 2026-02-25T11:43:40.019 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: kLZ4HCCompression supported: 1 2026-02-25T11:43:40.019 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: kSnappyCompression supported: 1 2026-02-25T11:43:40.019 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Fast CRC32 supported: Supported on x86 2026-02-25T11:43:40.019 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: DMutex implementation: pthread_mutex_t 2026-02-25T11:43:40.019 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2026-02-25T11:43:40.019 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2026-02-25T11:43:40.020 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2026-02-25T11:43:40.020 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.merge_operator: 2026-02-25T11:43:40.020 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compaction_filter: None 2026-02-25T11:43:40.020 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compaction_filter_factory: None 2026-02-25T11:43:40.020 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.sst_partitioner_factory: None 2026-02-25T11:43:40.020 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.memtable_factory: SkipListFactory 2026-02-25T11:43:40.020 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.table_factory: BlockBasedTable 2026-02-25T11:43:40.020 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x19806ac0) 2026-02-25T11:43:40.020 INFO:journalctl@ceph.mon.b.trial119.stdout: cache_index_and_filter_blocks: 1 2026-02-25T11:43:40.020 INFO:journalctl@ceph.mon.b.trial119.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2026-02-25T11:43:40.020 INFO:journalctl@ceph.mon.b.trial119.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2026-02-25T11:43:40.020 INFO:journalctl@ceph.mon.b.trial119.stdout: pin_top_level_index_and_filter: 1 2026-02-25T11:43:40.020 INFO:journalctl@ceph.mon.b.trial119.stdout: index_type: 0 2026-02-25T11:43:40.021 INFO:journalctl@ceph.mon.b.trial119.stdout: data_block_index_type: 0 2026-02-25T11:43:40.021 INFO:journalctl@ceph.mon.b.trial119.stdout: index_shortening: 1 2026-02-25T11:43:40.021 INFO:journalctl@ceph.mon.b.trial119.stdout: data_block_hash_table_util_ratio: 0.750000 2026-02-25T11:43:40.021 INFO:journalctl@ceph.mon.b.trial119.stdout: checksum: 4 2026-02-25T11:43:40.021 INFO:journalctl@ceph.mon.b.trial119.stdout: no_block_cache: 0 2026-02-25T11:43:40.021 INFO:journalctl@ceph.mon.b.trial119.stdout: block_cache: 0x19833a50 2026-02-25T11:43:40.021 INFO:journalctl@ceph.mon.b.trial119.stdout: block_cache_name: BinnedLRUCache 2026-02-25T11:43:40.021 INFO:journalctl@ceph.mon.b.trial119.stdout: block_cache_options: 2026-02-25T11:43:40.021 INFO:journalctl@ceph.mon.b.trial119.stdout: capacity : 536870912 2026-02-25T11:43:40.021 INFO:journalctl@ceph.mon.b.trial119.stdout: num_shard_bits : 4 2026-02-25T11:43:40.021 INFO:journalctl@ceph.mon.b.trial119.stdout: strict_capacity_limit : 0 2026-02-25T11:43:40.021 INFO:journalctl@ceph.mon.b.trial119.stdout: high_pri_pool_ratio: 0.000 2026-02-25T11:43:40.021 INFO:journalctl@ceph.mon.b.trial119.stdout: block_cache_compressed: (nil) 2026-02-25T11:43:40.022 INFO:journalctl@ceph.mon.b.trial119.stdout: persistent_cache: (nil) 2026-02-25T11:43:40.022 INFO:journalctl@ceph.mon.b.trial119.stdout: block_size: 4096 2026-02-25T11:43:40.022 INFO:journalctl@ceph.mon.b.trial119.stdout: block_size_deviation: 10 2026-02-25T11:43:40.022 INFO:journalctl@ceph.mon.b.trial119.stdout: block_restart_interval: 16 2026-02-25T11:43:40.022 INFO:journalctl@ceph.mon.b.trial119.stdout: index_block_restart_interval: 1 2026-02-25T11:43:40.022 INFO:journalctl@ceph.mon.b.trial119.stdout: metadata_block_size: 4096 2026-02-25T11:43:40.022 INFO:journalctl@ceph.mon.b.trial119.stdout: partition_filters: 0 2026-02-25T11:43:40.022 INFO:journalctl@ceph.mon.b.trial119.stdout: use_delta_encoding: 1 2026-02-25T11:43:40.022 INFO:journalctl@ceph.mon.b.trial119.stdout: filter_policy: bloomfilter 2026-02-25T11:43:40.022 INFO:journalctl@ceph.mon.b.trial119.stdout: whole_key_filtering: 1 2026-02-25T11:43:40.022 INFO:journalctl@ceph.mon.b.trial119.stdout: verify_compression: 0 2026-02-25T11:43:40.022 INFO:journalctl@ceph.mon.b.trial119.stdout: read_amp_bytes_per_bit: 0 2026-02-25T11:43:40.022 INFO:journalctl@ceph.mon.b.trial119.stdout: format_version: 5 2026-02-25T11:43:40.023 INFO:journalctl@ceph.mon.b.trial119.stdout: enable_index_compression: 1 2026-02-25T11:43:40.023 INFO:journalctl@ceph.mon.b.trial119.stdout: block_align: 0 2026-02-25T11:43:40.023 INFO:journalctl@ceph.mon.b.trial119.stdout: max_auto_readahead_size: 262144 2026-02-25T11:43:40.023 INFO:journalctl@ceph.mon.b.trial119.stdout: prepopulate_block_cache: 0 2026-02-25T11:43:40.023 INFO:journalctl@ceph.mon.b.trial119.stdout: initial_auto_readahead_size: 8192 2026-02-25T11:43:40.023 INFO:journalctl@ceph.mon.b.trial119.stdout: num_file_reads_for_auto_readahead: 2 2026-02-25T11:43:40.023 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.write_buffer_size: 33554432 2026-02-25T11:43:40.023 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_write_buffer_number: 2 2026-02-25T11:43:40.023 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compression: NoCompression 2026-02-25T11:43:40.023 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.bottommost_compression: Disabled 2026-02-25T11:43:40.023 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.prefix_extractor: nullptr 2026-02-25T11:43:40.023 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2026-02-25T11:43:40.023 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.num_levels: 7 2026-02-25T11:43:40.024 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2026-02-25T11:43:40.024 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2026-02-25T11:43:40.024 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2026-02-25T11:43:40.024 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2026-02-25T11:43:40.024 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.bottommost_compression_opts.level: 32767 2026-02-25T11:43:40.024 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2026-02-25T11:43:40.024 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2026-02-25T11:43:40.024 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2026-02-25T11:43:40.024 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2026-02-25T11:43:40.024 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.bottommost_compression_opts.enabled: false 2026-02-25T11:43:40.027 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2026-02-25T11:43:40.027 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2026-02-25T11:43:40.027 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compression_opts.window_bits: -14 2026-02-25T11:43:40.027 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compression_opts.level: 32767 2026-02-25T11:43:40.027 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compression_opts.strategy: 0 2026-02-25T11:43:40.027 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2026-02-25T11:43:40.027 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2026-02-25T11:43:40.027 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2026-02-25T11:43:40.027 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compression_opts.parallel_threads: 1 2026-02-25T11:43:40.027 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compression_opts.enabled: false 2026-02-25T11:43:40.028 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2026-02-25T11:43:40.028 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2026-02-25T11:43:40.028 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2026-02-25T11:43:40.028 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.level0_stop_writes_trigger: 36 2026-02-25T11:43:40.028 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.target_file_size_base: 67108864 2026-02-25T11:43:40.028 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.target_file_size_multiplier: 1 2026-02-25T11:43:40.028 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_bytes_for_level_base: 268435456 2026-02-25T11:43:40.028 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2026-02-25T11:43:40.028 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2026-02-25T11:43:40.028 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2026-02-25T11:43:40.028 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2026-02-25T11:43:40.028 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2026-02-25T11:43:40.028 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2026-02-25T11:43:40.029 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2026-02-25T11:43:40.029 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2026-02-25T11:43:40.029 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2026-02-25T11:43:40.029 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2026-02-25T11:43:40.029 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_compaction_bytes: 1677721600 2026-02-25T11:43:40.029 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2026-02-25T11:43:40.029 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.arena_block_size: 1048576 2026-02-25T11:43:40.029 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2026-02-25T11:43:40.029 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2026-02-25T11:43:40.029 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.disable_auto_compactions: 0 2026-02-25T11:43:40.029 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2026-02-25T11:43:40.029 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2026-02-25T11:43:40.029 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2026-02-25T11:43:40.030 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2026-02-25T11:43:40.030 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2026-02-25T11:43:40.030 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2026-02-25T11:43:40.030 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2026-02-25T11:43:40.030 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2026-02-25T11:43:40.030 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2026-02-25T11:43:40.030 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2026-02-25T11:43:40.030 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2026-02-25T11:43:40.030 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.inplace_update_support: 0 2026-02-25T11:43:40.030 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.inplace_update_num_locks: 10000 2026-02-25T11:43:40.030 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2026-02-25T11:43:40.030 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.memtable_whole_key_filtering: 0 2026-02-25T11:43:40.030 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.memtable_huge_page_size: 0 2026-02-25T11:43:40.031 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.bloom_locality: 0 2026-02-25T11:43:40.031 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.max_successive_merges: 0 2026-02-25T11:43:40.031 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.optimize_filters_for_hits: 0 2026-02-25T11:43:40.031 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.paranoid_file_checks: 0 2026-02-25T11:43:40.031 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.force_consistency_checks: 1 2026-02-25T11:43:40.031 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.report_bg_io_stats: 0 2026-02-25T11:43:40.031 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.ttl: 2592000 2026-02-25T11:43:40.031 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.periodic_compaction_seconds: 0 2026-02-25T11:43:40.031 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.preclude_last_level_data_seconds: 0 2026-02-25T11:43:40.031 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.preserve_internal_time_seconds: 0 2026-02-25T11:43:40.031 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.enable_blob_files: false 2026-02-25T11:43:40.031 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.min_blob_size: 0 2026-02-25T11:43:40.032 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.blob_file_size: 268435456 2026-02-25T11:43:40.032 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.blob_compression_type: NoCompression 2026-02-25T11:43:40.032 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.enable_blob_garbage_collection: false 2026-02-25T11:43:40.032 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2026-02-25T11:43:40.032 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2026-02-25T11:43:40.032 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.blob_compaction_readahead_size: 0 2026-02-25T11:43:40.032 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.blob_file_starting_level: 0 2026-02-25T11:43:40.032 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2026-02-25T11:43:40.032 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2026-02-25T11:43:40.032 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2026-02-25T11:43:40.032 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 58e58d08-0c4d-4c0f-b089-7a9378f3d199 2026-02-25T11:43:40.032 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: EVENT_LOG_v1 {"time_micros": 1772019819747723, "job": 1, "event": "recovery_started", "wal_files": [4]} 2026-02-25T11:43:40.033 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2026-02-25T11:43:40.033 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: EVENT_LOG_v1 {"time_micros": 1772019819748288, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1655, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 531, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 409, "raw_average_value_size": 81, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1772019819, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "58e58d08-0c4d-4c0f-b089-7a9378f3d199", "db_session_id": "I1103WFJL2XI8L1AQQPY", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2026-02-25T11:43:40.033 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: EVENT_LOG_v1 {"time_micros": 1772019819748403, "job": 1, "event": "recovery_finished"} 2026-02-25T11:43:40.033 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2026-02-25T11:43:40.033 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2026-02-25T11:43:40.033 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x19876e00 2026-02-25T11:43:40.033 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: DB pointer 0x19888000 2026-02-25T11:43:40.033 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2026-02-25T11:43:40.033 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: rocksdb: [db/db_impl/db_impl.cc:1111] 2026-02-25T11:43:40.033 INFO:journalctl@ceph.mon.b.trial119.stdout: ** DB Stats ** 2026-02-25T11:43:40.033 INFO:journalctl@ceph.mon.b.trial119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2026-02-25T11:43:40.034 INFO:journalctl@ceph.mon.b.trial119.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2026-02-25T11:43:40.034 INFO:journalctl@ceph.mon.b.trial119.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2026-02-25T11:43:40.034 INFO:journalctl@ceph.mon.b.trial119.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2026-02-25T11:43:40.034 INFO:journalctl@ceph.mon.b.trial119.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2026-02-25T11:43:40.034 INFO:journalctl@ceph.mon.b.trial119.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2026-02-25T11:43:40.034 INFO:journalctl@ceph.mon.b.trial119.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2026-02-25T11:43:40.034 INFO:journalctl@ceph.mon.b.trial119.stdout: 2026-02-25T11:43:40.034 INFO:journalctl@ceph.mon.b.trial119.stdout: ** Compaction Stats [default] ** 2026-02-25T11:43:40.034 INFO:journalctl@ceph.mon.b.trial119.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2026-02-25T11:43:40.034 INFO:journalctl@ceph.mon.b.trial119.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2026-02-25T11:43:40.034 INFO:journalctl@ceph.mon.b.trial119.stdout: L0 1/0 1.62 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-25T11:43:40.034 INFO:journalctl@ceph.mon.b.trial119.stdout: Sum 1/0 1.62 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-25T11:43:40.035 INFO:journalctl@ceph.mon.b.trial119.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-25T11:43:40.035 INFO:journalctl@ceph.mon.b.trial119.stdout: 2026-02-25T11:43:40.035 INFO:journalctl@ceph.mon.b.trial119.stdout: ** Compaction Stats [default] ** 2026-02-25T11:43:40.035 INFO:journalctl@ceph.mon.b.trial119.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2026-02-25T11:43:40.035 INFO:journalctl@ceph.mon.b.trial119.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2026-02-25T11:43:40.035 INFO:journalctl@ceph.mon.b.trial119.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2026-02-25T11:43:40.035 INFO:journalctl@ceph.mon.b.trial119.stdout: 2026-02-25T11:43:40.035 INFO:journalctl@ceph.mon.b.trial119.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2026-02-25T11:43:40.035 INFO:journalctl@ceph.mon.b.trial119.stdout: 2026-02-25T11:43:40.035 INFO:journalctl@ceph.mon.b.trial119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2026-02-25T11:43:40.035 INFO:journalctl@ceph.mon.b.trial119.stdout: Flush(GB): cumulative 0.000, interval 0.000 2026-02-25T11:43:40.035 INFO:journalctl@ceph.mon.b.trial119.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2026-02-25T11:43:40.036 INFO:journalctl@ceph.mon.b.trial119.stdout: AddFile(Total Files): cumulative 0, interval 0 2026-02-25T11:43:40.036 INFO:journalctl@ceph.mon.b.trial119.stdout: AddFile(L0 Files): cumulative 0, interval 0 2026-02-25T11:43:40.036 INFO:journalctl@ceph.mon.b.trial119.stdout: AddFile(Keys): cumulative 0, interval 0 2026-02-25T11:43:40.036 INFO:journalctl@ceph.mon.b.trial119.stdout: Cumulative compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2026-02-25T11:43:40.036 INFO:journalctl@ceph.mon.b.trial119.stdout: Interval compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2026-02-25T11:43:40.036 INFO:journalctl@ceph.mon.b.trial119.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2026-02-25T11:43:40.036 INFO:journalctl@ceph.mon.b.trial119.stdout: Block cache BinnedLRUCache@0x19833a50#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2026-02-25T11:43:40.036 INFO:journalctl@ceph.mon.b.trial119.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2026-02-25T11:43:40.036 INFO:journalctl@ceph.mon.b.trial119.stdout: 2026-02-25T11:43:40.036 INFO:journalctl@ceph.mon.b.trial119.stdout: ** File Read Latency Histogram By Level [default] ** 2026-02-25T11:43:40.036 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b does not exist in monmap, will attempt to join an existing cluster 2026-02-25T11:43:40.036 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: using public_addr v2:10.20.193.119:0/0 -> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] 2026-02-25T11:43:40.037 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: starting mon.b rank -1 at public addrs [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] at bind addrs [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:40.037 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(???) e0 preinit fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:40.037 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(synchronizing).mds e1 new map 2026-02-25T11:43:40.037 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(synchronizing).mds e1 print_map 2026-02-25T11:43:40.037 INFO:journalctl@ceph.mon.b.trial119.stdout: e1 2026-02-25T11:43:40.037 INFO:journalctl@ceph.mon.b.trial119.stdout: btime 2026-02-25T11:42:51:962673+0000 2026-02-25T11:43:40.037 INFO:journalctl@ceph.mon.b.trial119.stdout: enable_multiple, ever_enabled_multiple: 1,1 2026-02-25T11:43:40.037 INFO:journalctl@ceph.mon.b.trial119.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2026-02-25T11:43:40.037 INFO:journalctl@ceph.mon.b.trial119.stdout: legacy client fscid: -1 2026-02-25T11:43:40.037 INFO:journalctl@ceph.mon.b.trial119.stdout: 2026-02-25T11:43:40.037 INFO:journalctl@ceph.mon.b.trial119.stdout: No filesystems configured 2026-02-25T11:43:40.037 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2026-02-25T11:43:40.038 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2026-02-25T11:43:40.038 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2026-02-25T11:43:40.038 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2026-02-25T11:43:40.038 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2026-02-25T11:43:40.038 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2026-02-25T11:43:40.038 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2026-02-25T11:43:40.038 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2026-02-25T11:43:40.038 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2026-02-25T11:43:40.038 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2026-02-25T11:43:40.038 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mkfs 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:40.038 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.a is new leader, mons a in quorum (ranks 0) 2026-02-25T11:43:40.038 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.a is new leader, mons a in quorum (ranks 0) 2026-02-25T11:43:40.039 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: monmap epoch 1 2026-02-25T11:43:40.039 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:40.039 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: last_changed 2026-02-25T11:42:50.233340+0000 2026-02-25T11:43:40.039 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: created 2026-02-25T11:42:50.233340+0000 2026-02-25T11:43:40.039 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: min_mon_release 20 (tentacle) 2026-02-25T11:43:40.039 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: election_strategy: 1 2026-02-25T11:43:40.039 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: 0: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.a 2026-02-25T11:43:40.039 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: fsmap 2026-02-25T11:43:40.039 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: osdmap e1: 0 total, 0 up, 0 in 2026-02-25T11:43:40.039 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mgrmap e1: no daemons active 2026-02-25T11:43:40.039 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/68170403' entity='client.admin' cmd={"prefix": "status"} : dispatch 2026-02-25T11:43:40.039 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1173800642' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2026-02-25T11:43:40.039 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1173800642' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2026-02-25T11:43:40.040 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3475089742' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:40.040 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.a is new leader, mons a in quorum (ranks 0) 2026-02-25T11:43:40.040 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: monmap epoch 1 2026-02-25T11:43:40.040 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:40.040 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: last_changed 2026-02-25T11:42:50.233340+0000 2026-02-25T11:43:40.040 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: created 2026-02-25T11:42:50.233340+0000 2026-02-25T11:43:40.040 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: min_mon_release 20 (tentacle) 2026-02-25T11:43:40.040 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: election_strategy: 1 2026-02-25T11:43:40.040 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: 0: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.a 2026-02-25T11:43:40.040 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: fsmap 2026-02-25T11:43:40.040 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: osdmap e1: 0 total, 0 up, 0 in 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mgrmap e1: no daemons active 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/175157633' entity='client.admin' 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/4115412768' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2716198742' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Activating manager daemon x 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mgrmap e2: x(active, starting, since 0.00411603s) 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Manager daemon x is now available 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2026-02-25T11:43:40.041 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14100 10.20.193.28:0/3122814711' entity='mgr.x' 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mgrmap e3: x(active, since 1.00779s) 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2886477994' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1007823634' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3904241517' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3904241517' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mgrmap e4: x(active, since 2s) 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/526933894' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Active manager daemon x restarted 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Activating manager daemon x 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: osdmap e2: 0 total, 0 up, 0 in 2026-02-25T11:43:40.042 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mgrmap e5: x(active, starting, since 0.00293698s) 2026-02-25T11:43:40.043 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:43:40.043 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2026-02-25T11:43:40.043 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2026-02-25T11:43:40.043 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2026-02-25T11:43:40.043 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2026-02-25T11:43:40.043 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Manager daemon x is now available 2026-02-25T11:43:40.043 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.043 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.043 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Found migration_current of "None". Setting to last migration. 2026-02-25T11:43:40.043 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.043 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mgrmap e6: x(active, since 1.00559s) 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: [25/Feb/2026:11:43:05] ENGINE Bus STARTING 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: [25/Feb/2026:11:43:05] ENGINE Serving on https://10.20.193.28:7150 2026-02-25T11:43:40.044 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: [25/Feb/2026:11:43:05] ENGINE Client ('10.20.193.28', 47276) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)') 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2068688925' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: [25/Feb/2026:11:43:05] ENGINE Serving on http://10.20.193.28:8765 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: [25/Feb/2026:11:43:05] ENGINE Bus STARTED 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2068688925' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mgrmap e7: x(active, since 2s) 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Generating ssh key... 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.045 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial028", "addr": "10.20.193.28", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Deploying cephadm binary to trial028 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Added host trial028 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Saving service mon spec with placement count:5 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Saving service mgr spec with placement count:2 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2255955359' entity='client.admin' 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1308784641' entity='client.admin' 2026-02-25T11:43:40.046 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2796583430' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14118 10.20.193.28:0/3397039409' entity='mgr.x' 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2796583430' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mgrmap e8: x(active, since 9s) 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/924690553' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Active manager daemon x restarted 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Activating manager daemon x 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: osdmap e3: 0 total, 0 up, 0 in 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mgrmap e9: x(active, starting, since 0.0028656s) 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Manager daemon x is now available 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:40.047 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2026-02-25T11:43:40.048 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2026-02-25T11:43:40.048 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.048 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mgrmap e10: x(active, since 1.00624s) 2026-02-25T11:43:40.048 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: [25/Feb/2026:11:43:17] ENGINE Bus STARTING 2026-02-25T11:43:40.048 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: [25/Feb/2026:11:43:17] ENGINE Serving on https://10.20.193.28:7150 2026-02-25T11:43:40.048 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: [25/Feb/2026:11:43:17] ENGINE Client ('10.20.193.28', 55036) lost — peer dropped the TLS connection suddenly, during handshake: (6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:1147)') 2026-02-25T11:43:40.048 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: [25/Feb/2026:11:43:17] ENGINE Serving on http://10.20.193.28:8765 2026-02-25T11:43:40.048 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: [25/Feb/2026:11:43:17] ENGINE Bus STARTED 2026-02-25T11:43:40.048 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2026-02-25T11:43:40.048 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2026-02-25T11:43:40.048 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.048 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mgrmap e11: x(active, since 2s) 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/217289784' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2068748540' entity='client.admin' 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3763411681' entity='client.admin' 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mgrmap e12: x(active, since 6s) 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial119", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.049 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:trial028", "name": "osd_memory_target"} : dispatch 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Updating trial028:/etc/ceph/ceph.conf 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Updating trial028:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Deploying cephadm binary to trial119 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Updating trial028:/etc/ceph/ceph.client.admin.keyring 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Updating trial028:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.client.admin.keyring 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Added host trial119 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Certificate for "cephadm-signed_agent_cert (trial028)" is still valid for 1824 days. 2026-02-25T11:43:40.050 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "trial167", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Deploying cephadm binary to trial167 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:trial119", "name": "osd_memory_target"} : dispatch 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Updating trial119:/etc/ceph/ceph.conf 2026-02-25T11:43:40.051 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Added host trial167 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Updating trial119:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Updating trial119:/etc/ceph/ceph.client.admin.keyring 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Updating trial119:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.client.admin.keyring 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3825654374' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3825654374' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: osdmap e4: 0 total, 0 up, 0 in 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;trial028:10.20.193.28=a;trial119:10.20.193.119=b;trial167:10.20.193.167=c", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Saving service mon spec with placement trial028:10.20.193.28=a;trial119:10.20.193.119=b;trial167:10.20.193.167=c;count:3 2026-02-25T11:43:40.052 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.167:0/4106283337' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:trial167", "name": "osd_memory_target"} : dispatch 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Updating trial167:/etc/ceph/ceph.conf 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.167:0/2932256792' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Updating trial167:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Updating trial167:/etc/ceph/ceph.client.admin.keyring 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Updating trial167:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.client.admin.keyring 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.053 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:40.054 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-25T11:43:40.054 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:40.054 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: from='client.? 10.20.193.167:0/219839841' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-25T11:43:40.054 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: Deploying daemon mon.c on trial167 2026-02-25T11:43:40.054 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:39 trial119 ceph-mon[42710]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2026-02-25T11:43:42.643 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:43:42.643 INFO:teuthology.orchestra.run.trial167.stdout:{"epoch":2,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","modified":"2026-02-25T11:43:37.632685Z","created":"2026-02-25T11:42:50.233340Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:3300","nonce":0},{"type":"v1","addr":"10.20.193.28:6789","nonce":0}]},"addr":"10.20.193.28:6789/0","public_addr":"10.20.193.28:6789/0","priority":0,"weight":0,"time_added":"2026-02-25T11:42:50:233366+0000","crush_location":[]},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:3300","nonce":0},{"type":"v1","addr":"10.20.193.167:6789","nonce":0}]},"addr":"10.20.193.167:6789/0","public_addr":"10.20.193.167:6789/0","priority":0,"weight":0,"time_added":"2026-02-25T11:43:37:632683+0000","crush_location":[]}],"quorum":[0,1]} 2026-02-25T11:43:42.644 INFO:teuthology.orchestra.run.trial167.stderr:dumped monmap epoch 2 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: Deploying daemon mon.b on trial119 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: mon.a calling monitor election 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='client.? 10.20.193.167:0/3039535948' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: mon.c calling monitor election 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: monmap epoch 2 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: last_changed 2026-02-25T11:43:37.632685+0000 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: created 2026-02-25T11:42:50.233340+0000 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: min_mon_release 20 (tentacle) 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: election_strategy: 1 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: 0: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.a 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: 1: [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] mon.c 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: fsmap 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: osdmap e4: 0 total, 0 up, 0 in 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: mgrmap e12: x(active, since 25s) 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: overall HEALTH_OK 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:43.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:43.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: Deploying daemon mon.b on trial119 2026-02-25T11:43:43.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:43:43.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:43.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: mon.a calling monitor election 2026-02-25T11:43:43.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='client.? 10.20.193.167:0/3039535948' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-25T11:43:43.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:43.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:43.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:43.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: mon.c calling monitor election 2026-02-25T11:43:43.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:43.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:43.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:43.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:43.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:43.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:43.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:43.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2026-02-25T11:43:43.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: monmap epoch 2 2026-02-25T11:43:43.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:43.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: last_changed 2026-02-25T11:43:37.632685+0000 2026-02-25T11:43:43.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: created 2026-02-25T11:42:50.233340+0000 2026-02-25T11:43:43.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: min_mon_release 20 (tentacle) 2026-02-25T11:43:43.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: election_strategy: 1 2026-02-25T11:43:43.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: 0: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.a 2026-02-25T11:43:43.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: 1: [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] mon.c 2026-02-25T11:43:43.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: fsmap 2026-02-25T11:43:43.063 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: osdmap e4: 0 total, 0 up, 0 in 2026-02-25T11:43:43.063 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: mgrmap e12: x(active, since 25s) 2026-02-25T11:43:43.063 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: overall HEALTH_OK 2026-02-25T11:43:43.063 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:43.063 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:43.063 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:43.063 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:43.747 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2026-02-25T11:43:43.747 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph mon dump -f json 2026-02-25T11:43:43.931 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.c/config 2026-02-25T11:43:43.972 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:43.973 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:43.973 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:43 trial167 ceph-mon[43392]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:43.973 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:43.973 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:43.973 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:43.973 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:43.973 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:43.973 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:43.973 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:44.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:44.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:44.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:43 trial028 ceph-mon[39224]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:44.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:44.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:44.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:44.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:44.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:44.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:44.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:44.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: Deploying daemon mon.b on trial119 2026-02-25T11:43:44.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:43:44.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:44.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: mon.a calling monitor election 2026-02-25T11:43:44.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='client.? 10.20.193.167:0/3039535948' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-25T11:43:44.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:44.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:44.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:44.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: mon.c calling monitor election 2026-02-25T11:43:44.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:44.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:44.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:44.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:44.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:44.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:44.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:44.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2026-02-25T11:43:44.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: monmap epoch 2 2026-02-25T11:43:44.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:44.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: last_changed 2026-02-25T11:43:37.632685+0000 2026-02-25T11:43:44.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: created 2026-02-25T11:42:50.233340+0000 2026-02-25T11:43:44.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: min_mon_release 20 (tentacle) 2026-02-25T11:43:44.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: election_strategy: 1 2026-02-25T11:43:44.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: 0: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.a 2026-02-25T11:43:44.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: 1: [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] mon.c 2026-02-25T11:43:44.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: fsmap 2026-02-25T11:43:44.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: osdmap e4: 0 total, 0 up, 0 in 2026-02-25T11:43:44.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: mgrmap e12: x(active, since 25s) 2026-02-25T11:43:44.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: overall HEALTH_OK 2026-02-25T11:43:44.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:44.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:44.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:44.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:44.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:44.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:44.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:44.264 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:44.264 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:44.264 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:44.264 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:44.264 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:44.264 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:44.264 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:45.060 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:44 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: 2026-02-25T11:43:44.632+0000 7f016d678640 -1 mgr.server handle_report got status from non-daemon mon.c 2026-02-25T11:43:49.205 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: Updating trial028:/etc/ceph/ceph.conf 2026-02-25T11:43:49.206 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: Updating trial119:/etc/ceph/ceph.conf 2026-02-25T11:43:49.206 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: Updating trial167:/etc/ceph/ceph.conf 2026-02-25T11:43:49.206 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: Updating trial028:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:49.206 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: Updating trial119:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:49.206 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: Updating trial167:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:49.206 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:43:49.206 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.206 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:49.206 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: mon.a calling monitor election 2026-02-25T11:43:49.206 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: mon.c calling monitor election 2026-02-25T11:43:49.207 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.207 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:49.207 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.207 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: mon.b calling monitor election 2026-02-25T11:43:49.207 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.207 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:49.207 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.207 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.207 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2026-02-25T11:43:49.207 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: monmap epoch 3 2026-02-25T11:43:49.207 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:49.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: last_changed 2026-02-25T11:43:43.787068+0000 2026-02-25T11:43:49.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: created 2026-02-25T11:42:50.233340+0000 2026-02-25T11:43:49.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: min_mon_release 20 (tentacle) 2026-02-25T11:43:49.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: election_strategy: 1 2026-02-25T11:43:49.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: 0: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.a 2026-02-25T11:43:49.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: 1: [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] mon.c 2026-02-25T11:43:49.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: 2: [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] mon.b 2026-02-25T11:43:49.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: fsmap 2026-02-25T11:43:49.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: osdmap e4: 0 total, 0 up, 0 in 2026-02-25T11:43:49.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: mgrmap e12: x(active, since 31s) 2026-02-25T11:43:49.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: overall HEALTH_OK 2026-02-25T11:43:49.209 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.209 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.209 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.209 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.209 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.209 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.209 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.209 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: Updating trial028:/etc/ceph/ceph.conf 2026-02-25T11:43:49.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: Updating trial119:/etc/ceph/ceph.conf 2026-02-25T11:43:49.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: Updating trial167:/etc/ceph/ceph.conf 2026-02-25T11:43:49.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: Updating trial028:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:49.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: Updating trial119:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:49.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: Updating trial167:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:49.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:43:49.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:49.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: mon.a calling monitor election 2026-02-25T11:43:49.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: mon.c calling monitor election 2026-02-25T11:43:49.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:49.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: mon.b calling monitor election 2026-02-25T11:43:49.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:49.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2026-02-25T11:43:49.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: monmap epoch 3 2026-02-25T11:43:49.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:49.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: last_changed 2026-02-25T11:43:43.787068+0000 2026-02-25T11:43:49.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: created 2026-02-25T11:42:50.233340+0000 2026-02-25T11:43:49.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: min_mon_release 20 (tentacle) 2026-02-25T11:43:49.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: election_strategy: 1 2026-02-25T11:43:49.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: 0: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.a 2026-02-25T11:43:49.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: 1: [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] mon.c 2026-02-25T11:43:49.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: 2: [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] mon.b 2026-02-25T11:43:49.253 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: fsmap 2026-02-25T11:43:49.253 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: osdmap e4: 0 total, 0 up, 0 in 2026-02-25T11:43:49.253 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: mgrmap e12: x(active, since 31s) 2026-02-25T11:43:49.253 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: overall HEALTH_OK 2026-02-25T11:43:49.253 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.253 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.253 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.253 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.253 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.253 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.253 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.253 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: Updating trial028:/etc/ceph/ceph.conf 2026-02-25T11:43:49.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: Updating trial119:/etc/ceph/ceph.conf 2026-02-25T11:43:49.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: Updating trial167:/etc/ceph/ceph.conf 2026-02-25T11:43:49.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: Updating trial028:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:49.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: Updating trial119:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:49.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: Updating trial167:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:49.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:43:49.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:43:49.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: mon.a calling monitor election 2026-02-25T11:43:49.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: mon.c calling monitor election 2026-02-25T11:43:49.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:49.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: mon.b calling monitor election 2026-02-25T11:43:49.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:49.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:49.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2026-02-25T11:43:49.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: monmap epoch 3 2026-02-25T11:43:49.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:49.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: last_changed 2026-02-25T11:43:43.787068+0000 2026-02-25T11:43:49.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: created 2026-02-25T11:42:50.233340+0000 2026-02-25T11:43:49.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: min_mon_release 20 (tentacle) 2026-02-25T11:43:49.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: election_strategy: 1 2026-02-25T11:43:49.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: 0: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.a 2026-02-25T11:43:49.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: 1: [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] mon.c 2026-02-25T11:43:49.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: 2: [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] mon.b 2026-02-25T11:43:49.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: fsmap 2026-02-25T11:43:49.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: osdmap e4: 0 total, 0 up, 0 in 2026-02-25T11:43:49.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: mgrmap e12: x(active, since 31s) 2026-02-25T11:43:49.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: overall HEALTH_OK 2026-02-25T11:43:49.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:49.263 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:50.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:49 trial119 ceph-mon[42710]: Reconfiguring mon.a (unknown last config time)... 2026-02-25T11:43:50.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-25T11:43:50.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-25T11:43:50.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:50.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:49 trial119 ceph-mon[42710]: Reconfiguring daemon mon.a on trial028 2026-02-25T11:43:50.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:49 trial119 ceph-mon[42710]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:50.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:50.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:50.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-25T11:43:50.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-25T11:43:50.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:50.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:50.147 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:49 trial167 ceph-mon[43392]: Reconfiguring mon.a (unknown last config time)... 2026-02-25T11:43:50.147 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-25T11:43:50.147 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-25T11:43:50.147 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:50.147 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:49 trial167 ceph-mon[43392]: Reconfiguring daemon mon.a on trial028 2026-02-25T11:43:50.148 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:49 trial167 ceph-mon[43392]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:50.148 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:50.148 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:50.148 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-25T11:43:50.148 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-25T11:43:50.148 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:50.148 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:50.268 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:43:50.268 INFO:teuthology.orchestra.run.trial167.stdout:{"epoch":3,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","modified":"2026-02-25T11:43:43.787068Z","created":"2026-02-25T11:42:50.233340Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:3300","nonce":0},{"type":"v1","addr":"10.20.193.28:6789","nonce":0}]},"addr":"10.20.193.28:6789/0","public_addr":"10.20.193.28:6789/0","priority":0,"weight":0,"time_added":"2026-02-25T11:42:50:233366+0000","crush_location":[]},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:3300","nonce":0},{"type":"v1","addr":"10.20.193.167:6789","nonce":0}]},"addr":"10.20.193.167:6789/0","public_addr":"10.20.193.167:6789/0","priority":0,"weight":0,"time_added":"2026-02-25T11:43:37:632683+0000","crush_location":[]},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:3300","nonce":0},{"type":"v1","addr":"10.20.193.119:6789","nonce":0}]},"addr":"10.20.193.119:6789/0","public_addr":"10.20.193.119:6789/0","priority":0,"weight":0,"time_added":"2026-02-25T11:43:43:787066+0000","crush_location":[]}],"quorum":[0,1,2]} 2026-02-25T11:43:50.268 INFO:teuthology.orchestra.run.trial167.stderr:dumped monmap epoch 3 2026-02-25T11:43:50.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:49 trial028 ceph-mon[39224]: Reconfiguring mon.a (unknown last config time)... 2026-02-25T11:43:50.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-25T11:43:50.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-25T11:43:50.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:50.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:49 trial028 ceph-mon[39224]: Reconfiguring daemon mon.a on trial028 2026-02-25T11:43:50.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:49 trial028 ceph-mon[39224]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:50.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:50.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:50.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-25T11:43:50.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-25T11:43:50.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:50.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:43:50.395 INFO:tasks.cephadm:Generating final ceph.conf file... 2026-02-25T11:43:50.396 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config generate-minimal-conf 2026-02-25T11:43:50.579 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:50.881 INFO:teuthology.orchestra.run.trial028.stdout:# minimal ceph.conf for 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:50.881 INFO:teuthology.orchestra.run.trial028.stdout:[global] 2026-02-25T11:43:50.881 INFO:teuthology.orchestra.run.trial028.stdout: fsid = 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:43:50.881 INFO:teuthology.orchestra.run.trial028.stdout: mon_host = [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: Reconfiguring mon.b (monmap changed)... 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: Reconfiguring daemon mon.b on trial119 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: Reconfiguring mon.c (monmap changed)... 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: Reconfiguring daemon mon.c on trial167 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: from='client.? 10.20.193.167:0/2353520573' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:50.959 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:50.960 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:43:50 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: 2026-02-25T11:43:50.781+0000 7f016d678640 -1 mgr.server handle_report got status from non-daemon mon.b 2026-02-25T11:43:50.984 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2026-02-25T11:43:50.984 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:43:50.984 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/etc/ceph/ceph.conf 2026-02-25T11:43:51.043 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:43:51.043 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-25T11:43:51.140 DEBUG:teuthology.orchestra.run.trial119:> set -ex 2026-02-25T11:43:51.140 DEBUG:teuthology.orchestra.run.trial119:> sudo dd of=/etc/ceph/ceph.conf 2026-02-25T11:43:51.192 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: Reconfiguring mon.b (monmap changed)... 2026-02-25T11:43:51.192 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: Reconfiguring daemon mon.b on trial119 2026-02-25T11:43:51.192 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.192 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.192 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: Reconfiguring mon.c (monmap changed)... 2026-02-25T11:43:51.192 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-25T11:43:51.192 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-25T11:43:51.192 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:51.193 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: Reconfiguring daemon mon.c on trial167 2026-02-25T11:43:51.193 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: from='client.? 10.20.193.167:0/2353520573' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-25T11:43:51.193 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.193 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.193 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:51.193 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:51.193 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:51.200 DEBUG:teuthology.orchestra.run.trial119:> set -ex 2026-02-25T11:43:51.200 DEBUG:teuthology.orchestra.run.trial119:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-25T11:43:51.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: Reconfiguring mon.b (monmap changed)... 2026-02-25T11:43:51.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: Reconfiguring daemon mon.b on trial119 2026-02-25T11:43:51.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: Reconfiguring mon.c (monmap changed)... 2026-02-25T11:43:51.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2026-02-25T11:43:51.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2026-02-25T11:43:51.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:51.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: Reconfiguring daemon mon.c on trial167 2026-02-25T11:43:51.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: from='client.? 10.20.193.167:0/2353520573' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2026-02-25T11:43:51.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:51.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:51.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:51.294 DEBUG:teuthology.orchestra.run.trial167:> set -ex 2026-02-25T11:43:51.294 DEBUG:teuthology.orchestra.run.trial167:> sudo dd of=/etc/ceph/ceph.conf 2026-02-25T11:43:51.352 DEBUG:teuthology.orchestra.run.trial167:> set -ex 2026-02-25T11:43:51.353 DEBUG:teuthology.orchestra.run.trial167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2026-02-25T11:43:51.447 INFO:tasks.cephadm:Adding mgr.x on trial028 2026-02-25T11:43:51.448 INFO:tasks.cephadm:Adding mgr.y on trial119 2026-02-25T11:43:51.448 INFO:tasks.cephadm:Adding mgr.z on trial167 2026-02-25T11:43:51.448 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch apply mgr '3;trial028=x;trial119=y;trial167=z' 2026-02-25T11:43:51.669 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.c/config 2026-02-25T11:43:51.973 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: Updating trial028:/etc/ceph/ceph.conf 2026-02-25T11:43:51.973 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: Updating trial119:/etc/ceph/ceph.conf 2026-02-25T11:43:51.974 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: Updating trial167:/etc/ceph/ceph.conf 2026-02-25T11:43:51.974 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: Updating trial119:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:51.974 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: Updating trial167:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:51.974 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: Updating trial028:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:51.974 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/195570301' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:51.974 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.974 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.974 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.974 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.974 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.974 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.974 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:51.974 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:51 trial167 ceph-mon[43392]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:51.974 INFO:teuthology.orchestra.run.trial167.stdout:Scheduled mgr update... 2026-02-25T11:43:52.073 DEBUG:teuthology.orchestra.run.trial119:mgr.y> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mgr.y.service 2026-02-25T11:43:52.076 DEBUG:teuthology.orchestra.run.trial167:mgr.z> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mgr.z.service 2026-02-25T11:43:52.079 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2026-02-25T11:43:52.079 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set client client_mount_timeout 600 2026-02-25T11:43:52.083 INFO:tasks.cephadm:setting config [client] debug_client = 20 2026-02-25T11:43:52.083 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set client debug_client 20 2026-02-25T11:43:52.123 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: Updating trial028:/etc/ceph/ceph.conf 2026-02-25T11:43:52.123 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: Updating trial119:/etc/ceph/ceph.conf 2026-02-25T11:43:52.123 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: Updating trial167:/etc/ceph/ceph.conf 2026-02-25T11:43:52.123 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: Updating trial119:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:52.123 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: Updating trial167:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:52.123 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: Updating trial028:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:52.123 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/195570301' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:52.123 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.123 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.124 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.124 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.124 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.124 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.124 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.124 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:51 trial119 ceph-mon[42710]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:52.127 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2026-02-25T11:43:52.127 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set client debug_ms 1 2026-02-25T11:43:52.129 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: Updating trial028:/etc/ceph/ceph.conf 2026-02-25T11:43:52.129 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: Updating trial119:/etc/ceph/ceph.conf 2026-02-25T11:43:52.129 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: Updating trial167:/etc/ceph/ceph.conf 2026-02-25T11:43:52.130 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: Updating trial119:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:52.130 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: Updating trial167:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:52.130 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: Updating trial028:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/config/ceph.conf 2026-02-25T11:43:52.130 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/195570301' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:52.130 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.130 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.131 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.131 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.131 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.131 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.131 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:52.131 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:51 trial028 ceph-mon[39224]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:52.173 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2026-02-25T11:43:52.173 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set client rados_mon_op_timeout 900 2026-02-25T11:43:52.219 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2026-02-25T11:43:52.220 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set client rados_osd_op_timeout 900 2026-02-25T11:43:52.265 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:52.268 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2026-02-25T11:43:52.268 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds debug_mds 20 2026-02-25T11:43:52.317 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2026-02-25T11:43:52.317 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds debug_mds_balancer 20 2026-02-25T11:43:52.321 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:52.325 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2026-02-25T11:43:52.325 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds debug_ms 2 2026-02-25T11:43:52.368 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:52.374 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2026-02-25T11:43:52.374 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds mds_bal_fragment_size_max 10000 2026-02-25T11:43:52.422 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2026-02-25T11:43:52.423 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds mds_bal_merge_size 5 2026-02-25T11:43:52.429 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:52.433 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2026-02-25T11:43:52.433 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds mds_bal_split_bits 3 2026-02-25T11:43:52.478 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2026-02-25T11:43:52.478 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds mds_bal_split_size 100 2026-02-25T11:43:52.510 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:52.518 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2026-02-25T11:43:52.519 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds mds_debug_frag True 2026-02-25T11:43:52.569 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2026-02-25T11:43:52.570 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds mds_debug_scatterstat True 2026-02-25T11:43:52.601 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:52.606 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2026-02-25T11:43:52.606 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds mds_op_complaint_time 180 2026-02-25T11:43:52.627 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:52.635 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2026-02-25T11:43:52.635 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds mds_verify_scatter True 2026-02-25T11:43:52.684 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2026-02-25T11:43:52.684 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds osd_op_complaint_time 180 2026-02-25T11:43:52.718 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:52.728 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2026-02-25T11:43:52.728 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds rados_mon_op_timeout 900 2026-02-25T11:43:52.872 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2026-02-25T11:43:52.872 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mds rados_osd_op_timeout 900 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.797+0000 7f832b577640 1 -- 10.20.193.28:0/5551116 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f830403fa30 msgr2=0x7f830405fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.797+0000 7f832b577640 1 --2- 10.20.193.28:0/5551116 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f830403fa30 0x7f830405fee0 secure :-1 s=READY pgs=50 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f831c004740 tx=0x7f831c005b40 comp rx=0 tx=0).stop 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.797+0000 7f832b577640 1 -- 10.20.193.28:0/5551116 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f832c060020 msgr2=0x7f832c066dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.798+0000 7f832b577640 1 --2- 10.20.193.28:0/5551116 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f832c060020 0x7f832c066dd0 secure :-1 s=READY pgs=117 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8324006e90 tx=0x7f832400edf0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.798+0000 7f832b577640 1 -- 10.20.193.28:0/5551116 shutdown_connections 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.798+0000 7f832b577640 1 --2- 10.20.193.28:0/5551116 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f830403fa30 0x7f830405fee0 unknown :-1 s=CLOSED pgs=50 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.798+0000 7f832b577640 1 --2- 10.20.193.28:0/5551116 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f832c060020 0x7f832c066dd0 unknown :-1 s=CLOSED pgs=117 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.798+0000 7f832b577640 1 --2- 10.20.193.28:0/5551116 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f832c05f670 0x7f832c0661d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.798+0000 7f832b577640 1 --2- 10.20.193.28:0/5551116 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f832c05dcf0 0x7f832c065b80 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.798+0000 7f832b577640 1 -- 10.20.193.28:0/5551116 >> 10.20.193.28:0/5551116 conn(0x7f832c05b450 msgr2=0x7f832c069f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.798+0000 7f832b577640 1 -- 10.20.193.28:0/5551116 shutdown_connections 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.798+0000 7f832b577640 1 -- 10.20.193.28:0/5551116 wait complete. 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.860+0000 7fa402ffd640 1 -- 10.20.193.28:0/278919255 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fa3f4002e40 con 0x7fa4240bc4e0 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.866+0000 7fa4217fa640 1 -- 10.20.193.28:0/278919255 <== mon.2 v2:10.20.193.119:3300/0 9 ==== config(26 keys) ==== 1081+0+0 (secure 0 0 0) 0x7fa414007e80 con 0x7fa4240bc4e0 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.866+0000 7fa4217fa640 1 -- 10.20.193.28:0/278919255 <== mon.2 v2:10.20.193.119:3300/0 10 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v13)=0 v13) ==== 123+0+0 (secure 0 0 0) 0x7fa414013020 con 0x7fa4240bc4e0 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.868+0000 7fa402ffd640 1 -- 10.20.193.28:0/278919255 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa40803f9a0 msgr2=0x7fa40805fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.868+0000 7fa402ffd640 1 --2- 10.20.193.28:0/278919255 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa40803f9a0 0x7fa40805fe50 secure :-1 s=READY pgs=51 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa41c0098a0 tx=0x7fa41c03a040 comp rx=0 tx=0).stop 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.868+0000 7fa402ffd640 1 -- 10.20.193.28:0/278919255 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa4240bc4e0 msgr2=0x7fa42411d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:52.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.868+0000 7fa402ffd640 1 --2- 10.20.193.28:0/278919255 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa4240bc4e0 0x7fa42411d820 secure :-1 s=READY pgs=6 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa41400c9b0 tx=0x7fa41400ce80 comp rx=0 tx=0).stop 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.869+0000 7fa402ffd640 1 -- 10.20.193.28:0/278919255 shutdown_connections 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.869+0000 7fa402ffd640 1 --2- 10.20.193.28:0/278919255 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa40803f9a0 0x7fa40805fe50 unknown :-1 s=CLOSED pgs=51 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.869+0000 7fa402ffd640 1 --2- 10.20.193.28:0/278919255 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa4240bce90 0x7fa4240a6bf0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.869+0000 7fa402ffd640 1 --2- 10.20.193.28:0/278919255 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa4240bc4e0 0x7fa42411d820 unknown :-1 s=CLOSED pgs=6 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.869+0000 7fa402ffd640 1 --2- 10.20.193.28:0/278919255 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa4240bb350 0x7fa4240a6390 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.869+0000 7fa402ffd640 1 -- 10.20.193.28:0/278919255 >> 10.20.193.28:0/278919255 conn(0x7fa42400c2b0 msgr2=0x7fa42409c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.869+0000 7fa402ffd640 1 -- 10.20.193.28:0/278919255 shutdown_connections 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.869+0000 7fa402ffd640 1 -- 10.20.193.28:0/278919255 wait complete. 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.831+0000 7fad99e2c640 1 -- 10.20.193.28:0/3732729643 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fad9410f380 msgr2=0x7fad9410f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.831+0000 7fad99e2c640 1 --2- 10.20.193.28:0/3732729643 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fad9410f380 0x7fad9410f770 secure :-1 s=READY pgs=7 gs=1 cs=0 l=1 c_cookie=8610456951912272570 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fad84009950 tx=0x7fad8402f1b0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.832+0000 7fad99e2c640 1 -- 10.20.193.28:0/3732729643 shutdown_connections 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.832+0000 7fad99e2c640 1 --2- 10.20.193.28:0/3732729643 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad94110ec0 0x7fad94171850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.832+0000 7fad99e2c640 1 --2- 10.20.193.28:0/3732729643 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fad94110510 0x7fad94110980 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.833+0000 7fad99e2c640 1 --2- 10.20.193.28:0/3732729643 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fad9410f380 0x7fad9410f770 unknown :-1 s=CLOSED pgs=7 gs=1 cs=0 l=1 c_cookie=8610456951912272570 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.833+0000 7fad99e2c640 1 -- 10.20.193.28:0/3732729643 >> 10.20.193.28:0/3732729643 conn(0x7fad940ecad0 msgr2=0x7fad9410cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.833+0000 7fad99e2c640 1 -- 10.20.193.28:0/3732729643 shutdown_connections 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.833+0000 7fad99e2c640 1 -- 10.20.193.28:0/3732729643 wait complete. 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.834+0000 7fad99e2c640 1 Processor -- start 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.834+0000 7fad99e2c640 1 -- start start 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.835+0000 7fad99e2c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410f380 0x7fad940fd320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.835+0000 7fad99e2c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fad94110510 0x7fad94171850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.836+0000 7fad99e2c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fad94110ec0 0x7fad940fdb80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.836+0000 7fad99e2c640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fad940fe720 con 0x7fad9410f380 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.836+0000 7fad99e2c640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fad940fe890 con 0x7fad94110510 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.836+0000 7fad99e2c640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fad940fea50 con 0x7fad94110ec0 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.836+0000 7fad937fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410f380 0x7fad940fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.836+0000 7fad92ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fad94110510 0x7fad94171850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.836+0000 7fad92ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fad94110510 0x7fad94171850 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:51900/0 (socket says 10.20.193.28:51900) 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.836+0000 7fad92ffd640 1 -- 10.20.193.28:0/2789904224 learned_addr learned my addr 10.20.193.28:0/2789904224 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.836+0000 7fad93fff640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fad94110ec0 0x7fad940fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.837+0000 7fad92ffd640 1 -- 10.20.193.28:0/2789904224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fad94110ec0 msgr2=0x7fad940fdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:52.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.837+0000 7fad92ffd640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fad94110ec0 0x7fad940fdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.837+0000 7fad92ffd640 1 -- 10.20.193.28:0/2789904224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410f380 msgr2=0x7fad940fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.837+0000 7fad92ffd640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410f380 0x7fad940fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.837+0000 7fad92ffd640 1 -- 10.20.193.28:0/2789904224 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad84009570 con 0x7fad94110510 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.838+0000 7fad93fff640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fad94110ec0 0x7fad940fdb80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.838+0000 7fad937fe640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410f380 0x7fad940fd320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.838+0000 7fad92ffd640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fad94110510 0x7fad94171850 secure :-1 s=READY pgs=8 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fad8000d950 tx=0x7fad8000de20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.838+0000 7fad90ff9640 1 -- 10.20.193.28:0/2789904224 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fad80004540 con 0x7fad94110510 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.838+0000 7fad90ff9640 1 -- 10.20.193.28:0/2789904224 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(25 keys) ==== 1050+0+0 (secure 0 0 0) 0x7fad80009ca0 con 0x7fad94110510 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.839+0000 7fad90ff9640 1 -- 10.20.193.28:0/2789904224 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fad80010460 con 0x7fad94110510 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.839+0000 7fad99e2c640 1 -- 10.20.193.28:0/2789904224 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad940fed30 con 0x7fad94110510 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.839+0000 7fad99e2c640 1 -- 10.20.193.28:0/2789904224 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad940ff1b0 con 0x7fad94110510 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.841+0000 7fad90ff9640 1 -- 10.20.193.28:0/2789904224 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7fad800097c0 con 0x7fad94110510 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.841+0000 7fad99e2c640 1 -- 10.20.193.28:0/2789904224 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad941754d0 con 0x7fad94110510 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.842+0000 7fad90ff9640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fad6003f9d0 0x7fad6005fe80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.842+0000 7fad90ff9640 1 -- 10.20.193.28:0/2789904224 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7fad80053a90 con 0x7fad94110510 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.842+0000 7fad937fe640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fad6003f9d0 0x7fad6005fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.844+0000 7fad90ff9640 1 -- 10.20.193.28:0/2789904224 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fad80002730 con 0x7fad94110510 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.847+0000 7fad937fe640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fad6003f9d0 0x7fad6005fe80 secure :-1 s=READY pgs=52 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fad84002b30 tx=0x7fad84031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.864+0000 7fad90ff9640 1 -- 10.20.193.28:0/2789904224 <== mon.2 v2:10.20.193.119:3300/0 7 ==== config(26 keys) ==== 1081+0+0 (secure 0 0 0) 0x7fad800533d0 con 0x7fad94110510 2026-02-25T11:43:52.875 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:52.927 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2026-02-25T11:43:52.928 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mgr client_mount_timeout 30 2026-02-25T11:43:52.937 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:52.941 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2026-02-25T11:43:52.941 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mgr debug_client 20 2026-02-25T11:43:52.953 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2026-02-25T11:43:52.953 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mgr debug_mgr 20 2026-02-25T11:43:52.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.976+0000 7f78eaa08640 1 -- 10.20.193.28:0/1575974461 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f78e405e840 msgr2=0x7f78e41346e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:52.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.976+0000 7f78eaa08640 1 --2- 10.20.193.28:0/1575974461 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f78e405e840 0x7f78e41346e0 secure :-1 s=READY pgs=9 gs=1 cs=0 l=1 c_cookie=13244961524169534798 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f78e000aa30 tx=0x7f78e0030270 comp rx=0 tx=0).stop 2026-02-25T11:43:52.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.976+0000 7f78eaa08640 1 -- 10.20.193.28:0/1575974461 shutdown_connections 2026-02-25T11:43:52.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.976+0000 7f78eaa08640 1 --2- 10.20.193.28:0/1575974461 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f78e4155810 0x7f78e4175cc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.976+0000 7f78eaa08640 1 --2- 10.20.193.28:0/1575974461 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f78e4134c70 0x7f78e41550d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.977+0000 7f78eaa08640 1 --2- 10.20.193.28:0/1575974461 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f78e405e840 0x7f78e41346e0 unknown :-1 s=CLOSED pgs=9 gs=1 cs=0 l=1 c_cookie=13244961524169534798 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.977+0000 7f78eaa08640 1 -- 10.20.193.28:0/1575974461 >> 10.20.193.28:0/1575974461 conn(0x7f78e405b890 msgr2=0x7f78e405bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:52.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.977+0000 7f78eaa08640 1 -- 10.20.193.28:0/1575974461 shutdown_connections 2026-02-25T11:43:52.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.977+0000 7f78eaa08640 1 -- 10.20.193.28:0/1575974461 wait complete. 2026-02-25T11:43:52.988 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2026-02-25T11:43:52.988 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mgr debug_ms 1 2026-02-25T11:43:52.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.982+0000 7fad99e2c640 1 -- 10.20.193.28:0/2789904224 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fad9405f660 con 0x7fad94110510 2026-02-25T11:43:52.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.978+0000 7f78eaa08640 1 Processor -- start 2026-02-25T11:43:52.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.978+0000 7f78eaa08640 1 -- start start 2026-02-25T11:43:52.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.979+0000 7f78eaa08640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f78e405e840 0x7f78e4178630 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:52.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.979+0000 7f78eaa08640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f78e4134c70 0x7f78e41346e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:52.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.979+0000 7f78eaa08640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f78e4155810 0x7f78e4178b70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:52.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.979+0000 7f78eaa08640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4179260 con 0x7f78e4134c70 2026-02-25T11:43:52.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.979+0000 7f78eaa08640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f78e41793d0 con 0x7f78e405e840 2026-02-25T11:43:52.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.979+0000 7f78eaa08640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78e4164ff0 con 0x7f78e4155810 2026-02-25T11:43:52.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.979+0000 7f78e9a06640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f78e405e840 0x7f78e4178630 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:52.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.980+0000 7f78ea207640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f78e4155810 0x7f78e4178b70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:52.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.980+0000 7f78ea207640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f78e4155810 0x7f78e4178b70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56392/0 (socket says 10.20.193.28:56392) 2026-02-25T11:43:52.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.980+0000 7f78ea207640 1 -- 10.20.193.28:0/4002562033 learned_addr learned my addr 10.20.193.28:0/4002562033 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:52.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.980+0000 7f78ea207640 1 -- 10.20.193.28:0/4002562033 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f78e405e840 msgr2=0x7f78e4178630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:52.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.980+0000 7f78ea207640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f78e405e840 0x7f78e4178630 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.980+0000 7f78ea207640 1 -- 10.20.193.28:0/4002562033 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f78e4134c70 msgr2=0x7f78e41346e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:52.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.982+0000 7f78e9205640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f78e4134c70 0x7f78e41346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:52.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.982+0000 7f78ea207640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f78e4134c70 0x7f78e41346e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.982+0000 7f78ea207640 1 -- 10.20.193.28:0/4002562033 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e000a650 con 0x7f78e4155810 2026-02-25T11:43:52.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.982+0000 7f78e9205640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f78e4134c70 0x7f78e41346e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:43:52.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.982+0000 7f78ea207640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f78e4155810 0x7f78e4178b70 secure :-1 s=READY pgs=10 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f78d8005db0 tx=0x7f78d800ddf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:52.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.982+0000 7f78d2ffd640 1 -- 10.20.193.28:0/4002562033 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f78d8004500 con 0x7f78e4155810 2026-02-25T11:43:52.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.983+0000 7f78d2ffd640 1 -- 10.20.193.28:0/4002562033 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(26 keys) ==== 1081+0+0 (secure 0 0 0) 0x7f78d8004cf0 con 0x7f78e4155810 2026-02-25T11:43:52.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.983+0000 7f78d2ffd640 1 -- 10.20.193.28:0/4002562033 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f78d80052a0 con 0x7f78e4155810 2026-02-25T11:43:52.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.983+0000 7f78eaa08640 1 -- 10.20.193.28:0/4002562033 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e41652d0 con 0x7f78e4155810 2026-02-25T11:43:52.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.983+0000 7f78eaa08640 1 -- 10.20.193.28:0/4002562033 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e4165750 con 0x7f78e4155810 2026-02-25T11:43:52.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.985+0000 7f78eaa08640 1 -- 10.20.193.28:0/4002562033 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78e417bc40 con 0x7f78e4155810 2026-02-25T11:43:52.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.985+0000 7f78d2ffd640 1 -- 10.20.193.28:0/4002562033 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f78d8005430 con 0x7f78e4155810 2026-02-25T11:43:52.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.985+0000 7fad90ff9640 1 -- 10.20.193.28:0/2789904224 <== mon.2 v2:10.20.193.119:3300/0 8 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fad8001d120 con 0x7fad94110510 2026-02-25T11:43:52.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.985+0000 7fad90ff9640 1 -- 10.20.193.28:0/2789904224 <== mon.2 v2:10.20.193.119:3300/0 9 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7fad9405f660 con 0x7fad94110510 2026-02-25T11:43:52.994 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.985+0000 7f78d2ffd640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f78c003f9d0 0x7f78c005fe80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:52.994 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.986+0000 7f78d2ffd640 1 -- 10.20.193.28:0/4002562033 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f78d80543c0 con 0x7f78e4155810 2026-02-25T11:43:52.994 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.986+0000 7f78e9a06640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f78c003f9d0 0x7f78c005fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:52.994 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.986+0000 7f78d2ffd640 1 -- 10.20.193.28:0/4002562033 <== mon.1 v2:10.20.193.167:3300/0 6 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f78d8056c70 con 0x7f78e4155810 2026-02-25T11:43:52.994 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.987+0000 7f78e9a06640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f78c003f9d0 0x7f78c005fe80 secure :-1 s=READY pgs=53 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f78e0030780 tx=0x7f78e003b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:52.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.988+0000 7f78d2ffd640 1 -- 10.20.193.28:0/4002562033 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f78d80107f0 con 0x7f78e4155810 2026-02-25T11:43:52.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.988+0000 7fad99e2c640 1 -- 10.20.193.28:0/2789904224 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fad6003f9d0 msgr2=0x7fad6005fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:52.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.988+0000 7fad99e2c640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fad6003f9d0 0x7fad6005fe80 secure :-1 s=READY pgs=52 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fad84002b30 tx=0x7fad84031040 comp rx=0 tx=0).stop 2026-02-25T11:43:52.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.989+0000 7fad99e2c640 1 -- 10.20.193.28:0/2789904224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fad94110510 msgr2=0x7fad94171850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:52.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.989+0000 7fad99e2c640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fad94110510 0x7fad94171850 secure :-1 s=READY pgs=8 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fad8000d950 tx=0x7fad8000de20 comp rx=0 tx=0).stop 2026-02-25T11:43:52.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.989+0000 7fad99e2c640 1 -- 10.20.193.28:0/2789904224 shutdown_connections 2026-02-25T11:43:52.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.989+0000 7fad99e2c640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fad6003f9d0 0x7fad6005fe80 unknown :-1 s=CLOSED pgs=52 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.989+0000 7fad99e2c640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fad94110ec0 0x7fad940fdb80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.989+0000 7fad99e2c640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fad94110510 0x7fad94171850 unknown :-1 s=CLOSED pgs=8 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.989+0000 7fad99e2c640 1 --2- 10.20.193.28:0/2789904224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fad9410f380 0x7fad940fd320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:52.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.989+0000 7fad99e2c640 1 -- 10.20.193.28:0/2789904224 >> 10.20.193.28:0/2789904224 conn(0x7fad940ecad0 msgr2=0x7fad940ef440 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.003 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2026-02-25T11:43:53.003 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mgr mon_warn_on_pool_no_app False 2026-02-25T11:43:53.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.991+0000 7fad99e2c640 1 -- 10.20.193.28:0/2789904224 shutdown_connections 2026-02-25T11:43:53.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:52.991+0000 7fad99e2c640 1 -- 10.20.193.28:0/2789904224 wait complete. 2026-02-25T11:43:53.056 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2026-02-25T11:43:53.056 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set mon mon_op_complaint_time 120 2026-02-25T11:43:53.166 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2026-02-25T11:43:53.167 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config set osd osd_op_complaint_time 180 2026-02-25T11:43:53.168 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.168 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;trial028=x;trial119=y;trial167=z", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:53.169 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: Saving service mgr spec with placement trial028=x;trial119=y;trial167=z;count:3 2026-02-25T11:43:53.169 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:53.169 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:53.169 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:53.169 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:53.169 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:53.169 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-25T11:43:53.169 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2026-02-25T11:43:53.169 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-25T11:43:53.169 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:53.169 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: Deploying daemon mgr.z on trial167 2026-02-25T11:43:53.170 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:53.170 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/986921630' entity='client.admin' 2026-02-25T11:43:53.170 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/5551116' entity='client.admin' 2026-02-25T11:43:53.170 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:52 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:53.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.155+0000 7fd4f91be640 1 -- 10.20.193.28:0/559588520 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd4f4116320 msgr2=0x7fd4f4176cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.155+0000 7fd4f91be640 1 --2- 10.20.193.28:0/559588520 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd4f4116320 0x7fd4f4176cb0 secure :-1 s=READY pgs=11 gs=3 cs=0 l=1 c_cookie=16521263513275398784 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd4e800a970 tx=0x7fd4e80321f0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.156+0000 7fd4f91be640 1 -- 10.20.193.28:0/559588520 shutdown_connections 2026-02-25T11:43:53.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.156+0000 7fd4f91be640 1 --2- 10.20.193.28:0/559588520 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd4f4116320 0x7fd4f4176cb0 unknown :-1 s=CLOSED pgs=11 gs=3 cs=0 l=1 c_cookie=16521263513275398784 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.156+0000 7fd4f91be640 1 --2- 10.20.193.28:0/559588520 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd4f4115970 0x7fd4f4115de0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.156+0000 7fd4f91be640 1 --2- 10.20.193.28:0/559588520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd4f405df00 0x7fd4f405e2f0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.156+0000 7fd4f91be640 1 -- 10.20.193.28:0/559588520 >> 10.20.193.28:0/559588520 conn(0x7fd4f405b450 msgr2=0x7fd4f405b850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.156+0000 7fd4f91be640 1 -- 10.20.193.28:0/559588520 shutdown_connections 2026-02-25T11:43:53.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.156+0000 7fd4f91be640 1 -- 10.20.193.28:0/559588520 wait complete. 2026-02-25T11:43:53.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.157+0000 7fd4f91be640 1 Processor -- start 2026-02-25T11:43:53.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.158+0000 7fd4f91be640 1 -- start start 2026-02-25T11:43:53.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.158+0000 7fd4f91be640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd4f405df00 0x7fd4f4120230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.158+0000 7fd4f91be640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd4f4115970 0x7fd4f4120880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.158+0000 7fd4f91be640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd4f4116320 0x7fd4f4121480 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.158+0000 7fd4f3fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd4f405df00 0x7fd4f4120230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.158+0000 7fd4f3fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd4f405df00 0x7fd4f4120230 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:51938/0 (socket says 10.20.193.28:51938) 2026-02-25T11:43:53.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.158+0000 7fd4f3fff640 1 -- 10.20.193.28:0/2488061125 learned_addr learned my addr 10.20.193.28:0/2488061125 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:53.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.159+0000 7fd4f37fe640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd4f4115970 0x7fd4f4120880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.159+0000 7fd4f91be640 1 -- 10.20.193.28:0/2488061125 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4121bc0 con 0x7fd4f4116320 2026-02-25T11:43:53.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.159+0000 7fd4f91be640 1 -- 10.20.193.28:0/2488061125 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4121d30 con 0x7fd4f405df00 2026-02-25T11:43:53.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.159+0000 7fd4f91be640 1 -- 10.20.193.28:0/2488061125 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4121ef0 con 0x7fd4f4115970 2026-02-25T11:43:53.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.159+0000 7fd4f89bd640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd4f4116320 0x7fd4f4121480 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.159+0000 7fd4f3fff640 1 -- 10.20.193.28:0/2488061125 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd4f4115970 msgr2=0x7fd4f4120880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.159+0000 7fd4f3fff640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd4f4115970 0x7fd4f4120880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.159+0000 7fd4f3fff640 1 -- 10.20.193.28:0/2488061125 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd4f4116320 msgr2=0x7fd4f4121480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.160+0000 7fd4f3fff640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd4f4116320 0x7fd4f4121480 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.160+0000 7fd4f3fff640 1 -- 10.20.193.28:0/2488061125 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e00094b0 con 0x7fd4f405df00 2026-02-25T11:43:53.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.160+0000 7fd4f3fff640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd4f405df00 0x7fd4f4120230 secure :-1 s=READY pgs=9 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd4e0009fd0 tx=0x7fd4e0002bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.160+0000 7fd4f17fa640 1 -- 10.20.193.28:0/2488061125 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd4e000ed60 con 0x7fd4f405df00 2026-02-25T11:43:53.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.160+0000 7fd4f17fa640 1 -- 10.20.193.28:0/2488061125 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd4e0002e20 con 0x7fd4f405df00 2026-02-25T11:43:53.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.160+0000 7fd4f17fa640 1 -- 10.20.193.28:0/2488061125 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd4e00188b0 con 0x7fd4f405df00 2026-02-25T11:43:53.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.160+0000 7fd4f91be640 1 -- 10.20.193.28:0/2488061125 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e4009570 con 0x7fd4f405df00 2026-02-25T11:43:53.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.160+0000 7fd4f91be640 1 -- 10.20.193.28:0/2488061125 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e800a590 con 0x7fd4f405df00 2026-02-25T11:43:53.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.161+0000 7fd4f17fa640 1 -- 10.20.193.28:0/2488061125 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7fd4e0016020 con 0x7fd4f405df00 2026-02-25T11:43:53.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.162+0000 7fd4f91be640 1 -- 10.20.193.28:0/2488061125 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f4055e30 con 0x7fd4f405df00 2026-02-25T11:43:53.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.162+0000 7fd4f17fa640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd4d403f9d0 0x7fd4d405fe80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.162+0000 7fd4f17fa640 1 -- 10.20.193.28:0/2488061125 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7fd4e0054440 con 0x7fd4f405df00 2026-02-25T11:43:53.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.163+0000 7f78eaa08640 1 -- 10.20.193.28:0/4002562033 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f78e417bdd0 con 0x7f78e4155810 2026-02-25T11:43:53.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.163+0000 7fd4f37fe640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd4d403f9d0 0x7fd4d405fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.164+0000 7fd4f37fe640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd4d403f9d0 0x7fd4d405fe80 secure :-1 s=READY pgs=54 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd4e4005b30 tx=0x7fd4e4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.165+0000 7fd4f17fa640 1 -- 10.20.193.28:0/2488061125 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fd4e00215e0 con 0x7fd4f405df00 2026-02-25T11:43:53.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.169+0000 7f78d2ffd640 1 -- 10.20.193.28:0/4002562033 <== mon.1 v2:10.20.193.167:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v15)=0 v15) ==== 117+0+0 (secure 0 0 0) 0x7f78d8021be0 con 0x7f78e4155810 2026-02-25T11:43:53.183 DEBUG:tasks.cephadm:set 26 configs 2026-02-25T11:43:53.183 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='trial028') 2026-02-25T11:43:53.183 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='trial028') 2026-02-25T11:43:53.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.172+0000 7f78eaa08640 1 -- 10.20.193.28:0/4002562033 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f78c003f9d0 msgr2=0x7f78c005fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.172+0000 7f78eaa08640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f78c003f9d0 0x7f78c005fe80 secure :-1 s=READY pgs=53 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f78e0030780 tx=0x7f78e003b040 comp rx=0 tx=0).stop 2026-02-25T11:43:53.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.172+0000 7f78eaa08640 1 -- 10.20.193.28:0/4002562033 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f78e4155810 msgr2=0x7f78e4178b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.172+0000 7f78eaa08640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f78e4155810 0x7f78e4178b70 secure :-1 s=READY pgs=10 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f78d8005db0 tx=0x7f78d800ddf0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.172+0000 7f78eaa08640 1 -- 10.20.193.28:0/4002562033 shutdown_connections 2026-02-25T11:43:53.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.172+0000 7f78eaa08640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f78c003f9d0 0x7f78c005fe80 unknown :-1 s=CLOSED pgs=53 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.172+0000 7f78eaa08640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f78e4155810 0x7f78e4178b70 unknown :-1 s=CLOSED pgs=10 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.172+0000 7f78eaa08640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f78e4134c70 0x7f78e41346e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.172+0000 7f78eaa08640 1 --2- 10.20.193.28:0/4002562033 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f78e405e840 0x7f78e4178630 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.172+0000 7f78eaa08640 1 -- 10.20.193.28:0/4002562033 >> 10.20.193.28:0/4002562033 conn(0x7f78e405b890 msgr2=0x7f78e41583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.174+0000 7f78eaa08640 1 -- 10.20.193.28:0/4002562033 shutdown_connections 2026-02-25T11:43:53.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.174+0000 7f78eaa08640 1 -- 10.20.193.28:0/4002562033 wait complete. 2026-02-25T11:43:53.189 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.197 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.198 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;trial028=x;trial119=y;trial167=z", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:53.199 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: Saving service mgr spec with placement trial028=x;trial119=y;trial167=z;count:3 2026-02-25T11:43:53.199 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:53.199 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:53.199 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:53.199 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:53.199 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:53.199 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-25T11:43:53.199 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2026-02-25T11:43:53.199 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-25T11:43:53.199 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:53.199 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: Deploying daemon mgr.z on trial167 2026-02-25T11:43:53.200 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:53.200 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/986921630' entity='client.admin' 2026-02-25T11:43:53.200 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/5551116' entity='client.admin' 2026-02-25T11:43:53.200 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:52 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:53.200 INFO:journalctl@ceph.mgr.z.trial167.stdout:Feb 25 11:43:53 trial167 systemd[1]: Started Ceph mgr.z for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:43:53.200 INFO:journalctl@ceph.mgr.z.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mgr[44952]: -- 10.20.193.167:0/1752102765 <== mon.1 v2:10.20.193.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x2e543a00 con 0x2f2b5800 2026-02-25T11:43:53.213 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;trial028=x;trial119=y;trial167=z", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:43:53.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: Saving service mgr spec with placement trial028=x;trial119=y;trial167=z;count:3 2026-02-25T11:43:53.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:53.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:53.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:53.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:53.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:53.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-25T11:43:53.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2026-02-25T11:43:53.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-25T11:43:53.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:53.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: Deploying daemon mgr.z on trial167 2026-02-25T11:43:53.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:53.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/986921630' entity='client.admin' 2026-02-25T11:43:53.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/5551116' entity='client.admin' 2026-02-25T11:43:53.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:52 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:53.285 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='trial028') 2026-02-25T11:43:53.286 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='trial028') 2026-02-25T11:43:53.286 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='trial028') 2026-02-25T11:43:53.295 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='trial028') 2026-02-25T11:43:53.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.318+0000 7fd4f91be640 1 -- 10.20.193.28:0/2488061125 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fd4f405e340 con 0x7fd4f405df00 2026-02-25T11:43:53.333 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.330+0000 7fd4f17fa640 1 -- 10.20.193.28:0/2488061125 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v16)=0 v16) ==== 108+0+0 (secure 0 0 0) 0x7fd4e001f2f0 con 0x7fd4f405df00 2026-02-25T11:43:53.337 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.333+0000 7fd4f91be640 1 -- 10.20.193.28:0/2488061125 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd4d403f9d0 msgr2=0x7fd4d405fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.337 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.333+0000 7fd4f91be640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd4d403f9d0 0x7fd4d405fe80 secure :-1 s=READY pgs=54 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd4e4005b30 tx=0x7fd4e4009310 comp rx=0 tx=0).stop 2026-02-25T11:43:53.337 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.333+0000 7fd4f91be640 1 -- 10.20.193.28:0/2488061125 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd4f405df00 msgr2=0x7fd4f4120230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.337 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.333+0000 7fd4f91be640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd4f405df00 0x7fd4f4120230 secure :-1 s=READY pgs=9 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd4e0009fd0 tx=0x7fd4e0002bc0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.337 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.334+0000 7fd4f91be640 1 -- 10.20.193.28:0/2488061125 shutdown_connections 2026-02-25T11:43:53.337 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.334+0000 7fd4f91be640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd4d403f9d0 0x7fd4d405fe80 unknown :-1 s=CLOSED pgs=54 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.334+0000 7fd4f91be640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd4f4116320 0x7fd4f4121480 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.334+0000 7fd4f91be640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd4f4115970 0x7fd4f4120880 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.334+0000 7fd4f91be640 1 --2- 10.20.193.28:0/2488061125 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd4f405df00 0x7fd4f4120230 unknown :-1 s=CLOSED pgs=9 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.334+0000 7fd4f91be640 1 -- 10.20.193.28:0/2488061125 >> 10.20.193.28:0/2488061125 conn(0x7fd4f405b450 msgr2=0x7fd4f405c810 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.334+0000 7fd4f91be640 1 -- 10.20.193.28:0/2488061125 shutdown_connections 2026-02-25T11:43:53.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.334+0000 7fd4f91be640 1 -- 10.20.193.28:0/2488061125 wait complete. 2026-02-25T11:43:53.403 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.443 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.442+0000 7fca71b79640 1 -- 10.20.193.28:0/1585734696 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c0609c0 msgr2=0x7fca6c10b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.442+0000 7fca71b79640 1 --2- 10.20.193.28:0/1585734696 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c0609c0 0x7fca6c10b900 secure :-1 s=READY pgs=118 gs=1 cs=0 l=1 c_cookie=4170270536569446945 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca5401ace0 tx=0x7fca54040120 comp rx=0 tx=0).stop 2026-02-25T11:43:53.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.443+0000 7fca71b79640 1 -- 10.20.193.28:0/1585734696 shutdown_connections 2026-02-25T11:43:53.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.443+0000 7fca71b79640 1 --2- 10.20.193.28:0/1585734696 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c10c9d0 0x7fca6c10ce50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.443+0000 7fca71b79640 1 --2- 10.20.193.28:0/1585734696 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c10be40 0x7fca6c10c2b0 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.443+0000 7fca71b79640 1 --2- 10.20.193.28:0/1585734696 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c0609c0 0x7fca6c10b900 unknown :-1 s=CLOSED pgs=118 gs=1 cs=0 l=1 c_cookie=4170270536569446945 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.443+0000 7fca71b79640 1 -- 10.20.193.28:0/1585734696 >> 10.20.193.28:0/1585734696 conn(0x7fca6c0e9580 msgr2=0x7fca6c109990 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.443+0000 7fca71b79640 1 -- 10.20.193.28:0/1585734696 shutdown_connections 2026-02-25T11:43:53.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.443+0000 7fca71b79640 1 -- 10.20.193.28:0/1585734696 wait complete. 2026-02-25T11:43:53.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.443+0000 7fca71b79640 1 Processor -- start 2026-02-25T11:43:53.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.444+0000 7fca71b79640 1 -- start start 2026-02-25T11:43:53.449 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.444+0000 7fca71b79640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c0609c0 0x7fca6c0f5070 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.444+0000 7fca71b79640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c10be40 0x7fca6c0f56c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.444+0000 7fca71b79640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c10c9d0 0x7fca6c0f62c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.444+0000 7fca63fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c10be40 0x7fca6c0f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.444+0000 7fca63fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c10be40 0x7fca6c0f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:51964/0 (socket says 10.20.193.28:51964) 2026-02-25T11:43:53.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.444+0000 7fca71b79640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c0f6a00 con 0x7fca6c10c9d0 2026-02-25T11:43:53.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.444+0000 7fca71b79640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c0f6b70 con 0x7fca6c10be40 2026-02-25T11:43:53.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.444+0000 7fca71b79640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c0f6d30 con 0x7fca6c0609c0 2026-02-25T11:43:53.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.444+0000 7fca70b77640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c0609c0 0x7fca6c0f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.444+0000 7fca70b77640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c0609c0 0x7fca6c0f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56440/0 (socket says 10.20.193.28:56440) 2026-02-25T11:43:53.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.444+0000 7fca70b77640 1 -- 10.20.193.28:0/1042915961 learned_addr learned my addr 10.20.193.28:0/1042915961 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:53.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.444+0000 7fca71378640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c10c9d0 0x7fca6c0f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.445+0000 7fca63fff640 1 -- 10.20.193.28:0/1042915961 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c0609c0 msgr2=0x7fca6c0f5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.445+0000 7fca63fff640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c0609c0 0x7fca6c0f5070 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.445+0000 7fca63fff640 1 -- 10.20.193.28:0/1042915961 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c10c9d0 msgr2=0x7fca6c0f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.445+0000 7fca63fff640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c10c9d0 0x7fca6c0f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.445+0000 7fca63fff640 1 -- 10.20.193.28:0/1042915961 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5401a980 con 0x7fca6c10be40 2026-02-25T11:43:53.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.445+0000 7fca70b77640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c0609c0 0x7fca6c0f5070 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:43:53.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.445+0000 7fca71378640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c10c9d0 0x7fca6c0f62c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:43:53.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.445+0000 7fca63fff640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c10be40 0x7fca6c0f56c0 secure :-1 s=READY pgs=11 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca5c00c520 tx=0x7fca5c00c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.446+0000 7fca61ffb640 1 -- 10.20.193.28:0/1042915961 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fca5c0054a0 con 0x7fca6c10be40 2026-02-25T11:43:53.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.446+0000 7fca71b79640 1 -- 10.20.193.28:0/1042915961 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c0f7010 con 0x7fca6c10be40 2026-02-25T11:43:53.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.446+0000 7fca71b79640 1 -- 10.20.193.28:0/1042915961 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c0f7610 con 0x7fca6c10be40 2026-02-25T11:43:53.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.446+0000 7fca61ffb640 1 -- 10.20.193.28:0/1042915961 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fca5c005600 con 0x7fca6c10be40 2026-02-25T11:43:53.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.446+0000 7fca61ffb640 1 -- 10.20.193.28:0/1042915961 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fca5c010cf0 con 0x7fca6c10be40 2026-02-25T11:43:53.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.446+0000 7fca71b79640 1 -- 10.20.193.28:0/1042915961 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca34005300 con 0x7fca6c10be40 2026-02-25T11:43:53.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.447+0000 7fca61ffb640 1 -- 10.20.193.28:0/1042915961 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7fca5c00e040 con 0x7fca6c10be40 2026-02-25T11:43:53.454 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.447+0000 7fca61ffb640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca5003fa00 0x7fca5005feb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.454 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.448+0000 7fca61ffb640 1 -- 10.20.193.28:0/1042915961 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7fca5c054a50 con 0x7fca6c10be40 2026-02-25T11:43:53.454 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.448+0000 7fca70b77640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca5003fa00 0x7fca5005feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.454 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.448+0000 7fca70b77640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca5003fa00 0x7fca5005feb0 secure :-1 s=READY pgs=55 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca540023d0 tx=0x7fca5404b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.454 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.450+0000 7fca61ffb640 1 -- 10.20.193.28:0/1042915961 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fca5c00d310 con 0x7fca6c10be40 2026-02-25T11:43:53.472 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.469+0000 7f89ae91d640 1 -- 10.20.193.28:0/4058100135 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f89a8114c10 msgr2=0x7f89a8115080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.469+0000 7f89ae91d640 1 --2- 10.20.193.28:0/4058100135 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f89a8114c10 0x7f89a8115080 secure :-1 s=READY pgs=13 gs=2 cs=0 l=1 c_cookie=9164474229862222024 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8990009950 tx=0x7f899002f1b0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.469+0000 7f89ae91d640 1 -- 10.20.193.28:0/4058100135 shutdown_connections 2026-02-25T11:43:53.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.469+0000 7f89ae91d640 1 --2- 10.20.193.28:0/4058100135 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f89a8115820 0x7f89a8115cc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.469+0000 7f89ae91d640 1 --2- 10.20.193.28:0/4058100135 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f89a8114c10 0x7f89a8115080 unknown :-1 s=CLOSED pgs=13 gs=2 cs=0 l=1 c_cookie=9164474229862222024 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.470+0000 7f89ae91d640 1 --2- 10.20.193.28:0/4058100135 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f89a805e4b0 0x7f89a81146d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.470+0000 7f89ae91d640 1 -- 10.20.193.28:0/4058100135 >> 10.20.193.28:0/4058100135 conn(0x7f89a805b5d0 msgr2=0x7f89a805b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.470+0000 7f89ae91d640 1 -- 10.20.193.28:0/4058100135 shutdown_connections 2026-02-25T11:43:53.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.470+0000 7f89ae91d640 1 -- 10.20.193.28:0/4058100135 wait complete. 2026-02-25T11:43:53.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.471+0000 7f89ae91d640 1 Processor -- start 2026-02-25T11:43:53.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.471+0000 7f89ae91d640 1 -- start start 2026-02-25T11:43:53.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.471+0000 7f89ae91d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f89a805e4b0 0x7f89a80fa410 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.472+0000 7f89ae91d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f89a8114c10 0x7f89a80faa60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.472+0000 7f89ae91d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f89a8115820 0x7f89a80fb660 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.472+0000 7f89a77fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f89a8114c10 0x7f89a80faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.472+0000 7f89ae91d640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f89a80fbda0 con 0x7f89a8115820 2026-02-25T11:43:53.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.472+0000 7f89ae91d640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f89a80fbf10 con 0x7f89a805e4b0 2026-02-25T11:43:53.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.472+0000 7f89ae91d640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f89a80fc0d0 con 0x7f89a8114c10 2026-02-25T11:43:53.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.472+0000 7f89a7fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f89a805e4b0 0x7f89a80fa410 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.472+0000 7f89a7fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f89a805e4b0 0x7f89a80fa410 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:51978/0 (socket says 10.20.193.28:51978) 2026-02-25T11:43:53.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.472+0000 7f89a7fff640 1 -- 10.20.193.28:0/1540058417 learned_addr learned my addr 10.20.193.28:0/1540058417 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:53.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.472+0000 7f89a77fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f89a8114c10 0x7f89a80faa60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56462/0 (socket says 10.20.193.28:56462) 2026-02-25T11:43:53.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.472+0000 7f89ace93640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f89a8115820 0x7f89a80fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.473+0000 7f89a7fff640 1 -- 10.20.193.28:0/1540058417 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f89a8114c10 msgr2=0x7f89a80faa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.473+0000 7f89a7fff640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f89a8114c10 0x7f89a80faa60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.473+0000 7f89a7fff640 1 -- 10.20.193.28:0/1540058417 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f89a8115820 msgr2=0x7f89a80fb660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.473+0000 7f89a7fff640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f89a8115820 0x7f89a80fb660 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.473+0000 7f89a7fff640 1 -- 10.20.193.28:0/1540058417 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899800a590 con 0x7f89a805e4b0 2026-02-25T11:43:53.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.473+0000 7f89a77fe640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f89a8114c10 0x7f89a80faa60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:43:53.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.473+0000 7f89a7fff640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f89a805e4b0 0x7f89a80fa410 secure :-1 s=READY pgs=12 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f899c00d950 tx=0x7f899c00de20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.473+0000 7f89a57fa640 1 -- 10.20.193.28:0/1540058417 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f899c004540 con 0x7f89a805e4b0 2026-02-25T11:43:53.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.473+0000 7f89a57fa640 1 -- 10.20.193.28:0/1540058417 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f899c009ce0 con 0x7f89a805e4b0 2026-02-25T11:43:53.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.473+0000 7f89a57fa640 1 -- 10.20.193.28:0/1540058417 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f899c0106e0 con 0x7f89a805e4b0 2026-02-25T11:43:53.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.473+0000 7f89ae91d640 1 -- 10.20.193.28:0/1540058417 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8990009570 con 0x7f89a805e4b0 2026-02-25T11:43:53.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.474+0000 7f89ae91d640 1 -- 10.20.193.28:0/1540058417 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a80fc870 con 0x7f89a805e4b0 2026-02-25T11:43:53.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.474+0000 7f89ae91d640 1 -- 10.20.193.28:0/1540058417 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a8055e30 con 0x7f89a805e4b0 2026-02-25T11:43:53.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.475+0000 7f89a57fa640 1 -- 10.20.193.28:0/1540058417 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f899c009820 con 0x7f89a805e4b0 2026-02-25T11:43:53.480 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.475+0000 7f89a57fa640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f898003f9d0 0x7f898005fe80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.480 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.475+0000 7f89a57fa640 1 -- 10.20.193.28:0/1540058417 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f899c0541f0 con 0x7f89a805e4b0 2026-02-25T11:43:53.480 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.476+0000 7f89a77fe640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f898003f9d0 0x7f898005fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.480 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.476+0000 7f89a77fe640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f898003f9d0 0x7f898005fe80 secure :-1 s=READY pgs=56 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8990005b30 tx=0x7f8990031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.478+0000 7f89a57fa640 1 -- 10.20.193.28:0/1540058417 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f899c00e770 con 0x7f89a805e4b0 2026-02-25T11:43:53.528 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.530+0000 7ff259321640 1 -- 10.20.193.28:0/2750935883 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff25405e580 msgr2=0x7ff2541146d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.530+0000 7ff259321640 1 --2- 10.20.193.28:0/2750935883 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff25405e580 0x7ff2541146d0 secure :-1 s=READY pgs=14 gs=2 cs=0 l=1 c_cookie=11515194252908943771 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff23c009950 tx=0x7ff23c02f1b0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.530+0000 7ff259321640 1 -- 10.20.193.28:0/2750935883 shutdown_connections 2026-02-25T11:43:53.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.530+0000 7ff259321640 1 --2- 10.20.193.28:0/2750935883 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff2541155c0 0x7ff254115a60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.530+0000 7ff259321640 1 --2- 10.20.193.28:0/2750935883 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff254114c10 0x7ff254115080 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.530+0000 7ff259321640 1 --2- 10.20.193.28:0/2750935883 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff25405e580 0x7ff2541146d0 unknown :-1 s=CLOSED pgs=14 gs=2 cs=0 l=1 c_cookie=11515194252908943771 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.530+0000 7ff259321640 1 -- 10.20.193.28:0/2750935883 >> 10.20.193.28:0/2750935883 conn(0x7ff25405b5d0 msgr2=0x7ff25405b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.530+0000 7ff259321640 1 -- 10.20.193.28:0/2750935883 shutdown_connections 2026-02-25T11:43:53.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.530+0000 7ff259321640 1 -- 10.20.193.28:0/2750935883 wait complete. 2026-02-25T11:43:53.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.531+0000 7ff259321640 1 Processor -- start 2026-02-25T11:43:53.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.531+0000 7ff259321640 1 -- start start 2026-02-25T11:43:53.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.531+0000 7ff259321640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff25405e580 0x7ff2540ffca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.531+0000 7ff259321640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff254114c10 0x7ff2541002f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.531+0000 7ff259321640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff2541155c0 0x7ff254100ef0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.531+0000 7ff259321640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff254101630 con 0x7ff254114c10 2026-02-25T11:43:53.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.531+0000 7ff259321640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2541017a0 con 0x7ff2541155c0 2026-02-25T11:43:53.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.531+0000 7ff259321640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff254101960 con 0x7ff25405e580 2026-02-25T11:43:53.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.531+0000 7ff2527fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff254114c10 0x7ff2541002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.531+0000 7ff2527fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff254114c10 0x7ff2541002f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58638/0 (socket says 10.20.193.28:58638) 2026-02-25T11:43:53.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.531+0000 7ff2527fc640 1 -- 10.20.193.28:0/1981818117 learned_addr learned my addr 10.20.193.28:0/1981818117 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:53.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.532+0000 7ff2527fc640 1 -- 10.20.193.28:0/1981818117 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff25405e580 msgr2=0x7ff2540ffca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:43:53.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.532+0000 7ff2527fc640 1 --2- 10.20.193.28:0/1981818117 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff25405e580 0x7ff2540ffca0 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.532+0000 7ff2527fc640 1 -- 10.20.193.28:0/1981818117 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff2541155c0 msgr2=0x7ff254100ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:43:53.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.532+0000 7ff2527fc640 1 --2- 10.20.193.28:0/1981818117 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff2541155c0 0x7ff254100ef0 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.532+0000 7ff2527fc640 1 -- 10.20.193.28:0/1981818117 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff23c009570 con 0x7ff254114c10 2026-02-25T11:43:53.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.532+0000 7ff2527fc640 1 --2- 10.20.193.28:0/1981818117 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff254114c10 0x7ff2541002f0 secure :-1 s=READY pgs=119 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff24400a560 tx=0x7ff24400d6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.532+0000 7ff24bfff640 1 -- 10.20.193.28:0/1981818117 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff24400fe00 con 0x7ff254114c10 2026-02-25T11:43:53.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.532+0000 7ff24bfff640 1 -- 10.20.193.28:0/1981818117 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff244005df0 con 0x7ff254114c10 2026-02-25T11:43:53.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.533+0000 7ff24bfff640 1 -- 10.20.193.28:0/1981818117 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff24400fe00 con 0x7ff254114c10 2026-02-25T11:43:53.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.534+0000 7ff259321640 1 -- 10.20.193.28:0/1981818117 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff254101c40 con 0x7ff254114c10 2026-02-25T11:43:53.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.535+0000 7ff259321640 1 -- 10.20.193.28:0/1981818117 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25417a990 con 0x7ff254114c10 2026-02-25T11:43:53.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.535+0000 7ff24bfff640 1 -- 10.20.193.28:0/1981818117 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7ff244006290 con 0x7ff254114c10 2026-02-25T11:43:53.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.535+0000 7ff24bfff640 1 --2- 10.20.193.28:0/1981818117 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff23003f950 0x7ff23005fe00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.536+0000 7ff252ffd640 1 --2- 10.20.193.28:0/1981818117 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff23003f950 0x7ff23005fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.536+0000 7ff24bfff640 1 -- 10.20.193.28:0/1981818117 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7ff244015070 con 0x7ff254114c10 2026-02-25T11:43:53.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.536+0000 7ff252ffd640 1 --2- 10.20.193.28:0/1981818117 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff23003f950 0x7ff23005fe00 secure :-1 s=READY pgs=57 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff23c0098a0 tx=0x7ff23c002390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.541+0000 7ff259321640 1 -- 10.20.193.28:0/1981818117 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff254055e30 con 0x7ff254114c10 2026-02-25T11:43:53.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.544+0000 7ff24bfff640 1 -- 10.20.193.28:0/1981818117 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7ff244011b40 con 0x7ff254114c10 2026-02-25T11:43:53.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.585+0000 7fca71b79640 1 -- 10.20.193.28:0/1042915961 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fca34005b30 con 0x7fca6c10be40 2026-02-25T11:43:53.591 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.593+0000 7fca61ffb640 1 -- 10.20.193.28:0/1042915961 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v17)=0 v17) ==== 116+0+0 (secure 0 0 0) 0x7fca5c021860 con 0x7fca6c10be40 2026-02-25T11:43:53.604 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.619 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.624 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.627+0000 7fca71b79640 1 -- 10.20.193.28:0/1042915961 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca5003fa00 msgr2=0x7fca5005feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.627+0000 7fca71b79640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca5003fa00 0x7fca5005feb0 secure :-1 s=READY pgs=55 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca540023d0 tx=0x7fca5404b040 comp rx=0 tx=0).stop 2026-02-25T11:43:53.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.627+0000 7fca71b79640 1 -- 10.20.193.28:0/1042915961 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c10be40 msgr2=0x7fca6c0f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.627+0000 7fca71b79640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c10be40 0x7fca6c0f56c0 secure :-1 s=READY pgs=11 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca5c00c520 tx=0x7fca5c00c9f0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.627+0000 7fca71b79640 1 -- 10.20.193.28:0/1042915961 shutdown_connections 2026-02-25T11:43:53.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.627+0000 7fca71b79640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca5003fa00 0x7fca5005feb0 unknown :-1 s=CLOSED pgs=55 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.627+0000 7fca71b79640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c10c9d0 0x7fca6c0f62c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.627+0000 7fca71b79640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c10be40 0x7fca6c0f56c0 unknown :-1 s=CLOSED pgs=11 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.627+0000 7fca71b79640 1 --2- 10.20.193.28:0/1042915961 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c0609c0 0x7fca6c0f5070 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.627+0000 7fca71b79640 1 -- 10.20.193.28:0/1042915961 >> 10.20.193.28:0/1042915961 conn(0x7fca6c0e9580 msgr2=0x7fca6c0eb230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.627+0000 7fca71b79640 1 -- 10.20.193.28:0/1042915961 shutdown_connections 2026-02-25T11:43:53.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.627+0000 7fca71b79640 1 -- 10.20.193.28:0/1042915961 wait complete. 2026-02-25T11:43:53.656 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='trial028') 2026-02-25T11:43:53.656 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='trial028') 2026-02-25T11:43:53.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.657+0000 7f89ae91d640 1 -- 10.20.193.28:0/1540058417 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f89a8057010 con 0x7f89a805e4b0 2026-02-25T11:43:53.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.661+0000 7f89a57fa640 1 -- 10.20.193.28:0/1540058417 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v18)=0 v18) ==== 106+0+0 (secure 0 0 0) 0x7f899c0172c0 con 0x7f89a805e4b0 2026-02-25T11:43:53.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.666+0000 7f89ae91d640 1 -- 10.20.193.28:0/1540058417 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f898003f9d0 msgr2=0x7f898005fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.666+0000 7f89ae91d640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f898003f9d0 0x7f898005fe80 secure :-1 s=READY pgs=56 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8990005b30 tx=0x7f8990031040 comp rx=0 tx=0).stop 2026-02-25T11:43:53.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.666+0000 7f89ae91d640 1 -- 10.20.193.28:0/1540058417 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f89a805e4b0 msgr2=0x7f89a80fa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.666+0000 7f89ae91d640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f89a805e4b0 0x7f89a80fa410 secure :-1 s=READY pgs=12 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f899c00d950 tx=0x7f899c00de20 comp rx=0 tx=0).stop 2026-02-25T11:43:53.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.666+0000 7f89ae91d640 1 -- 10.20.193.28:0/1540058417 shutdown_connections 2026-02-25T11:43:53.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.666+0000 7f89ae91d640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f898003f9d0 0x7f898005fe80 unknown :-1 s=CLOSED pgs=56 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.666+0000 7f89ae91d640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f89a8115820 0x7f89a80fb660 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.666+0000 7f89ae91d640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f89a8114c10 0x7f89a80faa60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.666+0000 7f89ae91d640 1 --2- 10.20.193.28:0/1540058417 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f89a805e4b0 0x7f89a80fa410 unknown :-1 s=CLOSED pgs=12 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.666+0000 7f89ae91d640 1 -- 10.20.193.28:0/1540058417 >> 10.20.193.28:0/1540058417 conn(0x7f89a805b5d0 msgr2=0x7f89a810fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.666+0000 7f89ae91d640 1 -- 10.20.193.28:0/1540058417 shutdown_connections 2026-02-25T11:43:53.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.666+0000 7f89ae91d640 1 -- 10.20.193.28:0/1540058417 wait complete. 2026-02-25T11:43:53.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.714+0000 7ff259321640 1 -- 10.20.193.28:0/1981818117 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7ff254119bb0 con 0x7ff254114c10 2026-02-25T11:43:53.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.718+0000 7ff24bfff640 1 -- 10.20.193.28:0/1981818117 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v19)=0 v19) ==== 127+0+0 (secure 0 0 0) 0x7ff254119bb0 con 0x7ff254114c10 2026-02-25T11:43:53.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.724+0000 7ff259321640 1 -- 10.20.193.28:0/1981818117 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff23003f950 msgr2=0x7ff23005fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.724+0000 7ff259321640 1 --2- 10.20.193.28:0/1981818117 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff23003f950 0x7ff23005fe00 secure :-1 s=READY pgs=57 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff23c0098a0 tx=0x7ff23c002390 comp rx=0 tx=0).stop 2026-02-25T11:43:53.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.724+0000 7ff259321640 1 -- 10.20.193.28:0/1981818117 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff254114c10 msgr2=0x7ff2541002f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.724+0000 7ff259321640 1 --2- 10.20.193.28:0/1981818117 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff254114c10 0x7ff2541002f0 secure :-1 s=READY pgs=119 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff24400a560 tx=0x7ff24400d6a0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.725+0000 7ff259321640 1 -- 10.20.193.28:0/1981818117 shutdown_connections 2026-02-25T11:43:53.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.725+0000 7ff259321640 1 --2- 10.20.193.28:0/1981818117 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff23003f950 0x7ff23005fe00 unknown :-1 s=CLOSED pgs=57 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.725+0000 7ff259321640 1 --2- 10.20.193.28:0/1981818117 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff2541155c0 0x7ff254100ef0 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.725+0000 7ff259321640 1 --2- 10.20.193.28:0/1981818117 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff254114c10 0x7ff2541002f0 unknown :-1 s=CLOSED pgs=119 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.725+0000 7ff259321640 1 --2- 10.20.193.28:0/1981818117 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff25405e580 0x7ff2540ffca0 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.725+0000 7ff259321640 1 -- 10.20.193.28:0/1981818117 >> 10.20.193.28:0/1981818117 conn(0x7ff25405b5d0 msgr2=0x7ff2540f5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.725+0000 7ff259321640 1 -- 10.20.193.28:0/1981818117 shutdown_connections 2026-02-25T11:43:53.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.725+0000 7ff259321640 1 -- 10.20.193.28:0/1981818117 wait complete. 2026-02-25T11:43:53.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.737+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2478476637 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27a00bb370 msgr2=0x7f27a00bb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.737+0000 7f27ad8f6640 1 --2- 10.20.193.28:0/2478476637 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27a00bb370 0x7f27a00bb760 secure :-1 s=READY pgs=120 gs=1 cs=0 l=1 c_cookie=14066866235717359158 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f279c0099a0 tx=0x7f279c02f220 comp rx=0 tx=0).stop 2026-02-25T11:43:53.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.738+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2478476637 shutdown_connections 2026-02-25T11:43:53.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.738+0000 7f27ad8f6640 1 --2- 10.20.193.28:0/2478476637 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f27a00bceb0 0x7f27a011d840 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.738+0000 7f27ad8f6640 1 --2- 10.20.193.28:0/2478476637 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f27a00bc500 0x7f27a00bc970 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.738+0000 7f27ad8f6640 1 --2- 10.20.193.28:0/2478476637 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27a00bb370 0x7f27a00bb760 unknown :-1 s=CLOSED pgs=120 gs=1 cs=0 l=1 c_cookie=14066866235717359158 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.738+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2478476637 >> 10.20.193.28:0/2478476637 conn(0x7f27a000c2f0 msgr2=0x7f27a000c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.738+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2478476637 shutdown_connections 2026-02-25T11:43:53.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.738+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2478476637 wait complete. 2026-02-25T11:43:53.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.739+0000 7f27ad8f6640 1 Processor -- start 2026-02-25T11:43:53.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.739+0000 7f27ad8f6640 1 -- start start 2026-02-25T11:43:53.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.740+0000 7f27ad8f6640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f27a00bb370 0x7f27a00a5ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.740+0000 7f27ad8f6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27a00bc500 0x7f27a011d840 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.741+0000 7f27a67fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27a00bc500 0x7f27a011d840 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.741+0000 7f27a67fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27a00bc500 0x7f27a011d840 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58650/0 (socket says 10.20.193.28:58650) 2026-02-25T11:43:53.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.741+0000 7f27a6ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f27a00bb370 0x7f27a00a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.741+0000 7f27a6ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f27a00bb370 0x7f27a00a5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56496/0 (socket says 10.20.193.28:56496) 2026-02-25T11:43:53.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.741+0000 7f27a67fc640 1 -- 10.20.193.28:0/2882002016 learned_addr learned my addr 10.20.193.28:0/2882002016 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:53.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.741+0000 7f27ad8f6640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f27a00bceb0 0x7f27a00a6850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.741+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2882002016 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00a7440 con 0x7f27a00bc500 2026-02-25T11:43:53.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.741+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2882002016 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00a75b0 con 0x7f27a00bceb0 2026-02-25T11:43:53.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.742+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2882002016 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00a7770 con 0x7f27a00bb370 2026-02-25T11:43:53.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.742+0000 7f27a6ffd640 1 -- 10.20.193.28:0/2882002016 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f27a00bceb0 msgr2=0x7f27a00a6850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-25T11:43:53.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.742+0000 7f27a6ffd640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f27a00bceb0 0x7f27a00a6850 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.742+0000 7f27a6ffd640 1 -- 10.20.193.28:0/2882002016 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27a00bc500 msgr2=0x7f27a011d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.742+0000 7f27a6ffd640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27a00bc500 0x7f27a011d840 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.742+0000 7f27a6ffd640 1 -- 10.20.193.28:0/2882002016 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c009570 con 0x7f27a00bb370 2026-02-25T11:43:53.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.743+0000 7f27a67fc640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27a00bc500 0x7f27a011d840 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:43:53.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.744+0000 7f27a6ffd640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f27a00bb370 0x7f27a00a5ff0 secure :-1 s=READY pgs=15 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f279c0099a0 tx=0x7f279c0043d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.744+0000 7f27ac8f4640 1 -- 10.20.193.28:0/2882002016 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f279c03d070 con 0x7f27a00bb370 2026-02-25T11:43:53.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.744+0000 7f27ac8f4640 1 -- 10.20.193.28:0/2882002016 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f279c038960 con 0x7f27a00bb370 2026-02-25T11:43:53.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.744+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2882002016 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a00a79f0 con 0x7f27a00bb370 2026-02-25T11:43:53.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.745+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2882002016 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a00a7e10 con 0x7f27a00bb370 2026-02-25T11:43:53.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.746+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2882002016 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a0121790 con 0x7f27a00bb370 2026-02-25T11:43:53.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.747+0000 7f27ac8f4640 1 -- 10.20.193.28:0/2882002016 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f279c02fca0 con 0x7f27a00bb370 2026-02-25T11:43:53.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.748+0000 7f27ac8f4640 1 -- 10.20.193.28:0/2882002016 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f279c04a3f0 con 0x7f27a00bb370 2026-02-25T11:43:53.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.748+0000 7f27ac8f4640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f277c03fa20 0x7f277c05fed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.748+0000 7f27ac8f4640 1 -- 10.20.193.28:0/2882002016 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f279c0793a0 con 0x7f27a00bb370 2026-02-25T11:43:53.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.748+0000 7f27a67fc640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f277c03fa20 0x7f277c05fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.749+0000 7f27a67fc640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f277c03fa20 0x7f277c05fed0 secure :-1 s=READY pgs=58 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2798008320 tx=0x7f2798008500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.751+0000 7f27ac8f4640 1 -- 10.20.193.28:0/2882002016 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f279c04ae00 con 0x7f27a00bb370 2026-02-25T11:43:53.846 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.861 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.870 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:53.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.881+0000 7f20f1373640 1 -- 10.20.193.28:0/226601237 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f20ec062a60 msgr2=0x7f20ec0610f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.881+0000 7f20f1373640 1 --2- 10.20.193.28:0/226601237 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f20ec062a60 0x7f20ec0610f0 secure :-1 s=READY pgs=121 gs=1 cs=0 l=1 c_cookie=12196477643317575623 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f20d80099a0 tx=0x7f20d802f1e0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.893+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2882002016 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f27a00bc970 con 0x7f27a00bb370 2026-02-25T11:43:53.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.893+0000 7f20f1373640 1 -- 10.20.193.28:0/226601237 shutdown_connections 2026-02-25T11:43:53.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.893+0000 7f20f1373640 1 --2- 10.20.193.28:0/226601237 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f20ec10cb10 0x7f20ec10cfb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.893+0000 7f20f1373640 1 --2- 10.20.193.28:0/226601237 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f20ec061630 0x7f20ec061aa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.893+0000 7f20f1373640 1 --2- 10.20.193.28:0/226601237 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f20ec062a60 0x7f20ec0610f0 unknown :-1 s=CLOSED pgs=121 gs=1 cs=0 l=1 c_cookie=12196477643317575623 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.893+0000 7f20f1373640 1 -- 10.20.193.28:0/226601237 >> 10.20.193.28:0/226601237 conn(0x7f20ec0ea6d0 msgr2=0x7f20ec10aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.897+0000 7f20f1373640 1 -- 10.20.193.28:0/226601237 shutdown_connections 2026-02-25T11:43:53.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.897+0000 7f20f1373640 1 -- 10.20.193.28:0/226601237 wait complete. 2026-02-25T11:43:53.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.898+0000 7f20f1373640 1 Processor -- start 2026-02-25T11:43:53.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.898+0000 7f20f1373640 1 -- start start 2026-02-25T11:43:53.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.899+0000 7f20f1373640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f20ec061630 0x7f20ec0f6200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.899+0000 7f20f1373640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f20ec062a60 0x7f20ec0f6850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.899+0000 7f20f1373640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f20ec10cb10 0x7f20ec0f7450 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.899+0000 7f20f1373640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec0f7b90 con 0x7f20ec062a60 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.900+0000 7f20f1373640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec0f7d00 con 0x7f20ec061630 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.900+0000 7f20f1373640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec0f7ec0 con 0x7f20ec10cb10 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.900+0000 7f20eb7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f20ec062a60 0x7f20ec0f6850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.900+0000 7f20eb7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f20ec062a60 0x7f20ec0f6850 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58670/0 (socket says 10.20.193.28:58670) 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.900+0000 7f20eb7fe640 1 -- 10.20.193.28:0/342622757 learned_addr learned my addr 10.20.193.28:0/342622757 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.901+0000 7f20eb7fe640 1 -- 10.20.193.28:0/342622757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f20ec10cb10 msgr2=0x7f20ec0f7450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.901+0000 7f20eb7fe640 1 --2- 10.20.193.28:0/342622757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f20ec10cb10 0x7f20ec0f7450 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.901+0000 7f20eb7fe640 1 -- 10.20.193.28:0/342622757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f20ec061630 msgr2=0x7f20ec0f6200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.901+0000 7f20eb7fe640 1 --2- 10.20.193.28:0/342622757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f20ec061630 0x7f20ec0f6200 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.901+0000 7f20eb7fe640 1 -- 10.20.193.28:0/342622757 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d8009570 con 0x7f20ec062a60 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.901+0000 7f20eb7fe640 1 --2- 10.20.193.28:0/342622757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f20ec062a60 0x7f20ec0f6850 secure :-1 s=READY pgs=122 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f20e0006e90 tx=0x7f20e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.903+0000 7f20e97fa640 1 -- 10.20.193.28:0/342622757 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f20e0005500 con 0x7f20ec062a60 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.903+0000 7f20e97fa640 1 -- 10.20.193.28:0/342622757 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f20e000cd00 con 0x7f20ec062a60 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.903+0000 7f20e97fa640 1 -- 10.20.193.28:0/342622757 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f20e00116e0 con 0x7f20ec062a60 2026-02-25T11:43:53.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.903+0000 7f20f1373640 1 -- 10.20.193.28:0/342622757 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ec0f81a0 con 0x7f20ec062a60 2026-02-25T11:43:53.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.903+0000 7f20f1373640 1 -- 10.20.193.28:0/342622757 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ec0f86c0 con 0x7f20ec062a60 2026-02-25T11:43:53.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.904+0000 7f20f1373640 1 -- 10.20.193.28:0/342622757 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b8005300 con 0x7f20ec062a60 2026-02-25T11:43:53.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.906+0000 7f20e97fa640 1 -- 10.20.193.28:0/342622757 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f20e000c840 con 0x7f20ec062a60 2026-02-25T11:43:53.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.909+0000 7f20e97fa640 1 --2- 10.20.193.28:0/342622757 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f20b403fa20 0x7f20b405fed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.922 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.910+0000 7f20e97fa640 1 -- 10.20.193.28:0/342622757 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f20e0055470 con 0x7f20ec062a60 2026-02-25T11:43:53.922 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.910+0000 7f20ebfff640 1 --2- 10.20.193.28:0/342622757 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f20b403fa20 0x7f20b405fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.922 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.910+0000 7f20ebfff640 1 --2- 10.20.193.28:0/342622757 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f20b403fa20 0x7f20b405fed0 secure :-1 s=READY pgs=59 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f20d80098f0 tx=0x7f20d8002390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.922 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.911+0000 7f20e97fa640 1 -- 10.20.193.28:0/342622757 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f20e0011c00 con 0x7f20ec062a60 2026-02-25T11:43:53.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.923+0000 7f27ac8f4640 1 -- 10.20.193.28:0/2882002016 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v20)=0 v20) ==== 118+0+0 (secure 0 0 0) 0x7f279c046070 con 0x7f27a00bb370 2026-02-25T11:43:53.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.927+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2882002016 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f277c03fa20 msgr2=0x7f277c05fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.927+0000 7f27ad8f6640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f277c03fa20 0x7f277c05fed0 secure :-1 s=READY pgs=58 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2798008320 tx=0x7f2798008500 comp rx=0 tx=0).stop 2026-02-25T11:43:53.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.927+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2882002016 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f27a00bb370 msgr2=0x7f27a00a5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.927+0000 7f27ad8f6640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f27a00bb370 0x7f27a00a5ff0 secure :-1 s=READY pgs=15 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f279c0099a0 tx=0x7f279c0043d0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.928+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2882002016 shutdown_connections 2026-02-25T11:43:53.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.928+0000 7f27ad8f6640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f277c03fa20 0x7f277c05fed0 unknown :-1 s=CLOSED pgs=58 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.928+0000 7f27ad8f6640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f27a00bceb0 0x7f27a00a6850 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.928+0000 7f27ad8f6640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f27a00bc500 0x7f27a011d840 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.928+0000 7f27ad8f6640 1 --2- 10.20.193.28:0/2882002016 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f27a00bb370 0x7f27a00a5ff0 unknown :-1 s=CLOSED pgs=15 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.928+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2882002016 >> 10.20.193.28:0/2882002016 conn(0x7f27a000c2f0 msgr2=0x7f27a009c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.928+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2882002016 shutdown_connections 2026-02-25T11:43:53.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.928+0000 7f27ad8f6640 1 -- 10.20.193.28:0/2882002016 wait complete. 2026-02-25T11:43:53.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.962+0000 7f6f18884640 1 -- 10.20.193.28:0/2354391302 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f14115210 msgr2=0x7f6f14115b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.962+0000 7f66926fb640 1 -- 10.20.193.28:0/412507776 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f668c110510 msgr2=0x7f668c110980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.962+0000 7f66926fb640 1 --2- 10.20.193.28:0/412507776 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f668c110510 0x7f668c110980 secure :-1 s=READY pgs=123 gs=1 cs=0 l=1 c_cookie=9353794547627114180 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f667c009950 tx=0x7f667c02f190 comp rx=0 tx=0).stop 2026-02-25T11:43:53.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.962+0000 7f66926fb640 1 -- 10.20.193.28:0/412507776 shutdown_connections 2026-02-25T11:43:53.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.962+0000 7f66926fb640 1 --2- 10.20.193.28:0/412507776 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f668c110ec0 0x7f668c171850 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.962+0000 7f66926fb640 1 --2- 10.20.193.28:0/412507776 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f668c110510 0x7f668c110980 unknown :-1 s=CLOSED pgs=123 gs=1 cs=0 l=1 c_cookie=9353794547627114180 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.962+0000 7f66926fb640 1 --2- 10.20.193.28:0/412507776 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f668c10f380 0x7f668c10f770 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.962+0000 7f66926fb640 1 -- 10.20.193.28:0/412507776 >> 10.20.193.28:0/412507776 conn(0x7f668c0ecad0 msgr2=0x7f668c10cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.962+0000 7f6f18884640 1 --2- 10.20.193.28:0/2354391302 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f14115210 0x7f6f14115b70 secure :-1 s=READY pgs=124 gs=3 cs=0 l=1 c_cookie=16147321106221991786 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6f0400a9c0 tx=0x7f6f04030220 comp rx=0 tx=0).stop 2026-02-25T11:43:53.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.963+0000 7f66926fb640 1 -- 10.20.193.28:0/412507776 shutdown_connections 2026-02-25T11:43:53.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.963+0000 7f66926fb640 1 -- 10.20.193.28:0/412507776 wait complete. 2026-02-25T11:43:53.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.963+0000 7f66926fb640 1 Processor -- start 2026-02-25T11:43:53.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.964+0000 7f66926fb640 1 -- start start 2026-02-25T11:43:53.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.964+0000 7f66926fb640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f668c10f380 0x7f668c1017f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.964+0000 7f66926fb640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f668c110510 0x7f668c171830 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.965+0000 7f668bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f668c10f380 0x7f668c1017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.965+0000 7f66926fb640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f668c110ec0 0x7f668c102030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.965+0000 7f668bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f668c10f380 0x7f668c1017f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56542/0 (socket says 10.20.193.28:56542) 2026-02-25T11:43:53.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.965+0000 7f668bfff640 1 -- 10.20.193.28:0/3180908517 learned_addr learned my addr 10.20.193.28:0/3180908517 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:53.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.965+0000 7f66926fb640 1 -- 10.20.193.28:0/3180908517 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f668c102a60 con 0x7f668c110510 2026-02-25T11:43:53.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.965+0000 7f66926fb640 1 -- 10.20.193.28:0/3180908517 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f668c102bd0 con 0x7f668c110ec0 2026-02-25T11:43:53.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.965+0000 7f66926fb640 1 -- 10.20.193.28:0/3180908517 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f668c102d90 con 0x7f668c10f380 2026-02-25T11:43:53.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.965+0000 7f6690c71640 1 --2- 10.20.193.28:0/3180908517 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f668c110ec0 0x7f668c102030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.965+0000 7f6690c71640 1 -- 10.20.193.28:0/3180908517 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f668c10f380 msgr2=0x7f668c1017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.965+0000 7f6690c71640 1 --2- 10.20.193.28:0/3180908517 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f668c10f380 0x7f668c1017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.965+0000 7f6690c71640 1 -- 10.20.193.28:0/3180908517 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f668c110510 msgr2=0x7f668c171830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.965+0000 7f6690c71640 1 --2- 10.20.193.28:0/3180908517 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f668c110510 0x7f668c171830 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.965+0000 7f6690c71640 1 -- 10.20.193.28:0/3180908517 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c009570 con 0x7f668c110ec0 2026-02-25T11:43:53.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.966+0000 7f6690c71640 1 --2- 10.20.193.28:0/3180908517 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f668c110ec0 0x7f668c102030 secure :-1 s=READY pgs=13 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f668000f940 tx=0x7f668000fe10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.966+0000 7f66897fa640 1 -- 10.20.193.28:0/3180908517 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f668000dce0 con 0x7f668c110ec0 2026-02-25T11:43:53.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.966+0000 7f66897fa640 1 -- 10.20.193.28:0/3180908517 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f668000de40 con 0x7f668c110ec0 2026-02-25T11:43:53.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.966+0000 7f66897fa640 1 -- 10.20.193.28:0/3180908517 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f66800116b0 con 0x7f668c110ec0 2026-02-25T11:43:53.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.966+0000 7f66926fb640 1 -- 10.20.193.28:0/3180908517 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f668c103070 con 0x7f668c110ec0 2026-02-25T11:43:53.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.966+0000 7f66926fb640 1 -- 10.20.193.28:0/3180908517 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f668c1034f0 con 0x7f668c110ec0 2026-02-25T11:43:53.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.967+0000 7f6f18884640 1 -- 10.20.193.28:0/2354391302 shutdown_connections 2026-02-25T11:43:53.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.967+0000 7f6f18884640 1 --2- 10.20.193.28:0/2354391302 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f14115210 0x7f6f14115b70 unknown :-1 s=CLOSED pgs=124 gs=3 cs=0 l=1 c_cookie=16147321106221991786 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.967+0000 7f6f18884640 1 --2- 10.20.193.28:0/2354391302 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6f14114860 0x7f6f14114cd0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.967+0000 7f6f18884640 1 --2- 10.20.193.28:0/2354391302 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6f1405de60 0x7f6f1405e250 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.967+0000 7f6f18884640 1 -- 10.20.193.28:0/2354391302 >> 10.20.193.28:0/2354391302 conn(0x7f6f1405b890 msgr2=0x7f6f1405bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:53.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.967+0000 7f66897fa640 1 -- 10.20.193.28:0/3180908517 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f6680020060 con 0x7f668c110ec0 2026-02-25T11:43:53.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.968+0000 7f66897fa640 1 --2- 10.20.193.28:0/3180908517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f666003fa20 0x7f666005fed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.968+0000 7f66897fa640 1 -- 10.20.193.28:0/3180908517 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f6680015070 con 0x7f668c110ec0 2026-02-25T11:43:53.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.969+0000 7f668bfff640 1 --2- 10.20.193.28:0/3180908517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f666003fa20 0x7f666005fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.969+0000 7f668bfff640 1 --2- 10.20.193.28:0/3180908517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f666003fa20 0x7f666005fed0 secure :-1 s=READY pgs=60 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f66780095d0 tx=0x7f6678005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.970+0000 7f6f18884640 1 -- 10.20.193.28:0/2354391302 shutdown_connections 2026-02-25T11:43:53.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.970+0000 7f6f18884640 1 -- 10.20.193.28:0/2354391302 wait complete. 2026-02-25T11:43:53.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.970+0000 7f66926fb640 1 -- 10.20.193.28:0/3180908517 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6658005300 con 0x7f668c110ec0 2026-02-25T11:43:53.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.971+0000 7f6f18884640 1 Processor -- start 2026-02-25T11:43:53.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.971+0000 7f6f18884640 1 -- start start 2026-02-25T11:43:53.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.972+0000 7f6f18884640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6f1405de60 0x7f6f141b4990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.972+0000 7f6f18884640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f14114860 0x7f6f141b4fe0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.972+0000 7f6f18884640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6f14115210 0x7f6f141b5be0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.972+0000 7f6f12575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f14114860 0x7f6f141b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.972+0000 7f6f12575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f14114860 0x7f6f141b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58700/0 (socket says 10.20.193.28:58700) 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.972+0000 7f6f12575640 1 -- 10.20.193.28:0/1738781290 learned_addr learned my addr 10.20.193.28:0/1738781290 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.972+0000 7f6f12d76640 1 --2- 10.20.193.28:0/1738781290 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6f1405de60 0x7f6f141b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.972+0000 7f6f18884640 1 -- 10.20.193.28:0/1738781290 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6f141b6320 con 0x7f6f14114860 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.972+0000 7f6f18884640 1 -- 10.20.193.28:0/1738781290 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6f141b6490 con 0x7f6f1405de60 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.972+0000 7f6f18884640 1 -- 10.20.193.28:0/1738781290 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f141b6650 con 0x7f6f14115210 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.973+0000 7f6f12575640 1 -- 10.20.193.28:0/1738781290 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6f14115210 msgr2=0x7f6f141b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.973+0000 7f6f12575640 1 --2- 10.20.193.28:0/1738781290 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6f14115210 0x7f6f141b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.973+0000 7f6f12575640 1 -- 10.20.193.28:0/1738781290 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6f1405de60 msgr2=0x7f6f141b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.973+0000 7f6f12575640 1 --2- 10.20.193.28:0/1738781290 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6f1405de60 0x7f6f141b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:53.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.973+0000 7f6f12575640 1 -- 10.20.193.28:0/1738781290 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f08009570 con 0x7f6f14114860 2026-02-25T11:43:53.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.973+0000 7f6f12575640 1 --2- 10.20.193.28:0/1738781290 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f14114860 0x7f6f141b4fe0 secure :-1 s=READY pgs=125 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6efc009480 tx=0x7f6efc00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.973+0000 7f6ef3fff640 1 -- 10.20.193.28:0/1738781290 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6efc00ccc0 con 0x7f6f14114860 2026-02-25T11:43:53.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.973+0000 7f6ef3fff640 1 -- 10.20.193.28:0/1738781290 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6efc00ce20 con 0x7f6f14114860 2026-02-25T11:43:53.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.974+0000 7f66897fa640 1 -- 10.20.193.28:0/3180908517 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f6680022630 con 0x7f668c110ec0 2026-02-25T11:43:53.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.973+0000 7f6ef3fff640 1 -- 10.20.193.28:0/1738781290 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6efc0106b0 con 0x7f6f14114860 2026-02-25T11:43:53.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.973+0000 7f6f18884640 1 -- 10.20.193.28:0/1738781290 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0400a590 con 0x7f6f14114860 2026-02-25T11:43:53.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.973+0000 7f6f18884640 1 -- 10.20.193.28:0/1738781290 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f14216e20 con 0x7f6f14114860 2026-02-25T11:43:53.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.975+0000 7f6ef3fff640 1 -- 10.20.193.28:0/1738781290 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f6efc01f060 con 0x7f6f14114860 2026-02-25T11:43:53.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.975+0000 7f6ef3fff640 1 --2- 10.20.193.28:0/1738781290 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ee003fa20 0x7f6ee005fed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:53.983 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.975+0000 7f6ef3fff640 1 -- 10.20.193.28:0/1738781290 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f6efc014070 con 0x7f6f14114860 2026-02-25T11:43:53.983 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.975+0000 7f6f12d76640 1 --2- 10.20.193.28:0/1738781290 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ee003fa20 0x7f6ee005fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:53.983 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.977+0000 7f6f12d76640 1 --2- 10.20.193.28:0/1738781290 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ee003fa20 0x7f6ee005fed0 secure :-1 s=READY pgs=61 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6f08005b30 tx=0x7f6f08009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:53.983 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.978+0000 7f6f18884640 1 -- 10.20.193.28:0/1738781290 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee4005300 con 0x7f6f14114860 2026-02-25T11:43:53.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:53.993+0000 7f6ef3fff640 1 -- 10.20.193.28:0/1738781290 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f6efc004b00 con 0x7f6f14114860 2026-02-25T11:43:54.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 -- 10.20.193.28:0/916760026 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39300b4bf0 msgr2=0x7f39300b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 --2- 10.20.193.28:0/916760026 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39300b4bf0 0x7f39300b4fe0 secure :-1 s=READY pgs=126 gs=1 cs=0 l=1 c_cookie=6278151904166202154 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3924009950 tx=0x7f392402f190 comp rx=0 tx=0).stop 2026-02-25T11:43:54.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 -- 10.20.193.28:0/916760026 shutdown_connections 2026-02-25T11:43:54.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 --2- 10.20.193.28:0/916760026 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39300b6730 0x7f39301170c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 --2- 10.20.193.28:0/916760026 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f39300b5d80 0x7f39300b61f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 --2- 10.20.193.28:0/916760026 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39300b4bf0 0x7f39300b4fe0 unknown :-1 s=CLOSED pgs=126 gs=1 cs=0 l=1 c_cookie=6278151904166202154 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 -- 10.20.193.28:0/916760026 >> 10.20.193.28:0/916760026 conn(0x7f3930092320 msgr2=0x7f39300b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 -- 10.20.193.28:0/916760026 shutdown_connections 2026-02-25T11:43:54.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 -- 10.20.193.28:0/916760026 wait complete. 2026-02-25T11:43:54.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 Processor -- start 2026-02-25T11:43:54.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 -- start start 2026-02-25T11:43:54.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39300b4bf0 0x7f393009adc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f39300b5d80 0x7f39301170c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39300b6730 0x7f393009b620 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f39377fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39300b4bf0 0x7f393009adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f39377fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39300b4bf0 0x7f393009adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56562/0 (socket says 10.20.193.28:56562) 2026-02-25T11:43:54.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f3936ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f39300b5d80 0x7f39301170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f393009c180 con 0x7f39300b6730 2026-02-25T11:43:54.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 -- 10.20.193.28:0/3241838096 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f393009c2f0 con 0x7f39300b5d80 2026-02-25T11:43:54.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f39377fe640 1 -- 10.20.193.28:0/3241838096 learned_addr learned my addr 10.20.193.28:0/3241838096 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:54.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f393ce00640 1 -- 10.20.193.28:0/3241838096 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f393009c4b0 con 0x7f39300b4bf0 2026-02-25T11:43:54.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f3937fff640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39300b6730 0x7f393009b620 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f39377fe640 1 -- 10.20.193.28:0/3241838096 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f39300b5d80 msgr2=0x7f39301170c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f39377fe640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f39300b5d80 0x7f39301170c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f39377fe640 1 -- 10.20.193.28:0/3241838096 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39300b6730 msgr2=0x7f393009b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f39377fe640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39300b6730 0x7f393009b620 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f39377fe640 1 -- 10.20.193.28:0/3241838096 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3924009570 con 0x7f39300b4bf0 2026-02-25T11:43:54.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f3937fff640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39300b6730 0x7f393009b620 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:43:54.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f39377fe640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39300b4bf0 0x7f393009adc0 secure :-1 s=READY pgs=16 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39240098a0 tx=0x7f39240028d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.003+0000 7f3934ff9640 1 -- 10.20.193.28:0/3241838096 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f392403d070 con 0x7f39300b4bf0 2026-02-25T11:43:54.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.009+0000 7f3934ff9640 1 -- 10.20.193.28:0/3241838096 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f392402fe20 con 0x7f39300b4bf0 2026-02-25T11:43:54.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.009+0000 7f3934ff9640 1 -- 10.20.193.28:0/3241838096 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3924041a60 con 0x7f39300b4bf0 2026-02-25T11:43:54.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.009+0000 7f393ce00640 1 -- 10.20.193.28:0/3241838096 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393009c730 con 0x7f39300b4bf0 2026-02-25T11:43:54.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.009+0000 7f393ce00640 1 -- 10.20.193.28:0/3241838096 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393009cb20 con 0x7f39300b4bf0 2026-02-25T11:43:54.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.009+0000 7f393ce00640 1 -- 10.20.193.28:0/3241838096 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f393000a240 con 0x7f39300b4bf0 2026-02-25T11:43:54.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.009+0000 7f3934ff9640 1 -- 10.20.193.28:0/3241838096 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f3924048020 con 0x7f39300b4bf0 2026-02-25T11:43:54.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.011+0000 7f3934ff9640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f391003f950 0x7f391005fe00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.011+0000 7f3934ff9640 1 -- 10.20.193.28:0/3241838096 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f3924032180 con 0x7f39300b4bf0 2026-02-25T11:43:54.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.011+0000 7f3936ffd640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f391003f950 0x7f391005fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.011+0000 7f3936ffd640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f391003f950 0x7f391005fe00 secure :-1 s=READY pgs=62 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3928004740 tx=0x7f3928004420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.011+0000 7f3934ff9640 1 -- 10.20.193.28:0/3241838096 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f392404a430 con 0x7f39300b4bf0 2026-02-25T11:43:54.035 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='trial028') 2026-02-25T11:43:54.035 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='trial028') 2026-02-25T11:43:54.035 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='trial028') 2026-02-25T11:43:54.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.042+0000 7f20f1373640 1 -- 10.20.193.28:0/342622757 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f20b80055b0 con 0x7f20ec062a60 2026-02-25T11:43:54.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.050+0000 7f20e97fa640 1 -- 10.20.193.28:0/342622757 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v21)=0 v21) ==== 115+0+0 (secure 0 0 0) 0x7f20e0015070 con 0x7f20ec062a60 2026-02-25T11:43:54.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.058+0000 7f20f1373640 1 -- 10.20.193.28:0/342622757 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f20b403fa20 msgr2=0x7f20b405fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.058+0000 7f20f1373640 1 --2- 10.20.193.28:0/342622757 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f20b403fa20 0x7f20b405fed0 secure :-1 s=READY pgs=59 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f20d80098f0 tx=0x7f20d8002390 comp rx=0 tx=0).stop 2026-02-25T11:43:54.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.058+0000 7f20f1373640 1 -- 10.20.193.28:0/342622757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f20ec062a60 msgr2=0x7f20ec0f6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.058+0000 7f20f1373640 1 --2- 10.20.193.28:0/342622757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f20ec062a60 0x7f20ec0f6850 secure :-1 s=READY pgs=122 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f20e0006e90 tx=0x7f20e000edf0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.059+0000 7f20f1373640 1 -- 10.20.193.28:0/342622757 shutdown_connections 2026-02-25T11:43:54.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.059+0000 7f20f1373640 1 --2- 10.20.193.28:0/342622757 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f20b403fa20 0x7f20b405fed0 unknown :-1 s=CLOSED pgs=59 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.059+0000 7f20f1373640 1 --2- 10.20.193.28:0/342622757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f20ec10cb10 0x7f20ec0f7450 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.059+0000 7f20f1373640 1 --2- 10.20.193.28:0/342622757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f20ec062a60 0x7f20ec0f6850 unknown :-1 s=CLOSED pgs=122 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.059+0000 7f20f1373640 1 --2- 10.20.193.28:0/342622757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f20ec061630 0x7f20ec0f6200 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.059+0000 7f20f1373640 1 -- 10.20.193.28:0/342622757 >> 10.20.193.28:0/342622757 conn(0x7f20ec0ea6d0 msgr2=0x7f20ec0ec380 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.059+0000 7f20f1373640 1 -- 10.20.193.28:0/342622757 shutdown_connections 2026-02-25T11:43:54.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.059+0000 7f20f1373640 1 -- 10.20.193.28:0/342622757 wait complete. 2026-02-25T11:43:54.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.105+0000 7f66926fb640 1 -- 10.20.193.28:0/3180908517 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f6658005b30 con 0x7f668c110ec0 2026-02-25T11:43:54.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.109+0000 7f66897fa640 1 -- 10.20.193.28:0/3180908517 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f6680020850 con 0x7f668c110ec0 2026-02-25T11:43:54.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.122+0000 7f66926fb640 1 -- 10.20.193.28:0/3180908517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f666003fa20 msgr2=0x7f666005fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.122+0000 7f66926fb640 1 --2- 10.20.193.28:0/3180908517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f666003fa20 0x7f666005fed0 secure :-1 s=READY pgs=60 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f66780095d0 tx=0x7f6678005f90 comp rx=0 tx=0).stop 2026-02-25T11:43:54.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.122+0000 7f66926fb640 1 -- 10.20.193.28:0/3180908517 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f668c110ec0 msgr2=0x7f668c102030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.122+0000 7f66926fb640 1 --2- 10.20.193.28:0/3180908517 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f668c110ec0 0x7f668c102030 secure :-1 s=READY pgs=13 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f668000f940 tx=0x7f668000fe10 comp rx=0 tx=0).stop 2026-02-25T11:43:54.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.122+0000 7f66926fb640 1 -- 10.20.193.28:0/3180908517 shutdown_connections 2026-02-25T11:43:54.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.122+0000 7f66926fb640 1 --2- 10.20.193.28:0/3180908517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f666003fa20 0x7f666005fed0 unknown :-1 s=CLOSED pgs=60 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.122+0000 7f66926fb640 1 --2- 10.20.193.28:0/3180908517 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f668c110ec0 0x7f668c102030 unknown :-1 s=CLOSED pgs=13 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.122+0000 7f66926fb640 1 --2- 10.20.193.28:0/3180908517 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f668c110510 0x7f668c171830 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.122+0000 7f66926fb640 1 --2- 10.20.193.28:0/3180908517 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f668c10f380 0x7f668c1017f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.122+0000 7f66926fb640 1 -- 10.20.193.28:0/3180908517 >> 10.20.193.28:0/3180908517 conn(0x7f668c0ecad0 msgr2=0x7f668c0ef190 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.123+0000 7f66926fb640 1 -- 10.20.193.28:0/3180908517 shutdown_connections 2026-02-25T11:43:54.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.123+0000 7f66926fb640 1 -- 10.20.193.28:0/3180908517 wait complete. 2026-02-25T11:43:54.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.124+0000 7efe4e6e6640 1 -- 10.20.193.28:0/1819566757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efe48114860 msgr2=0x7efe48114cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.124+0000 7efe4e6e6640 1 --2- 10.20.193.28:0/1819566757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efe48114860 0x7efe48114cd0 secure :-1 s=READY pgs=127 gs=1 cs=0 l=1 c_cookie=5182850898515628497 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efe380099a0 tx=0x7efe3802f220 comp rx=0 tx=0).stop 2026-02-25T11:43:54.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.125+0000 7efe4e6e6640 1 -- 10.20.193.28:0/1819566757 shutdown_connections 2026-02-25T11:43:54.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.125+0000 7efe4e6e6640 1 --2- 10.20.193.28:0/1819566757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7efe48115210 0x7efe48115b70 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.125+0000 7efe4e6e6640 1 --2- 10.20.193.28:0/1819566757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efe48114860 0x7efe48114cd0 unknown :-1 s=CLOSED pgs=127 gs=1 cs=0 l=1 c_cookie=5182850898515628497 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.125+0000 7efe4e6e6640 1 --2- 10.20.193.28:0/1819566757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efe4805de60 0x7efe4805e250 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.125+0000 7efe4e6e6640 1 -- 10.20.193.28:0/1819566757 >> 10.20.193.28:0/1819566757 conn(0x7efe4805b890 msgr2=0x7efe4805bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.125+0000 7efe4e6e6640 1 -- 10.20.193.28:0/1819566757 shutdown_connections 2026-02-25T11:43:54.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.125+0000 7efe4e6e6640 1 -- 10.20.193.28:0/1819566757 wait complete. 2026-02-25T11:43:54.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.126+0000 7efe4e6e6640 1 Processor -- start 2026-02-25T11:43:54.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.127+0000 7efe4e6e6640 1 -- start start 2026-02-25T11:43:54.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.128+0000 7efe4e6e6640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7efe4805de60 0x7efe480fdd20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.128+0000 7efe4e6e6640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efe48114860 0x7efe480fe370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.128+0000 7efe4e6e6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efe48115210 0x7efe480fef70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.128+0000 7efe4e6e6640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7efe480ff6b0 con 0x7efe48115210 2026-02-25T11:43:54.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.128+0000 7efe4e6e6640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe480ff820 con 0x7efe48114860 2026-02-25T11:43:54.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.128+0000 7efe4e6e6640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe480ff9e0 con 0x7efe4805de60 2026-02-25T11:43:54.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.129+0000 7efe4dee5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efe48115210 0x7efe480fef70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.129+0000 7efe4cee3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efe48114860 0x7efe480fe370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.129+0000 7efe4cee3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efe48114860 0x7efe480fe370 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:52072/0 (socket says 10.20.193.28:52072) 2026-02-25T11:43:54.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.129+0000 7efe4cee3640 1 -- 10.20.193.28:0/2361979087 learned_addr learned my addr 10.20.193.28:0/2361979087 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:54.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.129+0000 7efe4d6e4640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7efe4805de60 0x7efe480fdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.130+0000 7efe4cee3640 1 -- 10.20.193.28:0/2361979087 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7efe4805de60 msgr2=0x7efe480fdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.130+0000 7efe4cee3640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7efe4805de60 0x7efe480fdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.130+0000 7efe4cee3640 1 -- 10.20.193.28:0/2361979087 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efe48115210 msgr2=0x7efe480fef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.130+0000 7efe4cee3640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efe48115210 0x7efe480fef70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.130+0000 7efe4cee3640 1 -- 10.20.193.28:0/2361979087 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe38009570 con 0x7efe48114860 2026-02-25T11:43:54.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.130+0000 7efe4dee5640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efe48115210 0x7efe480fef70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:43:54.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.131+0000 7efe4d6e4640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7efe4805de60 0x7efe480fdd20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:43:54.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.132+0000 7efe4cee3640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efe48114860 0x7efe480fe370 secure :-1 s=READY pgs=14 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efe38005970 tx=0x7efe3802fcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.132+0000 7efe367fc640 1 -- 10.20.193.28:0/2361979087 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efe38002ad0 con 0x7efe48114860 2026-02-25T11:43:54.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.139+0000 7efe367fc640 1 -- 10.20.193.28:0/2361979087 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7efe3802fe20 con 0x7efe48114860 2026-02-25T11:43:54.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.139+0000 7efe367fc640 1 -- 10.20.193.28:0/2361979087 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efe38041af0 con 0x7efe48114860 2026-02-25T11:43:54.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.139+0000 7efe4e6e6640 1 -- 10.20.193.28:0/2361979087 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe480ffc60 con 0x7efe48114860 2026-02-25T11:43:54.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.139+0000 7efe4e6e6640 1 -- 10.20.193.28:0/2361979087 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe48100180 con 0x7efe48114860 2026-02-25T11:43:54.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.143+0000 7efe367fc640 1 -- 10.20.193.28:0/2361979087 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7efe38041cc0 con 0x7efe48114860 2026-02-25T11:43:54.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.143+0000 7efe4e6e6640 1 -- 10.20.193.28:0/2361979087 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe48119bf0 con 0x7efe48114860 2026-02-25T11:43:54.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.143+0000 7efe367fc640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efe2403f9d0 0x7efe2405fe80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.143+0000 7efe367fc640 1 -- 10.20.193.28:0/2361979087 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7efe38079f90 con 0x7efe48114860 2026-02-25T11:43:54.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.151+0000 7efe367fc640 1 -- 10.20.193.28:0/2361979087 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7efe38049580 con 0x7efe48114860 2026-02-25T11:43:54.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.151+0000 7efe4d6e4640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efe2403f9d0 0x7efe2405fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.151+0000 7efe4d6e4640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efe2403f9d0 0x7efe2405fe80 secure :-1 s=READY pgs=63 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efe44005860 tx=0x7efe44003840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.154+0000 7f9950052640 1 -- 10.20.193.28:0/2360086998 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99400b4430 msgr2=0x7f99400b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.154+0000 7f9950052640 1 --2- 10.20.193.28:0/2360086998 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99400b4430 0x7f99400b4f00 secure :-1 s=READY pgs=15 gs=3 cs=0 l=1 c_cookie=2430144444683097498 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f994400aa30 tx=0x7f9944030290 comp rx=0 tx=0).stop 2026-02-25T11:43:54.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.158+0000 7f9950052640 1 -- 10.20.193.28:0/2360086998 shutdown_connections 2026-02-25T11:43:54.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.158+0000 7f9950052640 1 --2- 10.20.193.28:0/2360086998 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99400b4430 0x7f99400b4f00 unknown :-1 s=CLOSED pgs=15 gs=3 cs=0 l=1 c_cookie=2430144444683097498 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.158+0000 7f9950052640 1 --2- 10.20.193.28:0/2360086998 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f99400b3880 0x7f99400b3cf0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.158+0000 7f9950052640 1 --2- 10.20.193.28:0/2360086998 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99400b2e50 0x7f99400b3240 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.158+0000 7f9950052640 1 -- 10.20.193.28:0/2360086998 >> 10.20.193.28:0/2360086998 conn(0x7f9940090970 msgr2=0x7f99400b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.158+0000 7f9950052640 1 -- 10.20.193.28:0/2360086998 shutdown_connections 2026-02-25T11:43:54.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.158+0000 7f9950052640 1 -- 10.20.193.28:0/2360086998 wait complete. 2026-02-25T11:43:54.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.158+0000 7f9950052640 1 Processor -- start 2026-02-25T11:43:54.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.160+0000 7f9950052640 1 -- start start 2026-02-25T11:43:54.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.161+0000 7f9950052640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f99400b2e50 0x7f994009ca90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.161+0000 7f9950052640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99400b3880 0x7f994009d0e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.161+0000 7f9950052640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99400b4430 0x7f994009dce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.161+0000 7f9950052640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f994009e420 con 0x7f99400b4430 2026-02-25T11:43:54.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.161+0000 7f994d5c6640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99400b3880 0x7f994009d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.161+0000 7f9950052640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f994009e590 con 0x7f99400b3880 2026-02-25T11:43:54.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.161+0000 7f9950052640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f994009e750 con 0x7f99400b2e50 2026-02-25T11:43:54.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.161+0000 7f994d5c6640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99400b3880 0x7f994009d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:52096/0 (socket says 10.20.193.28:52096) 2026-02-25T11:43:54.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.161+0000 7f994d5c6640 1 -- 10.20.193.28:0/2620875365 learned_addr learned my addr 10.20.193.28:0/2620875365 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:54.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.161+0000 7f994ddc7640 1 --2- 10.20.193.28:0/2620875365 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f99400b2e50 0x7f994009ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f994d5c6640 1 -- 10.20.193.28:0/2620875365 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f99400b2e50 msgr2=0x7f994009ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f994d5c6640 1 --2- 10.20.193.28:0/2620875365 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f99400b2e50 0x7f994009ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f994d5c6640 1 -- 10.20.193.28:0/2620875365 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99400b4430 msgr2=0x7f994009dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f994d5c6640 1 --2- 10.20.193.28:0/2620875365 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99400b4430 0x7f994009dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f994d5c6640 1 -- 10.20.193.28:0/2620875365 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994400a650 con 0x7f99400b3880 2026-02-25T11:43:54.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f994d5c6640 1 --2- 10.20.193.28:0/2620875365 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99400b3880 0x7f994009d0e0 secure :-1 s=READY pgs=16 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f993c009480 tx=0x7f993c00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f9936ffd640 1 -- 10.20.193.28:0/2620875365 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f993c00cd40 con 0x7f99400b3880 2026-02-25T11:43:54.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f9936ffd640 1 -- 10.20.193.28:0/2620875365 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f993c004d10 con 0x7f99400b3880 2026-02-25T11:43:54.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f9936ffd640 1 -- 10.20.193.28:0/2620875365 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f993c0106b0 con 0x7f99400b3880 2026-02-25T11:43:54.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f9950052640 1 -- 10.20.193.28:0/2620875365 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994009ea30 con 0x7f99400b3880 2026-02-25T11:43:54.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f9950052640 1 -- 10.20.193.28:0/2620875365 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99401191f0 con 0x7f99400b3880 2026-02-25T11:43:54.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f9936ffd640 1 -- 10.20.193.28:0/2620875365 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f993c004770 con 0x7f99400b3880 2026-02-25T11:43:54.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f9950052640 1 -- 10.20.193.28:0/2620875365 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99400b8db0 con 0x7f99400b3880 2026-02-25T11:43:54.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f9936ffd640 1 --2- 10.20.193.28:0/2620875365 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f992403fa20 0x7f992405fed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.162+0000 7f9936ffd640 1 -- 10.20.193.28:0/2620875365 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f993c014070 con 0x7f99400b3880 2026-02-25T11:43:54.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.166+0000 7f994ddc7640 1 --2- 10.20.193.28:0/2620875365 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f992403fa20 0x7f992405fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.166+0000 7f393ce00640 1 -- 10.20.193.28:0/3241838096 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f39300b6240 con 0x7f39300b4bf0 2026-02-25T11:43:54.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.166+0000 7f994ddc7640 1 --2- 10.20.193.28:0/2620875365 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f992403fa20 0x7f992405fed0 secure :-1 s=READY pgs=64 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9938004740 tx=0x7f9938009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.166+0000 7f9936ffd640 1 -- 10.20.193.28:0/2620875365 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f993c018620 con 0x7f99400b3880 2026-02-25T11:43:54.176 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.176 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:54.177 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.177 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:54.177 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:54.177 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:54.177 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-25T11:43:54.177 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.177 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2026-02-25T11:43:54.177 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-25T11:43:54.177 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:54.177 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.178 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.178 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1981818117' entity='client.admin' 2026-02-25T11:43:54.178 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:53 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.177+0000 7f3934ff9640 1 -- 10.20.193.28:0/3241838096 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v23)=0 v23) ==== 122+0+0 (secure 0 0 0) 0x7f392404a5d0 con 0x7f39300b4bf0 2026-02-25T11:43:54.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.179+0000 7f9f52e41640 1 -- 10.20.193.28:0/3230132353 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9f4c05dcf0 msgr2=0x7f9f4c05e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.179+0000 7f9f52e41640 1 --2- 10.20.193.28:0/3230132353 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9f4c05dcf0 0x7f9f4c05e0e0 secure :-1 s=READY pgs=17 gs=2 cs=0 l=1 c_cookie=7331666818082224604 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9f48009950 tx=0x7f9f48030300 comp rx=0 tx=0).stop 2026-02-25T11:43:54.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.180+0000 7f9f52e41640 1 -- 10.20.193.28:0/3230132353 shutdown_connections 2026-02-25T11:43:54.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.180+0000 7f9f52e41640 1 --2- 10.20.193.28:0/3230132353 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9f4c060020 0x7f9f4c060980 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.180+0000 7f9f52e41640 1 --2- 10.20.193.28:0/3230132353 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9f4c05f670 0x7f9f4c05fae0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.180+0000 7f9f52e41640 1 --2- 10.20.193.28:0/3230132353 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9f4c05dcf0 0x7f9f4c05e0e0 unknown :-1 s=CLOSED pgs=17 gs=2 cs=0 l=1 c_cookie=7331666818082224604 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.180+0000 7f9f52e41640 1 -- 10.20.193.28:0/3230132353 >> 10.20.193.28:0/3230132353 conn(0x7f9f4c05b450 msgr2=0x7f9f4c05b850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.180+0000 7f9f52e41640 1 -- 10.20.193.28:0/3230132353 shutdown_connections 2026-02-25T11:43:54.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.180+0000 7f9f52e41640 1 -- 10.20.193.28:0/3230132353 wait complete. 2026-02-25T11:43:54.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.183+0000 7f9f52e41640 1 Processor -- start 2026-02-25T11:43:54.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.184+0000 7f9f52e41640 1 -- start start 2026-02-25T11:43:54.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.185+0000 7f9f52e41640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9f4c05dcf0 0x7f9f4c066320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.188 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.185+0000 7f9f52e41640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9f4c05f670 0x7f9f4c066970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.188 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.185+0000 7f9f52e41640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9f4c060020 0x7f9f4c067570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.185+0000 7f9f52e41640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c067cb0 con 0x7f9f4c05f670 2026-02-25T11:43:54.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.185+0000 7f9f52e41640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c067e20 con 0x7f9f4c05dcf0 2026-02-25T11:43:54.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.185+0000 7f9f52e41640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c067fe0 con 0x7f9f4c060020 2026-02-25T11:43:54.190 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.186+0000 7f9f5163e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9f4c05f670 0x7f9f4c066970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.190 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f52640640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9f4c060020 0x7f9f4c067570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.190 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f51e3f640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9f4c05dcf0 0x7f9f4c066320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.190 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f5163e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9f4c05f670 0x7f9f4c066970 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58768/0 (socket says 10.20.193.28:58768) 2026-02-25T11:43:54.190 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f5163e640 1 -- 10.20.193.28:0/116715386 learned_addr learned my addr 10.20.193.28:0/116715386 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:54.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f51e3f640 1 -- 10.20.193.28:0/116715386 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9f4c060020 msgr2=0x7f9f4c067570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f51e3f640 1 --2- 10.20.193.28:0/116715386 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9f4c060020 0x7f9f4c067570 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f51e3f640 1 -- 10.20.193.28:0/116715386 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9f4c05f670 msgr2=0x7f9f4c066970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f51e3f640 1 --2- 10.20.193.28:0/116715386 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9f4c05f670 0x7f9f4c066970 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f51e3f640 1 -- 10.20.193.28:0/116715386 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f48009570 con 0x7f9f4c05dcf0 2026-02-25T11:43:54.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f5163e640 1 --2- 10.20.193.28:0/116715386 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9f4c05f670 0x7f9f4c066970 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:43:54.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f51e3f640 1 --2- 10.20.193.28:0/116715386 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9f4c05dcf0 0x7f9f4c066320 secure :-1 s=READY pgs=18 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9f480023d0 tx=0x7f9f48032db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f42ffd640 1 -- 10.20.193.28:0/116715386 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9f4803e070 con 0x7f9f4c05dcf0 2026-02-25T11:43:54.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f42ffd640 1 -- 10.20.193.28:0/116715386 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9f48002b60 con 0x7f9f4c05dcf0 2026-02-25T11:43:54.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.187+0000 7f9f42ffd640 1 -- 10.20.193.28:0/116715386 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9f480052b0 con 0x7f9f4c05dcf0 2026-02-25T11:43:54.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.188+0000 7f9f52e41640 1 -- 10.20.193.28:0/116715386 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4c068260 con 0x7f9f4c05dcf0 2026-02-25T11:43:54.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.188+0000 7f9f52e41640 1 -- 10.20.193.28:0/116715386 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4c068780 con 0x7f9f4c05dcf0 2026-02-25T11:43:54.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.190+0000 7f9f40ff9640 1 -- 10.20.193.28:0/116715386 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f1c005300 con 0x7f9f4c05dcf0 2026-02-25T11:43:54.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.190+0000 7f9f42ffd640 1 -- 10.20.193.28:0/116715386 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f9f48005440 con 0x7f9f4c05dcf0 2026-02-25T11:43:54.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.190+0000 7f9f42ffd640 1 --2- 10.20.193.28:0/116715386 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9f2003f9d0 0x7f9f2005fe80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.190+0000 7f9f42ffd640 1 -- 10.20.193.28:0/116715386 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f9f48078d20 con 0x7f9f4c05dcf0 2026-02-25T11:43:54.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.191+0000 7f9f5163e640 1 --2- 10.20.193.28:0/116715386 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9f2003f9d0 0x7f9f2005fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.191+0000 7f9f5163e640 1 --2- 10.20.193.28:0/116715386 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9f2003f9d0 0x7f9f2005fe80 secure :-1 s=READY pgs=65 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9f3c0095d0 tx=0x7f9f3c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.195 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.192+0000 7f393ce00640 1 -- 10.20.193.28:0/3241838096 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f391003f950 msgr2=0x7f391005fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.195 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.192+0000 7f393ce00640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f391003f950 0x7f391005fe00 secure :-1 s=READY pgs=62 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3928004740 tx=0x7f3928004420 comp rx=0 tx=0).stop 2026-02-25T11:43:54.195 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.192+0000 7f393ce00640 1 -- 10.20.193.28:0/3241838096 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39300b4bf0 msgr2=0x7f393009adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.195 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.192+0000 7f393ce00640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39300b4bf0 0x7f393009adc0 secure :-1 s=READY pgs=16 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39240098a0 tx=0x7f39240028d0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.196 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.193+0000 7f393ce00640 1 -- 10.20.193.28:0/3241838096 shutdown_connections 2026-02-25T11:43:54.196 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.193+0000 7f393ce00640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f391003f950 0x7f391005fe00 unknown :-1 s=CLOSED pgs=62 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.196 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.193+0000 7f393ce00640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39300b6730 0x7f393009b620 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.196 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.193+0000 7f393ce00640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f39300b5d80 0x7f39301170c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.196 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.193+0000 7f393ce00640 1 --2- 10.20.193.28:0/3241838096 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39300b4bf0 0x7f393009adc0 unknown :-1 s=CLOSED pgs=16 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.196 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.193+0000 7f393ce00640 1 -- 10.20.193.28:0/3241838096 >> 10.20.193.28:0/3241838096 conn(0x7f3930092320 msgr2=0x7f3930007440 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.196 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.193+0000 7f393ce00640 1 -- 10.20.193.28:0/3241838096 shutdown_connections 2026-02-25T11:43:54.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.193+0000 7f393ce00640 1 -- 10.20.193.28:0/3241838096 wait complete. 2026-02-25T11:43:54.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.194+0000 7f9f42ffd640 1 -- 10.20.193.28:0/116715386 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f9f4807e020 con 0x7f9f4c05dcf0 2026-02-25T11:43:54.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.195+0000 7f6bb4822640 1 -- 10.20.193.28:0/3666911406 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb010f1e0 msgr2=0x7f6bb010f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.195+0000 7f6bb4822640 1 --2- 10.20.193.28:0/3666911406 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb010f1e0 0x7f6bb010f5d0 secure :-1 s=READY pgs=17 gs=1 cs=0 l=1 c_cookie=169177624662190165 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ba000a970 tx=0x7f6ba00301b0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.195+0000 7f6bb4822640 1 -- 10.20.193.28:0/3666911406 shutdown_connections 2026-02-25T11:43:54.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.195+0000 7f6bb4822640 1 --2- 10.20.193.28:0/3666911406 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6bb0110d20 0x7f6bb01716b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.195+0000 7f6bb4822640 1 --2- 10.20.193.28:0/3666911406 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6bb0110370 0x7f6bb01107e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.195+0000 7f6bb4822640 1 --2- 10.20.193.28:0/3666911406 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb010f1e0 0x7f6bb010f5d0 unknown :-1 s=CLOSED pgs=17 gs=1 cs=0 l=1 c_cookie=169177624662190165 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.195+0000 7f6bb4822640 1 -- 10.20.193.28:0/3666911406 >> 10.20.193.28:0/3666911406 conn(0x7f6bb00ec950 msgr2=0x7f6bb010cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.195+0000 7f6bb4822640 1 -- 10.20.193.28:0/3666911406 shutdown_connections 2026-02-25T11:43:54.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.196+0000 7f6bb4822640 1 -- 10.20.193.28:0/3666911406 wait complete. 2026-02-25T11:43:54.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.196+0000 7f6bb4822640 1 Processor -- start 2026-02-25T11:43:54.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.197+0000 7f6bb4822640 1 -- start start 2026-02-25T11:43:54.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.199+0000 7f6bb4822640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb010f1e0 0x7f6bb00eece0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.199+0000 7f6bb4822640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6bb0110370 0x7f6bb01716b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.199+0000 7f6bb4822640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6bb0110d20 0x7f6bb00ef540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.199+0000 7f6bb4822640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb00f00e0 con 0x7f6bb0110d20 2026-02-25T11:43:54.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.199+0000 7f6bb4822640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb00f0250 con 0x7f6bb0110370 2026-02-25T11:43:54.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.199+0000 7f6bb4822640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb00f0410 con 0x7f6bb010f1e0 2026-02-25T11:43:54.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.199+0000 7f6baf577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6bb0110d20 0x7f6bb00ef540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.199+0000 7f6baf577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6bb0110d20 0x7f6bb00ef540 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58780/0 (socket says 10.20.193.28:58780) 2026-02-25T11:43:54.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.199+0000 7f6baf577640 1 -- 10.20.193.28:0/2843286466 learned_addr learned my addr 10.20.193.28:0/2843286466 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:54.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.199+0000 7f6baed76640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb010f1e0 0x7f6bb00eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.200+0000 7f6baf577640 1 -- 10.20.193.28:0/2843286466 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb010f1e0 msgr2=0x7f6bb00eece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.200+0000 7f6baf577640 1 --2- 10.20.193.28:0/2843286466 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb010f1e0 0x7f6bb00eece0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.200+0000 7f6baf577640 1 -- 10.20.193.28:0/2843286466 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6bb0110370 msgr2=0x7f6bb01716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.200+0000 7f6baf577640 1 --2- 10.20.193.28:0/2843286466 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6bb0110370 0x7f6bb01716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.200+0000 7f6baf577640 1 -- 10.20.193.28:0/2843286466 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b98009570 con 0x7f6bb0110d20 2026-02-25T11:43:54.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.200+0000 7f6baf577640 1 --2- 10.20.193.28:0/2843286466 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6bb0110d20 0x7f6bb00ef540 secure :-1 s=READY pgs=128 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ba400d950 tx=0x7f6ba400de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.200+0000 7f6b8ffff640 1 -- 10.20.193.28:0/2843286466 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6ba4004500 con 0x7f6bb0110d20 2026-02-25T11:43:54.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.200+0000 7f6bb4822640 1 -- 10.20.193.28:0/2843286466 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba000a590 con 0x7f6bb0110d20 2026-02-25T11:43:54.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.200+0000 7f6bb4822640 1 -- 10.20.193.28:0/2843286466 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb00f0a80 con 0x7f6bb0110d20 2026-02-25T11:43:54.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.201+0000 7f6b8ffff640 1 -- 10.20.193.28:0/2843286466 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6ba400bd00 con 0x7f6bb0110d20 2026-02-25T11:43:54.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.201+0000 7f6b8ffff640 1 -- 10.20.193.28:0/2843286466 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6ba4004500 con 0x7f6bb0110d20 2026-02-25T11:43:54.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.202+0000 7f6bb4822640 1 -- 10.20.193.28:0/2843286466 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b78005300 con 0x7f6bb0110d20 2026-02-25T11:43:54.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.203+0000 7f6b8ffff640 1 -- 10.20.193.28:0/2843286466 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f6ba4020020 con 0x7f6bb0110d20 2026-02-25T11:43:54.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.205+0000 7f6b8ffff640 1 --2- 10.20.193.28:0/2843286466 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6b8803fa30 0x7f6b8805fee0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.205+0000 7f6b8ffff640 1 -- 10.20.193.28:0/2843286466 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f6ba4053cc0 con 0x7f6bb0110d20 2026-02-25T11:43:54.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.206+0000 7f6baed76640 1 --2- 10.20.193.28:0/2843286466 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6b8803fa30 0x7f6b8805fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.210 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.208+0000 7f6b8ffff640 1 -- 10.20.193.28:0/2843286466 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f6ba4018d80 con 0x7f6bb0110d20 2026-02-25T11:43:54.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.208+0000 7f6baed76640 1 --2- 10.20.193.28:0/2843286466 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6b8803fa30 0x7f6b8805fee0 secure :-1 s=READY pgs=66 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ba0003d30 tx=0x7f6ba003b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:54.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:54.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:54.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:54.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-25T11:43:54.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2026-02-25T11:43:54.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-25T11:43:54.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:54.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1981818117' entity='client.admin' 2026-02-25T11:43:54.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:53 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:54.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:54.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:54.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:54.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-25T11:43:54.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2026-02-25T11:43:54.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-25T11:43:54.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:54.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1981818117' entity='client.admin' 2026-02-25T11:43:54.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:53 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:54.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.286+0000 7f2da731e640 1 -- 10.20.193.28:0/3498239149 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2da01153a0 msgr2=0x7f2da0175d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.286+0000 7f2da731e640 1 --2- 10.20.193.28:0/3498239149 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2da01153a0 0x7f2da0175d30 secure :-1 s=READY pgs=18 gs=1 cs=0 l=1 c_cookie=78003472126569300 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2d90009950 tx=0x7f2d9002f190 comp rx=0 tx=0).stop 2026-02-25T11:43:54.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.286+0000 7f2da731e640 1 -- 10.20.193.28:0/3498239149 shutdown_connections 2026-02-25T11:43:54.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.286+0000 7f2da731e640 1 --2- 10.20.193.28:0/3498239149 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2da01153a0 0x7f2da0175d30 unknown :-1 s=CLOSED pgs=18 gs=1 cs=0 l=1 c_cookie=78003472126569300 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.288+0000 7f2da731e640 1 --2- 10.20.193.28:0/3498239149 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2da01149f0 0x7f2da0114e60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.288+0000 7f2da731e640 1 --2- 10.20.193.28:0/3498239149 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2da01138f0 0x7f2da0113ce0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.288+0000 7f2da731e640 1 -- 10.20.193.28:0/3498239149 >> 10.20.193.28:0/3498239149 conn(0x7f2da00647e0 msgr2=0x7f2da0064be0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.288+0000 7f2da731e640 1 -- 10.20.193.28:0/3498239149 shutdown_connections 2026-02-25T11:43:54.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.291+0000 7f2da731e640 1 -- 10.20.193.28:0/3498239149 wait complete. 2026-02-25T11:43:54.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.291+0000 7f2da731e640 1 Processor -- start 2026-02-25T11:43:54.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.291+0000 7f2da731e640 1 -- start start 2026-02-25T11:43:54.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.292+0000 7f2da731e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2da01138f0 0x7f2da0107560 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.292+0000 7f2da731e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2da01149f0 0x7f2da0175d30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.292+0000 7f2da731e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2da01153a0 0x7f2da0107dc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.294+0000 7f2da731e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2da01089b0 con 0x7f2da01138f0 2026-02-25T11:43:54.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.294+0000 7f2da731e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0108b20 con 0x7f2da01153a0 2026-02-25T11:43:54.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.294+0000 7f2da731e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2da0108ce0 con 0x7f2da01149f0 2026-02-25T11:43:54.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.294+0000 7f2da5093640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2da01138f0 0x7f2da0107560 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.294+0000 7f2da5894640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2da01153a0 0x7f2da0107dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.294+0000 7f2da5093640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2da01138f0 0x7f2da0107560 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58802/0 (socket says 10.20.193.28:58802) 2026-02-25T11:43:54.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.294+0000 7f2da5894640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2da01153a0 0x7f2da0107dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:52148/0 (socket says 10.20.193.28:52148) 2026-02-25T11:43:54.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.294+0000 7f2da5894640 1 -- 10.20.193.28:0/441820481 learned_addr learned my addr 10.20.193.28:0/441820481 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:54.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.294+0000 7f2da4892640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2da01149f0 0x7f2da0175d30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.295+0000 7f2da5093640 1 -- 10.20.193.28:0/441820481 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2da01149f0 msgr2=0x7f2da0175d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.295+0000 7f2da5093640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2da01149f0 0x7f2da0175d30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.295+0000 7f2da5093640 1 -- 10.20.193.28:0/441820481 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2da01153a0 msgr2=0x7f2da0107dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.295+0000 7f2da5093640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2da01153a0 0x7f2da0107dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.295+0000 7f2da5093640 1 -- 10.20.193.28:0/441820481 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d940094b0 con 0x7f2da01138f0 2026-02-25T11:43:54.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.297+0000 7f2da4892640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2da01149f0 0x7f2da0175d30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:43:54.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.297+0000 7f2da5093640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2da01138f0 0x7f2da0107560 secure :-1 s=READY pgs=129 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2d9c00e5c0 tx=0x7f2d9c00ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.299+0000 7f2d8e7fc640 1 -- 10.20.193.28:0/441820481 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2d9c00a9b0 con 0x7f2da01138f0 2026-02-25T11:43:54.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.299+0000 7f2d8e7fc640 1 -- 10.20.193.28:0/441820481 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2d9c011460 con 0x7f2da01138f0 2026-02-25T11:43:54.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.299+0000 7f2da731e640 1 -- 10.20.193.28:0/441820481 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d90009570 con 0x7f2da01138f0 2026-02-25T11:43:54.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.299+0000 7f2d8e7fc640 1 -- 10.20.193.28:0/441820481 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2d9c019680 con 0x7f2da01138f0 2026-02-25T11:43:54.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.299+0000 7f2da731e640 1 -- 10.20.193.28:0/441820481 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da01092b0 con 0x7f2da01138f0 2026-02-25T11:43:54.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.301+0000 7f2d6ffff640 1 -- 10.20.193.28:0/441820481 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d60005300 con 0x7f2da01138f0 2026-02-25T11:43:54.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.303+0000 7f2d8e7fc640 1 -- 10.20.193.28:0/441820481 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f2d9c017020 con 0x7f2da01138f0 2026-02-25T11:43:54.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.304+0000 7f2d8e7fc640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2d7c03fa20 0x7f2d7c05fed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.304+0000 7f2d8e7fc640 1 -- 10.20.193.28:0/441820481 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f2d9c055500 con 0x7f2da01138f0 2026-02-25T11:43:54.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.305+0000 7f2da4892640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2d7c03fa20 0x7f2d7c05fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.305+0000 7f2da4892640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2d7c03fa20 0x7f2d7c05fed0 secure :-1 s=READY pgs=67 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2d94004740 tx=0x7f2d94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.309 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.306+0000 7f2d8e7fc640 1 -- 10.20.193.28:0/441820481 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f2d9c00f530 con 0x7f2da01138f0 2026-02-25T11:43:54.342 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.338+0000 7f58e21ac640 1 -- 10.20.193.28:0/3467429164 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f58dc134c70 msgr2=0x7f58dc1550f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.343 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.338+0000 7f58da7fc640 1 -- 10.20.193.28:0/3467429164 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f58c802fcc0 con 0x7f58dc134c70 2026-02-25T11:43:54.343 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.338+0000 7f58e21ac640 1 --2- 10.20.193.28:0/3467429164 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f58dc134c70 0x7f58dc1550f0 secure :-1 s=READY pgs=19 gs=1 cs=0 l=1 c_cookie=6627190656857330425 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f58c8009950 tx=0x7f58c802f240 comp rx=0 tx=0).stop 2026-02-25T11:43:54.343 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.339+0000 7f58e21ac640 1 -- 10.20.193.28:0/3467429164 shutdown_connections 2026-02-25T11:43:54.343 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.339+0000 7f58e21ac640 1 --2- 10.20.193.28:0/3467429164 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f58dc155830 0x7f58dc175ce0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.343 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.339+0000 7f58e21ac640 1 --2- 10.20.193.28:0/3467429164 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f58dc134c70 0x7f58dc1550f0 unknown :-1 s=CLOSED pgs=19 gs=1 cs=0 l=1 c_cookie=6627190656857330425 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.343 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.339+0000 7f58e21ac640 1 --2- 10.20.193.28:0/3467429164 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f58dc05e850 0x7f58dc1346e0 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.343 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.339+0000 7f58e21ac640 1 -- 10.20.193.28:0/3467429164 >> 10.20.193.28:0/3467429164 conn(0x7f58dc05b440 msgr2=0x7f58dc05b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.343 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.340+0000 7f58e21ac640 1 -- 10.20.193.28:0/3467429164 shutdown_connections 2026-02-25T11:43:54.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.340+0000 7f58e21ac640 1 -- 10.20.193.28:0/3467429164 wait complete. 2026-02-25T11:43:54.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.341+0000 7f58e21ac640 1 Processor -- start 2026-02-25T11:43:54.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.341+0000 7f58e21ac640 1 -- start start 2026-02-25T11:43:54.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.342+0000 7f58e21ac640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f58dc05e850 0x7f58dc1588b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.342+0000 7f58e21ac640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f58dc134c70 0x7f58dc1346e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.342+0000 7f58e21ac640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f58dc155830 0x7f58dc159110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.342+0000 7f58e21ac640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc159d00 con 0x7f58dc155830 2026-02-25T11:43:54.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.342+0000 7f58e21ac640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc159e70 con 0x7f58dc134c70 2026-02-25T11:43:54.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.342+0000 7f58e21ac640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f58dc15a030 con 0x7f58dc05e850 2026-02-25T11:43:54.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.342+0000 7f58db7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f58dc05e850 0x7f58dc1588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.342+0000 7f58db7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f58dc05e850 0x7f58dc1588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56640/0 (socket says 10.20.193.28:56640) 2026-02-25T11:43:54.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.342+0000 7f58db7fe640 1 -- 10.20.193.28:0/1257160216 learned_addr learned my addr 10.20.193.28:0/1257160216 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:54.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.343+0000 7f58dbfff640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f58dc155830 0x7f58dc159110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.343+0000 7f58daffd640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f58dc134c70 0x7f58dc1346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.343+0000 7f58dbfff640 1 -- 10.20.193.28:0/1257160216 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f58dc05e850 msgr2=0x7f58dc1588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.343+0000 7f58dbfff640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f58dc05e850 0x7f58dc1588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.343+0000 7f58dbfff640 1 -- 10.20.193.28:0/1257160216 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f58dc134c70 msgr2=0x7f58dc1346e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.343+0000 7f58dbfff640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f58dc134c70 0x7f58dc1346e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.343+0000 7f58dbfff640 1 -- 10.20.193.28:0/1257160216 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c8009570 con 0x7f58dc155830 2026-02-25T11:43:54.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.344+0000 7f58dbfff640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f58dc155830 0x7f58dc159110 secure :-1 s=READY pgs=130 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f58c400d670 tx=0x7f58c400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.344+0000 7f58d8ff9640 1 -- 10.20.193.28:0/1257160216 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f58c40042f0 con 0x7f58dc155830 2026-02-25T11:43:54.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.344+0000 7f58d8ff9640 1 -- 10.20.193.28:0/1257160216 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f58c4004450 con 0x7f58dc155830 2026-02-25T11:43:54.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.344+0000 7f58d8ff9640 1 -- 10.20.193.28:0/1257160216 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f58c4010460 con 0x7f58dc155830 2026-02-25T11:43:54.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.344+0000 7f58e21ac640 1 -- 10.20.193.28:0/1257160216 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58dc15a310 con 0x7f58dc155830 2026-02-25T11:43:54.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.344+0000 7f58e21ac640 1 -- 10.20.193.28:0/1257160216 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58dc15a760 con 0x7f58dc155830 2026-02-25T11:43:54.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.345+0000 7f58e21ac640 1 -- 10.20.193.28:0/1257160216 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58dc055e30 con 0x7f58dc155830 2026-02-25T11:43:54.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.346+0000 7f76fd1b5640 1 -- 10.20.193.28:0/3692957420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76f00c5850 msgr2=0x7f76f00c5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.346+0000 7f76fd1b5640 1 --2- 10.20.193.28:0/3692957420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76f00c5850 0x7f76f00c5cf0 secure :-1 s=READY pgs=131 gs=1 cs=0 l=1 c_cookie=6299878542063110103 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f76ec00aa30 tx=0x7f76ec030270 comp rx=0 tx=0).stop 2026-02-25T11:43:54.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.347+0000 7f76fd1b5640 1 -- 10.20.193.28:0/3692957420 shutdown_connections 2026-02-25T11:43:54.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.347+0000 7f76fd1b5640 1 --2- 10.20.193.28:0/3692957420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76f00c5850 0x7f76f00c5cf0 unknown :-1 s=CLOSED pgs=131 gs=1 cs=0 l=1 c_cookie=6299878542063110103 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.347+0000 7f76fd1b5640 1 --2- 10.20.193.28:0/3692957420 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76f00c4750 0x7f76f00c5310 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.348+0000 7f76fd1b5640 1 --2- 10.20.193.28:0/3692957420 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f76f00c3460 0x7f76f00c4110 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.348+0000 7f76fd1b5640 1 -- 10.20.193.28:0/3692957420 >> 10.20.193.28:0/3692957420 conn(0x7f76f00a1470 msgr2=0x7f76f00c1880 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.348+0000 7f76fd1b5640 1 -- 10.20.193.28:0/3692957420 shutdown_connections 2026-02-25T11:43:54.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.348+0000 7f76fd1b5640 1 -- 10.20.193.28:0/3692957420 wait complete. 2026-02-25T11:43:54.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.349+0000 7f76fd1b5640 1 Processor -- start 2026-02-25T11:43:54.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.349+0000 7f76fd1b5640 1 -- start start 2026-02-25T11:43:54.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.350+0000 7f76fd1b5640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76f00c3460 0x7f76f00d27a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.350+0000 7f76fd1b5640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f76f00c4750 0x7f76f00d2df0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.350+0000 7f58d8ff9640 1 -- 10.20.193.28:0/1257160216 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f58c400b840 con 0x7f58dc155830 2026-02-25T11:43:54.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.350+0000 7f76f7fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76f00c3460 0x7f76f00d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.350+0000 7f76f7fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76f00c3460 0x7f76f00d27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56652/0 (socket says 10.20.193.28:56652) 2026-02-25T11:43:54.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.350+0000 7f76f7fff640 1 -- 10.20.193.28:0/4037226528 learned_addr learned my addr 10.20.193.28:0/4037226528 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:54.355 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.350+0000 7f76fd1b5640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76f00c5850 0x7f76f00d39f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.355 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.351+0000 7f76fd1b5640 1 -- 10.20.193.28:0/4037226528 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f76f00d4130 con 0x7f76f00c5850 2026-02-25T11:43:54.355 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.351+0000 7f58d8ff9640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f58ac03f9d0 0x7f58ac05fe80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.356 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.351+0000 7f76fd1b5640 1 -- 10.20.193.28:0/4037226528 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f76f00d42a0 con 0x7f76f00c4750 2026-02-25T11:43:54.356 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.351+0000 7f76fd1b5640 1 -- 10.20.193.28:0/4037226528 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f76f00d4460 con 0x7f76f00c3460 2026-02-25T11:43:54.356 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.351+0000 7f76f77fe640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f76f00c4750 0x7f76f00d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.356 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.351+0000 7f76f7fff640 1 -- 10.20.193.28:0/4037226528 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f76f00c4750 msgr2=0x7f76f00d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.356 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.351+0000 7f76f7fff640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f76f00c4750 0x7f76f00d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.356 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.351+0000 7f58d8ff9640 1 -- 10.20.193.28:0/1257160216 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f58c4054750 con 0x7f58dc155830 2026-02-25T11:43:54.357 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.351+0000 7f76f7fff640 1 -- 10.20.193.28:0/4037226528 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76f00c5850 msgr2=0x7f76f00d39f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:43:54.357 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.352+0000 7f76fc9b4640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76f00c5850 0x7f76f00d39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.357 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.352+0000 7f58d8ff9640 1 -- 10.20.193.28:0/1257160216 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f58c4054b70 con 0x7f58dc155830 2026-02-25T11:43:54.358 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.352+0000 7f76f7fff640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76f00c5850 0x7f76f00d39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.358 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.352+0000 7f76f7fff640 1 -- 10.20.193.28:0/4037226528 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e40094b0 con 0x7f76f00c3460 2026-02-25T11:43:54.359 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.352+0000 7f58db7fe640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f58ac03f9d0 0x7f58ac05fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.359 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.352+0000 7f76fc9b4640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76f00c5850 0x7f76f00d39f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:43:54.359 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.354+0000 7f76f7fff640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76f00c3460 0x7f76f00d27a0 secure :-1 s=READY pgs=20 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f76e4009fd0 tx=0x7f76e4002bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.355+0000 7f76f57fa640 1 -- 10.20.193.28:0/4037226528 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f76e400ed60 con 0x7f76f00c3460 2026-02-25T11:43:54.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.354+0000 7f83ba7fc640 1 -- 10.20.193.28:0/3525321397 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f83ac02fbf0 con 0x7f83bc10c820 2026-02-25T11:43:54.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.355+0000 7f83c09dd640 1 -- 10.20.193.28:0/3525321397 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f83bc10c820 msgr2=0x7f83bc10d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.355+0000 7f83c09dd640 1 --2- 10.20.193.28:0/3525321397 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f83bc10c820 0x7f83bc10d180 secure :-1 s=READY pgs=132 gs=3 cs=0 l=1 c_cookie=3515223414749283376 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f83ac0098e0 tx=0x7f83ac02f170 comp rx=0 tx=0).stop 2026-02-25T11:43:54.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.355+0000 7f76fd1b5640 1 -- 10.20.193.28:0/4037226528 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ec00a650 con 0x7f76f00c3460 2026-02-25T11:43:54.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.355+0000 7f76f57fa640 1 -- 10.20.193.28:0/4037226528 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f76e4002e20 con 0x7f76f00c3460 2026-02-25T11:43:54.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.355+0000 7f6bb4822640 1 -- 10.20.193.28:0/2843286466 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7f6b78005b30 con 0x7f6bb0110d20 2026-02-25T11:43:54.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.355+0000 7f76f57fa640 1 -- 10.20.193.28:0/4037226528 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f76e400ed60 con 0x7f76f00c3460 2026-02-25T11:43:54.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.355+0000 7f76fd1b5640 1 -- 10.20.193.28:0/4037226528 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f0129b10 con 0x7f76f00c3460 2026-02-25T11:43:54.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.356+0000 7f58db7fe640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f58ac03f9d0 0x7f58ac05fe80 secure :-1 s=READY pgs=68 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f58cc007fd0 tx=0x7f58cc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.356+0000 7f6b8ffff640 1 -- 10.20.193.28:0/2843286466 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v23)=0 v23) ==== 108+0+0 (secure 0 0 0) 0x7f6ba4018d80 con 0x7f6bb0110d20 2026-02-25T11:43:54.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.356+0000 7f76f57fa640 1 -- 10.20.193.28:0/4037226528 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f76e4016070 con 0x7f76f00c3460 2026-02-25T11:43:54.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.356+0000 7f76f57fa640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f76cc03fa30 0x7f76cc05fee0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.356+0000 7f76f57fa640 1 -- 10.20.193.28:0/4037226528 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f76e4053fa0 con 0x7f76f00c3460 2026-02-25T11:43:54.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.357+0000 7f76fd1b5640 1 -- 10.20.193.28:0/4037226528 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f012e650 con 0x7f76f00c3460 2026-02-25T11:43:54.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.357+0000 7f76f77fe640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f76cc03fa30 0x7f76cc05fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.358+0000 7f76f77fe640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f76cc03fa30 0x7f76cc05fee0 secure :-1 s=READY pgs=69 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f76e8005970 tx=0x7f76e8005d10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.359+0000 7f6bb4822640 1 -- 10.20.193.28:0/2843286466 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6b8803fa30 msgr2=0x7f6b8805fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.359+0000 7f6bb4822640 1 --2- 10.20.193.28:0/2843286466 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6b8803fa30 0x7f6b8805fee0 secure :-1 s=READY pgs=66 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ba0003d30 tx=0x7f6ba003b040 comp rx=0 tx=0).stop 2026-02-25T11:43:54.366 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.359+0000 7f6bb4822640 1 -- 10.20.193.28:0/2843286466 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6bb0110d20 msgr2=0x7f6bb00ef540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.366 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.359+0000 7f6bb4822640 1 --2- 10.20.193.28:0/2843286466 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6bb0110d20 0x7f6bb00ef540 secure :-1 s=READY pgs=128 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ba400d950 tx=0x7f6ba400de20 comp rx=0 tx=0).stop 2026-02-25T11:43:54.366 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.360+0000 7f6bb4822640 1 -- 10.20.193.28:0/2843286466 shutdown_connections 2026-02-25T11:43:54.366 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.360+0000 7f6bb4822640 1 --2- 10.20.193.28:0/2843286466 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6b8803fa30 0x7f6b8805fee0 unknown :-1 s=CLOSED pgs=66 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.366 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.360+0000 7f6bb4822640 1 --2- 10.20.193.28:0/2843286466 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6bb0110d20 0x7f6bb00ef540 unknown :-1 s=CLOSED pgs=128 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.366 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.360+0000 7f6bb4822640 1 --2- 10.20.193.28:0/2843286466 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6bb0110370 0x7f6bb01716b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.366 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.360+0000 7f76f57fa640 1 -- 10.20.193.28:0/4037226528 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f76e4005030 con 0x7f76f00c3460 2026-02-25T11:43:54.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.360+0000 7f6bb4822640 1 --2- 10.20.193.28:0/2843286466 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb010f1e0 0x7f6bb00eece0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.360+0000 7f6bb4822640 1 -- 10.20.193.28:0/2843286466 >> 10.20.193.28:0/2843286466 conn(0x7f6bb00ec950 msgr2=0x7f6bb0064330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.360+0000 7f6bb4822640 1 -- 10.20.193.28:0/2843286466 shutdown_connections 2026-02-25T11:43:54.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.361+0000 7f6bb4822640 1 -- 10.20.193.28:0/2843286466 wait complete. 2026-02-25T11:43:54.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.366+0000 7f9f40ff9640 1 -- 10.20.193.28:0/116715386 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f9f4c055e30 con 0x7f9f4c05dcf0 2026-02-25T11:43:54.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.371+0000 7f9f42ffd640 1 -- 10.20.193.28:0/116715386 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v24)=0 v24) ==== 120+0+0 (secure 0 0 0) 0x7f9f48037bc0 con 0x7f9f4c05dcf0 2026-02-25T11:43:54.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.376+0000 7f9f40ff9640 1 -- 10.20.193.28:0/116715386 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9f2003f9d0 msgr2=0x7f9f2005fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.376+0000 7f9f40ff9640 1 --2- 10.20.193.28:0/116715386 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9f2003f9d0 0x7f9f2005fe80 secure :-1 s=READY pgs=65 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9f3c0095d0 tx=0x7f9f3c005f90 comp rx=0 tx=0).stop 2026-02-25T11:43:54.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.376+0000 7f9f40ff9640 1 -- 10.20.193.28:0/116715386 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9f4c05dcf0 msgr2=0x7f9f4c066320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.376+0000 7f9f40ff9640 1 --2- 10.20.193.28:0/116715386 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9f4c05dcf0 0x7f9f4c066320 secure :-1 s=READY pgs=18 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9f480023d0 tx=0x7f9f48032db0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.376+0000 7f9f40ff9640 1 -- 10.20.193.28:0/116715386 shutdown_connections 2026-02-25T11:43:54.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.376+0000 7f9f40ff9640 1 --2- 10.20.193.28:0/116715386 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9f2003f9d0 0x7f9f2005fe80 unknown :-1 s=CLOSED pgs=65 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.376+0000 7f9f40ff9640 1 --2- 10.20.193.28:0/116715386 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9f4c060020 0x7f9f4c067570 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.376+0000 7f9f40ff9640 1 --2- 10.20.193.28:0/116715386 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9f4c05f670 0x7f9f4c066970 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.376+0000 7f9f40ff9640 1 --2- 10.20.193.28:0/116715386 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9f4c05dcf0 0x7f9f4c066320 unknown :-1 s=CLOSED pgs=18 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.376+0000 7f9f40ff9640 1 -- 10.20.193.28:0/116715386 >> 10.20.193.28:0/116715386 conn(0x7f9f4c05b450 msgr2=0x7f9f4c05c810 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.376+0000 7f9f40ff9640 1 -- 10.20.193.28:0/116715386 shutdown_connections 2026-02-25T11:43:54.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.377+0000 7f9f40ff9640 1 -- 10.20.193.28:0/116715386 wait complete. 2026-02-25T11:43:54.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.403+0000 7f83c09dd640 1 -- 10.20.193.28:0/3525321397 shutdown_connections 2026-02-25T11:43:54.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.403+0000 7f83c09dd640 1 --2- 10.20.193.28:0/3525321397 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f83bc10c820 0x7f83bc10d180 unknown :-1 s=CLOSED pgs=132 gs=3 cs=0 l=1 c_cookie=3515223414749283376 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.403+0000 7f83c09dd640 1 --2- 10.20.193.28:0/3525321397 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f83bc10be70 0x7f83bc10c2e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.403+0000 7f83c09dd640 1 --2- 10.20.193.28:0/3525321397 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f83bc10ae40 0x7f83bc10b230 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.404+0000 7f83c09dd640 1 -- 10.20.193.28:0/3525321397 >> 10.20.193.28:0/3525321397 conn(0x7f83bc0e8860 msgr2=0x7f83bc108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.405+0000 7f83c09dd640 1 -- 10.20.193.28:0/3525321397 shutdown_connections 2026-02-25T11:43:54.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.405+0000 7f83c09dd640 1 -- 10.20.193.28:0/3525321397 wait complete. 2026-02-25T11:43:54.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.406+0000 7f83c09dd640 1 Processor -- start 2026-02-25T11:43:54.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.406+0000 7f83c09dd640 1 -- start start 2026-02-25T11:43:54.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.407+0000 7f83c09dd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f83bc10ae40 0x7f83bc0f3320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.407+0000 7f83c09dd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f83bc10be70 0x7f83bc0f3970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.407+0000 7f83c09dd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f83bc10c820 0x7f83bc0f4570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.407+0000 7f83c09dd640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc0f4cb0 con 0x7f83bc10c820 2026-02-25T11:43:54.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.407+0000 7f83c09dd640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc0f4e20 con 0x7f83bc10ae40 2026-02-25T11:43:54.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.407+0000 7f83c09dd640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc0f4fe0 con 0x7f83bc10be70 2026-02-25T11:43:54.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.407+0000 7f83baffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f83bc10be70 0x7f83bc0f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.408+0000 7f83baffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f83bc10be70 0x7f83bc0f3970 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56660/0 (socket says 10.20.193.28:56660) 2026-02-25T11:43:54.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.408+0000 7f83baffd640 1 -- 10.20.193.28:0/4147960216 learned_addr learned my addr 10.20.193.28:0/4147960216 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:54.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.408+0000 7f83bbfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f83bc10c820 0x7f83bc0f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.408+0000 7f83bb7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f83bc10ae40 0x7f83bc0f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.410+0000 7f83baffd640 1 -- 10.20.193.28:0/4147960216 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f83bc10ae40 msgr2=0x7f83bc0f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.410+0000 7f83baffd640 1 --2- 10.20.193.28:0/4147960216 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f83bc10ae40 0x7f83bc0f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.410+0000 7f83baffd640 1 -- 10.20.193.28:0/4147960216 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f83bc10c820 msgr2=0x7f83bc0f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.410+0000 7f83baffd640 1 --2- 10.20.193.28:0/4147960216 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f83bc10c820 0x7f83bc0f4570 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.410+0000 7f83baffd640 1 -- 10.20.193.28:0/4147960216 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac0094b0 con 0x7f83bc10be70 2026-02-25T11:43:54.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.410+0000 7f83baffd640 1 --2- 10.20.193.28:0/4147960216 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f83bc10be70 0x7f83bc0f3970 secure :-1 s=READY pgs=21 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f83a800b500 tx=0x7f83a800b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.411+0000 7f83b8ff9640 1 -- 10.20.193.28:0/4147960216 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f83a80044a0 con 0x7f83bc10be70 2026-02-25T11:43:54.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.411+0000 7f83b8ff9640 1 -- 10.20.193.28:0/4147960216 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f83a8002b60 con 0x7f83bc10be70 2026-02-25T11:43:54.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.411+0000 7f83b8ff9640 1 -- 10.20.193.28:0/4147960216 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f83a800fd20 con 0x7f83bc10be70 2026-02-25T11:43:54.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.411+0000 7f83c09dd640 1 -- 10.20.193.28:0/4147960216 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83bc0f52c0 con 0x7f83bc10be70 2026-02-25T11:43:54.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.411+0000 7f83c09dd640 1 -- 10.20.193.28:0/4147960216 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83bc0f57e0 con 0x7f83bc10be70 2026-02-25T11:43:54.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.412+0000 7f83c09dd640 1 -- 10.20.193.28:0/4147960216 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83bc111140 con 0x7f83bc10be70 2026-02-25T11:43:54.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.413+0000 7f83b8ff9640 1 -- 10.20.193.28:0/4147960216 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f83a80026a0 con 0x7f83bc10be70 2026-02-25T11:43:54.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.414+0000 7f83b8ff9640 1 --2- 10.20.193.28:0/4147960216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f838c03fa20 0x7f838c05fed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.414+0000 7f83b8ff9640 1 -- 10.20.193.28:0/4147960216 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f83a8053670 con 0x7f83bc10be70 2026-02-25T11:43:54.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.415+0000 7f83bb7fe640 1 --2- 10.20.193.28:0/4147960216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f838c03fa20 0x7f838c05fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.415+0000 7f81164cc640 1 -- 10.20.193.28:0/1591820034 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f811010be80 msgr2=0x7f811010c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.415+0000 7f81164cc640 1 --2- 10.20.193.28:0/1591820034 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f811010be80 0x7f811010c2f0 secure :-1 s=READY pgs=133 gs=2 cs=0 l=1 c_cookie=17387937787397284438 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f810400a9c0 tx=0x7f8104030240 comp rx=0 tx=0).stop 2026-02-25T11:43:54.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.416+0000 7f81164cc640 1 -- 10.20.193.28:0/1591820034 shutdown_connections 2026-02-25T11:43:54.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.416+0000 7f81164cc640 1 --2- 10.20.193.28:0/1591820034 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f811010c9e0 0x7f811010ce80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.416+0000 7f81164cc640 1 --2- 10.20.193.28:0/1591820034 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f811010be80 0x7f811010c2f0 unknown :-1 s=CLOSED pgs=133 gs=2 cs=0 l=1 c_cookie=17387937787397284438 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.416+0000 7f81164cc640 1 --2- 10.20.193.28:0/1591820034 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f81100601b0 0x7f811010b940 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.416+0000 7f81164cc640 1 -- 10.20.193.28:0/1591820034 >> 10.20.193.28:0/1591820034 conn(0x7f81100e9560 msgr2=0x7f81101099b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.416+0000 7f81164cc640 1 -- 10.20.193.28:0/1591820034 shutdown_connections 2026-02-25T11:43:54.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.418+0000 7f81164cc640 1 -- 10.20.193.28:0/1591820034 wait complete. 2026-02-25T11:43:54.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.419+0000 7f81164cc640 1 Processor -- start 2026-02-25T11:43:54.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.419+0000 7f81164cc640 1 -- start start 2026-02-25T11:43:54.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.420+0000 7f81164cc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f81100601b0 0x7f81100f5b00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.420+0000 7f81164cc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f811010be80 0x7f81100f6150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.420+0000 7f81164cc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f811010c9e0 0x7f81100f6d50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.420+0000 7f81164cc640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f81100f7490 con 0x7f81100601b0 2026-02-25T11:43:54.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.420+0000 7f81164cc640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f81100f7600 con 0x7f811010c9e0 2026-02-25T11:43:54.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.420+0000 7f81164cc640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f81100f77c0 con 0x7f811010be80 2026-02-25T11:43:54.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.421+0000 7f810f7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f811010be80 0x7f81100f6150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.421+0000 7f810f7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f811010be80 0x7f81100f6150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56664/0 (socket says 10.20.193.28:56664) 2026-02-25T11:43:54.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.421+0000 7f810f7fe640 1 -- 10.20.193.28:0/4104857923 learned_addr learned my addr 10.20.193.28:0/4104857923 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:54.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.421+0000 7f810f7fe640 1 -- 10.20.193.28:0/4104857923 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f811010c9e0 msgr2=0x7f81100f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.421+0000 7f810f7fe640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f811010c9e0 0x7f81100f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.421+0000 7f810f7fe640 1 -- 10.20.193.28:0/4104857923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f81100601b0 msgr2=0x7f81100f5b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:43:54.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.423+0000 7f83bb7fe640 1 --2- 10.20.193.28:0/4147960216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f838c03fa20 0x7f838c05fed0 secure :-1 s=READY pgs=70 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f83b0005820 tx=0x7f83b0005180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.423+0000 7f83b8ff9640 1 -- 10.20.193.28:0/4147960216 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f83a8016770 con 0x7f83bc10be70 2026-02-25T11:43:54.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.423+0000 7f810ffff640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f81100601b0 0x7f81100f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.423+0000 7f810f7fe640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f81100601b0 0x7f81100f5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.423+0000 7f810f7fe640 1 -- 10.20.193.28:0/4104857923 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810400a590 con 0x7f811010be80 2026-02-25T11:43:54.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.424+0000 7f810ffff640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f81100601b0 0x7f81100f5b00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:43:54.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.424+0000 7f810f7fe640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f811010be80 0x7f81100f6150 secure :-1 s=READY pgs=22 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f810400a910 tx=0x7f8104030e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.424+0000 7f810d7fa640 1 -- 10.20.193.28:0/4104857923 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f810403e070 con 0x7f811010be80 2026-02-25T11:43:54.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.425+0000 7f810d7fa640 1 -- 10.20.193.28:0/4104857923 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f81040053b0 con 0x7f811010be80 2026-02-25T11:43:54.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.425+0000 7f810d7fa640 1 -- 10.20.193.28:0/4104857923 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8104037e70 con 0x7f811010be80 2026-02-25T11:43:54.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.425+0000 7f81164cc640 1 -- 10.20.193.28:0/4104857923 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81100f7a40 con 0x7f811010be80 2026-02-25T11:43:54.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.425+0000 7f81164cc640 1 -- 10.20.193.28:0/4104857923 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8110171490 con 0x7f811010be80 2026-02-25T11:43:54.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.426+0000 7f810d7fa640 1 -- 10.20.193.28:0/4104857923 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f81040039a0 con 0x7f811010be80 2026-02-25T11:43:54.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.427+0000 7f810d7fa640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80ec03f9d0 0x7f80ec05fe80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.427+0000 7f810d7fa640 1 -- 10.20.193.28:0/4104857923 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f810407a300 con 0x7f811010be80 2026-02-25T11:43:54.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.428+0000 7f810ffff640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80ec03f9d0 0x7f80ec05fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.428+0000 7f810ffff640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80ec03f9d0 0x7f80ec05fe80 secure :-1 s=READY pgs=71 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8100004740 tx=0x7f8100005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.429+0000 7f81164cc640 1 -- 10.20.193.28:0/4104857923 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81101110b0 con 0x7f811010be80 2026-02-25T11:43:54.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.432+0000 7f810d7fa640 1 -- 10.20.193.28:0/4104857923 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f8104047020 con 0x7f811010be80 2026-02-25T11:43:54.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.446+0000 7f2d6ffff640 1 -- 10.20.193.28:0/441820481 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f2d600055b0 con 0x7f2da01138f0 2026-02-25T11:43:54.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.449+0000 7f2d8e7fc640 1 -- 10.20.193.28:0/441820481 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v25)=0 v25) ==== 119+0+0 (secure 0 0 0) 0x7f2d9c0103c0 con 0x7f2da01138f0 2026-02-25T11:43:54.454 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.451+0000 7f6f18884640 1 -- 10.20.193.28:0/1738781290 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f6ee4005b30 con 0x7f6f14114860 2026-02-25T11:43:54.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.456+0000 7f2d6ffff640 1 -- 10.20.193.28:0/441820481 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2d7c03fa20 msgr2=0x7f2d7c05fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.456+0000 7f2d6ffff640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2d7c03fa20 0x7f2d7c05fed0 secure :-1 s=READY pgs=67 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2d94004740 tx=0x7f2d94005f90 comp rx=0 tx=0).stop 2026-02-25T11:43:54.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.456+0000 7f2d6ffff640 1 -- 10.20.193.28:0/441820481 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2da01138f0 msgr2=0x7f2da0107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.456+0000 7f2d6ffff640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2da01138f0 0x7f2da0107560 secure :-1 s=READY pgs=129 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2d9c00e5c0 tx=0x7f2d9c00ea90 comp rx=0 tx=0).stop 2026-02-25T11:43:54.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.457+0000 7f2d6ffff640 1 -- 10.20.193.28:0/441820481 shutdown_connections 2026-02-25T11:43:54.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.457+0000 7f2d6ffff640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2d7c03fa20 0x7f2d7c05fed0 unknown :-1 s=CLOSED pgs=67 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.457+0000 7f2d6ffff640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2da01153a0 0x7f2da0107dc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.457+0000 7f2d6ffff640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2da01149f0 0x7f2da0175d30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.457+0000 7f2d6ffff640 1 --2- 10.20.193.28:0/441820481 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2da01138f0 0x7f2da0107560 unknown :-1 s=CLOSED pgs=129 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.457+0000 7f2d6ffff640 1 -- 10.20.193.28:0/441820481 >> 10.20.193.28:0/441820481 conn(0x7f2da00647e0 msgr2=0x7f2da00562f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.457+0000 7f2d6ffff640 1 -- 10.20.193.28:0/441820481 shutdown_connections 2026-02-25T11:43:54.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.457+0000 7f2d6ffff640 1 -- 10.20.193.28:0/441820481 wait complete. 2026-02-25T11:43:54.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.457+0000 7f6ef3fff640 1 -- 10.20.193.28:0/1738781290 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v26)=0 v26) ==== 116+0+0 (secure 0 0 0) 0x7f6efc00fc10 con 0x7f6f14114860 2026-02-25T11:43:54.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.479+0000 7f9e36a92640 1 -- 10.20.193.28:0/1961934642 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3010af50 msgr2=0x7f9e3010b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.479+0000 7f9e36a92640 1 --2- 10.20.193.28:0/1961934642 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3010af50 0x7f9e3010b340 secure :-1 s=READY pgs=134 gs=2 cs=0 l=1 c_cookie=985937504949429520 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9e1801ac20 tx=0x7f9e18040080 comp rx=0 tx=0).stop 2026-02-25T11:43:54.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.483+0000 7f6f18884640 1 -- 10.20.193.28:0/1738781290 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ee003fa20 msgr2=0x7f6ee005fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.483+0000 7f6f18884640 1 --2- 10.20.193.28:0/1738781290 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ee003fa20 0x7f6ee005fed0 secure :-1 s=READY pgs=61 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6f08005b30 tx=0x7f6f08009310 comp rx=0 tx=0).stop 2026-02-25T11:43:54.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.483+0000 7f6f18884640 1 -- 10.20.193.28:0/1738781290 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f14114860 msgr2=0x7f6f141b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.483+0000 7f6f18884640 1 --2- 10.20.193.28:0/1738781290 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f14114860 0x7f6f141b4fe0 secure :-1 s=READY pgs=125 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6efc009480 tx=0x7f6efc00ee60 comp rx=0 tx=0).stop 2026-02-25T11:43:54.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.485+0000 7f6f18884640 1 -- 10.20.193.28:0/1738781290 shutdown_connections 2026-02-25T11:43:54.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.485+0000 7f6f18884640 1 --2- 10.20.193.28:0/1738781290 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ee003fa20 0x7f6ee005fed0 unknown :-1 s=CLOSED pgs=61 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.485+0000 7f6f18884640 1 --2- 10.20.193.28:0/1738781290 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6f14115210 0x7f6f141b5be0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.485+0000 7f6f18884640 1 --2- 10.20.193.28:0/1738781290 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f14114860 0x7f6f141b4fe0 unknown :-1 s=CLOSED pgs=125 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.485+0000 7f6f18884640 1 --2- 10.20.193.28:0/1738781290 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6f1405de60 0x7f6f141b4990 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.485+0000 7f6f18884640 1 -- 10.20.193.28:0/1738781290 >> 10.20.193.28:0/1738781290 conn(0x7f6f1405b890 msgr2=0x7f6f141ab620 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.486+0000 7f6f18884640 1 -- 10.20.193.28:0/1738781290 shutdown_connections 2026-02-25T11:43:54.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.486+0000 7f6f18884640 1 -- 10.20.193.28:0/1738781290 wait complete. 2026-02-25T11:43:54.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.487+0000 7f9e36a92640 1 -- 10.20.193.28:0/1961934642 shutdown_connections 2026-02-25T11:43:54.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.488+0000 7f9e36a92640 1 --2- 10.20.193.28:0/1961934642 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3010c9d0 0x7f9e3010d310 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.488+0000 7f9e36a92640 1 --2- 10.20.193.28:0/1961934642 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9e3010c020 0x7f9e3010c490 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.488+0000 7f9e36a92640 1 --2- 10.20.193.28:0/1961934642 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3010af50 0x7f9e3010b340 unknown :-1 s=CLOSED pgs=134 gs=2 cs=0 l=1 c_cookie=985937504949429520 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.488+0000 7f9e36a92640 1 -- 10.20.193.28:0/1961934642 >> 10.20.193.28:0/1961934642 conn(0x7f9e300e8a00 msgr2=0x7f9e30108e30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.485+0000 7f58e21ac640 1 -- 10.20.193.28:0/1257160216 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f58dc17b5a0 con 0x7f58dc155830 2026-02-25T11:43:54.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.488+0000 7f9e36a92640 1 -- 10.20.193.28:0/1961934642 shutdown_connections 2026-02-25T11:43:54.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.489+0000 7f58d8ff9640 1 -- 10.20.193.28:0/1257160216 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v26)=0 v26) ==== 106+0+0 (secure 0 0 0) 0x7f58c40460b0 con 0x7f58dc155830 2026-02-25T11:43:54.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.489+0000 7f9e36a92640 1 -- 10.20.193.28:0/1961934642 wait complete. 2026-02-25T11:43:54.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.491+0000 7f9e36a92640 1 Processor -- start 2026-02-25T11:43:54.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.491+0000 7f9e36a92640 1 -- start start 2026-02-25T11:43:54.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.491+0000 7f9e36a92640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3010af50 0x7f9e301144a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.492+0000 7f9e34807640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3010af50 0x7f9e301144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.492+0000 7f9e34807640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3010af50 0x7f9e301144a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58904/0 (socket says 10.20.193.28:58904) 2026-02-25T11:43:54.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.492+0000 7f9e34807640 1 -- 10.20.193.28:0/3742703022 learned_addr learned my addr 10.20.193.28:0/3742703022 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:54.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.492+0000 7f9e36a92640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3010c020 0x7f9e30114af0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.493+0000 7f9e36a92640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9e3010c9d0 0x7f9e301156f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.492+0000 7f58e21ac640 1 -- 10.20.193.28:0/1257160216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f58ac03f9d0 msgr2=0x7f58ac05fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.492+0000 7f58e21ac640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f58ac03f9d0 0x7f58ac05fe80 secure :-1 s=READY pgs=68 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f58cc007fd0 tx=0x7f58cc009040 comp rx=0 tx=0).stop 2026-02-25T11:43:54.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.492+0000 7f58e21ac640 1 -- 10.20.193.28:0/1257160216 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f58dc155830 msgr2=0x7f58dc159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.492+0000 7f58e21ac640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f58dc155830 0x7f58dc159110 secure :-1 s=READY pgs=130 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f58c400d670 tx=0x7f58c400db40 comp rx=0 tx=0).stop 2026-02-25T11:43:54.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.493+0000 7f9e36a92640 1 -- 10.20.193.28:0/3742703022 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30115da0 con 0x7f9e3010af50 2026-02-25T11:43:54.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.493+0000 7f58e21ac640 1 -- 10.20.193.28:0/1257160216 shutdown_connections 2026-02-25T11:43:54.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.493+0000 7f9e36a92640 1 -- 10.20.193.28:0/3742703022 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9e30115f10 con 0x7f9e3010c020 2026-02-25T11:43:54.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.493+0000 7f9e36a92640 1 -- 10.20.193.28:0/3742703022 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e301160d0 con 0x7f9e3010c9d0 2026-02-25T11:43:54.499 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.493+0000 7f58e21ac640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f58ac03f9d0 0x7f58ac05fe80 unknown :-1 s=CLOSED pgs=68 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.499 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.493+0000 7f58e21ac640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f58dc155830 0x7f58dc159110 unknown :-1 s=CLOSED pgs=130 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.499 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.493+0000 7f58e21ac640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f58dc134c70 0x7f58dc1346e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.499 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.493+0000 7f58e21ac640 1 --2- 10.20.193.28:0/1257160216 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f58dc05e850 0x7f58dc1588b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.499 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.493+0000 7f58e21ac640 1 -- 10.20.193.28:0/1257160216 >> 10.20.193.28:0/1257160216 conn(0x7f58dc05b440 msgr2=0x7f58dc157690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.499 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.494+0000 7f9e27fff640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3010c020 0x7f9e30114af0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.500 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.494+0000 7f58e21ac640 1 -- 10.20.193.28:0/1257160216 shutdown_connections 2026-02-25T11:43:54.500 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.494+0000 7f9e35008640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9e3010c9d0 0x7f9e301156f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.500 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.494+0000 7f58e21ac640 1 -- 10.20.193.28:0/1257160216 wait complete. 2026-02-25T11:43:54.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.495+0000 7f9e34807640 1 -- 10.20.193.28:0/3742703022 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9e3010c9d0 msgr2=0x7f9e301156f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.495+0000 7f9e34807640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9e3010c9d0 0x7f9e301156f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.495+0000 7f9e34807640 1 -- 10.20.193.28:0/3742703022 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3010c020 msgr2=0x7f9e30114af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.495+0000 7f9e34807640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3010c020 0x7f9e30114af0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.495+0000 7f9e34807640 1 -- 10.20.193.28:0/3742703022 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1801a8c0 con 0x7f9e3010af50 2026-02-25T11:43:54.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.496+0000 7f9e34807640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3010af50 0x7f9e301144a0 secure :-1 s=READY pgs=135 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9e18005e60 tx=0x7f9e1801d020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.497+0000 7f9e25ffb640 1 -- 10.20.193.28:0/3742703022 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9e18005210 con 0x7f9e3010af50 2026-02-25T11:43:54.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.497+0000 7f9e25ffb640 1 -- 10.20.193.28:0/3742703022 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9e180185d0 con 0x7f9e3010af50 2026-02-25T11:43:54.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.497+0000 7f9e25ffb640 1 -- 10.20.193.28:0/3742703022 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9e18049730 con 0x7f9e3010af50 2026-02-25T11:43:54.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.497+0000 7f9e36a92640 1 -- 10.20.193.28:0/3742703022 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e30116350 con 0x7f9e3010af50 2026-02-25T11:43:54.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.497+0000 7f9e36a92640 1 -- 10.20.193.28:0/3742703022 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e30176930 con 0x7f9e3010af50 2026-02-25T11:43:54.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.500+0000 7f9e25ffb640 1 -- 10.20.193.28:0/3742703022 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f9e18018ab0 con 0x7f9e3010af50 2026-02-25T11:43:54.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.500+0000 7f9e36a92640 1 -- 10.20.193.28:0/3742703022 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e30116570 con 0x7f9e3010af50 2026-02-25T11:43:54.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.504+0000 7f9e25ffb640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9e1c03fa50 0x7f9e1c05ff00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.505+0000 7f9e25ffb640 1 -- 10.20.193.28:0/3742703022 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f9e18089ec0 con 0x7f9e3010af50 2026-02-25T11:43:54.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.505+0000 7f9e25ffb640 1 -- 10.20.193.28:0/3742703022 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f9e18048770 con 0x7f9e3010af50 2026-02-25T11:43:54.510 INFO:journalctl@ceph.mgr.y.trial119.stdout:Feb 25 11:43:54 trial119 ceph-mgr[44053]: -- 10.20.193.119:0/1634445626 <== mon.2 v2:10.20.193.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x2a0cfa00 con 0x2ae41000 2026-02-25T11:43:54.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.506+0000 7f9e27fff640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9e1c03fa50 0x7f9e1c05ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.506+0000 7f9e27fff640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9e1c03fa50 0x7f9e1c05ff00 secure :-1 s=READY pgs=72 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9e20004740 tx=0x7f9e2000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.513+0000 7fec5a2f9640 1 -- 10.20.193.28:0/3098716362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec4c0c4180 msgr2=0x7fec4c0c4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.513+0000 7fec5a2f9640 1 --2- 10.20.193.28:0/3098716362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec4c0c4180 0x7fec4c0c4570 secure :-1 s=READY pgs=136 gs=1 cs=0 l=1 c_cookie=7074537979499471305 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec440099a0 tx=0x7fec4402f200 comp rx=0 tx=0).stop 2026-02-25T11:43:54.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.513+0000 7fec5a2f9640 1 -- 10.20.193.28:0/3098716362 shutdown_connections 2026-02-25T11:43:54.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.514+0000 7fec5a2f9640 1 --2- 10.20.193.28:0/3098716362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec4c0c5cc0 0x7fec4c126650 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.514+0000 7fec5a2f9640 1 --2- 10.20.193.28:0/3098716362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec4c0c5310 0x7fec4c0c5780 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.514+0000 7fec5a2f9640 1 --2- 10.20.193.28:0/3098716362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec4c0c4180 0x7fec4c0c4570 unknown :-1 s=CLOSED pgs=136 gs=1 cs=0 l=1 c_cookie=7074537979499471305 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.514+0000 7fec5a2f9640 1 -- 10.20.193.28:0/3098716362 >> 10.20.193.28:0/3098716362 conn(0x7fec4c0a1470 msgr2=0x7fec4c0c1880 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.514+0000 7fec5a2f9640 1 -- 10.20.193.28:0/3098716362 shutdown_connections 2026-02-25T11:43:54.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.514+0000 7fec5a2f9640 1 -- 10.20.193.28:0/3098716362 wait complete. 2026-02-25T11:43:54.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.515+0000 7fec5a2f9640 1 Processor -- start 2026-02-25T11:43:54.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.515+0000 7fec5a2f9640 1 -- start start 2026-02-25T11:43:54.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.516+0000 7fec5a2f9640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec4c0c4180 0x7fec4c0cdba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.516+0000 7fec5a2f9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec4c0c5310 0x7fec4c0ce1f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.516+0000 7fec5a2f9640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec4c0c5cc0 0x7fec4c0cedf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.516+0000 7fec5a2f9640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c0cf530 con 0x7fec4c0c5310 2026-02-25T11:43:54.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.516+0000 7fec5a2f9640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c0cf6a0 con 0x7fec4c0c5cc0 2026-02-25T11:43:54.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.516+0000 7fec5a2f9640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c0cf860 con 0x7fec4c0c4180 2026-02-25T11:43:54.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.517+0000 7fec592f7640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec4c0c4180 0x7fec4c0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.517+0000 7fec58af6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec4c0c5310 0x7fec4c0ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.517+0000 7fec592f7640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec4c0c4180 0x7fec4c0cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56700/0 (socket says 10.20.193.28:56700) 2026-02-25T11:43:54.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.517+0000 7fec592f7640 1 -- 10.20.193.28:0/1216333208 learned_addr learned my addr 10.20.193.28:0/1216333208 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:54.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.518+0000 7fec59af8640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec4c0c5cc0 0x7fec4c0cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.518+0000 7fec58af6640 1 -- 10.20.193.28:0/1216333208 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec4c0c4180 msgr2=0x7fec4c0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.518+0000 7f76fd1b5640 1 -- 10.20.193.28:0/4037226528 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f76f001a480 con 0x7f76f00c3460 2026-02-25T11:43:54.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.518+0000 7fec58af6640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec4c0c4180 0x7fec4c0cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.518+0000 7fec58af6640 1 -- 10.20.193.28:0/1216333208 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec4c0c5cc0 msgr2=0x7fec4c0cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.518+0000 7fec58af6640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec4c0c5cc0 0x7fec4c0cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.518+0000 7fec58af6640 1 -- 10.20.193.28:0/1216333208 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec44009570 con 0x7fec4c0c5310 2026-02-25T11:43:54.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.519+0000 7fec59af8640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec4c0c5cc0 0x7fec4c0cedf0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:43:54.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.520+0000 7fec58af6640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec4c0c5310 0x7fec4c0ce1f0 secure :-1 s=READY pgs=137 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec48004740 tx=0x7fec4800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.520+0000 7fec427fc640 1 -- 10.20.193.28:0/1216333208 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fec48004520 con 0x7fec4c0c5310 2026-02-25T11:43:54.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.520+0000 7fec427fc640 1 -- 10.20.193.28:0/1216333208 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fec48002b60 con 0x7fec4c0c5310 2026-02-25T11:43:54.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.520+0000 7fec427fc640 1 -- 10.20.193.28:0/1216333208 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fec48010cb0 con 0x7fec4c0c5310 2026-02-25T11:43:54.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.521+0000 7fec5a2f9640 1 -- 10.20.193.28:0/1216333208 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c0cfb40 con 0x7fec4c0c5310 2026-02-25T11:43:54.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.521+0000 7fec5a2f9640 1 -- 10.20.193.28:0/1216333208 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4c110100 con 0x7fec4c0c5310 2026-02-25T11:43:54.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.523+0000 7fec427fc640 1 -- 10.20.193.28:0/1216333208 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7fec48002cd0 con 0x7fec4c0c5310 2026-02-25T11:43:54.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.524+0000 7fec5a2f9640 1 -- 10.20.193.28:0/1216333208 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1c005300 con 0x7fec4c0c5310 2026-02-25T11:43:54.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.524+0000 7f76f57fa640 1 -- 10.20.193.28:0/4037226528 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v27)=0 v27) ==== 125+0+0 (secure 0 0 0) 0x7f76e4016be0 con 0x7f76f00c3460 2026-02-25T11:43:54.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.524+0000 7fec427fc640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec3803fa00 0x7fec3805feb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:54.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.525+0000 7fec592f7640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec3803fa00 0x7fec3805feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:54.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.527+0000 7fec592f7640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec3803fa00 0x7fec3805feb0 secure :-1 s=READY pgs=73 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec44005970 tx=0x7fec44005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:54.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.527+0000 7fec427fc640 1 -- 10.20.193.28:0/1216333208 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7fec4805c830 con 0x7fec4c0c5310 2026-02-25T11:43:54.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.528+0000 7f76fd1b5640 1 -- 10.20.193.28:0/4037226528 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f76cc03fa30 msgr2=0x7f76cc05fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.528+0000 7f76fd1b5640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f76cc03fa30 0x7f76cc05fee0 secure :-1 s=READY pgs=69 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f76e8005970 tx=0x7f76e8005d10 comp rx=0 tx=0).stop 2026-02-25T11:43:54.532 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.528+0000 7f76fd1b5640 1 -- 10.20.193.28:0/4037226528 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76f00c3460 msgr2=0x7f76f00d27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.532 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.528+0000 7f76fd1b5640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76f00c3460 0x7f76f00d27a0 secure :-1 s=READY pgs=20 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f76e4009fd0 tx=0x7f76e4002bc0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.532 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.528+0000 7fec427fc640 1 -- 10.20.193.28:0/1216333208 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fec48059060 con 0x7fec4c0c5310 2026-02-25T11:43:54.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.530+0000 7f76fd1b5640 1 -- 10.20.193.28:0/4037226528 shutdown_connections 2026-02-25T11:43:54.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.530+0000 7f76fd1b5640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f76cc03fa30 0x7f76cc05fee0 unknown :-1 s=CLOSED pgs=69 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.530+0000 7f76fd1b5640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76f00c5850 0x7f76f00d39f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.530+0000 7f76fd1b5640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f76f00c4750 0x7f76f00d2df0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.530+0000 7f76fd1b5640 1 --2- 10.20.193.28:0/4037226528 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76f00c3460 0x7f76f00d27a0 unknown :-1 s=CLOSED pgs=20 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.530+0000 7f76fd1b5640 1 -- 10.20.193.28:0/4037226528 >> 10.20.193.28:0/4037226528 conn(0x7f76f00a1470 msgr2=0x7f76f00c1850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.530+0000 7f76fd1b5640 1 -- 10.20.193.28:0/4037226528 shutdown_connections 2026-02-25T11:43:54.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.530+0000 7f76fd1b5640 1 -- 10.20.193.28:0/4037226528 wait complete. 2026-02-25T11:43:54.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.556+0000 7efe4e6e6640 1 -- 10.20.193.28:0/2361979087 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7efe481194e0 con 0x7efe48114860 2026-02-25T11:43:54.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.569+0000 7f83c09dd640 1 -- 10.20.193.28:0/4147960216 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f83bc10b230 con 0x7f83bc10be70 2026-02-25T11:43:54.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.569+0000 7efe367fc640 1 -- 10.20.193.28:0/2361979087 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v28)=0 v28) ==== 119+0+0 (secure 0 0 0) 0x7efe38045180 con 0x7efe48114860 2026-02-25T11:43:54.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.574+0000 7f9950052640 1 -- 10.20.193.28:0/2620875365 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f99100051c0 con 0x7f99400b3880 2026-02-25T11:43:54.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.574+0000 7efe4e6e6640 1 -- 10.20.193.28:0/2361979087 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efe2403f9d0 msgr2=0x7efe2405fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.574+0000 7efe4e6e6640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efe2403f9d0 0x7efe2405fe80 secure :-1 s=READY pgs=63 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efe44005860 tx=0x7efe44003840 comp rx=0 tx=0).stop 2026-02-25T11:43:54.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.575+0000 7efe4e6e6640 1 -- 10.20.193.28:0/2361979087 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efe48114860 msgr2=0x7efe480fe370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.575+0000 7efe4e6e6640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efe48114860 0x7efe480fe370 secure :-1 s=READY pgs=14 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efe38005970 tx=0x7efe3802fcb0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.575+0000 7efe4e6e6640 1 -- 10.20.193.28:0/2361979087 shutdown_connections 2026-02-25T11:43:54.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.575+0000 7efe4e6e6640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efe2403f9d0 0x7efe2405fe80 unknown :-1 s=CLOSED pgs=63 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.575+0000 7efe4e6e6640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efe48115210 0x7efe480fef70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.575+0000 7efe4e6e6640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efe48114860 0x7efe480fe370 unknown :-1 s=CLOSED pgs=14 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.575+0000 7efe4e6e6640 1 --2- 10.20.193.28:0/2361979087 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7efe4805de60 0x7efe480fdd20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.575+0000 7efe4e6e6640 1 -- 10.20.193.28:0/2361979087 >> 10.20.193.28:0/2361979087 conn(0x7efe4805b890 msgr2=0x7efe480f4790 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.574+0000 7f83b8ff9640 1 -- 10.20.193.28:0/4147960216 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v29)=0 v29) ==== 121+0+0 (secure 0 0 0) 0x7f83a800d380 con 0x7f83bc10be70 2026-02-25T11:43:54.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.577+0000 7efe4e6e6640 1 -- 10.20.193.28:0/2361979087 shutdown_connections 2026-02-25T11:43:54.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.577+0000 7efe4e6e6640 1 -- 10.20.193.28:0/2361979087 wait complete. 2026-02-25T11:43:54.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.580+0000 7f9936ffd640 1 -- 10.20.193.28:0/2620875365 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v30)=0 v30) ==== 111+0+0 (secure 0 0 0) 0x7f993c01f380 con 0x7f99400b3880 2026-02-25T11:43:54.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.583+0000 7f9950052640 1 -- 10.20.193.28:0/2620875365 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f992403fa20 msgr2=0x7f992405fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.583+0000 7f9950052640 1 --2- 10.20.193.28:0/2620875365 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f992403fa20 0x7f992405fed0 secure :-1 s=READY pgs=64 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9938004740 tx=0x7f9938009190 comp rx=0 tx=0).stop 2026-02-25T11:43:54.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.583+0000 7f9950052640 1 -- 10.20.193.28:0/2620875365 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99400b3880 msgr2=0x7f994009d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.583+0000 7f9950052640 1 --2- 10.20.193.28:0/2620875365 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99400b3880 0x7f994009d0e0 secure :-1 s=READY pgs=16 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f993c009480 tx=0x7f993c00ee60 comp rx=0 tx=0).stop 2026-02-25T11:43:54.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.587+0000 7f83c09dd640 1 -- 10.20.193.28:0/4147960216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f838c03fa20 msgr2=0x7f838c05fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.587+0000 7f83c09dd640 1 --2- 10.20.193.28:0/4147960216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f838c03fa20 0x7f838c05fed0 secure :-1 s=READY pgs=70 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f83b0005820 tx=0x7f83b0005180 comp rx=0 tx=0).stop 2026-02-25T11:43:54.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.587+0000 7f83c09dd640 1 -- 10.20.193.28:0/4147960216 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f83bc10be70 msgr2=0x7f83bc0f3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.587+0000 7f83c09dd640 1 --2- 10.20.193.28:0/4147960216 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f83bc10be70 0x7f83bc0f3970 secure :-1 s=READY pgs=21 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f83a800b500 tx=0x7f83a800b9d0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.593+0000 7f83c09dd640 1 -- 10.20.193.28:0/4147960216 shutdown_connections 2026-02-25T11:43:54.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.593+0000 7f83c09dd640 1 --2- 10.20.193.28:0/4147960216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f838c03fa20 0x7f838c05fed0 unknown :-1 s=CLOSED pgs=70 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.593+0000 7f83c09dd640 1 --2- 10.20.193.28:0/4147960216 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f83bc10c820 0x7f83bc0f4570 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.593+0000 7f83c09dd640 1 --2- 10.20.193.28:0/4147960216 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f83bc10be70 0x7f83bc0f3970 unknown :-1 s=CLOSED pgs=21 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.593+0000 7f83c09dd640 1 --2- 10.20.193.28:0/4147960216 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f83bc10ae40 0x7f83bc0f3320 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.593+0000 7f83c09dd640 1 -- 10.20.193.28:0/4147960216 >> 10.20.193.28:0/4147960216 conn(0x7f83bc0e8860 msgr2=0x7f83bc109c50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.594+0000 7f83c09dd640 1 -- 10.20.193.28:0/4147960216 shutdown_connections 2026-02-25T11:43:54.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.594+0000 7f83c09dd640 1 -- 10.20.193.28:0/4147960216 wait complete. 2026-02-25T11:43:54.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.612+0000 7f81164cc640 1 -- 10.20.193.28:0/4104857923 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f8110061920 con 0x7f811010be80 2026-02-25T11:43:54.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.615+0000 7f810d7fa640 1 -- 10.20.193.28:0/4104857923 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v31)=0 v31) ==== 121+0+0 (secure 0 0 0) 0x7f81040063c0 con 0x7f811010be80 2026-02-25T11:43:54.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.608+0000 7f9950052640 1 -- 10.20.193.28:0/2620875365 shutdown_connections 2026-02-25T11:43:54.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.608+0000 7f9950052640 1 --2- 10.20.193.28:0/2620875365 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f992403fa20 0x7f992405fed0 unknown :-1 s=CLOSED pgs=64 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.608+0000 7f9950052640 1 --2- 10.20.193.28:0/2620875365 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99400b4430 0x7f994009dce0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.608+0000 7f9950052640 1 --2- 10.20.193.28:0/2620875365 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99400b3880 0x7f994009d0e0 unknown :-1 s=CLOSED pgs=16 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.608+0000 7f9950052640 1 --2- 10.20.193.28:0/2620875365 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f99400b2e50 0x7f994009ca90 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.608+0000 7f9950052640 1 -- 10.20.193.28:0/2620875365 >> 10.20.193.28:0/2620875365 conn(0x7f9940090970 msgr2=0x7f99400933c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.617+0000 7f9950052640 1 -- 10.20.193.28:0/2620875365 shutdown_connections 2026-02-25T11:43:54.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.617+0000 7f9950052640 1 -- 10.20.193.28:0/2620875365 wait complete. 2026-02-25T11:43:54.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.618+0000 7f81164cc640 1 -- 10.20.193.28:0/4104857923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80ec03f9d0 msgr2=0x7f80ec05fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.619+0000 7f81164cc640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80ec03f9d0 0x7f80ec05fe80 secure :-1 s=READY pgs=71 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8100004740 tx=0x7f8100005ab0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.619+0000 7f81164cc640 1 -- 10.20.193.28:0/4104857923 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f811010be80 msgr2=0x7f81100f6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.619+0000 7f81164cc640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f811010be80 0x7f81100f6150 secure :-1 s=READY pgs=22 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f810400a910 tx=0x7f8104030e00 comp rx=0 tx=0).stop 2026-02-25T11:43:54.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.619+0000 7f81164cc640 1 -- 10.20.193.28:0/4104857923 shutdown_connections 2026-02-25T11:43:54.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.619+0000 7f81164cc640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80ec03f9d0 0x7f80ec05fe80 unknown :-1 s=CLOSED pgs=71 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.619+0000 7f81164cc640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f811010c9e0 0x7f81100f6d50 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.619+0000 7f81164cc640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f811010be80 0x7f81100f6150 unknown :-1 s=CLOSED pgs=22 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.619+0000 7f81164cc640 1 --2- 10.20.193.28:0/4104857923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f81100601b0 0x7f81100f5b00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.619+0000 7f81164cc640 1 -- 10.20.193.28:0/4104857923 >> 10.20.193.28:0/4104857923 conn(0x7f81100e9560 msgr2=0x7f81100ec150 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.619+0000 7f81164cc640 1 -- 10.20.193.28:0/4104857923 shutdown_connections 2026-02-25T11:43:54.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.619+0000 7f81164cc640 1 -- 10.20.193.28:0/4104857923 wait complete. 2026-02-25T11:43:54.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.654+0000 7f9e36a92640 1 -- 10.20.193.28:0/3742703022 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f9e301113e0 con 0x7f9e3010af50 2026-02-25T11:43:54.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.663+0000 7f9e25ffb640 1 -- 10.20.193.28:0/3742703022 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v32)=0 v32) ==== 120+0+0 (secure 0 0 0) 0x7f9e180489c0 con 0x7f9e3010af50 2026-02-25T11:43:54.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.666+0000 7f9e36a92640 1 -- 10.20.193.28:0/3742703022 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9e1c03fa50 msgr2=0x7f9e1c05ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.666+0000 7f9e36a92640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9e1c03fa50 0x7f9e1c05ff00 secure :-1 s=READY pgs=72 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9e20004740 tx=0x7f9e2000a380 comp rx=0 tx=0).stop 2026-02-25T11:43:54.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.667+0000 7f9e36a92640 1 -- 10.20.193.28:0/3742703022 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3010af50 msgr2=0x7f9e301144a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.667+0000 7f9e36a92640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3010af50 0x7f9e301144a0 secure :-1 s=READY pgs=135 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9e18005e60 tx=0x7f9e1801d020 comp rx=0 tx=0).stop 2026-02-25T11:43:54.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.667+0000 7f9e36a92640 1 -- 10.20.193.28:0/3742703022 shutdown_connections 2026-02-25T11:43:54.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.667+0000 7f9e36a92640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9e1c03fa50 0x7f9e1c05ff00 unknown :-1 s=CLOSED pgs=72 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.667+0000 7f9e36a92640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9e3010c9d0 0x7f9e301156f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.667+0000 7f9e36a92640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3010c020 0x7f9e30114af0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.667+0000 7f9e36a92640 1 --2- 10.20.193.28:0/3742703022 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3010af50 0x7f9e301144a0 unknown :-1 s=CLOSED pgs=135 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.667+0000 7f9e36a92640 1 -- 10.20.193.28:0/3742703022 >> 10.20.193.28:0/3742703022 conn(0x7f9e300e8a00 msgr2=0x7f9e30108de0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.668+0000 7f9e36a92640 1 -- 10.20.193.28:0/3742703022 shutdown_connections 2026-02-25T11:43:54.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.669+0000 7f9e36a92640 1 -- 10.20.193.28:0/3742703022 wait complete. 2026-02-25T11:43:54.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.677+0000 7fec5a2f9640 1 -- 10.20.193.28:0/1216333208 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fec1c005950 con 0x7fec4c0c5310 2026-02-25T11:43:54.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.682+0000 7fec427fc640 1 -- 10.20.193.28:0/1216333208 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v33)=0 v33) ==== 121+0+0 (secure 0 0 0) 0x7fec48059070 con 0x7fec4c0c5310 2026-02-25T11:43:54.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.684+0000 7fec5a2f9640 1 -- 10.20.193.28:0/1216333208 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec3803fa00 msgr2=0x7fec3805feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.684+0000 7fec5a2f9640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec3803fa00 0x7fec3805feb0 secure :-1 s=READY pgs=73 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec44005970 tx=0x7fec44005900 comp rx=0 tx=0).stop 2026-02-25T11:43:54.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.684+0000 7fec5a2f9640 1 -- 10.20.193.28:0/1216333208 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec4c0c5310 msgr2=0x7fec4c0ce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:54.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.684+0000 7fec5a2f9640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec4c0c5310 0x7fec4c0ce1f0 secure :-1 s=READY pgs=137 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec48004740 tx=0x7fec4800b8f0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.685+0000 7fec5a2f9640 1 -- 10.20.193.28:0/1216333208 shutdown_connections 2026-02-25T11:43:54.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.685+0000 7fec5a2f9640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec3803fa00 0x7fec3805feb0 unknown :-1 s=CLOSED pgs=73 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.685+0000 7fec5a2f9640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec4c0c5cc0 0x7fec4c0cedf0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.685+0000 7fec5a2f9640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec4c0c5310 0x7fec4c0ce1f0 unknown :-1 s=CLOSED pgs=137 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.685+0000 7fec5a2f9640 1 --2- 10.20.193.28:0/1216333208 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec4c0c4180 0x7fec4c0cdba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:54.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.685+0000 7fec5a2f9640 1 -- 10.20.193.28:0/1216333208 >> 10.20.193.28:0/1216333208 conn(0x7fec4c0a1470 msgr2=0x7fec4c0c1850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:54.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.685+0000 7fec5a2f9640 1 -- 10.20.193.28:0/1216333208 shutdown_connections 2026-02-25T11:43:54.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:54.685+0000 7fec5a2f9640 1 -- 10.20.193.28:0/1216333208 wait complete. 2026-02-25T11:43:54.812 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='trial028') 2026-02-25T11:43:54.813 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='trial028') 2026-02-25T11:43:54.813 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='trial028') 2026-02-25T11:43:54.813 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='trial028') 2026-02-25T11:43:55.000 INFO:journalctl@ceph.mgr.z.trial167.stdout:Feb 25 11:43:54 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-z[44948]: /lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-25T11:43:55.000 INFO:journalctl@ceph.mgr.z.trial167.stdout:Feb 25 11:43:54 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-z[44948]: Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-25T11:43:55.000 INFO:journalctl@ceph.mgr.z.trial167.stdout:Feb 25 11:43:54 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-z[44948]: from numpy import show_config as show_numpy_config 2026-02-25T11:43:55.110 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='trial028') 2026-02-25T11:43:55.111 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='trial028') 2026-02-25T11:43:55.111 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='trial028') 2026-02-25T11:43:55.111 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='trial028') 2026-02-25T11:43:55.111 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='trial028') 2026-02-25T11:43:55.111 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='trial028') 2026-02-25T11:43:55.111 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='trial028') 2026-02-25T11:43:55.111 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='trial028') 2026-02-25T11:43:55.111 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='trial028') 2026-02-25T11:43:55.111 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='trial028') 2026-02-25T11:43:55.111 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '123fa7bf-123f-11f1-80da-d404e6e7d460', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='trial028') 2026-02-25T11:43:55.111 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config dump 2026-02-25T11:43:55.201 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: Deploying daemon mgr.y on trial119 2026-02-25T11:43:55.201 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/342622757' entity='client.admin' 2026-02-25T11:43:55.202 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.202 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.202 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.202 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/441820481' entity='client.admin' 2026-02-25T11:43:55.202 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1738781290' entity='client.admin' 2026-02-25T11:43:55.202 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.202 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.202 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:55.202 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:55.202 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:55.203 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.203 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:55.203 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.203 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.203 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3742703022' entity='client.admin' 2026-02-25T11:43:55.203 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1216333208' entity='client.admin' 2026-02-25T11:43:55.203 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:55.335 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:55.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: Deploying daemon mgr.y on trial119 2026-02-25T11:43:55.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/342622757' entity='client.admin' 2026-02-25T11:43:55.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/441820481' entity='client.admin' 2026-02-25T11:43:55.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1738781290' entity='client.admin' 2026-02-25T11:43:55.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:55.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:55.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:55.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:55.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3742703022' entity='client.admin' 2026-02-25T11:43:55.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1216333208' entity='client.admin' 2026-02-25T11:43:55.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:55.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: Deploying daemon mgr.y on trial119 2026-02-25T11:43:55.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/342622757' entity='client.admin' 2026-02-25T11:43:55.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/441820481' entity='client.admin' 2026-02-25T11:43:55.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1738781290' entity='client.admin' 2026-02-25T11:43:55.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:55.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:55.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:55.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:55.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='client.? ' entity='client.admin' 2026-02-25T11:43:55.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3742703022' entity='client.admin' 2026-02-25T11:43:55.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1216333208' entity='client.admin' 2026-02-25T11:43:55.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:55.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.514+0000 7f5adb44a640 1 -- 10.20.193.28:0/2810342997 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad410b620 msgr2=0x7f5ad410ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:55.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.514+0000 7f5adb44a640 1 --2- 10.20.193.28:0/2810342997 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad410b620 0x7f5ad410ba90 secure :-1 s=READY pgs=138 gs=1 cs=0 l=1 c_cookie=11289646328354893144 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ad000aa30 tx=0x7f5ad0030270 comp rx=0 tx=0).stop 2026-02-25T11:43:55.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.514+0000 7f5adb44a640 1 -- 10.20.193.28:0/2810342997 shutdown_connections 2026-02-25T11:43:55.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.514+0000 7f5adb44a640 1 --2- 10.20.193.28:0/2810342997 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad410b620 0x7f5ad410ba90 unknown :-1 s=CLOSED pgs=138 gs=1 cs=0 l=1 c_cookie=11289646328354893144 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:55.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.514+0000 7f5adb44a640 1 --2- 10.20.193.28:0/2810342997 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ad410de40 0x7f5ad410b090 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:55.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.514+0000 7f5adb44a640 1 --2- 10.20.193.28:0/2810342997 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad410d510 0x7f5ad410d900 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:55.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.514+0000 7f5adb44a640 1 -- 10.20.193.28:0/2810342997 >> 10.20.193.28:0/2810342997 conn(0x7f5ad40e8cb0 msgr2=0x7f5ad41090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:55.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.514+0000 7f5adb44a640 1 -- 10.20.193.28:0/2810342997 shutdown_connections 2026-02-25T11:43:55.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.514+0000 7f5adb44a640 1 -- 10.20.193.28:0/2810342997 wait complete. 2026-02-25T11:43:55.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.515+0000 7f5adb44a640 1 Processor -- start 2026-02-25T11:43:55.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.515+0000 7f5adb44a640 1 -- start start 2026-02-25T11:43:55.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5adb44a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad410b620 0x7f5ad40f47d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:55.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5adb44a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad410d510 0x7f5ad40f4e20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:55.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5adb44a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ad410de40 0x7f5ad40f5a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:55.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5adb44a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad40f6160 con 0x7f5ad410d510 2026-02-25T11:43:55.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5adb44a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad40f62d0 con 0x7f5ad410b620 2026-02-25T11:43:55.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5adb44a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad40f6490 con 0x7f5ad410de40 2026-02-25T11:43:55.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5ad9c47640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad410d510 0x7f5ad40f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:55.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5ada448640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad410b620 0x7f5ad40f47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:55.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5ada448640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad410b620 0x7f5ad40f47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:52266/0 (socket says 10.20.193.28:52266) 2026-02-25T11:43:55.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5ada448640 1 -- 10.20.193.28:0/2514031362 learned_addr learned my addr 10.20.193.28:0/2514031362 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:43:55.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5adac49640 1 --2- 10.20.193.28:0/2514031362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ad410de40 0x7f5ad40f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:55.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5ad9c47640 1 -- 10.20.193.28:0/2514031362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ad410de40 msgr2=0x7f5ad40f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:55.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5ad9c47640 1 --2- 10.20.193.28:0/2514031362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ad410de40 0x7f5ad40f5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:55.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5ad9c47640 1 -- 10.20.193.28:0/2514031362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad410b620 msgr2=0x7f5ad40f47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:55.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5ad9c47640 1 --2- 10.20.193.28:0/2514031362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad410b620 0x7f5ad40f47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:55.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.516+0000 7f5ad9c47640 1 -- 10.20.193.28:0/2514031362 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad000a650 con 0x7f5ad410d510 2026-02-25T11:43:55.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.517+0000 7f5ad9c47640 1 --2- 10.20.193.28:0/2514031362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad410d510 0x7f5ad40f4e20 secure :-1 s=READY pgs=139 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac8004740 tx=0x7f5ac800de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:55.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.517+0000 7f5abb7fe640 1 -- 10.20.193.28:0/2514031362 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5ac80042f0 con 0x7f5ad410d510 2026-02-25T11:43:55.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.517+0000 7f5abb7fe640 1 -- 10.20.193.28:0/2514031362 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5ac8009da0 con 0x7f5ad410d510 2026-02-25T11:43:55.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.517+0000 7f5abb7fe640 1 -- 10.20.193.28:0/2514031362 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5ac8010460 con 0x7f5ad410d510 2026-02-25T11:43:55.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.517+0000 7f5adb44a640 1 -- 10.20.193.28:0/2514031362 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad40f6770 con 0x7f5ad410d510 2026-02-25T11:43:55.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.517+0000 7f5adb44a640 1 -- 10.20.193.28:0/2514031362 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad40f6c90 con 0x7f5ad410d510 2026-02-25T11:43:55.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.517+0000 7f5adb44a640 1 -- 10.20.193.28:0/2514031362 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad4061010 con 0x7f5ad410d510 2026-02-25T11:43:55.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.518+0000 7f5abb7fe640 1 -- 10.20.193.28:0/2514031362 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 12) ==== 52870+0+0 (secure 0 0 0) 0x7f5ac80098e0 con 0x7f5ad410d510 2026-02-25T11:43:55.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.519+0000 7f5abb7fe640 1 --2- 10.20.193.28:0/2514031362 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ab003f9d0 0x7f5ab005fe80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:43:55.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.519+0000 7f5abb7fe640 1 -- 10.20.193.28:0/2514031362 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7f5ac8053d90 con 0x7f5ad410d510 2026-02-25T11:43:55.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.520+0000 7f5ada448640 1 --2- 10.20.193.28:0/2514031362 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ab003f9d0 0x7f5ab005fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:43:55.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.520+0000 7f5ada448640 1 --2- 10.20.193.28:0/2514031362 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ab003f9d0 0x7f5ab005fe80 secure :-1 s=READY pgs=74 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac400b2c0 tx=0x7f5ac400b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:43:55.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.521+0000 7f5abb7fe640 1 -- 10.20.193.28:0/2514031362 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f5ac801a640 con 0x7f5ad410d510 2026-02-25T11:43:55.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.636+0000 7f5adb44a640 1 -- 10.20.193.28:0/2514031362 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f5ad41755c0 con 0x7f5ad410d510 2026-02-25T11:43:55.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.637+0000 7f5abb7fe640 1 -- 10.20.193.28:0/2514031362 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+18868 (secure 0 0 0) 0x7f5ac8015080 con 0x7f5ad410d510 2026-02-25T11:43:55.641 INFO:teuthology.orchestra.run.trial028.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-25T11:43:55.642 INFO:teuthology.orchestra.run.trial028.stdout:global dev auth_debug true 2026-02-25T11:43:55.642 INFO:teuthology.orchestra.run.trial028.stdout:global basic container_image quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 * 2026-02-25T11:43:55.642 INFO:teuthology.orchestra.run.trial028.stdout:global dev debug_asserts_on_shutdown true 2026-02-25T11:43:55.642 INFO:teuthology.orchestra.run.trial028.stdout:global basic log_to_file true 2026-02-25T11:43:55.642 INFO:teuthology.orchestra.run.trial028.stdout:global basic log_to_journald false 2026-02-25T11:43:55.642 INFO:teuthology.orchestra.run.trial028.stdout:global basic log_to_stderr false 2026-02-25T11:43:55.642 INFO:teuthology.orchestra.run.trial028.stdout:global advanced mon_allow_pool_delete true 2026-02-25T11:43:55.642 INFO:teuthology.orchestra.run.trial028.stdout:global advanced mon_clock_drift_allowed 1.000000 2026-02-25T11:43:55.642 INFO:teuthology.orchestra.run.trial028.stdout:global advanced mon_cluster_log_to_file true 2026-02-25T11:43:55.642 INFO:teuthology.orchestra.run.trial028.stdout:global advanced mon_max_pg_per_osd 10000 2026-02-25T11:43:55.643 INFO:teuthology.orchestra.run.trial028.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2026-02-25T11:43:55.643 INFO:teuthology.orchestra.run.trial028.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2026-02-25T11:43:55.643 INFO:teuthology.orchestra.run.trial028.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2026-02-25T11:43:55.643 INFO:teuthology.orchestra.run.trial028.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2026-02-25T11:43:55.643 INFO:teuthology.orchestra.run.trial028.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2026-02-25T11:43:55.643 INFO:teuthology.orchestra.run.trial028.stdout:global advanced mon_warn_on_too_few_osds false 2026-02-25T11:43:55.643 INFO:teuthology.orchestra.run.trial028.stdout:global dev ms_die_on_bug true 2026-02-25T11:43:55.643 INFO:teuthology.orchestra.run.trial028.stdout:global dev ms_die_on_old_message true 2026-02-25T11:43:55.643 INFO:teuthology.orchestra.run.trial028.stdout:global advanced osd_pool_default_erasure_code_profile plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-25T11:43:55.643 INFO:teuthology.orchestra.run.trial028.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2026-02-25T11:43:55.643 INFO:teuthology.orchestra.run.trial028.stdout:global advanced public_network 10.20.192.0/20 * 2026-02-25T11:43:55.644 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2026-02-25T11:43:55.644 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2026-02-25T11:43:55.644 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced auth_service_ticket_ttl 240.000000 2026-02-25T11:43:55.644 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced debug_mon 20/20 2026-02-25T11:43:55.644 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced debug_ms 1/1 2026-02-25T11:43:55.644 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced debug_paxos 20/20 2026-02-25T11:43:55.644 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced mon_data_avail_warn 5 2026-02-25T11:43:55.644 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced mon_down_mkfs_grace 300 2026-02-25T11:43:55.644 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced mon_mgr_mkfs_grace 240 2026-02-25T11:43:55.644 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced mon_op_complaint_time 120 2026-02-25T11:43:55.645 INFO:teuthology.orchestra.run.trial028.stdout:mon dev mon_osd_prime_pg_temp true 2026-02-25T11:43:55.645 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced mon_osd_reporter_subtree_level osd 2026-02-25T11:43:55.645 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2026-02-25T11:43:55.645 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2026-02-25T11:43:55.645 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2026-02-25T11:43:55.645 INFO:teuthology.orchestra.run.trial028.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2026-02-25T11:43:55.645 INFO:teuthology.orchestra.run.trial028.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2026-02-25T11:43:55.645 INFO:teuthology.orchestra.run.trial028.stdout:mgr advanced client_mount_timeout 30 2026-02-25T11:43:55.645 INFO:teuthology.orchestra.run.trial028.stdout:mgr advanced debug_client 20/20 2026-02-25T11:43:55.645 INFO:teuthology.orchestra.run.trial028.stdout:mgr advanced debug_mgr 20/20 2026-02-25T11:43:55.645 INFO:teuthology.orchestra.run.trial028.stdout:mgr advanced debug_ms 1/1 2026-02-25T11:43:55.645 INFO:teuthology.orchestra.run.trial028.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2026-02-25T11:43:55.646 INFO:teuthology.orchestra.run.trial028.stdout:mgr advanced mgr/cephadm/container_init True * 2026-02-25T11:43:55.646 INFO:teuthology.orchestra.run.trial028.stdout:mgr advanced mgr/cephadm/migration_current 8 * 2026-02-25T11:43:55.646 INFO:teuthology.orchestra.run.trial028.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2026-02-25T11:43:55.646 INFO:teuthology.orchestra.run.trial028.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2026-02-25T11:43:55.646 INFO:teuthology.orchestra.run.trial028.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2026-02-25T11:43:55.646 INFO:teuthology.orchestra.run.trial028.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2026-02-25T11:43:55.646 INFO:teuthology.orchestra.run.trial028.stdout:mgr dev mon_warn_on_pool_no_app false 2026-02-25T11:43:55.646 INFO:teuthology.orchestra.run.trial028.stdout:osd dev bdev_debug_aio true 2026-02-25T11:43:55.646 INFO:teuthology.orchestra.run.trial028.stdout:osd dev bluestore_block_size 96636764160 * 2026-02-25T11:43:55.646 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced bluestore_compression_mode aggressive 2026-02-25T11:43:55.647 INFO:teuthology.orchestra.run.trial028.stdout:osd dev bluestore_fsck_on_mount true 2026-02-25T11:43:55.647 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced debug_bluefs 20/20 2026-02-25T11:43:55.647 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced debug_bluestore 20/20 2026-02-25T11:43:55.647 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced debug_ms 1/1 2026-02-25T11:43:55.647 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced debug_osd 20/20 2026-02-25T11:43:55.647 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced debug_rocksdb 10/10 2026-02-25T11:43:55.647 INFO:teuthology.orchestra.run.trial028.stdout:osd dev osd_debug_misdirected_ops true 2026-02-25T11:43:55.647 INFO:teuthology.orchestra.run.trial028.stdout:osd dev osd_debug_op_order true 2026-02-25T11:43:55.647 INFO:teuthology.orchestra.run.trial028.stdout:osd dev osd_debug_pg_log_writeout true 2026-02-25T11:43:55.647 INFO:teuthology.orchestra.run.trial028.stdout:osd dev osd_debug_shutdown true 2026-02-25T11:43:55.648 INFO:teuthology.orchestra.run.trial028.stdout:osd dev osd_debug_verify_cached_snaps true 2026-02-25T11:43:55.648 INFO:teuthology.orchestra.run.trial028.stdout:osd dev osd_debug_verify_missing_on_start true 2026-02-25T11:43:55.648 INFO:teuthology.orchestra.run.trial028.stdout:osd dev osd_debug_verify_stray_on_activate true 2026-02-25T11:43:55.648 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2026-02-25T11:43:55.648 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2026-02-25T11:43:55.648 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_max_omap_entries_per_request 10000 2026-02-25T11:43:55.648 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_mclock_profile high_recovery_ops 2026-02-25T11:43:55.648 INFO:teuthology.orchestra.run.trial028.stdout:osd dev osd_mclock_skip_benchmark true 2026-02-25T11:43:55.648 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_memory_target_autotune true 2026-02-25T11:43:55.648 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_objectstore bluestore * 2026-02-25T11:43:55.648 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_op_complaint_time 180.000000 2026-02-25T11:43:55.648 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_op_queue debug_random * 2026-02-25T11:43:55.649 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_op_queue_cut_off debug_random * 2026-02-25T11:43:55.649 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_recover_clone_overlap true 2026-02-25T11:43:55.649 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_recovery_max_chunk 1048576 2026-02-25T11:43:55.649 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_scrub_load_threshold 5.000000 2026-02-25T11:43:55.649 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_scrub_max_interval 600.000000 2026-02-25T11:43:55.649 INFO:teuthology.orchestra.run.trial028.stdout:osd advanced osd_shutdown_pgref_assert true 2026-02-25T11:43:55.649 INFO:teuthology.orchestra.run.trial028.stdout:mds advanced debug_mds 20/20 2026-02-25T11:43:55.649 INFO:teuthology.orchestra.run.trial028.stdout:mds advanced debug_mds_balancer 20/20 2026-02-25T11:43:55.649 INFO:teuthology.orchestra.run.trial028.stdout:mds advanced debug_ms 2/2 2026-02-25T11:43:55.650 INFO:teuthology.orchestra.run.trial028.stdout:mds advanced mds_bal_fragment_size_max 10000 2026-02-25T11:43:55.650 INFO:teuthology.orchestra.run.trial028.stdout:mds advanced mds_bal_merge_size 5 2026-02-25T11:43:55.650 INFO:teuthology.orchestra.run.trial028.stdout:mds advanced mds_bal_replicate_threshold 1.000000 2026-02-25T11:43:55.650 INFO:teuthology.orchestra.run.trial028.stdout:mds advanced mds_bal_split_bits 3 2026-02-25T11:43:55.650 INFO:teuthology.orchestra.run.trial028.stdout:mds advanced mds_bal_split_size 100 2026-02-25T11:43:55.650 INFO:teuthology.orchestra.run.trial028.stdout:mds dev mds_debug_frag true 2026-02-25T11:43:55.650 INFO:teuthology.orchestra.run.trial028.stdout:mds dev mds_debug_scatterstat true 2026-02-25T11:43:55.650 INFO:teuthology.orchestra.run.trial028.stdout:mds advanced mds_op_complaint_time 180.000000 2026-02-25T11:43:55.650 INFO:teuthology.orchestra.run.trial028.stdout:mds dev mds_verify_scatter true 2026-02-25T11:43:55.650 INFO:teuthology.orchestra.run.trial028.stdout:mds advanced osd_op_complaint_time 180.000000 2026-02-25T11:43:55.650 INFO:teuthology.orchestra.run.trial028.stdout:mds advanced rados_mon_op_timeout 900 2026-02-25T11:43:55.650 INFO:teuthology.orchestra.run.trial028.stdout:mds advanced rados_osd_op_timeout 900 2026-02-25T11:43:55.651 INFO:teuthology.orchestra.run.trial028.stdout:client dev client_fscrypt_dummy_encryption true 2026-02-25T11:43:55.651 INFO:teuthology.orchestra.run.trial028.stdout:client advanced client_mount_timeout 600 2026-02-25T11:43:55.651 INFO:teuthology.orchestra.run.trial028.stdout:client advanced debug_client 20/20 2026-02-25T11:43:55.651 INFO:teuthology.orchestra.run.trial028.stdout:client advanced debug_ms 1/1 2026-02-25T11:43:55.651 INFO:teuthology.orchestra.run.trial028.stdout:client advanced rados_mon_op_timeout 900 2026-02-25T11:43:55.651 INFO:teuthology.orchestra.run.trial028.stdout:client advanced rados_osd_op_timeout 900 2026-02-25T11:43:55.651 INFO:teuthology.orchestra.run.trial028.stdout:client.rgw advanced rgw_cache_enabled true 2026-02-25T11:43:55.651 INFO:teuthology.orchestra.run.trial028.stdout:client.rgw advanced rgw_enable_ops_log true 2026-02-25T11:43:55.651 INFO:teuthology.orchestra.run.trial028.stdout:client.rgw advanced rgw_enable_usage_log true 2026-02-25T11:43:55.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.639+0000 7f5adb44a640 1 -- 10.20.193.28:0/2514031362 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ab003f9d0 msgr2=0x7f5ab005fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:55.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.639+0000 7f5adb44a640 1 --2- 10.20.193.28:0/2514031362 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ab003f9d0 0x7f5ab005fe80 secure :-1 s=READY pgs=74 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac400b2c0 tx=0x7f5ac400b4a0 comp rx=0 tx=0).stop 2026-02-25T11:43:55.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.639+0000 7f5adb44a640 1 -- 10.20.193.28:0/2514031362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad410d510 msgr2=0x7f5ad40f4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:43:55.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.639+0000 7f5adb44a640 1 --2- 10.20.193.28:0/2514031362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad410d510 0x7f5ad40f4e20 secure :-1 s=READY pgs=139 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac8004740 tx=0x7f5ac800de60 comp rx=0 tx=0).stop 2026-02-25T11:43:55.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.640+0000 7f5adb44a640 1 -- 10.20.193.28:0/2514031362 shutdown_connections 2026-02-25T11:43:55.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.640+0000 7f5adb44a640 1 --2- 10.20.193.28:0/2514031362 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ab003f9d0 0x7f5ab005fe80 unknown :-1 s=CLOSED pgs=74 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:55.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.640+0000 7f5adb44a640 1 --2- 10.20.193.28:0/2514031362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ad410de40 0x7f5ad40f5a20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:55.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.640+0000 7f5adb44a640 1 --2- 10.20.193.28:0/2514031362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad410d510 0x7f5ad40f4e20 unknown :-1 s=CLOSED pgs=139 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:55.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.640+0000 7f5adb44a640 1 --2- 10.20.193.28:0/2514031362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad410b620 0x7f5ad40f47d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:43:55.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.640+0000 7f5adb44a640 1 -- 10.20.193.28:0/2514031362 >> 10.20.193.28:0/2514031362 conn(0x7f5ad40e8cb0 msgr2=0x7f5ad40ea960 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:43:55.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.640+0000 7f5adb44a640 1 -- 10.20.193.28:0/2514031362 shutdown_connections 2026-02-25T11:43:55.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:43:55.640+0000 7f5adb44a640 1 -- 10.20.193.28:0/2514031362 wait complete. 2026-02-25T11:43:55.744 INFO:tasks.cephadm:Deploying OSDs... 2026-02-25T11:43:55.744 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:43:55.744 DEBUG:teuthology.orchestra.run.trial028:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T11:43:55.782 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T11:43:55.782 DEBUG:teuthology.orchestra.run.trial028:> stat /dev/vg_nvme/lv_1 2026-02-25T11:43:55.862 INFO:teuthology.orchestra.run.trial028.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T11:43:55.862 INFO:teuthology.orchestra.run.trial028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T11:43:55.862 INFO:teuthology.orchestra.run.trial028.stdout:Device: 6h/6d Inode: 732 Links: 1 2026-02-25T11:43:55.862 INFO:teuthology.orchestra.run.trial028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T11:43:55.862 INFO:teuthology.orchestra.run.trial028.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T11:43:55.862 INFO:teuthology.orchestra.run.trial028.stdout:Access: 2026-02-25 11:43:23.801561592 +0000 2026-02-25T11:43:55.862 INFO:teuthology.orchestra.run.trial028.stdout:Modify: 2026-02-25 11:40:47.282968342 +0000 2026-02-25T11:43:55.862 INFO:teuthology.orchestra.run.trial028.stdout:Change: 2026-02-25 11:40:47.282968342 +0000 2026-02-25T11:43:55.862 INFO:teuthology.orchestra.run.trial028.stdout: Birth: 2026-02-25 11:40:47.282968342 +0000 2026-02-25T11:43:55.863 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T11:43:55.957 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records in 2026-02-25T11:43:55.957 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records out 2026-02-25T11:43:55.957 INFO:teuthology.orchestra.run.trial028.stderr:512 bytes copied, 0.000259632 s, 2.0 MB/s 2026-02-25T11:43:55.960 DEBUG:teuthology.orchestra.run.trial028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T11:43:56.001 DEBUG:teuthology.orchestra.run.trial028:> stat /dev/vg_nvme/lv_2 2026-02-25T11:43:56.080 INFO:teuthology.orchestra.run.trial028.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T11:43:56.080 INFO:teuthology.orchestra.run.trial028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T11:43:56.080 INFO:teuthology.orchestra.run.trial028.stdout:Device: 6h/6d Inode: 769 Links: 1 2026-02-25T11:43:56.080 INFO:teuthology.orchestra.run.trial028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T11:43:56.081 INFO:teuthology.orchestra.run.trial028.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T11:43:56.081 INFO:teuthology.orchestra.run.trial028.stdout:Access: 2026-02-25 11:43:23.838560199 +0000 2026-02-25T11:43:56.081 INFO:teuthology.orchestra.run.trial028.stdout:Modify: 2026-02-25 11:40:47.646956993 +0000 2026-02-25T11:43:56.081 INFO:teuthology.orchestra.run.trial028.stdout:Change: 2026-02-25 11:40:47.646956993 +0000 2026-02-25T11:43:56.081 INFO:teuthology.orchestra.run.trial028.stdout: Birth: 2026-02-25 11:40:47.646956993 +0000 2026-02-25T11:43:56.081 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T11:43:56.173 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records in 2026-02-25T11:43:56.173 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records out 2026-02-25T11:43:56.173 INFO:teuthology.orchestra.run.trial028.stderr:512 bytes copied, 0.00022116 s, 2.3 MB/s 2026-02-25T11:43:56.176 DEBUG:teuthology.orchestra.run.trial028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T11:43:56.216 DEBUG:teuthology.orchestra.run.trial028:> stat /dev/vg_nvme/lv_3 2026-02-25T11:43:56.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:56.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:56.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:56.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-25T11:43:56.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-25T11:43:56.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:56.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2514031362' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2026-02-25T11:43:56.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:56.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:56.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:56.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.262 INFO:journalctl@ceph.mgr.y.trial119.stdout:Feb 25 11:43:55 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-y[44049]: /lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-25T11:43:56.263 INFO:journalctl@ceph.mgr.y.trial119.stdout:Feb 25 11:43:55 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-y[44049]: Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-25T11:43:56.263 INFO:journalctl@ceph.mgr.y.trial119.stdout:Feb 25 11:43:55 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-y[44049]: from numpy import show_config as show_numpy_config 2026-02-25T11:43:56.292 INFO:teuthology.orchestra.run.trial028.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T11:43:56.292 INFO:teuthology.orchestra.run.trial028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T11:43:56.292 INFO:teuthology.orchestra.run.trial028.stdout:Device: 6h/6d Inode: 798 Links: 1 2026-02-25T11:43:56.292 INFO:teuthology.orchestra.run.trial028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T11:43:56.292 INFO:teuthology.orchestra.run.trial028.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T11:43:56.292 INFO:teuthology.orchestra.run.trial028.stdout:Access: 2026-02-25 11:43:23.876558767 +0000 2026-02-25T11:43:56.292 INFO:teuthology.orchestra.run.trial028.stdout:Modify: 2026-02-25 11:40:49.010914467 +0000 2026-02-25T11:43:56.292 INFO:teuthology.orchestra.run.trial028.stdout:Change: 2026-02-25 11:40:49.010914467 +0000 2026-02-25T11:43:56.292 INFO:teuthology.orchestra.run.trial028.stdout: Birth: 2026-02-25 11:40:49.010914467 +0000 2026-02-25T11:43:56.292 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T11:43:56.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:56.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-25T11:43:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-25T11:43:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2514031362' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2026-02-25T11:43:56.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:56.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:56.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:56.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.364 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records in 2026-02-25T11:43:56.365 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records out 2026-02-25T11:43:56.365 INFO:teuthology.orchestra.run.trial028.stderr:512 bytes copied, 0.000293704 s, 1.7 MB/s 2026-02-25T11:43:56.367 DEBUG:teuthology.orchestra.run.trial028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T11:43:56.406 DEBUG:teuthology.orchestra.run.trial028:> stat /dev/vg_nvme/lv_4 2026-02-25T11:43:56.485 INFO:teuthology.orchestra.run.trial028.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T11:43:56.486 INFO:teuthology.orchestra.run.trial028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T11:43:56.486 INFO:teuthology.orchestra.run.trial028.stdout:Device: 6h/6d Inode: 825 Links: 1 2026-02-25T11:43:56.486 INFO:teuthology.orchestra.run.trial028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T11:43:56.486 INFO:teuthology.orchestra.run.trial028.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T11:43:56.486 INFO:teuthology.orchestra.run.trial028.stdout:Access: 2026-02-25 11:43:23.912557411 +0000 2026-02-25T11:43:56.486 INFO:teuthology.orchestra.run.trial028.stdout:Modify: 2026-02-25 11:40:50.410870820 +0000 2026-02-25T11:43:56.486 INFO:teuthology.orchestra.run.trial028.stdout:Change: 2026-02-25 11:40:50.410870820 +0000 2026-02-25T11:43:56.487 INFO:teuthology.orchestra.run.trial028.stdout: Birth: 2026-02-25 11:40:50.410870820 +0000 2026-02-25T11:43:56.487 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T11:43:56.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:56.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:56.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:56.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2026-02-25T11:43:56.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2026-02-25T11:43:56.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:56.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2514031362' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2026-02-25T11:43:56.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:43:56.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:43:56.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:43:56.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:56.580 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records in 2026-02-25T11:43:56.580 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records out 2026-02-25T11:43:56.580 INFO:teuthology.orchestra.run.trial028.stderr:512 bytes copied, 0.000254573 s, 2.0 MB/s 2026-02-25T11:43:56.582 DEBUG:teuthology.orchestra.run.trial028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T11:43:56.622 DEBUG:teuthology.orchestra.run.trial119:> set -ex 2026-02-25T11:43:56.623 DEBUG:teuthology.orchestra.run.trial119:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T11:43:56.664 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T11:43:56.664 DEBUG:teuthology.orchestra.run.trial119:> stat /dev/vg_nvme/lv_1 2026-02-25T11:43:56.745 INFO:teuthology.orchestra.run.trial119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T11:43:56.745 INFO:teuthology.orchestra.run.trial119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T11:43:56.745 INFO:teuthology.orchestra.run.trial119.stdout:Device: 6h/6d Inode: 746 Links: 1 2026-02-25T11:43:56.745 INFO:teuthology.orchestra.run.trial119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T11:43:56.745 INFO:teuthology.orchestra.run.trial119.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T11:43:56.745 INFO:teuthology.orchestra.run.trial119.stdout:Access: 2026-02-25 11:43:54.158805849 +0000 2026-02-25T11:43:56.745 INFO:teuthology.orchestra.run.trial119.stdout:Modify: 2026-02-25 11:40:46.550928749 +0000 2026-02-25T11:43:56.745 INFO:teuthology.orchestra.run.trial119.stdout:Change: 2026-02-25 11:40:46.550928749 +0000 2026-02-25T11:43:56.745 INFO:teuthology.orchestra.run.trial119.stdout: Birth: 2026-02-25 11:40:46.550928749 +0000 2026-02-25T11:43:56.746 DEBUG:teuthology.orchestra.run.trial119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T11:43:56.845 INFO:teuthology.orchestra.run.trial119.stderr:1+0 records in 2026-02-25T11:43:56.845 INFO:teuthology.orchestra.run.trial119.stderr:1+0 records out 2026-02-25T11:43:56.845 INFO:teuthology.orchestra.run.trial119.stderr:512 bytes copied, 0.000259272 s, 2.0 MB/s 2026-02-25T11:43:56.847 DEBUG:teuthology.orchestra.run.trial119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T11:43:56.889 DEBUG:teuthology.orchestra.run.trial119:> stat /dev/vg_nvme/lv_2 2026-02-25T11:43:56.968 INFO:teuthology.orchestra.run.trial119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T11:43:56.968 INFO:teuthology.orchestra.run.trial119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T11:43:56.968 INFO:teuthology.orchestra.run.trial119.stdout:Device: 6h/6d Inode: 777 Links: 1 2026-02-25T11:43:56.968 INFO:teuthology.orchestra.run.trial119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T11:43:56.968 INFO:teuthology.orchestra.run.trial119.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T11:43:56.968 INFO:teuthology.orchestra.run.trial119.stdout:Access: 2026-02-25 11:43:54.159805831 +0000 2026-02-25T11:43:56.968 INFO:teuthology.orchestra.run.trial119.stdout:Modify: 2026-02-25 11:40:47.874971581 +0000 2026-02-25T11:43:56.968 INFO:teuthology.orchestra.run.trial119.stdout:Change: 2026-02-25 11:40:47.874971581 +0000 2026-02-25T11:43:56.968 INFO:teuthology.orchestra.run.trial119.stdout: Birth: 2026-02-25 11:40:47.874971581 +0000 2026-02-25T11:43:56.969 DEBUG:teuthology.orchestra.run.trial119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T11:43:57.065 INFO:teuthology.orchestra.run.trial119.stderr:1+0 records in 2026-02-25T11:43:57.065 INFO:teuthology.orchestra.run.trial119.stderr:1+0 records out 2026-02-25T11:43:57.065 INFO:teuthology.orchestra.run.trial119.stderr:512 bytes copied, 0.000221843 s, 2.3 MB/s 2026-02-25T11:43:57.067 DEBUG:teuthology.orchestra.run.trial119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T11:43:57.110 DEBUG:teuthology.orchestra.run.trial119:> stat /dev/vg_nvme/lv_3 2026-02-25T11:43:57.192 INFO:teuthology.orchestra.run.trial119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T11:43:57.192 INFO:teuthology.orchestra.run.trial119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T11:43:57.192 INFO:teuthology.orchestra.run.trial119.stdout:Device: 6h/6d Inode: 804 Links: 1 2026-02-25T11:43:57.192 INFO:teuthology.orchestra.run.trial119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T11:43:57.192 INFO:teuthology.orchestra.run.trial119.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T11:43:57.193 INFO:teuthology.orchestra.run.trial119.stdout:Access: 2026-02-25 11:43:54.160805812 +0000 2026-02-25T11:43:57.193 INFO:teuthology.orchestra.run.trial119.stdout:Modify: 2026-02-25 11:40:48.237967290 +0000 2026-02-25T11:43:57.193 INFO:teuthology.orchestra.run.trial119.stdout:Change: 2026-02-25 11:40:48.237967290 +0000 2026-02-25T11:43:57.193 INFO:teuthology.orchestra.run.trial119.stdout: Birth: 2026-02-25 11:40:48.237967290 +0000 2026-02-25T11:43:57.193 DEBUG:teuthology.orchestra.run.trial119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T11:43:57.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:57 trial119 ceph-mon[42710]: Reconfiguring mgr.x (unknown last config time)... 2026-02-25T11:43:57.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:57 trial119 ceph-mon[42710]: Reconfiguring daemon mgr.x on trial028 2026-02-25T11:43:57.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:57 trial119 ceph-mon[42710]: Standby manager daemon z started 2026-02-25T11:43:57.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:57 trial119 ceph-mon[42710]: from='mgr.? 10.20.193.167:0/3769742069' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2026-02-25T11:43:57.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:57 trial119 ceph-mon[42710]: from='mgr.? 10.20.193.167:0/3769742069' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2026-02-25T11:43:57.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:57 trial119 ceph-mon[42710]: from='mgr.? 10.20.193.167:0/3769742069' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2026-02-25T11:43:57.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:57 trial119 ceph-mon[42710]: from='mgr.? 10.20.193.167:0/3769742069' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2026-02-25T11:43:57.290 INFO:teuthology.orchestra.run.trial119.stderr:1+0 records in 2026-02-25T11:43:57.290 INFO:teuthology.orchestra.run.trial119.stderr:1+0 records out 2026-02-25T11:43:57.290 INFO:teuthology.orchestra.run.trial119.stderr:512 bytes copied, 0.000290681 s, 1.8 MB/s 2026-02-25T11:43:57.293 DEBUG:teuthology.orchestra.run.trial119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T11:43:57.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:57 trial028 ceph-mon[39224]: Reconfiguring mgr.x (unknown last config time)... 2026-02-25T11:43:57.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:57 trial028 ceph-mon[39224]: Reconfiguring daemon mgr.x on trial028 2026-02-25T11:43:57.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:57 trial028 ceph-mon[39224]: Standby manager daemon z started 2026-02-25T11:43:57.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:57 trial028 ceph-mon[39224]: from='mgr.? 10.20.193.167:0/3769742069' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2026-02-25T11:43:57.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:57 trial028 ceph-mon[39224]: from='mgr.? 10.20.193.167:0/3769742069' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2026-02-25T11:43:57.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:57 trial028 ceph-mon[39224]: from='mgr.? 10.20.193.167:0/3769742069' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2026-02-25T11:43:57.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:57 trial028 ceph-mon[39224]: from='mgr.? 10.20.193.167:0/3769742069' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2026-02-25T11:43:57.379 DEBUG:teuthology.orchestra.run.trial119:> stat /dev/vg_nvme/lv_4 2026-02-25T11:43:57.463 INFO:teuthology.orchestra.run.trial119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T11:43:57.464 INFO:teuthology.orchestra.run.trial119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T11:43:57.464 INFO:teuthology.orchestra.run.trial119.stdout:Device: 6h/6d Inode: 829 Links: 1 2026-02-25T11:43:57.464 INFO:teuthology.orchestra.run.trial119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T11:43:57.464 INFO:teuthology.orchestra.run.trial119.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T11:43:57.464 INFO:teuthology.orchestra.run.trial119.stdout:Access: 2026-02-25 11:43:54.162805775 +0000 2026-02-25T11:43:57.464 INFO:teuthology.orchestra.run.trial119.stdout:Modify: 2026-02-25 11:40:48.614962833 +0000 2026-02-25T11:43:57.464 INFO:teuthology.orchestra.run.trial119.stdout:Change: 2026-02-25 11:40:48.614962833 +0000 2026-02-25T11:43:57.464 INFO:teuthology.orchestra.run.trial119.stdout: Birth: 2026-02-25 11:40:48.614962833 +0000 2026-02-25T11:43:57.465 DEBUG:teuthology.orchestra.run.trial119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T11:43:57.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:57 trial167 ceph-mon[43392]: Reconfiguring mgr.x (unknown last config time)... 2026-02-25T11:43:57.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:57 trial167 ceph-mon[43392]: Reconfiguring daemon mgr.x on trial028 2026-02-25T11:43:57.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:57 trial167 ceph-mon[43392]: Standby manager daemon z started 2026-02-25T11:43:57.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:57 trial167 ceph-mon[43392]: from='mgr.? 10.20.193.167:0/3769742069' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2026-02-25T11:43:57.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:57 trial167 ceph-mon[43392]: from='mgr.? 10.20.193.167:0/3769742069' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2026-02-25T11:43:57.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:57 trial167 ceph-mon[43392]: from='mgr.? 10.20.193.167:0/3769742069' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2026-02-25T11:43:57.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:57 trial167 ceph-mon[43392]: from='mgr.? 10.20.193.167:0/3769742069' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2026-02-25T11:43:57.560 INFO:teuthology.orchestra.run.trial119.stderr:1+0 records in 2026-02-25T11:43:57.560 INFO:teuthology.orchestra.run.trial119.stderr:1+0 records out 2026-02-25T11:43:57.560 INFO:teuthology.orchestra.run.trial119.stderr:512 bytes copied, 0.000228064 s, 2.2 MB/s 2026-02-25T11:43:57.562 DEBUG:teuthology.orchestra.run.trial119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T11:43:57.604 DEBUG:teuthology.orchestra.run.trial167:> set -ex 2026-02-25T11:43:57.604 DEBUG:teuthology.orchestra.run.trial167:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T11:43:57.644 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T11:43:57.644 DEBUG:teuthology.orchestra.run.trial167:> stat /dev/vg_nvme/lv_1 2026-02-25T11:43:57.722 INFO:teuthology.orchestra.run.trial167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T11:43:57.722 INFO:teuthology.orchestra.run.trial167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T11:43:57.723 INFO:teuthology.orchestra.run.trial167.stdout:Device: 6h/6d Inode: 736 Links: 1 2026-02-25T11:43:57.723 INFO:teuthology.orchestra.run.trial167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T11:43:57.723 INFO:teuthology.orchestra.run.trial167.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T11:43:57.723 INFO:teuthology.orchestra.run.trial167.stdout:Access: 2026-02-25 11:43:52.856130037 +0000 2026-02-25T11:43:57.723 INFO:teuthology.orchestra.run.trial167.stdout:Modify: 2026-02-25 11:40:47.129362892 +0000 2026-02-25T11:43:57.723 INFO:teuthology.orchestra.run.trial167.stdout:Change: 2026-02-25 11:40:47.129362892 +0000 2026-02-25T11:43:57.723 INFO:teuthology.orchestra.run.trial167.stdout: Birth: 2026-02-25 11:40:47.129362892 +0000 2026-02-25T11:43:57.723 DEBUG:teuthology.orchestra.run.trial167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T11:43:57.818 INFO:teuthology.orchestra.run.trial167.stderr:1+0 records in 2026-02-25T11:43:57.818 INFO:teuthology.orchestra.run.trial167.stderr:1+0 records out 2026-02-25T11:43:57.818 INFO:teuthology.orchestra.run.trial167.stderr:512 bytes copied, 0.000227923 s, 2.2 MB/s 2026-02-25T11:43:57.821 DEBUG:teuthology.orchestra.run.trial167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T11:43:57.861 DEBUG:teuthology.orchestra.run.trial167:> stat /dev/vg_nvme/lv_2 2026-02-25T11:43:57.939 INFO:teuthology.orchestra.run.trial167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T11:43:57.939 INFO:teuthology.orchestra.run.trial167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T11:43:57.940 INFO:teuthology.orchestra.run.trial167.stdout:Device: 6h/6d Inode: 762 Links: 1 2026-02-25T11:43:57.940 INFO:teuthology.orchestra.run.trial167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T11:43:57.940 INFO:teuthology.orchestra.run.trial167.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T11:43:57.940 INFO:teuthology.orchestra.run.trial167.stdout:Access: 2026-02-25 11:43:52.856130037 +0000 2026-02-25T11:43:57.940 INFO:teuthology.orchestra.run.trial167.stdout:Modify: 2026-02-25 11:40:47.481252219 +0000 2026-02-25T11:43:57.940 INFO:teuthology.orchestra.run.trial167.stdout:Change: 2026-02-25 11:40:47.481252219 +0000 2026-02-25T11:43:57.940 INFO:teuthology.orchestra.run.trial167.stdout: Birth: 2026-02-25 11:40:47.481252219 +0000 2026-02-25T11:43:57.940 DEBUG:teuthology.orchestra.run.trial167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T11:43:58.032 INFO:teuthology.orchestra.run.trial167.stderr:1+0 records in 2026-02-25T11:43:58.032 INFO:teuthology.orchestra.run.trial167.stderr:1+0 records out 2026-02-25T11:43:58.032 INFO:teuthology.orchestra.run.trial167.stderr:512 bytes copied, 0.000208788 s, 2.5 MB/s 2026-02-25T11:43:58.035 DEBUG:teuthology.orchestra.run.trial167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T11:43:58.075 DEBUG:teuthology.orchestra.run.trial167:> stat /dev/vg_nvme/lv_3 2026-02-25T11:43:58.154 INFO:teuthology.orchestra.run.trial167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T11:43:58.154 INFO:teuthology.orchestra.run.trial167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T11:43:58.154 INFO:teuthology.orchestra.run.trial167.stdout:Device: 6h/6d Inode: 791 Links: 1 2026-02-25T11:43:58.154 INFO:teuthology.orchestra.run.trial167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T11:43:58.154 INFO:teuthology.orchestra.run.trial167.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T11:43:58.154 INFO:teuthology.orchestra.run.trial167.stdout:Access: 2026-02-25 11:43:52.857130024 +0000 2026-02-25T11:43:58.154 INFO:teuthology.orchestra.run.trial167.stdout:Modify: 2026-02-25 11:40:47.843245229 +0000 2026-02-25T11:43:58.154 INFO:teuthology.orchestra.run.trial167.stdout:Change: 2026-02-25 11:40:47.843245229 +0000 2026-02-25T11:43:58.154 INFO:teuthology.orchestra.run.trial167.stdout: Birth: 2026-02-25 11:40:47.843245229 +0000 2026-02-25T11:43:58.155 DEBUG:teuthology.orchestra.run.trial167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T11:43:58.242 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:58 trial167 ceph-mon[43392]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:58.243 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:58 trial167 ceph-mon[43392]: mgrmap e13: x(active, since 40s), standbys: z 2026-02-25T11:43:58.243 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:58 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2026-02-25T11:43:58.247 INFO:teuthology.orchestra.run.trial167.stderr:1+0 records in 2026-02-25T11:43:58.247 INFO:teuthology.orchestra.run.trial167.stderr:1+0 records out 2026-02-25T11:43:58.247 INFO:teuthology.orchestra.run.trial167.stderr:512 bytes copied, 0.000277125 s, 1.8 MB/s 2026-02-25T11:43:58.250 DEBUG:teuthology.orchestra.run.trial167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T11:43:58.291 DEBUG:teuthology.orchestra.run.trial167:> stat /dev/vg_nvme/lv_4 2026-02-25T11:43:58.368 INFO:teuthology.orchestra.run.trial167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T11:43:58.369 INFO:teuthology.orchestra.run.trial167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T11:43:58.369 INFO:teuthology.orchestra.run.trial167.stdout:Device: 6h/6d Inode: 818 Links: 1 2026-02-25T11:43:58.369 INFO:teuthology.orchestra.run.trial167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T11:43:58.369 INFO:teuthology.orchestra.run.trial167.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T11:43:58.369 INFO:teuthology.orchestra.run.trial167.stdout:Access: 2026-02-25 11:43:52.858130011 +0000 2026-02-25T11:43:58.369 INFO:teuthology.orchestra.run.trial167.stdout:Modify: 2026-02-25 11:40:48.201238316 +0000 2026-02-25T11:43:58.369 INFO:teuthology.orchestra.run.trial167.stdout:Change: 2026-02-25 11:40:48.201238316 +0000 2026-02-25T11:43:58.369 INFO:teuthology.orchestra.run.trial167.stdout: Birth: 2026-02-25 11:40:48.201238316 +0000 2026-02-25T11:43:58.369 DEBUG:teuthology.orchestra.run.trial167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T11:43:58.463 INFO:teuthology.orchestra.run.trial167.stderr:1+0 records in 2026-02-25T11:43:58.464 INFO:teuthology.orchestra.run.trial167.stderr:1+0 records out 2026-02-25T11:43:58.464 INFO:teuthology.orchestra.run.trial167.stderr:512 bytes copied, 0.000246929 s, 2.1 MB/s 2026-02-25T11:43:58.466 DEBUG:teuthology.orchestra.run.trial167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T11:43:58.504 INFO:tasks.cephadm:Deploying osd.0 on trial028 with /dev/vg_nvme/lv_4... 2026-02-25T11:43:58.505 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_4 2026-02-25T11:43:58.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:58 trial119 ceph-mon[42710]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:58.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:58 trial119 ceph-mon[42710]: mgrmap e13: x(active, since 40s), standbys: z 2026-02-25T11:43:58.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:58 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2026-02-25T11:43:58.553 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:58 trial028 ceph-mon[39224]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:43:58.553 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:58 trial028 ceph-mon[39224]: mgrmap e13: x(active, since 40s), standbys: z 2026-02-25T11:43:58.553 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:58 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2026-02-25T11:43:58.687 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:43:59.072 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:59 trial028 ceph-mon[39224]: Standby manager daemon y started 2026-02-25T11:43:59.072 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:59 trial028 ceph-mon[39224]: from='mgr.? 10.20.193.119:0/618345500' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2026-02-25T11:43:59.072 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:59 trial028 ceph-mon[39224]: from='mgr.? 10.20.193.119:0/618345500' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2026-02-25T11:43:59.072 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:59 trial028 ceph-mon[39224]: from='mgr.? 10.20.193.119:0/618345500' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2026-02-25T11:43:59.072 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:59 trial028 ceph-mon[39224]: from='mgr.? 10.20.193.119:0/618345500' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2026-02-25T11:43:59.072 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:43:59 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:59.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:59 trial167 ceph-mon[43392]: Standby manager daemon y started 2026-02-25T11:43:59.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:59 trial167 ceph-mon[43392]: from='mgr.? 10.20.193.119:0/618345500' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2026-02-25T11:43:59.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:59 trial167 ceph-mon[43392]: from='mgr.? 10.20.193.119:0/618345500' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2026-02-25T11:43:59.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:59 trial167 ceph-mon[43392]: from='mgr.? 10.20.193.119:0/618345500' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2026-02-25T11:43:59.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:59 trial167 ceph-mon[43392]: from='mgr.? 10.20.193.119:0/618345500' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2026-02-25T11:43:59.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:43:59 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:59.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:59 trial119 ceph-mon[42710]: Standby manager daemon y started 2026-02-25T11:43:59.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:59 trial119 ceph-mon[42710]: from='mgr.? 10.20.193.119:0/618345500' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2026-02-25T11:43:59.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:59 trial119 ceph-mon[42710]: from='mgr.? 10.20.193.119:0/618345500' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2026-02-25T11:43:59.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:59 trial119 ceph-mon[42710]: from='mgr.? 10.20.193.119:0/618345500' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2026-02-25T11:43:59.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:59 trial119 ceph-mon[42710]: from='mgr.? 10.20.193.119:0/618345500' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2026-02-25T11:43:59.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:43:59 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:43:59.625 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:43:59.648 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch daemon add osd trial028:vg_nvme/lv_4 --skip-validation 2026-02-25T11:43:59.833 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:44:00.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.014+0000 7fbad94f7640 1 -- 10.20.193.28:0/785429039 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbad410f1e0 msgr2=0x7fbad410f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:00.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.014+0000 7fbad94f7640 1 --2- 10.20.193.28:0/785429039 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbad410f1e0 0x7fbad410f5d0 secure :-1 s=READY pgs=140 gs=1 cs=0 l=1 c_cookie=15097508014932061060 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbac0009890 tx=0x7fbac002f0e0 comp rx=0 tx=0).stop 2026-02-25T11:44:00.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.014+0000 7fbad94f7640 1 -- 10.20.193.28:0/785429039 shutdown_connections 2026-02-25T11:44:00.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.014+0000 7fbad94f7640 1 --2- 10.20.193.28:0/785429039 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbad4110d20 0x7fbad41716b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:00.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.014+0000 7fbad94f7640 1 --2- 10.20.193.28:0/785429039 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbad4110370 0x7fbad41107e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:00.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.014+0000 7fbad94f7640 1 --2- 10.20.193.28:0/785429039 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbad410f1e0 0x7fbad410f5d0 unknown :-1 s=CLOSED pgs=140 gs=1 cs=0 l=1 c_cookie=15097508014932061060 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:00.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.014+0000 7fbad94f7640 1 -- 10.20.193.28:0/785429039 >> 10.20.193.28:0/785429039 conn(0x7fbad40ec950 msgr2=0x7fbad410cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:44:00.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.014+0000 7fbad94f7640 1 -- 10.20.193.28:0/785429039 shutdown_connections 2026-02-25T11:44:00.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.014+0000 7fbad94f7640 1 -- 10.20.193.28:0/785429039 wait complete. 2026-02-25T11:44:00.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad94f7640 1 Processor -- start 2026-02-25T11:44:00.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad94f7640 1 -- start start 2026-02-25T11:44:00.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad94f7640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbad410f1e0 0x7fbad40f8a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:00.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad94f7640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbad4110370 0x7fbad41716b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:00.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad94f7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbad4110d20 0x7fbad40f9270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:00.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad94f7640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbad40f9e60 con 0x7fbad4110d20 2026-02-25T11:44:00.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad94f7640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbad40f9fd0 con 0x7fbad410f1e0 2026-02-25T11:44:00.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad94f7640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbad40fa190 con 0x7fbad4110370 2026-02-25T11:44:00.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad37fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbad4110370 0x7fbad41716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:00.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad8cf6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbad4110d20 0x7fbad40f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:00.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad37fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbad4110370 0x7fbad41716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:55938/0 (socket says 10.20.193.28:55938) 2026-02-25T11:44:00.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad37fe640 1 -- 10.20.193.28:0/1066130734 learned_addr learned my addr 10.20.193.28:0/1066130734 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:44:00.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad3fff640 1 --2- 10.20.193.28:0/1066130734 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbad410f1e0 0x7fbad40f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:00.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad8cf6640 1 -- 10.20.193.28:0/1066130734 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbad4110370 msgr2=0x7fbad41716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:00.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad8cf6640 1 --2- 10.20.193.28:0/1066130734 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbad4110370 0x7fbad41716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:00.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad8cf6640 1 -- 10.20.193.28:0/1066130734 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbad410f1e0 msgr2=0x7fbad40f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:00.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad8cf6640 1 --2- 10.20.193.28:0/1066130734 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbad410f1e0 0x7fbad40f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:00.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.015+0000 7fbad8cf6640 1 -- 10.20.193.28:0/1066130734 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac40094b0 con 0x7fbad4110d20 2026-02-25T11:44:00.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.016+0000 7fbad8cf6640 1 --2- 10.20.193.28:0/1066130734 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbad4110d20 0x7fbad40f9270 secure :-1 s=READY pgs=141 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbac8006c10 tx=0x7fbac800fde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:44:00.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.016+0000 7fbad17fa640 1 -- 10.20.193.28:0/1066130734 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbac800dda0 con 0x7fbad4110d20 2026-02-25T11:44:00.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.016+0000 7fbad17fa640 1 -- 10.20.193.28:0/1066130734 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fbac8005df0 con 0x7fbad4110d20 2026-02-25T11:44:00.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.016+0000 7fbad17fa640 1 -- 10.20.193.28:0/1066130734 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbac8004a30 con 0x7fbad4110d20 2026-02-25T11:44:00.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.016+0000 7fbad94f7640 1 -- 10.20.193.28:0/1066130734 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac00094b0 con 0x7fbad4110d20 2026-02-25T11:44:00.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.016+0000 7fbad94f7640 1 -- 10.20.193.28:0/1066130734 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad40fa760 con 0x7fbad4110d20 2026-02-25T11:44:00.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.017+0000 7fbad94f7640 1 -- 10.20.193.28:0/1066130734 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbad41755b0 con 0x7fbad4110d20 2026-02-25T11:44:00.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.019+0000 7fbad17fa640 1 -- 10.20.193.28:0/1066130734 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 14) ==== 157066+0+0 (secure 0 0 0) 0x7fbac8005150 con 0x7fbad4110d20 2026-02-25T11:44:00.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.020+0000 7fbad17fa640 1 --2- 10.20.193.28:0/1066130734 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbaa80b56d0 0x7fbaa80d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:00.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.020+0000 7fbad17fa640 1 -- 10.20.193.28:0/1066130734 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1150+0+0 (secure 0 0 0) 0x7fbac8015070 con 0x7fbad4110d20 2026-02-25T11:44:00.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.020+0000 7fbad17fa640 1 -- 10.20.193.28:0/1066130734 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fbac80e6240 con 0x7fbad4110d20 2026-02-25T11:44:00.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.021+0000 7fbad3fff640 1 --2- 10.20.193.28:0/1066130734 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbaa80b56d0 0x7fbaa80d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:00.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.021+0000 7fbad3fff640 1 --2- 10.20.193.28:0/1066130734 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbaa80b56d0 0x7fbaa80d5b80 secure :-1 s=READY pgs=79 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbac0002bc0 tx=0x7fbac003a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:44:00.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:00.139+0000 7fbad94f7640 1 -- 10.20.193.28:0/1066130734 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7fbad4174c50 con 0x7fbaa80b56d0 2026-02-25T11:44:00.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:00 trial028 ceph-mon[39224]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:00.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:00 trial028 ceph-mon[39224]: mgrmap e14: x(active, since 42s), standbys: z, y 2026-02-25T11:44:00.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2026-02-25T11:44:00.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:00 trial167 ceph-mon[43392]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:00.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:00 trial167 ceph-mon[43392]: mgrmap e14: x(active, since 42s), standbys: z, y 2026-02-25T11:44:00.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2026-02-25T11:44:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:00 trial119 ceph-mon[42710]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:00 trial119 ceph-mon[42710]: mgrmap e14: x(active, since 42s), standbys: z, y 2026-02-25T11:44:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2026-02-25T11:44:01.476 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:01.476 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: osd.default does not exist. Creating it now. 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: Creating OSDs with service ID: default on trial028:['vg_nvme/lv_4'] 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: Marking host: trial028 for OSDSpec preview refresh. 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: Saving service osd.default spec with placement trial028 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:01.477 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:01.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: osd.default does not exist. Creating it now. 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: Creating OSDs with service ID: default on trial028:['vg_nvme/lv_4'] 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: Marking host: trial028 for OSDSpec preview refresh. 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: Saving service osd.default spec with placement trial028 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:01.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:01.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:01.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: osd.default does not exist. Creating it now. 2026-02-25T11:44:01.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: Creating OSDs with service ID: default on trial028:['vg_nvme/lv_4'] 2026-02-25T11:44:01.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: Marking host: trial028 for OSDSpec preview refresh. 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: Saving service osd.default spec with placement trial028 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:01.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:02.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:02 trial167 ceph-mon[43392]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:02.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:02 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1165040083' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bf596df7-d1fb-4d10-8d4c-14a35210bab2"} : dispatch 2026-02-25T11:44:02.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:02 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1165040083' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf596df7-d1fb-4d10-8d4c-14a35210bab2"}]': finished 2026-02-25T11:44:02.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:02 trial167 ceph-mon[43392]: osdmap e5: 1 total, 0 up, 1 in 2026-02-25T11:44:02.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:02 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:02.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:02 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/49041382' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:02.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:02 trial119 ceph-mon[42710]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:02.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:02 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1165040083' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bf596df7-d1fb-4d10-8d4c-14a35210bab2"} : dispatch 2026-02-25T11:44:02.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:02 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1165040083' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf596df7-d1fb-4d10-8d4c-14a35210bab2"}]': finished 2026-02-25T11:44:02.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:02 trial119 ceph-mon[42710]: osdmap e5: 1 total, 0 up, 1 in 2026-02-25T11:44:02.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:02 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:02.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:02 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/49041382' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:02.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:02 trial028 ceph-mon[39224]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:02.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:02 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1165040083' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bf596df7-d1fb-4d10-8d4c-14a35210bab2"} : dispatch 2026-02-25T11:44:02.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:02 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1165040083' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf596df7-d1fb-4d10-8d4c-14a35210bab2"}]': finished 2026-02-25T11:44:02.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:02 trial028 ceph-mon[39224]: osdmap e5: 1 total, 0 up, 1 in 2026-02-25T11:44:02.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:02 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:02.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:02 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/49041382' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:04.394 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:04 trial028 ceph-mon[39224]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:04.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:04 trial167 ceph-mon[43392]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:04.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:04 trial119 ceph-mon[42710]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:05.187 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:05 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2026-02-25T11:44:05.187 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:05 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:05.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:05 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2026-02-25T11:44:05.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:05 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:05.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:05 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2026-02-25T11:44:05.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:05 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:06.400 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:06 trial028 ceph-mon[39224]: Deploying daemon osd.0 on trial028 2026-02-25T11:44:06.400 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:06 trial028 ceph-mon[39224]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:06.400 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:06 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2026-02-25T11:44:06.400 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:06 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:06.400 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:06 trial028 ceph-mon[39224]: Deploying daemon osd.0 on trial028 2026-02-25T11:44:06.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:06 trial167 ceph-mon[43392]: Deploying daemon osd.0 on trial028 2026-02-25T11:44:06.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:06 trial167 ceph-mon[43392]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:06.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:06 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2026-02-25T11:44:06.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:06 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:06.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:06 trial167 ceph-mon[43392]: Deploying daemon osd.0 on trial028 2026-02-25T11:44:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:06 trial119 ceph-mon[42710]: Deploying daemon osd.0 on trial028 2026-02-25T11:44:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:06 trial119 ceph-mon[42710]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:06 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2026-02-25T11:44:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:06 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:06 trial119 ceph-mon[42710]: Deploying daemon osd.0 on trial028 2026-02-25T11:44:08.183 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:08 trial028 ceph-mon[39224]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:08.183 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:08.183 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:08.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:08 trial167 ceph-mon[43392]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:08.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:08.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:08.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:08 trial119 ceph-mon[42710]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:08.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:08.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:09.405 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:09 trial028 ceph-mon[39224]: from='osd.0 [v2:10.20.193.28:6802/3259224338,v1:10.20.193.28:6803/3259224338]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2026-02-25T11:44:09.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:09 trial167 ceph-mon[43392]: from='osd.0 [v2:10.20.193.28:6802/3259224338,v1:10.20.193.28:6803/3259224338]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2026-02-25T11:44:09.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:09 trial119 ceph-mon[42710]: from='osd.0 [v2:10.20.193.28:6802/3259224338,v1:10.20.193.28:6803/3259224338]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2026-02-25T11:44:10.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:10.017+0000 7fbad9f7f640 1 -- 10.20.193.28:0/1066130734 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=5}) -- 0x7fbab4003560 con 0x7fbad4110d20 2026-02-25T11:44:10.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:10.017+0000 7fbad17fa640 1 -- 10.20.193.28:0/1066130734 <== mon.0 v2:10.20.193.28:3300/0 7 ==== osd_map(5..6 src has 1..6) ==== 1027+0+0 (secure 0 0 0) 0x7fbac800a480 con 0x7fbad4110d20 2026-02-25T11:44:10.281 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:10 trial028 ceph-mon[39224]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:10.281 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:10 trial028 ceph-mon[39224]: from='osd.0 [v2:10.20.193.28:6802/3259224338,v1:10.20.193.28:6803/3259224338]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2026-02-25T11:44:10.281 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:10 trial028 ceph-mon[39224]: osdmap e6: 1 total, 0 up, 1 in 2026-02-25T11:44:10.281 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:10.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:10 trial167 ceph-mon[43392]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:10.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:10 trial167 ceph-mon[43392]: from='osd.0 [v2:10.20.193.28:6802/3259224338,v1:10.20.193.28:6803/3259224338]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2026-02-25T11:44:10.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:10 trial167 ceph-mon[43392]: osdmap e6: 1 total, 0 up, 1 in 2026-02-25T11:44:10.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:10.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:10 trial119 ceph-mon[42710]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:10.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:10 trial119 ceph-mon[42710]: from='osd.0 [v2:10.20.193.28:6802/3259224338,v1:10.20.193.28:6803/3259224338]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2026-02-25T11:44:10.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:10 trial119 ceph-mon[42710]: osdmap e6: 1 total, 0 up, 1 in 2026-02-25T11:44:10.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:12.132 INFO:teuthology.orchestra.run.trial028.stdout:Created osd(s) 0 on host 'trial028' 2026-02-25T11:44:12.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:12.129+0000 7fbad17fa640 1 -- 10.20.193.28:0/1066130734 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7fbad4174c50 con 0x7fbaa80b56d0 2026-02-25T11:44:12.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:12.132+0000 7fbad94f7640 1 -- 10.20.193.28:0/1066130734 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbaa80b56d0 msgr2=0x7fbaa80d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:12.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:12.132+0000 7fbad94f7640 1 --2- 10.20.193.28:0/1066130734 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbaa80b56d0 0x7fbaa80d5b80 secure :-1 s=READY pgs=79 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbac0002bc0 tx=0x7fbac003a040 comp rx=0 tx=0).stop 2026-02-25T11:44:12.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:12.132+0000 7fbad94f7640 1 -- 10.20.193.28:0/1066130734 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbad4110d20 msgr2=0x7fbad40f9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:12.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:12.132+0000 7fbad94f7640 1 --2- 10.20.193.28:0/1066130734 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbad4110d20 0x7fbad40f9270 secure :-1 s=READY pgs=141 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbac8006c10 tx=0x7fbac800fde0 comp rx=0 tx=0).stop 2026-02-25T11:44:12.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:12.132+0000 7fbad94f7640 1 -- 10.20.193.28:0/1066130734 shutdown_connections 2026-02-25T11:44:12.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:12.132+0000 7fbad94f7640 1 --2- 10.20.193.28:0/1066130734 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbaa80b56d0 0x7fbaa80d5b80 unknown :-1 s=CLOSED pgs=79 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:12.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:12.132+0000 7fbad94f7640 1 --2- 10.20.193.28:0/1066130734 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbad4110d20 0x7fbad40f9270 unknown :-1 s=CLOSED pgs=141 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:12.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:12.132+0000 7fbad94f7640 1 --2- 10.20.193.28:0/1066130734 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbad4110370 0x7fbad41716b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:12.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:12.132+0000 7fbad94f7640 1 --2- 10.20.193.28:0/1066130734 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbad410f1e0 0x7fbad40f8a10 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:12.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:12.132+0000 7fbad94f7640 1 -- 10.20.193.28:0/1066130734 >> 10.20.193.28:0/1066130734 conn(0x7fbad40ec950 msgr2=0x7fbad40ef060 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:44:12.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:12.132+0000 7fbad94f7640 1 -- 10.20.193.28:0/1066130734 shutdown_connections 2026-02-25T11:44:12.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:12.132+0000 7fbad94f7640 1 -- 10.20.193.28:0/1066130734 wait complete. 2026-02-25T11:44:12.239 DEBUG:teuthology.orchestra.run.trial028:osd.0> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.0.service 2026-02-25T11:44:12.244 INFO:tasks.cephadm:Deploying osd.1 on trial119 with /dev/vg_nvme/lv_4... 2026-02-25T11:44:12.245 DEBUG:teuthology.orchestra.run.trial119:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_4 2026-02-25T11:44:12.431 INFO:teuthology.orchestra.run.trial119.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.b/config 2026-02-25T11:44:12.436 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:12 trial028 ceph-mon[39224]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:12.436 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:12 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:12.436 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:12 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:12.436 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:12 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:12.436 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:12 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:12.472 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:12 trial119 ceph-mon[42710]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:12.473 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:12 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:12.473 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:12 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:12.473 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:12 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:12.473 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:12 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:12.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:12 trial167 ceph-mon[43392]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:12.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:12 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:12.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:12 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:12.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:12 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:12.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:12 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:12.810 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:44:12 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-0[58013]: 2026-02-25T11:44:12.432+0000 7f98608fb780 -1 osd.0 0 log_to_monitors true 2026-02-25T11:44:13.299 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:44:13.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:13 trial028 ceph-mon[39224]: from='osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2026-02-25T11:44:13.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:13.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:13.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:13.323 DEBUG:teuthology.orchestra.run.trial119:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch daemon add osd trial119:vg_nvme/lv_4 --skip-validation 2026-02-25T11:44:13.373 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:13 trial119 ceph-mon[42710]: from='osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2026-02-25T11:44:13.374 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:13.374 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:13.374 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:13.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:13 trial167 ceph-mon[43392]: from='osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2026-02-25T11:44:13.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:13.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:13.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:13.508 INFO:teuthology.orchestra.run.trial119.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.b/config 2026-02-25T11:44:13.694 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.691+0000 7fa58ac9b640 1 -- 10.20.193.119:0/1363163047 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa584110fb0 msgr2=0x7fa584171940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:13.694 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.691+0000 7fa58ac9b640 1 --2- 10.20.193.119:0/1363163047 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa584110fb0 0x7fa584171940 secure :-1 s=READY pgs=23 gs=1 cs=0 l=1 c_cookie=17573702411351892901 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa58000aa30 tx=0x7fa580030270 comp rx=0 tx=0).stop 2026-02-25T11:44:13.694 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.692+0000 7fa58ac9b640 1 -- 10.20.193.119:0/1363163047 shutdown_connections 2026-02-25T11:44:13.694 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.692+0000 7fa58ac9b640 1 --2- 10.20.193.119:0/1363163047 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa584110fb0 0x7fa584171940 unknown :-1 s=CLOSED pgs=23 gs=1 cs=0 l=1 c_cookie=17573702411351892901 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:13.694 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.692+0000 7fa58ac9b640 1 --2- 10.20.193.119:0/1363163047 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa584110600 0x7fa584110a70 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:13.694 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.692+0000 7fa58ac9b640 1 --2- 10.20.193.119:0/1363163047 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa58410f470 0x7fa58410f860 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:13.694 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.692+0000 7fa58ac9b640 1 -- 10.20.193.119:0/1363163047 >> 10.20.193.119:0/1363163047 conn(0x7fa5840ecba0 msgr2=0x7fa58410cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:44:13.695 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.692+0000 7fa58ac9b640 1 -- 10.20.193.119:0/1363163047 shutdown_connections 2026-02-25T11:44:13.695 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.692+0000 7fa58ac9b640 1 -- 10.20.193.119:0/1363163047 wait complete. 2026-02-25T11:44:13.697 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.692+0000 7fa58ac9b640 1 Processor -- start 2026-02-25T11:44:13.697 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa58ac9b640 1 -- start start 2026-02-25T11:44:13.697 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa58ac9b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa58410f470 0x7fa5840f9550 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:13.697 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa58ac9b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa584110600 0x7fa584171940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:13.697 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa58ac9b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa584110fb0 0x7fa5840f9db0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:13.697 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa58ac9b640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa5840fa9a0 con 0x7fa584110600 2026-02-25T11:44:13.697 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa58ac9b640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa5840fab10 con 0x7fa584110fb0 2026-02-25T11:44:13.697 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa58ac9b640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa5840facd0 con 0x7fa58410f470 2026-02-25T11:44:13.697 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa57bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa584110600 0x7fa584171940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:13.697 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa57bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa584110600 0x7fa584171940 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.119:44610/0 (socket says 10.20.193.119:44610) 2026-02-25T11:44:13.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa57bfff640 1 -- 10.20.193.119:0/2192880774 learned_addr learned my addr 10.20.193.119:0/2192880774 (peer_addr_for_me v2:10.20.193.119:0/0) 2026-02-25T11:44:13.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa589211640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa584110fb0 0x7fa5840f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:13.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa588a10640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa58410f470 0x7fa5840f9550 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:13.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa589211640 1 -- 10.20.193.119:0/2192880774 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa58410f470 msgr2=0x7fa5840f9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:13.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa589211640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa58410f470 0x7fa5840f9550 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:13.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa589211640 1 -- 10.20.193.119:0/2192880774 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa584110600 msgr2=0x7fa584171940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:13.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa589211640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa584110600 0x7fa584171940 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:13.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.693+0000 7fa589211640 1 -- 10.20.193.119:0/2192880774 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58000a650 con 0x7fa584110fb0 2026-02-25T11:44:13.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.694+0000 7fa589211640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa584110fb0 0x7fa5840f9db0 secure :-1 s=READY pgs=24 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa58000aa30 tx=0x7fa5800039b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:44:13.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.694+0000 7fa579ffb640 1 -- 10.20.193.119:0/2192880774 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa58003e070 con 0x7fa584110fb0 2026-02-25T11:44:13.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.694+0000 7fa579ffb640 1 -- 10.20.193.119:0/2192880774 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa5800050c0 con 0x7fa584110fb0 2026-02-25T11:44:13.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.694+0000 7fa579ffb640 1 -- 10.20.193.119:0/2192880774 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa580042930 con 0x7fa584110fb0 2026-02-25T11:44:13.699 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.694+0000 7fa58ac9b640 1 -- 10.20.193.119:0/2192880774 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5840faf50 con 0x7fa584110fb0 2026-02-25T11:44:13.699 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.694+0000 7fa58ac9b640 1 -- 10.20.193.119:0/2192880774 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5840fb3c0 con 0x7fa584110fb0 2026-02-25T11:44:13.699 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.694+0000 7fa58ac9b640 1 -- 10.20.193.119:0/2192880774 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5840fb630 con 0x7fa584110fb0 2026-02-25T11:44:13.699 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.697+0000 7fa579ffb640 1 -- 10.20.193.119:0/2192880774 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 14) ==== 157066+0+0 (secure 0 0 0) 0x7fa580005230 con 0x7fa584110fb0 2026-02-25T11:44:13.701 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.699+0000 7fa579ffb640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5600b56d0 0x7fa5600d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:13.701 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.699+0000 7fa579ffb640 1 -- 10.20.193.119:0/2192880774 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1356+0+0 (secure 0 0 0) 0x7fa58010a890 con 0x7fa584110fb0 2026-02-25T11:44:13.701 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.699+0000 7fa579ffb640 1 -- 10.20.193.119:0/2192880774 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fa580143960 con 0x7fa584110fb0 2026-02-25T11:44:13.701 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.699+0000 7fa588a10640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5600b56d0 0x7fa5600d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:13.702 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.700+0000 7fa588a10640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5600b56d0 0x7fa5600d5b80 secure :-1 s=READY pgs=82 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa574004740 tx=0x7fa5740073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:44:13.820 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:13.819+0000 7fa58ac9b640 1 -- 10.20.193.119:0/2192880774 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7fa5840fb630 con 0x7fa5600b56d0 2026-02-25T11:44:14.340 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:44:14 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-0[58013]: 2026-02-25T11:44:14.194+0000 7f985c87d640 -1 osd.0 0 waiting for initial osdmap 2026-02-25T11:44:14.340 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:44:14 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-0[58013]: 2026-02-25T11:44:14.197+0000 7f9858744640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T11:44:14.341 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:14 trial028 ceph-mon[39224]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:14.341 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:14 trial028 ceph-mon[39224]: from='osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2026-02-25T11:44:14.341 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:14 trial028 ceph-mon[39224]: osdmap e7: 1 total, 0 up, 1 in 2026-02-25T11:44:14.341 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:14 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:14.341 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:14 trial028 ceph-mon[39224]: from='osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:44:14.341 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:14 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:14.341 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:14 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:14.341 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:14 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:14.341 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:14 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:14.341 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:14 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:14.341 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:14 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:14.341 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:14 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:14.359 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:14 trial119 ceph-mon[42710]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:14.359 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:14 trial119 ceph-mon[42710]: from='osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2026-02-25T11:44:14.359 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:14 trial119 ceph-mon[42710]: osdmap e7: 1 total, 0 up, 1 in 2026-02-25T11:44:14.359 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:14 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:14.359 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:14 trial119 ceph-mon[42710]: from='osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:44:14.359 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:14 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:14.359 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:14 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:14.359 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:14 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:14.359 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:14 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:14.359 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:14 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:14.359 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:14 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:14.359 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:14 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:14.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:14 trial167 ceph-mon[43392]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:14.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:14 trial167 ceph-mon[43392]: from='osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2026-02-25T11:44:14.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:14 trial167 ceph-mon[43392]: osdmap e7: 1 total, 0 up, 1 in 2026-02-25T11:44:14.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:14 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:14.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:14 trial167 ceph-mon[43392]: from='osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:44:14.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:14 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:14.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:14 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:14.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:14 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:14.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:14 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:14.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:14 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:14.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:14 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:14.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:14 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:15.402 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:15.403 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: osd.default does not exist. Creating it now. 2026-02-25T11:44:15.403 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: Creating OSDs with service ID: default on trial119:['vg_nvme/lv_4'] 2026-02-25T11:44:15.403 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: Marking host: trial119 for OSDSpec preview refresh. 2026-02-25T11:44:15.403 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: Saving service osd.default spec with placement trial119 2026-02-25T11:44:15.403 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: from='osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-25T11:44:15.403 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: osdmap e8: 1 total, 0 up, 1 in 2026-02-25T11:44:15.403 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:15.403 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:15.403 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "185719c0-7c2d-431f-9925-88bbee86a98d"} : dispatch 2026-02-25T11:44:15.404 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: from='client.? 10.20.193.119:0/3503168007' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "185719c0-7c2d-431f-9925-88bbee86a98d"} : dispatch 2026-02-25T11:44:15.404 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "185719c0-7c2d-431f-9925-88bbee86a98d"}]': finished 2026-02-25T11:44:15.404 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205] boot 2026-02-25T11:44:15.404 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: osdmap e9: 2 total, 1 up, 2 in 2026-02-25T11:44:15.404 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:15.404 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:15.455 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:15.455 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: osd.default does not exist. Creating it now. 2026-02-25T11:44:15.455 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: Creating OSDs with service ID: default on trial119:['vg_nvme/lv_4'] 2026-02-25T11:44:15.455 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: Marking host: trial119 for OSDSpec preview refresh. 2026-02-25T11:44:15.456 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: Saving service osd.default spec with placement trial119 2026-02-25T11:44:15.456 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: from='osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-25T11:44:15.456 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: osdmap e8: 1 total, 0 up, 1 in 2026-02-25T11:44:15.456 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:15.456 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:15.456 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "185719c0-7c2d-431f-9925-88bbee86a98d"} : dispatch 2026-02-25T11:44:15.456 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: from='client.? 10.20.193.119:0/3503168007' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "185719c0-7c2d-431f-9925-88bbee86a98d"} : dispatch 2026-02-25T11:44:15.456 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "185719c0-7c2d-431f-9925-88bbee86a98d"}]': finished 2026-02-25T11:44:15.456 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205] boot 2026-02-25T11:44:15.456 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: osdmap e9: 2 total, 1 up, 2 in 2026-02-25T11:44:15.457 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:15.457 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:15.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:15.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: osd.default does not exist. Creating it now. 2026-02-25T11:44:15.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: Creating OSDs with service ID: default on trial119:['vg_nvme/lv_4'] 2026-02-25T11:44:15.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: Marking host: trial119 for OSDSpec preview refresh. 2026-02-25T11:44:15.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: Saving service osd.default spec with placement trial119 2026-02-25T11:44:15.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: from='osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-25T11:44:15.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: osdmap e8: 1 total, 0 up, 1 in 2026-02-25T11:44:15.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:15.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:15.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "185719c0-7c2d-431f-9925-88bbee86a98d"} : dispatch 2026-02-25T11:44:15.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: from='client.? 10.20.193.119:0/3503168007' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "185719c0-7c2d-431f-9925-88bbee86a98d"} : dispatch 2026-02-25T11:44:15.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "185719c0-7c2d-431f-9925-88bbee86a98d"}]': finished 2026-02-25T11:44:15.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: osd.0 [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205] boot 2026-02-25T11:44:15.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: osdmap e9: 2 total, 1 up, 2 in 2026-02-25T11:44:15.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2026-02-25T11:44:15.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:16.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: purged_snaps scrub starts 2026-02-25T11:44:16.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: purged_snaps scrub ok 2026-02-25T11:44:16.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:16.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: from='client.? 10.20.193.119:0/907972196' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:16.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:16.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:16.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:16.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:16.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:16.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:16.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:16.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:16.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:16.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:16.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: purged_snaps scrub starts 2026-02-25T11:44:16.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: purged_snaps scrub ok 2026-02-25T11:44:16.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:16.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: from='client.? 10.20.193.119:0/907972196' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:16.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:16.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:16.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:16.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:16.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:16.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:16.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:16.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:16.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:16.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:16.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: purged_snaps scrub starts 2026-02-25T11:44:16.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: purged_snaps scrub ok 2026-02-25T11:44:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2026-02-25T11:44:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: from='client.? 10.20.193.119:0/907972196' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:16.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:17.332 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:17 trial119 ceph-mon[42710]: Detected new or changed devices on trial028 2026-02-25T11:44:17.333 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:17 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial028 to 84053M 2026-02-25T11:44:17.333 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:17 trial119 ceph-mon[42710]: osdmap e10: 2 total, 1 up, 2 in 2026-02-25T11:44:17.333 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:17.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:17 trial167 ceph-mon[43392]: Detected new or changed devices on trial028 2026-02-25T11:44:17.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:17 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial028 to 84053M 2026-02-25T11:44:17.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:17 trial167 ceph-mon[43392]: osdmap e10: 2 total, 1 up, 2 in 2026-02-25T11:44:17.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:17.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:17 trial028 ceph-mon[39224]: Detected new or changed devices on trial028 2026-02-25T11:44:17.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:17 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial028 to 84053M 2026-02-25T11:44:17.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:17 trial028 ceph-mon[39224]: osdmap e10: 2 total, 1 up, 2 in 2026-02-25T11:44:17.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:18.371 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:18 trial119 ceph-mon[42710]: pgmap v30: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:18.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:18 trial167 ceph-mon[43392]: pgmap v30: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:18.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:18 trial028 ceph-mon[39224]: pgmap v30: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:19.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:19 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2026-02-25T11:44:19.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:19 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:19.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:19 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2026-02-25T11:44:19.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:19 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:19.501 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:19 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2026-02-25T11:44:19.501 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:19 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:19.501 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:19 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2026-02-25T11:44:19.502 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:19 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:19.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:19 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2026-02-25T11:44:19.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:19 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:19.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:19 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2026-02-25T11:44:19.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:19 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:20.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:20 trial167 ceph-mon[43392]: Deploying daemon osd.1 on trial119 2026-02-25T11:44:20.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:20 trial167 ceph-mon[43392]: Deploying daemon osd.1 on trial119 2026-02-25T11:44:20.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:20 trial167 ceph-mon[43392]: pgmap v31: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:20.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:20 trial119 ceph-mon[42710]: Deploying daemon osd.1 on trial119 2026-02-25T11:44:20.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:20 trial119 ceph-mon[42710]: Deploying daemon osd.1 on trial119 2026-02-25T11:44:20.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:20 trial119 ceph-mon[42710]: pgmap v31: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:20 trial028 ceph-mon[39224]: Deploying daemon osd.1 on trial119 2026-02-25T11:44:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:20 trial028 ceph-mon[39224]: Deploying daemon osd.1 on trial119 2026-02-25T11:44:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:20 trial028 ceph-mon[39224]: pgmap v31: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:22.441 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:22 trial119 ceph-mon[42710]: pgmap v32: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:22.441 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:22.441 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:22.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:22 trial167 ceph-mon[43392]: pgmap v32: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:22.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:22.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:22.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:22 trial028 ceph-mon[39224]: pgmap v32: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:22.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:22.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:23.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:23 trial167 ceph-mon[43392]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-25T11:44:23.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:23 trial167 ceph-mon[43392]: from='osd.1 [v2:10.20.193.119:6800/2394072662,v1:10.20.193.119:6801/2394072662]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-25T11:44:23.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:23 trial119 ceph-mon[42710]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-25T11:44:23.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:23 trial119 ceph-mon[42710]: from='osd.1 [v2:10.20.193.119:6800/2394072662,v1:10.20.193.119:6801/2394072662]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-25T11:44:23.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:23 trial028 ceph-mon[39224]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-25T11:44:23.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:23 trial028 ceph-mon[39224]: from='osd.1 [v2:10.20.193.119:6800/2394072662,v1:10.20.193.119:6801/2394072662]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-25T11:44:23.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:23.696+0000 7fa58a49a640 1 -- 10.20.193.119:0/2192880774 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=8}) -- 0x7fa55c003560 con 0x7fa584110fb0 2026-02-25T11:44:23.698 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:23.696+0000 7fa579ffb640 1 -- 10.20.193.119:0/2192880774 <== mon.2 v2:10.20.193.119:3300/0 7 ==== osd_map(8..11 src has 1..11) ==== 2675+0+0 (secure 0 0 0) 0x7fa5800d8510 con 0x7fa584110fb0 2026-02-25T11:44:24.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:24 trial167 ceph-mon[43392]: pgmap v33: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:24.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:24 trial167 ceph-mon[43392]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2026-02-25T11:44:24.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:24 trial167 ceph-mon[43392]: osdmap e11: 2 total, 1 up, 2 in 2026-02-25T11:44:24.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:24 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:24.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:24 trial119 ceph-mon[42710]: pgmap v33: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:24.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:24 trial119 ceph-mon[42710]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2026-02-25T11:44:24.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:24 trial119 ceph-mon[42710]: osdmap e11: 2 total, 1 up, 2 in 2026-02-25T11:44:24.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:24 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:24.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:24 trial028 ceph-mon[39224]: pgmap v33: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:24.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:24 trial028 ceph-mon[39224]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2026-02-25T11:44:24.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:24 trial028 ceph-mon[39224]: osdmap e11: 2 total, 1 up, 2 in 2026-02-25T11:44:24.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:26.004 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:26.002+0000 7fa579ffb640 1 -- 10.20.193.119:0/2192880774 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7fa5840fb630 con 0x7fa5600b56d0 2026-02-25T11:44:26.004 INFO:teuthology.orchestra.run.trial119.stdout:Created osd(s) 1 on host 'trial119' 2026-02-25T11:44:26.007 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:26.004+0000 7fa58ac9b640 1 -- 10.20.193.119:0/2192880774 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5600b56d0 msgr2=0x7fa5600d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:26.007 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:26.004+0000 7fa58ac9b640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5600b56d0 0x7fa5600d5b80 secure :-1 s=READY pgs=82 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa574004740 tx=0x7fa5740073d0 comp rx=0 tx=0).stop 2026-02-25T11:44:26.007 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:26.004+0000 7fa58ac9b640 1 -- 10.20.193.119:0/2192880774 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa584110fb0 msgr2=0x7fa5840f9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:26.007 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:26.004+0000 7fa58ac9b640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa584110fb0 0x7fa5840f9db0 secure :-1 s=READY pgs=24 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa58000aa30 tx=0x7fa5800039b0 comp rx=0 tx=0).stop 2026-02-25T11:44:26.007 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:26.004+0000 7fa58ac9b640 1 -- 10.20.193.119:0/2192880774 shutdown_connections 2026-02-25T11:44:26.007 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:26.004+0000 7fa58ac9b640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5600b56d0 0x7fa5600d5b80 unknown :-1 s=CLOSED pgs=82 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:26.008 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:26.004+0000 7fa58ac9b640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa584110fb0 0x7fa5840f9db0 unknown :-1 s=CLOSED pgs=24 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:26.008 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:26.004+0000 7fa58ac9b640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa584110600 0x7fa584171940 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:26.008 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:26.004+0000 7fa58ac9b640 1 --2- 10.20.193.119:0/2192880774 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa58410f470 0x7fa5840f9550 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:26.008 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:26.004+0000 7fa58ac9b640 1 -- 10.20.193.119:0/2192880774 >> 10.20.193.119:0/2192880774 conn(0x7fa5840ecba0 msgr2=0x7fa5840efb70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:44:26.008 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:26.005+0000 7fa58ac9b640 1 -- 10.20.193.119:0/2192880774 shutdown_connections 2026-02-25T11:44:26.008 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:26.005+0000 7fa58ac9b640 1 -- 10.20.193.119:0/2192880774 wait complete. 2026-02-25T11:44:26.107 DEBUG:teuthology.orchestra.run.trial119:osd.1> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.1.service 2026-02-25T11:44:26.110 INFO:tasks.cephadm:Deploying osd.2 on trial167 with /dev/vg_nvme/lv_4... 2026-02-25T11:44:26.111 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_4 2026-02-25T11:44:26.299 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.c/config 2026-02-25T11:44:26.414 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:26 trial167 ceph-mon[43392]: pgmap v35: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:26.414 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:26 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:26.415 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:26 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:26.415 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:26 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:26.415 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:26 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:26.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:26 trial119 ceph-mon[42710]: pgmap v35: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:26.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:26 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:26.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:26 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:26.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:26 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:26.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:26 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:26.511 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:44:26 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-1[49059]: 2026-02-25T11:44:26.203+0000 7ff857c79780 -1 osd.1 0 log_to_monitors true 2026-02-25T11:44:26.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:26 trial028 ceph-mon[39224]: pgmap v35: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:26.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:26 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:26.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:26 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:26.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:26 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:26.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:26 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:27.392 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:44:27.417 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch daemon add osd trial167:vg_nvme/lv_4 --skip-validation 2026-02-25T11:44:27.469 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:27 trial167 ceph-mon[43392]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-25T11:44:27.469 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:27 trial167 ceph-mon[43392]: from='osd.1 [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-25T11:44:27.469 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:27 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:27.469 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:27 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:27.469 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:27 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:27.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:27 trial119 ceph-mon[42710]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-25T11:44:27.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:27 trial119 ceph-mon[42710]: from='osd.1 [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-25T11:44:27.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:27 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:27.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:27 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:27.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:27 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:27.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:27 trial028 ceph-mon[39224]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-25T11:44:27.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:27 trial028 ceph-mon[39224]: from='osd.1 [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2026-02-25T11:44:27.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:27 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:27.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:27 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:27.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:27 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:27.603 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.c/config 2026-02-25T11:44:27.772 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.769+0000 7fd7123fd640 1 -- 10.20.193.167:0/2127545651 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd70c10b2c0 msgr2=0x7fd70c10b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:27.772 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.769+0000 7fd7123fd640 1 --2- 10.20.193.167:0/2127545651 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd70c10b2c0 0x7fd70c10b6b0 secure :-1 s=READY pgs=23 gs=1 cs=0 l=1 c_cookie=9758571925836822070 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd6fc009950 tx=0x7fd6fc02f1d0 comp rx=0 tx=0).stop 2026-02-25T11:44:27.772 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.769+0000 7fd7123fd640 1 -- 10.20.193.167:0/2127545651 shutdown_connections 2026-02-25T11:44:27.772 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.769+0000 7fd7123fd640 1 --2- 10.20.193.167:0/2127545651 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd70c14ec50 0x7fd70c16f100 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:27.772 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.769+0000 7fd7123fd640 1 --2- 10.20.193.167:0/2127545651 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd70c10bcd0 0x7fd70c14e510 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:27.772 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.769+0000 7fd7123fd640 1 --2- 10.20.193.167:0/2127545651 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd70c10b2c0 0x7fd70c10b6b0 unknown :-1 s=CLOSED pgs=23 gs=1 cs=0 l=1 c_cookie=9758571925836822070 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:27.772 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.769+0000 7fd7123fd640 1 -- 10.20.193.167:0/2127545651 >> 10.20.193.167:0/2127545651 conn(0x7fd70c0e8cb0 msgr2=0x7fd70c1090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:44:27.772 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.769+0000 7fd7123fd640 1 -- 10.20.193.167:0/2127545651 shutdown_connections 2026-02-25T11:44:27.772 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.769+0000 7fd7123fd640 1 -- 10.20.193.167:0/2127545651 wait complete. 2026-02-25T11:44:27.772 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.770+0000 7fd7123fd640 1 Processor -- start 2026-02-25T11:44:27.772 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.770+0000 7fd7123fd640 1 -- start start 2026-02-25T11:44:27.772 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.770+0000 7fd7123fd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd70c10b2c0 0x7fd70c16f100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:27.772 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.770+0000 7fd7123fd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd70c10bcd0 0x7fd70c109090 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.770+0000 7fd7123fd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd70c14ec50 0x7fd70c11c4e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.770+0000 7fd7123fd640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c11cc70 con 0x7fd70c10bcd0 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.770+0000 7fd7123fd640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c11cde0 con 0x7fd70c14ec50 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.770+0000 7fd7123fd640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c11cfa0 con 0x7fd70c10b2c0 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.770+0000 7fd7113fb640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd70c10b2c0 0x7fd70c16f100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd7113fb640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd70c10b2c0 0x7fd70c16f100 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.167:37864/0 (socket says 10.20.193.167:37864) 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd7113fb640 1 -- 10.20.193.167:0/1663047604 learned_addr learned my addr 10.20.193.167:0/1663047604 (peer_addr_for_me v2:10.20.193.167:0/0) 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd711bfc640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd70c14ec50 0x7fd70c11c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd710bfa640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd70c10bcd0 0x7fd70c109090 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd7113fb640 1 -- 10.20.193.167:0/1663047604 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd70c14ec50 msgr2=0x7fd70c11c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd7113fb640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd70c14ec50 0x7fd70c11c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd7113fb640 1 -- 10.20.193.167:0/1663047604 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd70c10bcd0 msgr2=0x7fd70c109090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd7113fb640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd70c10bcd0 0x7fd70c109090 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd7113fb640 1 -- 10.20.193.167:0/1663047604 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6fc009570 con 0x7fd70c10b2c0 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd7113fb640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd70c10b2c0 0x7fd70c16f100 secure :-1 s=READY pgs=24 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd6fc0098a0 tx=0x7fd6fc0028d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd6fa7fc640 1 -- 10.20.193.167:0/1663047604 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd6fc03d070 con 0x7fd70c10b2c0 2026-02-25T11:44:27.773 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd7123fd640 1 -- 10.20.193.167:0/1663047604 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70c11d680 con 0x7fd70c10b2c0 2026-02-25T11:44:27.774 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd6fa7fc640 1 -- 10.20.193.167:0/1663047604 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd6fc005dd0 con 0x7fd70c10b2c0 2026-02-25T11:44:27.774 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd6fa7fc640 1 -- 10.20.193.167:0/1663047604 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd6fc0418c0 con 0x7fd70c10b2c0 2026-02-25T11:44:27.774 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.771+0000 7fd7123fd640 1 -- 10.20.193.167:0/1663047604 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70c11db40 con 0x7fd70c10b2c0 2026-02-25T11:44:27.774 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.772+0000 7fd7123fd640 1 -- 10.20.193.167:0/1663047604 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd70c061010 con 0x7fd70c10b2c0 2026-02-25T11:44:27.777 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.774+0000 7fd6fa7fc640 1 -- 10.20.193.167:0/1663047604 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 14) ==== 157066+0+0 (secure 0 0 0) 0x7fd6fc041a50 con 0x7fd70c10b2c0 2026-02-25T11:44:27.778 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.775+0000 7fd6fa7fc640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd6e80b56d0 0x7fd6e80d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:27.778 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.776+0000 7fd6fa7fc640 1 -- 10.20.193.167:0/1663047604 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 1896+0+0 (secure 0 0 0) 0x7fd6fc109740 con 0x7fd70c10b2c0 2026-02-25T11:44:27.778 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.776+0000 7fd710bfa640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd6e80b56d0 0x7fd6e80d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:27.778 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.776+0000 7fd710bfa640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd6e80b56d0 0x7fd6e80d5b80 secure :-1 s=READY pgs=87 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd700004740 tx=0x7fd700005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:44:27.780 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.778+0000 7fd6fa7fc640 1 -- 10.20.193.167:0/1663047604 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fd6fc0cdad0 con 0x7fd70c10b2c0 2026-02-25T11:44:27.898 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:27.896+0000 7fd7123fd640 1 -- 10.20.193.167:0/1663047604 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7fd70c172ae0 con 0x7fd6e80b56d0 2026-02-25T11:44:28.420 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: pgmap v36: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:28.420 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2026-02-25T11:44:28.420 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: osdmap e12: 2 total, 1 up, 2 in 2026-02-25T11:44:28.421 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:28.421 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:44:28.421 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: from='osd.1 [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:44:28.421 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.421 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.421 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.421 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.421 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.422 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.422 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:28.422 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:28.422 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:28.448 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:44:28 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-1[49059]: 2026-02-25T11:44:28.199+0000 7ff853bfb640 -1 osd.1 0 waiting for initial osdmap 2026-02-25T11:44:28.449 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:44:28 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-1[49059]: 2026-02-25T11:44:28.202+0000 7ff84fac2640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T11:44:28.449 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: pgmap v36: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:28.449 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2026-02-25T11:44:28.449 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: osdmap e12: 2 total, 1 up, 2 in 2026-02-25T11:44:28.449 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:28.449 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:44:28.450 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: from='osd.1 [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:44:28.450 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.450 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.450 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.450 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.450 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.450 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.450 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:28.450 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:28.451 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:28.457 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: pgmap v36: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:28.457 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2026-02-25T11:44:28.458 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: osdmap e12: 2 total, 1 up, 2 in 2026-02-25T11:44:28.458 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:28.458 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:44:28.458 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: from='osd.1 [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:44:28.458 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.458 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.459 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.459 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.459 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.459 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:28.459 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:28.459 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:28.460 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:29.200 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:29.200 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: osd.default does not exist. Creating it now. 2026-02-25T11:44:29.201 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: Creating OSDs with service ID: default on trial167:['vg_nvme/lv_4'] 2026-02-25T11:44:29.201 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: Marking host: trial167 for OSDSpec preview refresh. 2026-02-25T11:44:29.201 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: Saving service osd.default spec with placement trial167 2026-02-25T11:44:29.201 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial119", "root=default"]}]': finished 2026-02-25T11:44:29.201 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: osdmap e13: 2 total, 1 up, 2 in 2026-02-25T11:44:29.201 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:29.201 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:29.201 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4de13b90-1c3c-4614-9fef-2d7315626433"} : dispatch 2026-02-25T11:44:29.201 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: from='client.? 10.20.193.167:0/938136464' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4de13b90-1c3c-4614-9fef-2d7315626433"} : dispatch 2026-02-25T11:44:29.201 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4de13b90-1c3c-4614-9fef-2d7315626433"}]': finished 2026-02-25T11:44:29.201 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: osd.1 [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242] boot 2026-02-25T11:44:29.202 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: osdmap e14: 3 total, 2 up, 3 in 2026-02-25T11:44:29.202 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:29.202 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:29 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:29.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:29.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: osd.default does not exist. Creating it now. 2026-02-25T11:44:29.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: Creating OSDs with service ID: default on trial167:['vg_nvme/lv_4'] 2026-02-25T11:44:29.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: Marking host: trial167 for OSDSpec preview refresh. 2026-02-25T11:44:29.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: Saving service osd.default spec with placement trial167 2026-02-25T11:44:29.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial119", "root=default"]}]': finished 2026-02-25T11:44:29.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: osdmap e13: 2 total, 1 up, 2 in 2026-02-25T11:44:29.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:29.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:29.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4de13b90-1c3c-4614-9fef-2d7315626433"} : dispatch 2026-02-25T11:44:29.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: from='client.? 10.20.193.167:0/938136464' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4de13b90-1c3c-4614-9fef-2d7315626433"} : dispatch 2026-02-25T11:44:29.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4de13b90-1c3c-4614-9fef-2d7315626433"}]': finished 2026-02-25T11:44:29.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: osd.1 [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242] boot 2026-02-25T11:44:29.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: osdmap e14: 3 total, 2 up, 3 in 2026-02-25T11:44:29.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:29.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:29 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:29.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: from='client.24307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_4", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:29.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: osd.default does not exist. Creating it now. 2026-02-25T11:44:29.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: Creating OSDs with service ID: default on trial167:['vg_nvme/lv_4'] 2026-02-25T11:44:29.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: Marking host: trial167 for OSDSpec preview refresh. 2026-02-25T11:44:29.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: Saving service osd.default spec with placement trial167 2026-02-25T11:44:29.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.6836, "args": ["host=trial119", "root=default"]}]': finished 2026-02-25T11:44:29.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: osdmap e13: 2 total, 1 up, 2 in 2026-02-25T11:44:29.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:29.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:29.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4de13b90-1c3c-4614-9fef-2d7315626433"} : dispatch 2026-02-25T11:44:29.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: from='client.? 10.20.193.167:0/938136464' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "4de13b90-1c3c-4614-9fef-2d7315626433"} : dispatch 2026-02-25T11:44:29.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4de13b90-1c3c-4614-9fef-2d7315626433"}]': finished 2026-02-25T11:44:29.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: osd.1 [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242] boot 2026-02-25T11:44:29.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: osdmap e14: 3 total, 2 up, 3 in 2026-02-25T11:44:29.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2026-02-25T11:44:29.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:29 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: purged_snaps scrub starts 2026-02-25T11:44:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: purged_snaps scrub ok 2026-02-25T11:44:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: pgmap v39: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='client.? 10.20.193.167:0/2071650374' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:30.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:30.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:30.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:30.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:30.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: purged_snaps scrub starts 2026-02-25T11:44:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: purged_snaps scrub ok 2026-02-25T11:44:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: pgmap v39: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='client.? 10.20.193.167:0/2071650374' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:30.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:30.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:30.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:30.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:30.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: purged_snaps scrub starts 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: purged_snaps scrub ok 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: pgmap v39: 0 pgs: ; 0 B data, 27 MiB used, 700 GiB / 700 GiB avail 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='client.? 10.20.193.167:0/2071650374' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:30.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:30.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:31.445 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:31 trial167 ceph-mon[43392]: Detected new or changed devices on trial119 2026-02-25T11:44:31.445 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:31 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial119 to 84053M 2026-02-25T11:44:31.445 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:31 trial167 ceph-mon[43392]: osdmap e15: 3 total, 2 up, 3 in 2026-02-25T11:44:31.445 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:31.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:31 trial119 ceph-mon[42710]: Detected new or changed devices on trial119 2026-02-25T11:44:31.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:31 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial119 to 84053M 2026-02-25T11:44:31.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:31 trial119 ceph-mon[42710]: osdmap e15: 3 total, 2 up, 3 in 2026-02-25T11:44:31.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:31.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:31 trial028 ceph-mon[39224]: Detected new or changed devices on trial119 2026-02-25T11:44:31.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:31 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial119 to 84053M 2026-02-25T11:44:31.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:31 trial028 ceph-mon[39224]: osdmap e15: 3 total, 2 up, 3 in 2026-02-25T11:44:31.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:32.266 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:32 trial167 ceph-mon[43392]: pgmap v42: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:32.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:32 trial119 ceph-mon[42710]: pgmap v42: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:32.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:32 trial028 ceph-mon[39224]: pgmap v42: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:33.453 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:33 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2026-02-25T11:44:33.454 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:33 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:33.454 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:33 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2026-02-25T11:44:33.454 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:33 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:33.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:33 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2026-02-25T11:44:33.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:33 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:33.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:33 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2026-02-25T11:44:33.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:33 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:33.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:33 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2026-02-25T11:44:33.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:33 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:33.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:33 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2026-02-25T11:44:33.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:33 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:34.494 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:34 trial167 ceph-mon[43392]: Deploying daemon osd.2 on trial167 2026-02-25T11:44:34.495 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:34 trial167 ceph-mon[43392]: Deploying daemon osd.2 on trial167 2026-02-25T11:44:34.495 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:34 trial167 ceph-mon[43392]: pgmap v43: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:34.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:34 trial119 ceph-mon[42710]: Deploying daemon osd.2 on trial167 2026-02-25T11:44:34.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:34 trial119 ceph-mon[42710]: Deploying daemon osd.2 on trial167 2026-02-25T11:44:34.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:34 trial119 ceph-mon[42710]: pgmap v43: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:34.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:34 trial028 ceph-mon[39224]: Deploying daemon osd.2 on trial167 2026-02-25T11:44:34.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:34 trial028 ceph-mon[39224]: Deploying daemon osd.2 on trial167 2026-02-25T11:44:34.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:34 trial028 ceph-mon[39224]: pgmap v43: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:36.389 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:36 trial167 ceph-mon[43392]: pgmap v44: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:36.389 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:36 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:36.389 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:36 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:36.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:36 trial119 ceph-mon[42710]: pgmap v44: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:36.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:36 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:36.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:36 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:36.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:36 trial028 ceph-mon[39224]: pgmap v44: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:36.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:36.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:37.343 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:37 trial167 ceph-mon[43392]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-25T11:44:37.344 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:37 trial167 ceph-mon[43392]: from='osd.2 [v2:10.20.193.167:6800/1113929535,v1:10.20.193.167:6801/1113929535]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-25T11:44:37.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:37 trial119 ceph-mon[42710]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-25T11:44:37.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:37 trial119 ceph-mon[42710]: from='osd.2 [v2:10.20.193.167:6800/1113929535,v1:10.20.193.167:6801/1113929535]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-25T11:44:37.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:37 trial028 ceph-mon[39224]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-25T11:44:37.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:37 trial028 ceph-mon[39224]: from='osd.2 [v2:10.20.193.167:6800/1113929535,v1:10.20.193.167:6801/1113929535]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-25T11:44:37.776 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:37.773+0000 7fd712e85640 1 -- 10.20.193.167:0/1663047604 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=13}) -- 0x7fd6e4003560 con 0x7fd70c10b2c0 2026-02-25T11:44:37.776 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:37.773+0000 7fd6fa7fc640 1 -- 10.20.193.167:0/1663047604 <== mon.1 v2:10.20.193.167:3300/0 7 ==== osd_map(13..16 src has 1..16) ==== 2965+0+0 (secure 0 0 0) 0x7fd6fc0d1e80 con 0x7fd70c10b2c0 2026-02-25T11:44:38.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:38 trial167 ceph-mon[43392]: pgmap v45: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:38.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:38 trial167 ceph-mon[43392]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2026-02-25T11:44:38.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:38 trial167 ceph-mon[43392]: osdmap e16: 3 total, 2 up, 3 in 2026-02-25T11:44:38.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:38.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:38 trial119 ceph-mon[42710]: pgmap v45: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:38.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:38 trial119 ceph-mon[42710]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2026-02-25T11:44:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:38 trial119 ceph-mon[42710]: osdmap e16: 3 total, 2 up, 3 in 2026-02-25T11:44:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:38.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:38 trial028 ceph-mon[39224]: pgmap v45: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:38.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:38 trial028 ceph-mon[39224]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2026-02-25T11:44:38.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:38 trial028 ceph-mon[39224]: osdmap e16: 3 total, 2 up, 3 in 2026-02-25T11:44:38.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:39.739 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:39.736+0000 7fd6fa7fc640 1 -- 10.20.193.167:0/1663047604 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7fd70c172ae0 con 0x7fd6e80b56d0 2026-02-25T11:44:39.739 INFO:teuthology.orchestra.run.trial167.stdout:Created osd(s) 2 on host 'trial167' 2026-02-25T11:44:39.742 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:39.739+0000 7fd7123fd640 1 -- 10.20.193.167:0/1663047604 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd6e80b56d0 msgr2=0x7fd6e80d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:39.742 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:39.739+0000 7fd7123fd640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd6e80b56d0 0x7fd6e80d5b80 secure :-1 s=READY pgs=87 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd700004740 tx=0x7fd700005f90 comp rx=0 tx=0).stop 2026-02-25T11:44:39.742 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:39.739+0000 7fd7123fd640 1 -- 10.20.193.167:0/1663047604 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd70c10b2c0 msgr2=0x7fd70c16f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:39.742 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:39.739+0000 7fd7123fd640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd70c10b2c0 0x7fd70c16f100 secure :-1 s=READY pgs=24 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd6fc0098a0 tx=0x7fd6fc0028d0 comp rx=0 tx=0).stop 2026-02-25T11:44:39.742 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:39.739+0000 7fd7123fd640 1 -- 10.20.193.167:0/1663047604 shutdown_connections 2026-02-25T11:44:39.742 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:39.739+0000 7fd7123fd640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd6e80b56d0 0x7fd6e80d5b80 unknown :-1 s=CLOSED pgs=87 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:39.743 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:39.739+0000 7fd7123fd640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd70c14ec50 0x7fd70c11c4e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:39.743 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:39.739+0000 7fd7123fd640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd70c10bcd0 0x7fd70c109090 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:39.743 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:39.739+0000 7fd7123fd640 1 --2- 10.20.193.167:0/1663047604 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd70c10b2c0 0x7fd70c16f100 unknown :-1 s=CLOSED pgs=24 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:39.743 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:39.739+0000 7fd7123fd640 1 -- 10.20.193.167:0/1663047604 >> 10.20.193.167:0/1663047604 conn(0x7fd70c0e8cb0 msgr2=0x7fd70c10c210 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:44:39.743 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:39.739+0000 7fd7123fd640 1 -- 10.20.193.167:0/1663047604 shutdown_connections 2026-02-25T11:44:39.743 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:44:39.739+0000 7fd7123fd640 1 -- 10.20.193.167:0/1663047604 wait complete. 2026-02-25T11:44:39.843 DEBUG:teuthology.orchestra.run.trial167:osd.2> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.2.service 2026-02-25T11:44:39.847 INFO:tasks.cephadm:Deploying osd.3 on trial028 with /dev/vg_nvme/lv_3... 2026-02-25T11:44:39.847 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_3 2026-02-25T11:44:40.039 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:44:40.158 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:39 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:40.158 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:39 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:40.158 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:39 trial028 ceph-mon[39224]: pgmap v47: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:40.158 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:39 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:40.158 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:39 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:40.159 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:39 trial028 ceph-mon[39224]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-25T11:44:40.159 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:39 trial028 ceph-mon[39224]: from='osd.2 [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-25T11:44:40.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:39 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:40.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:39 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:40.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:39 trial167 ceph-mon[43392]: pgmap v47: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:40.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:39 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:40.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:39 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:40.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:39 trial167 ceph-mon[43392]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-25T11:44:40.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:39 trial167 ceph-mon[43392]: from='osd.2 [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-25T11:44:40.251 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:44:39 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2[49994]: 2026-02-25T11:44:39.949+0000 7f24ec411780 -1 osd.2 0 log_to_monitors true 2026-02-25T11:44:40.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:40.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:40.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:39 trial119 ceph-mon[42710]: pgmap v47: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:40.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:40.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:40.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:39 trial119 ceph-mon[42710]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-25T11:44:40.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:39 trial119 ceph-mon[42710]: from='osd.2 [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2026-02-25T11:44:40.987 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:44:41.011 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch daemon add osd trial028:vg_nvme/lv_3 --skip-validation 2026-02-25T11:44:41.198 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:44:41.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.372+0000 7fa0b3fff640 1 -- 10.20.193.28:0/2144214674 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa0b410be60 msgr2=0x7fa0b410c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:41.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.372+0000 7fa0b3fff640 1 --2- 10.20.193.28:0/2144214674 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa0b410be60 0x7fa0b410c2d0 secure :-1 s=READY pgs=149 gs=1 cs=0 l=1 c_cookie=8473099492156770161 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa09c009950 tx=0x7fa09c02f190 comp rx=0 tx=0).stop 2026-02-25T11:44:41.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.372+0000 7fa0b3fff640 1 -- 10.20.193.28:0/2144214674 shutdown_connections 2026-02-25T11:44:41.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.372+0000 7fa0b3fff640 1 --2- 10.20.193.28:0/2144214674 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa0b410c9c0 0x7fa0b410ce60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:41.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.372+0000 7fa0b3fff640 1 --2- 10.20.193.28:0/2144214674 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa0b410be60 0x7fa0b410c2d0 unknown :-1 s=CLOSED pgs=149 gs=1 cs=0 l=1 c_cookie=8473099492156770161 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:41.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.372+0000 7fa0b3fff640 1 --2- 10.20.193.28:0/2144214674 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa0b40609a0 0x7fa0b410b920 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:41.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.372+0000 7fa0b3fff640 1 -- 10.20.193.28:0/2144214674 >> 10.20.193.28:0/2144214674 conn(0x7fa0b40e9580 msgr2=0x7fa0b4109990 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:44:41.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.373+0000 7fa0b3fff640 1 -- 10.20.193.28:0/2144214674 shutdown_connections 2026-02-25T11:44:41.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.373+0000 7fa0b3fff640 1 -- 10.20.193.28:0/2144214674 wait complete. 2026-02-25T11:44:41.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.373+0000 7fa0b3fff640 1 Processor -- start 2026-02-25T11:44:41.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b3fff640 1 -- start start 2026-02-25T11:44:41.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa0b40609a0 0x7fa0b40f4c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:41.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b3fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa0b410be60 0x7fa0b40f52d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:41.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b3fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa0b410c9c0 0x7fa0b40f5ed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:41.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b3fff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b40f6610 con 0x7fa0b40609a0 2026-02-25T11:44:41.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b3fff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b40f6780 con 0x7fa0b410be60 2026-02-25T11:44:41.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b3fff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b40f6940 con 0x7fa0b410c9c0 2026-02-25T11:44:41.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa0b40609a0 0x7fa0b40f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:41.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa0b40609a0 0x7fa0b40f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:54048/0 (socket says 10.20.193.28:54048) 2026-02-25T11:44:41.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b2ffd640 1 -- 10.20.193.28:0/101006714 learned_addr learned my addr 10.20.193.28:0/101006714 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:44:41.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b27fc640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa0b410be60 0x7fa0b40f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:41.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b37fe640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa0b410c9c0 0x7fa0b40f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:41.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b2ffd640 1 -- 10.20.193.28:0/101006714 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa0b410c9c0 msgr2=0x7fa0b40f5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:41.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b2ffd640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa0b410c9c0 0x7fa0b40f5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:41.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b2ffd640 1 -- 10.20.193.28:0/101006714 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa0b410be60 msgr2=0x7fa0b40f52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:41.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b2ffd640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa0b410be60 0x7fa0b40f52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:41.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.374+0000 7fa0b2ffd640 1 -- 10.20.193.28:0/101006714 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09c009570 con 0x7fa0b40609a0 2026-02-25T11:44:41.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.375+0000 7fa0b2ffd640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa0b40609a0 0x7fa0b40f4c80 secure :-1 s=READY pgs=150 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa0a8005db0 tx=0x7fa0a800ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:44:41.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.375+0000 7fa093fff640 1 -- 10.20.193.28:0/101006714 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa0a8004500 con 0x7fa0b40609a0 2026-02-25T11:44:41.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.375+0000 7fa093fff640 1 -- 10.20.193.28:0/101006714 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa0a8004d10 con 0x7fa0b40609a0 2026-02-25T11:44:41.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.375+0000 7fa093fff640 1 -- 10.20.193.28:0/101006714 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa0a8005280 con 0x7fa0b40609a0 2026-02-25T11:44:41.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.375+0000 7fa0b3fff640 1 -- 10.20.193.28:0/101006714 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b40f6c20 con 0x7fa0b40609a0 2026-02-25T11:44:41.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.375+0000 7fa0b3fff640 1 -- 10.20.193.28:0/101006714 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b40f7140 con 0x7fa0b40609a0 2026-02-25T11:44:41.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.376+0000 7fa0b3fff640 1 -- 10.20.193.28:0/101006714 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa078005300 con 0x7fa0b40609a0 2026-02-25T11:44:41.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.377+0000 7fa093fff640 1 -- 10.20.193.28:0/101006714 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 14) ==== 157066+0+0 (secure 0 0 0) 0x7fa0a8005410 con 0x7fa0b40609a0 2026-02-25T11:44:41.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.379+0000 7fa093fff640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa0880b56d0 0x7fa0880d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:41.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.379+0000 7fa093fff640 1 -- 10.20.193.28:0/101006714 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2436+0+0 (secure 0 0 0) 0x7fa0a80e50a0 con 0x7fa0b40609a0 2026-02-25T11:44:41.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.379+0000 7fa0b27fc640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa0880b56d0 0x7fa0880d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:41.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.379+0000 7fa0b27fc640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa0880b56d0 0x7fa0880d5b80 secure :-1 s=READY pgs=92 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa09c0023d0 tx=0x7fa09c031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:44:41.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.380+0000 7fa093fff640 1 -- 10.20.193.28:0/101006714 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fa0a80a9210 con 0x7fa0b40609a0 2026-02-25T11:44:41.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:41.499+0000 7fa0b3fff640 1 -- 10.20.193.28:0/101006714 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7fa078002bb0 con 0x7fa0880b56d0 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: osdmap e17: 3 total, 2 up, 3 in 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='osd.2 [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: pgmap v49: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.837 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:41.838 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:41.838 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: osdmap e17: 3 total, 2 up, 3 in 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='osd.2 [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: pgmap v49: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:41.862 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:42.000 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:44:41 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2[49994]: 2026-02-25T11:44:41.743+0000 7f24e8393640 -1 osd.2 0 waiting for initial osdmap 2026-02-25T11:44:42.000 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:44:41 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2[49994]: 2026-02-25T11:44:41.748+0000 7f24e425a640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T11:44:42.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:42.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:42.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:42.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2026-02-25T11:44:42.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: osdmap e17: 3 total, 2 up, 3 in 2026-02-25T11:44:42.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:42.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:44:42.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='osd.2 [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:44:42.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: pgmap v49: 0 pgs: ; 0 B data, 53 MiB used, 1.4 TiB / 1.4 TiB avail 2026-02-25T11:44:42.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:42.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:42.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:42.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:42.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:42.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:42.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:42.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:42.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: from='client.14451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: osd.default does not exist. Creating it now. 2026-02-25T11:44:43.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: Creating OSDs with service ID: default on trial028:['vg_nvme/lv_3'] 2026-02-25T11:44:43.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: Marking host: trial028 for OSDSpec preview refresh. 2026-02-25T11:44:43.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: Saving service osd.default spec with placement trial028 2026-02-25T11:44:43.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial167", "root=default"]}]': finished 2026-02-25T11:44:43.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: osdmap e18: 3 total, 2 up, 3 in 2026-02-25T11:44:43.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:43.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:43.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/528106902' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "45d9ea8e-5a77-4738-96da-ada98515fc02"} : dispatch 2026-02-25T11:44:43.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/528106902' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45d9ea8e-5a77-4738-96da-ada98515fc02"}]': finished 2026-02-25T11:44:43.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: osd.2 [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559] boot 2026-02-25T11:44:43.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: osdmap e19: 4 total, 3 up, 4 in 2026-02-25T11:44:43.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:43.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:42 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:43.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: from='client.14451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:43.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: osd.default does not exist. Creating it now. 2026-02-25T11:44:43.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: Creating OSDs with service ID: default on trial028:['vg_nvme/lv_3'] 2026-02-25T11:44:43.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: Marking host: trial028 for OSDSpec preview refresh. 2026-02-25T11:44:43.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: Saving service osd.default spec with placement trial028 2026-02-25T11:44:43.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial167", "root=default"]}]': finished 2026-02-25T11:44:43.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: osdmap e18: 3 total, 2 up, 3 in 2026-02-25T11:44:43.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:43.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:43.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/528106902' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "45d9ea8e-5a77-4738-96da-ada98515fc02"} : dispatch 2026-02-25T11:44:43.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/528106902' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45d9ea8e-5a77-4738-96da-ada98515fc02"}]': finished 2026-02-25T11:44:43.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: osd.2 [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559] boot 2026-02-25T11:44:43.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: osdmap e19: 4 total, 3 up, 4 in 2026-02-25T11:44:43.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:43.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:42 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:43.041 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: from='client.14451 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:43.041 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: osd.default does not exist. Creating it now. 2026-02-25T11:44:43.041 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: Creating OSDs with service ID: default on trial028:['vg_nvme/lv_3'] 2026-02-25T11:44:43.041 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: Marking host: trial028 for OSDSpec preview refresh. 2026-02-25T11:44:43.041 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: Saving service osd.default spec with placement trial028 2026-02-25T11:44:43.042 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.6836, "args": ["host=trial167", "root=default"]}]': finished 2026-02-25T11:44:43.042 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: osdmap e18: 3 total, 2 up, 3 in 2026-02-25T11:44:43.042 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:43.042 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:43.042 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/528106902' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "45d9ea8e-5a77-4738-96da-ada98515fc02"} : dispatch 2026-02-25T11:44:43.042 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/528106902' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45d9ea8e-5a77-4738-96da-ada98515fc02"}]': finished 2026-02-25T11:44:43.042 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: osd.2 [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559] boot 2026-02-25T11:44:43.042 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: osdmap e19: 4 total, 3 up, 4 in 2026-02-25T11:44:43.042 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2026-02-25T11:44:43.042 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:42 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:44.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: purged_snaps scrub starts 2026-02-25T11:44:44.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: purged_snaps scrub ok 2026-02-25T11:44:44.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1339465662' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:44.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: pgmap v52: 0 pgs: ; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:44.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2026-02-25T11:44:44.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:44.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:44.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:44.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:44.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:44.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: purged_snaps scrub starts 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: purged_snaps scrub ok 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1339465662' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: pgmap v52: 0 pgs: ; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:44.008 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:44.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: purged_snaps scrub starts 2026-02-25T11:44:44.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: purged_snaps scrub ok 2026-02-25T11:44:44.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1339465662' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:44.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: pgmap v52: 0 pgs: ; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:44.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2026-02-25T11:44:44.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:44.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:44.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:44.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:44.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:44.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:44.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:44.930 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:44 trial028 ceph-mon[39224]: Detected new or changed devices on trial167 2026-02-25T11:44:44.930 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:44 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial167 to 84053M 2026-02-25T11:44:44.930 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:44 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2026-02-25T11:44:44.930 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:44 trial028 ceph-mon[39224]: osdmap e20: 4 total, 3 up, 4 in 2026-02-25T11:44:44.930 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:44 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:44.931 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:44 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2026-02-25T11:44:44.931 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:44 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2026-02-25T11:44:44.931 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:44 trial028 ceph-mon[39224]: osdmap e21: 4 total, 3 up, 4 in 2026-02-25T11:44:44.931 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:44 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:44.931 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:44:44 trial028 sudo[67559]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2026-02-25T11:44:44.931 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:44:44 trial028 sudo[67559]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-25T11:44:44.931 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:44:44 trial028 sudo[67559]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-25T11:44:44.931 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:44:44 trial028 sudo[67559]: pam_unix(sudo:session): session closed for user root 2026-02-25T11:44:44.931 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:44:44 trial028 sudo[67565]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme2n1 2026-02-25T11:44:44.932 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:44:44 trial028 sudo[67565]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-25T11:44:44.932 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:44:44 trial028 sudo[67565]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-25T11:44:44.932 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:44:44 trial028 sudo[67565]: pam_unix(sudo:session): session closed for user root 2026-02-25T11:44:45.000 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:44:44 trial167 sudo[54750]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2026-02-25T11:44:45.000 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:44:44 trial167 sudo[54750]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-25T11:44:45.000 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:44:44 trial167 sudo[54750]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-25T11:44:45.000 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:44:44 trial167 sudo[54750]: pam_unix(sudo:session): session closed for user root 2026-02-25T11:44:45.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:44 trial167 ceph-mon[43392]: Detected new or changed devices on trial167 2026-02-25T11:44:45.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:44 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial167 to 84053M 2026-02-25T11:44:45.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:44 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2026-02-25T11:44:45.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:44 trial167 ceph-mon[43392]: osdmap e20: 4 total, 3 up, 4 in 2026-02-25T11:44:45.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:44 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:45.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:44 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2026-02-25T11:44:45.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:44 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2026-02-25T11:44:45.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:44 trial167 ceph-mon[43392]: osdmap e21: 4 total, 3 up, 4 in 2026-02-25T11:44:45.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:44 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:45.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:44 trial119 ceph-mon[42710]: Detected new or changed devices on trial167 2026-02-25T11:44:45.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:44 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial167 to 84053M 2026-02-25T11:44:45.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:44 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2026-02-25T11:44:45.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:44 trial119 ceph-mon[42710]: osdmap e20: 4 total, 3 up, 4 in 2026-02-25T11:44:45.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:44 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:45.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:44 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2026-02-25T11:44:45.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:44 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2026-02-25T11:44:45.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:44 trial119 ceph-mon[42710]: osdmap e21: 4 total, 3 up, 4 in 2026-02-25T11:44:45.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:44 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:45.011 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:44:44 trial119 sudo[56543]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2026-02-25T11:44:45.011 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:44:44 trial119 sudo[56543]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-25T11:44:45.011 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:44:44 trial119 sudo[56543]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-25T11:44:45.011 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:44:44 trial119 sudo[56543]: pam_unix(sudo:session): session closed for user root 2026-02-25T11:44:45.011 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:44:44 trial119 sudo[56547]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme2n1 2026-02-25T11:44:45.011 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:44:44 trial119 sudo[56547]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-25T11:44:45.011 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:44:44 trial119 sudo[56547]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-25T11:44:45.011 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:44:44 trial119 sudo[56547]: pam_unix(sudo:session): session closed for user root 2026-02-25T11:44:45.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 sudo[67645]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2026-02-25T11:44:45.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 sudo[67645]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-25T11:44:45.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 sudo[67645]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-25T11:44:45.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 sudo[67645]: pam_unix(sudo:session): session closed for user root 2026-02-25T11:44:45.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 sudo[67660]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme1n1 2026-02-25T11:44:45.208 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 sudo[67660]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-25T11:44:45.209 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 sudo[67660]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-25T11:44:45.209 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 sudo[67660]: pam_unix(sudo:session): session closed for user root 2026-02-25T11:44:45.500 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:44:45 trial167 sudo[54754]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme1n1 2026-02-25T11:44:45.500 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:44:45 trial167 sudo[54754]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-25T11:44:45.500 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:44:45 trial167 sudo[54754]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-25T11:44:45.500 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:44:45 trial167 sudo[54754]: pam_unix(sudo:session): session closed for user root 2026-02-25T11:44:45.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 sudo[54758]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2026-02-25T11:44:45.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 sudo[54758]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-25T11:44:45.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 sudo[54758]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-25T11:44:45.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 sudo[54758]: pam_unix(sudo:session): session closed for user root 2026-02-25T11:44:45.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 sudo[54762]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme0n1 2026-02-25T11:44:45.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 sudo[54762]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-25T11:44:45.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 sudo[54762]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-25T11:44:45.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 sudo[54762]: pam_unix(sudo:session): session closed for user root 2026-02-25T11:44:45.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 sudo[56551]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2026-02-25T11:44:45.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 sudo[56551]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-25T11:44:45.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 sudo[56551]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-25T11:44:45.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 sudo[56551]: pam_unix(sudo:session): session closed for user root 2026-02-25T11:44:45.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 sudo[56555]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme mtfdkcc1t9tgp-1bk1jabyy smart-log-add --json /dev/nvme1n1 2026-02-25T11:44:45.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 sudo[56555]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2026-02-25T11:44:45.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 sudo[56555]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2026-02-25T11:44:45.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 sudo[56555]: pam_unix(sudo:session): session closed for user root 2026-02-25T11:44:46.036 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.037 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.037 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.037 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-25T11:44:46.037 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: pgmap v55: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:46.037 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-25T11:44:46.037 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:44:46.037 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:44:46.037 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:44:46.037 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.037 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:44:46.037 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:44:46.038 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:44:46.038 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-25T11:44:46.038 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-25T11:44:46.038 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.038 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:44:46.038 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:44:46.038 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:44:46.038 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-25T11:44:46.038 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-25T11:44:46.038 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.038 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: osdmap e22: 4 total, 3 up, 4 in 2026-02-25T11:44:46.038 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:46.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:46.050+0000 7fa093fff640 1 -- 10.20.193.28:0/101006714 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7fa0a80a8e30 con 0x7fa0b40609a0 2026-02-25T11:44:46.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-25T11:44:46.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: pgmap v55: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:46.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-25T11:44:46.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:44:46.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:44:46.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:44:46.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:44:46.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:44:46.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:44:46.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-25T11:44:46.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-25T11:44:46.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:44:46.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:44:46.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:44:46.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-25T11:44:46.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-25T11:44:46.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: osdmap e22: 4 total, 3 up, 4 in 2026-02-25T11:44:46.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:46.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-25T11:44:46.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: pgmap v55: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:46.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-25T11:44:46.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:44:46.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:44:46.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:44:46.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:44:46.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:44:46.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:44:46.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-25T11:44:46.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-25T11:44:46.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2026-02-25T11:44:46.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2026-02-25T11:44:46.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2026-02-25T11:44:46.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2026-02-25T11:44:46.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2026-02-25T11:44:46.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:46.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: osdmap e22: 4 total, 3 up, 4 in 2026-02-25T11:44:46.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:47.049 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:47 trial028 ceph-mon[39224]: mgrmap e15: x(active, since 89s), standbys: z, y 2026-02-25T11:44:47.049 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:47 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2026-02-25T11:44:47.049 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:47 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:47.049 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:47 trial028 ceph-mon[39224]: Deploying daemon osd.3 on trial028 2026-02-25T11:44:47.049 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:47 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2026-02-25T11:44:47.049 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:47 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:47.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:47 trial167 ceph-mon[43392]: mgrmap e15: x(active, since 89s), standbys: z, y 2026-02-25T11:44:47.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:47 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2026-02-25T11:44:47.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:47 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:47.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:47 trial167 ceph-mon[43392]: Deploying daemon osd.3 on trial028 2026-02-25T11:44:47.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:47 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2026-02-25T11:44:47.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:47 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:47.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:47 trial119 ceph-mon[42710]: mgrmap e15: x(active, since 89s), standbys: z, y 2026-02-25T11:44:47.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:47 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2026-02-25T11:44:47.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:47 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:47.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:47 trial119 ceph-mon[42710]: Deploying daemon osd.3 on trial028 2026-02-25T11:44:47.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:47 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2026-02-25T11:44:47.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:47 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:48.211 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:48 trial028 ceph-mon[39224]: Deploying daemon osd.3 on trial028 2026-02-25T11:44:48.211 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:48 trial028 ceph-mon[39224]: pgmap v57: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:48.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:48 trial167 ceph-mon[43392]: Deploying daemon osd.3 on trial028 2026-02-25T11:44:48.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:48 trial167 ceph-mon[43392]: pgmap v57: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:48.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:48 trial119 ceph-mon[42710]: Deploying daemon osd.3 on trial028 2026-02-25T11:44:48.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:48 trial119 ceph-mon[42710]: pgmap v57: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:50.428 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:50 trial028 ceph-mon[39224]: pgmap v58: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:50.428 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:50.428 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:50.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:50 trial167 ceph-mon[43392]: pgmap v58: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:50.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:50.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:50.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:50 trial119 ceph-mon[42710]: pgmap v58: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:50.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:50.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:51.259 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:51 trial028 ceph-mon[39224]: from='osd.3 [v2:10.20.193.28:6810/1736953726,v1:10.20.193.28:6811/1736953726]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2026-02-25T11:44:51.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:51.377+0000 7fa0b8b20640 1 -- 10.20.193.28:0/101006714 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=18}) -- 0x7fa08c003560 con 0x7fa0b40609a0 2026-02-25T11:44:51.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:51.377+0000 7fa093fff640 1 -- 10.20.193.28:0/101006714 <== mon.0 v2:10.20.193.28:3300/0 8 ==== osd_map(18..23 src has 1..23) ==== 4891+0+0 (secure 0 0 0) 0x7fa0a80ad5c0 con 0x7fa0b40609a0 2026-02-25T11:44:51.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:51 trial167 ceph-mon[43392]: from='osd.3 [v2:10.20.193.28:6810/1736953726,v1:10.20.193.28:6811/1736953726]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2026-02-25T11:44:51.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:51 trial119 ceph-mon[42710]: from='osd.3 [v2:10.20.193.28:6810/1736953726,v1:10.20.193.28:6811/1736953726]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2026-02-25T11:44:52.258 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:52 trial028 ceph-mon[39224]: pgmap v59: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:52.258 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:52 trial028 ceph-mon[39224]: from='osd.3 [v2:10.20.193.28:6810/1736953726,v1:10.20.193.28:6811/1736953726]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2026-02-25T11:44:52.258 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:52 trial028 ceph-mon[39224]: osdmap e23: 4 total, 3 up, 4 in 2026-02-25T11:44:52.259 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:52.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:52 trial167 ceph-mon[43392]: pgmap v59: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:52.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:52 trial167 ceph-mon[43392]: from='osd.3 [v2:10.20.193.28:6810/1736953726,v1:10.20.193.28:6811/1736953726]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2026-02-25T11:44:52.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:52 trial167 ceph-mon[43392]: osdmap e23: 4 total, 3 up, 4 in 2026-02-25T11:44:52.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:52.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:52 trial119 ceph-mon[42710]: pgmap v59: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:52.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:52 trial119 ceph-mon[42710]: from='osd.3 [v2:10.20.193.28:6810/1736953726,v1:10.20.193.28:6811/1736953726]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2026-02-25T11:44:52.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:52 trial119 ceph-mon[42710]: osdmap e23: 4 total, 3 up, 4 in 2026-02-25T11:44:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:53.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:53.266+0000 7fa093fff640 1 -- 10.20.193.28:0/101006714 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7fa078002bb0 con 0x7fa0880b56d0 2026-02-25T11:44:53.269 INFO:teuthology.orchestra.run.trial028.stdout:Created osd(s) 3 on host 'trial028' 2026-02-25T11:44:53.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:53.269+0000 7fa0b3fff640 1 -- 10.20.193.28:0/101006714 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa0880b56d0 msgr2=0x7fa0880d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:53.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:53.269+0000 7fa0b3fff640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa0880b56d0 0x7fa0880d5b80 secure :-1 s=READY pgs=92 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa09c0023d0 tx=0x7fa09c031040 comp rx=0 tx=0).stop 2026-02-25T11:44:53.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:53.269+0000 7fa0b3fff640 1 -- 10.20.193.28:0/101006714 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa0b40609a0 msgr2=0x7fa0b40f4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:53.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:53.269+0000 7fa0b3fff640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa0b40609a0 0x7fa0b40f4c80 secure :-1 s=READY pgs=150 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa0a8005db0 tx=0x7fa0a800ddf0 comp rx=0 tx=0).stop 2026-02-25T11:44:53.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:53.269+0000 7fa0b3fff640 1 -- 10.20.193.28:0/101006714 shutdown_connections 2026-02-25T11:44:53.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:53.269+0000 7fa0b3fff640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa0880b56d0 0x7fa0880d5b80 unknown :-1 s=CLOSED pgs=92 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:53.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:53.269+0000 7fa0b3fff640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa0b410c9c0 0x7fa0b40f5ed0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:53.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:53.269+0000 7fa0b3fff640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa0b410be60 0x7fa0b40f52d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:53.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:53.269+0000 7fa0b3fff640 1 --2- 10.20.193.28:0/101006714 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa0b40609a0 0x7fa0b40f4c80 unknown :-1 s=CLOSED pgs=150 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:53.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:53.269+0000 7fa0b3fff640 1 -- 10.20.193.28:0/101006714 >> 10.20.193.28:0/101006714 conn(0x7fa0b40e9580 msgr2=0x7fa0b40eb230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:44:53.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:53.269+0000 7fa0b3fff640 1 -- 10.20.193.28:0/101006714 shutdown_connections 2026-02-25T11:44:53.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:44:53.269+0000 7fa0b3fff640 1 -- 10.20.193.28:0/101006714 wait complete. 2026-02-25T11:44:53.409 DEBUG:teuthology.orchestra.run.trial028:osd.3> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.3.service 2026-02-25T11:44:53.413 INFO:tasks.cephadm:Deploying osd.4 on trial119 with /dev/vg_nvme/lv_3... 2026-02-25T11:44:53.413 DEBUG:teuthology.orchestra.run.trial119:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_3 2026-02-25T11:44:53.600 INFO:teuthology.orchestra.run.trial119.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.b/config 2026-02-25T11:44:53.716 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:53.716 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:53.716 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:53 trial119 ceph-mon[42710]: pgmap v61: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:53.716 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:53.716 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:53.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:53.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:53.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:53 trial028 ceph-mon[39224]: pgmap v61: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:53.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:53.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:53.810 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:44:53 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3[70080]: 2026-02-25T11:44:53.537+0000 7f3e5c26a780 -1 osd.3 0 log_to_monitors true 2026-02-25T11:44:54.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:54.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:54.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:53 trial167 ceph-mon[43392]: pgmap v61: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:54.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:54.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:54.638 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:44:54.660 DEBUG:teuthology.orchestra.run.trial119:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch daemon add osd trial119:vg_nvme/lv_3 --skip-validation 2026-02-25T11:44:54.711 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:54 trial119 ceph-mon[42710]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2026-02-25T11:44:54.711 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:54 trial119 ceph-mon[42710]: from='osd.3 [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2026-02-25T11:44:54.712 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:54 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:54.712 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:54 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:54.712 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:54 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:54.712 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:54 trial119 ceph-mon[42710]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2026-02-25T11:44:54.712 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:54 trial119 ceph-mon[42710]: osdmap e24: 4 total, 3 up, 4 in 2026-02-25T11:44:54.712 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:54 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:54.712 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:54 trial119 ceph-mon[42710]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:44:54.712 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:54 trial119 ceph-mon[42710]: from='osd.3 [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:44:54.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:54 trial028 ceph-mon[39224]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2026-02-25T11:44:54.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:54 trial028 ceph-mon[39224]: from='osd.3 [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2026-02-25T11:44:54.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:54 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:54.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:54 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:54.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:54 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:54.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:54 trial028 ceph-mon[39224]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2026-02-25T11:44:54.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:54 trial028 ceph-mon[39224]: osdmap e24: 4 total, 3 up, 4 in 2026-02-25T11:44:54.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:54 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:54.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:54 trial028 ceph-mon[39224]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:44:54.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:54 trial028 ceph-mon[39224]: from='osd.3 [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:44:54.848 INFO:teuthology.orchestra.run.trial119.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.b/config 2026-02-25T11:44:54.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:54 trial167 ceph-mon[43392]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2026-02-25T11:44:55.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:54 trial167 ceph-mon[43392]: from='osd.3 [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2026-02-25T11:44:55.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:54 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:55.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:54 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:55.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:54 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:44:55.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:54 trial167 ceph-mon[43392]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2026-02-25T11:44:55.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:54 trial167 ceph-mon[43392]: osdmap e24: 4 total, 3 up, 4 in 2026-02-25T11:44:55.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:54 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:55.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:54 trial167 ceph-mon[43392]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:44:55.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:54 trial167 ceph-mon[43392]: from='osd.3 [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:44:55.035 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.032+0000 7f9886029640 1 -- 10.20.193.119:0/4216460757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f988010d510 msgr2=0x7f988010d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:55.036 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.032+0000 7f9886029640 1 --2- 10.20.193.119:0/4216460757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f988010d510 0x7f988010d900 secure :-1 s=READY pgs=37 gs=3 cs=0 l=1 c_cookie=2688932727733499682 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9870009950 tx=0x7f987002f190 comp rx=0 tx=0).stop 2026-02-25T11:44:55.036 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.032+0000 7f9886029640 1 -- 10.20.193.119:0/4216460757 shutdown_connections 2026-02-25T11:44:55.036 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.032+0000 7f9886029640 1 --2- 10.20.193.119:0/4216460757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f988010b620 0x7f988010ba90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:55.036 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.032+0000 7f9886029640 1 --2- 10.20.193.119:0/4216460757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f988010de40 0x7f988010b090 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:55.036 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.032+0000 7f9886029640 1 --2- 10.20.193.119:0/4216460757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f988010d510 0x7f988010d900 unknown :-1 s=CLOSED pgs=37 gs=3 cs=0 l=1 c_cookie=2688932727733499682 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:55.036 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.032+0000 7f9886029640 1 -- 10.20.193.119:0/4216460757 >> 10.20.193.119:0/4216460757 conn(0x7f98800e8cb0 msgr2=0x7f98801090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:44:55.036 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.032+0000 7f9886029640 1 -- 10.20.193.119:0/4216460757 shutdown_connections 2026-02-25T11:44:55.036 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.032+0000 7f9886029640 1 -- 10.20.193.119:0/4216460757 wait complete. 2026-02-25T11:44:55.036 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.032+0000 7f9886029640 1 Processor -- start 2026-02-25T11:44:55.036 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9886029640 1 -- start start 2026-02-25T11:44:55.037 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9886029640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f988010b620 0x7f98800f4820 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:55.037 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9886029640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f988010d510 0x7f98800f4e70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:55.037 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9886029640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f988010de40 0x7f98800f5a70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:55.037 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9886029640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f98800f61b0 con 0x7f988010d510 2026-02-25T11:44:55.037 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9886029640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f98800f6320 con 0x7f988010de40 2026-02-25T11:44:55.037 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9886029640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f98800f64e0 con 0x7f988010b620 2026-02-25T11:44:55.037 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9885828640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f988010de40 0x7f98800f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:55.037 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9885027640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f988010b620 0x7f98800f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:55.037 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9885828640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f988010de40 0x7f98800f5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.119:55078/0 (socket says 10.20.193.119:55078) 2026-02-25T11:44:55.037 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9885828640 1 -- 10.20.193.119:0/2217587839 learned_addr learned my addr 10.20.193.119:0/2217587839 (peer_addr_for_me v2:10.20.193.119:0/0) 2026-02-25T11:44:55.037 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9885828640 1 -- 10.20.193.119:0/2217587839 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f988010b620 msgr2=0x7f98800f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:55.037 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9885828640 1 --2- 10.20.193.119:0/2217587839 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f988010b620 0x7f98800f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:55.038 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9885828640 1 -- 10.20.193.119:0/2217587839 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f988010d510 msgr2=0x7f98800f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:44:55.038 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9885828640 1 --2- 10.20.193.119:0/2217587839 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f988010d510 0x7f98800f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:44:55.038 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.033+0000 7f9885828640 1 -- 10.20.193.119:0/2217587839 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9870009570 con 0x7f988010de40 2026-02-25T11:44:55.038 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.034+0000 7f9885828640 1 --2- 10.20.193.119:0/2217587839 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f988010de40 0x7f98800f5a70 secure :-1 s=READY pgs=38 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f987c006e90 tx=0x7f987c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:44:55.038 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.034+0000 7f986e7fc640 1 -- 10.20.193.119:0/2217587839 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f987c005500 con 0x7f988010de40 2026-02-25T11:44:55.038 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.034+0000 7f9886029640 1 -- 10.20.193.119:0/2217587839 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98800f67c0 con 0x7f988010de40 2026-02-25T11:44:55.039 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.034+0000 7f9886029640 1 -- 10.20.193.119:0/2217587839 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98800f6d30 con 0x7f988010de40 2026-02-25T11:44:55.039 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.034+0000 7f986e7fc640 1 -- 10.20.193.119:0/2217587839 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f987c005df0 con 0x7f988010de40 2026-02-25T11:44:55.039 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.034+0000 7f986e7fc640 1 -- 10.20.193.119:0/2217587839 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f987c003a00 con 0x7f988010de40 2026-02-25T11:44:55.039 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.034+0000 7f9886029640 1 -- 10.20.193.119:0/2217587839 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9848005300 con 0x7f988010de40 2026-02-25T11:44:55.039 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.036+0000 7f986e7fc640 1 -- 10.20.193.119:0/2217587839 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f987c003b60 con 0x7f988010de40 2026-02-25T11:44:55.040 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.038+0000 7f986e7fc640 1 --2- 10.20.193.119:0/2217587839 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f985c0b5760 0x7f985c0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:44:55.040 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.038+0000 7f986e7fc640 1 -- 10.20.193.119:0/2217587839 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3365+0+0 (secure 0 0 0) 0x7f987c0e68a0 con 0x7f988010de40 2026-02-25T11:44:55.040 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.039+0000 7f9885027640 1 --2- 10.20.193.119:0/2217587839 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f985c0b5760 0x7f985c0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:44:55.041 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.039+0000 7f986e7fc640 1 -- 10.20.193.119:0/2217587839 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f987c0aa550 con 0x7f988010de40 2026-02-25T11:44:55.041 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.039+0000 7f9885027640 1 --2- 10.20.193.119:0/2217587839 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f985c0b5760 0x7f985c0d5c10 secure :-1 s=READY pgs=98 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f98700098a0 tx=0x7f987003a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:44:55.159 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:44:55.158+0000 7f9886029640 1 -- 10.20.193.119:0/2217587839 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7f9848002bb0 con 0x7f985c0b5760 2026-02-25T11:44:55.423 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:44:55 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3[70080]: 2026-02-25T11:44:55.297+0000 7f3e581ec640 -1 osd.3 0 waiting for initial osdmap 2026-02-25T11:44:55.423 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:44:55 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3[70080]: 2026-02-25T11:44:55.304+0000 7f3e500ab640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T11:44:56.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: pgmap v63: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:56.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:56.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: osd.default does not exist. Creating it now. 2026-02-25T11:44:56.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: Creating OSDs with service ID: default on trial119:['vg_nvme/lv_3'] 2026-02-25T11:44:56.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: Marking host: trial119 for OSDSpec preview refresh. 2026-02-25T11:44:56.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: Saving service osd.default spec with placement trial119 2026-02-25T11:44:56.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:56.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:56.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:56.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-25T11:44:56.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: osdmap e25: 4 total, 3 up, 4 in 2026-02-25T11:44:56.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:56.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: pgmap v63: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: osd.default does not exist. Creating it now. 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: Creating OSDs with service ID: default on trial119:['vg_nvme/lv_3'] 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: Marking host: trial119 for OSDSpec preview refresh. 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: Saving service osd.default spec with placement trial119 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:56.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:56.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-25T11:44:56.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: osdmap e25: 4 total, 3 up, 4 in 2026-02-25T11:44:56.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:56.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:56.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: pgmap v63: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 2.1 TiB / 2.1 TiB avail 2026-02-25T11:44:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: from='client.24377 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:44:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: osd.default does not exist. Creating it now. 2026-02-25T11:44:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: Creating OSDs with service ID: default on trial119:['vg_nvme/lv_3'] 2026-02-25T11:44:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: Marking host: trial119 for OSDSpec preview refresh. 2026-02-25T11:44:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: Saving service osd.default spec with placement trial119 2026-02-25T11:44:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:56.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:56.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:56.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:56.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-25T11:44:56.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: osdmap e25: 4 total, 3 up, 4 in 2026-02-25T11:44:56.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:56.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:57.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: purged_snaps scrub starts 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: purged_snaps scrub ok 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: osd.3 [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436] boot 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: osdmap e26: 4 total, 4 up, 4 in 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c9c3f512-665b-44b2-a6f0-f3356a4b661e"} : dispatch 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='client.? 10.20.193.119:0/2809780276' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c9c3f512-665b-44b2-a6f0-f3356a4b661e"} : dispatch 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9c3f512-665b-44b2-a6f0-f3356a4b661e"}]': finished 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: osdmap e27: 5 total, 4 up, 5 in 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:57.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:57.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:57.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:57.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:57.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:57 trial028 ceph-mon[39224]: from='client.? 10.20.193.119:0/1202422946' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:57.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: purged_snaps scrub starts 2026-02-25T11:44:57.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: purged_snaps scrub ok 2026-02-25T11:44:57.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: osd.3 [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436] boot 2026-02-25T11:44:57.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: osdmap e26: 4 total, 4 up, 4 in 2026-02-25T11:44:57.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:57.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c9c3f512-665b-44b2-a6f0-f3356a4b661e"} : dispatch 2026-02-25T11:44:57.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='client.? 10.20.193.119:0/2809780276' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c9c3f512-665b-44b2-a6f0-f3356a4b661e"} : dispatch 2026-02-25T11:44:57.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9c3f512-665b-44b2-a6f0-f3356a4b661e"}]': finished 2026-02-25T11:44:57.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: osdmap e27: 5 total, 4 up, 5 in 2026-02-25T11:44:57.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:44:57.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:57.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:57.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.752 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:57.752 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:57.752 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.752 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:57.752 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:57.752 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:57.752 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:57 trial167 ceph-mon[43392]: from='client.? 10.20.193.119:0/1202422946' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:57.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: purged_snaps scrub starts 2026-02-25T11:44:57.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: purged_snaps scrub ok 2026-02-25T11:44:57.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: osd.3 [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436] boot 2026-02-25T11:44:57.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: osdmap e26: 4 total, 4 up, 4 in 2026-02-25T11:44:57.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2026-02-25T11:44:57.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c9c3f512-665b-44b2-a6f0-f3356a4b661e"} : dispatch 2026-02-25T11:44:57.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='client.? 10.20.193.119:0/2809780276' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c9c3f512-665b-44b2-a6f0-f3356a4b661e"} : dispatch 2026-02-25T11:44:57.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c9c3f512-665b-44b2-a6f0-f3356a4b661e"}]': finished 2026-02-25T11:44:57.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: osdmap e27: 5 total, 4 up, 5 in 2026-02-25T11:44:57.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:44:57.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:57.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-25T11:44:57.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:57.762 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:44:57.762 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:44:57.762 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:44:57.762 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:44:57.762 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:44:57.762 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:57 trial119 ceph-mon[42710]: from='client.? 10.20.193.119:0/1202422946' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:44:58.555 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:58 trial119 ceph-mon[42710]: Detected new or changed devices on trial028 2026-02-25T11:44:58.555 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:58 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial028 to 42026M 2026-02-25T11:44:58.555 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:58 trial119 ceph-mon[42710]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-25T11:44:58.555 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:58 trial119 ceph-mon[42710]: osdmap e28: 5 total, 4 up, 5 in 2026-02-25T11:44:58.556 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:58 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:44:58.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:58 trial028 ceph-mon[39224]: Detected new or changed devices on trial028 2026-02-25T11:44:58.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:58 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial028 to 42026M 2026-02-25T11:44:58.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:58 trial028 ceph-mon[39224]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-25T11:44:58.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:58 trial028 ceph-mon[39224]: osdmap e28: 5 total, 4 up, 5 in 2026-02-25T11:44:58.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:58 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:44:58.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:58 trial167 ceph-mon[43392]: Detected new or changed devices on trial028 2026-02-25T11:44:58.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:58 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial028 to 42026M 2026-02-25T11:44:58.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:58 trial167 ceph-mon[43392]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-25T11:44:58.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:58 trial167 ceph-mon[43392]: osdmap e28: 5 total, 4 up, 5 in 2026-02-25T11:44:58.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:58 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:44:59.673 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:59 trial119 ceph-mon[42710]: osdmap e29: 5 total, 4 up, 5 in 2026-02-25T11:44:59.673 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:59 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:44:59.673 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:44:59 trial119 ceph-mon[42710]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-25T11:44:59.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:59 trial028 ceph-mon[39224]: osdmap e29: 5 total, 4 up, 5 in 2026-02-25T11:44:59.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:59 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:44:59.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:44:59 trial028 ceph-mon[39224]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-25T11:45:00.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:59 trial167 ceph-mon[43392]: osdmap e29: 5 total, 4 up, 5 in 2026-02-25T11:45:00.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:59 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:00.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:44:59 trial167 ceph-mon[43392]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-25T11:45:00.793 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:00 trial119 ceph-mon[42710]: osdmap e30: 5 total, 4 up, 5 in 2026-02-25T11:45:00.793 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:00.793 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2026-02-25T11:45:00.793 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:00.794 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:00 trial119 ceph-mon[42710]: Deploying daemon osd.4 on trial119 2026-02-25T11:45:00.794 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2026-02-25T11:45:00.794 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:00.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:00 trial028 ceph-mon[39224]: osdmap e30: 5 total, 4 up, 5 in 2026-02-25T11:45:00.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:00.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2026-02-25T11:45:00.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:00.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:00 trial028 ceph-mon[39224]: Deploying daemon osd.4 on trial119 2026-02-25T11:45:00.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2026-02-25T11:45:00.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:01.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:00 trial167 ceph-mon[43392]: osdmap e30: 5 total, 4 up, 5 in 2026-02-25T11:45:01.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:01.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2026-02-25T11:45:01.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:01.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:00 trial167 ceph-mon[43392]: Deploying daemon osd.4 on trial119 2026-02-25T11:45:01.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2026-02-25T11:45:01.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:01.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:01 trial028 ceph-mon[39224]: Deploying daemon osd.4 on trial119 2026-02-25T11:45:01.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:01 trial028 ceph-mon[39224]: pgmap v72: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail; 123 KiB/s, 0 objects/s recovering 2026-02-25T11:45:01.820 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:01 trial119 ceph-mon[42710]: Deploying daemon osd.4 on trial119 2026-02-25T11:45:01.820 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:01 trial119 ceph-mon[42710]: pgmap v72: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail; 123 KiB/s, 0 objects/s recovering 2026-02-25T11:45:01.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:01 trial167 ceph-mon[43392]: Deploying daemon osd.4 on trial119 2026-02-25T11:45:01.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:01 trial167 ceph-mon[43392]: pgmap v72: 1 pgs: 1 peering; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail; 123 KiB/s, 0 objects/s recovering 2026-02-25T11:45:04.162 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:04.162 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:04.162 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:03 trial119 ceph-mon[42710]: pgmap v73: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 96 KiB/s, 0 objects/s recovering 2026-02-25T11:45:04.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:04.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:04.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:03 trial167 ceph-mon[43392]: pgmap v73: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 96 KiB/s, 0 objects/s recovering 2026-02-25T11:45:04.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:04.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:04.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:03 trial028 ceph-mon[39224]: pgmap v73: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 96 KiB/s, 0 objects/s recovering 2026-02-25T11:45:05.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:04 trial119 ceph-mon[42710]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-25T11:45:05.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:04 trial119 ceph-mon[42710]: from='osd.4 [v2:10.20.193.119:6808/3711746550,v1:10.20.193.119:6809/3711746550]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-25T11:45:05.038 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:05.036+0000 7f9886ab1640 1 -- 10.20.193.119:0/2217587839 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=25}) -- 0x7f9858003560 con 0x7f988010de40 2026-02-25T11:45:05.038 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:05.036+0000 7f986e7fc640 1 -- 10.20.193.119:0/2217587839 <== mon.2 v2:10.20.193.119:3300/0 7 ==== osd_map(25..31 src has 1..31) ==== 4229+0+0 (secure 0 0 0) 0x7f987c003e40 con 0x7f988010de40 2026-02-25T11:45:05.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:04 trial167 ceph-mon[43392]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-25T11:45:05.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:04 trial167 ceph-mon[43392]: from='osd.4 [v2:10.20.193.119:6808/3711746550,v1:10.20.193.119:6809/3711746550]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-25T11:45:05.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:04 trial028 ceph-mon[39224]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-25T11:45:05.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:04 trial028 ceph-mon[39224]: from='osd.4 [v2:10.20.193.119:6808/3711746550,v1:10.20.193.119:6809/3711746550]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-25T11:45:05.983 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:05 trial119 ceph-mon[42710]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2026-02-25T11:45:05.983 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:05 trial119 ceph-mon[42710]: osdmap e31: 5 total, 4 up, 5 in 2026-02-25T11:45:05.983 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:05 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:05.984 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:05 trial119 ceph-mon[42710]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 89 KiB/s, 0 objects/s recovering 2026-02-25T11:45:06.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:05 trial167 ceph-mon[43392]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2026-02-25T11:45:06.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:05 trial167 ceph-mon[43392]: osdmap e31: 5 total, 4 up, 5 in 2026-02-25T11:45:06.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:05 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:06.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:05 trial167 ceph-mon[43392]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 89 KiB/s, 0 objects/s recovering 2026-02-25T11:45:06.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:05 trial028 ceph-mon[39224]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2026-02-25T11:45:06.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:05 trial028 ceph-mon[39224]: osdmap e31: 5 total, 4 up, 5 in 2026-02-25T11:45:06.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:05 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:06.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:05 trial028 ceph-mon[39224]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 2.7 TiB / 2.7 TiB avail; 89 KiB/s, 0 objects/s recovering 2026-02-25T11:45:07.030 INFO:teuthology.orchestra.run.trial119.stdout:Created osd(s) 4 on host 'trial119' 2026-02-25T11:45:07.030 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:07.028+0000 7f986e7fc640 1 -- 10.20.193.119:0/2217587839 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f9848002bb0 con 0x7f985c0b5760 2026-02-25T11:45:07.032 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:07.031+0000 7f9886029640 1 -- 10.20.193.119:0/2217587839 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f985c0b5760 msgr2=0x7f985c0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:07.032 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:07.031+0000 7f9886029640 1 --2- 10.20.193.119:0/2217587839 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f985c0b5760 0x7f985c0d5c10 secure :-1 s=READY pgs=98 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f98700098a0 tx=0x7f987003a040 comp rx=0 tx=0).stop 2026-02-25T11:45:07.032 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:07.031+0000 7f9886029640 1 -- 10.20.193.119:0/2217587839 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f988010de40 msgr2=0x7f98800f5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:07.032 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:07.031+0000 7f9886029640 1 --2- 10.20.193.119:0/2217587839 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f988010de40 0x7f98800f5a70 secure :-1 s=READY pgs=38 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f987c006e90 tx=0x7f987c00edf0 comp rx=0 tx=0).stop 2026-02-25T11:45:07.033 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:07.031+0000 7f9886029640 1 -- 10.20.193.119:0/2217587839 shutdown_connections 2026-02-25T11:45:07.033 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:07.031+0000 7f9886029640 1 --2- 10.20.193.119:0/2217587839 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f985c0b5760 0x7f985c0d5c10 unknown :-1 s=CLOSED pgs=98 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:07.033 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:07.031+0000 7f9886029640 1 --2- 10.20.193.119:0/2217587839 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f988010de40 0x7f98800f5a70 unknown :-1 s=CLOSED pgs=38 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:07.033 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:07.031+0000 7f9886029640 1 --2- 10.20.193.119:0/2217587839 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f988010d510 0x7f98800f4e70 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:07.033 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:07.031+0000 7f9886029640 1 --2- 10.20.193.119:0/2217587839 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f988010b620 0x7f98800f4820 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:07.033 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:07.031+0000 7f9886029640 1 -- 10.20.193.119:0/2217587839 >> 10.20.193.119:0/2217587839 conn(0x7f98800e8cb0 msgr2=0x7f98800ea9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:45:07.033 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:07.031+0000 7f9886029640 1 -- 10.20.193.119:0/2217587839 shutdown_connections 2026-02-25T11:45:07.033 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:07.031+0000 7f9886029640 1 -- 10.20.193.119:0/2217587839 wait complete. 2026-02-25T11:45:07.176 DEBUG:teuthology.orchestra.run.trial119:osd.4> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.4.service 2026-02-25T11:45:07.180 INFO:tasks.cephadm:Deploying osd.5 on trial167 with /dev/vg_nvme/lv_3... 2026-02-25T11:45:07.180 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_3 2026-02-25T11:45:07.369 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.c/config 2026-02-25T11:45:07.484 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:07 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:07.484 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:07 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:07.484 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:07 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:07.484 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:07 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:07.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:07 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:07.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:07 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:07.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:07 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:07.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:07 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:07.510 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:45:07 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4[60946]: 2026-02-25T11:45:07.319+0000 7f509b91d780 -1 osd.4 0 log_to_monitors true 2026-02-25T11:45:07.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:07 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:07.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:07 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:07.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:07 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:07.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:07 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:08.319 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:45:08.343 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch daemon add osd trial167:vg_nvme/lv_3 --skip-validation 2026-02-25T11:45:08.394 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:08 trial167 ceph-mon[43392]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail; 72 KiB/s, 0 objects/s recovering 2026-02-25T11:45:08.394 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:08 trial167 ceph-mon[43392]: from='osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-25T11:45:08.394 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:08.395 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:08.395 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:45:08.528 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.c/config 2026-02-25T11:45:08.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:08 trial028 ceph-mon[39224]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail; 72 KiB/s, 0 objects/s recovering 2026-02-25T11:45:08.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:08 trial028 ceph-mon[39224]: from='osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-25T11:45:08.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:08.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:08.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:45:08.717 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.714+0000 7f563f69d640 1 -- 10.20.193.167:0/2115062478 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f563810f1e0 msgr2=0x7f563810f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:08.718 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.714+0000 7f563f69d640 1 --2- 10.20.193.167:0/2115062478 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f563810f1e0 0x7f563810f5d0 secure :-1 s=READY pgs=35 gs=1 cs=0 l=1 c_cookie=7667484357531124960 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5630009950 tx=0x7f563002f190 comp rx=0 tx=0).stop 2026-02-25T11:45:08.718 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.714+0000 7f563f69d640 1 -- 10.20.193.167:0/2115062478 shutdown_connections 2026-02-25T11:45:08.718 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.714+0000 7f563f69d640 1 --2- 10.20.193.167:0/2115062478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5638110d20 0x7f56381716b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:08.718 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.714+0000 7f563f69d640 1 --2- 10.20.193.167:0/2115062478 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5638110370 0x7f56381107e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:08.718 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.714+0000 7f563f69d640 1 --2- 10.20.193.167:0/2115062478 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f563810f1e0 0x7f563810f5d0 unknown :-1 s=CLOSED pgs=35 gs=1 cs=0 l=1 c_cookie=7667484357531124960 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:08.718 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.714+0000 7f563f69d640 1 -- 10.20.193.167:0/2115062478 >> 10.20.193.167:0/2115062478 conn(0x7f56380ec950 msgr2=0x7f563810cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:45:08.719 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.714+0000 7f563f69d640 1 -- 10.20.193.167:0/2115062478 shutdown_connections 2026-02-25T11:45:08.719 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.714+0000 7f563f69d640 1 -- 10.20.193.167:0/2115062478 wait complete. 2026-02-25T11:45:08.719 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.715+0000 7f563f69d640 1 Processor -- start 2026-02-25T11:45:08.719 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.715+0000 7f563f69d640 1 -- start start 2026-02-25T11:45:08.719 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.715+0000 7f563f69d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f563810f1e0 0x7f56380f89f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:08.719 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.715+0000 7f563f69d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5638110370 0x7f56381716b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:08.719 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.715+0000 7f563f69d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5638110d20 0x7f56380f9250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:08.719 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.715+0000 7f563f69d640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f56380f9e40 con 0x7f5638110370 2026-02-25T11:45:08.719 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.715+0000 7f563f69d640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56380f9fb0 con 0x7f5638110d20 2026-02-25T11:45:08.719 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.715+0000 7f563f69d640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56380fa170 con 0x7f563810f1e0 2026-02-25T11:45:08.721 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.715+0000 7f563e69b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f563810f1e0 0x7f56380f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:08.721 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f563e69b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f563810f1e0 0x7f56380f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.167:40130/0 (socket says 10.20.193.167:40130) 2026-02-25T11:45:08.722 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f563e69b640 1 -- 10.20.193.167:0/1375688193 learned_addr learned my addr 10.20.193.167:0/1375688193 (peer_addr_for_me v2:10.20.193.167:0/0) 2026-02-25T11:45:08.722 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f563de9a640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5638110370 0x7f56381716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:08.722 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f563ee9c640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5638110d20 0x7f56380f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:08.722 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f563e69b640 1 -- 10.20.193.167:0/1375688193 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5638110d20 msgr2=0x7f56380f9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:08.722 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f563e69b640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5638110d20 0x7f56380f9250 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:08.722 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f563e69b640 1 -- 10.20.193.167:0/1375688193 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5638110370 msgr2=0x7f56381716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:08.722 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f563e69b640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5638110370 0x7f56381716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:08.722 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f563e69b640 1 -- 10.20.193.167:0/1375688193 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5630009570 con 0x7f563810f1e0 2026-02-25T11:45:08.722 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f563e69b640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f563810f1e0 0x7f56380f89f0 secure :-1 s=READY pgs=36 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56300098a0 tx=0x7f56300028d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:45:08.723 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f56237fe640 1 -- 10.20.193.167:0/1375688193 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f563003d070 con 0x7f563810f1e0 2026-02-25T11:45:08.723 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f56237fe640 1 -- 10.20.193.167:0/1375688193 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5630002a70 con 0x7f563810f1e0 2026-02-25T11:45:08.723 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f56237fe640 1 -- 10.20.193.167:0/1375688193 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56300419f0 con 0x7f563810f1e0 2026-02-25T11:45:08.723 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f563f69d640 1 -- 10.20.193.167:0/1375688193 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56380fa3f0 con 0x7f563810f1e0 2026-02-25T11:45:08.723 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.716+0000 7f563f69d640 1 -- 10.20.193.167:0/1375688193 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56380fa8b0 con 0x7f563810f1e0 2026-02-25T11:45:08.723 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.717+0000 7f563f69d640 1 -- 10.20.193.167:0/1375688193 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5600005300 con 0x7f563810f1e0 2026-02-25T11:45:08.723 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.719+0000 7f56237fe640 1 -- 10.20.193.167:0/1375688193 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f5630041b80 con 0x7f563810f1e0 2026-02-25T11:45:08.724 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.720+0000 7f56237fe640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56140b5760 0x7f56140d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:08.724 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.720+0000 7f56237fe640 1 -- 10.20.193.167:0/1375688193 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 3797+0+0 (secure 0 0 0) 0x7f563010a2d0 con 0x7f563810f1e0 2026-02-25T11:45:08.725 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.721+0000 7f563de9a640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56140b5760 0x7f56140d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:08.725 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.721+0000 7f56237fe640 1 -- 10.20.193.167:0/1375688193 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f56300cde80 con 0x7f563810f1e0 2026-02-25T11:45:08.725 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.721+0000 7f563de9a640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56140b5760 0x7f56140d5c10 secure :-1 s=READY pgs=103 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5624004740 tx=0x7f562400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:45:08.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:08 trial119 ceph-mon[42710]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail; 72 KiB/s, 0 objects/s recovering 2026-02-25T11:45:08.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:08 trial119 ceph-mon[42710]: from='osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2026-02-25T11:45:08.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:08.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:08.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:45:08.845 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:08.842+0000 7f563f69d640 1 -- 10.20.193.167:0/1375688193 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7f5600002bb0 con 0x7f56140b5760 2026-02-25T11:45:09.371 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: osdmap e32: 5 total, 4 up, 5 in 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial119", "root=default"]}]': finished 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: osdmap e33: 5 total, 4 up, 5 in 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:09.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:09.510 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:45:09 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4[60946]: 2026-02-25T11:45:09.273+0000 7f509789f640 -1 osd.4 0 waiting for initial osdmap 2026-02-25T11:45:09.510 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:45:09 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4[60946]: 2026-02-25T11:45:09.280+0000 7f508f75e640 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T11:45:09.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2026-02-25T11:45:09.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: osdmap e32: 5 total, 4 up, 5 in 2026-02-25T11:45:09.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:45:09.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:09.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:09.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:09.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:09.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial119", "root=default"]}]': finished 2026-02-25T11:45:09.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: osdmap e33: 5 total, 4 up, 5 in 2026-02-25T11:45:09.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:09.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:09.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996]' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2026-02-25T11:45:09.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: osdmap e32: 5 total, 4 up, 5 in 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996]' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.6836, "args": ["host=trial119", "root=default"]}]': finished 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: osdmap e33: 5 total, 4 up, 5 in 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:09.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:10.458 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:45:10.458 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: osd.default does not exist. Creating it now. 2026-02-25T11:45:10.458 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: Creating OSDs with service ID: default on trial167:['vg_nvme/lv_3'] 2026-02-25T11:45:10.458 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: Marking host: trial167 for OSDSpec preview refresh. 2026-02-25T11:45:10.459 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: Saving service osd.default spec with placement trial167 2026-02-25T11:45:10.459 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-25T11:45:10.459 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "777c3819-5c90-4b2b-a545-c024884c2eb4"} : dispatch 2026-02-25T11:45:10.459 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: from='client.? 10.20.193.167:0/413053574' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "777c3819-5c90-4b2b-a545-c024884c2eb4"} : dispatch 2026-02-25T11:45:10.459 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "777c3819-5c90-4b2b-a545-c024884c2eb4"}]': finished 2026-02-25T11:45:10.459 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996] boot 2026-02-25T11:45:10.459 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: osdmap e34: 6 total, 5 up, 6 in 2026-02-25T11:45:10.459 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:10.460 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:10.460 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.460 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.460 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.460 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.460 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:10.460 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:10.460 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:45:10.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: osd.default does not exist. Creating it now. 2026-02-25T11:45:10.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: Creating OSDs with service ID: default on trial167:['vg_nvme/lv_3'] 2026-02-25T11:45:10.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: Marking host: trial167 for OSDSpec preview refresh. 2026-02-25T11:45:10.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: Saving service osd.default spec with placement trial167 2026-02-25T11:45:10.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-25T11:45:10.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "777c3819-5c90-4b2b-a545-c024884c2eb4"} : dispatch 2026-02-25T11:45:10.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: from='client.? 10.20.193.167:0/413053574' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "777c3819-5c90-4b2b-a545-c024884c2eb4"} : dispatch 2026-02-25T11:45:10.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "777c3819-5c90-4b2b-a545-c024884c2eb4"}]': finished 2026-02-25T11:45:10.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996] boot 2026-02-25T11:45:10.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: osdmap e34: 6 total, 5 up, 6 in 2026-02-25T11:45:10.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:10.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:10.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:10.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:10.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_3", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:45:10.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: osd.default does not exist. Creating it now. 2026-02-25T11:45:10.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: Creating OSDs with service ID: default on trial167:['vg_nvme/lv_3'] 2026-02-25T11:45:10.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: Marking host: trial167 for OSDSpec preview refresh. 2026-02-25T11:45:10.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: Saving service osd.default spec with placement trial167 2026-02-25T11:45:10.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 2.7 TiB / 2.7 TiB avail 2026-02-25T11:45:10.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "777c3819-5c90-4b2b-a545-c024884c2eb4"} : dispatch 2026-02-25T11:45:10.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: from='client.? 10.20.193.167:0/413053574' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "777c3819-5c90-4b2b-a545-c024884c2eb4"} : dispatch 2026-02-25T11:45:10.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "777c3819-5c90-4b2b-a545-c024884c2eb4"}]': finished 2026-02-25T11:45:10.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: osd.4 [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996] boot 2026-02-25T11:45:10.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: osdmap e34: 6 total, 5 up, 6 in 2026-02-25T11:45:10.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2026-02-25T11:45:10.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:10.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:10.563 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:10.563 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:10.563 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:11.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:11 trial028 ceph-mon[39224]: purged_snaps scrub starts 2026-02-25T11:45:11.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:11 trial028 ceph-mon[39224]: purged_snaps scrub ok 2026-02-25T11:45:11.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:11 trial028 ceph-mon[39224]: Detected new or changed devices on trial119 2026-02-25T11:45:11.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:11 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial119 to 42026M 2026-02-25T11:45:11.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:11.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:11.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:11.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:45:11.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:11.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:11.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:11.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:11.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:11 trial028 ceph-mon[39224]: from='client.? 10.20.193.167:0/2099756948' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:45:11.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:11 trial167 ceph-mon[43392]: purged_snaps scrub starts 2026-02-25T11:45:11.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:11 trial167 ceph-mon[43392]: purged_snaps scrub ok 2026-02-25T11:45:11.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:11 trial167 ceph-mon[43392]: Detected new or changed devices on trial119 2026-02-25T11:45:11.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:11 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial119 to 42026M 2026-02-25T11:45:11.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:11.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:11.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:11.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:45:11.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:11.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:11.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:11.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:11.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:11 trial167 ceph-mon[43392]: from='client.? 10.20.193.167:0/2099756948' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:45:11.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:11 trial119 ceph-mon[42710]: purged_snaps scrub starts 2026-02-25T11:45:11.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:11 trial119 ceph-mon[42710]: purged_snaps scrub ok 2026-02-25T11:45:11.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:11 trial119 ceph-mon[42710]: Detected new or changed devices on trial119 2026-02-25T11:45:11.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:11 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial119 to 42026M 2026-02-25T11:45:11.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:11.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:11.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:11.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:45:11.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:11.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:11.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:11.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:11.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:11 trial119 ceph-mon[42710]: from='client.? 10.20.193.167:0/2099756948' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:45:12.356 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:12 trial167 ceph-mon[43392]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:12.356 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:12 trial167 ceph-mon[43392]: osdmap e35: 6 total, 5 up, 6 in 2026-02-25T11:45:12.356 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:12 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:12.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:12 trial028 ceph-mon[39224]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:12.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:12 trial028 ceph-mon[39224]: osdmap e35: 6 total, 5 up, 6 in 2026-02-25T11:45:12.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:12 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:12.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:12 trial119 ceph-mon[42710]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:12.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:12 trial119 ceph-mon[42710]: osdmap e35: 6 total, 5 up, 6 in 2026-02-25T11:45:12.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:12 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:14.439 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:14 trial167 ceph-mon[43392]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:14.440 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:14 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2026-02-25T11:45:14.440 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:14 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:14.440 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:14 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2026-02-25T11:45:14.440 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:14 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:14.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:14 trial028 ceph-mon[39224]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:14.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:14 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2026-02-25T11:45:14.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:14 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:14.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:14 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2026-02-25T11:45:14.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:14 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:14.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:14 trial119 ceph-mon[42710]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:14.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:14 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2026-02-25T11:45:14.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:14 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:14.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:14 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2026-02-25T11:45:14.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:14 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:15.530 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:15 trial167 ceph-mon[43392]: Deploying daemon osd.5 on trial167 2026-02-25T11:45:15.530 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:15 trial167 ceph-mon[43392]: Deploying daemon osd.5 on trial167 2026-02-25T11:45:15.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:15 trial028 ceph-mon[39224]: Deploying daemon osd.5 on trial167 2026-02-25T11:45:15.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:15 trial028 ceph-mon[39224]: Deploying daemon osd.5 on trial167 2026-02-25T11:45:15.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:15 trial119 ceph-mon[42710]: Deploying daemon osd.5 on trial167 2026-02-25T11:45:15.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:15 trial119 ceph-mon[42710]: Deploying daemon osd.5 on trial167 2026-02-25T11:45:16.471 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:16 trial167 ceph-mon[43392]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:16.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:16 trial028 ceph-mon[39224]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:16.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:16 trial119 ceph-mon[42710]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:17.633 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:17.633 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:17.633 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:17 trial167 ceph-mon[43392]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:17.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:17.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:17.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:17 trial028 ceph-mon[39224]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:18.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:18.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:18.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:17 trial119 ceph-mon[42710]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:18.720 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:18.718+0000 7f5640125640 1 -- 10.20.193.167:0/1375688193 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=33}) -- 0x7f5610003560 con 0x7f563810f1e0 2026-02-25T11:45:18.720 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:18.718+0000 7f56237fe640 1 -- 10.20.193.167:0/1375688193 <== mon.1 v2:10.20.193.167:3300/0 7 ==== osd_map(33..36 src has 1..36) ==== 3403+0+0 (secure 0 0 0) 0x7f5630005bd0 con 0x7f563810f1e0 2026-02-25T11:45:18.796 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:18 trial167 ceph-mon[43392]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-25T11:45:18.797 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:18 trial167 ceph-mon[43392]: from='osd.5 [v2:10.20.193.167:6808/278922660,v1:10.20.193.167:6809/278922660]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-25T11:45:18.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:18 trial028 ceph-mon[39224]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-25T11:45:18.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:18 trial028 ceph-mon[39224]: from='osd.5 [v2:10.20.193.167:6808/278922660,v1:10.20.193.167:6809/278922660]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-25T11:45:19.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:18 trial119 ceph-mon[42710]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-25T11:45:19.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:18 trial119 ceph-mon[42710]: from='osd.5 [v2:10.20.193.167:6808/278922660,v1:10.20.193.167:6809/278922660]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-25T11:45:19.594 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:19 trial167 ceph-mon[43392]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2026-02-25T11:45:19.594 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:19 trial167 ceph-mon[43392]: osdmap e36: 6 total, 5 up, 6 in 2026-02-25T11:45:19.594 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:19 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:19.594 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:19 trial167 ceph-mon[43392]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:19.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:19 trial028 ceph-mon[39224]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2026-02-25T11:45:19.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:19 trial028 ceph-mon[39224]: osdmap e36: 6 total, 5 up, 6 in 2026-02-25T11:45:19.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:19 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:19.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:19 trial028 ceph-mon[39224]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:20.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:19 trial119 ceph-mon[42710]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2026-02-25T11:45:20.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:19 trial119 ceph-mon[42710]: osdmap e36: 6 total, 5 up, 6 in 2026-02-25T11:45:20.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:19 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:20.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:19 trial119 ceph-mon[42710]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:20.669 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:20.667+0000 7f56237fe640 1 -- 10.20.193.167:0/1375688193 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f5600002bb0 con 0x7f56140b5760 2026-02-25T11:45:20.669 INFO:teuthology.orchestra.run.trial167.stdout:Created osd(s) 5 on host 'trial167' 2026-02-25T11:45:20.672 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:20.671+0000 7f563f69d640 1 -- 10.20.193.167:0/1375688193 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56140b5760 msgr2=0x7f56140d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:20.672 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:20.671+0000 7f563f69d640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56140b5760 0x7f56140d5c10 secure :-1 s=READY pgs=103 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5624004740 tx=0x7f562400b410 comp rx=0 tx=0).stop 2026-02-25T11:45:20.672 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:20.671+0000 7f563f69d640 1 -- 10.20.193.167:0/1375688193 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f563810f1e0 msgr2=0x7f56380f89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:20.672 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:20.671+0000 7f563f69d640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f563810f1e0 0x7f56380f89f0 secure :-1 s=READY pgs=36 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56300098a0 tx=0x7f56300028d0 comp rx=0 tx=0).stop 2026-02-25T11:45:20.673 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:20.671+0000 7f563f69d640 1 -- 10.20.193.167:0/1375688193 shutdown_connections 2026-02-25T11:45:20.673 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:20.671+0000 7f563f69d640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56140b5760 0x7f56140d5c10 unknown :-1 s=CLOSED pgs=103 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:20.673 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:20.671+0000 7f563f69d640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5638110d20 0x7f56380f9250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:20.673 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:20.671+0000 7f563f69d640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5638110370 0x7f56381716b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:20.673 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:20.671+0000 7f563f69d640 1 --2- 10.20.193.167:0/1375688193 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f563810f1e0 0x7f56380f89f0 unknown :-1 s=CLOSED pgs=36 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:20.673 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:20.671+0000 7f563f69d640 1 -- 10.20.193.167:0/1375688193 >> 10.20.193.167:0/1375688193 conn(0x7f56380ec950 msgr2=0x7f56380ef010 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:45:20.673 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:20.671+0000 7f563f69d640 1 -- 10.20.193.167:0/1375688193 shutdown_connections 2026-02-25T11:45:20.673 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:20.671+0000 7f563f69d640 1 -- 10.20.193.167:0/1375688193 wait complete. 2026-02-25T11:45:20.773 DEBUG:teuthology.orchestra.run.trial167:osd.5> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.5.service 2026-02-25T11:45:20.776 INFO:tasks.cephadm:Deploying osd.6 on trial028 with /dev/vg_nvme/lv_2... 2026-02-25T11:45:20.777 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_2 2026-02-25T11:45:20.967 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:45:21.087 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:21.087 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:21.087 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:21.087 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:21.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:21.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:21.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:21.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:21.250 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:45:20 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:45:20.900+0000 7f169593c780 -1 osd.5 0 log_to_monitors true 2026-02-25T11:45:21.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:21.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:21.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:21.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:21.868 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:45:21.891 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch daemon add osd trial028:vg_nvme/lv_2 --skip-validation 2026-02-25T11:45:22.076 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:45:22.147 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:21 trial028 ceph-mon[39224]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-25T11:45:22.147 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:21 trial028 ceph-mon[39224]: from='osd.5 [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-25T11:45:22.147 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:21 trial028 ceph-mon[39224]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:22.147 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:21 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:22.147 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:21 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:22.147 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:21 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:45:22.160 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:21 trial119 ceph-mon[42710]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-25T11:45:22.161 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:21 trial119 ceph-mon[42710]: from='osd.5 [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-25T11:45:22.161 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:21 trial119 ceph-mon[42710]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:22.161 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:21 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:22.161 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:21 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:22.161 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:21 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:45:22.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:21 trial167 ceph-mon[43392]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-25T11:45:22.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:21 trial167 ceph-mon[43392]: from='osd.5 [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2026-02-25T11:45:22.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:21 trial167 ceph-mon[43392]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:22.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:21 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:22.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:21 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:22.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:21 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:45:22.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.262+0000 7fc9fc987640 1 -- 10.20.193.28:0/3648760145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9f8110d20 msgr2=0x7fc9f81716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:22.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.262+0000 7fc9fc987640 1 --2- 10.20.193.28:0/3648760145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9f8110d20 0x7fc9f81716b0 secure :-1 s=READY pgs=163 gs=1 cs=0 l=1 c_cookie=616031440962712574 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9e800aa30 tx=0x7fc9e8030270 comp rx=0 tx=0).stop 2026-02-25T11:45:22.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.263+0000 7fc9fc987640 1 -- 10.20.193.28:0/3648760145 shutdown_connections 2026-02-25T11:45:22.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.263+0000 7fc9fc987640 1 --2- 10.20.193.28:0/3648760145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9f8110d20 0x7fc9f81716b0 unknown :-1 s=CLOSED pgs=163 gs=1 cs=0 l=1 c_cookie=616031440962712574 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:22.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.263+0000 7fc9fc987640 1 --2- 10.20.193.28:0/3648760145 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc9f8110370 0x7fc9f81107e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:22.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.263+0000 7fc9fc987640 1 --2- 10.20.193.28:0/3648760145 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9f810f1e0 0x7fc9f810f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:22.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.263+0000 7fc9fc987640 1 -- 10.20.193.28:0/3648760145 >> 10.20.193.28:0/3648760145 conn(0x7fc9f80ec950 msgr2=0x7fc9f810cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:45:22.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.263+0000 7fc9fc987640 1 -- 10.20.193.28:0/3648760145 shutdown_connections 2026-02-25T11:45:22.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.263+0000 7fc9fc987640 1 -- 10.20.193.28:0/3648760145 wait complete. 2026-02-25T11:45:22.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9fc987640 1 Processor -- start 2026-02-25T11:45:22.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9fc987640 1 -- start start 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9fc987640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9f810f1e0 0x7fc9f80f8970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9fc987640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9f8110370 0x7fc9f81716b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9fc987640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc9f8110d20 0x7fc9f80f91d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9fc987640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f80f9dc0 con 0x7fc9f810f1e0 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9fc987640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f80f9f30 con 0x7fc9f8110d20 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9fc987640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f80fa0f0 con 0x7fc9f8110370 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9f6d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9f810f1e0 0x7fc9f80f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9f6575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9f8110370 0x7fc9f81716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9f6d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9f810f1e0 0x7fc9f80f8970 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44448/0 (socket says 10.20.193.28:44448) 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9f6575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9f8110370 0x7fc9f81716b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:42598/0 (socket says 10.20.193.28:42598) 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9f7577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc9f8110d20 0x7fc9f80f91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.264+0000 7fc9f6d76640 1 -- 10.20.193.28:0/4097092342 learned_addr learned my addr 10.20.193.28:0/4097092342 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.265+0000 7fc9f6d76640 1 -- 10.20.193.28:0/4097092342 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9f8110370 msgr2=0x7fc9f81716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:22.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.265+0000 7fc9f6d76640 1 --2- 10.20.193.28:0/4097092342 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9f8110370 0x7fc9f81716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:22.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.265+0000 7fc9f6d76640 1 -- 10.20.193.28:0/4097092342 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc9f8110d20 msgr2=0x7fc9f80f91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:22.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.265+0000 7fc9f6d76640 1 --2- 10.20.193.28:0/4097092342 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc9f8110d20 0x7fc9f80f91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:22.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.265+0000 7fc9f6d76640 1 -- 10.20.193.28:0/4097092342 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e800a650 con 0x7fc9f810f1e0 2026-02-25T11:45:22.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.265+0000 7fc9f6d76640 1 --2- 10.20.193.28:0/4097092342 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9f810f1e0 0x7fc9f80f8970 secure :-1 s=READY pgs=164 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9ec005db0 tx=0x7fc9ec00ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:45:22.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.265+0000 7fc9d7fff640 1 -- 10.20.193.28:0/4097092342 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc9ec004500 con 0x7fc9f810f1e0 2026-02-25T11:45:22.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.265+0000 7fc9d7fff640 1 -- 10.20.193.28:0/4097092342 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc9ec004d10 con 0x7fc9f810f1e0 2026-02-25T11:45:22.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.265+0000 7fc9d7fff640 1 -- 10.20.193.28:0/4097092342 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc9ec0052a0 con 0x7fc9f810f1e0 2026-02-25T11:45:22.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.265+0000 7fc9fc987640 1 -- 10.20.193.28:0/4097092342 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f80fa3d0 con 0x7fc9f810f1e0 2026-02-25T11:45:22.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.265+0000 7fc9fc987640 1 -- 10.20.193.28:0/4097092342 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f80fa8a0 con 0x7fc9f810f1e0 2026-02-25T11:45:22.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.266+0000 7fc9fc987640 1 -- 10.20.193.28:0/4097092342 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f81755d0 con 0x7fc9f810f1e0 2026-02-25T11:45:22.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.268+0000 7fc9d7fff640 1 -- 10.20.193.28:0/4097092342 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7fc9ec005430 con 0x7fc9f810f1e0 2026-02-25T11:45:22.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.270+0000 7fc9d7fff640 1 --2- 10.20.193.28:0/4097092342 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9cc0b5760 0x7fc9cc0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:22.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.270+0000 7fc9d7fff640 1 -- 10.20.193.28:0/4097092342 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4229+0+0 (secure 0 0 0) 0x7fc9ec0e5f00 con 0x7fc9f810f1e0 2026-02-25T11:45:22.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.270+0000 7fc9d7fff640 1 -- 10.20.193.28:0/4097092342 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fc9ec0acbe0 con 0x7fc9f810f1e0 2026-02-25T11:45:22.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.270+0000 7fc9f6575640 1 --2- 10.20.193.28:0/4097092342 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9cc0b5760 0x7fc9cc0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:22.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.271+0000 7fc9f6575640 1 --2- 10.20.193.28:0/4097092342 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9cc0b5760 0x7fc9cc0d5c10 secure :-1 s=READY pgs=108 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9e0004740 tx=0x7fc9e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:45:22.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:22.390+0000 7fc9fc987640 1 -- 10.20.193.28:0/4097092342 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7fc9f8174c90 con 0x7fc9cc0b5760 2026-02-25T11:45:22.917 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2026-02-25T11:45:22.917 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: osdmap e37: 6 total, 5 up, 6 in 2026-02-25T11:45:22.917 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:22.917 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:45:22.917 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: from='osd.5 [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:45:22.917 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:22.917 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:22.917 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:22.918 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:22.918 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:22.918 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:22.918 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:22.918 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:22.918 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:23.250 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:45:22 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:45:22.917+0000 7f16918be640 -1 osd.5 0 waiting for initial osdmap 2026-02-25T11:45:23.250 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:45:22 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:45:22.923+0000 7f168d785640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T11:45:23.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2026-02-25T11:45:23.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: osdmap e37: 6 total, 5 up, 6 in 2026-02-25T11:45:23.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:23.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:45:23.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: from='osd.5 [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:45:23.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:23.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:23.252 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:23.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2026-02-25T11:45:23.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: osdmap e37: 6 total, 5 up, 6 in 2026-02-25T11:45:23.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:23.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:45:23.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: from='osd.5 [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:45:23.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:23.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:23.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:23.983 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: purged_snaps scrub starts 2026-02-25T11:45:23.983 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: purged_snaps scrub ok 2026-02-25T11:45:23.983 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:45:23.983 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: osd.default does not exist. Creating it now. 2026-02-25T11:45:23.983 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: Creating OSDs with service ID: default on trial028:['vg_nvme/lv_2'] 2026-02-25T11:45:23.983 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: Marking host: trial028 for OSDSpec preview refresh. 2026-02-25T11:45:23.983 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: Saving service osd.default spec with placement trial028 2026-02-25T11:45:23.983 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial167", "root=default"]}]': finished 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: osdmap e38: 6 total, 5 up, 6 in 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: osd.5 [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884] boot 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: osdmap e39: 6 total, 6 up, 6 in 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1640730910' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bbd9de64-b9fe-499e-b451-c16828be2c01"} : dispatch 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1640730910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbd9de64-b9fe-499e-b451-c16828be2c01"}]': finished 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: osdmap e40: 7 total, 6 up, 7 in 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:23.984 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:23 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:24.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: purged_snaps scrub starts 2026-02-25T11:45:24.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: purged_snaps scrub ok 2026-02-25T11:45:24.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:45:24.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: osd.default does not exist. Creating it now. 2026-02-25T11:45:24.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: Creating OSDs with service ID: default on trial028:['vg_nvme/lv_2'] 2026-02-25T11:45:24.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: Marking host: trial028 for OSDSpec preview refresh. 2026-02-25T11:45:24.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: Saving service osd.default spec with placement trial028 2026-02-25T11:45:24.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial167", "root=default"]}]': finished 2026-02-25T11:45:24.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: osdmap e38: 6 total, 5 up, 6 in 2026-02-25T11:45:24.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:24.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: osd.5 [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884] boot 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: osdmap e39: 6 total, 6 up, 6 in 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1640730910' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bbd9de64-b9fe-499e-b451-c16828be2c01"} : dispatch 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1640730910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbd9de64-b9fe-499e-b451-c16828be2c01"}]': finished 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: osdmap e40: 7 total, 6 up, 7 in 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:24.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:23 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:24.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: purged_snaps scrub starts 2026-02-25T11:45:24.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: purged_snaps scrub ok 2026-02-25T11:45:24.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='client.14547 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:45:24.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: osd.default does not exist. Creating it now. 2026-02-25T11:45:24.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: Creating OSDs with service ID: default on trial028:['vg_nvme/lv_2'] 2026-02-25T11:45:24.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: Marking host: trial028 for OSDSpec preview refresh. 2026-02-25T11:45:24.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: Saving service osd.default spec with placement trial028 2026-02-25T11:45:24.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.6836, "args": ["host=trial167", "root=default"]}]': finished 2026-02-25T11:45:24.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: osdmap e38: 6 total, 5 up, 6 in 2026-02-25T11:45:24.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:24.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:24.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 3.4 TiB / 3.4 TiB avail 2026-02-25T11:45:24.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: osd.5 [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884] boot 2026-02-25T11:45:24.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: osdmap e39: 6 total, 6 up, 6 in 2026-02-25T11:45:24.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2026-02-25T11:45:24.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1640730910' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "bbd9de64-b9fe-499e-b451-c16828be2c01"} : dispatch 2026-02-25T11:45:24.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1640730910' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbd9de64-b9fe-499e-b451-c16828be2c01"}]': finished 2026-02-25T11:45:24.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: osdmap e40: 7 total, 6 up, 7 in 2026-02-25T11:45:24.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:24.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:24.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:24.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:24.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:24.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:24.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:24.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:24.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:23 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:25.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:24 trial167 ceph-mon[43392]: Detected new or changed devices on trial167 2026-02-25T11:45:25.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:24 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial167 to 42026M 2026-02-25T11:45:25.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:24 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:25.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:24 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:25.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:24 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:45:25.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:24 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:25.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:24 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:25.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:24 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:25.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:24 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:25.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:24 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/82487978' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:45:25.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:24 trial167 ceph-mon[43392]: osdmap e41: 7 total, 6 up, 7 in 2026-02-25T11:45:25.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:24 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:25.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:24 trial119 ceph-mon[42710]: Detected new or changed devices on trial167 2026-02-25T11:45:25.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:24 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial167 to 42026M 2026-02-25T11:45:25.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:24 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:25.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:24 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:25.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:24 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:45:25.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:24 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:25.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:24 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:25.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:24 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:25.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:24 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:25.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:24 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/82487978' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:45:25.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:24 trial119 ceph-mon[42710]: osdmap e41: 7 total, 6 up, 7 in 2026-02-25T11:45:25.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:24 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:25.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:24 trial028 ceph-mon[39224]: Detected new or changed devices on trial167 2026-02-25T11:45:25.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:24 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial167 to 42026M 2026-02-25T11:45:25.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:25.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:25.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:45:25.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:25.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:25.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:25.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:25.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:24 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/82487978' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:45:25.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:24 trial028 ceph-mon[39224]: osdmap e41: 7 total, 6 up, 7 in 2026-02-25T11:45:25.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:24 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:26.177 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:25 trial028 ceph-mon[39224]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-25T11:45:26.177 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:25 trial028 ceph-mon[39224]: osdmap e42: 7 total, 6 up, 7 in 2026-02-25T11:45:26.177 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:25 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:26.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:25 trial167 ceph-mon[43392]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-25T11:45:26.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:25 trial167 ceph-mon[43392]: osdmap e42: 7 total, 6 up, 7 in 2026-02-25T11:45:26.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:25 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:26.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:25 trial119 ceph-mon[42710]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-25T11:45:26.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:25 trial119 ceph-mon[42710]: osdmap e42: 7 total, 6 up, 7 in 2026-02-25T11:45:26.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:25 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:27.731 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:27 trial028 ceph-mon[39224]: osdmap e43: 7 total, 6 up, 7 in 2026-02-25T11:45:27.732 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:27 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:27.732 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:27 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2026-02-25T11:45:27.732 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:27 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:27.732 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:27 trial028 ceph-mon[39224]: Deploying daemon osd.6 on trial028 2026-02-25T11:45:27.732 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:27 trial028 ceph-mon[39224]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-25T11:45:27.732 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:27 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2026-02-25T11:45:27.732 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:27 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:27.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:27 trial167 ceph-mon[43392]: osdmap e43: 7 total, 6 up, 7 in 2026-02-25T11:45:27.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:27 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:27.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:27 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2026-02-25T11:45:27.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:27 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:27.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:27 trial167 ceph-mon[43392]: Deploying daemon osd.6 on trial028 2026-02-25T11:45:27.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:27 trial167 ceph-mon[43392]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-25T11:45:27.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:27 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2026-02-25T11:45:28.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:27 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:28.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:27 trial119 ceph-mon[42710]: osdmap e43: 7 total, 6 up, 7 in 2026-02-25T11:45:28.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:27 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:28.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:27 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2026-02-25T11:45:28.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:27 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:28.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:27 trial119 ceph-mon[42710]: Deploying daemon osd.6 on trial028 2026-02-25T11:45:28.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:27 trial119 ceph-mon[42710]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-25T11:45:28.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:27 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2026-02-25T11:45:28.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:27 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:28.702 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:28 trial028 ceph-mon[39224]: Deploying daemon osd.6 on trial028 2026-02-25T11:45:28.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:28 trial167 ceph-mon[43392]: Deploying daemon osd.6 on trial028 2026-02-25T11:45:29.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:28 trial119 ceph-mon[42710]: Deploying daemon osd.6 on trial028 2026-02-25T11:45:29.851 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:29 trial028 ceph-mon[39224]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-25T11:45:29.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:29 trial167 ceph-mon[43392]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-25T11:45:30.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:29 trial119 ceph-mon[42710]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-25T11:45:31.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:31.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:31.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:31.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:31.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:31.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:32.218 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:32 trial028 ceph-mon[39224]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 89 KiB/s, 0 objects/s recovering 2026-02-25T11:45:32.219 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:32 trial028 ceph-mon[39224]: from='osd.6 [v2:10.20.193.28:6818/2507357211,v1:10.20.193.28:6819/2507357211]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2026-02-25T11:45:32.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:32.267+0000 7fc9f7fff640 1 -- 10.20.193.28:0/4097092342 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=38}) -- 0x7fc9d0003560 con 0x7fc9f810f1e0 2026-02-25T11:45:32.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:32.267+0000 7fc9d7fff640 1 -- 10.20.193.28:0/4097092342 <== mon.0 v2:10.20.193.28:3300/0 7 ==== osd_map(38..44 src has 1..44) ==== 4377+0+0 (secure 0 0 0) 0x7fc9ec0adc00 con 0x7fc9f810f1e0 2026-02-25T11:45:32.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:32 trial167 ceph-mon[43392]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 89 KiB/s, 0 objects/s recovering 2026-02-25T11:45:32.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:32 trial167 ceph-mon[43392]: from='osd.6 [v2:10.20.193.28:6818/2507357211,v1:10.20.193.28:6819/2507357211]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2026-02-25T11:45:32.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:32 trial119 ceph-mon[42710]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 89 KiB/s, 0 objects/s recovering 2026-02-25T11:45:32.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:32 trial119 ceph-mon[42710]: from='osd.6 [v2:10.20.193.28:6818/2507357211,v1:10.20.193.28:6819/2507357211]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2026-02-25T11:45:33.454 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:33 trial028 ceph-mon[39224]: from='osd.6 [v2:10.20.193.28:6818/2507357211,v1:10.20.193.28:6819/2507357211]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2026-02-25T11:45:33.454 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:33 trial028 ceph-mon[39224]: osdmap e44: 7 total, 6 up, 7 in 2026-02-25T11:45:33.455 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:33 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:33.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:33 trial167 ceph-mon[43392]: from='osd.6 [v2:10.20.193.28:6818/2507357211,v1:10.20.193.28:6819/2507357211]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2026-02-25T11:45:33.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:33 trial167 ceph-mon[43392]: osdmap e44: 7 total, 6 up, 7 in 2026-02-25T11:45:33.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:33 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:33.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:33 trial119 ceph-mon[42710]: from='osd.6 [v2:10.20.193.28:6818/2507357211,v1:10.20.193.28:6819/2507357211]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2026-02-25T11:45:33.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:33 trial119 ceph-mon[42710]: osdmap e44: 7 total, 6 up, 7 in 2026-02-25T11:45:33.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:33 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:34.260 INFO:teuthology.orchestra.run.trial028.stdout:Created osd(s) 6 on host 'trial028' 2026-02-25T11:45:34.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:34.258+0000 7fc9d7fff640 1 -- 10.20.193.28:0/4097092342 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7fc9f8174c90 con 0x7fc9cc0b5760 2026-02-25T11:45:34.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:34.261+0000 7fc9fc987640 1 -- 10.20.193.28:0/4097092342 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9cc0b5760 msgr2=0x7fc9cc0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:34.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:34.261+0000 7fc9fc987640 1 --2- 10.20.193.28:0/4097092342 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9cc0b5760 0x7fc9cc0d5c10 secure :-1 s=READY pgs=108 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9e0004740 tx=0x7fc9e000b410 comp rx=0 tx=0).stop 2026-02-25T11:45:34.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:34.261+0000 7fc9fc987640 1 -- 10.20.193.28:0/4097092342 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9f810f1e0 msgr2=0x7fc9f80f8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:34.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:34.261+0000 7fc9fc987640 1 --2- 10.20.193.28:0/4097092342 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9f810f1e0 0x7fc9f80f8970 secure :-1 s=READY pgs=164 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9ec005db0 tx=0x7fc9ec00ddf0 comp rx=0 tx=0).stop 2026-02-25T11:45:34.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:34.262+0000 7fc9fc987640 1 -- 10.20.193.28:0/4097092342 shutdown_connections 2026-02-25T11:45:34.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:34.262+0000 7fc9fc987640 1 --2- 10.20.193.28:0/4097092342 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9cc0b5760 0x7fc9cc0d5c10 unknown :-1 s=CLOSED pgs=108 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:34.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:34.262+0000 7fc9fc987640 1 --2- 10.20.193.28:0/4097092342 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc9f8110d20 0x7fc9f80f91d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:34.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:34.262+0000 7fc9fc987640 1 --2- 10.20.193.28:0/4097092342 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9f8110370 0x7fc9f81716b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:34.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:34.262+0000 7fc9fc987640 1 --2- 10.20.193.28:0/4097092342 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9f810f1e0 0x7fc9f80f8970 unknown :-1 s=CLOSED pgs=164 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:34.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:34.262+0000 7fc9fc987640 1 -- 10.20.193.28:0/4097092342 >> 10.20.193.28:0/4097092342 conn(0x7fc9f80ec950 msgr2=0x7fc9f80eefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:45:34.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:34.262+0000 7fc9fc987640 1 -- 10.20.193.28:0/4097092342 shutdown_connections 2026-02-25T11:45:34.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:45:34.262+0000 7fc9fc987640 1 -- 10.20.193.28:0/4097092342 wait complete. 2026-02-25T11:45:34.364 DEBUG:teuthology.orchestra.run.trial028:osd.6> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.6.service 2026-02-25T11:45:34.367 INFO:tasks.cephadm:Deploying osd.7 on trial119 with /dev/vg_nvme/lv_2... 2026-02-25T11:45:34.367 DEBUG:teuthology.orchestra.run.trial119:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_2 2026-02-25T11:45:34.403 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:34 trial028 ceph-mon[39224]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 77 KiB/s, 0 objects/s recovering 2026-02-25T11:45:34.403 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:34 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:34.403 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:34 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:34.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:34 trial167 ceph-mon[43392]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 77 KiB/s, 0 objects/s recovering 2026-02-25T11:45:34.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:34 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:34.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:34 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:34.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:34 trial119 ceph-mon[42710]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 77 KiB/s, 0 objects/s recovering 2026-02-25T11:45:34.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:34 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:34.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:34 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:34.554 INFO:teuthology.orchestra.run.trial119.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.b/config 2026-02-25T11:45:34.811 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:45:34 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:45:34.490+0000 7f694e4ff780 -1 osd.6 0 log_to_monitors true 2026-02-25T11:45:35.498 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:45:35.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:35 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:35.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:35 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:35.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:35 trial119 ceph-mon[42710]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2026-02-25T11:45:35.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:35 trial119 ceph-mon[42710]: from='osd.6 [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2026-02-25T11:45:35.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:35 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:35.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:35 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:35.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:35 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:45:35.522 DEBUG:teuthology.orchestra.run.trial119:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch daemon add osd trial119:vg_nvme/lv_2 --skip-validation 2026-02-25T11:45:35.533 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:35.534 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:35.534 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:35 trial028 ceph-mon[39224]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2026-02-25T11:45:35.534 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:35 trial028 ceph-mon[39224]: from='osd.6 [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2026-02-25T11:45:35.534 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:35.534 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:35.534 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:45:35.546 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:35 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:35.546 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:35 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:35.546 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:35 trial167 ceph-mon[43392]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2026-02-25T11:45:35.546 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:35 trial167 ceph-mon[43392]: from='osd.6 [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2026-02-25T11:45:35.547 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:35 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:35.547 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:35 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:35.547 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:35 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:45:35.710 INFO:teuthology.orchestra.run.trial119.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.b/config 2026-02-25T11:45:35.882 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.879+0000 7fc88da31640 1 -- 10.20.193.119:0/3972803410 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc88810ae40 msgr2=0x7fc88810b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:35.882 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.879+0000 7fc88da31640 1 --2- 10.20.193.119:0/3972803410 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc88810ae40 0x7fc88810b230 secure :-1 s=READY pgs=48 gs=1 cs=0 l=1 c_cookie=8629364482211841692 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc878009950 tx=0x7fc87802f190 comp rx=0 tx=0).stop 2026-02-25T11:45:35.882 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.879+0000 7fc88da31640 1 -- 10.20.193.119:0/3972803410 shutdown_connections 2026-02-25T11:45:35.882 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.879+0000 7fc88da31640 1 --2- 10.20.193.119:0/3972803410 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc88810c820 0x7fc88810d180 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:35.882 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.879+0000 7fc88da31640 1 --2- 10.20.193.119:0/3972803410 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc88810be70 0x7fc88810c2e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:35.882 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.879+0000 7fc88da31640 1 --2- 10.20.193.119:0/3972803410 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc88810ae40 0x7fc88810b230 unknown :-1 s=CLOSED pgs=48 gs=1 cs=0 l=1 c_cookie=8629364482211841692 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:35.882 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.879+0000 7fc88da31640 1 -- 10.20.193.119:0/3972803410 >> 10.20.193.119:0/3972803410 conn(0x7fc8880e8860 msgr2=0x7fc888108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:45:35.882 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.879+0000 7fc88da31640 1 -- 10.20.193.119:0/3972803410 shutdown_connections 2026-02-25T11:45:35.882 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.879+0000 7fc88da31640 1 -- 10.20.193.119:0/3972803410 wait complete. 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.880+0000 7fc88da31640 1 Processor -- start 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.880+0000 7fc88da31640 1 -- start start 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.880+0000 7fc88da31640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc88810ae40 0x7fc8880f3390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.880+0000 7fc88da31640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc88810be70 0x7fc8880f39e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.880+0000 7fc88da31640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc88810c820 0x7fc8880f45e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.880+0000 7fc88da31640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc8880f4d20 con 0x7fc88810c820 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.880+0000 7fc88da31640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8880f4e90 con 0x7fc88810be70 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.880+0000 7fc88da31640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8880f5050 con 0x7fc88810ae40 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.880+0000 7fc87ffff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc88810be70 0x7fc8880f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.880+0000 7fc88ca2f640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc88810ae40 0x7fc8880f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.880+0000 7fc87ffff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc88810be70 0x7fc8880f39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.119:42226/0 (socket says 10.20.193.119:42226) 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.880+0000 7fc87ffff640 1 -- 10.20.193.119:0/569528894 learned_addr learned my addr 10.20.193.119:0/569528894 (peer_addr_for_me v2:10.20.193.119:0/0) 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.880+0000 7fc88d230640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc88810c820 0x7fc8880f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.881+0000 7fc88ca2f640 1 -- 10.20.193.119:0/569528894 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc88810be70 msgr2=0x7fc8880f39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:35.883 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.881+0000 7fc88ca2f640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc88810be70 0x7fc8880f39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:35.884 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.881+0000 7fc88ca2f640 1 -- 10.20.193.119:0/569528894 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc88810c820 msgr2=0x7fc8880f45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:35.884 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.881+0000 7fc88ca2f640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc88810c820 0x7fc8880f45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:35.884 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.881+0000 7fc88ca2f640 1 -- 10.20.193.119:0/569528894 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc878009570 con 0x7fc88810ae40 2026-02-25T11:45:35.884 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.881+0000 7fc87ffff640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc88810be70 0x7fc8880f39e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:45:35.884 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.881+0000 7fc88ca2f640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc88810ae40 0x7fc8880f3390 secure :-1 s=READY pgs=45 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc878009950 tx=0x7fc8780028b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:45:35.884 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.881+0000 7fc87dffb640 1 -- 10.20.193.119:0/569528894 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc87803d070 con 0x7fc88810ae40 2026-02-25T11:45:35.884 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.881+0000 7fc87dffb640 1 -- 10.20.193.119:0/569528894 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc878005dd0 con 0x7fc88810ae40 2026-02-25T11:45:35.884 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.881+0000 7fc88da31640 1 -- 10.20.193.119:0/569528894 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8880f52d0 con 0x7fc88810ae40 2026-02-25T11:45:35.884 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.881+0000 7fc87dffb640 1 -- 10.20.193.119:0/569528894 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc878041930 con 0x7fc88810ae40 2026-02-25T11:45:35.884 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.881+0000 7fc88da31640 1 -- 10.20.193.119:0/569528894 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8880f57f0 con 0x7fc88810ae40 2026-02-25T11:45:35.884 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.882+0000 7fc88da31640 1 -- 10.20.193.119:0/569528894 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc888111070 con 0x7fc88810ae40 2026-02-25T11:45:35.887 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.884+0000 7fc87dffb640 1 -- 10.20.193.119:0/569528894 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7fc878041ac0 con 0x7fc88810ae40 2026-02-25T11:45:35.888 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.886+0000 7fc87dffb640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc8640b56e0 0x7fc8640d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:35.888 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.886+0000 7fc87dffb640 1 -- 10.20.193.119:0/569528894 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4661+0+0 (secure 0 0 0) 0x7fc87810a7d0 con 0x7fc88810ae40 2026-02-25T11:45:35.888 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.886+0000 7fc87dffb640 1 -- 10.20.193.119:0/569528894 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fc878143960 con 0x7fc88810ae40 2026-02-25T11:45:35.889 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.886+0000 7fc87ffff640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc8640b56e0 0x7fc8640d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:35.889 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:35.887+0000 7fc87ffff640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc8640b56e0 0x7fc8640d5b90 secure :-1 s=READY pgs=113 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc87000b2c0 tx=0x7fc87000b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:45:36.009 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:36.007+0000 7fc88da31640 1 -- 10.20.193.119:0/569528894 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7fc8881108f0 con 0x7fc8640b56e0 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 68 KiB/s, 0 objects/s recovering 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: osdmap e45: 7 total, 6 up, 7 in 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: from='osd.6 [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:36.352 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:36 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 68 KiB/s, 0 objects/s recovering 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: osdmap e45: 7 total, 6 up, 7 in 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: from='osd.6 [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:36.543 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:36.544 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:36 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:36.560 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:45:36 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:45:36.267+0000 7f694a481640 -1 osd.6 0 waiting for initial osdmap 2026-02-25T11:45:36.560 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:45:36 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:45:36.275+0000 7f6946348640 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T11:45:36.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail; 68 KiB/s, 0 objects/s recovering 2026-02-25T11:45:36.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2026-02-25T11:45:36.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: osdmap e45: 7 total, 6 up, 7 in 2026-02-25T11:45:36.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:36.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:45:36.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: from='osd.6 [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:45:36.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:36.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:36.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:36.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:37.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:45:37.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: osd.default does not exist. Creating it now. 2026-02-25T11:45:37.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: Creating OSDs with service ID: default on trial119:['vg_nvme/lv_2'] 2026-02-25T11:45:37.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: Marking host: trial119 for OSDSpec preview refresh. 2026-02-25T11:45:37.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: Saving service osd.default spec with placement trial119 2026-02-25T11:45:37.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-25T11:45:37.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: osdmap e46: 7 total, 6 up, 7 in 2026-02-25T11:45:37.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:37.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:37.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "546d9f28-597b-454c-8456-fcc2d46c69f3"} : dispatch 2026-02-25T11:45:37.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: from='client.? 10.20.193.119:0/1569522700' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "546d9f28-597b-454c-8456-fcc2d46c69f3"} : dispatch 2026-02-25T11:45:37.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "546d9f28-597b-454c-8456-fcc2d46c69f3"}]': finished 2026-02-25T11:45:37.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: osd.6 [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999] boot 2026-02-25T11:45:37.563 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: osdmap e47: 8 total, 7 up, 8 in 2026-02-25T11:45:37.563 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:37.563 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:37 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:37.616 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:45:37.617 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: osd.default does not exist. Creating it now. 2026-02-25T11:45:37.617 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: Creating OSDs with service ID: default on trial119:['vg_nvme/lv_2'] 2026-02-25T11:45:37.617 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: Marking host: trial119 for OSDSpec preview refresh. 2026-02-25T11:45:37.617 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: Saving service osd.default spec with placement trial119 2026-02-25T11:45:37.617 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-25T11:45:37.617 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: osdmap e46: 7 total, 6 up, 7 in 2026-02-25T11:45:37.617 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:37.617 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:37.617 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "546d9f28-597b-454c-8456-fcc2d46c69f3"} : dispatch 2026-02-25T11:45:37.617 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: from='client.? 10.20.193.119:0/1569522700' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "546d9f28-597b-454c-8456-fcc2d46c69f3"} : dispatch 2026-02-25T11:45:37.618 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "546d9f28-597b-454c-8456-fcc2d46c69f3"}]': finished 2026-02-25T11:45:37.618 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: osd.6 [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999] boot 2026-02-25T11:45:37.618 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: osdmap e47: 8 total, 7 up, 8 in 2026-02-25T11:45:37.618 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:37.618 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:37 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:37.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: from='client.24472 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:45:37.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: osd.default does not exist. Creating it now. 2026-02-25T11:45:37.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: Creating OSDs with service ID: default on trial119:['vg_nvme/lv_2'] 2026-02-25T11:45:37.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: Marking host: trial119 for OSDSpec preview refresh. 2026-02-25T11:45:37.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: Saving service osd.default spec with placement trial119 2026-02-25T11:45:37.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-25T11:45:37.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: osdmap e46: 7 total, 6 up, 7 in 2026-02-25T11:45:37.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:37.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:37.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "546d9f28-597b-454c-8456-fcc2d46c69f3"} : dispatch 2026-02-25T11:45:37.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: from='client.? 10.20.193.119:0/1569522700' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "546d9f28-597b-454c-8456-fcc2d46c69f3"} : dispatch 2026-02-25T11:45:37.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "546d9f28-597b-454c-8456-fcc2d46c69f3"}]': finished 2026-02-25T11:45:37.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: osd.6 [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999] boot 2026-02-25T11:45:37.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: osdmap e47: 8 total, 7 up, 8 in 2026-02-25T11:45:37.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2026-02-25T11:45:37.752 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:37 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:38.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: purged_snaps scrub starts 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: purged_snaps scrub ok 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='client.? 10.20.193.119:0/2309037821' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:38.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:38.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:38.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: purged_snaps scrub starts 2026-02-25T11:45:38.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: purged_snaps scrub ok 2026-02-25T11:45:38.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-25T11:45:38.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='client.? 10.20.193.119:0/2309037821' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:45:38.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:38.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:38.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:38.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:38.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:45:38.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:38.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:38.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:38.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: purged_snaps scrub starts 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: purged_snaps scrub ok 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 4.1 TiB / 4.1 TiB avail 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='client.? 10.20.193.119:0/2309037821' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:38.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:39.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:39 trial028 ceph-mon[39224]: Detected new or changed devices on trial028 2026-02-25T11:45:39.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:39 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial028 to 28017M 2026-02-25T11:45:39.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:39 trial028 ceph-mon[39224]: osdmap e48: 8 total, 7 up, 8 in 2026-02-25T11:45:39.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:39 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:39.677 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:39 trial119 ceph-mon[42710]: Detected new or changed devices on trial028 2026-02-25T11:45:39.678 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:39 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial028 to 28017M 2026-02-25T11:45:39.678 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:39 trial119 ceph-mon[42710]: osdmap e48: 8 total, 7 up, 8 in 2026-02-25T11:45:39.678 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:39 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:39.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:39 trial167 ceph-mon[43392]: Detected new or changed devices on trial028 2026-02-25T11:45:39.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:39 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial028 to 28017M 2026-02-25T11:45:39.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:39 trial167 ceph-mon[43392]: osdmap e48: 8 total, 7 up, 8 in 2026-02-25T11:45:39.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:39 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:40.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:40 trial119 ceph-mon[42710]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:40.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:40 trial028 ceph-mon[39224]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:40.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:40 trial167 ceph-mon[43392]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:41.325 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2026-02-25T11:45:41.325 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:41.325 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2026-02-25T11:45:41.325 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:41.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2026-02-25T11:45:41.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:41.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2026-02-25T11:45:41.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:41.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2026-02-25T11:45:41.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:41.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2026-02-25T11:45:41.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:42.370 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:42 trial119 ceph-mon[42710]: Deploying daemon osd.7 on trial119 2026-02-25T11:45:42.370 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:42 trial119 ceph-mon[42710]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:42.370 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:42 trial119 ceph-mon[42710]: Deploying daemon osd.7 on trial119 2026-02-25T11:45:42.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:42 trial028 ceph-mon[39224]: Deploying daemon osd.7 on trial119 2026-02-25T11:45:42.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:42 trial028 ceph-mon[39224]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:42.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:42 trial028 ceph-mon[39224]: Deploying daemon osd.7 on trial119 2026-02-25T11:45:42.598 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:42 trial167 ceph-mon[43392]: Deploying daemon osd.7 on trial119 2026-02-25T11:45:42.598 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:42 trial167 ceph-mon[43392]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:42.599 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:42 trial167 ceph-mon[43392]: Deploying daemon osd.7 on trial119 2026-02-25T11:45:44.420 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:44 trial119 ceph-mon[42710]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:44.420 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:44 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:44.420 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:44 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:44.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:44 trial028 ceph-mon[39224]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:44.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:44 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:44.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:44 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:44.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:44 trial167 ceph-mon[43392]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:44.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:44 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:44.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:44 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:45.374 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:45 trial119 ceph-mon[42710]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-25T11:45:45.375 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:45 trial119 ceph-mon[42710]: from='osd.7 [v2:10.20.193.119:6816/1460867170,v1:10.20.193.119:6817/1460867170]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-25T11:45:45.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:45 trial028 ceph-mon[39224]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-25T11:45:45.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:45 trial028 ceph-mon[39224]: from='osd.7 [v2:10.20.193.119:6816/1460867170,v1:10.20.193.119:6817/1460867170]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-25T11:45:45.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:45 trial167 ceph-mon[43392]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-25T11:45:45.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:45 trial167 ceph-mon[43392]: from='osd.7 [v2:10.20.193.119:6816/1460867170,v1:10.20.193.119:6817/1460867170]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-25T11:45:45.885 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:45.883+0000 7fc88e4b9640 1 -- 10.20.193.119:0/569528894 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=46}) -- 0x7fc860003560 con 0x7fc88810ae40 2026-02-25T11:45:45.885 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:45.883+0000 7fc87dffb640 1 -- 10.20.193.119:0/569528894 <== mon.1 v2:10.20.193.167:3300/0 7 ==== osd_map(46..49 src has 1..49) ==== 3551+0+0 (secure 0 0 0) 0x7fc8780d2360 con 0x7fc88810ae40 2026-02-25T11:45:46.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:46 trial119 ceph-mon[42710]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:46.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:46 trial119 ceph-mon[42710]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2026-02-25T11:45:46.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:46 trial119 ceph-mon[42710]: osdmap e49: 8 total, 7 up, 8 in 2026-02-25T11:45:46.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:46.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:46 trial028 ceph-mon[39224]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:46.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:46 trial028 ceph-mon[39224]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2026-02-25T11:45:46.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:46 trial028 ceph-mon[39224]: osdmap e49: 8 total, 7 up, 8 in 2026-02-25T11:45:46.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:46.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:46 trial167 ceph-mon[43392]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:46.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:46 trial167 ceph-mon[43392]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2026-02-25T11:45:46.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:46 trial167 ceph-mon[43392]: osdmap e49: 8 total, 7 up, 8 in 2026-02-25T11:45:46.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:47.898 INFO:teuthology.orchestra.run.trial119.stdout:Created osd(s) 7 on host 'trial119' 2026-02-25T11:45:47.898 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:47.896+0000 7fc87dffb640 1 -- 10.20.193.119:0/569528894 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7fc8881108f0 con 0x7fc8640b56e0 2026-02-25T11:45:47.900 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:47.899+0000 7fc88da31640 1 -- 10.20.193.119:0/569528894 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc8640b56e0 msgr2=0x7fc8640d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:47.901 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:47.899+0000 7fc88da31640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc8640b56e0 0x7fc8640d5b90 secure :-1 s=READY pgs=113 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc87000b2c0 tx=0x7fc87000b4a0 comp rx=0 tx=0).stop 2026-02-25T11:45:47.901 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:47.899+0000 7fc88da31640 1 -- 10.20.193.119:0/569528894 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc88810ae40 msgr2=0x7fc8880f3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:47.901 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:47.899+0000 7fc88da31640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc88810ae40 0x7fc8880f3390 secure :-1 s=READY pgs=45 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc878009950 tx=0x7fc8780028b0 comp rx=0 tx=0).stop 2026-02-25T11:45:47.901 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:47.899+0000 7fc88da31640 1 -- 10.20.193.119:0/569528894 shutdown_connections 2026-02-25T11:45:47.901 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:47.899+0000 7fc88da31640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc8640b56e0 0x7fc8640d5b90 unknown :-1 s=CLOSED pgs=113 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:47.901 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:47.899+0000 7fc88da31640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc88810c820 0x7fc8880f45e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:47.901 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:47.899+0000 7fc88da31640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc88810be70 0x7fc8880f39e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:47.901 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:47.899+0000 7fc88da31640 1 --2- 10.20.193.119:0/569528894 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc88810ae40 0x7fc8880f3390 unknown :-1 s=CLOSED pgs=45 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:47.901 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:47.899+0000 7fc88da31640 1 -- 10.20.193.119:0/569528894 >> 10.20.193.119:0/569528894 conn(0x7fc8880e8860 msgr2=0x7fc888112470 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:45:47.901 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:47.899+0000 7fc88da31640 1 -- 10.20.193.119:0/569528894 shutdown_connections 2026-02-25T11:45:47.901 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:45:47.899+0000 7fc88da31640 1 -- 10.20.193.119:0/569528894 wait complete. 2026-02-25T11:45:48.006 DEBUG:teuthology.orchestra.run.trial119:osd.7> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.7.service 2026-02-25T11:45:48.010 INFO:tasks.cephadm:Deploying osd.8 on trial167 with /dev/vg_nvme/lv_2... 2026-02-25T11:45:48.010 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_2 2026-02-25T11:45:48.201 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.c/config 2026-02-25T11:45:48.318 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:48 trial167 ceph-mon[43392]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:48.318 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:48.318 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:48.318 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:48.318 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:48.319 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:48 trial167 ceph-mon[43392]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-25T11:45:48.319 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:48 trial167 ceph-mon[43392]: from='osd.7 [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-25T11:45:48.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:48 trial119 ceph-mon[42710]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:48.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:48.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:48.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:48.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:48.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:48 trial119 ceph-mon[42710]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-25T11:45:48.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:48 trial119 ceph-mon[42710]: from='osd.7 [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-25T11:45:48.511 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 11:45:48 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T11:45:48.136+0000 7fb26bf7d780 -1 osd.7 0 log_to_monitors true 2026-02-25T11:45:48.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:48 trial028 ceph-mon[39224]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:48.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:48.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:48.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:48.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:48.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:48 trial028 ceph-mon[39224]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-25T11:45:48.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:48 trial028 ceph-mon[39224]: from='osd.7 [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2026-02-25T11:45:49.114 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:45:49.139 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch daemon add osd trial167:vg_nvme/lv_2 --skip-validation 2026-02-25T11:45:49.327 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.c/config 2026-02-25T11:45:49.513 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.511+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/203581115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0fa0110370 msgr2=0x7f0fa01107e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:49.514 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.511+0000 7f0fa5ffc640 1 --2- 10.20.193.167:0/203581115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0fa0110370 0x7f0fa01107e0 secure :-1 s=READY pgs=47 gs=1 cs=0 l=1 c_cookie=1230856351745815321 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0f88009950 tx=0x7f0f8802f190 comp rx=0 tx=0).stop 2026-02-25T11:45:49.514 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.511+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/203581115 shutdown_connections 2026-02-25T11:45:49.514 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.511+0000 7f0fa5ffc640 1 --2- 10.20.193.167:0/203581115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0fa0110d20 0x7f0fa01716b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:49.514 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.511+0000 7f0fa5ffc640 1 --2- 10.20.193.167:0/203581115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0fa0110370 0x7f0fa01107e0 unknown :-1 s=CLOSED pgs=47 gs=1 cs=0 l=1 c_cookie=1230856351745815321 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:49.514 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.511+0000 7f0fa5ffc640 1 --2- 10.20.193.167:0/203581115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0fa010f1e0 0x7f0fa010f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:49.514 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.511+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/203581115 >> 10.20.193.167:0/203581115 conn(0x7f0fa00ec950 msgr2=0x7f0fa010cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:45:49.514 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.511+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/203581115 shutdown_connections 2026-02-25T11:45:49.514 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.511+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/203581115 wait complete. 2026-02-25T11:45:49.514 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.512+0000 7f0fa5ffc640 1 Processor -- start 2026-02-25T11:45:49.514 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.512+0000 7f0fa5ffc640 1 -- start start 2026-02-25T11:45:49.515 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.512+0000 7f0fa5ffc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0fa010f1e0 0x7f0fa00f8be0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:49.515 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa5ffc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0fa0110370 0x7f0fa01716b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:49.515 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa5ffc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0fa0110d20 0x7f0fa00f9440 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:49.515 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa5ffc640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa00fa030 con 0x7f0fa010f1e0 2026-02-25T11:45:49.515 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa5ffc640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa00fa1a0 con 0x7f0fa0110370 2026-02-25T11:45:49.515 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa5ffc640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa00fa360 con 0x7f0fa0110d20 2026-02-25T11:45:49.515 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa57fb640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0fa0110d20 0x7f0fa00f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:49.515 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa57fb640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0fa0110d20 0x7f0fa00f9440 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.167:56366/0 (socket says 10.20.193.167:56366) 2026-02-25T11:45:49.516 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa57fb640 1 -- 10.20.193.167:0/3084529320 learned_addr learned my addr 10.20.193.167:0/3084529320 (peer_addr_for_me v2:10.20.193.167:0/0) 2026-02-25T11:45:49.516 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa4ffa640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0fa010f1e0 0x7f0fa00f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:49.516 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0f97fff640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0fa0110370 0x7f0fa01716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:49.516 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa57fb640 1 -- 10.20.193.167:0/3084529320 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0fa0110370 msgr2=0x7f0fa01716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:49.516 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa57fb640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0fa0110370 0x7f0fa01716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:49.516 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa57fb640 1 -- 10.20.193.167:0/3084529320 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0fa010f1e0 msgr2=0x7f0fa00f8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:45:49.516 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa57fb640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0fa010f1e0 0x7f0fa00f8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:45:49.516 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa57fb640 1 -- 10.20.193.167:0/3084529320 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f88009570 con 0x7f0fa0110d20 2026-02-25T11:45:49.516 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa57fb640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0fa0110d20 0x7f0fa00f9440 secure :-1 s=READY pgs=48 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0f9c00c750 tx=0x7f0f9c00cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:45:49.516 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0f95ffb640 1 -- 10.20.193.167:0/3084529320 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0f9c005500 con 0x7f0fa0110d20 2026-02-25T11:45:49.516 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/3084529320 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa00fa640 con 0x7f0fa0110d20 2026-02-25T11:45:49.516 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0f95ffb640 1 -- 10.20.193.167:0/3084529320 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0f9c003c40 con 0x7f0fa0110d20 2026-02-25T11:45:49.516 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.513+0000 7f0f95ffb640 1 -- 10.20.193.167:0/3084529320 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0f9c00dd40 con 0x7f0fa0110d20 2026-02-25T11:45:49.517 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.514+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/3084529320 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa00fab60 con 0x7f0fa0110d20 2026-02-25T11:45:49.517 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.514+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/3084529320 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa01756b0 con 0x7f0fa0110d20 2026-02-25T11:45:49.518 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.516+0000 7f0f95ffb640 1 -- 10.20.193.167:0/3084529320 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f0f9c003780 con 0x7f0fa0110d20 2026-02-25T11:45:49.520 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.518+0000 7f0f95ffb640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0f7c0b5760 0x7f0f7c0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:45:49.521 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.518+0000 7f0f95ffb640 1 -- 10.20.193.167:0/3084529320 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5093+0+0 (secure 0 0 0) 0x7f0f9c0e6340 con 0x7f0fa0110d20 2026-02-25T11:45:49.521 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.518+0000 7f0f95ffb640 1 -- 10.20.193.167:0/3084529320 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f0f9c0e67c0 con 0x7f0fa0110d20 2026-02-25T11:45:49.521 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.518+0000 7f0fa4ffa640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0f7c0b5760 0x7f0f7c0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:45:49.521 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.519+0000 7f0fa4ffa640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0f7c0b5760 0x7f0f7c0d5c10 secure :-1 s=READY pgs=118 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0f90006fd0 tx=0x7f0f90008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:45:49.640 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:49.637+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/3084529320 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7f0fa0174d50 con 0x7f0f7c0b5760 2026-02-25T11:45:49.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:45:49.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2026-02-25T11:45:49.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: osdmap e50: 8 total, 7 up, 8 in 2026-02-25T11:45:49.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:49.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:45:49.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='osd.7 [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:45:49.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:49.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:49.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:49.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:49.916 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 11:45:49 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T11:45:49.906+0000 7fb267eff640 -1 osd.7 0 waiting for initial osdmap 2026-02-25T11:45:49.916 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.916 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.916 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:45:49.917 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2026-02-25T11:45:49.917 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: osdmap e50: 8 total, 7 up, 8 in 2026-02-25T11:45:49.917 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:49.917 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:45:49.917 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='osd.7 [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:45:49.917 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:49.917 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.917 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.917 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.917 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.917 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.918 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.918 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:49.918 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:49.918 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:49.921 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.921 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.921 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:45:49.921 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2026-02-25T11:45:49.921 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: osdmap e50: 8 total, 7 up, 8 in 2026-02-25T11:45:49.921 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:49.922 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:45:49.922 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='osd.7 [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:45:49.922 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 4.8 TiB / 4.8 TiB avail 2026-02-25T11:45:49.922 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.922 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.922 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.922 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.922 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.922 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:49.922 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:49.923 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:49.923 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:50.259 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 11:45:49 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T11:45:49.913+0000 7fb25fdbe640 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T11:45:51.216 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: from='client.24505 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:45:51.217 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: osd.default does not exist. Creating it now. 2026-02-25T11:45:51.217 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: Creating OSDs with service ID: default on trial167:['vg_nvme/lv_2'] 2026-02-25T11:45:51.217 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: Marking host: trial167 for OSDSpec preview refresh. 2026-02-25T11:45:51.217 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: Saving service osd.default spec with placement trial167 2026-02-25T11:45:51.217 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial119", "root=default"]}]': finished 2026-02-25T11:45:51.217 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: osdmap e51: 8 total, 7 up, 8 in 2026-02-25T11:45:51.217 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:51.217 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:51.218 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a1a7cea6-3586-4bb2-aa4c-bf6f7799a501"} : dispatch 2026-02-25T11:45:51.218 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: from='client.? 10.20.193.167:0/925226283' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a1a7cea6-3586-4bb2-aa4c-bf6f7799a501"} : dispatch 2026-02-25T11:45:51.218 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1a7cea6-3586-4bb2-aa4c-bf6f7799a501"}]': finished 2026-02-25T11:45:51.218 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: osd.7 [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747] boot 2026-02-25T11:45:51.218 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: osdmap e52: 9 total, 8 up, 9 in 2026-02-25T11:45:51.218 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:51.218 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:51.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: from='client.24505 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:45:51.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: osd.default does not exist. Creating it now. 2026-02-25T11:45:51.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: Creating OSDs with service ID: default on trial167:['vg_nvme/lv_2'] 2026-02-25T11:45:51.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: Marking host: trial167 for OSDSpec preview refresh. 2026-02-25T11:45:51.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: Saving service osd.default spec with placement trial167 2026-02-25T11:45:51.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial119", "root=default"]}]': finished 2026-02-25T11:45:51.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: osdmap e51: 8 total, 7 up, 8 in 2026-02-25T11:45:51.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:51.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:51.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a1a7cea6-3586-4bb2-aa4c-bf6f7799a501"} : dispatch 2026-02-25T11:45:51.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: from='client.? 10.20.193.167:0/925226283' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a1a7cea6-3586-4bb2-aa4c-bf6f7799a501"} : dispatch 2026-02-25T11:45:51.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1a7cea6-3586-4bb2-aa4c-bf6f7799a501"}]': finished 2026-02-25T11:45:51.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: osd.7 [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747] boot 2026-02-25T11:45:51.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: osdmap e52: 9 total, 8 up, 9 in 2026-02-25T11:45:51.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:51.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:51.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: from='client.24505 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_2", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:45:51.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: osd.default does not exist. Creating it now. 2026-02-25T11:45:51.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: Creating OSDs with service ID: default on trial167:['vg_nvme/lv_2'] 2026-02-25T11:45:51.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: Marking host: trial167 for OSDSpec preview refresh. 2026-02-25T11:45:51.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: Saving service osd.default spec with placement trial167 2026-02-25T11:45:51.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.6836, "args": ["host=trial119", "root=default"]}]': finished 2026-02-25T11:45:51.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: osdmap e51: 8 total, 7 up, 8 in 2026-02-25T11:45:51.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:51.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:51.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a1a7cea6-3586-4bb2-aa4c-bf6f7799a501"} : dispatch 2026-02-25T11:45:51.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: from='client.? 10.20.193.167:0/925226283' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a1a7cea6-3586-4bb2-aa4c-bf6f7799a501"} : dispatch 2026-02-25T11:45:51.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1a7cea6-3586-4bb2-aa4c-bf6f7799a501"}]': finished 2026-02-25T11:45:51.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: osd.7 [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747] boot 2026-02-25T11:45:51.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: osdmap e52: 9 total, 8 up, 9 in 2026-02-25T11:45:51.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2026-02-25T11:45:51.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:52.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: purged_snaps scrub starts 2026-02-25T11:45:52.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: purged_snaps scrub ok 2026-02-25T11:45:52.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: Detected new or changed devices on trial119 2026-02-25T11:45:52.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:52.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:52.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:52.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial119 to 28017M 2026-02-25T11:45:52.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:45:52.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:52.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:45:52.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:52.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:52.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:52.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='client.? 10.20.193.167:0/2685949046' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:45:52.313 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: osdmap e53: 9 total, 8 up, 9 in 2026-02-25T11:45:52.313 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:52.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: purged_snaps scrub starts 2026-02-25T11:45:52.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: purged_snaps scrub ok 2026-02-25T11:45:52.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: Detected new or changed devices on trial119 2026-02-25T11:45:52.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:52.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:52.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:52.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial119 to 28017M 2026-02-25T11:45:52.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:45:52.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:52.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:45:52.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:52.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:52.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:52.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='client.? 10.20.193.167:0/2685949046' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:45:52.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: osdmap e53: 9 total, 8 up, 9 in 2026-02-25T11:45:52.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: purged_snaps scrub starts 2026-02-25T11:45:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: purged_snaps scrub ok 2026-02-25T11:45:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: Detected new or changed devices on trial119 2026-02-25T11:45:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-25T11:45:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial119 to 28017M 2026-02-25T11:45:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:45:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:45:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:52.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:45:52.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:45:52.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:52.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='client.? 10.20.193.167:0/2685949046' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:45:52.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: osdmap e53: 9 total, 8 up, 9 in 2026-02-25T11:45:52.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:53.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:53 trial167 ceph-mon[43392]: osdmap e54: 9 total, 8 up, 9 in 2026-02-25T11:45:53.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:53.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:53 trial167 ceph-mon[43392]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:45:53.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:53 trial167 ceph-mon[43392]: osdmap e55: 9 total, 8 up, 9 in 2026-02-25T11:45:53.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:54.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:53 trial119 ceph-mon[42710]: osdmap e54: 9 total, 8 up, 9 in 2026-02-25T11:45:54.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:54.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:53 trial119 ceph-mon[42710]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:45:54.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:53 trial119 ceph-mon[42710]: osdmap e55: 9 total, 8 up, 9 in 2026-02-25T11:45:54.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:54.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:53 trial028 ceph-mon[39224]: osdmap e54: 9 total, 8 up, 9 in 2026-02-25T11:45:54.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:54.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:53 trial028 ceph-mon[39224]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:45:54.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:53 trial028 ceph-mon[39224]: osdmap e55: 9 total, 8 up, 9 in 2026-02-25T11:45:54.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:55.151 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:54 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2026-02-25T11:45:55.151 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:54 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:55.151 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:54 trial167 ceph-mon[43392]: Deploying daemon osd.8 on trial167 2026-02-25T11:45:55.151 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:54 trial167 ceph-mon[43392]: osdmap e56: 9 total, 8 up, 9 in 2026-02-25T11:45:55.152 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:54 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:55.152 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:54 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2026-02-25T11:45:55.152 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:54 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:55.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:54 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2026-02-25T11:45:55.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:54 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:55.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:54 trial119 ceph-mon[42710]: Deploying daemon osd.8 on trial167 2026-02-25T11:45:55.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:54 trial119 ceph-mon[42710]: osdmap e56: 9 total, 8 up, 9 in 2026-02-25T11:45:55.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:54 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:55.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:54 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2026-02-25T11:45:55.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:54 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:55.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:54 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2026-02-25T11:45:55.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:54 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:55.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:54 trial028 ceph-mon[39224]: Deploying daemon osd.8 on trial167 2026-02-25T11:45:55.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:54 trial028 ceph-mon[39224]: osdmap e56: 9 total, 8 up, 9 in 2026-02-25T11:45:55.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:54 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:45:55.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:54 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2026-02-25T11:45:55.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:54 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:45:56.186 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:55 trial167 ceph-mon[43392]: Deploying daemon osd.8 on trial167 2026-02-25T11:45:56.187 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:55 trial167 ceph-mon[43392]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:45:56.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:55 trial119 ceph-mon[42710]: Deploying daemon osd.8 on trial167 2026-02-25T11:45:56.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:55 trial119 ceph-mon[42710]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:45:56.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:55 trial028 ceph-mon[39224]: Deploying daemon osd.8 on trial167 2026-02-25T11:45:56.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:55 trial028 ceph-mon[39224]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:45:58.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:58 trial167 ceph-mon[43392]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:45:58.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:58 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:58.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:58 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:58.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:58 trial028 ceph-mon[39224]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:45:58.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:58 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:58.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:58 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:58.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:58 trial119 ceph-mon[42710]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:45:58.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:58 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:58.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:58 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:45:59.413 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:59 trial167 ceph-mon[43392]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-25T11:45:59.414 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:45:59 trial167 ceph-mon[43392]: from='osd.8 [v2:10.20.193.167:6816/1547003380,v1:10.20.193.167:6817/1547003380]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-25T11:45:59.519 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:59.516+0000 7f0fa6a84640 1 -- 10.20.193.167:0/3084529320 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=51}) -- 0x7f0f78003560 con 0x7f0fa0110d20 2026-02-25T11:45:59.519 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:45:59.516+0000 7f0f95ffb640 1 -- 10.20.193.167:0/3084529320 <== mon.1 v2:10.20.193.167:3300/0 7 ==== osd_map(51..57 src has 1..57) ==== 4525+0+0 (secure 0 0 0) 0x7f0f9c003a60 con 0x7f0fa0110d20 2026-02-25T11:45:59.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:59 trial028 ceph-mon[39224]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-25T11:45:59.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:45:59 trial028 ceph-mon[39224]: from='osd.8 [v2:10.20.193.167:6816/1547003380,v1:10.20.193.167:6817/1547003380]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-25T11:45:59.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:59 trial119 ceph-mon[42710]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-25T11:45:59.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:45:59 trial119 ceph-mon[42710]: from='osd.8 [v2:10.20.193.167:6816/1547003380,v1:10.20.193.167:6817/1547003380]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-25T11:46:00.389 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:00 trial167 ceph-mon[43392]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:46:00.390 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:00 trial167 ceph-mon[43392]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2026-02-25T11:46:00.390 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:00 trial167 ceph-mon[43392]: osdmap e57: 9 total, 8 up, 9 in 2026-02-25T11:46:00.390 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:00.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:00 trial028 ceph-mon[39224]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:46:00.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:00 trial028 ceph-mon[39224]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2026-02-25T11:46:00.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:00 trial028 ceph-mon[39224]: osdmap e57: 9 total, 8 up, 9 in 2026-02-25T11:46:00.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:00.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:00 trial119 ceph-mon[42710]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:46:00.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:00 trial119 ceph-mon[42710]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2026-02-25T11:46:00.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:00 trial119 ceph-mon[42710]: osdmap e57: 9 total, 8 up, 9 in 2026-02-25T11:46:00.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:01.450 INFO:teuthology.orchestra.run.trial167.stdout:Created osd(s) 8 on host 'trial167' 2026-02-25T11:46:01.451 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:01.448+0000 7f0f95ffb640 1 -- 10.20.193.167:0/3084529320 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f0fa0174d50 con 0x7f0f7c0b5760 2026-02-25T11:46:01.453 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:01.451+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/3084529320 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0f7c0b5760 msgr2=0x7f0f7c0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:01.454 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:01.451+0000 7f0fa5ffc640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0f7c0b5760 0x7f0f7c0d5c10 secure :-1 s=READY pgs=118 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0f90006fd0 tx=0x7f0f90008040 comp rx=0 tx=0).stop 2026-02-25T11:46:01.454 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:01.451+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/3084529320 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0fa0110d20 msgr2=0x7f0fa00f9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:01.454 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:01.451+0000 7f0fa5ffc640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0fa0110d20 0x7f0fa00f9440 secure :-1 s=READY pgs=48 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0f9c00c750 tx=0x7f0f9c00cc20 comp rx=0 tx=0).stop 2026-02-25T11:46:01.454 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:01.451+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/3084529320 shutdown_connections 2026-02-25T11:46:01.454 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:01.451+0000 7f0fa5ffc640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0f7c0b5760 0x7f0f7c0d5c10 unknown :-1 s=CLOSED pgs=118 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:01.454 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:01.451+0000 7f0fa5ffc640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0fa0110d20 0x7f0fa00f9440 unknown :-1 s=CLOSED pgs=48 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:01.454 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:01.451+0000 7f0fa5ffc640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0fa0110370 0x7f0fa01716b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:01.454 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:01.451+0000 7f0fa5ffc640 1 --2- 10.20.193.167:0/3084529320 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0fa010f1e0 0x7f0fa00f8be0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:01.454 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:01.451+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/3084529320 >> 10.20.193.167:0/3084529320 conn(0x7f0fa00ec950 msgr2=0x7f0fa00ef180 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:01.454 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:01.451+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/3084529320 shutdown_connections 2026-02-25T11:46:01.455 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:01.451+0000 7f0fa5ffc640 1 -- 10.20.193.167:0/3084529320 wait complete. 2026-02-25T11:46:01.597 DEBUG:teuthology.orchestra.run.trial167:osd.8> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.8.service 2026-02-25T11:46:01.600 INFO:tasks.cephadm:Deploying osd.9 on trial028 with /dev/vg_nvme/lv_1... 2026-02-25T11:46:01.600 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_1 2026-02-25T11:46:01.790 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:46:01.920 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:01.920 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:01.920 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:01 trial028 ceph-mon[39224]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:46:01.920 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:01.920 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:01.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:01.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:01.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:01 trial167 ceph-mon[43392]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:46:01.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:01.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:02.000 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:46:01 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:46:01.762+0000 7f22f2536780 -1 osd.8 0 log_to_monitors true 2026-02-25T11:46:02.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:02.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:02.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:01 trial119 ceph-mon[42710]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:46:02.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:02.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:02.713 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:46:02.735 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch daemon add osd trial028:vg_nvme/lv_1 --skip-validation 2026-02-25T11:46:02.786 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:02 trial028 ceph-mon[39224]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-25T11:46:02.787 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:02 trial028 ceph-mon[39224]: from='osd.8 [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-25T11:46:02.787 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:02 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:02.787 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:02 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:02.787 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:02 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:02.922 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:46:02.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:02 trial167 ceph-mon[43392]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-25T11:46:02.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:02 trial167 ceph-mon[43392]: from='osd.8 [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-25T11:46:03.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:02 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:03.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:02 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:03.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:02 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:03.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:02 trial119 ceph-mon[42710]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-25T11:46:03.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:02 trial119 ceph-mon[42710]: from='osd.8 [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2026-02-25T11:46:03.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:02 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:03.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:02 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:03.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:02 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:03.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.113+0000 7f457902c640 1 -- 10.20.193.28:0/636551385 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4574110d20 msgr2=0x7f45741716b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:03.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.113+0000 7f457902c640 1 --2- 10.20.193.28:0/636551385 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4574110d20 0x7f45741716b0 secure :-1 s=READY pgs=174 gs=1 cs=0 l=1 c_cookie=3642935943159650861 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f456800aa30 tx=0x7f4568030270 comp rx=0 tx=0).stop 2026-02-25T11:46:03.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.113+0000 7f457902c640 1 -- 10.20.193.28:0/636551385 shutdown_connections 2026-02-25T11:46:03.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.113+0000 7f457902c640 1 --2- 10.20.193.28:0/636551385 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4574110d20 0x7f45741716b0 unknown :-1 s=CLOSED pgs=174 gs=1 cs=0 l=1 c_cookie=3642935943159650861 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:03.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.113+0000 7f457902c640 1 --2- 10.20.193.28:0/636551385 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4574110370 0x7f45741107e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:03.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.113+0000 7f457902c640 1 --2- 10.20.193.28:0/636551385 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f457410f1e0 0x7f457410f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:03.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.113+0000 7f457902c640 1 -- 10.20.193.28:0/636551385 >> 10.20.193.28:0/636551385 conn(0x7f45740ec950 msgr2=0x7f457410cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:03.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.114+0000 7f457902c640 1 -- 10.20.193.28:0/636551385 shutdown_connections 2026-02-25T11:46:03.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.114+0000 7f457902c640 1 -- 10.20.193.28:0/636551385 wait complete. 2026-02-25T11:46:03.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.114+0000 7f457902c640 1 Processor -- start 2026-02-25T11:46:03.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.114+0000 7f457902c640 1 -- start start 2026-02-25T11:46:03.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457902c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f457410f1e0 0x7f45740f89a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:03.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457902c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4574110370 0x7f4574171690 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:03.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457902c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4574110d20 0x7f45740f91e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:03.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457902c640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f45740f9e40 con 0x7f4574110d20 2026-02-25T11:46:03.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457902c640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f45740f9fb0 con 0x7f457410f1e0 2026-02-25T11:46:03.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457902c640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f45740fa170 con 0x7f4574110370 2026-02-25T11:46:03.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457882b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4574110d20 0x7f45740f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:03.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457882b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4574110d20 0x7f45740f91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38598/0 (socket says 10.20.193.28:38598) 2026-02-25T11:46:03.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457882b640 1 -- 10.20.193.28:0/3818537600 learned_addr learned my addr 10.20.193.28:0/3818537600 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:46:03.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f4573fff640 1 --2- 10.20.193.28:0/3818537600 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f457410f1e0 0x7f45740f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:03.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457882b640 1 -- 10.20.193.28:0/3818537600 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4574110370 msgr2=0x7f4574171690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:03.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457882b640 1 --2- 10.20.193.28:0/3818537600 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4574110370 0x7f4574171690 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:03.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457882b640 1 -- 10.20.193.28:0/3818537600 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f457410f1e0 msgr2=0x7f45740f89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:03.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457882b640 1 --2- 10.20.193.28:0/3818537600 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f457410f1e0 0x7f45740f89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:03.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457882b640 1 -- 10.20.193.28:0/3818537600 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456800a650 con 0x7f4574110d20 2026-02-25T11:46:03.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.115+0000 7f457882b640 1 --2- 10.20.193.28:0/3818537600 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4574110d20 0x7f45740f91e0 secure :-1 s=READY pgs=175 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f456800a980 tx=0x7f45680039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:03.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.116+0000 7f45717fa640 1 -- 10.20.193.28:0/3818537600 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f456803e070 con 0x7f4574110d20 2026-02-25T11:46:03.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.116+0000 7f45717fa640 1 -- 10.20.193.28:0/3818537600 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4568003aa0 con 0x7f4574110d20 2026-02-25T11:46:03.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.116+0000 7f45717fa640 1 -- 10.20.193.28:0/3818537600 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4568042a60 con 0x7f4574110d20 2026-02-25T11:46:03.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.116+0000 7f457902c640 1 -- 10.20.193.28:0/3818537600 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45740fa3f0 con 0x7f4574110d20 2026-02-25T11:46:03.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.116+0000 7f457902c640 1 -- 10.20.193.28:0/3818537600 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45740fa810 con 0x7f4574110d20 2026-02-25T11:46:03.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.116+0000 7f457902c640 1 -- 10.20.193.28:0/3818537600 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45741755d0 con 0x7f4574110d20 2026-02-25T11:46:03.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.118+0000 7f45717fa640 1 -- 10.20.193.28:0/3818537600 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f45680050c0 con 0x7f4574110d20 2026-02-25T11:46:03.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.120+0000 7f45717fa640 1 --2- 10.20.193.28:0/3818537600 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f45480b5760 0x7f45480d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:03.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.120+0000 7f45717fa640 1 -- 10.20.193.28:0/3818537600 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5525+0+0 (secure 0 0 0) 0x7f456810c620 con 0x7f4574110d20 2026-02-25T11:46:03.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.120+0000 7f4573fff640 1 --2- 10.20.193.28:0/3818537600 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f45480b5760 0x7f45480d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:03.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.120+0000 7f45717fa640 1 -- 10.20.193.28:0/3818537600 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f45680cfa60 con 0x7f4574110d20 2026-02-25T11:46:03.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.121+0000 7f4573fff640 1 --2- 10.20.193.28:0/3818537600 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f45480b5760 0x7f45480d5c10 secure :-1 s=READY pgs=123 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4560005970 tx=0x7f4560005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:03.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:03.240+0000 7f457902c640 1 -- 10.20.193.28:0/3818537600 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7f4574174c90 con 0x7f45480b5760 2026-02-25T11:46:03.778 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2026-02-25T11:46:03.779 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: osdmap e58: 9 total, 8 up, 9 in 2026-02-25T11:46:03.779 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:03.779 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:46:03.779 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='osd.8 [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:46:03.779 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:46:03.779 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:03.779 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:03.779 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:03.779 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:03.779 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='client.14643 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:46:03.780 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: osd.default does not exist. Creating it now. 2026-02-25T11:46:03.780 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: Creating OSDs with service ID: default on trial028:['vg_nvme/lv_1'] 2026-02-25T11:46:03.780 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:03.780 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: Marking host: trial028 for OSDSpec preview refresh. 2026-02-25T11:46:03.780 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: Saving service osd.default spec with placement trial028 2026-02-25T11:46:03.780 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:03.780 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:03.780 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:03.780 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:04.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2026-02-25T11:46:04.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: osdmap e58: 9 total, 8 up, 9 in 2026-02-25T11:46:04.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:04.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:46:04.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='osd.8 [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:46:04.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:46:04.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='client.14643 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:46:04.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: osd.default does not exist. Creating it now. 2026-02-25T11:46:04.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: Creating OSDs with service ID: default on trial028:['vg_nvme/lv_1'] 2026-02-25T11:46:04.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: Marking host: trial028 for OSDSpec preview refresh. 2026-02-25T11:46:04.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: Saving service osd.default spec with placement trial028 2026-02-25T11:46:04.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:04.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:04.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:04.002 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:46:03 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:46:03.707+0000 7f22ee4b8640 -1 osd.8 0 waiting for initial osdmap 2026-02-25T11:46:04.002 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:46:03 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:46:03.716+0000 7f22ea37f640 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T11:46:04.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2026-02-25T11:46:04.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: osdmap e58: 9 total, 8 up, 9 in 2026-02-25T11:46:04.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:04.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:46:04.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='osd.8 [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:46:04.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 5.5 TiB / 5.5 TiB avail 2026-02-25T11:46:04.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='client.14643 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial028:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:46:04.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: osd.default does not exist. Creating it now. 2026-02-25T11:46:04.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: Creating OSDs with service ID: default on trial028:['vg_nvme/lv_1'] 2026-02-25T11:46:04.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: Marking host: trial028 for OSDSpec preview refresh. 2026-02-25T11:46:04.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: Saving service osd.default spec with placement trial028 2026-02-25T11:46:04.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:04.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:04.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:04.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial167", "root=default"]}]': finished 2026-02-25T11:46:04.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: osdmap e59: 9 total, 8 up, 9 in 2026-02-25T11:46:04.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:04.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:04.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2985029322' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03fef134-6946-43d5-9c26-dc0e59fffe51"} : dispatch 2026-02-25T11:46:04.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2985029322' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03fef134-6946-43d5-9c26-dc0e59fffe51"}]': finished 2026-02-25T11:46:04.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: osd.8 [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317] boot 2026-02-25T11:46:04.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: osdmap e60: 10 total, 9 up, 10 in 2026-02-25T11:46:04.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:04.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:04.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:04.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:04.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:04.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:04.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:04.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:04.813 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:04.813 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:04.813 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:05.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial167", "root=default"]}]': finished 2026-02-25T11:46:05.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: osdmap e59: 9 total, 8 up, 9 in 2026-02-25T11:46:05.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:05.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:05.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2985029322' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03fef134-6946-43d5-9c26-dc0e59fffe51"} : dispatch 2026-02-25T11:46:05.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2985029322' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03fef134-6946-43d5-9c26-dc0e59fffe51"}]': finished 2026-02-25T11:46:05.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: osd.8 [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317] boot 2026-02-25T11:46:05.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: osdmap e60: 10 total, 9 up, 10 in 2026-02-25T11:46:05.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:05.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:05.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:05.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:05.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:05.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:05.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:05.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.002 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:05.003 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:05.003 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:05.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.6836, "args": ["host=trial167", "root=default"]}]': finished 2026-02-25T11:46:05.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: osdmap e59: 9 total, 8 up, 9 in 2026-02-25T11:46:05.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:05.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:05.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2985029322' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03fef134-6946-43d5-9c26-dc0e59fffe51"} : dispatch 2026-02-25T11:46:05.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2985029322' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03fef134-6946-43d5-9c26-dc0e59fffe51"}]': finished 2026-02-25T11:46:05.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: osd.8 [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317] boot 2026-02-25T11:46:05.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: osdmap e60: 10 total, 9 up, 10 in 2026-02-25T11:46:05.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2026-02-25T11:46:05.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:05.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:05.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:05.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:05.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:05.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:05.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:05.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:05.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:05.012 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:06.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:05 trial167 ceph-mon[43392]: purged_snaps scrub starts 2026-02-25T11:46:06.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:05 trial167 ceph-mon[43392]: purged_snaps scrub ok 2026-02-25T11:46:06.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:05 trial167 ceph-mon[43392]: Detected new or changed devices on trial167 2026-02-25T11:46:06.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:05 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial167 to 28017M 2026-02-25T11:46:06.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:05 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/711720846' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:46:06.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:05 trial167 ceph-mon[43392]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-25T11:46:06.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:05 trial167 ceph-mon[43392]: osdmap e61: 10 total, 9 up, 10 in 2026-02-25T11:46:06.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:05 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:06.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:05 trial119 ceph-mon[42710]: purged_snaps scrub starts 2026-02-25T11:46:06.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:05 trial119 ceph-mon[42710]: purged_snaps scrub ok 2026-02-25T11:46:06.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:05 trial119 ceph-mon[42710]: Detected new or changed devices on trial167 2026-02-25T11:46:06.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:05 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial167 to 28017M 2026-02-25T11:46:06.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:05 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/711720846' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:46:06.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:05 trial119 ceph-mon[42710]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-25T11:46:06.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:05 trial119 ceph-mon[42710]: osdmap e61: 10 total, 9 up, 10 in 2026-02-25T11:46:06.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:05 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:06.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:05 trial028 ceph-mon[39224]: purged_snaps scrub starts 2026-02-25T11:46:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:05 trial028 ceph-mon[39224]: purged_snaps scrub ok 2026-02-25T11:46:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:05 trial028 ceph-mon[39224]: Detected new or changed devices on trial167 2026-02-25T11:46:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:05 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial167 to 28017M 2026-02-25T11:46:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:05 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/711720846' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:46:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:05 trial028 ceph-mon[39224]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-25T11:46:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:05 trial028 ceph-mon[39224]: osdmap e61: 10 total, 9 up, 10 in 2026-02-25T11:46:06.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:05 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:07.597 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:07 trial028 ceph-mon[39224]: osdmap e62: 10 total, 9 up, 10 in 2026-02-25T11:46:07.597 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:07 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:07.597 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:07 trial028 ceph-mon[39224]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-25T11:46:07.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:07 trial167 ceph-mon[43392]: osdmap e62: 10 total, 9 up, 10 in 2026-02-25T11:46:07.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:07 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:07.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:07 trial167 ceph-mon[43392]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-25T11:46:07.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:07 trial119 ceph-mon[42710]: osdmap e62: 10 total, 9 up, 10 in 2026-02-25T11:46:07.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:07 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:07.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:07 trial119 ceph-mon[42710]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-25T11:46:08.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:08 trial167 ceph-mon[43392]: osdmap e63: 10 total, 9 up, 10 in 2026-02-25T11:46:08.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:08.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2026-02-25T11:46:08.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:08.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:08 trial167 ceph-mon[43392]: Deploying daemon osd.9 on trial028 2026-02-25T11:46:08.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2026-02-25T11:46:08.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:08.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:08 trial119 ceph-mon[42710]: osdmap e63: 10 total, 9 up, 10 in 2026-02-25T11:46:08.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:08.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2026-02-25T11:46:08.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:08.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:08 trial119 ceph-mon[42710]: Deploying daemon osd.9 on trial028 2026-02-25T11:46:08.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2026-02-25T11:46:08.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:08.799 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:08 trial028 ceph-mon[39224]: osdmap e63: 10 total, 9 up, 10 in 2026-02-25T11:46:08.799 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:08.799 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2026-02-25T11:46:08.799 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:08.799 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:08 trial028 ceph-mon[39224]: Deploying daemon osd.9 on trial028 2026-02-25T11:46:08.799 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2026-02-25T11:46:08.799 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:09.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:09 trial167 ceph-mon[43392]: Deploying daemon osd.9 on trial028 2026-02-25T11:46:09.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:09 trial167 ceph-mon[43392]: osdmap e64: 10 total, 9 up, 10 in 2026-02-25T11:46:09.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:09.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:09 trial167 ceph-mon[43392]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-25T11:46:09.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:09 trial119 ceph-mon[42710]: Deploying daemon osd.9 on trial028 2026-02-25T11:46:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:09 trial119 ceph-mon[42710]: osdmap e64: 10 total, 9 up, 10 in 2026-02-25T11:46:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:09 trial119 ceph-mon[42710]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-25T11:46:09.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:09 trial028 ceph-mon[39224]: Deploying daemon osd.9 on trial028 2026-02-25T11:46:09.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:09 trial028 ceph-mon[39224]: osdmap e64: 10 total, 9 up, 10 in 2026-02-25T11:46:09.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:09.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:09 trial028 ceph-mon[39224]: pgmap v140: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail 2026-02-25T11:46:12.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:12.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:12.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:11 trial028 ceph-mon[39224]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail; 103 KiB/s, 0 objects/s recovering 2026-02-25T11:46:12.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:12.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:12.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:11 trial167 ceph-mon[43392]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail; 103 KiB/s, 0 objects/s recovering 2026-02-25T11:46:12.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:12.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:12.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:11 trial119 ceph-mon[42710]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail; 103 KiB/s, 0 objects/s recovering 2026-02-25T11:46:13.025 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:12 trial028 ceph-mon[39224]: from='osd.9 [v2:10.20.193.28:6826/1989630537,v1:10.20.193.28:6827/1989630537]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2026-02-25T11:46:13.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:13.118+0000 7f4579ab4640 1 -- 10.20.193.28:0/3818537600 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=59}) -- 0x7f4554003560 con 0x7f4574110d20 2026-02-25T11:46:13.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:13.118+0000 7f45717fa640 1 -- 10.20.193.28:0/3818537600 <== mon.0 v2:10.20.193.28:3300/0 7 ==== osd_map(59..65 src has 1..65) ==== 4599+0+0 (secure 0 0 0) 0x7f45680d3e10 con 0x7f4574110d20 2026-02-25T11:46:13.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:12 trial167 ceph-mon[43392]: from='osd.9 [v2:10.20.193.28:6826/1989630537,v1:10.20.193.28:6827/1989630537]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2026-02-25T11:46:13.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:12 trial119 ceph-mon[42710]: from='osd.9 [v2:10.20.193.28:6826/1989630537,v1:10.20.193.28:6827/1989630537]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2026-02-25T11:46:13.985 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:13 trial028 ceph-mon[39224]: from='osd.9 [v2:10.20.193.28:6826/1989630537,v1:10.20.193.28:6827/1989630537]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2026-02-25T11:46:13.985 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:13 trial028 ceph-mon[39224]: osdmap e65: 10 total, 9 up, 10 in 2026-02-25T11:46:13.986 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:13.986 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:13 trial028 ceph-mon[39224]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail; 96 KiB/s, 0 objects/s recovering 2026-02-25T11:46:14.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:13 trial167 ceph-mon[43392]: from='osd.9 [v2:10.20.193.28:6826/1989630537,v1:10.20.193.28:6827/1989630537]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2026-02-25T11:46:14.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:13 trial167 ceph-mon[43392]: osdmap e65: 10 total, 9 up, 10 in 2026-02-25T11:46:14.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:14.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:13 trial167 ceph-mon[43392]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail; 96 KiB/s, 0 objects/s recovering 2026-02-25T11:46:14.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:13 trial119 ceph-mon[42710]: from='osd.9 [v2:10.20.193.28:6826/1989630537,v1:10.20.193.28:6827/1989630537]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2026-02-25T11:46:14.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:13 trial119 ceph-mon[42710]: osdmap e65: 10 total, 9 up, 10 in 2026-02-25T11:46:14.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:14.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:13 trial119 ceph-mon[42710]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail; 96 KiB/s, 0 objects/s recovering 2026-02-25T11:46:15.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:15.030+0000 7f45717fa640 1 -- 10.20.193.28:0/3818537600 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+36 (secure 0 0 0) 0x7f4574174c90 con 0x7f45480b5760 2026-02-25T11:46:15.033 INFO:teuthology.orchestra.run.trial028.stdout:Created osd(s) 9 on host 'trial028' 2026-02-25T11:46:15.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:15.032+0000 7f457902c640 1 -- 10.20.193.28:0/3818537600 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f45480b5760 msgr2=0x7f45480d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:15.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:15.032+0000 7f457902c640 1 --2- 10.20.193.28:0/3818537600 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f45480b5760 0x7f45480d5c10 secure :-1 s=READY pgs=123 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4560005970 tx=0x7f4560005900 comp rx=0 tx=0).stop 2026-02-25T11:46:15.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:15.032+0000 7f457902c640 1 -- 10.20.193.28:0/3818537600 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4574110d20 msgr2=0x7f45740f91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:15.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:15.032+0000 7f457902c640 1 --2- 10.20.193.28:0/3818537600 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4574110d20 0x7f45740f91e0 secure :-1 s=READY pgs=175 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f456800a980 tx=0x7f45680039b0 comp rx=0 tx=0).stop 2026-02-25T11:46:15.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:15.033+0000 7f457902c640 1 -- 10.20.193.28:0/3818537600 shutdown_connections 2026-02-25T11:46:15.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:15.033+0000 7f457902c640 1 --2- 10.20.193.28:0/3818537600 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f45480b5760 0x7f45480d5c10 unknown :-1 s=CLOSED pgs=123 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:15.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:15.033+0000 7f457902c640 1 --2- 10.20.193.28:0/3818537600 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4574110d20 0x7f45740f91e0 unknown :-1 s=CLOSED pgs=175 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:15.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:15.033+0000 7f457902c640 1 --2- 10.20.193.28:0/3818537600 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4574110370 0x7f4574171690 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:15.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:15.033+0000 7f457902c640 1 --2- 10.20.193.28:0/3818537600 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f457410f1e0 0x7f45740f89a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:15.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:15.033+0000 7f457902c640 1 -- 10.20.193.28:0/3818537600 >> 10.20.193.28:0/3818537600 conn(0x7f45740ec950 msgr2=0x7f45740eefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:15.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:15.033+0000 7f457902c640 1 -- 10.20.193.28:0/3818537600 shutdown_connections 2026-02-25T11:46:15.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:15.033+0000 7f457902c640 1 -- 10.20.193.28:0/3818537600 wait complete. 2026-02-25T11:46:15.140 DEBUG:teuthology.orchestra.run.trial028:osd.9> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.9.service 2026-02-25T11:46:15.143 INFO:tasks.cephadm:Deploying osd.10 on trial119 with /dev/vg_nvme/lv_1... 2026-02-25T11:46:15.144 DEBUG:teuthology.orchestra.run.trial119:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_1 2026-02-25T11:46:15.333 INFO:teuthology.orchestra.run.trial119.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.b/config 2026-02-25T11:46:15.466 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:15.466 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:15.466 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:15.466 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:15.466 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:15 trial119 ceph-mon[42710]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail; 76 KiB/s, 0 objects/s recovering 2026-02-25T11:46:15.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:15.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:15.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:15.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:15.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:15 trial028 ceph-mon[39224]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail; 76 KiB/s, 0 objects/s recovering 2026-02-25T11:46:15.562 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:46:15 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:46:15.331+0000 7f00c5741780 -1 osd.9 0 log_to_monitors true 2026-02-25T11:46:15.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:15.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:15.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:15.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:15.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:15 trial167 ceph-mon[43392]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail; 76 KiB/s, 0 objects/s recovering 2026-02-25T11:46:16.276 INFO:teuthology.orchestra.run.trial119.stdout: 2026-02-25T11:46:16.303 DEBUG:teuthology.orchestra.run.trial119:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch daemon add osd trial119:vg_nvme/lv_1 --skip-validation 2026-02-25T11:46:16.488 INFO:teuthology.orchestra.run.trial119.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.b/config 2026-02-25T11:46:16.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:16 trial167 ceph-mon[43392]: from='osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2026-02-25T11:46:16.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:16.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:16.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:16.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:16 trial028 ceph-mon[39224]: from='osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2026-02-25T11:46:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:16.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:16.575 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:16 trial119 ceph-mon[42710]: from='osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2026-02-25T11:46:16.575 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:16.575 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:16.576 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:16.670 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.666+0000 7f2639c83640 1 -- 10.20.193.119:0/801663584 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2634110370 msgr2=0x7f26341107e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:16.670 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.666+0000 7f2639c83640 1 --2- 10.20.193.119:0/801663584 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2634110370 0x7f26341107e0 secure :-1 s=READY pgs=57 gs=1 cs=0 l=1 c_cookie=9445658363490765054 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f261c009950 tx=0x7f261c02f190 comp rx=0 tx=0).stop 2026-02-25T11:46:16.670 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.667+0000 7f2639c83640 1 -- 10.20.193.119:0/801663584 shutdown_connections 2026-02-25T11:46:16.670 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.667+0000 7f2639c83640 1 --2- 10.20.193.119:0/801663584 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2634110d20 0x7f26341716b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:16.670 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.667+0000 7f2639c83640 1 --2- 10.20.193.119:0/801663584 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2634110370 0x7f26341107e0 unknown :-1 s=CLOSED pgs=57 gs=1 cs=0 l=1 c_cookie=9445658363490765054 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:16.670 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.667+0000 7f2639c83640 1 --2- 10.20.193.119:0/801663584 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f263410f1e0 0x7f263410f5d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:16.671 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.667+0000 7f2639c83640 1 -- 10.20.193.119:0/801663584 >> 10.20.193.119:0/801663584 conn(0x7f26340ec950 msgr2=0x7f263410cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:16.671 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.667+0000 7f2639c83640 1 -- 10.20.193.119:0/801663584 shutdown_connections 2026-02-25T11:46:16.671 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.667+0000 7f2639c83640 1 -- 10.20.193.119:0/801663584 wait complete. 2026-02-25T11:46:16.672 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.667+0000 7f2639c83640 1 Processor -- start 2026-02-25T11:46:16.672 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2639c83640 1 -- start start 2026-02-25T11:46:16.673 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2639c83640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f263410f1e0 0x7f26340f89a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:16.673 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2639c83640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2634110370 0x7f26341716b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:16.673 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2639c83640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2634110d20 0x7f26340f9200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:16.673 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2639c83640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f26340f9df0 con 0x7f2634110370 2026-02-25T11:46:16.673 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2639c83640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26340f9f60 con 0x7f263410f1e0 2026-02-25T11:46:16.673 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2639c83640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26340fa120 con 0x7f2634110d20 2026-02-25T11:46:16.673 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2638c81640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f263410f1e0 0x7f26340f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:16.673 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2638c81640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f263410f1e0 0x7f26340f89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.119:45510/0 (socket says 10.20.193.119:45510) 2026-02-25T11:46:16.673 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2638c81640 1 -- 10.20.193.119:0/1985863529 learned_addr learned my addr 10.20.193.119:0/1985863529 (peer_addr_for_me v2:10.20.193.119:0/0) 2026-02-25T11:46:16.673 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f262bfff640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2634110370 0x7f26341716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:16.673 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2639482640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2634110d20 0x7f26340f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:16.674 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2638c81640 1 -- 10.20.193.119:0/1985863529 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2634110d20 msgr2=0x7f26340f9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:16.674 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2638c81640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2634110d20 0x7f26340f9200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:16.674 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2638c81640 1 -- 10.20.193.119:0/1985863529 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2634110370 msgr2=0x7f26341716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:16.674 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2638c81640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2634110370 0x7f26341716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:16.674 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.668+0000 7f2638c81640 1 -- 10.20.193.119:0/1985863529 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261c009570 con 0x7f263410f1e0 2026-02-25T11:46:16.674 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.669+0000 7f2638c81640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f263410f1e0 0x7f26340f89a0 secure :-1 s=READY pgs=58 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26240072a0 tx=0x7f262400ddf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:16.674 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.669+0000 7f2629ffb640 1 -- 10.20.193.119:0/1985863529 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2624004500 con 0x7f263410f1e0 2026-02-25T11:46:16.674 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.669+0000 7f2629ffb640 1 -- 10.20.193.119:0/1985863529 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f262400bd00 con 0x7f263410f1e0 2026-02-25T11:46:16.674 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.669+0000 7f2629ffb640 1 -- 10.20.193.119:0/1985863529 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f26240106e0 con 0x7f263410f1e0 2026-02-25T11:46:16.674 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.669+0000 7f2639c83640 1 -- 10.20.193.119:0/1985863529 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26340fa400 con 0x7f263410f1e0 2026-02-25T11:46:16.674 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.669+0000 7f2639c83640 1 -- 10.20.193.119:0/1985863529 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26340fa880 con 0x7f263410f1e0 2026-02-25T11:46:16.675 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.669+0000 7f2639c83640 1 -- 10.20.193.119:0/1985863529 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2634175630 con 0x7f263410f1e0 2026-02-25T11:46:16.675 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.671+0000 7f2629ffb640 1 -- 10.20.193.119:0/1985863529 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f262400b840 con 0x7f263410f1e0 2026-02-25T11:46:16.676 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.673+0000 7f2629ffb640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f26100b5760 0x7f26100d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:16.676 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.673+0000 7f2629ffb640 1 -- 10.20.193.119:0/1985863529 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5957+0+0 (secure 0 0 0) 0x7f26240e5cb0 con 0x7f263410f1e0 2026-02-25T11:46:16.676 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.673+0000 7f262bfff640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f26100b5760 0x7f26100d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:16.676 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.674+0000 7f262bfff640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f26100b5760 0x7f26100d5c10 secure :-1 s=READY pgs=128 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f261c0023d0 tx=0x7f261c031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:16.677 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.674+0000 7f2629ffb640 1 -- 10.20.193.119:0/1985863529 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f26240a8f40 con 0x7f263410f1e0 2026-02-25T11:46:16.798 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:16.795+0000 7f2639c83640 1 -- 10.20.193.119:0/1985863529 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7f2634174cd0 con 0x7f26100b5760 2026-02-25T11:46:17.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: from='osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2026-02-25T11:46:17.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: osdmap e66: 10 total, 9 up, 10 in 2026-02-25T11:46:17.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: from='osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:46:17.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:17.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: from='client.24563 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:46:17.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: osd.default does not exist. Creating it now. 2026-02-25T11:46:17.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: Creating OSDs with service ID: default on trial119:['vg_nvme/lv_1'] 2026-02-25T11:46:17.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: Marking host: trial119 for OSDSpec preview refresh. 2026-02-25T11:46:17.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: Saving service osd.default spec with placement trial119 2026-02-25T11:46:17.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.502 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:17.503 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:17.503 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:17.503 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:17 trial167 ceph-mon[43392]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail; 72 KiB/s, 0 objects/s recovering 2026-02-25T11:46:17.561 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:46:17 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:46:17.333+0000 7f00c16c3640 -1 osd.9 0 waiting for initial osdmap 2026-02-25T11:46:17.561 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:46:17 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:46:17.342+0000 7f00bd58a640 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T11:46:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: from='osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2026-02-25T11:46:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: osdmap e66: 10 total, 9 up, 10 in 2026-02-25T11:46:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: from='osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:46:17.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:17.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: from='client.24563 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:46:17.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: osd.default does not exist. Creating it now. 2026-02-25T11:46:17.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: Creating OSDs with service ID: default on trial119:['vg_nvme/lv_1'] 2026-02-25T11:46:17.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: Marking host: trial119 for OSDSpec preview refresh. 2026-02-25T11:46:17.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: Saving service osd.default spec with placement trial119 2026-02-25T11:46:17.563 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.563 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:17.563 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:17.563 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:17.563 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:17 trial028 ceph-mon[39224]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail; 72 KiB/s, 0 objects/s recovering 2026-02-25T11:46:17.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: from='osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2026-02-25T11:46:17.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: osdmap e66: 10 total, 9 up, 10 in 2026-02-25T11:46:17.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: from='osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.6836, "args": ["host=trial028", "root=default"]} : dispatch 2026-02-25T11:46:17.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:17.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: from='client.24563 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial119:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:46:17.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: osd.default does not exist. Creating it now. 2026-02-25T11:46:17.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: Creating OSDs with service ID: default on trial119:['vg_nvme/lv_1'] 2026-02-25T11:46:17.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: Marking host: trial119 for OSDSpec preview refresh. 2026-02-25T11:46:17.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: Saving service osd.default spec with placement trial119 2026-02-25T11:46:17.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:17.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:17.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:17.762 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:17.762 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:17 trial119 ceph-mon[42710]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 6.2 TiB / 6.2 TiB avail; 72 KiB/s, 0 objects/s recovering 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: osdmap e67: 10 total, 9 up, 10 in 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b46c29ef-285c-4184-be28-d99b76bc0e1d"} : dispatch 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='client.? 10.20.193.119:0/3023477954' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b46c29ef-285c-4184-be28-d99b76bc0e1d"} : dispatch 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b46c29ef-285c-4184-be28-d99b76bc0e1d"}]': finished 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625] boot 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: osdmap e68: 11 total, 10 up, 11 in 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: Detected new or changed devices on trial028 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:18.335 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:18.407 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-25T11:46:18.407 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: osdmap e67: 10 total, 9 up, 10 in 2026-02-25T11:46:18.407 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:18.407 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:18.408 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b46c29ef-285c-4184-be28-d99b76bc0e1d"} : dispatch 2026-02-25T11:46:18.408 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='client.? 10.20.193.119:0/3023477954' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b46c29ef-285c-4184-be28-d99b76bc0e1d"} : dispatch 2026-02-25T11:46:18.408 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b46c29ef-285c-4184-be28-d99b76bc0e1d"}]': finished 2026-02-25T11:46:18.408 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625] boot 2026-02-25T11:46:18.408 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: osdmap e68: 11 total, 10 up, 11 in 2026-02-25T11:46:18.408 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:18.408 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:18.408 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: Detected new or changed devices on trial028 2026-02-25T11:46:18.408 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.408 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.408 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.409 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.409 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:18.409 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:18.409 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:18.409 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:18.409 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial028 to 21013M 2026-02-25T11:46:18.409 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.409 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.409 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.409 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:18.409 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:18.410 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.410 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:18.410 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:18.410 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial028 to 21013M 2026-02-25T11:46:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:18.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.6836, "args": ["host=trial028", "root=default"]}]': finished 2026-02-25T11:46:18.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: osdmap e67: 10 total, 9 up, 10 in 2026-02-25T11:46:18.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:18.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:18.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b46c29ef-285c-4184-be28-d99b76bc0e1d"} : dispatch 2026-02-25T11:46:18.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='client.? 10.20.193.119:0/3023477954' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b46c29ef-285c-4184-be28-d99b76bc0e1d"} : dispatch 2026-02-25T11:46:18.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b46c29ef-285c-4184-be28-d99b76bc0e1d"}]': finished 2026-02-25T11:46:18.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: osd.9 [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625] boot 2026-02-25T11:46:18.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: osdmap e68: 11 total, 10 up, 11 in 2026-02-25T11:46:18.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2026-02-25T11:46:18.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:18.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: Detected new or changed devices on trial028 2026-02-25T11:46:18.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:18.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:18.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:18.812 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:18.813 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial028 to 21013M 2026-02-25T11:46:18.813 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.813 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.813 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.813 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:18.813 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:18.813 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:18.813 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:18.813 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:18.813 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:19.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:19 trial167 ceph-mon[43392]: purged_snaps scrub starts 2026-02-25T11:46:19.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:19 trial167 ceph-mon[43392]: purged_snaps scrub ok 2026-02-25T11:46:19.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:19 trial167 ceph-mon[43392]: from='client.? 10.20.193.119:0/2359132256' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:46:19.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:19 trial167 ceph-mon[43392]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:19.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:19 trial119 ceph-mon[42710]: purged_snaps scrub starts 2026-02-25T11:46:19.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:19 trial119 ceph-mon[42710]: purged_snaps scrub ok 2026-02-25T11:46:19.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:19 trial119 ceph-mon[42710]: from='client.? 10.20.193.119:0/2359132256' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:46:19.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:19 trial119 ceph-mon[42710]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:19.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:19 trial028 ceph-mon[39224]: purged_snaps scrub starts 2026-02-25T11:46:19.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:19 trial028 ceph-mon[39224]: purged_snaps scrub ok 2026-02-25T11:46:19.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:19 trial028 ceph-mon[39224]: from='client.? 10.20.193.119:0/2359132256' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:46:19.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:19 trial028 ceph-mon[39224]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:20.602 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:20 trial119 ceph-mon[42710]: osdmap e69: 11 total, 10 up, 11 in 2026-02-25T11:46:20.602 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:20 trial167 ceph-mon[43392]: osdmap e69: 11 total, 10 up, 11 in 2026-02-25T11:46:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:20.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:20 trial028 ceph-mon[39224]: osdmap e69: 11 total, 10 up, 11 in 2026-02-25T11:46:20.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:21.411 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:21 trial119 ceph-mon[42710]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:21.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:21 trial167 ceph-mon[43392]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:21.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:21 trial028 ceph-mon[39224]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:22.482 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2026-02-25T11:46:22.482 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:22.483 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:22 trial119 ceph-mon[42710]: Deploying daemon osd.10 on trial119 2026-02-25T11:46:22.483 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2026-02-25T11:46:22.483 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:22 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:22.483 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:22 trial119 ceph-mon[42710]: Deploying daemon osd.10 on trial119 2026-02-25T11:46:22.651 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2026-02-25T11:46:22.651 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:22.651 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:22 trial167 ceph-mon[43392]: Deploying daemon osd.10 on trial119 2026-02-25T11:46:22.651 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2026-02-25T11:46:22.651 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:22 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:22.651 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:22 trial167 ceph-mon[43392]: Deploying daemon osd.10 on trial119 2026-02-25T11:46:22.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2026-02-25T11:46:22.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:22.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:22 trial028 ceph-mon[39224]: Deploying daemon osd.10 on trial119 2026-02-25T11:46:22.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2026-02-25T11:46:22.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:22 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:22.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:22 trial028 ceph-mon[39224]: Deploying daemon osd.10 on trial119 2026-02-25T11:46:23.594 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:23 trial119 ceph-mon[42710]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:23.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:23 trial167 ceph-mon[43392]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:23.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:23 trial028 ceph-mon[39224]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:25.787 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:25 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:25.787 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:25 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:25.787 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:25 trial119 ceph-mon[42710]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:25.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:25 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:25.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:25 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:25.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:25 trial028 ceph-mon[39224]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:25.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:25 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:25.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:25 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:25.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:25 trial167 ceph-mon[43392]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:26.646 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:26 trial119 ceph-mon[42710]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-25T11:46:26.646 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:26 trial119 ceph-mon[42710]: from='osd.10 [v2:10.20.193.119:6824/2829916271,v1:10.20.193.119:6825/2829916271]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-25T11:46:26.674 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:26.671+0000 7f263a70b640 1 -- 10.20.193.119:0/1985863529 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=67}) -- 0x7f260c003560 con 0x7f263410f1e0 2026-02-25T11:46:26.674 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:26.671+0000 7f2629ffb640 1 -- 10.20.193.119:0/1985863529 <== mon.2 v2:10.20.193.119:3300/0 7 ==== osd_map(67..70 src has 1..70) ==== 3774+0+0 (secure 0 0 0) 0x7f2624002e30 con 0x7f263410f1e0 2026-02-25T11:46:26.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:26 trial028 ceph-mon[39224]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-25T11:46:26.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:26 trial028 ceph-mon[39224]: from='osd.10 [v2:10.20.193.119:6824/2829916271,v1:10.20.193.119:6825/2829916271]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-25T11:46:26.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:26 trial167 ceph-mon[43392]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-25T11:46:26.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:26 trial167 ceph-mon[43392]: from='osd.10 [v2:10.20.193.119:6824/2829916271,v1:10.20.193.119:6825/2829916271]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-25T11:46:27.566 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:27 trial119 ceph-mon[42710]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2026-02-25T11:46:27.566 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:27 trial119 ceph-mon[42710]: osdmap e70: 11 total, 10 up, 11 in 2026-02-25T11:46:27.566 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:27 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:27.566 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:27 trial119 ceph-mon[42710]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:27.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:27 trial028 ceph-mon[39224]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2026-02-25T11:46:27.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:27 trial028 ceph-mon[39224]: osdmap e70: 11 total, 10 up, 11 in 2026-02-25T11:46:27.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:27 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:27.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:27 trial028 ceph-mon[39224]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:27.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:27 trial167 ceph-mon[43392]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2026-02-25T11:46:28.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:27 trial167 ceph-mon[43392]: osdmap e70: 11 total, 10 up, 11 in 2026-02-25T11:46:28.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:27 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:28.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:27 trial167 ceph-mon[43392]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:28.658 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:28.656+0000 7f2629ffb640 1 -- 10.20.193.119:0/1985863529 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2634174cd0 con 0x7f26100b5760 2026-02-25T11:46:28.659 INFO:teuthology.orchestra.run.trial119.stdout:Created osd(s) 10 on host 'trial119' 2026-02-25T11:46:28.661 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:28.659+0000 7f2639c83640 1 -- 10.20.193.119:0/1985863529 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f26100b5760 msgr2=0x7f26100d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:28.662 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:28.659+0000 7f2639c83640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f26100b5760 0x7f26100d5c10 secure :-1 s=READY pgs=128 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f261c0023d0 tx=0x7f261c031040 comp rx=0 tx=0).stop 2026-02-25T11:46:28.662 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:28.659+0000 7f2639c83640 1 -- 10.20.193.119:0/1985863529 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f263410f1e0 msgr2=0x7f26340f89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:28.662 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:28.659+0000 7f2639c83640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f263410f1e0 0x7f26340f89a0 secure :-1 s=READY pgs=58 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26240072a0 tx=0x7f262400ddf0 comp rx=0 tx=0).stop 2026-02-25T11:46:28.662 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:28.659+0000 7f2639c83640 1 -- 10.20.193.119:0/1985863529 shutdown_connections 2026-02-25T11:46:28.662 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:28.659+0000 7f2639c83640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f26100b5760 0x7f26100d5c10 unknown :-1 s=CLOSED pgs=128 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:28.662 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:28.659+0000 7f2639c83640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2634110d20 0x7f26340f9200 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:28.662 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:28.659+0000 7f2639c83640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2634110370 0x7f26341716b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:28.662 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:28.659+0000 7f2639c83640 1 --2- 10.20.193.119:0/1985863529 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f263410f1e0 0x7f26340f89a0 unknown :-1 s=CLOSED pgs=58 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:28.662 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:28.659+0000 7f2639c83640 1 -- 10.20.193.119:0/1985863529 >> 10.20.193.119:0/1985863529 conn(0x7f26340ec950 msgr2=0x7f26340ef060 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:28.662 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:28.660+0000 7f2639c83640 1 -- 10.20.193.119:0/1985863529 shutdown_connections 2026-02-25T11:46:28.663 INFO:teuthology.orchestra.run.trial119.stderr:2026-02-25T11:46:28.660+0000 7f2639c83640 1 -- 10.20.193.119:0/1985863529 wait complete. 2026-02-25T11:46:28.765 DEBUG:teuthology.orchestra.run.trial119:osd.10> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.10.service 2026-02-25T11:46:28.767 INFO:tasks.cephadm:Deploying osd.11 on trial167 with /dev/vg_nvme/lv_1... 2026-02-25T11:46:28.768 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- lvm zap /dev/vg_nvme/lv_1 2026-02-25T11:46:28.954 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.c/config 2026-02-25T11:46:29.087 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.088 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.088 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.088 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:28 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.219 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.219 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.219 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.219 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:28 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.219 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:46:28 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:46:28.913+0000 7f1c5753c780 -1 osd.10 0 log_to_monitors true 2026-02-25T11:46:29.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:28 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.900 INFO:teuthology.orchestra.run.trial167.stdout: 2026-02-25T11:46:29.924 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch daemon add osd trial167:vg_nvme/lv_1 --skip-validation 2026-02-25T11:46:29.975 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:29 trial167 ceph-mon[43392]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-25T11:46:29.975 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:29 trial167 ceph-mon[43392]: from='osd.10 [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-25T11:46:29.975 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:29 trial167 ceph-mon[43392]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:29.975 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:29 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.975 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:29 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:29.975 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:29 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:30.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:29 trial028 ceph-mon[39224]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-25T11:46:30.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:29 trial028 ceph-mon[39224]: from='osd.10 [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-25T11:46:30.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:29 trial028 ceph-mon[39224]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:30.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:29 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:30.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:29 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:30.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:29 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:30.109 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.c/config 2026-02-25T11:46:30.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:29 trial119 ceph-mon[42710]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-25T11:46:30.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:29 trial119 ceph-mon[42710]: from='osd.10 [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2026-02-25T11:46:30.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:29 trial119 ceph-mon[42710]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:30.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:29 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:30.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:29 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:30.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:29 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:30.292 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.287+0000 7f1157fff640 1 -- 10.20.193.167:0/63098235 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1158062ac0 msgr2=0x7f1158061150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:30.292 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.287+0000 7f1157fff640 1 --2- 10.20.193.167:0/63098235 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1158062ac0 0x7f1158061150 secure :-1 s=READY pgs=59 gs=1 cs=0 l=1 c_cookie=14644896928159247130 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f114c009950 tx=0x7f114c02f190 comp rx=0 tx=0).stop 2026-02-25T11:46:30.293 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.288+0000 7f1157fff640 1 -- 10.20.193.167:0/63098235 shutdown_connections 2026-02-25T11:46:30.293 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.288+0000 7f1157fff640 1 --2- 10.20.193.167:0/63098235 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f115810ca80 0x7f115810cf20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:30.293 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.288+0000 7f1157fff640 1 --2- 10.20.193.167:0/63098235 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1158061690 0x7f1158061b00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:30.293 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.288+0000 7f1157fff640 1 --2- 10.20.193.167:0/63098235 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1158062ac0 0x7f1158061150 unknown :-1 s=CLOSED pgs=59 gs=1 cs=0 l=1 c_cookie=14644896928159247130 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:30.293 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.288+0000 7f1157fff640 1 -- 10.20.193.167:0/63098235 >> 10.20.193.167:0/63098235 conn(0x7f11580ea6a0 msgr2=0x7f115810aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:30.293 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.288+0000 7f1157fff640 1 -- 10.20.193.167:0/63098235 shutdown_connections 2026-02-25T11:46:30.293 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.288+0000 7f1157fff640 1 -- 10.20.193.167:0/63098235 wait complete. 2026-02-25T11:46:30.293 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.288+0000 7f1157fff640 1 Processor -- start 2026-02-25T11:46:30.293 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.288+0000 7f1157fff640 1 -- start start 2026-02-25T11:46:30.294 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.288+0000 7f1157fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1158061690 0x7f11580f61e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:30.294 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.288+0000 7f1157fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1158062ac0 0x7f11580f6830 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:30.294 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1157fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f115810ca80 0x7f11580f7430 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:30.294 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1157fff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f11580f7b70 con 0x7f1158062ac0 2026-02-25T11:46:30.294 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1157fff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f11580f7ce0 con 0x7f115810ca80 2026-02-25T11:46:30.294 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1157fff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f11580f7ea0 con 0x7f1158061690 2026-02-25T11:46:30.294 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1156ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1158061690 0x7f11580f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:30.294 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1156ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1158061690 0x7f11580f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.167:50252/0 (socket says 10.20.193.167:50252) 2026-02-25T11:46:30.294 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1156ffd640 1 -- 10.20.193.167:0/1403339743 learned_addr learned my addr 10.20.193.167:0/1403339743 (peer_addr_for_me v2:10.20.193.167:0/0) 2026-02-25T11:46:30.294 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f11577fe640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f115810ca80 0x7f11580f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:30.295 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f11567fc640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1158062ac0 0x7f11580f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:30.295 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1156ffd640 1 -- 10.20.193.167:0/1403339743 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f115810ca80 msgr2=0x7f11580f7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:30.295 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1156ffd640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f115810ca80 0x7f11580f7430 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:30.295 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1156ffd640 1 -- 10.20.193.167:0/1403339743 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1158062ac0 msgr2=0x7f11580f6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:30.295 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1156ffd640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1158062ac0 0x7f11580f6830 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:30.295 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1156ffd640 1 -- 10.20.193.167:0/1403339743 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c009570 con 0x7f1158061690 2026-02-25T11:46:30.295 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1156ffd640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1158061690 0x7f11580f61e0 secure :-1 s=READY pgs=60 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f114c0098a0 tx=0x7f114c0028b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:30.295 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1137fff640 1 -- 10.20.193.167:0/1403339743 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f114c03d070 con 0x7f1158061690 2026-02-25T11:46:30.295 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1137fff640 1 -- 10.20.193.167:0/1403339743 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f114c002a50 con 0x7f1158061690 2026-02-25T11:46:30.295 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.289+0000 7f1157fff640 1 -- 10.20.193.167:0/1403339743 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11580f8120 con 0x7f1158061690 2026-02-25T11:46:30.295 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.290+0000 7f1137fff640 1 -- 10.20.193.167:0/1403339743 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f114c0418a0 con 0x7f1158061690 2026-02-25T11:46:30.296 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.290+0000 7f1157fff640 1 -- 10.20.193.167:0/1403339743 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11580f8640 con 0x7f1158061690 2026-02-25T11:46:30.296 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.290+0000 7f1157fff640 1 -- 10.20.193.167:0/1403339743 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1158179ae0 con 0x7f1158061690 2026-02-25T11:46:30.297 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.292+0000 7f1137fff640 1 -- 10.20.193.167:0/1403339743 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f114c038bf0 con 0x7f1158061690 2026-02-25T11:46:30.297 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.293+0000 7f1137fff640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f112c0b5760 0x7f112c0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:30.297 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.293+0000 7f1137fff640 1 -- 10.20.193.167:0/1403339743 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6390+0+0 (secure 0 0 0) 0x7f114c10aa60 con 0x7f1158061690 2026-02-25T11:46:30.297 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.293+0000 7f11567fc640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f112c0b5760 0x7f112c0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:30.297 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.294+0000 7f11567fc640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f112c0b5760 0x7f112c0d5c10 secure :-1 s=READY pgs=133 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1140004740 tx=0x7f1140005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:30.298 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.294+0000 7f1137fff640 1 -- 10.20.193.167:0/1403339743 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f114c0cdb40 con 0x7f1158061690 2026-02-25T11:46:30.420 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:30.418+0000 7f1157fff640 1 -- 10.20.193.167:0/1403339743 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}) -- 0x7f11581107b0 con 0x7f112c0b5760 2026-02-25T11:46:30.941 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2026-02-25T11:46:30.941 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: osdmap e71: 11 total, 10 up, 11 in 2026-02-25T11:46:30.941 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:30.941 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:46:30.941 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: from='osd.10 [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:46:30.941 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:30.941 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:30.942 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:30.942 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:30.942 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:30.942 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:30.942 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:30.942 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:30.942 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:30 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:31.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2026-02-25T11:46:31.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: osdmap e71: 11 total, 10 up, 11 in 2026-02-25T11:46:31.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:31.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:46:31.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: from='osd.10 [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:46:31.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:31.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:31.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:30 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:31.260 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:46:30 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:46:30.853+0000 7f1c534be640 -1 osd.10 0 waiting for initial osdmap 2026-02-25T11:46:31.260 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:46:30 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:46:30.862+0000 7f1c4b37d640 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T11:46:31.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2026-02-25T11:46:31.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: osdmap e71: 11 total, 10 up, 11 in 2026-02-25T11:46:31.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:31.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:46:31.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: from='osd.10 [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial119", "root=default"]} : dispatch 2026-02-25T11:46:31.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:31.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:31.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:30 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='client.24598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: osd.default does not exist. Creating it now. 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: Creating OSDs with service ID: default on trial167:['vg_nvme/lv_1'] 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: Marking host: trial167 for OSDSpec preview refresh. 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: Saving service osd.default spec with placement trial167 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial119", "root=default"]}]': finished 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: osdmap e72: 11 total, 10 up, 11 in 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f6696d8c-5de7-4fd9-b693-524f204e4e11"} : dispatch 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='client.? 10.20.193.167:0/2321886611' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f6696d8c-5de7-4fd9-b693-524f204e4e11"} : dispatch 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6696d8c-5de7-4fd9-b693-524f204e4e11"}]': finished 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: osd.10 [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116] boot 2026-02-25T11:46:31.987 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: osdmap e73: 12 total, 11 up, 12 in 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:31.988 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:31 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='client.24598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: osd.default does not exist. Creating it now. 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: Creating OSDs with service ID: default on trial167:['vg_nvme/lv_1'] 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: Marking host: trial167 for OSDSpec preview refresh. 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: Saving service osd.default spec with placement trial167 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial119", "root=default"]}]': finished 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: osdmap e72: 11 total, 10 up, 11 in 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f6696d8c-5de7-4fd9-b693-524f204e4e11"} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='client.? 10.20.193.167:0/2321886611' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f6696d8c-5de7-4fd9-b693-524f204e4e11"} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6696d8c-5de7-4fd9-b693-524f204e4e11"}]': finished 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: osd.10 [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116] boot 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: osdmap e73: 12 total, 11 up, 12 in 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:32.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:32.011 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:31 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:32.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='client.24598 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "trial167:vg_nvme/lv_1", "skip_validation": true, "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:46:32.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: osd.default does not exist. Creating it now. 2026-02-25T11:46:32.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: Creating OSDs with service ID: default on trial167:['vg_nvme/lv_1'] 2026-02-25T11:46:32.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: Marking host: trial167 for OSDSpec preview refresh. 2026-02-25T11:46:32.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: Saving service osd.default spec with placement trial167 2026-02-25T11:46:32.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.6836, "args": ["host=trial119", "root=default"]}]': finished 2026-02-25T11:46:32.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: osdmap e72: 11 total, 10 up, 11 in 2026-02-25T11:46:32.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:32.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:32.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 273 MiB used, 6.8 TiB / 6.8 TiB avail 2026-02-25T11:46:32.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f6696d8c-5de7-4fd9-b693-524f204e4e11"} : dispatch 2026-02-25T11:46:32.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='client.? 10.20.193.167:0/2321886611' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f6696d8c-5de7-4fd9-b693-524f204e4e11"} : dispatch 2026-02-25T11:46:32.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f6696d8c-5de7-4fd9-b693-524f204e4e11"}]': finished 2026-02-25T11:46:32.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: osd.10 [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116] boot 2026-02-25T11:46:32.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: osdmap e73: 12 total, 11 up, 12 in 2026-02-25T11:46:32.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2026-02-25T11:46:32.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:32.312 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.313 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.313 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.313 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.313 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.313 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.313 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:32.313 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:32.313 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:32.313 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:32.313 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.313 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:32.314 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:32.314 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:32.314 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:32.314 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:32.314 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:31 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:33.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:32 trial167 ceph-mon[43392]: purged_snaps scrub starts 2026-02-25T11:46:33.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:32 trial167 ceph-mon[43392]: purged_snaps scrub ok 2026-02-25T11:46:33.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:32 trial167 ceph-mon[43392]: Detected new or changed devices on trial119 2026-02-25T11:46:33.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:32 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial119 to 21013M 2026-02-25T11:46:33.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:32 trial167 ceph-mon[43392]: from='client.? 10.20.193.167:0/2299350426' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:46:33.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:32 trial167 ceph-mon[43392]: osdmap e74: 12 total, 11 up, 12 in 2026-02-25T11:46:33.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:32 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:33.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:32 trial119 ceph-mon[42710]: purged_snaps scrub starts 2026-02-25T11:46:33.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:32 trial119 ceph-mon[42710]: purged_snaps scrub ok 2026-02-25T11:46:33.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:32 trial119 ceph-mon[42710]: Detected new or changed devices on trial119 2026-02-25T11:46:33.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:32 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial119 to 21013M 2026-02-25T11:46:33.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:32 trial119 ceph-mon[42710]: from='client.? 10.20.193.167:0/2299350426' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:46:33.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:32 trial119 ceph-mon[42710]: osdmap e74: 12 total, 11 up, 12 in 2026-02-25T11:46:33.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:32 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:33.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:32 trial028 ceph-mon[39224]: purged_snaps scrub starts 2026-02-25T11:46:33.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:32 trial028 ceph-mon[39224]: purged_snaps scrub ok 2026-02-25T11:46:33.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:32 trial028 ceph-mon[39224]: Detected new or changed devices on trial119 2026-02-25T11:46:33.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:32 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial119 to 21013M 2026-02-25T11:46:33.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:32 trial028 ceph-mon[39224]: from='client.? 10.20.193.167:0/2299350426' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2026-02-25T11:46:33.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:32 trial028 ceph-mon[39224]: osdmap e74: 12 total, 11 up, 12 in 2026-02-25T11:46:33.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:32 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:34.113 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:33 trial167 ceph-mon[43392]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-25T11:46:34.113 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:33 trial167 ceph-mon[43392]: osdmap e75: 12 total, 11 up, 12 in 2026-02-25T11:46:34.113 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:33 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:34.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:33 trial119 ceph-mon[42710]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-25T11:46:34.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:33 trial119 ceph-mon[42710]: osdmap e75: 12 total, 11 up, 12 in 2026-02-25T11:46:34.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:33 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:34.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:33 trial028 ceph-mon[39224]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 300 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-25T11:46:34.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:33 trial028 ceph-mon[39224]: osdmap e75: 12 total, 11 up, 12 in 2026-02-25T11:46:34.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:33 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:36.125 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:35 trial167 ceph-mon[43392]: osdmap e76: 12 total, 11 up, 12 in 2026-02-25T11:46:36.126 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:35 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:36.126 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:35 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2026-02-25T11:46:36.126 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:35 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:36.126 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:35 trial167 ceph-mon[43392]: Deploying daemon osd.11 on trial167 2026-02-25T11:46:36.126 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:35 trial167 ceph-mon[43392]: pgmap v165: 1 pgs: 1 peering; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-25T11:46:36.127 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:35 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2026-02-25T11:46:36.127 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:35 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:36.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:35 trial119 ceph-mon[42710]: osdmap e76: 12 total, 11 up, 12 in 2026-02-25T11:46:36.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:35 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:36.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:35 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2026-02-25T11:46:36.261 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:35 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:36.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:35 trial119 ceph-mon[42710]: Deploying daemon osd.11 on trial167 2026-02-25T11:46:36.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:35 trial119 ceph-mon[42710]: pgmap v165: 1 pgs: 1 peering; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-25T11:46:36.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:35 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2026-02-25T11:46:36.262 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:35 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:36.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:35 trial028 ceph-mon[39224]: osdmap e76: 12 total, 11 up, 12 in 2026-02-25T11:46:36.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:36.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2026-02-25T11:46:36.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:36.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:35 trial028 ceph-mon[39224]: Deploying daemon osd.11 on trial167 2026-02-25T11:46:36.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:35 trial028 ceph-mon[39224]: pgmap v165: 1 pgs: 1 peering; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-25T11:46:36.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2026-02-25T11:46:36.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:35 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:37.132 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:36 trial167 ceph-mon[43392]: Deploying daemon osd.11 on trial167 2026-02-25T11:46:37.132 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:36 trial167 ceph-mon[43392]: osdmap e77: 12 total, 11 up, 12 in 2026-02-25T11:46:37.132 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:36 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:37.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:36 trial119 ceph-mon[42710]: Deploying daemon osd.11 on trial167 2026-02-25T11:46:37.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:36 trial119 ceph-mon[42710]: osdmap e77: 12 total, 11 up, 12 in 2026-02-25T11:46:37.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:36 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:37.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:36 trial028 ceph-mon[39224]: Deploying daemon osd.11 on trial167 2026-02-25T11:46:37.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:36 trial028 ceph-mon[39224]: osdmap e77: 12 total, 11 up, 12 in 2026-02-25T11:46:37.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:36 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:38.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:37 trial167 ceph-mon[43392]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail; 129 KiB/s, 0 objects/s recovering 2026-02-25T11:46:38.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:37 trial119 ceph-mon[42710]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail; 129 KiB/s, 0 objects/s recovering 2026-02-25T11:46:38.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:37 trial028 ceph-mon[39224]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail; 129 KiB/s, 0 objects/s recovering 2026-02-25T11:46:39.232 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:39.232 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:38 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:39.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:39.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:38 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:39.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:39.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:38 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:40.114 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:39 trial167 ceph-mon[43392]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail; 96 KiB/s, 0 objects/s recovering 2026-02-25T11:46:40.114 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:39 trial167 ceph-mon[43392]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-25T11:46:40.114 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:39 trial167 ceph-mon[43392]: from='osd.11 [v2:10.20.193.167:6824/2824297643,v1:10.20.193.167:6825/2824297643]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-25T11:46:40.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:39 trial119 ceph-mon[42710]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail; 96 KiB/s, 0 objects/s recovering 2026-02-25T11:46:40.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:39 trial119 ceph-mon[42710]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-25T11:46:40.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:39 trial119 ceph-mon[42710]: from='osd.11 [v2:10.20.193.167:6824/2824297643,v1:10.20.193.167:6825/2824297643]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-25T11:46:40.294 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:40.292+0000 7f115cdc4640 1 -- 10.20.193.167:0/1403339743 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=72}) -- 0x7f1130003560 con 0x7f1158061690 2026-02-25T11:46:40.295 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:40.292+0000 7f1137fff640 1 -- 10.20.193.167:0/1403339743 <== mon.1 v2:10.20.193.167:3300/0 7 ==== osd_map(72..78 src has 1..78) ==== 4750+0+0 (secure 0 0 0) 0x7f114c005bd0 con 0x7f1158061690 2026-02-25T11:46:40.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:39 trial028 ceph-mon[39224]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail; 96 KiB/s, 0 objects/s recovering 2026-02-25T11:46:40.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:39 trial028 ceph-mon[39224]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-25T11:46:40.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:39 trial028 ceph-mon[39224]: from='osd.11 [v2:10.20.193.167:6824/2824297643,v1:10.20.193.167:6825/2824297643]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-25T11:46:41.064 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:40 trial167 ceph-mon[43392]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2026-02-25T11:46:41.064 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:40 trial167 ceph-mon[43392]: osdmap e78: 12 total, 11 up, 12 in 2026-02-25T11:46:41.064 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:40 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:41.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:40 trial119 ceph-mon[42710]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2026-02-25T11:46:41.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:40 trial119 ceph-mon[42710]: osdmap e78: 12 total, 11 up, 12 in 2026-02-25T11:46:41.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:40 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:41.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:40 trial028 ceph-mon[39224]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2026-02-25T11:46:41.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:40 trial028 ceph-mon[39224]: osdmap e78: 12 total, 11 up, 12 in 2026-02-25T11:46:41.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:40 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:42.022 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:41 trial167 ceph-mon[43392]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail; 93 KiB/s, 0 objects/s recovering 2026-02-25T11:46:42.022 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:42.022 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:41 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:42.153 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:42.150+0000 7f1137fff640 1 -- 10.20.193.167:0/1403339743 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f11581107b0 con 0x7f112c0b5760 2026-02-25T11:46:42.153 INFO:teuthology.orchestra.run.trial167.stdout:Created osd(s) 11 on host 'trial167' 2026-02-25T11:46:42.156 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:42.154+0000 7f1157fff640 1 -- 10.20.193.167:0/1403339743 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f112c0b5760 msgr2=0x7f112c0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:42.156 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:42.154+0000 7f1157fff640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f112c0b5760 0x7f112c0d5c10 secure :-1 s=READY pgs=133 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1140004740 tx=0x7f1140005d00 comp rx=0 tx=0).stop 2026-02-25T11:46:42.157 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:42.154+0000 7f1157fff640 1 -- 10.20.193.167:0/1403339743 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1158061690 msgr2=0x7f11580f61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:42.157 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:42.154+0000 7f1157fff640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1158061690 0x7f11580f61e0 secure :-1 s=READY pgs=60 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f114c0098a0 tx=0x7f114c0028b0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.157 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:42.154+0000 7f1157fff640 1 -- 10.20.193.167:0/1403339743 shutdown_connections 2026-02-25T11:46:42.157 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:42.154+0000 7f1157fff640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f112c0b5760 0x7f112c0d5c10 unknown :-1 s=CLOSED pgs=133 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.157 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:42.154+0000 7f1157fff640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f115810ca80 0x7f11580f7430 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.157 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:42.154+0000 7f1157fff640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1158062ac0 0x7f11580f6830 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.157 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:42.154+0000 7f1157fff640 1 --2- 10.20.193.167:0/1403339743 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1158061690 0x7f11580f61e0 unknown :-1 s=CLOSED pgs=60 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.157 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:42.154+0000 7f1157fff640 1 -- 10.20.193.167:0/1403339743 >> 10.20.193.167:0/1403339743 conn(0x7f11580ea6a0 msgr2=0x7f11580ec410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:42.157 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:42.154+0000 7f1157fff640 1 -- 10.20.193.167:0/1403339743 shutdown_connections 2026-02-25T11:46:42.157 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:42.154+0000 7f1157fff640 1 -- 10.20.193.167:0/1403339743 wait complete. 2026-02-25T11:46:42.256 DEBUG:teuthology.orchestra.run.trial167:osd.11> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.11.service 2026-02-25T11:46:42.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:41 trial119 ceph-mon[42710]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail; 93 KiB/s, 0 objects/s recovering 2026-02-25T11:46:42.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:42.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:41 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:42.261 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2026-02-25T11:46:42.261 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd stat -f json 2026-02-25T11:46:42.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:41 trial028 ceph-mon[39224]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail; 93 KiB/s, 0 objects/s recovering 2026-02-25T11:46:42.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:42.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:41 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:42.450 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:46:42.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.621+0000 7f742ab38640 1 -- 10.20.193.28:0/2048159836 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f742410b580 msgr2=0x7f742410b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:42.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.621+0000 7f742ab38640 1 --2- 10.20.193.28:0/2048159836 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f742410b580 0x7f742410b970 secure :-1 s=READY pgs=65 gs=2 cs=0 l=1 c_cookie=11582094529212683560 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7414009950 tx=0x7f741402f190 comp rx=0 tx=0).stop 2026-02-25T11:46:42.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.621+0000 7f742ab38640 1 -- 10.20.193.28:0/2048159836 shutdown_connections 2026-02-25T11:46:42.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.621+0000 7f742ab38640 1 --2- 10.20.193.28:0/2048159836 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f742410d0c0 0x7f742416da50 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.621+0000 7f742ab38640 1 --2- 10.20.193.28:0/2048159836 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f742410c710 0x7f742410cb80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.621+0000 7f742ab38640 1 --2- 10.20.193.28:0/2048159836 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f742410b580 0x7f742410b970 unknown :-1 s=CLOSED pgs=65 gs=2 cs=0 l=1 c_cookie=11582094529212683560 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.621+0000 7f742ab38640 1 -- 10.20.193.28:0/2048159836 >> 10.20.193.28:0/2048159836 conn(0x7f74240e8cb0 msgr2=0x7f74241090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:42.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.621+0000 7f742ab38640 1 -- 10.20.193.28:0/2048159836 shutdown_connections 2026-02-25T11:46:42.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.621+0000 7f742ab38640 1 -- 10.20.193.28:0/2048159836 wait complete. 2026-02-25T11:46:42.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f742ab38640 1 Processor -- start 2026-02-25T11:46:42.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f742ab38640 1 -- start start 2026-02-25T11:46:42.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f742ab38640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f742410b580 0x7f74240f4c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:42.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f742ab38640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f742410c710 0x7f742416da50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:42.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f742ab38640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f742410d0c0 0x7f74240f54f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:42.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f742ab38640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f74240f60e0 con 0x7f742410d0c0 2026-02-25T11:46:42.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f742ab38640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f74240f6250 con 0x7f742410b580 2026-02-25T11:46:42.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f742ab38640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f74240f6410 con 0x7f742410c710 2026-02-25T11:46:42.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f7429335640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f742410c710 0x7f742416da50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:42.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f7429b36640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f742410b580 0x7f74240f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:42.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f7429335640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f742410c710 0x7f742416da50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:39742/0 (socket says 10.20.193.28:39742) 2026-02-25T11:46:42.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f7429335640 1 -- 10.20.193.28:0/497471450 learned_addr learned my addr 10.20.193.28:0/497471450 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:46:42.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f7429b36640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f742410b580 0x7f74240f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:57648/0 (socket says 10.20.193.28:57648) 2026-02-25T11:46:42.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f7429b36640 1 -- 10.20.193.28:0/497471450 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f742410c710 msgr2=0x7f742416da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:42.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f7429b36640 1 --2- 10.20.193.28:0/497471450 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f742410c710 0x7f742416da50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f7429b36640 1 -- 10.20.193.28:0/497471450 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f742410d0c0 msgr2=0x7f74240f54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:42.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f7429b36640 1 --2- 10.20.193.28:0/497471450 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f742410d0c0 0x7f74240f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.622+0000 7f7429b36640 1 -- 10.20.193.28:0/497471450 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742000b6e0 con 0x7f742410b580 2026-02-25T11:46:42.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.623+0000 7f7429b36640 1 --2- 10.20.193.28:0/497471450 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f742410b580 0x7f74240f4c90 secure :-1 s=READY pgs=66 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7414009950 tx=0x7f74140028d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:42.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.623+0000 7f7412ffd640 1 -- 10.20.193.28:0/497471450 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f741403d070 con 0x7f742410b580 2026-02-25T11:46:42.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.623+0000 7f7412ffd640 1 -- 10.20.193.28:0/497471450 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7414005dd0 con 0x7f742410b580 2026-02-25T11:46:42.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.623+0000 7f7412ffd640 1 -- 10.20.193.28:0/497471450 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7414041960 con 0x7f742410b580 2026-02-25T11:46:42.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.623+0000 7f742ab38640 1 -- 10.20.193.28:0/497471450 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7414009570 con 0x7f742410b580 2026-02-25T11:46:42.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.623+0000 7f742ab38640 1 -- 10.20.193.28:0/497471450 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74240f6920 con 0x7f742410b580 2026-02-25T11:46:42.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.624+0000 7f742ab38640 1 -- 10.20.193.28:0/497471450 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7424061010 con 0x7f742410b580 2026-02-25T11:46:42.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.626+0000 7f7412ffd640 1 -- 10.20.193.28:0/497471450 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f7414041af0 con 0x7f742410b580 2026-02-25T11:46:42.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.628+0000 7f7412ffd640 1 --2- 10.20.193.28:0/497471450 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74000b5760 0x7f74000d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:42.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.628+0000 7f7412ffd640 1 -- 10.20.193.28:0/497471450 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6823+0+0 (secure 0 0 0) 0x7f741410af10 con 0x7f742410b580 2026-02-25T11:46:42.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.628+0000 7f7412ffd640 1 -- 10.20.193.28:0/497471450 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f741410b400 con 0x7f742410b580 2026-02-25T11:46:42.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.628+0000 7f7429335640 1 --2- 10.20.193.28:0/497471450 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74000b5760 0x7f74000d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:42.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.628+0000 7f7429335640 1 --2- 10.20.193.28:0/497471450 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74000b5760 0x7f74000d5c10 secure :-1 s=READY pgs=138 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f741800b2a0 tx=0x7f741800b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:42.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.742+0000 7f742ab38640 1 -- 10.20.193.28:0/497471450 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f742405fda0 con 0x7f742410b580 2026-02-25T11:46:42.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.743+0000 7f7412ffd640 1 -- 10.20.193.28:0/497471450 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7f742405fda0 con 0x7f742410b580 2026-02-25T11:46:42.746 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:46:42.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.744+0000 7f742ab38640 1 -- 10.20.193.28:0/497471450 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74000b5760 msgr2=0x7f74000d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:42.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.744+0000 7f742ab38640 1 --2- 10.20.193.28:0/497471450 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74000b5760 0x7f74000d5c10 secure :-1 s=READY pgs=138 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f741800b2a0 tx=0x7f741800b480 comp rx=0 tx=0).stop 2026-02-25T11:46:42.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.744+0000 7f742ab38640 1 -- 10.20.193.28:0/497471450 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f742410b580 msgr2=0x7f74240f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:42.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.744+0000 7f742ab38640 1 --2- 10.20.193.28:0/497471450 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f742410b580 0x7f74240f4c90 secure :-1 s=READY pgs=66 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7414009950 tx=0x7f74140028d0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.744+0000 7f742ab38640 1 -- 10.20.193.28:0/497471450 shutdown_connections 2026-02-25T11:46:42.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.744+0000 7f742ab38640 1 --2- 10.20.193.28:0/497471450 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74000b5760 0x7f74000d5c10 unknown :-1 s=CLOSED pgs=138 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.744+0000 7f742ab38640 1 --2- 10.20.193.28:0/497471450 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f742410d0c0 0x7f74240f54f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.744+0000 7f742ab38640 1 --2- 10.20.193.28:0/497471450 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f742410c710 0x7f742416da50 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.744+0000 7f742ab38640 1 --2- 10.20.193.28:0/497471450 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f742410b580 0x7f74240f4c90 unknown :-1 s=CLOSED pgs=66 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:42.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.744+0000 7f742ab38640 1 -- 10.20.193.28:0/497471450 >> 10.20.193.28:0/497471450 conn(0x7f74240e8cb0 msgr2=0x7f74240eb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:42.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.744+0000 7f742ab38640 1 -- 10.20.193.28:0/497471450 shutdown_connections 2026-02-25T11:46:42.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:42.744+0000 7f742ab38640 1 -- 10.20.193.28:0/497471450 wait complete. 2026-02-25T11:46:42.750 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:46:42 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:46:42.388+0000 7f1649797780 -1 osd.11 0 log_to_monitors true 2026-02-25T11:46:42.884 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":78,"num_osds":12,"num_up_osds":11,"osd_up_since":1772019991,"num_in_osds":12,"osd_in_since":1772019991,"num_remapped_pgs":0} 2026-02-25T11:46:43.151 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:43.151 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:43.151 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:43 trial167 ceph-mon[43392]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-25T11:46:43.151 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:43 trial167 ceph-mon[43392]: from='osd.11 [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-25T11:46:43.151 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:43 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/497471450' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-25T11:46:43.151 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:43.151 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:43.151 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:43 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:43.415 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:43.415 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:43.415 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:43 trial119 ceph-mon[42710]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-25T11:46:43.415 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:43 trial119 ceph-mon[42710]: from='osd.11 [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-25T11:46:43.416 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:43 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/497471450' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-25T11:46:43.416 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:43.416 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:43.416 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:43 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:43.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:43.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:43.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:43 trial028 ceph-mon[39224]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-25T11:46:43.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:43 trial028 ceph-mon[39224]: from='osd.11 [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2026-02-25T11:46:43.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:43 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/497471450' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-25T11:46:43.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:43.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:43.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:43 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:43.886 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd stat -f json 2026-02-25T11:46:44.074 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:46:44.192 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:44 trial028 ceph-mon[39224]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail; 72 KiB/s, 0 objects/s recovering 2026-02-25T11:46:44.192 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:44 trial028 ceph-mon[39224]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2026-02-25T11:46:44.192 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:44 trial028 ceph-mon[39224]: osdmap e79: 12 total, 11 up, 12 in 2026-02-25T11:46:44.192 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:44 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:44.192 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:44 trial028 ceph-mon[39224]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:46:44.192 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:44 trial028 ceph-mon[39224]: from='osd.11 [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:46:44.192 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:44 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:44.193 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:44 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:44.193 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:44 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:44.193 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:44 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:44.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.257+0000 7f683489f640 1 -- 10.20.193.28:0/695132387 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f682c110600 msgr2=0x7f682c110a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:44.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.257+0000 7f683489f640 1 --2- 10.20.193.28:0/695132387 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f682c110600 0x7f682c110a70 secure :-1 s=READY pgs=188 gs=1 cs=0 l=1 c_cookie=18157431653013685562 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6818009950 tx=0x7f681802f190 comp rx=0 tx=0).stop 2026-02-25T11:46:44.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.257+0000 7f683489f640 1 -- 10.20.193.28:0/695132387 shutdown_connections 2026-02-25T11:46:44.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.257+0000 7f683489f640 1 --2- 10.20.193.28:0/695132387 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f682c110fb0 0x7f682c171940 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:44.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.257+0000 7f683489f640 1 --2- 10.20.193.28:0/695132387 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f682c110600 0x7f682c110a70 unknown :-1 s=CLOSED pgs=188 gs=1 cs=0 l=1 c_cookie=18157431653013685562 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:44.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.257+0000 7f683489f640 1 --2- 10.20.193.28:0/695132387 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f682c10f470 0x7f682c10f860 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:44.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.257+0000 7f683489f640 1 -- 10.20.193.28:0/695132387 >> 10.20.193.28:0/695132387 conn(0x7f682c0ecba0 msgr2=0x7f682c10cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:44.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.257+0000 7f683489f640 1 -- 10.20.193.28:0/695132387 shutdown_connections 2026-02-25T11:46:44.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.257+0000 7f683489f640 1 -- 10.20.193.28:0/695132387 wait complete. 2026-02-25T11:46:44.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.258+0000 7f683489f640 1 Processor -- start 2026-02-25T11:46:44.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.258+0000 7f683489f640 1 -- start start 2026-02-25T11:46:44.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.258+0000 7f683489f640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f682c10f470 0x7f682c0f95c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:44.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.258+0000 7f683489f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f682c110600 0x7f682c171940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:44.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.258+0000 7f683489f640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f682c110fb0 0x7f682c0f9e20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:44.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.258+0000 7f683489f640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f682c0faa10 con 0x7f682c110600 2026-02-25T11:46:44.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.258+0000 7f683489f640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f682c0fab80 con 0x7f682c110fb0 2026-02-25T11:46:44.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.258+0000 7f683489f640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f682c0fad40 con 0x7f682c10f470 2026-02-25T11:46:44.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f6831e13640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f682c110600 0x7f682c171940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:44.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f6831e13640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f682c110600 0x7f682c171940 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47050/0 (socket says 10.20.193.28:47050) 2026-02-25T11:46:44.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f6831e13640 1 -- 10.20.193.28:0/1580820452 learned_addr learned my addr 10.20.193.28:0/1580820452 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:46:44.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f6832614640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f682c10f470 0x7f682c0f95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:44.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f6832e15640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f682c110fb0 0x7f682c0f9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:44.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f6831e13640 1 -- 10.20.193.28:0/1580820452 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f682c10f470 msgr2=0x7f682c0f95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:44.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f6831e13640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f682c10f470 0x7f682c0f95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:44.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f6831e13640 1 -- 10.20.193.28:0/1580820452 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f682c110fb0 msgr2=0x7f682c0f9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:44.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f6831e13640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f682c110fb0 0x7f682c0f9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:44.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f6831e13640 1 -- 10.20.193.28:0/1580820452 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6818009570 con 0x7f682c110600 2026-02-25T11:46:44.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f6831e13640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f682c110600 0x7f682c171940 secure :-1 s=READY pgs=189 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6818004820 tx=0x7f68180028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:44.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f68177fe640 1 -- 10.20.193.28:0/1580820452 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f681803d070 con 0x7f682c110600 2026-02-25T11:46:44.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f68177fe640 1 -- 10.20.193.28:0/1580820452 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f681802fe20 con 0x7f682c110600 2026-02-25T11:46:44.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f68177fe640 1 -- 10.20.193.28:0/1580820452 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6818041a20 con 0x7f682c110600 2026-02-25T11:46:44.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f683489f640 1 -- 10.20.193.28:0/1580820452 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682c0fafc0 con 0x7f682c110600 2026-02-25T11:46:44.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.259+0000 7f683489f640 1 -- 10.20.193.28:0/1580820452 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f682c0fb460 con 0x7f682c110600 2026-02-25T11:46:44.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.260+0000 7f683489f640 1 -- 10.20.193.28:0/1580820452 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f682c0fb680 con 0x7f682c110600 2026-02-25T11:46:44.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.264+0000 7f68177fe640 1 -- 10.20.193.28:0/1580820452 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f6818041bb0 con 0x7f682c110600 2026-02-25T11:46:44.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.266+0000 7f68177fe640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f68080b5760 0x7f68080d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:44.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.266+0000 7f68177fe640 1 -- 10.20.193.28:0/1580820452 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6839+0+0 (secure 0 0 0) 0x7f681810b1c0 con 0x7f682c110600 2026-02-25T11:46:44.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.266+0000 7f68177fe640 1 -- 10.20.193.28:0/1580820452 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f681810b640 con 0x7f682c110600 2026-02-25T11:46:44.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.266+0000 7f6832614640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f68080b5760 0x7f68080d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:44.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.267+0000 7f6832614640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f68080b5760 0x7f68080d5c10 secure :-1 s=READY pgs=140 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6824006fd0 tx=0x7f6824008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:44.349 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:44 trial167 ceph-mon[43392]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail; 72 KiB/s, 0 objects/s recovering 2026-02-25T11:46:44.350 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:44 trial167 ceph-mon[43392]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2026-02-25T11:46:44.350 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:44 trial167 ceph-mon[43392]: osdmap e79: 12 total, 11 up, 12 in 2026-02-25T11:46:44.350 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:44 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:44.350 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:44 trial167 ceph-mon[43392]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:46:44.350 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:44 trial167 ceph-mon[43392]: from='osd.11 [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:46:44.350 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:44 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:44.350 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:44 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:44.350 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:44 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:44.350 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:44 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:44.351 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:46:44 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:46:44.159+0000 7f1645719640 -1 osd.11 0 waiting for initial osdmap 2026-02-25T11:46:44.351 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:46:44 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:46:44.171+0000 7f163d5d8640 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-25T11:46:44.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.382+0000 7f683489f640 1 -- 10.20.193.28:0/1580820452 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f682c10f840 con 0x7f682c110600 2026-02-25T11:46:44.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.382+0000 7f68177fe640 1 -- 10.20.193.28:0/1580820452 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f68180ce0e0 con 0x7f682c110600 2026-02-25T11:46:44.386 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:46:44.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.384+0000 7f683489f640 1 -- 10.20.193.28:0/1580820452 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f68080b5760 msgr2=0x7f68080d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:44.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.384+0000 7f683489f640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f68080b5760 0x7f68080d5c10 secure :-1 s=READY pgs=140 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6824006fd0 tx=0x7f6824008040 comp rx=0 tx=0).stop 2026-02-25T11:46:44.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.384+0000 7f683489f640 1 -- 10.20.193.28:0/1580820452 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f682c110600 msgr2=0x7f682c171940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:44.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.384+0000 7f683489f640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f682c110600 0x7f682c171940 secure :-1 s=READY pgs=189 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6818004820 tx=0x7f68180028d0 comp rx=0 tx=0).stop 2026-02-25T11:46:44.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.384+0000 7f683489f640 1 -- 10.20.193.28:0/1580820452 shutdown_connections 2026-02-25T11:46:44.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.384+0000 7f683489f640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f68080b5760 0x7f68080d5c10 unknown :-1 s=CLOSED pgs=140 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:44.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.384+0000 7f683489f640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f682c110fb0 0x7f682c0f9e20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:44.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.384+0000 7f683489f640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f682c110600 0x7f682c171940 unknown :-1 s=CLOSED pgs=189 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:44.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.384+0000 7f683489f640 1 --2- 10.20.193.28:0/1580820452 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f682c10f470 0x7f682c0f95c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:44.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.384+0000 7f683489f640 1 -- 10.20.193.28:0/1580820452 >> 10.20.193.28:0/1580820452 conn(0x7f682c0ecba0 msgr2=0x7f682c0efb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:44.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.384+0000 7f683489f640 1 -- 10.20.193.28:0/1580820452 shutdown_connections 2026-02-25T11:46:44.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:44.384+0000 7f683489f640 1 -- 10.20.193.28:0/1580820452 wait complete. 2026-02-25T11:46:44.489 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1772019991,"num_in_osds":12,"osd_in_since":1772019991,"num_remapped_pgs":0} 2026-02-25T11:46:44.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:44 trial119 ceph-mon[42710]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail; 72 KiB/s, 0 objects/s recovering 2026-02-25T11:46:44.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:44 trial119 ceph-mon[42710]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2026-02-25T11:46:44.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:44 trial119 ceph-mon[42710]: osdmap e79: 12 total, 11 up, 12 in 2026-02-25T11:46:44.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:44 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:44.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:44 trial119 ceph-mon[42710]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:46:44.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:44 trial119 ceph-mon[42710]: from='osd.11 [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial167", "root=default"]} : dispatch 2026-02-25T11:46:44.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:44 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:44.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:44 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:44.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:44 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:44.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:44 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:45.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:45 trial119 ceph-mon[42710]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial167", "root=default"]}]': finished 2026-02-25T11:46:45.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:45 trial119 ceph-mon[42710]: osdmap e80: 12 total, 11 up, 12 in 2026-02-25T11:46:45.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:45.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:45 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:45.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:45 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1580820452' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-25T11:46:45.489 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd stat -f json 2026-02-25T11:46:45.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:45 trial167 ceph-mon[43392]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial167", "root=default"]}]': finished 2026-02-25T11:46:45.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:45 trial167 ceph-mon[43392]: osdmap e80: 12 total, 11 up, 12 in 2026-02-25T11:46:45.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:45.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:45 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:45.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:45 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1580820452' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-25T11:46:45.540 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:45 trial028 ceph-mon[39224]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.6836, "args": ["host=trial167", "root=default"]}]': finished 2026-02-25T11:46:45.540 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:45 trial028 ceph-mon[39224]: osdmap e80: 12 total, 11 up, 12 in 2026-02-25T11:46:45.540 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:45.540 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:45 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:45.540 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:45 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1580820452' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-25T11:46:45.673 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:46:45.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.856+0000 7f8af08ef640 1 -- 10.20.193.28:0/926833044 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8aec113e70 msgr2=0x7f8aec134250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:45.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.856+0000 7f8af08ef640 1 --2- 10.20.193.28:0/926833044 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8aec113e70 0x7f8aec134250 secure :-1 s=READY pgs=68 gs=3 cs=0 l=1 c_cookie=5177249731998765999 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8ae0009950 tx=0x7f8ae002f190 comp rx=0 tx=0).stop 2026-02-25T11:46:45.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.856+0000 7f8af08ef640 1 -- 10.20.193.28:0/926833044 shutdown_connections 2026-02-25T11:46:45.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.856+0000 7f8af08ef640 1 --2- 10.20.193.28:0/926833044 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8aec1553c0 0x7f8aec175870 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:45.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.856+0000 7f8af08ef640 1 --2- 10.20.193.28:0/926833044 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8aec134820 0x7f8aec154c80 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:45.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.856+0000 7f8af08ef640 1 --2- 10.20.193.28:0/926833044 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8aec113e70 0x7f8aec134250 unknown :-1 s=CLOSED pgs=68 gs=3 cs=0 l=1 c_cookie=5177249731998765999 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:45.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.856+0000 7f8af08ef640 1 -- 10.20.193.28:0/926833044 >> 10.20.193.28:0/926833044 conn(0x7f8aec064670 msgr2=0x7f8aec064a70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:45.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.856+0000 7f8af08ef640 1 -- 10.20.193.28:0/926833044 shutdown_connections 2026-02-25T11:46:45.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.857+0000 7f8af08ef640 1 -- 10.20.193.28:0/926833044 wait complete. 2026-02-25T11:46:45.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.858+0000 7f8af08ef640 1 Processor -- start 2026-02-25T11:46:45.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.858+0000 7f8af08ef640 1 -- start start 2026-02-25T11:46:45.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.858+0000 7f8af08ef640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8aec113e70 0x7f8aec175870 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:45.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.858+0000 7f8af08ef640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8aec134820 0x7f8aec154c50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:45.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.858+0000 7f8af08ef640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8aec1553c0 0x7f8aec0ffc80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:45.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.858+0000 7f8af08ef640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec100410 con 0x7f8aec113e70 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.858+0000 7f8af08ef640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec100580 con 0x7f8aec1553c0 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.858+0000 7f8af08ef640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8aec100bd0 con 0x7f8aec134820 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.858+0000 7f8aeb577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8aec1553c0 0x7f8aec0ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.858+0000 7f8aeb577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8aec1553c0 0x7f8aec0ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:57684/0 (socket says 10.20.193.28:57684) 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.858+0000 7f8aeb577640 1 -- 10.20.193.28:0/2693264258 learned_addr learned my addr 10.20.193.28:0/2693264258 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.858+0000 7f8aea575640 1 --2- 10.20.193.28:0/2693264258 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8aec134820 0x7f8aec154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.859+0000 7f8aeb577640 1 -- 10.20.193.28:0/2693264258 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8aec134820 msgr2=0x7f8aec154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.859+0000 7f8aeb577640 1 --2- 10.20.193.28:0/2693264258 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8aec134820 0x7f8aec154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.859+0000 7f8aeb577640 1 -- 10.20.193.28:0/2693264258 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8aec113e70 msgr2=0x7f8aec175870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.859+0000 7f8aeb577640 1 --2- 10.20.193.28:0/2693264258 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8aec113e70 0x7f8aec175870 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.859+0000 7f8aeb577640 1 -- 10.20.193.28:0/2693264258 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8adc00a590 con 0x7f8aec1553c0 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.859+0000 7f8aeb577640 1 --2- 10.20.193.28:0/2693264258 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8aec1553c0 0x7f8aec0ffc80 secure :-1 s=READY pgs=67 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8adc005790 tx=0x7f8adc003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.859+0000 7f8acbfff640 1 -- 10.20.193.28:0/2693264258 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8adc00fd60 con 0x7f8aec1553c0 2026-02-25T11:46:45.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.859+0000 7f8acbfff640 1 -- 10.20.193.28:0/2693264258 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8adc003e80 con 0x7f8aec1553c0 2026-02-25T11:46:45.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.859+0000 7f8acbfff640 1 -- 10.20.193.28:0/2693264258 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8adc019930 con 0x7f8aec1553c0 2026-02-25T11:46:45.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.859+0000 7f8af08ef640 1 -- 10.20.193.28:0/2693264258 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae0009570 con 0x7f8aec1553c0 2026-02-25T11:46:45.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.859+0000 7f8af08ef640 1 -- 10.20.193.28:0/2693264258 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aec101170 con 0x7f8aec1553c0 2026-02-25T11:46:45.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.860+0000 7f8af08ef640 1 -- 10.20.193.28:0/2693264258 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aec05f4f0 con 0x7f8aec1553c0 2026-02-25T11:46:45.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.861+0000 7f8acbfff640 1 -- 10.20.193.28:0/2693264258 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f8adc017020 con 0x7f8aec1553c0 2026-02-25T11:46:45.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.863+0000 7f8acbfff640 1 --2- 10.20.193.28:0/2693264258 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8ac00b5760 0x7f8ac00d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:45.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.863+0000 7f8acbfff640 1 -- 10.20.193.28:0/2693264258 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7119+0+0 (secure 0 0 0) 0x7f8adc0e7a70 con 0x7f8aec1553c0 2026-02-25T11:46:45.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.863+0000 7f8aead76640 1 --2- 10.20.193.28:0/2693264258 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8ac00b5760 0x7f8ac00d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:45.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.863+0000 7f8acbfff640 1 -- 10.20.193.28:0/2693264258 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f8adc0aa880 con 0x7f8aec1553c0 2026-02-25T11:46:45.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.864+0000 7f8aead76640 1 --2- 10.20.193.28:0/2693264258 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8ac00b5760 0x7f8ac00d5c10 secure :-1 s=READY pgs=142 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8ae002f6a0 tx=0x7f8ae0005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:45.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.978+0000 7f8af08ef640 1 -- 10.20.193.28:0/2693264258 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8aec179c90 con 0x7f8aec1553c0 2026-02-25T11:46:45.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.979+0000 7f8acbfff640 1 -- 10.20.193.28:0/2693264258 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f8adc0ada30 con 0x7f8aec1553c0 2026-02-25T11:46:45.981 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:46:45.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.980+0000 7f8af08ef640 1 -- 10.20.193.28:0/2693264258 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8ac00b5760 msgr2=0x7f8ac00d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:45.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.980+0000 7f8af08ef640 1 --2- 10.20.193.28:0/2693264258 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8ac00b5760 0x7f8ac00d5c10 secure :-1 s=READY pgs=142 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8ae002f6a0 tx=0x7f8ae0005bf0 comp rx=0 tx=0).stop 2026-02-25T11:46:45.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.980+0000 7f8af08ef640 1 -- 10.20.193.28:0/2693264258 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8aec1553c0 msgr2=0x7f8aec0ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:45.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.980+0000 7f8af08ef640 1 --2- 10.20.193.28:0/2693264258 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8aec1553c0 0x7f8aec0ffc80 secure :-1 s=READY pgs=67 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8adc005790 tx=0x7f8adc003c20 comp rx=0 tx=0).stop 2026-02-25T11:46:45.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.980+0000 7f8af08ef640 1 -- 10.20.193.28:0/2693264258 shutdown_connections 2026-02-25T11:46:45.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.980+0000 7f8af08ef640 1 --2- 10.20.193.28:0/2693264258 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8ac00b5760 0x7f8ac00d5c10 unknown :-1 s=CLOSED pgs=142 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:45.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.980+0000 7f8af08ef640 1 --2- 10.20.193.28:0/2693264258 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8aec1553c0 0x7f8aec0ffc80 unknown :-1 s=CLOSED pgs=67 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:45.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.980+0000 7f8af08ef640 1 --2- 10.20.193.28:0/2693264258 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8aec134820 0x7f8aec154c50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:45.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.980+0000 7f8af08ef640 1 --2- 10.20.193.28:0/2693264258 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8aec113e70 0x7f8aec175870 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:45.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.980+0000 7f8af08ef640 1 -- 10.20.193.28:0/2693264258 >> 10.20.193.28:0/2693264258 conn(0x7f8aec064670 msgr2=0x7f8aec0f6530 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:45.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.980+0000 7f8af08ef640 1 -- 10.20.193.28:0/2693264258 shutdown_connections 2026-02-25T11:46:45.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:45.980+0000 7f8af08ef640 1 -- 10.20.193.28:0/2693264258 wait complete. 2026-02-25T11:46:46.081 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1772020005,"num_in_osds":12,"osd_in_since":1772019991,"num_remapped_pgs":0} 2026-02-25T11:46:46.081 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd dump --format=json 2026-02-25T11:46:46.265 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:46:46.308 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: purged_snaps scrub starts 2026-02-25T11:46:46.308 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: purged_snaps scrub ok 2026-02-25T11:46:46.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-25T11:46:46.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: osd.11 [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700] boot 2026-02-25T11:46:46.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: osdmap e81: 12 total, 12 up, 12 in 2026-02-25T11:46:46.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:46.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: Detected new or changed devices on trial167 2026-02-25T11:46:46.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:46.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:46.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:46.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:46.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial167 to 21013M 2026-02-25T11:46:46.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:46.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:46.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:46.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:46.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:46.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:46 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2693264258' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-25T11:46:46.371 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: purged_snaps scrub starts 2026-02-25T11:46:46.371 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: purged_snaps scrub ok 2026-02-25T11:46:46.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-25T11:46:46.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: osd.11 [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700] boot 2026-02-25T11:46:46.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: osdmap e81: 12 total, 12 up, 12 in 2026-02-25T11:46:46.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:46.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: Detected new or changed devices on trial167 2026-02-25T11:46:46.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:46.372 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:46.373 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:46.373 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:46.373 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial167 to 21013M 2026-02-25T11:46:46.373 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.373 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.373 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.373 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:46.373 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:46.374 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.374 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:46.374 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:46.374 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:46.374 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:46 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2693264258' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-25T11:46:46.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.453+0000 7f5bcba8e640 1 -- 10.20.193.28:0/267343916 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5bc4110230 msgr2=0x7f5bc4115280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:46.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.453+0000 7f5bcba8e640 1 --2- 10.20.193.28:0/267343916 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5bc4110230 0x7f5bc4115280 secure :-1 s=READY pgs=190 gs=1 cs=0 l=1 c_cookie=17941079825767557888 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5bb8009950 tx=0x7f5bb802f190 comp rx=0 tx=0).stop 2026-02-25T11:46:46.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.453+0000 7f5bcba8e640 1 -- 10.20.193.28:0/267343916 shutdown_connections 2026-02-25T11:46:46.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.453+0000 7f5bcba8e640 1 --2- 10.20.193.28:0/267343916 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5bc4115a30 0x7f5bc4176400 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:46.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.453+0000 7f5bcba8e640 1 --2- 10.20.193.28:0/267343916 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5bc4110230 0x7f5bc4115280 unknown :-1 s=CLOSED pgs=190 gs=1 cs=0 l=1 c_cookie=17941079825767557888 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:46.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.453+0000 7f5bcba8e640 1 --2- 10.20.193.28:0/267343916 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5bc410f840 0x7f5bc410fc10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:46.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.453+0000 7f5bcba8e640 1 -- 10.20.193.28:0/267343916 >> 10.20.193.28:0/267343916 conn(0x7f5bc40ed130 msgr2=0x7f5bc410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:46.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.453+0000 7f5bcba8e640 1 -- 10.20.193.28:0/267343916 shutdown_connections 2026-02-25T11:46:46.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.453+0000 7f5bcba8e640 1 -- 10.20.193.28:0/267343916 wait complete. 2026-02-25T11:46:46.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.454+0000 7f5bcba8e640 1 Processor -- start 2026-02-25T11:46:46.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.454+0000 7f5bcba8e640 1 -- start start 2026-02-25T11:46:46.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bcba8e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5bc410f840 0x7f5bc40f5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:46.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bcba8e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5bc4110230 0x7f5bc4176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:46.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bcba8e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5bc4115a30 0x7f5bc40f62c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:46.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bcba8e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc40fb4d0 con 0x7f5bc4115a30 2026-02-25T11:46:46.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bcba8e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc40fb640 con 0x7f5bc4110230 2026-02-25T11:46:46.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bcba8e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc40fb800 con 0x7f5bc410f840 2026-02-25T11:46:46.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bca004640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5bc4115a30 0x7f5bc40f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:46.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bc9803640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5bc410f840 0x7f5bc40f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:46.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bca004640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5bc4115a30 0x7f5bc40f62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:45534/0 (socket says 10.20.193.28:45534) 2026-02-25T11:46:46.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bca004640 1 -- 10.20.193.28:0/2164739299 learned_addr learned my addr 10.20.193.28:0/2164739299 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:46:46.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bc9002640 1 --2- 10.20.193.28:0/2164739299 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5bc4110230 0x7f5bc4176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:46.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bca004640 1 -- 10.20.193.28:0/2164739299 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5bc410f840 msgr2=0x7f5bc40f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:46.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bca004640 1 --2- 10.20.193.28:0/2164739299 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5bc410f840 0x7f5bc40f5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:46.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bca004640 1 -- 10.20.193.28:0/2164739299 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5bc4110230 msgr2=0x7f5bc4176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:46.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bca004640 1 --2- 10.20.193.28:0/2164739299 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5bc4110230 0x7f5bc4176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:46.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bca004640 1 -- 10.20.193.28:0/2164739299 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb8009570 con 0x7f5bc4115a30 2026-02-25T11:46:46.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bca004640 1 --2- 10.20.193.28:0/2164739299 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5bc4115a30 0x7f5bc40f62c0 secure :-1 s=READY pgs=191 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5bc000e650 tx=0x7f5bc000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:46.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bb2ffd640 1 -- 10.20.193.28:0/2164739299 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5bc000aa20 con 0x7f5bc4115a30 2026-02-25T11:46:46.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bb2ffd640 1 -- 10.20.193.28:0/2164739299 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5bc000ab80 con 0x7f5bc4115a30 2026-02-25T11:46:46.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bcba8e640 1 -- 10.20.193.28:0/2164739299 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc40fbae0 con 0x7f5bc4115a30 2026-02-25T11:46:46.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.455+0000 7f5bb2ffd640 1 -- 10.20.193.28:0/2164739299 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5bc0010640 con 0x7f5bc4115a30 2026-02-25T11:46:46.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.456+0000 7f5bcba8e640 1 -- 10.20.193.28:0/2164739299 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc40fbfb0 con 0x7f5bc4115a30 2026-02-25T11:46:46.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.456+0000 7f5bcba8e640 1 -- 10.20.193.28:0/2164739299 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc4110b20 con 0x7f5bc4115a30 2026-02-25T11:46:46.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.458+0000 7f5bb2ffd640 1 -- 10.20.193.28:0/2164739299 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f5bc0011460 con 0x7f5bc4115a30 2026-02-25T11:46:46.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.460+0000 7f5bb2ffd640 1 --2- 10.20.193.28:0/2164739299 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ba00b5820 0x7f5ba00d5cd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:46.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.460+0000 7f5bb2ffd640 1 -- 10.20.193.28:0/2164739299 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7f5bc00e7a30 con 0x7f5bc4115a30 2026-02-25T11:46:46.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.460+0000 7f5bb2ffd640 1 -- 10.20.193.28:0/2164739299 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f5bc0120940 con 0x7f5bc4115a30 2026-02-25T11:46:46.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.460+0000 7f5bc9803640 1 --2- 10.20.193.28:0/2164739299 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ba00b5820 0x7f5ba00d5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:46.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.461+0000 7f5bc9803640 1 --2- 10.20.193.28:0/2164739299 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ba00b5820 0x7f5ba00d5cd0 secure :-1 s=READY pgs=144 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5bb4004740 tx=0x7f5bb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:46.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: purged_snaps scrub starts 2026-02-25T11:46:46.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: purged_snaps scrub ok 2026-02-25T11:46:46.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 7.5 TiB / 7.5 TiB avail 2026-02-25T11:46:46.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: osd.11 [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700] boot 2026-02-25T11:46:46.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: osdmap e81: 12 total, 12 up, 12 in 2026-02-25T11:46:46.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2026-02-25T11:46:46.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: Detected new or changed devices on trial167 2026-02-25T11:46:46.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:46.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:46.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:46.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2026-02-25T11:46:46.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial167 to 21013M 2026-02-25T11:46:46.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:46.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:46.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:46.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:46.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2026-02-25T11:46:46.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:46.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:46 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2693264258' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2026-02-25T11:46:46.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.577+0000 7f5bcba8e640 1 -- 10.20.193.28:0/2164739299 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bc40f6d20 con 0x7f5bc4115a30 2026-02-25T11:46:46.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.578+0000 7f5bb2ffd640 1 -- 10.20.193.28:0/2164739299 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18479 (secure 0 0 0) 0x7f5bc00aa8f0 con 0x7f5bc4115a30 2026-02-25T11:46:46.581 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:46:46.581 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":82,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","created":"2026-02-25T11:42:51.962844+0000","modified":"2026-02-25T11:46:46.231278+0000","last_up_change":"2026-02-25T11:46:45.156629+0000","last_in_change":"2026-02-25T11:46:31.585002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":38,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T11:44:43.266336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf596df7-d1fb-4d10-8d4c-14a35210bab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6803","nonce":668944205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6805","nonce":668944205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6809","nonce":668944205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6807","nonce":668944205}]},"public_addr":"10.20.193.28:6803/668944205","cluster_addr":"10.20.193.28:6805/668944205","heartbeat_back_addr":"10.20.193.28:6809/668944205","heartbeat_front_addr":"10.20.193.28:6807/668944205","state":["exists","up"]},{"osd":1,"uuid":"185719c0-7c2d-431f-9925-88bbee86a98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6800","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6801","nonce":3486566242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6802","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6803","nonce":3486566242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6806","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6807","nonce":3486566242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6804","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6805","nonce":3486566242}]},"public_addr":"10.20.193.119:6801/3486566242","cluster_addr":"10.20.193.119:6803/3486566242","heartbeat_back_addr":"10.20.193.119:6807/3486566242","heartbeat_front_addr":"10.20.193.119:6805/3486566242","state":["exists","up"]},{"osd":2,"uuid":"4de13b90-1c3c-4614-9fef-2d7315626433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6800","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6801","nonce":484781559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6802","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6803","nonce":484781559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6806","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6807","nonce":484781559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6804","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6805","nonce":484781559}]},"public_addr":"10.20.193.167:6801/484781559","cluster_addr":"10.20.193.167:6803/484781559","heartbeat_back_addr":"10.20.193.167:6807/484781559","heartbeat_front_addr":"10.20.193.167:6805/484781559","state":["exists","up"]},{"osd":3,"uuid":"45d9ea8e-5a77-4738-96da-ada98515fc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6811","nonce":2452541436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6813","nonce":2452541436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6817","nonce":2452541436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6815","nonce":2452541436}]},"public_addr":"10.20.193.28:6811/2452541436","cluster_addr":"10.20.193.28:6813/2452541436","heartbeat_back_addr":"10.20.193.28:6817/2452541436","heartbeat_front_addr":"10.20.193.28:6815/2452541436","state":["exists","up"]},{"osd":4,"uuid":"c9c3f512-665b-44b2-a6f0-f3356a4b661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6808","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6809","nonce":135971996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6810","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6811","nonce":135971996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6814","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6815","nonce":135971996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6812","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6813","nonce":135971996}]},"public_addr":"10.20.193.119:6809/135971996","cluster_addr":"10.20.193.119:6811/135971996","heartbeat_back_addr":"10.20.193.119:6815/135971996","heartbeat_front_addr":"10.20.193.119:6813/135971996","state":["exists","up"]},{"osd":5,"uuid":"777c3819-5c90-4b2b-a545-c024884c2eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6808","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6809","nonce":4203420884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6810","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6811","nonce":4203420884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6814","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6815","nonce":4203420884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6812","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6813","nonce":4203420884}]},"public_addr":"10.20.193.167:6809/4203420884","cluster_addr":"10.20.193.167:6811/4203420884","heartbeat_back_addr":"10.20.193.167:6815/4203420884","heartbeat_front_addr":"10.20.193.167:6813/4203420884","state":["exists","up"]},{"osd":6,"uuid":"bbd9de64-b9fe-499e-b451-c16828be2c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6819","nonce":621484999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6821","nonce":621484999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6825","nonce":621484999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6823","nonce":621484999}]},"public_addr":"10.20.193.28:6819/621484999","cluster_addr":"10.20.193.28:6821/621484999","heartbeat_back_addr":"10.20.193.28:6825/621484999","heartbeat_front_addr":"10.20.193.28:6823/621484999","state":["exists","up"]},{"osd":7,"uuid":"546d9f28-597b-454c-8456-fcc2d46c69f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6816","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6817","nonce":2724541747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6818","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6819","nonce":2724541747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6822","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6823","nonce":2724541747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6820","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6821","nonce":2724541747}]},"public_addr":"10.20.193.119:6817/2724541747","cluster_addr":"10.20.193.119:6819/2724541747","heartbeat_back_addr":"10.20.193.119:6823/2724541747","heartbeat_front_addr":"10.20.193.119:6821/2724541747","state":["exists","up"]},{"osd":8,"uuid":"a1a7cea6-3586-4bb2-aa4c-bf6f7799a501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6816","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6817","nonce":1328734317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6818","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6819","nonce":1328734317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6822","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6823","nonce":1328734317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6820","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6821","nonce":1328734317}]},"public_addr":"10.20.193.167:6817/1328734317","cluster_addr":"10.20.193.167:6819/1328734317","heartbeat_back_addr":"10.20.193.167:6823/1328734317","heartbeat_front_addr":"10.20.193.167:6821/1328734317","state":["exists","up"]},{"osd":9,"uuid":"03fef134-6946-43d5-9c26-dc0e59fffe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6827","nonce":354500625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6829","nonce":354500625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6833","nonce":354500625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6831","nonce":354500625}]},"public_addr":"10.20.193.28:6827/354500625","cluster_addr":"10.20.193.28:6829/354500625","heartbeat_back_addr":"10.20.193.28:6833/354500625","heartbeat_front_addr":"10.20.193.28:6831/354500625","state":["exists","up"]},{"osd":10,"uuid":"b46c29ef-285c-4184-be28-d99b76bc0e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6824","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6825","nonce":2996581116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6826","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6827","nonce":2996581116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6830","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6831","nonce":2996581116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6828","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6829","nonce":2996581116}]},"public_addr":"10.20.193.119:6825/2996581116","cluster_addr":"10.20.193.119:6827/2996581116","heartbeat_back_addr":"10.20.193.119:6831/2996581116","heartbeat_front_addr":"10.20.193.119:6829/2996581116","state":["exists","up"]},{"osd":11,"uuid":"f6696d8c-5de7-4fd9-b693-524f204e4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6824","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6825","nonce":2182906700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6826","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6827","nonce":2182906700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6830","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6831","nonce":2182906700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6828","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6829","nonce":2182906700}]},"public_addr":"10.20.193.167:6825/2182906700","cluster_addr":"10.20.193.167:6827/2182906700","heartbeat_back_addr":"10.20.193.167:6831/2182906700","heartbeat_front_addr":"10.20.193.167:6829/2182906700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:13.410235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:27.254773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:40.956615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:54.529866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:08.306772+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:21.899498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:35.500407+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:49.094965+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:02.790138+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:16.286937+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:29.922722+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:43.396028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.28:0/1467902196":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6800/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/2274270913":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/60087422":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:6800/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/1725332841":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/3082630924":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/2391715078":"2026-02-26T11:43:04.102029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T11:46:46.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.579+0000 7f5bcba8e640 1 -- 10.20.193.28:0/2164739299 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ba00b5820 msgr2=0x7f5ba00d5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:46.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.579+0000 7f5bcba8e640 1 --2- 10.20.193.28:0/2164739299 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ba00b5820 0x7f5ba00d5cd0 secure :-1 s=READY pgs=144 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5bb4004740 tx=0x7f5bb4005f90 comp rx=0 tx=0).stop 2026-02-25T11:46:46.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.579+0000 7f5bcba8e640 1 -- 10.20.193.28:0/2164739299 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5bc4115a30 msgr2=0x7f5bc40f62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:46.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.579+0000 7f5bcba8e640 1 --2- 10.20.193.28:0/2164739299 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5bc4115a30 0x7f5bc40f62c0 secure :-1 s=READY pgs=191 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5bc000e650 tx=0x7f5bc000eb20 comp rx=0 tx=0).stop 2026-02-25T11:46:46.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.579+0000 7f5bcba8e640 1 -- 10.20.193.28:0/2164739299 shutdown_connections 2026-02-25T11:46:46.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.579+0000 7f5bcba8e640 1 --2- 10.20.193.28:0/2164739299 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ba00b5820 0x7f5ba00d5cd0 unknown :-1 s=CLOSED pgs=144 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:46.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.579+0000 7f5bcba8e640 1 --2- 10.20.193.28:0/2164739299 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5bc4115a30 0x7f5bc40f62c0 unknown :-1 s=CLOSED pgs=191 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:46.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.579+0000 7f5bcba8e640 1 --2- 10.20.193.28:0/2164739299 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5bc4110230 0x7f5bc4176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:46.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.579+0000 7f5bcba8e640 1 --2- 10.20.193.28:0/2164739299 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5bc410f840 0x7f5bc40f5bc0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:46.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.579+0000 7f5bcba8e640 1 -- 10.20.193.28:0/2164739299 >> 10.20.193.28:0/2164739299 conn(0x7f5bc40ed130 msgr2=0x7f5bc4112040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:46.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.579+0000 7f5bcba8e640 1 -- 10.20.193.28:0/2164739299 shutdown_connections 2026-02-25T11:46:46.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:46.579+0000 7f5bcba8e640 1 -- 10.20.193.28:0/2164739299 wait complete. 2026-02-25T11:46:46.679 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2026-02-25T11:44:43.266336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2026-02-25T11:46:46.679 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd pool get .mgr pg_num 2026-02-25T11:46:46.865 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:46:47.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.044+0000 7fbb676ab640 1 -- 10.20.193.28:0/4099319521 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbb601135c0 msgr2=0x7fbb60173f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:47.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.044+0000 7fbb676ab640 1 --2- 10.20.193.28:0/4099319521 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbb601135c0 0x7fbb60173f90 secure :-1 s=READY pgs=68 gs=2 cs=0 l=1 c_cookie=15131052796650162150 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbb5400aa30 tx=0x7fbb54030270 comp rx=0 tx=0).stop 2026-02-25T11:46:47.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.044+0000 7fbb676ab640 1 -- 10.20.193.28:0/4099319521 shutdown_connections 2026-02-25T11:46:47.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.044+0000 7fbb676ab640 1 --2- 10.20.193.28:0/4099319521 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbb601135c0 0x7fbb60173f90 unknown :-1 s=CLOSED pgs=68 gs=2 cs=0 l=1 c_cookie=15131052796650162150 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.044+0000 7fbb676ab640 1 --2- 10.20.193.28:0/4099319521 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbb6010ddc0 0x7fbb60112e10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.044+0000 7fbb676ab640 1 --2- 10.20.193.28:0/4099319521 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbb6010d3d0 0x7fbb6010d7a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.044+0000 7fbb676ab640 1 -- 10.20.193.28:0/4099319521 >> 10.20.193.28:0/4099319521 conn(0x7fbb600eaca0 msgr2=0x7fbb6010b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:47.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.044+0000 7fbb676ab640 1 -- 10.20.193.28:0/4099319521 shutdown_connections 2026-02-25T11:46:47.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.045+0000 7fbb676ab640 1 -- 10.20.193.28:0/4099319521 wait complete. 2026-02-25T11:46:47.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.045+0000 7fbb676ab640 1 Processor -- start 2026-02-25T11:46:47.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.045+0000 7fbb676ab640 1 -- start start 2026-02-25T11:46:47.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb676ab640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbb6010d3d0 0x7fbb600f5980 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:47.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb676ab640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbb6010ddc0 0x7fbb60173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:47.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb676ab640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbb601135c0 0x7fbb600f6080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:47.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb676ab640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbb600fb240 con 0x7fbb601135c0 2026-02-25T11:46:47.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb676ab640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbb600fb3b0 con 0x7fbb6010ddc0 2026-02-25T11:46:47.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb676ab640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbb600fb570 con 0x7fbb6010d3d0 2026-02-25T11:46:47.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb66eaa640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbb601135c0 0x7fbb600f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:47.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb65ea8640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbb6010ddc0 0x7fbb60173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:47.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb666a9640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbb6010d3d0 0x7fbb600f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:47.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb66eaa640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbb601135c0 0x7fbb600f6080 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:45546/0 (socket says 10.20.193.28:45546) 2026-02-25T11:46:47.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb66eaa640 1 -- 10.20.193.28:0/3056240727 learned_addr learned my addr 10.20.193.28:0/3056240727 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:46:47.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb66eaa640 1 -- 10.20.193.28:0/3056240727 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbb6010d3d0 msgr2=0x7fbb600f5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:47.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb66eaa640 1 --2- 10.20.193.28:0/3056240727 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbb6010d3d0 0x7fbb600f5980 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb66eaa640 1 -- 10.20.193.28:0/3056240727 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbb6010ddc0 msgr2=0x7fbb60173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:47.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb66eaa640 1 --2- 10.20.193.28:0/3056240727 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbb6010ddc0 0x7fbb60173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb66eaa640 1 -- 10.20.193.28:0/3056240727 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb580094b0 con 0x7fbb601135c0 2026-02-25T11:46:47.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb65ea8640 1 --2- 10.20.193.28:0/3056240727 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbb6010ddc0 0x7fbb60173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:46:47.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.046+0000 7fbb66eaa640 1 --2- 10.20.193.28:0/3056240727 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbb601135c0 0x7fbb600f6080 secure :-1 s=READY pgs=192 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbb54032fe0 tx=0x7fbb54030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:47.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.047+0000 7fbb4b7fe640 1 -- 10.20.193.28:0/3056240727 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbb5403e070 con 0x7fbb601135c0 2026-02-25T11:46:47.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.047+0000 7fbb676ab640 1 -- 10.20.193.28:0/3056240727 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5400a650 con 0x7fbb601135c0 2026-02-25T11:46:47.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.047+0000 7fbb4b7fe640 1 -- 10.20.193.28:0/3056240727 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fbb54039de0 con 0x7fbb601135c0 2026-02-25T11:46:47.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.047+0000 7fbb4b7fe640 1 -- 10.20.193.28:0/3056240727 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbb54042960 con 0x7fbb601135c0 2026-02-25T11:46:47.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.047+0000 7fbb676ab640 1 -- 10.20.193.28:0/3056240727 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb600fbad0 con 0x7fbb601135c0 2026-02-25T11:46:47.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.047+0000 7fbb676ab640 1 -- 10.20.193.28:0/3056240727 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb28005300 con 0x7fbb601135c0 2026-02-25T11:46:47.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.049+0000 7fbb4b7fe640 1 -- 10.20.193.28:0/3056240727 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7fbb540050c0 con 0x7fbb601135c0 2026-02-25T11:46:47.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.051+0000 7fbb4b7fe640 1 --2- 10.20.193.28:0/3056240727 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbb3c0b5750 0x7fbb3c0d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:47.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.051+0000 7fbb4b7fe640 1 -- 10.20.193.28:0/3056240727 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7fbb5410c5c0 con 0x7fbb601135c0 2026-02-25T11:46:47.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.051+0000 7fbb4b7fe640 1 -- 10.20.193.28:0/3056240727 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fbb5410ca40 con 0x7fbb601135c0 2026-02-25T11:46:47.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.051+0000 7fbb666a9640 1 --2- 10.20.193.28:0/3056240727 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbb3c0b5750 0x7fbb3c0d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:47.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.052+0000 7fbb666a9640 1 --2- 10.20.193.28:0/3056240727 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbb3c0b5750 0x7fbb3c0d5c00 secure :-1 s=READY pgs=145 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbb58004740 tx=0x7fbb58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:47.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.172+0000 7fbb676ab640 1 -- 10.20.193.28:0/3056240727 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb28005b30 con 0x7fbb601135c0 2026-02-25T11:46:47.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.173+0000 7fbb4b7fe640 1 -- 10.20.193.28:0/3056240727 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fbb540cf480 con 0x7fbb601135c0 2026-02-25T11:46:47.175 INFO:teuthology.orchestra.run.trial028.stdout:pg_num: 1 2026-02-25T11:46:47.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.174+0000 7fbb676ab640 1 -- 10.20.193.28:0/3056240727 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbb3c0b5750 msgr2=0x7fbb3c0d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:47.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.174+0000 7fbb676ab640 1 --2- 10.20.193.28:0/3056240727 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbb3c0b5750 0x7fbb3c0d5c00 secure :-1 s=READY pgs=145 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbb58004740 tx=0x7fbb58005f90 comp rx=0 tx=0).stop 2026-02-25T11:46:47.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.174+0000 7fbb676ab640 1 -- 10.20.193.28:0/3056240727 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbb601135c0 msgr2=0x7fbb600f6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:47.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.174+0000 7fbb676ab640 1 --2- 10.20.193.28:0/3056240727 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbb601135c0 0x7fbb600f6080 secure :-1 s=READY pgs=192 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbb54032fe0 tx=0x7fbb54030d00 comp rx=0 tx=0).stop 2026-02-25T11:46:47.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.174+0000 7fbb676ab640 1 -- 10.20.193.28:0/3056240727 shutdown_connections 2026-02-25T11:46:47.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.174+0000 7fbb676ab640 1 --2- 10.20.193.28:0/3056240727 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbb3c0b5750 0x7fbb3c0d5c00 unknown :-1 s=CLOSED pgs=145 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.174+0000 7fbb676ab640 1 --2- 10.20.193.28:0/3056240727 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbb601135c0 0x7fbb600f6080 unknown :-1 s=CLOSED pgs=192 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.174+0000 7fbb676ab640 1 --2- 10.20.193.28:0/3056240727 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbb6010ddc0 0x7fbb60173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.174+0000 7fbb676ab640 1 --2- 10.20.193.28:0/3056240727 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbb6010d3d0 0x7fbb600f5980 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.174+0000 7fbb676ab640 1 -- 10.20.193.28:0/3056240727 >> 10.20.193.28:0/3056240727 conn(0x7fbb600eaca0 msgr2=0x7fbb6010b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:47.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.175+0000 7fbb676ab640 1 -- 10.20.193.28:0/3056240727 shutdown_connections 2026-02-25T11:46:47.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:47.175+0000 7fbb676ab640 1 -- 10.20.193.28:0/3056240727 wait complete. 2026-02-25T11:46:47.276 INFO:tasks.cephadm:Adding mds.a on trial028 2026-02-25T11:46:47.276 INFO:tasks.cephadm:Adding mds.d on trial028 2026-02-25T11:46:47.277 INFO:tasks.cephadm:Adding mds.g on trial028 2026-02-25T11:46:47.277 INFO:tasks.cephadm:Adding mds.j on trial028 2026-02-25T11:46:47.277 INFO:tasks.cephadm:Adding mds.b on trial119 2026-02-25T11:46:47.277 INFO:tasks.cephadm:Adding mds.e on trial119 2026-02-25T11:46:47.277 INFO:tasks.cephadm:Adding mds.h on trial119 2026-02-25T11:46:47.277 INFO:tasks.cephadm:Adding mds.k on trial119 2026-02-25T11:46:47.277 INFO:tasks.cephadm:Adding mds.c on trial167 2026-02-25T11:46:47.277 INFO:tasks.cephadm:Adding mds.f on trial167 2026-02-25T11:46:47.277 INFO:tasks.cephadm:Adding mds.i on trial167 2026-02-25T11:46:47.277 INFO:tasks.cephadm:Adding mds.l on trial167 2026-02-25T11:46:47.278 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph orch apply mds all '12;trial028=a;trial028=d;trial028=g;trial028=j;trial119=b;trial119=e;trial119=h;trial119=k;trial167=c;trial167=f;trial167=i;trial167=l' 2026-02-25T11:46:47.296 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:47 trial167 ceph-mon[43392]: osdmap e82: 12 total, 12 up, 12 in 2026-02-25T11:46:47.296 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:47 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2164739299' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:46:47.296 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:47 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3056240727' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2026-02-25T11:46:47.475 INFO:teuthology.orchestra.run.trial167.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.c/config 2026-02-25T11:46:47.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:47 trial119 ceph-mon[42710]: osdmap e82: 12 total, 12 up, 12 in 2026-02-25T11:46:47.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:47 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2164739299' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:46:47.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:47 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3056240727' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2026-02-25T11:46:47.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:47 trial028 ceph-mon[39224]: osdmap e82: 12 total, 12 up, 12 in 2026-02-25T11:46:47.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:47 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2164739299' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:46:47.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:47 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3056240727' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2026-02-25T11:46:47.646 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.644+0000 7fb68ca4c640 1 -- 10.20.193.167:0/1716938405 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb688060810 msgr2=0x7fb68810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:47.647 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.644+0000 7fb68ca4c640 1 --2- 10.20.193.167:0/1716938405 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb688060810 0x7fb68810b9a0 secure :-1 s=READY pgs=69 gs=1 cs=0 l=1 c_cookie=3493576318121654094 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb674009950 tx=0x7fb67402f1d0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.647 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.644+0000 7fb68ca4c640 1 -- 10.20.193.167:0/1716938405 shutdown_connections 2026-02-25T11:46:47.647 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.644+0000 7fb68ca4c640 1 --2- 10.20.193.167:0/1716938405 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb68810c890 0x7fb68810cd30 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.647 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.644+0000 7fb68ca4c640 1 --2- 10.20.193.167:0/1716938405 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb68810bee0 0x7fb68810c350 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.647 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.644+0000 7fb68ca4c640 1 --2- 10.20.193.167:0/1716938405 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb688060810 0x7fb68810b9a0 unknown :-1 s=CLOSED pgs=69 gs=1 cs=0 l=1 c_cookie=3493576318121654094 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.647 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.644+0000 7fb68ca4c640 1 -- 10.20.193.167:0/1716938405 >> 10.20.193.167:0/1716938405 conn(0x7fb6880e9580 msgr2=0x7fb688109990 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:47.647 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.644+0000 7fb68ca4c640 1 -- 10.20.193.167:0/1716938405 shutdown_connections 2026-02-25T11:46:47.647 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.644+0000 7fb68ca4c640 1 -- 10.20.193.167:0/1716938405 wait complete. 2026-02-25T11:46:47.649 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.645+0000 7fb68ca4c640 1 Processor -- start 2026-02-25T11:46:47.649 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.645+0000 7fb68ca4c640 1 -- start start 2026-02-25T11:46:47.650 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb68ca4c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb688060810 0x7fb68810e010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:47.650 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb68ca4c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb68810bee0 0x7fb68810e660 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:47.650 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb68ca4c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb68810c890 0x7fb68810f260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:47.650 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb68ca4c640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb68810f9a0 con 0x7fb68810c890 2026-02-25T11:46:47.650 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb68ca4c640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb68810fb10 con 0x7fb68810bee0 2026-02-25T11:46:47.650 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb68ca4c640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb68810fcd0 con 0x7fb688060810 2026-02-25T11:46:47.650 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb6877fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb688060810 0x7fb68810e010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:47.650 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb6877fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb688060810 0x7fb68810e010 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.167:55682/0 (socket says 10.20.193.167:55682) 2026-02-25T11:46:47.650 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb6877fe640 1 -- 10.20.193.167:0/2550868705 learned_addr learned my addr 10.20.193.167:0/2550868705 (peer_addr_for_me v2:10.20.193.167:0/0) 2026-02-25T11:46:47.650 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb6877fe640 1 -- 10.20.193.167:0/2550868705 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb68810bee0 msgr2=0x7fb68810e660 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-25T11:46:47.650 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb6877fe640 1 --2- 10.20.193.167:0/2550868705 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb68810bee0 0x7fb68810e660 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.651 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb6877fe640 1 -- 10.20.193.167:0/2550868705 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb68810c890 msgr2=0x7fb68810f260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-25T11:46:47.651 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb6877fe640 1 --2- 10.20.193.167:0/2550868705 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb68810c890 0x7fb68810f260 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.651 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.646+0000 7fb6877fe640 1 -- 10.20.193.167:0/2550868705 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb674009570 con 0x7fb688060810 2026-02-25T11:46:47.651 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.647+0000 7fb6877fe640 1 --2- 10.20.193.167:0/2550868705 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb688060810 0x7fb68810e010 secure :-1 s=READY pgs=70 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb6740098a0 tx=0x7fb674002a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:47.651 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.647+0000 7fb684ff9640 1 -- 10.20.193.167:0/2550868705 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb67403d070 con 0x7fb688060810 2026-02-25T11:46:47.651 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.647+0000 7fb684ff9640 1 -- 10.20.193.167:0/2550868705 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb674038ac0 con 0x7fb688060810 2026-02-25T11:46:47.651 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.647+0000 7fb684ff9640 1 -- 10.20.193.167:0/2550868705 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb674041ac0 con 0x7fb688060810 2026-02-25T11:46:47.651 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.647+0000 7fb68ca4c640 1 -- 10.20.193.167:0/2550868705 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb68810ff50 con 0x7fb688060810 2026-02-25T11:46:47.651 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.647+0000 7fb68ca4c640 1 -- 10.20.193.167:0/2550868705 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb688110470 con 0x7fb688060810 2026-02-25T11:46:47.651 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.647+0000 7fb68ca4c640 1 -- 10.20.193.167:0/2550868705 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb688175710 con 0x7fb688060810 2026-02-25T11:46:47.653 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.650+0000 7fb684ff9640 1 -- 10.20.193.167:0/2550868705 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7fb674038c30 con 0x7fb688060810 2026-02-25T11:46:47.653 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.651+0000 7fb684ff9640 1 --2- 10.20.193.167:0/2550868705 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb65c0b5760 0x7fb65c0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:47.657 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.651+0000 7fb684ff9640 1 -- 10.20.193.167:0/2550868705 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7fb67410bfb0 con 0x7fb688060810 2026-02-25T11:46:47.657 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.652+0000 7fb684ff9640 1 -- 10.20.193.167:0/2550868705 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fb6740d2150 con 0x7fb688060810 2026-02-25T11:46:47.657 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.655+0000 7fb686ffd640 1 --2- 10.20.193.167:0/2550868705 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb65c0b5760 0x7fb65c0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:47.658 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.656+0000 7fb686ffd640 1 --2- 10.20.193.167:0/2550868705 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb65c0b5760 0x7fb65c0d5c10 secure :-1 s=READY pgs=146 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb678004760 tx=0x7fb678006c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:47.778 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.776+0000 7fb68ca4c640 1 -- 10.20.193.167:0/2550868705 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;trial028=a;trial028=d;trial028=g;trial028=j;trial119=b;trial119=e;trial119=h;trial119=k;trial167=c;trial167=f;trial167=i;trial167=l", "target": ["mon-mgr", ""]}) -- 0x7fb688062180 con 0x7fb65c0b5760 2026-02-25T11:46:47.785 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.783+0000 7fb684ff9640 1 -- 10.20.193.167:0/2550868705 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fb688062180 con 0x7fb65c0b5760 2026-02-25T11:46:47.786 INFO:teuthology.orchestra.run.trial167.stdout:Scheduled mds.all update... 2026-02-25T11:46:47.788 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.785+0000 7fb68ca4c640 1 -- 10.20.193.167:0/2550868705 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb65c0b5760 msgr2=0x7fb65c0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:47.788 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.785+0000 7fb68ca4c640 1 --2- 10.20.193.167:0/2550868705 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb65c0b5760 0x7fb65c0d5c10 secure :-1 s=READY pgs=146 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb678004760 tx=0x7fb678006c80 comp rx=0 tx=0).stop 2026-02-25T11:46:47.788 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.785+0000 7fb68ca4c640 1 -- 10.20.193.167:0/2550868705 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb688060810 msgr2=0x7fb68810e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:47.788 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.785+0000 7fb68ca4c640 1 --2- 10.20.193.167:0/2550868705 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb688060810 0x7fb68810e010 secure :-1 s=READY pgs=70 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb6740098a0 tx=0x7fb674002a10 comp rx=0 tx=0).stop 2026-02-25T11:46:47.788 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.785+0000 7fb68ca4c640 1 -- 10.20.193.167:0/2550868705 shutdown_connections 2026-02-25T11:46:47.788 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.785+0000 7fb68ca4c640 1 --2- 10.20.193.167:0/2550868705 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb65c0b5760 0x7fb65c0d5c10 unknown :-1 s=CLOSED pgs=146 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.788 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.785+0000 7fb68ca4c640 1 --2- 10.20.193.167:0/2550868705 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb68810c890 0x7fb68810f260 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.788 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.785+0000 7fb68ca4c640 1 --2- 10.20.193.167:0/2550868705 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb68810bee0 0x7fb68810e660 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.789 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.785+0000 7fb68ca4c640 1 --2- 10.20.193.167:0/2550868705 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb688060810 0x7fb68810e010 unknown :-1 s=CLOSED pgs=70 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:47.789 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.785+0000 7fb68ca4c640 1 -- 10.20.193.167:0/2550868705 >> 10.20.193.167:0/2550868705 conn(0x7fb6880e9580 msgr2=0x7fb6880eb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:47.789 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.786+0000 7fb68ca4c640 1 -- 10.20.193.167:0/2550868705 shutdown_connections 2026-02-25T11:46:47.789 INFO:teuthology.orchestra.run.trial167.stderr:2026-02-25T11:46:47.786+0000 7fb68ca4c640 1 -- 10.20.193.167:0/2550868705 wait complete. 2026-02-25T11:46:47.888 DEBUG:teuthology.orchestra.run.trial028:mds.a> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.a.service 2026-02-25T11:46:47.892 DEBUG:teuthology.orchestra.run.trial028:mds.d> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.d.service 2026-02-25T11:46:47.937 DEBUG:teuthology.orchestra.run.trial028:mds.g> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.g.service 2026-02-25T11:46:47.982 DEBUG:teuthology.orchestra.run.trial028:mds.j> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.j.service 2026-02-25T11:46:48.028 DEBUG:teuthology.orchestra.run.trial119:mds.b> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.b.service 2026-02-25T11:46:48.034 DEBUG:teuthology.orchestra.run.trial119:mds.e> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.e.service 2026-02-25T11:46:48.078 DEBUG:teuthology.orchestra.run.trial119:mds.h> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.h.service 2026-02-25T11:46:48.122 DEBUG:teuthology.orchestra.run.trial119:mds.k> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.k.service 2026-02-25T11:46:48.166 DEBUG:teuthology.orchestra.run.trial167:mds.c> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.c.service 2026-02-25T11:46:48.170 DEBUG:teuthology.orchestra.run.trial167:mds.f> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.f.service 2026-02-25T11:46:48.214 DEBUG:teuthology.orchestra.run.trial167:mds.i> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.i.service 2026-02-25T11:46:48.258 DEBUG:teuthology.orchestra.run.trial167:mds.l> sudo journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.l.service 2026-02-25T11:46:48.302 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2026-02-25T11:46:48.302 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:46:48.349 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:48 trial167 ceph-mon[43392]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:48.349 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:48 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:48.486 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:46:48.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:48 trial119 ceph-mon[42710]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:48.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:48 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:48.529 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:48 trial028 ceph-mon[39224]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:48.529 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:48 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:48.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.653+0000 7fd8f7337640 1 -- 10.20.193.28:0/4232438318 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd8f01135c0 msgr2=0x7fd8f0173f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:48.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.653+0000 7fd8f7337640 1 --2- 10.20.193.28:0/4232438318 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd8f01135c0 0x7fd8f0173f90 secure :-1 s=READY pgs=193 gs=2 cs=0 l=1 c_cookie=4321815361431030575 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd8ec00aa30 tx=0x7fd8ec030270 comp rx=0 tx=0).stop 2026-02-25T11:46:48.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.653+0000 7fd8f7337640 1 -- 10.20.193.28:0/4232438318 shutdown_connections 2026-02-25T11:46:48.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.653+0000 7fd8f7337640 1 --2- 10.20.193.28:0/4232438318 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd8f01135c0 0x7fd8f0173f90 unknown :-1 s=CLOSED pgs=193 gs=2 cs=0 l=1 c_cookie=4321815361431030575 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:48.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.653+0000 7fd8f7337640 1 --2- 10.20.193.28:0/4232438318 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd8f010ddc0 0x7fd8f0112e10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:48.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.653+0000 7fd8f7337640 1 --2- 10.20.193.28:0/4232438318 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd8f010d3d0 0x7fd8f010d7a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:48.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.653+0000 7fd8f7337640 1 -- 10.20.193.28:0/4232438318 >> 10.20.193.28:0/4232438318 conn(0x7fd8f00eaca0 msgr2=0x7fd8f010b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:48.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.653+0000 7fd8f7337640 1 -- 10.20.193.28:0/4232438318 shutdown_connections 2026-02-25T11:46:48.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.653+0000 7fd8f7337640 1 -- 10.20.193.28:0/4232438318 wait complete. 2026-02-25T11:46:48.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.653+0000 7fd8f7337640 1 Processor -- start 2026-02-25T11:46:48.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.654+0000 7fd8f7337640 1 -- start start 2026-02-25T11:46:48.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.654+0000 7fd8f7337640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd8f010d3d0 0x7fd8f00f59b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:48.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.654+0000 7fd8f7337640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd8f010ddc0 0x7fd8f0173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:48.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.654+0000 7fd8f7337640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd8f01135c0 0x7fd8f00f60b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:48.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.654+0000 7fd8f7337640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f00fb2c0 con 0x7fd8f010d3d0 2026-02-25T11:46:48.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.654+0000 7fd8f7337640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f00fb430 con 0x7fd8f010ddc0 2026-02-25T11:46:48.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.654+0000 7fd8f7337640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f00fb5f0 con 0x7fd8f01135c0 2026-02-25T11:46:48.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.654+0000 7fd8f6335640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd8f010d3d0 0x7fd8f00f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:48.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.654+0000 7fd8f6b36640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd8f01135c0 0x7fd8f00f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:48.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.654+0000 7fd8f6b36640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd8f01135c0 0x7fd8f00f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:43006/0 (socket says 10.20.193.28:43006) 2026-02-25T11:46:48.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.654+0000 7fd8f5b34640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd8f010ddc0 0x7fd8f0173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:48.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.654+0000 7fd8f6b36640 1 -- 10.20.193.28:0/3503267296 learned_addr learned my addr 10.20.193.28:0/3503267296 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:46:48.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.655+0000 7fd8f6b36640 1 -- 10.20.193.28:0/3503267296 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd8f010ddc0 msgr2=0x7fd8f0173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:48.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.655+0000 7fd8f6b36640 1 --2- 10.20.193.28:0/3503267296 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd8f010ddc0 0x7fd8f0173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:48.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.655+0000 7fd8f6b36640 1 -- 10.20.193.28:0/3503267296 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd8f010d3d0 msgr2=0x7fd8f00f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:48.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.655+0000 7fd8f6b36640 1 --2- 10.20.193.28:0/3503267296 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd8f010d3d0 0x7fd8f00f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:48.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.655+0000 7fd8f6b36640 1 -- 10.20.193.28:0/3503267296 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec00a650 con 0x7fd8f01135c0 2026-02-25T11:46:48.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.655+0000 7fd8f6b36640 1 --2- 10.20.193.28:0/3503267296 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd8f01135c0 0x7fd8f00f60b0 secure :-1 s=READY pgs=71 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd8ec032fe0 tx=0x7fd8ec030d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:48.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.655+0000 7fd8d77fe640 1 -- 10.20.193.28:0/3503267296 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd8ec03e070 con 0x7fd8f01135c0 2026-02-25T11:46:48.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.655+0000 7fd8f7337640 1 -- 10.20.193.28:0/3503267296 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f00fb870 con 0x7fd8f01135c0 2026-02-25T11:46:48.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.655+0000 7fd8d77fe640 1 -- 10.20.193.28:0/3503267296 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd8ec039de0 con 0x7fd8f01135c0 2026-02-25T11:46:48.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.655+0000 7fd8d77fe640 1 -- 10.20.193.28:0/3503267296 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd8ec042960 con 0x7fd8f01135c0 2026-02-25T11:46:48.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.655+0000 7fd8f7337640 1 -- 10.20.193.28:0/3503267296 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f00fbd30 con 0x7fd8f01135c0 2026-02-25T11:46:48.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.656+0000 7fd8d57fa640 1 -- 10.20.193.28:0/3503267296 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f010e6b0 con 0x7fd8f01135c0 2026-02-25T11:46:48.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.658+0000 7fd8d77fe640 1 -- 10.20.193.28:0/3503267296 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7fd8ec0050c0 con 0x7fd8f01135c0 2026-02-25T11:46:48.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.660+0000 7fd8d77fe640 1 --2- 10.20.193.28:0/3503267296 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd8cc0b5750 0x7fd8cc0d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:48.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.660+0000 7fd8d77fe640 1 -- 10.20.193.28:0/3503267296 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7fd8ec10c440 con 0x7fd8f01135c0 2026-02-25T11:46:48.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.660+0000 7fd8d77fe640 1 -- 10.20.193.28:0/3503267296 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fd8ec10c8c0 con 0x7fd8f01135c0 2026-02-25T11:46:48.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.660+0000 7fd8f6335640 1 --2- 10.20.193.28:0/3503267296 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd8cc0b5750 0x7fd8cc0d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:48.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.661+0000 7fd8f6335640 1 --2- 10.20.193.28:0/3503267296 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd8cc0b5750 0x7fd8cc0d5c00 secure :-1 s=READY pgs=147 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd8e0004740 tx=0x7fd8e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:48.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.802+0000 7fd8d57fa640 1 -- 10.20.193.28:0/3503267296 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8f00f6b10 con 0x7fd8f01135c0 2026-02-25T11:46:48.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.802+0000 7fd8d77fe640 1 -- 10.20.193.28:0/3503267296 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fd8ec0cf300 con 0x7fd8f01135c0 2026-02-25T11:46:48.805 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:46:48.805 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":1,"btime":"2026-02-25T11:42:51:962673+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2026-02-25T11:46:48.806 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 1 2026-02-25T11:46:48.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.804+0000 7fd8d57fa640 1 -- 10.20.193.28:0/3503267296 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd8cc0b5750 msgr2=0x7fd8cc0d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:48.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.804+0000 7fd8d57fa640 1 --2- 10.20.193.28:0/3503267296 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd8cc0b5750 0x7fd8cc0d5c00 secure :-1 s=READY pgs=147 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd8e0004740 tx=0x7fd8e0005f90 comp rx=0 tx=0).stop 2026-02-25T11:46:48.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.804+0000 7fd8d57fa640 1 -- 10.20.193.28:0/3503267296 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd8f01135c0 msgr2=0x7fd8f00f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:48.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.804+0000 7fd8d57fa640 1 --2- 10.20.193.28:0/3503267296 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd8f01135c0 0x7fd8f00f60b0 secure :-1 s=READY pgs=71 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd8ec032fe0 tx=0x7fd8ec030d00 comp rx=0 tx=0).stop 2026-02-25T11:46:48.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.804+0000 7fd8d57fa640 1 -- 10.20.193.28:0/3503267296 shutdown_connections 2026-02-25T11:46:48.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.804+0000 7fd8d57fa640 1 --2- 10.20.193.28:0/3503267296 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd8cc0b5750 0x7fd8cc0d5c00 unknown :-1 s=CLOSED pgs=147 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:48.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.804+0000 7fd8d57fa640 1 --2- 10.20.193.28:0/3503267296 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd8f01135c0 0x7fd8f00f60b0 unknown :-1 s=CLOSED pgs=71 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:48.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.804+0000 7fd8d57fa640 1 --2- 10.20.193.28:0/3503267296 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd8f010ddc0 0x7fd8f0173e90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:48.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.804+0000 7fd8d57fa640 1 --2- 10.20.193.28:0/3503267296 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd8f010d3d0 0x7fd8f00f59b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:48.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.804+0000 7fd8d57fa640 1 -- 10.20.193.28:0/3503267296 >> 10.20.193.28:0/3503267296 conn(0x7fd8f00eaca0 msgr2=0x7fd8f010b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:48.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.804+0000 7fd8d57fa640 1 -- 10.20.193.28:0/3503267296 shutdown_connections 2026-02-25T11:46:48.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:48.804+0000 7fd8d57fa640 1 -- 10.20.193.28:0/3503267296 wait complete. 2026-02-25T11:46:49.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:49 trial167 ceph-mon[43392]: from='client.24658 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;trial028=a;trial028=d;trial028=g;trial028=j;trial119=b;trial119=e;trial119=h;trial119=k;trial167=c;trial167=f;trial167=i;trial167=l", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:46:49.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:49 trial167 ceph-mon[43392]: Saving service mds.all spec with placement trial028=a;trial028=d;trial028=g;trial028=j;trial119=b;trial119=e;trial119=h;trial119=k;trial167=c;trial167=f;trial167=i;trial167=l;count:12 2026-02-25T11:46:49.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:49 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3503267296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:46:49.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:49.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:49.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:49 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:49.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:49 trial119 ceph-mon[42710]: from='client.24658 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;trial028=a;trial028=d;trial028=g;trial028=j;trial119=b;trial119=e;trial119=h;trial119=k;trial167=c;trial167=f;trial167=i;trial167=l", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:46:49.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:49 trial119 ceph-mon[42710]: Saving service mds.all spec with placement trial028=a;trial028=d;trial028=g;trial028=j;trial119=b;trial119=e;trial119=h;trial119=k;trial167=c;trial167=f;trial167=i;trial167=l;count:12 2026-02-25T11:46:49.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:49 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3503267296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:46:49.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:49.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:49.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:49 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:49.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:49 trial028 ceph-mon[39224]: from='client.24658 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;trial028=a;trial028=d;trial028=g;trial028=j;trial119=b;trial119=e;trial119=h;trial119=k;trial167=c;trial167=f;trial167=i;trial167=l", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:46:49.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:49 trial028 ceph-mon[39224]: Saving service mds.all spec with placement trial028=a;trial028=d;trial028=g;trial028=j;trial119=b;trial119=e;trial119=h;trial119=k;trial167=c;trial167=f;trial167=i;trial167=l;count:12 2026-02-25T11:46:49.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:49 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3503267296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:46:49.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:49.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:49.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:49 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:46:50.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:50 trial167 ceph-mon[43392]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:50.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:50.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:50 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:50.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:50 trial119 ceph-mon[42710]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:50.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:50.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:50 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:50.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:50 trial028 ceph-mon[39224]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:50.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:50.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:50 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:50.910 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:46:51.095 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:46:51.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.272+0000 7ffbb3506640 1 -- 10.20.193.28:0/2464306370 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffbac10f690 msgr2=0x7ffbac10fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:51.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.272+0000 7ffbb3506640 1 --2- 10.20.193.28:0/2464306370 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffbac10f690 0x7ffbac10fa60 secure :-1 s=READY pgs=194 gs=1 cs=0 l=1 c_cookie=7593899683269455454 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb9c009950 tx=0x7ffb9c02f190 comp rx=0 tx=0).stop 2026-02-25T11:46:51.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.272+0000 7ffbb3506640 1 -- 10.20.193.28:0/2464306370 shutdown_connections 2026-02-25T11:46:51.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.272+0000 7ffbb3506640 1 --2- 10.20.193.28:0/2464306370 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffbac115880 0x7ffbac176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:51.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.272+0000 7ffbb3506640 1 --2- 10.20.193.28:0/2464306370 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffbac110080 0x7ffbac1150d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:51.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.272+0000 7ffbb3506640 1 --2- 10.20.193.28:0/2464306370 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffbac10f690 0x7ffbac10fa60 unknown :-1 s=CLOSED pgs=194 gs=1 cs=0 l=1 c_cookie=7593899683269455454 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:51.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.272+0000 7ffbb3506640 1 -- 10.20.193.28:0/2464306370 >> 10.20.193.28:0/2464306370 conn(0x7ffbac0ecf00 msgr2=0x7ffbac10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:51.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.272+0000 7ffbb3506640 1 -- 10.20.193.28:0/2464306370 shutdown_connections 2026-02-25T11:46:51.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.272+0000 7ffbb3506640 1 -- 10.20.193.28:0/2464306370 wait complete. 2026-02-25T11:46:51.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.273+0000 7ffbb3506640 1 Processor -- start 2026-02-25T11:46:51.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.273+0000 7ffbb3506640 1 -- start start 2026-02-25T11:46:51.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.273+0000 7ffbb3506640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffbac10f690 0x7ffbac0f5960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:51.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.273+0000 7ffbb3506640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffbac110080 0x7ffbac176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:51.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.273+0000 7ffbb3506640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffbac115880 0x7ffbac0f6060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:51.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.273+0000 7ffbb3506640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac0fb270 con 0x7ffbac10f690 2026-02-25T11:46:51.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.273+0000 7ffbb3506640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac0fb3e0 con 0x7ffbac110080 2026-02-25T11:46:51.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.273+0000 7ffbb3506640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffbac0fb5a0 con 0x7ffbac115880 2026-02-25T11:46:51.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.273+0000 7ffbb2504640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffbac10f690 0x7ffbac0f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:51.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffbb2504640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffbac10f690 0x7ffbac0f5960 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:45582/0 (socket says 10.20.193.28:45582) 2026-02-25T11:46:51.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffbb2504640 1 -- 10.20.193.28:0/3058675568 learned_addr learned my addr 10.20.193.28:0/3058675568 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:46:51.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffbb2504640 1 -- 10.20.193.28:0/3058675568 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffbac115880 msgr2=0x7ffbac0f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:51.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffbb2504640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffbac115880 0x7ffbac0f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:51.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffbb2504640 1 -- 10.20.193.28:0/3058675568 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffbac110080 msgr2=0x7ffbac176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:51.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffbb1d03640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffbac110080 0x7ffbac176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:51.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffbb2504640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffbac110080 0x7ffbac176150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:51.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffbb2504640 1 -- 10.20.193.28:0/3058675568 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c009570 con 0x7ffbac10f690 2026-02-25T11:46:51.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffbb1d03640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffbac110080 0x7ffbac176150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:46:51.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffbb2504640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffbac10f690 0x7ffbac0f5960 secure :-1 s=READY pgs=195 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb9c0098a0 tx=0x7ffb9c0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:51.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffb937fe640 1 -- 10.20.193.28:0/3058675568 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb9c03d070 con 0x7ffbac10f690 2026-02-25T11:46:51.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffb937fe640 1 -- 10.20.193.28:0/3058675568 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ffb9c002a70 con 0x7ffbac10f690 2026-02-25T11:46:51.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffb937fe640 1 -- 10.20.193.28:0/3058675568 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb9c041890 con 0x7ffbac10f690 2026-02-25T11:46:51.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffbb3506640 1 -- 10.20.193.28:0/3058675568 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbac0fb820 con 0x7ffbac10f690 2026-02-25T11:46:51.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.274+0000 7ffbb3506640 1 -- 10.20.193.28:0/3058675568 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbac0fbc40 con 0x7ffbac10f690 2026-02-25T11:46:51.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.275+0000 7ffbb3506640 1 -- 10.20.193.28:0/3058675568 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbac110970 con 0x7ffbac10f690 2026-02-25T11:46:51.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.277+0000 7ffb937fe640 1 -- 10.20.193.28:0/3058675568 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7ffb9c041a20 con 0x7ffbac10f690 2026-02-25T11:46:51.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.279+0000 7ffb937fe640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffb880b5750 0x7ffb880d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:51.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.279+0000 7ffb937fe640 1 -- 10.20.193.28:0/3058675568 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7ffb9c10b0d0 con 0x7ffbac10f690 2026-02-25T11:46:51.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.279+0000 7ffb937fe640 1 -- 10.20.193.28:0/3058675568 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7ffb9c10b550 con 0x7ffbac10f690 2026-02-25T11:46:51.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.279+0000 7ffbb1d03640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffb880b5750 0x7ffb880d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:51.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.280+0000 7ffbb1d03640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffb880b5750 0x7ffb880d5c00 secure :-1 s=READY pgs=148 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffbac0fc300 tx=0x7ffba0005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:51.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.422+0000 7ffbb3506640 1 -- 10.20.193.28:0/3058675568 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbac10fa60 con 0x7ffbac10f690 2026-02-25T11:46:51.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.423+0000 7ffb937fe640 1 -- 10.20.193.28:0/3058675568 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7ffb9c0cdee0 con 0x7ffbac10f690 2026-02-25T11:46:51.424 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:46:51.424 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":1,"btime":"2026-02-25T11:42:51:962673+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2026-02-25T11:46:51.424 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 1 2026-02-25T11:46:51.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.424+0000 7ffbb3506640 1 -- 10.20.193.28:0/3058675568 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffb880b5750 msgr2=0x7ffb880d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:51.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.424+0000 7ffbb3506640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffb880b5750 0x7ffb880d5c00 secure :-1 s=READY pgs=148 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffbac0fc300 tx=0x7ffba0005f00 comp rx=0 tx=0).stop 2026-02-25T11:46:51.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.424+0000 7ffbb3506640 1 -- 10.20.193.28:0/3058675568 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffbac10f690 msgr2=0x7ffbac0f5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:51.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.424+0000 7ffbb3506640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffbac10f690 0x7ffbac0f5960 secure :-1 s=READY pgs=195 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb9c0098a0 tx=0x7ffb9c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:46:51.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.424+0000 7ffbb3506640 1 -- 10.20.193.28:0/3058675568 shutdown_connections 2026-02-25T11:46:51.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.424+0000 7ffbb3506640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffb880b5750 0x7ffb880d5c00 unknown :-1 s=CLOSED pgs=148 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:51.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.424+0000 7ffbb3506640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffbac115880 0x7ffbac0f6060 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:51.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.424+0000 7ffbb3506640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffbac110080 0x7ffbac176150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:51.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.424+0000 7ffbb3506640 1 --2- 10.20.193.28:0/3058675568 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffbac10f690 0x7ffbac0f5960 unknown :-1 s=CLOSED pgs=195 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:51.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.424+0000 7ffbb3506640 1 -- 10.20.193.28:0/3058675568 >> 10.20.193.28:0/3058675568 conn(0x7ffbac0ecf00 msgr2=0x7ffbac111b30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:51.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.424+0000 7ffbb3506640 1 -- 10.20.193.28:0/3058675568 shutdown_connections 2026-02-25T11:46:51.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:51.424+0000 7ffbb3506640 1 -- 10.20.193.28:0/3058675568 wait complete. 2026-02-25T11:46:52.422 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:52 trial167 ceph-mon[43392]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:52.422 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:52 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3058675568' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:46:52.422 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:52.422 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:52.422 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:52.422 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:52.423 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:52.423 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:52.423 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:52.423 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:52.423 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:52 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:52 trial119 ceph-mon[42710]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:52 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3058675568' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:46:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:52.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:52 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:52.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:52 trial028 ceph-mon[39224]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:52.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:52 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3058675568' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:46:52.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:52.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:52.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:52.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:46:52.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:52.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:46:52.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:52.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:52.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:52 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:53.250 INFO:journalctl@ceph.mds.l.trial167.stdout:Feb 25 11:46:53 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-l[97423]: starting mds.l at 2026-02-25T11:46:53.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:53 trial119 ceph-mon[42710]: Deploying daemon mds.l on trial167 2026-02-25T11:46:53.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:53 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.167:6832/478500593,v1:10.20.193.167:6833/478500593] up:boot 2026-02-25T11:46:53.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:53 trial119 ceph-mon[42710]: fsmap 1 up:standby 2026-02-25T11:46:53.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2026-02-25T11:46:53.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:53.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:53.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:53.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:53.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:53.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:53 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:53.523 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:46:53.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:53 trial028 ceph-mon[39224]: Deploying daemon mds.l on trial167 2026-02-25T11:46:53.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:53 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.167:6832/478500593,v1:10.20.193.167:6833/478500593] up:boot 2026-02-25T11:46:53.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:53 trial028 ceph-mon[39224]: fsmap 1 up:standby 2026-02-25T11:46:53.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2026-02-25T11:46:53.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:53.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:53.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:53.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:53.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:53.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:53 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:53.708 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:46:53.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:53 trial167 ceph-mon[43392]: Deploying daemon mds.l on trial167 2026-02-25T11:46:53.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:53 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.167:6832/478500593,v1:10.20.193.167:6833/478500593] up:boot 2026-02-25T11:46:53.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:53 trial167 ceph-mon[43392]: fsmap 1 up:standby 2026-02-25T11:46:53.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2026-02-25T11:46:53.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:53.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:53.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:53.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:53.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:53.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:53 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:53.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.872+0000 7f735134f640 1 -- 10.20.193.28:0/1555684604 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f734c10b860 msgr2=0x7f734c10bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:53.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.872+0000 7f735134f640 1 --2- 10.20.193.28:0/1555684604 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f734c10b860 0x7f734c10bc30 secure :-1 s=READY pgs=196 gs=1 cs=0 l=1 c_cookie=8668216566484783148 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7338009950 tx=0x7f733802f190 comp rx=0 tx=0).stop 2026-02-25T11:46:53.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.872+0000 7f735134f640 1 -- 10.20.193.28:0/1555684604 shutdown_connections 2026-02-25T11:46:53.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.872+0000 7f735134f640 1 --2- 10.20.193.28:0/1555684604 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f734c1135b0 0x7f734c173f80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:53.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.872+0000 7f735134f640 1 --2- 10.20.193.28:0/1555684604 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f734c10c250 0x7f734c112e00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:53.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.872+0000 7f735134f640 1 --2- 10.20.193.28:0/1555684604 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f734c10b860 0x7f734c10bc30 unknown :-1 s=CLOSED pgs=196 gs=1 cs=0 l=1 c_cookie=8668216566484783148 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:53.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.872+0000 7f735134f640 1 -- 10.20.193.28:0/1555684604 >> 10.20.193.28:0/1555684604 conn(0x7f734c0e9290 msgr2=0x7f734c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:53.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.872+0000 7f735134f640 1 -- 10.20.193.28:0/1555684604 shutdown_connections 2026-02-25T11:46:53.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.872+0000 7f735134f640 1 -- 10.20.193.28:0/1555684604 wait complete. 2026-02-25T11:46:53.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.873+0000 7f735134f640 1 Processor -- start 2026-02-25T11:46:53.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.873+0000 7f735134f640 1 -- start start 2026-02-25T11:46:53.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.873+0000 7f735134f640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f734c10b860 0x7f734c0f4020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:53.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.873+0000 7f735134f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f734c10c250 0x7f734c173e80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:53.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.873+0000 7f735134f640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f734c1135b0 0x7f734c0f4720 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:53.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.873+0000 7f735134f640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f734c0f98e0 con 0x7f734c10c250 2026-02-25T11:46:53.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.873+0000 7f735134f640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f734c0f9a50 con 0x7f734c10b860 2026-02-25T11:46:53.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.873+0000 7f735134f640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f734c0f9c10 con 0x7f734c1135b0 2026-02-25T11:46:53.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.874+0000 7f734b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f734c10c250 0x7f734c173e80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:53.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.874+0000 7f734bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f734c10b860 0x7f734c0f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:53.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.874+0000 7f734b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f734c10c250 0x7f734c173e80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:45596/0 (socket says 10.20.193.28:45596) 2026-02-25T11:46:53.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.874+0000 7f734b7fe640 1 -- 10.20.193.28:0/225899923 learned_addr learned my addr 10.20.193.28:0/225899923 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:46:53.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.874+0000 7f734b7fe640 1 -- 10.20.193.28:0/225899923 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f734c1135b0 msgr2=0x7f734c0f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:53.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.874+0000 7f734b7fe640 1 --2- 10.20.193.28:0/225899923 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f734c1135b0 0x7f734c0f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:53.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.874+0000 7f734b7fe640 1 -- 10.20.193.28:0/225899923 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f734c10b860 msgr2=0x7f734c0f4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:53.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.874+0000 7f734b7fe640 1 --2- 10.20.193.28:0/225899923 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f734c10b860 0x7f734c0f4020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:53.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.874+0000 7f734b7fe640 1 -- 10.20.193.28:0/225899923 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7338009570 con 0x7f734c10c250 2026-02-25T11:46:53.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.874+0000 7f734b7fe640 1 --2- 10.20.193.28:0/225899923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f734c10c250 0x7f734c173e80 secure :-1 s=READY pgs=197 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f733c00c8c0 tx=0x7f733c00cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:53.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.875+0000 7f73497fa640 1 -- 10.20.193.28:0/225899923 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f733c0044a0 con 0x7f734c10c250 2026-02-25T11:46:53.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.875+0000 7f73497fa640 1 -- 10.20.193.28:0/225899923 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f733c007c60 con 0x7f734c10c250 2026-02-25T11:46:53.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.875+0000 7f73497fa640 1 -- 10.20.193.28:0/225899923 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f733c00f4b0 con 0x7f734c10c250 2026-02-25T11:46:53.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.875+0000 7f735134f640 1 -- 10.20.193.28:0/225899923 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c0f9ef0 con 0x7f734c10c250 2026-02-25T11:46:53.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.875+0000 7f735134f640 1 -- 10.20.193.28:0/225899923 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c0fa370 con 0x7f734c10c250 2026-02-25T11:46:53.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.875+0000 7f735134f640 1 -- 10.20.193.28:0/225899923 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7310005300 con 0x7f734c10c250 2026-02-25T11:46:53.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.878+0000 7f73497fa640 1 -- 10.20.193.28:0/225899923 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f733c0077a0 con 0x7f734c10c250 2026-02-25T11:46:53.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.879+0000 7f73497fa640 1 --2- 10.20.193.28:0/225899923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73200b56d0 0x7f73200d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:53.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.879+0000 7f73497fa640 1 -- 10.20.193.28:0/225899923 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7f733c0e6ad0 con 0x7f734c10c250 2026-02-25T11:46:53.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.879+0000 7f73497fa640 1 -- 10.20.193.28:0/225899923 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f733c11f940 con 0x7f734c10c250 2026-02-25T11:46:53.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.879+0000 7f734bfff640 1 --2- 10.20.193.28:0/225899923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73200b56d0 0x7f73200d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:53.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:53.880+0000 7f734bfff640 1 --2- 10.20.193.28:0/225899923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73200b56d0 0x7f73200d5b80 secure :-1 s=READY pgs=150 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f73380098a0 tx=0x7f733803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:54.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.021+0000 7f735134f640 1 -- 10.20.193.28:0/225899923 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7310005950 con 0x7f734c10c250 2026-02-25T11:46:54.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.021+0000 7f73497fa640 1 -- 10.20.193.28:0/225899923 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f733c0a98e0 con 0x7f734c10c250 2026-02-25T11:46:54.024 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:46:54.024 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":2,"btime":"2026-02-25T11:46:53:127270+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2026-02-25T11:46:54.025 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 2 2026-02-25T11:46:54.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.023+0000 7f735134f640 1 -- 10.20.193.28:0/225899923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73200b56d0 msgr2=0x7f73200d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:54.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.023+0000 7f735134f640 1 --2- 10.20.193.28:0/225899923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73200b56d0 0x7f73200d5b80 secure :-1 s=READY pgs=150 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f73380098a0 tx=0x7f733803a040 comp rx=0 tx=0).stop 2026-02-25T11:46:54.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.023+0000 7f735134f640 1 -- 10.20.193.28:0/225899923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f734c10c250 msgr2=0x7f734c173e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:54.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.023+0000 7f735134f640 1 --2- 10.20.193.28:0/225899923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f734c10c250 0x7f734c173e80 secure :-1 s=READY pgs=197 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f733c00c8c0 tx=0x7f733c00cd90 comp rx=0 tx=0).stop 2026-02-25T11:46:54.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.023+0000 7f735134f640 1 -- 10.20.193.28:0/225899923 shutdown_connections 2026-02-25T11:46:54.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.023+0000 7f735134f640 1 --2- 10.20.193.28:0/225899923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73200b56d0 0x7f73200d5b80 unknown :-1 s=CLOSED pgs=150 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:54.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.023+0000 7f735134f640 1 --2- 10.20.193.28:0/225899923 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f734c1135b0 0x7f734c0f4720 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:54.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.023+0000 7f735134f640 1 --2- 10.20.193.28:0/225899923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f734c10c250 0x7f734c173e80 unknown :-1 s=CLOSED pgs=197 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:54.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.023+0000 7f735134f640 1 --2- 10.20.193.28:0/225899923 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f734c10b860 0x7f734c0f4020 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:54.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.023+0000 7f735134f640 1 -- 10.20.193.28:0/225899923 >> 10.20.193.28:0/225899923 conn(0x7f734c0e9290 msgr2=0x7f734c10f660 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:54.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.023+0000 7f735134f640 1 -- 10.20.193.28:0/225899923 shutdown_connections 2026-02-25T11:46:54.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:54.023+0000 7f735134f640 1 -- 10.20.193.28:0/225899923 wait complete. 2026-02-25T11:46:54.254 INFO:journalctl@ceph.mds.k.trial119.stdout:Feb 25 11:46:54 trial119 systemd[1]: Starting Ceph mds.k for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:46:54.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:54 trial119 ceph-mon[42710]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:54.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:54 trial119 ceph-mon[42710]: Deploying daemon mds.k on trial119 2026-02-25T11:46:54.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:54 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/225899923' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:46:54.510 INFO:journalctl@ceph.mds.k.trial119.stdout:Feb 25 11:46:54 trial119 podman[94887]: 2026-02-25 11:46:54.318791947 +0000 UTC m=+0.028844427 container create 6b629be09a3681ed448327c26a24f5c9a24039b6fe0a0ec55f11f7fa2e0e1422 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-k, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.41.3, CEPH_REF=wip-vshankar-testing-20260224.100235, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa) 2026-02-25T11:46:54.511 INFO:journalctl@ceph.mds.k.trial119.stdout:Feb 25 11:46:54 trial119 podman[94887]: 2026-02-25 11:46:54.360216474 +0000 UTC m=+0.070268944 container init 6b629be09a3681ed448327c26a24f5c9a24039b6fe0a0ec55f11f7fa2e0e1422 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-k, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.vendor=CentOS, io.buildah.version=1.41.3, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.license=GPLv2, OSD_FLAVOR=default, ceph=True, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223, org.label-schema.name=CentOS Stream 9 Base Image, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0, CEPH_REF=wip-vshankar-testing-20260224.100235) 2026-02-25T11:46:54.511 INFO:journalctl@ceph.mds.k.trial119.stdout:Feb 25 11:46:54 trial119 podman[94887]: 2026-02-25 11:46:54.365644385 +0000 UTC m=+0.075696865 container start 6b629be09a3681ed448327c26a24f5c9a24039b6fe0a0ec55f11f7fa2e0e1422 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.41.3, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.vendor=CentOS, CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.build-date=20260223, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, ceph=True, org.label-schema.license=GPLv2, OSD_FLAVOR=default) 2026-02-25T11:46:54.511 INFO:journalctl@ceph.mds.k.trial119.stdout:Feb 25 11:46:54 trial119 bash[94887]: 6b629be09a3681ed448327c26a24f5c9a24039b6fe0a0ec55f11f7fa2e0e1422 2026-02-25T11:46:54.511 INFO:journalctl@ceph.mds.k.trial119.stdout:Feb 25 11:46:54 trial119 podman[94887]: 2026-02-25 11:46:54.30816966 +0000 UTC m=+0.018222160 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:46:54.511 INFO:journalctl@ceph.mds.k.trial119.stdout:Feb 25 11:46:54 trial119 systemd[1]: Started Ceph mds.k for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:46:54.511 INFO:journalctl@ceph.mds.k.trial119.stdout:Feb 25 11:46:54 trial119 ceph-mds[94906]: -- 10.20.193.119:0/1107111251 <== mon.2 v2:10.20.193.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x3d4f5860 con 0x3d638400 2026-02-25T11:46:54.511 INFO:journalctl@ceph.mds.k.trial119.stdout:Feb 25 11:46:54 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-k[94902]: starting mds.k at 2026-02-25T11:46:54.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:54 trial028 ceph-mon[39224]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:54.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:54 trial028 ceph-mon[39224]: Deploying daemon mds.k on trial119 2026-02-25T11:46:54.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:54 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/225899923' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:46:54.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:54 trial167 ceph-mon[43392]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:54.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:54 trial167 ceph-mon[43392]: Deploying daemon mds.k on trial119 2026-02-25T11:46:54.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:54 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/225899923' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:46:55.426 INFO:journalctl@ceph.mds.j.trial028.stdout:Feb 25 11:46:55 trial028 systemd[1]: Starting Ceph mds.j for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:46:55.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:55.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:55.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:55.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:55.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:55.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:55.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:55 trial167 ceph-mon[43392]: Deploying daemon mds.j on trial028 2026-02-25T11:46:55.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:55 trial167 ceph-mon[43392]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:55.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:55 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.119:6832/4164243082,v1:10.20.193.119:6833/4164243082] up:boot 2026-02-25T11:46:55.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:55 trial167 ceph-mon[43392]: fsmap 2 up:standby 2026-02-25T11:46:55.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:55 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2026-02-25T11:46:55.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:55.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:55.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:55.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:55.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:55.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:55.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:55 trial119 ceph-mon[42710]: Deploying daemon mds.j on trial028 2026-02-25T11:46:55.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:55 trial119 ceph-mon[42710]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:55.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:55 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.119:6832/4164243082,v1:10.20.193.119:6833/4164243082] up:boot 2026-02-25T11:46:55.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:55 trial119 ceph-mon[42710]: fsmap 2 up:standby 2026-02-25T11:46:55.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:55 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:55 trial028 ceph-mon[39224]: Deploying daemon mds.j on trial028 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:55 trial028 ceph-mon[39224]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:55 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.119:6832/4164243082,v1:10.20.193.119:6833/4164243082] up:boot 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:55 trial028 ceph-mon[39224]: fsmap 2 up:standby 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:55 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mds.j.trial028.stdout:Feb 25 11:46:55 trial028 podman[104179]: 2026-02-25 11:46:55.564746454 +0000 UTC m=+0.027848940 container create 5dbafe81b84fc005056d252acace4fa4a500cf608ba6bf036a9b1b7a155515de (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-j, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.41.3, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.build-date=20260223, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, FROM_IMAGE=quay.io/centos/centos:stream9) 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mds.j.trial028.stdout:Feb 25 11:46:55 trial028 podman[104179]: 2026-02-25 11:46:55.607924019 +0000 UTC m=+0.071026515 container init 5dbafe81b84fc005056d252acace4fa4a500cf608ba6bf036a9b1b7a155515de (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-j, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.build-date=20260223, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.41.3) 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mds.j.trial028.stdout:Feb 25 11:46:55 trial028 podman[104179]: 2026-02-25 11:46:55.613011803 +0000 UTC m=+0.076114289 container start 5dbafe81b84fc005056d252acace4fa4a500cf608ba6bf036a9b1b7a155515de (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-j, io.buildah.version=1.41.3, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.license=GPLv2, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.build-date=20260223, ceph=True, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default) 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mds.j.trial028.stdout:Feb 25 11:46:55 trial028 bash[104179]: 5dbafe81b84fc005056d252acace4fa4a500cf608ba6bf036a9b1b7a155515de 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mds.j.trial028.stdout:Feb 25 11:46:55 trial028 podman[104179]: 2026-02-25 11:46:55.554495053 +0000 UTC m=+0.017597539 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mds.j.trial028.stdout:Feb 25 11:46:55 trial028 systemd[1]: Started Ceph mds.j for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mds.j.trial028.stdout:Feb 25 11:46:55 trial028 ceph-mds[104199]: -- 10.20.193.28:0/2920024992 <== mon.0 v2:10.20.193.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x1c8e9860 con 0x1ca2c000 2026-02-25T11:46:55.810 INFO:journalctl@ceph.mds.j.trial028.stdout:Feb 25 11:46:55 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-j[104195]: starting mds.j at 2026-02-25T11:46:56.126 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:46:56.316 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:46:56.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.554+0000 7fb341b8b640 1 -- 10.20.193.28:0/3449562088 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb33c10cc70 msgr2=0x7fb33c111cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:56.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.554+0000 7fb341b8b640 1 --2- 10.20.193.28:0/3449562088 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb33c10cc70 0x7fb33c111cc0 secure :-1 s=READY pgs=200 gs=1 cs=0 l=1 c_cookie=16122850271986047009 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb324009950 tx=0x7fb32402f190 comp rx=0 tx=0).stop 2026-02-25T11:46:56.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.554+0000 7fb341b8b640 1 -- 10.20.193.28:0/3449562088 shutdown_connections 2026-02-25T11:46:56.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.554+0000 7fb341b8b640 1 --2- 10.20.193.28:0/3449562088 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb33c112470 0x7fb33c172e40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:56.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.554+0000 7fb341b8b640 1 --2- 10.20.193.28:0/3449562088 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb33c10cc70 0x7fb33c111cc0 unknown :-1 s=CLOSED pgs=200 gs=1 cs=0 l=1 c_cookie=16122850271986047009 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:56.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.554+0000 7fb341b8b640 1 --2- 10.20.193.28:0/3449562088 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb33c10c280 0x7fb33c10c650 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:56.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.554+0000 7fb341b8b640 1 -- 10.20.193.28:0/3449562088 >> 10.20.193.28:0/3449562088 conn(0x7fb33c0e9b50 msgr2=0x7fb33c109f60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:56.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.554+0000 7fb341b8b640 1 -- 10.20.193.28:0/3449562088 shutdown_connections 2026-02-25T11:46:56.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.554+0000 7fb341b8b640 1 -- 10.20.193.28:0/3449562088 wait complete. 2026-02-25T11:46:56.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.555+0000 7fb341b8b640 1 Processor -- start 2026-02-25T11:46:56.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.555+0000 7fb341b8b640 1 -- start start 2026-02-25T11:46:56.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.555+0000 7fb341b8b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb33c10c280 0x7fb33c0f4510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:56.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.555+0000 7fb341b8b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb33c10cc70 0x7fb33c172d40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:56.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.555+0000 7fb341b8b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb33c112470 0x7fb33c0f4c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:56.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.555+0000 7fb341b8b640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c0f9e20 con 0x7fb33c10cc70 2026-02-25T11:46:56.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.555+0000 7fb341b8b640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c0f9f90 con 0x7fb33c112470 2026-02-25T11:46:56.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.555+0000 7fb341b8b640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c0fa150 con 0x7fb33c10c280 2026-02-25T11:46:56.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.555+0000 7fb340b89640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb33c10c280 0x7fb33c0f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:56.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.555+0000 7fb340b89640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb33c10c280 0x7fb33c0f4510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:46858/0 (socket says 10.20.193.28:46858) 2026-02-25T11:46:56.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.555+0000 7fb340b89640 1 -- 10.20.193.28:0/1434939118 learned_addr learned my addr 10.20.193.28:0/1434939118 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:46:56.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.556+0000 7fb333fff640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb33c10cc70 0x7fb33c172d40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:56.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.556+0000 7fb34138a640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb33c112470 0x7fb33c0f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:56.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.556+0000 7fb340b89640 1 -- 10.20.193.28:0/1434939118 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb33c112470 msgr2=0x7fb33c0f4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:56.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.556+0000 7fb340b89640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb33c112470 0x7fb33c0f4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:56.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.556+0000 7fb340b89640 1 -- 10.20.193.28:0/1434939118 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb33c10cc70 msgr2=0x7fb33c172d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:56.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.556+0000 7fb340b89640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb33c10cc70 0x7fb33c172d40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:56.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.556+0000 7fb340b89640 1 -- 10.20.193.28:0/1434939118 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb324009570 con 0x7fb33c10c280 2026-02-25T11:46:56.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.556+0000 7fb340b89640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb33c10c280 0x7fb33c0f4510 secure :-1 s=READY pgs=74 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb32c0072a0 tx=0x7fb32c00ddf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:56.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.556+0000 7fb331ffb640 1 -- 10.20.193.28:0/1434939118 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb32c004500 con 0x7fb33c10c280 2026-02-25T11:46:56.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.556+0000 7fb331ffb640 1 -- 10.20.193.28:0/1434939118 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb32c00bd00 con 0x7fb33c10c280 2026-02-25T11:46:56.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.556+0000 7fb331ffb640 1 -- 10.20.193.28:0/1434939118 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb32c0106e0 con 0x7fb33c10c280 2026-02-25T11:46:56.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.556+0000 7fb341b8b640 1 -- 10.20.193.28:0/1434939118 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33c0fa430 con 0x7fb33c10c280 2026-02-25T11:46:56.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.556+0000 7fb341b8b640 1 -- 10.20.193.28:0/1434939118 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33c0fa8b0 con 0x7fb33c10c280 2026-02-25T11:46:56.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.557+0000 7fb341b8b640 1 -- 10.20.193.28:0/1434939118 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb33c10d560 con 0x7fb33c10c280 2026-02-25T11:46:56.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.559+0000 7fb331ffb640 1 -- 10.20.193.28:0/1434939118 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7fb32c00b840 con 0x7fb33c10c280 2026-02-25T11:46:56.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.561+0000 7fb331ffb640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb3180b5820 0x7fb3180d5cd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:56.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.561+0000 7fb331ffb640 1 -- 10.20.193.28:0/1434939118 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7fb32c0e66e0 con 0x7fb33c10c280 2026-02-25T11:46:56.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.561+0000 7fb331ffb640 1 -- 10.20.193.28:0/1434939118 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fb32c0e6b60 con 0x7fb33c10c280 2026-02-25T11:46:56.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.561+0000 7fb333fff640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb3180b5820 0x7fb3180d5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:56.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.562+0000 7fb333fff640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb3180b5820 0x7fb3180d5cd0 secure :-1 s=READY pgs=153 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb33c0faeb0 tx=0x7fb32403a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:56.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.703+0000 7fb341b8b640 1 -- 10.20.193.28:0/1434939118 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb33c060c00 con 0x7fb33c10c280 2026-02-25T11:46:56.706 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.703+0000 7fb331ffb640 1 -- 10.20.193.28:0/1434939118 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7fb32c0a94f0 con 0x7fb33c10c280 2026-02-25T11:46:56.707 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:46:56.707 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":4,"btime":"2026-02-25T11:46:56:432504+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2026-02-25T11:46:56.707 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 4 2026-02-25T11:46:56.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.705+0000 7fb341b8b640 1 -- 10.20.193.28:0/1434939118 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb3180b5820 msgr2=0x7fb3180d5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:56.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.705+0000 7fb341b8b640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb3180b5820 0x7fb3180d5cd0 secure :-1 s=READY pgs=153 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb33c0faeb0 tx=0x7fb32403a040 comp rx=0 tx=0).stop 2026-02-25T11:46:56.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.705+0000 7fb341b8b640 1 -- 10.20.193.28:0/1434939118 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb33c10c280 msgr2=0x7fb33c0f4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:56.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.705+0000 7fb341b8b640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb33c10c280 0x7fb33c0f4510 secure :-1 s=READY pgs=74 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb32c0072a0 tx=0x7fb32c00ddf0 comp rx=0 tx=0).stop 2026-02-25T11:46:56.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.706+0000 7fb341b8b640 1 -- 10.20.193.28:0/1434939118 shutdown_connections 2026-02-25T11:46:56.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.706+0000 7fb341b8b640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb3180b5820 0x7fb3180d5cd0 unknown :-1 s=CLOSED pgs=153 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:56.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.706+0000 7fb341b8b640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb33c112470 0x7fb33c0f4c10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:56.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.706+0000 7fb341b8b640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb33c10cc70 0x7fb33c172d40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:56.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.706+0000 7fb341b8b640 1 --2- 10.20.193.28:0/1434939118 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb33c10c280 0x7fb33c0f4510 unknown :-1 s=CLOSED pgs=74 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:56.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.706+0000 7fb341b8b640 1 -- 10.20.193.28:0/1434939118 >> 10.20.193.28:0/1434939118 conn(0x7fb33c0e9b50 msgr2=0x7fb33c1102c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:56.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.706+0000 7fb341b8b640 1 -- 10.20.193.28:0/1434939118 shutdown_connections 2026-02-25T11:46:56.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:56.706+0000 7fb341b8b640 1 -- 10.20.193.28:0/1434939118 wait complete. 2026-02-25T11:46:56.942 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:56.943 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:56.943 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:56.943 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:56.943 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:56.943 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:56.943 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:56 trial028 ceph-mon[39224]: Deploying daemon mds.d on trial028 2026-02-25T11:46:56.943 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:56 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.28:6834/685551537,v1:10.20.193.28:6835/685551537] up:boot 2026-02-25T11:46:56.943 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:56 trial028 ceph-mon[39224]: fsmap 3 up:standby 2026-02-25T11:46:56.944 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:56 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2026-02-25T11:46:56.944 INFO:journalctl@ceph.mds.d.trial028.stdout:Feb 25 11:46:56 trial028 podman[104566]: 2026-02-25 11:46:56.905983516 +0000 UTC m=+0.027171389 container create b798644ac44309fbba1627d580ed58343f76af4a028b4a474b568e59a254c809 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-d, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, io.buildah.version=1.41.3, org.label-schema.license=GPLv2, OSD_FLAVOR=default, org.label-schema.build-date=20260223, ceph=True, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T11:46:56.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:56.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:57.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:57.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:57.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:57.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:57.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:56 trial167 ceph-mon[43392]: Deploying daemon mds.d on trial028 2026-02-25T11:46:57.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:56 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.28:6834/685551537,v1:10.20.193.28:6835/685551537] up:boot 2026-02-25T11:46:57.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:56 trial167 ceph-mon[43392]: fsmap 3 up:standby 2026-02-25T11:46:57.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:56 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2026-02-25T11:46:57.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:57.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:57.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:57.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:57.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:57.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:57.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:56 trial119 ceph-mon[42710]: Deploying daemon mds.d on trial028 2026-02-25T11:46:57.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:56 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.28:6834/685551537,v1:10.20.193.28:6835/685551537] up:boot 2026-02-25T11:46:57.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:56 trial119 ceph-mon[42710]: fsmap 3 up:standby 2026-02-25T11:46:57.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:56 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2026-02-25T11:46:57.310 INFO:journalctl@ceph.mds.d.trial028.stdout:Feb 25 11:46:56 trial028 podman[104566]: 2026-02-25 11:46:56.944315296 +0000 UTC m=+0.065503179 container init b798644ac44309fbba1627d580ed58343f76af4a028b4a474b568e59a254c809 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-d, ceph=True, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, io.buildah.version=1.41.3, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, org.label-schema.build-date=20260223, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2026-02-25T11:46:57.310 INFO:journalctl@ceph.mds.d.trial028.stdout:Feb 25 11:46:56 trial028 podman[104566]: 2026-02-25 11:46:56.9496067 +0000 UTC m=+0.070794573 container start b798644ac44309fbba1627d580ed58343f76af4a028b4a474b568e59a254c809 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-d, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.name=CentOS Stream 9 Base Image, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, ceph=True, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.build-date=20260223, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-25T11:46:57.310 INFO:journalctl@ceph.mds.d.trial028.stdout:Feb 25 11:46:56 trial028 bash[104566]: b798644ac44309fbba1627d580ed58343f76af4a028b4a474b568e59a254c809 2026-02-25T11:46:57.310 INFO:journalctl@ceph.mds.d.trial028.stdout:Feb 25 11:46:56 trial028 podman[104566]: 2026-02-25 11:46:56.89572964 +0000 UTC m=+0.016917513 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:46:57.310 INFO:journalctl@ceph.mds.d.trial028.stdout:Feb 25 11:46:56 trial028 systemd[1]: Started Ceph mds.d for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:46:57.310 INFO:journalctl@ceph.mds.d.trial028.stdout:Feb 25 11:46:56 trial028 ceph-mds[104585]: -- 10.20.193.28:0/2582502832 <== mon.0 v2:10.20.193.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x19d8d860 con 0x19ec4400 2026-02-25T11:46:57.310 INFO:journalctl@ceph.mds.d.trial028.stdout:Feb 25 11:46:56 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-d[104581]: starting mds.d at 2026-02-25T11:46:57.959 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:57 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1434939118' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:46:57.959 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:57.960 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:57.960 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:57.960 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:57.960 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:57.960 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:57 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:57.960 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:57 trial119 ceph-mon[42710]: Deploying daemon mds.e on trial119 2026-02-25T11:46:57.960 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:57 trial119 ceph-mon[42710]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:57.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:57 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1434939118' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:46:57.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:57.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:57.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:57.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:57.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:57.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:57 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:57.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:57 trial167 ceph-mon[43392]: Deploying daemon mds.e on trial119 2026-02-25T11:46:57.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:57 trial167 ceph-mon[43392]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:58.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:57 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1434939118' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:46:58.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:58.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:58.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:58.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:58.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:58.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:57 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:58.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:57 trial028 ceph-mon[39224]: Deploying daemon mds.e on trial119 2026-02-25T11:46:58.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:57 trial028 ceph-mon[39224]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:46:58.260 INFO:journalctl@ceph.mds.e.trial119.stdout:Feb 25 11:46:58 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-e[95192]: starting mds.e at 2026-02-25T11:46:58.846 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:46:58.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:58 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.28:6836/281937174,v1:10.20.193.28:6837/281937174] up:boot 2026-02-25T11:46:59.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:58 trial167 ceph-mon[43392]: fsmap 4 up:standby 2026-02-25T11:46:59.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:58 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2026-02-25T11:46:59.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:58 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:59.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:58 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:59.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:58 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:59.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:58 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:59.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:58 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:59.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:58 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:59.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:46:58 trial167 ceph-mon[43392]: Deploying daemon mds.f on trial167 2026-02-25T11:46:59.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:58 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.28:6836/281937174,v1:10.20.193.28:6837/281937174] up:boot 2026-02-25T11:46:59.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:58 trial119 ceph-mon[42710]: fsmap 4 up:standby 2026-02-25T11:46:59.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:58 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2026-02-25T11:46:59.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:58 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:59.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:58 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:59.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:58 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:59.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:58 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:59.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:58 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:59.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:58 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:59.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:46:58 trial119 ceph-mon[42710]: Deploying daemon mds.f on trial167 2026-02-25T11:46:59.032 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:46:59.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:58 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.28:6836/281937174,v1:10.20.193.28:6837/281937174] up:boot 2026-02-25T11:46:59.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:58 trial028 ceph-mon[39224]: fsmap 4 up:standby 2026-02-25T11:46:59.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:58 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2026-02-25T11:46:59.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:58 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:59.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:58 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:59.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:58 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:46:59.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:58 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:46:59.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:58 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:46:59.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:58 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:46:59.060 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:46:58 trial028 ceph-mon[39224]: Deploying daemon mds.f on trial167 2026-02-25T11:46:59.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.211+0000 7f7339b7a640 1 -- 10.20.193.28:0/438777534 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f73341135c0 msgr2=0x7f7334173f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:59.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.211+0000 7f7339b7a640 1 --2- 10.20.193.28:0/438777534 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f73341135c0 0x7f7334173f90 secure :-1 s=READY pgs=203 gs=1 cs=0 l=1 c_cookie=9482544602226407685 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f733000aa30 tx=0x7f7330030270 comp rx=0 tx=0).stop 2026-02-25T11:46:59.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.211+0000 7f7339b7a640 1 -- 10.20.193.28:0/438777534 shutdown_connections 2026-02-25T11:46:59.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.211+0000 7f7339b7a640 1 --2- 10.20.193.28:0/438777534 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f73341135c0 0x7f7334173f90 unknown :-1 s=CLOSED pgs=203 gs=1 cs=0 l=1 c_cookie=9482544602226407685 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:59.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.211+0000 7f7339b7a640 1 --2- 10.20.193.28:0/438777534 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f733410ddc0 0x7f7334112e10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:59.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.211+0000 7f7339b7a640 1 --2- 10.20.193.28:0/438777534 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f733410d3d0 0x7f733410d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:59.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.211+0000 7f7339b7a640 1 -- 10.20.193.28:0/438777534 >> 10.20.193.28:0/438777534 conn(0x7f73340eaca0 msgr2=0x7f733410b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:59.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.211+0000 7f7339b7a640 1 -- 10.20.193.28:0/438777534 shutdown_connections 2026-02-25T11:46:59.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.211+0000 7f7339b7a640 1 -- 10.20.193.28:0/438777534 wait complete. 2026-02-25T11:46:59.217 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.212+0000 7f7339b7a640 1 Processor -- start 2026-02-25T11:46:59.217 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.212+0000 7f7339b7a640 1 -- start start 2026-02-25T11:46:59.217 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7339b7a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f733410d3d0 0x7f73340f5930 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:59.217 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7339b7a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f733410ddc0 0x7f7334173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:59.217 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7339b7a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f73341135c0 0x7f73340f6030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:59.218 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7339b7a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f73340fb240 con 0x7f733410d3d0 2026-02-25T11:46:59.218 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7339b7a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f73340fb3b0 con 0x7f73341135c0 2026-02-25T11:46:59.218 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7339b7a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f73340fb570 con 0x7f733410ddc0 2026-02-25T11:46:59.218 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7338b78640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f733410d3d0 0x7f73340f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:59.218 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7338b78640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f733410d3d0 0x7f73340f5930 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:36444/0 (socket says 10.20.193.28:36444) 2026-02-25T11:46:59.218 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7338b78640 1 -- 10.20.193.28:0/2516819783 learned_addr learned my addr 10.20.193.28:0/2516819783 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:46:59.218 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f732bfff640 1 --2- 10.20.193.28:0/2516819783 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f733410ddc0 0x7f7334173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:59.218 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7338b78640 1 -- 10.20.193.28:0/2516819783 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f733410ddc0 msgr2=0x7f7334173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:59.218 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7338b78640 1 --2- 10.20.193.28:0/2516819783 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f733410ddc0 0x7f7334173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:59.218 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7338b78640 1 -- 10.20.193.28:0/2516819783 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f73341135c0 msgr2=0x7f73340f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:59.218 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7338b78640 1 --2- 10.20.193.28:0/2516819783 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f73341135c0 0x7f73340f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:59.219 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7338b78640 1 -- 10.20.193.28:0/2516819783 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f733000a650 con 0x7f733410d3d0 2026-02-25T11:46:59.219 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7338b78640 1 --2- 10.20.193.28:0/2516819783 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f733410d3d0 0x7f73340f5930 secure :-1 s=READY pgs=204 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f732400d950 tx=0x7f732400de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:59.219 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.213+0000 7f7329ffb640 1 -- 10.20.193.28:0/2516819783 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7324004540 con 0x7f733410d3d0 2026-02-25T11:46:59.219 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.214+0000 7f7329ffb640 1 -- 10.20.193.28:0/2516819783 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7324009ce0 con 0x7f733410d3d0 2026-02-25T11:46:59.219 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.214+0000 7f7339b7a640 1 -- 10.20.193.28:0/2516819783 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73340fb850 con 0x7f733410d3d0 2026-02-25T11:46:59.219 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.214+0000 7f7329ffb640 1 -- 10.20.193.28:0/2516819783 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f73240106e0 con 0x7f733410d3d0 2026-02-25T11:46:59.219 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.214+0000 7f7339b7a640 1 -- 10.20.193.28:0/2516819783 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73340fbd20 con 0x7f733410d3d0 2026-02-25T11:46:59.219 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.214+0000 7f7339b7a640 1 -- 10.20.193.28:0/2516819783 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f733410e6b0 con 0x7f733410d3d0 2026-02-25T11:46:59.220 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.216+0000 7f7329ffb640 1 -- 10.20.193.28:0/2516819783 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f7324009820 con 0x7f733410d3d0 2026-02-25T11:46:59.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.218+0000 7f7329ffb640 1 --2- 10.20.193.28:0/2516819783 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73100b56d0 0x7f73100d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:46:59.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.218+0000 7f7329ffb640 1 -- 10.20.193.28:0/2516819783 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7f73240e6850 con 0x7f733410d3d0 2026-02-25T11:46:59.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.218+0000 7f732bfff640 1 --2- 10.20.193.28:0/2516819783 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73100b56d0 0x7f73100d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:46:59.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.218+0000 7f7329ffb640 1 -- 10.20.193.28:0/2516819783 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f732411f940 con 0x7f733410d3d0 2026-02-25T11:46:59.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.219+0000 7f732bfff640 1 --2- 10.20.193.28:0/2516819783 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73100b56d0 0x7f73100d5b80 secure :-1 s=READY pgs=156 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f73340fc2d0 tx=0x7f731c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:46:59.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.360+0000 7f7339b7a640 1 -- 10.20.193.28:0/2516819783 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f733410d7a0 con 0x7f733410d3d0 2026-02-25T11:46:59.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.360+0000 7f7329ffb640 1 -- 10.20.193.28:0/2516819783 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7f73240a9660 con 0x7f733410d3d0 2026-02-25T11:46:59.363 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:46:59.363 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":6,"btime":"2026-02-25T11:46:59:015655+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2026-02-25T11:46:59.364 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 6 2026-02-25T11:46:59.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.361+0000 7f7339b7a640 1 -- 10.20.193.28:0/2516819783 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73100b56d0 msgr2=0x7f73100d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:59.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.361+0000 7f7339b7a640 1 --2- 10.20.193.28:0/2516819783 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73100b56d0 0x7f73100d5b80 secure :-1 s=READY pgs=156 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f73340fc2d0 tx=0x7f731c005f90 comp rx=0 tx=0).stop 2026-02-25T11:46:59.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.361+0000 7f7339b7a640 1 -- 10.20.193.28:0/2516819783 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f733410d3d0 msgr2=0x7f73340f5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:46:59.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.361+0000 7f7339b7a640 1 --2- 10.20.193.28:0/2516819783 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f733410d3d0 0x7f73340f5930 secure :-1 s=READY pgs=204 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f732400d950 tx=0x7f732400de20 comp rx=0 tx=0).stop 2026-02-25T11:46:59.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.361+0000 7f7339b7a640 1 -- 10.20.193.28:0/2516819783 shutdown_connections 2026-02-25T11:46:59.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.361+0000 7f7339b7a640 1 --2- 10.20.193.28:0/2516819783 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73100b56d0 0x7f73100d5b80 unknown :-1 s=CLOSED pgs=156 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:59.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.361+0000 7f7339b7a640 1 --2- 10.20.193.28:0/2516819783 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f73341135c0 0x7f73340f6030 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:59.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.361+0000 7f7339b7a640 1 --2- 10.20.193.28:0/2516819783 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f733410ddc0 0x7f7334173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:59.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.361+0000 7f7339b7a640 1 --2- 10.20.193.28:0/2516819783 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f733410d3d0 0x7f73340f5930 unknown :-1 s=CLOSED pgs=204 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:46:59.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.361+0000 7f7339b7a640 1 -- 10.20.193.28:0/2516819783 >> 10.20.193.28:0/2516819783 conn(0x7f73340eaca0 msgr2=0x7f733410fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:46:59.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.362+0000 7f7339b7a640 1 -- 10.20.193.28:0/2516819783 shutdown_connections 2026-02-25T11:46:59.366 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:46:59.362+0000 7f7339b7a640 1 -- 10.20.193.28:0/2516819783 wait complete. 2026-02-25T11:46:59.400 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 11:46:59 trial167 systemd[1]: Starting Ceph mds.f for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:46:59.750 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 11:46:59 trial167 podman[97701]: 2026-02-25 11:46:59.399184881 +0000 UTC m=+0.027939073 container create 2ea3e1280d4794438608e93d942bc456a4ed212b3b028cdc0b3ae423c175c867 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-f, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , ceph=True, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.build-date=20260223, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.license=GPLv2, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa) 2026-02-25T11:46:59.750 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 11:46:59 trial167 podman[97701]: 2026-02-25 11:46:59.441605787 +0000 UTC m=+0.070359988 container init 2ea3e1280d4794438608e93d942bc456a4ed212b3b028cdc0b3ae423c175c867 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-f, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , ceph=True, org.label-schema.build-date=20260223, CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T11:46:59.750 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 11:46:59 trial167 podman[97701]: 2026-02-25 11:46:59.44658223 +0000 UTC m=+0.075336431 container start 2ea3e1280d4794438608e93d942bc456a4ed212b3b028cdc0b3ae423c175c867 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-f, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.vendor=CentOS, io.buildah.version=1.41.3, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , ceph=True, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.license=GPLv2, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T11:46:59.750 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 11:46:59 trial167 bash[97701]: 2ea3e1280d4794438608e93d942bc456a4ed212b3b028cdc0b3ae423c175c867 2026-02-25T11:46:59.750 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 11:46:59 trial167 podman[97701]: 2026-02-25 11:46:59.389036613 +0000 UTC m=+0.017790824 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:46:59.750 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 11:46:59 trial167 systemd[1]: Started Ceph mds.f for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:46:59.751 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 11:46:59 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-f[97716]: starting mds.f at 2026-02-25T11:47:00.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:00 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.119:6834/3262473702,v1:10.20.193.119:6835/3262473702] up:boot 2026-02-25T11:47:00.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:00 trial167 ceph-mon[43392]: fsmap 5 up:standby 2026-02-25T11:47:00.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2026-02-25T11:47:00.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:00 trial167 ceph-mon[43392]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:00.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:00 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2516819783' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:00.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:00.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:00.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:00.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:00.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:00.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:00 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:00.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:00 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.119:6834/3262473702,v1:10.20.193.119:6835/3262473702] up:boot 2026-02-25T11:47:00.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:00 trial028 ceph-mon[39224]: fsmap 5 up:standby 2026-02-25T11:47:00.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2026-02-25T11:47:00.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:00 trial028 ceph-mon[39224]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:00.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:00 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2516819783' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:00.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:00.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:00.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:00.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:00.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:00.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:00 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:00.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:00 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.119:6834/3262473702,v1:10.20.193.119:6835/3262473702] up:boot 2026-02-25T11:47:00.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:00 trial119 ceph-mon[42710]: fsmap 5 up:standby 2026-02-25T11:47:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2026-02-25T11:47:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:00 trial119 ceph-mon[42710]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:00 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2516819783' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:00 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:00.681 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 11:47:00 trial167 systemd[1]: Starting Ceph mds.c for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:47:01.000 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 11:47:00 trial167 podman[98003]: 2026-02-25 11:47:00.680170589 +0000 UTC m=+0.029201061 container create 26211a9b34e7e029f94610c569a68325c1a61b4cd8ab3d0cfb51d7fe199d84ce (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-c, ceph=True, org.label-schema.vendor=CentOS, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.license=GPLv2, CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.41.3, org.label-schema.build-date=20260223, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-25T11:47:01.000 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 11:47:00 trial167 podman[98003]: 2026-02-25 11:47:00.718621696 +0000 UTC m=+0.067652168 container init 26211a9b34e7e029f94610c569a68325c1a61b4cd8ab3d0cfb51d7fe199d84ce (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-c, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, ceph=True, io.buildah.version=1.41.3, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.vendor=CentOS, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20260223) 2026-02-25T11:47:01.000 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 11:47:00 trial167 podman[98003]: 2026-02-25 11:47:00.723328097 +0000 UTC m=+0.072358569 container start 26211a9b34e7e029f94610c569a68325c1a61b4cd8ab3d0cfb51d7fe199d84ce (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-c, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, io.buildah.version=1.41.3, OSD_FLAVOR=default, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.build-date=20260223, org.label-schema.vendor=CentOS, CEPH_REF=wip-vshankar-testing-20260224.100235) 2026-02-25T11:47:01.000 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 11:47:00 trial167 bash[98003]: 26211a9b34e7e029f94610c569a68325c1a61b4cd8ab3d0cfb51d7fe199d84ce 2026-02-25T11:47:01.000 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 11:47:00 trial167 podman[98003]: 2026-02-25 11:47:00.668704198 +0000 UTC m=+0.017734680 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:47:01.000 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 11:47:00 trial167 systemd[1]: Started Ceph mds.c for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:47:01.000 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 11:47:00 trial167 ceph-mds[98023]: -- 10.20.193.167:0/1536728809 <== mon.0 v2:10.20.193.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x1ad9860 con 0x1c1a000 2026-02-25T11:47:01.000 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 11:47:00 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-c[98019]: starting mds.c at 2026-02-25T11:47:01.290 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:01 trial119 ceph-mon[42710]: Deploying daemon mds.c on trial167 2026-02-25T11:47:01.290 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:01 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.167:6834/3681115733,v1:10.20.193.167:6835/3681115733] up:boot 2026-02-25T11:47:01.290 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:01 trial119 ceph-mon[42710]: fsmap 6 up:standby 2026-02-25T11:47:01.291 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2026-02-25T11:47:01.291 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:01.291 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:01.291 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:01.291 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:01.291 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:01.291 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:01 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:01.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:01 trial028 ceph-mon[39224]: Deploying daemon mds.c on trial167 2026-02-25T11:47:01.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:01 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.167:6834/3681115733,v1:10.20.193.167:6835/3681115733] up:boot 2026-02-25T11:47:01.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:01 trial028 ceph-mon[39224]: fsmap 6 up:standby 2026-02-25T11:47:01.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2026-02-25T11:47:01.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:01.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:01.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:01.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:01.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:01.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:01 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:01.463 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:47:01.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:01 trial167 ceph-mon[43392]: Deploying daemon mds.c on trial167 2026-02-25T11:47:01.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:01 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.167:6834/3681115733,v1:10.20.193.167:6835/3681115733] up:boot 2026-02-25T11:47:01.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:01 trial167 ceph-mon[43392]: fsmap 6 up:standby 2026-02-25T11:47:01.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2026-02-25T11:47:01.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:01.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:01.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:01.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:01.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:01.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:01 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:01.649 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:01.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.817+0000 7f2fd702b640 1 -- 10.20.193.28:0/2376943648 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2fd010ce70 msgr2=0x7f2fd0111ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:01.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.817+0000 7f2fd702b640 1 --2- 10.20.193.28:0/2376943648 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2fd010ce70 0x7f2fd0111ec0 secure :-1 s=READY pgs=206 gs=1 cs=0 l=1 c_cookie=3386031926399338938 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2fb8009950 tx=0x7f2fb802f190 comp rx=0 tx=0).stop 2026-02-25T11:47:01.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.817+0000 7f2fd702b640 1 -- 10.20.193.28:0/2376943648 shutdown_connections 2026-02-25T11:47:01.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.817+0000 7f2fd702b640 1 --2- 10.20.193.28:0/2376943648 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2fd0112670 0x7f2fd0173040 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:01.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.817+0000 7f2fd702b640 1 --2- 10.20.193.28:0/2376943648 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2fd010ce70 0x7f2fd0111ec0 unknown :-1 s=CLOSED pgs=206 gs=1 cs=0 l=1 c_cookie=3386031926399338938 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:01.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.817+0000 7f2fd702b640 1 --2- 10.20.193.28:0/2376943648 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2fd010c480 0x7f2fd010c850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:01.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.817+0000 7f2fd702b640 1 -- 10.20.193.28:0/2376943648 >> 10.20.193.28:0/2376943648 conn(0x7f2fd00e9d50 msgr2=0x7f2fd010a160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:01.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.817+0000 7f2fd702b640 1 -- 10.20.193.28:0/2376943648 shutdown_connections 2026-02-25T11:47:01.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.817+0000 7f2fd702b640 1 -- 10.20.193.28:0/2376943648 wait complete. 2026-02-25T11:47:01.821 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.817+0000 7f2fd702b640 1 Processor -- start 2026-02-25T11:47:01.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd702b640 1 -- start start 2026-02-25T11:47:01.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd702b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2fd010c480 0x7f2fd00f49d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:01.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd702b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2fd010ce70 0x7f2fd0172f40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:01.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd702b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2fd0112670 0x7f2fd00f50d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:01.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd702b640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd00fa2e0 con 0x7f2fd010c480 2026-02-25T11:47:01.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd702b640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd00fa450 con 0x7f2fd010ce70 2026-02-25T11:47:01.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd702b640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2fd00fa610 con 0x7f2fd0112670 2026-02-25T11:47:01.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd4da0640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2fd010c480 0x7f2fd00f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:01.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd4da0640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2fd010c480 0x7f2fd00f49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:36464/0 (socket says 10.20.193.28:36464) 2026-02-25T11:47:01.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd4da0640 1 -- 10.20.193.28:0/1497312344 learned_addr learned my addr 10.20.193.28:0/1497312344 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:01.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fc7fff640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2fd010ce70 0x7f2fd0172f40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:01.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd55a1640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2fd0112670 0x7f2fd00f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:01.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd4da0640 1 -- 10.20.193.28:0/1497312344 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2fd0112670 msgr2=0x7f2fd00f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:01.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd4da0640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2fd0112670 0x7f2fd00f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:01.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd4da0640 1 -- 10.20.193.28:0/1497312344 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2fd010ce70 msgr2=0x7f2fd0172f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:01.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd4da0640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2fd010ce70 0x7f2fd0172f40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:01.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fd4da0640 1 -- 10.20.193.28:0/1497312344 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb8009570 con 0x7f2fd010c480 2026-02-25T11:47:01.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.818+0000 7f2fc7fff640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2fd010ce70 0x7f2fd0172f40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:01.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.819+0000 7f2fd4da0640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2fd010c480 0x7f2fd00f49d0 secure :-1 s=READY pgs=207 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2fc0009480 tx=0x7f2fc000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:01.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.819+0000 7f2fc5ffb640 1 -- 10.20.193.28:0/1497312344 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2fc000cd40 con 0x7f2fd010c480 2026-02-25T11:47:01.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.819+0000 7f2fc5ffb640 1 -- 10.20.193.28:0/1497312344 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2fc0004d10 con 0x7f2fd010c480 2026-02-25T11:47:01.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.819+0000 7f2fc5ffb640 1 -- 10.20.193.28:0/1497312344 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2fc00106b0 con 0x7f2fd010c480 2026-02-25T11:47:01.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.819+0000 7f2fd702b640 1 -- 10.20.193.28:0/1497312344 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fd00fa8f0 con 0x7f2fd010c480 2026-02-25T11:47:01.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.819+0000 7f2fd702b640 1 -- 10.20.193.28:0/1497312344 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fd00fad70 con 0x7f2fd010c480 2026-02-25T11:47:01.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.819+0000 7f2fd702b640 1 -- 10.20.193.28:0/1497312344 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fd010d760 con 0x7f2fd010c480 2026-02-25T11:47:01.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.822+0000 7f2fc5ffb640 1 -- 10.20.193.28:0/1497312344 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f2fc001f060 con 0x7f2fd010c480 2026-02-25T11:47:01.826 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.823+0000 7f2fc5ffb640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2fac0b57d0 0x7f2fac0d5c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:01.826 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.823+0000 7f2fc5ffb640 1 -- 10.20.193.28:0/1497312344 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7f2fc0014070 con 0x7f2fd010c480 2026-02-25T11:47:01.826 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.823+0000 7f2fc7fff640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2fac0b57d0 0x7f2fac0d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:01.826 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.823+0000 7f2fc5ffb640 1 -- 10.20.193.28:0/1497312344 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f2fc011f940 con 0x7f2fd010c480 2026-02-25T11:47:01.826 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.824+0000 7f2fc7fff640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2fac0b57d0 0x7f2fac0d5c80 secure :-1 s=READY pgs=159 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2fb80023d0 tx=0x7f2fb8031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:01.964 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 11:47:01 trial119 systemd[1]: Starting Ceph mds.h for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:47:01.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.968+0000 7f2fd702b640 1 -- 10.20.193.28:0/1497312344 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fd00f5b30 con 0x7f2fd010c480 2026-02-25T11:47:01.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.969+0000 7f2fc5ffb640 1 -- 10.20.193.28:0/1497312344 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6391 (secure 0 0 0) 0x7f2fc00a9860 con 0x7f2fd010c480 2026-02-25T11:47:01.971 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:01.971 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":8,"btime":"2026-02-25T11:47:01:020719+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24706,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2026-02-25T11:47:01.972 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 8 2026-02-25T11:47:01.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.970+0000 7f2fd702b640 1 -- 10.20.193.28:0/1497312344 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2fac0b57d0 msgr2=0x7f2fac0d5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:01.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.970+0000 7f2fd702b640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2fac0b57d0 0x7f2fac0d5c80 secure :-1 s=READY pgs=159 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2fb80023d0 tx=0x7f2fb8031040 comp rx=0 tx=0).stop 2026-02-25T11:47:01.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.970+0000 7f2fd702b640 1 -- 10.20.193.28:0/1497312344 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2fd010c480 msgr2=0x7f2fd00f49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:01.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.970+0000 7f2fd702b640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2fd010c480 0x7f2fd00f49d0 secure :-1 s=READY pgs=207 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2fc0009480 tx=0x7f2fc000ee60 comp rx=0 tx=0).stop 2026-02-25T11:47:01.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.970+0000 7f2fd702b640 1 -- 10.20.193.28:0/1497312344 shutdown_connections 2026-02-25T11:47:01.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.970+0000 7f2fd702b640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2fac0b57d0 0x7f2fac0d5c80 unknown :-1 s=CLOSED pgs=159 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:01.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.970+0000 7f2fd702b640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2fd0112670 0x7f2fd00f50d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:01.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.970+0000 7f2fd702b640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2fd010ce70 0x7f2fd0172f40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:01.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.970+0000 7f2fd702b640 1 --2- 10.20.193.28:0/1497312344 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2fd010c480 0x7f2fd00f49d0 unknown :-1 s=CLOSED pgs=207 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:01.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.970+0000 7f2fd702b640 1 -- 10.20.193.28:0/1497312344 >> 10.20.193.28:0/1497312344 conn(0x7f2fd00e9d50 msgr2=0x7f2fd010a910 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:01.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.970+0000 7f2fd702b640 1 -- 10.20.193.28:0/1497312344 shutdown_connections 2026-02-25T11:47:01.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:01.970+0000 7f2fd702b640 1 -- 10.20.193.28:0/1497312344 wait complete. 2026-02-25T11:47:02.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:02 trial119 ceph-mon[42710]: Deploying daemon mds.h on trial119 2026-02-25T11:47:02.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:02 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.167:6836/3206735083,v1:10.20.193.167:6837/3206735083] up:boot 2026-02-25T11:47:02.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:02 trial119 ceph-mon[42710]: fsmap 7 up:standby 2026-02-25T11:47:02.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:02 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2026-02-25T11:47:02.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:02 trial119 ceph-mon[42710]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:02.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:02 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1497312344' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:02.260 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 11:47:01 trial119 podman[95473]: 2026-02-25 11:47:01.962289809 +0000 UTC m=+0.028161997 container create 7aae0e577c666a13b5f230a6bebe7c5a99a71ffe8118dda9ebdcb68d0574d931 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-h, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True, org.label-schema.build-date=20260223, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, CEPH_REF=wip-vshankar-testing-20260224.100235) 2026-02-25T11:47:02.260 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 11:47:02 trial119 podman[95473]: 2026-02-25 11:47:02.003226889 +0000 UTC m=+0.069099067 container init 7aae0e577c666a13b5f230a6bebe7c5a99a71ffe8118dda9ebdcb68d0574d931 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-h, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.build-date=20260223, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS, FROM_IMAGE=quay.io/centos/centos:stream9, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, org.label-schema.schema-version=1.0) 2026-02-25T11:47:02.260 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 11:47:02 trial119 podman[95473]: 2026-02-25 11:47:02.00863312 +0000 UTC m=+0.074505308 container start 7aae0e577c666a13b5f230a6bebe7c5a99a71ffe8118dda9ebdcb68d0574d931 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-h, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.41.3, ceph=True, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, org.label-schema.build-date=20260223, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.schema-version=1.0) 2026-02-25T11:47:02.260 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 11:47:02 trial119 bash[95473]: 7aae0e577c666a13b5f230a6bebe7c5a99a71ffe8118dda9ebdcb68d0574d931 2026-02-25T11:47:02.261 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 11:47:02 trial119 podman[95473]: 2026-02-25 11:47:01.95117159 +0000 UTC m=+0.017043778 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:47:02.261 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 11:47:02 trial119 systemd[1]: Started Ceph mds.h for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:47:02.261 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 11:47:02 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-h[95489]: starting mds.h at 2026-02-25T11:47:02.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:02 trial028 ceph-mon[39224]: Deploying daemon mds.h on trial119 2026-02-25T11:47:02.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:02 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.167:6836/3206735083,v1:10.20.193.167:6837/3206735083] up:boot 2026-02-25T11:47:02.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:02 trial028 ceph-mon[39224]: fsmap 7 up:standby 2026-02-25T11:47:02.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:02 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2026-02-25T11:47:02.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:02 trial028 ceph-mon[39224]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:02.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:02 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1497312344' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:02.425 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:02 trial167 ceph-mon[43392]: Deploying daemon mds.h on trial119 2026-02-25T11:47:02.425 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:02 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.167:6836/3206735083,v1:10.20.193.167:6837/3206735083] up:boot 2026-02-25T11:47:02.425 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:02 trial167 ceph-mon[43392]: fsmap 7 up:standby 2026-02-25T11:47:02.425 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:02 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2026-02-25T11:47:02.425 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:02 trial167 ceph-mon[43392]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:02.425 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:02 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1497312344' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:03.233 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:03.233 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:03.233 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:03.233 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:03.233 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:03.233 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:03.233 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:03 trial167 ceph-mon[43392]: Deploying daemon mds.i on trial167 2026-02-25T11:47:03.234 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:03 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.119:6836/392187157,v1:10.20.193.119:6837/392187157] up:boot 2026-02-25T11:47:03.234 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:03 trial167 ceph-mon[43392]: fsmap 8 up:standby 2026-02-25T11:47:03.234 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:03 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2026-02-25T11:47:03.500 INFO:journalctl@ceph.mds.i.trial167.stdout:Feb 25 11:47:03 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-i[98316]: starting mds.i at 2026-02-25T11:47:03.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:03.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:03.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:03.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:03.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:03.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:03.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:03 trial119 ceph-mon[42710]: Deploying daemon mds.i on trial167 2026-02-25T11:47:03.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:03 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.119:6836/392187157,v1:10.20.193.119:6837/392187157] up:boot 2026-02-25T11:47:03.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:03 trial119 ceph-mon[42710]: fsmap 8 up:standby 2026-02-25T11:47:03.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:03 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2026-02-25T11:47:03.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:03.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:03.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:03.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:03.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:03.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:03.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:03 trial028 ceph-mon[39224]: Deploying daemon mds.i on trial167 2026-02-25T11:47:03.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:03 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.119:6836/392187157,v1:10.20.193.119:6837/392187157] up:boot 2026-02-25T11:47:03.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:03 trial028 ceph-mon[39224]: fsmap 8 up:standby 2026-02-25T11:47:03.562 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:03 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2026-02-25T11:47:04.109 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:47:04.331 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:04 trial028 ceph-mon[39224]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:04.331 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:04.331 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:04.332 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:04.332 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:04.332 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:04.332 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:04 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:04.438 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:04.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:04 trial167 ceph-mon[43392]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:04.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:04.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:04.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:04.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:04.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:04.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:04 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:04.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:04 trial119 ceph-mon[42710]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:04.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:04.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:04.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:04.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:04.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:04.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:04 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:04.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.621+0000 7f6105189640 1 -- 10.20.193.28:0/1015451849 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f610010d3d0 msgr2=0x7f610010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:04.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.621+0000 7f6105189640 1 --2- 10.20.193.28:0/1015451849 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f610010d3d0 0x7f610010d7a0 secure :-1 s=READY pgs=80 gs=1 cs=0 l=1 c_cookie=11168296074981989023 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60ec009890 tx=0x7f60ec02f120 comp rx=0 tx=0).stop 2026-02-25T11:47:04.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.621+0000 7f6105189640 1 -- 10.20.193.28:0/1015451849 shutdown_connections 2026-02-25T11:47:04.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.621+0000 7f6105189640 1 --2- 10.20.193.28:0/1015451849 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61001135c0 0x7f6100173f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:04.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.621+0000 7f6105189640 1 --2- 10.20.193.28:0/1015451849 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f610010ddc0 0x7f6100112e10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:04.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.621+0000 7f6105189640 1 --2- 10.20.193.28:0/1015451849 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f610010d3d0 0x7f610010d7a0 unknown :-1 s=CLOSED pgs=80 gs=1 cs=0 l=1 c_cookie=11168296074981989023 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:04.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.621+0000 7f6105189640 1 -- 10.20.193.28:0/1015451849 >> 10.20.193.28:0/1015451849 conn(0x7f61000eaca0 msgr2=0x7f610010b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:04.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.621+0000 7f6105189640 1 -- 10.20.193.28:0/1015451849 shutdown_connections 2026-02-25T11:47:04.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.621+0000 7f6105189640 1 -- 10.20.193.28:0/1015451849 wait complete. 2026-02-25T11:47:04.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.623+0000 7f6105189640 1 Processor -- start 2026-02-25T11:47:04.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.623+0000 7f6105189640 1 -- start start 2026-02-25T11:47:04.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f6105189640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f610010d3d0 0x7f61000eef20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:04.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f6105189640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f610010ddc0 0x7f6100173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:04.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f6105189640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61001135c0 0x7f61000eb1f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:04.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f6105189640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f61000eb980 con 0x7f610010d3d0 2026-02-25T11:47:04.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f6105189640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f61000ebaf0 con 0x7f61001135c0 2026-02-25T11:47:04.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f6105189640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f61000ebcb0 con 0x7f610010ddc0 2026-02-25T11:47:04.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f60fffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f610010d3d0 0x7f61000eef20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:04.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f60fffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f610010d3d0 0x7f61000eef20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:36508/0 (socket says 10.20.193.28:36508) 2026-02-25T11:47:04.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f60fffff640 1 -- 10.20.193.28:0/55784293 learned_addr learned my addr 10.20.193.28:0/55784293 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:04.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f6104988640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61001135c0 0x7f61000eb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:04.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f60ff7fe640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f610010ddc0 0x7f6100173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:04.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f60fffff640 1 -- 10.20.193.28:0/55784293 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f610010ddc0 msgr2=0x7f6100173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:04.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f60fffff640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f610010ddc0 0x7f6100173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:04.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f60fffff640 1 -- 10.20.193.28:0/55784293 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61001135c0 msgr2=0x7f61000eb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:04.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f60fffff640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61001135c0 0x7f61000eb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:04.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f60fffff640 1 -- 10.20.193.28:0/55784293 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f400a650 con 0x7f610010d3d0 2026-02-25T11:47:04.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.624+0000 7f6104988640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61001135c0 0x7f61000eb1f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:04.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.625+0000 7f60fffff640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f610010d3d0 0x7f61000eef20 secure :-1 s=READY pgs=209 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60ec004640 tx=0x7f60ec004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:04.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.625+0000 7f60fd7fa640 1 -- 10.20.193.28:0/55784293 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f60ec03d070 con 0x7f610010d3d0 2026-02-25T11:47:04.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.625+0000 7f60fd7fa640 1 -- 10.20.193.28:0/55784293 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f60ec0026a0 con 0x7f610010d3d0 2026-02-25T11:47:04.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.625+0000 7f60fd7fa640 1 -- 10.20.193.28:0/55784293 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f60ec0418a0 con 0x7f610010d3d0 2026-02-25T11:47:04.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.625+0000 7f6105189640 1 -- 10.20.193.28:0/55784293 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ec0094b0 con 0x7f610010d3d0 2026-02-25T11:47:04.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.625+0000 7f6105189640 1 -- 10.20.193.28:0/55784293 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6100100260 con 0x7f610010d3d0 2026-02-25T11:47:04.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.625+0000 7f6105189640 1 -- 10.20.193.28:0/55784293 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f610010e6b0 con 0x7f610010d3d0 2026-02-25T11:47:04.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.628+0000 7f60fd7fa640 1 -- 10.20.193.28:0/55784293 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f60ec002810 con 0x7f610010d3d0 2026-02-25T11:47:04.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.629+0000 7f60fd7fa640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60d40b57e0 0x7f60d40d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:04.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.629+0000 7f60fd7fa640 1 -- 10.20.193.28:0/55784293 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7f60ec10af40 con 0x7f610010d3d0 2026-02-25T11:47:04.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.629+0000 7f60fd7fa640 1 -- 10.20.193.28:0/55784293 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f60ec10b340 con 0x7f610010d3d0 2026-02-25T11:47:04.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.629+0000 7f60ff7fe640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60d40b57e0 0x7f60d40d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:04.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.630+0000 7f60ff7fe640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60d40b57e0 0x7f60d40d5c90 secure :-1 s=READY pgs=162 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60f0006fd0 tx=0x7f60f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:04.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.772+0000 7f6105189640 1 -- 10.20.193.28:0/55784293 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f610010d7a0 con 0x7f610010d3d0 2026-02-25T11:47:04.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.772+0000 7f60fd7fa640 1 -- 10.20.193.28:0/55784293 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8043 (secure 0 0 0) 0x7f60ec0cdde0 con 0x7f610010d3d0 2026-02-25T11:47:04.775 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:04.775 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":10,"btime":"2026-02-25T11:47:04:070599+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24706,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24719,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24730,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2026-02-25T11:47:04.775 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 10 2026-02-25T11:47:04.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.775+0000 7f6105189640 1 -- 10.20.193.28:0/55784293 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60d40b57e0 msgr2=0x7f60d40d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:04.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.775+0000 7f6105189640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60d40b57e0 0x7f60d40d5c90 secure :-1 s=READY pgs=162 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60f0006fd0 tx=0x7f60f0008040 comp rx=0 tx=0).stop 2026-02-25T11:47:04.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.775+0000 7f6105189640 1 -- 10.20.193.28:0/55784293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f610010d3d0 msgr2=0x7f61000eef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:04.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.775+0000 7f6105189640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f610010d3d0 0x7f61000eef20 secure :-1 s=READY pgs=209 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60ec004640 tx=0x7f60ec004670 comp rx=0 tx=0).stop 2026-02-25T11:47:04.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.775+0000 7f6105189640 1 -- 10.20.193.28:0/55784293 shutdown_connections 2026-02-25T11:47:04.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.775+0000 7f6105189640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60d40b57e0 0x7f60d40d5c90 unknown :-1 s=CLOSED pgs=162 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:04.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.775+0000 7f6105189640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61001135c0 0x7f61000eb1f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:04.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.775+0000 7f6105189640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f610010ddc0 0x7f6100173e90 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:04.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.775+0000 7f6105189640 1 --2- 10.20.193.28:0/55784293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f610010d3d0 0x7f61000eef20 unknown :-1 s=CLOSED pgs=209 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:04.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.775+0000 7f6105189640 1 -- 10.20.193.28:0/55784293 >> 10.20.193.28:0/55784293 conn(0x7f61000eaca0 msgr2=0x7f610010b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:04.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.775+0000 7f6105189640 1 -- 10.20.193.28:0/55784293 shutdown_connections 2026-02-25T11:47:04.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:04.775+0000 7f6105189640 1 -- 10.20.193.28:0/55784293 wait complete. 2026-02-25T11:47:05.107 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:05 trial028 ceph-mon[39224]: Deploying daemon mds.g on trial028 2026-02-25T11:47:05.107 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:05 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] up:boot 2026-02-25T11:47:05.107 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:05 trial028 ceph-mon[39224]: fsmap 9 up:standby 2026-02-25T11:47:05.107 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:05 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2026-02-25T11:47:05.107 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:05 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:05.107 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:05 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:05.107 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:05 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:05.108 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:05 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:05.108 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:05 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:05.108 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:05 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:05.108 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:05 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/55784293' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:05.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:05 trial167 ceph-mon[43392]: Deploying daemon mds.g on trial028 2026-02-25T11:47:05.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:05 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] up:boot 2026-02-25T11:47:05.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:05 trial167 ceph-mon[43392]: fsmap 9 up:standby 2026-02-25T11:47:05.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:05 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2026-02-25T11:47:05.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:05 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:05.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:05 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:05.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:05 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:05.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:05 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:05.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:05 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:05.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:05 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:05.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:05 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/55784293' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:05.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:05 trial119 ceph-mon[42710]: Deploying daemon mds.g on trial028 2026-02-25T11:47:05.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:05 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] up:boot 2026-02-25T11:47:05.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:05 trial119 ceph-mon[42710]: fsmap 9 up:standby 2026-02-25T11:47:05.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:05 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2026-02-25T11:47:05.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:05 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:05.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:05 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:05.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:05 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:05.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:05 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:05.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:05 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:05.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:05 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:05.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:05 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/55784293' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:05.789 INFO:journalctl@ceph.mds.a.trial028.stdout:Feb 25 11:47:05 trial028 podman[105429]: 2026-02-25 11:47:05.78626972 +0000 UTC m=+0.030456130 container create 23038f7d2dd3b554bbacededc45dd0c1945251c68ebee0dbdec2d855032f7d2c (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-a, CEPH_REF=wip-vshankar-testing-20260224.100235, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, io.buildah.version=1.41.3, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, org.label-schema.vendor=CentOS, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, org.label-schema.schema-version=1.0) 2026-02-25T11:47:06.060 INFO:journalctl@ceph.mds.a.trial028.stdout:Feb 25 11:47:05 trial028 podman[105429]: 2026-02-25 11:47:05.826792115 +0000 UTC m=+0.070978535 container init 23038f7d2dd3b554bbacededc45dd0c1945251c68ebee0dbdec2d855032f7d2c (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-a, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.vendor=CentOS, io.buildah.version=1.41.3, ceph=True, org.label-schema.license=GPLv2, CEPH_REF=wip-vshankar-testing-20260224.100235) 2026-02-25T11:47:06.060 INFO:journalctl@ceph.mds.a.trial028.stdout:Feb 25 11:47:05 trial028 podman[105429]: 2026-02-25 11:47:05.831659259 +0000 UTC m=+0.075845669 container start 23038f7d2dd3b554bbacededc45dd0c1945251c68ebee0dbdec2d855032f7d2c (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-a, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, org.label-schema.schema-version=1.0, ceph=True, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.41.3, CEPH_REF=wip-vshankar-testing-20260224.100235, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-25T11:47:06.060 INFO:journalctl@ceph.mds.a.trial028.stdout:Feb 25 11:47:05 trial028 bash[105429]: 23038f7d2dd3b554bbacededc45dd0c1945251c68ebee0dbdec2d855032f7d2c 2026-02-25T11:47:06.060 INFO:journalctl@ceph.mds.a.trial028.stdout:Feb 25 11:47:05 trial028 podman[105429]: 2026-02-25 11:47:05.773912631 +0000 UTC m=+0.018099041 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:47:06.060 INFO:journalctl@ceph.mds.a.trial028.stdout:Feb 25 11:47:05 trial028 systemd[1]: Started Ceph mds.a for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:47:06.060 INFO:journalctl@ceph.mds.a.trial028.stdout:Feb 25 11:47:05 trial028 ceph-mds[105448]: -- 10.20.193.28:0/1949776399 <== mon.0 v2:10.20.193.28:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x18c1b860 con 0x18d5e000 2026-02-25T11:47:06.060 INFO:journalctl@ceph.mds.a.trial028.stdout:Feb 25 11:47:05 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-a[105444]: starting mds.a at 2026-02-25T11:47:06.334 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:06 trial119 ceph-mon[42710]: Deploying daemon mds.a on trial028 2026-02-25T11:47:06.335 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:06 trial119 ceph-mon[42710]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:06.335 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:06 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] up:boot 2026-02-25T11:47:06.335 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:06 trial119 ceph-mon[42710]: fsmap 10 up:standby 2026-02-25T11:47:06.335 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:06 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2026-02-25T11:47:06.335 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:06 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:06.335 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:06 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:06.335 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:06 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:06.335 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:06 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:06.335 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:06 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:06.336 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:06 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:06.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:06 trial167 ceph-mon[43392]: Deploying daemon mds.a on trial028 2026-02-25T11:47:06.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:06 trial167 ceph-mon[43392]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:06.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:06 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] up:boot 2026-02-25T11:47:06.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:06 trial167 ceph-mon[43392]: fsmap 10 up:standby 2026-02-25T11:47:06.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:06 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2026-02-25T11:47:06.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:06 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:06.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:06 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:06.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:06 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:06.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:06 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:06.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:06 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:06.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:06 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:06.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:06 trial028 ceph-mon[39224]: Deploying daemon mds.a on trial028 2026-02-25T11:47:06.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:06 trial028 ceph-mon[39224]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:06.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:06 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] up:boot 2026-02-25T11:47:06.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:06 trial028 ceph-mon[39224]: fsmap 10 up:standby 2026-02-25T11:47:06.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:06 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2026-02-25T11:47:06.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:06 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:06.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:06 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:06.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:06 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:06.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:06 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2026-02-25T11:47:06.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:06 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2026-02-25T11:47:06.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:06 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:06.882 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:47:07.010 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:47:06 trial119 systemd[1]: Starting Ceph mds.b for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:47:07.079 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:07.212 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:07 trial028 ceph-mon[39224]: Deploying daemon mds.b on trial119 2026-02-25T11:47:07.212 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:07 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.28:6840/1424760953,v1:10.20.193.28:6841/1424760953] up:boot 2026-02-25T11:47:07.212 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:07 trial028 ceph-mon[39224]: fsmap 11 up:standby 2026-02-25T11:47:07.212 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:07 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2026-02-25T11:47:07.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.250+0000 7f8b455bc640 1 -- 10.20.193.28:0/3915764916 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b4010cc80 msgr2=0x7f8b40111cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:07.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.250+0000 7f8b455bc640 1 --2- 10.20.193.28:0/3915764916 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b4010cc80 0x7f8b40111cd0 secure :-1 s=READY pgs=212 gs=1 cs=0 l=1 c_cookie=14854145736594123179 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b30009950 tx=0x7f8b3002f1d0 comp rx=0 tx=0).stop 2026-02-25T11:47:07.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.250+0000 7f8b455bc640 1 -- 10.20.193.28:0/3915764916 shutdown_connections 2026-02-25T11:47:07.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.250+0000 7f8b455bc640 1 --2- 10.20.193.28:0/3915764916 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8b40112480 0x7f8b40172e50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:07.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.250+0000 7f8b455bc640 1 --2- 10.20.193.28:0/3915764916 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b4010cc80 0x7f8b40111cd0 unknown :-1 s=CLOSED pgs=212 gs=1 cs=0 l=1 c_cookie=14854145736594123179 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:07.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.250+0000 7f8b455bc640 1 --2- 10.20.193.28:0/3915764916 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4010c290 0x7f8b4010c660 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:07.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.250+0000 7f8b455bc640 1 -- 10.20.193.28:0/3915764916 >> 10.20.193.28:0/3915764916 conn(0x7f8b400e9b40 msgr2=0x7f8b40109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:07.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.250+0000 7f8b455bc640 1 -- 10.20.193.28:0/3915764916 shutdown_connections 2026-02-25T11:47:07.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.250+0000 7f8b455bc640 1 -- 10.20.193.28:0/3915764916 wait complete. 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.251+0000 7f8b455bc640 1 Processor -- start 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.251+0000 7f8b455bc640 1 -- start start 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.251+0000 7f8b455bc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4010c290 0x7f8b400f5930 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.251+0000 7f8b455bc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8b4010cc80 0x7f8b40172d50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.251+0000 7f8b455bc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b40112480 0x7f8b400f6030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.251+0000 7f8b455bc640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8b400fb240 con 0x7f8b40112480 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.251+0000 7f8b455bc640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8b400fb3b0 con 0x7f8b4010c290 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.251+0000 7f8b455bc640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b400fb570 con 0x7f8b4010cc80 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b44dbb640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b40112480 0x7f8b400f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b3ffff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4010c290 0x7f8b400f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b3ffff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4010c290 0x7f8b400f5930 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41228/0 (socket says 10.20.193.28:41228) 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b44dbb640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b40112480 0x7f8b400f6030 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59276/0 (socket says 10.20.193.28:59276) 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b3ffff640 1 -- 10.20.193.28:0/3088369248 learned_addr learned my addr 10.20.193.28:0/3088369248 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:07.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b3f7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8b4010cc80 0x7f8b40172d50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:07.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b44dbb640 1 -- 10.20.193.28:0/3088369248 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8b4010cc80 msgr2=0x7f8b40172d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:07.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b44dbb640 1 --2- 10.20.193.28:0/3088369248 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8b4010cc80 0x7f8b40172d50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:07.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b44dbb640 1 -- 10.20.193.28:0/3088369248 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4010c290 msgr2=0x7f8b400f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:07.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b44dbb640 1 --2- 10.20.193.28:0/3088369248 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4010c290 0x7f8b400f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:07.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b44dbb640 1 -- 10.20.193.28:0/3088369248 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b30009570 con 0x7f8b40112480 2026-02-25T11:47:07.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b44dbb640 1 --2- 10.20.193.28:0/3088369248 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b40112480 0x7f8b400f6030 secure :-1 s=READY pgs=213 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b3400e720 tx=0x7f8b3400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:07.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b3d7fa640 1 -- 10.20.193.28:0/3088369248 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8b34005500 con 0x7f8b40112480 2026-02-25T11:47:07.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b3d7fa640 1 -- 10.20.193.28:0/3088369248 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8b34005660 con 0x7f8b40112480 2026-02-25T11:47:07.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b3d7fa640 1 -- 10.20.193.28:0/3088369248 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8b34011460 con 0x7f8b40112480 2026-02-25T11:47:07.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b455bc640 1 -- 10.20.193.28:0/3088369248 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b400fb850 con 0x7f8b40112480 2026-02-25T11:47:07.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.252+0000 7f8b455bc640 1 -- 10.20.193.28:0/3088369248 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b400fbcd0 con 0x7f8b40112480 2026-02-25T11:47:07.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.253+0000 7f8b12ffd640 1 -- 10.20.193.28:0/3088369248 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4010d570 con 0x7f8b40112480 2026-02-25T11:47:07.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.255+0000 7f8b3d7fa640 1 -- 10.20.193.28:0/3088369248 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f8b3400c7e0 con 0x7f8b40112480 2026-02-25T11:47:07.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.257+0000 7f8b3d7fa640 1 --2- 10.20.193.28:0/3088369248 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8b140b57d0 0x7f8b140d5c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:07.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.257+0000 7f8b3d7fa640 1 -- 10.20.193.28:0/3088369248 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7f8b340e75b0 con 0x7f8b40112480 2026-02-25T11:47:07.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.257+0000 7f8b3d7fa640 1 -- 10.20.193.28:0/3088369248 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f8b340aa470 con 0x7f8b40112480 2026-02-25T11:47:07.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.257+0000 7f8b3ffff640 1 --2- 10.20.193.28:0/3088369248 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8b140b57d0 0x7f8b140d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:07.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.258+0000 7f8b3ffff640 1 --2- 10.20.193.28:0/3088369248 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8b140b57d0 0x7f8b140d5c80 secure :-1 s=READY pgs=165 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b2c00b2c0 tx=0x7f8b2c00b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:07.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.399+0000 7f8b12ffd640 1 -- 10.20.193.28:0/3088369248 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b400f6a90 con 0x7f8b40112480 2026-02-25T11:47:07.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.400+0000 7f8b3d7fa640 1 -- 10.20.193.28:0/3088369248 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7f8b340ad620 con 0x7f8b40112480 2026-02-25T11:47:07.403 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:07.403 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":12,"btime":"2026-02-25T11:47:06:624414+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24706,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24719,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24730,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24731,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2026-02-25T11:47:07.403 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 12 2026-02-25T11:47:07.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.402+0000 7f8b12ffd640 1 -- 10.20.193.28:0/3088369248 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8b140b57d0 msgr2=0x7f8b140d5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:07.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.402+0000 7f8b12ffd640 1 --2- 10.20.193.28:0/3088369248 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8b140b57d0 0x7f8b140d5c80 secure :-1 s=READY pgs=165 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b2c00b2c0 tx=0x7f8b2c00b4a0 comp rx=0 tx=0).stop 2026-02-25T11:47:07.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.402+0000 7f8b12ffd640 1 -- 10.20.193.28:0/3088369248 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b40112480 msgr2=0x7f8b400f6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:07.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.402+0000 7f8b12ffd640 1 --2- 10.20.193.28:0/3088369248 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b40112480 0x7f8b400f6030 secure :-1 s=READY pgs=213 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b3400e720 tx=0x7f8b3400ebf0 comp rx=0 tx=0).stop 2026-02-25T11:47:07.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.402+0000 7f8b12ffd640 1 -- 10.20.193.28:0/3088369248 shutdown_connections 2026-02-25T11:47:07.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.402+0000 7f8b12ffd640 1 --2- 10.20.193.28:0/3088369248 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8b140b57d0 0x7f8b140d5c80 unknown :-1 s=CLOSED pgs=165 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:07.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.402+0000 7f8b12ffd640 1 --2- 10.20.193.28:0/3088369248 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b40112480 0x7f8b400f6030 unknown :-1 s=CLOSED pgs=213 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:07.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.402+0000 7f8b12ffd640 1 --2- 10.20.193.28:0/3088369248 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8b4010cc80 0x7f8b40172d50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:07.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.402+0000 7f8b12ffd640 1 --2- 10.20.193.28:0/3088369248 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4010c290 0x7f8b400f5930 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:07.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.402+0000 7f8b12ffd640 1 -- 10.20.193.28:0/3088369248 >> 10.20.193.28:0/3088369248 conn(0x7f8b400e9b40 msgr2=0x7f8b4010f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:07.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.402+0000 7f8b12ffd640 1 -- 10.20.193.28:0/3088369248 shutdown_connections 2026-02-25T11:47:07.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:07.402+0000 7f8b12ffd640 1 -- 10.20.193.28:0/3088369248 wait complete. 2026-02-25T11:47:07.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:07 trial167 ceph-mon[43392]: Deploying daemon mds.b on trial119 2026-02-25T11:47:07.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:07 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.28:6840/1424760953,v1:10.20.193.28:6841/1424760953] up:boot 2026-02-25T11:47:07.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:07 trial167 ceph-mon[43392]: fsmap 11 up:standby 2026-02-25T11:47:07.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:07 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2026-02-25T11:47:07.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:07 trial119 ceph-mon[42710]: Deploying daemon mds.b on trial119 2026-02-25T11:47:07.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:07 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.28:6840/1424760953,v1:10.20.193.28:6841/1424760953] up:boot 2026-02-25T11:47:07.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:07 trial119 ceph-mon[42710]: fsmap 11 up:standby 2026-02-25T11:47:07.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:07 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2026-02-25T11:47:07.509 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:47:07 trial119 podman[95768]: 2026-02-25 11:47:07.07170903 +0000 UTC m=+0.028494526 container create c566a26541b54b83b0b1af176d324f3991653ec498f4f5573424bb57c02942ae (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-b, org.opencontainers.image.authors=Ceph Release Team , io.buildah.version=1.41.3, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.vendor=CentOS, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, OSD_FLAVOR=default, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.build-date=20260223, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T11:47:07.510 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:47:07 trial119 podman[95768]: 2026-02-25 11:47:07.112191805 +0000 UTC m=+0.068977301 container init c566a26541b54b83b0b1af176d324f3991653ec498f4f5573424bb57c02942ae (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-b, org.label-schema.build-date=20260223, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.vendor=CentOS, io.buildah.version=1.41.3, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, OSD_FLAVOR=default, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team , ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-25T11:47:07.510 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:47:07 trial119 podman[95768]: 2026-02-25 11:47:07.117280975 +0000 UTC m=+0.074066461 container start c566a26541b54b83b0b1af176d324f3991653ec498f4f5573424bb57c02942ae (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-b, OSD_FLAVOR=default, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.documentation=https://docs.ceph.com/, io.buildah.version=1.41.3, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20260223, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-vshankar-testing-20260224.100235, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa) 2026-02-25T11:47:07.510 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:47:07 trial119 bash[95768]: c566a26541b54b83b0b1af176d324f3991653ec498f4f5573424bb57c02942ae 2026-02-25T11:47:07.510 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:47:07 trial119 podman[95768]: 2026-02-25 11:47:07.060538743 +0000 UTC m=+0.017324239 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:47:07.510 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:47:07 trial119 systemd[1]: Started Ceph mds.b for 123fa7bf-123f-11f1-80da-d404e6e7d460. 2026-02-25T11:47:07.510 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:47:07 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-b[95783]: starting mds.b at 2026-02-25T11:47:08.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:08 trial167 ceph-mon[43392]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:08.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:47:08.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:08 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3088369248' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:08.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:08 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] up:boot 2026-02-25T11:47:08.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:08 trial167 ceph-mon[43392]: fsmap 12 up:standby 2026-02-25T11:47:08.501 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:08 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2026-02-25T11:47:08.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:08 trial119 ceph-mon[42710]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:08.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:47:08.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:08 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3088369248' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:08.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:08 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] up:boot 2026-02-25T11:47:08.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:08 trial119 ceph-mon[42710]: fsmap 12 up:standby 2026-02-25T11:47:08.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:08 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2026-02-25T11:47:08.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:08 trial028 ceph-mon[39224]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:08.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:08.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:47:08.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:08 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3088369248' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:08.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:08 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] up:boot 2026-02-25T11:47:08.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:08 trial028 ceph-mon[39224]: fsmap 12 up:standby 2026-02-25T11:47:08.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:08 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2026-02-25T11:47:09.511 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:47:09.703 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:09.746 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.747 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.747 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.747 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.747 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.747 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.747 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:09.747 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:47:09.747 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.747 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:47:09.747 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:47:09.747 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:09 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.747 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:09 trial028 ceph-mon[39224]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:09.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:09.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:47:09.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:47:09.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:47:09.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:09 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:09 trial167 ceph-mon[43392]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:09.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:47:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:47:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:47:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:09 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:09 trial119 ceph-mon[42710]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:09.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.880+0000 7fb1c71e9640 1 -- 10.20.193.28:0/980620395 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb1c010ff90 msgr2=0x7fb1c0114fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:09.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.880+0000 7fb1c71e9640 1 --2- 10.20.193.28:0/980620395 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb1c010ff90 0x7fb1c0114fe0 secure :-1 s=READY pgs=214 gs=1 cs=0 l=1 c_cookie=7500841431948779662 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb1a8009950 tx=0x7fb1a802f190 comp rx=0 tx=0).stop 2026-02-25T11:47:09.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.881+0000 7fb1c71e9640 1 -- 10.20.193.28:0/980620395 shutdown_connections 2026-02-25T11:47:09.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.881+0000 7fb1c71e9640 1 --2- 10.20.193.28:0/980620395 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb1c0115790 0x7fb1c0176160 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:09.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.881+0000 7fb1c71e9640 1 --2- 10.20.193.28:0/980620395 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb1c010ff90 0x7fb1c0114fe0 unknown :-1 s=CLOSED pgs=214 gs=1 cs=0 l=1 c_cookie=7500841431948779662 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:09.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.881+0000 7fb1c71e9640 1 --2- 10.20.193.28:0/980620395 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb1c010f5a0 0x7fb1c010f970 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:09.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.881+0000 7fb1c71e9640 1 -- 10.20.193.28:0/980620395 >> 10.20.193.28:0/980620395 conn(0x7fb1c00ece30 msgr2=0x7fb1c010d240 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:09.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.881+0000 7fb1c71e9640 1 -- 10.20.193.28:0/980620395 shutdown_connections 2026-02-25T11:47:09.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.881+0000 7fb1c71e9640 1 -- 10.20.193.28:0/980620395 wait complete. 2026-02-25T11:47:09.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.881+0000 7fb1c71e9640 1 Processor -- start 2026-02-25T11:47:09.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.881+0000 7fb1c71e9640 1 -- start start 2026-02-25T11:47:09.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c71e9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb1c010f5a0 0x7fb1c00f5850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:09.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c71e9640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb1c010ff90 0x7fb1c0176060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:09.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c71e9640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb1c0115790 0x7fb1c00f5f50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:09.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c71e9640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c00fb160 con 0x7fb1c010f5a0 2026-02-25T11:47:09.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c71e9640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c00fb2d0 con 0x7fb1c0115790 2026-02-25T11:47:09.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c71e9640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c00fb490 con 0x7fb1c010ff90 2026-02-25T11:47:09.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c4f5e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb1c010f5a0 0x7fb1c00f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:09.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c4f5e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb1c010f5a0 0x7fb1c00f5850 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59290/0 (socket says 10.20.193.28:59290) 2026-02-25T11:47:09.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c4f5e640 1 -- 10.20.193.28:0/4206797057 learned_addr learned my addr 10.20.193.28:0/4206797057 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:09.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c575f640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb1c0115790 0x7fb1c00f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:09.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1b7fff640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb1c010ff90 0x7fb1c0176060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:09.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c4f5e640 1 -- 10.20.193.28:0/4206797057 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb1c010ff90 msgr2=0x7fb1c0176060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:09.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c4f5e640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb1c010ff90 0x7fb1c0176060 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:09.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c4f5e640 1 -- 10.20.193.28:0/4206797057 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb1c0115790 msgr2=0x7fb1c00f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:09.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c4f5e640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb1c0115790 0x7fb1c00f5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:09.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.882+0000 7fb1c4f5e640 1 -- 10.20.193.28:0/4206797057 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a8009570 con 0x7fb1c010f5a0 2026-02-25T11:47:09.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.883+0000 7fb1c4f5e640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb1c010f5a0 0x7fb1c00f5850 secure :-1 s=READY pgs=215 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb1b000c9b0 tx=0x7fb1b000ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:09.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.883+0000 7fb1b5ffb640 1 -- 10.20.193.28:0/4206797057 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb1b00044a0 con 0x7fb1c010f5a0 2026-02-25T11:47:09.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.883+0000 7fb1b5ffb640 1 -- 10.20.193.28:0/4206797057 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb1b0007d80 con 0x7fb1c010f5a0 2026-02-25T11:47:09.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.883+0000 7fb1b5ffb640 1 -- 10.20.193.28:0/4206797057 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb1b000f6d0 con 0x7fb1c010f5a0 2026-02-25T11:47:09.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.883+0000 7fb1c71e9640 1 -- 10.20.193.28:0/4206797057 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c00fb770 con 0x7fb1c010f5a0 2026-02-25T11:47:09.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.883+0000 7fb1c71e9640 1 -- 10.20.193.28:0/4206797057 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c00fbbf0 con 0x7fb1c010f5a0 2026-02-25T11:47:09.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.883+0000 7fb1c71e9640 1 -- 10.20.193.28:0/4206797057 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb188005300 con 0x7fb1c010f5a0 2026-02-25T11:47:09.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.886+0000 7fb1b5ffb640 1 -- 10.20.193.28:0/4206797057 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7fb1b00078c0 con 0x7fb1c010f5a0 2026-02-25T11:47:09.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.887+0000 7fb1b5ffb640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb19c0b5830 0x7fb19c0d5ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:09.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.888+0000 7fb1b7fff640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb19c0b5830 0x7fb19c0d5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:09.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.888+0000 7fb1b5ffb640 1 -- 10.20.193.28:0/4206797057 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7fb1b00e6840 con 0x7fb1c010f5a0 2026-02-25T11:47:09.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.888+0000 7fb1b5ffb640 1 -- 10.20.193.28:0/4206797057 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fb1b011f960 con 0x7fb1c010f5a0 2026-02-25T11:47:09.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:09.888+0000 7fb1b7fff640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb19c0b5830 0x7fb19c0d5ce0 secure :-1 s=READY pgs=167 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb1c00fc1f0 tx=0x7fb1a803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:10.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.031+0000 7fb1c71e9640 1 -- 10.20.193.28:0/4206797057 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb188005950 con 0x7fb1c010f5a0 2026-02-25T11:47:10.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.032+0000 7fb1b5ffb640 1 -- 10.20.193.28:0/4206797057 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10524 (secure 0 0 0) 0x7fb1b00a9740 con 0x7fb1c010f5a0 2026-02-25T11:47:10.034 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:10.035 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":13,"btime":"2026-02-25T11:47:08:081017+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24706,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24719,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24730,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24731,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24749,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2026-02-25T11:47:10.035 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 13 2026-02-25T11:47:10.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.033+0000 7fb1c71e9640 1 -- 10.20.193.28:0/4206797057 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb19c0b5830 msgr2=0x7fb19c0d5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:10.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.033+0000 7fb1c71e9640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb19c0b5830 0x7fb19c0d5ce0 secure :-1 s=READY pgs=167 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb1c00fc1f0 tx=0x7fb1a803a040 comp rx=0 tx=0).stop 2026-02-25T11:47:10.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.033+0000 7fb1c71e9640 1 -- 10.20.193.28:0/4206797057 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb1c010f5a0 msgr2=0x7fb1c00f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:10.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.033+0000 7fb1c71e9640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb1c010f5a0 0x7fb1c00f5850 secure :-1 s=READY pgs=215 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb1b000c9b0 tx=0x7fb1b000ce80 comp rx=0 tx=0).stop 2026-02-25T11:47:10.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.033+0000 7fb1c71e9640 1 -- 10.20.193.28:0/4206797057 shutdown_connections 2026-02-25T11:47:10.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.033+0000 7fb1c71e9640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb19c0b5830 0x7fb19c0d5ce0 unknown :-1 s=CLOSED pgs=167 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:10.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.033+0000 7fb1c71e9640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb1c0115790 0x7fb1c00f5f50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:10.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.033+0000 7fb1c71e9640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb1c010ff90 0x7fb1c0176060 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:10.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.033+0000 7fb1c71e9640 1 --2- 10.20.193.28:0/4206797057 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb1c010f5a0 0x7fb1c00f5850 unknown :-1 s=CLOSED pgs=215 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:10.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.033+0000 7fb1c71e9640 1 -- 10.20.193.28:0/4206797057 >> 10.20.193.28:0/4206797057 conn(0x7fb1c00ece30 msgr2=0x7fb1c0111da0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:10.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.033+0000 7fb1c71e9640 1 -- 10.20.193.28:0/4206797057 shutdown_connections 2026-02-25T11:47:10.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.033+0000 7fb1c71e9640 1 -- 10.20.193.28:0/4206797057 wait complete. 2026-02-25T11:47:10.137 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph --format=json-pretty osd lspools 2026-02-25T11:47:10.320 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:10.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.508+0000 7f66ae2b7640 1 -- 10.20.193.28:0/3697148155 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f66a8060b00 msgr2=0x7f66a8112e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:10.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.508+0000 7f66ae2b7640 1 --2- 10.20.193.28:0/3697148155 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f66a8060b00 0x7f66a8112e00 secure :-1 s=READY pgs=216 gs=1 cs=0 l=1 c_cookie=7685848114945955434 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f669c009950 tx=0x7f669c02f190 comp rx=0 tx=0).stop 2026-02-25T11:47:10.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.508+0000 7f66ae2b7640 1 -- 10.20.193.28:0/3697148155 shutdown_connections 2026-02-25T11:47:10.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.508+0000 7f66ae2b7640 1 --2- 10.20.193.28:0/3697148155 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f66a81135b0 0x7f66a8173f80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:10.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.508+0000 7f66ae2b7640 1 --2- 10.20.193.28:0/3697148155 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f66a8060b00 0x7f66a8112e00 unknown :-1 s=CLOSED pgs=216 gs=1 cs=0 l=1 c_cookie=7685848114945955434 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:10.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.508+0000 7f66ae2b7640 1 --2- 10.20.193.28:0/3697148155 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f66a8060110 0x7f66a80604e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:10.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.508+0000 7f66ae2b7640 1 -- 10.20.193.28:0/3697148155 >> 10.20.193.28:0/3697148155 conn(0x7f66a80e9b50 msgr2=0x7f66a8109f60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:10.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.508+0000 7f66ae2b7640 1 -- 10.20.193.28:0/3697148155 shutdown_connections 2026-02-25T11:47:10.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.508+0000 7f66ae2b7640 1 -- 10.20.193.28:0/3697148155 wait complete. 2026-02-25T11:47:10.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.509+0000 7f66ae2b7640 1 Processor -- start 2026-02-25T11:47:10.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.509+0000 7f66ae2b7640 1 -- start start 2026-02-25T11:47:10.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.509+0000 7f66ae2b7640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f66a8060110 0x7f66a810d990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:10.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.509+0000 7f66ae2b7640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f66a8060b00 0x7f66a8173e80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:10.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.509+0000 7f66ae2b7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f66a81135b0 0x7f66a810bf10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:10.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.509+0000 7f66ae2b7640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f66a810c510 con 0x7f66a81135b0 2026-02-25T11:47:10.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.509+0000 7f66ae2b7640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f66a810c680 con 0x7f66a8060110 2026-02-25T11:47:10.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.509+0000 7f66ae2b7640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f66a810c840 con 0x7f66a8060b00 2026-02-25T11:47:10.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.509+0000 7f66adab6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f66a81135b0 0x7f66a810bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:10.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66ad2b5640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f66a8060110 0x7f66a810d990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:10.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66acab4640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f66a8060b00 0x7f66a8173e80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:10.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66ad2b5640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f66a8060110 0x7f66a810d990 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41274/0 (socket says 10.20.193.28:41274) 2026-02-25T11:47:10.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66ad2b5640 1 -- 10.20.193.28:0/1802010144 learned_addr learned my addr 10.20.193.28:0/1802010144 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:10.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66adab6640 1 -- 10.20.193.28:0/1802010144 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f66a8060b00 msgr2=0x7f66a8173e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:10.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66adab6640 1 --2- 10.20.193.28:0/1802010144 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f66a8060b00 0x7f66a8173e80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:10.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66adab6640 1 -- 10.20.193.28:0/1802010144 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f66a8060110 msgr2=0x7f66a810d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:10.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66adab6640 1 --2- 10.20.193.28:0/1802010144 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f66a8060110 0x7f66a810d990 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:10.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66adab6640 1 -- 10.20.193.28:0/1802010144 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669c009570 con 0x7f66a81135b0 2026-02-25T11:47:10.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66adab6640 1 --2- 10.20.193.28:0/1802010144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f66a81135b0 0x7f66a810bf10 secure :-1 s=READY pgs=217 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f66a400f940 tx=0x7f66a400fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:10.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66967fc640 1 -- 10.20.193.28:0/1802010144 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f66a400dce0 con 0x7f66a81135b0 2026-02-25T11:47:10.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66967fc640 1 -- 10.20.193.28:0/1802010144 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f66a400de40 con 0x7f66a81135b0 2026-02-25T11:47:10.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66967fc640 1 -- 10.20.193.28:0/1802010144 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f66a40116b0 con 0x7f66a81135b0 2026-02-25T11:47:10.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66ae2b7640 1 -- 10.20.193.28:0/1802010144 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a810cac0 con 0x7f66a81135b0 2026-02-25T11:47:10.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.510+0000 7f66ae2b7640 1 -- 10.20.193.28:0/1802010144 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a80ff4a0 con 0x7f66a81135b0 2026-02-25T11:47:10.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.511+0000 7f66ae2b7640 1 -- 10.20.193.28:0/1802010144 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a810e6a0 con 0x7f66a81135b0 2026-02-25T11:47:10.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.513+0000 7f66967fc640 1 -- 10.20.193.28:0/1802010144 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f66a4020060 con 0x7f66a81135b0 2026-02-25T11:47:10.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.515+0000 7f66967fc640 1 --2- 10.20.193.28:0/1802010144 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f66840b56e0 0x7f66840d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:10.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.515+0000 7f66967fc640 1 -- 10.20.193.28:0/1802010144 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7f66a4015070 con 0x7f66a81135b0 2026-02-25T11:47:10.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.515+0000 7f66ad2b5640 1 --2- 10.20.193.28:0/1802010144 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f66840b56e0 0x7f66840d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:10.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.515+0000 7f66967fc640 1 -- 10.20.193.28:0/1802010144 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f66a40e7a90 con 0x7f66a81135b0 2026-02-25T11:47:10.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.515+0000 7f66ad2b5640 1 --2- 10.20.193.28:0/1802010144 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f66840b56e0 0x7f66840d5b90 secure :-1 s=READY pgs=168 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f66980095b0 tx=0x7f6698005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:10.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.631+0000 7f66ae2b7640 1 -- 10.20.193.28:0/1802010144 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f66a80604e0 con 0x7f66a81135b0 2026-02-25T11:47:10.634 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:10.634 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:10.634 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:10.635 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:10.635 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:10.635 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:10.635 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:10.635 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:47:10.635 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:10.635 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:47:10.635 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:10 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/4206797057' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:10.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.631+0000 7f66967fc640 1 -- 10.20.193.28:0/1802010144 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f66a40aa640 con 0x7f66a81135b0 2026-02-25T11:47:10.636 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:10.636 INFO:teuthology.orchestra.run.trial028.stdout:[ 2026-02-25T11:47:10.636 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:47:10.636 INFO:teuthology.orchestra.run.trial028.stdout: "poolnum": 1, 2026-02-25T11:47:10.636 INFO:teuthology.orchestra.run.trial028.stdout: "poolname": ".mgr" 2026-02-25T11:47:10.637 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:47:10.637 INFO:teuthology.orchestra.run.trial028.stdout:] 2026-02-25T11:47:10.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.633+0000 7f66ae2b7640 1 -- 10.20.193.28:0/1802010144 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f66840b56e0 msgr2=0x7f66840d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:10.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.633+0000 7f66ae2b7640 1 --2- 10.20.193.28:0/1802010144 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f66840b56e0 0x7f66840d5b90 secure :-1 s=READY pgs=168 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f66980095b0 tx=0x7f6698005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:10.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.633+0000 7f66ae2b7640 1 -- 10.20.193.28:0/1802010144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f66a81135b0 msgr2=0x7f66a810bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:10.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.633+0000 7f66ae2b7640 1 --2- 10.20.193.28:0/1802010144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f66a81135b0 0x7f66a810bf10 secure :-1 s=READY pgs=217 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f66a400f940 tx=0x7f66a400fe10 comp rx=0 tx=0).stop 2026-02-25T11:47:10.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.633+0000 7f66ae2b7640 1 -- 10.20.193.28:0/1802010144 shutdown_connections 2026-02-25T11:47:10.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.633+0000 7f66ae2b7640 1 --2- 10.20.193.28:0/1802010144 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f66840b56e0 0x7f66840d5b90 unknown :-1 s=CLOSED pgs=168 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:10.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.633+0000 7f66ae2b7640 1 --2- 10.20.193.28:0/1802010144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f66a81135b0 0x7f66a810bf10 unknown :-1 s=CLOSED pgs=217 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:10.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.633+0000 7f66ae2b7640 1 --2- 10.20.193.28:0/1802010144 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f66a8060b00 0x7f66a8173e80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:10.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.633+0000 7f66ae2b7640 1 --2- 10.20.193.28:0/1802010144 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f66a8060110 0x7f66a810d990 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:10.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.633+0000 7f66ae2b7640 1 -- 10.20.193.28:0/1802010144 >> 10.20.193.28:0/1802010144 conn(0x7f66a80e9b50 msgr2=0x7f66a8110c10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:10.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.634+0000 7f66ae2b7640 1 -- 10.20.193.28:0/1802010144 shutdown_connections 2026-02-25T11:47:10.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:10.634+0000 7f66ae2b7640 1 -- 10.20.193.28:0/1802010144 wait complete. 2026-02-25T11:47:10.736 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2026-02-25T11:47:10.736 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2026-02-25T11:47:10.920 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:10.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:10.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:10.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:10.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:10.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:10.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:11.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:11.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:47:11.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:11.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:47:11.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:10 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/4206797057' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:11.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:11.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:11.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:11.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:11.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:11.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:11.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:47:11.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:47:11.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:47:11.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:47:11.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:10 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/4206797057' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:11.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.100+0000 7f21c5937640 1 -- 10.20.193.28:0/3528446324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21c010f840 msgr2=0x7f21c010fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:11.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.100+0000 7f21c5937640 1 --2- 10.20.193.28:0/3528446324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21c010f840 0x7f21c010fc10 secure :-1 s=READY pgs=218 gs=1 cs=0 l=1 c_cookie=17021078008638344636 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f21b4009950 tx=0x7f21b402f190 comp rx=0 tx=0).stop 2026-02-25T11:47:11.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.101+0000 7f21c5937640 1 -- 10.20.193.28:0/3528446324 shutdown_connections 2026-02-25T11:47:11.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.101+0000 7f21c5937640 1 --2- 10.20.193.28:0/3528446324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21c0115a30 0x7f21c0176400 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:11.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.101+0000 7f21c5937640 1 --2- 10.20.193.28:0/3528446324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f21c0110230 0x7f21c0115280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:11.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.101+0000 7f21c5937640 1 --2- 10.20.193.28:0/3528446324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21c010f840 0x7f21c010fc10 unknown :-1 s=CLOSED pgs=218 gs=1 cs=0 l=1 c_cookie=17021078008638344636 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:11.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.101+0000 7f21c5937640 1 -- 10.20.193.28:0/3528446324 >> 10.20.193.28:0/3528446324 conn(0x7f21c00ed130 msgr2=0x7f21c010d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:11.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.101+0000 7f21c5937640 1 -- 10.20.193.28:0/3528446324 shutdown_connections 2026-02-25T11:47:11.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.101+0000 7f21c5937640 1 -- 10.20.193.28:0/3528446324 wait complete. 2026-02-25T11:47:11.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.101+0000 7f21c5937640 1 Processor -- start 2026-02-25T11:47:11.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21c5937640 1 -- start start 2026-02-25T11:47:11.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21c5937640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21c010f840 0x7f21c011e190 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:11.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21c5937640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f21c0110230 0x7f21c011e6d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:11.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21c5937640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21c0115a30 0x7f21c0123aa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:11.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21c5937640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f21c01240c0 con 0x7f21c0115a30 2026-02-25T11:47:11.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21c5937640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0124230 con 0x7f21c0110230 2026-02-25T11:47:11.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21c5937640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f21c01243f0 con 0x7f21c010f840 2026-02-25T11:47:11.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21bf7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21c0115a30 0x7f21c0123aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:11.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21bf7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21c0115a30 0x7f21c0123aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59326/0 (socket says 10.20.193.28:59326) 2026-02-25T11:47:11.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21bf7fe640 1 -- 10.20.193.28:0/2538845496 learned_addr learned my addr 10.20.193.28:0/2538845496 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:11.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21beffd640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21c010f840 0x7f21c011e190 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:11.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21be7fc640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f21c0110230 0x7f21c011e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:11.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21bf7fe640 1 -- 10.20.193.28:0/2538845496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21c010f840 msgr2=0x7f21c011e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:11.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21bf7fe640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21c010f840 0x7f21c011e190 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:11.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21bf7fe640 1 -- 10.20.193.28:0/2538845496 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f21c0110230 msgr2=0x7f21c011e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:11.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21bf7fe640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f21c0110230 0x7f21c011e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:11.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21bf7fe640 1 -- 10.20.193.28:0/2538845496 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b4009570 con 0x7f21c0115a30 2026-02-25T11:47:11.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.102+0000 7f21beffd640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21c010f840 0x7f21c011e190 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:11.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.103+0000 7f21bf7fe640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21c0115a30 0x7f21c0123aa0 secure :-1 s=READY pgs=219 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f21b0006e90 tx=0x7f21b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:11.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.103+0000 7f21be7fc640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f21c0110230 0x7f21c011e6d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:11.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.103+0000 7f21c4935640 1 -- 10.20.193.28:0/2538845496 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f21b0005500 con 0x7f21c0115a30 2026-02-25T11:47:11.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.103+0000 7f21c5937640 1 -- 10.20.193.28:0/2538845496 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c01246d0 con 0x7f21c0115a30 2026-02-25T11:47:11.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.103+0000 7f21c5937640 1 -- 10.20.193.28:0/2538845496 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c0164dd0 con 0x7f21c0115a30 2026-02-25T11:47:11.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.103+0000 7f21c4935640 1 -- 10.20.193.28:0/2538845496 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f21b0005df0 con 0x7f21c0115a30 2026-02-25T11:47:11.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.103+0000 7f21c4935640 1 -- 10.20.193.28:0/2538845496 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f21b0003a00 con 0x7f21c0115a30 2026-02-25T11:47:11.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.103+0000 7f21c5937640 1 -- 10.20.193.28:0/2538845496 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c0110b20 con 0x7f21c0115a30 2026-02-25T11:47:11.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.105+0000 7f21c4935640 1 -- 10.20.193.28:0/2538845496 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f21b0003b60 con 0x7f21c0115a30 2026-02-25T11:47:11.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.107+0000 7f21c4935640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f21940b5760 0x7f21940d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:11.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.107+0000 7f21c4935640 1 -- 10.20.193.28:0/2538845496 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7119+0+0 (secure 0 0 0) 0x7f21b00e77d0 con 0x7f21c0115a30 2026-02-25T11:47:11.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.107+0000 7f21beffd640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f21940b5760 0x7f21940d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:11.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.108+0000 7f21c4935640 1 -- 10.20.193.28:0/2538845496 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f21b00e7c50 con 0x7f21c0115a30 2026-02-25T11:47:11.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.108+0000 7f21beffd640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f21940b5760 0x7f21940d5c10 secure :-1 s=READY pgs=169 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f21b40023d0 tx=0x7f21b4031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:11.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.226+0000 7f21c5937640 1 -- 10.20.193.28:0/2538845496 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f21c011f2f0 con 0x7f21c0115a30 2026-02-25T11:47:11.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.597+0000 7f21c4935640 1 -- 10.20.193.28:0/2538845496 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f21b00aa5e0 con 0x7f21c0115a30 2026-02-25T11:47:11.600 INFO:teuthology.orchestra.run.trial028.stderr:pool 'cephfs_metadata' created 2026-02-25T11:47:11.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.600+0000 7f21c5937640 1 -- 10.20.193.28:0/2538845496 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f21940b5760 msgr2=0x7f21940d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:11.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.600+0000 7f21c5937640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f21940b5760 0x7f21940d5c10 secure :-1 s=READY pgs=169 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f21b40023d0 tx=0x7f21b4031040 comp rx=0 tx=0).stop 2026-02-25T11:47:11.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.600+0000 7f21c5937640 1 -- 10.20.193.28:0/2538845496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21c0115a30 msgr2=0x7f21c0123aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:11.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.600+0000 7f21c5937640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21c0115a30 0x7f21c0123aa0 secure :-1 s=READY pgs=219 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f21b0006e90 tx=0x7f21b000edf0 comp rx=0 tx=0).stop 2026-02-25T11:47:11.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.600+0000 7f21c5937640 1 -- 10.20.193.28:0/2538845496 shutdown_connections 2026-02-25T11:47:11.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.600+0000 7f21c5937640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f21940b5760 0x7f21940d5c10 unknown :-1 s=CLOSED pgs=169 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:11.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.600+0000 7f21c5937640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f21c0115a30 0x7f21c0123aa0 unknown :-1 s=CLOSED pgs=219 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:11.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.600+0000 7f21c5937640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f21c0110230 0x7f21c011e6d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:11.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.600+0000 7f21c5937640 1 --2- 10.20.193.28:0/2538845496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21c010f840 0x7f21c011e190 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:11.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.600+0000 7f21c5937640 1 -- 10.20.193.28:0/2538845496 >> 10.20.193.28:0/2538845496 conn(0x7f21c00ed130 msgr2=0x7f21c010d510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:11.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.600+0000 7f21c5937640 1 -- 10.20.193.28:0/2538845496 shutdown_connections 2026-02-25T11:47:11.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:11.601+0000 7f21c5937640 1 -- 10.20.193.28:0/2538845496 wait complete. 2026-02-25T11:47:11.704 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2026-02-25T11:47:11.887 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:11.930 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:11 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1802010144' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2026-02-25T11:47:11.931 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:11 trial028 ceph-mon[39224]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:11.931 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:11 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2538845496' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2026-02-25T11:47:11.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:11 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1802010144' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2026-02-25T11:47:11.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:11 trial167 ceph-mon[43392]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:11.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:11 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2538845496' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2026-02-25T11:47:12.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:11 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1802010144' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2026-02-25T11:47:12.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:11 trial119 ceph-mon[42710]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:12.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:11 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2538845496' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2026-02-25T11:47:12.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.065+0000 7fc38c801640 1 -- 10.20.193.28:0/197622539 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc38810eb80 msgr2=0x7fc388153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:12.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.065+0000 7fc38c801640 1 --2- 10.20.193.28:0/197622539 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc38810eb80 0x7fc388153200 secure :-1 s=READY pgs=220 gs=1 cs=0 l=1 c_cookie=3163692013184680432 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc370009950 tx=0x7fc37002f190 comp rx=0 tx=0).stop 2026-02-25T11:47:12.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.065+0000 7fc38c801640 1 -- 10.20.193.28:0/197622539 shutdown_connections 2026-02-25T11:47:12.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.065+0000 7fc38c801640 1 --2- 10.20.193.28:0/197622539 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc388153740 0x7fc388173b60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:12.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.065+0000 7fc38c801640 1 --2- 10.20.193.28:0/197622539 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc38810eb80 0x7fc388153200 unknown :-1 s=CLOSED pgs=220 gs=1 cs=0 l=1 c_cookie=3163692013184680432 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:12.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.065+0000 7fc38c801640 1 --2- 10.20.193.28:0/197622539 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc38810e190 0x7fc38810e560 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:12.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.065+0000 7fc38c801640 1 -- 10.20.193.28:0/197622539 >> 10.20.193.28:0/197622539 conn(0x7fc3880e9b40 msgr2=0x7fc388109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:12.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.065+0000 7fc38c801640 1 -- 10.20.193.28:0/197622539 shutdown_connections 2026-02-25T11:47:12.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.065+0000 7fc38c801640 1 -- 10.20.193.28:0/197622539 wait complete. 2026-02-25T11:47:12.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.066+0000 7fc38c801640 1 Processor -- start 2026-02-25T11:47:12.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.066+0000 7fc38c801640 1 -- start start 2026-02-25T11:47:12.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.066+0000 7fc38c801640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc38810e190 0x7fc3880f1900 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:12.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.066+0000 7fc38c801640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc38810eb80 0x7fc3880f1e40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:12.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.066+0000 7fc38c801640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc388153740 0x7fc3880f71c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:12.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.066+0000 7fc38c801640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880f77e0 con 0x7fc388153740 2026-02-25T11:47:12.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.066+0000 7fc38c801640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880f7950 con 0x7fc38810e190 2026-02-25T11:47:12.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.066+0000 7fc38c801640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc3880f7b10 con 0x7fc38810eb80 2026-02-25T11:47:12.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc387577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc388153740 0x7fc3880f71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc386575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc38810eb80 0x7fc3880f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc386575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc38810eb80 0x7fc3880f1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:33028/0 (socket says 10.20.193.28:33028) 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc386575640 1 -- 10.20.193.28:0/1070499557 learned_addr learned my addr 10.20.193.28:0/1070499557 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc386d76640 1 --2- 10.20.193.28:0/1070499557 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc38810e190 0x7fc3880f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc387577640 1 -- 10.20.193.28:0/1070499557 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc38810eb80 msgr2=0x7fc3880f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc387577640 1 --2- 10.20.193.28:0/1070499557 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc38810eb80 0x7fc3880f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc387577640 1 -- 10.20.193.28:0/1070499557 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc38810e190 msgr2=0x7fc3880f1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc387577640 1 --2- 10.20.193.28:0/1070499557 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc38810e190 0x7fc3880f1900 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc387577640 1 -- 10.20.193.28:0/1070499557 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc370009570 con 0x7fc388153740 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc387577640 1 --2- 10.20.193.28:0/1070499557 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc388153740 0x7fc3880f71c0 secure :-1 s=READY pgs=221 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc37800e5e0 tx=0x7fc37800eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc367fff640 1 -- 10.20.193.28:0/1070499557 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc37800a960 con 0x7fc388153740 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc367fff640 1 -- 10.20.193.28:0/1070499557 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc378011460 con 0x7fc388153740 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc367fff640 1 -- 10.20.193.28:0/1070499557 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc378019680 con 0x7fc388153740 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc38c801640 1 -- 10.20.193.28:0/1070499557 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3880f7df0 con 0x7fc388153740 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.067+0000 7fc38c801640 1 -- 10.20.193.28:0/1070499557 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc38814f360 con 0x7fc388153740 2026-02-25T11:47:12.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.068+0000 7fc38c801640 1 -- 10.20.193.28:0/1070499557 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc34c005300 con 0x7fc388153740 2026-02-25T11:47:12.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.071+0000 7fc367fff640 1 -- 10.20.193.28:0/1070499557 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7fc378017020 con 0x7fc388153740 2026-02-25T11:47:12.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.072+0000 7fc367fff640 1 --2- 10.20.193.28:0/1070499557 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc35c0b5760 0x7fc35c0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:12.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.073+0000 7fc367fff640 1 -- 10.20.193.28:0/1070499557 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7492+0+0 (secure 0 0 0) 0x7fc3780e7700 con 0x7fc388153740 2026-02-25T11:47:12.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.073+0000 7fc367fff640 1 -- 10.20.193.28:0/1070499557 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fc3780e7b80 con 0x7fc388153740 2026-02-25T11:47:12.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.073+0000 7fc386d76640 1 --2- 10.20.193.28:0/1070499557 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc35c0b5760 0x7fc35c0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:12.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.073+0000 7fc386d76640 1 --2- 10.20.193.28:0/1070499557 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc35c0b5760 0x7fc35c0d5c10 secure :-1 s=READY pgs=170 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc37c004740 tx=0x7fc37c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:12.190 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.188+0000 7fc38c801640 1 -- 10.20.193.28:0/1070499557 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc34c0055b0 con 0x7fc388153740 2026-02-25T11:47:12.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.600+0000 7fc367fff640 1 -- 10.20.193.28:0/1070499557 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7fc3780aa420 con 0x7fc388153740 2026-02-25T11:47:12.602 INFO:teuthology.orchestra.run.trial028.stderr:pool 'cephfs_data' created 2026-02-25T11:47:12.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.603+0000 7fc38c801640 1 -- 10.20.193.28:0/1070499557 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc35c0b5760 msgr2=0x7fc35c0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:12.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.603+0000 7fc38c801640 1 --2- 10.20.193.28:0/1070499557 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc35c0b5760 0x7fc35c0d5c10 secure :-1 s=READY pgs=170 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc37c004740 tx=0x7fc37c005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:12.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.603+0000 7fc38c801640 1 -- 10.20.193.28:0/1070499557 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc388153740 msgr2=0x7fc3880f71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:12.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.603+0000 7fc38c801640 1 --2- 10.20.193.28:0/1070499557 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc388153740 0x7fc3880f71c0 secure :-1 s=READY pgs=221 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc37800e5e0 tx=0x7fc37800eab0 comp rx=0 tx=0).stop 2026-02-25T11:47:12.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.603+0000 7fc38c801640 1 -- 10.20.193.28:0/1070499557 shutdown_connections 2026-02-25T11:47:12.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.603+0000 7fc38c801640 1 --2- 10.20.193.28:0/1070499557 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc35c0b5760 0x7fc35c0d5c10 unknown :-1 s=CLOSED pgs=170 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:12.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.603+0000 7fc38c801640 1 --2- 10.20.193.28:0/1070499557 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc388153740 0x7fc3880f71c0 unknown :-1 s=CLOSED pgs=221 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:12.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.603+0000 7fc38c801640 1 --2- 10.20.193.28:0/1070499557 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc38810eb80 0x7fc3880f1e40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:12.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.603+0000 7fc38c801640 1 --2- 10.20.193.28:0/1070499557 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc38810e190 0x7fc3880f1900 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:12.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.603+0000 7fc38c801640 1 -- 10.20.193.28:0/1070499557 >> 10.20.193.28:0/1070499557 conn(0x7fc3880e9b40 msgr2=0x7fc388177cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:12.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.603+0000 7fc38c801640 1 -- 10.20.193.28:0/1070499557 shutdown_connections 2026-02-25T11:47:12.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:12.604+0000 7fc38c801640 1 -- 10.20.193.28:0/1070499557 wait complete. 2026-02-25T11:47:12.707 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs new cephfs cephfs_metadata cephfs_data 2026-02-25T11:47:12.889 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:12.931 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:12 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2538845496' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2026-02-25T11:47:12.931 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:12 trial028 ceph-mon[39224]: osdmap e83: 12 total, 12 up, 12 in 2026-02-25T11:47:12.932 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:12 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1070499557' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2026-02-25T11:47:12.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:12 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2538845496' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2026-02-25T11:47:13.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:12 trial167 ceph-mon[43392]: osdmap e83: 12 total, 12 up, 12 in 2026-02-25T11:47:13.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:12 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1070499557' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2026-02-25T11:47:13.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:12 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2538845496' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2026-02-25T11:47:13.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:12 trial119 ceph-mon[42710]: osdmap e83: 12 total, 12 up, 12 in 2026-02-25T11:47:13.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:12 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1070499557' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2026-02-25T11:47:13.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.070+0000 7f0afbaa8640 1 -- 10.20.193.28:0/1739397623 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0af4110130 msgr2=0x7f0af4115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:13.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.070+0000 7f0afbaa8640 1 --2- 10.20.193.28:0/1739397623 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0af4110130 0x7f0af4115180 secure :-1 s=READY pgs=222 gs=1 cs=0 l=1 c_cookie=146195452772852189 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ae8009950 tx=0x7f0ae802f190 comp rx=0 tx=0).stop 2026-02-25T11:47:13.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.070+0000 7f0afbaa8640 1 -- 10.20.193.28:0/1739397623 shutdown_connections 2026-02-25T11:47:13.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.070+0000 7f0afbaa8640 1 --2- 10.20.193.28:0/1739397623 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0af4115930 0x7f0af4176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:13.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.070+0000 7f0afbaa8640 1 --2- 10.20.193.28:0/1739397623 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0af4110130 0x7f0af4115180 unknown :-1 s=CLOSED pgs=222 gs=1 cs=0 l=1 c_cookie=146195452772852189 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:13.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.070+0000 7f0afbaa8640 1 --2- 10.20.193.28:0/1739397623 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0af410f740 0x7f0af410fb10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:13.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.070+0000 7f0afbaa8640 1 -- 10.20.193.28:0/1739397623 >> 10.20.193.28:0/1739397623 conn(0x7f0af40ecff0 msgr2=0x7f0af410d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:13.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.070+0000 7f0afbaa8640 1 -- 10.20.193.28:0/1739397623 shutdown_connections 2026-02-25T11:47:13.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.070+0000 7f0afbaa8640 1 -- 10.20.193.28:0/1739397623 wait complete. 2026-02-25T11:47:13.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.071+0000 7f0afbaa8640 1 Processor -- start 2026-02-25T11:47:13.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.071+0000 7f0afbaa8640 1 -- start start 2026-02-25T11:47:13.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0afbaa8640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0af410f740 0x7f0af40f5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:13.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0afbaa8640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0af4110130 0x7f0af4176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:13.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0afbaa8640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0af4115930 0x7f0af40f6210 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:13.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0afbaa8640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0af40fb490 con 0x7f0af4110130 2026-02-25T11:47:13.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0afbaa8640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0af40fb600 con 0x7f0af4115930 2026-02-25T11:47:13.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0afbaa8640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0af40fb770 con 0x7f0af410f740 2026-02-25T11:47:13.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0af901c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0af4110130 0x7f0af4176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:13.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0af901c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0af4110130 0x7f0af4176200 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59354/0 (socket says 10.20.193.28:59354) 2026-02-25T11:47:13.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0af901c640 1 -- 10.20.193.28:0/3332340655 learned_addr learned my addr 10.20.193.28:0/3332340655 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:13.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0af981d640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0af410f740 0x7f0af40f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:13.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0afa01e640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0af4115930 0x7f0af40f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:13.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0af901c640 1 -- 10.20.193.28:0/3332340655 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0af410f740 msgr2=0x7f0af40f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:13.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0af901c640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0af410f740 0x7f0af40f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:13.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0af901c640 1 -- 10.20.193.28:0/3332340655 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0af4115930 msgr2=0x7f0af40f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:13.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0af901c640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0af4115930 0x7f0af40f6210 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:13.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0af901c640 1 -- 10.20.193.28:0/3332340655 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae8009570 con 0x7f0af4110130 2026-02-25T11:47:13.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0afa01e640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0af4115930 0x7f0af40f6210 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:13.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.072+0000 7f0af901c640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0af4110130 0x7f0af4176200 secure :-1 s=READY pgs=223 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ae80098a0 tx=0x7f0ae80028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:13.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.073+0000 7f0ae2ffd640 1 -- 10.20.193.28:0/3332340655 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0ae803d070 con 0x7f0af4110130 2026-02-25T11:47:13.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.073+0000 7f0ae2ffd640 1 -- 10.20.193.28:0/3332340655 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0ae802fe20 con 0x7f0af4110130 2026-02-25T11:47:13.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.073+0000 7f0ae2ffd640 1 -- 10.20.193.28:0/3332340655 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0ae80418c0 con 0x7f0af4110130 2026-02-25T11:47:13.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.073+0000 7f0afbaa8640 1 -- 10.20.193.28:0/3332340655 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af40fb9f0 con 0x7f0af4110130 2026-02-25T11:47:13.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.073+0000 7f0afbaa8640 1 -- 10.20.193.28:0/3332340655 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af40fbe10 con 0x7f0af4110130 2026-02-25T11:47:13.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.073+0000 7f0afbaa8640 1 -- 10.20.193.28:0/3332340655 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0abc005300 con 0x7f0af4110130 2026-02-25T11:47:13.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.076+0000 7f0ae2ffd640 1 -- 10.20.193.28:0/3332340655 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f0ae8041a50 con 0x7f0af4110130 2026-02-25T11:47:13.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.078+0000 7f0ae2ffd640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0ad00b5760 0x7f0ad00d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:13.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.078+0000 7f0ae2ffd640 1 -- 10.20.193.28:0/3332340655 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7877+0+0 (secure 0 0 0) 0x7f0ae810bce0 con 0x7f0af4110130 2026-02-25T11:47:13.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.078+0000 7f0af981d640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0ad00b5760 0x7f0ad00d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:13.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.078+0000 7f0ae2ffd640 1 -- 10.20.193.28:0/3332340655 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f0ae810c160 con 0x7f0af4110130 2026-02-25T11:47:13.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.078+0000 7f0af981d640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0ad00b5760 0x7f0ad00d5c10 secure :-1 s=READY pgs=171 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ae4004740 tx=0x7f0ae4004420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:13.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.222+0000 7f0afbaa8640 1 -- 10.20.193.28:0/3332340655 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f0abc0055b0 con 0x7f0af4110130 2026-02-25T11:47:13.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.624+0000 7f0ae2ffd640 1 -- 10.20.193.28:0/3332340655 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f0ae80f5e60 con 0x7f0af4110130 2026-02-25T11:47:13.628 INFO:teuthology.orchestra.run.trial028.stderr:new fs with metadata pool 2 and data pool 3 2026-02-25T11:47:13.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.627+0000 7f0afbaa8640 1 -- 10.20.193.28:0/3332340655 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0ad00b5760 msgr2=0x7f0ad00d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:13.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.627+0000 7f0afbaa8640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0ad00b5760 0x7f0ad00d5c10 secure :-1 s=READY pgs=171 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ae4004740 tx=0x7f0ae4004420 comp rx=0 tx=0).stop 2026-02-25T11:47:13.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.627+0000 7f0afbaa8640 1 -- 10.20.193.28:0/3332340655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0af4110130 msgr2=0x7f0af4176200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:13.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.627+0000 7f0afbaa8640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0af4110130 0x7f0af4176200 secure :-1 s=READY pgs=223 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ae80098a0 tx=0x7f0ae80028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:13.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.628+0000 7f0afbaa8640 1 -- 10.20.193.28:0/3332340655 shutdown_connections 2026-02-25T11:47:13.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.628+0000 7f0afbaa8640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0ad00b5760 0x7f0ad00d5c10 unknown :-1 s=CLOSED pgs=171 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:13.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.628+0000 7f0afbaa8640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0af4115930 0x7f0af40f6210 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:13.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.628+0000 7f0afbaa8640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0af4110130 0x7f0af4176200 unknown :-1 s=CLOSED pgs=223 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:13.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.628+0000 7f0afbaa8640 1 --2- 10.20.193.28:0/3332340655 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0af410f740 0x7f0af40f5b80 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:13.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.628+0000 7f0afbaa8640 1 -- 10.20.193.28:0/3332340655 >> 10.20.193.28:0/3332340655 conn(0x7f0af40ecff0 msgr2=0x7f0af410dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:13.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.628+0000 7f0afbaa8640 1 -- 10.20.193.28:0/3332340655 shutdown_connections 2026-02-25T11:47:13.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:13.628+0000 7f0afbaa8640 1 -- 10.20.193.28:0/3332340655 wait complete. 2026-02-25T11:47:13.731 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd dump --format=json 2026-02-25T11:47:13.916 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:13.960 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:13 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1070499557' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2026-02-25T11:47:13.960 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:13 trial028 ceph-mon[39224]: osdmap e84: 12 total, 12 up, 12 in 2026-02-25T11:47:13.960 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:13 trial028 ceph-mon[39224]: pgmap v192: 97 pgs: 10 creating+peering, 83 unknown, 4 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:13.960 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:13 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3332340655' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2026-02-25T11:47:13.960 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:13 trial028 ceph-mon[39224]: daemon mds.b assigned to filesystem cephfs as rank 0 2026-02-25T11:47:13.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:13 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1070499557' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2026-02-25T11:47:13.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:13 trial167 ceph-mon[43392]: osdmap e84: 12 total, 12 up, 12 in 2026-02-25T11:47:13.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:13 trial167 ceph-mon[43392]: pgmap v192: 97 pgs: 10 creating+peering, 83 unknown, 4 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:13.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:13 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3332340655' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2026-02-25T11:47:13.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:13 trial167 ceph-mon[43392]: daemon mds.b assigned to filesystem cephfs as rank 0 2026-02-25T11:47:14.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:13 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1070499557' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2026-02-25T11:47:14.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:13 trial119 ceph-mon[42710]: osdmap e84: 12 total, 12 up, 12 in 2026-02-25T11:47:14.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:13 trial119 ceph-mon[42710]: pgmap v192: 97 pgs: 10 creating+peering, 83 unknown, 4 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail 2026-02-25T11:47:14.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:13 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3332340655' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2026-02-25T11:47:14.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:13 trial119 ceph-mon[42710]: daemon mds.b assigned to filesystem cephfs as rank 0 2026-02-25T11:47:14.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.091+0000 7f3302039640 1 -- 10.20.193.28:0/499955863 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32fc10bd50 msgr2=0x7f32fc110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:14.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.091+0000 7f3302039640 1 --2- 10.20.193.28:0/499955863 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32fc10bd50 0x7f32fc110be0 secure :-1 s=READY pgs=224 gs=1 cs=0 l=1 c_cookie=4136674221282975214 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f32f0009950 tx=0x7f32f002f190 comp rx=0 tx=0).stop 2026-02-25T11:47:14.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.091+0000 7f3302039640 1 -- 10.20.193.28:0/499955863 shutdown_connections 2026-02-25T11:47:14.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.091+0000 7f3302039640 1 --2- 10.20.193.28:0/499955863 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f32fc111370 0x7f32fc111d50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.091+0000 7f3302039640 1 --2- 10.20.193.28:0/499955863 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32fc10bd50 0x7f32fc110be0 unknown :-1 s=CLOSED pgs=224 gs=1 cs=0 l=1 c_cookie=4136674221282975214 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.091+0000 7f3302039640 1 --2- 10.20.193.28:0/499955863 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f32fc10b360 0x7f32fc10b730 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.091+0000 7f3302039640 1 -- 10.20.193.28:0/499955863 >> 10.20.193.28:0/499955863 conn(0x7f32fc0e8e70 msgr2=0x7f32fc109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:14.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.091+0000 7f3302039640 1 -- 10.20.193.28:0/499955863 shutdown_connections 2026-02-25T11:47:14.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.092+0000 7f3302039640 1 -- 10.20.193.28:0/499955863 wait complete. 2026-02-25T11:47:14.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.092+0000 7f3302039640 1 Processor -- start 2026-02-25T11:47:14.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.092+0000 7f3302039640 1 -- start start 2026-02-25T11:47:14.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3302039640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32fc10b360 0x7f32fc123680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:14.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3302039640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f32fc10bd50 0x7f32fc123bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:14.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3302039640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f32fc111370 0x7f32fc128f90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:14.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3302039640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1295b0 con 0x7f32fc10b360 2026-02-25T11:47:14.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3302039640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc129720 con 0x7f32fc10bd50 2026-02-25T11:47:14.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3302039640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc1298e0 con 0x7f32fc111370 2026-02-25T11:47:14.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3300836640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f32fc10bd50 0x7f32fc123bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:14.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3300836640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f32fc10bd50 0x7f32fc123bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41356/0 (socket says 10.20.193.28:41356) 2026-02-25T11:47:14.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3300836640 1 -- 10.20.193.28:0/3031772233 learned_addr learned my addr 10.20.193.28:0/3031772233 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:14.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3301037640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32fc10b360 0x7f32fc123680 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:14.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3301838640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f32fc111370 0x7f32fc128f90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:14.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3300836640 1 -- 10.20.193.28:0/3031772233 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f32fc111370 msgr2=0x7f32fc128f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:14.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3300836640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f32fc111370 0x7f32fc128f90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3300836640 1 -- 10.20.193.28:0/3031772233 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32fc10b360 msgr2=0x7f32fc123680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:14.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3300836640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32fc10b360 0x7f32fc123680 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.093+0000 7f3300836640 1 -- 10.20.193.28:0/3031772233 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f0009570 con 0x7f32fc10bd50 2026-02-25T11:47:14.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.094+0000 7f3300836640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f32fc10bd50 0x7f32fc123bc0 secure :-1 s=READY pgs=78 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f32f00098a0 tx=0x7f32f00028d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:14.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.094+0000 7f32ea7fc640 1 -- 10.20.193.28:0/3031772233 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f32f003d070 con 0x7f32fc10bd50 2026-02-25T11:47:14.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.094+0000 7f3302039640 1 -- 10.20.193.28:0/3031772233 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32fc129b60 con 0x7f32fc10bd50 2026-02-25T11:47:14.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.094+0000 7f32ea7fc640 1 -- 10.20.193.28:0/3031772233 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f32f0002a70 con 0x7f32fc10bd50 2026-02-25T11:47:14.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.094+0000 7f32ea7fc640 1 -- 10.20.193.28:0/3031772233 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f32f00418c0 con 0x7f32fc10bd50 2026-02-25T11:47:14.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.094+0000 7f3302039640 1 -- 10.20.193.28:0/3031772233 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32fc16a2c0 con 0x7f32fc10bd50 2026-02-25T11:47:14.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.095+0000 7f32cbfff640 1 -- 10.20.193.28:0/3031772233 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32fc10c480 con 0x7f32fc10bd50 2026-02-25T11:47:14.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.097+0000 7f32ea7fc640 1 -- 10.20.193.28:0/3031772233 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f32f0041a50 con 0x7f32fc10bd50 2026-02-25T11:47:14.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.099+0000 7f32ea7fc640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f32d80b5760 0x7f32d80d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:14.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.099+0000 7f32ea7fc640 1 -- 10.20.193.28:0/3031772233 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7977+0+0 (secure 0 0 0) 0x7f32f010b2f0 con 0x7f32fc10bd50 2026-02-25T11:47:14.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.099+0000 7f32ea7fc640 1 -- 10.20.193.28:0/3031772233 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f32f010b770 con 0x7f32fc10bd50 2026-02-25T11:47:14.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.099+0000 7f3301037640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f32d80b5760 0x7f32d80d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:14.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.100+0000 7f3301037640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f32d80b5760 0x7f32d80d5c10 secure :-1 s=READY pgs=172 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f32ec004740 tx=0x7f32ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:14.217 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.214+0000 7f32cbfff640 1 -- 10.20.193.28:0/3031772233 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32fc110be0 con 0x7f32fc10bd50 2026-02-25T11:47:14.219 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:14.219 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.215+0000 7f32ea7fc640 1 -- 10.20.193.28:0/3031772233 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22353 (secure 0 0 0) 0x7f32f00cdda0 con 0x7f32fc10bd50 2026-02-25T11:47:14.220 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":85,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","created":"2026-02-25T11:42:51.962844+0000","modified":"2026-02-25T11:47:13.223646+0000","last_up_change":"2026-02-25T11:46:45.156629+0000","last_in_change":"2026-02-25T11:46:31.585002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":38,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T11:44:43.266336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-25T11:47:11.236575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-25T11:47:12.196775+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf596df7-d1fb-4d10-8d4c-14a35210bab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6803","nonce":668944205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6805","nonce":668944205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6809","nonce":668944205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6807","nonce":668944205}]},"public_addr":"10.20.193.28:6803/668944205","cluster_addr":"10.20.193.28:6805/668944205","heartbeat_back_addr":"10.20.193.28:6809/668944205","heartbeat_front_addr":"10.20.193.28:6807/668944205","state":["exists","up"]},{"osd":1,"uuid":"185719c0-7c2d-431f-9925-88bbee86a98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6800","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6801","nonce":3486566242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6802","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6803","nonce":3486566242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6806","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6807","nonce":3486566242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6804","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6805","nonce":3486566242}]},"public_addr":"10.20.193.119:6801/3486566242","cluster_addr":"10.20.193.119:6803/3486566242","heartbeat_back_addr":"10.20.193.119:6807/3486566242","heartbeat_front_addr":"10.20.193.119:6805/3486566242","state":["exists","up"]},{"osd":2,"uuid":"4de13b90-1c3c-4614-9fef-2d7315626433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6800","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6801","nonce":484781559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6802","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6803","nonce":484781559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6806","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6807","nonce":484781559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6804","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6805","nonce":484781559}]},"public_addr":"10.20.193.167:6801/484781559","cluster_addr":"10.20.193.167:6803/484781559","heartbeat_back_addr":"10.20.193.167:6807/484781559","heartbeat_front_addr":"10.20.193.167:6805/484781559","state":["exists","up"]},{"osd":3,"uuid":"45d9ea8e-5a77-4738-96da-ada98515fc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6811","nonce":2452541436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6813","nonce":2452541436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6817","nonce":2452541436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6815","nonce":2452541436}]},"public_addr":"10.20.193.28:6811/2452541436","cluster_addr":"10.20.193.28:6813/2452541436","heartbeat_back_addr":"10.20.193.28:6817/2452541436","heartbeat_front_addr":"10.20.193.28:6815/2452541436","state":["exists","up"]},{"osd":4,"uuid":"c9c3f512-665b-44b2-a6f0-f3356a4b661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6808","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6809","nonce":135971996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6810","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6811","nonce":135971996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6814","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6815","nonce":135971996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6812","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6813","nonce":135971996}]},"public_addr":"10.20.193.119:6809/135971996","cluster_addr":"10.20.193.119:6811/135971996","heartbeat_back_addr":"10.20.193.119:6815/135971996","heartbeat_front_addr":"10.20.193.119:6813/135971996","state":["exists","up"]},{"osd":5,"uuid":"777c3819-5c90-4b2b-a545-c024884c2eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6808","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6809","nonce":4203420884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6810","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6811","nonce":4203420884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6814","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6815","nonce":4203420884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6812","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6813","nonce":4203420884}]},"public_addr":"10.20.193.167:6809/4203420884","cluster_addr":"10.20.193.167:6811/4203420884","heartbeat_back_addr":"10.20.193.167:6815/4203420884","heartbeat_front_addr":"10.20.193.167:6813/4203420884","state":["exists","up"]},{"osd":6,"uuid":"bbd9de64-b9fe-499e-b451-c16828be2c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6819","nonce":621484999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6821","nonce":621484999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6825","nonce":621484999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6823","nonce":621484999}]},"public_addr":"10.20.193.28:6819/621484999","cluster_addr":"10.20.193.28:6821/621484999","heartbeat_back_addr":"10.20.193.28:6825/621484999","heartbeat_front_addr":"10.20.193.28:6823/621484999","state":["exists","up"]},{"osd":7,"uuid":"546d9f28-597b-454c-8456-fcc2d46c69f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6816","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6817","nonce":2724541747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6818","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6819","nonce":2724541747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6822","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6823","nonce":2724541747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6820","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6821","nonce":2724541747}]},"public_addr":"10.20.193.119:6817/2724541747","cluster_addr":"10.20.193.119:6819/2724541747","heartbeat_back_addr":"10.20.193.119:6823/2724541747","heartbeat_front_addr":"10.20.193.119:6821/2724541747","state":["exists","up"]},{"osd":8,"uuid":"a1a7cea6-3586-4bb2-aa4c-bf6f7799a501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6816","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6817","nonce":1328734317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6818","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6819","nonce":1328734317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6822","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6823","nonce":1328734317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6820","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6821","nonce":1328734317}]},"public_addr":"10.20.193.167:6817/1328734317","cluster_addr":"10.20.193.167:6819/1328734317","heartbeat_back_addr":"10.20.193.167:6823/1328734317","heartbeat_front_addr":"10.20.193.167:6821/1328734317","state":["exists","up"]},{"osd":9,"uuid":"03fef134-6946-43d5-9c26-dc0e59fffe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6827","nonce":354500625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6829","nonce":354500625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6833","nonce":354500625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6831","nonce":354500625}]},"public_addr":"10.20.193.28:6827/354500625","cluster_addr":"10.20.193.28:6829/354500625","heartbeat_back_addr":"10.20.193.28:6833/354500625","heartbeat_front_addr":"10.20.193.28:6831/354500625","state":["exists","up"]},{"osd":10,"uuid":"b46c29ef-285c-4184-be28-d99b76bc0e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6824","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6825","nonce":2996581116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6826","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6827","nonce":2996581116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6830","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6831","nonce":2996581116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6828","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6829","nonce":2996581116}]},"public_addr":"10.20.193.119:6825/2996581116","cluster_addr":"10.20.193.119:6827/2996581116","heartbeat_back_addr":"10.20.193.119:6831/2996581116","heartbeat_front_addr":"10.20.193.119:6829/2996581116","state":["exists","up"]},{"osd":11,"uuid":"f6696d8c-5de7-4fd9-b693-524f204e4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6824","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6825","nonce":2182906700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6826","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6827","nonce":2182906700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6830","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6831","nonce":2182906700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6828","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6829","nonce":2182906700}]},"public_addr":"10.20.193.167:6825/2182906700","cluster_addr":"10.20.193.167:6827/2182906700","heartbeat_back_addr":"10.20.193.167:6831/2182906700","heartbeat_front_addr":"10.20.193.167:6829/2182906700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:13.410235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:27.254773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:40.956615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:54.529866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:08.306772+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:21.899498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:35.500407+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:49.094965+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:02.790138+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:16.286937+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:29.922722+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:43.396028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.28:0/1467902196":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6800/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/2274270913":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/60087422":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:6800/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/1725332841":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/3082630924":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/2391715078":"2026-02-26T11:43:04.102029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T11:47:14.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.217+0000 7f32cbfff640 1 -- 10.20.193.28:0/3031772233 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f32d80b5760 msgr2=0x7f32d80d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:14.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.217+0000 7f32cbfff640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f32d80b5760 0x7f32d80d5c10 secure :-1 s=READY pgs=172 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f32ec004740 tx=0x7f32ec00b410 comp rx=0 tx=0).stop 2026-02-25T11:47:14.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.217+0000 7f32cbfff640 1 -- 10.20.193.28:0/3031772233 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f32fc10bd50 msgr2=0x7f32fc123bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:14.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.217+0000 7f32cbfff640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f32fc10bd50 0x7f32fc123bc0 secure :-1 s=READY pgs=78 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f32f00098a0 tx=0x7f32f00028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.217+0000 7f32cbfff640 1 -- 10.20.193.28:0/3031772233 shutdown_connections 2026-02-25T11:47:14.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.217+0000 7f32cbfff640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f32d80b5760 0x7f32d80d5c10 unknown :-1 s=CLOSED pgs=172 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.217+0000 7f32cbfff640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f32fc111370 0x7f32fc128f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.217+0000 7f32cbfff640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f32fc10bd50 0x7f32fc123bc0 unknown :-1 s=CLOSED pgs=78 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.217+0000 7f32cbfff640 1 --2- 10.20.193.28:0/3031772233 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f32fc10b360 0x7f32fc123680 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.217+0000 7f32cbfff640 1 -- 10.20.193.28:0/3031772233 >> 10.20.193.28:0/3031772233 conn(0x7f32fc0e8e70 msgr2=0x7f32fc109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:14.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.217+0000 7f32cbfff640 1 -- 10.20.193.28:0/3031772233 shutdown_connections 2026-02-25T11:47:14.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.217+0000 7f32cbfff640 1 -- 10.20.193.28:0/3031772233 wait complete. 2026-02-25T11:47:14.319 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd dump --format=json 2026-02-25T11:47:14.504 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:14.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.671+0000 7f08d976d640 1 -- 10.20.193.28:0/732493380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f08d4115880 msgr2=0x7f08d4176250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:14.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.671+0000 7f08d976d640 1 --2- 10.20.193.28:0/732493380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f08d4115880 0x7f08d4176250 secure :-1 s=READY pgs=225 gs=1 cs=0 l=1 c_cookie=16940444318759298290 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f08c800aa30 tx=0x7f08c8030270 comp rx=0 tx=0).stop 2026-02-25T11:47:14.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.671+0000 7f08d976d640 1 -- 10.20.193.28:0/732493380 shutdown_connections 2026-02-25T11:47:14.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.671+0000 7f08d976d640 1 --2- 10.20.193.28:0/732493380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f08d4115880 0x7f08d4176250 unknown :-1 s=CLOSED pgs=225 gs=1 cs=0 l=1 c_cookie=16940444318759298290 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.671+0000 7f08d976d640 1 --2- 10.20.193.28:0/732493380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f08d4110080 0x7f08d41150d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.671+0000 7f08d976d640 1 --2- 10.20.193.28:0/732493380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f08d410f690 0x7f08d410fa60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.671+0000 7f08d976d640 1 -- 10.20.193.28:0/732493380 >> 10.20.193.28:0/732493380 conn(0x7f08d40ecf00 msgr2=0x7f08d410d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:14.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.671+0000 7f08d976d640 1 -- 10.20.193.28:0/732493380 shutdown_connections 2026-02-25T11:47:14.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.671+0000 7f08d976d640 1 -- 10.20.193.28:0/732493380 wait complete. 2026-02-25T11:47:14.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.672+0000 7f08d976d640 1 Processor -- start 2026-02-25T11:47:14.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.672+0000 7f08d976d640 1 -- start start 2026-02-25T11:47:14.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.672+0000 7f08d976d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f08d410f690 0x7f08d411b600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:14.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.672+0000 7f08d976d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f08d4110080 0x7f08d411bb40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:14.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.672+0000 7f08d976d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f08d4115880 0x7f08d4120f10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:14.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.672+0000 7f08d976d640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4121530 con 0x7f08d4110080 2026-02-25T11:47:14.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.672+0000 7f08d976d640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f08d41216a0 con 0x7f08d410f690 2026-02-25T11:47:14.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.672+0000 7f08d976d640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4121860 con 0x7f08d4115880 2026-02-25T11:47:14.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.673+0000 7f08d8f6c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f08d4115880 0x7f08d4120f10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:14.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.673+0000 7f08d8f6c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f08d4115880 0x7f08d4120f10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:33076/0 (socket says 10.20.193.28:33076) 2026-02-25T11:47:14.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.673+0000 7f08d8f6c640 1 -- 10.20.193.28:0/4052302875 learned_addr learned my addr 10.20.193.28:0/4052302875 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:14.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.673+0000 7f08d3fff640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f08d410f690 0x7f08d411b600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.673+0000 7f08d37fe640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f08d4110080 0x7f08d411bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.673+0000 7f08d8f6c640 1 -- 10.20.193.28:0/4052302875 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f08d410f690 msgr2=0x7f08d411b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.673+0000 7f08d8f6c640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f08d410f690 0x7f08d411b600 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.673+0000 7f08d8f6c640 1 -- 10.20.193.28:0/4052302875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f08d4110080 msgr2=0x7f08d411bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.673+0000 7f08d8f6c640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f08d4110080 0x7f08d411bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.673+0000 7f08d8f6c640 1 -- 10.20.193.28:0/4052302875 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c800a650 con 0x7f08d4115880 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.673+0000 7f08d37fe640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f08d4110080 0x7f08d411bb40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.673+0000 7f08d8f6c640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f08d4115880 0x7f08d4120f10 secure :-1 s=READY pgs=81 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f08c8006a50 tx=0x7f08c80039b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.673+0000 7f08d17fa640 1 -- 10.20.193.28:0/4052302875 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f08c803e070 con 0x7f08d4115880 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.674+0000 7f08d976d640 1 -- 10.20.193.28:0/4052302875 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d4121ae0 con 0x7f08d4115880 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.674+0000 7f08d976d640 1 -- 10.20.193.28:0/4052302875 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d4162240 con 0x7f08d4115880 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.674+0000 7f08d17fa640 1 -- 10.20.193.28:0/4052302875 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f08c8003aa0 con 0x7f08d4115880 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.674+0000 7f08d17fa640 1 -- 10.20.193.28:0/4052302875 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f08c8042820 con 0x7f08d4115880 2026-02-25T11:47:14.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.674+0000 7f08d976d640 1 -- 10.20.193.28:0/4052302875 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d4121db0 con 0x7f08d4115880 2026-02-25T11:47:14.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.676+0000 7f08d17fa640 1 -- 10.20.193.28:0/4052302875 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f08c8039880 con 0x7f08d4115880 2026-02-25T11:47:14.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.678+0000 7f08d17fa640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f08a80b5760 0x7f08a80d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:14.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.678+0000 7f08d17fa640 1 -- 10.20.193.28:0/4052302875 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f08c810c760 con 0x7f08d4115880 2026-02-25T11:47:14.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.678+0000 7f08d17fa640 1 -- 10.20.193.28:0/4052302875 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f08c810cbe0 con 0x7f08d4115880 2026-02-25T11:47:14.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.678+0000 7f08d3fff640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f08a80b5760 0x7f08a80d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:14.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.679+0000 7f08d3fff640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f08a80b5760 0x7f08a80d5c10 secure :-1 s=READY pgs=173 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f08c0004740 tx=0x7f08c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:14.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.794+0000 7f08d976d640 1 -- 10.20.193.28:0/4052302875 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08d410fa60 con 0x7f08d4115880 2026-02-25T11:47:14.797 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:14 trial028 ceph-mon[39224]: osdmap e85: 12 total, 12 up, 12 in 2026-02-25T11:47:14.797 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:14 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3332340655' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2026-02-25T11:47:14.797 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:14 trial028 ceph-mon[39224]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2026-02-25T11:47:14.797 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:14 trial028 ceph-mon[39224]: daemon mds.b is now active in filesystem cephfs as rank 0 2026-02-25T11:47:14.797 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:14 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3031772233' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:14.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.795+0000 7f08d17fa640 1 -- 10.20.193.28:0/4052302875 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22340 (secure 0 0 0) 0x7f08c80cf210 con 0x7f08d4115880 2026-02-25T11:47:14.798 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:14.798 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":86,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","created":"2026-02-25T11:42:51.962844+0000","modified":"2026-02-25T11:47:14.624486+0000","last_up_change":"2026-02-25T11:46:45.156629+0000","last_in_change":"2026-02-25T11:46:31.585002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":38,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T11:44:43.266336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-25T11:47:11.236575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-25T11:47:12.196775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf596df7-d1fb-4d10-8d4c-14a35210bab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6803","nonce":668944205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6805","nonce":668944205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6809","nonce":668944205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6807","nonce":668944205}]},"public_addr":"10.20.193.28:6803/668944205","cluster_addr":"10.20.193.28:6805/668944205","heartbeat_back_addr":"10.20.193.28:6809/668944205","heartbeat_front_addr":"10.20.193.28:6807/668944205","state":["exists","up"]},{"osd":1,"uuid":"185719c0-7c2d-431f-9925-88bbee86a98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6800","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6801","nonce":3486566242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6802","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6803","nonce":3486566242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6806","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6807","nonce":3486566242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6804","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6805","nonce":3486566242}]},"public_addr":"10.20.193.119:6801/3486566242","cluster_addr":"10.20.193.119:6803/3486566242","heartbeat_back_addr":"10.20.193.119:6807/3486566242","heartbeat_front_addr":"10.20.193.119:6805/3486566242","state":["exists","up"]},{"osd":2,"uuid":"4de13b90-1c3c-4614-9fef-2d7315626433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6800","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6801","nonce":484781559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6802","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6803","nonce":484781559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6806","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6807","nonce":484781559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6804","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6805","nonce":484781559}]},"public_addr":"10.20.193.167:6801/484781559","cluster_addr":"10.20.193.167:6803/484781559","heartbeat_back_addr":"10.20.193.167:6807/484781559","heartbeat_front_addr":"10.20.193.167:6805/484781559","state":["exists","up"]},{"osd":3,"uuid":"45d9ea8e-5a77-4738-96da-ada98515fc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6811","nonce":2452541436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6813","nonce":2452541436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6817","nonce":2452541436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6815","nonce":2452541436}]},"public_addr":"10.20.193.28:6811/2452541436","cluster_addr":"10.20.193.28:6813/2452541436","heartbeat_back_addr":"10.20.193.28:6817/2452541436","heartbeat_front_addr":"10.20.193.28:6815/2452541436","state":["exists","up"]},{"osd":4,"uuid":"c9c3f512-665b-44b2-a6f0-f3356a4b661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6808","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6809","nonce":135971996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6810","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6811","nonce":135971996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6814","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6815","nonce":135971996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6812","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6813","nonce":135971996}]},"public_addr":"10.20.193.119:6809/135971996","cluster_addr":"10.20.193.119:6811/135971996","heartbeat_back_addr":"10.20.193.119:6815/135971996","heartbeat_front_addr":"10.20.193.119:6813/135971996","state":["exists","up"]},{"osd":5,"uuid":"777c3819-5c90-4b2b-a545-c024884c2eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6808","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6809","nonce":4203420884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6810","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6811","nonce":4203420884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6814","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6815","nonce":4203420884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6812","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6813","nonce":4203420884}]},"public_addr":"10.20.193.167:6809/4203420884","cluster_addr":"10.20.193.167:6811/4203420884","heartbeat_back_addr":"10.20.193.167:6815/4203420884","heartbeat_front_addr":"10.20.193.167:6813/4203420884","state":["exists","up"]},{"osd":6,"uuid":"bbd9de64-b9fe-499e-b451-c16828be2c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6819","nonce":621484999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6821","nonce":621484999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6825","nonce":621484999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6823","nonce":621484999}]},"public_addr":"10.20.193.28:6819/621484999","cluster_addr":"10.20.193.28:6821/621484999","heartbeat_back_addr":"10.20.193.28:6825/621484999","heartbeat_front_addr":"10.20.193.28:6823/621484999","state":["exists","up"]},{"osd":7,"uuid":"546d9f28-597b-454c-8456-fcc2d46c69f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6816","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6817","nonce":2724541747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6818","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6819","nonce":2724541747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6822","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6823","nonce":2724541747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6820","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6821","nonce":2724541747}]},"public_addr":"10.20.193.119:6817/2724541747","cluster_addr":"10.20.193.119:6819/2724541747","heartbeat_back_addr":"10.20.193.119:6823/2724541747","heartbeat_front_addr":"10.20.193.119:6821/2724541747","state":["exists","up"]},{"osd":8,"uuid":"a1a7cea6-3586-4bb2-aa4c-bf6f7799a501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6816","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6817","nonce":1328734317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6818","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6819","nonce":1328734317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6822","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6823","nonce":1328734317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6820","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6821","nonce":1328734317}]},"public_addr":"10.20.193.167:6817/1328734317","cluster_addr":"10.20.193.167:6819/1328734317","heartbeat_back_addr":"10.20.193.167:6823/1328734317","heartbeat_front_addr":"10.20.193.167:6821/1328734317","state":["exists","up"]},{"osd":9,"uuid":"03fef134-6946-43d5-9c26-dc0e59fffe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6827","nonce":354500625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6829","nonce":354500625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6833","nonce":354500625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6831","nonce":354500625}]},"public_addr":"10.20.193.28:6827/354500625","cluster_addr":"10.20.193.28:6829/354500625","heartbeat_back_addr":"10.20.193.28:6833/354500625","heartbeat_front_addr":"10.20.193.28:6831/354500625","state":["exists","up"]},{"osd":10,"uuid":"b46c29ef-285c-4184-be28-d99b76bc0e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6824","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6825","nonce":2996581116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6826","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6827","nonce":2996581116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6830","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6831","nonce":2996581116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6828","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6829","nonce":2996581116}]},"public_addr":"10.20.193.119:6825/2996581116","cluster_addr":"10.20.193.119:6827/2996581116","heartbeat_back_addr":"10.20.193.119:6831/2996581116","heartbeat_front_addr":"10.20.193.119:6829/2996581116","state":["exists","up"]},{"osd":11,"uuid":"f6696d8c-5de7-4fd9-b693-524f204e4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6824","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6825","nonce":2182906700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6826","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6827","nonce":2182906700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6830","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6831","nonce":2182906700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6828","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6829","nonce":2182906700}]},"public_addr":"10.20.193.167:6825/2182906700","cluster_addr":"10.20.193.167:6827/2182906700","heartbeat_back_addr":"10.20.193.167:6831/2182906700","heartbeat_front_addr":"10.20.193.167:6829/2182906700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:13.410235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:27.254773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:40.956615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:54.529866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:08.306772+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:21.899498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:35.500407+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:49.094965+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:02.790138+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:16.286937+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:29.922722+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:43.396028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.28:0/1467902196":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6800/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/2274270913":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/60087422":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:6800/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/1725332841":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/3082630924":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/2391715078":"2026-02-26T11:43:04.102029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T11:47:14.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.797+0000 7f08d976d640 1 -- 10.20.193.28:0/4052302875 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f08a80b5760 msgr2=0x7f08a80d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:14.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.797+0000 7f08d976d640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f08a80b5760 0x7f08a80d5c10 secure :-1 s=READY pgs=173 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f08c0004740 tx=0x7f08c0005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:14.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.797+0000 7f08d976d640 1 -- 10.20.193.28:0/4052302875 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f08d4115880 msgr2=0x7f08d4120f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:14.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.797+0000 7f08d976d640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f08d4115880 0x7f08d4120f10 secure :-1 s=READY pgs=81 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f08c8006a50 tx=0x7f08c80039b0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.797+0000 7f08d976d640 1 -- 10.20.193.28:0/4052302875 shutdown_connections 2026-02-25T11:47:14.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.797+0000 7f08d976d640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f08a80b5760 0x7f08a80d5c10 unknown :-1 s=CLOSED pgs=173 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.797+0000 7f08d976d640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f08d4115880 0x7f08d4120f10 unknown :-1 s=CLOSED pgs=81 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.797+0000 7f08d976d640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f08d4110080 0x7f08d411bb40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.797+0000 7f08d976d640 1 --2- 10.20.193.28:0/4052302875 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f08d410f690 0x7f08d411b600 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:14.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.797+0000 7f08d976d640 1 -- 10.20.193.28:0/4052302875 >> 10.20.193.28:0/4052302875 conn(0x7f08d40ecf00 msgr2=0x7f08d410d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:14.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.797+0000 7f08d976d640 1 -- 10.20.193.28:0/4052302875 shutdown_connections 2026-02-25T11:47:14.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:14.797+0000 7f08d976d640 1 -- 10.20.193.28:0/4052302875 wait complete. 2026-02-25T11:47:14.900 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs set cephfs standby_count_wanted 0 2026-02-25T11:47:14.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:14 trial167 ceph-mon[43392]: osdmap e85: 12 total, 12 up, 12 in 2026-02-25T11:47:14.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:14 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3332340655' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2026-02-25T11:47:14.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:14 trial167 ceph-mon[43392]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2026-02-25T11:47:14.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:14 trial167 ceph-mon[43392]: daemon mds.b is now active in filesystem cephfs as rank 0 2026-02-25T11:47:14.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:14 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3031772233' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:15.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:14 trial119 ceph-mon[42710]: osdmap e85: 12 total, 12 up, 12 in 2026-02-25T11:47:15.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:14 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3332340655' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2026-02-25T11:47:15.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:14 trial119 ceph-mon[42710]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2026-02-25T11:47:15.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:14 trial119 ceph-mon[42710]: daemon mds.b is now active in filesystem cephfs as rank 0 2026-02-25T11:47:15.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:14 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3031772233' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:15.083 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:15.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.257+0000 7fba33fff640 1 -- 10.20.193.28:0/543032745 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba34115880 msgr2=0x7fba34176250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:15.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.257+0000 7fba33fff640 1 --2- 10.20.193.28:0/543032745 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba34115880 0x7fba34176250 secure :-1 s=READY pgs=226 gs=1 cs=0 l=1 c_cookie=7517706450944861053 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fba2400aa30 tx=0x7fba24030280 comp rx=0 tx=0).stop 2026-02-25T11:47:15.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.257+0000 7fba33fff640 1 -- 10.20.193.28:0/543032745 shutdown_connections 2026-02-25T11:47:15.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.257+0000 7fba33fff640 1 --2- 10.20.193.28:0/543032745 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba34115880 0x7fba34176250 unknown :-1 s=CLOSED pgs=226 gs=1 cs=0 l=1 c_cookie=7517706450944861053 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.257+0000 7fba33fff640 1 --2- 10.20.193.28:0/543032745 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fba34110080 0x7fba341150d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.257+0000 7fba33fff640 1 --2- 10.20.193.28:0/543032745 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fba3410f690 0x7fba3410fa60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.257+0000 7fba33fff640 1 -- 10.20.193.28:0/543032745 >> 10.20.193.28:0/543032745 conn(0x7fba340ecf00 msgr2=0x7fba3410d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.257+0000 7fba33fff640 1 -- 10.20.193.28:0/543032745 shutdown_connections 2026-02-25T11:47:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.257+0000 7fba33fff640 1 -- 10.20.193.28:0/543032745 wait complete. 2026-02-25T11:47:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.258+0000 7fba33fff640 1 Processor -- start 2026-02-25T11:47:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.258+0000 7fba33fff640 1 -- start start 2026-02-25T11:47:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.258+0000 7fba33fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fba3410f690 0x7fba340f5940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.258+0000 7fba33fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba34110080 0x7fba34176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.258+0000 7fba33fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fba34115880 0x7fba340f6040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.258+0000 7fba33fff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fba340fb200 con 0x7fba34110080 2026-02-25T11:47:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.258+0000 7fba33fff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fba340fb370 con 0x7fba34115880 2026-02-25T11:47:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.258+0000 7fba33fff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fba340fb530 con 0x7fba3410f690 2026-02-25T11:47:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba327fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba34110080 0x7fba34176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:15.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba327fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba34110080 0x7fba34176150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59406/0 (socket says 10.20.193.28:59406) 2026-02-25T11:47:15.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba327fc640 1 -- 10.20.193.28:0/3633810986 learned_addr learned my addr 10.20.193.28:0/3633810986 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:15.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba32ffd640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fba3410f690 0x7fba340f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:15.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba337fe640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fba34115880 0x7fba340f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:15.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba327fc640 1 -- 10.20.193.28:0/3633810986 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fba3410f690 msgr2=0x7fba340f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:15.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba327fc640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fba3410f690 0x7fba340f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:15.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba327fc640 1 -- 10.20.193.28:0/3633810986 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fba34115880 msgr2=0x7fba340f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:15.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba327fc640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fba34115880 0x7fba340f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:15.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba327fc640 1 -- 10.20.193.28:0/3633810986 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2400a650 con 0x7fba34110080 2026-02-25T11:47:15.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba327fc640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba34110080 0x7fba34176150 secure :-1 s=READY pgs=227 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fba1c005db0 tx=0x7fba1c00ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:15.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba389d3640 1 -- 10.20.193.28:0/3633810986 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fba1c004500 con 0x7fba34110080 2026-02-25T11:47:15.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba389d3640 1 -- 10.20.193.28:0/3633810986 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fba1c004d10 con 0x7fba34110080 2026-02-25T11:47:15.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba389d3640 1 -- 10.20.193.28:0/3633810986 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fba1c005280 con 0x7fba34110080 2026-02-25T11:47:15.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba33fff640 1 -- 10.20.193.28:0/3633810986 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba340fb810 con 0x7fba34110080 2026-02-25T11:47:15.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.259+0000 7fba33fff640 1 -- 10.20.193.28:0/3633810986 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba340fbc90 con 0x7fba34110080 2026-02-25T11:47:15.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.260+0000 7fba33fff640 1 -- 10.20.193.28:0/3633810986 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba34110970 con 0x7fba34110080 2026-02-25T11:47:15.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.262+0000 7fba389d3640 1 -- 10.20.193.28:0/3633810986 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7fba1c005410 con 0x7fba34110080 2026-02-25T11:47:15.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.264+0000 7fba389d3640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fba080b5750 0x7fba080d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:15.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.264+0000 7fba389d3640 1 -- 10.20.193.28:0/3633810986 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fba1c0e6f70 con 0x7fba34110080 2026-02-25T11:47:15.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.264+0000 7fba389d3640 1 -- 10.20.193.28:0/3633810986 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fba1c0e73f0 con 0x7fba34110080 2026-02-25T11:47:15.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.264+0000 7fba32ffd640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fba080b5750 0x7fba080d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.264+0000 7fba32ffd640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fba080b5750 0x7fba080d5c00 secure :-1 s=READY pgs=174 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fba28004740 tx=0x7fba2800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:15.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.407+0000 7fba33fff640 1 -- 10.20.193.28:0/3633810986 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fba3410fa60 con 0x7fba34110080 2026-02-25T11:47:15.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.633+0000 7fba389d3640 1 -- 10.20.193.28:0/3633810986 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fba1c0a9a20 con 0x7fba34110080 2026-02-25T11:47:15.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.635+0000 7fba33fff640 1 -- 10.20.193.28:0/3633810986 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fba080b5750 msgr2=0x7fba080d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:15.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.635+0000 7fba33fff640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fba080b5750 0x7fba080d5c00 secure :-1 s=READY pgs=174 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fba28004740 tx=0x7fba2800b410 comp rx=0 tx=0).stop 2026-02-25T11:47:15.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.635+0000 7fba33fff640 1 -- 10.20.193.28:0/3633810986 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba34110080 msgr2=0x7fba34176150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:15.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.635+0000 7fba33fff640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba34110080 0x7fba34176150 secure :-1 s=READY pgs=227 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fba1c005db0 tx=0x7fba1c00ddf0 comp rx=0 tx=0).stop 2026-02-25T11:47:15.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.635+0000 7fba33fff640 1 -- 10.20.193.28:0/3633810986 shutdown_connections 2026-02-25T11:47:15.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.635+0000 7fba33fff640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fba080b5750 0x7fba080d5c00 unknown :-1 s=CLOSED pgs=174 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:15.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.635+0000 7fba33fff640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fba34115880 0x7fba340f6040 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:15.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.635+0000 7fba33fff640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba34110080 0x7fba34176150 unknown :-1 s=CLOSED pgs=227 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:15.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.635+0000 7fba33fff640 1 --2- 10.20.193.28:0/3633810986 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fba3410f690 0x7fba340f5940 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:15.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.635+0000 7fba33fff640 1 -- 10.20.193.28:0/3633810986 >> 10.20.193.28:0/3633810986 conn(0x7fba340ecf00 msgr2=0x7fba34111e90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:15.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.635+0000 7fba33fff640 1 -- 10.20.193.28:0/3633810986 shutdown_connections 2026-02-25T11:47:15.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:15.635+0000 7fba33fff640 1 -- 10.20.193.28:0/3633810986 wait complete. 2026-02-25T11:47:15.661 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:15 trial028 ceph-mon[39224]: osdmap e86: 12 total, 12 up, 12 in 2026-02-25T11:47:15.661 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:15 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] up:active 2026-02-25T11:47:15.661 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:15 trial028 ceph-mon[39224]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-25T11:47:15.661 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:15 trial028 ceph-mon[39224]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-25T11:47:15.661 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:15 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/4052302875' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:15.661 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:15 trial028 ceph-mon[39224]: pgmap v195: 97 pgs: 12 creating+peering, 51 unknown, 34 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail; 767 B/s wr, 6 op/s 2026-02-25T11:47:15.661 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:15 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3633810986' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2026-02-25T11:47:15.735 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2026-02-25T11:47:15.735 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs set cephfs max_mds 3 --yes-i-really-mean-it 2026-02-25T11:47:15.920 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:15.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:15 trial167 ceph-mon[43392]: osdmap e86: 12 total, 12 up, 12 in 2026-02-25T11:47:15.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:15 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] up:active 2026-02-25T11:47:15.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:15 trial167 ceph-mon[43392]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-25T11:47:15.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:15 trial167 ceph-mon[43392]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-25T11:47:15.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:15 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/4052302875' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:15.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:15 trial167 ceph-mon[43392]: pgmap v195: 97 pgs: 12 creating+peering, 51 unknown, 34 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail; 767 B/s wr, 6 op/s 2026-02-25T11:47:15.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:15 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3633810986' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2026-02-25T11:47:16.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:15 trial119 ceph-mon[42710]: osdmap e86: 12 total, 12 up, 12 in 2026-02-25T11:47:16.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:15 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] up:active 2026-02-25T11:47:16.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:15 trial119 ceph-mon[42710]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-25T11:47:16.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:15 trial119 ceph-mon[42710]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-25T11:47:16.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:15 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/4052302875' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:16.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:15 trial119 ceph-mon[42710]: pgmap v195: 97 pgs: 12 creating+peering, 51 unknown, 34 active+clean; 577 KiB data, 329 MiB used, 8.2 TiB / 8.2 TiB avail; 767 B/s wr, 6 op/s 2026-02-25T11:47:16.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:15 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3633810986' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2026-02-25T11:47:16.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.104+0000 7f36e9dca640 1 -- 10.20.193.28:0/1423187701 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f36e4119d30 msgr2=0x7f36e417a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:16.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.104+0000 7f36e9dca640 1 --2- 10.20.193.28:0/1423187701 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f36e4119d30 0x7f36e417a700 secure :-1 s=READY pgs=228 gs=1 cs=0 l=1 c_cookie=11900174354873846520 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f36e000aa30 tx=0x7f36e0030270 comp rx=0 tx=0).stop 2026-02-25T11:47:16.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.104+0000 7f36e9dca640 1 -- 10.20.193.28:0/1423187701 shutdown_connections 2026-02-25T11:47:16.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.104+0000 7f36e9dca640 1 --2- 10.20.193.28:0/1423187701 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f36e4119d30 0x7f36e417a700 unknown :-1 s=CLOSED pgs=228 gs=1 cs=0 l=1 c_cookie=11900174354873846520 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:16.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.104+0000 7f36e9dca640 1 --2- 10.20.193.28:0/1423187701 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f36e4114a90 0x7f36e4119580 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:16.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.104+0000 7f36e9dca640 1 --2- 10.20.193.28:0/1423187701 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f36e41140a0 0x7f36e4114470 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:16.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.104+0000 7f36e9dca640 1 -- 10.20.193.28:0/1423187701 >> 10.20.193.28:0/1423187701 conn(0x7f36e4064c40 msgr2=0x7f36e4065040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:16.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.104+0000 7f36e9dca640 1 -- 10.20.193.28:0/1423187701 shutdown_connections 2026-02-25T11:47:16.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.105+0000 7f36e9dca640 1 -- 10.20.193.28:0/1423187701 wait complete. 2026-02-25T11:47:16.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.105+0000 7f36e9dca640 1 Processor -- start 2026-02-25T11:47:16.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e9dca640 1 -- start start 2026-02-25T11:47:16.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e9dca640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f36e41140a0 0x7f36e41224e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:16.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e9dca640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f36e4114a90 0x7f36e4122a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:16.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e9dca640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f36e4119d30 0x7f36e4127df0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:16.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e9dca640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4128410 con 0x7f36e41140a0 2026-02-25T11:47:16.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e9dca640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4128580 con 0x7f36e4114a90 2026-02-25T11:47:16.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e9dca640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4128740 con 0x7f36e4119d30 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e8dc8640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f36e41140a0 0x7f36e41224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e8dc8640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f36e41140a0 0x7f36e41224e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:33952/0 (socket says 10.20.193.28:33952) 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e8dc8640 1 -- 10.20.193.28:0/1722411581 learned_addr learned my addr 10.20.193.28:0/1722411581 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36dbfff640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f36e4114a90 0x7f36e4122a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e95c9640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f36e4119d30 0x7f36e4127df0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e8dc8640 1 -- 10.20.193.28:0/1722411581 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f36e4119d30 msgr2=0x7f36e4127df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e8dc8640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f36e4119d30 0x7f36e4127df0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e8dc8640 1 -- 10.20.193.28:0/1722411581 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f36e4114a90 msgr2=0x7f36e4122a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e8dc8640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f36e4114a90 0x7f36e4122a20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.106+0000 7f36e8dc8640 1 -- 10.20.193.28:0/1722411581 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e000a650 con 0x7f36e41140a0 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.107+0000 7f36e8dc8640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f36e41140a0 0x7f36e41224e0 secure :-1 s=READY pgs=229 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f36d400d950 tx=0x7f36d400de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.107+0000 7f36d9ffb640 1 -- 10.20.193.28:0/1722411581 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f36d4004540 con 0x7f36e41140a0 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.107+0000 7f36e9dca640 1 -- 10.20.193.28:0/1722411581 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e4128a20 con 0x7f36e41140a0 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.107+0000 7f36d9ffb640 1 -- 10.20.193.28:0/1722411581 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f36d4009ce0 con 0x7f36e41140a0 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.107+0000 7f36d9ffb640 1 -- 10.20.193.28:0/1722411581 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f36d40106e0 con 0x7f36e41140a0 2026-02-25T11:47:16.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.107+0000 7f36e9dca640 1 -- 10.20.193.28:0/1722411581 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e4169120 con 0x7f36e41140a0 2026-02-25T11:47:16.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.108+0000 7f36e9dca640 1 -- 10.20.193.28:0/1722411581 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e4114e80 con 0x7f36e41140a0 2026-02-25T11:47:16.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.109+0000 7f36d9ffb640 1 -- 10.20.193.28:0/1722411581 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f36d40097c0 con 0x7f36e41140a0 2026-02-25T11:47:16.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.111+0000 7f36d9ffb640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f36c00b5760 0x7f36c00d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:16.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.111+0000 7f36d9ffb640 1 -- 10.20.193.28:0/1722411581 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f36d40e66a0 con 0x7f36e41140a0 2026-02-25T11:47:16.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.111+0000 7f36dbfff640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f36c00b5760 0x7f36c00d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:16.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.112+0000 7f36dbfff640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f36c00b5760 0x7f36c00d5c10 secure :-1 s=READY pgs=175 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f36e4128e80 tx=0x7f36cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:16.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.112+0000 7f36d9ffb640 1 -- 10.20.193.28:0/1722411581 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f36d40a9150 con 0x7f36e41140a0 2026-02-25T11:47:16.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.255+0000 7f36e9dca640 1 -- 10.20.193.28:0/1722411581 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3", "yes_i_really_mean_it": true} v 0) -- 0x7f36e4123640 con 0x7f36e41140a0 2026-02-25T11:47:16.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.639+0000 7f36d9ffb640 1 -- 10.20.193.28:0/1722411581 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3", "yes_i_really_mean_it": true}]=0 v18) ==== 135+0+0 (secure 0 0 0) 0x7f36d40ac300 con 0x7f36e41140a0 2026-02-25T11:47:16.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.641+0000 7f36e9dca640 1 -- 10.20.193.28:0/1722411581 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f36c00b5760 msgr2=0x7f36c00d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:16.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.641+0000 7f36e9dca640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f36c00b5760 0x7f36c00d5c10 secure :-1 s=READY pgs=175 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f36e4128e80 tx=0x7f36cc00b410 comp rx=0 tx=0).stop 2026-02-25T11:47:16.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.641+0000 7f36e9dca640 1 -- 10.20.193.28:0/1722411581 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f36e41140a0 msgr2=0x7f36e41224e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:16.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.641+0000 7f36e9dca640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f36e41140a0 0x7f36e41224e0 secure :-1 s=READY pgs=229 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f36d400d950 tx=0x7f36d400de20 comp rx=0 tx=0).stop 2026-02-25T11:47:16.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.641+0000 7f36e9dca640 1 -- 10.20.193.28:0/1722411581 shutdown_connections 2026-02-25T11:47:16.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.641+0000 7f36e9dca640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f36c00b5760 0x7f36c00d5c10 unknown :-1 s=CLOSED pgs=175 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:16.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.641+0000 7f36e9dca640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f36e4119d30 0x7f36e4127df0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:16.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.641+0000 7f36e9dca640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f36e4114a90 0x7f36e4122a20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:16.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.641+0000 7f36e9dca640 1 --2- 10.20.193.28:0/1722411581 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f36e41140a0 0x7f36e41224e0 unknown :-1 s=CLOSED pgs=229 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:16.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.641+0000 7f36e9dca640 1 -- 10.20.193.28:0/1722411581 >> 10.20.193.28:0/1722411581 conn(0x7f36e4064c40 msgr2=0x7f36e4118240 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:16.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.641+0000 7f36e9dca640 1 -- 10.20.193.28:0/1722411581 shutdown_connections 2026-02-25T11:47:16.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:16.641+0000 7f36e9dca640 1 -- 10.20.193.28:0/1722411581 wait complete. 2026-02-25T11:47:16.745 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs set cephfs allow_standby_replay true 2026-02-25T11:47:16.928 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:16.973 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:16 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3633810986' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2026-02-25T11:47:16.973 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:16 trial028 ceph-mon[39224]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-25T11:47:16.974 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:16 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1722411581' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3", "yes_i_really_mean_it": true} : dispatch 2026-02-25T11:47:16.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:16 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3633810986' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2026-02-25T11:47:16.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:16 trial167 ceph-mon[43392]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-25T11:47:17.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:16 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1722411581' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3", "yes_i_really_mean_it": true} : dispatch 2026-02-25T11:47:17.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:16 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3633810986' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2026-02-25T11:47:17.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:16 trial119 ceph-mon[42710]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-25T11:47:17.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:16 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1722411581' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3", "yes_i_really_mean_it": true} : dispatch 2026-02-25T11:47:17.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.107+0000 7f9d15ca9640 1 -- 10.20.193.28:0/1750687295 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d10110080 msgr2=0x7f9d101150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:17.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.107+0000 7f9d15ca9640 1 --2- 10.20.193.28:0/1750687295 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d10110080 0x7f9d101150d0 secure :-1 s=READY pgs=230 gs=1 cs=0 l=1 c_cookie=12498529416809259830 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9cf8009950 tx=0x7f9cf802f190 comp rx=0 tx=0).stop 2026-02-25T11:47:17.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.107+0000 7f9d15ca9640 1 -- 10.20.193.28:0/1750687295 shutdown_connections 2026-02-25T11:47:17.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.107+0000 7f9d15ca9640 1 --2- 10.20.193.28:0/1750687295 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9d10115880 0x7f9d10176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:17.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.107+0000 7f9d15ca9640 1 --2- 10.20.193.28:0/1750687295 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d10110080 0x7f9d101150d0 unknown :-1 s=CLOSED pgs=230 gs=1 cs=0 l=1 c_cookie=12498529416809259830 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:17.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.107+0000 7f9d15ca9640 1 --2- 10.20.193.28:0/1750687295 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9d1010f690 0x7f9d1010fa60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:17.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.107+0000 7f9d15ca9640 1 -- 10.20.193.28:0/1750687295 >> 10.20.193.28:0/1750687295 conn(0x7f9d100ecf00 msgr2=0x7f9d1010d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:17.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.107+0000 7f9d15ca9640 1 -- 10.20.193.28:0/1750687295 shutdown_connections 2026-02-25T11:47:17.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.108+0000 7f9d15ca9640 1 -- 10.20.193.28:0/1750687295 wait complete. 2026-02-25T11:47:17.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.109+0000 7f9d15ca9640 1 Processor -- start 2026-02-25T11:47:17.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.109+0000 7f9d15ca9640 1 -- start start 2026-02-25T11:47:17.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.109+0000 7f9d15ca9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d1010f690 0x7f9d100f5a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:17.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.109+0000 7f9d15ca9640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9d10110080 0x7f9d10176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:17.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.109+0000 7f9d15ca9640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9d10115880 0x7f9d100f6120 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:17.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.109+0000 7f9d15ca9640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9d100fb330 con 0x7f9d1010f690 2026-02-25T11:47:17.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.109+0000 7f9d15ca9640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d100fb4a0 con 0x7f9d10110080 2026-02-25T11:47:17.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.109+0000 7f9d15ca9640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d100fb660 con 0x7f9d10115880 2026-02-25T11:47:17.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.109+0000 7f9d14ca7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d1010f690 0x7f9d100f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:17.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.109+0000 7f9d14ca7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d1010f690 0x7f9d100f5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:33974/0 (socket says 10.20.193.28:33974) 2026-02-25T11:47:17.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.109+0000 7f9d14ca7640 1 -- 10.20.193.28:0/2865624120 learned_addr learned my addr 10.20.193.28:0/2865624120 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:17.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.109+0000 7f9d154a8640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9d10115880 0x7f9d100f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:17.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.109+0000 7f9d07fff640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9d10110080 0x7f9d10176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:17.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.110+0000 7f9d14ca7640 1 -- 10.20.193.28:0/2865624120 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9d10115880 msgr2=0x7f9d100f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:17.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.110+0000 7f9d14ca7640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9d10115880 0x7f9d100f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:17.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.110+0000 7f9d14ca7640 1 -- 10.20.193.28:0/2865624120 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9d10110080 msgr2=0x7f9d10176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:17.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.110+0000 7f9d14ca7640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9d10110080 0x7f9d10176150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:17.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.110+0000 7f9d14ca7640 1 -- 10.20.193.28:0/2865624120 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf8009570 con 0x7f9d1010f690 2026-02-25T11:47:17.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.110+0000 7f9d07fff640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9d10110080 0x7f9d10176150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:17.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.110+0000 7f9d14ca7640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d1010f690 0x7f9d100f5a20 secure :-1 s=READY pgs=231 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d00005db0 tx=0x7f9d0000ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:17.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.110+0000 7f9d05ffb640 1 -- 10.20.193.28:0/2865624120 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9d00004500 con 0x7f9d1010f690 2026-02-25T11:47:17.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.110+0000 7f9d05ffb640 1 -- 10.20.193.28:0/2865624120 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9d00004d10 con 0x7f9d1010f690 2026-02-25T11:47:17.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.110+0000 7f9d05ffb640 1 -- 10.20.193.28:0/2865624120 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9d00005280 con 0x7f9d1010f690 2026-02-25T11:47:17.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.110+0000 7f9d15ca9640 1 -- 10.20.193.28:0/2865624120 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d100fb940 con 0x7f9d1010f690 2026-02-25T11:47:17.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.110+0000 7f9d15ca9640 1 -- 10.20.193.28:0/2865624120 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d100fbdc0 con 0x7f9d1010f690 2026-02-25T11:47:17.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.111+0000 7f9d15ca9640 1 -- 10.20.193.28:0/2865624120 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d10110970 con 0x7f9d1010f690 2026-02-25T11:47:17.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.113+0000 7f9d05ffb640 1 -- 10.20.193.28:0/2865624120 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f9d00005410 con 0x7f9d1010f690 2026-02-25T11:47:17.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.115+0000 7f9d05ffb640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9cec0b5750 0x7f9cec0d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:17.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.115+0000 7f9d05ffb640 1 -- 10.20.193.28:0/2865624120 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f9d000e6c60 con 0x7f9d1010f690 2026-02-25T11:47:17.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.115+0000 7f9d05ffb640 1 -- 10.20.193.28:0/2865624120 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f9d000e70e0 con 0x7f9d1010f690 2026-02-25T11:47:17.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.115+0000 7f9d07fff640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9cec0b5750 0x7f9cec0d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:17.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.116+0000 7f9d07fff640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9cec0b5750 0x7f9cec0d5c00 secure :-1 s=READY pgs=176 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9cf80023d0 tx=0x7f9cf8031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:17.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.256+0000 7f9d15ca9640 1 -- 10.20.193.28:0/2865624120 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f9d1010fa60 con 0x7f9d1010f690 2026-02-25T11:47:17.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.644+0000 7f9d05ffb640 1 -- 10.20.193.28:0/2865624120 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f9d000a9710 con 0x7f9d1010f690 2026-02-25T11:47:17.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.647+0000 7f9d15ca9640 1 -- 10.20.193.28:0/2865624120 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9cec0b5750 msgr2=0x7f9cec0d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:17.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.647+0000 7f9d15ca9640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9cec0b5750 0x7f9cec0d5c00 secure :-1 s=READY pgs=176 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9cf80023d0 tx=0x7f9cf8031040 comp rx=0 tx=0).stop 2026-02-25T11:47:17.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.647+0000 7f9d15ca9640 1 -- 10.20.193.28:0/2865624120 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d1010f690 msgr2=0x7f9d100f5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:17.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.647+0000 7f9d15ca9640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d1010f690 0x7f9d100f5a20 secure :-1 s=READY pgs=231 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d00005db0 tx=0x7f9d0000ddf0 comp rx=0 tx=0).stop 2026-02-25T11:47:17.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.647+0000 7f9d15ca9640 1 -- 10.20.193.28:0/2865624120 shutdown_connections 2026-02-25T11:47:17.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.647+0000 7f9d15ca9640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9cec0b5750 0x7f9cec0d5c00 unknown :-1 s=CLOSED pgs=176 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:17.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.647+0000 7f9d15ca9640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9d10115880 0x7f9d100f6120 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:17.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.647+0000 7f9d15ca9640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9d10110080 0x7f9d10176150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:17.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.647+0000 7f9d15ca9640 1 --2- 10.20.193.28:0/2865624120 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d1010f690 0x7f9d100f5a20 unknown :-1 s=CLOSED pgs=231 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:17.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.647+0000 7f9d15ca9640 1 -- 10.20.193.28:0/2865624120 >> 10.20.193.28:0/2865624120 conn(0x7f9d100ecf00 msgr2=0x7f9d10111e90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:17.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.647+0000 7f9d15ca9640 1 -- 10.20.193.28:0/2865624120 shutdown_connections 2026-02-25T11:47:17.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:17.647+0000 7f9d15ca9640 1 -- 10.20.193.28:0/2865624120 wait complete. 2026-02-25T11:47:17.747 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs set cephfs session_timeout 300 2026-02-25T11:47:17.931 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:17.973 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:17 trial028 ceph-mon[39224]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2026-02-25T11:47:17.973 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:17 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1722411581' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3", "yes_i_really_mean_it": true}]': finished 2026-02-25T11:47:17.973 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:17 trial028 ceph-mon[39224]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2026-02-25T11:47:17.973 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:17 trial028 ceph-mon[39224]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-25T11:47:17.973 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:17 trial028 ceph-mon[39224]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2026-02-25T11:47:17.973 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:17 trial028 ceph-mon[39224]: daemon mds.g is now active in filesystem cephfs as rank 1 2026-02-25T11:47:17.973 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:17 trial028 ceph-mon[39224]: pgmap v196: 97 pgs: 9 creating+peering, 6 unknown, 82 active+clean; 578 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 1.3 KiB/s wr, 7 op/s 2026-02-25T11:47:17.973 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:17 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2865624120' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2026-02-25T11:47:17.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:17 trial167 ceph-mon[43392]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2026-02-25T11:47:18.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:17 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1722411581' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3", "yes_i_really_mean_it": true}]': finished 2026-02-25T11:47:18.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:17 trial167 ceph-mon[43392]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2026-02-25T11:47:18.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:17 trial167 ceph-mon[43392]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-25T11:47:18.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:17 trial167 ceph-mon[43392]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2026-02-25T11:47:18.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:17 trial167 ceph-mon[43392]: daemon mds.g is now active in filesystem cephfs as rank 1 2026-02-25T11:47:18.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:17 trial167 ceph-mon[43392]: pgmap v196: 97 pgs: 9 creating+peering, 6 unknown, 82 active+clean; 578 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 1.3 KiB/s wr, 7 op/s 2026-02-25T11:47:18.001 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:17 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2865624120' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2026-02-25T11:47:18.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:17 trial119 ceph-mon[42710]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2026-02-25T11:47:18.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:17 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1722411581' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3", "yes_i_really_mean_it": true}]': finished 2026-02-25T11:47:18.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:17 trial119 ceph-mon[42710]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2026-02-25T11:47:18.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:17 trial119 ceph-mon[42710]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2026-02-25T11:47:18.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:17 trial119 ceph-mon[42710]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2026-02-25T11:47:18.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:17 trial119 ceph-mon[42710]: daemon mds.g is now active in filesystem cephfs as rank 1 2026-02-25T11:47:18.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:17 trial119 ceph-mon[42710]: pgmap v196: 97 pgs: 9 creating+peering, 6 unknown, 82 active+clean; 578 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 1.3 KiB/s wr, 7 op/s 2026-02-25T11:47:18.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:17 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2865624120' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2026-02-25T11:47:18.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.114+0000 7f39b4db5640 1 -- 10.20.193.28:0/1478092715 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39b010ddc0 msgr2=0x7f39b0112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:18.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.114+0000 7f39b4db5640 1 --2- 10.20.193.28:0/1478092715 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39b010ddc0 0x7f39b0112e10 secure :-1 s=READY pgs=232 gs=1 cs=0 l=1 c_cookie=123399791297874069 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39a0009950 tx=0x7f39a002f190 comp rx=0 tx=0).stop 2026-02-25T11:47:18.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.114+0000 7f39b4db5640 1 -- 10.20.193.28:0/1478092715 shutdown_connections 2026-02-25T11:47:18.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.114+0000 7f39b4db5640 1 --2- 10.20.193.28:0/1478092715 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f39b01135c0 0x7f39b0173f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:18.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.114+0000 7f39b4db5640 1 --2- 10.20.193.28:0/1478092715 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39b010ddc0 0x7f39b0112e10 unknown :-1 s=CLOSED pgs=232 gs=1 cs=0 l=1 c_cookie=123399791297874069 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:18.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.114+0000 7f39b4db5640 1 --2- 10.20.193.28:0/1478092715 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39b010d3d0 0x7f39b010d7a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:18.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.114+0000 7f39b4db5640 1 -- 10.20.193.28:0/1478092715 >> 10.20.193.28:0/1478092715 conn(0x7f39b00eaca0 msgr2=0x7f39b010b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:18.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.114+0000 7f39b4db5640 1 -- 10.20.193.28:0/1478092715 shutdown_connections 2026-02-25T11:47:18.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.114+0000 7f39b4db5640 1 -- 10.20.193.28:0/1478092715 wait complete. 2026-02-25T11:47:18.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.115+0000 7f39b4db5640 1 Processor -- start 2026-02-25T11:47:18.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.115+0000 7f39b4db5640 1 -- start start 2026-02-25T11:47:18.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.115+0000 7f39b4db5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39b010d3d0 0x7f39b00f59a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:18.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.115+0000 7f39b4db5640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f39b010ddc0 0x7f39b0173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:18.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.115+0000 7f39b4db5640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39b01135c0 0x7f39b00f60a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:18.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.115+0000 7f39b4db5640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f39b00fb2b0 con 0x7f39b010d3d0 2026-02-25T11:47:18.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.115+0000 7f39b4db5640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f39b00fb420 con 0x7f39b010ddc0 2026-02-25T11:47:18.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.115+0000 7f39b4db5640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f39b00fb5e0 con 0x7f39b01135c0 2026-02-25T11:47:18.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.115+0000 7f39af7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39b010d3d0 0x7f39b00f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:18.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.115+0000 7f39af7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39b010d3d0 0x7f39b00f59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34006/0 (socket says 10.20.193.28:34006) 2026-02-25T11:47:18.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.115+0000 7f39af7fe640 1 -- 10.20.193.28:0/1279155509 learned_addr learned my addr 10.20.193.28:0/1279155509 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:18.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.115+0000 7f39aeffd640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f39b010ddc0 0x7f39b0173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:18.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.115+0000 7f39affff640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39b01135c0 0x7f39b00f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:18.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.116+0000 7f39af7fe640 1 -- 10.20.193.28:0/1279155509 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39b01135c0 msgr2=0x7f39b00f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:18.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.116+0000 7f39af7fe640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39b01135c0 0x7f39b00f60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:18.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.116+0000 7f39af7fe640 1 -- 10.20.193.28:0/1279155509 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f39b010ddc0 msgr2=0x7f39b0173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:18.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.116+0000 7f39af7fe640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f39b010ddc0 0x7f39b0173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:18.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.116+0000 7f39af7fe640 1 -- 10.20.193.28:0/1279155509 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a0009570 con 0x7f39b010d3d0 2026-02-25T11:47:18.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.116+0000 7f39af7fe640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39b010d3d0 0x7f39b00f59a0 secure :-1 s=READY pgs=233 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f399c005db0 tx=0x7f399c00ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:18.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.116+0000 7f39acff9640 1 -- 10.20.193.28:0/1279155509 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f399c004500 con 0x7f39b010d3d0 2026-02-25T11:47:18.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.116+0000 7f39acff9640 1 -- 10.20.193.28:0/1279155509 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f399c004d10 con 0x7f39b010d3d0 2026-02-25T11:47:18.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.116+0000 7f39acff9640 1 -- 10.20.193.28:0/1279155509 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f399c005280 con 0x7f39b010d3d0 2026-02-25T11:47:18.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.116+0000 7f39b4db5640 1 -- 10.20.193.28:0/1279155509 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b00fb8c0 con 0x7f39b010d3d0 2026-02-25T11:47:18.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.116+0000 7f39b4db5640 1 -- 10.20.193.28:0/1279155509 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b00fbd40 con 0x7f39b010d3d0 2026-02-25T11:47:18.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.117+0000 7f398a7fc640 1 -- 10.20.193.28:0/1279155509 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b010e6b0 con 0x7f39b010d3d0 2026-02-25T11:47:18.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.119+0000 7f39acff9640 1 -- 10.20.193.28:0/1279155509 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f399c005410 con 0x7f39b010d3d0 2026-02-25T11:47:18.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.121+0000 7f39acff9640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f39840b5750 0x7f39840d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:18.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.121+0000 7f39acff9640 1 -- 10.20.193.28:0/1279155509 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f399c0e6fc0 con 0x7f39b010d3d0 2026-02-25T11:47:18.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.121+0000 7f39aeffd640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f39840b5750 0x7f39840d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:18.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.121+0000 7f39acff9640 1 -- 10.20.193.28:0/1279155509 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f399c0e7440 con 0x7f39b010d3d0 2026-02-25T11:47:18.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.121+0000 7f39aeffd640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f39840b5750 0x7f39840d5c00 secure :-1 s=READY pgs=177 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39a00023d0 tx=0x7f39a0031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:18.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.264+0000 7f398a7fc640 1 -- 10.20.193.28:0/1279155509 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f39b010d7a0 con 0x7f39b010d3d0 2026-02-25T11:47:18.510 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-b[95783]: 2026-02-25T11:47:18.090+0000 7f8d3cb6c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2026-02-25T11:47:18.510 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-b[95783]: 2026-02-25T11:47:18.090+0000 7f8d3cb6c640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2026-02-25T11:47:18.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.568+0000 7f39acff9640 1 -- 10.20.193.28:0/1279155509 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f399c0a9a70 con 0x7f39b010d3d0 2026-02-25T11:47:18.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.570+0000 7f398a7fc640 1 -- 10.20.193.28:0/1279155509 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f39840b5750 msgr2=0x7f39840d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:18.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.570+0000 7f398a7fc640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f39840b5750 0x7f39840d5c00 secure :-1 s=READY pgs=177 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39a00023d0 tx=0x7f39a0031040 comp rx=0 tx=0).stop 2026-02-25T11:47:18.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.570+0000 7f398a7fc640 1 -- 10.20.193.28:0/1279155509 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39b010d3d0 msgr2=0x7f39b00f59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:18.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.570+0000 7f398a7fc640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39b010d3d0 0x7f39b00f59a0 secure :-1 s=READY pgs=233 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f399c005db0 tx=0x7f399c00ddf0 comp rx=0 tx=0).stop 2026-02-25T11:47:18.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.570+0000 7f398a7fc640 1 -- 10.20.193.28:0/1279155509 shutdown_connections 2026-02-25T11:47:18.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.570+0000 7f398a7fc640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f39840b5750 0x7f39840d5c00 unknown :-1 s=CLOSED pgs=177 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:18.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.570+0000 7f398a7fc640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f39b01135c0 0x7f39b00f60a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:18.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.570+0000 7f398a7fc640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f39b010ddc0 0x7f39b0173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:18.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.570+0000 7f398a7fc640 1 --2- 10.20.193.28:0/1279155509 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f39b010d3d0 0x7f39b00f59a0 unknown :-1 s=CLOSED pgs=233 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:18.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.570+0000 7f398a7fc640 1 -- 10.20.193.28:0/1279155509 >> 10.20.193.28:0/1279155509 conn(0x7f39b00eaca0 msgr2=0x7f39b010fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:18.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.571+0000 7f398a7fc640 1 -- 10.20.193.28:0/1279155509 shutdown_connections 2026-02-25T11:47:18.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:18.571+0000 7f398a7fc640 1 -- 10.20.193.28:0/1279155509 wait complete. 2026-02-25T11:47:18.674 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2026-02-25T11:47:18.674 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs subvolumegroup create cephfs qa 2026-02-25T11:47:18.856 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:18.898 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:18 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2865624120' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2026-02-25T11:47:18.898 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:18 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] up:active 2026-02-25T11:47:18.898 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:18 trial028 ceph-mon[39224]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2026-02-25T11:47:18.898 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:18 trial028 ceph-mon[39224]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2026-02-25T11:47:18.898 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:18 trial028 ceph-mon[39224]: Cluster is now healthy 2026-02-25T11:47:18.899 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:18 trial028 ceph-mon[39224]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2026-02-25T11:47:18.899 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:18 trial028 ceph-mon[39224]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2026-02-25T11:47:18.899 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:18 trial028 ceph-mon[39224]: daemon mds.i is now active in filesystem cephfs as rank 2 2026-02-25T11:47:18.899 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:18 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1279155509' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2026-02-25T11:47:18.899 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:18 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1279155509' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2026-02-25T11:47:18.899 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:18 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] up:active 2026-02-25T11:47:18.899 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:18 trial028 ceph-mon[39224]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2026-02-25T11:47:18.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:18 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2865624120' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2026-02-25T11:47:18.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:18 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] up:active 2026-02-25T11:47:18.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:18 trial167 ceph-mon[43392]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2026-02-25T11:47:18.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:18 trial167 ceph-mon[43392]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2026-02-25T11:47:18.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:18 trial167 ceph-mon[43392]: Cluster is now healthy 2026-02-25T11:47:18.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:18 trial167 ceph-mon[43392]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2026-02-25T11:47:18.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:18 trial167 ceph-mon[43392]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2026-02-25T11:47:18.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:18 trial167 ceph-mon[43392]: daemon mds.i is now active in filesystem cephfs as rank 2 2026-02-25T11:47:18.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:18 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1279155509' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2026-02-25T11:47:18.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:18 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1279155509' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2026-02-25T11:47:18.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:18 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] up:active 2026-02-25T11:47:18.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:18 trial167 ceph-mon[43392]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2026-02-25T11:47:19.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2865624120' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2026-02-25T11:47:19.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] up:active 2026-02-25T11:47:19.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-mon[42710]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2026-02-25T11:47:19.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-mon[42710]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2026-02-25T11:47:19.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-mon[42710]: Cluster is now healthy 2026-02-25T11:47:19.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-mon[42710]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2026-02-25T11:47:19.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-mon[42710]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2026-02-25T11:47:19.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-mon[42710]: daemon mds.i is now active in filesystem cephfs as rank 2 2026-02-25T11:47:19.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1279155509' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2026-02-25T11:47:19.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1279155509' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2026-02-25T11:47:19.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] up:active 2026-02-25T11:47:19.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:18 trial119 ceph-mon[42710]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2026-02-25T11:47:19.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.035+0000 7fc9f1896640 1 -- 10.20.193.28:0/3949146836 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9ec10f690 msgr2=0x7fc9ec10fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:19.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.035+0000 7fc9f1896640 1 --2- 10.20.193.28:0/3949146836 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9ec10f690 0x7fc9ec10fa60 secure :-1 s=READY pgs=234 gs=1 cs=0 l=1 c_cookie=16058471071019278254 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9dc009950 tx=0x7fc9dc02f190 comp rx=0 tx=0).stop 2026-02-25T11:47:19.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.035+0000 7fc9f1896640 1 -- 10.20.193.28:0/3949146836 shutdown_connections 2026-02-25T11:47:19.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.035+0000 7fc9f1896640 1 --2- 10.20.193.28:0/3949146836 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc9ec115880 0x7fc9ec176250 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.035+0000 7fc9f1896640 1 --2- 10.20.193.28:0/3949146836 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9ec110080 0x7fc9ec1150d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.035+0000 7fc9f1896640 1 --2- 10.20.193.28:0/3949146836 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9ec10f690 0x7fc9ec10fa60 unknown :-1 s=CLOSED pgs=234 gs=1 cs=0 l=1 c_cookie=16058471071019278254 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.035+0000 7fc9f1896640 1 -- 10.20.193.28:0/3949146836 >> 10.20.193.28:0/3949146836 conn(0x7fc9ec0ecf00 msgr2=0x7fc9ec10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:19.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.035+0000 7fc9f1896640 1 -- 10.20.193.28:0/3949146836 shutdown_connections 2026-02-25T11:47:19.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.035+0000 7fc9f1896640 1 -- 10.20.193.28:0/3949146836 wait complete. 2026-02-25T11:47:19.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.036+0000 7fc9f1896640 1 Processor -- start 2026-02-25T11:47:19.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.036+0000 7fc9f1896640 1 -- start start 2026-02-25T11:47:19.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.036+0000 7fc9f1896640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9ec10f690 0x7fc9ec0f5960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:19.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.036+0000 7fc9f1896640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9ec110080 0x7fc9ec176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:19.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9f1896640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc9ec115880 0x7fc9ec0f6060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:19.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9f1896640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec0fb270 con 0x7fc9ec110080 2026-02-25T11:47:19.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9f1896640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec0fb3e0 con 0x7fc9ec115880 2026-02-25T11:47:19.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9f1896640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec0fb5a0 con 0x7fc9ec10f690 2026-02-25T11:47:19.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9e3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9ec110080 0x7fc9ec176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:19.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9e3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9ec110080 0x7fc9ec176150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34020/0 (socket says 10.20.193.28:34020) 2026-02-25T11:47:19.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9e3fff640 1 -- 10.20.193.28:0/2982542882 learned_addr learned my addr 10.20.193.28:0/2982542882 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:19.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9f0894640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9ec10f690 0x7fc9ec0f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:19.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9f1095640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc9ec115880 0x7fc9ec0f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:19.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9e3fff640 1 -- 10.20.193.28:0/2982542882 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9ec10f690 msgr2=0x7fc9ec0f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:19.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9e3fff640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9ec10f690 0x7fc9ec0f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9e3fff640 1 -- 10.20.193.28:0/2982542882 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc9ec115880 msgr2=0x7fc9ec0f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:19.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9e3fff640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc9ec115880 0x7fc9ec0f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9e3fff640 1 -- 10.20.193.28:0/2982542882 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9dc009570 con 0x7fc9ec110080 2026-02-25T11:47:19.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9e3fff640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9ec110080 0x7fc9ec176150 secure :-1 s=READY pgs=235 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9d400d950 tx=0x7fc9d400de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:19.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.037+0000 7fc9e1ffb640 1 -- 10.20.193.28:0/2982542882 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc9d4004540 con 0x7fc9ec110080 2026-02-25T11:47:19.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.038+0000 7fc9e1ffb640 1 -- 10.20.193.28:0/2982542882 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc9d4009ce0 con 0x7fc9ec110080 2026-02-25T11:47:19.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.038+0000 7fc9e1ffb640 1 -- 10.20.193.28:0/2982542882 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc9d40106e0 con 0x7fc9ec110080 2026-02-25T11:47:19.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.038+0000 7fc9f1896640 1 -- 10.20.193.28:0/2982542882 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ec0fb880 con 0x7fc9ec110080 2026-02-25T11:47:19.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.038+0000 7fc9f1896640 1 -- 10.20.193.28:0/2982542882 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ec0fbda0 con 0x7fc9ec110080 2026-02-25T11:47:19.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.038+0000 7fc9f1896640 1 -- 10.20.193.28:0/2982542882 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ec110970 con 0x7fc9ec110080 2026-02-25T11:47:19.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.041+0000 7fc9e1ffb640 1 -- 10.20.193.28:0/2982542882 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7fc9d40097c0 con 0x7fc9ec110080 2026-02-25T11:47:19.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.042+0000 7fc9e1ffb640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9c80b5750 0x7fc9c80d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:19.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.042+0000 7fc9f0894640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9c80b5750 0x7fc9c80d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:19.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.042+0000 7fc9e1ffb640 1 -- 10.20.193.28:0/2982542882 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fc9d40e7180 con 0x7fc9ec110080 2026-02-25T11:47:19.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.042+0000 7fc9e1ffb640 1 -- 10.20.193.28:0/2982542882 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fc9d40e7600 con 0x7fc9ec110080 2026-02-25T11:47:19.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.043+0000 7fc9f0894640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9c80b5750 0x7fc9c80d5c00 secure :-1 s=READY pgs=178 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9dc0098a0 tx=0x7fc9dc03a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:19.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.178+0000 7fc9f1896640 1 -- 10.20.193.28:0/2982542882 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc9ec0f6ac0 con 0x7fc9c80b5750 2026-02-25T11:47:19.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.196+0000 7fc9e1ffb640 1 -- 10.20.193.28:0/2982542882 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc9ec0f6ac0 con 0x7fc9c80b5750 2026-02-25T11:47:19.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.198+0000 7fc9f1896640 1 -- 10.20.193.28:0/2982542882 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9c80b5750 msgr2=0x7fc9c80d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:19.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.198+0000 7fc9f1896640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9c80b5750 0x7fc9c80d5c00 secure :-1 s=READY pgs=178 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9dc0098a0 tx=0x7fc9dc03a040 comp rx=0 tx=0).stop 2026-02-25T11:47:19.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.198+0000 7fc9f1896640 1 -- 10.20.193.28:0/2982542882 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9ec110080 msgr2=0x7fc9ec176150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:19.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.198+0000 7fc9f1896640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9ec110080 0x7fc9ec176150 secure :-1 s=READY pgs=235 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9d400d950 tx=0x7fc9d400de20 comp rx=0 tx=0).stop 2026-02-25T11:47:19.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.198+0000 7fc9f1896640 1 -- 10.20.193.28:0/2982542882 shutdown_connections 2026-02-25T11:47:19.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.198+0000 7fc9f1896640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9c80b5750 0x7fc9c80d5c00 unknown :-1 s=CLOSED pgs=178 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.198+0000 7fc9f1896640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc9ec115880 0x7fc9ec0f6060 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.198+0000 7fc9f1896640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc9ec110080 0x7fc9ec176150 unknown :-1 s=CLOSED pgs=235 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.198+0000 7fc9f1896640 1 --2- 10.20.193.28:0/2982542882 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9ec10f690 0x7fc9ec0f5960 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.198+0000 7fc9f1896640 1 -- 10.20.193.28:0/2982542882 >> 10.20.193.28:0/2982542882 conn(0x7fc9ec0ecf00 msgr2=0x7fc9ec112d50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:19.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.198+0000 7fc9f1896640 1 -- 10.20.193.28:0/2982542882 shutdown_connections 2026-02-25T11:47:19.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.198+0000 7fc9f1896640 1 -- 10.20.193.28:0/2982542882 wait complete. 2026-02-25T11:47:19.301 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2026-02-25T11:47:19.483 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:19.525 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:47:19 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: 2026-02-25T11:47:19.185+0000 7f013b4c0640 -1 client.14937 error registering admin socket command: (17) File exists 2026-02-25T11:47:19.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.659+0000 7fba6f577640 1 -- 10.20.193.28:0/2003452660 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba7010d3d0 msgr2=0x7fba7010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:19.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.659+0000 7fba6f577640 1 --2- 10.20.193.28:0/2003452660 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba7010d3d0 0x7fba7010d7a0 secure :-1 s=READY pgs=238 gs=1 cs=0 l=1 c_cookie=2064980482606979042 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fba64009950 tx=0x7fba6402f190 comp rx=0 tx=0).stop 2026-02-25T11:47:19.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.659+0000 7fba6f577640 1 -- 10.20.193.28:0/2003452660 shutdown_connections 2026-02-25T11:47:19.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.659+0000 7fba6f577640 1 --2- 10.20.193.28:0/2003452660 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fba701135c0 0x7fba70173f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.659+0000 7fba6f577640 1 --2- 10.20.193.28:0/2003452660 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fba7010ddc0 0x7fba70112e10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.659+0000 7fba6f577640 1 --2- 10.20.193.28:0/2003452660 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba7010d3d0 0x7fba7010d7a0 unknown :-1 s=CLOSED pgs=238 gs=1 cs=0 l=1 c_cookie=2064980482606979042 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.659+0000 7fba6f577640 1 -- 10.20.193.28:0/2003452660 >> 10.20.193.28:0/2003452660 conn(0x7fba700eaca0 msgr2=0x7fba7010b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:19.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.659+0000 7fba6f577640 1 -- 10.20.193.28:0/2003452660 shutdown_connections 2026-02-25T11:47:19.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.659+0000 7fba6f577640 1 -- 10.20.193.28:0/2003452660 wait complete. 2026-02-25T11:47:19.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.660+0000 7fba6f577640 1 Processor -- start 2026-02-25T11:47:19.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.660+0000 7fba6f577640 1 -- start start 2026-02-25T11:47:19.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.660+0000 7fba6f577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fba7010d3d0 0x7fba700f5970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:19.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.660+0000 7fba6f577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fba7010ddc0 0x7fba70173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:19.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.660+0000 7fba6f577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba701135c0 0x7fba700f6070 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:19.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.660+0000 7fba6f577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fba700fb230 con 0x7fba701135c0 2026-02-25T11:47:19.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.660+0000 7fba6f577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fba700fb3a0 con 0x7fba7010d3d0 2026-02-25T11:47:19.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.660+0000 7fba6f577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fba700fb560 con 0x7fba7010ddc0 2026-02-25T11:47:19.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.660+0000 7fba6ed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba701135c0 0x7fba700f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:19.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.660+0000 7fba6ed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba701135c0 0x7fba700f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34050/0 (socket says 10.20.193.28:34050) 2026-02-25T11:47:19.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.660+0000 7fba6ed76640 1 -- 10.20.193.28:0/2422714664 learned_addr learned my addr 10.20.193.28:0/2422714664 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:19.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.660+0000 7fba6e575640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fba7010d3d0 0x7fba700f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:19.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.660+0000 7fba6dd74640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fba7010ddc0 0x7fba70173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:19.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.661+0000 7fba6ed76640 1 -- 10.20.193.28:0/2422714664 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fba7010ddc0 msgr2=0x7fba70173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:19.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.661+0000 7fba6ed76640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fba7010ddc0 0x7fba70173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.661+0000 7fba6ed76640 1 -- 10.20.193.28:0/2422714664 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fba7010d3d0 msgr2=0x7fba700f5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:19.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.661+0000 7fba6ed76640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fba7010d3d0 0x7fba700f5970 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.661+0000 7fba6ed76640 1 -- 10.20.193.28:0/2422714664 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba64009570 con 0x7fba701135c0 2026-02-25T11:47:19.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.661+0000 7fba6dd74640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fba7010ddc0 0x7fba70173e90 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:19.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.661+0000 7fba6ed76640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba701135c0 0x7fba700f6070 secure :-1 s=READY pgs=239 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fba6000f8d0 tx=0x7fba6000fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:19.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.661+0000 7fba577fe640 1 -- 10.20.193.28:0/2422714664 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fba6000dce0 con 0x7fba701135c0 2026-02-25T11:47:19.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.661+0000 7fba577fe640 1 -- 10.20.193.28:0/2422714664 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fba6000de40 con 0x7fba701135c0 2026-02-25T11:47:19.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.661+0000 7fba577fe640 1 -- 10.20.193.28:0/2422714664 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fba600116b0 con 0x7fba701135c0 2026-02-25T11:47:19.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.661+0000 7fba6f577640 1 -- 10.20.193.28:0/2422714664 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba700fb840 con 0x7fba701135c0 2026-02-25T11:47:19.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.661+0000 7fba6f577640 1 -- 10.20.193.28:0/2422714664 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba700fbcc0 con 0x7fba701135c0 2026-02-25T11:47:19.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.662+0000 7fba6f577640 1 -- 10.20.193.28:0/2422714664 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7010e6b0 con 0x7fba701135c0 2026-02-25T11:47:19.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.664+0000 7fba577fe640 1 -- 10.20.193.28:0/2422714664 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7fba60020060 con 0x7fba701135c0 2026-02-25T11:47:19.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.665+0000 7fba577fe640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fba480b5750 0x7fba480d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:19.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.665+0000 7fba577fe640 1 -- 10.20.193.28:0/2422714664 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fba60015070 con 0x7fba701135c0 2026-02-25T11:47:19.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.666+0000 7fba6e575640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fba480b5750 0x7fba480d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:19.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.666+0000 7fba6e575640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fba480b5750 0x7fba480d5c00 secure :-1 s=READY pgs=179 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fba640098a0 tx=0x7fba6403a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:19.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.666+0000 7fba577fe640 1 -- 10.20.193.28:0/2422714664 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fba600aa620 con 0x7fba701135c0 2026-02-25T11:47:19.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.805+0000 7fba6f577640 1 -- 10.20.193.28:0/2422714664 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fba700f6ad0 con 0x7fba480b5750 2026-02-25T11:47:19.808 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:19 trial028 ceph-mon[39224]: pgmap v197: 97 pgs: 97 active+clean; 581 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 633 B/s rd, 2.9 KiB/s wr, 12 op/s 2026-02-25T11:47:19.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:19 trial028 ceph-mon[39224]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:19.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.821+0000 7fba577fe640 1 -- 10.20.193.28:0/2422714664 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fba700f6ad0 con 0x7fba480b5750 2026-02-25T11:47:19.826 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.823+0000 7fba6f577640 1 -- 10.20.193.28:0/2422714664 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fba480b5750 msgr2=0x7fba480d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:19.826 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.823+0000 7fba6f577640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fba480b5750 0x7fba480d5c00 secure :-1 s=READY pgs=179 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fba640098a0 tx=0x7fba6403a040 comp rx=0 tx=0).stop 2026-02-25T11:47:19.826 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.823+0000 7fba6f577640 1 -- 10.20.193.28:0/2422714664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba701135c0 msgr2=0x7fba700f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:19.826 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.823+0000 7fba6f577640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba701135c0 0x7fba700f6070 secure :-1 s=READY pgs=239 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fba6000f8d0 tx=0x7fba6000fda0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.826 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.823+0000 7fba6f577640 1 -- 10.20.193.28:0/2422714664 shutdown_connections 2026-02-25T11:47:19.826 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.823+0000 7fba6f577640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fba480b5750 0x7fba480d5c00 unknown :-1 s=CLOSED pgs=179 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.826 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.823+0000 7fba6f577640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fba701135c0 0x7fba700f6070 unknown :-1 s=CLOSED pgs=239 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.823+0000 7fba6f577640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fba7010ddc0 0x7fba70173e90 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.823+0000 7fba6f577640 1 --2- 10.20.193.28:0/2422714664 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fba7010d3d0 0x7fba700f5970 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:19.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.823+0000 7fba6f577640 1 -- 10.20.193.28:0/2422714664 >> 10.20.193.28:0/2422714664 conn(0x7fba700eaca0 msgr2=0x7fba701105c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:19.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.823+0000 7fba6f577640 1 -- 10.20.193.28:0/2422714664 shutdown_connections 2026-02-25T11:47:19.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:19.823+0000 7fba6f577640 1 -- 10.20.193.28:0/2422714664 wait complete. 2026-02-25T11:47:19.924 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2026-02-25T11:47:19.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:19 trial167 ceph-mon[43392]: pgmap v197: 97 pgs: 97 active+clean; 581 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 633 B/s rd, 2.9 KiB/s wr, 12 op/s 2026-02-25T11:47:19.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:19 trial167 ceph-mon[43392]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:20.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:19 trial119 ceph-mon[42710]: pgmap v197: 97 pgs: 97 active+clean; 581 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 633 B/s rd, 2.9 KiB/s wr, 12 op/s 2026-02-25T11:47:20.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:19 trial119 ceph-mon[42710]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:20.108 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:20.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.281+0000 7fc46b837640 1 -- 10.20.193.28:0/3705834371 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc464115a30 msgr2=0x7fc464176400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:20.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.281+0000 7fc46b837640 1 --2- 10.20.193.28:0/3705834371 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc464115a30 0x7fc464176400 secure :-1 s=READY pgs=240 gs=1 cs=0 l=1 c_cookie=1532805425323335709 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc46000aa30 tx=0x7fc460030270 comp rx=0 tx=0).stop 2026-02-25T11:47:20.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.281+0000 7fc46b837640 1 -- 10.20.193.28:0/3705834371 shutdown_connections 2026-02-25T11:47:20.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.281+0000 7fc46b837640 1 --2- 10.20.193.28:0/3705834371 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc464115a30 0x7fc464176400 unknown :-1 s=CLOSED pgs=240 gs=1 cs=0 l=1 c_cookie=1532805425323335709 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.281+0000 7fc46b837640 1 --2- 10.20.193.28:0/3705834371 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc464110230 0x7fc464115280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.281+0000 7fc46b837640 1 --2- 10.20.193.28:0/3705834371 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc46410f840 0x7fc46410fc10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.281+0000 7fc46b837640 1 -- 10.20.193.28:0/3705834371 >> 10.20.193.28:0/3705834371 conn(0x7fc4640ed130 msgr2=0x7fc46410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:20.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.281+0000 7fc46b837640 1 -- 10.20.193.28:0/3705834371 shutdown_connections 2026-02-25T11:47:20.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.281+0000 7fc46b837640 1 -- 10.20.193.28:0/3705834371 wait complete. 2026-02-25T11:47:20.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.282+0000 7fc46b837640 1 Processor -- start 2026-02-25T11:47:20.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.282+0000 7fc46b837640 1 -- start start 2026-02-25T11:47:20.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.282+0000 7fc46b837640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc46410f840 0x7fc4640f5ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:20.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.282+0000 7fc46b837640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc464110230 0x7fc464176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:20.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.282+0000 7fc46b837640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc464115a30 0x7fc4640f61d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:20.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.282+0000 7fc46b837640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc4640fb3e0 con 0x7fc46410f840 2026-02-25T11:47:20.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.282+0000 7fc46b837640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc4640fb550 con 0x7fc464110230 2026-02-25T11:47:20.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.282+0000 7fc46b837640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc4640fb710 con 0x7fc464115a30 2026-02-25T11:47:20.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.282+0000 7fc4695ac640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc46410f840 0x7fc4640f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:20.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.282+0000 7fc4695ac640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc46410f840 0x7fc4640f5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34060/0 (socket says 10.20.193.28:34060) 2026-02-25T11:47:20.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.282+0000 7fc4695ac640 1 -- 10.20.193.28:0/1742034758 learned_addr learned my addr 10.20.193.28:0/1742034758 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:20.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc469dad640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc464115a30 0x7fc4640f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:20.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc468dab640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc464110230 0x7fc464176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:20.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc4695ac640 1 -- 10.20.193.28:0/1742034758 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc464115a30 msgr2=0x7fc4640f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:20.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc4695ac640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc464115a30 0x7fc4640f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc4695ac640 1 -- 10.20.193.28:0/1742034758 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc464110230 msgr2=0x7fc464176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:20.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc4695ac640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc464110230 0x7fc464176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc4695ac640 1 -- 10.20.193.28:0/1742034758 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46000a650 con 0x7fc46410f840 2026-02-25T11:47:20.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc468dab640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc464110230 0x7fc464176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:20.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc469dad640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc464115a30 0x7fc4640f61d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:20.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc4695ac640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc46410f840 0x7fc4640f5ad0 secure :-1 s=READY pgs=241 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc45400d950 tx=0x7fc45400de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:20.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc4527fc640 1 -- 10.20.193.28:0/1742034758 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc454004540 con 0x7fc46410f840 2026-02-25T11:47:20.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc4527fc640 1 -- 10.20.193.28:0/1742034758 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc454009ce0 con 0x7fc46410f840 2026-02-25T11:47:20.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc4527fc640 1 -- 10.20.193.28:0/1742034758 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc4540106e0 con 0x7fc46410f840 2026-02-25T11:47:20.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc46b837640 1 -- 10.20.193.28:0/1742034758 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4640fb9f0 con 0x7fc46410f840 2026-02-25T11:47:20.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.283+0000 7fc46b837640 1 -- 10.20.193.28:0/1742034758 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4640fbe70 con 0x7fc46410f840 2026-02-25T11:47:20.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.284+0000 7fc46b837640 1 -- 10.20.193.28:0/1742034758 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc42c005300 con 0x7fc46410f840 2026-02-25T11:47:20.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.286+0000 7fc4527fc640 1 -- 10.20.193.28:0/1742034758 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7fc4540097c0 con 0x7fc46410f840 2026-02-25T11:47:20.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.287+0000 7fc4527fc640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4400b5750 0x7fc4400d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:20.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.288+0000 7fc4527fc640 1 -- 10.20.193.28:0/1742034758 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fc4540e6d60 con 0x7fc46410f840 2026-02-25T11:47:20.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.288+0000 7fc468dab640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4400b5750 0x7fc4400d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:20.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.289+0000 7fc4527fc640 1 -- 10.20.193.28:0/1742034758 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fc4540a9810 con 0x7fc46410f840 2026-02-25T11:47:20.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.289+0000 7fc468dab640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4400b5750 0x7fc4400d5c00 secure :-1 s=READY pgs=180 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc4640fc470 tx=0x7fc458005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:20.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.427+0000 7fc46b837640 1 -- 10.20.193.28:0/1742034758 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fc42c002bb0 con 0x7fc4400b5750 2026-02-25T11:47:20.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.430+0000 7fc4527fc640 1 -- 10.20.193.28:0/1742034758 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc42c002bb0 con 0x7fc4400b5750 2026-02-25T11:47:20.433 INFO:teuthology.orchestra.run.trial028.stdout:/volumes/qa/sv_0/d31804c4-30cc-4124-85d4-92e1f982c86a 2026-02-25T11:47:20.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.432+0000 7fc46b837640 1 -- 10.20.193.28:0/1742034758 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4400b5750 msgr2=0x7fc4400d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:20.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.432+0000 7fc46b837640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4400b5750 0x7fc4400d5c00 secure :-1 s=READY pgs=180 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc4640fc470 tx=0x7fc458005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:20.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.432+0000 7fc46b837640 1 -- 10.20.193.28:0/1742034758 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc46410f840 msgr2=0x7fc4640f5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:20.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.432+0000 7fc46b837640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc46410f840 0x7fc4640f5ad0 secure :-1 s=READY pgs=241 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc45400d950 tx=0x7fc45400de20 comp rx=0 tx=0).stop 2026-02-25T11:47:20.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.432+0000 7fc46b837640 1 -- 10.20.193.28:0/1742034758 shutdown_connections 2026-02-25T11:47:20.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.432+0000 7fc46b837640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4400b5750 0x7fc4400d5c00 unknown :-1 s=CLOSED pgs=180 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.432+0000 7fc46b837640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc464115a30 0x7fc4640f61d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.432+0000 7fc46b837640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc464110230 0x7fc464176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.432+0000 7fc46b837640 1 --2- 10.20.193.28:0/1742034758 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc46410f840 0x7fc4640f5ad0 unknown :-1 s=CLOSED pgs=241 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.432+0000 7fc46b837640 1 -- 10.20.193.28:0/1742034758 >> 10.20.193.28:0/1742034758 conn(0x7fc4640ed130 msgr2=0x7fc46410dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:20.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.432+0000 7fc46b837640 1 -- 10.20.193.28:0/1742034758 shutdown_connections 2026-02-25T11:47:20.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.432+0000 7fc46b837640 1 -- 10.20.193.28:0/1742034758 wait complete. 2026-02-25T11:47:20.533 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2026-02-25T11:47:20.718 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:20.764 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:20 trial028 ceph-mon[39224]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:20.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.910+0000 7f2714f1a640 1 -- 10.20.193.28:0/3792550359 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2710110230 msgr2=0x7f2710115280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:20.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.910+0000 7f2714f1a640 1 --2- 10.20.193.28:0/3792550359 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2710110230 0x7f2710115280 secure :-1 s=READY pgs=242 gs=1 cs=0 l=1 c_cookie=315909776453288819 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26f8009950 tx=0x7f26f802f190 comp rx=0 tx=0).stop 2026-02-25T11:47:20.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.910+0000 7f2714f1a640 1 -- 10.20.193.28:0/3792550359 shutdown_connections 2026-02-25T11:47:20.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.910+0000 7f2714f1a640 1 --2- 10.20.193.28:0/3792550359 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2710115a30 0x7f2710176400 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.910+0000 7f2714f1a640 1 --2- 10.20.193.28:0/3792550359 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2710110230 0x7f2710115280 unknown :-1 s=CLOSED pgs=242 gs=1 cs=0 l=1 c_cookie=315909776453288819 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.910+0000 7f2714f1a640 1 --2- 10.20.193.28:0/3792550359 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f271010f840 0x7f271010fc10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.910+0000 7f2714f1a640 1 -- 10.20.193.28:0/3792550359 >> 10.20.193.28:0/3792550359 conn(0x7f27100ed130 msgr2=0x7f271010d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:20.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.910+0000 7f2714f1a640 1 -- 10.20.193.28:0/3792550359 shutdown_connections 2026-02-25T11:47:20.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.910+0000 7f2714f1a640 1 -- 10.20.193.28:0/3792550359 wait complete. 2026-02-25T11:47:20.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.911+0000 7f2714f1a640 1 Processor -- start 2026-02-25T11:47:20.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.911+0000 7f2714f1a640 1 -- start start 2026-02-25T11:47:20.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.911+0000 7f2714f1a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f271010f840 0x7f27100f5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:20.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.911+0000 7f2714f1a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2710110230 0x7f2710176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:20.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.911+0000 7f2714f1a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2710115a30 0x7f27100f6240 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:20.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.911+0000 7f2714f1a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f27100fb450 con 0x7f271010f840 2026-02-25T11:47:20.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.911+0000 7f2714f1a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f27100fb5c0 con 0x7f2710110230 2026-02-25T11:47:20.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.911+0000 7f2714f1a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27100fb780 con 0x7f2710115a30 2026-02-25T11:47:20.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.911+0000 7f270e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f271010f840 0x7f27100f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:20.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f270e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f271010f840 0x7f27100f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34072/0 (socket says 10.20.193.28:34072) 2026-02-25T11:47:20.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f270e575640 1 -- 10.20.193.28:0/1998582826 learned_addr learned my addr 10.20.193.28:0/1998582826 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:20.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f270ed76640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2710115a30 0x7f27100f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:20.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f270dd74640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2710110230 0x7f2710176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:20.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f270e575640 1 -- 10.20.193.28:0/1998582826 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2710115a30 msgr2=0x7f27100f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:20.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f270e575640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2710115a30 0x7f27100f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f270e575640 1 -- 10.20.193.28:0/1998582826 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2710110230 msgr2=0x7f2710176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:20.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f270e575640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2710110230 0x7f2710176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:20.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f270e575640 1 -- 10.20.193.28:0/1998582826 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f8009570 con 0x7f271010f840 2026-02-25T11:47:20.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f270e575640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f271010f840 0x7f27100f5b40 secure :-1 s=READY pgs=243 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2704009480 tx=0x7f270400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:20.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f26f77fe640 1 -- 10.20.193.28:0/1998582826 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f270400cd40 con 0x7f271010f840 2026-02-25T11:47:20.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f26f77fe640 1 -- 10.20.193.28:0/1998582826 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2704004d10 con 0x7f271010f840 2026-02-25T11:47:20.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f26f77fe640 1 -- 10.20.193.28:0/1998582826 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f27040106b0 con 0x7f271010f840 2026-02-25T11:47:20.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f2714f1a640 1 -- 10.20.193.28:0/1998582826 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27100fba60 con 0x7f271010f840 2026-02-25T11:47:20.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.912+0000 7f2714f1a640 1 -- 10.20.193.28:0/1998582826 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27100fbee0 con 0x7f271010f840 2026-02-25T11:47:20.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.913+0000 7f2714f1a640 1 -- 10.20.193.28:0/1998582826 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2710110b20 con 0x7f271010f840 2026-02-25T11:47:20.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.915+0000 7f26f77fe640 1 -- 10.20.193.28:0/1998582826 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f270401f060 con 0x7f271010f840 2026-02-25T11:47:20.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.917+0000 7f26f77fe640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f26e80b56d0 0x7f26e80d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:20.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.917+0000 7f26f77fe640 1 -- 10.20.193.28:0/1998582826 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f2704014070 con 0x7f271010f840 2026-02-25T11:47:20.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.917+0000 7f270dd74640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f26e80b56d0 0x7f26e80d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:20.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.917+0000 7f26f77fe640 1 -- 10.20.193.28:0/1998582826 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f270411f940 con 0x7f271010f840 2026-02-25T11:47:20.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:20.917+0000 7f270dd74640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f26e80b56d0 0x7f26e80d5b80 secure :-1 s=READY pgs=181 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f27100fc4e0 tx=0x7f26f803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:20.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:20 trial167 ceph-mon[43392]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:21.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:20 trial119 ceph-mon[42710]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:21.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.052+0000 7f2714f1a640 1 -- 10.20.193.28:0/1998582826 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f27100f6ca0 con 0x7f26e80b56d0 2026-02-25T11:47:21.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.068+0000 7f26f77fe640 1 -- 10.20.193.28:0/1998582826 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f27100f6ca0 con 0x7f26e80b56d0 2026-02-25T11:47:21.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.069+0000 7f2714f1a640 1 -- 10.20.193.28:0/1998582826 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f26e80b56d0 msgr2=0x7f26e80d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:21.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.069+0000 7f2714f1a640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f26e80b56d0 0x7f26e80d5b80 secure :-1 s=READY pgs=181 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f27100fc4e0 tx=0x7f26f803a040 comp rx=0 tx=0).stop 2026-02-25T11:47:21.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.069+0000 7f2714f1a640 1 -- 10.20.193.28:0/1998582826 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f271010f840 msgr2=0x7f27100f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:21.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.069+0000 7f2714f1a640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f271010f840 0x7f27100f5b40 secure :-1 s=READY pgs=243 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2704009480 tx=0x7f270400ee60 comp rx=0 tx=0).stop 2026-02-25T11:47:21.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.069+0000 7f2714f1a640 1 -- 10.20.193.28:0/1998582826 shutdown_connections 2026-02-25T11:47:21.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.069+0000 7f2714f1a640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f26e80b56d0 0x7f26e80d5b80 unknown :-1 s=CLOSED pgs=181 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:21.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.069+0000 7f2714f1a640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2710115a30 0x7f27100f6240 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:21.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.069+0000 7f2714f1a640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2710110230 0x7f2710176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:21.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.069+0000 7f2714f1a640 1 --2- 10.20.193.28:0/1998582826 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f271010f840 0x7f27100f5b40 unknown :-1 s=CLOSED pgs=243 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:21.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.069+0000 7f2714f1a640 1 -- 10.20.193.28:0/1998582826 >> 10.20.193.28:0/1998582826 conn(0x7f27100ed130 msgr2=0x7f271010dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:21.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.069+0000 7f2714f1a640 1 -- 10.20.193.28:0/1998582826 shutdown_connections 2026-02-25T11:47:21.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.069+0000 7f2714f1a640 1 -- 10.20.193.28:0/1998582826 wait complete. 2026-02-25T11:47:21.173 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2026-02-25T11:47:21.355 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:21.532 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.530+0000 7f13e0f5c640 1 -- 10.20.193.28:0/2677705857 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f13dc10f840 msgr2=0x7f13dc10fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:21.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.530+0000 7f13e0f5c640 1 --2- 10.20.193.28:0/2677705857 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f13dc10f840 0x7f13dc10fc10 secure :-1 s=READY pgs=244 gs=1 cs=0 l=1 c_cookie=14187605847336419117 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f13d0009950 tx=0x7f13d002f190 comp rx=0 tx=0).stop 2026-02-25T11:47:21.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.530+0000 7f13e0f5c640 1 -- 10.20.193.28:0/2677705857 shutdown_connections 2026-02-25T11:47:21.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.530+0000 7f13e0f5c640 1 --2- 10.20.193.28:0/2677705857 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f13dc115a30 0x7f13dc176400 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:21.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.530+0000 7f13e0f5c640 1 --2- 10.20.193.28:0/2677705857 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f13dc110230 0x7f13dc115280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:21.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.530+0000 7f13e0f5c640 1 --2- 10.20.193.28:0/2677705857 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f13dc10f840 0x7f13dc10fc10 unknown :-1 s=CLOSED pgs=244 gs=1 cs=0 l=1 c_cookie=14187605847336419117 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:21.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.530+0000 7f13e0f5c640 1 -- 10.20.193.28:0/2677705857 >> 10.20.193.28:0/2677705857 conn(0x7f13dc0ed130 msgr2=0x7f13dc10d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:21.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.530+0000 7f13e0f5c640 1 -- 10.20.193.28:0/2677705857 shutdown_connections 2026-02-25T11:47:21.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.530+0000 7f13e0f5c640 1 -- 10.20.193.28:0/2677705857 wait complete. 2026-02-25T11:47:21.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.531+0000 7f13e0f5c640 1 Processor -- start 2026-02-25T11:47:21.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.531+0000 7f13e0f5c640 1 -- start start 2026-02-25T11:47:21.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13e0f5c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f13dc10f840 0x7f13dc0f5b50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:21.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13e0f5c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f13dc110230 0x7f13dc176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:21.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13e0f5c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f13dc115a30 0x7f13dc0f6250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:21.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13e0f5c640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc0fb460 con 0x7f13dc115a30 2026-02-25T11:47:21.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13e0f5c640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc0fb5d0 con 0x7f13dc10f840 2026-02-25T11:47:21.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13e0f5c640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc0fb790 con 0x7f13dc110230 2026-02-25T11:47:21.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13dad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f13dc115a30 0x7f13dc0f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:21.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13d9d74640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f13dc110230 0x7f13dc176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:21.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13d9d74640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f13dc110230 0x7f13dc176300 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:35200/0 (socket says 10.20.193.28:35200) 2026-02-25T11:47:21.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13d9d74640 1 -- 10.20.193.28:0/4091398938 learned_addr learned my addr 10.20.193.28:0/4091398938 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:21.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13da575640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f13dc10f840 0x7f13dc0f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:21.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13dad76640 1 -- 10.20.193.28:0/4091398938 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f13dc110230 msgr2=0x7f13dc176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:21.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13dad76640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f13dc110230 0x7f13dc176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:21.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13dad76640 1 -- 10.20.193.28:0/4091398938 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f13dc10f840 msgr2=0x7f13dc0f5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:21.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13dad76640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f13dc10f840 0x7f13dc0f5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:21.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13dad76640 1 -- 10.20.193.28:0/4091398938 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d0009570 con 0x7f13dc115a30 2026-02-25T11:47:21.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13da575640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f13dc10f840 0x7f13dc0f5b50 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:21.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.532+0000 7f13d9d74640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f13dc110230 0x7f13dc176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:21.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.533+0000 7f13dad76640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f13dc115a30 0x7f13dc0f6250 secure :-1 s=READY pgs=245 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f13cc00f940 tx=0x7f13cc00fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:21.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.533+0000 7f13c37fe640 1 -- 10.20.193.28:0/4091398938 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f13cc00dce0 con 0x7f13dc115a30 2026-02-25T11:47:21.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.533+0000 7f13e0f5c640 1 -- 10.20.193.28:0/4091398938 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13dc0fba70 con 0x7f13dc115a30 2026-02-25T11:47:21.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.533+0000 7f13e0f5c640 1 -- 10.20.193.28:0/4091398938 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13dc0fbf90 con 0x7f13dc115a30 2026-02-25T11:47:21.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.533+0000 7f13c37fe640 1 -- 10.20.193.28:0/4091398938 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f13cc00de40 con 0x7f13dc115a30 2026-02-25T11:47:21.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.533+0000 7f13c37fe640 1 -- 10.20.193.28:0/4091398938 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f13cc019650 con 0x7f13dc115a30 2026-02-25T11:47:21.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.533+0000 7f13e0f5c640 1 -- 10.20.193.28:0/4091398938 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a0005300 con 0x7f13dc115a30 2026-02-25T11:47:21.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.535+0000 7f13c37fe640 1 -- 10.20.193.28:0/4091398938 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 15) ==== 157122+0+0 (secure 0 0 0) 0x7f13cc020060 con 0x7f13dc115a30 2026-02-25T11:47:21.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.537+0000 7f13c37fe640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f13b40b5750 0x7f13b40d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:21.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.537+0000 7f13c37fe640 1 -- 10.20.193.28:0/4091398938 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f13cc015070 con 0x7f13dc115a30 2026-02-25T11:47:21.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.537+0000 7f13c37fe640 1 -- 10.20.193.28:0/4091398938 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f13cc0e8160 con 0x7f13dc115a30 2026-02-25T11:47:21.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.537+0000 7f13da575640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f13b40b5750 0x7f13b40d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:21.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.537+0000 7f13da575640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f13b40b5750 0x7f13b40d5c00 secure :-1 s=READY pgs=182 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f13d0005b30 tx=0x7f13d003a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:21.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.668+0000 7f13c37fe640 1 -- 10.20.193.28:0/4091398938 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f13cc006780 con 0x7f13dc115a30 2026-02-25T11:47:21.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.680+0000 7f13e0f5c640 1 -- 10.20.193.28:0/4091398938 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f13a0002bb0 con 0x7f13b40b5750 2026-02-25T11:47:21.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.683+0000 7f13c37fe640 1 -- 10.20.193.28:0/4091398938 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f13a0002bb0 con 0x7f13b40b5750 2026-02-25T11:47:21.685 INFO:teuthology.orchestra.run.trial028.stdout:/volumes/qa/sv_1/1f4a3d66-4941-4d67-8bf4-dd7baf64696f 2026-02-25T11:47:21.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.686+0000 7f13e0f5c640 1 -- 10.20.193.28:0/4091398938 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f13b40b5750 msgr2=0x7f13b40d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:21.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.686+0000 7f13e0f5c640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f13b40b5750 0x7f13b40d5c00 secure :-1 s=READY pgs=182 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f13d0005b30 tx=0x7f13d003a040 comp rx=0 tx=0).stop 2026-02-25T11:47:21.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.686+0000 7f13e0f5c640 1 -- 10.20.193.28:0/4091398938 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f13dc115a30 msgr2=0x7f13dc0f6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:21.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.686+0000 7f13e0f5c640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f13dc115a30 0x7f13dc0f6250 secure :-1 s=READY pgs=245 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f13cc00f940 tx=0x7f13cc00fe10 comp rx=0 tx=0).stop 2026-02-25T11:47:21.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.686+0000 7f13e0f5c640 1 -- 10.20.193.28:0/4091398938 shutdown_connections 2026-02-25T11:47:21.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.686+0000 7f13e0f5c640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f13b40b5750 0x7f13b40d5c00 unknown :-1 s=CLOSED pgs=182 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:21.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.686+0000 7f13e0f5c640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f13dc115a30 0x7f13dc0f6250 unknown :-1 s=CLOSED pgs=245 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:21.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.686+0000 7f13e0f5c640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f13dc110230 0x7f13dc176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:21.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.686+0000 7f13e0f5c640 1 --2- 10.20.193.28:0/4091398938 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f13dc10f840 0x7f13dc0f5b50 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:21.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.686+0000 7f13e0f5c640 1 -- 10.20.193.28:0/4091398938 >> 10.20.193.28:0/4091398938 conn(0x7f13dc0ed130 msgr2=0x7f13dc111ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:21.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.686+0000 7f13e0f5c640 1 -- 10.20.193.28:0/4091398938 shutdown_connections 2026-02-25T11:47:21.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:21.686+0000 7f13e0f5c640 1 -- 10.20.193.28:0/4091398938 wait complete. 2026-02-25T11:47:21.786 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:47:21.968 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:21.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:21 trial167 ceph-mon[43392]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:21.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:21 trial167 ceph-mon[43392]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:21.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:21 trial167 ceph-mon[43392]: pgmap v198: 97 pgs: 97 active+clean; 582 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 2.0 KiB/s rd, 4.6 KiB/s wr, 16 op/s 2026-02-25T11:47:22.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:21 trial119 ceph-mon[42710]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:22.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:21 trial119 ceph-mon[42710]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:22.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:21 trial119 ceph-mon[42710]: pgmap v198: 97 pgs: 97 active+clean; 582 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 2.0 KiB/s rd, 4.6 KiB/s wr, 16 op/s 2026-02-25T11:47:22.011 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:21 trial028 ceph-mon[39224]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:22.012 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:21 trial028 ceph-mon[39224]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:22.012 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:21 trial028 ceph-mon[39224]: pgmap v198: 97 pgs: 97 active+clean; 582 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 2.0 KiB/s rd, 4.6 KiB/s wr, 16 op/s 2026-02-25T11:47:22.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.142+0000 7fea6f468640 1 -- 10.20.193.28:0/4035400125 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea68115a30 msgr2=0x7fea68176400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:22.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.142+0000 7fea6f468640 1 --2- 10.20.193.28:0/4035400125 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea68115a30 0x7fea68176400 secure :-1 s=READY pgs=246 gs=1 cs=0 l=1 c_cookie=37957074804111941 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fea6400aa30 tx=0x7fea64030270 comp rx=0 tx=0).stop 2026-02-25T11:47:22.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.143+0000 7fea6f468640 1 -- 10.20.193.28:0/4035400125 shutdown_connections 2026-02-25T11:47:22.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.143+0000 7fea6f468640 1 --2- 10.20.193.28:0/4035400125 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea68115a30 0x7fea68176400 unknown :-1 s=CLOSED pgs=246 gs=1 cs=0 l=1 c_cookie=37957074804111941 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.143+0000 7fea6f468640 1 --2- 10.20.193.28:0/4035400125 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fea68110230 0x7fea68115280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.143+0000 7fea6f468640 1 --2- 10.20.193.28:0/4035400125 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fea6810f840 0x7fea6810fc10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.143+0000 7fea6f468640 1 -- 10.20.193.28:0/4035400125 >> 10.20.193.28:0/4035400125 conn(0x7fea680ed130 msgr2=0x7fea6810d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:22.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.143+0000 7fea6f468640 1 -- 10.20.193.28:0/4035400125 shutdown_connections 2026-02-25T11:47:22.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.143+0000 7fea6f468640 1 -- 10.20.193.28:0/4035400125 wait complete. 2026-02-25T11:47:22.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.144+0000 7fea6f468640 1 Processor -- start 2026-02-25T11:47:22.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.144+0000 7fea6f468640 1 -- start start 2026-02-25T11:47:22.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.144+0000 7fea6f468640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fea6810f840 0x7fea680f5ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:22.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.144+0000 7fea6f468640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea68110230 0x7fea68176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:22.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.144+0000 7fea6f468640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fea68115a30 0x7fea680f62a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:22.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.144+0000 7fea6f468640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fea680fb4b0 con 0x7fea68110230 2026-02-25T11:47:22.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.144+0000 7fea6f468640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fea680fb620 con 0x7fea68115a30 2026-02-25T11:47:22.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.144+0000 7fea6f468640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea680fb7e0 con 0x7fea6810f840 2026-02-25T11:47:22.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.144+0000 7fea6c9dc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea68110230 0x7fea68176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:22.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.144+0000 7fea6c9dc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea68110230 0x7fea68176300 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34100/0 (socket says 10.20.193.28:34100) 2026-02-25T11:47:22.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea6c9dc640 1 -- 10.20.193.28:0/4218379184 learned_addr learned my addr 10.20.193.28:0/4218379184 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:22.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea6d9de640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fea68115a30 0x7fea680f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:22.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea6d1dd640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fea6810f840 0x7fea680f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:22.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea6c9dc640 1 -- 10.20.193.28:0/4218379184 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fea6810f840 msgr2=0x7fea680f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:22.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea6c9dc640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fea6810f840 0x7fea680f5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea6c9dc640 1 -- 10.20.193.28:0/4218379184 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fea68115a30 msgr2=0x7fea680f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:22.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea6c9dc640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fea68115a30 0x7fea680f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea6c9dc640 1 -- 10.20.193.28:0/4218379184 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea6400a650 con 0x7fea68110230 2026-02-25T11:47:22.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea6c9dc640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea68110230 0x7fea68176300 secure :-1 s=READY pgs=247 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fea5c00b500 tx=0x7fea5c00b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:22.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea567fc640 1 -- 10.20.193.28:0/4218379184 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fea5c0044a0 con 0x7fea68110230 2026-02-25T11:47:22.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea567fc640 1 -- 10.20.193.28:0/4218379184 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fea5c002b60 con 0x7fea68110230 2026-02-25T11:47:22.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea567fc640 1 -- 10.20.193.28:0/4218379184 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fea5c00fd20 con 0x7fea68110230 2026-02-25T11:47:22.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea6f468640 1 -- 10.20.193.28:0/4218379184 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea680fbac0 con 0x7fea68110230 2026-02-25T11:47:22.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.145+0000 7fea6f468640 1 -- 10.20.193.28:0/4218379184 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea680fbfe0 con 0x7fea68110230 2026-02-25T11:47:22.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.146+0000 7fea6f468640 1 -- 10.20.193.28:0/4218379184 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea68110b20 con 0x7fea68110230 2026-02-25T11:47:22.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.148+0000 7fea567fc640 1 -- 10.20.193.28:0/4218379184 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fea5c002cd0 con 0x7fea68110230 2026-02-25T11:47:22.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.149+0000 7fea567fc640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fea440b57e0 0x7fea440d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:22.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.149+0000 7fea567fc640 1 -- 10.20.193.28:0/4218379184 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fea5c0e5f60 con 0x7fea68110230 2026-02-25T11:47:22.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.150+0000 7fea6d1dd640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fea440b57e0 0x7fea440d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:22.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.150+0000 7fea6d1dd640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fea440b57e0 0x7fea440d5c90 secure :-1 s=READY pgs=183 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fea58004740 tx=0x7fea58004420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:22.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.151+0000 7fea567fc640 1 -- 10.20.193.28:0/4218379184 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fea5c0a89c0 con 0x7fea68110230 2026-02-25T11:47:22.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.293+0000 7fea6f468640 1 -- 10.20.193.28:0/4218379184 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea680f6d00 con 0x7fea68110230 2026-02-25T11:47:22.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.294+0000 7fea567fc640 1 -- 10.20.193.28:0/4218379184 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11940 (secure 0 0 0) 0x7fea5c0abb70 con 0x7fea68110230 2026-02-25T11:47:22.295 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:22.295 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":22,"btime":"2026-02-25T11:47:18:567831+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:18.567827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24730,24731,24749]},"id":1}]} 2026-02-25T11:47:22.296 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 22 2026-02-25T11:47:22.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.295+0000 7fea6f468640 1 -- 10.20.193.28:0/4218379184 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fea440b57e0 msgr2=0x7fea440d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:22.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.295+0000 7fea6f468640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fea440b57e0 0x7fea440d5c90 secure :-1 s=READY pgs=183 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fea58004740 tx=0x7fea58004420 comp rx=0 tx=0).stop 2026-02-25T11:47:22.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.295+0000 7fea6f468640 1 -- 10.20.193.28:0/4218379184 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea68110230 msgr2=0x7fea68176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:22.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.295+0000 7fea6f468640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea68110230 0x7fea68176300 secure :-1 s=READY pgs=247 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fea5c00b500 tx=0x7fea5c00b9d0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.295+0000 7fea6f468640 1 -- 10.20.193.28:0/4218379184 shutdown_connections 2026-02-25T11:47:22.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.295+0000 7fea6f468640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fea440b57e0 0x7fea440d5c90 unknown :-1 s=CLOSED pgs=183 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.295+0000 7fea6f468640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fea68115a30 0x7fea680f62a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.295+0000 7fea6f468640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fea68110230 0x7fea68176300 unknown :-1 s=CLOSED pgs=247 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.295+0000 7fea6f468640 1 --2- 10.20.193.28:0/4218379184 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fea6810f840 0x7fea680f5ba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.295+0000 7fea6f468640 1 -- 10.20.193.28:0/4218379184 >> 10.20.193.28:0/4218379184 conn(0x7fea680ed130 msgr2=0x7fea68112040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:22.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.295+0000 7fea6f468640 1 -- 10.20.193.28:0/4218379184 shutdown_connections 2026-02-25T11:47:22.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.295+0000 7fea6f468640 1 -- 10.20.193.28:0/4218379184 wait complete. 2026-02-25T11:47:22.396 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd dump --format=json 2026-02-25T11:47:22.580 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:22.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.760+0000 7f303752a640 1 -- 10.20.193.28:0/2480489463 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3030111370 msgr2=0x7f3030111d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:22.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.760+0000 7f303752a640 1 --2- 10.20.193.28:0/2480489463 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3030111370 0x7f3030111d50 secure :-1 s=READY pgs=248 gs=1 cs=0 l=1 c_cookie=5886242434571678266 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f302c00aa30 tx=0x7f302c030270 comp rx=0 tx=0).stop 2026-02-25T11:47:22.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.760+0000 7f303752a640 1 -- 10.20.193.28:0/2480489463 shutdown_connections 2026-02-25T11:47:22.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.760+0000 7f303752a640 1 --2- 10.20.193.28:0/2480489463 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3030111370 0x7f3030111d50 unknown :-1 s=CLOSED pgs=248 gs=1 cs=0 l=1 c_cookie=5886242434571678266 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.760+0000 7f303752a640 1 --2- 10.20.193.28:0/2480489463 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f303010bd50 0x7f3030110be0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.760+0000 7f303752a640 1 --2- 10.20.193.28:0/2480489463 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f303010b360 0x7f303010b730 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.760+0000 7f303752a640 1 -- 10.20.193.28:0/2480489463 >> 10.20.193.28:0/2480489463 conn(0x7f30300e8e70 msgr2=0x7f3030109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:22.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.760+0000 7f303752a640 1 -- 10.20.193.28:0/2480489463 shutdown_connections 2026-02-25T11:47:22.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.760+0000 7f303752a640 1 -- 10.20.193.28:0/2480489463 wait complete. 2026-02-25T11:47:22.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.760+0000 7f303752a640 1 Processor -- start 2026-02-25T11:47:22.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f303752a640 1 -- start start 2026-02-25T11:47:22.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f303752a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f303010b360 0x7f30301235c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:22.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f303752a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f303010bd50 0x7f3030123b00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:22.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f303752a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3030111370 0x7f3030128ed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:22.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f303752a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f30301294f0 con 0x7f303010b360 2026-02-25T11:47:22.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f303752a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3030129660 con 0x7f303010bd50 2026-02-25T11:47:22.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f303752a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3030129820 con 0x7f3030111370 2026-02-25T11:47:22.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f3036528640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f303010b360 0x7f30301235c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:22.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f3036528640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f303010b360 0x7f30301235c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34122/0 (socket says 10.20.193.28:34122) 2026-02-25T11:47:22.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f3036528640 1 -- 10.20.193.28:0/373154967 learned_addr learned my addr 10.20.193.28:0/373154967 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:22.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f3036d29640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3030111370 0x7f3030128ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:22.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f3035d27640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f303010bd50 0x7f3030123b00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:22.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f3036528640 1 -- 10.20.193.28:0/373154967 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3030111370 msgr2=0x7f3030128ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:22.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f3036528640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3030111370 0x7f3030128ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f3036528640 1 -- 10.20.193.28:0/373154967 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f303010bd50 msgr2=0x7f3030123b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:22.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f3036528640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f303010bd50 0x7f3030123b00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f3036528640 1 -- 10.20.193.28:0/373154967 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c00a650 con 0x7f303010b360 2026-02-25T11:47:22.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.761+0000 7f3036528640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f303010b360 0x7f30301235c0 secure :-1 s=READY pgs=249 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f302000d950 tx=0x7f302000de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:22.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.762+0000 7f30177fe640 1 -- 10.20.193.28:0/373154967 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3020004540 con 0x7f303010b360 2026-02-25T11:47:22.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.762+0000 7f30177fe640 1 -- 10.20.193.28:0/373154967 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3020009ce0 con 0x7f303010b360 2026-02-25T11:47:22.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.762+0000 7f303752a640 1 -- 10.20.193.28:0/373154967 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3030129b00 con 0x7f303010b360 2026-02-25T11:47:22.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.762+0000 7f303752a640 1 -- 10.20.193.28:0/373154967 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303016a200 con 0x7f303010b360 2026-02-25T11:47:22.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.762+0000 7f30177fe640 1 -- 10.20.193.28:0/373154967 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f30200106e0 con 0x7f303010b360 2026-02-25T11:47:22.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.762+0000 7f303752a640 1 -- 10.20.193.28:0/373154967 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f303010c480 con 0x7f303010b360 2026-02-25T11:47:22.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.764+0000 7f30177fe640 1 -- 10.20.193.28:0/373154967 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f30200097c0 con 0x7f303010b360 2026-02-25T11:47:22.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.766+0000 7f30177fe640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f300c0b57f0 0x7f300c0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:22.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.766+0000 7f3035d27640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f300c0b57f0 0x7f300c0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:22.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.766+0000 7f30177fe640 1 -- 10.20.193.28:0/373154967 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f302000f3c0 con 0x7f303010b360 2026-02-25T11:47:22.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.766+0000 7f30177fe640 1 -- 10.20.193.28:0/373154967 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f30200a9e50 con 0x7f303010b360 2026-02-25T11:47:22.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.766+0000 7f3035d27640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f300c0b57f0 0x7f300c0d5ca0 secure :-1 s=READY pgs=184 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3030129f60 tx=0x7f3024004420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:22.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.881+0000 7f303752a640 1 -- 10.20.193.28:0/373154967 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3030124720 con 0x7f303010b360 2026-02-25T11:47:22.883 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:22 trial028 ceph-mon[39224]: mgrmap e16: x(active, since 4m), standbys: z, y 2026-02-25T11:47:22.884 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:22 trial028 ceph-mon[39224]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:22.884 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:22 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/4218379184' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:22.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.882+0000 7f30177fe640 1 -- 10.20.193.28:0/373154967 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22340 (secure 0 0 0) 0x7f30200a9ff0 con 0x7f303010b360 2026-02-25T11:47:22.885 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:22.885 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":86,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","created":"2026-02-25T11:42:51.962844+0000","modified":"2026-02-25T11:47:14.624486+0000","last_up_change":"2026-02-25T11:46:45.156629+0000","last_in_change":"2026-02-25T11:46:31.585002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":38,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T11:44:43.266336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-25T11:47:11.236575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-25T11:47:12.196775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf596df7-d1fb-4d10-8d4c-14a35210bab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6803","nonce":668944205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6805","nonce":668944205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6809","nonce":668944205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6807","nonce":668944205}]},"public_addr":"10.20.193.28:6803/668944205","cluster_addr":"10.20.193.28:6805/668944205","heartbeat_back_addr":"10.20.193.28:6809/668944205","heartbeat_front_addr":"10.20.193.28:6807/668944205","state":["exists","up"]},{"osd":1,"uuid":"185719c0-7c2d-431f-9925-88bbee86a98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6800","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6801","nonce":3486566242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6802","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6803","nonce":3486566242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6806","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6807","nonce":3486566242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6804","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6805","nonce":3486566242}]},"public_addr":"10.20.193.119:6801/3486566242","cluster_addr":"10.20.193.119:6803/3486566242","heartbeat_back_addr":"10.20.193.119:6807/3486566242","heartbeat_front_addr":"10.20.193.119:6805/3486566242","state":["exists","up"]},{"osd":2,"uuid":"4de13b90-1c3c-4614-9fef-2d7315626433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6800","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6801","nonce":484781559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6802","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6803","nonce":484781559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6806","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6807","nonce":484781559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6804","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6805","nonce":484781559}]},"public_addr":"10.20.193.167:6801/484781559","cluster_addr":"10.20.193.167:6803/484781559","heartbeat_back_addr":"10.20.193.167:6807/484781559","heartbeat_front_addr":"10.20.193.167:6805/484781559","state":["exists","up"]},{"osd":3,"uuid":"45d9ea8e-5a77-4738-96da-ada98515fc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6811","nonce":2452541436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6813","nonce":2452541436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6817","nonce":2452541436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6815","nonce":2452541436}]},"public_addr":"10.20.193.28:6811/2452541436","cluster_addr":"10.20.193.28:6813/2452541436","heartbeat_back_addr":"10.20.193.28:6817/2452541436","heartbeat_front_addr":"10.20.193.28:6815/2452541436","state":["exists","up"]},{"osd":4,"uuid":"c9c3f512-665b-44b2-a6f0-f3356a4b661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6808","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6809","nonce":135971996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6810","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6811","nonce":135971996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6814","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6815","nonce":135971996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6812","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6813","nonce":135971996}]},"public_addr":"10.20.193.119:6809/135971996","cluster_addr":"10.20.193.119:6811/135971996","heartbeat_back_addr":"10.20.193.119:6815/135971996","heartbeat_front_addr":"10.20.193.119:6813/135971996","state":["exists","up"]},{"osd":5,"uuid":"777c3819-5c90-4b2b-a545-c024884c2eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6808","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6809","nonce":4203420884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6810","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6811","nonce":4203420884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6814","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6815","nonce":4203420884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6812","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6813","nonce":4203420884}]},"public_addr":"10.20.193.167:6809/4203420884","cluster_addr":"10.20.193.167:6811/4203420884","heartbeat_back_addr":"10.20.193.167:6815/4203420884","heartbeat_front_addr":"10.20.193.167:6813/4203420884","state":["exists","up"]},{"osd":6,"uuid":"bbd9de64-b9fe-499e-b451-c16828be2c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6819","nonce":621484999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6821","nonce":621484999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6825","nonce":621484999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6823","nonce":621484999}]},"public_addr":"10.20.193.28:6819/621484999","cluster_addr":"10.20.193.28:6821/621484999","heartbeat_back_addr":"10.20.193.28:6825/621484999","heartbeat_front_addr":"10.20.193.28:6823/621484999","state":["exists","up"]},{"osd":7,"uuid":"546d9f28-597b-454c-8456-fcc2d46c69f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6816","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6817","nonce":2724541747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6818","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6819","nonce":2724541747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6822","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6823","nonce":2724541747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6820","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6821","nonce":2724541747}]},"public_addr":"10.20.193.119:6817/2724541747","cluster_addr":"10.20.193.119:6819/2724541747","heartbeat_back_addr":"10.20.193.119:6823/2724541747","heartbeat_front_addr":"10.20.193.119:6821/2724541747","state":["exists","up"]},{"osd":8,"uuid":"a1a7cea6-3586-4bb2-aa4c-bf6f7799a501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6816","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6817","nonce":1328734317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6818","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6819","nonce":1328734317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6822","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6823","nonce":1328734317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6820","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6821","nonce":1328734317}]},"public_addr":"10.20.193.167:6817/1328734317","cluster_addr":"10.20.193.167:6819/1328734317","heartbeat_back_addr":"10.20.193.167:6823/1328734317","heartbeat_front_addr":"10.20.193.167:6821/1328734317","state":["exists","up"]},{"osd":9,"uuid":"03fef134-6946-43d5-9c26-dc0e59fffe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6827","nonce":354500625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6829","nonce":354500625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6833","nonce":354500625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6831","nonce":354500625}]},"public_addr":"10.20.193.28:6827/354500625","cluster_addr":"10.20.193.28:6829/354500625","heartbeat_back_addr":"10.20.193.28:6833/354500625","heartbeat_front_addr":"10.20.193.28:6831/354500625","state":["exists","up"]},{"osd":10,"uuid":"b46c29ef-285c-4184-be28-d99b76bc0e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6824","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6825","nonce":2996581116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6826","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6827","nonce":2996581116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6830","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6831","nonce":2996581116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6828","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6829","nonce":2996581116}]},"public_addr":"10.20.193.119:6825/2996581116","cluster_addr":"10.20.193.119:6827/2996581116","heartbeat_back_addr":"10.20.193.119:6831/2996581116","heartbeat_front_addr":"10.20.193.119:6829/2996581116","state":["exists","up"]},{"osd":11,"uuid":"f6696d8c-5de7-4fd9-b693-524f204e4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6824","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6825","nonce":2182906700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6826","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6827","nonce":2182906700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6830","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6831","nonce":2182906700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6828","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6829","nonce":2182906700}]},"public_addr":"10.20.193.167:6825/2182906700","cluster_addr":"10.20.193.167:6827/2182906700","heartbeat_back_addr":"10.20.193.167:6831/2182906700","heartbeat_front_addr":"10.20.193.167:6829/2182906700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:13.410235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:27.254773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:40.956615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:54.529866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:08.306772+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:21.899498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:35.500407+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:49.094965+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:02.790138+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:16.286937+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:29.922722+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:43.396028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.28:0/1467902196":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6800/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/2274270913":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/60087422":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:6800/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/1725332841":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/3082630924":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/2391715078":"2026-02-26T11:43:04.102029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T11:47:22.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.883+0000 7f303752a640 1 -- 10.20.193.28:0/373154967 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f300c0b57f0 msgr2=0x7f300c0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:22.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.883+0000 7f303752a640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f300c0b57f0 0x7f300c0d5ca0 secure :-1 s=READY pgs=184 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3030129f60 tx=0x7f3024004420 comp rx=0 tx=0).stop 2026-02-25T11:47:22.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.883+0000 7f303752a640 1 -- 10.20.193.28:0/373154967 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f303010b360 msgr2=0x7f30301235c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:22.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.883+0000 7f303752a640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f303010b360 0x7f30301235c0 secure :-1 s=READY pgs=249 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f302000d950 tx=0x7f302000de20 comp rx=0 tx=0).stop 2026-02-25T11:47:22.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.884+0000 7f303752a640 1 -- 10.20.193.28:0/373154967 shutdown_connections 2026-02-25T11:47:22.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.884+0000 7f303752a640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f300c0b57f0 0x7f300c0d5ca0 unknown :-1 s=CLOSED pgs=184 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.884+0000 7f303752a640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3030111370 0x7f3030128ed0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.884+0000 7f303752a640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f303010bd50 0x7f3030123b00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.884+0000 7f303752a640 1 --2- 10.20.193.28:0/373154967 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f303010b360 0x7f30301235c0 unknown :-1 s=CLOSED pgs=249 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:22.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.884+0000 7f303752a640 1 -- 10.20.193.28:0/373154967 >> 10.20.193.28:0/373154967 conn(0x7f30300e8e70 msgr2=0x7f3030109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:22.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.884+0000 7f303752a640 1 -- 10.20.193.28:0/373154967 shutdown_connections 2026-02-25T11:47:22.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:22.884+0000 7f303752a640 1 -- 10.20.193.28:0/373154967 wait complete. 2026-02-25T11:47:22.985 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2026-02-25T11:47:22.985 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph pg dump --format=json 2026-02-25T11:47:22.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:22 trial167 ceph-mon[43392]: mgrmap e16: x(active, since 4m), standbys: z, y 2026-02-25T11:47:22.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:22 trial167 ceph-mon[43392]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:22.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:22 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/4218379184' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:23.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:22 trial119 ceph-mon[42710]: mgrmap e16: x(active, since 4m), standbys: z, y 2026-02-25T11:47:23.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:22 trial119 ceph-mon[42710]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:23.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:22 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/4218379184' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:23.169 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:23.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.346+0000 7f2a97fff640 1 -- 10.20.193.28:0/1483323845 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a9810f690 msgr2=0x7f2a9810fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:23.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.346+0000 7f2a97fff640 1 --2- 10.20.193.28:0/1483323845 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a9810f690 0x7f2a9810fa60 secure :-1 s=READY pgs=250 gs=1 cs=0 l=1 c_cookie=11477310528130321305 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2a8c009950 tx=0x7f2a8c02f190 comp rx=0 tx=0).stop 2026-02-25T11:47:23.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.346+0000 7f2a97fff640 1 -- 10.20.193.28:0/1483323845 shutdown_connections 2026-02-25T11:47:23.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.346+0000 7f2a97fff640 1 --2- 10.20.193.28:0/1483323845 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2a98115880 0x7f2a98176250 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.346+0000 7f2a97fff640 1 --2- 10.20.193.28:0/1483323845 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2a98110080 0x7f2a981150d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.346+0000 7f2a97fff640 1 --2- 10.20.193.28:0/1483323845 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a9810f690 0x7f2a9810fa60 unknown :-1 s=CLOSED pgs=250 gs=1 cs=0 l=1 c_cookie=11477310528130321305 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.346+0000 7f2a97fff640 1 -- 10.20.193.28:0/1483323845 >> 10.20.193.28:0/1483323845 conn(0x7f2a980ecf00 msgr2=0x7f2a9810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:23.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.346+0000 7f2a97fff640 1 -- 10.20.193.28:0/1483323845 shutdown_connections 2026-02-25T11:47:23.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.346+0000 7f2a97fff640 1 -- 10.20.193.28:0/1483323845 wait complete. 2026-02-25T11:47:23.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.347+0000 7f2a97fff640 1 Processor -- start 2026-02-25T11:47:23.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.347+0000 7f2a97fff640 1 -- start start 2026-02-25T11:47:23.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.347+0000 7f2a97fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a9810f690 0x7f2a9811e100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:23.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.347+0000 7f2a97fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2a98110080 0x7f2a9811e640 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:23.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a97fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2a98115880 0x7f2a98123a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:23.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a97fff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98124030 con 0x7f2a9810f690 2026-02-25T11:47:23.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a97fff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a981241a0 con 0x7f2a98110080 2026-02-25T11:47:23.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a97fff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a98124360 con 0x7f2a98115880 2026-02-25T11:47:23.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a96ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a9810f690 0x7f2a9811e100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:23.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a96ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a9810f690 0x7f2a9811e100 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34140/0 (socket says 10.20.193.28:34140) 2026-02-25T11:47:23.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a96ffd640 1 -- 10.20.193.28:0/140005649 learned_addr learned my addr 10.20.193.28:0/140005649 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:23.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a967fc640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2a98110080 0x7f2a9811e640 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:23.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a977fe640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2a98115880 0x7f2a98123a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:23.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a96ffd640 1 -- 10.20.193.28:0/140005649 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2a98115880 msgr2=0x7f2a98123a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:23.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a96ffd640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2a98115880 0x7f2a98123a10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a96ffd640 1 -- 10.20.193.28:0/140005649 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2a98110080 msgr2=0x7f2a9811e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:23.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a96ffd640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2a98110080 0x7f2a9811e640 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a96ffd640 1 -- 10.20.193.28:0/140005649 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c009570 con 0x7f2a9810f690 2026-02-25T11:47:23.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.348+0000 7f2a96ffd640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a9810f690 0x7f2a9811e100 secure :-1 s=READY pgs=251 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2a8c0098a0 tx=0x7f2a8c0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:23.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.349+0000 7f2a77fff640 1 -- 10.20.193.28:0/140005649 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2a8c03d070 con 0x7f2a9810f690 2026-02-25T11:47:23.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.349+0000 7f2a97fff640 1 -- 10.20.193.28:0/140005649 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a981245e0 con 0x7f2a9810f690 2026-02-25T11:47:23.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.349+0000 7f2a77fff640 1 -- 10.20.193.28:0/140005649 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2a8c002a40 con 0x7f2a9810f690 2026-02-25T11:47:23.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.349+0000 7f2a77fff640 1 -- 10.20.193.28:0/140005649 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2a8c041820 con 0x7f2a9810f690 2026-02-25T11:47:23.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.349+0000 7f2a97fff640 1 -- 10.20.193.28:0/140005649 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a98164d40 con 0x7f2a9810f690 2026-02-25T11:47:23.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.350+0000 7f2a97fff640 1 -- 10.20.193.28:0/140005649 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a98110970 con 0x7f2a9810f690 2026-02-25T11:47:23.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.352+0000 7f2a77fff640 1 -- 10.20.193.28:0/140005649 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f2a8c0419c0 con 0x7f2a9810f690 2026-02-25T11:47:23.355 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.354+0000 7f2a77fff640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2a6c0b57f0 0x7f2a6c0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:23.356 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.354+0000 7f2a77fff640 1 -- 10.20.193.28:0/140005649 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f2a8c10b2d0 con 0x7f2a9810f690 2026-02-25T11:47:23.356 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.354+0000 7f2a967fc640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2a6c0b57f0 0x7f2a6c0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:23.356 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.354+0000 7f2a77fff640 1 -- 10.20.193.28:0/140005649 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f2a8c10b750 con 0x7f2a9810f690 2026-02-25T11:47:23.356 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.354+0000 7f2a967fc640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2a6c0b57f0 0x7f2a6c0d5ca0 secure :-1 s=READY pgs=185 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2a98124aa0 tx=0x7f2a80005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:23.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.468+0000 7f2a97fff640 1 -- 10.20.193.28:0/140005649 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a9811f260 con 0x7f2a6c0b57f0 2026-02-25T11:47:23.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.472+0000 7f2a77fff640 1 -- 10.20.193.28:0/140005649 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258942 (secure 0 0 0) 0x7f2a9811f260 con 0x7f2a6c0b57f0 2026-02-25T11:47:23.477 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:23.479 INFO:teuthology.orchestra.run.trial028.stderr:dumped all 2026-02-25T11:47:23.481 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.475+0000 7f2a97fff640 1 -- 10.20.193.28:0/140005649 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2a6c0b57f0 msgr2=0x7f2a6c0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:23.481 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.475+0000 7f2a97fff640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2a6c0b57f0 0x7f2a6c0d5ca0 secure :-1 s=READY pgs=185 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2a98124aa0 tx=0x7f2a80005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:23.481 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.475+0000 7f2a97fff640 1 -- 10.20.193.28:0/140005649 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a9810f690 msgr2=0x7f2a9811e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:23.481 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.475+0000 7f2a97fff640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a9810f690 0x7f2a9811e100 secure :-1 s=READY pgs=251 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2a8c0098a0 tx=0x7f2a8c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.481 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.475+0000 7f2a97fff640 1 -- 10.20.193.28:0/140005649 shutdown_connections 2026-02-25T11:47:23.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.475+0000 7f2a97fff640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2a6c0b57f0 0x7f2a6c0d5ca0 unknown :-1 s=CLOSED pgs=185 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.475+0000 7f2a97fff640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2a98115880 0x7f2a98123a10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.475+0000 7f2a97fff640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2a98110080 0x7f2a9811e640 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.475+0000 7f2a97fff640 1 --2- 10.20.193.28:0/140005649 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2a9810f690 0x7f2a9811e100 unknown :-1 s=CLOSED pgs=251 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.475+0000 7f2a97fff640 1 -- 10.20.193.28:0/140005649 >> 10.20.193.28:0/140005649 conn(0x7f2a980ecf00 msgr2=0x7f2a9810d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:23.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.475+0000 7f2a97fff640 1 -- 10.20.193.28:0/140005649 shutdown_connections 2026-02-25T11:47:23.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.475+0000 7f2a97fff640 1 -- 10.20.193.28:0/140005649 wait complete. 2026-02-25T11:47:23.575 INFO:teuthology.orchestra.run.trial028.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2026-02-25T11:47:23.071120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643578,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":148,"num_read_kb":170,"num_write":329,"num_write_kb":3813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":8808038400,"kb_used":338024,"kb_used_data":11384,"kb_used_omap":457,"kb_used_meta":325942,"kb_avail":8807700376,"statfs":{"total":9019431321600,"available":9019085185024,"internally_reserved":0,"allocated":11657216,"data_stored":7347049,"data_compressed":114024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":468743,"internal_metadata":333764857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53210,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":42,"num_read_kb":79,"num_write":140,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.453894"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627807+0000","last_change":"2026-02-25T11:47:13.630492+0000","last_active":"2026-02-25T11:47:14.627807+0000","last_peered":"2026-02-25T11:47:14.627807+0000","last_clean":"2026-02-25T11:47:14.627807+0000","last_became_active":"2026-02-25T11:47:13.630388+0000","last_became_peered":"2026-02-25T11:47:13.630388+0000","last_unstale":"2026-02-25T11:47:14.627807+0000","last_undegraded":"2026-02-25T11:47:14.627807+0000","last_fullsized":"2026-02-25T11:47:14.627807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:01:17.024058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627657+0000","last_change":"2026-02-25T11:47:13.629507+0000","last_active":"2026-02-25T11:47:14.627657+0000","last_peered":"2026-02-25T11:47:14.627657+0000","last_clean":"2026-02-25T11:47:14.627657+0000","last_became_active":"2026-02-25T11:47:13.628937+0000","last_became_peered":"2026-02-25T11:47:13.628937+0000","last_unstale":"2026-02-25T11:47:14.627657+0000","last_undegraded":"2026-02-25T11:47:14.627657+0000","last_fullsized":"2026-02-25T11:47:14.627657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:47:20.441522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627713+0000","last_change":"2026-02-25T11:47:13.630415+0000","last_active":"2026-02-25T11:47:14.627713+0000","last_peered":"2026-02-25T11:47:14.627713+0000","last_clean":"2026-02-25T11:47:14.627713+0000","last_became_active":"2026-02-25T11:47:13.630319+0000","last_became_peered":"2026-02-25T11:47:13.630319+0000","last_unstale":"2026-02-25T11:47:14.627713+0000","last_undegraded":"2026-02-25T11:47:14.627713+0000","last_fullsized":"2026-02-25T11:47:14.627713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:18:57.675760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627800+0000","last_change":"2026-02-25T11:47:13.633017+0000","last_active":"2026-02-25T11:47:14.627800+0000","last_peered":"2026-02-25T11:47:14.627800+0000","last_clean":"2026-02-25T11:47:14.627800+0000","last_became_active":"2026-02-25T11:47:13.632934+0000","last_became_peered":"2026-02-25T11:47:13.632934+0000","last_unstale":"2026-02-25T11:47:14.627800+0000","last_undegraded":"2026-02-25T11:47:14.627800+0000","last_fullsized":"2026-02-25T11:47:14.627800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:08:54.195980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628667+0000","last_change":"2026-02-25T11:47:13.638057+0000","last_active":"2026-02-25T11:47:14.628667+0000","last_peered":"2026-02-25T11:47:14.628667+0000","last_clean":"2026-02-25T11:47:14.628667+0000","last_became_active":"2026-02-25T11:47:13.636387+0000","last_became_peered":"2026-02-25T11:47:13.636387+0000","last_unstale":"2026-02-25T11:47:14.628667+0000","last_undegraded":"2026-02-25T11:47:14.628667+0000","last_fullsized":"2026-02-25T11:47:14.628667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:34:29.570622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750630+0000","last_change":"2026-02-25T11:47:13.631824+0000","last_active":"2026-02-25T11:47:14.750630+0000","last_peered":"2026-02-25T11:47:14.750630+0000","last_clean":"2026-02-25T11:47:14.750630+0000","last_became_active":"2026-02-25T11:47:13.631656+0000","last_became_peered":"2026-02-25T11:47:13.631656+0000","last_unstale":"2026-02-25T11:47:14.750630+0000","last_undegraded":"2026-02-25T11:47:14.750630+0000","last_fullsized":"2026-02-25T11:47:14.750630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:15:05.237015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628186+0000","last_change":"2026-02-25T11:47:13.626877+0000","last_active":"2026-02-25T11:47:14.628186+0000","last_peered":"2026-02-25T11:47:14.628186+0000","last_clean":"2026-02-25T11:47:14.628186+0000","last_became_active":"2026-02-25T11:47:13.626543+0000","last_became_peered":"2026-02-25T11:47:13.626543+0000","last_unstale":"2026-02-25T11:47:14.628186+0000","last_undegraded":"2026-02-25T11:47:14.628186+0000","last_fullsized":"2026-02-25T11:47:14.628186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:56:58.194911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635224+0000","last_change":"2026-02-25T11:47:13.637661+0000","last_active":"2026-02-25T11:47:14.635224+0000","last_peered":"2026-02-25T11:47:14.635224+0000","last_clean":"2026-02-25T11:47:14.635224+0000","last_became_active":"2026-02-25T11:47:13.637230+0000","last_became_peered":"2026-02-25T11:47:13.637230+0000","last_unstale":"2026-02-25T11:47:14.635224+0000","last_undegraded":"2026-02-25T11:47:14.635224+0000","last_fullsized":"2026-02-25T11:47:14.635224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:44:52.310360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751287+0000","last_change":"2026-02-25T11:47:13.625160+0000","last_active":"2026-02-25T11:47:14.751287+0000","last_peered":"2026-02-25T11:47:14.751287+0000","last_clean":"2026-02-25T11:47:14.751287+0000","last_became_active":"2026-02-25T11:47:13.625020+0000","last_became_peered":"2026-02-25T11:47:13.625020+0000","last_unstale":"2026-02-25T11:47:14.751287+0000","last_undegraded":"2026-02-25T11:47:14.751287+0000","last_fullsized":"2026-02-25T11:47:14.751287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:35:44.509551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750832+0000","last_change":"2026-02-25T11:47:13.637252+0000","last_active":"2026-02-25T11:47:14.750832+0000","last_peered":"2026-02-25T11:47:14.750832+0000","last_clean":"2026-02-25T11:47:14.750832+0000","last_became_active":"2026-02-25T11:47:13.636670+0000","last_became_peered":"2026-02-25T11:47:13.636670+0000","last_unstale":"2026-02-25T11:47:14.750832+0000","last_undegraded":"2026-02-25T11:47:14.750832+0000","last_fullsized":"2026-02-25T11:47:14.750832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:01:53.119101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628443+0000","last_change":"2026-02-25T11:47:13.628252+0000","last_active":"2026-02-25T11:47:14.628443+0000","last_peered":"2026-02-25T11:47:14.628443+0000","last_clean":"2026-02-25T11:47:14.628443+0000","last_became_active":"2026-02-25T11:47:13.628183+0000","last_became_peered":"2026-02-25T11:47:13.628183+0000","last_unstale":"2026-02-25T11:47:14.628443+0000","last_undegraded":"2026-02-25T11:47:14.628443+0000","last_fullsized":"2026-02-25T11:47:14.628443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:12:00.576747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.632358+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.631487+0000","last_became_peered":"2026-02-25T11:47:13.631487+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:41:17.150851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749735+0000","last_change":"2026-02-25T11:47:13.629675+0000","last_active":"2026-02-25T11:47:14.749735+0000","last_peered":"2026-02-25T11:47:14.749735+0000","last_clean":"2026-02-25T11:47:14.749735+0000","last_became_active":"2026-02-25T11:47:13.628686+0000","last_became_peered":"2026-02-25T11:47:13.628686+0000","last_unstale":"2026-02-25T11:47:14.749735+0000","last_undegraded":"2026-02-25T11:47:14.749735+0000","last_fullsized":"2026-02-25T11:47:14.749735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:00:54.430287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627615+0000","last_change":"2026-02-25T11:47:13.628842+0000","last_active":"2026-02-25T11:47:14.627615+0000","last_peered":"2026-02-25T11:47:14.627615+0000","last_clean":"2026-02-25T11:47:14.627615+0000","last_became_active":"2026-02-25T11:47:13.628695+0000","last_became_peered":"2026-02-25T11:47:13.628695+0000","last_unstale":"2026-02-25T11:47:14.627615+0000","last_undegraded":"2026-02-25T11:47:14.627615+0000","last_fullsized":"2026-02-25T11:47:14.627615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:10:05.404740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.635652+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.635505+0000","last_became_peered":"2026-02-25T11:47:13.635505+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:46:03.598577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749928+0000","last_change":"2026-02-25T11:47:13.628651+0000","last_active":"2026-02-25T11:47:14.749928+0000","last_peered":"2026-02-25T11:47:14.749928+0000","last_clean":"2026-02-25T11:47:14.749928+0000","last_became_active":"2026-02-25T11:47:13.628280+0000","last_became_peered":"2026-02-25T11:47:13.628280+0000","last_unstale":"2026-02-25T11:47:14.749928+0000","last_undegraded":"2026-02-25T11:47:14.749928+0000","last_fullsized":"2026-02-25T11:47:14.749928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:18:56.238078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628351+0000","last_change":"2026-02-25T11:47:13.630656+0000","last_active":"2026-02-25T11:47:14.628351+0000","last_peered":"2026-02-25T11:47:14.628351+0000","last_clean":"2026-02-25T11:47:14.628351+0000","last_became_active":"2026-02-25T11:47:13.630513+0000","last_became_peered":"2026-02-25T11:47:13.630513+0000","last_unstale":"2026-02-25T11:47:14.628351+0000","last_undegraded":"2026-02-25T11:47:14.628351+0000","last_fullsized":"2026-02-25T11:47:14.628351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:03:33.580576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.634036+0000","last_change":"2026-02-25T11:47:13.629202+0000","last_active":"2026-02-25T11:47:14.634036+0000","last_peered":"2026-02-25T11:47:14.634036+0000","last_clean":"2026-02-25T11:47:14.634036+0000","last_became_active":"2026-02-25T11:47:13.628860+0000","last_became_peered":"2026-02-25T11:47:13.628860+0000","last_unstale":"2026-02-25T11:47:14.634036+0000","last_undegraded":"2026-02-25T11:47:14.634036+0000","last_fullsized":"2026-02-25T11:47:14.634036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:47:14.635796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628235+0000","last_change":"2026-02-25T11:47:13.638872+0000","last_active":"2026-02-25T11:47:14.628235+0000","last_peered":"2026-02-25T11:47:14.628235+0000","last_clean":"2026-02-25T11:47:14.628235+0000","last_became_active":"2026-02-25T11:47:13.638443+0000","last_became_peered":"2026-02-25T11:47:13.638443+0000","last_unstale":"2026-02-25T11:47:14.628235+0000","last_undegraded":"2026-02-25T11:47:14.628235+0000","last_fullsized":"2026-02-25T11:47:14.628235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:51.699838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628595+0000","last_change":"2026-02-25T11:47:13.635894+0000","last_active":"2026-02-25T11:47:14.628595+0000","last_peered":"2026-02-25T11:47:14.628595+0000","last_clean":"2026-02-25T11:47:14.628595+0000","last_became_active":"2026-02-25T11:47:13.635725+0000","last_became_peered":"2026-02-25T11:47:13.635725+0000","last_unstale":"2026-02-25T11:47:14.628595+0000","last_undegraded":"2026-02-25T11:47:14.628595+0000","last_fullsized":"2026-02-25T11:47:14.628595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:28:20.134454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749794+0000","last_change":"2026-02-25T11:47:13.628117+0000","last_active":"2026-02-25T11:47:14.749794+0000","last_peered":"2026-02-25T11:47:14.749794+0000","last_clean":"2026-02-25T11:47:14.749794+0000","last_became_active":"2026-02-25T11:47:13.628037+0000","last_became_peered":"2026-02-25T11:47:13.628037+0000","last_unstale":"2026-02-25T11:47:14.749794+0000","last_undegraded":"2026-02-25T11:47:14.749794+0000","last_fullsized":"2026-02-25T11:47:14.749794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:28:22.976133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750830+0000","last_change":"2026-02-25T11:47:13.634652+0000","last_active":"2026-02-25T11:47:14.750830+0000","last_peered":"2026-02-25T11:47:14.750830+0000","last_clean":"2026-02-25T11:47:14.750830+0000","last_became_active":"2026-02-25T11:47:13.634433+0000","last_became_peered":"2026-02-25T11:47:13.634433+0000","last_unstale":"2026-02-25T11:47:14.750830+0000","last_undegraded":"2026-02-25T11:47:14.750830+0000","last_fullsized":"2026-02-25T11:47:14.750830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:01:29.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635319+0000","last_change":"2026-02-25T11:47:13.632721+0000","last_active":"2026-02-25T11:47:14.635319+0000","last_peered":"2026-02-25T11:47:14.635319+0000","last_clean":"2026-02-25T11:47:14.635319+0000","last_became_active":"2026-02-25T11:47:13.632626+0000","last_became_peered":"2026-02-25T11:47:13.632626+0000","last_unstale":"2026-02-25T11:47:14.635319+0000","last_undegraded":"2026-02-25T11:47:14.635319+0000","last_fullsized":"2026-02-25T11:47:14.635319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:28:35.201966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749844+0000","last_change":"2026-02-25T11:47:13.637504+0000","last_active":"2026-02-25T11:47:14.749844+0000","last_peered":"2026-02-25T11:47:14.749844+0000","last_clean":"2026-02-25T11:47:14.749844+0000","last_became_active":"2026-02-25T11:47:13.636824+0000","last_became_peered":"2026-02-25T11:47:13.636824+0000","last_unstale":"2026-02-25T11:47:14.749844+0000","last_undegraded":"2026-02-25T11:47:14.749844+0000","last_fullsized":"2026-02-25T11:47:14.749844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:20:31.416302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635724+0000","last_change":"2026-02-25T11:47:13.633135+0000","last_active":"2026-02-25T11:47:14.635724+0000","last_peered":"2026-02-25T11:47:14.635724+0000","last_clean":"2026-02-25T11:47:14.635724+0000","last_became_active":"2026-02-25T11:47:13.632966+0000","last_became_peered":"2026-02-25T11:47:13.632966+0000","last_unstale":"2026-02-25T11:47:14.635724+0000","last_undegraded":"2026-02-25T11:47:14.635724+0000","last_fullsized":"2026-02-25T11:47:14.635724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:26:17.052924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628538+0000","last_change":"2026-02-25T11:47:13.633210+0000","last_active":"2026-02-25T11:47:14.628538+0000","last_peered":"2026-02-25T11:47:14.628538+0000","last_clean":"2026-02-25T11:47:14.628538+0000","last_became_active":"2026-02-25T11:47:13.633099+0000","last_became_peered":"2026-02-25T11:47:13.633099+0000","last_unstale":"2026-02-25T11:47:14.628538+0000","last_undegraded":"2026-02-25T11:47:14.628538+0000","last_fullsized":"2026-02-25T11:47:14.628538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:07:46.884629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628656+0000","last_change":"2026-02-25T11:47:13.634757+0000","last_active":"2026-02-25T11:47:14.628656+0000","last_peered":"2026-02-25T11:47:14.628656+0000","last_clean":"2026-02-25T11:47:14.628656+0000","last_became_active":"2026-02-25T11:47:13.634643+0000","last_became_peered":"2026-02-25T11:47:13.634643+0000","last_unstale":"2026-02-25T11:47:14.628656+0000","last_undegraded":"2026-02-25T11:47:14.628656+0000","last_fullsized":"2026-02-25T11:47:14.628656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:20:23.383359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749674+0000","last_change":"2026-02-25T11:47:13.630515+0000","last_active":"2026-02-25T11:47:14.749674+0000","last_peered":"2026-02-25T11:47:14.749674+0000","last_clean":"2026-02-25T11:47:14.749674+0000","last_became_active":"2026-02-25T11:47:13.630202+0000","last_became_peered":"2026-02-25T11:47:13.630202+0000","last_unstale":"2026-02-25T11:47:14.749674+0000","last_undegraded":"2026-02-25T11:47:14.749674+0000","last_fullsized":"2026-02-25T11:47:14.749674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:51:24.646614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627367+0000","last_change":"2026-02-25T11:47:13.635749+0000","last_active":"2026-02-25T11:47:14.627367+0000","last_peered":"2026-02-25T11:47:14.627367+0000","last_clean":"2026-02-25T11:47:14.627367+0000","last_became_active":"2026-02-25T11:47:13.635284+0000","last_became_peered":"2026-02-25T11:47:13.635284+0000","last_unstale":"2026-02-25T11:47:14.627367+0000","last_undegraded":"2026-02-25T11:47:14.627367+0000","last_fullsized":"2026-02-25T11:47:14.627367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:23:45.042142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628732+0000","last_change":"2026-02-25T11:47:13.635448+0000","last_active":"2026-02-25T11:47:14.628732+0000","last_peered":"2026-02-25T11:47:14.628732+0000","last_clean":"2026-02-25T11:47:14.628732+0000","last_became_active":"2026-02-25T11:47:13.635232+0000","last_became_peered":"2026-02-25T11:47:13.635232+0000","last_unstale":"2026-02-25T11:47:14.628732+0000","last_undegraded":"2026-02-25T11:47:14.628732+0000","last_fullsized":"2026-02-25T11:47:14.628732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:33:38.020605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628332+0000","last_change":"2026-02-25T11:47:13.634237+0000","last_active":"2026-02-25T11:47:14.628332+0000","last_peered":"2026-02-25T11:47:14.628332+0000","last_clean":"2026-02-25T11:47:14.628332+0000","last_became_active":"2026-02-25T11:47:13.634054+0000","last_became_peered":"2026-02-25T11:47:13.634054+0000","last_unstale":"2026-02-25T11:47:14.628332+0000","last_undegraded":"2026-02-25T11:47:14.628332+0000","last_fullsized":"2026-02-25T11:47:14.628332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:38:34.603077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627786+0000","last_change":"2026-02-25T11:47:13.638347+0000","last_active":"2026-02-25T11:47:14.627786+0000","last_peered":"2026-02-25T11:47:14.627786+0000","last_clean":"2026-02-25T11:47:14.627786+0000","last_became_active":"2026-02-25T11:47:13.637984+0000","last_became_peered":"2026-02-25T11:47:13.637984+0000","last_unstale":"2026-02-25T11:47:14.627786+0000","last_undegraded":"2026-02-25T11:47:14.627786+0000","last_fullsized":"2026-02-25T11:47:14.627786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:36:11.273078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628202+0000","last_change":"2026-02-25T11:47:13.634523+0000","last_active":"2026-02-25T11:47:14.628202+0000","last_peered":"2026-02-25T11:47:14.628202+0000","last_clean":"2026-02-25T11:47:14.628202+0000","last_became_active":"2026-02-25T11:47:13.634184+0000","last_became_peered":"2026-02-25T11:47:13.634184+0000","last_unstale":"2026-02-25T11:47:14.628202+0000","last_undegraded":"2026-02-25T11:47:14.628202+0000","last_fullsized":"2026-02-25T11:47:14.628202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:52:23.917090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.576471+0000","last_change":"2026-02-25T11:47:12.613380+0000","last_active":"2026-02-25T11:47:18.576471+0000","last_peered":"2026-02-25T11:47:18.576471+0000","last_clean":"2026-02-25T11:47:18.576471+0000","last_became_active":"2026-02-25T11:47:12.613224+0000","last_became_peered":"2026-02-25T11:47:12.613224+0000","last_unstale":"2026-02-25T11:47:18.576471+0000","last_undegraded":"2026-02-25T11:47:18.576471+0000","last_fullsized":"2026-02-25T11:47:18.576471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:59:39.894067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749915+0000","last_change":"2026-02-25T11:47:13.636227+0000","last_active":"2026-02-25T11:47:14.749915+0000","last_peered":"2026-02-25T11:47:14.749915+0000","last_clean":"2026-02-25T11:47:14.749915+0000","last_became_active":"2026-02-25T11:47:13.635212+0000","last_became_peered":"2026-02-25T11:47:13.635212+0000","last_unstale":"2026-02-25T11:47:14.749915+0000","last_undegraded":"2026-02-25T11:47:14.749915+0000","last_fullsized":"2026-02-25T11:47:14.749915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:51:07.635399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657308+0000","last_change":"2026-02-25T11:47:12.613789+0000","last_active":"2026-02-25T11:47:17.657308+0000","last_peered":"2026-02-25T11:47:17.657308+0000","last_clean":"2026-02-25T11:47:17.657308+0000","last_became_active":"2026-02-25T11:47:12.613635+0000","last_became_peered":"2026-02-25T11:47:12.613635+0000","last_unstale":"2026-02-25T11:47:17.657308+0000","last_undegraded":"2026-02-25T11:47:17.657308+0000","last_fullsized":"2026-02-25T11:47:17.657308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:13:32.216593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657439+0000","last_change":"2026-02-25T11:47:12.612012+0000","last_active":"2026-02-25T11:47:17.657439+0000","last_peered":"2026-02-25T11:47:17.657439+0000","last_clean":"2026-02-25T11:47:17.657439+0000","last_became_active":"2026-02-25T11:47:12.611862+0000","last_became_peered":"2026-02-25T11:47:12.611862+0000","last_unstale":"2026-02-25T11:47:17.657439+0000","last_undegraded":"2026-02-25T11:47:17.657439+0000","last_fullsized":"2026-02-25T11:47:17.657439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:08:17.490884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635279+0000","last_change":"2026-02-25T11:47:13.627129+0000","last_active":"2026-02-25T11:47:14.635279+0000","last_peered":"2026-02-25T11:47:14.635279+0000","last_clean":"2026-02-25T11:47:14.635279+0000","last_became_active":"2026-02-25T11:47:13.626987+0000","last_became_peered":"2026-02-25T11:47:13.626987+0000","last_unstale":"2026-02-25T11:47:14.635279+0000","last_undegraded":"2026-02-25T11:47:14.635279+0000","last_fullsized":"2026-02-25T11:47:14.635279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:53:45.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749940+0000","last_change":"2026-02-25T11:47:12.611992+0000","last_active":"2026-02-25T11:47:14.749940+0000","last_peered":"2026-02-25T11:47:14.749940+0000","last_clean":"2026-02-25T11:47:14.749940+0000","last_became_active":"2026-02-25T11:47:12.611502+0000","last_became_peered":"2026-02-25T11:47:12.611502+0000","last_unstale":"2026-02-25T11:47:14.749940+0000","last_undegraded":"2026-02-25T11:47:14.749940+0000","last_fullsized":"2026-02-25T11:47:14.749940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:15:45.718628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633835+0000","last_change":"2026-02-25T11:47:13.635344+0000","last_active":"2026-02-25T11:47:14.633835+0000","last_peered":"2026-02-25T11:47:14.633835+0000","last_clean":"2026-02-25T11:47:14.633835+0000","last_became_active":"2026-02-25T11:47:13.634327+0000","last_became_peered":"2026-02-25T11:47:13.634327+0000","last_unstale":"2026-02-25T11:47:14.633835+0000","last_undegraded":"2026-02-25T11:47:14.633835+0000","last_fullsized":"2026-02-25T11:47:14.633835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:30:06.778398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:20.578577+0000","last_change":"2026-02-25T11:47:12.611130+0000","last_active":"2026-02-25T11:47:20.578577+0000","last_peered":"2026-02-25T11:47:20.578577+0000","last_clean":"2026-02-25T11:47:20.578577+0000","last_became_active":"2026-02-25T11:47:12.610838+0000","last_became_peered":"2026-02-25T11:47:12.610838+0000","last_unstale":"2026-02-25T11:47:20.578577+0000","last_undegraded":"2026-02-25T11:47:20.578577+0000","last_fullsized":"2026-02-25T11:47:20.578577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:34:19.366424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751244+0000","last_change":"2026-02-25T11:47:13.629263+0000","last_active":"2026-02-25T11:47:14.751244+0000","last_peered":"2026-02-25T11:47:14.751244+0000","last_clean":"2026-02-25T11:47:14.751244+0000","last_became_active":"2026-02-25T11:47:13.628765+0000","last_became_peered":"2026-02-25T11:47:13.628765+0000","last_unstale":"2026-02-25T11:47:14.751244+0000","last_undegraded":"2026-02-25T11:47:14.751244+0000","last_fullsized":"2026-02-25T11:47:14.751244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:49:04.773052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.649872+0000","last_change":"2026-02-25T11:47:12.613455+0000","last_active":"2026-02-25T11:47:16.649872+0000","last_peered":"2026-02-25T11:47:16.649872+0000","last_clean":"2026-02-25T11:47:16.649872+0000","last_became_active":"2026-02-25T11:47:12.612904+0000","last_became_peered":"2026-02-25T11:47:12.612904+0000","last_unstale":"2026-02-25T11:47:16.649872+0000","last_undegraded":"2026-02-25T11:47:16.649872+0000","last_fullsized":"2026-02-25T11:47:16.649872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:16:13.149517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627673+0000","last_change":"2026-02-25T11:47:13.635178+0000","last_active":"2026-02-25T11:47:14.627673+0000","last_peered":"2026-02-25T11:47:14.627673+0000","last_clean":"2026-02-25T11:47:14.627673+0000","last_became_active":"2026-02-25T11:47:13.634985+0000","last_became_peered":"2026-02-25T11:47:13.634985+0000","last_unstale":"2026-02-25T11:47:14.627673+0000","last_undegraded":"2026-02-25T11:47:14.627673+0000","last_fullsized":"2026-02-25T11:47:14.627673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:08.068457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653282+0000","last_change":"2026-02-25T11:47:12.610853+0000","last_active":"2026-02-25T11:47:16.653282+0000","last_peered":"2026-02-25T11:47:16.653282+0000","last_clean":"2026-02-25T11:47:16.653282+0000","last_became_active":"2026-02-25T11:47:12.610639+0000","last_became_peered":"2026-02-25T11:47:12.610639+0000","last_unstale":"2026-02-25T11:47:16.653282+0000","last_undegraded":"2026-02-25T11:47:16.653282+0000","last_fullsized":"2026-02-25T11:47:16.653282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:00:06.042790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":694,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750953+0000","last_change":"2026-02-25T11:47:13.630063+0000","last_active":"2026-02-25T11:47:14.750953+0000","last_peered":"2026-02-25T11:47:14.750953+0000","last_clean":"2026-02-25T11:47:14.750953+0000","last_became_active":"2026-02-25T11:47:13.629906+0000","last_became_peered":"2026-02-25T11:47:13.629906+0000","last_unstale":"2026-02-25T11:47:14.750953+0000","last_undegraded":"2026-02-25T11:47:14.750953+0000","last_fullsized":"2026-02-25T11:47:14.750953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:09:13.614948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'21","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:21.581481+0000","last_change":"2026-02-25T11:47:12.615044+0000","last_active":"2026-02-25T11:47:21.581481+0000","last_peered":"2026-02-25T11:47:21.581481+0000","last_clean":"2026-02-25T11:47:21.581481+0000","last_became_active":"2026-02-25T11:47:12.614698+0000","last_became_peered":"2026-02-25T11:47:12.614698+0000","last_unstale":"2026-02-25T11:47:21.581481+0000","last_undegraded":"2026-02-25T11:47:21.581481+0000","last_fullsized":"2026-02-25T11:47:21.581481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:52:32.148801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49007,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":53,"num_write":22,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628510+0000","last_change":"2026-02-25T11:47:13.630945+0000","last_active":"2026-02-25T11:47:14.628510+0000","last_peered":"2026-02-25T11:47:14.628510+0000","last_clean":"2026-02-25T11:47:14.628510+0000","last_became_active":"2026-02-25T11:47:13.630693+0000","last_became_peered":"2026-02-25T11:47:13.630693+0000","last_unstale":"2026-02-25T11:47:14.628510+0000","last_undegraded":"2026-02-25T11:47:14.628510+0000","last_fullsized":"2026-02-25T11:47:14.628510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:39:52.744366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656758+0000","last_change":"2026-02-25T11:47:12.614950+0000","last_active":"2026-02-25T11:47:17.656758+0000","last_peered":"2026-02-25T11:47:17.656758+0000","last_clean":"2026-02-25T11:47:17.656758+0000","last_became_active":"2026-02-25T11:47:12.614272+0000","last_became_peered":"2026-02-25T11:47:12.614272+0000","last_unstale":"2026-02-25T11:47:17.656758+0000","last_undegraded":"2026-02-25T11:47:17.656758+0000","last_fullsized":"2026-02-25T11:47:17.656758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:45:50.928166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749596+0000","last_change":"2026-02-25T11:47:13.633128+0000","last_active":"2026-02-25T11:47:14.749596+0000","last_peered":"2026-02-25T11:47:14.749596+0000","last_clean":"2026-02-25T11:47:14.749596+0000","last_became_active":"2026-02-25T11:47:13.633047+0000","last_became_peered":"2026-02-25T11:47:13.633047+0000","last_unstale":"2026-02-25T11:47:14.749596+0000","last_undegraded":"2026-02-25T11:47:14.749596+0000","last_fullsized":"2026-02-25T11:47:14.749596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:56:33.191193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653783+0000","last_change":"2026-02-25T11:47:12.616898+0000","last_active":"2026-02-25T11:47:16.653783+0000","last_peered":"2026-02-25T11:47:16.653783+0000","last_clean":"2026-02-25T11:47:16.653783+0000","last_became_active":"2026-02-25T11:47:12.616774+0000","last_became_peered":"2026-02-25T11:47:12.616774+0000","last_unstale":"2026-02-25T11:47:16.653783+0000","last_undegraded":"2026-02-25T11:47:16.653783+0000","last_fullsized":"2026-02-25T11:47:16.653783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:59:33.930544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628464+0000","last_change":"2026-02-25T11:47:13.630377+0000","last_active":"2026-02-25T11:47:14.628464+0000","last_peered":"2026-02-25T11:47:14.628464+0000","last_clean":"2026-02-25T11:47:14.628464+0000","last_became_active":"2026-02-25T11:47:13.630124+0000","last_became_peered":"2026-02-25T11:47:13.630124+0000","last_unstale":"2026-02-25T11:47:14.628464+0000","last_undegraded":"2026-02-25T11:47:14.628464+0000","last_fullsized":"2026-02-25T11:47:14.628464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:42:12.028258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:19.580555+0000","last_change":"2026-02-25T11:47:12.613208+0000","last_active":"2026-02-25T11:47:19.580555+0000","last_peered":"2026-02-25T11:47:19.580555+0000","last_clean":"2026-02-25T11:47:19.580555+0000","last_became_active":"2026-02-25T11:47:12.612460+0000","last_became_peered":"2026-02-25T11:47:12.612460+0000","last_unstale":"2026-02-25T11:47:19.580555+0000","last_undegraded":"2026-02-25T11:47:19.580555+0000","last_fullsized":"2026-02-25T11:47:19.580555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:15:59.810632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":998,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628121+0000","last_change":"2026-02-25T11:47:13.631123+0000","last_active":"2026-02-25T11:47:14.628121+0000","last_peered":"2026-02-25T11:47:14.628121+0000","last_clean":"2026-02-25T11:47:14.628121+0000","last_became_active":"2026-02-25T11:47:13.631031+0000","last_became_peered":"2026-02-25T11:47:13.631031+0000","last_unstale":"2026-02-25T11:47:14.628121+0000","last_undegraded":"2026-02-25T11:47:14.628121+0000","last_fullsized":"2026-02-25T11:47:14.628121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:47:54.073226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657937+0000","last_change":"2026-02-25T11:47:12.612153+0000","last_active":"2026-02-25T11:47:17.657937+0000","last_peered":"2026-02-25T11:47:17.657937+0000","last_clean":"2026-02-25T11:47:17.657937+0000","last_became_active":"2026-02-25T11:47:12.611107+0000","last_became_peered":"2026-02-25T11:47:12.611107+0000","last_unstale":"2026-02-25T11:47:17.657937+0000","last_undegraded":"2026-02-25T11:47:17.657937+0000","last_fullsized":"2026-02-25T11:47:17.657937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:37:21.791728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749858+0000","last_change":"2026-02-25T11:47:13.635800+0000","last_active":"2026-02-25T11:47:14.749858+0000","last_peered":"2026-02-25T11:47:14.749858+0000","last_clean":"2026-02-25T11:47:14.749858+0000","last_became_active":"2026-02-25T11:47:13.635590+0000","last_became_peered":"2026-02-25T11:47:13.635590+0000","last_unstale":"2026-02-25T11:47:14.749858+0000","last_undegraded":"2026-02-25T11:47:14.749858+0000","last_fullsized":"2026-02-25T11:47:14.749858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:46:36.324518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575342+0000","last_change":"2026-02-25T11:47:12.616473+0000","last_active":"2026-02-25T11:47:18.575342+0000","last_peered":"2026-02-25T11:47:18.575342+0000","last_clean":"2026-02-25T11:47:18.575342+0000","last_became_active":"2026-02-25T11:47:12.616039+0000","last_became_peered":"2026-02-25T11:47:12.616039+0000","last_unstale":"2026-02-25T11:47:18.575342+0000","last_undegraded":"2026-02-25T11:47:18.575342+0000","last_fullsized":"2026-02-25T11:47:18.575342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:28.324889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628413+0000","last_change":"2026-02-25T11:47:13.634771+0000","last_active":"2026-02-25T11:47:14.628413+0000","last_peered":"2026-02-25T11:47:14.628413+0000","last_clean":"2026-02-25T11:47:14.628413+0000","last_became_active":"2026-02-25T11:47:13.634672+0000","last_became_peered":"2026-02-25T11:47:13.634672+0000","last_unstale":"2026-02-25T11:47:14.628413+0000","last_undegraded":"2026-02-25T11:47:14.628413+0000","last_fullsized":"2026-02-25T11:47:14.628413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:19:03.692423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749863+0000","last_change":"2026-02-25T11:47:12.616685+0000","last_active":"2026-02-25T11:47:14.749863+0000","last_peered":"2026-02-25T11:47:14.749863+0000","last_clean":"2026-02-25T11:47:14.749863+0000","last_became_active":"2026-02-25T11:47:12.616518+0000","last_became_peered":"2026-02-25T11:47:12.616518+0000","last_unstale":"2026-02-25T11:47:14.749863+0000","last_undegraded":"2026-02-25T11:47:14.749863+0000","last_fullsized":"2026-02-25T11:47:14.749863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:28:50.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628462+0000","last_change":"2026-02-25T11:47:13.627734+0000","last_active":"2026-02-25T11:47:14.628462+0000","last_peered":"2026-02-25T11:47:14.628462+0000","last_clean":"2026-02-25T11:47:14.628462+0000","last_became_active":"2026-02-25T11:47:13.627632+0000","last_became_peered":"2026-02-25T11:47:13.627632+0000","last_unstale":"2026-02-25T11:47:14.628462+0000","last_undegraded":"2026-02-25T11:47:14.628462+0000","last_fullsized":"2026-02-25T11:47:14.628462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:01:20.907699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750104+0000","last_change":"2026-02-25T11:47:12.612830+0000","last_active":"2026-02-25T11:47:14.750104+0000","last_peered":"2026-02-25T11:47:14.750104+0000","last_clean":"2026-02-25T11:47:14.750104+0000","last_became_active":"2026-02-25T11:47:12.612661+0000","last_became_peered":"2026-02-25T11:47:12.612661+0000","last_unstale":"2026-02-25T11:47:14.750104+0000","last_undegraded":"2026-02-25T11:47:14.750104+0000","last_fullsized":"2026-02-25T11:47:14.750104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:05:15.367058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628390+0000","last_change":"2026-02-25T11:47:13.633998+0000","last_active":"2026-02-25T11:47:14.628390+0000","last_peered":"2026-02-25T11:47:14.628390+0000","last_clean":"2026-02-25T11:47:14.628390+0000","last_became_active":"2026-02-25T11:47:13.633820+0000","last_became_peered":"2026-02-25T11:47:13.633820+0000","last_unstale":"2026-02-25T11:47:14.628390+0000","last_undegraded":"2026-02-25T11:47:14.628390+0000","last_fullsized":"2026-02-25T11:47:14.628390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:02:38.693106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.658417+0000","last_change":"2026-02-25T11:47:12.614091+0000","last_active":"2026-02-25T11:47:17.658417+0000","last_peered":"2026-02-25T11:47:17.658417+0000","last_clean":"2026-02-25T11:47:17.658417+0000","last_became_active":"2026-02-25T11:47:12.612962+0000","last_became_peered":"2026-02-25T11:47:12.612962+0000","last_unstale":"2026-02-25T11:47:17.658417+0000","last_undegraded":"2026-02-25T11:47:17.658417+0000","last_fullsized":"2026-02-25T11:47:17.658417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:11:40.214146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628504+0000","last_change":"2026-02-25T11:47:13.627288+0000","last_active":"2026-02-25T11:47:14.628504+0000","last_peered":"2026-02-25T11:47:14.628504+0000","last_clean":"2026-02-25T11:47:14.628504+0000","last_became_active":"2026-02-25T11:47:13.626902+0000","last_became_peered":"2026-02-25T11:47:13.626902+0000","last_unstale":"2026-02-25T11:47:14.628504+0000","last_undegraded":"2026-02-25T11:47:14.628504+0000","last_fullsized":"2026-02-25T11:47:14.628504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:38:19.137242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'152","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628095+0000","last_change":"2026-02-25T11:46:35.865282+0000","last_active":"2026-02-25T11:47:14.628095+0000","last_peered":"2026-02-25T11:47:14.628095+0000","last_clean":"2026-02-25T11:47:14.628095+0000","last_became_active":"2026-02-25T11:46:35.865139+0000","last_became_peered":"2026-02-25T11:46:35.865139+0000","last_unstale":"2026-02-25T11:47:14.628095+0000","last_undegraded":"2026-02-25T11:47:14.628095+0000","last_fullsized":"2026-02-25T11:47:14.628095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_clean_scrub_stamp":"2026-02-25T11:44:43.739673+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:29:57.573039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575170+0000","last_change":"2026-02-25T11:47:12.612834+0000","last_active":"2026-02-25T11:47:18.575170+0000","last_peered":"2026-02-25T11:47:18.575170+0000","last_clean":"2026-02-25T11:47:18.575170+0000","last_became_active":"2026-02-25T11:47:12.611827+0000","last_became_peered":"2026-02-25T11:47:12.611827+0000","last_unstale":"2026-02-25T11:47:18.575170+0000","last_undegraded":"2026-02-25T11:47:18.575170+0000","last_fullsized":"2026-02-25T11:47:18.575170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:05:32.112908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635504+0000","last_change":"2026-02-25T11:47:13.634534+0000","last_active":"2026-02-25T11:47:14.635504+0000","last_peered":"2026-02-25T11:47:14.635504+0000","last_clean":"2026-02-25T11:47:14.635504+0000","last_became_active":"2026-02-25T11:47:13.633769+0000","last_became_peered":"2026-02-25T11:47:13.633769+0000","last_unstale":"2026-02-25T11:47:14.635504+0000","last_undegraded":"2026-02-25T11:47:14.635504+0000","last_fullsized":"2026-02-25T11:47:14.635504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:16:09.255508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656541+0000","last_change":"2026-02-25T11:47:12.612621+0000","last_active":"2026-02-25T11:47:17.656541+0000","last_peered":"2026-02-25T11:47:17.656541+0000","last_clean":"2026-02-25T11:47:17.656541+0000","last_became_active":"2026-02-25T11:47:12.612284+0000","last_became_peered":"2026-02-25T11:47:12.612284+0000","last_unstale":"2026-02-25T11:47:17.656541+0000","last_undegraded":"2026-02-25T11:47:17.656541+0000","last_fullsized":"2026-02-25T11:47:17.656541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:49:09.299914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749514+0000","last_change":"2026-02-25T11:47:13.629107+0000","last_active":"2026-02-25T11:47:14.749514+0000","last_peered":"2026-02-25T11:47:14.749514+0000","last_clean":"2026-02-25T11:47:14.749514+0000","last_became_active":"2026-02-25T11:47:13.629031+0000","last_became_peered":"2026-02-25T11:47:13.629031+0000","last_unstale":"2026-02-25T11:47:14.749514+0000","last_undegraded":"2026-02-25T11:47:14.749514+0000","last_fullsized":"2026-02-25T11:47:14.749514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:07:58.915416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:20.580996+0000","last_change":"2026-02-25T11:47:12.616543+0000","last_active":"2026-02-25T11:47:20.580996+0000","last_peered":"2026-02-25T11:47:20.580996+0000","last_clean":"2026-02-25T11:47:20.580996+0000","last_became_active":"2026-02-25T11:47:12.616345+0000","last_became_peered":"2026-02-25T11:47:12.616345+0000","last_unstale":"2026-02-25T11:47:20.580996+0000","last_undegraded":"2026-02-25T11:47:20.580996+0000","last_fullsized":"2026-02-25T11:47:20.580996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:27:30.862168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750014+0000","last_change":"2026-02-25T11:47:13.637385+0000","last_active":"2026-02-25T11:47:14.750014+0000","last_peered":"2026-02-25T11:47:14.750014+0000","last_clean":"2026-02-25T11:47:14.750014+0000","last_became_active":"2026-02-25T11:47:13.634653+0000","last_became_peered":"2026-02-25T11:47:13.634653+0000","last_unstale":"2026-02-25T11:47:14.750014+0000","last_undegraded":"2026-02-25T11:47:14.750014+0000","last_fullsized":"2026-02-25T11:47:14.750014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:05:08.662208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:15.914921+0000","last_change":"2026-02-25T11:47:12.614640+0000","last_active":"2026-02-25T11:47:15.914921+0000","last_peered":"2026-02-25T11:47:15.914921+0000","last_clean":"2026-02-25T11:47:15.914921+0000","last_became_active":"2026-02-25T11:47:12.614486+0000","last_became_peered":"2026-02-25T11:47:12.614486+0000","last_unstale":"2026-02-25T11:47:15.914921+0000","last_undegraded":"2026-02-25T11:47:15.914921+0000","last_fullsized":"2026-02-25T11:47:15.914921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:16:06.314709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750327+0000","last_change":"2026-02-25T11:47:13.631828+0000","last_active":"2026-02-25T11:47:14.750327+0000","last_peered":"2026-02-25T11:47:14.750327+0000","last_clean":"2026-02-25T11:47:14.750327+0000","last_became_active":"2026-02-25T11:47:13.631590+0000","last_became_peered":"2026-02-25T11:47:13.631590+0000","last_unstale":"2026-02-25T11:47:14.750327+0000","last_undegraded":"2026-02-25T11:47:14.750327+0000","last_fullsized":"2026-02-25T11:47:14.750327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:25.518794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633984+0000","last_change":"2026-02-25T11:47:12.614032+0000","last_active":"2026-02-25T11:47:14.633984+0000","last_peered":"2026-02-25T11:47:14.633984+0000","last_clean":"2026-02-25T11:47:14.633984+0000","last_became_active":"2026-02-25T11:47:12.613936+0000","last_became_peered":"2026-02-25T11:47:12.613936+0000","last_unstale":"2026-02-25T11:47:14.633984+0000","last_undegraded":"2026-02-25T11:47:14.633984+0000","last_fullsized":"2026-02-25T11:47:14.633984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:16:46.397485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.859010+0000","last_change":"2026-02-25T11:47:13.636173+0000","last_active":"2026-02-25T11:47:16.859010+0000","last_peered":"2026-02-25T11:47:16.859010+0000","last_clean":"2026-02-25T11:47:16.859010+0000","last_became_active":"2026-02-25T11:47:13.635745+0000","last_became_peered":"2026-02-25T11:47:13.635745+0000","last_unstale":"2026-02-25T11:47:16.859010+0000","last_undegraded":"2026-02-25T11:47:16.859010+0000","last_fullsized":"2026-02-25T11:47:16.859010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:03:33.671318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651871+0000","last_change":"2026-02-25T11:47:12.614003+0000","last_active":"2026-02-25T11:47:16.651871+0000","last_peered":"2026-02-25T11:47:16.651871+0000","last_clean":"2026-02-25T11:47:16.651871+0000","last_became_active":"2026-02-25T11:47:12.613893+0000","last_became_peered":"2026-02-25T11:47:12.613893+0000","last_unstale":"2026-02-25T11:47:16.651871+0000","last_undegraded":"2026-02-25T11:47:16.651871+0000","last_fullsized":"2026-02-25T11:47:16.651871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:33:21.133364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749746+0000","last_change":"2026-02-25T11:47:13.632632+0000","last_active":"2026-02-25T11:47:14.749746+0000","last_peered":"2026-02-25T11:47:14.749746+0000","last_clean":"2026-02-25T11:47:14.749746+0000","last_became_active":"2026-02-25T11:47:13.632408+0000","last_became_peered":"2026-02-25T11:47:13.632408+0000","last_unstale":"2026-02-25T11:47:14.749746+0000","last_undegraded":"2026-02-25T11:47:14.749746+0000","last_fullsized":"2026-02-25T11:47:14.749746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:38:45.448637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657145+0000","last_change":"2026-02-25T11:47:12.613816+0000","last_active":"2026-02-25T11:47:17.657145+0000","last_peered":"2026-02-25T11:47:17.657145+0000","last_clean":"2026-02-25T11:47:17.657145+0000","last_became_active":"2026-02-25T11:47:12.613196+0000","last_became_peered":"2026-02-25T11:47:12.613196+0000","last_unstale":"2026-02-25T11:47:17.657145+0000","last_undegraded":"2026-02-25T11:47:17.657145+0000","last_fullsized":"2026-02-25T11:47:17.657145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:23:52.372808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635752+0000","last_change":"2026-02-25T11:47:13.632431+0000","last_active":"2026-02-25T11:47:14.635752+0000","last_peered":"2026-02-25T11:47:14.635752+0000","last_clean":"2026-02-25T11:47:14.635752+0000","last_became_active":"2026-02-25T11:47:13.631972+0000","last_became_peered":"2026-02-25T11:47:13.631972+0000","last_unstale":"2026-02-25T11:47:14.635752+0000","last_undegraded":"2026-02-25T11:47:14.635752+0000","last_fullsized":"2026-02-25T11:47:14.635752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:59:34.655028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651131+0000","last_change":"2026-02-25T11:47:12.612805+0000","last_active":"2026-02-25T11:47:16.651131+0000","last_peered":"2026-02-25T11:47:16.651131+0000","last_clean":"2026-02-25T11:47:16.651131+0000","last_became_active":"2026-02-25T11:47:12.612155+0000","last_became_peered":"2026-02-25T11:47:12.612155+0000","last_unstale":"2026-02-25T11:47:16.651131+0000","last_undegraded":"2026-02-25T11:47:16.651131+0000","last_fullsized":"2026-02-25T11:47:16.651131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:58:00.429639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627853+0000","last_change":"2026-02-25T11:47:13.636089+0000","last_active":"2026-02-25T11:47:14.627853+0000","last_peered":"2026-02-25T11:47:14.627853+0000","last_clean":"2026-02-25T11:47:14.627853+0000","last_became_active":"2026-02-25T11:47:13.635963+0000","last_became_peered":"2026-02-25T11:47:13.635963+0000","last_unstale":"2026-02-25T11:47:14.627853+0000","last_undegraded":"2026-02-25T11:47:14.627853+0000","last_fullsized":"2026-02-25T11:47:14.627853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:50:28.012023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650081+0000","last_change":"2026-02-25T11:47:12.609749+0000","last_active":"2026-02-25T11:47:16.650081+0000","last_peered":"2026-02-25T11:47:16.650081+0000","last_clean":"2026-02-25T11:47:16.650081+0000","last_became_active":"2026-02-25T11:47:12.609190+0000","last_became_peered":"2026-02-25T11:47:12.609190+0000","last_unstale":"2026-02-25T11:47:16.650081+0000","last_undegraded":"2026-02-25T11:47:16.650081+0000","last_fullsized":"2026-02-25T11:47:16.650081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:12:42.204122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628201+0000","last_change":"2026-02-25T11:47:13.629339+0000","last_active":"2026-02-25T11:47:14.628201+0000","last_peered":"2026-02-25T11:47:14.628201+0000","last_clean":"2026-02-25T11:47:14.628201+0000","last_became_active":"2026-02-25T11:47:13.629107+0000","last_became_peered":"2026-02-25T11:47:13.629107+0000","last_unstale":"2026-02-25T11:47:14.628201+0000","last_undegraded":"2026-02-25T11:47:14.628201+0000","last_fullsized":"2026-02-25T11:47:14.628201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:19:50.087771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.652673+0000","last_change":"2026-02-25T11:47:12.615249+0000","last_active":"2026-02-25T11:47:16.652673+0000","last_peered":"2026-02-25T11:47:16.652673+0000","last_clean":"2026-02-25T11:47:16.652673+0000","last_became_active":"2026-02-25T11:47:12.615165+0000","last_became_peered":"2026-02-25T11:47:12.615165+0000","last_unstale":"2026-02-25T11:47:16.652673+0000","last_undegraded":"2026-02-25T11:47:16.652673+0000","last_fullsized":"2026-02-25T11:47:16.652673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:40:52.452074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627665+0000","last_change":"2026-02-25T11:47:13.631460+0000","last_active":"2026-02-25T11:47:14.627665+0000","last_peered":"2026-02-25T11:47:14.627665+0000","last_clean":"2026-02-25T11:47:14.627665+0000","last_became_active":"2026-02-25T11:47:13.631298+0000","last_became_peered":"2026-02-25T11:47:13.631298+0000","last_unstale":"2026-02-25T11:47:14.627665+0000","last_undegraded":"2026-02-25T11:47:14.627665+0000","last_fullsized":"2026-02-25T11:47:14.627665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:26:03.362106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628432+0000","last_change":"2026-02-25T11:47:12.615779+0000","last_active":"2026-02-25T11:47:14.628432+0000","last_peered":"2026-02-25T11:47:14.628432+0000","last_clean":"2026-02-25T11:47:14.628432+0000","last_became_active":"2026-02-25T11:47:12.615073+0000","last_became_peered":"2026-02-25T11:47:12.615073+0000","last_unstale":"2026-02-25T11:47:14.628432+0000","last_undegraded":"2026-02-25T11:47:14.628432+0000","last_fullsized":"2026-02-25T11:47:14.628432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:28:51.468199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635380+0000","last_change":"2026-02-25T11:47:13.632351+0000","last_active":"2026-02-25T11:47:14.635380+0000","last_peered":"2026-02-25T11:47:14.635380+0000","last_clean":"2026-02-25T11:47:14.635380+0000","last_became_active":"2026-02-25T11:47:13.632081+0000","last_became_peered":"2026-02-25T11:47:13.632081+0000","last_unstale":"2026-02-25T11:47:14.635380+0000","last_undegraded":"2026-02-25T11:47:14.635380+0000","last_fullsized":"2026-02-25T11:47:14.635380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:38:59.143253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650353+0000","last_change":"2026-02-25T11:47:12.616555+0000","last_active":"2026-02-25T11:47:16.650353+0000","last_peered":"2026-02-25T11:47:16.650353+0000","last_clean":"2026-02-25T11:47:16.650353+0000","last_became_active":"2026-02-25T11:47:12.616174+0000","last_became_peered":"2026-02-25T11:47:12.616174+0000","last_unstale":"2026-02-25T11:47:16.650353+0000","last_undegraded":"2026-02-25T11:47:16.650353+0000","last_fullsized":"2026-02-25T11:47:16.650353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:01:17.494129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628458+0000","last_change":"2026-02-25T11:47:13.627400+0000","last_active":"2026-02-25T11:47:14.628458+0000","last_peered":"2026-02-25T11:47:14.628458+0000","last_clean":"2026-02-25T11:47:14.628458+0000","last_became_active":"2026-02-25T11:47:13.626782+0000","last_became_peered":"2026-02-25T11:47:13.626782+0000","last_unstale":"2026-02-25T11:47:14.628458+0000","last_undegraded":"2026-02-25T11:47:14.628458+0000","last_fullsized":"2026-02-25T11:47:14.628458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:42:39.996935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750027+0000","last_change":"2026-02-25T11:47:13.634030+0000","last_active":"2026-02-25T11:47:14.750027+0000","last_peered":"2026-02-25T11:47:14.750027+0000","last_clean":"2026-02-25T11:47:14.750027+0000","last_became_active":"2026-02-25T11:47:13.632844+0000","last_became_peered":"2026-02-25T11:47:13.632844+0000","last_unstale":"2026-02-25T11:47:14.750027+0000","last_undegraded":"2026-02-25T11:47:14.750027+0000","last_fullsized":"2026-02-25T11:47:14.750027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:52:16.013596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628805+0000","last_change":"2026-02-25T11:47:12.613666+0000","last_active":"2026-02-25T11:47:14.628805+0000","last_peered":"2026-02-25T11:47:14.628805+0000","last_clean":"2026-02-25T11:47:14.628805+0000","last_became_active":"2026-02-25T11:47:12.613570+0000","last_became_peered":"2026-02-25T11:47:12.613570+0000","last_unstale":"2026-02-25T11:47:14.628805+0000","last_undegraded":"2026-02-25T11:47:14.628805+0000","last_fullsized":"2026-02-25T11:47:14.628805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:28:39.750737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750924+0000","last_change":"2026-02-25T11:47:13.631515+0000","last_active":"2026-02-25T11:47:14.750924+0000","last_peered":"2026-02-25T11:47:14.750924+0000","last_clean":"2026-02-25T11:47:14.750924+0000","last_became_active":"2026-02-25T11:47:13.631161+0000","last_became_peered":"2026-02-25T11:47:13.631161+0000","last_unstale":"2026-02-25T11:47:14.750924+0000","last_undegraded":"2026-02-25T11:47:14.750924+0000","last_fullsized":"2026-02-25T11:47:14.750924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:13:30.496521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656891+0000","last_change":"2026-02-25T11:47:12.615904+0000","last_active":"2026-02-25T11:47:17.656891+0000","last_peered":"2026-02-25T11:47:17.656891+0000","last_clean":"2026-02-25T11:47:17.656891+0000","last_became_active":"2026-02-25T11:47:12.615784+0000","last_became_peered":"2026-02-25T11:47:12.615784+0000","last_unstale":"2026-02-25T11:47:17.656891+0000","last_undegraded":"2026-02-25T11:47:17.656891+0000","last_fullsized":"2026-02-25T11:47:17.656891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:32:26.802870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627754+0000","last_change":"2026-02-25T11:47:13.630065+0000","last_active":"2026-02-25T11:47:14.627754+0000","last_peered":"2026-02-25T11:47:14.627754+0000","last_clean":"2026-02-25T11:47:14.627754+0000","last_became_active":"2026-02-25T11:47:13.629214+0000","last_became_peered":"2026-02-25T11:47:13.629214+0000","last_unstale":"2026-02-25T11:47:14.627754+0000","last_undegraded":"2026-02-25T11:47:14.627754+0000","last_fullsized":"2026-02-25T11:47:14.627754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:53:45.639526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.310632+0000","last_change":"2026-02-25T11:47:12.612956+0000","last_active":"2026-02-25T11:47:16.310632+0000","last_peered":"2026-02-25T11:47:16.310632+0000","last_clean":"2026-02-25T11:47:16.310632+0000","last_became_active":"2026-02-25T11:47:12.612843+0000","last_became_peered":"2026-02-25T11:47:12.612843+0000","last_unstale":"2026-02-25T11:47:16.310632+0000","last_undegraded":"2026-02-25T11:47:16.310632+0000","last_fullsized":"2026-02-25T11:47:16.310632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:41:30.867465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635780+0000","last_change":"2026-02-25T11:47:13.637712+0000","last_active":"2026-02-25T11:47:14.635780+0000","last_peered":"2026-02-25T11:47:14.635780+0000","last_clean":"2026-02-25T11:47:14.635780+0000","last_became_active":"2026-02-25T11:47:13.635030+0000","last_became_peered":"2026-02-25T11:47:13.635030+0000","last_unstale":"2026-02-25T11:47:14.635780+0000","last_undegraded":"2026-02-25T11:47:14.635780+0000","last_fullsized":"2026-02-25T11:47:14.635780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:01:22.896129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656680+0000","last_change":"2026-02-25T11:47:12.614149+0000","last_active":"2026-02-25T11:47:17.656680+0000","last_peered":"2026-02-25T11:47:17.656680+0000","last_clean":"2026-02-25T11:47:17.656680+0000","last_became_active":"2026-02-25T11:47:12.613137+0000","last_became_peered":"2026-02-25T11:47:12.613137+0000","last_unstale":"2026-02-25T11:47:17.656680+0000","last_undegraded":"2026-02-25T11:47:17.656680+0000","last_fullsized":"2026-02-25T11:47:17.656680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:49:49.451435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53093,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":42,"num_read_kb":79,"num_write":139,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":123366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":221240,"internal_metadata":0},"log_size":123,"ondisk_log_size":123,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":262884,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350985,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28132,"kb_used_data":912,"kb_used_omap":9,"kb_used_meta":27190,"kb_avail":733975068,"statfs":{"total":751619276800,"available":751590469632,"internally_reserved":0,"allocated":933888,"data_stored":455512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10109,"internal_metadata":27842691},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612619,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28268,"kb_used_data":1048,"kb_used_omap":28,"kb_used_meta":27171,"kb_avail":733974932,"statfs":{"total":751619276800,"available":751590330368,"internally_reserved":0,"allocated":1073152,"data_stored":1096410,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":29123,"internal_metadata":27823677},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776141,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28144,"kb_used_data":924,"kb_used_omap":24,"kb_used_meta":27175,"kb_avail":733975056,"statfs":{"total":751619276800,"available":751590457344,"internally_reserved":0,"allocated":946176,"data_stored":455714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24746,"internal_metadata":27828054},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037776,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28148,"kb_used_data":928,"kb_used_omap":33,"kb_used_meta":27166,"kb_avail":733975052,"statfs":{"total":751619276800,"available":751590453248,"internally_reserved":0,"allocated":950272,"data_stored":455668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":34197,"internal_metadata":27818603},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299411,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28192,"kb_used_data":972,"kb_used_omap":35,"kb_used_meta":27164,"kb_avail":733975008,"statfs":{"total":751619276800,"available":751590408192,"internally_reserved":0,"allocated":995328,"data_stored":1047500,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36168,"internal_metadata":27816632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462935,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28136,"kb_used_data":916,"kb_used_omap":31,"kb_used_meta":27168,"kb_avail":733975064,"statfs":{"total":751619276800,"available":751590465536,"internally_reserved":0,"allocated":937984,"data_stored":455115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32619,"internal_metadata":27820181},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724568,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28204,"kb_used_data":984,"kb_used_omap":36,"kb_used_meta":27163,"kb_avail":733974996,"statfs":{"total":751619276800,"available":751590395904,"internally_reserved":0,"allocated":1007616,"data_stored":1046774,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36952,"internal_metadata":27815848},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888091,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28140,"kb_used_data":920,"kb_used_omap":45,"kb_used_meta":27154,"kb_avail":733975060,"statfs":{"total":751619276800,"available":751590461440,"internally_reserved":0,"allocated":942080,"data_stored":455962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":46332,"internal_metadata":27806468},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149726,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28152,"kb_used_data":932,"kb_used_omap":50,"kb_used_meta":27149,"kb_avail":733975048,"statfs":{"total":751619276800,"available":751590449152,"internally_reserved":0,"allocated":954368,"data_stored":455426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51675,"internal_metadata":27801125},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378657,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28212,"kb_used_data":992,"kb_used_omap":64,"kb_used_meta":27135,"kb_avail":733974988,"statfs":{"total":751619276800,"available":751590387712,"internally_reserved":0,"allocated":1015808,"data_stored":509996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":66234,"internal_metadata":27786566},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28140,"kb_used_data":920,"kb_used_omap":43,"kb_used_meta":27156,"kb_avail":733975060,"statfs":{"total":751619276800,"available":751590461440,"internally_reserved":0,"allocated":942080,"data_stored":456542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44488,"internal_metadata":27808312},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28156,"kb_used_data":936,"kb_used_omap":54,"kb_used_meta":27145,"kb_avail":733975044,"statfs":{"total":751619276800,"available":751590445056,"internally_reserved":0,"allocated":958464,"data_stored":456430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56100,"internal_metadata":27796700},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8853,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1407,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":55088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56835,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48464,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6449,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3478,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14040,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9858,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24153,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19148,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":51017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21693,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6862,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T11:47:23.583 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph pg dump --format=json 2026-02-25T11:47:23.768 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:23.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:23 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/373154967' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:23.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:23 trial028 ceph-mon[39224]: pgmap v199: 97 pgs: 97 active+clean; 628 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 8.4 KiB/s rd, 10 KiB/s wr, 19 op/s 2026-02-25T11:47:23.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.937+0000 7fa3530a7640 1 -- 10.20.193.28:0/2345191636 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa34c1135c0 msgr2=0x7fa34c173f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:23.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.937+0000 7fa3530a7640 1 --2- 10.20.193.28:0/2345191636 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa34c1135c0 0x7fa34c173f90 secure :-1 s=READY pgs=252 gs=1 cs=0 l=1 c_cookie=16586098018698244399 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa34800aa30 tx=0x7fa348030270 comp rx=0 tx=0).stop 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.937+0000 7fa3530a7640 1 -- 10.20.193.28:0/2345191636 shutdown_connections 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.937+0000 7fa3530a7640 1 --2- 10.20.193.28:0/2345191636 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa34c1135c0 0x7fa34c173f90 unknown :-1 s=CLOSED pgs=252 gs=1 cs=0 l=1 c_cookie=16586098018698244399 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.937+0000 7fa3530a7640 1 --2- 10.20.193.28:0/2345191636 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa34c10ddc0 0x7fa34c112e10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.937+0000 7fa3530a7640 1 --2- 10.20.193.28:0/2345191636 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa34c10d3d0 0x7fa34c10d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.937+0000 7fa3530a7640 1 -- 10.20.193.28:0/2345191636 >> 10.20.193.28:0/2345191636 conn(0x7fa34c0eaca0 msgr2=0x7fa34c10b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.937+0000 7fa3530a7640 1 -- 10.20.193.28:0/2345191636 shutdown_connections 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.937+0000 7fa3530a7640 1 -- 10.20.193.28:0/2345191636 wait complete. 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3530a7640 1 Processor -- start 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3530a7640 1 -- start start 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3530a7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa34c10d3d0 0x7fa34c0f5a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3530a7640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa34c10ddc0 0x7fa34c173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3530a7640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa34c1135c0 0x7fa34c0f6110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3530a7640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c0fb2d0 con 0x7fa34c10d3d0 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3530a7640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c0fb440 con 0x7fa34c10ddc0 2026-02-25T11:47:23.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3530a7640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa34c0fb600 con 0x7fa34c1135c0 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3520a5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa34c10d3d0 0x7fa34c0f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3520a5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa34c10d3d0 0x7fa34c0f5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34156/0 (socket says 10.20.193.28:34156) 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3520a5640 1 -- 10.20.193.28:0/2470470727 learned_addr learned my addr 10.20.193.28:0/2470470727 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3518a4640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa34c10ddc0 0x7fa34c173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3528a6640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa34c1135c0 0x7fa34c0f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3520a5640 1 -- 10.20.193.28:0/2470470727 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa34c1135c0 msgr2=0x7fa34c0f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3520a5640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa34c1135c0 0x7fa34c0f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3520a5640 1 -- 10.20.193.28:0/2470470727 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa34c10ddc0 msgr2=0x7fa34c173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3520a5640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa34c10ddc0 0x7fa34c173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.938+0000 7fa3520a5640 1 -- 10.20.193.28:0/2470470727 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa34800a650 con 0x7fa34c10d3d0 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.939+0000 7fa3520a5640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa34c10d3d0 0x7fa34c0f5a10 secure :-1 s=READY pgs=253 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa33c009480 tx=0x7fa33c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.939+0000 7fa3337fe640 1 -- 10.20.193.28:0/2470470727 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa33c00cd40 con 0x7fa34c10d3d0 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.939+0000 7fa3337fe640 1 -- 10.20.193.28:0/2470470727 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa33c004d10 con 0x7fa34c10d3d0 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.939+0000 7fa3530a7640 1 -- 10.20.193.28:0/2470470727 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa34c0fb8e0 con 0x7fa34c10d3d0 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.939+0000 7fa3337fe640 1 -- 10.20.193.28:0/2470470727 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa33c0106b0 con 0x7fa34c10d3d0 2026-02-25T11:47:23.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.939+0000 7fa3530a7640 1 -- 10.20.193.28:0/2470470727 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa34c0fbe00 con 0x7fa34c10d3d0 2026-02-25T11:47:23.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.940+0000 7fa3530a7640 1 -- 10.20.193.28:0/2470470727 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa314005300 con 0x7fa34c10d3d0 2026-02-25T11:47:23.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.941+0000 7fa3337fe640 1 -- 10.20.193.28:0/2470470727 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fa33c01f060 con 0x7fa34c10d3d0 2026-02-25T11:47:23.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.943+0000 7fa3337fe640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa3280b57e0 0x7fa3280d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:23.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.943+0000 7fa3337fe640 1 -- 10.20.193.28:0/2470470727 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fa33c014070 con 0x7fa34c10d3d0 2026-02-25T11:47:23.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.943+0000 7fa3337fe640 1 -- 10.20.193.28:0/2470470727 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fa33c0e71a0 con 0x7fa34c10d3d0 2026-02-25T11:47:23.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.943+0000 7fa3518a4640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa3280b57e0 0x7fa3280d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:23.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:23.944+0000 7fa3518a4640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa3280b57e0 0x7fa3280d5c90 secure :-1 s=READY pgs=186 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa34c0fc360 tx=0x7fa34000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:23.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:23 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/373154967' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:23.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:23 trial167 ceph-mon[43392]: pgmap v199: 97 pgs: 97 active+clean; 628 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 8.4 KiB/s rd, 10 KiB/s wr, 19 op/s 2026-02-25T11:47:24.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:23 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/373154967' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:24.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:23 trial119 ceph-mon[42710]: pgmap v199: 97 pgs: 97 active+clean; 628 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 8.4 KiB/s rd, 10 KiB/s wr, 19 op/s 2026-02-25T11:47:24.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.060+0000 7fa3530a7640 1 -- 10.20.193.28:0/2470470727 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa314002bb0 con 0x7fa3280b57e0 2026-02-25T11:47:24.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.063+0000 7fa3337fe640 1 -- 10.20.193.28:0/2470470727 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258942 (secure 0 0 0) 0x7fa314002bb0 con 0x7fa3280b57e0 2026-02-25T11:47:24.066 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:24.066 INFO:teuthology.orchestra.run.trial028.stderr:dumped all 2026-02-25T11:47:24.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.066+0000 7fa3530a7640 1 -- 10.20.193.28:0/2470470727 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa3280b57e0 msgr2=0x7fa3280d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:24.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.066+0000 7fa3530a7640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa3280b57e0 0x7fa3280d5c90 secure :-1 s=READY pgs=186 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa34c0fc360 tx=0x7fa34000b410 comp rx=0 tx=0).stop 2026-02-25T11:47:24.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.066+0000 7fa3530a7640 1 -- 10.20.193.28:0/2470470727 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa34c10d3d0 msgr2=0x7fa34c0f5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:24.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.066+0000 7fa3530a7640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa34c10d3d0 0x7fa34c0f5a10 secure :-1 s=READY pgs=253 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa33c009480 tx=0x7fa33c00ee60 comp rx=0 tx=0).stop 2026-02-25T11:47:24.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.066+0000 7fa3530a7640 1 -- 10.20.193.28:0/2470470727 shutdown_connections 2026-02-25T11:47:24.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.066+0000 7fa3530a7640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa3280b57e0 0x7fa3280d5c90 unknown :-1 s=CLOSED pgs=186 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:24.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.066+0000 7fa3530a7640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa34c1135c0 0x7fa34c0f6110 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:24.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.066+0000 7fa3530a7640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa34c10ddc0 0x7fa34c173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:24.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.066+0000 7fa3530a7640 1 --2- 10.20.193.28:0/2470470727 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa34c10d3d0 0x7fa34c0f5a10 unknown :-1 s=CLOSED pgs=253 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:24.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.066+0000 7fa3530a7640 1 -- 10.20.193.28:0/2470470727 >> 10.20.193.28:0/2470470727 conn(0x7fa34c0eaca0 msgr2=0x7fa34c10b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:24.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.066+0000 7fa3530a7640 1 -- 10.20.193.28:0/2470470727 shutdown_connections 2026-02-25T11:47:24.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.066+0000 7fa3530a7640 1 -- 10.20.193.28:0/2470470727 wait complete. 2026-02-25T11:47:24.165 INFO:teuthology.orchestra.run.trial028.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2026-02-25T11:47:23.071120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":643578,"num_objects":63,"num_object_clones":0,"num_object_copies":189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":0,"num_read":148,"num_read_kb":170,"num_write":329,"num_write_kb":3813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":8808038400,"kb_used":338024,"kb_used_data":11384,"kb_used_omap":457,"kb_used_meta":325942,"kb_avail":8807700376,"statfs":{"total":9019431321600,"available":9019085185024,"internally_reserved":0,"allocated":11657216,"data_stored":7347049,"data_compressed":114024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":468743,"internal_metadata":333764857},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53210,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":42,"num_read_kb":79,"num_write":140,"num_write_kb":97,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.453894"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627807+0000","last_change":"2026-02-25T11:47:13.630492+0000","last_active":"2026-02-25T11:47:14.627807+0000","last_peered":"2026-02-25T11:47:14.627807+0000","last_clean":"2026-02-25T11:47:14.627807+0000","last_became_active":"2026-02-25T11:47:13.630388+0000","last_became_peered":"2026-02-25T11:47:13.630388+0000","last_unstale":"2026-02-25T11:47:14.627807+0000","last_undegraded":"2026-02-25T11:47:14.627807+0000","last_fullsized":"2026-02-25T11:47:14.627807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:01:17.024058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627657+0000","last_change":"2026-02-25T11:47:13.629507+0000","last_active":"2026-02-25T11:47:14.627657+0000","last_peered":"2026-02-25T11:47:14.627657+0000","last_clean":"2026-02-25T11:47:14.627657+0000","last_became_active":"2026-02-25T11:47:13.628937+0000","last_became_peered":"2026-02-25T11:47:13.628937+0000","last_unstale":"2026-02-25T11:47:14.627657+0000","last_undegraded":"2026-02-25T11:47:14.627657+0000","last_fullsized":"2026-02-25T11:47:14.627657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:47:20.441522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627713+0000","last_change":"2026-02-25T11:47:13.630415+0000","last_active":"2026-02-25T11:47:14.627713+0000","last_peered":"2026-02-25T11:47:14.627713+0000","last_clean":"2026-02-25T11:47:14.627713+0000","last_became_active":"2026-02-25T11:47:13.630319+0000","last_became_peered":"2026-02-25T11:47:13.630319+0000","last_unstale":"2026-02-25T11:47:14.627713+0000","last_undegraded":"2026-02-25T11:47:14.627713+0000","last_fullsized":"2026-02-25T11:47:14.627713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:18:57.675760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627800+0000","last_change":"2026-02-25T11:47:13.633017+0000","last_active":"2026-02-25T11:47:14.627800+0000","last_peered":"2026-02-25T11:47:14.627800+0000","last_clean":"2026-02-25T11:47:14.627800+0000","last_became_active":"2026-02-25T11:47:13.632934+0000","last_became_peered":"2026-02-25T11:47:13.632934+0000","last_unstale":"2026-02-25T11:47:14.627800+0000","last_undegraded":"2026-02-25T11:47:14.627800+0000","last_fullsized":"2026-02-25T11:47:14.627800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:08:54.195980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628667+0000","last_change":"2026-02-25T11:47:13.638057+0000","last_active":"2026-02-25T11:47:14.628667+0000","last_peered":"2026-02-25T11:47:14.628667+0000","last_clean":"2026-02-25T11:47:14.628667+0000","last_became_active":"2026-02-25T11:47:13.636387+0000","last_became_peered":"2026-02-25T11:47:13.636387+0000","last_unstale":"2026-02-25T11:47:14.628667+0000","last_undegraded":"2026-02-25T11:47:14.628667+0000","last_fullsized":"2026-02-25T11:47:14.628667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:34:29.570622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750630+0000","last_change":"2026-02-25T11:47:13.631824+0000","last_active":"2026-02-25T11:47:14.750630+0000","last_peered":"2026-02-25T11:47:14.750630+0000","last_clean":"2026-02-25T11:47:14.750630+0000","last_became_active":"2026-02-25T11:47:13.631656+0000","last_became_peered":"2026-02-25T11:47:13.631656+0000","last_unstale":"2026-02-25T11:47:14.750630+0000","last_undegraded":"2026-02-25T11:47:14.750630+0000","last_fullsized":"2026-02-25T11:47:14.750630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:15:05.237015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628186+0000","last_change":"2026-02-25T11:47:13.626877+0000","last_active":"2026-02-25T11:47:14.628186+0000","last_peered":"2026-02-25T11:47:14.628186+0000","last_clean":"2026-02-25T11:47:14.628186+0000","last_became_active":"2026-02-25T11:47:13.626543+0000","last_became_peered":"2026-02-25T11:47:13.626543+0000","last_unstale":"2026-02-25T11:47:14.628186+0000","last_undegraded":"2026-02-25T11:47:14.628186+0000","last_fullsized":"2026-02-25T11:47:14.628186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:56:58.194911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635224+0000","last_change":"2026-02-25T11:47:13.637661+0000","last_active":"2026-02-25T11:47:14.635224+0000","last_peered":"2026-02-25T11:47:14.635224+0000","last_clean":"2026-02-25T11:47:14.635224+0000","last_became_active":"2026-02-25T11:47:13.637230+0000","last_became_peered":"2026-02-25T11:47:13.637230+0000","last_unstale":"2026-02-25T11:47:14.635224+0000","last_undegraded":"2026-02-25T11:47:14.635224+0000","last_fullsized":"2026-02-25T11:47:14.635224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:44:52.310360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751287+0000","last_change":"2026-02-25T11:47:13.625160+0000","last_active":"2026-02-25T11:47:14.751287+0000","last_peered":"2026-02-25T11:47:14.751287+0000","last_clean":"2026-02-25T11:47:14.751287+0000","last_became_active":"2026-02-25T11:47:13.625020+0000","last_became_peered":"2026-02-25T11:47:13.625020+0000","last_unstale":"2026-02-25T11:47:14.751287+0000","last_undegraded":"2026-02-25T11:47:14.751287+0000","last_fullsized":"2026-02-25T11:47:14.751287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:35:44.509551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750832+0000","last_change":"2026-02-25T11:47:13.637252+0000","last_active":"2026-02-25T11:47:14.750832+0000","last_peered":"2026-02-25T11:47:14.750832+0000","last_clean":"2026-02-25T11:47:14.750832+0000","last_became_active":"2026-02-25T11:47:13.636670+0000","last_became_peered":"2026-02-25T11:47:13.636670+0000","last_unstale":"2026-02-25T11:47:14.750832+0000","last_undegraded":"2026-02-25T11:47:14.750832+0000","last_fullsized":"2026-02-25T11:47:14.750832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:01:53.119101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628443+0000","last_change":"2026-02-25T11:47:13.628252+0000","last_active":"2026-02-25T11:47:14.628443+0000","last_peered":"2026-02-25T11:47:14.628443+0000","last_clean":"2026-02-25T11:47:14.628443+0000","last_became_active":"2026-02-25T11:47:13.628183+0000","last_became_peered":"2026-02-25T11:47:13.628183+0000","last_unstale":"2026-02-25T11:47:14.628443+0000","last_undegraded":"2026-02-25T11:47:14.628443+0000","last_fullsized":"2026-02-25T11:47:14.628443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:12:00.576747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.632358+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.631487+0000","last_became_peered":"2026-02-25T11:47:13.631487+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:41:17.150851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749735+0000","last_change":"2026-02-25T11:47:13.629675+0000","last_active":"2026-02-25T11:47:14.749735+0000","last_peered":"2026-02-25T11:47:14.749735+0000","last_clean":"2026-02-25T11:47:14.749735+0000","last_became_active":"2026-02-25T11:47:13.628686+0000","last_became_peered":"2026-02-25T11:47:13.628686+0000","last_unstale":"2026-02-25T11:47:14.749735+0000","last_undegraded":"2026-02-25T11:47:14.749735+0000","last_fullsized":"2026-02-25T11:47:14.749735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:00:54.430287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627615+0000","last_change":"2026-02-25T11:47:13.628842+0000","last_active":"2026-02-25T11:47:14.627615+0000","last_peered":"2026-02-25T11:47:14.627615+0000","last_clean":"2026-02-25T11:47:14.627615+0000","last_became_active":"2026-02-25T11:47:13.628695+0000","last_became_peered":"2026-02-25T11:47:13.628695+0000","last_unstale":"2026-02-25T11:47:14.627615+0000","last_undegraded":"2026-02-25T11:47:14.627615+0000","last_fullsized":"2026-02-25T11:47:14.627615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:10:05.404740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.635652+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.635505+0000","last_became_peered":"2026-02-25T11:47:13.635505+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:46:03.598577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749928+0000","last_change":"2026-02-25T11:47:13.628651+0000","last_active":"2026-02-25T11:47:14.749928+0000","last_peered":"2026-02-25T11:47:14.749928+0000","last_clean":"2026-02-25T11:47:14.749928+0000","last_became_active":"2026-02-25T11:47:13.628280+0000","last_became_peered":"2026-02-25T11:47:13.628280+0000","last_unstale":"2026-02-25T11:47:14.749928+0000","last_undegraded":"2026-02-25T11:47:14.749928+0000","last_fullsized":"2026-02-25T11:47:14.749928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:18:56.238078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628351+0000","last_change":"2026-02-25T11:47:13.630656+0000","last_active":"2026-02-25T11:47:14.628351+0000","last_peered":"2026-02-25T11:47:14.628351+0000","last_clean":"2026-02-25T11:47:14.628351+0000","last_became_active":"2026-02-25T11:47:13.630513+0000","last_became_peered":"2026-02-25T11:47:13.630513+0000","last_unstale":"2026-02-25T11:47:14.628351+0000","last_undegraded":"2026-02-25T11:47:14.628351+0000","last_fullsized":"2026-02-25T11:47:14.628351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:03:33.580576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.634036+0000","last_change":"2026-02-25T11:47:13.629202+0000","last_active":"2026-02-25T11:47:14.634036+0000","last_peered":"2026-02-25T11:47:14.634036+0000","last_clean":"2026-02-25T11:47:14.634036+0000","last_became_active":"2026-02-25T11:47:13.628860+0000","last_became_peered":"2026-02-25T11:47:13.628860+0000","last_unstale":"2026-02-25T11:47:14.634036+0000","last_undegraded":"2026-02-25T11:47:14.634036+0000","last_fullsized":"2026-02-25T11:47:14.634036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:47:14.635796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628235+0000","last_change":"2026-02-25T11:47:13.638872+0000","last_active":"2026-02-25T11:47:14.628235+0000","last_peered":"2026-02-25T11:47:14.628235+0000","last_clean":"2026-02-25T11:47:14.628235+0000","last_became_active":"2026-02-25T11:47:13.638443+0000","last_became_peered":"2026-02-25T11:47:13.638443+0000","last_unstale":"2026-02-25T11:47:14.628235+0000","last_undegraded":"2026-02-25T11:47:14.628235+0000","last_fullsized":"2026-02-25T11:47:14.628235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:51.699838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628595+0000","last_change":"2026-02-25T11:47:13.635894+0000","last_active":"2026-02-25T11:47:14.628595+0000","last_peered":"2026-02-25T11:47:14.628595+0000","last_clean":"2026-02-25T11:47:14.628595+0000","last_became_active":"2026-02-25T11:47:13.635725+0000","last_became_peered":"2026-02-25T11:47:13.635725+0000","last_unstale":"2026-02-25T11:47:14.628595+0000","last_undegraded":"2026-02-25T11:47:14.628595+0000","last_fullsized":"2026-02-25T11:47:14.628595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:28:20.134454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749794+0000","last_change":"2026-02-25T11:47:13.628117+0000","last_active":"2026-02-25T11:47:14.749794+0000","last_peered":"2026-02-25T11:47:14.749794+0000","last_clean":"2026-02-25T11:47:14.749794+0000","last_became_active":"2026-02-25T11:47:13.628037+0000","last_became_peered":"2026-02-25T11:47:13.628037+0000","last_unstale":"2026-02-25T11:47:14.749794+0000","last_undegraded":"2026-02-25T11:47:14.749794+0000","last_fullsized":"2026-02-25T11:47:14.749794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:28:22.976133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750830+0000","last_change":"2026-02-25T11:47:13.634652+0000","last_active":"2026-02-25T11:47:14.750830+0000","last_peered":"2026-02-25T11:47:14.750830+0000","last_clean":"2026-02-25T11:47:14.750830+0000","last_became_active":"2026-02-25T11:47:13.634433+0000","last_became_peered":"2026-02-25T11:47:13.634433+0000","last_unstale":"2026-02-25T11:47:14.750830+0000","last_undegraded":"2026-02-25T11:47:14.750830+0000","last_fullsized":"2026-02-25T11:47:14.750830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:01:29.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635319+0000","last_change":"2026-02-25T11:47:13.632721+0000","last_active":"2026-02-25T11:47:14.635319+0000","last_peered":"2026-02-25T11:47:14.635319+0000","last_clean":"2026-02-25T11:47:14.635319+0000","last_became_active":"2026-02-25T11:47:13.632626+0000","last_became_peered":"2026-02-25T11:47:13.632626+0000","last_unstale":"2026-02-25T11:47:14.635319+0000","last_undegraded":"2026-02-25T11:47:14.635319+0000","last_fullsized":"2026-02-25T11:47:14.635319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:28:35.201966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749844+0000","last_change":"2026-02-25T11:47:13.637504+0000","last_active":"2026-02-25T11:47:14.749844+0000","last_peered":"2026-02-25T11:47:14.749844+0000","last_clean":"2026-02-25T11:47:14.749844+0000","last_became_active":"2026-02-25T11:47:13.636824+0000","last_became_peered":"2026-02-25T11:47:13.636824+0000","last_unstale":"2026-02-25T11:47:14.749844+0000","last_undegraded":"2026-02-25T11:47:14.749844+0000","last_fullsized":"2026-02-25T11:47:14.749844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:20:31.416302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635724+0000","last_change":"2026-02-25T11:47:13.633135+0000","last_active":"2026-02-25T11:47:14.635724+0000","last_peered":"2026-02-25T11:47:14.635724+0000","last_clean":"2026-02-25T11:47:14.635724+0000","last_became_active":"2026-02-25T11:47:13.632966+0000","last_became_peered":"2026-02-25T11:47:13.632966+0000","last_unstale":"2026-02-25T11:47:14.635724+0000","last_undegraded":"2026-02-25T11:47:14.635724+0000","last_fullsized":"2026-02-25T11:47:14.635724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:26:17.052924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628538+0000","last_change":"2026-02-25T11:47:13.633210+0000","last_active":"2026-02-25T11:47:14.628538+0000","last_peered":"2026-02-25T11:47:14.628538+0000","last_clean":"2026-02-25T11:47:14.628538+0000","last_became_active":"2026-02-25T11:47:13.633099+0000","last_became_peered":"2026-02-25T11:47:13.633099+0000","last_unstale":"2026-02-25T11:47:14.628538+0000","last_undegraded":"2026-02-25T11:47:14.628538+0000","last_fullsized":"2026-02-25T11:47:14.628538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:07:46.884629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628656+0000","last_change":"2026-02-25T11:47:13.634757+0000","last_active":"2026-02-25T11:47:14.628656+0000","last_peered":"2026-02-25T11:47:14.628656+0000","last_clean":"2026-02-25T11:47:14.628656+0000","last_became_active":"2026-02-25T11:47:13.634643+0000","last_became_peered":"2026-02-25T11:47:13.634643+0000","last_unstale":"2026-02-25T11:47:14.628656+0000","last_undegraded":"2026-02-25T11:47:14.628656+0000","last_fullsized":"2026-02-25T11:47:14.628656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:20:23.383359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749674+0000","last_change":"2026-02-25T11:47:13.630515+0000","last_active":"2026-02-25T11:47:14.749674+0000","last_peered":"2026-02-25T11:47:14.749674+0000","last_clean":"2026-02-25T11:47:14.749674+0000","last_became_active":"2026-02-25T11:47:13.630202+0000","last_became_peered":"2026-02-25T11:47:13.630202+0000","last_unstale":"2026-02-25T11:47:14.749674+0000","last_undegraded":"2026-02-25T11:47:14.749674+0000","last_fullsized":"2026-02-25T11:47:14.749674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:51:24.646614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627367+0000","last_change":"2026-02-25T11:47:13.635749+0000","last_active":"2026-02-25T11:47:14.627367+0000","last_peered":"2026-02-25T11:47:14.627367+0000","last_clean":"2026-02-25T11:47:14.627367+0000","last_became_active":"2026-02-25T11:47:13.635284+0000","last_became_peered":"2026-02-25T11:47:13.635284+0000","last_unstale":"2026-02-25T11:47:14.627367+0000","last_undegraded":"2026-02-25T11:47:14.627367+0000","last_fullsized":"2026-02-25T11:47:14.627367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:23:45.042142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628732+0000","last_change":"2026-02-25T11:47:13.635448+0000","last_active":"2026-02-25T11:47:14.628732+0000","last_peered":"2026-02-25T11:47:14.628732+0000","last_clean":"2026-02-25T11:47:14.628732+0000","last_became_active":"2026-02-25T11:47:13.635232+0000","last_became_peered":"2026-02-25T11:47:13.635232+0000","last_unstale":"2026-02-25T11:47:14.628732+0000","last_undegraded":"2026-02-25T11:47:14.628732+0000","last_fullsized":"2026-02-25T11:47:14.628732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:33:38.020605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628332+0000","last_change":"2026-02-25T11:47:13.634237+0000","last_active":"2026-02-25T11:47:14.628332+0000","last_peered":"2026-02-25T11:47:14.628332+0000","last_clean":"2026-02-25T11:47:14.628332+0000","last_became_active":"2026-02-25T11:47:13.634054+0000","last_became_peered":"2026-02-25T11:47:13.634054+0000","last_unstale":"2026-02-25T11:47:14.628332+0000","last_undegraded":"2026-02-25T11:47:14.628332+0000","last_fullsized":"2026-02-25T11:47:14.628332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:38:34.603077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627786+0000","last_change":"2026-02-25T11:47:13.638347+0000","last_active":"2026-02-25T11:47:14.627786+0000","last_peered":"2026-02-25T11:47:14.627786+0000","last_clean":"2026-02-25T11:47:14.627786+0000","last_became_active":"2026-02-25T11:47:13.637984+0000","last_became_peered":"2026-02-25T11:47:13.637984+0000","last_unstale":"2026-02-25T11:47:14.627786+0000","last_undegraded":"2026-02-25T11:47:14.627786+0000","last_fullsized":"2026-02-25T11:47:14.627786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:36:11.273078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628202+0000","last_change":"2026-02-25T11:47:13.634523+0000","last_active":"2026-02-25T11:47:14.628202+0000","last_peered":"2026-02-25T11:47:14.628202+0000","last_clean":"2026-02-25T11:47:14.628202+0000","last_became_active":"2026-02-25T11:47:13.634184+0000","last_became_peered":"2026-02-25T11:47:13.634184+0000","last_unstale":"2026-02-25T11:47:14.628202+0000","last_undegraded":"2026-02-25T11:47:14.628202+0000","last_fullsized":"2026-02-25T11:47:14.628202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:52:23.917090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.576471+0000","last_change":"2026-02-25T11:47:12.613380+0000","last_active":"2026-02-25T11:47:18.576471+0000","last_peered":"2026-02-25T11:47:18.576471+0000","last_clean":"2026-02-25T11:47:18.576471+0000","last_became_active":"2026-02-25T11:47:12.613224+0000","last_became_peered":"2026-02-25T11:47:12.613224+0000","last_unstale":"2026-02-25T11:47:18.576471+0000","last_undegraded":"2026-02-25T11:47:18.576471+0000","last_fullsized":"2026-02-25T11:47:18.576471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:59:39.894067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749915+0000","last_change":"2026-02-25T11:47:13.636227+0000","last_active":"2026-02-25T11:47:14.749915+0000","last_peered":"2026-02-25T11:47:14.749915+0000","last_clean":"2026-02-25T11:47:14.749915+0000","last_became_active":"2026-02-25T11:47:13.635212+0000","last_became_peered":"2026-02-25T11:47:13.635212+0000","last_unstale":"2026-02-25T11:47:14.749915+0000","last_undegraded":"2026-02-25T11:47:14.749915+0000","last_fullsized":"2026-02-25T11:47:14.749915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:51:07.635399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657308+0000","last_change":"2026-02-25T11:47:12.613789+0000","last_active":"2026-02-25T11:47:17.657308+0000","last_peered":"2026-02-25T11:47:17.657308+0000","last_clean":"2026-02-25T11:47:17.657308+0000","last_became_active":"2026-02-25T11:47:12.613635+0000","last_became_peered":"2026-02-25T11:47:12.613635+0000","last_unstale":"2026-02-25T11:47:17.657308+0000","last_undegraded":"2026-02-25T11:47:17.657308+0000","last_fullsized":"2026-02-25T11:47:17.657308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:13:32.216593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657439+0000","last_change":"2026-02-25T11:47:12.612012+0000","last_active":"2026-02-25T11:47:17.657439+0000","last_peered":"2026-02-25T11:47:17.657439+0000","last_clean":"2026-02-25T11:47:17.657439+0000","last_became_active":"2026-02-25T11:47:12.611862+0000","last_became_peered":"2026-02-25T11:47:12.611862+0000","last_unstale":"2026-02-25T11:47:17.657439+0000","last_undegraded":"2026-02-25T11:47:17.657439+0000","last_fullsized":"2026-02-25T11:47:17.657439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:08:17.490884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635279+0000","last_change":"2026-02-25T11:47:13.627129+0000","last_active":"2026-02-25T11:47:14.635279+0000","last_peered":"2026-02-25T11:47:14.635279+0000","last_clean":"2026-02-25T11:47:14.635279+0000","last_became_active":"2026-02-25T11:47:13.626987+0000","last_became_peered":"2026-02-25T11:47:13.626987+0000","last_unstale":"2026-02-25T11:47:14.635279+0000","last_undegraded":"2026-02-25T11:47:14.635279+0000","last_fullsized":"2026-02-25T11:47:14.635279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:53:45.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749940+0000","last_change":"2026-02-25T11:47:12.611992+0000","last_active":"2026-02-25T11:47:14.749940+0000","last_peered":"2026-02-25T11:47:14.749940+0000","last_clean":"2026-02-25T11:47:14.749940+0000","last_became_active":"2026-02-25T11:47:12.611502+0000","last_became_peered":"2026-02-25T11:47:12.611502+0000","last_unstale":"2026-02-25T11:47:14.749940+0000","last_undegraded":"2026-02-25T11:47:14.749940+0000","last_fullsized":"2026-02-25T11:47:14.749940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:15:45.718628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633835+0000","last_change":"2026-02-25T11:47:13.635344+0000","last_active":"2026-02-25T11:47:14.633835+0000","last_peered":"2026-02-25T11:47:14.633835+0000","last_clean":"2026-02-25T11:47:14.633835+0000","last_became_active":"2026-02-25T11:47:13.634327+0000","last_became_peered":"2026-02-25T11:47:13.634327+0000","last_unstale":"2026-02-25T11:47:14.633835+0000","last_undegraded":"2026-02-25T11:47:14.633835+0000","last_fullsized":"2026-02-25T11:47:14.633835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:30:06.778398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:20.578577+0000","last_change":"2026-02-25T11:47:12.611130+0000","last_active":"2026-02-25T11:47:20.578577+0000","last_peered":"2026-02-25T11:47:20.578577+0000","last_clean":"2026-02-25T11:47:20.578577+0000","last_became_active":"2026-02-25T11:47:12.610838+0000","last_became_peered":"2026-02-25T11:47:12.610838+0000","last_unstale":"2026-02-25T11:47:20.578577+0000","last_undegraded":"2026-02-25T11:47:20.578577+0000","last_fullsized":"2026-02-25T11:47:20.578577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:34:19.366424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751244+0000","last_change":"2026-02-25T11:47:13.629263+0000","last_active":"2026-02-25T11:47:14.751244+0000","last_peered":"2026-02-25T11:47:14.751244+0000","last_clean":"2026-02-25T11:47:14.751244+0000","last_became_active":"2026-02-25T11:47:13.628765+0000","last_became_peered":"2026-02-25T11:47:13.628765+0000","last_unstale":"2026-02-25T11:47:14.751244+0000","last_undegraded":"2026-02-25T11:47:14.751244+0000","last_fullsized":"2026-02-25T11:47:14.751244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:49:04.773052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.649872+0000","last_change":"2026-02-25T11:47:12.613455+0000","last_active":"2026-02-25T11:47:16.649872+0000","last_peered":"2026-02-25T11:47:16.649872+0000","last_clean":"2026-02-25T11:47:16.649872+0000","last_became_active":"2026-02-25T11:47:12.612904+0000","last_became_peered":"2026-02-25T11:47:12.612904+0000","last_unstale":"2026-02-25T11:47:16.649872+0000","last_undegraded":"2026-02-25T11:47:16.649872+0000","last_fullsized":"2026-02-25T11:47:16.649872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:16:13.149517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627673+0000","last_change":"2026-02-25T11:47:13.635178+0000","last_active":"2026-02-25T11:47:14.627673+0000","last_peered":"2026-02-25T11:47:14.627673+0000","last_clean":"2026-02-25T11:47:14.627673+0000","last_became_active":"2026-02-25T11:47:13.634985+0000","last_became_peered":"2026-02-25T11:47:13.634985+0000","last_unstale":"2026-02-25T11:47:14.627673+0000","last_undegraded":"2026-02-25T11:47:14.627673+0000","last_fullsized":"2026-02-25T11:47:14.627673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:08.068457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653282+0000","last_change":"2026-02-25T11:47:12.610853+0000","last_active":"2026-02-25T11:47:16.653282+0000","last_peered":"2026-02-25T11:47:16.653282+0000","last_clean":"2026-02-25T11:47:16.653282+0000","last_became_active":"2026-02-25T11:47:12.610639+0000","last_became_peered":"2026-02-25T11:47:12.610639+0000","last_unstale":"2026-02-25T11:47:16.653282+0000","last_undegraded":"2026-02-25T11:47:16.653282+0000","last_fullsized":"2026-02-25T11:47:16.653282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:00:06.042790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":694,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750953+0000","last_change":"2026-02-25T11:47:13.630063+0000","last_active":"2026-02-25T11:47:14.750953+0000","last_peered":"2026-02-25T11:47:14.750953+0000","last_clean":"2026-02-25T11:47:14.750953+0000","last_became_active":"2026-02-25T11:47:13.629906+0000","last_became_peered":"2026-02-25T11:47:13.629906+0000","last_unstale":"2026-02-25T11:47:14.750953+0000","last_undegraded":"2026-02-25T11:47:14.750953+0000","last_fullsized":"2026-02-25T11:47:14.750953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:09:13.614948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'21","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:21.581481+0000","last_change":"2026-02-25T11:47:12.615044+0000","last_active":"2026-02-25T11:47:21.581481+0000","last_peered":"2026-02-25T11:47:21.581481+0000","last_clean":"2026-02-25T11:47:21.581481+0000","last_became_active":"2026-02-25T11:47:12.614698+0000","last_became_peered":"2026-02-25T11:47:12.614698+0000","last_unstale":"2026-02-25T11:47:21.581481+0000","last_undegraded":"2026-02-25T11:47:21.581481+0000","last_fullsized":"2026-02-25T11:47:21.581481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:52:32.148801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":49007,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":13,"num_read_kb":53,"num_write":22,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628510+0000","last_change":"2026-02-25T11:47:13.630945+0000","last_active":"2026-02-25T11:47:14.628510+0000","last_peered":"2026-02-25T11:47:14.628510+0000","last_clean":"2026-02-25T11:47:14.628510+0000","last_became_active":"2026-02-25T11:47:13.630693+0000","last_became_peered":"2026-02-25T11:47:13.630693+0000","last_unstale":"2026-02-25T11:47:14.628510+0000","last_undegraded":"2026-02-25T11:47:14.628510+0000","last_fullsized":"2026-02-25T11:47:14.628510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:39:52.744366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656758+0000","last_change":"2026-02-25T11:47:12.614950+0000","last_active":"2026-02-25T11:47:17.656758+0000","last_peered":"2026-02-25T11:47:17.656758+0000","last_clean":"2026-02-25T11:47:17.656758+0000","last_became_active":"2026-02-25T11:47:12.614272+0000","last_became_peered":"2026-02-25T11:47:12.614272+0000","last_unstale":"2026-02-25T11:47:17.656758+0000","last_undegraded":"2026-02-25T11:47:17.656758+0000","last_fullsized":"2026-02-25T11:47:17.656758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:45:50.928166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749596+0000","last_change":"2026-02-25T11:47:13.633128+0000","last_active":"2026-02-25T11:47:14.749596+0000","last_peered":"2026-02-25T11:47:14.749596+0000","last_clean":"2026-02-25T11:47:14.749596+0000","last_became_active":"2026-02-25T11:47:13.633047+0000","last_became_peered":"2026-02-25T11:47:13.633047+0000","last_unstale":"2026-02-25T11:47:14.749596+0000","last_undegraded":"2026-02-25T11:47:14.749596+0000","last_fullsized":"2026-02-25T11:47:14.749596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:56:33.191193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653783+0000","last_change":"2026-02-25T11:47:12.616898+0000","last_active":"2026-02-25T11:47:16.653783+0000","last_peered":"2026-02-25T11:47:16.653783+0000","last_clean":"2026-02-25T11:47:16.653783+0000","last_became_active":"2026-02-25T11:47:12.616774+0000","last_became_peered":"2026-02-25T11:47:12.616774+0000","last_unstale":"2026-02-25T11:47:16.653783+0000","last_undegraded":"2026-02-25T11:47:16.653783+0000","last_fullsized":"2026-02-25T11:47:16.653783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:59:33.930544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628464+0000","last_change":"2026-02-25T11:47:13.630377+0000","last_active":"2026-02-25T11:47:14.628464+0000","last_peered":"2026-02-25T11:47:14.628464+0000","last_clean":"2026-02-25T11:47:14.628464+0000","last_became_active":"2026-02-25T11:47:13.630124+0000","last_became_peered":"2026-02-25T11:47:13.630124+0000","last_unstale":"2026-02-25T11:47:14.628464+0000","last_undegraded":"2026-02-25T11:47:14.628464+0000","last_fullsized":"2026-02-25T11:47:14.628464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:42:12.028258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:19.580555+0000","last_change":"2026-02-25T11:47:12.613208+0000","last_active":"2026-02-25T11:47:19.580555+0000","last_peered":"2026-02-25T11:47:19.580555+0000","last_clean":"2026-02-25T11:47:19.580555+0000","last_became_active":"2026-02-25T11:47:12.612460+0000","last_became_peered":"2026-02-25T11:47:12.612460+0000","last_unstale":"2026-02-25T11:47:19.580555+0000","last_undegraded":"2026-02-25T11:47:19.580555+0000","last_fullsized":"2026-02-25T11:47:19.580555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:15:59.810632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":998,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628121+0000","last_change":"2026-02-25T11:47:13.631123+0000","last_active":"2026-02-25T11:47:14.628121+0000","last_peered":"2026-02-25T11:47:14.628121+0000","last_clean":"2026-02-25T11:47:14.628121+0000","last_became_active":"2026-02-25T11:47:13.631031+0000","last_became_peered":"2026-02-25T11:47:13.631031+0000","last_unstale":"2026-02-25T11:47:14.628121+0000","last_undegraded":"2026-02-25T11:47:14.628121+0000","last_fullsized":"2026-02-25T11:47:14.628121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:47:54.073226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657937+0000","last_change":"2026-02-25T11:47:12.612153+0000","last_active":"2026-02-25T11:47:17.657937+0000","last_peered":"2026-02-25T11:47:17.657937+0000","last_clean":"2026-02-25T11:47:17.657937+0000","last_became_active":"2026-02-25T11:47:12.611107+0000","last_became_peered":"2026-02-25T11:47:12.611107+0000","last_unstale":"2026-02-25T11:47:17.657937+0000","last_undegraded":"2026-02-25T11:47:17.657937+0000","last_fullsized":"2026-02-25T11:47:17.657937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:37:21.791728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749858+0000","last_change":"2026-02-25T11:47:13.635800+0000","last_active":"2026-02-25T11:47:14.749858+0000","last_peered":"2026-02-25T11:47:14.749858+0000","last_clean":"2026-02-25T11:47:14.749858+0000","last_became_active":"2026-02-25T11:47:13.635590+0000","last_became_peered":"2026-02-25T11:47:13.635590+0000","last_unstale":"2026-02-25T11:47:14.749858+0000","last_undegraded":"2026-02-25T11:47:14.749858+0000","last_fullsized":"2026-02-25T11:47:14.749858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:46:36.324518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575342+0000","last_change":"2026-02-25T11:47:12.616473+0000","last_active":"2026-02-25T11:47:18.575342+0000","last_peered":"2026-02-25T11:47:18.575342+0000","last_clean":"2026-02-25T11:47:18.575342+0000","last_became_active":"2026-02-25T11:47:12.616039+0000","last_became_peered":"2026-02-25T11:47:12.616039+0000","last_unstale":"2026-02-25T11:47:18.575342+0000","last_undegraded":"2026-02-25T11:47:18.575342+0000","last_fullsized":"2026-02-25T11:47:18.575342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:28.324889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628413+0000","last_change":"2026-02-25T11:47:13.634771+0000","last_active":"2026-02-25T11:47:14.628413+0000","last_peered":"2026-02-25T11:47:14.628413+0000","last_clean":"2026-02-25T11:47:14.628413+0000","last_became_active":"2026-02-25T11:47:13.634672+0000","last_became_peered":"2026-02-25T11:47:13.634672+0000","last_unstale":"2026-02-25T11:47:14.628413+0000","last_undegraded":"2026-02-25T11:47:14.628413+0000","last_fullsized":"2026-02-25T11:47:14.628413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:19:03.692423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749863+0000","last_change":"2026-02-25T11:47:12.616685+0000","last_active":"2026-02-25T11:47:14.749863+0000","last_peered":"2026-02-25T11:47:14.749863+0000","last_clean":"2026-02-25T11:47:14.749863+0000","last_became_active":"2026-02-25T11:47:12.616518+0000","last_became_peered":"2026-02-25T11:47:12.616518+0000","last_unstale":"2026-02-25T11:47:14.749863+0000","last_undegraded":"2026-02-25T11:47:14.749863+0000","last_fullsized":"2026-02-25T11:47:14.749863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:28:50.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628462+0000","last_change":"2026-02-25T11:47:13.627734+0000","last_active":"2026-02-25T11:47:14.628462+0000","last_peered":"2026-02-25T11:47:14.628462+0000","last_clean":"2026-02-25T11:47:14.628462+0000","last_became_active":"2026-02-25T11:47:13.627632+0000","last_became_peered":"2026-02-25T11:47:13.627632+0000","last_unstale":"2026-02-25T11:47:14.628462+0000","last_undegraded":"2026-02-25T11:47:14.628462+0000","last_fullsized":"2026-02-25T11:47:14.628462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:01:20.907699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750104+0000","last_change":"2026-02-25T11:47:12.612830+0000","last_active":"2026-02-25T11:47:14.750104+0000","last_peered":"2026-02-25T11:47:14.750104+0000","last_clean":"2026-02-25T11:47:14.750104+0000","last_became_active":"2026-02-25T11:47:12.612661+0000","last_became_peered":"2026-02-25T11:47:12.612661+0000","last_unstale":"2026-02-25T11:47:14.750104+0000","last_undegraded":"2026-02-25T11:47:14.750104+0000","last_fullsized":"2026-02-25T11:47:14.750104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:05:15.367058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628390+0000","last_change":"2026-02-25T11:47:13.633998+0000","last_active":"2026-02-25T11:47:14.628390+0000","last_peered":"2026-02-25T11:47:14.628390+0000","last_clean":"2026-02-25T11:47:14.628390+0000","last_became_active":"2026-02-25T11:47:13.633820+0000","last_became_peered":"2026-02-25T11:47:13.633820+0000","last_unstale":"2026-02-25T11:47:14.628390+0000","last_undegraded":"2026-02-25T11:47:14.628390+0000","last_fullsized":"2026-02-25T11:47:14.628390+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:02:38.693106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.658417+0000","last_change":"2026-02-25T11:47:12.614091+0000","last_active":"2026-02-25T11:47:17.658417+0000","last_peered":"2026-02-25T11:47:17.658417+0000","last_clean":"2026-02-25T11:47:17.658417+0000","last_became_active":"2026-02-25T11:47:12.612962+0000","last_became_peered":"2026-02-25T11:47:12.612962+0000","last_unstale":"2026-02-25T11:47:17.658417+0000","last_undegraded":"2026-02-25T11:47:17.658417+0000","last_fullsized":"2026-02-25T11:47:17.658417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:11:40.214146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628504+0000","last_change":"2026-02-25T11:47:13.627288+0000","last_active":"2026-02-25T11:47:14.628504+0000","last_peered":"2026-02-25T11:47:14.628504+0000","last_clean":"2026-02-25T11:47:14.628504+0000","last_became_active":"2026-02-25T11:47:13.626902+0000","last_became_peered":"2026-02-25T11:47:13.626902+0000","last_unstale":"2026-02-25T11:47:14.628504+0000","last_undegraded":"2026-02-25T11:47:14.628504+0000","last_fullsized":"2026-02-25T11:47:14.628504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:38:19.137242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'152","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628095+0000","last_change":"2026-02-25T11:46:35.865282+0000","last_active":"2026-02-25T11:47:14.628095+0000","last_peered":"2026-02-25T11:47:14.628095+0000","last_clean":"2026-02-25T11:47:14.628095+0000","last_became_active":"2026-02-25T11:46:35.865139+0000","last_became_peered":"2026-02-25T11:46:35.865139+0000","last_unstale":"2026-02-25T11:47:14.628095+0000","last_undegraded":"2026-02-25T11:47:14.628095+0000","last_fullsized":"2026-02-25T11:47:14.628095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_clean_scrub_stamp":"2026-02-25T11:44:43.739673+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:29:57.573039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575170+0000","last_change":"2026-02-25T11:47:12.612834+0000","last_active":"2026-02-25T11:47:18.575170+0000","last_peered":"2026-02-25T11:47:18.575170+0000","last_clean":"2026-02-25T11:47:18.575170+0000","last_became_active":"2026-02-25T11:47:12.611827+0000","last_became_peered":"2026-02-25T11:47:12.611827+0000","last_unstale":"2026-02-25T11:47:18.575170+0000","last_undegraded":"2026-02-25T11:47:18.575170+0000","last_fullsized":"2026-02-25T11:47:18.575170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:05:32.112908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635504+0000","last_change":"2026-02-25T11:47:13.634534+0000","last_active":"2026-02-25T11:47:14.635504+0000","last_peered":"2026-02-25T11:47:14.635504+0000","last_clean":"2026-02-25T11:47:14.635504+0000","last_became_active":"2026-02-25T11:47:13.633769+0000","last_became_peered":"2026-02-25T11:47:13.633769+0000","last_unstale":"2026-02-25T11:47:14.635504+0000","last_undegraded":"2026-02-25T11:47:14.635504+0000","last_fullsized":"2026-02-25T11:47:14.635504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:16:09.255508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656541+0000","last_change":"2026-02-25T11:47:12.612621+0000","last_active":"2026-02-25T11:47:17.656541+0000","last_peered":"2026-02-25T11:47:17.656541+0000","last_clean":"2026-02-25T11:47:17.656541+0000","last_became_active":"2026-02-25T11:47:12.612284+0000","last_became_peered":"2026-02-25T11:47:12.612284+0000","last_unstale":"2026-02-25T11:47:17.656541+0000","last_undegraded":"2026-02-25T11:47:17.656541+0000","last_fullsized":"2026-02-25T11:47:17.656541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:49:09.299914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749514+0000","last_change":"2026-02-25T11:47:13.629107+0000","last_active":"2026-02-25T11:47:14.749514+0000","last_peered":"2026-02-25T11:47:14.749514+0000","last_clean":"2026-02-25T11:47:14.749514+0000","last_became_active":"2026-02-25T11:47:13.629031+0000","last_became_peered":"2026-02-25T11:47:13.629031+0000","last_unstale":"2026-02-25T11:47:14.749514+0000","last_undegraded":"2026-02-25T11:47:14.749514+0000","last_fullsized":"2026-02-25T11:47:14.749514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:07:58.915416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:20.580996+0000","last_change":"2026-02-25T11:47:12.616543+0000","last_active":"2026-02-25T11:47:20.580996+0000","last_peered":"2026-02-25T11:47:20.580996+0000","last_clean":"2026-02-25T11:47:20.580996+0000","last_became_active":"2026-02-25T11:47:12.616345+0000","last_became_peered":"2026-02-25T11:47:12.616345+0000","last_unstale":"2026-02-25T11:47:20.580996+0000","last_undegraded":"2026-02-25T11:47:20.580996+0000","last_fullsized":"2026-02-25T11:47:20.580996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:27:30.862168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750014+0000","last_change":"2026-02-25T11:47:13.637385+0000","last_active":"2026-02-25T11:47:14.750014+0000","last_peered":"2026-02-25T11:47:14.750014+0000","last_clean":"2026-02-25T11:47:14.750014+0000","last_became_active":"2026-02-25T11:47:13.634653+0000","last_became_peered":"2026-02-25T11:47:13.634653+0000","last_unstale":"2026-02-25T11:47:14.750014+0000","last_undegraded":"2026-02-25T11:47:14.750014+0000","last_fullsized":"2026-02-25T11:47:14.750014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:05:08.662208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:15.914921+0000","last_change":"2026-02-25T11:47:12.614640+0000","last_active":"2026-02-25T11:47:15.914921+0000","last_peered":"2026-02-25T11:47:15.914921+0000","last_clean":"2026-02-25T11:47:15.914921+0000","last_became_active":"2026-02-25T11:47:12.614486+0000","last_became_peered":"2026-02-25T11:47:12.614486+0000","last_unstale":"2026-02-25T11:47:15.914921+0000","last_undegraded":"2026-02-25T11:47:15.914921+0000","last_fullsized":"2026-02-25T11:47:15.914921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:16:06.314709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750327+0000","last_change":"2026-02-25T11:47:13.631828+0000","last_active":"2026-02-25T11:47:14.750327+0000","last_peered":"2026-02-25T11:47:14.750327+0000","last_clean":"2026-02-25T11:47:14.750327+0000","last_became_active":"2026-02-25T11:47:13.631590+0000","last_became_peered":"2026-02-25T11:47:13.631590+0000","last_unstale":"2026-02-25T11:47:14.750327+0000","last_undegraded":"2026-02-25T11:47:14.750327+0000","last_fullsized":"2026-02-25T11:47:14.750327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:25.518794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633984+0000","last_change":"2026-02-25T11:47:12.614032+0000","last_active":"2026-02-25T11:47:14.633984+0000","last_peered":"2026-02-25T11:47:14.633984+0000","last_clean":"2026-02-25T11:47:14.633984+0000","last_became_active":"2026-02-25T11:47:12.613936+0000","last_became_peered":"2026-02-25T11:47:12.613936+0000","last_unstale":"2026-02-25T11:47:14.633984+0000","last_undegraded":"2026-02-25T11:47:14.633984+0000","last_fullsized":"2026-02-25T11:47:14.633984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:16:46.397485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.859010+0000","last_change":"2026-02-25T11:47:13.636173+0000","last_active":"2026-02-25T11:47:16.859010+0000","last_peered":"2026-02-25T11:47:16.859010+0000","last_clean":"2026-02-25T11:47:16.859010+0000","last_became_active":"2026-02-25T11:47:13.635745+0000","last_became_peered":"2026-02-25T11:47:13.635745+0000","last_unstale":"2026-02-25T11:47:16.859010+0000","last_undegraded":"2026-02-25T11:47:16.859010+0000","last_fullsized":"2026-02-25T11:47:16.859010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:03:33.671318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651871+0000","last_change":"2026-02-25T11:47:12.614003+0000","last_active":"2026-02-25T11:47:16.651871+0000","last_peered":"2026-02-25T11:47:16.651871+0000","last_clean":"2026-02-25T11:47:16.651871+0000","last_became_active":"2026-02-25T11:47:12.613893+0000","last_became_peered":"2026-02-25T11:47:12.613893+0000","last_unstale":"2026-02-25T11:47:16.651871+0000","last_undegraded":"2026-02-25T11:47:16.651871+0000","last_fullsized":"2026-02-25T11:47:16.651871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:33:21.133364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749746+0000","last_change":"2026-02-25T11:47:13.632632+0000","last_active":"2026-02-25T11:47:14.749746+0000","last_peered":"2026-02-25T11:47:14.749746+0000","last_clean":"2026-02-25T11:47:14.749746+0000","last_became_active":"2026-02-25T11:47:13.632408+0000","last_became_peered":"2026-02-25T11:47:13.632408+0000","last_unstale":"2026-02-25T11:47:14.749746+0000","last_undegraded":"2026-02-25T11:47:14.749746+0000","last_fullsized":"2026-02-25T11:47:14.749746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:38:45.448637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657145+0000","last_change":"2026-02-25T11:47:12.613816+0000","last_active":"2026-02-25T11:47:17.657145+0000","last_peered":"2026-02-25T11:47:17.657145+0000","last_clean":"2026-02-25T11:47:17.657145+0000","last_became_active":"2026-02-25T11:47:12.613196+0000","last_became_peered":"2026-02-25T11:47:12.613196+0000","last_unstale":"2026-02-25T11:47:17.657145+0000","last_undegraded":"2026-02-25T11:47:17.657145+0000","last_fullsized":"2026-02-25T11:47:17.657145+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:23:52.372808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635752+0000","last_change":"2026-02-25T11:47:13.632431+0000","last_active":"2026-02-25T11:47:14.635752+0000","last_peered":"2026-02-25T11:47:14.635752+0000","last_clean":"2026-02-25T11:47:14.635752+0000","last_became_active":"2026-02-25T11:47:13.631972+0000","last_became_peered":"2026-02-25T11:47:13.631972+0000","last_unstale":"2026-02-25T11:47:14.635752+0000","last_undegraded":"2026-02-25T11:47:14.635752+0000","last_fullsized":"2026-02-25T11:47:14.635752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:59:34.655028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651131+0000","last_change":"2026-02-25T11:47:12.612805+0000","last_active":"2026-02-25T11:47:16.651131+0000","last_peered":"2026-02-25T11:47:16.651131+0000","last_clean":"2026-02-25T11:47:16.651131+0000","last_became_active":"2026-02-25T11:47:12.612155+0000","last_became_peered":"2026-02-25T11:47:12.612155+0000","last_unstale":"2026-02-25T11:47:16.651131+0000","last_undegraded":"2026-02-25T11:47:16.651131+0000","last_fullsized":"2026-02-25T11:47:16.651131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:58:00.429639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627853+0000","last_change":"2026-02-25T11:47:13.636089+0000","last_active":"2026-02-25T11:47:14.627853+0000","last_peered":"2026-02-25T11:47:14.627853+0000","last_clean":"2026-02-25T11:47:14.627853+0000","last_became_active":"2026-02-25T11:47:13.635963+0000","last_became_peered":"2026-02-25T11:47:13.635963+0000","last_unstale":"2026-02-25T11:47:14.627853+0000","last_undegraded":"2026-02-25T11:47:14.627853+0000","last_fullsized":"2026-02-25T11:47:14.627853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:50:28.012023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650081+0000","last_change":"2026-02-25T11:47:12.609749+0000","last_active":"2026-02-25T11:47:16.650081+0000","last_peered":"2026-02-25T11:47:16.650081+0000","last_clean":"2026-02-25T11:47:16.650081+0000","last_became_active":"2026-02-25T11:47:12.609190+0000","last_became_peered":"2026-02-25T11:47:12.609190+0000","last_unstale":"2026-02-25T11:47:16.650081+0000","last_undegraded":"2026-02-25T11:47:16.650081+0000","last_fullsized":"2026-02-25T11:47:16.650081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:12:42.204122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628201+0000","last_change":"2026-02-25T11:47:13.629339+0000","last_active":"2026-02-25T11:47:14.628201+0000","last_peered":"2026-02-25T11:47:14.628201+0000","last_clean":"2026-02-25T11:47:14.628201+0000","last_became_active":"2026-02-25T11:47:13.629107+0000","last_became_peered":"2026-02-25T11:47:13.629107+0000","last_unstale":"2026-02-25T11:47:14.628201+0000","last_undegraded":"2026-02-25T11:47:14.628201+0000","last_fullsized":"2026-02-25T11:47:14.628201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:19:50.087771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.652673+0000","last_change":"2026-02-25T11:47:12.615249+0000","last_active":"2026-02-25T11:47:16.652673+0000","last_peered":"2026-02-25T11:47:16.652673+0000","last_clean":"2026-02-25T11:47:16.652673+0000","last_became_active":"2026-02-25T11:47:12.615165+0000","last_became_peered":"2026-02-25T11:47:12.615165+0000","last_unstale":"2026-02-25T11:47:16.652673+0000","last_undegraded":"2026-02-25T11:47:16.652673+0000","last_fullsized":"2026-02-25T11:47:16.652673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:40:52.452074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627665+0000","last_change":"2026-02-25T11:47:13.631460+0000","last_active":"2026-02-25T11:47:14.627665+0000","last_peered":"2026-02-25T11:47:14.627665+0000","last_clean":"2026-02-25T11:47:14.627665+0000","last_became_active":"2026-02-25T11:47:13.631298+0000","last_became_peered":"2026-02-25T11:47:13.631298+0000","last_unstale":"2026-02-25T11:47:14.627665+0000","last_undegraded":"2026-02-25T11:47:14.627665+0000","last_fullsized":"2026-02-25T11:47:14.627665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:26:03.362106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628432+0000","last_change":"2026-02-25T11:47:12.615779+0000","last_active":"2026-02-25T11:47:14.628432+0000","last_peered":"2026-02-25T11:47:14.628432+0000","last_clean":"2026-02-25T11:47:14.628432+0000","last_became_active":"2026-02-25T11:47:12.615073+0000","last_became_peered":"2026-02-25T11:47:12.615073+0000","last_unstale":"2026-02-25T11:47:14.628432+0000","last_undegraded":"2026-02-25T11:47:14.628432+0000","last_fullsized":"2026-02-25T11:47:14.628432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:28:51.468199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635380+0000","last_change":"2026-02-25T11:47:13.632351+0000","last_active":"2026-02-25T11:47:14.635380+0000","last_peered":"2026-02-25T11:47:14.635380+0000","last_clean":"2026-02-25T11:47:14.635380+0000","last_became_active":"2026-02-25T11:47:13.632081+0000","last_became_peered":"2026-02-25T11:47:13.632081+0000","last_unstale":"2026-02-25T11:47:14.635380+0000","last_undegraded":"2026-02-25T11:47:14.635380+0000","last_fullsized":"2026-02-25T11:47:14.635380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:38:59.143253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650353+0000","last_change":"2026-02-25T11:47:12.616555+0000","last_active":"2026-02-25T11:47:16.650353+0000","last_peered":"2026-02-25T11:47:16.650353+0000","last_clean":"2026-02-25T11:47:16.650353+0000","last_became_active":"2026-02-25T11:47:12.616174+0000","last_became_peered":"2026-02-25T11:47:12.616174+0000","last_unstale":"2026-02-25T11:47:16.650353+0000","last_undegraded":"2026-02-25T11:47:16.650353+0000","last_fullsized":"2026-02-25T11:47:16.650353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:01:17.494129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628458+0000","last_change":"2026-02-25T11:47:13.627400+0000","last_active":"2026-02-25T11:47:14.628458+0000","last_peered":"2026-02-25T11:47:14.628458+0000","last_clean":"2026-02-25T11:47:14.628458+0000","last_became_active":"2026-02-25T11:47:13.626782+0000","last_became_peered":"2026-02-25T11:47:13.626782+0000","last_unstale":"2026-02-25T11:47:14.628458+0000","last_undegraded":"2026-02-25T11:47:14.628458+0000","last_fullsized":"2026-02-25T11:47:14.628458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:42:39.996935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750027+0000","last_change":"2026-02-25T11:47:13.634030+0000","last_active":"2026-02-25T11:47:14.750027+0000","last_peered":"2026-02-25T11:47:14.750027+0000","last_clean":"2026-02-25T11:47:14.750027+0000","last_became_active":"2026-02-25T11:47:13.632844+0000","last_became_peered":"2026-02-25T11:47:13.632844+0000","last_unstale":"2026-02-25T11:47:14.750027+0000","last_undegraded":"2026-02-25T11:47:14.750027+0000","last_fullsized":"2026-02-25T11:47:14.750027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:52:16.013596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628805+0000","last_change":"2026-02-25T11:47:12.613666+0000","last_active":"2026-02-25T11:47:14.628805+0000","last_peered":"2026-02-25T11:47:14.628805+0000","last_clean":"2026-02-25T11:47:14.628805+0000","last_became_active":"2026-02-25T11:47:12.613570+0000","last_became_peered":"2026-02-25T11:47:12.613570+0000","last_unstale":"2026-02-25T11:47:14.628805+0000","last_undegraded":"2026-02-25T11:47:14.628805+0000","last_fullsized":"2026-02-25T11:47:14.628805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:28:39.750737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750924+0000","last_change":"2026-02-25T11:47:13.631515+0000","last_active":"2026-02-25T11:47:14.750924+0000","last_peered":"2026-02-25T11:47:14.750924+0000","last_clean":"2026-02-25T11:47:14.750924+0000","last_became_active":"2026-02-25T11:47:13.631161+0000","last_became_peered":"2026-02-25T11:47:13.631161+0000","last_unstale":"2026-02-25T11:47:14.750924+0000","last_undegraded":"2026-02-25T11:47:14.750924+0000","last_fullsized":"2026-02-25T11:47:14.750924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:13:30.496521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656891+0000","last_change":"2026-02-25T11:47:12.615904+0000","last_active":"2026-02-25T11:47:17.656891+0000","last_peered":"2026-02-25T11:47:17.656891+0000","last_clean":"2026-02-25T11:47:17.656891+0000","last_became_active":"2026-02-25T11:47:12.615784+0000","last_became_peered":"2026-02-25T11:47:12.615784+0000","last_unstale":"2026-02-25T11:47:17.656891+0000","last_undegraded":"2026-02-25T11:47:17.656891+0000","last_fullsized":"2026-02-25T11:47:17.656891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:32:26.802870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627754+0000","last_change":"2026-02-25T11:47:13.630065+0000","last_active":"2026-02-25T11:47:14.627754+0000","last_peered":"2026-02-25T11:47:14.627754+0000","last_clean":"2026-02-25T11:47:14.627754+0000","last_became_active":"2026-02-25T11:47:13.629214+0000","last_became_peered":"2026-02-25T11:47:13.629214+0000","last_unstale":"2026-02-25T11:47:14.627754+0000","last_undegraded":"2026-02-25T11:47:14.627754+0000","last_fullsized":"2026-02-25T11:47:14.627754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:53:45.639526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.310632+0000","last_change":"2026-02-25T11:47:12.612956+0000","last_active":"2026-02-25T11:47:16.310632+0000","last_peered":"2026-02-25T11:47:16.310632+0000","last_clean":"2026-02-25T11:47:16.310632+0000","last_became_active":"2026-02-25T11:47:12.612843+0000","last_became_peered":"2026-02-25T11:47:12.612843+0000","last_unstale":"2026-02-25T11:47:16.310632+0000","last_undegraded":"2026-02-25T11:47:16.310632+0000","last_fullsized":"2026-02-25T11:47:16.310632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:41:30.867465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635780+0000","last_change":"2026-02-25T11:47:13.637712+0000","last_active":"2026-02-25T11:47:14.635780+0000","last_peered":"2026-02-25T11:47:14.635780+0000","last_clean":"2026-02-25T11:47:14.635780+0000","last_became_active":"2026-02-25T11:47:13.635030+0000","last_became_peered":"2026-02-25T11:47:13.635030+0000","last_unstale":"2026-02-25T11:47:14.635780+0000","last_undegraded":"2026-02-25T11:47:14.635780+0000","last_fullsized":"2026-02-25T11:47:14.635780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:01:22.896129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656680+0000","last_change":"2026-02-25T11:47:12.614149+0000","last_active":"2026-02-25T11:47:17.656680+0000","last_peered":"2026-02-25T11:47:17.656680+0000","last_clean":"2026-02-25T11:47:17.656680+0000","last_became_active":"2026-02-25T11:47:12.613137+0000","last_became_peered":"2026-02-25T11:47:12.613137+0000","last_unstale":"2026-02-25T11:47:17.656680+0000","last_undegraded":"2026-02-25T11:47:17.656680+0000","last_fullsized":"2026-02-25T11:47:17.656680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:49:49.451435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":53093,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":42,"num_read_kb":79,"num_write":139,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":123366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":221240,"internal_metadata":0},"log_size":123,"ondisk_log_size":123,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":262884,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350985,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28132,"kb_used_data":912,"kb_used_omap":9,"kb_used_meta":27190,"kb_avail":733975068,"statfs":{"total":751619276800,"available":751590469632,"internally_reserved":0,"allocated":933888,"data_stored":455512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10109,"internal_metadata":27842691},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612619,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28268,"kb_used_data":1048,"kb_used_omap":28,"kb_used_meta":27171,"kb_avail":733974932,"statfs":{"total":751619276800,"available":751590330368,"internally_reserved":0,"allocated":1073152,"data_stored":1096410,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":29123,"internal_metadata":27823677},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776141,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28144,"kb_used_data":924,"kb_used_omap":24,"kb_used_meta":27175,"kb_avail":733975056,"statfs":{"total":751619276800,"available":751590457344,"internally_reserved":0,"allocated":946176,"data_stored":455714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24746,"internal_metadata":27828054},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037776,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28148,"kb_used_data":928,"kb_used_omap":33,"kb_used_meta":27166,"kb_avail":733975052,"statfs":{"total":751619276800,"available":751590453248,"internally_reserved":0,"allocated":950272,"data_stored":455668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":34197,"internal_metadata":27818603},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299411,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28192,"kb_used_data":972,"kb_used_omap":35,"kb_used_meta":27164,"kb_avail":733975008,"statfs":{"total":751619276800,"available":751590408192,"internally_reserved":0,"allocated":995328,"data_stored":1047500,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36168,"internal_metadata":27816632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462935,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28136,"kb_used_data":916,"kb_used_omap":31,"kb_used_meta":27168,"kb_avail":733975064,"statfs":{"total":751619276800,"available":751590465536,"internally_reserved":0,"allocated":937984,"data_stored":455115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32619,"internal_metadata":27820181},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724568,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28204,"kb_used_data":984,"kb_used_omap":36,"kb_used_meta":27163,"kb_avail":733974996,"statfs":{"total":751619276800,"available":751590395904,"internally_reserved":0,"allocated":1007616,"data_stored":1046774,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36952,"internal_metadata":27815848},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888091,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28140,"kb_used_data":920,"kb_used_omap":45,"kb_used_meta":27154,"kb_avail":733975060,"statfs":{"total":751619276800,"available":751590461440,"internally_reserved":0,"allocated":942080,"data_stored":455962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":46332,"internal_metadata":27806468},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149726,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28152,"kb_used_data":932,"kb_used_omap":50,"kb_used_meta":27149,"kb_avail":733975048,"statfs":{"total":751619276800,"available":751590449152,"internally_reserved":0,"allocated":954368,"data_stored":455426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":51675,"internal_metadata":27801125},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378657,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28212,"kb_used_data":992,"kb_used_omap":64,"kb_used_meta":27135,"kb_avail":733974988,"statfs":{"total":751619276800,"available":751590387712,"internally_reserved":0,"allocated":1015808,"data_stored":509996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":66234,"internal_metadata":27786566},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28140,"kb_used_data":920,"kb_used_omap":43,"kb_used_meta":27156,"kb_avail":733975060,"statfs":{"total":751619276800,"available":751590461440,"internally_reserved":0,"allocated":942080,"data_stored":456542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44488,"internal_metadata":27808312},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28156,"kb_used_data":936,"kb_used_omap":54,"kb_used_meta":27145,"kb_avail":733975044,"statfs":{"total":751619276800,"available":751590445056,"internally_reserved":0,"allocated":958464,"data_stored":456430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56100,"internal_metadata":27796700},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8853,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":8385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1407,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":55088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56835,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48464,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6449,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3478,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14040,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9858,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24153,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19148,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":51017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21693,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6862,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T11:47:24.168 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2026-02-25T11:47:24.168 INFO:tasks.cephadm:Setting up client nodes... 2026-02-25T11:47:24.168 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2026-02-25T11:47:24.390 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:24.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.570+0000 7fd3019b8640 1 -- 10.20.193.28:0/65925664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd2fc10cf80 msgr2=0x7fd2fc10d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:24.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.570+0000 7fd3019b8640 1 --2- 10.20.193.28:0/65925664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd2fc10cf80 0x7fd2fc10d370 secure :-1 s=READY pgs=254 gs=1 cs=0 l=1 c_cookie=15589314897720370420 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd2ec009950 tx=0x7fd2ec02f190 comp rx=0 tx=0).stop 2026-02-25T11:47:24.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.570+0000 7fd3019b8640 1 -- 10.20.193.28:0/65925664 shutdown_connections 2026-02-25T11:47:24.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.570+0000 7fd3019b8640 1 --2- 10.20.193.28:0/65925664 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd2fc10eac0 0x7fd2fc16f450 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:24.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.570+0000 7fd3019b8640 1 --2- 10.20.193.28:0/65925664 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd2fc10e110 0x7fd2fc10e580 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:24.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.570+0000 7fd3019b8640 1 --2- 10.20.193.28:0/65925664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd2fc10cf80 0x7fd2fc10d370 unknown :-1 s=CLOSED pgs=254 gs=1 cs=0 l=1 c_cookie=15589314897720370420 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:24.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.570+0000 7fd3019b8640 1 -- 10.20.193.28:0/65925664 >> 10.20.193.28:0/65925664 conn(0x7fd2fc0ea6d0 msgr2=0x7fd2fc10aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:24.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.570+0000 7fd3019b8640 1 -- 10.20.193.28:0/65925664 shutdown_connections 2026-02-25T11:47:24.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.570+0000 7fd3019b8640 1 -- 10.20.193.28:0/65925664 wait complete. 2026-02-25T11:47:24.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.571+0000 7fd3019b8640 1 Processor -- start 2026-02-25T11:47:24.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.571+0000 7fd3019b8640 1 -- start start 2026-02-25T11:47:24.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.571+0000 7fd3019b8640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd2fc10cf80 0x7fd2fc0625e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:24.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.571+0000 7fd3019b8640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd2fc10e110 0x7fd2fc16f450 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:24.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.571+0000 7fd3019b8640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd2fc10eac0 0x7fd2fc060ef0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:24.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.571+0000 7fd3019b8640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc061430 con 0x7fd2fc10e110 2026-02-25T11:47:24.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.571+0000 7fd3019b8640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc0615a0 con 0x7fd2fc10cf80 2026-02-25T11:47:24.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.571+0000 7fd3019b8640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc061760 con 0x7fd2fc10eac0 2026-02-25T11:47:24.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd2f3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd2fc10e110 0x7fd2fc16f450 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:24.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd3009b6640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd2fc10cf80 0x7fd2fc0625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:24.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd2f3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd2fc10e110 0x7fd2fc16f450 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34166/0 (socket says 10.20.193.28:34166) 2026-02-25T11:47:24.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd2f3fff640 1 -- 10.20.193.28:0/1690485396 learned_addr learned my addr 10.20.193.28:0/1690485396 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:24.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd3011b7640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd2fc10eac0 0x7fd2fc060ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:24.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd2f3fff640 1 -- 10.20.193.28:0/1690485396 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd2fc10eac0 msgr2=0x7fd2fc060ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:24.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd2f3fff640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd2fc10eac0 0x7fd2fc060ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:24.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd2f3fff640 1 -- 10.20.193.28:0/1690485396 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd2fc10cf80 msgr2=0x7fd2fc0625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:24.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd2f3fff640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd2fc10cf80 0x7fd2fc0625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:24.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd2f3fff640 1 -- 10.20.193.28:0/1690485396 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ec009570 con 0x7fd2fc10e110 2026-02-25T11:47:24.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd3009b6640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd2fc10cf80 0x7fd2fc0625e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:24.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd2f3fff640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd2fc10e110 0x7fd2fc16f450 secure :-1 s=READY pgs=255 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd2e400d950 tx=0x7fd2e400de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:24.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd2f1ffb640 1 -- 10.20.193.28:0/1690485396 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd2e4004540 con 0x7fd2fc10e110 2026-02-25T11:47:24.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd2f1ffb640 1 -- 10.20.193.28:0/1690485396 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd2e4009ce0 con 0x7fd2fc10e110 2026-02-25T11:47:24.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd2f1ffb640 1 -- 10.20.193.28:0/1690485396 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd2e40106e0 con 0x7fd2fc10e110 2026-02-25T11:47:24.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd3019b8640 1 -- 10.20.193.28:0/1690485396 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc061a40 con 0x7fd2fc10e110 2026-02-25T11:47:24.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.572+0000 7fd3019b8640 1 -- 10.20.193.28:0/1690485396 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc0f6510 con 0x7fd2fc10e110 2026-02-25T11:47:24.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.573+0000 7fd2cb7fe640 1 -- 10.20.193.28:0/1690485396 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2fc173350 con 0x7fd2fc10e110 2026-02-25T11:47:24.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.575+0000 7fd2f1ffb640 1 -- 10.20.193.28:0/1690485396 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fd2e40097c0 con 0x7fd2fc10e110 2026-02-25T11:47:24.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.576+0000 7fd2f1ffb640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd2d80b57f0 0x7fd2d80d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:24.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.576+0000 7fd2f1ffb640 1 -- 10.20.193.28:0/1690485396 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fd2e40e6630 con 0x7fd2fc10e110 2026-02-25T11:47:24.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.577+0000 7fd3009b6640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd2d80b57f0 0x7fd2d80d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:24.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.577+0000 7fd3009b6640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd2d80b57f0 0x7fd2d80d5ca0 secure :-1 s=READY pgs=187 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd2ec0098a0 tx=0x7fd2ec03a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:24.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.577+0000 7fd2f1ffb640 1 -- 10.20.193.28:0/1690485396 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fd2e40a9090 con 0x7fd2fc10e110 2026-02-25T11:47:24.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.735+0000 7fd2cb7fe640 1 -- 10.20.193.28:0/1690485396 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fd2fc10d350 con 0x7fd2fc10e110 2026-02-25T11:47:24.738 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:24 trial028 ceph-mon[39224]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:24.738 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:24 trial028 ceph-mon[39224]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:24.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.737+0000 7fd2f1ffb640 1 -- 10.20.193.28:0/1690485396 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) ==== 170+0+59 (secure 0 0 0) 0x7fd2e40ac240 con 0x7fd2fc10e110 2026-02-25T11:47:24.739 INFO:teuthology.orchestra.run.trial028.stdout:[client.0] 2026-02-25T11:47:24.739 INFO:teuthology.orchestra.run.trial028.stdout: key = AQBM4Z5pwg7eKxAAVWaiK0N1LizlQDccvGvXow== 2026-02-25T11:47:24.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.738+0000 7fd2cb7fe640 1 -- 10.20.193.28:0/1690485396 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd2d80b57f0 msgr2=0x7fd2d80d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:24.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.738+0000 7fd2cb7fe640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd2d80b57f0 0x7fd2d80d5ca0 secure :-1 s=READY pgs=187 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd2ec0098a0 tx=0x7fd2ec03a040 comp rx=0 tx=0).stop 2026-02-25T11:47:24.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.738+0000 7fd2cb7fe640 1 -- 10.20.193.28:0/1690485396 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd2fc10e110 msgr2=0x7fd2fc16f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:24.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.738+0000 7fd2cb7fe640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd2fc10e110 0x7fd2fc16f450 secure :-1 s=READY pgs=255 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd2e400d950 tx=0x7fd2e400de20 comp rx=0 tx=0).stop 2026-02-25T11:47:24.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.738+0000 7fd2cb7fe640 1 -- 10.20.193.28:0/1690485396 shutdown_connections 2026-02-25T11:47:24.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.738+0000 7fd2cb7fe640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd2d80b57f0 0x7fd2d80d5ca0 unknown :-1 s=CLOSED pgs=187 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:24.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.738+0000 7fd2cb7fe640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd2fc10eac0 0x7fd2fc060ef0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:24.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.738+0000 7fd2cb7fe640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd2fc10e110 0x7fd2fc16f450 unknown :-1 s=CLOSED pgs=255 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:24.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.738+0000 7fd2cb7fe640 1 --2- 10.20.193.28:0/1690485396 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd2fc10cf80 0x7fd2fc0625e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:24.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.738+0000 7fd2cb7fe640 1 -- 10.20.193.28:0/1690485396 >> 10.20.193.28:0/1690485396 conn(0x7fd2fc0ea6d0 msgr2=0x7fd2fc0ecde0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:24.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.738+0000 7fd2cb7fe640 1 -- 10.20.193.28:0/1690485396 shutdown_connections 2026-02-25T11:47:24.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:24.738+0000 7fd2cb7fe640 1 -- 10.20.193.28:0/1690485396 wait complete. 2026-02-25T11:47:24.840 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:47:24.840 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2026-02-25T11:47:24.840 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-25T11:47:24.918 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph config log 1 --format=json 2026-02-25T11:47:24.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:24 trial167 ceph-mon[43392]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:24.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:24 trial167 ceph-mon[43392]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:25.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:24 trial119 ceph-mon[42710]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:25.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:24 trial119 ceph-mon[42710]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:25.140 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.320+0000 7ff9d3448640 1 -- 10.20.193.28:0/931089338 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9cc110080 msgr2=0x7ff9cc1150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.320+0000 7ff9d3448640 1 --2- 10.20.193.28:0/931089338 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9cc110080 0x7ff9cc1150d0 secure :-1 s=READY pgs=256 gs=1 cs=0 l=1 c_cookie=12771058586736545011 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff9c0009950 tx=0x7ff9c002f190 comp rx=0 tx=0).stop 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.320+0000 7ff9d3448640 1 -- 10.20.193.28:0/931089338 shutdown_connections 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.320+0000 7ff9d3448640 1 --2- 10.20.193.28:0/931089338 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff9cc115880 0x7ff9cc176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.320+0000 7ff9d3448640 1 --2- 10.20.193.28:0/931089338 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9cc110080 0x7ff9cc1150d0 unknown :-1 s=CLOSED pgs=256 gs=1 cs=0 l=1 c_cookie=12771058586736545011 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.320+0000 7ff9d3448640 1 --2- 10.20.193.28:0/931089338 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff9cc10f690 0x7ff9cc10fa60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.320+0000 7ff9d3448640 1 -- 10.20.193.28:0/931089338 >> 10.20.193.28:0/931089338 conn(0x7ff9cc0ecf00 msgr2=0x7ff9cc10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.320+0000 7ff9d3448640 1 -- 10.20.193.28:0/931089338 shutdown_connections 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.320+0000 7ff9d3448640 1 -- 10.20.193.28:0/931089338 wait complete. 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.321+0000 7ff9d3448640 1 Processor -- start 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.321+0000 7ff9d3448640 1 -- start start 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.321+0000 7ff9d3448640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff9cc10f690 0x7ff9cc0f5a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.321+0000 7ff9d3448640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff9cc110080 0x7ff9cc176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:25.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.321+0000 7ff9d3448640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9cc115880 0x7ff9cc0f6120 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.321+0000 7ff9d3448640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc0fb330 con 0x7ff9cc115880 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.321+0000 7ff9d3448640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc0fb4a0 con 0x7ff9cc110080 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.321+0000 7ff9d3448640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc0fb660 con 0x7ff9cc10f690 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9d2c47640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9cc115880 0x7ff9cc0f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9d2c47640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9cc115880 0x7ff9cc0f6120 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34186/0 (socket says 10.20.193.28:34186) 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9d2c47640 1 -- 10.20.193.28:0/3598746608 learned_addr learned my addr 10.20.193.28:0/3598746608 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9d2446640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff9cc10f690 0x7ff9cc0f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9d1c45640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff9cc110080 0x7ff9cc176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9d2c47640 1 -- 10.20.193.28:0/3598746608 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff9cc10f690 msgr2=0x7ff9cc0f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9d2c47640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff9cc10f690 0x7ff9cc0f5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9d2c47640 1 -- 10.20.193.28:0/3598746608 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff9cc110080 msgr2=0x7ff9cc176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9d2c47640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff9cc110080 0x7ff9cc176150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9d2c47640 1 -- 10.20.193.28:0/3598746608 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c0009570 con 0x7ff9cc115880 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9d2c47640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9cc115880 0x7ff9cc0f6120 secure :-1 s=READY pgs=257 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff9c800f8d0 tx=0x7ff9c800fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9b37fe640 1 -- 10.20.193.28:0/3598746608 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff9c800dce0 con 0x7ff9cc115880 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9b37fe640 1 -- 10.20.193.28:0/3598746608 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff9c800de40 con 0x7ff9cc115880 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9b37fe640 1 -- 10.20.193.28:0/3598746608 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff9c80116b0 con 0x7ff9cc115880 2026-02-25T11:47:25.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9d3448640 1 -- 10.20.193.28:0/3598746608 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9cc0fb940 con 0x7ff9cc115880 2026-02-25T11:47:25.324 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.322+0000 7ff9d3448640 1 -- 10.20.193.28:0/3598746608 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9cc0fbdc0 con 0x7ff9cc115880 2026-02-25T11:47:25.324 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.323+0000 7ff9d3448640 1 -- 10.20.193.28:0/3598746608 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9cc110970 con 0x7ff9cc115880 2026-02-25T11:47:25.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.325+0000 7ff9b37fe640 1 -- 10.20.193.28:0/3598746608 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7ff9c8020060 con 0x7ff9cc115880 2026-02-25T11:47:25.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.327+0000 7ff9b37fe640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff9a80b5760 0x7ff9a80d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:25.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.327+0000 7ff9b37fe640 1 -- 10.20.193.28:0/3598746608 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7ff9c8015070 con 0x7ff9cc115880 2026-02-25T11:47:25.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.327+0000 7ff9b37fe640 1 -- 10.20.193.28:0/3598746608 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7ff9c8120960 con 0x7ff9cc115880 2026-02-25T11:47:25.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.327+0000 7ff9d2446640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff9a80b5760 0x7ff9a80d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:25.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.327+0000 7ff9d2446640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff9a80b5760 0x7ff9a80d5c10 secure :-1 s=READY pgs=188 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff9bc004740 tx=0x7ff9bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:25.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.443+0000 7ff9d3448640 1 -- 10.20.193.28:0/3598746608 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7ff9cc0f6b80 con 0x7ff9cc115880 2026-02-25T11:47:25.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.444+0000 7ff9b37fe640 1 -- 10.20.193.28:0/3598746608 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v46) ==== 86+0+133 (secure 0 0 0) 0x7ff9c80970a0 con 0x7ff9cc115880 2026-02-25T11:47:25.446 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:25.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.445+0000 7ff9d3448640 1 -- 10.20.193.28:0/3598746608 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff9a80b5760 msgr2=0x7ff9a80d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:25.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.445+0000 7ff9d3448640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff9a80b5760 0x7ff9a80d5c10 secure :-1 s=READY pgs=188 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff9bc004740 tx=0x7ff9bc005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:25.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.445+0000 7ff9d3448640 1 -- 10.20.193.28:0/3598746608 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9cc115880 msgr2=0x7ff9cc0f6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:25.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.445+0000 7ff9d3448640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9cc115880 0x7ff9cc0f6120 secure :-1 s=READY pgs=257 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff9c800f8d0 tx=0x7ff9c800fda0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.445+0000 7ff9d3448640 1 -- 10.20.193.28:0/3598746608 shutdown_connections 2026-02-25T11:47:25.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.445+0000 7ff9d3448640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff9a80b5760 0x7ff9a80d5c10 unknown :-1 s=CLOSED pgs=188 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.445+0000 7ff9d3448640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9cc115880 0x7ff9cc0f6120 unknown :-1 s=CLOSED pgs=257 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.445+0000 7ff9d3448640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff9cc110080 0x7ff9cc176150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.445+0000 7ff9d3448640 1 --2- 10.20.193.28:0/3598746608 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff9cc10f690 0x7ff9cc0f5a20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.445+0000 7ff9d3448640 1 -- 10.20.193.28:0/3598746608 >> 10.20.193.28:0/3598746608 conn(0x7ff9cc0ecf00 msgr2=0x7ff9cc111e90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:25.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.445+0000 7ff9d3448640 1 -- 10.20.193.28:0/3598746608 shutdown_connections 2026-02-25T11:47:25.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.445+0000 7ff9d3448640 1 -- 10.20.193.28:0/3598746608 wait complete. 2026-02-25T11:47:25.546 INFO:teuthology.orchestra.run.trial028.stdout:[{"version":46,"timestamp":"2026-02-25T11:47:07.186991+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2026-02-25T11:47:25.547 INFO:tasks.ceph_manager:config epoch is 46 2026-02-25T11:47:25.547 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-25T11:47:25.547 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2026-02-25T11:47:25.547 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph mgr dump --format=json 2026-02-25T11:47:25.731 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:25.774 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:25 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1690485396' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2026-02-25T11:47:25.775 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:25 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1690485396' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2026-02-25T11:47:25.775 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:25 trial028 ceph-mon[39224]: pgmap v200: 97 pgs: 97 active+clean; 628 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 7.7 KiB/s rd, 9.5 KiB/s wr, 18 op/s 2026-02-25T11:47:25.775 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:25 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3598746608' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2026-02-25T11:47:25.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.907+0000 7f3da0658640 1 -- 10.20.193.28:0/1033963415 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d9810c450 msgr2=0x7f3d9810c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:25.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.907+0000 7f3da0658640 1 --2- 10.20.193.28:0/1033963415 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d9810c450 0x7f3d9810c820 secure :-1 s=READY pgs=258 gs=1 cs=0 l=1 c_cookie=10386222813141263541 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d88009950 tx=0x7f3d8802f1d0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.907+0000 7f3da0658640 1 -- 10.20.193.28:0/1033963415 shutdown_connections 2026-02-25T11:47:25.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.907+0000 7f3da0658640 1 --2- 10.20.193.28:0/1033963415 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3d98112640 0x7f3d98173010 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.907+0000 7f3da0658640 1 --2- 10.20.193.28:0/1033963415 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3d9810ce40 0x7f3d98111e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.907+0000 7f3da0658640 1 --2- 10.20.193.28:0/1033963415 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d9810c450 0x7f3d9810c820 unknown :-1 s=CLOSED pgs=258 gs=1 cs=0 l=1 c_cookie=10386222813141263541 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.907+0000 7f3da0658640 1 -- 10.20.193.28:0/1033963415 >> 10.20.193.28:0/1033963415 conn(0x7f3d980e9d40 msgr2=0x7f3d9810a150 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:25.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.907+0000 7f3da0658640 1 -- 10.20.193.28:0/1033963415 shutdown_connections 2026-02-25T11:47:25.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.907+0000 7f3da0658640 1 -- 10.20.193.28:0/1033963415 wait complete. 2026-02-25T11:47:25.910 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.907+0000 7f3da0658640 1 Processor -- start 2026-02-25T11:47:25.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3da0658640 1 -- start start 2026-02-25T11:47:25.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3da0658640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3d9810c450 0x7f3d980f4d60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:25.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3da0658640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3d9810ce40 0x7f3d98172f10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:25.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3da0658640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d98112640 0x7f3d980f5460 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:25.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3da0658640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d980fa670 con 0x7f3d98112640 2026-02-25T11:47:25.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3da0658640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3d980fa7e0 con 0x7f3d9810ce40 2026-02-25T11:47:25.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3da0658640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d980fa9a0 con 0x7f3d9810c450 2026-02-25T11:47:25.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3d9ebce640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d98112640 0x7f3d980f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:25.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3d9ebce640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d98112640 0x7f3d980f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40570/0 (socket says 10.20.193.28:40570) 2026-02-25T11:47:25.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3d9ebce640 1 -- 10.20.193.28:0/2232770096 learned_addr learned my addr 10.20.193.28:0/2232770096 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:25.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3d9e3cd640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3d9810c450 0x7f3d980f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:25.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3d9dbcc640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3d9810ce40 0x7f3d98172f10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:25.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3d9ebce640 1 -- 10.20.193.28:0/2232770096 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3d9810c450 msgr2=0x7f3d980f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:25.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3d9ebce640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3d9810c450 0x7f3d980f4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3d9ebce640 1 -- 10.20.193.28:0/2232770096 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3d9810ce40 msgr2=0x7f3d98172f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:25.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3d9ebce640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3d9810ce40 0x7f3d98172f10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:25.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3d9ebce640 1 -- 10.20.193.28:0/2232770096 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d88009570 con 0x7f3d98112640 2026-02-25T11:47:25.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.908+0000 7f3d9dbcc640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3d9810ce40 0x7f3d98172f10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:25.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.909+0000 7f3d9ebce640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d98112640 0x7f3d980f5460 secure :-1 s=READY pgs=259 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d94006e90 tx=0x7f3d9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:25.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.909+0000 7f3d7f7fe640 1 -- 10.20.193.28:0/2232770096 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3d94005500 con 0x7f3d98112640 2026-02-25T11:47:25.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.909+0000 7f3d7f7fe640 1 -- 10.20.193.28:0/2232770096 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3d94005df0 con 0x7f3d98112640 2026-02-25T11:47:25.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.909+0000 7f3d7f7fe640 1 -- 10.20.193.28:0/2232770096 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3d94003a00 con 0x7f3d98112640 2026-02-25T11:47:25.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.909+0000 7f3da0658640 1 -- 10.20.193.28:0/2232770096 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d980fac80 con 0x7f3d98112640 2026-02-25T11:47:25.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.909+0000 7f3da0658640 1 -- 10.20.193.28:0/2232770096 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d980fb100 con 0x7f3d98112640 2026-02-25T11:47:25.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.909+0000 7f3da0658640 1 -- 10.20.193.28:0/2232770096 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9810d730 con 0x7f3d98112640 2026-02-25T11:47:25.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.911+0000 7f3d7f7fe640 1 -- 10.20.193.28:0/2232770096 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f3d94003b60 con 0x7f3d98112640 2026-02-25T11:47:25.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.913+0000 7f3d7f7fe640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3d740b57e0 0x7f3d740d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:25.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.913+0000 7f3d7f7fe640 1 -- 10.20.193.28:0/2232770096 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f3d940e7c20 con 0x7f3d98112640 2026-02-25T11:47:25.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.913+0000 7f3d7f7fe640 1 -- 10.20.193.28:0/2232770096 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f3d94120940 con 0x7f3d98112640 2026-02-25T11:47:25.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.913+0000 7f3d9e3cd640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3d740b57e0 0x7f3d740d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:25.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:25.914+0000 7f3d9e3cd640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3d740b57e0 0x7f3d740d5c90 secure :-1 s=READY pgs=189 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d880098a0 tx=0x7f3d8803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:25.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:25 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1690485396' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2026-02-25T11:47:25.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:25 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1690485396' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2026-02-25T11:47:26.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:25 trial167 ceph-mon[43392]: pgmap v200: 97 pgs: 97 active+clean; 628 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 7.7 KiB/s rd, 9.5 KiB/s wr, 18 op/s 2026-02-25T11:47:26.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:25 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3598746608' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2026-02-25T11:47:26.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:25 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1690485396' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2026-02-25T11:47:26.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:25 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1690485396' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2026-02-25T11:47:26.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:25 trial119 ceph-mon[42710]: pgmap v200: 97 pgs: 97 active+clean; 628 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 7.7 KiB/s rd, 9.5 KiB/s wr, 18 op/s 2026-02-25T11:47:26.010 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:25 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3598746608' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2026-02-25T11:47:26.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.058+0000 7f3da0658640 1 -- 10.20.193.28:0/2232770096 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f3d980f5ef0 con 0x7f3d98112640 2026-02-25T11:47:26.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.062+0000 7f3d7f7fe640 1 -- 10.20.193.28:0/2232770096 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) ==== 74+0+297629 (secure 0 0 0) 0x7f3d940aa570 con 0x7f3d98112640 2026-02-25T11:47:26.069 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:26.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.065+0000 7f3da0658640 1 -- 10.20.193.28:0/2232770096 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3d740b57e0 msgr2=0x7f3d740d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:26.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.065+0000 7f3da0658640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3d740b57e0 0x7f3d740d5c90 secure :-1 s=READY pgs=189 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d880098a0 tx=0x7f3d8803a040 comp rx=0 tx=0).stop 2026-02-25T11:47:26.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.065+0000 7f3da0658640 1 -- 10.20.193.28:0/2232770096 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d98112640 msgr2=0x7f3d980f5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:26.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.065+0000 7f3da0658640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d98112640 0x7f3d980f5460 secure :-1 s=READY pgs=259 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d94006e90 tx=0x7f3d9400edf0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.066+0000 7f3da0658640 1 -- 10.20.193.28:0/2232770096 shutdown_connections 2026-02-25T11:47:26.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.066+0000 7f3da0658640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3d740b57e0 0x7f3d740d5c90 unknown :-1 s=CLOSED pgs=189 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.066+0000 7f3da0658640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d98112640 0x7f3d980f5460 unknown :-1 s=CLOSED pgs=259 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.066+0000 7f3da0658640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3d9810ce40 0x7f3d98172f10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.066+0000 7f3da0658640 1 --2- 10.20.193.28:0/2232770096 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3d9810c450 0x7f3d980f4d60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.066+0000 7f3da0658640 1 -- 10.20.193.28:0/2232770096 >> 10.20.193.28:0/2232770096 conn(0x7f3d980e9d40 msgr2=0x7f3d9810e890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:26.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.066+0000 7f3da0658640 1 -- 10.20.193.28:0/2232770096 shutdown_connections 2026-02-25T11:47:26.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.066+0000 7f3da0658640 1 -- 10.20.193.28:0/2232770096 wait complete. 2026-02-25T11:47:26.168 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":16,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6800","nonce":3140694734},{"type":"v1","addr":"10.20.193.28:6801","nonce":3140694734}]},"active_addr":"10.20.193.28:6801/3140694734","active_change":"2026-02-25T11:43:17.037590+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[{"gid":24152,"name":"z","mgr_features":4544132024016699391,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.7","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24233,"name":"y","mgr_features":4544132024016699391,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.7","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.7","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://10.20.193.28:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.28:0","nonce":628429394}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.28:0","nonce":825682998}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.28:0","nonce":1871763065}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.28:0","nonce":1120515357}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.28:0","nonce":1694391857}]}]} 2026-02-25T11:47:26.179 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2026-02-25T11:47:26.179 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2026-02-25T11:47:26.180 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd dump --format=json 2026-02-25T11:47:26.362 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:26.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.538+0000 7f0f5c66c640 1 -- 10.20.193.28:0/981588781 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0f54110230 msgr2=0x7f0f54115280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:26.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.538+0000 7f0f5c66c640 1 --2- 10.20.193.28:0/981588781 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0f54110230 0x7f0f54115280 secure :-1 s=READY pgs=260 gs=1 cs=0 l=1 c_cookie=12552782324204336905 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0f48009950 tx=0x7f0f4802f190 comp rx=0 tx=0).stop 2026-02-25T11:47:26.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.538+0000 7f0f5c66c640 1 -- 10.20.193.28:0/981588781 shutdown_connections 2026-02-25T11:47:26.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.538+0000 7f0f5c66c640 1 --2- 10.20.193.28:0/981588781 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0f54115a30 0x7f0f54176400 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.538+0000 7f0f5c66c640 1 --2- 10.20.193.28:0/981588781 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0f54110230 0x7f0f54115280 unknown :-1 s=CLOSED pgs=260 gs=1 cs=0 l=1 c_cookie=12552782324204336905 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.538+0000 7f0f5c66c640 1 --2- 10.20.193.28:0/981588781 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0f5410f840 0x7f0f5410fc10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.538+0000 7f0f5c66c640 1 -- 10.20.193.28:0/981588781 >> 10.20.193.28:0/981588781 conn(0x7f0f540ed130 msgr2=0x7f0f5410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:26.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.539+0000 7f0f5c66c640 1 -- 10.20.193.28:0/981588781 shutdown_connections 2026-02-25T11:47:26.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.539+0000 7f0f5c66c640 1 -- 10.20.193.28:0/981588781 wait complete. 2026-02-25T11:47:26.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.539+0000 7f0f5c66c640 1 Processor -- start 2026-02-25T11:47:26.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.539+0000 7f0f5c66c640 1 -- start start 2026-02-25T11:47:26.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5c66c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0f5410f840 0x7f0f540fa150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:26.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5c66c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0f54110230 0x7f0f54176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:26.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5c66c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0f54115a30 0x7f0f540fa850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:26.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5c66c640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0f540ffa10 con 0x7f0f5410f840 2026-02-25T11:47:26.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5c66c640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0f540ffb80 con 0x7f0f54115a30 2026-02-25T11:47:26.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5c66c640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0f540ffd40 con 0x7f0f54110230 2026-02-25T11:47:26.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5a3e1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0f5410f840 0x7f0f540fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:26.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5a3e1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0f5410f840 0x7f0f540fa150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40592/0 (socket says 10.20.193.28:40592) 2026-02-25T11:47:26.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5a3e1640 1 -- 10.20.193.28:0/2281946853 learned_addr learned my addr 10.20.193.28:0/2281946853 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:26.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5abe2640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0f54115a30 0x7f0f540fa850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:26.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f59be0640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0f54110230 0x7f0f54176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:26.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5a3e1640 1 -- 10.20.193.28:0/2281946853 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0f54110230 msgr2=0x7f0f54176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:26.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5a3e1640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0f54110230 0x7f0f54176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5a3e1640 1 -- 10.20.193.28:0/2281946853 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0f54115a30 msgr2=0x7f0f540fa850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:26.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5a3e1640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0f54115a30 0x7f0f540fa850 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5a3e1640 1 -- 10.20.193.28:0/2281946853 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f48009570 con 0x7f0f5410f840 2026-02-25T11:47:26.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f59be0640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0f54110230 0x7f0f54176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:26.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.540+0000 7f0f5a3e1640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0f5410f840 0x7f0f540fa150 secure :-1 s=READY pgs=261 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0f44005db0 tx=0x7f0f4400ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:26.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.541+0000 7f0f3b7fe640 1 -- 10.20.193.28:0/2281946853 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0f44004500 con 0x7f0f5410f840 2026-02-25T11:47:26.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.541+0000 7f0f3b7fe640 1 -- 10.20.193.28:0/2281946853 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0f44004d10 con 0x7f0f5410f840 2026-02-25T11:47:26.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.541+0000 7f0f3b7fe640 1 -- 10.20.193.28:0/2281946853 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0f44005280 con 0x7f0f5410f840 2026-02-25T11:47:26.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.541+0000 7f0f5c66c640 1 -- 10.20.193.28:0/2281946853 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f54100020 con 0x7f0f5410f840 2026-02-25T11:47:26.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.541+0000 7f0f5c66c640 1 -- 10.20.193.28:0/2281946853 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f541004a0 con 0x7f0f5410f840 2026-02-25T11:47:26.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.541+0000 7f0f5c66c640 1 -- 10.20.193.28:0/2281946853 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f24005300 con 0x7f0f5410f840 2026-02-25T11:47:26.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.543+0000 7f0f3b7fe640 1 -- 10.20.193.28:0/2281946853 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f0f44005410 con 0x7f0f5410f840 2026-02-25T11:47:26.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.545+0000 7f0f3b7fe640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0f280b57e0 0x7f0f280d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:26.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.545+0000 7f0f3b7fe640 1 -- 10.20.193.28:0/2281946853 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f0f440e70a0 con 0x7f0f5410f840 2026-02-25T11:47:26.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.545+0000 7f0f59be0640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0f280b57e0 0x7f0f280d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:26.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.545+0000 7f0f3b7fe640 1 -- 10.20.193.28:0/2281946853 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f0f440e7520 con 0x7f0f5410f840 2026-02-25T11:47:26.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.546+0000 7f0f59be0640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0f280b57e0 0x7f0f280d5c90 secure :-1 s=READY pgs=190 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0f480098a0 tx=0x7f0f48031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:26.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.661+0000 7f0f5c66c640 1 -- 10.20.193.28:0/2281946853 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f24005880 con 0x7f0f5410f840 2026-02-25T11:47:26.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.662+0000 7f0f3b7fe640 1 -- 10.20.193.28:0/2281946853 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22340 (secure 0 0 0) 0x7f0f440a99f0 con 0x7f0f5410f840 2026-02-25T11:47:26.664 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:26.664 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":86,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","created":"2026-02-25T11:42:51.962844+0000","modified":"2026-02-25T11:47:14.624486+0000","last_up_change":"2026-02-25T11:46:45.156629+0000","last_in_change":"2026-02-25T11:46:31.585002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":38,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T11:44:43.266336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-25T11:47:11.236575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-25T11:47:12.196775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf596df7-d1fb-4d10-8d4c-14a35210bab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6803","nonce":668944205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6805","nonce":668944205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6809","nonce":668944205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6807","nonce":668944205}]},"public_addr":"10.20.193.28:6803/668944205","cluster_addr":"10.20.193.28:6805/668944205","heartbeat_back_addr":"10.20.193.28:6809/668944205","heartbeat_front_addr":"10.20.193.28:6807/668944205","state":["exists","up"]},{"osd":1,"uuid":"185719c0-7c2d-431f-9925-88bbee86a98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6800","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6801","nonce":3486566242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6802","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6803","nonce":3486566242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6806","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6807","nonce":3486566242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6804","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6805","nonce":3486566242}]},"public_addr":"10.20.193.119:6801/3486566242","cluster_addr":"10.20.193.119:6803/3486566242","heartbeat_back_addr":"10.20.193.119:6807/3486566242","heartbeat_front_addr":"10.20.193.119:6805/3486566242","state":["exists","up"]},{"osd":2,"uuid":"4de13b90-1c3c-4614-9fef-2d7315626433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6800","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6801","nonce":484781559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6802","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6803","nonce":484781559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6806","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6807","nonce":484781559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6804","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6805","nonce":484781559}]},"public_addr":"10.20.193.167:6801/484781559","cluster_addr":"10.20.193.167:6803/484781559","heartbeat_back_addr":"10.20.193.167:6807/484781559","heartbeat_front_addr":"10.20.193.167:6805/484781559","state":["exists","up"]},{"osd":3,"uuid":"45d9ea8e-5a77-4738-96da-ada98515fc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6811","nonce":2452541436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6813","nonce":2452541436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6817","nonce":2452541436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6815","nonce":2452541436}]},"public_addr":"10.20.193.28:6811/2452541436","cluster_addr":"10.20.193.28:6813/2452541436","heartbeat_back_addr":"10.20.193.28:6817/2452541436","heartbeat_front_addr":"10.20.193.28:6815/2452541436","state":["exists","up"]},{"osd":4,"uuid":"c9c3f512-665b-44b2-a6f0-f3356a4b661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6808","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6809","nonce":135971996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6810","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6811","nonce":135971996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6814","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6815","nonce":135971996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6812","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6813","nonce":135971996}]},"public_addr":"10.20.193.119:6809/135971996","cluster_addr":"10.20.193.119:6811/135971996","heartbeat_back_addr":"10.20.193.119:6815/135971996","heartbeat_front_addr":"10.20.193.119:6813/135971996","state":["exists","up"]},{"osd":5,"uuid":"777c3819-5c90-4b2b-a545-c024884c2eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6808","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6809","nonce":4203420884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6810","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6811","nonce":4203420884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6814","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6815","nonce":4203420884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6812","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6813","nonce":4203420884}]},"public_addr":"10.20.193.167:6809/4203420884","cluster_addr":"10.20.193.167:6811/4203420884","heartbeat_back_addr":"10.20.193.167:6815/4203420884","heartbeat_front_addr":"10.20.193.167:6813/4203420884","state":["exists","up"]},{"osd":6,"uuid":"bbd9de64-b9fe-499e-b451-c16828be2c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6819","nonce":621484999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6821","nonce":621484999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6825","nonce":621484999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6823","nonce":621484999}]},"public_addr":"10.20.193.28:6819/621484999","cluster_addr":"10.20.193.28:6821/621484999","heartbeat_back_addr":"10.20.193.28:6825/621484999","heartbeat_front_addr":"10.20.193.28:6823/621484999","state":["exists","up"]},{"osd":7,"uuid":"546d9f28-597b-454c-8456-fcc2d46c69f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6816","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6817","nonce":2724541747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6818","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6819","nonce":2724541747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6822","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6823","nonce":2724541747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6820","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6821","nonce":2724541747}]},"public_addr":"10.20.193.119:6817/2724541747","cluster_addr":"10.20.193.119:6819/2724541747","heartbeat_back_addr":"10.20.193.119:6823/2724541747","heartbeat_front_addr":"10.20.193.119:6821/2724541747","state":["exists","up"]},{"osd":8,"uuid":"a1a7cea6-3586-4bb2-aa4c-bf6f7799a501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6816","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6817","nonce":1328734317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6818","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6819","nonce":1328734317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6822","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6823","nonce":1328734317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6820","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6821","nonce":1328734317}]},"public_addr":"10.20.193.167:6817/1328734317","cluster_addr":"10.20.193.167:6819/1328734317","heartbeat_back_addr":"10.20.193.167:6823/1328734317","heartbeat_front_addr":"10.20.193.167:6821/1328734317","state":["exists","up"]},{"osd":9,"uuid":"03fef134-6946-43d5-9c26-dc0e59fffe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6827","nonce":354500625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6829","nonce":354500625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6833","nonce":354500625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6831","nonce":354500625}]},"public_addr":"10.20.193.28:6827/354500625","cluster_addr":"10.20.193.28:6829/354500625","heartbeat_back_addr":"10.20.193.28:6833/354500625","heartbeat_front_addr":"10.20.193.28:6831/354500625","state":["exists","up"]},{"osd":10,"uuid":"b46c29ef-285c-4184-be28-d99b76bc0e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6824","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6825","nonce":2996581116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6826","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6827","nonce":2996581116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6830","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6831","nonce":2996581116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6828","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6829","nonce":2996581116}]},"public_addr":"10.20.193.119:6825/2996581116","cluster_addr":"10.20.193.119:6827/2996581116","heartbeat_back_addr":"10.20.193.119:6831/2996581116","heartbeat_front_addr":"10.20.193.119:6829/2996581116","state":["exists","up"]},{"osd":11,"uuid":"f6696d8c-5de7-4fd9-b693-524f204e4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6824","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6825","nonce":2182906700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6826","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6827","nonce":2182906700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6830","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6831","nonce":2182906700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6828","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6829","nonce":2182906700}]},"public_addr":"10.20.193.167:6825/2182906700","cluster_addr":"10.20.193.167:6827/2182906700","heartbeat_back_addr":"10.20.193.167:6831/2182906700","heartbeat_front_addr":"10.20.193.167:6829/2182906700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:13.410235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:27.254773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:40.956615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:54.529866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:08.306772+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:21.899498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:35.500407+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:49.094965+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:02.790138+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:16.286937+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:29.922722+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:43.396028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.28:0/1467902196":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6800/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/2274270913":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/60087422":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:6800/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/1725332841":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/3082630924":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/2391715078":"2026-02-26T11:43:04.102029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T11:47:26.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.663+0000 7f0f5c66c640 1 -- 10.20.193.28:0/2281946853 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0f280b57e0 msgr2=0x7f0f280d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:26.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.663+0000 7f0f5c66c640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0f280b57e0 0x7f0f280d5c90 secure :-1 s=READY pgs=190 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0f480098a0 tx=0x7f0f48031040 comp rx=0 tx=0).stop 2026-02-25T11:47:26.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.663+0000 7f0f5c66c640 1 -- 10.20.193.28:0/2281946853 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0f5410f840 msgr2=0x7f0f540fa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:26.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.663+0000 7f0f5c66c640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0f5410f840 0x7f0f540fa150 secure :-1 s=READY pgs=261 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0f44005db0 tx=0x7f0f4400ddf0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.663+0000 7f0f5c66c640 1 -- 10.20.193.28:0/2281946853 shutdown_connections 2026-02-25T11:47:26.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.663+0000 7f0f5c66c640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0f280b57e0 0x7f0f280d5c90 unknown :-1 s=CLOSED pgs=190 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.663+0000 7f0f5c66c640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0f54115a30 0x7f0f540fa850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.663+0000 7f0f5c66c640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0f54110230 0x7f0f54176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.663+0000 7f0f5c66c640 1 --2- 10.20.193.28:0/2281946853 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0f5410f840 0x7f0f540fa150 unknown :-1 s=CLOSED pgs=261 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:26.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.663+0000 7f0f5c66c640 1 -- 10.20.193.28:0/2281946853 >> 10.20.193.28:0/2281946853 conn(0x7f0f540ed130 msgr2=0x7f0f54112040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:26.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.664+0000 7f0f5c66c640 1 -- 10.20.193.28:0/2281946853 shutdown_connections 2026-02-25T11:47:26.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:26.664+0000 7f0f5c66c640 1 -- 10.20.193.28:0/2281946853 wait complete. 2026-02-25T11:47:26.766 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2026-02-25T11:47:26.767 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd dump --format=json 2026-02-25T11:47:26.950 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:26.993 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:26 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2232770096' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2026-02-25T11:47:26.993 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:26 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2281946853' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:26.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:26 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2232770096' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2026-02-25T11:47:26.999 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:26 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2281946853' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:27.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:26 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2232770096' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2026-02-25T11:47:27.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:26 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2281946853' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.125+0000 7faae50ba640 1 -- 10.20.193.28:0/1162960577 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faae0110230 msgr2=0x7faae0115280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.125+0000 7faae50ba640 1 --2- 10.20.193.28:0/1162960577 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faae0110230 0x7faae0115280 secure :-1 s=READY pgs=82 gs=1 cs=0 l=1 c_cookie=14970373937243747267 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faac8009890 tx=0x7faac802f120 comp rx=0 tx=0).stop 2026-02-25T11:47:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.125+0000 7faae50ba640 1 -- 10.20.193.28:0/1162960577 shutdown_connections 2026-02-25T11:47:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.125+0000 7faae50ba640 1 --2- 10.20.193.28:0/1162960577 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7faae0115a30 0x7faae0176400 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.125+0000 7faae50ba640 1 --2- 10.20.193.28:0/1162960577 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faae0110230 0x7faae0115280 unknown :-1 s=CLOSED pgs=82 gs=1 cs=0 l=1 c_cookie=14970373937243747267 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.125+0000 7faae50ba640 1 --2- 10.20.193.28:0/1162960577 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faae010f840 0x7faae010fc10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.125+0000 7faae50ba640 1 -- 10.20.193.28:0/1162960577 >> 10.20.193.28:0/1162960577 conn(0x7faae00ed130 msgr2=0x7faae010d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.125+0000 7faae50ba640 1 -- 10.20.193.28:0/1162960577 shutdown_connections 2026-02-25T11:47:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.125+0000 7faae50ba640 1 -- 10.20.193.28:0/1162960577 wait complete. 2026-02-25T11:47:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.126+0000 7faae50ba640 1 Processor -- start 2026-02-25T11:47:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.126+0000 7faae50ba640 1 -- start start 2026-02-25T11:47:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.126+0000 7faae50ba640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faae010f840 0x7faae00f5b60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.126+0000 7faae50ba640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7faae0110230 0x7faae0176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.126+0000 7faae50ba640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faae0115a30 0x7faae00f6260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.126+0000 7faae50ba640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7faae00fb470 con 0x7faae010f840 2026-02-25T11:47:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.126+0000 7faae50ba640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7faae00fb5e0 con 0x7faae0110230 2026-02-25T11:47:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.126+0000 7faae50ba640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7faae00fb7a0 con 0x7faae0115a30 2026-02-25T11:47:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faaded76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faae010f840 0x7faae00f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faaded76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faae010f840 0x7faae00f5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40616/0 (socket says 10.20.193.28:40616) 2026-02-25T11:47:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faaded76640 1 -- 10.20.193.28:0/418693799 learned_addr learned my addr 10.20.193.28:0/418693799 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faade575640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7faae0110230 0x7faae0176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faadf577640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faae0115a30 0x7faae00f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faaded76640 1 -- 10.20.193.28:0/418693799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faae0115a30 msgr2=0x7faae00f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faaded76640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faae0115a30 0x7faae00f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faaded76640 1 -- 10.20.193.28:0/418693799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7faae0110230 msgr2=0x7faae0176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faaded76640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7faae0110230 0x7faae0176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faaded76640 1 -- 10.20.193.28:0/418693799 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad4009570 con 0x7faae010f840 2026-02-25T11:47:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faadf577640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faae0115a30 0x7faae00f6260 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faaded76640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faae010f840 0x7faae00f5b60 secure :-1 s=READY pgs=262 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faad4005b30 tx=0x7faad4002c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faabffff640 1 -- 10.20.193.28:0/418693799 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7faad400ed60 con 0x7faae010f840 2026-02-25T11:47:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faabffff640 1 -- 10.20.193.28:0/418693799 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7faad4002e70 con 0x7faae010f840 2026-02-25T11:47:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faabffff640 1 -- 10.20.193.28:0/418693799 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7faad400f7b0 con 0x7faae010f840 2026-02-25T11:47:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faae50ba640 1 -- 10.20.193.28:0/418693799 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac80094b0 con 0x7faae010f840 2026-02-25T11:47:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.127+0000 7faae50ba640 1 -- 10.20.193.28:0/418693799 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae00fbd70 con 0x7faae010f840 2026-02-25T11:47:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.128+0000 7faae50ba640 1 -- 10.20.193.28:0/418693799 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae0110b20 con 0x7faae010f840 2026-02-25T11:47:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.130+0000 7faabffff640 1 -- 10.20.193.28:0/418693799 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7faad401f060 con 0x7faae010f840 2026-02-25T11:47:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.131+0000 7faabffff640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7faab40b57e0 0x7faab40d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.131+0000 7faabffff640 1 -- 10.20.193.28:0/418693799 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7faad40e71f0 con 0x7faae010f840 2026-02-25T11:47:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.132+0000 7faade575640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7faab40b57e0 0x7faab40d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.132+0000 7faabffff640 1 -- 10.20.193.28:0/418693799 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7faad40e7670 con 0x7faae010f840 2026-02-25T11:47:27.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.132+0000 7faade575640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7faab40b57e0 0x7faab40d5c90 secure :-1 s=READY pgs=191 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faac80097e0 tx=0x7faac803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.248+0000 7faae50ba640 1 -- 10.20.193.28:0/418693799 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faae010fc10 con 0x7faae010f840 2026-02-25T11:47:27.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.249+0000 7faabffff640 1 -- 10.20.193.28:0/418693799 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22340 (secure 0 0 0) 0x7faad40a9b40 con 0x7faae010f840 2026-02-25T11:47:27.250 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:27.251 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":86,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","created":"2026-02-25T11:42:51.962844+0000","modified":"2026-02-25T11:47:14.624486+0000","last_up_change":"2026-02-25T11:46:45.156629+0000","last_in_change":"2026-02-25T11:46:31.585002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":38,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T11:44:43.266336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-25T11:47:11.236575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-25T11:47:12.196775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf596df7-d1fb-4d10-8d4c-14a35210bab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6803","nonce":668944205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6805","nonce":668944205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6809","nonce":668944205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6807","nonce":668944205}]},"public_addr":"10.20.193.28:6803/668944205","cluster_addr":"10.20.193.28:6805/668944205","heartbeat_back_addr":"10.20.193.28:6809/668944205","heartbeat_front_addr":"10.20.193.28:6807/668944205","state":["exists","up"]},{"osd":1,"uuid":"185719c0-7c2d-431f-9925-88bbee86a98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6800","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6801","nonce":3486566242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6802","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6803","nonce":3486566242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6806","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6807","nonce":3486566242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6804","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6805","nonce":3486566242}]},"public_addr":"10.20.193.119:6801/3486566242","cluster_addr":"10.20.193.119:6803/3486566242","heartbeat_back_addr":"10.20.193.119:6807/3486566242","heartbeat_front_addr":"10.20.193.119:6805/3486566242","state":["exists","up"]},{"osd":2,"uuid":"4de13b90-1c3c-4614-9fef-2d7315626433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6800","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6801","nonce":484781559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6802","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6803","nonce":484781559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6806","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6807","nonce":484781559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6804","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6805","nonce":484781559}]},"public_addr":"10.20.193.167:6801/484781559","cluster_addr":"10.20.193.167:6803/484781559","heartbeat_back_addr":"10.20.193.167:6807/484781559","heartbeat_front_addr":"10.20.193.167:6805/484781559","state":["exists","up"]},{"osd":3,"uuid":"45d9ea8e-5a77-4738-96da-ada98515fc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6811","nonce":2452541436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6813","nonce":2452541436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6817","nonce":2452541436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6815","nonce":2452541436}]},"public_addr":"10.20.193.28:6811/2452541436","cluster_addr":"10.20.193.28:6813/2452541436","heartbeat_back_addr":"10.20.193.28:6817/2452541436","heartbeat_front_addr":"10.20.193.28:6815/2452541436","state":["exists","up"]},{"osd":4,"uuid":"c9c3f512-665b-44b2-a6f0-f3356a4b661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6808","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6809","nonce":135971996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6810","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6811","nonce":135971996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6814","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6815","nonce":135971996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6812","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6813","nonce":135971996}]},"public_addr":"10.20.193.119:6809/135971996","cluster_addr":"10.20.193.119:6811/135971996","heartbeat_back_addr":"10.20.193.119:6815/135971996","heartbeat_front_addr":"10.20.193.119:6813/135971996","state":["exists","up"]},{"osd":5,"uuid":"777c3819-5c90-4b2b-a545-c024884c2eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6808","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6809","nonce":4203420884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6810","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6811","nonce":4203420884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6814","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6815","nonce":4203420884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6812","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6813","nonce":4203420884}]},"public_addr":"10.20.193.167:6809/4203420884","cluster_addr":"10.20.193.167:6811/4203420884","heartbeat_back_addr":"10.20.193.167:6815/4203420884","heartbeat_front_addr":"10.20.193.167:6813/4203420884","state":["exists","up"]},{"osd":6,"uuid":"bbd9de64-b9fe-499e-b451-c16828be2c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6819","nonce":621484999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6821","nonce":621484999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6825","nonce":621484999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6823","nonce":621484999}]},"public_addr":"10.20.193.28:6819/621484999","cluster_addr":"10.20.193.28:6821/621484999","heartbeat_back_addr":"10.20.193.28:6825/621484999","heartbeat_front_addr":"10.20.193.28:6823/621484999","state":["exists","up"]},{"osd":7,"uuid":"546d9f28-597b-454c-8456-fcc2d46c69f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6816","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6817","nonce":2724541747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6818","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6819","nonce":2724541747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6822","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6823","nonce":2724541747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6820","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6821","nonce":2724541747}]},"public_addr":"10.20.193.119:6817/2724541747","cluster_addr":"10.20.193.119:6819/2724541747","heartbeat_back_addr":"10.20.193.119:6823/2724541747","heartbeat_front_addr":"10.20.193.119:6821/2724541747","state":["exists","up"]},{"osd":8,"uuid":"a1a7cea6-3586-4bb2-aa4c-bf6f7799a501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6816","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6817","nonce":1328734317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6818","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6819","nonce":1328734317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6822","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6823","nonce":1328734317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6820","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6821","nonce":1328734317}]},"public_addr":"10.20.193.167:6817/1328734317","cluster_addr":"10.20.193.167:6819/1328734317","heartbeat_back_addr":"10.20.193.167:6823/1328734317","heartbeat_front_addr":"10.20.193.167:6821/1328734317","state":["exists","up"]},{"osd":9,"uuid":"03fef134-6946-43d5-9c26-dc0e59fffe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6827","nonce":354500625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6829","nonce":354500625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6833","nonce":354500625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6831","nonce":354500625}]},"public_addr":"10.20.193.28:6827/354500625","cluster_addr":"10.20.193.28:6829/354500625","heartbeat_back_addr":"10.20.193.28:6833/354500625","heartbeat_front_addr":"10.20.193.28:6831/354500625","state":["exists","up"]},{"osd":10,"uuid":"b46c29ef-285c-4184-be28-d99b76bc0e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6824","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6825","nonce":2996581116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6826","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6827","nonce":2996581116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6830","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6831","nonce":2996581116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6828","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6829","nonce":2996581116}]},"public_addr":"10.20.193.119:6825/2996581116","cluster_addr":"10.20.193.119:6827/2996581116","heartbeat_back_addr":"10.20.193.119:6831/2996581116","heartbeat_front_addr":"10.20.193.119:6829/2996581116","state":["exists","up"]},{"osd":11,"uuid":"f6696d8c-5de7-4fd9-b693-524f204e4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6824","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6825","nonce":2182906700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6826","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6827","nonce":2182906700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6830","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6831","nonce":2182906700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6828","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6829","nonce":2182906700}]},"public_addr":"10.20.193.167:6825/2182906700","cluster_addr":"10.20.193.167:6827/2182906700","heartbeat_back_addr":"10.20.193.167:6831/2182906700","heartbeat_front_addr":"10.20.193.167:6829/2182906700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:13.410235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:27.254773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:40.956615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:54.529866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:08.306772+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:21.899498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:35.500407+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:49.094965+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:02.790138+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:16.286937+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:29.922722+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:43.396028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.28:0/1467902196":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6800/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/2274270913":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/60087422":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:6800/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/1725332841":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/3082630924":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/2391715078":"2026-02-26T11:43:04.102029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T11:47:27.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.250+0000 7faae50ba640 1 -- 10.20.193.28:0/418693799 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7faab40b57e0 msgr2=0x7faab40d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.250+0000 7faae50ba640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7faab40b57e0 0x7faab40d5c90 secure :-1 s=READY pgs=191 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faac80097e0 tx=0x7faac803a040 comp rx=0 tx=0).stop 2026-02-25T11:47:27.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.250+0000 7faae50ba640 1 -- 10.20.193.28:0/418693799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faae010f840 msgr2=0x7faae00f5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.250+0000 7faae50ba640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faae010f840 0x7faae00f5b60 secure :-1 s=READY pgs=262 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faad4005b30 tx=0x7faad4002c10 comp rx=0 tx=0).stop 2026-02-25T11:47:27.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.250+0000 7faae50ba640 1 -- 10.20.193.28:0/418693799 shutdown_connections 2026-02-25T11:47:27.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.250+0000 7faae50ba640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7faab40b57e0 0x7faab40d5c90 unknown :-1 s=CLOSED pgs=191 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.250+0000 7faae50ba640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faae0115a30 0x7faae00f6260 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.250+0000 7faae50ba640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7faae0110230 0x7faae0176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.250+0000 7faae50ba640 1 --2- 10.20.193.28:0/418693799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faae010f840 0x7faae00f5b60 unknown :-1 s=CLOSED pgs=262 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.250+0000 7faae50ba640 1 -- 10.20.193.28:0/418693799 >> 10.20.193.28:0/418693799 conn(0x7faae00ed130 msgr2=0x7faae0112040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:27.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.250+0000 7faae50ba640 1 -- 10.20.193.28:0/418693799 shutdown_connections 2026-02-25T11:47:27.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.251+0000 7faae50ba640 1 -- 10.20.193.28:0/418693799 wait complete. 2026-02-25T11:47:27.350 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell osd.0 flush_pg_stats 2026-02-25T11:47:27.350 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell osd.1 flush_pg_stats 2026-02-25T11:47:27.351 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell osd.2 flush_pg_stats 2026-02-25T11:47:27.351 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell osd.3 flush_pg_stats 2026-02-25T11:47:27.351 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell osd.4 flush_pg_stats 2026-02-25T11:47:27.352 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell osd.5 flush_pg_stats 2026-02-25T11:47:27.352 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell osd.6 flush_pg_stats 2026-02-25T11:47:27.352 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell osd.7 flush_pg_stats 2026-02-25T11:47:27.352 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell osd.8 flush_pg_stats 2026-02-25T11:47:27.353 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell osd.9 flush_pg_stats 2026-02-25T11:47:27.353 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell osd.10 flush_pg_stats 2026-02-25T11:47:27.353 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell osd.11 flush_pg_stats 2026-02-25T11:47:27.601 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:27.608 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:27.632 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:27.674 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:27.677 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:27.682 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:27.685 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:27.705 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:27.788 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:27.806 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:27.831 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:27.876 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:27.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.878+0000 7fb3af055640 1 -- 10.20.193.28:0/1946519485 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a81152f0 msgr2=0x7fb3a8116940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.878+0000 7fb3af055640 1 --2- 10.20.193.28:0/1946519485 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a81152f0 0x7fb3a8116940 secure :-1 s=READY pgs=263 gs=1 cs=0 l=1 c_cookie=13006403487815303415 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb390009890 tx=0x7fb39002f100 comp rx=0 tx=0).stop 2026-02-25T11:47:27.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.879+0000 7fb3af055640 1 -- 10.20.193.28:0/1946519485 shutdown_connections 2026-02-25T11:47:27.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.879+0000 7fb3af055640 1 --2- 10.20.193.28:0/1946519485 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb3a8117120 0x7fb3a817ace0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.879+0000 7fb3af055640 1 --2- 10.20.193.28:0/1946519485 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a81152f0 0x7fb3a8116940 unknown :-1 s=CLOSED pgs=263 gs=1 cs=0 l=1 c_cookie=13006403487815303415 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.879+0000 7fb3af055640 1 --2- 10.20.193.28:0/1946519485 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb3a8114900 0x7fb3a8114cd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.879+0000 7fb3af055640 1 -- 10.20.193.28:0/1946519485 >> 10.20.193.28:0/1946519485 conn(0x7fb3a8057d30 msgr2=0x7fb3a8058130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:27.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.879+0000 7fb3af055640 1 -- 10.20.193.28:0/1946519485 shutdown_connections 2026-02-25T11:47:27.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.879+0000 7fb3af055640 1 -- 10.20.193.28:0/1946519485 wait complete. 2026-02-25T11:47:27.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.879+0000 7fb3af055640 1 Processor -- start 2026-02-25T11:47:27.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.880+0000 7fb3af055640 1 -- start start 2026-02-25T11:47:27.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.880+0000 7fb3af055640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb3a8114900 0x7fb3a81220a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.880+0000 7fb3af055640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a81152f0 0x7fb3a81225e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb3af055640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb3a8117120 0x7fb3a81279b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb39ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a81152f0 0x7fb3a81225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb39ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a81152f0 0x7fb3a81225e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40642/0 (socket says 10.20.193.28:40642) 2026-02-25T11:47:27.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb39ffff640 1 -- 10.20.193.28:0/2814656582 learned_addr learned my addr 10.20.193.28:0/2814656582 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:27.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a8127fd0 con 0x7fb3a81152f0 2026-02-25T11:47:27.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a8128140 con 0x7fb3a8117120 2026-02-25T11:47:27.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a8128300 con 0x7fb3a8114900 2026-02-25T11:47:27.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb3acdca640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb3a8114900 0x7fb3a81220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb3ad5cb640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb3a8117120 0x7fb3a81279b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb3acdca640 1 -- 10.20.193.28:0/2814656582 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb3a8117120 msgr2=0x7fb3a81279b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb3acdca640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb3a8117120 0x7fb3a81279b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb3acdca640 1 -- 10.20.193.28:0/2814656582 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a81152f0 msgr2=0x7fb3a81225e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb3acdca640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb3a81152f0 0x7fb3a81225e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.881+0000 7fb3acdca640 1 -- 10.20.193.28:0/2814656582 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3900094b0 con 0x7fb3a8114900 2026-02-25T11:47:27.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.883+0000 7fb3acdca640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb3a8114900 0x7fb3a81220a0 secure :-1 s=READY pgs=83 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb398005820 tx=0x7fb39800fe60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.883+0000 7fb39dffb640 1 -- 10.20.193.28:0/2814656582 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb39800de00 con 0x7fb3a8114900 2026-02-25T11:47:27.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.883+0000 7fb39dffb640 1 -- 10.20.193.28:0/2814656582 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb398005df0 con 0x7fb3a8114900 2026-02-25T11:47:27.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.883+0000 7fb39dffb640 1 -- 10.20.193.28:0/2814656582 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb3980116b0 con 0x7fb3a8114900 2026-02-25T11:47:27.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.883+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a81285e0 con 0x7fb3a8114900 2026-02-25T11:47:27.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.884+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a8168ce0 con 0x7fb3a8114900 2026-02-25T11:47:27.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.889+0000 7fb39dffb640 1 -- 10.20.193.28:0/2814656582 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fb398020060 con 0x7fb3a8114900 2026-02-25T11:47:27.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.889+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/3780423785 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb810f840 msgr2=0x7f6bb810fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.889+0000 7f6bbe3f1640 1 --2- 10.20.193.28:0/3780423785 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb810f840 0x7f6bb810fc10 secure :-1 s=READY pgs=84 gs=1 cs=0 l=1 c_cookie=10959494472778981571 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ba8009950 tx=0x7f6ba802f1d0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.890+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/3780423785 shutdown_connections 2026-02-25T11:47:27.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.890+0000 7f6bbe3f1640 1 --2- 10.20.193.28:0/3780423785 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6bb8115a30 0x7f6bb8176400 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.890+0000 7f6bbe3f1640 1 --2- 10.20.193.28:0/3780423785 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6bb8110230 0x7f6bb8115280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.890+0000 7f6bbe3f1640 1 --2- 10.20.193.28:0/3780423785 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb810f840 0x7f6bb810fc10 unknown :-1 s=CLOSED pgs=84 gs=1 cs=0 l=1 c_cookie=10959494472778981571 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.890+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/3780423785 >> 10.20.193.28:0/3780423785 conn(0x7f6bb80ed130 msgr2=0x7f6bb810d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:27.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.891+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/3780423785 shutdown_connections 2026-02-25T11:47:27.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.891+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/3780423785 wait complete. 2026-02-25T11:47:27.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.891+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb370000fd0 con 0x7fb3a8114900 2026-02-25T11:47:27.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.892+0000 7fb39dffb640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb3840b5870 0x7fb3840d5d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.892+0000 7fb39dffb640 1 -- 10.20.193.28:0/2814656582 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fb398015070 con 0x7fb3a8114900 2026-02-25T11:47:27.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.892+0000 7fb39dffb640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996] conn(0x7fb3840dcfe0 0x7fb3840fd3f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.892+0000 7fb39dffb640 1 -- 10.20.193.28:0/2814656582 --> [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3840fdaa0 con 0x7fb3840dcfe0 2026-02-25T11:47:27.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.892+0000 7fb39dffb640 1 -- 10.20.193.28:0/2814656582 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fb3980ec2a0 con 0x7fb3a8114900 2026-02-25T11:47:27.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.892+0000 7fb39ffff640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb3840b5870 0x7fb3840d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.893+0000 7fb3ad5cb640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996] conn(0x7fb3840dcfe0 0x7fb3840fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.893+0000 7fb39ffff640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb3840b5870 0x7fb3840d5d20 secure :-1 s=READY pgs=192 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb3a8128a40 tx=0x7fb390002390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.893+0000 7fb3ad5cb640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996] conn(0x7fb3840dcfe0 0x7fb3840fd3f0 crc :-1 s=READY pgs=11 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.893+0000 7ffa381d4640 1 -- 10.20.193.28:0/3154357665 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa280b6da0 msgr2=0x7ffa280b7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.893+0000 7ffa381d4640 1 --2- 10.20.193.28:0/3154357665 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa280b6da0 0x7ffa280b7170 secure :-1 s=READY pgs=264 gs=3 cs=0 l=1 c_cookie=10036189744117840869 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa2000a970 tx=0x7ffa200301b0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.894+0000 7ffa381d4640 1 -- 10.20.193.28:0/3154357665 shutdown_connections 2026-02-25T11:47:27.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.894+0000 7ffa381d4640 1 --2- 10.20.193.28:0/3154357665 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffa280bcf90 0x7ffa28008e00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.894+0000 7ffa381d4640 1 --2- 10.20.193.28:0/3154357665 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffa280b7790 0x7ffa280bc7e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.894+0000 7ffa381d4640 1 --2- 10.20.193.28:0/3154357665 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa280b6da0 0x7ffa280b7170 unknown :-1 s=CLOSED pgs=264 gs=3 cs=0 l=1 c_cookie=10036189744117840869 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.894+0000 7ffa381d4640 1 -- 10.20.193.28:0/3154357665 >> 10.20.193.28:0/3154357665 conn(0x7ffa28094630 msgr2=0x7ffa280b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:27.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.894+0000 7fb39dffb640 1 -- 10.20.193.28:0/2814656582 <== osd.4 v2:10.20.193.119:6808/135971996 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (crc 0 0 0) 0x7fb3840fdaa0 con 0x7fb3840dcfe0 2026-02-25T11:47:27.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.894+0000 7ffa381d4640 1 -- 10.20.193.28:0/3154357665 shutdown_connections 2026-02-25T11:47:27.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.894+0000 7ffa381d4640 1 -- 10.20.193.28:0/3154357665 wait complete. 2026-02-25T11:47:27.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.895+0000 7ffa381d4640 1 Processor -- start 2026-02-25T11:47:27.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.896+0000 7ffa381d4640 1 -- start start 2026-02-25T11:47:27.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.896+0000 7ffa381d4640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffa280b6da0 0x7ffa280bdfe0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.896+0000 7ffa381d4640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa280b7790 0x7ffa280be520 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.896+0000 7ffa381d4640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffa280bcf90 0x7ffa280c38f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.896+0000 7ffa381d4640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffa280c3f10 con 0x7ffa280b7790 2026-02-25T11:47:27.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.896+0000 7ffa381d4640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffa280c4080 con 0x7ffa280bcf90 2026-02-25T11:47:27.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.896+0000 7ffa381d4640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffa280c4240 con 0x7ffa280b6da0 2026-02-25T11:47:27.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.897+0000 7ffa3674a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffa280bcf90 0x7ffa280c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.897+0000 7ffa3674a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffa280bcf90 0x7ffa280c38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41652/0 (socket says 10.20.193.28:41652) 2026-02-25T11:47:27.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.897+0000 7ffa3674a640 1 -- 10.20.193.28:0/1567261165 learned_addr learned my addr 10.20.193.28:0/1567261165 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:27.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.897+0000 7ffa3674a640 1 -- 10.20.193.28:0/1567261165 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffa280b6da0 msgr2=0x7ffa280bdfe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:47:27.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.897+0000 7ffa3674a640 1 --2- 10.20.193.28:0/1567261165 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffa280b6da0 0x7ffa280bdfe0 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.897+0000 7ffa3674a640 1 -- 10.20.193.28:0/1567261165 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa280b7790 msgr2=0x7ffa280be520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:47:27.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.895+0000 7f6bbe3f1640 1 Processor -- start 2026-02-25T11:47:27.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.896+0000 7f6bbe3f1640 1 -- start start 2026-02-25T11:47:27.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.896+0000 7f6bbe3f1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6bb810f840 0x7f6bb8067030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.897+0000 7ffa3674a640 1 --2- 10.20.193.28:0/1567261165 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa280b7790 0x7ffa280be520 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.897+0000 7ffa3674a640 1 -- 10.20.193.28:0/1567261165 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa240094b0 con 0x7ffa280bcf90 2026-02-25T11:47:27.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.898+0000 7ffa3674a640 1 --2- 10.20.193.28:0/1567261165 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffa280bcf90 0x7ffa280c38f0 secure :-1 s=READY pgs=79 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa2c0026f0 tx=0x7ffa2c002bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.898+0000 7ffa1effd640 1 -- 10.20.193.28:0/1567261165 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffa2c00ed60 con 0x7ffa280bcf90 2026-02-25T11:47:27.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.898+0000 7ffa381d4640 1 -- 10.20.193.28:0/1567261165 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2c009570 con 0x7ffa280bcf90 2026-02-25T11:47:27.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.898+0000 7ffa1effd640 1 -- 10.20.193.28:0/1567261165 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ffa2c00eec0 con 0x7ffa280bcf90 2026-02-25T11:47:27.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.898+0000 7ffa1effd640 1 -- 10.20.193.28:0/1567261165 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffa2c00f7d0 con 0x7ffa280bcf90 2026-02-25T11:47:27.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.898+0000 7ffa381d4640 1 -- 10.20.193.28:0/1567261165 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2000a590 con 0x7ffa280bcf90 2026-02-25T11:47:27.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.902+0000 7ffa1effd640 1 -- 10.20.193.28:0/1567261165 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7ffa2c01f060 con 0x7ffa280bcf90 2026-02-25T11:47:27.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.900+0000 7f6bbe3f1640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6bb8110230 0x7f6bb80632a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.900+0000 7f6bb7fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6bb810f840 0x7f6bb8067030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.900+0000 7f6bbe3f1640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb8115a30 0x7f6bb8063950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.900+0000 7f6bb77fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6bb8110230 0x7f6bb80632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.900+0000 7f6bb77fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6bb8110230 0x7f6bb80632a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41656/0 (socket says 10.20.193.28:41656) 2026-02-25T11:47:27.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.900+0000 7f6bb77fe640 1 -- 10.20.193.28:0/1641992177 learned_addr learned my addr 10.20.193.28:0/1641992177 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:27.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.900+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb8064050 con 0x7f6bb810f840 2026-02-25T11:47:27.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.900+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb8168980 con 0x7f6bb8110230 2026-02-25T11:47:27.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.900+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb8168b40 con 0x7f6bb8115a30 2026-02-25T11:47:27.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.900+0000 7f6bbc967640 1 --2- 10.20.193.28:0/1641992177 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb8115a30 0x7f6bb8063950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.901+0000 7f6bbc967640 1 -- 10.20.193.28:0/1641992177 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6bb8110230 msgr2=0x7f6bb80632a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.901+0000 7f6bbc967640 1 --2- 10.20.193.28:0/1641992177 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6bb8110230 0x7f6bb80632a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.901+0000 7f6bbc967640 1 -- 10.20.193.28:0/1641992177 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6bb810f840 msgr2=0x7f6bb8067030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.901+0000 7f6bbc967640 1 --2- 10.20.193.28:0/1641992177 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6bb810f840 0x7f6bb8067030 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.901+0000 7f6bbc967640 1 -- 10.20.193.28:0/1641992177 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba400a590 con 0x7f6bb8115a30 2026-02-25T11:47:27.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.902+0000 7f6bbc967640 1 --2- 10.20.193.28:0/1641992177 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb8115a30 0x7f6bb8063950 secure :-1 s=READY pgs=85 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6bac00c940 tx=0x7f6bac00ce10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.902+0000 7f6bb57fa640 1 -- 10.20.193.28:0/1641992177 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6bac0044a0 con 0x7f6bb8115a30 2026-02-25T11:47:27.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.902+0000 7f6bb57fa640 1 -- 10.20.193.28:0/1641992177 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6bac007c60 con 0x7f6bb8115a30 2026-02-25T11:47:27.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.902+0000 7f6bb57fa640 1 -- 10.20.193.28:0/1641992177 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6bac00f450 con 0x7f6bb8115a30 2026-02-25T11:47:27.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.902+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba8009570 con 0x7f6bb8115a30 2026-02-25T11:47:27.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.902+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb8169110 con 0x7f6bb8115a30 2026-02-25T11:47:27.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.903+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bb80533d0 con 0x7f6bb8115a30 2026-02-25T11:47:27.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.904+0000 7ffa1effd640 1 --2- 10.20.193.28:0/1567261165 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffa0c0b57f0 0x7ffa0c0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.904+0000 7ffa1effd640 1 -- 10.20.193.28:0/1567261165 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7ffa2c0e6ff0 con 0x7ffa280bcf90 2026-02-25T11:47:27.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.905+0000 7ffa35f49640 1 --2- 10.20.193.28:0/1567261165 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffa0c0b57f0 0x7ffa0c0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.905+0000 7ffa35f49640 1 --2- 10.20.193.28:0/1567261165 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffa0c0b57f0 0x7ffa0c0d5ca0 secure :-1 s=READY pgs=193 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa200306c0 tx=0x7ffa20003470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.905+0000 7f6bb57fa640 1 -- 10.20.193.28:0/1641992177 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f6bac0077a0 con 0x7f6bb8115a30 2026-02-25T11:47:27.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.908+0000 7ffa381d4640 1 --2- 10.20.193.28:0/1567261165 >> [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884] conn(0x7ff9f80015f0 0x7ff9f8021aa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.907+0000 7f6bb57fa640 1 --2- 10.20.193.28:0/1641992177 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6b8c0b5870 0x7f6b8c0d5d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.907+0000 7f6bb57fa640 1 -- 10.20.193.28:0/1641992177 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f6bac0e6f00 con 0x7f6bb8115a30 2026-02-25T11:47:27.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.907+0000 7f6bb7fff640 1 --2- 10.20.193.28:0/1641992177 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6b8c0b5870 0x7f6b8c0d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.907+0000 7f6bb57fa640 1 --2- 10.20.193.28:0/1641992177 >> [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700] conn(0x7f6b8c0dcfe0 0x7f6b8c0fd3f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.907+0000 7f6bb57fa640 1 -- 10.20.193.28:0/1641992177 --> [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b8c0fdaa0 con 0x7f6b8c0dcfe0 2026-02-25T11:47:27.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.907+0000 7f6bb57fa640 1 -- 10.20.193.28:0/1641992177 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f6bac0e7230 con 0x7f6bb8115a30 2026-02-25T11:47:27.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.907+0000 7f6bb77fe640 1 --2- 10.20.193.28:0/1641992177 >> [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700] conn(0x7f6b8c0dcfe0 0x7f6b8c0fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.908+0000 7f6bb7fff640 1 --2- 10.20.193.28:0/1641992177 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6b8c0b5870 0x7f6b8c0d5d20 secure :-1 s=READY pgs=194 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ba8005b30 tx=0x7f6ba8002390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.908+0000 7f6bb77fe640 1 --2- 10.20.193.28:0/1641992177 >> [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700] conn(0x7f6b8c0dcfe0 0x7f6b8c0fd3f0 crc :-1 s=READY pgs=7 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.909+0000 7f6bb57fa640 1 -- 10.20.193.28:0/1641992177 <== osd.11 v2:10.20.193.167:6824/2182906700 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (crc 0 0 0) 0x7f6b8c0fdaa0 con 0x7f6b8c0dcfe0 2026-02-25T11:47:27.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.911+0000 7ffa381d4640 1 -- 10.20.193.28:0/1567261165 --> [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9f8024c00 con 0x7ff9f80015f0 2026-02-25T11:47:27.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.911+0000 7ffa35748640 1 --2- 10.20.193.28:0/1567261165 >> [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884] conn(0x7ff9f80015f0 0x7ff9f8021aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.911+0000 7ffa35748640 1 --2- 10.20.193.28:0/1567261165 >> [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884] conn(0x7ff9f80015f0 0x7ff9f8021aa0 crc :-1 s=READY pgs=10 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.912+0000 7ffa1effd640 1 -- 10.20.193.28:0/1567261165 <== osd.5 v2:10.20.193.167:6808/4203420884 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (crc 0 0 0) 0x7ff9f8024c00 con 0x7ff9f80015f0 2026-02-25T11:47:27.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.918+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 --> [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6bb8179850 con 0x7f6b8c0dcfe0 2026-02-25T11:47:27.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.919+0000 7f6bb57fa640 1 -- 10.20.193.28:0/1641992177 <== osd.11 v2:10.20.193.167:6824/2182906700 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6bb8179850 con 0x7f6b8c0dcfe0 2026-02-25T11:47:27.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.920+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 >> [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700] conn(0x7f6b8c0dcfe0 msgr2=0x7f6b8c0fd3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.922 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.920+0000 7f6bbe3f1640 1 --2- 10.20.193.28:0/1641992177 >> [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700] conn(0x7f6b8c0dcfe0 0x7f6b8c0fd3f0 crc :-1 s=READY pgs=7 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.920+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6b8c0b5870 msgr2=0x7f6b8c0d5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.920+0000 7f6bbe3f1640 1 --2- 10.20.193.28:0/1641992177 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6b8c0b5870 0x7f6b8c0d5d20 secure :-1 s=READY pgs=194 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ba8005b30 tx=0x7f6ba8002390 comp rx=0 tx=0).stop 2026-02-25T11:47:27.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.920+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb8115a30 msgr2=0x7f6bb8063950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.920+0000 7f6bbe3f1640 1 --2- 10.20.193.28:0/1641992177 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6bb8115a30 0x7f6bb8063950 secure :-1 s=READY pgs=85 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6bac00c940 tx=0x7f6bac00ce10 comp rx=0 tx=0).stop 2026-02-25T11:47:27.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.920+0000 7f6bbc967640 1 -- 10.20.193.28:0/1641992177 reap_dead start 2026-02-25T11:47:27.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.920+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 shutdown_connections 2026-02-25T11:47:27.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.920+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 >> 10.20.193.28:0/1641992177 conn(0x7f6bb80ed130 msgr2=0x7f6bb810d510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:27.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.921+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 shutdown_connections 2026-02-25T11:47:27.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.921+0000 7f6bbe3f1640 1 -- 10.20.193.28:0/1641992177 wait complete. 2026-02-25T11:47:27.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.921+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 --> [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb370002d10 con 0x7fb3840dcfe0 2026-02-25T11:47:27.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.921+0000 7fb39dffb640 1 -- 10.20.193.28:0/2814656582 <== osd.4 v2:10.20.193.119:6808/135971996 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb370002d10 con 0x7fb3840dcfe0 2026-02-25T11:47:27.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.922+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 >> [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996] conn(0x7fb3840dcfe0 msgr2=0x7fb3840fd3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.922+0000 7fb3af055640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996] conn(0x7fb3840dcfe0 0x7fb3840fd3f0 crc :-1 s=READY pgs=11 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.922+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb3840b5870 msgr2=0x7fb3840d5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.922+0000 7fb3af055640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb3840b5870 0x7fb3840d5d20 secure :-1 s=READY pgs=192 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb3a8128a40 tx=0x7fb390002390 comp rx=0 tx=0).stop 2026-02-25T11:47:27.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.922+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb3a8114900 msgr2=0x7fb3a81220a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.922+0000 7fb3af055640 1 --2- 10.20.193.28:0/2814656582 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb3a8114900 0x7fb3a81220a0 secure :-1 s=READY pgs=83 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb398005820 tx=0x7fb39800fe60 comp rx=0 tx=0).stop 2026-02-25T11:47:27.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.923+0000 7fb3ad5cb640 1 -- 10.20.193.28:0/2814656582 reap_dead start 2026-02-25T11:47:27.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.923+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 shutdown_connections 2026-02-25T11:47:27.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.923+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 >> 10.20.193.28:0/2814656582 conn(0x7fb3a8057d30 msgr2=0x7fb3a8117970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:27.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.924+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 shutdown_connections 2026-02-25T11:47:27.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.924+0000 7fb3af055640 1 -- 10.20.193.28:0/2814656582 wait complete. 2026-02-25T11:47:27.933 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:27 trial028 ceph-mon[39224]: pgmap v201: 97 pgs: 97 active+clean; 633 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 7.9 KiB/s rd, 8.7 KiB/s wr, 16 op/s 2026-02-25T11:47:27.933 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:27 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/418693799' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:27.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.950+0000 7ffa381d4640 1 -- 10.20.193.28:0/1567261165 --> [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff9f8023cf0 con 0x7ff9f80015f0 2026-02-25T11:47:27.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.951+0000 7ffa1effd640 1 -- 10.20.193.28:0/1567261165 <== osd.5 v2:10.20.193.167:6808/4203420884 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff9f8023cf0 con 0x7ff9f80015f0 2026-02-25T11:47:27.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.955+0000 7ffa381d4640 1 -- 10.20.193.28:0/1567261165 >> [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884] conn(0x7ff9f80015f0 msgr2=0x7ff9f8021aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.955+0000 7ffa381d4640 1 --2- 10.20.193.28:0/1567261165 >> [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884] conn(0x7ff9f80015f0 0x7ff9f8021aa0 crc :-1 s=READY pgs=10 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.955+0000 7ffa381d4640 1 -- 10.20.193.28:0/1567261165 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffa0c0b57f0 msgr2=0x7ffa0c0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.955+0000 7ffa381d4640 1 --2- 10.20.193.28:0/1567261165 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffa0c0b57f0 0x7ffa0c0d5ca0 secure :-1 s=READY pgs=193 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa200306c0 tx=0x7ffa20003470 comp rx=0 tx=0).stop 2026-02-25T11:47:27.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.955+0000 7ffa381d4640 1 -- 10.20.193.28:0/1567261165 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffa280bcf90 msgr2=0x7ffa280c38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.955+0000 7ffa381d4640 1 --2- 10.20.193.28:0/1567261165 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffa280bcf90 0x7ffa280c38f0 secure :-1 s=READY pgs=79 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa2c0026f0 tx=0x7ffa2c002bc0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.955+0000 7ffa3674a640 1 -- 10.20.193.28:0/1567261165 reap_dead start 2026-02-25T11:47:27.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.955+0000 7ffa381d4640 1 -- 10.20.193.28:0/1567261165 shutdown_connections 2026-02-25T11:47:27.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.955+0000 7ffa381d4640 1 -- 10.20.193.28:0/1567261165 >> 10.20.193.28:0/1567261165 conn(0x7ffa28094630 msgr2=0x7ffa280b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:27.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.955+0000 7ffa381d4640 1 -- 10.20.193.28:0/1567261165 shutdown_connections 2026-02-25T11:47:27.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.955+0000 7ffa381d4640 1 -- 10.20.193.28:0/1567261165 wait complete. 2026-02-25T11:47:27.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.967+0000 7f60e77ea640 1 -- 10.20.193.28:0/2162900095 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f60e01152f0 msgr2=0x7f60e0116940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.967+0000 7f60e77ea640 1 --2- 10.20.193.28:0/2162900095 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f60e01152f0 0x7f60e0116940 secure :-1 s=READY pgs=265 gs=1 cs=0 l=1 c_cookie=6002078606866089945 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60c80098c0 tx=0x7f60c802f0e0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.969+0000 7f60e77ea640 1 -- 10.20.193.28:0/2162900095 shutdown_connections 2026-02-25T11:47:27.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.969+0000 7f60e77ea640 1 --2- 10.20.193.28:0/2162900095 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f60e0117120 0x7f60e017ace0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.969+0000 7f60e77ea640 1 --2- 10.20.193.28:0/2162900095 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f60e01152f0 0x7f60e0116940 unknown :-1 s=CLOSED pgs=265 gs=1 cs=0 l=1 c_cookie=6002078606866089945 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.969+0000 7f60e77ea640 1 --2- 10.20.193.28:0/2162900095 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f60e0114900 0x7f60e0114cd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.969+0000 7f60e77ea640 1 -- 10.20.193.28:0/2162900095 >> 10.20.193.28:0/2162900095 conn(0x7f60e0057d30 msgr2=0x7f60e0058130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:27.984 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.970+0000 7f60e77ea640 1 -- 10.20.193.28:0/2162900095 shutdown_connections 2026-02-25T11:47:27.984 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.970+0000 7f60e77ea640 1 -- 10.20.193.28:0/2162900095 wait complete. 2026-02-25T11:47:27.984 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.971+0000 7f60e77ea640 1 Processor -- start 2026-02-25T11:47:27.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.972+0000 7f60e77ea640 1 -- start start 2026-02-25T11:47:27.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.972+0000 7f60e77ea640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f60e0114900 0x7f60e0122990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.972+0000 7f60e77ea640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f60e01152f0 0x7f60e0122ed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.972+0000 7f60e77ea640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f60e0117120 0x7f60e0128250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.973+0000 7f60e555f640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f60e0114900 0x7f60e0122990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.973+0000 7f60e77ea640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f60e0128870 con 0x7f60e0117120 2026-02-25T11:47:27.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.973+0000 7f60e77ea640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f60e01289e0 con 0x7f60e01152f0 2026-02-25T11:47:27.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.973+0000 7f60e77ea640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60e0128ba0 con 0x7f60e0114900 2026-02-25T11:47:27.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.973+0000 7f60e4d5e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f60e01152f0 0x7f60e0122ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.973+0000 7f60e4d5e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f60e01152f0 0x7f60e0122ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41688/0 (socket says 10.20.193.28:41688) 2026-02-25T11:47:27.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.973+0000 7f60e4d5e640 1 -- 10.20.193.28:0/1769662128 learned_addr learned my addr 10.20.193.28:0/1769662128 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:27.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.974+0000 7f60e5d60640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f60e0117120 0x7f60e0128250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.974+0000 7f60e4d5e640 1 -- 10.20.193.28:0/1769662128 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f60e0114900 msgr2=0x7f60e0122990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.974+0000 7f60e4d5e640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f60e0114900 0x7f60e0122990 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.974+0000 7f60e4d5e640 1 -- 10.20.193.28:0/1769662128 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f60e0117120 msgr2=0x7f60e0128250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.974+0000 7f60e4d5e640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f60e0117120 0x7f60e0128250 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.974+0000 7f60e4d5e640 1 -- 10.20.193.28:0/1769662128 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d00071c0 con 0x7f60e01152f0 2026-02-25T11:47:27.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.974+0000 7f60e555f640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f60e0114900 0x7f60e0122990 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:27.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.975+0000 7f60e4d5e640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f60e01152f0 0x7f60e0122ed0 secure :-1 s=READY pgs=80 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60c8009890 tx=0x7f60c80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.975+0000 7f60d67fc640 1 -- 10.20.193.28:0/1769662128 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f60c803d070 con 0x7f60e01152f0 2026-02-25T11:47:27.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.975+0000 7f60d67fc640 1 -- 10.20.193.28:0/1769662128 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f60c8002ae0 con 0x7f60e01152f0 2026-02-25T11:47:27.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.975+0000 7f60d67fc640 1 -- 10.20.193.28:0/1769662128 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f60c8041a60 con 0x7f60e01152f0 2026-02-25T11:47:27.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.975+0000 7f60e77ea640 1 -- 10.20.193.28:0/1769662128 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60c80094b0 con 0x7f60e01152f0 2026-02-25T11:47:27.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.975+0000 7f60e77ea640 1 -- 10.20.193.28:0/1769662128 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e01290b0 con 0x7f60e01152f0 2026-02-25T11:47:27.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.977+0000 7f60e77ea640 1 -- 10.20.193.28:0/1769662128 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60e00533d0 con 0x7f60e01152f0 2026-02-25T11:47:27.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.978+0000 7f60d67fc640 1 -- 10.20.193.28:0/1769662128 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f60c804b440 con 0x7f60e01152f0 2026-02-25T11:47:27.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.980+0000 7f60d67fc640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60b80b5870 0x7f60b80d5d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.980+0000 7f60e555f640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60b80b5870 0x7f60b80d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.980+0000 7f60d67fc640 1 -- 10.20.193.28:0/1769662128 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f60c810c930 con 0x7f60e01152f0 2026-02-25T11:47:27.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.981+0000 7f60d67fc640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116] conn(0x7f60b80dcfe0 0x7f60b80fd3f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:27.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.981+0000 7f60d67fc640 1 -- 10.20.193.28:0/1769662128 --> [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60b80fdaa0 con 0x7f60b80dcfe0 2026-02-25T11:47:27.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.981+0000 7f60d67fc640 1 -- 10.20.193.28:0/1769662128 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f60c8038710 con 0x7f60e01152f0 2026-02-25T11:47:27.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.982+0000 7f60e5d60640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116] conn(0x7f60b80dcfe0 0x7f60b80fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:27.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.982+0000 7f60e555f640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60b80b5870 0x7f60b80d5d20 secure :-1 s=READY pgs=195 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60dc006a50 tx=0x7f60dc005050 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.988 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.982+0000 7f60e5d60640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116] conn(0x7f60b80dcfe0 0x7f60b80fd3f0 crc :-1 s=READY pgs=12 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:27.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.986+0000 7f60d67fc640 1 -- 10.20.193.28:0/1769662128 <== osd.10 v2:10.20.193.119:6824/2996581116 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (crc 0 0 0) 0x7f60b80fdaa0 con 0x7f60b80dcfe0 2026-02-25T11:47:27.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.995+0000 7f60e77ea640 1 -- 10.20.193.28:0/1769662128 --> [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f60e017dd40 con 0x7f60b80dcfe0 2026-02-25T11:47:27.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.995+0000 7f60d67fc640 1 -- 10.20.193.28:0/1769662128 <== osd.10 v2:10.20.193.119:6824/2996581116 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f60e017dd40 con 0x7f60b80dcfe0 2026-02-25T11:47:27.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.996+0000 7f60e77ea640 1 -- 10.20.193.28:0/1769662128 >> [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116] conn(0x7f60b80dcfe0 msgr2=0x7f60b80fd3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.996+0000 7f60e77ea640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116] conn(0x7f60b80dcfe0 0x7f60b80fd3f0 crc :-1 s=READY pgs=12 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.996+0000 7f60e77ea640 1 -- 10.20.193.28:0/1769662128 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60b80b5870 msgr2=0x7f60b80d5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.996+0000 7f60e77ea640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60b80b5870 0x7f60b80d5d20 secure :-1 s=READY pgs=195 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60dc006a50 tx=0x7f60dc005050 comp rx=0 tx=0).stop 2026-02-25T11:47:27.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.996+0000 7f60e77ea640 1 -- 10.20.193.28:0/1769662128 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f60e01152f0 msgr2=0x7f60e0122ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:27.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.996+0000 7f60e77ea640 1 --2- 10.20.193.28:0/1769662128 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f60e01152f0 0x7f60e0122ed0 secure :-1 s=READY pgs=80 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60c8009890 tx=0x7f60c80042f0 comp rx=0 tx=0).stop 2026-02-25T11:47:27.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.996+0000 7f60e5d60640 1 -- 10.20.193.28:0/1769662128 reap_dead start 2026-02-25T11:47:27.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.997+0000 7f60e77ea640 1 -- 10.20.193.28:0/1769662128 shutdown_connections 2026-02-25T11:47:27.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.997+0000 7f60e77ea640 1 -- 10.20.193.28:0/1769662128 >> 10.20.193.28:0/1769662128 conn(0x7f60e0057d30 msgr2=0x7f60e0116120 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:27.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.997+0000 7f60e77ea640 1 -- 10.20.193.28:0/1769662128 shutdown_connections 2026-02-25T11:47:27.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.997+0000 7f60e77ea640 1 -- 10.20.193.28:0/1769662128 wait complete. 2026-02-25T11:47:28.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:27 trial167 ceph-mon[43392]: pgmap v201: 97 pgs: 97 active+clean; 633 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 7.9 KiB/s rd, 8.7 KiB/s wr, 16 op/s 2026-02-25T11:47:28.000 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:27 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/418693799' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:28.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.998+0000 7f0c13fff640 1 -- 10.20.193.28:0/4188653475 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0c14116fe0 msgr2=0x7f0c1417aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.998+0000 7f0c13fff640 1 --2- 10.20.193.28:0/4188653475 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0c14116fe0 0x7f0c1417aba0 secure :-1 s=READY pgs=81 gs=1 cs=0 l=1 c_cookie=15409077428827012263 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0c0400aa30 tx=0x7f0c04030290 comp rx=0 tx=0).stop 2026-02-25T11:47:28.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.999+0000 7f0c13fff640 1 -- 10.20.193.28:0/4188653475 shutdown_connections 2026-02-25T11:47:28.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.999+0000 7f0c13fff640 1 --2- 10.20.193.28:0/4188653475 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0c14116fe0 0x7f0c1417aba0 unknown :-1 s=CLOSED pgs=81 gs=1 cs=0 l=1 c_cookie=15409077428827012263 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.999+0000 7f0c13fff640 1 --2- 10.20.193.28:0/4188653475 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c141152f0 0x7f0c14116800 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.999+0000 7f0c13fff640 1 --2- 10.20.193.28:0/4188653475 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0c14114900 0x7f0c14114cd0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:27.999+0000 7f0c13fff640 1 -- 10.20.193.28:0/4188653475 >> 10.20.193.28:0/4188653475 conn(0x7f0c1405ba30 msgr2=0x7f0c1405be30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.000+0000 7f0c13fff640 1 -- 10.20.193.28:0/4188653475 shutdown_connections 2026-02-25T11:47:28.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.001+0000 7f0c13fff640 1 -- 10.20.193.28:0/4188653475 wait complete. 2026-02-25T11:47:28.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:27 trial119 ceph-mon[42710]: pgmap v201: 97 pgs: 97 active+clean; 633 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 7.9 KiB/s rd, 8.7 KiB/s wr, 16 op/s 2026-02-25T11:47:28.009 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:27 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/418693799' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:28.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.010+0000 7f0c13fff640 1 Processor -- start 2026-02-25T11:47:28.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.011+0000 7f0c13fff640 1 -- start start 2026-02-25T11:47:28.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.012+0000 7f0c13fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0c14114900 0x7f0c140fc0f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.013+0000 7f0c13fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0c141152f0 0x7f0c140fc630 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.013+0000 7f0c12ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0c14114900 0x7f0c140fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.013+0000 7f0c12ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0c14114900 0x7f0c140fc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52210/0 (socket says 10.20.193.28:52210) 2026-02-25T11:47:28.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.013+0000 7f0c13fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c14116fe0 0x7f0c14101a00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.013+0000 7f0c12ffd640 1 -- 10.20.193.28:0/1619913277 learned_addr learned my addr 10.20.193.28:0/1619913277 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:28.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.014+0000 7f0c13fff640 1 -- 10.20.193.28:0/1619913277 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14102020 con 0x7f0c14116fe0 2026-02-25T11:47:28.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.014+0000 7f0c13fff640 1 -- 10.20.193.28:0/1619913277 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14102190 con 0x7f0c141152f0 2026-02-25T11:47:28.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.014+0000 7f0c13fff640 1 -- 10.20.193.28:0/1619913277 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14102350 con 0x7f0c14114900 2026-02-25T11:47:28.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.013+0000 7f0c127fc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0c141152f0 0x7f0c140fc630 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.015+0000 7f0c12ffd640 1 -- 10.20.193.28:0/1619913277 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0c141152f0 msgr2=0x7f0c140fc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.015+0000 7f0c12ffd640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0c141152f0 0x7f0c140fc630 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.015+0000 7f0c12ffd640 1 -- 10.20.193.28:0/1619913277 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c14116fe0 msgr2=0x7f0c14101a00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:47:28.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.015+0000 7f0c137fe640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c14116fe0 0x7f0c14101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.015+0000 7f0c12ffd640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c14116fe0 0x7f0c14101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.015+0000 7f0c12ffd640 1 -- 10.20.193.28:0/1619913277 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0400a650 con 0x7f0c14114900 2026-02-25T11:47:28.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.015+0000 7f0c127fc640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0c141152f0 0x7f0c140fc630 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:28.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.016+0000 7f0c137fe640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0c14116fe0 0x7f0c14101a00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:28.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.016+0000 7f0c12ffd640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0c14114900 0x7f0c140fc0f0 secure :-1 s=READY pgs=86 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0c080072a0 tx=0x7f0c0800ddf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.016+0000 7f0bf3fff640 1 -- 10.20.193.28:0/1619913277 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0c08004500 con 0x7f0c14114900 2026-02-25T11:47:28.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.017+0000 7f0c13fff640 1 -- 10.20.193.28:0/1619913277 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c14102630 con 0x7f0c14114900 2026-02-25T11:47:28.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.017+0000 7f0c13fff640 1 -- 10.20.193.28:0/1619913277 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1416c140 con 0x7f0c14114900 2026-02-25T11:47:28.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.017+0000 7f0bf3fff640 1 -- 10.20.193.28:0/1619913277 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0c0800bd00 con 0x7f0c14114900 2026-02-25T11:47:28.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.017+0000 7f0bf3fff640 1 -- 10.20.193.28:0/1619913277 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0c08004500 con 0x7f0c14114900 2026-02-25T11:47:28.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.018+0000 7f0bf1ffb640 1 -- 10.20.193.28:0/1619913277 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c140533d0 con 0x7f0c14114900 2026-02-25T11:47:28.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.019+0000 7f6eae541640 1 -- 10.20.193.28:0/114939462 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6ea8114900 msgr2=0x7f6ea8114cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.019+0000 7f6eae541640 1 --2- 10.20.193.28:0/114939462 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6ea8114900 0x7f6ea8114cd0 secure :-1 s=READY pgs=82 gs=1 cs=0 l=1 c_cookie=14490301194501394241 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6e9c009950 tx=0x7f6e9c02f1d0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.022+0000 7f6eae541640 1 -- 10.20.193.28:0/114939462 shutdown_connections 2026-02-25T11:47:28.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.022+0000 7f6eae541640 1 --2- 10.20.193.28:0/114939462 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ea8116fe0 0x7f6ea817aba0 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.022+0000 7f6eae541640 1 --2- 10.20.193.28:0/114939462 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6ea81152f0 0x7f6ea8116800 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.022+0000 7f6eae541640 1 --2- 10.20.193.28:0/114939462 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6ea8114900 0x7f6ea8114cd0 unknown :-1 s=CLOSED pgs=82 gs=1 cs=0 l=1 c_cookie=14490301194501394241 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.022+0000 7f6eae541640 1 -- 10.20.193.28:0/114939462 >> 10.20.193.28:0/114939462 conn(0x7f6ea805ba30 msgr2=0x7f6ea805be30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.022+0000 7f6eae541640 1 -- 10.20.193.28:0/114939462 shutdown_connections 2026-02-25T11:47:28.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.021+0000 7f0bf3fff640 1 -- 10.20.193.28:0/1619913277 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f0c080489f0 con 0x7f0c14114900 2026-02-25T11:47:28.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.022+0000 7f6eae541640 1 -- 10.20.193.28:0/114939462 wait complete. 2026-02-25T11:47:28.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.022+0000 7f0bf3fff640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0be00b5850 0x7f0be00d5d00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.023+0000 7f0bf3fff640 1 -- 10.20.193.28:0/1619913277 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f0c080e6a50 con 0x7f0c14114900 2026-02-25T11:47:28.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.023+0000 7f0c127fc640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0be00b5850 0x7f0be00d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.023+0000 7f6eae541640 1 Processor -- start 2026-02-25T11:47:28.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.023+0000 7f6eae541640 1 -- start start 2026-02-25T11:47:28.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.023+0000 7f6eae541640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ea8114900 0x7f6ea8069e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.023+0000 7f6eae541640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6ea81152f0 0x7f6ea806a3d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.023+0000 7f6eae541640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6ea8116fe0 0x7f6ea817aaa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.023+0000 7f6eae541640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea806f710 con 0x7f6ea8114900 2026-02-25T11:47:28.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.023+0000 7f6eae541640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea806f880 con 0x7f6ea8116fe0 2026-02-25T11:47:28.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.023+0000 7f6eae541640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea806fa40 con 0x7f6ea81152f0 2026-02-25T11:47:28.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.024+0000 7f6ead53f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ea8114900 0x7f6ea8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.024+0000 7f6ead53f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ea8114900 0x7f6ea8069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40708/0 (socket says 10.20.193.28:40708) 2026-02-25T11:47:28.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.024+0000 7f6ead53f640 1 -- 10.20.193.28:0/3091103360 learned_addr learned my addr 10.20.193.28:0/3091103360 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:28.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.024+0000 7f6ead53f640 1 -- 10.20.193.28:0/3091103360 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6ea81152f0 msgr2=0x7f6ea806a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.024+0000 7f6eacd3e640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6ea81152f0 0x7f6ea806a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.025+0000 7f6eadd40640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6ea8116fe0 0x7f6ea817aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.024+0000 7f0c127fc640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0be00b5850 0x7f0be00d5d00 secure :-1 s=READY pgs=196 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0c14102a90 tx=0x7f0bfc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.025+0000 7f6ead53f640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6ea81152f0 0x7f6ea806a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.025+0000 7f6ead53f640 1 -- 10.20.193.28:0/3091103360 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6ea8116fe0 msgr2=0x7f6ea817aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.025+0000 7f6ead53f640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6ea8116fe0 0x7f6ea817aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.025+0000 7f6ead53f640 1 -- 10.20.193.28:0/3091103360 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9c009570 con 0x7f6ea8114900 2026-02-25T11:47:28.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.025+0000 7f6eacd3e640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6ea81152f0 0x7f6ea806a3d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:28.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.025+0000 7f0bf3fff640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242] conn(0x7f0be00dcfc0 0x7f0be00fd3d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.025+0000 7f6ead53f640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ea8114900 0x7f6ea8069e90 secure :-1 s=READY pgs=266 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6e9c0098a0 tx=0x7f6e9c0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.026+0000 7f0bf3fff640 1 -- 10.20.193.28:0/1619913277 --> [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0be00fda80 con 0x7f0be00dcfc0 2026-02-25T11:47:28.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.026+0000 7f0bf3fff640 1 -- 10.20.193.28:0/1619913277 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f0c080e6e70 con 0x7f0c14114900 2026-02-25T11:47:28.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.026+0000 7f0c137fe640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242] conn(0x7f0be00dcfc0 0x7f0be00fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.026+0000 7f6e967fc640 1 -- 10.20.193.28:0/3091103360 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6e9c03d070 con 0x7f6ea8114900 2026-02-25T11:47:28.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.026+0000 7f6e967fc640 1 -- 10.20.193.28:0/3091103360 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6e9c002a80 con 0x7f6ea8114900 2026-02-25T11:47:28.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.026+0000 7f6e967fc640 1 -- 10.20.193.28:0/3091103360 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6e9c041a60 con 0x7f6ea8114900 2026-02-25T11:47:28.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.026+0000 7f6eae541640 1 -- 10.20.193.28:0/3091103360 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea806fc60 con 0x7f6ea8114900 2026-02-25T11:47:28.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.027+0000 7f6eae541640 1 -- 10.20.193.28:0/3091103360 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea8070080 con 0x7f6ea8114900 2026-02-25T11:47:28.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.028+0000 7f0c137fe640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242] conn(0x7f0be00dcfc0 0x7f0be00fd3d0 crc :-1 s=READY pgs=10 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.029+0000 7f0bf3fff640 1 -- 10.20.193.28:0/1619913277 <== osd.1 v2:10.20.193.119:6800/3486566242 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (crc 0 0 0) 0x7f0be00fda80 con 0x7f0be00dcfc0 2026-02-25T11:47:28.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.031+0000 7f6e967fc640 1 -- 10.20.193.28:0/3091103360 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f6e9c0386e0 con 0x7f6ea8114900 2026-02-25T11:47:28.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.033+0000 7f6e967fc640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6e800b57f0 0x7f6e800d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.033+0000 7f6e967fc640 1 -- 10.20.193.28:0/3091103360 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f6e9c10b5e0 con 0x7f6ea8114900 2026-02-25T11:47:28.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.034+0000 7f6eacd3e640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6e800b57f0 0x7f6e800d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.034+0000 7f6eacd3e640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6e800b57f0 0x7f6e800d5ca0 secure :-1 s=READY pgs=197 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ea81189d0 tx=0x7f6e98008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.040+0000 7f0bf1ffb640 1 -- 10.20.193.28:0/1619913277 --> [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c14056680 con 0x7f0be00dcfc0 2026-02-25T11:47:28.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.041+0000 7f0bf3fff640 1 -- 10.20.193.28:0/1619913277 <== osd.1 v2:10.20.193.119:6800/3486566242 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0c04003d30 con 0x7f0be00dcfc0 2026-02-25T11:47:28.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.038+0000 7f6eae541640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317] conn(0x7f6ea804cb40 0x7f6ea804cf10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.038+0000 7f6eae541640 1 -- 10.20.193.28:0/3091103360 --> [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ea8056680 con 0x7f6ea804cb40 2026-02-25T11:47:28.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.039+0000 7f6eadd40640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317] conn(0x7f6ea804cb40 0x7f6ea804cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.040+0000 7f6eadd40640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317] conn(0x7f6ea804cb40 0x7f6ea804cf10 crc :-1 s=READY pgs=12 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.040+0000 7f6e967fc640 1 -- 10.20.193.28:0/3091103360 <== osd.8 v2:10.20.193.167:6816/1328734317 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (crc 0 0 0) 0x7f6ea400b4c0 con 0x7f6ea804cb40 2026-02-25T11:47:28.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.041+0000 7f0bf1ffb640 1 -- 10.20.193.28:0/1619913277 >> [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242] conn(0x7f0be00dcfc0 msgr2=0x7f0be00fd3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.041+0000 7f0bf1ffb640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242] conn(0x7f0be00dcfc0 0x7f0be00fd3d0 crc :-1 s=READY pgs=10 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.042+0000 7f0bf1ffb640 1 -- 10.20.193.28:0/1619913277 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0be00b5850 msgr2=0x7f0be00d5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.042+0000 7f0bf1ffb640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0be00b5850 0x7f0be00d5d00 secure :-1 s=READY pgs=196 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0c14102a90 tx=0x7f0bfc008040 comp rx=0 tx=0).stop 2026-02-25T11:47:28.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.042+0000 7f0bf1ffb640 1 -- 10.20.193.28:0/1619913277 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0c14114900 msgr2=0x7f0c140fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.042+0000 7f0bf1ffb640 1 --2- 10.20.193.28:0/1619913277 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0c14114900 0x7f0c140fc0f0 secure :-1 s=READY pgs=86 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0c080072a0 tx=0x7f0c0800ddf0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.042+0000 7f0c137fe640 1 -- 10.20.193.28:0/1619913277 reap_dead start 2026-02-25T11:47:28.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.042+0000 7f0bf1ffb640 1 -- 10.20.193.28:0/1619913277 shutdown_connections 2026-02-25T11:47:28.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.042+0000 7f0bf1ffb640 1 -- 10.20.193.28:0/1619913277 >> 10.20.193.28:0/1619913277 conn(0x7f0c1405ba30 msgr2=0x7f0c141157f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.045+0000 7f0bf1ffb640 1 -- 10.20.193.28:0/1619913277 shutdown_connections 2026-02-25T11:47:28.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.045+0000 7f0bf1ffb640 1 -- 10.20.193.28:0/1619913277 wait complete. 2026-02-25T11:47:28.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.052+0000 7f6eae541640 1 -- 10.20.193.28:0/3091103360 --> [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6ea805e8e0 con 0x7f6ea804cb40 2026-02-25T11:47:28.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.052+0000 7f6e967fc640 1 -- 10.20.193.28:0/3091103360 <== osd.8 v2:10.20.193.167:6816/1328734317 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6ea805e8e0 con 0x7f6ea804cb40 2026-02-25T11:47:28.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.055+0000 7f391a95a640 1 -- 10.20.193.28:0/3174009503 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391410b360 msgr2=0x7f391410b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.055+0000 7f391a95a640 1 --2- 10.20.193.28:0/3174009503 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391410b360 0x7f391410b730 secure :-1 s=READY pgs=83 gs=1 cs=0 l=1 c_cookie=17982370059647460977 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f391000aa30 tx=0x7f39100302b0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.055+0000 7f391a95a640 1 -- 10.20.193.28:0/3174009503 shutdown_connections 2026-02-25T11:47:28.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.055+0000 7f391a95a640 1 --2- 10.20.193.28:0/3174009503 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3914113620 0x7f3914173ff0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.055+0000 7f391a95a640 1 --2- 10.20.193.28:0/3174009503 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f391410bd50 0x7f3914112e70 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.055+0000 7f391a95a640 1 --2- 10.20.193.28:0/3174009503 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391410b360 0x7f391410b730 unknown :-1 s=CLOSED pgs=83 gs=1 cs=0 l=1 c_cookie=17982370059647460977 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.055+0000 7f391a95a640 1 -- 10.20.193.28:0/3174009503 >> 10.20.193.28:0/3174009503 conn(0x7f39140e8e70 msgr2=0x7f3914109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.055+0000 7f391a95a640 1 -- 10.20.193.28:0/3174009503 shutdown_connections 2026-02-25T11:47:28.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.055+0000 7f391a95a640 1 -- 10.20.193.28:0/3174009503 wait complete. 2026-02-25T11:47:28.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.056+0000 7f391a95a640 1 Processor -- start 2026-02-25T11:47:28.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.056+0000 7f61b1157640 1 -- 10.20.193.28:0/3744589399 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f61ac1117f0 msgr2=0x7f61ac1721c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.056+0000 7f391a95a640 1 -- start start 2026-02-25T11:47:28.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.056+0000 7f61b1157640 1 --2- 10.20.193.28:0/3744589399 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f61ac1117f0 0x7f61ac1721c0 secure :-1 s=READY pgs=267 gs=3 cs=0 l=1 c_cookie=11769140747077702399 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f619c0098e0 tx=0x7f619c02f170 comp rx=0 tx=0).stop 2026-02-25T11:47:28.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.057+0000 7f61b1157640 1 -- 10.20.193.28:0/3744589399 shutdown_connections 2026-02-25T11:47:28.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.057+0000 7f61b1157640 1 --2- 10.20.193.28:0/3744589399 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f61ac1117f0 0x7f61ac1721c0 unknown :-1 s=CLOSED pgs=267 gs=3 cs=0 l=1 c_cookie=11769140747077702399 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.057+0000 7f391a95a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f391410b360 0x7f39141071e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.057+0000 7f6eae541640 1 -- 10.20.193.28:0/3091103360 >> [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317] conn(0x7f6ea804cb40 msgr2=0x7f6ea804cf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.057+0000 7f6eae541640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317] conn(0x7f6ea804cb40 0x7f6ea804cf10 crc :-1 s=READY pgs=12 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.057+0000 7f391a95a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391410bd50 0x7f3914173ef0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.057+0000 7f391a95a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3914113620 0x7f3914107910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.057+0000 7f61b1157640 1 --2- 10.20.193.28:0/3744589399 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f61ac10bff0 0x7f61ac111040 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.058+0000 7f3919958640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f391410b360 0x7f39141071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.058+0000 7f3919157640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391410bd50 0x7f3914173ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.058+0000 7f391a159640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3914113620 0x7f3914107910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.058+0000 7f391a95a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f39140eb320 con 0x7f3914113620 2026-02-25T11:47:28.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.058+0000 7f3919157640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391410bd50 0x7f3914173ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41756/0 (socket says 10.20.193.28:41756) 2026-02-25T11:47:28.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.058+0000 7f3919157640 1 -- 10.20.193.28:0/1927573420 learned_addr learned my addr 10.20.193.28:0/1927573420 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:28.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.058+0000 7f391a95a640 1 -- 10.20.193.28:0/1927573420 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f39140eb490 con 0x7f391410bd50 2026-02-25T11:47:28.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.058+0000 7f391a95a640 1 -- 10.20.193.28:0/1927573420 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f39140eb650 con 0x7f391410b360 2026-02-25T11:47:28.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.058+0000 7f61b1157640 1 --2- 10.20.193.28:0/3744589399 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61ac10b600 0x7f61ac10b9d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.058+0000 7f61b1157640 1 -- 10.20.193.28:0/3744589399 >> 10.20.193.28:0/3744589399 conn(0x7f61ac0e8e70 msgr2=0x7f61ac109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.058+0000 7f61b1157640 1 -- 10.20.193.28:0/3744589399 shutdown_connections 2026-02-25T11:47:28.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.059+0000 7f61b1157640 1 -- 10.20.193.28:0/3744589399 wait complete. 2026-02-25T11:47:28.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.059+0000 7f391a159640 1 -- 10.20.193.28:0/1927573420 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f391410b360 msgr2=0x7f39141071e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.059+0000 7f391a159640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f391410b360 0x7f39141071e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.060+0000 7f61b1157640 1 Processor -- start 2026-02-25T11:47:28.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.059+0000 7f391a159640 1 -- 10.20.193.28:0/1927573420 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391410bd50 msgr2=0x7f3914173ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.059+0000 7f391a159640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391410bd50 0x7f3914173ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.059+0000 7f391a159640 1 -- 10.20.193.28:0/1927573420 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39080094b0 con 0x7f3914113620 2026-02-25T11:47:28.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.060+0000 7f3919157640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391410bd50 0x7f3914173ef0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:28.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.060+0000 7f3919958640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f391410b360 0x7f39141071e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:28.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.060+0000 7f61b1157640 1 -- start start 2026-02-25T11:47:28.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.060+0000 7f391a159640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3914113620 0x7f3914107910 secure :-1 s=READY pgs=268 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3908009f70 tx=0x7f3908002b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.060+0000 7f61b1157640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61ac10b600 0x7f61ac1152d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.060+0000 7f61b1157640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f61ac10bff0 0x7f61ac115810 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.060+0000 7f61b1157640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f61ac1117f0 0x7f61ac11abe0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.060+0000 7f61b1157640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac11b200 con 0x7f61ac1117f0 2026-02-25T11:47:28.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.060+0000 7f61b1157640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac11b370 con 0x7f61ac10b600 2026-02-25T11:47:28.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.060+0000 7f61b1157640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac11b530 con 0x7f61ac10bff0 2026-02-25T11:47:28.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.061+0000 7f3902ffd640 1 -- 10.20.193.28:0/1927573420 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f390800ea70 con 0x7f3914113620 2026-02-25T11:47:28.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.061+0000 7f3902ffd640 1 -- 10.20.193.28:0/1927573420 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f390800ebd0 con 0x7f3914113620 2026-02-25T11:47:28.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.061+0000 7f391a95a640 1 -- 10.20.193.28:0/1927573420 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391000a650 con 0x7f3914113620 2026-02-25T11:47:28.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.061+0000 7f391a95a640 1 -- 10.20.193.28:0/1927573420 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39140ebc70 con 0x7f3914113620 2026-02-25T11:47:28.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.061+0000 7f3902ffd640 1 -- 10.20.193.28:0/1927573420 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f390800f6d0 con 0x7f3914113620 2026-02-25T11:47:28.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.061+0000 7f61abfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61ac10b600 0x7f61ac1152d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.061+0000 7f61b0956640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f61ac1117f0 0x7f61ac11abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.062+0000 7f61abfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61ac10b600 0x7f61ac1152d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41770/0 (socket says 10.20.193.28:41770) 2026-02-25T11:47:28.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.062+0000 7f61abfff640 1 -- 10.20.193.28:0/589824126 learned_addr learned my addr 10.20.193.28:0/589824126 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:28.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.061+0000 7f61ab7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f61ac10bff0 0x7f61ac115810 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.062+0000 7f61abfff640 1 -- 10.20.193.28:0/589824126 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f61ac10bff0 msgr2=0x7f61ac115810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.062+0000 7f391a95a640 1 -- 10.20.193.28:0/1927573420 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f39140533d0 con 0x7f3914113620 2026-02-25T11:47:28.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.062+0000 7f61abfff640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f61ac10bff0 0x7f61ac115810 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.062+0000 7f61abfff640 1 -- 10.20.193.28:0/589824126 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f61ac1117f0 msgr2=0x7f61ac11abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.063+0000 7f61abfff640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f61ac1117f0 0x7f61ac11abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.063+0000 7f61abfff640 1 -- 10.20.193.28:0/589824126 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c0094b0 con 0x7f61ac10b600 2026-02-25T11:47:28.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.063+0000 7f61ab7fe640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f61ac10bff0 0x7f61ac115810 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:28.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.063+0000 7f61b0956640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f61ac1117f0 0x7f61ac11abe0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:28.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.063+0000 7f61abfff640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61ac10b600 0x7f61ac1152d0 secure :-1 s=READY pgs=84 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f61a000d9d0 tx=0x7f61a000dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.064+0000 7f61a97fa640 1 -- 10.20.193.28:0/589824126 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f61a0008cf0 con 0x7f61ac10b600 2026-02-25T11:47:28.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.064+0000 7f61a97fa640 1 -- 10.20.193.28:0/589824126 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f61a0008e50 con 0x7f61ac10b600 2026-02-25T11:47:28.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.064+0000 7f61a97fa640 1 -- 10.20.193.28:0/589824126 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f61a0019980 con 0x7f61ac10b600 2026-02-25T11:47:28.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.064+0000 7f61b1157640 1 -- 10.20.193.28:0/589824126 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ac11b810 con 0x7f61ac10b600 2026-02-25T11:47:28.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.064+0000 7f61b1157640 1 -- 10.20.193.28:0/589824126 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ac15bf10 con 0x7f61ac10b600 2026-02-25T11:47:28.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.064+0000 7f617effd640 1 -- 10.20.193.28:0/589824126 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f61ac0533d0 con 0x7f61ac10b600 2026-02-25T11:47:28.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.063+0000 7f6eae541640 1 -- 10.20.193.28:0/3091103360 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6e800b57f0 msgr2=0x7f6e800d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.063+0000 7f6eae541640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6e800b57f0 0x7f6e800d5ca0 secure :-1 s=READY pgs=197 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ea81189d0 tx=0x7f6e98008040 comp rx=0 tx=0).stop 2026-02-25T11:47:28.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.063+0000 7f6eae541640 1 -- 10.20.193.28:0/3091103360 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ea8114900 msgr2=0x7f6ea8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.063+0000 7f6eae541640 1 --2- 10.20.193.28:0/3091103360 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ea8114900 0x7f6ea8069e90 secure :-1 s=READY pgs=266 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6e9c0098a0 tx=0x7f6e9c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.064+0000 7f6eadd40640 1 -- 10.20.193.28:0/3091103360 reap_dead start 2026-02-25T11:47:28.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.064+0000 7f6eae541640 1 -- 10.20.193.28:0/3091103360 shutdown_connections 2026-02-25T11:47:28.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.064+0000 7f6eae541640 1 -- 10.20.193.28:0/3091103360 >> 10.20.193.28:0/3091103360 conn(0x7f6ea805ba30 msgr2=0x7f6ea8119970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.065+0000 7f6eae541640 1 -- 10.20.193.28:0/3091103360 shutdown_connections 2026-02-25T11:47:28.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.069+0000 7f3902ffd640 1 -- 10.20.193.28:0/1927573420 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f390801f060 con 0x7f3914113620 2026-02-25T11:47:28.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.071+0000 7f3902ffd640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f38f00b57f0 0x7f38f00d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.065+0000 7f6eae541640 1 -- 10.20.193.28:0/3091103360 wait complete. 2026-02-25T11:47:28.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.071+0000 7f3902ffd640 1 -- 10.20.193.28:0/1927573420 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f39080e6ea0 con 0x7f3914113620 2026-02-25T11:47:28.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.073+0000 7f3919958640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f38f00b57f0 0x7f38f00d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.073+0000 7f3902ffd640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747] conn(0x7f38f00dcf60 0x7f38f00fd370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.073+0000 7f3902ffd640 1 -- 10.20.193.28:0/1927573420 --> [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f38f00fda20 con 0x7f38f00dcf60 2026-02-25T11:47:28.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.073+0000 7f3902ffd640 1 -- 10.20.193.28:0/1927573420 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f39080e9530 con 0x7f3914113620 2026-02-25T11:47:28.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.073+0000 7f3919958640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f38f00b57f0 0x7f38f00d5ca0 secure :-1 s=READY pgs=198 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39100307c0 tx=0x7f3910003470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.080+0000 7f3919157640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747] conn(0x7f38f00dcf60 0x7f38f00fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.083+0000 7f3919157640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747] conn(0x7f38f00dcf60 0x7f38f00fd370 crc :-1 s=READY pgs=9 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.084+0000 7f3902ffd640 1 -- 10.20.193.28:0/1927573420 <== osd.7 v2:10.20.193.119:6816/2724541747 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (crc 0 0 0) 0x7f38f00fda20 con 0x7f38f00dcf60 2026-02-25T11:47:28.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.087+0000 7f61a97fa640 1 -- 10.20.193.28:0/589824126 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f61a0017020 con 0x7f61ac10b600 2026-02-25T11:47:28.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.088+0000 7f61a97fa640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f61800b58c0 0x7f61800d5d70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.092+0000 7f61ab7fe640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f61800b58c0 0x7f61800d5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.093+0000 7f61a97fa640 1 -- 10.20.193.28:0/589824126 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f61a00e7fb0 con 0x7f61ac10b600 2026-02-25T11:47:28.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.093+0000 7f61a97fa640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205] conn(0x7f61800dd030 0x7f61800fd440 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.093+0000 7f61a97fa640 1 -- 10.20.193.28:0/589824126 --> [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f61800fdaf0 con 0x7f61800dd030 2026-02-25T11:47:28.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.093+0000 7f61a97fa640 1 -- 10.20.193.28:0/589824126 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f61a00e8360 con 0x7f61ac10b600 2026-02-25T11:47:28.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.093+0000 7f61ab7fe640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f61800b58c0 0x7f61800d5d70 secure :-1 s=READY pgs=199 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6198005970 tx=0x7f6198005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.093+0000 7f61b0956640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205] conn(0x7f61800dd030 0x7f61800fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.094+0000 7f61b0956640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205] conn(0x7f61800dd030 0x7f61800fd440 crc :-1 s=READY pgs=7 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.095+0000 7f61a97fa640 1 -- 10.20.193.28:0/589824126 <== osd.0 v2:10.20.193.28:6802/668944205 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (crc 0 0 0) 0x7f61800fdaf0 con 0x7f61800dd030 2026-02-25T11:47:28.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.104+0000 7f391a95a640 1 -- 10.20.193.28:0/1927573420 --> [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f391410c480 con 0x7f38f00dcf60 2026-02-25T11:47:28.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.108+0000 7f617effd640 1 -- 10.20.193.28:0/589824126 --> [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f61ac175250 con 0x7f61800dd030 2026-02-25T11:47:28.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.108+0000 7f61a97fa640 1 -- 10.20.193.28:0/589824126 <== osd.0 v2:10.20.193.28:6802/668944205 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f61ac175250 con 0x7f61800dd030 2026-02-25T11:47:28.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.108+0000 7f617effd640 1 -- 10.20.193.28:0/589824126 >> [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205] conn(0x7f61800dd030 msgr2=0x7f61800fd440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.108+0000 7f617effd640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205] conn(0x7f61800dd030 0x7f61800fd440 crc :-1 s=READY pgs=7 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.110+0000 7f3902ffd640 1 -- 10.20.193.28:0/1927573420 <== osd.7 v2:10.20.193.119:6816/2724541747 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f391410c480 con 0x7f38f00dcf60 2026-02-25T11:47:28.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.110+0000 7f617effd640 1 -- 10.20.193.28:0/589824126 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f61800b58c0 msgr2=0x7f61800d5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.110+0000 7f617effd640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f61800b58c0 0x7f61800d5d70 secure :-1 s=READY pgs=199 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6198005970 tx=0x7f6198005900 comp rx=0 tx=0).stop 2026-02-25T11:47:28.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.110+0000 7f617effd640 1 -- 10.20.193.28:0/589824126 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61ac10b600 msgr2=0x7f61ac1152d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.110+0000 7f617effd640 1 --2- 10.20.193.28:0/589824126 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f61ac10b600 0x7f61ac1152d0 secure :-1 s=READY pgs=84 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f61a000d9d0 tx=0x7f61a000dea0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.110+0000 7f391a95a640 1 -- 10.20.193.28:0/1927573420 >> [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747] conn(0x7f38f00dcf60 msgr2=0x7f38f00fd370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.110+0000 7f391a95a640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747] conn(0x7f38f00dcf60 0x7f38f00fd370 crc :-1 s=READY pgs=9 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.110+0000 7f61b0956640 1 -- 10.20.193.28:0/589824126 reap_dead start 2026-02-25T11:47:28.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.110+0000 7f391a95a640 1 -- 10.20.193.28:0/1927573420 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f38f00b57f0 msgr2=0x7f38f00d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.111+0000 7f391a95a640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f38f00b57f0 0x7f38f00d5ca0 secure :-1 s=READY pgs=198 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39100307c0 tx=0x7f3910003470 comp rx=0 tx=0).stop 2026-02-25T11:47:28.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.111+0000 7f391a95a640 1 -- 10.20.193.28:0/1927573420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3914113620 msgr2=0x7f3914107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.111+0000 7f391a95a640 1 --2- 10.20.193.28:0/1927573420 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3914113620 0x7f3914107910 secure :-1 s=READY pgs=268 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3908009f70 tx=0x7f3908002b40 comp rx=0 tx=0).stop 2026-02-25T11:47:28.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.111+0000 7f617effd640 1 -- 10.20.193.28:0/589824126 shutdown_connections 2026-02-25T11:47:28.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.111+0000 7f617effd640 1 -- 10.20.193.28:0/589824126 >> 10.20.193.28:0/589824126 conn(0x7f61ac0e8e70 msgr2=0x7f61ac109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.113+0000 7f391a159640 1 -- 10.20.193.28:0/1927573420 reap_dead start 2026-02-25T11:47:28.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.113+0000 7f391a95a640 1 -- 10.20.193.28:0/1927573420 shutdown_connections 2026-02-25T11:47:28.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.113+0000 7f391a95a640 1 -- 10.20.193.28:0/1927573420 >> 10.20.193.28:0/1927573420 conn(0x7f39140e8e70 msgr2=0x7f39141121d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.113+0000 7f391a95a640 1 -- 10.20.193.28:0/1927573420 shutdown_connections 2026-02-25T11:47:28.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.113+0000 7f391a95a640 1 -- 10.20.193.28:0/1927573420 wait complete. 2026-02-25T11:47:28.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.115+0000 7f617effd640 1 -- 10.20.193.28:0/589824126 shutdown_connections 2026-02-25T11:47:28.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.116+0000 7f617effd640 1 -- 10.20.193.28:0/589824126 wait complete. 2026-02-25T11:47:28.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.125+0000 7f5a2c47e640 1 -- 10.20.193.28:0/366018035 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a24115a30 msgr2=0x7f5a24176400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.125+0000 7f5a2c47e640 1 --2- 10.20.193.28:0/366018035 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a24115a30 0x7f5a24176400 secure :-1 s=READY pgs=269 gs=2 cs=0 l=1 c_cookie=121712787956911835 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a2000aa80 tx=0x7f5a20038d00 comp rx=0 tx=0).stop 2026-02-25T11:47:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.125+0000 7f5a2c47e640 1 -- 10.20.193.28:0/366018035 shutdown_connections 2026-02-25T11:47:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.125+0000 7f5a2c47e640 1 --2- 10.20.193.28:0/366018035 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a24115a30 0x7f5a24176400 unknown :-1 s=CLOSED pgs=269 gs=2 cs=0 l=1 c_cookie=121712787956911835 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.125+0000 7f5a2c47e640 1 --2- 10.20.193.28:0/366018035 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5a24110230 0x7f5a24115280 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.125+0000 7f5a2c47e640 1 --2- 10.20.193.28:0/366018035 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a2410f840 0x7f5a2410fc10 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.125+0000 7f5a2c47e640 1 -- 10.20.193.28:0/366018035 >> 10.20.193.28:0/366018035 conn(0x7f5a240ed130 msgr2=0x7f5a2410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.125+0000 7f5a2c47e640 1 -- 10.20.193.28:0/366018035 shutdown_connections 2026-02-25T11:47:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.125+0000 7f5a2c47e640 1 -- 10.20.193.28:0/366018035 wait complete. 2026-02-25T11:47:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.126+0000 7f5a2c47e640 1 Processor -- start 2026-02-25T11:47:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.126+0000 7f5a2c47e640 1 -- start start 2026-02-25T11:47:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.126+0000 7f5a2c47e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a2410f840 0x7f5a2411ce90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a2c47e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a24110230 0x7f5a2411d3d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a2c47e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5a24115a30 0x7f5a241227a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a2c47e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5a24122dc0 con 0x7f5a24110230 2026-02-25T11:47:28.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a2c47e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5a24122f30 con 0x7f5a2410f840 2026-02-25T11:47:28.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a2c47e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5a241230f0 con 0x7f5a24115a30 2026-02-25T11:47:28.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a299f2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a24110230 0x7f5a2411d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a2a1f3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a2410f840 0x7f5a2411ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a299f2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a24110230 0x7f5a2411d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40746/0 (socket says 10.20.193.28:40746) 2026-02-25T11:47:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a299f2640 1 -- 10.20.193.28:0/3622576087 learned_addr learned my addr 10.20.193.28:0/3622576087 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a299f2640 1 -- 10.20.193.28:0/3622576087 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5a24115a30 msgr2=0x7f5a241227a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a299f2640 1 --2- 10.20.193.28:0/3622576087 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5a24115a30 0x7f5a241227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a299f2640 1 -- 10.20.193.28:0/3622576087 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a2410f840 msgr2=0x7f5a2411ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a299f2640 1 --2- 10.20.193.28:0/3622576087 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a2410f840 0x7f5a2411ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a299f2640 1 -- 10.20.193.28:0/3622576087 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2000a650 con 0x7f5a24110230 2026-02-25T11:47:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a2a1f3640 1 --2- 10.20.193.28:0/3622576087 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a2410f840 0x7f5a2411ce90 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.127+0000 7f5a299f2640 1 --2- 10.20.193.28:0/3622576087 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a24110230 0x7f5a2411d3d0 secure :-1 s=READY pgs=270 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a18009480 tx=0x7f5a1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.128+0000 7f5a137fe640 1 -- 10.20.193.28:0/3622576087 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a1800ccd0 con 0x7f5a24110230 2026-02-25T11:47:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.128+0000 7f5a137fe640 1 -- 10.20.193.28:0/3622576087 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5a1800ce30 con 0x7f5a24110230 2026-02-25T11:47:28.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.128+0000 7f5a137fe640 1 -- 10.20.193.28:0/3622576087 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a180106b0 con 0x7f5a24110230 2026-02-25T11:47:28.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.128+0000 7f5a2c47e640 1 -- 10.20.193.28:0/3622576087 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a241233d0 con 0x7f5a24110230 2026-02-25T11:47:28.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.128+0000 7f5a2c47e640 1 -- 10.20.193.28:0/3622576087 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a24174ee0 con 0x7f5a24110230 2026-02-25T11:47:28.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.128+0000 7f5a2c47e640 1 -- 10.20.193.28:0/3622576087 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59f4000fd0 con 0x7f5a24110230 2026-02-25T11:47:28.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.131+0000 7f5a137fe640 1 -- 10.20.193.28:0/3622576087 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f5a1801f060 con 0x7f5a24110230 2026-02-25T11:47:28.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.134+0000 7f5a137fe640 1 --2- 10.20.193.28:0/3622576087 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f59f80b57f0 0x7f59f80d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.134+0000 7f5a137fe640 1 -- 10.20.193.28:0/3622576087 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f5a18014070 con 0x7f5a24110230 2026-02-25T11:47:28.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.134+0000 7f5a137fe640 1 --2- 10.20.193.28:0/3622576087 >> [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999] conn(0x7f59f80dcf60 0x7f59f80fd370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.135+0000 7f5a137fe640 1 -- 10.20.193.28:0/3622576087 --> [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59f80fda20 con 0x7f59f80dcf60 2026-02-25T11:47:28.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.135+0000 7f5a137fe640 1 -- 10.20.193.28:0/3622576087 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f5a180e7110 con 0x7f5a24110230 2026-02-25T11:47:28.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.138+0000 7f5a2a9f4640 1 --2- 10.20.193.28:0/3622576087 >> [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999] conn(0x7f59f80dcf60 0x7f59f80fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.138+0000 7f5a2a1f3640 1 --2- 10.20.193.28:0/3622576087 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f59f80b57f0 0x7f59f80d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.145+0000 7f5a2a9f4640 1 --2- 10.20.193.28:0/3622576087 >> [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999] conn(0x7f59f80dcf60 0x7f59f80fd370 crc :-1 s=READY pgs=10 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.146+0000 7f5a2a1f3640 1 --2- 10.20.193.28:0/3622576087 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f59f80b57f0 0x7f59f80d5ca0 secure :-1 s=READY pgs=200 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a14004780 tx=0x7f5a14008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.146+0000 7f5a137fe640 1 -- 10.20.193.28:0/3622576087 <== osd.6 v2:10.20.193.28:6818/621484999 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (crc 0 0 0) 0x7f59f80fda20 con 0x7f59f80dcf60 2026-02-25T11:47:28.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.156+0000 7f5a2c47e640 1 -- 10.20.193.28:0/3622576087 --> [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f59f4002d10 con 0x7f59f80dcf60 2026-02-25T11:47:28.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.157+0000 7f5a137fe640 1 -- 10.20.193.28:0/3622576087 <== osd.6 v2:10.20.193.28:6818/621484999 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f59f4002d10 con 0x7f59f80dcf60 2026-02-25T11:47:28.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.159+0000 7f5a2c47e640 1 -- 10.20.193.28:0/3622576087 >> [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999] conn(0x7f59f80dcf60 msgr2=0x7f59f80fd370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.159+0000 7f5a2c47e640 1 --2- 10.20.193.28:0/3622576087 >> [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999] conn(0x7f59f80dcf60 0x7f59f80fd370 crc :-1 s=READY pgs=10 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.159+0000 7f5a2c47e640 1 -- 10.20.193.28:0/3622576087 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f59f80b57f0 msgr2=0x7f59f80d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.159+0000 7f5a2c47e640 1 --2- 10.20.193.28:0/3622576087 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f59f80b57f0 0x7f59f80d5ca0 secure :-1 s=READY pgs=200 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a14004780 tx=0x7f5a14008040 comp rx=0 tx=0).stop 2026-02-25T11:47:28.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.159+0000 7f5a2c47e640 1 -- 10.20.193.28:0/3622576087 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a24110230 msgr2=0x7f5a2411d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.159+0000 7f5a2c47e640 1 --2- 10.20.193.28:0/3622576087 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a24110230 0x7f5a2411d3d0 secure :-1 s=READY pgs=270 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a18009480 tx=0x7f5a1800c2d0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.159+0000 7f5a2a9f4640 1 -- 10.20.193.28:0/3622576087 reap_dead start 2026-02-25T11:47:28.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.160+0000 7f5a2c47e640 1 -- 10.20.193.28:0/3622576087 shutdown_connections 2026-02-25T11:47:28.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.160+0000 7f5a2c47e640 1 -- 10.20.193.28:0/3622576087 >> 10.20.193.28:0/3622576087 conn(0x7f5a240ed130 msgr2=0x7f5a2410d510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.160+0000 7f5a2c47e640 1 -- 10.20.193.28:0/3622576087 shutdown_connections 2026-02-25T11:47:28.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.160+0000 7f5a2c47e640 1 -- 10.20.193.28:0/3622576087 wait complete. 2026-02-25T11:47:28.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.171+0000 7f4f5c845640 1 -- 10.20.193.28:0/2510291756 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4f500c4530 msgr2=0x7f4f500c4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.171+0000 7f4f5c845640 1 --2- 10.20.193.28:0/2510291756 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4f500c4530 0x7f4f500c4900 secure :-1 s=READY pgs=87 gs=2 cs=0 l=1 c_cookie=9752914877063154800 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f4c007340 tx=0x7f4f4c02ecf0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.172+0000 7f4f5c845640 1 -- 10.20.193.28:0/2510291756 shutdown_connections 2026-02-25T11:47:28.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.172+0000 7f4f5c845640 1 --2- 10.20.193.28:0/2510291756 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4f500ca720 0x7f4f5012b0f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.172+0000 7f4f5c845640 1 --2- 10.20.193.28:0/2510291756 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f500c4f20 0x7f4f500c9f70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.172+0000 7f4f5c845640 1 --2- 10.20.193.28:0/2510291756 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4f500c4530 0x7f4f500c4900 unknown :-1 s=CLOSED pgs=87 gs=2 cs=0 l=1 c_cookie=9752914877063154800 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.172+0000 7f4f5c845640 1 -- 10.20.193.28:0/2510291756 >> 10.20.193.28:0/2510291756 conn(0x7f4f5001c630 msgr2=0x7f4f5001cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.172+0000 7f4f5c845640 1 -- 10.20.193.28:0/2510291756 shutdown_connections 2026-02-25T11:47:28.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.172+0000 7f4f5c845640 1 -- 10.20.193.28:0/2510291756 wait complete. 2026-02-25T11:47:28.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.174+0000 7f4f5c845640 1 Processor -- start 2026-02-25T11:47:28.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.174+0000 7f4f5c845640 1 -- start start 2026-02-25T11:47:28.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.175+0000 7f4f5c845640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4f500c4530 0x7f4f500a9240 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.175+0000 7f4f5c845640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f500c4f20 0x7f4f5012aff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.175+0000 7f4f5c845640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4f500ca720 0x7f4f500a5510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.175+0000 7f4f5c845640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4f500a5c10 con 0x7f4f500c4f20 2026-02-25T11:47:28.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.175+0000 7f4f5c845640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4f500a5d80 con 0x7f4f500c4530 2026-02-25T11:47:28.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.175+0000 7f4f5c845640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f500a5f40 con 0x7f4f500ca720 2026-02-25T11:47:28.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.175+0000 7f4f56d76640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4f500c4530 0x7f4f500a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.175+0000 7f4f56d76640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4f500c4530 0x7f4f500a9240 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41782/0 (socket says 10.20.193.28:41782) 2026-02-25T11:47:28.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.175+0000 7f4f56d76640 1 -- 10.20.193.28:0/1320682689 learned_addr learned my addr 10.20.193.28:0/1320682689 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:28.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.176+0000 7f4f56d76640 1 -- 10.20.193.28:0/1320682689 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4f500ca720 msgr2=0x7f4f500a5510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:47:28.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.176+0000 7f4f56d76640 1 --2- 10.20.193.28:0/1320682689 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4f500ca720 0x7f4f500a5510 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.176+0000 7f4f56d76640 1 -- 10.20.193.28:0/1320682689 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f500c4f20 msgr2=0x7f4f5012aff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:47:28.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.176+0000 7f4f56d76640 1 --2- 10.20.193.28:0/1320682689 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f500c4f20 0x7f4f5012aff0 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.176+0000 7f4f56d76640 1 -- 10.20.193.28:0/1320682689 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4000a600 con 0x7f4f500c4530 2026-02-25T11:47:28.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.176+0000 7f4f56d76640 1 --2- 10.20.193.28:0/1320682689 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4f500c4530 0x7f4f500a9240 secure :-1 s=READY pgs=85 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f4c005920 tx=0x7f4f4c008ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.176+0000 7f4f37fff640 1 -- 10.20.193.28:0/1320682689 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f4c03d070 con 0x7f4f500c4530 2026-02-25T11:47:28.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.176+0000 7f4f37fff640 1 -- 10.20.193.28:0/1320682689 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4f4c004390 con 0x7f4f500c4530 2026-02-25T11:47:28.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.177+0000 7f4f37fff640 1 -- 10.20.193.28:0/1320682689 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f4c03d070 con 0x7f4f500c4530 2026-02-25T11:47:28.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.178+0000 7f4f5c845640 1 -- 10.20.193.28:0/1320682689 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c009e20 con 0x7f4f500c4530 2026-02-25T11:47:28.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.178+0000 7f4f5c845640 1 -- 10.20.193.28:0/1320682689 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f500ba450 con 0x7f4f500c4530 2026-02-25T11:47:28.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.181+0000 7f4f37fff640 1 -- 10.20.193.28:0/1320682689 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f4f4c0026a0 con 0x7f4f500c4530 2026-02-25T11:47:28.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.182+0000 7f4f5c845640 1 -- 10.20.193.28:0/1320682689 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f50000e20 con 0x7f4f500c4530 2026-02-25T11:47:28.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.182+0000 7f4f37fff640 1 --2- 10.20.193.28:0/1320682689 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f2c0b57d0 0x7f4f2c0d5c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.183+0000 7f4f37fff640 1 -- 10.20.193.28:0/1320682689 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f4f4c10a5e0 con 0x7f4f500c4530 2026-02-25T11:47:28.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.183+0000 7f4f37fff640 1 --2- 10.20.193.28:0/1320682689 >> [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436] conn(0x7f4f2c0dcf40 0x7f4f2c0fd350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.183+0000 7f4f37fff640 1 -- 10.20.193.28:0/1320682689 --> [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f2c0fda00 con 0x7f4f2c0dcf40 2026-02-25T11:47:28.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.183+0000 7f4f37fff640 1 -- 10.20.193.28:0/1320682689 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f4f4c10cc70 con 0x7f4f500c4530 2026-02-25T11:47:28.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.183+0000 7f4f57577640 1 --2- 10.20.193.28:0/1320682689 >> [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436] conn(0x7f4f2c0dcf40 0x7f4f2c0fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.184+0000 7f4f56575640 1 --2- 10.20.193.28:0/1320682689 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f2c0b57d0 0x7f4f2c0d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.184+0000 7f4f57577640 1 --2- 10.20.193.28:0/1320682689 >> [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436] conn(0x7f4f2c0dcf40 0x7f4f2c0fd350 crc :-1 s=READY pgs=9 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.184+0000 7f4f56575640 1 --2- 10.20.193.28:0/1320682689 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f2c0b57d0 0x7f4f2c0d5c80 secure :-1 s=READY pgs=201 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f500a8fc0 tx=0x7f4f40005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.185+0000 7f4f37fff640 1 -- 10.20.193.28:0/1320682689 <== osd.3 v2:10.20.193.28:6810/2452541436 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (crc 0 0 0) 0x7f4f2c0fda00 con 0x7f4f2c0dcf40 2026-02-25T11:47:28.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.197+0000 7f4f5c845640 1 -- 10.20.193.28:0/1320682689 --> [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f5012e160 con 0x7f4f2c0dcf40 2026-02-25T11:47:28.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.197+0000 7f4f37fff640 1 -- 10.20.193.28:0/1320682689 <== osd.3 v2:10.20.193.28:6810/2452541436 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4f5012e160 con 0x7f4f2c0dcf40 2026-02-25T11:47:28.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.208+0000 7f4f35ffb640 1 -- 10.20.193.28:0/1320682689 >> [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436] conn(0x7f4f2c0dcf40 msgr2=0x7f4f2c0fd350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.208+0000 7f4f35ffb640 1 --2- 10.20.193.28:0/1320682689 >> [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436] conn(0x7f4f2c0dcf40 0x7f4f2c0fd350 crc :-1 s=READY pgs=9 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.208+0000 7f4f35ffb640 1 -- 10.20.193.28:0/1320682689 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f2c0b57d0 msgr2=0x7f4f2c0d5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.208+0000 7f4f35ffb640 1 --2- 10.20.193.28:0/1320682689 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f2c0b57d0 0x7f4f2c0d5c80 secure :-1 s=READY pgs=201 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f500a8fc0 tx=0x7f4f40005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:28.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.208+0000 7f4f35ffb640 1 -- 10.20.193.28:0/1320682689 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4f500c4530 msgr2=0x7f4f500a9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.208+0000 7f4f35ffb640 1 --2- 10.20.193.28:0/1320682689 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4f500c4530 0x7f4f500a9240 secure :-1 s=READY pgs=85 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f4c005920 tx=0x7f4f4c008ec0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.210+0000 7f4f57577640 1 -- 10.20.193.28:0/1320682689 reap_dead start 2026-02-25T11:47:28.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.210+0000 7f4f35ffb640 1 -- 10.20.193.28:0/1320682689 shutdown_connections 2026-02-25T11:47:28.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.210+0000 7f4f35ffb640 1 -- 10.20.193.28:0/1320682689 >> 10.20.193.28:0/1320682689 conn(0x7f4f5001c630 msgr2=0x7f4f500c7720 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.210+0000 7f4f35ffb640 1 -- 10.20.193.28:0/1320682689 shutdown_connections 2026-02-25T11:47:28.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.210+0000 7f4f35ffb640 1 -- 10.20.193.28:0/1320682689 wait complete. 2026-02-25T11:47:28.217 INFO:teuthology.orchestra.run.trial028.stdout:167503724570 2026-02-25T11:47:28.218 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.5 2026-02-25T11:47:28.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.216+0000 7f99cb69c640 1 -- 10.20.193.28:0/467471286 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f99c4110230 msgr2=0x7f99c4115280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.216+0000 7f99cb69c640 1 --2- 10.20.193.28:0/467471286 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f99c4110230 0x7f99c4115280 secure :-1 s=READY pgs=88 gs=1 cs=0 l=1 c_cookie=10950145819186377951 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f99b4009890 tx=0x7f99b402f0e0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.216+0000 7f99cb69c640 1 -- 10.20.193.28:0/467471286 shutdown_connections 2026-02-25T11:47:28.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.216+0000 7f99cb69c640 1 --2- 10.20.193.28:0/467471286 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99c4115a30 0x7f99c4176400 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.216+0000 7f99cb69c640 1 --2- 10.20.193.28:0/467471286 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f99c4110230 0x7f99c4115280 unknown :-1 s=CLOSED pgs=88 gs=1 cs=0 l=1 c_cookie=10950145819186377951 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.216+0000 7f99cb69c640 1 --2- 10.20.193.28:0/467471286 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99c410f840 0x7f99c410fc10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.216+0000 7f99cb69c640 1 -- 10.20.193.28:0/467471286 >> 10.20.193.28:0/467471286 conn(0x7f99c40ed130 msgr2=0x7f99c410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.217+0000 7f99cb69c640 1 -- 10.20.193.28:0/467471286 shutdown_connections 2026-02-25T11:47:28.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.217+0000 7f99cb69c640 1 -- 10.20.193.28:0/467471286 wait complete. 2026-02-25T11:47:28.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.218+0000 7f99cb69c640 1 Processor -- start 2026-02-25T11:47:28.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.218+0000 7f99cb69c640 1 -- start start 2026-02-25T11:47:28.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.218+0000 7f99cb69c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99c410f840 0x7f99c4116470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.218+0000 7f99cb69c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99c4110230 0x7f99c41169b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.218+0000 7f99cb69c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f99c4115a30 0x7f99c415b340 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.219+0000 7f99c9411640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99c410f840 0x7f99c4116470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.219+0000 7f99c9411640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99c410f840 0x7f99c4116470 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40764/0 (socket says 10.20.193.28:40764) 2026-02-25T11:47:28.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.219+0000 7f99c8c10640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99c4110230 0x7f99c41169b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.219+0000 7f99c8c10640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99c4110230 0x7f99c41169b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41794/0 (socket says 10.20.193.28:41794) 2026-02-25T11:47:28.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.219+0000 7f99c9411640 1 -- 10.20.193.28:0/303492257 learned_addr learned my addr 10.20.193.28:0/303492257 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:28.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.219+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f99c415b880 con 0x7f99c410f840 2026-02-25T11:47:28.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.219+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f99c415b9f0 con 0x7f99c4110230 2026-02-25T11:47:28.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.219+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f99c415bbb0 con 0x7f99c4115a30 2026-02-25T11:47:28.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.219+0000 7f99c9c12640 1 --2- 10.20.193.28:0/303492257 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f99c4115a30 0x7f99c415b340 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.219+0000 7f99c9411640 1 -- 10.20.193.28:0/303492257 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f99c4115a30 msgr2=0x7f99c415b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.219+0000 7f99c9411640 1 --2- 10.20.193.28:0/303492257 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f99c4115a30 0x7f99c415b340 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.219+0000 7f99c9411640 1 -- 10.20.193.28:0/303492257 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99c4110230 msgr2=0x7f99c41169b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.219+0000 7f99c9411640 1 --2- 10.20.193.28:0/303492257 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99c4110230 0x7f99c41169b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.220+0000 7f99c9411640 1 -- 10.20.193.28:0/303492257 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b40094b0 con 0x7f99c410f840 2026-02-25T11:47:28.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.220+0000 7f99c9411640 1 --2- 10.20.193.28:0/303492257 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99c410f840 0x7f99c4116470 secure :-1 s=READY pgs=271 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f99b800d950 tx=0x7f99b800de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.221+0000 7f99b27fc640 1 -- 10.20.193.28:0/303492257 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f99b8004500 con 0x7f99c410f840 2026-02-25T11:47:28.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.221+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c415be90 con 0x7f99c410f840 2026-02-25T11:47:28.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.221+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c415c3b0 con 0x7f99c410f840 2026-02-25T11:47:28.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.221+0000 7f99b27fc640 1 -- 10.20.193.28:0/303492257 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f99b800bc80 con 0x7f99c410f840 2026-02-25T11:47:28.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.221+0000 7f99b27fc640 1 -- 10.20.193.28:0/303492257 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f99b8010460 con 0x7f99c410f840 2026-02-25T11:47:28.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.223+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99c40533d0 con 0x7f99c410f840 2026-02-25T11:47:28.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.224+0000 7f99b27fc640 1 -- 10.20.193.28:0/303492257 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f99b80105f0 con 0x7f99c410f840 2026-02-25T11:47:28.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.224+0000 7f9e425df640 1 -- 10.20.193.28:0/4116549448 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3c115a00 msgr2=0x7f9e3c1763d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.224+0000 7f9e425df640 1 --2- 10.20.193.28:0/4116549448 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3c115a00 0x7f9e3c1763d0 secure :-1 s=READY pgs=86 gs=2 cs=0 l=1 c_cookie=316006834883762607 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9e3000a970 tx=0x7f9e300301b0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.225+0000 7f9e425df640 1 -- 10.20.193.28:0/4116549448 shutdown_connections 2026-02-25T11:47:28.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.225+0000 7f9e425df640 1 --2- 10.20.193.28:0/4116549448 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3c115a00 0x7f9e3c1763d0 unknown :-1 s=CLOSED pgs=86 gs=2 cs=0 l=1 c_cookie=316006834883762607 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.225+0000 7f9e425df640 1 --2- 10.20.193.28:0/4116549448 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3c110200 0x7f9e3c115250 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.225+0000 7f9e425df640 1 --2- 10.20.193.28:0/4116549448 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9e3c10f810 0x7f9e3c10fbe0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.225+0000 7f9e425df640 1 -- 10.20.193.28:0/4116549448 >> 10.20.193.28:0/4116549448 conn(0x7f9e3c0ed100 msgr2=0x7f9e3c10d510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.225+0000 7f9e425df640 1 -- 10.20.193.28:0/4116549448 shutdown_connections 2026-02-25T11:47:28.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.225+0000 7f99b27fc640 1 --2- 10.20.193.28:0/303492257 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f99940b57f0 0x7f99940d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.225+0000 7f9e425df640 1 -- 10.20.193.28:0/4116549448 wait complete. 2026-02-25T11:47:28.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.225+0000 7f99b27fc640 1 -- 10.20.193.28:0/303492257 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f99b80e6aa0 con 0x7f99c410f840 2026-02-25T11:47:28.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.225+0000 7f99c8c10640 1 --2- 10.20.193.28:0/303492257 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f99940b57f0 0x7f99940d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.225+0000 7f99b27fc640 1 --2- 10.20.193.28:0/303492257 >> [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625] conn(0x7f99940dcf60 0x7f99940fd370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.225+0000 7f99b27fc640 1 -- 10.20.193.28:0/303492257 --> [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99940fda20 con 0x7f99940dcf60 2026-02-25T11:47:28.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.225+0000 7f99b27fc640 1 -- 10.20.193.28:0/303492257 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f99b80eb2c0 con 0x7f99c410f840 2026-02-25T11:47:28.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.226+0000 7f99c8c10640 1 --2- 10.20.193.28:0/303492257 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f99940b57f0 0x7f99940d5ca0 secure :-1 s=READY pgs=202 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f99c4114650 tx=0x7f99b403a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.227+0000 7f9e425df640 1 Processor -- start 2026-02-25T11:47:28.234 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.226+0000 7f99c9c12640 1 --2- 10.20.193.28:0/303492257 >> [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625] conn(0x7f99940dcf60 0x7f99940fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.234 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.227+0000 7f9e425df640 1 -- start start 2026-02-25T11:47:28.234 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.227+0000 7f99c9c12640 1 --2- 10.20.193.28:0/303492257 >> [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625] conn(0x7f99940dcf60 0x7f99940fd370 crc :-1 s=READY pgs=9 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.228+0000 7f9e425df640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3c10f810 0x7f9e3c0f5ec0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.228+0000 7f9e425df640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3c110200 0x7f9e3c1762d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.228+0000 7f9e425df640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9e3c115a00 0x7f9e3c0f65c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.228+0000 7f9e425df640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c0fb7d0 con 0x7f9e3c110200 2026-02-25T11:47:28.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.228+0000 7f9e425df640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c0fb940 con 0x7f9e3c10f810 2026-02-25T11:47:28.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.228+0000 7f9e425df640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c0fbb00 con 0x7f9e3c115a00 2026-02-25T11:47:28.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.228+0000 7f9e3bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3c10f810 0x7f9e3c0f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.228+0000 7f9e3bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3c10f810 0x7f9e3c0f5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41818/0 (socket says 10.20.193.28:41818) 2026-02-25T11:47:28.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.228+0000 7f9e3bfff640 1 -- 10.20.193.28:0/3599542594 learned_addr learned my addr 10.20.193.28:0/3599542594 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:28.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.228+0000 7f99b27fc640 1 -- 10.20.193.28:0/303492257 <== osd.9 v2:10.20.193.28:6826/354500625 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (crc 0 0 0) 0x7f99940fda20 con 0x7f99940dcf60 2026-02-25T11:47:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.229+0000 7f9e3bfff640 1 -- 10.20.193.28:0/3599542594 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9e3c115a00 msgr2=0x7f9e3c0f65c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:47:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.229+0000 7f9e3bfff640 1 --2- 10.20.193.28:0/3599542594 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9e3c115a00 0x7f9e3c0f65c0 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.229+0000 7f9e3bfff640 1 -- 10.20.193.28:0/3599542594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3c110200 msgr2=0x7f9e3c1762d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:47:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.229+0000 7f9e3bfff640 1 --2- 10.20.193.28:0/3599542594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9e3c110200 0x7f9e3c1762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.229+0000 7f9e3bfff640 1 -- 10.20.193.28:0/3599542594 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3000a590 con 0x7f9e3c10f810 2026-02-25T11:47:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.229+0000 7f9e3bfff640 1 --2- 10.20.193.28:0/3599542594 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3c10f810 0x7f9e3c0f5ec0 secure :-1 s=READY pgs=87 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9e2c00d950 tx=0x7f9e2c00de20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.229+0000 7f9e397fa640 1 -- 10.20.193.28:0/3599542594 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9e2c004540 con 0x7f9e3c10f810 2026-02-25T11:47:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.229+0000 7f9e397fa640 1 -- 10.20.193.28:0/3599542594 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9e2c009ce0 con 0x7f9e3c10f810 2026-02-25T11:47:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.230+0000 7f9e397fa640 1 -- 10.20.193.28:0/3599542594 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9e2c0106e0 con 0x7f9e3c10f810 2026-02-25T11:47:28.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.230+0000 7f9e425df640 1 -- 10.20.193.28:0/3599542594 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3c0fbde0 con 0x7f9e3c10f810 2026-02-25T11:47:28.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.230+0000 7f9e425df640 1 -- 10.20.193.28:0/3599542594 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e3c0fc260 con 0x7f9e3c10f810 2026-02-25T11:47:28.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.232+0000 7f9e0effd640 1 -- 10.20.193.28:0/3599542594 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e3c0533d0 con 0x7f9e3c10f810 2026-02-25T11:47:28.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.235+0000 7f9e397fa640 1 -- 10.20.193.28:0/3599542594 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f9e2c0097c0 con 0x7f9e3c10f810 2026-02-25T11:47:28.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.237+0000 7f9e397fa640 1 --2- 10.20.193.28:0/3599542594 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9e100b57e0 0x7f9e100d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.237+0000 7f9e397fa640 1 -- 10.20.193.28:0/3599542594 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f9e2c0e6f70 con 0x7f9e3c10f810 2026-02-25T11:47:28.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.237+0000 7f9e397fa640 1 --2- 10.20.193.28:0/3599542594 >> [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559] conn(0x7f9e100dcf50 0x7f9e100fd360 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.238+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 --> [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f99c41798a0 con 0x7f99940dcf60 2026-02-25T11:47:28.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.237+0000 7f9e397fa640 1 -- 10.20.193.28:0/3599542594 --> [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e100fda10 con 0x7f9e100dcf50 2026-02-25T11:47:28.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.237+0000 7f9e397fa640 1 -- 10.20.193.28:0/3599542594 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9e2c0e72a0 con 0x7f9e3c10f810 2026-02-25T11:47:28.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.238+0000 7f9e40b55640 1 --2- 10.20.193.28:0/3599542594 >> [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559] conn(0x7f9e100dcf50 0x7f9e100fd360 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.238+0000 7f99b27fc640 1 -- 10.20.193.28:0/303492257 <== osd.9 v2:10.20.193.28:6826/354500625 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f99c41798a0 con 0x7f99940dcf60 2026-02-25T11:47:28.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.239+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 >> [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625] conn(0x7f99940dcf60 msgr2=0x7f99940fd370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.239+0000 7f99cb69c640 1 --2- 10.20.193.28:0/303492257 >> [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625] conn(0x7f99940dcf60 0x7f99940fd370 crc :-1 s=READY pgs=9 gs=5 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.239+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f99940b57f0 msgr2=0x7f99940d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.239+0000 7f99cb69c640 1 --2- 10.20.193.28:0/303492257 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f99940b57f0 0x7f99940d5ca0 secure :-1 s=READY pgs=202 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f99c4114650 tx=0x7f99b403a040 comp rx=0 tx=0).stop 2026-02-25T11:47:28.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.239+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99c410f840 msgr2=0x7f99c4116470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.239+0000 7f99cb69c640 1 --2- 10.20.193.28:0/303492257 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99c410f840 0x7f99c4116470 secure :-1 s=READY pgs=271 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f99b800d950 tx=0x7f99b800de20 comp rx=0 tx=0).stop 2026-02-25T11:47:28.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.239+0000 7f9e3b7fe640 1 --2- 10.20.193.28:0/3599542594 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9e100b57e0 0x7f9e100d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.240+0000 7f99c9c12640 1 -- 10.20.193.28:0/303492257 reap_dead start 2026-02-25T11:47:28.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.240+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 shutdown_connections 2026-02-25T11:47:28.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.240+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 >> 10.20.193.28:0/303492257 conn(0x7f99c40ed130 msgr2=0x7f99c410d510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.240+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 shutdown_connections 2026-02-25T11:47:28.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.240+0000 7f99cb69c640 1 -- 10.20.193.28:0/303492257 wait complete. 2026-02-25T11:47:28.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.241+0000 7f9e3b7fe640 1 --2- 10.20.193.28:0/3599542594 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9e100b57e0 0x7f9e100d5c90 secure :-1 s=READY pgs=203 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9e28005b30 tx=0x7f9e28005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.241+0000 7f9e40b55640 1 --2- 10.20.193.28:0/3599542594 >> [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559] conn(0x7f9e100dcf50 0x7f9e100fd360 crc :-1 s=READY pgs=12 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.243+0000 7f9e397fa640 1 -- 10.20.193.28:0/3599542594 <== osd.2 v2:10.20.193.167:6800/484781559 1 ==== command_reply(tid 1: 0 ) ==== 8+0+33658 (crc 0 0 0) 0x7f9e100fda10 con 0x7f9e100dcf50 2026-02-25T11:47:28.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.255+0000 7f9e0effd640 1 -- 10.20.193.28:0/3599542594 --> [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9e3c060c10 con 0x7f9e100dcf50 2026-02-25T11:47:28.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.255+0000 7f9e397fa640 1 -- 10.20.193.28:0/3599542594 <== osd.2 v2:10.20.193.167:6800/484781559 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9e3c060c10 con 0x7f9e100dcf50 2026-02-25T11:47:28.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.256+0000 7f9e0effd640 1 -- 10.20.193.28:0/3599542594 >> [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559] conn(0x7f9e100dcf50 msgr2=0x7f9e100fd360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.256+0000 7f9e0effd640 1 --2- 10.20.193.28:0/3599542594 >> [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559] conn(0x7f9e100dcf50 0x7f9e100fd360 crc :-1 s=READY pgs=12 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.256+0000 7f9e0effd640 1 -- 10.20.193.28:0/3599542594 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9e100b57e0 msgr2=0x7f9e100d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.256+0000 7f9e0effd640 1 --2- 10.20.193.28:0/3599542594 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9e100b57e0 0x7f9e100d5c90 secure :-1 s=READY pgs=203 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9e28005b30 tx=0x7f9e28005900 comp rx=0 tx=0).stop 2026-02-25T11:47:28.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.256+0000 7f9e0effd640 1 -- 10.20.193.28:0/3599542594 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3c10f810 msgr2=0x7f9e3c0f5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.256+0000 7f9e0effd640 1 --2- 10.20.193.28:0/3599542594 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9e3c10f810 0x7f9e3c0f5ec0 secure :-1 s=READY pgs=87 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9e2c00d950 tx=0x7f9e2c00de20 comp rx=0 tx=0).stop 2026-02-25T11:47:28.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.256+0000 7f9e40b55640 1 -- 10.20.193.28:0/3599542594 reap_dead start 2026-02-25T11:47:28.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.259+0000 7f9e0effd640 1 -- 10.20.193.28:0/3599542594 shutdown_connections 2026-02-25T11:47:28.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.259+0000 7f9e0effd640 1 -- 10.20.193.28:0/3599542594 >> 10.20.193.28:0/3599542594 conn(0x7f9e3c0ed100 msgr2=0x7f9e3c10dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.260+0000 7f9e0effd640 1 -- 10.20.193.28:0/3599542594 shutdown_connections 2026-02-25T11:47:28.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.260+0000 7f9e0effd640 1 -- 10.20.193.28:0/3599542594 wait complete. 2026-02-25T11:47:28.264 INFO:teuthology.orchestra.run.trial028.stdout:347892350987 2026-02-25T11:47:28.264 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.11 2026-02-25T11:47:28.316 INFO:teuthology.orchestra.run.trial028.stdout:257698037778 2026-02-25T11:47:28.317 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.8 2026-02-25T11:47:28.417 INFO:teuthology.orchestra.run.trial028.stdout:146028888093 2026-02-25T11:47:28.418 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.4 2026-02-25T11:47:28.459 INFO:teuthology.orchestra.run.trial028.stdout:313532612621 2026-02-25T11:47:28.459 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.10 2026-02-25T11:47:28.555 INFO:teuthology.orchestra.run.trial028.stdout:201863462937 2026-02-25T11:47:28.555 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.6 2026-02-25T11:47:28.566 INFO:teuthology.orchestra.run.trial028.stdout:38654705704 2026-02-25T11:47:28.567 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.0 2026-02-25T11:47:28.587 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:28.606 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:28.607 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:28.613 INFO:teuthology.orchestra.run.trial028.stdout:60129542181 2026-02-25T11:47:28.614 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.1 2026-02-25T11:47:28.620 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:28.665 INFO:teuthology.orchestra.run.trial028.stdout:111669149728 2026-02-25T11:47:28.665 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.3 2026-02-25T11:47:28.670 INFO:teuthology.orchestra.run.trial028.stdout:223338299413 2026-02-25T11:47:28.670 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.7 2026-02-25T11:47:28.671 INFO:teuthology.orchestra.run.trial028.stdout:292057776143 2026-02-25T11:47:28.671 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.9 2026-02-25T11:47:28.677 INFO:teuthology.orchestra.run.trial028.stdout:81604378659 2026-02-25T11:47:28.690 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.2 2026-02-25T11:47:28.777 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.876+0000 7fcfcd91e640 1 -- 10.20.193.28:0/2388555266 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcfc8114900 msgr2=0x7fcfc8114cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.876+0000 7fcfcd91e640 1 --2- 10.20.193.28:0/2388555266 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcfc8114900 0x7fcfc8114cd0 secure :-1 s=READY pgs=88 gs=2 cs=0 l=1 c_cookie=14357233940283665985 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcfb8007520 tx=0x7fcfb802fd80 comp rx=0 tx=0).stop 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.876+0000 7fcfcd91e640 1 -- 10.20.193.28:0/2388555266 shutdown_connections 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.876+0000 7fcfcd91e640 1 --2- 10.20.193.28:0/2388555266 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcfc8117120 0x7fcfc817ace0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.876+0000 7fcfcd91e640 1 --2- 10.20.193.28:0/2388555266 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfc81152f0 0x7fcfc8116940 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.876+0000 7fcfcd91e640 1 --2- 10.20.193.28:0/2388555266 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcfc8114900 0x7fcfc8114cd0 unknown :-1 s=CLOSED pgs=88 gs=2 cs=0 l=1 c_cookie=14357233940283665985 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.876+0000 7fcfcd91e640 1 -- 10.20.193.28:0/2388555266 >> 10.20.193.28:0/2388555266 conn(0x7fcfc8057d30 msgr2=0x7fcfc8058130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.876+0000 7fcfcd91e640 1 -- 10.20.193.28:0/2388555266 shutdown_connections 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.877+0000 7fcfcd91e640 1 -- 10.20.193.28:0/2388555266 wait complete. 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.878+0000 7fcfcd91e640 1 Processor -- start 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.878+0000 7fcfcd91e640 1 -- start start 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.878+0000 7fcfcd91e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcfc8114900 0x7fcfc811ffd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.878+0000 7fcfcd91e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcfc81152f0 0x7fcfc8120510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfcd91e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfc8117120 0x7fcfc8125890 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfc6ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcfc8114900 0x7fcfc811ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfc6ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcfc8114900 0x7fcfc811ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41848/0 (socket says 10.20.193.28:41848) 2026-02-25T11:47:28.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfcd91e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc8125eb0 con 0x7fcfc8117120 2026-02-25T11:47:28.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfcd91e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc8126020 con 0x7fcfc8114900 2026-02-25T11:47:28.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfcd91e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc81261e0 con 0x7fcfc81152f0 2026-02-25T11:47:28.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfc6ffd640 1 -- 10.20.193.28:0/1793927631 learned_addr learned my addr 10.20.193.28:0/1793927631 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:28.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fca71b5c640 1 -- 10.20.193.28:0/4216231724 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c1152f0 msgr2=0x7fca6c116800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fca71b5c640 1 --2- 10.20.193.28:0/4216231724 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c1152f0 0x7fca6c116800 secure :-1 s=READY pgs=89 gs=1 cs=0 l=1 c_cookie=2817926311031680646 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca60009950 tx=0x7fca6002f1b0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fca71b5c640 1 -- 10.20.193.28:0/4216231724 shutdown_connections 2026-02-25T11:47:28.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fca71b5c640 1 --2- 10.20.193.28:0/4216231724 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c116fe0 0x7fca6c17aba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fca71b5c640 1 --2- 10.20.193.28:0/4216231724 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c1152f0 0x7fca6c116800 unknown :-1 s=CLOSED pgs=89 gs=1 cs=0 l=1 c_cookie=2817926311031680646 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.880+0000 7fca71b5c640 1 --2- 10.20.193.28:0/4216231724 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c114900 0x7fca6c114cd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.880+0000 7fca71b5c640 1 -- 10.20.193.28:0/4216231724 >> 10.20.193.28:0/4216231724 conn(0x7fca6c05ba30 msgr2=0x7fca6c05be30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.880+0000 7fca71b5c640 1 -- 10.20.193.28:0/4216231724 shutdown_connections 2026-02-25T11:47:28.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.880+0000 7fca71b5c640 1 -- 10.20.193.28:0/4216231724 wait complete. 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfc77fe640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfc8117120 0x7fcfc8125890 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfc67fc640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcfc81152f0 0x7fcfc8120510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfc6ffd640 1 -- 10.20.193.28:0/1793927631 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcfc81152f0 msgr2=0x7fcfc8120510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfc6ffd640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcfc81152f0 0x7fcfc8120510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfc6ffd640 1 -- 10.20.193.28:0/1793927631 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfc8117120 msgr2=0x7fcfc8125890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfc6ffd640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfc8117120 0x7fcfc8125890 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfc6ffd640 1 -- 10.20.193.28:0/1793927631 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc000a650 con 0x7fcfc8114900 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.879+0000 7fcfc77fe640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfc8117120 0x7fcfc8125890 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.880+0000 7fcfc6ffd640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcfc8114900 0x7fcfc811ffd0 secure :-1 s=READY pgs=90 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcfb80074f0 tx=0x7fcfb8030890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.880+0000 7fcfcc91c640 1 -- 10.20.193.28:0/1793927631 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcfb8031d80 con 0x7fcfc8114900 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.880+0000 7fcfcc91c640 1 -- 10.20.193.28:0/1793927631 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcfb8004510 con 0x7fcfc8114900 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.880+0000 7fcfcc91c640 1 -- 10.20.193.28:0/1793927631 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcfb800a450 con 0x7fcfc8114900 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.880+0000 7fcfcd91e640 1 -- 10.20.193.28:0/1793927631 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb80071c0 con 0x7fcfc8114900 2026-02-25T11:47:28.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.881+0000 7fcfcd91e640 1 -- 10.20.193.28:0/1793927631 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc8166a20 con 0x7fcfc8114900 2026-02-25T11:47:28.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.882+0000 7fca71b5c640 1 Processor -- start 2026-02-25T11:47:28.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.882+0000 7fca71b5c640 1 -- start start 2026-02-25T11:47:28.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.882+0000 7fca71b5c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c114900 0x7fca6c12a2a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.882+0000 7fca71b5c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c1152f0 0x7fca6c1228f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.883+0000 7fca70b5a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c114900 0x7fca6c12a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.883+0000 7fca70b5a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c114900 0x7fca6c12a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52346/0 (socket says 10.20.193.28:52346) 2026-02-25T11:47:28.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.883+0000 7fca70b5a640 1 -- 10.20.193.28:0/369915339 learned_addr learned my addr 10.20.193.28:0/369915339 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:28.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.883+0000 7fca71b5c640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c116fe0 0x7fca6c122fa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.883+0000 7fca6bfff640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c1152f0 0x7fca6c1228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.883+0000 7fca71b5c640 1 -- 10.20.193.28:0/369915339 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c1236a0 con 0x7fca6c116fe0 2026-02-25T11:47:28.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.883+0000 7fca71b5c640 1 -- 10.20.193.28:0/369915339 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c126e70 con 0x7fca6c1152f0 2026-02-25T11:47:28.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.883+0000 7fca71b5c640 1 -- 10.20.193.28:0/369915339 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c127030 con 0x7fca6c114900 2026-02-25T11:47:28.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.883+0000 7fca7135b640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c116fe0 0x7fca6c122fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.883+0000 7fcfcd91e640 1 -- 10.20.193.28:0/1793927631 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc805aba0 con 0x7fcfc8114900 2026-02-25T11:47:28.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.884+0000 7fcfcc91c640 1 -- 10.20.193.28:0/1793927631 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fcfb8004050 con 0x7fcfc8114900 2026-02-25T11:47:28.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.886+0000 7fcfcc91c640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcf9c0b57f0 0x7fcf9c0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.886+0000 7fcfcc91c640 1 -- 10.20.193.28:0/1793927631 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fcfb810b290 con 0x7fcfc8114900 2026-02-25T11:47:28.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.884+0000 7fca7135b640 1 -- 10.20.193.28:0/369915339 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c114900 msgr2=0x7fca6c12a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.884+0000 7fca7135b640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c114900 0x7fca6c12a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.884+0000 7fca7135b640 1 -- 10.20.193.28:0/369915339 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c1152f0 msgr2=0x7fca6c1228f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.884+0000 7fca7135b640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c1152f0 0x7fca6c1228f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.884+0000 7fca7135b640 1 -- 10.20.193.28:0/369915339 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca60009570 con 0x7fca6c116fe0 2026-02-25T11:47:28.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.884+0000 7fca7135b640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c116fe0 0x7fca6c122fa0 secure :-1 s=READY pgs=272 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca5c00f940 tx=0x7fca5c00fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.885+0000 7fca69ffb640 1 -- 10.20.193.28:0/369915339 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fca5c00dce0 con 0x7fca6c116fe0 2026-02-25T11:47:28.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.885+0000 7fca69ffb640 1 -- 10.20.193.28:0/369915339 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fca5c00de40 con 0x7fca6c116fe0 2026-02-25T11:47:28.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.886+0000 7fca69ffb640 1 -- 10.20.193.28:0/369915339 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fca5c0116b0 con 0x7fca6c116fe0 2026-02-25T11:47:28.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.886+0000 7fca71b5c640 1 -- 10.20.193.28:0/369915339 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c127310 con 0x7fca6c116fe0 2026-02-25T11:47:28.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.886+0000 7fca71b5c640 1 -- 10.20.193.28:0/369915339 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c127790 con 0x7fca6c116fe0 2026-02-25T11:47:28.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.887+0000 7fca71b5c640 1 -- 10.20.193.28:0/369915339 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca6c05e8e0 con 0x7fca6c116fe0 2026-02-25T11:47:28.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.889+0000 7fcfcc91c640 1 -- 10.20.193.28:0/1793927631 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fcfb80cdcf0 con 0x7fcfc8114900 2026-02-25T11:47:28.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.892+0000 7fcfc67fc640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcf9c0b57f0 0x7fcf9c0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.894+0000 7fcfc67fc640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcf9c0b57f0 0x7fcf9c0d5ca0 secure :-1 s=READY pgs=205 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcfc0006c10 tx=0x7fcfc000a390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.897 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:28.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.890+0000 7fca69ffb640 1 -- 10.20.193.28:0/369915339 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fca5c020060 con 0x7fca6c116fe0 2026-02-25T11:47:28.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.892+0000 7fca69ffb640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca340b58c0 0x7fca340d5d70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.892+0000 7fca69ffb640 1 -- 10.20.193.28:0/369915339 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fca5c015070 con 0x7fca6c116fe0 2026-02-25T11:47:28.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.892+0000 7fca70b5a640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca340b58c0 0x7fca340d5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.893+0000 7fca70b5a640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca340b58c0 0x7fca340d5d70 secure :-1 s=READY pgs=204 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca54005fd0 tx=0x7fca5400a480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.895+0000 7fca69ffb640 1 -- 10.20.193.28:0/369915339 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fca5c0aa590 con 0x7fca6c116fe0 2026-02-25T11:47:28.939 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:28.965 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:28.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.966+0000 7fdab83b5640 1 -- 10.20.193.28:0/2679314860 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdaa80c4520 msgr2=0x7fdaa80c48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.966+0000 7fdab83b5640 1 --2- 10.20.193.28:0/2679314860 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdaa80c4520 0x7fdaa80c48f0 secure :-1 s=READY pgs=273 gs=1 cs=0 l=1 c_cookie=12899661239052245049 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdaa0009950 tx=0x7fdaa002f1d0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.971+0000 7fdab83b5640 1 -- 10.20.193.28:0/2679314860 shutdown_connections 2026-02-25T11:47:28.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.971+0000 7fdab83b5640 1 --2- 10.20.193.28:0/2679314860 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdaa80ca710 0x7fdaa812b0e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.972+0000 7fdab83b5640 1 --2- 10.20.193.28:0/2679314860 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdaa80c4f10 0x7fdaa80c9f60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.972+0000 7fdab83b5640 1 --2- 10.20.193.28:0/2679314860 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdaa80c4520 0x7fdaa80c48f0 unknown :-1 s=CLOSED pgs=273 gs=1 cs=0 l=1 c_cookie=12899661239052245049 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.972+0000 7fdab83b5640 1 -- 10.20.193.28:0/2679314860 >> 10.20.193.28:0/2679314860 conn(0x7fdaa801c630 msgr2=0x7fdaa801cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.972+0000 7fdab83b5640 1 -- 10.20.193.28:0/2679314860 shutdown_connections 2026-02-25T11:47:28.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.972+0000 7fdab83b5640 1 -- 10.20.193.28:0/2679314860 wait complete. 2026-02-25T11:47:28.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.972+0000 7fdab83b5640 1 Processor -- start 2026-02-25T11:47:28.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.973+0000 7fdab83b5640 1 -- start start 2026-02-25T11:47:28.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.973+0000 7fdab83b5640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdaa80c4520 0x7fdaa80d1950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.973+0000 7fdab83b5640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdaa80c4f10 0x7fdaa80d1e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.973+0000 7fdab83b5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdaa80ca710 0x7fdaa80d5a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.967+0000 7f65b5e4b640 1 -- 10.20.193.28:0/134688134 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f65b01119b0 msgr2=0x7f65b0172380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.967+0000 7f65b5e4b640 1 --2- 10.20.193.28:0/134688134 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f65b01119b0 0x7f65b0172380 secure :-1 s=READY pgs=91 gs=2 cs=0 l=1 c_cookie=8192591322950884260 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f65a0009890 tx=0x7f65a002f140 comp rx=0 tx=0).stop 2026-02-25T11:47:28.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.967+0000 7f65b5e4b640 1 -- 10.20.193.28:0/134688134 shutdown_connections 2026-02-25T11:47:28.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.967+0000 7f65b5e4b640 1 --2- 10.20.193.28:0/134688134 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f65b01119b0 0x7f65b0172380 unknown :-1 s=CLOSED pgs=91 gs=2 cs=0 l=1 c_cookie=8192591322950884260 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.967+0000 7f65b5e4b640 1 --2- 10.20.193.28:0/134688134 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f65b010c1b0 0x7f65b0111200 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.980 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.967+0000 7f65b5e4b640 1 --2- 10.20.193.28:0/134688134 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f65b010b7c0 0x7f65b010bb90 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.967+0000 7f65b5e4b640 1 -- 10.20.193.28:0/134688134 >> 10.20.193.28:0/134688134 conn(0x7f65b00e93e0 msgr2=0x7f65b01097c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:28.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.967+0000 7f65b5e4b640 1 -- 10.20.193.28:0/134688134 shutdown_connections 2026-02-25T11:47:28.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.967+0000 7f65b5e4b640 1 -- 10.20.193.28:0/134688134 wait complete. 2026-02-25T11:47:28.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.970+0000 7f65b5e4b640 1 Processor -- start 2026-02-25T11:47:28.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.971+0000 7f65b5e4b640 1 -- start start 2026-02-25T11:47:28.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.971+0000 7f65b5e4b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f65b010b7c0 0x7f65b0114bd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.971+0000 7f65b5e4b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f65b010c1b0 0x7f65b0115110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.971+0000 7f65b5e4b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f65b01119b0 0x7f65b011a4e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.971+0000 7f65b5e4b640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f65b011ab00 con 0x7f65b010b7c0 2026-02-25T11:47:28.981 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.971+0000 7f65b5e4b640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f65b011ac70 con 0x7f65b010c1b0 2026-02-25T11:47:28.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.971+0000 7f65b5e4b640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f65b011ae30 con 0x7f65b01119b0 2026-02-25T11:47:28.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.972+0000 7f65affff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f65b01119b0 0x7f65b011a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.972+0000 7f65affff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f65b01119b0 0x7f65b011a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52364/0 (socket says 10.20.193.28:52364) 2026-02-25T11:47:28.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.972+0000 7f65affff640 1 -- 10.20.193.28:0/185692718 learned_addr learned my addr 10.20.193.28:0/185692718 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:28.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.972+0000 7f65aeffd640 1 --2- 10.20.193.28:0/185692718 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f65b010c1b0 0x7f65b0115110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.974+0000 7f65affff640 1 -- 10.20.193.28:0/185692718 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f65b010c1b0 msgr2=0x7f65b0115110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.974+0000 7f65affff640 1 --2- 10.20.193.28:0/185692718 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f65b010c1b0 0x7f65b0115110 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.974+0000 7f65affff640 1 -- 10.20.193.28:0/185692718 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f65b010b7c0 msgr2=0x7f65b0114bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.974+0000 7f65affff640 1 --2- 10.20.193.28:0/185692718 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f65b010b7c0 0x7f65b0114bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.982 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.974+0000 7f65affff640 1 -- 10.20.193.28:0/185692718 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c009570 con 0x7f65b01119b0 2026-02-25T11:47:28.983 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.974+0000 7f65affff640 1 --2- 10.20.193.28:0/185692718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f65b01119b0 0x7f65b011a4e0 secure :-1 s=READY pgs=89 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f65a0002d50 tx=0x7f65a0002d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.983 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.975+0000 7f65acff9640 1 -- 10.20.193.28:0/185692718 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f65a003d070 con 0x7f65b01119b0 2026-02-25T11:47:28.983 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.975+0000 7f65acff9640 1 -- 10.20.193.28:0/185692718 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f65a00044f0 con 0x7f65b01119b0 2026-02-25T11:47:28.983 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.975+0000 7f65acff9640 1 -- 10.20.193.28:0/185692718 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f65a0005710 con 0x7f65b01119b0 2026-02-25T11:47:28.983 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.975+0000 7f65b5e4b640 1 -- 10.20.193.28:0/185692718 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a00094b0 con 0x7f65b01119b0 2026-02-25T11:47:28.983 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.975+0000 7f65b5e4b640 1 -- 10.20.193.28:0/185692718 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b011b2b0 con 0x7f65b01119b0 2026-02-25T11:47:28.983 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.977+0000 7f65b5e4b640 1 -- 10.20.193.28:0/185692718 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f657c005300 con 0x7f65b01119b0 2026-02-25T11:47:28.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.981+0000 7f65acff9640 1 -- 10.20.193.28:0/185692718 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f65a0048060 con 0x7f65b01119b0 2026-02-25T11:47:28.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.982+0000 7f65acff9640 1 --2- 10.20.193.28:0/185692718 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65840b57f0 0x7f65840d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.983+0000 7f65acff9640 1 -- 10.20.193.28:0/185692718 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f65a010b250 con 0x7f65b01119b0 2026-02-25T11:47:28.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.983+0000 7f65acff9640 1 -- 10.20.193.28:0/185692718 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f65a010b6d0 con 0x7f65b01119b0 2026-02-25T11:47:28.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.983+0000 7f65af7fe640 1 --2- 10.20.193.28:0/185692718 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65840b57f0 0x7f65840d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.983+0000 7f65af7fe640 1 --2- 10.20.193.28:0/185692718 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65840b57f0 0x7f65840d5ca0 secure :-1 s=READY pgs=206 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f659c004740 tx=0x7f659c005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.987 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:28.988 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.974+0000 7fdab83b5640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa80d6040 con 0x7fdaa80ca710 2026-02-25T11:47:28.988 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.974+0000 7fdab83b5640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa80d61b0 con 0x7fdaa80c4520 2026-02-25T11:47:28.988 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.974+0000 7fdab83b5640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa80d6370 con 0x7fdaa80c4f10 2026-02-25T11:47:28.988 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.974+0000 7fdab612a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdaa80c4520 0x7fdaa80d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.988 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.974+0000 7fdab612a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdaa80c4520 0x7fdaa80d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41912/0 (socket says 10.20.193.28:41912) 2026-02-25T11:47:28.988 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.974+0000 7fdab612a640 1 -- 10.20.193.28:0/3213037380 learned_addr learned my addr 10.20.193.28:0/3213037380 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:28.988 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.974+0000 7fdab5929640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdaa80c4f10 0x7fdaa80d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.978+0000 7fdab612a640 1 -- 10.20.193.28:0/3213037380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdaa80c4f10 msgr2=0x7fdaa80d1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.978+0000 7fdab612a640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdaa80c4f10 0x7fdaa80d1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.978+0000 7fdab612a640 1 -- 10.20.193.28:0/3213037380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdaa80ca710 msgr2=0x7fdaa80d5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:28.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.978+0000 7fdab612a640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdaa80ca710 0x7fdaa80d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:28.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.978+0000 7fdab612a640 1 -- 10.20.193.28:0/3213037380 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa0009570 con 0x7fdaa80c4520 2026-02-25T11:47:28.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.978+0000 7fdab5929640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdaa80c4f10 0x7fdaa80d1e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:28.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.978+0000 7fdab612a640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdaa80c4520 0x7fdaa80d1950 secure :-1 s=READY pgs=92 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdaa00098a0 tx=0x7fdaa00028b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:28.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.980+0000 7fda977fe640 1 -- 10.20.193.28:0/3213037380 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdaa003d070 con 0x7fdaa80c4520 2026-02-25T11:47:28.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.980+0000 7fda977fe640 1 -- 10.20.193.28:0/3213037380 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdaa002fe60 con 0x7fdaa80c4520 2026-02-25T11:47:28.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.980+0000 7fda977fe640 1 -- 10.20.193.28:0/3213037380 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdaa0041a00 con 0x7fdaa80c4520 2026-02-25T11:47:28.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.980+0000 7fdab83b5640 1 -- 10.20.193.28:0/3213037380 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa80d65f0 con 0x7fdaa80c4520 2026-02-25T11:47:28.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.980+0000 7fdab83b5640 1 -- 10.20.193.28:0/3213037380 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa81318c0 con 0x7fdaa80c4520 2026-02-25T11:47:28.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.981+0000 7fdab83b5640 1 -- 10.20.193.28:0/3213037380 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa80c5800 con 0x7fdaa80c4520 2026-02-25T11:47:28.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.983+0000 7fda977fe640 1 -- 10.20.193.28:0/3213037380 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fdaa00386b0 con 0x7fdaa80c4520 2026-02-25T11:47:28.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.984+0000 7fda977fe640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fda840b5870 0x7fda840d5d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:28.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.985+0000 7fda977fe640 1 -- 10.20.193.28:0/3213037380 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fdaa010b660 con 0x7fdaa80c4520 2026-02-25T11:47:28.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.985+0000 7fdab5929640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fda840b5870 0x7fda840d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:28.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.987+0000 7fda977fe640 1 -- 10.20.193.28:0/3213037380 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fdaa00ce0c0 con 0x7fdaa80c4520 2026-02-25T11:47:28.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:28.987+0000 7fdab5929640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fda840b5870 0x7fda840d5d20 secure :-1 s=READY pgs=207 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdaa80c7360 tx=0x7fdaa4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.012 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:29.024 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:29.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.029+0000 7fca71b5c640 1 -- 10.20.193.28:0/369915339 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fca6c114d30 con 0x7fca6c116fe0 2026-02-25T11:47:29.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.032+0000 7fca69ffb640 1 -- 10.20.193.28:0/369915339 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fca5c0ad740 con 0x7fca6c116fe0 2026-02-25T11:47:29.036 INFO:teuthology.orchestra.run.trial028.stdout:167503724569 2026-02-25T11:47:29.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.038+0000 7fca71b5c640 1 -- 10.20.193.28:0/369915339 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca340b58c0 msgr2=0x7fca340d5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.038+0000 7fca71b5c640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca340b58c0 0x7fca340d5d70 secure :-1 s=READY pgs=204 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca54005fd0 tx=0x7fca5400a480 comp rx=0 tx=0).stop 2026-02-25T11:47:29.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.038+0000 7fca71b5c640 1 -- 10.20.193.28:0/369915339 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c116fe0 msgr2=0x7fca6c122fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.038+0000 7fca71b5c640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c116fe0 0x7fca6c122fa0 secure :-1 s=READY pgs=272 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fca5c00f940 tx=0x7fca5c00fe10 comp rx=0 tx=0).stop 2026-02-25T11:47:29.040 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:29.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.038+0000 7fca71b5c640 1 -- 10.20.193.28:0/369915339 shutdown_connections 2026-02-25T11:47:29.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.038+0000 7fca71b5c640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca340b58c0 0x7fca340d5d70 unknown :-1 s=CLOSED pgs=204 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.038+0000 7fca71b5c640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fca6c116fe0 0x7fca6c122fa0 unknown :-1 s=CLOSED pgs=272 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.038+0000 7fca71b5c640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fca6c1152f0 0x7fca6c1228f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.038+0000 7fca71b5c640 1 --2- 10.20.193.28:0/369915339 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fca6c114900 0x7fca6c12a2a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.038+0000 7fca71b5c640 1 -- 10.20.193.28:0/369915339 >> 10.20.193.28:0/369915339 conn(0x7fca6c05ba30 msgr2=0x7fca6c1157f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.041+0000 7fca71b5c640 1 -- 10.20.193.28:0/369915339 shutdown_connections 2026-02-25T11:47:29.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.041+0000 7fca71b5c640 1 -- 10.20.193.28:0/369915339 wait complete. 2026-02-25T11:47:29.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.053+0000 7fcfcd91e640 1 -- 10.20.193.28:0/1793927631 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fcfc8114d00 con 0x7fcfc8114900 2026-02-25T11:47:29.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.054+0000 7fcfcc91c640 1 -- 10.20.193.28:0/1793927631 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcfb80d0ea0 con 0x7fcfc8114900 2026-02-25T11:47:29.056 INFO:teuthology.orchestra.run.trial028.stdout:257698037777 2026-02-25T11:47:29.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.058+0000 7fcfcd91e640 1 -- 10.20.193.28:0/1793927631 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcf9c0b57f0 msgr2=0x7fcf9c0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.058+0000 7fcfcd91e640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcf9c0b57f0 0x7fcf9c0d5ca0 secure :-1 s=READY pgs=205 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcfc0006c10 tx=0x7fcfc000a390 comp rx=0 tx=0).stop 2026-02-25T11:47:29.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.058+0000 7fcfcd91e640 1 -- 10.20.193.28:0/1793927631 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcfc8114900 msgr2=0x7fcfc811ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.058+0000 7fcfcd91e640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcfc8114900 0x7fcfc811ffd0 secure :-1 s=READY pgs=90 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcfb80074f0 tx=0x7fcfb8030890 comp rx=0 tx=0).stop 2026-02-25T11:47:29.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.059+0000 7fcfcd91e640 1 -- 10.20.193.28:0/1793927631 shutdown_connections 2026-02-25T11:47:29.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.059+0000 7fcfcd91e640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcf9c0b57f0 0x7fcf9c0d5ca0 unknown :-1 s=CLOSED pgs=205 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.059+0000 7fcfcd91e640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcfc8117120 0x7fcfc8125890 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.059+0000 7fcfcd91e640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcfc81152f0 0x7fcfc8120510 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.059+0000 7fcfcd91e640 1 --2- 10.20.193.28:0/1793927631 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcfc8114900 0x7fcfc811ffd0 unknown :-1 s=CLOSED pgs=90 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.059+0000 7fcfcd91e640 1 -- 10.20.193.28:0/1793927631 >> 10.20.193.28:0/1793927631 conn(0x7fcfc8057d30 msgr2=0x7fcfc8115930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.059+0000 7fcfcd91e640 1 -- 10.20.193.28:0/1793927631 shutdown_connections 2026-02-25T11:47:29.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.060+0000 7fcfcd91e640 1 -- 10.20.193.28:0/1793927631 wait complete. 2026-02-25T11:47:29.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.113+0000 7f76248c7640 1 -- 10.20.193.28:0/2861191768 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76180b7780 msgr2=0x7f76180bc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.113+0000 7f76248c7640 1 --2- 10.20.193.28:0/2861191768 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76180b7780 0x7f76180bc7d0 secure :-1 s=READY pgs=274 gs=3 cs=0 l=1 c_cookie=1385054823589835034 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f76140099a0 tx=0x7f761402f220 comp rx=0 tx=0).stop 2026-02-25T11:47:29.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.114+0000 7f76248c7640 1 -- 10.20.193.28:0/2861191768 shutdown_connections 2026-02-25T11:47:29.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.114+0000 7f76248c7640 1 --2- 10.20.193.28:0/2861191768 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f76180bcf80 0x7f761811d950 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.114+0000 7f76248c7640 1 --2- 10.20.193.28:0/2861191768 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76180b7780 0x7f76180bc7d0 unknown :-1 s=CLOSED pgs=274 gs=3 cs=0 l=1 c_cookie=1385054823589835034 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.114+0000 7f76248c7640 1 --2- 10.20.193.28:0/2861191768 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76180b6d90 0x7f76180b7160 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.114+0000 7f76248c7640 1 -- 10.20.193.28:0/2861191768 >> 10.20.193.28:0/2861191768 conn(0x7f7618094660 msgr2=0x7f76180b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.114+0000 7f76248c7640 1 -- 10.20.193.28:0/2861191768 shutdown_connections 2026-02-25T11:47:29.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.115+0000 7f76248c7640 1 -- 10.20.193.28:0/2861191768 wait complete. 2026-02-25T11:47:29.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.116+0000 7f76248c7640 1 Processor -- start 2026-02-25T11:47:29.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.118+0000 7f76248c7640 1 -- start start 2026-02-25T11:47:29.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.120+0000 7f76248c7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76180b6d90 0x7f76180b4730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.120+0000 7f762263c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76180b6d90 0x7f76180b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.120+0000 7f762263c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76180b6d90 0x7f76180b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40892/0 (socket says 10.20.193.28:40892) 2026-02-25T11:47:29.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.120+0000 7f762263c640 1 -- 10.20.193.28:0/179015499 learned_addr learned my addr 10.20.193.28:0/179015499 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:29.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.120+0000 7f76248c7640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76180b7780 0x7f761811d850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.121+0000 7f76248c7640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f76180bcf80 0x7f76180ad9e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.121+0000 7f7621e3b640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76180b7780 0x7f761811d850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.123+0000 7f76248c7640 1 -- 10.20.193.28:0/179015499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f76180adf20 con 0x7f76180b6d90 2026-02-25T11:47:29.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.123+0000 7f76248c7640 1 -- 10.20.193.28:0/179015499 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f76180ae090 con 0x7f76180bcf80 2026-02-25T11:47:29.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.123+0000 7f76248c7640 1 -- 10.20.193.28:0/179015499 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f76180ae250 con 0x7f76180b7780 2026-02-25T11:47:29.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.123+0000 7f7622e3d640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f76180bcf80 0x7f76180ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.123+0000 7f762263c640 1 -- 10.20.193.28:0/179015499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76180b7780 msgr2=0x7f761811d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.123+0000 7f762263c640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76180b7780 0x7f761811d850 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.124+0000 7f762263c640 1 -- 10.20.193.28:0/179015499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f76180bcf80 msgr2=0x7f76180ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.124+0000 7f762263c640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f76180bcf80 0x7f76180ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.124+0000 7f762263c640 1 -- 10.20.193.28:0/179015499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7614009570 con 0x7f76180b6d90 2026-02-25T11:47:29.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.124+0000 7f7621e3b640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76180b7780 0x7f761811d850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:29.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.124+0000 7f7622e3d640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f76180bcf80 0x7f76180ad9e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:29.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.124+0000 7f762263c640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76180b6d90 0x7f76180b4730 secure :-1 s=READY pgs=275 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f760800d950 tx=0x7f760800de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.124+0000 7f76077fe640 1 -- 10.20.193.28:0/179015499 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7608004540 con 0x7f76180b6d90 2026-02-25T11:47:29.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.125+0000 7f76248c7640 1 -- 10.20.193.28:0/179015499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76180ae530 con 0x7f76180b6d90 2026-02-25T11:47:29.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.125+0000 7f76248c7640 1 -- 10.20.193.28:0/179015499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76180ae860 con 0x7f76180b6d90 2026-02-25T11:47:29.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.125+0000 7f76077fe640 1 -- 10.20.193.28:0/179015499 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7608009ce0 con 0x7f76180b6d90 2026-02-25T11:47:29.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.125+0000 7f76077fe640 1 -- 10.20.193.28:0/179015499 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f76080106e0 con 0x7f76180b6d90 2026-02-25T11:47:29.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.128+0000 7f76248c7640 1 -- 10.20.193.28:0/179015499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76180b8070 con 0x7f76180b6d90 2026-02-25T11:47:29.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.134+0000 7f76077fe640 1 -- 10.20.193.28:0/179015499 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f76080097c0 con 0x7f76180b6d90 2026-02-25T11:47:29.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.137+0000 7f76077fe640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f75f40b5940 0x7f75f40d5df0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.138+0000 7f7621e3b640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f75f40b5940 0x7f75f40d5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.138+0000 7f76077fe640 1 -- 10.20.193.28:0/179015499 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f76080e6de0 con 0x7f76180b6d90 2026-02-25T11:47:29.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.140+0000 7f76077fe640 1 -- 10.20.193.28:0/179015499 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f76080eb370 con 0x7f76180b6d90 2026-02-25T11:47:29.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.144+0000 7f7621e3b640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f75f40b5940 0x7f75f40d5df0 secure :-1 s=READY pgs=208 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f76140023d0 tx=0x7f7614031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.165+0000 7fdab83b5640 1 -- 10.20.193.28:0/3213037380 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fdaa80c48f0 con 0x7fdaa80c4520 2026-02-25T11:47:29.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.166+0000 7fda977fe640 1 -- 10.20.193.28:0/3213037380 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdaa00d1270 con 0x7fdaa80c4520 2026-02-25T11:47:29.168 INFO:teuthology.orchestra.run.trial028.stdout:146028888093 2026-02-25T11:47:29.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.166+0000 7f65b5e4b640 1 -- 10.20.193.28:0/185692718 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f657c005880 con 0x7f65b01119b0 2026-02-25T11:47:29.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.167+0000 7f65acff9640 1 -- 10.20.193.28:0/185692718 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f65a00cdcb0 con 0x7f65b01119b0 2026-02-25T11:47:29.171 INFO:teuthology.orchestra.run.trial028.stdout:347892350987 2026-02-25T11:47:29.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7f65b5e4b640 1 -- 10.20.193.28:0/185692718 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65840b57f0 msgr2=0x7f65840d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7f65b5e4b640 1 --2- 10.20.193.28:0/185692718 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65840b57f0 0x7f65840d5ca0 secure :-1 s=READY pgs=206 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f659c004740 tx=0x7f659c005ab0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7f65b5e4b640 1 -- 10.20.193.28:0/185692718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f65b01119b0 msgr2=0x7f65b011a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7f65b5e4b640 1 --2- 10.20.193.28:0/185692718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f65b01119b0 0x7f65b011a4e0 secure :-1 s=READY pgs=89 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f65a0002d50 tx=0x7f65a0002d80 comp rx=0 tx=0).stop 2026-02-25T11:47:29.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.171+0000 7f65b5e4b640 1 -- 10.20.193.28:0/185692718 shutdown_connections 2026-02-25T11:47:29.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.171+0000 7f65b5e4b640 1 --2- 10.20.193.28:0/185692718 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65840b57f0 0x7f65840d5ca0 unknown :-1 s=CLOSED pgs=206 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.171+0000 7f65b5e4b640 1 --2- 10.20.193.28:0/185692718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f65b01119b0 0x7f65b011a4e0 unknown :-1 s=CLOSED pgs=89 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.171+0000 7f65b5e4b640 1 --2- 10.20.193.28:0/185692718 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f65b010c1b0 0x7f65b0115110 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.171+0000 7f65b5e4b640 1 --2- 10.20.193.28:0/185692718 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f65b010b7c0 0x7f65b0114bd0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.171+0000 7f65b5e4b640 1 -- 10.20.193.28:0/185692718 >> 10.20.193.28:0/185692718 conn(0x7f65b00e93e0 msgr2=0x7f65b010dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.171+0000 7f65b5e4b640 1 -- 10.20.193.28:0/185692718 shutdown_connections 2026-02-25T11:47:29.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.171+0000 7f65b5e4b640 1 -- 10.20.193.28:0/185692718 wait complete. 2026-02-25T11:47:29.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7fdab83b5640 1 -- 10.20.193.28:0/3213037380 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fda840b5870 msgr2=0x7fda840d5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7fdab83b5640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fda840b5870 0x7fda840d5d20 secure :-1 s=READY pgs=207 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdaa80c7360 tx=0x7fdaa4009190 comp rx=0 tx=0).stop 2026-02-25T11:47:29.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7fdab83b5640 1 -- 10.20.193.28:0/3213037380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdaa80c4520 msgr2=0x7fdaa80d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7fdab83b5640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdaa80c4520 0x7fdaa80d1950 secure :-1 s=READY pgs=92 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdaa00098a0 tx=0x7fdaa00028b0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7fdab83b5640 1 -- 10.20.193.28:0/3213037380 shutdown_connections 2026-02-25T11:47:29.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7fdab83b5640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fda840b5870 0x7fda840d5d20 unknown :-1 s=CLOSED pgs=207 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7fdab83b5640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdaa80ca710 0x7fdaa80d5a20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7fdab83b5640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdaa80c4f10 0x7fdaa80d1e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7fdab83b5640 1 --2- 10.20.193.28:0/3213037380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdaa80c4520 0x7fdaa80d1950 unknown :-1 s=CLOSED pgs=92 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.170+0000 7fdab83b5640 1 -- 10.20.193.28:0/3213037380 >> 10.20.193.28:0/3213037380 conn(0x7fdaa801c630 msgr2=0x7fdaa80c8560 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.171+0000 7fdab83b5640 1 -- 10.20.193.28:0/3213037380 shutdown_connections 2026-02-25T11:47:29.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.171+0000 7fdab83b5640 1 -- 10.20.193.28:0/3213037380 wait complete. 2026-02-25T11:47:29.214 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:29 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/369915339' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2026-02-25T11:47:29.214 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:29 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1793927631' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-25T11:47:29.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.220+0000 7f80a25db640 1 -- 10.20.193.28:0/2447686387 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f809c10ddc0 msgr2=0x7f809c112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.220+0000 7f80a25db640 1 --2- 10.20.193.28:0/2447686387 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f809c10ddc0 0x7f809c112e10 secure :-1 s=READY pgs=276 gs=1 cs=0 l=1 c_cookie=5360420671478587045 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8090009950 tx=0x7f809002f190 comp rx=0 tx=0).stop 2026-02-25T11:47:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.220+0000 7f80a25db640 1 -- 10.20.193.28:0/2447686387 shutdown_connections 2026-02-25T11:47:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.220+0000 7f80a25db640 1 --2- 10.20.193.28:0/2447686387 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f809c1135c0 0x7f809c173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.220+0000 7f80a25db640 1 --2- 10.20.193.28:0/2447686387 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f809c10ddc0 0x7f809c112e10 unknown :-1 s=CLOSED pgs=276 gs=1 cs=0 l=1 c_cookie=5360420671478587045 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.220+0000 7f80a25db640 1 --2- 10.20.193.28:0/2447686387 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f809c10d3d0 0x7f809c10d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.220+0000 7f80a25db640 1 -- 10.20.193.28:0/2447686387 >> 10.20.193.28:0/2447686387 conn(0x7f809c0eaca0 msgr2=0x7f809c10b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.221+0000 7f80a25db640 1 -- 10.20.193.28:0/2447686387 shutdown_connections 2026-02-25T11:47:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.221+0000 7f80a25db640 1 -- 10.20.193.28:0/2447686387 wait complete. 2026-02-25T11:47:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.223+0000 7f80a25db640 1 Processor -- start 2026-02-25T11:47:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.223+0000 7f80a25db640 1 -- start start 2026-02-25T11:47:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.224+0000 7f80a25db640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f809c10d3d0 0x7f809c0f0160 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.224+0000 7f80a25db640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f809c10ddc0 0x7f809c173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.224+0000 7f80a25db640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f809c1135c0 0x7f809c0f0860 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.224+0000 7f80a15d9640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f809c10d3d0 0x7f809c0f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.224+0000 7f80a15d9640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f809c10d3d0 0x7f809c0f0160 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52412/0 (socket says 10.20.193.28:52412) 2026-02-25T11:47:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.224+0000 7f80a25db640 1 -- 10.20.193.28:0/78789496 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f809c0f5a70 con 0x7f809c10ddc0 2026-02-25T11:47:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.224+0000 7f80a15d9640 1 -- 10.20.193.28:0/78789496 learned_addr learned my addr 10.20.193.28:0/78789496 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.224+0000 7f80a1dda640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f809c1135c0 0x7f809c0f0860 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.225+0000 7f80a25db640 1 -- 10.20.193.28:0/78789496 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f809c0f5be0 con 0x7f809c1135c0 2026-02-25T11:47:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.225+0000 7f80a0dd8640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f809c10ddc0 0x7f809c173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.225+0000 7f80a25db640 1 -- 10.20.193.28:0/78789496 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f809c0f5da0 con 0x7f809c10d3d0 2026-02-25T11:47:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.226+0000 7f80a15d9640 1 -- 10.20.193.28:0/78789496 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f809c1135c0 msgr2=0x7f809c0f0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.226+0000 7f80a15d9640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f809c1135c0 0x7f809c0f0860 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.226+0000 7f80a15d9640 1 -- 10.20.193.28:0/78789496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f809c10ddc0 msgr2=0x7f809c173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.226+0000 7f80a15d9640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f809c10ddc0 0x7f809c173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.226+0000 7f80a15d9640 1 -- 10.20.193.28:0/78789496 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8090009570 con 0x7f809c10d3d0 2026-02-25T11:47:29.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.226+0000 7f80a0dd8640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f809c10ddc0 0x7f809c173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:29.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.226+0000 7f80a15d9640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f809c10d3d0 0x7f809c0f0160 secure :-1 s=READY pgs=90 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f808c00c8f0 tx=0x7f808c00cdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.227+0000 7f808a7fc640 1 -- 10.20.193.28:0/78789496 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f808c0044a0 con 0x7f809c10d3d0 2026-02-25T11:47:29.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.227+0000 7f808a7fc640 1 -- 10.20.193.28:0/78789496 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f808c007d20 con 0x7f809c10d3d0 2026-02-25T11:47:29.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.227+0000 7f808a7fc640 1 -- 10.20.193.28:0/78789496 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f808c00f450 con 0x7f809c10d3d0 2026-02-25T11:47:29.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.227+0000 7f80a25db640 1 -- 10.20.193.28:0/78789496 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809c0f6080 con 0x7f809c10d3d0 2026-02-25T11:47:29.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.227+0000 7f80a25db640 1 -- 10.20.193.28:0/78789496 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f809c0f6500 con 0x7f809c10d3d0 2026-02-25T11:47:29.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.228+0000 7f80a25db640 1 -- 10.20.193.28:0/78789496 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8064005300 con 0x7f809c10d3d0 2026-02-25T11:47:29.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.231+0000 7f808a7fc640 1 -- 10.20.193.28:0/78789496 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f808c0026a0 con 0x7f809c10d3d0 2026-02-25T11:47:29.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.232+0000 7f808a7fc640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80740b58c0 0x7f80740d5d70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.234 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.232+0000 7f808a7fc640 1 -- 10.20.193.28:0/78789496 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f808c0e66f0 con 0x7f809c10d3d0 2026-02-25T11:47:29.234 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.233+0000 7f80a0dd8640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80740b58c0 0x7f80740d5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.234+0000 7f808a7fc640 1 -- 10.20.193.28:0/78789496 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f808c0a9040 con 0x7f809c10d3d0 2026-02-25T11:47:29.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.234+0000 7f80a0dd8640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80740b58c0 0x7f80740d5d70 secure :-1 s=READY pgs=209 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f809c0f6b00 tx=0x7f8090002390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.236+0000 7f69f99ba640 1 -- 10.20.193.28:0/2421734452 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69f4117120 msgr2=0x7f69f417ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.236+0000 7f69f99ba640 1 --2- 10.20.193.28:0/2421734452 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69f4117120 0x7f69f417ace0 secure :-1 s=READY pgs=91 gs=2 cs=0 l=1 c_cookie=4482522709578454956 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69e400a9a0 tx=0x7f69e40301d0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.237+0000 7f69f99ba640 1 -- 10.20.193.28:0/2421734452 shutdown_connections 2026-02-25T11:47:29.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.237+0000 7f69f99ba640 1 --2- 10.20.193.28:0/2421734452 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69f4117120 0x7f69f417ace0 unknown :-1 s=CLOSED pgs=91 gs=2 cs=0 l=1 c_cookie=4482522709578454956 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.237+0000 7f69f99ba640 1 --2- 10.20.193.28:0/2421734452 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69f41152f0 0x7f69f4116940 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.237+0000 7f69f99ba640 1 --2- 10.20.193.28:0/2421734452 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69f4114900 0x7f69f4114cd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.237+0000 7f69f99ba640 1 -- 10.20.193.28:0/2421734452 >> 10.20.193.28:0/2421734452 conn(0x7f69f4057d30 msgr2=0x7f69f4058130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.237+0000 7f69f99ba640 1 -- 10.20.193.28:0/2421734452 shutdown_connections 2026-02-25T11:47:29.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.237+0000 7f69f99ba640 1 -- 10.20.193.28:0/2421734452 wait complete. 2026-02-25T11:47:29.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.239+0000 7f69f99ba640 1 Processor -- start 2026-02-25T11:47:29.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.239+0000 7f69f99ba640 1 -- start start 2026-02-25T11:47:29.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.240+0000 7f69f99ba640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69f4114900 0x7f69f4124250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.240+0000 7f69f99ba640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69f41152f0 0x7f69f41227d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.240+0000 7f69f99ba640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69f4117120 0x7f69f4122d10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.240+0000 7f69f99ba640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f69f41233b0 con 0x7f69f4114900 2026-02-25T11:47:29.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.240+0000 7f69f99ba640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69f4123520 con 0x7f69f4117120 2026-02-25T11:47:29.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.240+0000 7f69f99ba640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69f4128fb0 con 0x7f69f41152f0 2026-02-25T11:47:29.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.240+0000 7f69f27fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69f41152f0 0x7f69f41227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.240+0000 7f69f27fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69f41152f0 0x7f69f41227d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52432/0 (socket says 10.20.193.28:52432) 2026-02-25T11:47:29.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.240+0000 7f69f27fc640 1 -- 10.20.193.28:0/3750376105 learned_addr learned my addr 10.20.193.28:0/3750376105 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.241+0000 7f69f27fc640 1 -- 10.20.193.28:0/3750376105 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69f4117120 msgr2=0x7f69f4122d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:47:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.241+0000 7f69f27fc640 1 --2- 10.20.193.28:0/3750376105 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69f4117120 0x7f69f4122d10 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.241+0000 7f69f27fc640 1 -- 10.20.193.28:0/3750376105 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69f4114900 msgr2=0x7f69f4124250 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:47:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.241+0000 7f69f27fc640 1 --2- 10.20.193.28:0/3750376105 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69f4114900 0x7f69f4124250 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.241+0000 7f69f27fc640 1 -- 10.20.193.28:0/3750376105 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e400a590 con 0x7f69f41152f0 2026-02-25T11:47:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.242+0000 7f69f27fc640 1 --2- 10.20.193.28:0/3750376105 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69f41152f0 0x7f69f41227d0 secure :-1 s=READY pgs=92 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69e8004740 tx=0x7f69e800da80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.242+0000 7f69f89b8640 1 -- 10.20.193.28:0/3750376105 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f69e80099b0 con 0x7f69f41152f0 2026-02-25T11:47:29.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.242+0000 7f69f89b8640 1 -- 10.20.193.28:0/3750376105 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f69e8010460 con 0x7f69f41152f0 2026-02-25T11:47:29.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.243+0000 7f69f89b8640 1 -- 10.20.193.28:0/3750376105 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f69e8018660 con 0x7f69f41152f0 2026-02-25T11:47:29.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.243+0000 7f69f99ba640 1 -- 10.20.193.28:0/3750376105 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f4129290 con 0x7f69f41152f0 2026-02-25T11:47:29.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.244+0000 7f69f99ba640 1 -- 10.20.193.28:0/3750376105 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f4129760 con 0x7f69f41152f0 2026-02-25T11:47:29.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.245+0000 7f69f99ba640 1 -- 10.20.193.28:0/3750376105 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69f405aba0 con 0x7f69f41152f0 2026-02-25T11:47:29.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.248+0000 7f69f89b8640 1 -- 10.20.193.28:0/3750376105 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f69e8016020 con 0x7f69f41152f0 2026-02-25T11:47:29.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.250+0000 7f69f89b8640 1 --2- 10.20.193.28:0/3750376105 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69c00b58c0 0x7f69c00d5d70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.251+0000 7f69f89b8640 1 -- 10.20.193.28:0/3750376105 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f69e80e7030 con 0x7f69f41152f0 2026-02-25T11:47:29.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.251+0000 7f69f89b8640 1 -- 10.20.193.28:0/3750376105 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f69e80e7520 con 0x7f69f41152f0 2026-02-25T11:47:29.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.253+0000 7f69f2ffd640 1 --2- 10.20.193.28:0/3750376105 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69c00b58c0 0x7f69c00d5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.257+0000 7f69f2ffd640 1 --2- 10.20.193.28:0/3750376105 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69c00b58c0 0x7f69c00d5d70 secure :-1 s=READY pgs=210 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69dc004740 tx=0x7f69dc00c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.273 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037778 got 257698037777 for osd.8 2026-02-25T11:47:29.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.284+0000 7f76248c7640 1 -- 10.20.193.28:0/179015499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f76180aec90 con 0x7f76180b6d90 2026-02-25T11:47:29.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.286+0000 7f76077fe640 1 -- 10.20.193.28:0/179015499 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f76080a9730 con 0x7f76180b6d90 2026-02-25T11:47:29.288 INFO:teuthology.orchestra.run.trial028.stdout:313532612621 2026-02-25T11:47:29.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.290+0000 7f76248c7640 1 -- 10.20.193.28:0/179015499 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f75f40b5940 msgr2=0x7f75f40d5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.290+0000 7f76248c7640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f75f40b5940 0x7f75f40d5df0 secure :-1 s=READY pgs=208 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f76140023d0 tx=0x7f7614031040 comp rx=0 tx=0).stop 2026-02-25T11:47:29.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.290+0000 7f76248c7640 1 -- 10.20.193.28:0/179015499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76180b6d90 msgr2=0x7f76180b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.290+0000 7f76248c7640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76180b6d90 0x7f76180b4730 secure :-1 s=READY pgs=275 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f760800d950 tx=0x7f760800de20 comp rx=0 tx=0).stop 2026-02-25T11:47:29.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.291+0000 7f76248c7640 1 -- 10.20.193.28:0/179015499 shutdown_connections 2026-02-25T11:47:29.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.291+0000 7f76248c7640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f75f40b5940 0x7f75f40d5df0 unknown :-1 s=CLOSED pgs=208 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.291+0000 7f76248c7640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f76180bcf80 0x7f76180ad9e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.291+0000 7f76248c7640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f76180b7780 0x7f761811d850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.291+0000 7f76248c7640 1 --2- 10.20.193.28:0/179015499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f76180b6d90 0x7f76180b4730 unknown :-1 s=CLOSED pgs=275 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.291+0000 7f76248c7640 1 -- 10.20.193.28:0/179015499 >> 10.20.193.28:0/179015499 conn(0x7f7618094660 msgr2=0x7f76180b5500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.291+0000 7f76248c7640 1 -- 10.20.193.28:0/179015499 shutdown_connections 2026-02-25T11:47:29.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.291+0000 7f76248c7640 1 -- 10.20.193.28:0/179015499 wait complete. 2026-02-25T11:47:29.295 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724570 got 167503724569 for osd.5 2026-02-25T11:47:29.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.361+0000 7f80a25db640 1 -- 10.20.193.28:0/78789496 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f80640055b0 con 0x7f809c10d3d0 2026-02-25T11:47:29.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.362+0000 7f808a7fc640 1 -- 10.20.193.28:0/78789496 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f808c0ac1f0 con 0x7f809c10d3d0 2026-02-25T11:47:29.366 INFO:teuthology.orchestra.run.trial028.stdout:201863462937 2026-02-25T11:47:29.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.365+0000 7f80a25db640 1 -- 10.20.193.28:0/78789496 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80740b58c0 msgr2=0x7f80740d5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.365+0000 7f80a25db640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80740b58c0 0x7f80740d5d70 secure :-1 s=READY pgs=209 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f809c0f6b00 tx=0x7f8090002390 comp rx=0 tx=0).stop 2026-02-25T11:47:29.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.365+0000 7f80a25db640 1 -- 10.20.193.28:0/78789496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f809c10d3d0 msgr2=0x7f809c0f0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.365+0000 7f80a25db640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f809c10d3d0 0x7f809c0f0160 secure :-1 s=READY pgs=90 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f808c00c8f0 tx=0x7f808c00cdc0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.371+0000 7f80a25db640 1 -- 10.20.193.28:0/78789496 shutdown_connections 2026-02-25T11:47:29.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.371+0000 7f80a25db640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80740b58c0 0x7f80740d5d70 unknown :-1 s=CLOSED pgs=209 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.371+0000 7f80a25db640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f809c1135c0 0x7f809c0f0860 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.371+0000 7f80a25db640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f809c10ddc0 0x7f809c173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.371+0000 7f80a25db640 1 --2- 10.20.193.28:0/78789496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f809c10d3d0 0x7f809c0f0160 unknown :-1 s=CLOSED pgs=90 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.371+0000 7f80a25db640 1 -- 10.20.193.28:0/78789496 >> 10.20.193.28:0/78789496 conn(0x7f809c0eaca0 msgr2=0x7f809c1103c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.373+0000 7f80a25db640 1 -- 10.20.193.28:0/78789496 shutdown_connections 2026-02-25T11:47:29.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.373+0000 7f80a25db640 1 -- 10.20.193.28:0/78789496 wait complete. 2026-02-25T11:47:29.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.382+0000 7fd01720d640 1 -- 10.20.193.28:0/124452682 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd010111240 msgr2=0x7fd010111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.382+0000 7fd01720d640 1 --2- 10.20.193.28:0/124452682 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd010111240 0x7fd010111610 secure :-1 s=READY pgs=277 gs=1 cs=0 l=1 c_cookie=16816476350381537682 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcffc009950 tx=0x7fcffc02f1d0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.382+0000 7fd01720d640 1 -- 10.20.193.28:0/124452682 shutdown_connections 2026-02-25T11:47:29.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.382+0000 7fd01720d640 1 --2- 10.20.193.28:0/124452682 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd010115a10 0x7fd0100631f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.382+0000 7fd01720d640 1 --2- 10.20.193.28:0/124452682 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd010111c30 0x7fd010115280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.382+0000 7fd01720d640 1 --2- 10.20.193.28:0/124452682 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd010111240 0x7fd010111610 unknown :-1 s=CLOSED pgs=277 gs=1 cs=0 l=1 c_cookie=16816476350381537682 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.382+0000 7fd01720d640 1 -- 10.20.193.28:0/124452682 >> 10.20.193.28:0/124452682 conn(0x7fd0100ed130 msgr2=0x7fd01010d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.383+0000 7fd01720d640 1 -- 10.20.193.28:0/124452682 shutdown_connections 2026-02-25T11:47:29.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.383+0000 7fd01720d640 1 -- 10.20.193.28:0/124452682 wait complete. 2026-02-25T11:47:29.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.385+0000 7fd01720d640 1 Processor -- start 2026-02-25T11:47:29.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.385+0000 7fd01720d640 1 -- start start 2026-02-25T11:47:29.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd01720d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd010111240 0x7fd0100f4ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd01720d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd010111c30 0x7fd0100f51e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd01720d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd010115a10 0x7fd0100fa5b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd01720d640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd0100fab40 con 0x7fd010111c30 2026-02-25T11:47:29.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd01720d640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd0100facb0 con 0x7fd010115a10 2026-02-25T11:47:29.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd01720d640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd0100fae70 con 0x7fd010111240 2026-02-25T11:47:29.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd00ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd010111c30 0x7fd0100f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd00ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd010111c30 0x7fd0100f51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40966/0 (socket says 10.20.193.28:40966) 2026-02-25T11:47:29.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd00ffff640 1 -- 10.20.193.28:0/1221891285 learned_addr learned my addr 10.20.193.28:0/1221891285 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:29.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd015783640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd010115a10 0x7fd0100fa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd014f82640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd010111240 0x7fd0100f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd00ffff640 1 -- 10.20.193.28:0/1221891285 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd010111240 msgr2=0x7fd0100f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd00ffff640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd010111240 0x7fd0100f4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.386+0000 7fd00ffff640 1 -- 10.20.193.28:0/1221891285 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd010115a10 msgr2=0x7fd0100fa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.387+0000 7fd00ffff640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd010115a10 0x7fd0100fa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.387+0000 7fd00ffff640 1 -- 10.20.193.28:0/1221891285 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc009570 con 0x7fd010111c30 2026-02-25T11:47:29.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.387+0000 7fd00ffff640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd010111c30 0x7fd0100f51e0 secure :-1 s=READY pgs=279 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd00000c940 tx=0x7fd00000ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.387+0000 7fd015783640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd010115a10 0x7fd0100fa5b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:29.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.388+0000 7fd00dffb640 1 -- 10.20.193.28:0/1221891285 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd000004560 con 0x7fd010111c30 2026-02-25T11:47:29.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.388+0000 7fd00dffb640 1 -- 10.20.193.28:0/1221891285 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd000007d20 con 0x7fd010111c30 2026-02-25T11:47:29.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.388+0000 7f71a2a4c640 1 -- 10.20.193.28:0/3939066115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f719c1152f0 msgr2=0x7f719c116940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.388+0000 7f71a2a4c640 1 --2- 10.20.193.28:0/3939066115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f719c1152f0 0x7f719c116940 secure :-1 s=READY pgs=278 gs=1 cs=0 l=1 c_cookie=13611983396262683846 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f719000d9c0 tx=0x7f71900311d0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.388+0000 7fd00dffb640 1 -- 10.20.193.28:0/1221891285 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd00000f6d0 con 0x7fd010111c30 2026-02-25T11:47:29.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.388+0000 7fd01720d640 1 -- 10.20.193.28:0/1221891285 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0100fb150 con 0x7fd010111c30 2026-02-25T11:47:29.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.391+0000 7fd01720d640 1 -- 10.20.193.28:0/1221891285 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0100fb430 con 0x7fd010111c30 2026-02-25T11:47:29.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.394+0000 7f71a2a4c640 1 -- 10.20.193.28:0/3939066115 shutdown_connections 2026-02-25T11:47:29.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.394+0000 7f71a2a4c640 1 --2- 10.20.193.28:0/3939066115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f719c117120 0x7f719c17ace0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.394+0000 7f71a2a4c640 1 --2- 10.20.193.28:0/3939066115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f719c1152f0 0x7f719c116940 unknown :-1 s=CLOSED pgs=278 gs=1 cs=0 l=1 c_cookie=13611983396262683846 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.394+0000 7f71a2a4c640 1 --2- 10.20.193.28:0/3939066115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f719c114900 0x7f719c114cd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.394+0000 7f71a2a4c640 1 -- 10.20.193.28:0/3939066115 >> 10.20.193.28:0/3939066115 conn(0x7f719c057d30 msgr2=0x7f719c058130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.394+0000 7fcfe37fe640 1 -- 10.20.193.28:0/1221891285 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd010110b20 con 0x7fd010111c30 2026-02-25T11:47:29.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.395+0000 7fd00dffb640 1 -- 10.20.193.28:0/1221891285 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fd000007860 con 0x7fd010111c30 2026-02-25T11:47:29.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.395+0000 7f71a2a4c640 1 -- 10.20.193.28:0/3939066115 shutdown_connections 2026-02-25T11:47:29.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.397+0000 7f71a2a4c640 1 -- 10.20.193.28:0/3939066115 wait complete. 2026-02-25T11:47:29.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.397+0000 7f69f99ba640 1 -- 10.20.193.28:0/3750376105 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f69f4114d30 con 0x7f69f41152f0 2026-02-25T11:47:29.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.398+0000 7f69f89b8640 1 -- 10.20.193.28:0/3750376105 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f69e80a99c0 con 0x7f69f41152f0 2026-02-25T11:47:29.401 INFO:teuthology.orchestra.run.trial028.stdout:38654705704 2026-02-25T11:47:29.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.400+0000 7f71a2a4c640 1 Processor -- start 2026-02-25T11:47:29.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.400+0000 7f71a2a4c640 1 -- start start 2026-02-25T11:47:29.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.400+0000 7f71a2a4c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f719c114900 0x7f719c11fff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.401+0000 7f71a2a4c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f719c1152f0 0x7f719c120530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.401+0000 7f71a2a4c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f719c117120 0x7f719c1258b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.401+0000 7f71a2a4c640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f719c125e40 con 0x7f719c1152f0 2026-02-25T11:47:29.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.401+0000 7f71a2a4c640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f719c125fb0 con 0x7f719c117120 2026-02-25T11:47:29.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.401+0000 7f71a2a4c640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f719c126170 con 0x7f719c114900 2026-02-25T11:47:29.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.401+0000 7f69f99ba640 1 -- 10.20.193.28:0/3750376105 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69c00b58c0 msgr2=0x7f69c00d5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.402+0000 7f719bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f719c114900 0x7f719c11fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.402+0000 7f719bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f719c114900 0x7f719c11fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52470/0 (socket says 10.20.193.28:52470) 2026-02-25T11:47:29.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.402+0000 7f719bfff640 1 -- 10.20.193.28:0/1991297655 learned_addr learned my addr 10.20.193.28:0/1991297655 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:29.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.401+0000 7f69f99ba640 1 --2- 10.20.193.28:0/3750376105 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69c00b58c0 0x7f69c00d5d70 secure :-1 s=READY pgs=210 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69dc004740 tx=0x7f69dc00c040 comp rx=0 tx=0).stop 2026-02-25T11:47:29.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.401+0000 7f69f99ba640 1 -- 10.20.193.28:0/3750376105 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69f41152f0 msgr2=0x7f69f41227d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.401+0000 7f69f99ba640 1 --2- 10.20.193.28:0/3750376105 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69f41152f0 0x7f69f41227d0 secure :-1 s=READY pgs=92 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69e8004740 tx=0x7f69e800da80 comp rx=0 tx=0).stop 2026-02-25T11:47:29.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.402+0000 7f69f99ba640 1 -- 10.20.193.28:0/3750376105 shutdown_connections 2026-02-25T11:47:29.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.402+0000 7f69f99ba640 1 --2- 10.20.193.28:0/3750376105 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69c00b58c0 0x7f69c00d5d70 unknown :-1 s=CLOSED pgs=210 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.402+0000 7f69f99ba640 1 --2- 10.20.193.28:0/3750376105 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69f4117120 0x7f69f4122d10 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.402+0000 7f69f99ba640 1 --2- 10.20.193.28:0/3750376105 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69f41152f0 0x7f69f41227d0 unknown :-1 s=CLOSED pgs=92 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.402+0000 7f69f99ba640 1 --2- 10.20.193.28:0/3750376105 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69f4114900 0x7f69f4124250 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.402+0000 7f69f99ba640 1 -- 10.20.193.28:0/3750376105 >> 10.20.193.28:0/3750376105 conn(0x7f69f4057d30 msgr2=0x7f69f4115930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.402+0000 7f69f99ba640 1 -- 10.20.193.28:0/3750376105 shutdown_connections 2026-02-25T11:47:29.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.403+0000 7f69f99ba640 1 -- 10.20.193.28:0/3750376105 wait complete. 2026-02-25T11:47:29.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.403+0000 7f71a0fc2640 1 --2- 10.20.193.28:0/1991297655 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f719c117120 0x7f719c1258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.403+0000 7f719bfff640 1 -- 10.20.193.28:0/1991297655 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f719c117120 msgr2=0x7f719c1258b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.403+0000 7f719bfff640 1 --2- 10.20.193.28:0/1991297655 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f719c117120 0x7f719c1258b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.403+0000 7f719bfff640 1 -- 10.20.193.28:0/1991297655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f719c1152f0 msgr2=0x7f719c120530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2026-02-25T11:47:29.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.403+0000 7fd00dffb640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcfe40b58c0 0x7fcfe40d5d70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.403+0000 7f719bfff640 1 --2- 10.20.193.28:0/1991297655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f719c1152f0 0x7f719c120530 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.404+0000 7fd00dffb640 1 -- 10.20.193.28:0/1221891285 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fd0000ee3e0 con 0x7fd010111c30 2026-02-25T11:47:29.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.404+0000 7fd014f82640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcfe40b58c0 0x7fcfe40d5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.403+0000 7f719bfff640 1 -- 10.20.193.28:0/1991297655 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71940091d0 con 0x7f719c114900 2026-02-25T11:47:29.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.404+0000 7fd00dffb640 1 -- 10.20.193.28:0/1221891285 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fd0000ee690 con 0x7fd010111c30 2026-02-25T11:47:29.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.404+0000 7f719bfff640 1 --2- 10.20.193.28:0/1991297655 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f719c114900 0x7f719c11fff0 secure :-1 s=READY pgs=93 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f718c00c9b0 tx=0x7f718c00ce80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.404+0000 7f71997fa640 1 -- 10.20.193.28:0/1991297655 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f718c004560 con 0x7f719c114900 2026-02-25T11:47:29.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.404+0000 7fd014f82640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcfe40b58c0 0x7fcfe40d5d70 secure :-1 s=READY pgs=211 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcffc0098a0 tx=0x7fcffc03c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.404+0000 7f71997fa640 1 -- 10.20.193.28:0/1991297655 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f718c007d80 con 0x7f719c114900 2026-02-25T11:47:29.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.404+0000 7f71997fa640 1 -- 10.20.193.28:0/1991297655 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f718c00f6d0 con 0x7f719c114900 2026-02-25T11:47:29.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.404+0000 7f71a2a4c640 1 -- 10.20.193.28:0/1991297655 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719000d5b0 con 0x7f719c114900 2026-02-25T11:47:29.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.404+0000 7f71a2a4c640 1 -- 10.20.193.28:0/1991297655 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719c166a40 con 0x7f719c114900 2026-02-25T11:47:29.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.412+0000 7f7172ffd640 1 -- 10.20.193.28:0/1991297655 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719c05aba0 con 0x7f719c114900 2026-02-25T11:47:29.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.412+0000 7f71997fa640 1 -- 10.20.193.28:0/1991297655 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f718c0028a0 con 0x7f719c114900 2026-02-25T11:47:29.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.412+0000 7f71997fa640 1 --2- 10.20.193.28:0/1991297655 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f71780b57f0 0x7f71780d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.412+0000 7f71997fa640 1 -- 10.20.193.28:0/1991297655 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f718c0e6be0 con 0x7f719c114900 2026-02-25T11:47:29.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.412+0000 7f719b7fe640 1 --2- 10.20.193.28:0/1991297655 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f71780b57f0 0x7f71780d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.412+0000 7f719b7fe640 1 --2- 10.20.193.28:0/1991297655 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f71780b57f0 0x7f71780d5ca0 secure :-1 s=READY pgs=212 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f719000d8e0 tx=0x7f7190002390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.417+0000 7f71997fa640 1 -- 10.20.193.28:0/1991297655 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f718c0a9530 con 0x7f719c114900 2026-02-25T11:47:29.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.433+0000 7f143509a640 1 -- 10.20.193.28:0/545162101 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1430110230 msgr2=0x7f1430115280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.433+0000 7f143509a640 1 --2- 10.20.193.28:0/545162101 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1430110230 0x7f1430115280 secure :-1 s=READY pgs=94 gs=1 cs=0 l=1 c_cookie=533106954367787643 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1418009950 tx=0x7f141802f190 comp rx=0 tx=0).stop 2026-02-25T11:47:29.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.433+0000 7f143509a640 1 -- 10.20.193.28:0/545162101 shutdown_connections 2026-02-25T11:47:29.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.433+0000 7f143509a640 1 --2- 10.20.193.28:0/545162101 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1430115a30 0x7f1430176400 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.433+0000 7f143509a640 1 --2- 10.20.193.28:0/545162101 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1430110230 0x7f1430115280 unknown :-1 s=CLOSED pgs=94 gs=1 cs=0 l=1 c_cookie=533106954367787643 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.433+0000 7f143509a640 1 --2- 10.20.193.28:0/545162101 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f143010f840 0x7f143010fc10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.433+0000 7f143509a640 1 -- 10.20.193.28:0/545162101 >> 10.20.193.28:0/545162101 conn(0x7f14300ed130 msgr2=0x7f143010d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.433+0000 7f143509a640 1 -- 10.20.193.28:0/545162101 shutdown_connections 2026-02-25T11:47:29.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.433+0000 7f143509a640 1 -- 10.20.193.28:0/545162101 wait complete. 2026-02-25T11:47:29.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.433+0000 7f143509a640 1 Processor -- start 2026-02-25T11:47:29.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.433+0000 7f143509a640 1 -- start start 2026-02-25T11:47:29.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f143509a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f143010f840 0x7f143005d8d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f143509a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1430110230 0x7f1430176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f143509a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1430115a30 0x7f143005e000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f143509a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1430100d30 con 0x7f1430110230 2026-02-25T11:47:29.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f143509a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1430100ea0 con 0x7f143010f840 2026-02-25T11:47:29.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f143509a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1430101060 con 0x7f1430115a30 2026-02-25T11:47:29.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f142f577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1430115a30 0x7f143005e000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f142f577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1430115a30 0x7f143005e000 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52488/0 (socket says 10.20.193.28:52488) 2026-02-25T11:47:29.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f142f577640 1 -- 10.20.193.28:0/2569678163 learned_addr learned my addr 10.20.193.28:0/2569678163 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:29.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f142f577640 1 -- 10.20.193.28:0/2569678163 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f143010f840 msgr2=0x7f143005d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f142f577640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f143010f840 0x7f143005d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f142f577640 1 -- 10.20.193.28:0/2569678163 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1430110230 msgr2=0x7f1430176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f142e575640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1430110230 0x7f1430176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f142f577640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1430110230 0x7f1430176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f142f577640 1 -- 10.20.193.28:0/2569678163 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1418009570 con 0x7f1430115a30 2026-02-25T11:47:29.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f142e575640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1430110230 0x7f1430176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:29.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f142f577640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1430115a30 0x7f143005e000 secure :-1 s=READY pgs=95 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f142000e650 tx=0x7f142000eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f140ffff640 1 -- 10.20.193.28:0/2569678163 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f142000aa20 con 0x7f1430115a30 2026-02-25T11:47:29.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f140ffff640 1 -- 10.20.193.28:0/2569678163 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f142000ab80 con 0x7f1430115a30 2026-02-25T11:47:29.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f140ffff640 1 -- 10.20.193.28:0/2569678163 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1420010640 con 0x7f1430115a30 2026-02-25T11:47:29.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f143509a640 1 -- 10.20.193.28:0/2569678163 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1430101340 con 0x7f1430115a30 2026-02-25T11:47:29.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f143509a640 1 -- 10.20.193.28:0/2569678163 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1430101790 con 0x7f1430115a30 2026-02-25T11:47:29.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f143509a640 1 -- 10.20.193.28:0/2569678163 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1430110b20 con 0x7f1430115a30 2026-02-25T11:47:29.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.435+0000 7f140ffff640 1 -- 10.20.193.28:0/2569678163 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f1420021020 con 0x7f1430115a30 2026-02-25T11:47:29.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.441+0000 7f140ffff640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f14040b57f0 0x7f14040d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.441+0000 7f140ffff640 1 -- 10.20.193.28:0/2569678163 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f14200e7ac0 con 0x7f1430115a30 2026-02-25T11:47:29.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.441+0000 7f142ed76640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f14040b57f0 0x7f14040d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.441+0000 7f140ffff640 1 -- 10.20.193.28:0/2569678163 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f14200e9e60 con 0x7f1430115a30 2026-02-25T11:47:29.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.441+0000 7f142ed76640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f14040b57f0 0x7f14040d5ca0 secure :-1 s=READY pgs=213 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1424004740 tx=0x7f1424009690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.476+0000 7eff360e3640 1 -- 10.20.193.28:0/447105520 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff30112640 msgr2=0x7eff30173010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.476+0000 7eff360e3640 1 --2- 10.20.193.28:0/447105520 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff30112640 0x7eff30173010 secure :-1 s=READY pgs=93 gs=3 cs=0 l=1 c_cookie=10458747311526746042 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7eff2400aa30 tx=0x7eff24030290 comp rx=0 tx=0).stop 2026-02-25T11:47:29.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.476+0000 7eff2e7fc640 1 -- 10.20.193.28:0/447105520 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7eff240396f0 con 0x7eff30112640 2026-02-25T11:47:29.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.477+0000 7eff360e3640 1 -- 10.20.193.28:0/447105520 shutdown_connections 2026-02-25T11:47:29.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.477+0000 7eff360e3640 1 --2- 10.20.193.28:0/447105520 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff30112640 0x7eff30173010 unknown :-1 s=CLOSED pgs=93 gs=3 cs=0 l=1 c_cookie=10458747311526746042 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.477+0000 7eff360e3640 1 --2- 10.20.193.28:0/447105520 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff3010ce40 0x7eff30111e90 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.477+0000 7eff360e3640 1 --2- 10.20.193.28:0/447105520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff3010c450 0x7eff3010c820 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.477+0000 7eff360e3640 1 -- 10.20.193.28:0/447105520 >> 10.20.193.28:0/447105520 conn(0x7eff300e9d40 msgr2=0x7eff3010a150 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.480 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.478+0000 7eff360e3640 1 -- 10.20.193.28:0/447105520 shutdown_connections 2026-02-25T11:47:29.480 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.478+0000 7eff360e3640 1 -- 10.20.193.28:0/447105520 wait complete. 2026-02-25T11:47:29.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.479+0000 7eff360e3640 1 Processor -- start 2026-02-25T11:47:29.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.481+0000 7eff360e3640 1 -- start start 2026-02-25T11:47:29.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.481+0000 7eff360e3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff3010c450 0x7eff300ed9a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.481+0000 7eff360e3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff3010ce40 0x7eff30172f10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.481+0000 7eff360e3640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff30112640 0x7eff300edee0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.481+0000 7eff360e3640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7eff300ee5b0 con 0x7eff3010c450 2026-02-25T11:47:29.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.481+0000 7eff360e3640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7eff300ee720 con 0x7eff3010ce40 2026-02-25T11:47:29.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.481+0000 7eff360e3640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff30100020 con 0x7eff30112640 2026-02-25T11:47:29.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.482+0000 7eff2ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff30112640 0x7eff300edee0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.482+0000 7eff2ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff30112640 0x7eff300edee0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52506/0 (socket says 10.20.193.28:52506) 2026-02-25T11:47:29.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.482+0000 7eff2ffff640 1 -- 10.20.193.28:0/4115230148 learned_addr learned my addr 10.20.193.28:0/4115230148 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:29.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.482+0000 7eff2effd640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff3010ce40 0x7eff30172f10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.482+0000 7eff2f7fe640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff3010c450 0x7eff300ed9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.483+0000 7eff2ffff640 1 -- 10.20.193.28:0/4115230148 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff3010ce40 msgr2=0x7eff30172f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.483+0000 7eff2ffff640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff3010ce40 0x7eff30172f10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.483+0000 7eff2ffff640 1 -- 10.20.193.28:0/4115230148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff3010c450 msgr2=0x7eff300ed9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.483+0000 7eff2ffff640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff3010c450 0x7eff300ed9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.483+0000 7eff2ffff640 1 -- 10.20.193.28:0/4115230148 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff2400a650 con 0x7eff30112640 2026-02-25T11:47:29.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.484+0000 7eff2ffff640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff30112640 0x7eff300edee0 secure :-1 s=READY pgs=96 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7eff24006950 tx=0x7eff24003a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.484+0000 7eff2cff9640 1 -- 10.20.193.28:0/4115230148 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7eff2403e070 con 0x7eff30112640 2026-02-25T11:47:29.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.485+0000 7eff2cff9640 1 -- 10.20.193.28:0/4115230148 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7eff24005500 con 0x7eff30112640 2026-02-25T11:47:29.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.485+0000 7eff360e3640 1 -- 10.20.193.28:0/4115230148 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff301002a0 con 0x7eff30112640 2026-02-25T11:47:29.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.485+0000 7eff360e3640 1 -- 10.20.193.28:0/4115230148 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff301006c0 con 0x7eff30112640 2026-02-25T11:47:29.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.485+0000 7eff2cff9640 1 -- 10.20.193.28:0/4115230148 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7eff24042d50 con 0x7eff30112640 2026-02-25T11:47:29.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.488+0000 7eff2cff9640 1 -- 10.20.193.28:0/4115230148 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7eff24049020 con 0x7eff30112640 2026-02-25T11:47:29.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.490+0000 7eff360e3640 1 -- 10.20.193.28:0/4115230148 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef4005300 con 0x7eff30112640 2026-02-25T11:47:29.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.490+0000 7eff2cff9640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7eff040b5940 0x7eff040d5df0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.490+0000 7eff2cff9640 1 -- 10.20.193.28:0/4115230148 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7eff2410cfb0 con 0x7eff30112640 2026-02-25T11:47:29.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.491+0000 7eff2f7fe640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7eff040b5940 0x7eff040d5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.492+0000 7eff2f7fe640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7eff040b5940 0x7eff040d5df0 secure :-1 s=READY pgs=214 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7eff1c004740 tx=0x7eff1c004420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.493+0000 7eff2cff9640 1 -- 10.20.193.28:0/4115230148 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7eff240cfa10 con 0x7eff30112640 2026-02-25T11:47:29.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:29 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/369915339' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2026-02-25T11:47:29.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:29 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1793927631' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-25T11:47:29.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.500+0000 7f0c02582640 1 -- 10.20.193.28:0/3952021213 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0bf40b71b0 msgr2=0x7f0bf40b7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.500+0000 7f0c02582640 1 --2- 10.20.193.28:0/3952021213 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0bf40b71b0 0x7f0bf40b7580 secure :-1 s=READY pgs=94 gs=2 cs=0 l=1 c_cookie=4858998668946464919 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0bec009950 tx=0x7f0bec02f190 comp rx=0 tx=0).stop 2026-02-25T11:47:29.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.501+0000 7f0c02582640 1 -- 10.20.193.28:0/3952021213 shutdown_connections 2026-02-25T11:47:29.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.501+0000 7f0c02582640 1 --2- 10.20.193.28:0/3952021213 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0bf40bd3a0 0x7f0bf411dd70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.501+0000 7f0c02582640 1 --2- 10.20.193.28:0/3952021213 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0bf40b7ba0 0x7f0bf40bcbf0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.501+0000 7f0c02582640 1 --2- 10.20.193.28:0/3952021213 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0bf40b71b0 0x7f0bf40b7580 unknown :-1 s=CLOSED pgs=94 gs=2 cs=0 l=1 c_cookie=4858998668946464919 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.501+0000 7f0c02582640 1 -- 10.20.193.28:0/3952021213 >> 10.20.193.28:0/3952021213 conn(0x7f0bf4094aa0 msgr2=0x7f0bf40b4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.501+0000 7f0c02582640 1 -- 10.20.193.28:0/3952021213 shutdown_connections 2026-02-25T11:47:29.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.501+0000 7f0c02582640 1 -- 10.20.193.28:0/3952021213 wait complete. 2026-02-25T11:47:29.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.501+0000 7f0c02582640 1 Processor -- start 2026-02-25T11:47:29.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.502+0000 7f0c02582640 1 -- start start 2026-02-25T11:47:29.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.502+0000 7f0c02582640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0bf40b71b0 0x7f0bf40a1040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.502+0000 7f0c02582640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0bf40b7ba0 0x7f0bf411dc70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.502+0000 7f0c02582640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0bf40bd3a0 0x7f0bf40a1740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.502+0000 7f0c02582640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf40a6950 con 0x7f0bf40b7ba0 2026-02-25T11:47:29.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.502+0000 7f0c02582640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf40a6ac0 con 0x7f0bf40bd3a0 2026-02-25T11:47:29.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.502+0000 7f0c02582640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf40a6c80 con 0x7f0bf40b71b0 2026-02-25T11:47:29.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.503+0000 7f0c01d81640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0bf40bd3a0 0x7f0bf40a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.503+0000 7f0c01580640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0bf40b71b0 0x7f0bf40a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.503+0000 7f0c01580640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0bf40b71b0 0x7f0bf40a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52528/0 (socket says 10.20.193.28:52528) 2026-02-25T11:47:29.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.503+0000 7f0c01580640 1 -- 10.20.193.28:0/3658571408 learned_addr learned my addr 10.20.193.28:0/3658571408 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:29.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.503+0000 7f0c00d7f640 1 --2- 10.20.193.28:0/3658571408 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0bf40b7ba0 0x7f0bf411dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.504+0000 7f0c00d7f640 1 -- 10.20.193.28:0/3658571408 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0bf40b71b0 msgr2=0x7f0bf40a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.504+0000 7f0c00d7f640 1 --2- 10.20.193.28:0/3658571408 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0bf40b71b0 0x7f0bf40a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.504+0000 7f0c00d7f640 1 -- 10.20.193.28:0/3658571408 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0bf40bd3a0 msgr2=0x7f0bf40a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.504+0000 7f0c00d7f640 1 --2- 10.20.193.28:0/3658571408 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0bf40bd3a0 0x7f0bf40a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.504+0000 7f0c00d7f640 1 -- 10.20.193.28:0/3658571408 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf00094b0 con 0x7f0bf40b7ba0 2026-02-25T11:47:29.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.504+0000 7f0c00d7f640 1 --2- 10.20.193.28:0/3658571408 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0bf40b7ba0 0x7f0bf411dc70 secure :-1 s=READY pgs=280 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0bf0004680 tx=0x7f0bf0002b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.505+0000 7f0bea7fc640 1 -- 10.20.193.28:0/3658571408 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0bf000eb20 con 0x7f0bf40b7ba0 2026-02-25T11:47:29.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.505+0000 7f0bea7fc640 1 -- 10.20.193.28:0/3658571408 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0bf000ec80 con 0x7f0bf40b7ba0 2026-02-25T11:47:29.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.505+0000 7f0bea7fc640 1 -- 10.20.193.28:0/3658571408 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0bf000f7d0 con 0x7f0bf40b7ba0 2026-02-25T11:47:29.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.505+0000 7f0c02582640 1 -- 10.20.193.28:0/3658571408 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bec009570 con 0x7f0bf40b7ba0 2026-02-25T11:47:29.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.505+0000 7f0c02582640 1 -- 10.20.193.28:0/3658571408 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf40a72d0 con 0x7f0bf40b7ba0 2026-02-25T11:47:29.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.507+0000 7f0c02582640 1 -- 10.20.193.28:0/3658571408 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bf40b8490 con 0x7f0bf40b7ba0 2026-02-25T11:47:29.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:29 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/369915339' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2026-02-25T11:47:29.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:29 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1793927631' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-25T11:47:29.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.508+0000 7f0bea7fc640 1 -- 10.20.193.28:0/3658571408 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f0bf001f060 con 0x7f0bf40b7ba0 2026-02-25T11:47:29.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.512+0000 7f0bea7fc640 1 --2- 10.20.193.28:0/3658571408 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0bcc0b57e0 0x7f0bcc0d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:29.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.512+0000 7f0bea7fc640 1 -- 10.20.193.28:0/3658571408 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f0bf00e6e70 con 0x7f0bf40b7ba0 2026-02-25T11:47:29.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.513+0000 7f0c01580640 1 --2- 10.20.193.28:0/3658571408 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0bcc0b57e0 0x7f0bcc0d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:29.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.514+0000 7f0c01580640 1 --2- 10.20.193.28:0/3658571408 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0bcc0b57e0 0x7f0bcc0d5c90 secure :-1 s=READY pgs=215 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0bec0023b0 tx=0x7f0bec03a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:29.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.514+0000 7f0bea7fc640 1 -- 10.20.193.28:0/3658571408 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f0bf00a9870 con 0x7f0bf40b7ba0 2026-02-25T11:47:29.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.546+0000 7f7172ffd640 1 -- 10.20.193.28:0/1991297655 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f719c114d50 con 0x7f719c114900 2026-02-25T11:47:29.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.547+0000 7f71997fa640 1 -- 10.20.193.28:0/1991297655 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f718c0ac6e0 con 0x7f719c114900 2026-02-25T11:47:29.550 INFO:teuthology.orchestra.run.trial028.stdout:223338299413 2026-02-25T11:47:29.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.551+0000 7f7172ffd640 1 -- 10.20.193.28:0/1991297655 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f71780b57f0 msgr2=0x7f71780d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.551+0000 7f7172ffd640 1 --2- 10.20.193.28:0/1991297655 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f71780b57f0 0x7f71780d5ca0 secure :-1 s=READY pgs=212 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f719000d8e0 tx=0x7f7190002390 comp rx=0 tx=0).stop 2026-02-25T11:47:29.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.551+0000 7f7172ffd640 1 -- 10.20.193.28:0/1991297655 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f719c114900 msgr2=0x7f719c11fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.551+0000 7f7172ffd640 1 --2- 10.20.193.28:0/1991297655 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f719c114900 0x7f719c11fff0 secure :-1 s=READY pgs=93 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f718c00c9b0 tx=0x7f718c00ce80 comp rx=0 tx=0).stop 2026-02-25T11:47:29.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.551+0000 7f7172ffd640 1 -- 10.20.193.28:0/1991297655 shutdown_connections 2026-02-25T11:47:29.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.551+0000 7f7172ffd640 1 --2- 10.20.193.28:0/1991297655 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f71780b57f0 0x7f71780d5ca0 unknown :-1 s=CLOSED pgs=212 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.551+0000 7f7172ffd640 1 --2- 10.20.193.28:0/1991297655 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f719c117120 0x7f719c1258b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.551+0000 7f7172ffd640 1 --2- 10.20.193.28:0/1991297655 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f719c1152f0 0x7f719c120530 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.551+0000 7f7172ffd640 1 --2- 10.20.193.28:0/1991297655 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f719c114900 0x7f719c11fff0 unknown :-1 s=CLOSED pgs=93 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.551+0000 7f7172ffd640 1 -- 10.20.193.28:0/1991297655 >> 10.20.193.28:0/1991297655 conn(0x7f719c057d30 msgr2=0x7f719c115930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.552+0000 7f7172ffd640 1 -- 10.20.193.28:0/1991297655 shutdown_connections 2026-02-25T11:47:29.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.552+0000 7f7172ffd640 1 -- 10.20.193.28:0/1991297655 wait complete. 2026-02-25T11:47:29.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.563+0000 7fcfe37fe640 1 -- 10.20.193.28:0/1221891285 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd010115280 con 0x7fd010111c30 2026-02-25T11:47:29.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.564+0000 7fd00dffb640 1 -- 10.20.193.28:0/1221891285 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd0000a8920 con 0x7fd010111c30 2026-02-25T11:47:29.566 INFO:teuthology.orchestra.run.trial028.stdout:60129542181 2026-02-25T11:47:29.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.571+0000 7fcfe37fe640 1 -- 10.20.193.28:0/1221891285 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcfe40b58c0 msgr2=0x7fcfe40d5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.571+0000 7fcfe37fe640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcfe40b58c0 0x7fcfe40d5d70 secure :-1 s=READY pgs=211 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcffc0098a0 tx=0x7fcffc03c040 comp rx=0 tx=0).stop 2026-02-25T11:47:29.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.571+0000 7fcfe37fe640 1 -- 10.20.193.28:0/1221891285 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd010111c30 msgr2=0x7fd0100f51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.571+0000 7fcfe37fe640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd010111c30 0x7fd0100f51e0 secure :-1 s=READY pgs=279 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd00000c940 tx=0x7fd00000ce10 comp rx=0 tx=0).stop 2026-02-25T11:47:29.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.572+0000 7fcfe37fe640 1 -- 10.20.193.28:0/1221891285 shutdown_connections 2026-02-25T11:47:29.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.572+0000 7fcfe37fe640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcfe40b58c0 0x7fcfe40d5d70 unknown :-1 s=CLOSED pgs=211 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.572+0000 7fcfe37fe640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd010115a10 0x7fd0100fa5b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.572+0000 7fcfe37fe640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd010111c30 0x7fd0100f51e0 unknown :-1 s=CLOSED pgs=279 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.572+0000 7fcfe37fe640 1 --2- 10.20.193.28:0/1221891285 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd010111240 0x7fd0100f4ca0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.572+0000 7fcfe37fe640 1 -- 10.20.193.28:0/1221891285 >> 10.20.193.28:0/1221891285 conn(0x7fd0100ed130 msgr2=0x7fd01010dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.574+0000 7fcfe37fe640 1 -- 10.20.193.28:0/1221891285 shutdown_connections 2026-02-25T11:47:29.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.575+0000 7fcfe37fe640 1 -- 10.20.193.28:0/1221891285 wait complete. 2026-02-25T11:47:29.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.577+0000 7f143509a640 1 -- 10.20.193.28:0/2569678163 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f143010fc10 con 0x7f1430115a30 2026-02-25T11:47:29.584 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888093 got 146028888093 for osd.4 2026-02-25T11:47:29.585 DEBUG:teuthology.parallel:result is None 2026-02-25T11:47:29.593 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705704 got 38654705704 for osd.0 2026-02-25T11:47:29.594 DEBUG:teuthology.parallel:result is None 2026-02-25T11:47:29.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.592+0000 7f140ffff640 1 -- 10.20.193.28:0/2569678163 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f14200aa410 con 0x7f1430115a30 2026-02-25T11:47:29.597 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462937 got 201863462937 for osd.6 2026-02-25T11:47:29.598 INFO:teuthology.orchestra.run.trial028.stdout:292057776144 2026-02-25T11:47:29.598 DEBUG:teuthology.parallel:result is None 2026-02-25T11:47:29.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.596+0000 7f143509a640 1 -- 10.20.193.28:0/2569678163 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f14040b57f0 msgr2=0x7f14040d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.596+0000 7f143509a640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f14040b57f0 0x7f14040d5ca0 secure :-1 s=READY pgs=213 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1424004740 tx=0x7f1424009690 comp rx=0 tx=0).stop 2026-02-25T11:47:29.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.596+0000 7f143509a640 1 -- 10.20.193.28:0/2569678163 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1430115a30 msgr2=0x7f143005e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.596+0000 7f143509a640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1430115a30 0x7f143005e000 secure :-1 s=READY pgs=95 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f142000e650 tx=0x7f142000eb20 comp rx=0 tx=0).stop 2026-02-25T11:47:29.600 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350987 got 347892350987 for osd.11 2026-02-25T11:47:29.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.597+0000 7f143509a640 1 -- 10.20.193.28:0/2569678163 shutdown_connections 2026-02-25T11:47:29.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.597+0000 7f143509a640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f14040b57f0 0x7f14040d5ca0 unknown :-1 s=CLOSED pgs=213 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.597+0000 7f143509a640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1430115a30 0x7f143005e000 unknown :-1 s=CLOSED pgs=95 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.597+0000 7f143509a640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1430110230 0x7f1430176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.597+0000 7f143509a640 1 --2- 10.20.193.28:0/2569678163 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f143010f840 0x7f143005d8d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.597+0000 7f143509a640 1 -- 10.20.193.28:0/2569678163 >> 10.20.193.28:0/2569678163 conn(0x7f14300ed130 msgr2=0x7f143010dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.602 DEBUG:teuthology.parallel:result is None 2026-02-25T11:47:29.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.598+0000 7f143509a640 1 -- 10.20.193.28:0/2569678163 shutdown_connections 2026-02-25T11:47:29.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.598+0000 7f143509a640 1 -- 10.20.193.28:0/2569678163 wait complete. 2026-02-25T11:47:29.603 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612621 got 313532612621 for osd.10 2026-02-25T11:47:29.604 DEBUG:teuthology.parallel:result is None 2026-02-25T11:47:29.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.620+0000 7eff360e3640 1 -- 10.20.193.28:0/4115230148 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7efef40055b0 con 0x7eff30112640 2026-02-25T11:47:29.627 INFO:teuthology.orchestra.run.trial028.stdout:111669149728 2026-02-25T11:47:29.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.620+0000 7eff2cff9640 1 -- 10.20.193.28:0/4115230148 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7eff240d2bc0 con 0x7eff30112640 2026-02-25T11:47:29.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.626+0000 7eff360e3640 1 -- 10.20.193.28:0/4115230148 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7eff040b5940 msgr2=0x7eff040d5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.626+0000 7eff360e3640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7eff040b5940 0x7eff040d5df0 secure :-1 s=READY pgs=214 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7eff1c004740 tx=0x7eff1c004420 comp rx=0 tx=0).stop 2026-02-25T11:47:29.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.626+0000 7eff360e3640 1 -- 10.20.193.28:0/4115230148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff30112640 msgr2=0x7eff300edee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.626+0000 7eff360e3640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff30112640 0x7eff300edee0 secure :-1 s=READY pgs=96 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7eff24006950 tx=0x7eff24003a40 comp rx=0 tx=0).stop 2026-02-25T11:47:29.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.628+0000 7eff360e3640 1 -- 10.20.193.28:0/4115230148 shutdown_connections 2026-02-25T11:47:29.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.628+0000 7eff360e3640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7eff040b5940 0x7eff040d5df0 unknown :-1 s=CLOSED pgs=214 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.628+0000 7eff360e3640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff30112640 0x7eff300edee0 unknown :-1 s=CLOSED pgs=96 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.628+0000 7eff360e3640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff3010ce40 0x7eff30172f10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.628+0000 7eff360e3640 1 --2- 10.20.193.28:0/4115230148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff3010c450 0x7eff300ed9a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.628+0000 7eff360e3640 1 -- 10.20.193.28:0/4115230148 >> 10.20.193.28:0/4115230148 conn(0x7eff300e9d40 msgr2=0x7eff3005fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.628+0000 7eff360e3640 1 -- 10.20.193.28:0/4115230148 shutdown_connections 2026-02-25T11:47:29.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.628+0000 7eff360e3640 1 -- 10.20.193.28:0/4115230148 wait complete. 2026-02-25T11:47:29.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.651+0000 7f0c02582640 1 -- 10.20.193.28:0/3658571408 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0bf40a21a0 con 0x7f0bf40b7ba0 2026-02-25T11:47:29.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.652+0000 7f0bea7fc640 1 -- 10.20.193.28:0/3658571408 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0bf00092e0 con 0x7f0bf40b7ba0 2026-02-25T11:47:29.654 INFO:teuthology.orchestra.run.trial028.stdout:81604378659 2026-02-25T11:47:29.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.655+0000 7f0c02582640 1 -- 10.20.193.28:0/3658571408 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0bcc0b57e0 msgr2=0x7f0bcc0d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.655+0000 7f0c02582640 1 --2- 10.20.193.28:0/3658571408 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0bcc0b57e0 0x7f0bcc0d5c90 secure :-1 s=READY pgs=215 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0bec0023b0 tx=0x7f0bec03a040 comp rx=0 tx=0).stop 2026-02-25T11:47:29.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.655+0000 7f0c02582640 1 -- 10.20.193.28:0/3658571408 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0bf40b7ba0 msgr2=0x7f0bf411dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:29.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.655+0000 7f0c02582640 1 --2- 10.20.193.28:0/3658571408 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0bf40b7ba0 0x7f0bf411dc70 secure :-1 s=READY pgs=280 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0bf0004680 tx=0x7f0bf0002b40 comp rx=0 tx=0).stop 2026-02-25T11:47:29.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.655+0000 7f0c02582640 1 -- 10.20.193.28:0/3658571408 shutdown_connections 2026-02-25T11:47:29.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.655+0000 7f0c02582640 1 --2- 10.20.193.28:0/3658571408 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0bcc0b57e0 0x7f0bcc0d5c90 unknown :-1 s=CLOSED pgs=215 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.655+0000 7f0c02582640 1 --2- 10.20.193.28:0/3658571408 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0bf40bd3a0 0x7f0bf40a1740 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.655+0000 7f0c02582640 1 --2- 10.20.193.28:0/3658571408 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0bf40b7ba0 0x7f0bf411dc70 unknown :-1 s=CLOSED pgs=280 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.655+0000 7f0c02582640 1 --2- 10.20.193.28:0/3658571408 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0bf40b71b0 0x7f0bf40a1040 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:29.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.655+0000 7f0c02582640 1 -- 10.20.193.28:0/3658571408 >> 10.20.193.28:0/3658571408 conn(0x7f0bf4094aa0 msgr2=0x7f0bf40ba260 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:29.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.655+0000 7f0c02582640 1 -- 10.20.193.28:0/3658571408 shutdown_connections 2026-02-25T11:47:29.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:29.655+0000 7f0c02582640 1 -- 10.20.193.28:0/3658571408 wait complete. 2026-02-25T11:47:29.732 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299413 got 223338299413 for osd.7 2026-02-25T11:47:29.732 DEBUG:teuthology.parallel:result is None 2026-02-25T11:47:29.821 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149728 got 111669149728 for osd.3 2026-02-25T11:47:29.821 DEBUG:teuthology.parallel:result is None 2026-02-25T11:47:29.823 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542181 got 60129542181 for osd.1 2026-02-25T11:47:29.824 DEBUG:teuthology.parallel:result is None 2026-02-25T11:47:29.872 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378659 got 81604378659 for osd.2 2026-02-25T11:47:29.872 DEBUG:teuthology.parallel:result is None 2026-02-25T11:47:29.879 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776143 got 292057776144 for osd.9 2026-02-25T11:47:29.879 DEBUG:teuthology.parallel:result is None 2026-02-25T11:47:30.274 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.8 2026-02-25T11:47:30.296 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd last-stat-seq osd.5 2026-02-25T11:47:30.325 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:30 trial028 ceph-mon[39224]: pgmap v202: 97 pgs: 97 active+clean; 641 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 9.3 KiB/s rd, 9.2 KiB/s wr, 17 op/s 2026-02-25T11:47:30.325 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:30 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3213037380' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2026-02-25T11:47:30.325 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:30 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/185692718' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2026-02-25T11:47:30.325 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:30 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/179015499' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2026-02-25T11:47:30.325 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:30 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/78789496' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2026-02-25T11:47:30.326 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:30 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3750376105' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-25T11:47:30.326 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:30 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1991297655' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2026-02-25T11:47:30.326 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:30 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1221891285' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-25T11:47:30.326 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:30 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2569678163' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2026-02-25T11:47:30.326 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:30 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/4115230148' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-25T11:47:30.326 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:30 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3658571408' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-25T11:47:30.457 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:30.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:30 trial167 ceph-mon[43392]: pgmap v202: 97 pgs: 97 active+clean; 641 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 9.3 KiB/s rd, 9.2 KiB/s wr, 17 op/s 2026-02-25T11:47:30.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:30 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3213037380' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2026-02-25T11:47:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:30 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/185692718' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2026-02-25T11:47:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:30 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/179015499' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2026-02-25T11:47:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:30 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/78789496' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2026-02-25T11:47:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:30 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3750376105' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-25T11:47:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:30 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1991297655' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2026-02-25T11:47:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:30 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1221891285' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-25T11:47:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:30 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2569678163' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2026-02-25T11:47:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:30 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/4115230148' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-25T11:47:30.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:30 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3658571408' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-25T11:47:30.501 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:30 trial119 ceph-mon[42710]: pgmap v202: 97 pgs: 97 active+clean; 641 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 9.3 KiB/s rd, 9.2 KiB/s wr, 17 op/s 2026-02-25T11:47:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:30 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3213037380' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2026-02-25T11:47:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:30 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/185692718' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2026-02-25T11:47:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:30 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/179015499' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2026-02-25T11:47:30.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:30 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/78789496' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2026-02-25T11:47:30.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:30 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3750376105' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2026-02-25T11:47:30.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:30 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1991297655' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2026-02-25T11:47:30.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:30 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1221891285' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2026-02-25T11:47:30.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:30 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2569678163' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2026-02-25T11:47:30.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:30 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/4115230148' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2026-02-25T11:47:30.512 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:30 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3658571408' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2026-02-25T11:47:30.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.633+0000 7f1b99fae640 1 -- 10.20.193.28:0/535203893 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1b9410d3d0 msgr2=0x7f1b9410d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:30.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.633+0000 7f1b99fae640 1 --2- 10.20.193.28:0/535203893 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1b9410d3d0 0x7f1b9410d7a0 secure :-1 s=READY pgs=281 gs=1 cs=0 l=1 c_cookie=14469236044726572941 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1b84009950 tx=0x7f1b8402f190 comp rx=0 tx=0).stop 2026-02-25T11:47:30.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.633+0000 7f1b99fae640 1 -- 10.20.193.28:0/535203893 shutdown_connections 2026-02-25T11:47:30.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.633+0000 7f1b99fae640 1 --2- 10.20.193.28:0/535203893 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1b941135c0 0x7f1b94173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.633+0000 7f1b99fae640 1 --2- 10.20.193.28:0/535203893 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1b9410ddc0 0x7f1b94112e10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.633+0000 7f1b99fae640 1 --2- 10.20.193.28:0/535203893 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1b9410d3d0 0x7f1b9410d7a0 unknown :-1 s=CLOSED pgs=281 gs=1 cs=0 l=1 c_cookie=14469236044726572941 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.633+0000 7f1b99fae640 1 -- 10.20.193.28:0/535203893 >> 10.20.193.28:0/535203893 conn(0x7f1b940eaca0 msgr2=0x7f1b9410b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:30.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.633+0000 7f1b99fae640 1 -- 10.20.193.28:0/535203893 shutdown_connections 2026-02-25T11:47:30.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.633+0000 7f1b99fae640 1 -- 10.20.193.28:0/535203893 wait complete. 2026-02-25T11:47:30.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b99fae640 1 Processor -- start 2026-02-25T11:47:30.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b99fae640 1 -- start start 2026-02-25T11:47:30.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b99fae640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1b9410d3d0 0x7f1b940f1060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:30.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b99fae640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1b9410ddc0 0x7f1b94173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:30.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b99fae640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1b941135c0 0x7f1b940f1760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:30.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b99fae640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1b940f6920 con 0x7f1b9410d3d0 2026-02-25T11:47:30.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b99fae640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1b940f6a90 con 0x7f1b9410ddc0 2026-02-25T11:47:30.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b99fae640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b940f6c50 con 0x7f1b941135c0 2026-02-25T11:47:30.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b98fac640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1b9410d3d0 0x7f1b940f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:30.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b997ad640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1b941135c0 0x7f1b940f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:30.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b98fac640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1b9410d3d0 0x7f1b940f1060 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41050/0 (socket says 10.20.193.28:41050) 2026-02-25T11:47:30.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b98fac640 1 -- 10.20.193.28:0/3122973527 learned_addr learned my addr 10.20.193.28:0/3122973527 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:30.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b997ad640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1b941135c0 0x7f1b940f1760 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52534/0 (socket says 10.20.193.28:52534) 2026-02-25T11:47:30.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.634+0000 7f1b8bfff640 1 --2- 10.20.193.28:0/3122973527 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1b9410ddc0 0x7f1b94173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:30.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.635+0000 7f1b98fac640 1 -- 10.20.193.28:0/3122973527 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1b941135c0 msgr2=0x7f1b940f1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:30.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.635+0000 7f1b98fac640 1 --2- 10.20.193.28:0/3122973527 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1b941135c0 0x7f1b940f1760 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.635+0000 7f1b98fac640 1 -- 10.20.193.28:0/3122973527 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1b9410ddc0 msgr2=0x7f1b94173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:30.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.635+0000 7f1b98fac640 1 --2- 10.20.193.28:0/3122973527 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1b9410ddc0 0x7f1b94173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.635+0000 7f1b98fac640 1 -- 10.20.193.28:0/3122973527 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b84009570 con 0x7f1b9410d3d0 2026-02-25T11:47:30.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.635+0000 7f1b98fac640 1 --2- 10.20.193.28:0/3122973527 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1b9410d3d0 0x7f1b940f1060 secure :-1 s=READY pgs=282 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1b840098a0 tx=0x7f1b840028b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:30.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.635+0000 7f1b89ffb640 1 -- 10.20.193.28:0/3122973527 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1b8403d070 con 0x7f1b9410d3d0 2026-02-25T11:47:30.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.635+0000 7f1b99fae640 1 -- 10.20.193.28:0/3122973527 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b940f6ed0 con 0x7f1b9410d3d0 2026-02-25T11:47:30.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.635+0000 7f1b99fae640 1 -- 10.20.193.28:0/3122973527 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b940f7390 con 0x7f1b9410d3d0 2026-02-25T11:47:30.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.635+0000 7f1b89ffb640 1 -- 10.20.193.28:0/3122973527 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1b84002a50 con 0x7f1b9410d3d0 2026-02-25T11:47:30.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.635+0000 7f1b89ffb640 1 -- 10.20.193.28:0/3122973527 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1b84041690 con 0x7f1b9410d3d0 2026-02-25T11:47:30.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.636+0000 7f1b637fe640 1 -- 10.20.193.28:0/3122973527 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b9410e6b0 con 0x7f1b9410d3d0 2026-02-25T11:47:30.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.638+0000 7f1b89ffb640 1 -- 10.20.193.28:0/3122973527 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f1b840386a0 con 0x7f1b9410d3d0 2026-02-25T11:47:30.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.640+0000 7f1b89ffb640 1 --2- 10.20.193.28:0/3122973527 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1b700b57e0 0x7f1b700d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:30.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.640+0000 7f1b89ffb640 1 -- 10.20.193.28:0/3122973527 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f1b8410b190 con 0x7f1b9410d3d0 2026-02-25T11:47:30.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.640+0000 7f1b89ffb640 1 -- 10.20.193.28:0/3122973527 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f1b8410b610 con 0x7f1b9410d3d0 2026-02-25T11:47:30.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.640+0000 7f1b8bfff640 1 --2- 10.20.193.28:0/3122973527 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1b700b57e0 0x7f1b700d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:30.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.640+0000 7f1b8bfff640 1 --2- 10.20.193.28:0/3122973527 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1b700b57e0 0x7f1b700d5c90 secure :-1 s=READY pgs=216 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1b940f79b0 tx=0x7f1b7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:30.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.678+0000 7f448cc7a640 1 -- 10.20.193.28:0/3821314598 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4488118820 msgr2=0x7f4488118cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:30.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.678+0000 7f448cc7a640 1 --2- 10.20.193.28:0/3821314598 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4488118820 0x7f4488118cc0 secure :-1 s=READY pgs=283 gs=1 cs=0 l=1 c_cookie=4510856478366993682 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f447800aa30 tx=0x7f4478030270 comp rx=0 tx=0).stop 2026-02-25T11:47:30.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.678+0000 7f448cc7a640 1 -- 10.20.193.28:0/3821314598 shutdown_connections 2026-02-25T11:47:30.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.678+0000 7f448cc7a640 1 --2- 10.20.193.28:0/3821314598 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4488118820 0x7f4488118cc0 unknown :-1 s=CLOSED pgs=283 gs=1 cs=0 l=1 c_cookie=4510856478366993682 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.678+0000 7f448cc7a640 1 --2- 10.20.193.28:0/3821314598 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f448811ad20 0x7f448811b150 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.678+0000 7f448cc7a640 1 --2- 10.20.193.28:0/3821314598 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f448811bab0 0x7f448811a7e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.678+0000 7f448cc7a640 1 -- 10.20.193.28:0/3821314598 >> 10.20.193.28:0/3821314598 conn(0x7f4488058190 msgr2=0x7f4488058590 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:30.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.678+0000 7f448cc7a640 1 -- 10.20.193.28:0/3821314598 shutdown_connections 2026-02-25T11:47:30.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.679+0000 7f448cc7a640 1 -- 10.20.193.28:0/3821314598 wait complete. 2026-02-25T11:47:30.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.679+0000 7f448cc7a640 1 Processor -- start 2026-02-25T11:47:30.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.680+0000 7f448cc7a640 1 -- start start 2026-02-25T11:47:30.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.680+0000 7f448cc7a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4488118820 0x7f44880fffc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:30.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.680+0000 7f448cc7a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f448811ad20 0x7f4488100500 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:30.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.680+0000 7f448cc7a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f448811bab0 0x7f44881058d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:30.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.681+0000 7f448cc7a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4488105ef0 con 0x7f4488118820 2026-02-25T11:47:30.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.681+0000 7f448cc7a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4488106060 con 0x7f448811ad20 2026-02-25T11:47:30.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.681+0000 7f448cc7a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4488106220 con 0x7f448811bab0 2026-02-25T11:47:30.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.681+0000 7f4486575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4488118820 0x7f44880fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:30.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.681+0000 7f4485d74640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f448811ad20 0x7f4488100500 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:30.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.681+0000 7f4485d74640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f448811ad20 0x7f4488100500 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:42082/0 (socket says 10.20.193.28:42082) 2026-02-25T11:47:30.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.681+0000 7f4485d74640 1 -- 10.20.193.28:0/1495280074 learned_addr learned my addr 10.20.193.28:0/1495280074 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:30.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.681+0000 7f4486d76640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f448811bab0 0x7f44881058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:30.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.682+0000 7f4485d74640 1 -- 10.20.193.28:0/1495280074 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f448811bab0 msgr2=0x7f44881058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:30.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.682+0000 7f4485d74640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f448811bab0 0x7f44881058d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.682+0000 7f4485d74640 1 -- 10.20.193.28:0/1495280074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4488118820 msgr2=0x7f44880fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:30.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.682+0000 7f4485d74640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4488118820 0x7f44880fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.682+0000 7f4485d74640 1 -- 10.20.193.28:0/1495280074 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f447800a650 con 0x7f448811ad20 2026-02-25T11:47:30.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.682+0000 7f4486575640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4488118820 0x7f44880fffc0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:30.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.682+0000 7f4485d74640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f448811ad20 0x7f4488100500 secure :-1 s=READY pgs=95 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4470009480 tx=0x7f447000ede0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:30.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.682+0000 7f446f7fe640 1 -- 10.20.193.28:0/1495280074 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f447000cce0 con 0x7f448811ad20 2026-02-25T11:47:30.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.682+0000 7f448cc7a640 1 -- 10.20.193.28:0/1495280074 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4488106500 con 0x7f448811ad20 2026-02-25T11:47:30.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.682+0000 7f446f7fe640 1 -- 10.20.193.28:0/1495280074 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f447000ce40 con 0x7f448811ad20 2026-02-25T11:47:30.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.683+0000 7f446f7fe640 1 -- 10.20.193.28:0/1495280074 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f44700106b0 con 0x7f448811ad20 2026-02-25T11:47:30.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.683+0000 7f448cc7a640 1 -- 10.20.193.28:0/1495280074 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4488106be0 con 0x7f448811ad20 2026-02-25T11:47:30.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.683+0000 7f448cc7a640 1 -- 10.20.193.28:0/1495280074 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f448805a610 con 0x7f448811ad20 2026-02-25T11:47:30.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.685+0000 7f446f7fe640 1 -- 10.20.193.28:0/1495280074 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f447001f060 con 0x7f448811ad20 2026-02-25T11:47:30.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.687+0000 7f446f7fe640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f44600b57f0 0x7f44600d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:30.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.687+0000 7f446f7fe640 1 -- 10.20.193.28:0/1495280074 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f4470014070 con 0x7f448811ad20 2026-02-25T11:47:30.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.687+0000 7f446f7fe640 1 -- 10.20.193.28:0/1495280074 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f447011f940 con 0x7f448811ad20 2026-02-25T11:47:30.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.688+0000 7f4486575640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f44600b57f0 0x7f44600d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:30.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.688+0000 7f4486575640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f44600b57f0 0x7f44600d5ca0 secure :-1 s=READY pgs=217 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f447c004740 tx=0x7f447c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:30.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.757+0000 7f1b637fe640 1 -- 10.20.193.28:0/3122973527 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f1b940f2170 con 0x7f1b9410d3d0 2026-02-25T11:47:30.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.757+0000 7f1b89ffb640 1 -- 10.20.193.28:0/3122973527 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1b840cdca0 con 0x7f1b9410d3d0 2026-02-25T11:47:30.759 INFO:teuthology.orchestra.run.trial028.stdout:257698037778 2026-02-25T11:47:30.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.759+0000 7f1b637fe640 1 -- 10.20.193.28:0/3122973527 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1b700b57e0 msgr2=0x7f1b700d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:30.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.759+0000 7f1b637fe640 1 --2- 10.20.193.28:0/3122973527 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1b700b57e0 0x7f1b700d5c90 secure :-1 s=READY pgs=216 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1b940f79b0 tx=0x7f1b7c005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:30.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.759+0000 7f1b637fe640 1 -- 10.20.193.28:0/3122973527 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1b9410d3d0 msgr2=0x7f1b940f1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:30.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.759+0000 7f1b637fe640 1 --2- 10.20.193.28:0/3122973527 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1b9410d3d0 0x7f1b940f1060 secure :-1 s=READY pgs=282 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1b840098a0 tx=0x7f1b840028b0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.759+0000 7f1b637fe640 1 -- 10.20.193.28:0/3122973527 shutdown_connections 2026-02-25T11:47:30.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.759+0000 7f1b637fe640 1 --2- 10.20.193.28:0/3122973527 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1b700b57e0 0x7f1b700d5c90 unknown :-1 s=CLOSED pgs=216 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.759+0000 7f1b637fe640 1 --2- 10.20.193.28:0/3122973527 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1b941135c0 0x7f1b940f1760 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.759+0000 7f1b637fe640 1 --2- 10.20.193.28:0/3122973527 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1b9410ddc0 0x7f1b94173e90 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.759+0000 7f1b637fe640 1 --2- 10.20.193.28:0/3122973527 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1b9410d3d0 0x7f1b940f1060 unknown :-1 s=CLOSED pgs=282 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.759+0000 7f1b637fe640 1 -- 10.20.193.28:0/3122973527 >> 10.20.193.28:0/3122973527 conn(0x7f1b940eaca0 msgr2=0x7f1b94110480 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:30.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.760+0000 7f1b637fe640 1 -- 10.20.193.28:0/3122973527 shutdown_connections 2026-02-25T11:47:30.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.760+0000 7f1b637fe640 1 -- 10.20.193.28:0/3122973527 wait complete. 2026-02-25T11:47:30.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.804+0000 7f448cc7a640 1 -- 10.20.193.28:0/1495280074 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4488101120 con 0x7f448811ad20 2026-02-25T11:47:30.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.805+0000 7f446f7fe640 1 -- 10.20.193.28:0/1495280074 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f44700a95e0 con 0x7f448811ad20 2026-02-25T11:47:30.807 INFO:teuthology.orchestra.run.trial028.stdout:167503724571 2026-02-25T11:47:30.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.808+0000 7f448cc7a640 1 -- 10.20.193.28:0/1495280074 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f44600b57f0 msgr2=0x7f44600d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:30.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.808+0000 7f448cc7a640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f44600b57f0 0x7f44600d5ca0 secure :-1 s=READY pgs=217 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f447c004740 tx=0x7f447c005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:30.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.808+0000 7f448cc7a640 1 -- 10.20.193.28:0/1495280074 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f448811ad20 msgr2=0x7f4488100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:30.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.808+0000 7f448cc7a640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f448811ad20 0x7f4488100500 secure :-1 s=READY pgs=95 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4470009480 tx=0x7f447000ede0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.808+0000 7f448cc7a640 1 -- 10.20.193.28:0/1495280074 shutdown_connections 2026-02-25T11:47:30.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.808+0000 7f448cc7a640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f44600b57f0 0x7f44600d5ca0 unknown :-1 s=CLOSED pgs=217 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.808+0000 7f448cc7a640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f448811bab0 0x7f44881058d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.808+0000 7f448cc7a640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f448811ad20 0x7f4488100500 unknown :-1 s=CLOSED pgs=95 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.808+0000 7f448cc7a640 1 --2- 10.20.193.28:0/1495280074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4488118820 0x7f44880fffc0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:30.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.808+0000 7f448cc7a640 1 -- 10.20.193.28:0/1495280074 >> 10.20.193.28:0/1495280074 conn(0x7f4488058190 msgr2=0x7f44880f5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:30.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.810+0000 7f448cc7a640 1 -- 10.20.193.28:0/1495280074 shutdown_connections 2026-02-25T11:47:30.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:30.810+0000 7f448cc7a640 1 -- 10.20.193.28:0/1495280074 wait complete. 2026-02-25T11:47:30.898 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037778 got 257698037778 for osd.8 2026-02-25T11:47:30.898 DEBUG:teuthology.parallel:result is None 2026-02-25T11:47:30.953 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724570 got 167503724571 for osd.5 2026-02-25T11:47:30.953 DEBUG:teuthology.parallel:result is None 2026-02-25T11:47:30.954 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2026-02-25T11:47:30.954 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph pg dump --format=json 2026-02-25T11:47:31.135 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:31.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.319+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/3492081074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd6810f690 msgr2=0x7fbd6810fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:31.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.319+0000 7fbd6cfd5640 1 --2- 10.20.193.28:0/3492081074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd6810f690 0x7fbd6810fa60 secure :-1 s=READY pgs=284 gs=1 cs=0 l=1 c_cookie=33733431042895329 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbd54009950 tx=0x7fbd5402f190 comp rx=0 tx=0).stop 2026-02-25T11:47:31.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.319+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/3492081074 shutdown_connections 2026-02-25T11:47:31.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.319+0000 7fbd6cfd5640 1 --2- 10.20.193.28:0/3492081074 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd68115880 0x7fbd68176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.319+0000 7fbd6cfd5640 1 --2- 10.20.193.28:0/3492081074 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbd68110080 0x7fbd681150d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.319+0000 7fbd6cfd5640 1 --2- 10.20.193.28:0/3492081074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd6810f690 0x7fbd6810fa60 unknown :-1 s=CLOSED pgs=284 gs=1 cs=0 l=1 c_cookie=33733431042895329 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.319+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/3492081074 >> 10.20.193.28:0/3492081074 conn(0x7fbd680ecf00 msgr2=0x7fbd6810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:31.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.319+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/3492081074 shutdown_connections 2026-02-25T11:47:31.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.319+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/3492081074 wait complete. 2026-02-25T11:47:31.324 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.320+0000 7fbd6cfd5640 1 Processor -- start 2026-02-25T11:47:31.324 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.320+0000 7fbd6cfd5640 1 -- start start 2026-02-25T11:47:31.324 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.320+0000 7fbd6cfd5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd6810f690 0x7fbd6811e100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:31.324 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.320+0000 7fbd6cfd5640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd68110080 0x7fbd6811e640 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:31.324 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.320+0000 7fbd6cfd5640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbd68115880 0x7fbd68123a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:31.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.320+0000 7fbd6cfd5640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd68124030 con 0x7fbd6810f690 2026-02-25T11:47:31.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.320+0000 7fbd6cfd5640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbd681241a0 con 0x7fbd68115880 2026-02-25T11:47:31.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.320+0000 7fbd6cfd5640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd68124360 con 0x7fbd68110080 2026-02-25T11:47:31.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd677fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd6810f690 0x7fbd6811e100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:31.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd677fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd6810f690 0x7fbd6811e100 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41084/0 (socket says 10.20.193.28:41084) 2026-02-25T11:47:31.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd677fe640 1 -- 10.20.193.28:0/2160594238 learned_addr learned my addr 10.20.193.28:0/2160594238 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:31.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd66ffd640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd68110080 0x7fbd6811e640 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:31.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd67fff640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbd68115880 0x7fbd68123a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:31.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd677fe640 1 -- 10.20.193.28:0/2160594238 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd68110080 msgr2=0x7fbd6811e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:31.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd677fe640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd68110080 0x7fbd6811e640 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd677fe640 1 -- 10.20.193.28:0/2160594238 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbd68115880 msgr2=0x7fbd68123a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:31.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd677fe640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbd68115880 0x7fbd68123a10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd677fe640 1 -- 10.20.193.28:0/2160594238 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd54009570 con 0x7fbd6810f690 2026-02-25T11:47:31.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd677fe640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd6810f690 0x7fbd6811e100 secure :-1 s=READY pgs=285 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbd540098a0 tx=0x7fbd540028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:31.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd64ff9640 1 -- 10.20.193.28:0/2160594238 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbd5403d070 con 0x7fbd6810f690 2026-02-25T11:47:31.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd64ff9640 1 -- 10.20.193.28:0/2160594238 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fbd54002a40 con 0x7fbd6810f690 2026-02-25T11:47:31.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd64ff9640 1 -- 10.20.193.28:0/2160594238 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbd54041820 con 0x7fbd6810f690 2026-02-25T11:47:31.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/2160594238 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd681245e0 con 0x7fbd6810f690 2026-02-25T11:47:31.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.321+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/2160594238 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd68164d40 con 0x7fbd6810f690 2026-02-25T11:47:31.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.322+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/2160594238 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd68110970 con 0x7fbd6810f690 2026-02-25T11:47:31.327 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.324+0000 7fbd64ff9640 1 -- 10.20.193.28:0/2160594238 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fbd540419c0 con 0x7fbd6810f690 2026-02-25T11:47:31.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.326+0000 7fbd64ff9640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbd3c0b57f0 0x7fbd3c0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:31.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.326+0000 7fbd64ff9640 1 -- 10.20.193.28:0/2160594238 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fbd5410b470 con 0x7fbd6810f690 2026-02-25T11:47:31.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.326+0000 7fbd66ffd640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbd3c0b57f0 0x7fbd3c0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:31.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.326+0000 7fbd64ff9640 1 -- 10.20.193.28:0/2160594238 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fbd5410b8f0 con 0x7fbd6810f690 2026-02-25T11:47:31.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.326+0000 7fbd66ffd640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbd3c0b57f0 0x7fbd3c0d5ca0 secure :-1 s=READY pgs=218 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbd68124aa0 tx=0x7fbd58005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:31.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.438+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/2160594238 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd6811f260 con 0x7fbd3c0b57f0 2026-02-25T11:47:31.441 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:31 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3122973527' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-25T11:47:31.441 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:31 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1495280074' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2026-02-25T11:47:31.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.442+0000 7fbd64ff9640 1 -- 10.20.193.28:0/2160594238 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258970 (secure 0 0 0) 0x7fbd6811f260 con 0x7fbd3c0b57f0 2026-02-25T11:47:31.447 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:31.448 INFO:teuthology.orchestra.run.trial028.stderr:dumped all 2026-02-25T11:47:31.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.444+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/2160594238 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbd3c0b57f0 msgr2=0x7fbd3c0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:31.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.444+0000 7fbd6cfd5640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbd3c0b57f0 0x7fbd3c0d5ca0 secure :-1 s=READY pgs=218 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbd68124aa0 tx=0x7fbd58005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:31.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.444+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/2160594238 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd6810f690 msgr2=0x7fbd6811e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:31.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.444+0000 7fbd6cfd5640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd6810f690 0x7fbd6811e100 secure :-1 s=READY pgs=285 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbd540098a0 tx=0x7fbd540028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.444+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/2160594238 shutdown_connections 2026-02-25T11:47:31.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.444+0000 7fbd6cfd5640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbd3c0b57f0 0x7fbd3c0d5ca0 unknown :-1 s=CLOSED pgs=218 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.444+0000 7fbd6cfd5640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbd68115880 0x7fbd68123a10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.444+0000 7fbd6cfd5640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd68110080 0x7fbd6811e640 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.444+0000 7fbd6cfd5640 1 --2- 10.20.193.28:0/2160594238 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd6810f690 0x7fbd6811e100 unknown :-1 s=CLOSED pgs=285 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.444+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/2160594238 >> 10.20.193.28:0/2160594238 conn(0x7fbd680ecf00 msgr2=0x7fbd6810d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:31.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.444+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/2160594238 shutdown_connections 2026-02-25T11:47:31.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.444+0000 7fbd6cfd5640 1 -- 10.20.193.28:0/2160594238 wait complete. 2026-02-25T11:47:31.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:31 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3122973527' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-25T11:47:31.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:31 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1495280074' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2026-02-25T11:47:31.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:31 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3122973527' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2026-02-25T11:47:31.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:31 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1495280074' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2026-02-25T11:47:31.546 INFO:teuthology.orchestra.run.trial028.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2026-02-25T11:47:31.076972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656776,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":195,"num_read_kb":203,"num_write":349,"num_write_kb":3834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":289,"ondisk_log_size":289,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":8808038400,"kb_used":338104,"kb_used_data":11464,"kb_used_omap":464,"kb_used_meta":325935,"kb_avail":8807700296,"statfs":{"total":9019431321600,"available":9019085103104,"internally_reserved":0,"allocated":11739136,"data_stored":7420852,"data_compressed":114024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":475320,"internal_metadata":333758280},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":62034,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":84,"num_read_kb":108,"num_write":82,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006497"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627807+0000","last_change":"2026-02-25T11:47:13.630492+0000","last_active":"2026-02-25T11:47:14.627807+0000","last_peered":"2026-02-25T11:47:14.627807+0000","last_clean":"2026-02-25T11:47:14.627807+0000","last_became_active":"2026-02-25T11:47:13.630388+0000","last_became_peered":"2026-02-25T11:47:13.630388+0000","last_unstale":"2026-02-25T11:47:14.627807+0000","last_undegraded":"2026-02-25T11:47:14.627807+0000","last_fullsized":"2026-02-25T11:47:14.627807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:01:17.024058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627657+0000","last_change":"2026-02-25T11:47:13.629507+0000","last_active":"2026-02-25T11:47:14.627657+0000","last_peered":"2026-02-25T11:47:14.627657+0000","last_clean":"2026-02-25T11:47:14.627657+0000","last_became_active":"2026-02-25T11:47:13.628937+0000","last_became_peered":"2026-02-25T11:47:13.628937+0000","last_unstale":"2026-02-25T11:47:14.627657+0000","last_undegraded":"2026-02-25T11:47:14.627657+0000","last_fullsized":"2026-02-25T11:47:14.627657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:47:20.441522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627713+0000","last_change":"2026-02-25T11:47:13.630415+0000","last_active":"2026-02-25T11:47:14.627713+0000","last_peered":"2026-02-25T11:47:14.627713+0000","last_clean":"2026-02-25T11:47:14.627713+0000","last_became_active":"2026-02-25T11:47:13.630319+0000","last_became_peered":"2026-02-25T11:47:13.630319+0000","last_unstale":"2026-02-25T11:47:14.627713+0000","last_undegraded":"2026-02-25T11:47:14.627713+0000","last_fullsized":"2026-02-25T11:47:14.627713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:18:57.675760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627800+0000","last_change":"2026-02-25T11:47:13.633017+0000","last_active":"2026-02-25T11:47:14.627800+0000","last_peered":"2026-02-25T11:47:14.627800+0000","last_clean":"2026-02-25T11:47:14.627800+0000","last_became_active":"2026-02-25T11:47:13.632934+0000","last_became_peered":"2026-02-25T11:47:13.632934+0000","last_unstale":"2026-02-25T11:47:14.627800+0000","last_undegraded":"2026-02-25T11:47:14.627800+0000","last_fullsized":"2026-02-25T11:47:14.627800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:08:54.195980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628667+0000","last_change":"2026-02-25T11:47:13.638057+0000","last_active":"2026-02-25T11:47:14.628667+0000","last_peered":"2026-02-25T11:47:14.628667+0000","last_clean":"2026-02-25T11:47:14.628667+0000","last_became_active":"2026-02-25T11:47:13.636387+0000","last_became_peered":"2026-02-25T11:47:13.636387+0000","last_unstale":"2026-02-25T11:47:14.628667+0000","last_undegraded":"2026-02-25T11:47:14.628667+0000","last_fullsized":"2026-02-25T11:47:14.628667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:34:29.570622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750630+0000","last_change":"2026-02-25T11:47:13.631824+0000","last_active":"2026-02-25T11:47:14.750630+0000","last_peered":"2026-02-25T11:47:14.750630+0000","last_clean":"2026-02-25T11:47:14.750630+0000","last_became_active":"2026-02-25T11:47:13.631656+0000","last_became_peered":"2026-02-25T11:47:13.631656+0000","last_unstale":"2026-02-25T11:47:14.750630+0000","last_undegraded":"2026-02-25T11:47:14.750630+0000","last_fullsized":"2026-02-25T11:47:14.750630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:15:05.237015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628186+0000","last_change":"2026-02-25T11:47:13.626877+0000","last_active":"2026-02-25T11:47:14.628186+0000","last_peered":"2026-02-25T11:47:14.628186+0000","last_clean":"2026-02-25T11:47:14.628186+0000","last_became_active":"2026-02-25T11:47:13.626543+0000","last_became_peered":"2026-02-25T11:47:13.626543+0000","last_unstale":"2026-02-25T11:47:14.628186+0000","last_undegraded":"2026-02-25T11:47:14.628186+0000","last_fullsized":"2026-02-25T11:47:14.628186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:56:58.194911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635224+0000","last_change":"2026-02-25T11:47:13.637661+0000","last_active":"2026-02-25T11:47:14.635224+0000","last_peered":"2026-02-25T11:47:14.635224+0000","last_clean":"2026-02-25T11:47:14.635224+0000","last_became_active":"2026-02-25T11:47:13.637230+0000","last_became_peered":"2026-02-25T11:47:13.637230+0000","last_unstale":"2026-02-25T11:47:14.635224+0000","last_undegraded":"2026-02-25T11:47:14.635224+0000","last_fullsized":"2026-02-25T11:47:14.635224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:44:52.310360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751287+0000","last_change":"2026-02-25T11:47:13.625160+0000","last_active":"2026-02-25T11:47:14.751287+0000","last_peered":"2026-02-25T11:47:14.751287+0000","last_clean":"2026-02-25T11:47:14.751287+0000","last_became_active":"2026-02-25T11:47:13.625020+0000","last_became_peered":"2026-02-25T11:47:13.625020+0000","last_unstale":"2026-02-25T11:47:14.751287+0000","last_undegraded":"2026-02-25T11:47:14.751287+0000","last_fullsized":"2026-02-25T11:47:14.751287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:35:44.509551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750832+0000","last_change":"2026-02-25T11:47:13.637252+0000","last_active":"2026-02-25T11:47:14.750832+0000","last_peered":"2026-02-25T11:47:14.750832+0000","last_clean":"2026-02-25T11:47:14.750832+0000","last_became_active":"2026-02-25T11:47:13.636670+0000","last_became_peered":"2026-02-25T11:47:13.636670+0000","last_unstale":"2026-02-25T11:47:14.750832+0000","last_undegraded":"2026-02-25T11:47:14.750832+0000","last_fullsized":"2026-02-25T11:47:14.750832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:01:53.119101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628443+0000","last_change":"2026-02-25T11:47:13.628252+0000","last_active":"2026-02-25T11:47:14.628443+0000","last_peered":"2026-02-25T11:47:14.628443+0000","last_clean":"2026-02-25T11:47:14.628443+0000","last_became_active":"2026-02-25T11:47:13.628183+0000","last_became_peered":"2026-02-25T11:47:13.628183+0000","last_unstale":"2026-02-25T11:47:14.628443+0000","last_undegraded":"2026-02-25T11:47:14.628443+0000","last_fullsized":"2026-02-25T11:47:14.628443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:12:00.576747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.632358+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.631487+0000","last_became_peered":"2026-02-25T11:47:13.631487+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:41:17.150851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749735+0000","last_change":"2026-02-25T11:47:13.629675+0000","last_active":"2026-02-25T11:47:14.749735+0000","last_peered":"2026-02-25T11:47:14.749735+0000","last_clean":"2026-02-25T11:47:14.749735+0000","last_became_active":"2026-02-25T11:47:13.628686+0000","last_became_peered":"2026-02-25T11:47:13.628686+0000","last_unstale":"2026-02-25T11:47:14.749735+0000","last_undegraded":"2026-02-25T11:47:14.749735+0000","last_fullsized":"2026-02-25T11:47:14.749735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:00:54.430287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627615+0000","last_change":"2026-02-25T11:47:13.628842+0000","last_active":"2026-02-25T11:47:14.627615+0000","last_peered":"2026-02-25T11:47:14.627615+0000","last_clean":"2026-02-25T11:47:14.627615+0000","last_became_active":"2026-02-25T11:47:13.628695+0000","last_became_peered":"2026-02-25T11:47:13.628695+0000","last_unstale":"2026-02-25T11:47:14.627615+0000","last_undegraded":"2026-02-25T11:47:14.627615+0000","last_fullsized":"2026-02-25T11:47:14.627615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:10:05.404740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.635652+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.635505+0000","last_became_peered":"2026-02-25T11:47:13.635505+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:46:03.598577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749928+0000","last_change":"2026-02-25T11:47:13.628651+0000","last_active":"2026-02-25T11:47:14.749928+0000","last_peered":"2026-02-25T11:47:14.749928+0000","last_clean":"2026-02-25T11:47:14.749928+0000","last_became_active":"2026-02-25T11:47:13.628280+0000","last_became_peered":"2026-02-25T11:47:13.628280+0000","last_unstale":"2026-02-25T11:47:14.749928+0000","last_undegraded":"2026-02-25T11:47:14.749928+0000","last_fullsized":"2026-02-25T11:47:14.749928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:18:56.238078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628351+0000","last_change":"2026-02-25T11:47:13.630656+0000","last_active":"2026-02-25T11:47:14.628351+0000","last_peered":"2026-02-25T11:47:14.628351+0000","last_clean":"2026-02-25T11:47:14.628351+0000","last_became_active":"2026-02-25T11:47:13.630513+0000","last_became_peered":"2026-02-25T11:47:13.630513+0000","last_unstale":"2026-02-25T11:47:14.628351+0000","last_undegraded":"2026-02-25T11:47:14.628351+0000","last_fullsized":"2026-02-25T11:47:14.628351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:03:33.580576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.634036+0000","last_change":"2026-02-25T11:47:13.629202+0000","last_active":"2026-02-25T11:47:14.634036+0000","last_peered":"2026-02-25T11:47:14.634036+0000","last_clean":"2026-02-25T11:47:14.634036+0000","last_became_active":"2026-02-25T11:47:13.628860+0000","last_became_peered":"2026-02-25T11:47:13.628860+0000","last_unstale":"2026-02-25T11:47:14.634036+0000","last_undegraded":"2026-02-25T11:47:14.634036+0000","last_fullsized":"2026-02-25T11:47:14.634036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:47:14.635796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628235+0000","last_change":"2026-02-25T11:47:13.638872+0000","last_active":"2026-02-25T11:47:14.628235+0000","last_peered":"2026-02-25T11:47:14.628235+0000","last_clean":"2026-02-25T11:47:14.628235+0000","last_became_active":"2026-02-25T11:47:13.638443+0000","last_became_peered":"2026-02-25T11:47:13.638443+0000","last_unstale":"2026-02-25T11:47:14.628235+0000","last_undegraded":"2026-02-25T11:47:14.628235+0000","last_fullsized":"2026-02-25T11:47:14.628235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:51.699838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628595+0000","last_change":"2026-02-25T11:47:13.635894+0000","last_active":"2026-02-25T11:47:14.628595+0000","last_peered":"2026-02-25T11:47:14.628595+0000","last_clean":"2026-02-25T11:47:14.628595+0000","last_became_active":"2026-02-25T11:47:13.635725+0000","last_became_peered":"2026-02-25T11:47:13.635725+0000","last_unstale":"2026-02-25T11:47:14.628595+0000","last_undegraded":"2026-02-25T11:47:14.628595+0000","last_fullsized":"2026-02-25T11:47:14.628595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:28:20.134454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749794+0000","last_change":"2026-02-25T11:47:13.628117+0000","last_active":"2026-02-25T11:47:14.749794+0000","last_peered":"2026-02-25T11:47:14.749794+0000","last_clean":"2026-02-25T11:47:14.749794+0000","last_became_active":"2026-02-25T11:47:13.628037+0000","last_became_peered":"2026-02-25T11:47:13.628037+0000","last_unstale":"2026-02-25T11:47:14.749794+0000","last_undegraded":"2026-02-25T11:47:14.749794+0000","last_fullsized":"2026-02-25T11:47:14.749794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:28:22.976133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750830+0000","last_change":"2026-02-25T11:47:13.634652+0000","last_active":"2026-02-25T11:47:14.750830+0000","last_peered":"2026-02-25T11:47:14.750830+0000","last_clean":"2026-02-25T11:47:14.750830+0000","last_became_active":"2026-02-25T11:47:13.634433+0000","last_became_peered":"2026-02-25T11:47:13.634433+0000","last_unstale":"2026-02-25T11:47:14.750830+0000","last_undegraded":"2026-02-25T11:47:14.750830+0000","last_fullsized":"2026-02-25T11:47:14.750830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:01:29.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635319+0000","last_change":"2026-02-25T11:47:13.632721+0000","last_active":"2026-02-25T11:47:14.635319+0000","last_peered":"2026-02-25T11:47:14.635319+0000","last_clean":"2026-02-25T11:47:14.635319+0000","last_became_active":"2026-02-25T11:47:13.632626+0000","last_became_peered":"2026-02-25T11:47:13.632626+0000","last_unstale":"2026-02-25T11:47:14.635319+0000","last_undegraded":"2026-02-25T11:47:14.635319+0000","last_fullsized":"2026-02-25T11:47:14.635319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:28:35.201966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749844+0000","last_change":"2026-02-25T11:47:13.637504+0000","last_active":"2026-02-25T11:47:14.749844+0000","last_peered":"2026-02-25T11:47:14.749844+0000","last_clean":"2026-02-25T11:47:14.749844+0000","last_became_active":"2026-02-25T11:47:13.636824+0000","last_became_peered":"2026-02-25T11:47:13.636824+0000","last_unstale":"2026-02-25T11:47:14.749844+0000","last_undegraded":"2026-02-25T11:47:14.749844+0000","last_fullsized":"2026-02-25T11:47:14.749844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:20:31.416302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635724+0000","last_change":"2026-02-25T11:47:13.633135+0000","last_active":"2026-02-25T11:47:14.635724+0000","last_peered":"2026-02-25T11:47:14.635724+0000","last_clean":"2026-02-25T11:47:14.635724+0000","last_became_active":"2026-02-25T11:47:13.632966+0000","last_became_peered":"2026-02-25T11:47:13.632966+0000","last_unstale":"2026-02-25T11:47:14.635724+0000","last_undegraded":"2026-02-25T11:47:14.635724+0000","last_fullsized":"2026-02-25T11:47:14.635724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:26:17.052924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628538+0000","last_change":"2026-02-25T11:47:13.633210+0000","last_active":"2026-02-25T11:47:14.628538+0000","last_peered":"2026-02-25T11:47:14.628538+0000","last_clean":"2026-02-25T11:47:14.628538+0000","last_became_active":"2026-02-25T11:47:13.633099+0000","last_became_peered":"2026-02-25T11:47:13.633099+0000","last_unstale":"2026-02-25T11:47:14.628538+0000","last_undegraded":"2026-02-25T11:47:14.628538+0000","last_fullsized":"2026-02-25T11:47:14.628538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:07:46.884629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628656+0000","last_change":"2026-02-25T11:47:13.634757+0000","last_active":"2026-02-25T11:47:14.628656+0000","last_peered":"2026-02-25T11:47:14.628656+0000","last_clean":"2026-02-25T11:47:14.628656+0000","last_became_active":"2026-02-25T11:47:13.634643+0000","last_became_peered":"2026-02-25T11:47:13.634643+0000","last_unstale":"2026-02-25T11:47:14.628656+0000","last_undegraded":"2026-02-25T11:47:14.628656+0000","last_fullsized":"2026-02-25T11:47:14.628656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:20:23.383359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749674+0000","last_change":"2026-02-25T11:47:13.630515+0000","last_active":"2026-02-25T11:47:14.749674+0000","last_peered":"2026-02-25T11:47:14.749674+0000","last_clean":"2026-02-25T11:47:14.749674+0000","last_became_active":"2026-02-25T11:47:13.630202+0000","last_became_peered":"2026-02-25T11:47:13.630202+0000","last_unstale":"2026-02-25T11:47:14.749674+0000","last_undegraded":"2026-02-25T11:47:14.749674+0000","last_fullsized":"2026-02-25T11:47:14.749674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:51:24.646614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627367+0000","last_change":"2026-02-25T11:47:13.635749+0000","last_active":"2026-02-25T11:47:14.627367+0000","last_peered":"2026-02-25T11:47:14.627367+0000","last_clean":"2026-02-25T11:47:14.627367+0000","last_became_active":"2026-02-25T11:47:13.635284+0000","last_became_peered":"2026-02-25T11:47:13.635284+0000","last_unstale":"2026-02-25T11:47:14.627367+0000","last_undegraded":"2026-02-25T11:47:14.627367+0000","last_fullsized":"2026-02-25T11:47:14.627367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:23:45.042142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628732+0000","last_change":"2026-02-25T11:47:13.635448+0000","last_active":"2026-02-25T11:47:14.628732+0000","last_peered":"2026-02-25T11:47:14.628732+0000","last_clean":"2026-02-25T11:47:14.628732+0000","last_became_active":"2026-02-25T11:47:13.635232+0000","last_became_peered":"2026-02-25T11:47:13.635232+0000","last_unstale":"2026-02-25T11:47:14.628732+0000","last_undegraded":"2026-02-25T11:47:14.628732+0000","last_fullsized":"2026-02-25T11:47:14.628732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:33:38.020605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628332+0000","last_change":"2026-02-25T11:47:13.634237+0000","last_active":"2026-02-25T11:47:14.628332+0000","last_peered":"2026-02-25T11:47:14.628332+0000","last_clean":"2026-02-25T11:47:14.628332+0000","last_became_active":"2026-02-25T11:47:13.634054+0000","last_became_peered":"2026-02-25T11:47:13.634054+0000","last_unstale":"2026-02-25T11:47:14.628332+0000","last_undegraded":"2026-02-25T11:47:14.628332+0000","last_fullsized":"2026-02-25T11:47:14.628332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:38:34.603077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627786+0000","last_change":"2026-02-25T11:47:13.638347+0000","last_active":"2026-02-25T11:47:14.627786+0000","last_peered":"2026-02-25T11:47:14.627786+0000","last_clean":"2026-02-25T11:47:14.627786+0000","last_became_active":"2026-02-25T11:47:13.637984+0000","last_became_peered":"2026-02-25T11:47:13.637984+0000","last_unstale":"2026-02-25T11:47:14.627786+0000","last_undegraded":"2026-02-25T11:47:14.627786+0000","last_fullsized":"2026-02-25T11:47:14.627786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:36:11.273078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628202+0000","last_change":"2026-02-25T11:47:13.634523+0000","last_active":"2026-02-25T11:47:14.628202+0000","last_peered":"2026-02-25T11:47:14.628202+0000","last_clean":"2026-02-25T11:47:14.628202+0000","last_became_active":"2026-02-25T11:47:13.634184+0000","last_became_peered":"2026-02-25T11:47:13.634184+0000","last_unstale":"2026-02-25T11:47:14.628202+0000","last_undegraded":"2026-02-25T11:47:14.628202+0000","last_fullsized":"2026-02-25T11:47:14.628202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:52:23.917090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:27.584183+0000","last_change":"2026-02-25T11:47:12.613380+0000","last_active":"2026-02-25T11:47:27.584183+0000","last_peered":"2026-02-25T11:47:27.584183+0000","last_clean":"2026-02-25T11:47:27.584183+0000","last_became_active":"2026-02-25T11:47:12.613224+0000","last_became_peered":"2026-02-25T11:47:12.613224+0000","last_unstale":"2026-02-25T11:47:27.584183+0000","last_undegraded":"2026-02-25T11:47:27.584183+0000","last_fullsized":"2026-02-25T11:47:27.584183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:59:39.894067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749915+0000","last_change":"2026-02-25T11:47:13.636227+0000","last_active":"2026-02-25T11:47:14.749915+0000","last_peered":"2026-02-25T11:47:14.749915+0000","last_clean":"2026-02-25T11:47:14.749915+0000","last_became_active":"2026-02-25T11:47:13.635212+0000","last_became_peered":"2026-02-25T11:47:13.635212+0000","last_unstale":"2026-02-25T11:47:14.749915+0000","last_undegraded":"2026-02-25T11:47:14.749915+0000","last_fullsized":"2026-02-25T11:47:14.749915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:51:07.635399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657308+0000","last_change":"2026-02-25T11:47:12.613789+0000","last_active":"2026-02-25T11:47:17.657308+0000","last_peered":"2026-02-25T11:47:17.657308+0000","last_clean":"2026-02-25T11:47:17.657308+0000","last_became_active":"2026-02-25T11:47:12.613635+0000","last_became_peered":"2026-02-25T11:47:12.613635+0000","last_unstale":"2026-02-25T11:47:17.657308+0000","last_undegraded":"2026-02-25T11:47:17.657308+0000","last_fullsized":"2026-02-25T11:47:17.657308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:13:32.216593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657439+0000","last_change":"2026-02-25T11:47:12.612012+0000","last_active":"2026-02-25T11:47:17.657439+0000","last_peered":"2026-02-25T11:47:17.657439+0000","last_clean":"2026-02-25T11:47:17.657439+0000","last_became_active":"2026-02-25T11:47:12.611862+0000","last_became_peered":"2026-02-25T11:47:12.611862+0000","last_unstale":"2026-02-25T11:47:17.657439+0000","last_undegraded":"2026-02-25T11:47:17.657439+0000","last_fullsized":"2026-02-25T11:47:17.657439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:08:17.490884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635279+0000","last_change":"2026-02-25T11:47:13.627129+0000","last_active":"2026-02-25T11:47:14.635279+0000","last_peered":"2026-02-25T11:47:14.635279+0000","last_clean":"2026-02-25T11:47:14.635279+0000","last_became_active":"2026-02-25T11:47:13.626987+0000","last_became_peered":"2026-02-25T11:47:13.626987+0000","last_unstale":"2026-02-25T11:47:14.635279+0000","last_undegraded":"2026-02-25T11:47:14.635279+0000","last_fullsized":"2026-02-25T11:47:14.635279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:53:45.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749940+0000","last_change":"2026-02-25T11:47:12.611992+0000","last_active":"2026-02-25T11:47:14.749940+0000","last_peered":"2026-02-25T11:47:14.749940+0000","last_clean":"2026-02-25T11:47:14.749940+0000","last_became_active":"2026-02-25T11:47:12.611502+0000","last_became_peered":"2026-02-25T11:47:12.611502+0000","last_unstale":"2026-02-25T11:47:14.749940+0000","last_undegraded":"2026-02-25T11:47:14.749940+0000","last_fullsized":"2026-02-25T11:47:14.749940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:15:45.718628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633835+0000","last_change":"2026-02-25T11:47:13.635344+0000","last_active":"2026-02-25T11:47:14.633835+0000","last_peered":"2026-02-25T11:47:14.633835+0000","last_clean":"2026-02-25T11:47:14.633835+0000","last_became_active":"2026-02-25T11:47:13.634327+0000","last_became_peered":"2026-02-25T11:47:13.634327+0000","last_unstale":"2026-02-25T11:47:14.633835+0000","last_undegraded":"2026-02-25T11:47:14.633835+0000","last_fullsized":"2026-02-25T11:47:14.633835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:30:06.778398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:26.582979+0000","last_change":"2026-02-25T11:47:12.611130+0000","last_active":"2026-02-25T11:47:26.582979+0000","last_peered":"2026-02-25T11:47:26.582979+0000","last_clean":"2026-02-25T11:47:26.582979+0000","last_became_active":"2026-02-25T11:47:12.610838+0000","last_became_peered":"2026-02-25T11:47:12.610838+0000","last_unstale":"2026-02-25T11:47:26.582979+0000","last_undegraded":"2026-02-25T11:47:26.582979+0000","last_fullsized":"2026-02-25T11:47:26.582979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:34:19.366424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751244+0000","last_change":"2026-02-25T11:47:13.629263+0000","last_active":"2026-02-25T11:47:14.751244+0000","last_peered":"2026-02-25T11:47:14.751244+0000","last_clean":"2026-02-25T11:47:14.751244+0000","last_became_active":"2026-02-25T11:47:13.628765+0000","last_became_peered":"2026-02-25T11:47:13.628765+0000","last_unstale":"2026-02-25T11:47:14.751244+0000","last_undegraded":"2026-02-25T11:47:14.751244+0000","last_fullsized":"2026-02-25T11:47:14.751244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:49:04.773052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.649872+0000","last_change":"2026-02-25T11:47:12.613455+0000","last_active":"2026-02-25T11:47:16.649872+0000","last_peered":"2026-02-25T11:47:16.649872+0000","last_clean":"2026-02-25T11:47:16.649872+0000","last_became_active":"2026-02-25T11:47:12.612904+0000","last_became_peered":"2026-02-25T11:47:12.612904+0000","last_unstale":"2026-02-25T11:47:16.649872+0000","last_undegraded":"2026-02-25T11:47:16.649872+0000","last_fullsized":"2026-02-25T11:47:16.649872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:16:13.149517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627673+0000","last_change":"2026-02-25T11:47:13.635178+0000","last_active":"2026-02-25T11:47:14.627673+0000","last_peered":"2026-02-25T11:47:14.627673+0000","last_clean":"2026-02-25T11:47:14.627673+0000","last_became_active":"2026-02-25T11:47:13.634985+0000","last_became_peered":"2026-02-25T11:47:13.634985+0000","last_unstale":"2026-02-25T11:47:14.627673+0000","last_undegraded":"2026-02-25T11:47:14.627673+0000","last_fullsized":"2026-02-25T11:47:14.627673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:08.068457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653282+0000","last_change":"2026-02-25T11:47:12.610853+0000","last_active":"2026-02-25T11:47:16.653282+0000","last_peered":"2026-02-25T11:47:16.653282+0000","last_clean":"2026-02-25T11:47:16.653282+0000","last_became_active":"2026-02-25T11:47:12.610639+0000","last_became_peered":"2026-02-25T11:47:12.610639+0000","last_unstale":"2026-02-25T11:47:16.653282+0000","last_undegraded":"2026-02-25T11:47:16.653282+0000","last_fullsized":"2026-02-25T11:47:16.653282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:00:06.042790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":694,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750953+0000","last_change":"2026-02-25T11:47:13.630063+0000","last_active":"2026-02-25T11:47:14.750953+0000","last_peered":"2026-02-25T11:47:14.750953+0000","last_clean":"2026-02-25T11:47:14.750953+0000","last_became_active":"2026-02-25T11:47:13.629906+0000","last_became_peered":"2026-02-25T11:47:13.629906+0000","last_unstale":"2026-02-25T11:47:14.750953+0000","last_undegraded":"2026-02-25T11:47:14.750953+0000","last_fullsized":"2026-02-25T11:47:14.750953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:09:13.614948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'26","reported_seq":69,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:27.586413+0000","last_change":"2026-02-25T11:47:12.615044+0000","last_active":"2026-02-25T11:47:27.586413+0000","last_peered":"2026-02-25T11:47:27.586413+0000","last_clean":"2026-02-25T11:47:27.586413+0000","last_became_active":"2026-02-25T11:47:12.614698+0000","last_became_peered":"2026-02-25T11:47:12.614698+0000","last_unstale":"2026-02-25T11:47:27.586413+0000","last_undegraded":"2026-02-25T11:47:27.586413+0000","last_fullsized":"2026-02-25T11:47:27.586413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:52:32.148801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62088,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":27,"num_read_kb":73,"num_write":27,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628510+0000","last_change":"2026-02-25T11:47:13.630945+0000","last_active":"2026-02-25T11:47:14.628510+0000","last_peered":"2026-02-25T11:47:14.628510+0000","last_clean":"2026-02-25T11:47:14.628510+0000","last_became_active":"2026-02-25T11:47:13.630693+0000","last_became_peered":"2026-02-25T11:47:13.630693+0000","last_unstale":"2026-02-25T11:47:14.628510+0000","last_undegraded":"2026-02-25T11:47:14.628510+0000","last_fullsized":"2026-02-25T11:47:14.628510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:39:52.744366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656758+0000","last_change":"2026-02-25T11:47:12.614950+0000","last_active":"2026-02-25T11:47:17.656758+0000","last_peered":"2026-02-25T11:47:17.656758+0000","last_clean":"2026-02-25T11:47:17.656758+0000","last_became_active":"2026-02-25T11:47:12.614272+0000","last_became_peered":"2026-02-25T11:47:12.614272+0000","last_unstale":"2026-02-25T11:47:17.656758+0000","last_undegraded":"2026-02-25T11:47:17.656758+0000","last_fullsized":"2026-02-25T11:47:17.656758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:45:50.928166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749596+0000","last_change":"2026-02-25T11:47:13.633128+0000","last_active":"2026-02-25T11:47:14.749596+0000","last_peered":"2026-02-25T11:47:14.749596+0000","last_clean":"2026-02-25T11:47:14.749596+0000","last_became_active":"2026-02-25T11:47:13.633047+0000","last_became_peered":"2026-02-25T11:47:13.633047+0000","last_unstale":"2026-02-25T11:47:14.749596+0000","last_undegraded":"2026-02-25T11:47:14.749596+0000","last_fullsized":"2026-02-25T11:47:14.749596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:56:33.191193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653783+0000","last_change":"2026-02-25T11:47:12.616898+0000","last_active":"2026-02-25T11:47:16.653783+0000","last_peered":"2026-02-25T11:47:16.653783+0000","last_clean":"2026-02-25T11:47:16.653783+0000","last_became_active":"2026-02-25T11:47:12.616774+0000","last_became_peered":"2026-02-25T11:47:12.616774+0000","last_unstale":"2026-02-25T11:47:16.653783+0000","last_undegraded":"2026-02-25T11:47:16.653783+0000","last_fullsized":"2026-02-25T11:47:16.653783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:59:33.930544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628464+0000","last_change":"2026-02-25T11:47:13.630377+0000","last_active":"2026-02-25T11:47:14.628464+0000","last_peered":"2026-02-25T11:47:14.628464+0000","last_clean":"2026-02-25T11:47:14.628464+0000","last_became_active":"2026-02-25T11:47:13.630124+0000","last_became_peered":"2026-02-25T11:47:13.630124+0000","last_unstale":"2026-02-25T11:47:14.628464+0000","last_undegraded":"2026-02-25T11:47:14.628464+0000","last_fullsized":"2026-02-25T11:47:14.628464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:42:12.028258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:28.588594+0000","last_change":"2026-02-25T11:47:12.613208+0000","last_active":"2026-02-25T11:47:28.588594+0000","last_peered":"2026-02-25T11:47:28.588594+0000","last_clean":"2026-02-25T11:47:28.588594+0000","last_became_active":"2026-02-25T11:47:12.612460+0000","last_became_peered":"2026-02-25T11:47:12.612460+0000","last_unstale":"2026-02-25T11:47:28.588594+0000","last_undegraded":"2026-02-25T11:47:28.588594+0000","last_fullsized":"2026-02-25T11:47:28.588594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:15:59.810632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":998,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628121+0000","last_change":"2026-02-25T11:47:13.631123+0000","last_active":"2026-02-25T11:47:14.628121+0000","last_peered":"2026-02-25T11:47:14.628121+0000","last_clean":"2026-02-25T11:47:14.628121+0000","last_became_active":"2026-02-25T11:47:13.631031+0000","last_became_peered":"2026-02-25T11:47:13.631031+0000","last_unstale":"2026-02-25T11:47:14.628121+0000","last_undegraded":"2026-02-25T11:47:14.628121+0000","last_fullsized":"2026-02-25T11:47:14.628121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:47:54.073226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657937+0000","last_change":"2026-02-25T11:47:12.612153+0000","last_active":"2026-02-25T11:47:17.657937+0000","last_peered":"2026-02-25T11:47:17.657937+0000","last_clean":"2026-02-25T11:47:17.657937+0000","last_became_active":"2026-02-25T11:47:12.611107+0000","last_became_peered":"2026-02-25T11:47:12.611107+0000","last_unstale":"2026-02-25T11:47:17.657937+0000","last_undegraded":"2026-02-25T11:47:17.657937+0000","last_fullsized":"2026-02-25T11:47:17.657937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:37:21.791728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749858+0000","last_change":"2026-02-25T11:47:13.635800+0000","last_active":"2026-02-25T11:47:14.749858+0000","last_peered":"2026-02-25T11:47:14.749858+0000","last_clean":"2026-02-25T11:47:14.749858+0000","last_became_active":"2026-02-25T11:47:13.635590+0000","last_became_peered":"2026-02-25T11:47:13.635590+0000","last_unstale":"2026-02-25T11:47:14.749858+0000","last_undegraded":"2026-02-25T11:47:14.749858+0000","last_fullsized":"2026-02-25T11:47:14.749858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:46:36.324518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575342+0000","last_change":"2026-02-25T11:47:12.616473+0000","last_active":"2026-02-25T11:47:18.575342+0000","last_peered":"2026-02-25T11:47:18.575342+0000","last_clean":"2026-02-25T11:47:18.575342+0000","last_became_active":"2026-02-25T11:47:12.616039+0000","last_became_peered":"2026-02-25T11:47:12.616039+0000","last_unstale":"2026-02-25T11:47:18.575342+0000","last_undegraded":"2026-02-25T11:47:18.575342+0000","last_fullsized":"2026-02-25T11:47:18.575342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:28.324889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628413+0000","last_change":"2026-02-25T11:47:13.634771+0000","last_active":"2026-02-25T11:47:14.628413+0000","last_peered":"2026-02-25T11:47:14.628413+0000","last_clean":"2026-02-25T11:47:14.628413+0000","last_became_active":"2026-02-25T11:47:13.634672+0000","last_became_peered":"2026-02-25T11:47:13.634672+0000","last_unstale":"2026-02-25T11:47:14.628413+0000","last_undegraded":"2026-02-25T11:47:14.628413+0000","last_fullsized":"2026-02-25T11:47:14.628413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:19:03.692423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749863+0000","last_change":"2026-02-25T11:47:12.616685+0000","last_active":"2026-02-25T11:47:14.749863+0000","last_peered":"2026-02-25T11:47:14.749863+0000","last_clean":"2026-02-25T11:47:14.749863+0000","last_became_active":"2026-02-25T11:47:12.616518+0000","last_became_peered":"2026-02-25T11:47:12.616518+0000","last_unstale":"2026-02-25T11:47:14.749863+0000","last_undegraded":"2026-02-25T11:47:14.749863+0000","last_fullsized":"2026-02-25T11:47:14.749863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:28:50.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628462+0000","last_change":"2026-02-25T11:47:13.627734+0000","last_active":"2026-02-25T11:47:14.628462+0000","last_peered":"2026-02-25T11:47:14.628462+0000","last_clean":"2026-02-25T11:47:14.628462+0000","last_became_active":"2026-02-25T11:47:13.627632+0000","last_became_peered":"2026-02-25T11:47:13.627632+0000","last_unstale":"2026-02-25T11:47:14.628462+0000","last_undegraded":"2026-02-25T11:47:14.628462+0000","last_fullsized":"2026-02-25T11:47:14.628462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:01:20.907699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750104+0000","last_change":"2026-02-25T11:47:12.612830+0000","last_active":"2026-02-25T11:47:14.750104+0000","last_peered":"2026-02-25T11:47:14.750104+0000","last_clean":"2026-02-25T11:47:14.750104+0000","last_became_active":"2026-02-25T11:47:12.612661+0000","last_became_peered":"2026-02-25T11:47:12.612661+0000","last_unstale":"2026-02-25T11:47:14.750104+0000","last_undegraded":"2026-02-25T11:47:14.750104+0000","last_fullsized":"2026-02-25T11:47:14.750104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:05:15.367058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:15.914932+0000","last_change":"2026-02-25T11:47:13.633998+0000","last_active":"2026-02-25T11:47:15.914932+0000","last_peered":"2026-02-25T11:47:15.914932+0000","last_clean":"2026-02-25T11:47:15.914932+0000","last_became_active":"2026-02-25T11:47:13.633820+0000","last_became_peered":"2026-02-25T11:47:13.633820+0000","last_unstale":"2026-02-25T11:47:15.914932+0000","last_undegraded":"2026-02-25T11:47:15.914932+0000","last_fullsized":"2026-02-25T11:47:15.914932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:02:38.693106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.658417+0000","last_change":"2026-02-25T11:47:12.614091+0000","last_active":"2026-02-25T11:47:17.658417+0000","last_peered":"2026-02-25T11:47:17.658417+0000","last_clean":"2026-02-25T11:47:17.658417+0000","last_became_active":"2026-02-25T11:47:12.612962+0000","last_became_peered":"2026-02-25T11:47:12.612962+0000","last_unstale":"2026-02-25T11:47:17.658417+0000","last_undegraded":"2026-02-25T11:47:17.658417+0000","last_fullsized":"2026-02-25T11:47:17.658417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:11:40.214146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628504+0000","last_change":"2026-02-25T11:47:13.627288+0000","last_active":"2026-02-25T11:47:14.628504+0000","last_peered":"2026-02-25T11:47:14.628504+0000","last_clean":"2026-02-25T11:47:14.628504+0000","last_became_active":"2026-02-25T11:47:13.626902+0000","last_became_peered":"2026-02-25T11:47:13.626902+0000","last_unstale":"2026-02-25T11:47:14.628504+0000","last_undegraded":"2026-02-25T11:47:14.628504+0000","last_fullsized":"2026-02-25T11:47:14.628504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:38:19.137242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'152","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628095+0000","last_change":"2026-02-25T11:46:35.865282+0000","last_active":"2026-02-25T11:47:14.628095+0000","last_peered":"2026-02-25T11:47:14.628095+0000","last_clean":"2026-02-25T11:47:14.628095+0000","last_became_active":"2026-02-25T11:46:35.865139+0000","last_became_peered":"2026-02-25T11:46:35.865139+0000","last_unstale":"2026-02-25T11:47:14.628095+0000","last_undegraded":"2026-02-25T11:47:14.628095+0000","last_fullsized":"2026-02-25T11:47:14.628095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_clean_scrub_stamp":"2026-02-25T11:44:43.739673+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:29:57.573039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575170+0000","last_change":"2026-02-25T11:47:12.612834+0000","last_active":"2026-02-25T11:47:18.575170+0000","last_peered":"2026-02-25T11:47:18.575170+0000","last_clean":"2026-02-25T11:47:18.575170+0000","last_became_active":"2026-02-25T11:47:12.611827+0000","last_became_peered":"2026-02-25T11:47:12.611827+0000","last_unstale":"2026-02-25T11:47:18.575170+0000","last_undegraded":"2026-02-25T11:47:18.575170+0000","last_fullsized":"2026-02-25T11:47:18.575170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:05:32.112908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635504+0000","last_change":"2026-02-25T11:47:13.634534+0000","last_active":"2026-02-25T11:47:14.635504+0000","last_peered":"2026-02-25T11:47:14.635504+0000","last_clean":"2026-02-25T11:47:14.635504+0000","last_became_active":"2026-02-25T11:47:13.633769+0000","last_became_peered":"2026-02-25T11:47:13.633769+0000","last_unstale":"2026-02-25T11:47:14.635504+0000","last_undegraded":"2026-02-25T11:47:14.635504+0000","last_fullsized":"2026-02-25T11:47:14.635504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:16:09.255508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656541+0000","last_change":"2026-02-25T11:47:12.612621+0000","last_active":"2026-02-25T11:47:17.656541+0000","last_peered":"2026-02-25T11:47:17.656541+0000","last_clean":"2026-02-25T11:47:17.656541+0000","last_became_active":"2026-02-25T11:47:12.612284+0000","last_became_peered":"2026-02-25T11:47:12.612284+0000","last_unstale":"2026-02-25T11:47:17.656541+0000","last_undegraded":"2026-02-25T11:47:17.656541+0000","last_fullsized":"2026-02-25T11:47:17.656541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:49:09.299914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749514+0000","last_change":"2026-02-25T11:47:13.629107+0000","last_active":"2026-02-25T11:47:14.749514+0000","last_peered":"2026-02-25T11:47:14.749514+0000","last_clean":"2026-02-25T11:47:14.749514+0000","last_became_active":"2026-02-25T11:47:13.629031+0000","last_became_peered":"2026-02-25T11:47:13.629031+0000","last_unstale":"2026-02-25T11:47:14.749514+0000","last_undegraded":"2026-02-25T11:47:14.749514+0000","last_fullsized":"2026-02-25T11:47:14.749514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:07:58.915416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:26.585836+0000","last_change":"2026-02-25T11:47:12.616543+0000","last_active":"2026-02-25T11:47:26.585836+0000","last_peered":"2026-02-25T11:47:26.585836+0000","last_clean":"2026-02-25T11:47:26.585836+0000","last_became_active":"2026-02-25T11:47:12.616345+0000","last_became_peered":"2026-02-25T11:47:12.616345+0000","last_unstale":"2026-02-25T11:47:26.585836+0000","last_undegraded":"2026-02-25T11:47:26.585836+0000","last_fullsized":"2026-02-25T11:47:26.585836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:27:30.862168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750014+0000","last_change":"2026-02-25T11:47:13.637385+0000","last_active":"2026-02-25T11:47:14.750014+0000","last_peered":"2026-02-25T11:47:14.750014+0000","last_clean":"2026-02-25T11:47:14.750014+0000","last_became_active":"2026-02-25T11:47:13.634653+0000","last_became_peered":"2026-02-25T11:47:13.634653+0000","last_unstale":"2026-02-25T11:47:14.750014+0000","last_undegraded":"2026-02-25T11:47:14.750014+0000","last_fullsized":"2026-02-25T11:47:14.750014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:05:08.662208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:15.914921+0000","last_change":"2026-02-25T11:47:12.614640+0000","last_active":"2026-02-25T11:47:15.914921+0000","last_peered":"2026-02-25T11:47:15.914921+0000","last_clean":"2026-02-25T11:47:15.914921+0000","last_became_active":"2026-02-25T11:47:12.614486+0000","last_became_peered":"2026-02-25T11:47:12.614486+0000","last_unstale":"2026-02-25T11:47:15.914921+0000","last_undegraded":"2026-02-25T11:47:15.914921+0000","last_fullsized":"2026-02-25T11:47:15.914921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:16:06.314709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750327+0000","last_change":"2026-02-25T11:47:13.631828+0000","last_active":"2026-02-25T11:47:14.750327+0000","last_peered":"2026-02-25T11:47:14.750327+0000","last_clean":"2026-02-25T11:47:14.750327+0000","last_became_active":"2026-02-25T11:47:13.631590+0000","last_became_peered":"2026-02-25T11:47:13.631590+0000","last_unstale":"2026-02-25T11:47:14.750327+0000","last_undegraded":"2026-02-25T11:47:14.750327+0000","last_fullsized":"2026-02-25T11:47:14.750327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:25.518794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633984+0000","last_change":"2026-02-25T11:47:12.614032+0000","last_active":"2026-02-25T11:47:14.633984+0000","last_peered":"2026-02-25T11:47:14.633984+0000","last_clean":"2026-02-25T11:47:14.633984+0000","last_became_active":"2026-02-25T11:47:12.613936+0000","last_became_peered":"2026-02-25T11:47:12.613936+0000","last_unstale":"2026-02-25T11:47:14.633984+0000","last_undegraded":"2026-02-25T11:47:14.633984+0000","last_fullsized":"2026-02-25T11:47:14.633984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:16:46.397485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.859010+0000","last_change":"2026-02-25T11:47:13.636173+0000","last_active":"2026-02-25T11:47:16.859010+0000","last_peered":"2026-02-25T11:47:16.859010+0000","last_clean":"2026-02-25T11:47:16.859010+0000","last_became_active":"2026-02-25T11:47:13.635745+0000","last_became_peered":"2026-02-25T11:47:13.635745+0000","last_unstale":"2026-02-25T11:47:16.859010+0000","last_undegraded":"2026-02-25T11:47:16.859010+0000","last_fullsized":"2026-02-25T11:47:16.859010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:03:33.671318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651871+0000","last_change":"2026-02-25T11:47:12.614003+0000","last_active":"2026-02-25T11:47:16.651871+0000","last_peered":"2026-02-25T11:47:16.651871+0000","last_clean":"2026-02-25T11:47:16.651871+0000","last_became_active":"2026-02-25T11:47:12.613893+0000","last_became_peered":"2026-02-25T11:47:12.613893+0000","last_unstale":"2026-02-25T11:47:16.651871+0000","last_undegraded":"2026-02-25T11:47:16.651871+0000","last_fullsized":"2026-02-25T11:47:16.651871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:33:21.133364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749746+0000","last_change":"2026-02-25T11:47:13.632632+0000","last_active":"2026-02-25T11:47:14.749746+0000","last_peered":"2026-02-25T11:47:14.749746+0000","last_clean":"2026-02-25T11:47:14.749746+0000","last_became_active":"2026-02-25T11:47:13.632408+0000","last_became_peered":"2026-02-25T11:47:13.632408+0000","last_unstale":"2026-02-25T11:47:14.749746+0000","last_undegraded":"2026-02-25T11:47:14.749746+0000","last_fullsized":"2026-02-25T11:47:14.749746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:38:45.448637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:26.638117+0000","last_change":"2026-02-25T11:47:12.613816+0000","last_active":"2026-02-25T11:47:26.638117+0000","last_peered":"2026-02-25T11:47:26.638117+0000","last_clean":"2026-02-25T11:47:26.638117+0000","last_became_active":"2026-02-25T11:47:12.613196+0000","last_became_peered":"2026-02-25T11:47:12.613196+0000","last_unstale":"2026-02-25T11:47:26.638117+0000","last_undegraded":"2026-02-25T11:47:26.638117+0000","last_fullsized":"2026-02-25T11:47:26.638117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:23:52.372808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635752+0000","last_change":"2026-02-25T11:47:13.632431+0000","last_active":"2026-02-25T11:47:14.635752+0000","last_peered":"2026-02-25T11:47:14.635752+0000","last_clean":"2026-02-25T11:47:14.635752+0000","last_became_active":"2026-02-25T11:47:13.631972+0000","last_became_peered":"2026-02-25T11:47:13.631972+0000","last_unstale":"2026-02-25T11:47:14.635752+0000","last_undegraded":"2026-02-25T11:47:14.635752+0000","last_fullsized":"2026-02-25T11:47:14.635752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:59:34.655028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651131+0000","last_change":"2026-02-25T11:47:12.612805+0000","last_active":"2026-02-25T11:47:16.651131+0000","last_peered":"2026-02-25T11:47:16.651131+0000","last_clean":"2026-02-25T11:47:16.651131+0000","last_became_active":"2026-02-25T11:47:12.612155+0000","last_became_peered":"2026-02-25T11:47:12.612155+0000","last_unstale":"2026-02-25T11:47:16.651131+0000","last_undegraded":"2026-02-25T11:47:16.651131+0000","last_fullsized":"2026-02-25T11:47:16.651131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:58:00.429639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627853+0000","last_change":"2026-02-25T11:47:13.636089+0000","last_active":"2026-02-25T11:47:14.627853+0000","last_peered":"2026-02-25T11:47:14.627853+0000","last_clean":"2026-02-25T11:47:14.627853+0000","last_became_active":"2026-02-25T11:47:13.635963+0000","last_became_peered":"2026-02-25T11:47:13.635963+0000","last_unstale":"2026-02-25T11:47:14.627853+0000","last_undegraded":"2026-02-25T11:47:14.627853+0000","last_fullsized":"2026-02-25T11:47:14.627853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:50:28.012023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650081+0000","last_change":"2026-02-25T11:47:12.609749+0000","last_active":"2026-02-25T11:47:16.650081+0000","last_peered":"2026-02-25T11:47:16.650081+0000","last_clean":"2026-02-25T11:47:16.650081+0000","last_became_active":"2026-02-25T11:47:12.609190+0000","last_became_peered":"2026-02-25T11:47:12.609190+0000","last_unstale":"2026-02-25T11:47:16.650081+0000","last_undegraded":"2026-02-25T11:47:16.650081+0000","last_fullsized":"2026-02-25T11:47:16.650081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:12:42.204122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628201+0000","last_change":"2026-02-25T11:47:13.629339+0000","last_active":"2026-02-25T11:47:14.628201+0000","last_peered":"2026-02-25T11:47:14.628201+0000","last_clean":"2026-02-25T11:47:14.628201+0000","last_became_active":"2026-02-25T11:47:13.629107+0000","last_became_peered":"2026-02-25T11:47:13.629107+0000","last_unstale":"2026-02-25T11:47:14.628201+0000","last_undegraded":"2026-02-25T11:47:14.628201+0000","last_fullsized":"2026-02-25T11:47:14.628201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:19:50.087771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.652673+0000","last_change":"2026-02-25T11:47:12.615249+0000","last_active":"2026-02-25T11:47:16.652673+0000","last_peered":"2026-02-25T11:47:16.652673+0000","last_clean":"2026-02-25T11:47:16.652673+0000","last_became_active":"2026-02-25T11:47:12.615165+0000","last_became_peered":"2026-02-25T11:47:12.615165+0000","last_unstale":"2026-02-25T11:47:16.652673+0000","last_undegraded":"2026-02-25T11:47:16.652673+0000","last_fullsized":"2026-02-25T11:47:16.652673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:40:52.452074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627665+0000","last_change":"2026-02-25T11:47:13.631460+0000","last_active":"2026-02-25T11:47:14.627665+0000","last_peered":"2026-02-25T11:47:14.627665+0000","last_clean":"2026-02-25T11:47:14.627665+0000","last_became_active":"2026-02-25T11:47:13.631298+0000","last_became_peered":"2026-02-25T11:47:13.631298+0000","last_unstale":"2026-02-25T11:47:14.627665+0000","last_undegraded":"2026-02-25T11:47:14.627665+0000","last_fullsized":"2026-02-25T11:47:14.627665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:26:03.362106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628432+0000","last_change":"2026-02-25T11:47:12.615779+0000","last_active":"2026-02-25T11:47:14.628432+0000","last_peered":"2026-02-25T11:47:14.628432+0000","last_clean":"2026-02-25T11:47:14.628432+0000","last_became_active":"2026-02-25T11:47:12.615073+0000","last_became_peered":"2026-02-25T11:47:12.615073+0000","last_unstale":"2026-02-25T11:47:14.628432+0000","last_undegraded":"2026-02-25T11:47:14.628432+0000","last_fullsized":"2026-02-25T11:47:14.628432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:28:51.468199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635380+0000","last_change":"2026-02-25T11:47:13.632351+0000","last_active":"2026-02-25T11:47:14.635380+0000","last_peered":"2026-02-25T11:47:14.635380+0000","last_clean":"2026-02-25T11:47:14.635380+0000","last_became_active":"2026-02-25T11:47:13.632081+0000","last_became_peered":"2026-02-25T11:47:13.632081+0000","last_unstale":"2026-02-25T11:47:14.635380+0000","last_undegraded":"2026-02-25T11:47:14.635380+0000","last_fullsized":"2026-02-25T11:47:14.635380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:38:59.143253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650353+0000","last_change":"2026-02-25T11:47:12.616555+0000","last_active":"2026-02-25T11:47:16.650353+0000","last_peered":"2026-02-25T11:47:16.650353+0000","last_clean":"2026-02-25T11:47:16.650353+0000","last_became_active":"2026-02-25T11:47:12.616174+0000","last_became_peered":"2026-02-25T11:47:12.616174+0000","last_unstale":"2026-02-25T11:47:16.650353+0000","last_undegraded":"2026-02-25T11:47:16.650353+0000","last_fullsized":"2026-02-25T11:47:16.650353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:01:17.494129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628458+0000","last_change":"2026-02-25T11:47:13.627400+0000","last_active":"2026-02-25T11:47:14.628458+0000","last_peered":"2026-02-25T11:47:14.628458+0000","last_clean":"2026-02-25T11:47:14.628458+0000","last_became_active":"2026-02-25T11:47:13.626782+0000","last_became_peered":"2026-02-25T11:47:13.626782+0000","last_unstale":"2026-02-25T11:47:14.628458+0000","last_undegraded":"2026-02-25T11:47:14.628458+0000","last_fullsized":"2026-02-25T11:47:14.628458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:42:39.996935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750027+0000","last_change":"2026-02-25T11:47:13.634030+0000","last_active":"2026-02-25T11:47:14.750027+0000","last_peered":"2026-02-25T11:47:14.750027+0000","last_clean":"2026-02-25T11:47:14.750027+0000","last_became_active":"2026-02-25T11:47:13.632844+0000","last_became_peered":"2026-02-25T11:47:13.632844+0000","last_unstale":"2026-02-25T11:47:14.750027+0000","last_undegraded":"2026-02-25T11:47:14.750027+0000","last_fullsized":"2026-02-25T11:47:14.750027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:52:16.013596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628805+0000","last_change":"2026-02-25T11:47:12.613666+0000","last_active":"2026-02-25T11:47:14.628805+0000","last_peered":"2026-02-25T11:47:14.628805+0000","last_clean":"2026-02-25T11:47:14.628805+0000","last_became_active":"2026-02-25T11:47:12.613570+0000","last_became_peered":"2026-02-25T11:47:12.613570+0000","last_unstale":"2026-02-25T11:47:14.628805+0000","last_undegraded":"2026-02-25T11:47:14.628805+0000","last_fullsized":"2026-02-25T11:47:14.628805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:28:39.750737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750924+0000","last_change":"2026-02-25T11:47:13.631515+0000","last_active":"2026-02-25T11:47:14.750924+0000","last_peered":"2026-02-25T11:47:14.750924+0000","last_clean":"2026-02-25T11:47:14.750924+0000","last_became_active":"2026-02-25T11:47:13.631161+0000","last_became_peered":"2026-02-25T11:47:13.631161+0000","last_unstale":"2026-02-25T11:47:14.750924+0000","last_undegraded":"2026-02-25T11:47:14.750924+0000","last_fullsized":"2026-02-25T11:47:14.750924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:13:30.496521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656891+0000","last_change":"2026-02-25T11:47:12.615904+0000","last_active":"2026-02-25T11:47:17.656891+0000","last_peered":"2026-02-25T11:47:17.656891+0000","last_clean":"2026-02-25T11:47:17.656891+0000","last_became_active":"2026-02-25T11:47:12.615784+0000","last_became_peered":"2026-02-25T11:47:12.615784+0000","last_unstale":"2026-02-25T11:47:17.656891+0000","last_undegraded":"2026-02-25T11:47:17.656891+0000","last_fullsized":"2026-02-25T11:47:17.656891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:32:26.802870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627754+0000","last_change":"2026-02-25T11:47:13.630065+0000","last_active":"2026-02-25T11:47:14.627754+0000","last_peered":"2026-02-25T11:47:14.627754+0000","last_clean":"2026-02-25T11:47:14.627754+0000","last_became_active":"2026-02-25T11:47:13.629214+0000","last_became_peered":"2026-02-25T11:47:13.629214+0000","last_unstale":"2026-02-25T11:47:14.627754+0000","last_undegraded":"2026-02-25T11:47:14.627754+0000","last_fullsized":"2026-02-25T11:47:14.627754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:53:45.639526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.310632+0000","last_change":"2026-02-25T11:47:12.612956+0000","last_active":"2026-02-25T11:47:16.310632+0000","last_peered":"2026-02-25T11:47:16.310632+0000","last_clean":"2026-02-25T11:47:16.310632+0000","last_became_active":"2026-02-25T11:47:12.612843+0000","last_became_peered":"2026-02-25T11:47:12.612843+0000","last_unstale":"2026-02-25T11:47:16.310632+0000","last_undegraded":"2026-02-25T11:47:16.310632+0000","last_fullsized":"2026-02-25T11:47:16.310632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:41:30.867465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635780+0000","last_change":"2026-02-25T11:47:13.637712+0000","last_active":"2026-02-25T11:47:14.635780+0000","last_peered":"2026-02-25T11:47:14.635780+0000","last_clean":"2026-02-25T11:47:14.635780+0000","last_became_active":"2026-02-25T11:47:13.635030+0000","last_became_peered":"2026-02-25T11:47:13.635030+0000","last_unstale":"2026-02-25T11:47:14.635780+0000","last_undegraded":"2026-02-25T11:47:14.635780+0000","last_fullsized":"2026-02-25T11:47:14.635780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:01:22.896129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656736+0000","last_change":"2026-02-25T11:47:12.614149+0000","last_active":"2026-02-25T11:47:17.656736+0000","last_peered":"2026-02-25T11:47:17.656736+0000","last_clean":"2026-02-25T11:47:17.656736+0000","last_became_active":"2026-02-25T11:47:12.613137+0000","last_became_peered":"2026-02-25T11:47:12.613137+0000","last_unstale":"2026-02-25T11:47:17.656736+0000","last_undegraded":"2026-02-25T11:47:17.656736+0000","last_fullsized":"2026-02-25T11:47:17.656736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:49:49.451435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66174,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":89,"num_read_kb":112,"num_write":158,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":227293,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":262884,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350987,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28132,"kb_used_data":912,"kb_used_omap":9,"kb_used_meta":27190,"kb_avail":733975068,"statfs":{"total":751619276800,"available":751590469632,"internally_reserved":0,"allocated":933888,"data_stored":455512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10109,"internal_metadata":27842691},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612621,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28272,"kb_used_data":1052,"kb_used_omap":28,"kb_used_meta":27171,"kb_avail":733974928,"statfs":{"total":751619276800,"available":751590326272,"internally_reserved":0,"allocated":1077248,"data_stored":1101119,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":29123,"internal_metadata":27823677},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776144,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28148,"kb_used_data":928,"kb_used_omap":27,"kb_used_meta":27172,"kb_avail":733975052,"statfs":{"total":751619276800,"available":751590453248,"internally_reserved":0,"allocated":950272,"data_stored":455748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28025,"internal_metadata":27824775},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037779,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28148,"kb_used_data":928,"kb_used_omap":35,"kb_used_meta":27164,"kb_avail":733975052,"statfs":{"total":751619276800,"available":751590453248,"internally_reserved":0,"allocated":950272,"data_stored":455668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36257,"internal_metadata":27816543},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299414,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28196,"kb_used_data":976,"kb_used_omap":35,"kb_used_meta":27164,"kb_avail":733975004,"statfs":{"total":751619276800,"available":751590404096,"internally_reserved":0,"allocated":999424,"data_stored":1047617,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36168,"internal_metadata":27816632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462937,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28136,"kb_used_data":916,"kb_used_omap":31,"kb_used_meta":27168,"kb_avail":733975064,"statfs":{"total":751619276800,"available":751590465536,"internally_reserved":0,"allocated":937984,"data_stored":455115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32619,"internal_metadata":27820181},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724571,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28208,"kb_used_data":988,"kb_used_omap":36,"kb_used_meta":27163,"kb_avail":733974992,"statfs":{"total":751619276800,"available":751590391808,"internally_reserved":0,"allocated":1011712,"data_stored":1046891,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36952,"internal_metadata":27815848},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888094,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28140,"kb_used_data":920,"kb_used_omap":45,"kb_used_meta":27154,"kb_avail":733975060,"statfs":{"total":751619276800,"available":751590461440,"internally_reserved":0,"allocated":942080,"data_stored":455962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":46332,"internal_metadata":27806468},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149728,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28152,"kb_used_data":932,"kb_used_omap":51,"kb_used_meta":27148,"kb_avail":733975048,"statfs":{"total":751619276800,"available":751590449152,"internally_reserved":0,"allocated":954368,"data_stored":455426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52913,"internal_metadata":27799887},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28220,"kb_used_data":1000,"kb_used_omap":64,"kb_used_meta":27135,"kb_avail":733974980,"statfs":{"total":751619276800,"available":751590379520,"internally_reserved":0,"allocated":1024000,"data_stored":518368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":66234,"internal_metadata":27786566},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542182,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28196,"kb_used_data":976,"kb_used_omap":43,"kb_used_meta":27156,"kb_avail":733975004,"statfs":{"total":751619276800,"available":751590404096,"internally_reserved":0,"allocated":999424,"data_stored":516996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44488,"internal_metadata":27808312},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705705,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28156,"kb_used_data":936,"kb_used_omap":54,"kb_used_meta":27145,"kb_avail":733975044,"statfs":{"total":751619276800,"available":751590445056,"internally_reserved":0,"allocated":958464,"data_stored":456430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56100,"internal_metadata":27796700},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8853,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1407,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56835,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49702,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6449,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3478,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14040,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9858,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25902,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22214,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21693,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6862,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T11:47:31.554 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph pg dump --format=json 2026-02-25T11:47:31.737 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:31.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.908+0000 7fc9556ba640 1 -- 10.20.193.28:0/1877705942 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc95010b860 msgr2=0x7fc95010bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:31.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.908+0000 7fc9556ba640 1 --2- 10.20.193.28:0/1877705942 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc95010b860 0x7fc95010bc30 secure :-1 s=READY pgs=286 gs=1 cs=0 l=1 c_cookie=2712257894100206718 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc93c009950 tx=0x7fc93c02f190 comp rx=0 tx=0).stop 2026-02-25T11:47:31.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.908+0000 7fc9556ba640 1 -- 10.20.193.28:0/1877705942 shutdown_connections 2026-02-25T11:47:31.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.908+0000 7fc9556ba640 1 --2- 10.20.193.28:0/1877705942 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9501135b0 0x7fc950173f80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.908+0000 7fc9556ba640 1 --2- 10.20.193.28:0/1877705942 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc95010c250 0x7fc950112e00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.908+0000 7fc9556ba640 1 --2- 10.20.193.28:0/1877705942 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc95010b860 0x7fc95010bc30 unknown :-1 s=CLOSED pgs=286 gs=1 cs=0 l=1 c_cookie=2712257894100206718 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.908+0000 7fc9556ba640 1 -- 10.20.193.28:0/1877705942 >> 10.20.193.28:0/1877705942 conn(0x7fc9500e9290 msgr2=0x7fc9501096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:31.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.909+0000 7fc9556ba640 1 -- 10.20.193.28:0/1877705942 shutdown_connections 2026-02-25T11:47:31.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.909+0000 7fc9556ba640 1 -- 10.20.193.28:0/1877705942 wait complete. 2026-02-25T11:47:31.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.909+0000 7fc9556ba640 1 Processor -- start 2026-02-25T11:47:31.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.909+0000 7fc9556ba640 1 -- start start 2026-02-25T11:47:31.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc9556ba640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc95010b860 0x7fc9500f3f50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:31.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc9556ba640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc95010c250 0x7fc950173e80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:31.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc9556ba640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9501135b0 0x7fc9500f4650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:31.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc9556ba640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc9500f9860 con 0x7fc95010b860 2026-02-25T11:47:31.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc9556ba640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc9500f99d0 con 0x7fc95010c250 2026-02-25T11:47:31.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc9556ba640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc9500f9b90 con 0x7fc9501135b0 2026-02-25T11:47:31.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc94ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc95010b860 0x7fc9500f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:31.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc94ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc95010b860 0x7fc9500f3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41112/0 (socket says 10.20.193.28:41112) 2026-02-25T11:47:31.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc94ffff640 1 -- 10.20.193.28:0/3728858721 learned_addr learned my addr 10.20.193.28:0/3728858721 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:31.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc94f7fe640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc95010c250 0x7fc950173e80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:31.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc954eb9640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9501135b0 0x7fc9500f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:31.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc94ffff640 1 -- 10.20.193.28:0/3728858721 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9501135b0 msgr2=0x7fc9500f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:31.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc94ffff640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9501135b0 0x7fc9500f4650 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc94ffff640 1 -- 10.20.193.28:0/3728858721 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc95010c250 msgr2=0x7fc950173e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:31.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc94ffff640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc95010c250 0x7fc950173e80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:31.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc94ffff640 1 -- 10.20.193.28:0/3728858721 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc93c009570 con 0x7fc95010b860 2026-02-25T11:47:31.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc954eb9640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9501135b0 0x7fc9500f4650 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:31.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc94ffff640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc95010b860 0x7fc9500f3f50 secure :-1 s=READY pgs=287 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc93c0098a0 tx=0x7fc93c0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:31.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.910+0000 7fc94d7fa640 1 -- 10.20.193.28:0/3728858721 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc93c03d070 con 0x7fc95010b860 2026-02-25T11:47:31.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.911+0000 7fc94d7fa640 1 -- 10.20.193.28:0/3728858721 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc93c002a40 con 0x7fc95010b860 2026-02-25T11:47:31.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.911+0000 7fc94d7fa640 1 -- 10.20.193.28:0/3728858721 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc93c041820 con 0x7fc95010b860 2026-02-25T11:47:31.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.911+0000 7fc9556ba640 1 -- 10.20.193.28:0/3728858721 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9500f9e10 con 0x7fc95010b860 2026-02-25T11:47:31.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.911+0000 7fc9556ba640 1 -- 10.20.193.28:0/3728858721 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9500fa230 con 0x7fc95010b860 2026-02-25T11:47:31.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.911+0000 7fc9556ba640 1 -- 10.20.193.28:0/3728858721 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc95005fa30 con 0x7fc95010b860 2026-02-25T11:47:31.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.914+0000 7fc94d7fa640 1 -- 10.20.193.28:0/3728858721 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fc93c0419c0 con 0x7fc95010b860 2026-02-25T11:47:31.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.916+0000 7fc94d7fa640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9240b57e0 0x7fc9240d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:31.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.916+0000 7fc94d7fa640 1 -- 10.20.193.28:0/3728858721 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fc93c10b4b0 con 0x7fc95010b860 2026-02-25T11:47:31.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.916+0000 7fc94f7fe640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9240b57e0 0x7fc9240d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:31.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.916+0000 7fc94d7fa640 1 -- 10.20.193.28:0/3728858721 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fc93c10b930 con 0x7fc95010b860 2026-02-25T11:47:31.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:31.916+0000 7fc94f7fe640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9240b57e0 0x7fc9240d5c90 secure :-1 s=READY pgs=219 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9500fa8f0 tx=0x7fc940005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:32.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.031+0000 7fc9556ba640 1 -- 10.20.193.28:0/3728858721 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc9500f50b0 con 0x7fc9240b57e0 2026-02-25T11:47:32.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.033+0000 7fc94d7fa640 1 -- 10.20.193.28:0/3728858721 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258970 (secure 0 0 0) 0x7fc9500f50b0 con 0x7fc9240b57e0 2026-02-25T11:47:32.041 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:32.043 INFO:teuthology.orchestra.run.trial028.stderr:dumped all 2026-02-25T11:47:32.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.036+0000 7fc9556ba640 1 -- 10.20.193.28:0/3728858721 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9240b57e0 msgr2=0x7fc9240d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:32.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.036+0000 7fc9556ba640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9240b57e0 0x7fc9240d5c90 secure :-1 s=READY pgs=219 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9500fa8f0 tx=0x7fc940005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:32.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.036+0000 7fc9556ba640 1 -- 10.20.193.28:0/3728858721 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc95010b860 msgr2=0x7fc9500f3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:32.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.036+0000 7fc9556ba640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc95010b860 0x7fc9500f3f50 secure :-1 s=READY pgs=287 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc93c0098a0 tx=0x7fc93c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.036+0000 7fc9556ba640 1 -- 10.20.193.28:0/3728858721 shutdown_connections 2026-02-25T11:47:32.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.036+0000 7fc9556ba640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9240b57e0 0x7fc9240d5c90 unknown :-1 s=CLOSED pgs=219 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.036+0000 7fc9556ba640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc9501135b0 0x7fc9500f4650 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.036+0000 7fc9556ba640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc95010c250 0x7fc950173e80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.036+0000 7fc9556ba640 1 --2- 10.20.193.28:0/3728858721 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc95010b860 0x7fc9500f3f50 unknown :-1 s=CLOSED pgs=287 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.036+0000 7fc9556ba640 1 -- 10.20.193.28:0/3728858721 >> 10.20.193.28:0/3728858721 conn(0x7fc9500e9290 msgr2=0x7fc95010fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:32.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.036+0000 7fc9556ba640 1 -- 10.20.193.28:0/3728858721 shutdown_connections 2026-02-25T11:47:32.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.036+0000 7fc9556ba640 1 -- 10.20.193.28:0/3728858721 wait complete. 2026-02-25T11:47:32.138 INFO:teuthology.orchestra.run.trial028.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2026-02-25T11:47:31.076972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656776,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":195,"num_read_kb":203,"num_write":349,"num_write_kb":3834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":289,"ondisk_log_size":289,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":8808038400,"kb_used":338104,"kb_used_data":11464,"kb_used_omap":464,"kb_used_meta":325935,"kb_avail":8807700296,"statfs":{"total":9019431321600,"available":9019085103104,"internally_reserved":0,"allocated":11739136,"data_stored":7420852,"data_compressed":114024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":475320,"internal_metadata":333758280},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":62034,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":84,"num_read_kb":108,"num_write":82,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006497"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627807+0000","last_change":"2026-02-25T11:47:13.630492+0000","last_active":"2026-02-25T11:47:14.627807+0000","last_peered":"2026-02-25T11:47:14.627807+0000","last_clean":"2026-02-25T11:47:14.627807+0000","last_became_active":"2026-02-25T11:47:13.630388+0000","last_became_peered":"2026-02-25T11:47:13.630388+0000","last_unstale":"2026-02-25T11:47:14.627807+0000","last_undegraded":"2026-02-25T11:47:14.627807+0000","last_fullsized":"2026-02-25T11:47:14.627807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:01:17.024058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627657+0000","last_change":"2026-02-25T11:47:13.629507+0000","last_active":"2026-02-25T11:47:14.627657+0000","last_peered":"2026-02-25T11:47:14.627657+0000","last_clean":"2026-02-25T11:47:14.627657+0000","last_became_active":"2026-02-25T11:47:13.628937+0000","last_became_peered":"2026-02-25T11:47:13.628937+0000","last_unstale":"2026-02-25T11:47:14.627657+0000","last_undegraded":"2026-02-25T11:47:14.627657+0000","last_fullsized":"2026-02-25T11:47:14.627657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:47:20.441522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627713+0000","last_change":"2026-02-25T11:47:13.630415+0000","last_active":"2026-02-25T11:47:14.627713+0000","last_peered":"2026-02-25T11:47:14.627713+0000","last_clean":"2026-02-25T11:47:14.627713+0000","last_became_active":"2026-02-25T11:47:13.630319+0000","last_became_peered":"2026-02-25T11:47:13.630319+0000","last_unstale":"2026-02-25T11:47:14.627713+0000","last_undegraded":"2026-02-25T11:47:14.627713+0000","last_fullsized":"2026-02-25T11:47:14.627713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:18:57.675760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627800+0000","last_change":"2026-02-25T11:47:13.633017+0000","last_active":"2026-02-25T11:47:14.627800+0000","last_peered":"2026-02-25T11:47:14.627800+0000","last_clean":"2026-02-25T11:47:14.627800+0000","last_became_active":"2026-02-25T11:47:13.632934+0000","last_became_peered":"2026-02-25T11:47:13.632934+0000","last_unstale":"2026-02-25T11:47:14.627800+0000","last_undegraded":"2026-02-25T11:47:14.627800+0000","last_fullsized":"2026-02-25T11:47:14.627800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:08:54.195980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628667+0000","last_change":"2026-02-25T11:47:13.638057+0000","last_active":"2026-02-25T11:47:14.628667+0000","last_peered":"2026-02-25T11:47:14.628667+0000","last_clean":"2026-02-25T11:47:14.628667+0000","last_became_active":"2026-02-25T11:47:13.636387+0000","last_became_peered":"2026-02-25T11:47:13.636387+0000","last_unstale":"2026-02-25T11:47:14.628667+0000","last_undegraded":"2026-02-25T11:47:14.628667+0000","last_fullsized":"2026-02-25T11:47:14.628667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:34:29.570622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750630+0000","last_change":"2026-02-25T11:47:13.631824+0000","last_active":"2026-02-25T11:47:14.750630+0000","last_peered":"2026-02-25T11:47:14.750630+0000","last_clean":"2026-02-25T11:47:14.750630+0000","last_became_active":"2026-02-25T11:47:13.631656+0000","last_became_peered":"2026-02-25T11:47:13.631656+0000","last_unstale":"2026-02-25T11:47:14.750630+0000","last_undegraded":"2026-02-25T11:47:14.750630+0000","last_fullsized":"2026-02-25T11:47:14.750630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:15:05.237015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628186+0000","last_change":"2026-02-25T11:47:13.626877+0000","last_active":"2026-02-25T11:47:14.628186+0000","last_peered":"2026-02-25T11:47:14.628186+0000","last_clean":"2026-02-25T11:47:14.628186+0000","last_became_active":"2026-02-25T11:47:13.626543+0000","last_became_peered":"2026-02-25T11:47:13.626543+0000","last_unstale":"2026-02-25T11:47:14.628186+0000","last_undegraded":"2026-02-25T11:47:14.628186+0000","last_fullsized":"2026-02-25T11:47:14.628186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:56:58.194911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635224+0000","last_change":"2026-02-25T11:47:13.637661+0000","last_active":"2026-02-25T11:47:14.635224+0000","last_peered":"2026-02-25T11:47:14.635224+0000","last_clean":"2026-02-25T11:47:14.635224+0000","last_became_active":"2026-02-25T11:47:13.637230+0000","last_became_peered":"2026-02-25T11:47:13.637230+0000","last_unstale":"2026-02-25T11:47:14.635224+0000","last_undegraded":"2026-02-25T11:47:14.635224+0000","last_fullsized":"2026-02-25T11:47:14.635224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:44:52.310360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751287+0000","last_change":"2026-02-25T11:47:13.625160+0000","last_active":"2026-02-25T11:47:14.751287+0000","last_peered":"2026-02-25T11:47:14.751287+0000","last_clean":"2026-02-25T11:47:14.751287+0000","last_became_active":"2026-02-25T11:47:13.625020+0000","last_became_peered":"2026-02-25T11:47:13.625020+0000","last_unstale":"2026-02-25T11:47:14.751287+0000","last_undegraded":"2026-02-25T11:47:14.751287+0000","last_fullsized":"2026-02-25T11:47:14.751287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:35:44.509551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750832+0000","last_change":"2026-02-25T11:47:13.637252+0000","last_active":"2026-02-25T11:47:14.750832+0000","last_peered":"2026-02-25T11:47:14.750832+0000","last_clean":"2026-02-25T11:47:14.750832+0000","last_became_active":"2026-02-25T11:47:13.636670+0000","last_became_peered":"2026-02-25T11:47:13.636670+0000","last_unstale":"2026-02-25T11:47:14.750832+0000","last_undegraded":"2026-02-25T11:47:14.750832+0000","last_fullsized":"2026-02-25T11:47:14.750832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:01:53.119101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628443+0000","last_change":"2026-02-25T11:47:13.628252+0000","last_active":"2026-02-25T11:47:14.628443+0000","last_peered":"2026-02-25T11:47:14.628443+0000","last_clean":"2026-02-25T11:47:14.628443+0000","last_became_active":"2026-02-25T11:47:13.628183+0000","last_became_peered":"2026-02-25T11:47:13.628183+0000","last_unstale":"2026-02-25T11:47:14.628443+0000","last_undegraded":"2026-02-25T11:47:14.628443+0000","last_fullsized":"2026-02-25T11:47:14.628443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:12:00.576747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.632358+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.631487+0000","last_became_peered":"2026-02-25T11:47:13.631487+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:41:17.150851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749735+0000","last_change":"2026-02-25T11:47:13.629675+0000","last_active":"2026-02-25T11:47:14.749735+0000","last_peered":"2026-02-25T11:47:14.749735+0000","last_clean":"2026-02-25T11:47:14.749735+0000","last_became_active":"2026-02-25T11:47:13.628686+0000","last_became_peered":"2026-02-25T11:47:13.628686+0000","last_unstale":"2026-02-25T11:47:14.749735+0000","last_undegraded":"2026-02-25T11:47:14.749735+0000","last_fullsized":"2026-02-25T11:47:14.749735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:00:54.430287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627615+0000","last_change":"2026-02-25T11:47:13.628842+0000","last_active":"2026-02-25T11:47:14.627615+0000","last_peered":"2026-02-25T11:47:14.627615+0000","last_clean":"2026-02-25T11:47:14.627615+0000","last_became_active":"2026-02-25T11:47:13.628695+0000","last_became_peered":"2026-02-25T11:47:13.628695+0000","last_unstale":"2026-02-25T11:47:14.627615+0000","last_undegraded":"2026-02-25T11:47:14.627615+0000","last_fullsized":"2026-02-25T11:47:14.627615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:10:05.404740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.635652+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.635505+0000","last_became_peered":"2026-02-25T11:47:13.635505+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:46:03.598577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749928+0000","last_change":"2026-02-25T11:47:13.628651+0000","last_active":"2026-02-25T11:47:14.749928+0000","last_peered":"2026-02-25T11:47:14.749928+0000","last_clean":"2026-02-25T11:47:14.749928+0000","last_became_active":"2026-02-25T11:47:13.628280+0000","last_became_peered":"2026-02-25T11:47:13.628280+0000","last_unstale":"2026-02-25T11:47:14.749928+0000","last_undegraded":"2026-02-25T11:47:14.749928+0000","last_fullsized":"2026-02-25T11:47:14.749928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:18:56.238078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628351+0000","last_change":"2026-02-25T11:47:13.630656+0000","last_active":"2026-02-25T11:47:14.628351+0000","last_peered":"2026-02-25T11:47:14.628351+0000","last_clean":"2026-02-25T11:47:14.628351+0000","last_became_active":"2026-02-25T11:47:13.630513+0000","last_became_peered":"2026-02-25T11:47:13.630513+0000","last_unstale":"2026-02-25T11:47:14.628351+0000","last_undegraded":"2026-02-25T11:47:14.628351+0000","last_fullsized":"2026-02-25T11:47:14.628351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:03:33.580576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.634036+0000","last_change":"2026-02-25T11:47:13.629202+0000","last_active":"2026-02-25T11:47:14.634036+0000","last_peered":"2026-02-25T11:47:14.634036+0000","last_clean":"2026-02-25T11:47:14.634036+0000","last_became_active":"2026-02-25T11:47:13.628860+0000","last_became_peered":"2026-02-25T11:47:13.628860+0000","last_unstale":"2026-02-25T11:47:14.634036+0000","last_undegraded":"2026-02-25T11:47:14.634036+0000","last_fullsized":"2026-02-25T11:47:14.634036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:47:14.635796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628235+0000","last_change":"2026-02-25T11:47:13.638872+0000","last_active":"2026-02-25T11:47:14.628235+0000","last_peered":"2026-02-25T11:47:14.628235+0000","last_clean":"2026-02-25T11:47:14.628235+0000","last_became_active":"2026-02-25T11:47:13.638443+0000","last_became_peered":"2026-02-25T11:47:13.638443+0000","last_unstale":"2026-02-25T11:47:14.628235+0000","last_undegraded":"2026-02-25T11:47:14.628235+0000","last_fullsized":"2026-02-25T11:47:14.628235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:51.699838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628595+0000","last_change":"2026-02-25T11:47:13.635894+0000","last_active":"2026-02-25T11:47:14.628595+0000","last_peered":"2026-02-25T11:47:14.628595+0000","last_clean":"2026-02-25T11:47:14.628595+0000","last_became_active":"2026-02-25T11:47:13.635725+0000","last_became_peered":"2026-02-25T11:47:13.635725+0000","last_unstale":"2026-02-25T11:47:14.628595+0000","last_undegraded":"2026-02-25T11:47:14.628595+0000","last_fullsized":"2026-02-25T11:47:14.628595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:28:20.134454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749794+0000","last_change":"2026-02-25T11:47:13.628117+0000","last_active":"2026-02-25T11:47:14.749794+0000","last_peered":"2026-02-25T11:47:14.749794+0000","last_clean":"2026-02-25T11:47:14.749794+0000","last_became_active":"2026-02-25T11:47:13.628037+0000","last_became_peered":"2026-02-25T11:47:13.628037+0000","last_unstale":"2026-02-25T11:47:14.749794+0000","last_undegraded":"2026-02-25T11:47:14.749794+0000","last_fullsized":"2026-02-25T11:47:14.749794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:28:22.976133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750830+0000","last_change":"2026-02-25T11:47:13.634652+0000","last_active":"2026-02-25T11:47:14.750830+0000","last_peered":"2026-02-25T11:47:14.750830+0000","last_clean":"2026-02-25T11:47:14.750830+0000","last_became_active":"2026-02-25T11:47:13.634433+0000","last_became_peered":"2026-02-25T11:47:13.634433+0000","last_unstale":"2026-02-25T11:47:14.750830+0000","last_undegraded":"2026-02-25T11:47:14.750830+0000","last_fullsized":"2026-02-25T11:47:14.750830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:01:29.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635319+0000","last_change":"2026-02-25T11:47:13.632721+0000","last_active":"2026-02-25T11:47:14.635319+0000","last_peered":"2026-02-25T11:47:14.635319+0000","last_clean":"2026-02-25T11:47:14.635319+0000","last_became_active":"2026-02-25T11:47:13.632626+0000","last_became_peered":"2026-02-25T11:47:13.632626+0000","last_unstale":"2026-02-25T11:47:14.635319+0000","last_undegraded":"2026-02-25T11:47:14.635319+0000","last_fullsized":"2026-02-25T11:47:14.635319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:28:35.201966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749844+0000","last_change":"2026-02-25T11:47:13.637504+0000","last_active":"2026-02-25T11:47:14.749844+0000","last_peered":"2026-02-25T11:47:14.749844+0000","last_clean":"2026-02-25T11:47:14.749844+0000","last_became_active":"2026-02-25T11:47:13.636824+0000","last_became_peered":"2026-02-25T11:47:13.636824+0000","last_unstale":"2026-02-25T11:47:14.749844+0000","last_undegraded":"2026-02-25T11:47:14.749844+0000","last_fullsized":"2026-02-25T11:47:14.749844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:20:31.416302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635724+0000","last_change":"2026-02-25T11:47:13.633135+0000","last_active":"2026-02-25T11:47:14.635724+0000","last_peered":"2026-02-25T11:47:14.635724+0000","last_clean":"2026-02-25T11:47:14.635724+0000","last_became_active":"2026-02-25T11:47:13.632966+0000","last_became_peered":"2026-02-25T11:47:13.632966+0000","last_unstale":"2026-02-25T11:47:14.635724+0000","last_undegraded":"2026-02-25T11:47:14.635724+0000","last_fullsized":"2026-02-25T11:47:14.635724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:26:17.052924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628538+0000","last_change":"2026-02-25T11:47:13.633210+0000","last_active":"2026-02-25T11:47:14.628538+0000","last_peered":"2026-02-25T11:47:14.628538+0000","last_clean":"2026-02-25T11:47:14.628538+0000","last_became_active":"2026-02-25T11:47:13.633099+0000","last_became_peered":"2026-02-25T11:47:13.633099+0000","last_unstale":"2026-02-25T11:47:14.628538+0000","last_undegraded":"2026-02-25T11:47:14.628538+0000","last_fullsized":"2026-02-25T11:47:14.628538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:07:46.884629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628656+0000","last_change":"2026-02-25T11:47:13.634757+0000","last_active":"2026-02-25T11:47:14.628656+0000","last_peered":"2026-02-25T11:47:14.628656+0000","last_clean":"2026-02-25T11:47:14.628656+0000","last_became_active":"2026-02-25T11:47:13.634643+0000","last_became_peered":"2026-02-25T11:47:13.634643+0000","last_unstale":"2026-02-25T11:47:14.628656+0000","last_undegraded":"2026-02-25T11:47:14.628656+0000","last_fullsized":"2026-02-25T11:47:14.628656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:20:23.383359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749674+0000","last_change":"2026-02-25T11:47:13.630515+0000","last_active":"2026-02-25T11:47:14.749674+0000","last_peered":"2026-02-25T11:47:14.749674+0000","last_clean":"2026-02-25T11:47:14.749674+0000","last_became_active":"2026-02-25T11:47:13.630202+0000","last_became_peered":"2026-02-25T11:47:13.630202+0000","last_unstale":"2026-02-25T11:47:14.749674+0000","last_undegraded":"2026-02-25T11:47:14.749674+0000","last_fullsized":"2026-02-25T11:47:14.749674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:51:24.646614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627367+0000","last_change":"2026-02-25T11:47:13.635749+0000","last_active":"2026-02-25T11:47:14.627367+0000","last_peered":"2026-02-25T11:47:14.627367+0000","last_clean":"2026-02-25T11:47:14.627367+0000","last_became_active":"2026-02-25T11:47:13.635284+0000","last_became_peered":"2026-02-25T11:47:13.635284+0000","last_unstale":"2026-02-25T11:47:14.627367+0000","last_undegraded":"2026-02-25T11:47:14.627367+0000","last_fullsized":"2026-02-25T11:47:14.627367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:23:45.042142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628732+0000","last_change":"2026-02-25T11:47:13.635448+0000","last_active":"2026-02-25T11:47:14.628732+0000","last_peered":"2026-02-25T11:47:14.628732+0000","last_clean":"2026-02-25T11:47:14.628732+0000","last_became_active":"2026-02-25T11:47:13.635232+0000","last_became_peered":"2026-02-25T11:47:13.635232+0000","last_unstale":"2026-02-25T11:47:14.628732+0000","last_undegraded":"2026-02-25T11:47:14.628732+0000","last_fullsized":"2026-02-25T11:47:14.628732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:33:38.020605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628332+0000","last_change":"2026-02-25T11:47:13.634237+0000","last_active":"2026-02-25T11:47:14.628332+0000","last_peered":"2026-02-25T11:47:14.628332+0000","last_clean":"2026-02-25T11:47:14.628332+0000","last_became_active":"2026-02-25T11:47:13.634054+0000","last_became_peered":"2026-02-25T11:47:13.634054+0000","last_unstale":"2026-02-25T11:47:14.628332+0000","last_undegraded":"2026-02-25T11:47:14.628332+0000","last_fullsized":"2026-02-25T11:47:14.628332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:38:34.603077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627786+0000","last_change":"2026-02-25T11:47:13.638347+0000","last_active":"2026-02-25T11:47:14.627786+0000","last_peered":"2026-02-25T11:47:14.627786+0000","last_clean":"2026-02-25T11:47:14.627786+0000","last_became_active":"2026-02-25T11:47:13.637984+0000","last_became_peered":"2026-02-25T11:47:13.637984+0000","last_unstale":"2026-02-25T11:47:14.627786+0000","last_undegraded":"2026-02-25T11:47:14.627786+0000","last_fullsized":"2026-02-25T11:47:14.627786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:36:11.273078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628202+0000","last_change":"2026-02-25T11:47:13.634523+0000","last_active":"2026-02-25T11:47:14.628202+0000","last_peered":"2026-02-25T11:47:14.628202+0000","last_clean":"2026-02-25T11:47:14.628202+0000","last_became_active":"2026-02-25T11:47:13.634184+0000","last_became_peered":"2026-02-25T11:47:13.634184+0000","last_unstale":"2026-02-25T11:47:14.628202+0000","last_undegraded":"2026-02-25T11:47:14.628202+0000","last_fullsized":"2026-02-25T11:47:14.628202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:52:23.917090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:27.584183+0000","last_change":"2026-02-25T11:47:12.613380+0000","last_active":"2026-02-25T11:47:27.584183+0000","last_peered":"2026-02-25T11:47:27.584183+0000","last_clean":"2026-02-25T11:47:27.584183+0000","last_became_active":"2026-02-25T11:47:12.613224+0000","last_became_peered":"2026-02-25T11:47:12.613224+0000","last_unstale":"2026-02-25T11:47:27.584183+0000","last_undegraded":"2026-02-25T11:47:27.584183+0000","last_fullsized":"2026-02-25T11:47:27.584183+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:59:39.894067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749915+0000","last_change":"2026-02-25T11:47:13.636227+0000","last_active":"2026-02-25T11:47:14.749915+0000","last_peered":"2026-02-25T11:47:14.749915+0000","last_clean":"2026-02-25T11:47:14.749915+0000","last_became_active":"2026-02-25T11:47:13.635212+0000","last_became_peered":"2026-02-25T11:47:13.635212+0000","last_unstale":"2026-02-25T11:47:14.749915+0000","last_undegraded":"2026-02-25T11:47:14.749915+0000","last_fullsized":"2026-02-25T11:47:14.749915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:51:07.635399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657308+0000","last_change":"2026-02-25T11:47:12.613789+0000","last_active":"2026-02-25T11:47:17.657308+0000","last_peered":"2026-02-25T11:47:17.657308+0000","last_clean":"2026-02-25T11:47:17.657308+0000","last_became_active":"2026-02-25T11:47:12.613635+0000","last_became_peered":"2026-02-25T11:47:12.613635+0000","last_unstale":"2026-02-25T11:47:17.657308+0000","last_undegraded":"2026-02-25T11:47:17.657308+0000","last_fullsized":"2026-02-25T11:47:17.657308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:13:32.216593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657439+0000","last_change":"2026-02-25T11:47:12.612012+0000","last_active":"2026-02-25T11:47:17.657439+0000","last_peered":"2026-02-25T11:47:17.657439+0000","last_clean":"2026-02-25T11:47:17.657439+0000","last_became_active":"2026-02-25T11:47:12.611862+0000","last_became_peered":"2026-02-25T11:47:12.611862+0000","last_unstale":"2026-02-25T11:47:17.657439+0000","last_undegraded":"2026-02-25T11:47:17.657439+0000","last_fullsized":"2026-02-25T11:47:17.657439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:08:17.490884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635279+0000","last_change":"2026-02-25T11:47:13.627129+0000","last_active":"2026-02-25T11:47:14.635279+0000","last_peered":"2026-02-25T11:47:14.635279+0000","last_clean":"2026-02-25T11:47:14.635279+0000","last_became_active":"2026-02-25T11:47:13.626987+0000","last_became_peered":"2026-02-25T11:47:13.626987+0000","last_unstale":"2026-02-25T11:47:14.635279+0000","last_undegraded":"2026-02-25T11:47:14.635279+0000","last_fullsized":"2026-02-25T11:47:14.635279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:53:45.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749940+0000","last_change":"2026-02-25T11:47:12.611992+0000","last_active":"2026-02-25T11:47:14.749940+0000","last_peered":"2026-02-25T11:47:14.749940+0000","last_clean":"2026-02-25T11:47:14.749940+0000","last_became_active":"2026-02-25T11:47:12.611502+0000","last_became_peered":"2026-02-25T11:47:12.611502+0000","last_unstale":"2026-02-25T11:47:14.749940+0000","last_undegraded":"2026-02-25T11:47:14.749940+0000","last_fullsized":"2026-02-25T11:47:14.749940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:15:45.718628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633835+0000","last_change":"2026-02-25T11:47:13.635344+0000","last_active":"2026-02-25T11:47:14.633835+0000","last_peered":"2026-02-25T11:47:14.633835+0000","last_clean":"2026-02-25T11:47:14.633835+0000","last_became_active":"2026-02-25T11:47:13.634327+0000","last_became_peered":"2026-02-25T11:47:13.634327+0000","last_unstale":"2026-02-25T11:47:14.633835+0000","last_undegraded":"2026-02-25T11:47:14.633835+0000","last_fullsized":"2026-02-25T11:47:14.633835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:30:06.778398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:26.582979+0000","last_change":"2026-02-25T11:47:12.611130+0000","last_active":"2026-02-25T11:47:26.582979+0000","last_peered":"2026-02-25T11:47:26.582979+0000","last_clean":"2026-02-25T11:47:26.582979+0000","last_became_active":"2026-02-25T11:47:12.610838+0000","last_became_peered":"2026-02-25T11:47:12.610838+0000","last_unstale":"2026-02-25T11:47:26.582979+0000","last_undegraded":"2026-02-25T11:47:26.582979+0000","last_fullsized":"2026-02-25T11:47:26.582979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:34:19.366424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751244+0000","last_change":"2026-02-25T11:47:13.629263+0000","last_active":"2026-02-25T11:47:14.751244+0000","last_peered":"2026-02-25T11:47:14.751244+0000","last_clean":"2026-02-25T11:47:14.751244+0000","last_became_active":"2026-02-25T11:47:13.628765+0000","last_became_peered":"2026-02-25T11:47:13.628765+0000","last_unstale":"2026-02-25T11:47:14.751244+0000","last_undegraded":"2026-02-25T11:47:14.751244+0000","last_fullsized":"2026-02-25T11:47:14.751244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:49:04.773052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.649872+0000","last_change":"2026-02-25T11:47:12.613455+0000","last_active":"2026-02-25T11:47:16.649872+0000","last_peered":"2026-02-25T11:47:16.649872+0000","last_clean":"2026-02-25T11:47:16.649872+0000","last_became_active":"2026-02-25T11:47:12.612904+0000","last_became_peered":"2026-02-25T11:47:12.612904+0000","last_unstale":"2026-02-25T11:47:16.649872+0000","last_undegraded":"2026-02-25T11:47:16.649872+0000","last_fullsized":"2026-02-25T11:47:16.649872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:16:13.149517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627673+0000","last_change":"2026-02-25T11:47:13.635178+0000","last_active":"2026-02-25T11:47:14.627673+0000","last_peered":"2026-02-25T11:47:14.627673+0000","last_clean":"2026-02-25T11:47:14.627673+0000","last_became_active":"2026-02-25T11:47:13.634985+0000","last_became_peered":"2026-02-25T11:47:13.634985+0000","last_unstale":"2026-02-25T11:47:14.627673+0000","last_undegraded":"2026-02-25T11:47:14.627673+0000","last_fullsized":"2026-02-25T11:47:14.627673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:08.068457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653282+0000","last_change":"2026-02-25T11:47:12.610853+0000","last_active":"2026-02-25T11:47:16.653282+0000","last_peered":"2026-02-25T11:47:16.653282+0000","last_clean":"2026-02-25T11:47:16.653282+0000","last_became_active":"2026-02-25T11:47:12.610639+0000","last_became_peered":"2026-02-25T11:47:12.610639+0000","last_unstale":"2026-02-25T11:47:16.653282+0000","last_undegraded":"2026-02-25T11:47:16.653282+0000","last_fullsized":"2026-02-25T11:47:16.653282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:00:06.042790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":694,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750953+0000","last_change":"2026-02-25T11:47:13.630063+0000","last_active":"2026-02-25T11:47:14.750953+0000","last_peered":"2026-02-25T11:47:14.750953+0000","last_clean":"2026-02-25T11:47:14.750953+0000","last_became_active":"2026-02-25T11:47:13.629906+0000","last_became_peered":"2026-02-25T11:47:13.629906+0000","last_unstale":"2026-02-25T11:47:14.750953+0000","last_undegraded":"2026-02-25T11:47:14.750953+0000","last_fullsized":"2026-02-25T11:47:14.750953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:09:13.614948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'26","reported_seq":69,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:27.586413+0000","last_change":"2026-02-25T11:47:12.615044+0000","last_active":"2026-02-25T11:47:27.586413+0000","last_peered":"2026-02-25T11:47:27.586413+0000","last_clean":"2026-02-25T11:47:27.586413+0000","last_became_active":"2026-02-25T11:47:12.614698+0000","last_became_peered":"2026-02-25T11:47:12.614698+0000","last_unstale":"2026-02-25T11:47:27.586413+0000","last_undegraded":"2026-02-25T11:47:27.586413+0000","last_fullsized":"2026-02-25T11:47:27.586413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:52:32.148801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62088,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":27,"num_read_kb":73,"num_write":27,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628510+0000","last_change":"2026-02-25T11:47:13.630945+0000","last_active":"2026-02-25T11:47:14.628510+0000","last_peered":"2026-02-25T11:47:14.628510+0000","last_clean":"2026-02-25T11:47:14.628510+0000","last_became_active":"2026-02-25T11:47:13.630693+0000","last_became_peered":"2026-02-25T11:47:13.630693+0000","last_unstale":"2026-02-25T11:47:14.628510+0000","last_undegraded":"2026-02-25T11:47:14.628510+0000","last_fullsized":"2026-02-25T11:47:14.628510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:39:52.744366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656758+0000","last_change":"2026-02-25T11:47:12.614950+0000","last_active":"2026-02-25T11:47:17.656758+0000","last_peered":"2026-02-25T11:47:17.656758+0000","last_clean":"2026-02-25T11:47:17.656758+0000","last_became_active":"2026-02-25T11:47:12.614272+0000","last_became_peered":"2026-02-25T11:47:12.614272+0000","last_unstale":"2026-02-25T11:47:17.656758+0000","last_undegraded":"2026-02-25T11:47:17.656758+0000","last_fullsized":"2026-02-25T11:47:17.656758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:45:50.928166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749596+0000","last_change":"2026-02-25T11:47:13.633128+0000","last_active":"2026-02-25T11:47:14.749596+0000","last_peered":"2026-02-25T11:47:14.749596+0000","last_clean":"2026-02-25T11:47:14.749596+0000","last_became_active":"2026-02-25T11:47:13.633047+0000","last_became_peered":"2026-02-25T11:47:13.633047+0000","last_unstale":"2026-02-25T11:47:14.749596+0000","last_undegraded":"2026-02-25T11:47:14.749596+0000","last_fullsized":"2026-02-25T11:47:14.749596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:56:33.191193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653783+0000","last_change":"2026-02-25T11:47:12.616898+0000","last_active":"2026-02-25T11:47:16.653783+0000","last_peered":"2026-02-25T11:47:16.653783+0000","last_clean":"2026-02-25T11:47:16.653783+0000","last_became_active":"2026-02-25T11:47:12.616774+0000","last_became_peered":"2026-02-25T11:47:12.616774+0000","last_unstale":"2026-02-25T11:47:16.653783+0000","last_undegraded":"2026-02-25T11:47:16.653783+0000","last_fullsized":"2026-02-25T11:47:16.653783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:59:33.930544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628464+0000","last_change":"2026-02-25T11:47:13.630377+0000","last_active":"2026-02-25T11:47:14.628464+0000","last_peered":"2026-02-25T11:47:14.628464+0000","last_clean":"2026-02-25T11:47:14.628464+0000","last_became_active":"2026-02-25T11:47:13.630124+0000","last_became_peered":"2026-02-25T11:47:13.630124+0000","last_unstale":"2026-02-25T11:47:14.628464+0000","last_undegraded":"2026-02-25T11:47:14.628464+0000","last_fullsized":"2026-02-25T11:47:14.628464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:42:12.028258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:28.588594+0000","last_change":"2026-02-25T11:47:12.613208+0000","last_active":"2026-02-25T11:47:28.588594+0000","last_peered":"2026-02-25T11:47:28.588594+0000","last_clean":"2026-02-25T11:47:28.588594+0000","last_became_active":"2026-02-25T11:47:12.612460+0000","last_became_peered":"2026-02-25T11:47:12.612460+0000","last_unstale":"2026-02-25T11:47:28.588594+0000","last_undegraded":"2026-02-25T11:47:28.588594+0000","last_fullsized":"2026-02-25T11:47:28.588594+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:15:59.810632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":998,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628121+0000","last_change":"2026-02-25T11:47:13.631123+0000","last_active":"2026-02-25T11:47:14.628121+0000","last_peered":"2026-02-25T11:47:14.628121+0000","last_clean":"2026-02-25T11:47:14.628121+0000","last_became_active":"2026-02-25T11:47:13.631031+0000","last_became_peered":"2026-02-25T11:47:13.631031+0000","last_unstale":"2026-02-25T11:47:14.628121+0000","last_undegraded":"2026-02-25T11:47:14.628121+0000","last_fullsized":"2026-02-25T11:47:14.628121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:47:54.073226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657937+0000","last_change":"2026-02-25T11:47:12.612153+0000","last_active":"2026-02-25T11:47:17.657937+0000","last_peered":"2026-02-25T11:47:17.657937+0000","last_clean":"2026-02-25T11:47:17.657937+0000","last_became_active":"2026-02-25T11:47:12.611107+0000","last_became_peered":"2026-02-25T11:47:12.611107+0000","last_unstale":"2026-02-25T11:47:17.657937+0000","last_undegraded":"2026-02-25T11:47:17.657937+0000","last_fullsized":"2026-02-25T11:47:17.657937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:37:21.791728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749858+0000","last_change":"2026-02-25T11:47:13.635800+0000","last_active":"2026-02-25T11:47:14.749858+0000","last_peered":"2026-02-25T11:47:14.749858+0000","last_clean":"2026-02-25T11:47:14.749858+0000","last_became_active":"2026-02-25T11:47:13.635590+0000","last_became_peered":"2026-02-25T11:47:13.635590+0000","last_unstale":"2026-02-25T11:47:14.749858+0000","last_undegraded":"2026-02-25T11:47:14.749858+0000","last_fullsized":"2026-02-25T11:47:14.749858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:46:36.324518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575342+0000","last_change":"2026-02-25T11:47:12.616473+0000","last_active":"2026-02-25T11:47:18.575342+0000","last_peered":"2026-02-25T11:47:18.575342+0000","last_clean":"2026-02-25T11:47:18.575342+0000","last_became_active":"2026-02-25T11:47:12.616039+0000","last_became_peered":"2026-02-25T11:47:12.616039+0000","last_unstale":"2026-02-25T11:47:18.575342+0000","last_undegraded":"2026-02-25T11:47:18.575342+0000","last_fullsized":"2026-02-25T11:47:18.575342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:28.324889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628413+0000","last_change":"2026-02-25T11:47:13.634771+0000","last_active":"2026-02-25T11:47:14.628413+0000","last_peered":"2026-02-25T11:47:14.628413+0000","last_clean":"2026-02-25T11:47:14.628413+0000","last_became_active":"2026-02-25T11:47:13.634672+0000","last_became_peered":"2026-02-25T11:47:13.634672+0000","last_unstale":"2026-02-25T11:47:14.628413+0000","last_undegraded":"2026-02-25T11:47:14.628413+0000","last_fullsized":"2026-02-25T11:47:14.628413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:19:03.692423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749863+0000","last_change":"2026-02-25T11:47:12.616685+0000","last_active":"2026-02-25T11:47:14.749863+0000","last_peered":"2026-02-25T11:47:14.749863+0000","last_clean":"2026-02-25T11:47:14.749863+0000","last_became_active":"2026-02-25T11:47:12.616518+0000","last_became_peered":"2026-02-25T11:47:12.616518+0000","last_unstale":"2026-02-25T11:47:14.749863+0000","last_undegraded":"2026-02-25T11:47:14.749863+0000","last_fullsized":"2026-02-25T11:47:14.749863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:28:50.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628462+0000","last_change":"2026-02-25T11:47:13.627734+0000","last_active":"2026-02-25T11:47:14.628462+0000","last_peered":"2026-02-25T11:47:14.628462+0000","last_clean":"2026-02-25T11:47:14.628462+0000","last_became_active":"2026-02-25T11:47:13.627632+0000","last_became_peered":"2026-02-25T11:47:13.627632+0000","last_unstale":"2026-02-25T11:47:14.628462+0000","last_undegraded":"2026-02-25T11:47:14.628462+0000","last_fullsized":"2026-02-25T11:47:14.628462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:01:20.907699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750104+0000","last_change":"2026-02-25T11:47:12.612830+0000","last_active":"2026-02-25T11:47:14.750104+0000","last_peered":"2026-02-25T11:47:14.750104+0000","last_clean":"2026-02-25T11:47:14.750104+0000","last_became_active":"2026-02-25T11:47:12.612661+0000","last_became_peered":"2026-02-25T11:47:12.612661+0000","last_unstale":"2026-02-25T11:47:14.750104+0000","last_undegraded":"2026-02-25T11:47:14.750104+0000","last_fullsized":"2026-02-25T11:47:14.750104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:05:15.367058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:15.914932+0000","last_change":"2026-02-25T11:47:13.633998+0000","last_active":"2026-02-25T11:47:15.914932+0000","last_peered":"2026-02-25T11:47:15.914932+0000","last_clean":"2026-02-25T11:47:15.914932+0000","last_became_active":"2026-02-25T11:47:13.633820+0000","last_became_peered":"2026-02-25T11:47:13.633820+0000","last_unstale":"2026-02-25T11:47:15.914932+0000","last_undegraded":"2026-02-25T11:47:15.914932+0000","last_fullsized":"2026-02-25T11:47:15.914932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:02:38.693106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.658417+0000","last_change":"2026-02-25T11:47:12.614091+0000","last_active":"2026-02-25T11:47:17.658417+0000","last_peered":"2026-02-25T11:47:17.658417+0000","last_clean":"2026-02-25T11:47:17.658417+0000","last_became_active":"2026-02-25T11:47:12.612962+0000","last_became_peered":"2026-02-25T11:47:12.612962+0000","last_unstale":"2026-02-25T11:47:17.658417+0000","last_undegraded":"2026-02-25T11:47:17.658417+0000","last_fullsized":"2026-02-25T11:47:17.658417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:11:40.214146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628504+0000","last_change":"2026-02-25T11:47:13.627288+0000","last_active":"2026-02-25T11:47:14.628504+0000","last_peered":"2026-02-25T11:47:14.628504+0000","last_clean":"2026-02-25T11:47:14.628504+0000","last_became_active":"2026-02-25T11:47:13.626902+0000","last_became_peered":"2026-02-25T11:47:13.626902+0000","last_unstale":"2026-02-25T11:47:14.628504+0000","last_undegraded":"2026-02-25T11:47:14.628504+0000","last_fullsized":"2026-02-25T11:47:14.628504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:38:19.137242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'152","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628095+0000","last_change":"2026-02-25T11:46:35.865282+0000","last_active":"2026-02-25T11:47:14.628095+0000","last_peered":"2026-02-25T11:47:14.628095+0000","last_clean":"2026-02-25T11:47:14.628095+0000","last_became_active":"2026-02-25T11:46:35.865139+0000","last_became_peered":"2026-02-25T11:46:35.865139+0000","last_unstale":"2026-02-25T11:47:14.628095+0000","last_undegraded":"2026-02-25T11:47:14.628095+0000","last_fullsized":"2026-02-25T11:47:14.628095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_clean_scrub_stamp":"2026-02-25T11:44:43.739673+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:29:57.573039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575170+0000","last_change":"2026-02-25T11:47:12.612834+0000","last_active":"2026-02-25T11:47:18.575170+0000","last_peered":"2026-02-25T11:47:18.575170+0000","last_clean":"2026-02-25T11:47:18.575170+0000","last_became_active":"2026-02-25T11:47:12.611827+0000","last_became_peered":"2026-02-25T11:47:12.611827+0000","last_unstale":"2026-02-25T11:47:18.575170+0000","last_undegraded":"2026-02-25T11:47:18.575170+0000","last_fullsized":"2026-02-25T11:47:18.575170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:05:32.112908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635504+0000","last_change":"2026-02-25T11:47:13.634534+0000","last_active":"2026-02-25T11:47:14.635504+0000","last_peered":"2026-02-25T11:47:14.635504+0000","last_clean":"2026-02-25T11:47:14.635504+0000","last_became_active":"2026-02-25T11:47:13.633769+0000","last_became_peered":"2026-02-25T11:47:13.633769+0000","last_unstale":"2026-02-25T11:47:14.635504+0000","last_undegraded":"2026-02-25T11:47:14.635504+0000","last_fullsized":"2026-02-25T11:47:14.635504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:16:09.255508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656541+0000","last_change":"2026-02-25T11:47:12.612621+0000","last_active":"2026-02-25T11:47:17.656541+0000","last_peered":"2026-02-25T11:47:17.656541+0000","last_clean":"2026-02-25T11:47:17.656541+0000","last_became_active":"2026-02-25T11:47:12.612284+0000","last_became_peered":"2026-02-25T11:47:12.612284+0000","last_unstale":"2026-02-25T11:47:17.656541+0000","last_undegraded":"2026-02-25T11:47:17.656541+0000","last_fullsized":"2026-02-25T11:47:17.656541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:49:09.299914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749514+0000","last_change":"2026-02-25T11:47:13.629107+0000","last_active":"2026-02-25T11:47:14.749514+0000","last_peered":"2026-02-25T11:47:14.749514+0000","last_clean":"2026-02-25T11:47:14.749514+0000","last_became_active":"2026-02-25T11:47:13.629031+0000","last_became_peered":"2026-02-25T11:47:13.629031+0000","last_unstale":"2026-02-25T11:47:14.749514+0000","last_undegraded":"2026-02-25T11:47:14.749514+0000","last_fullsized":"2026-02-25T11:47:14.749514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:07:58.915416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:26.585836+0000","last_change":"2026-02-25T11:47:12.616543+0000","last_active":"2026-02-25T11:47:26.585836+0000","last_peered":"2026-02-25T11:47:26.585836+0000","last_clean":"2026-02-25T11:47:26.585836+0000","last_became_active":"2026-02-25T11:47:12.616345+0000","last_became_peered":"2026-02-25T11:47:12.616345+0000","last_unstale":"2026-02-25T11:47:26.585836+0000","last_undegraded":"2026-02-25T11:47:26.585836+0000","last_fullsized":"2026-02-25T11:47:26.585836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:27:30.862168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750014+0000","last_change":"2026-02-25T11:47:13.637385+0000","last_active":"2026-02-25T11:47:14.750014+0000","last_peered":"2026-02-25T11:47:14.750014+0000","last_clean":"2026-02-25T11:47:14.750014+0000","last_became_active":"2026-02-25T11:47:13.634653+0000","last_became_peered":"2026-02-25T11:47:13.634653+0000","last_unstale":"2026-02-25T11:47:14.750014+0000","last_undegraded":"2026-02-25T11:47:14.750014+0000","last_fullsized":"2026-02-25T11:47:14.750014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:05:08.662208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:15.914921+0000","last_change":"2026-02-25T11:47:12.614640+0000","last_active":"2026-02-25T11:47:15.914921+0000","last_peered":"2026-02-25T11:47:15.914921+0000","last_clean":"2026-02-25T11:47:15.914921+0000","last_became_active":"2026-02-25T11:47:12.614486+0000","last_became_peered":"2026-02-25T11:47:12.614486+0000","last_unstale":"2026-02-25T11:47:15.914921+0000","last_undegraded":"2026-02-25T11:47:15.914921+0000","last_fullsized":"2026-02-25T11:47:15.914921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:16:06.314709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750327+0000","last_change":"2026-02-25T11:47:13.631828+0000","last_active":"2026-02-25T11:47:14.750327+0000","last_peered":"2026-02-25T11:47:14.750327+0000","last_clean":"2026-02-25T11:47:14.750327+0000","last_became_active":"2026-02-25T11:47:13.631590+0000","last_became_peered":"2026-02-25T11:47:13.631590+0000","last_unstale":"2026-02-25T11:47:14.750327+0000","last_undegraded":"2026-02-25T11:47:14.750327+0000","last_fullsized":"2026-02-25T11:47:14.750327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:25.518794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633984+0000","last_change":"2026-02-25T11:47:12.614032+0000","last_active":"2026-02-25T11:47:14.633984+0000","last_peered":"2026-02-25T11:47:14.633984+0000","last_clean":"2026-02-25T11:47:14.633984+0000","last_became_active":"2026-02-25T11:47:12.613936+0000","last_became_peered":"2026-02-25T11:47:12.613936+0000","last_unstale":"2026-02-25T11:47:14.633984+0000","last_undegraded":"2026-02-25T11:47:14.633984+0000","last_fullsized":"2026-02-25T11:47:14.633984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:16:46.397485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.859010+0000","last_change":"2026-02-25T11:47:13.636173+0000","last_active":"2026-02-25T11:47:16.859010+0000","last_peered":"2026-02-25T11:47:16.859010+0000","last_clean":"2026-02-25T11:47:16.859010+0000","last_became_active":"2026-02-25T11:47:13.635745+0000","last_became_peered":"2026-02-25T11:47:13.635745+0000","last_unstale":"2026-02-25T11:47:16.859010+0000","last_undegraded":"2026-02-25T11:47:16.859010+0000","last_fullsized":"2026-02-25T11:47:16.859010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:03:33.671318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651871+0000","last_change":"2026-02-25T11:47:12.614003+0000","last_active":"2026-02-25T11:47:16.651871+0000","last_peered":"2026-02-25T11:47:16.651871+0000","last_clean":"2026-02-25T11:47:16.651871+0000","last_became_active":"2026-02-25T11:47:12.613893+0000","last_became_peered":"2026-02-25T11:47:12.613893+0000","last_unstale":"2026-02-25T11:47:16.651871+0000","last_undegraded":"2026-02-25T11:47:16.651871+0000","last_fullsized":"2026-02-25T11:47:16.651871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:33:21.133364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749746+0000","last_change":"2026-02-25T11:47:13.632632+0000","last_active":"2026-02-25T11:47:14.749746+0000","last_peered":"2026-02-25T11:47:14.749746+0000","last_clean":"2026-02-25T11:47:14.749746+0000","last_became_active":"2026-02-25T11:47:13.632408+0000","last_became_peered":"2026-02-25T11:47:13.632408+0000","last_unstale":"2026-02-25T11:47:14.749746+0000","last_undegraded":"2026-02-25T11:47:14.749746+0000","last_fullsized":"2026-02-25T11:47:14.749746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:38:45.448637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'10","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:26.638117+0000","last_change":"2026-02-25T11:47:12.613816+0000","last_active":"2026-02-25T11:47:26.638117+0000","last_peered":"2026-02-25T11:47:26.638117+0000","last_clean":"2026-02-25T11:47:26.638117+0000","last_became_active":"2026-02-25T11:47:12.613196+0000","last_became_peered":"2026-02-25T11:47:12.613196+0000","last_unstale":"2026-02-25T11:47:26.638117+0000","last_undegraded":"2026-02-25T11:47:26.638117+0000","last_fullsized":"2026-02-25T11:47:26.638117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:23:52.372808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635752+0000","last_change":"2026-02-25T11:47:13.632431+0000","last_active":"2026-02-25T11:47:14.635752+0000","last_peered":"2026-02-25T11:47:14.635752+0000","last_clean":"2026-02-25T11:47:14.635752+0000","last_became_active":"2026-02-25T11:47:13.631972+0000","last_became_peered":"2026-02-25T11:47:13.631972+0000","last_unstale":"2026-02-25T11:47:14.635752+0000","last_undegraded":"2026-02-25T11:47:14.635752+0000","last_fullsized":"2026-02-25T11:47:14.635752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:59:34.655028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651131+0000","last_change":"2026-02-25T11:47:12.612805+0000","last_active":"2026-02-25T11:47:16.651131+0000","last_peered":"2026-02-25T11:47:16.651131+0000","last_clean":"2026-02-25T11:47:16.651131+0000","last_became_active":"2026-02-25T11:47:12.612155+0000","last_became_peered":"2026-02-25T11:47:12.612155+0000","last_unstale":"2026-02-25T11:47:16.651131+0000","last_undegraded":"2026-02-25T11:47:16.651131+0000","last_fullsized":"2026-02-25T11:47:16.651131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:58:00.429639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627853+0000","last_change":"2026-02-25T11:47:13.636089+0000","last_active":"2026-02-25T11:47:14.627853+0000","last_peered":"2026-02-25T11:47:14.627853+0000","last_clean":"2026-02-25T11:47:14.627853+0000","last_became_active":"2026-02-25T11:47:13.635963+0000","last_became_peered":"2026-02-25T11:47:13.635963+0000","last_unstale":"2026-02-25T11:47:14.627853+0000","last_undegraded":"2026-02-25T11:47:14.627853+0000","last_fullsized":"2026-02-25T11:47:14.627853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:50:28.012023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650081+0000","last_change":"2026-02-25T11:47:12.609749+0000","last_active":"2026-02-25T11:47:16.650081+0000","last_peered":"2026-02-25T11:47:16.650081+0000","last_clean":"2026-02-25T11:47:16.650081+0000","last_became_active":"2026-02-25T11:47:12.609190+0000","last_became_peered":"2026-02-25T11:47:12.609190+0000","last_unstale":"2026-02-25T11:47:16.650081+0000","last_undegraded":"2026-02-25T11:47:16.650081+0000","last_fullsized":"2026-02-25T11:47:16.650081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:12:42.204122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628201+0000","last_change":"2026-02-25T11:47:13.629339+0000","last_active":"2026-02-25T11:47:14.628201+0000","last_peered":"2026-02-25T11:47:14.628201+0000","last_clean":"2026-02-25T11:47:14.628201+0000","last_became_active":"2026-02-25T11:47:13.629107+0000","last_became_peered":"2026-02-25T11:47:13.629107+0000","last_unstale":"2026-02-25T11:47:14.628201+0000","last_undegraded":"2026-02-25T11:47:14.628201+0000","last_fullsized":"2026-02-25T11:47:14.628201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:19:50.087771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.652673+0000","last_change":"2026-02-25T11:47:12.615249+0000","last_active":"2026-02-25T11:47:16.652673+0000","last_peered":"2026-02-25T11:47:16.652673+0000","last_clean":"2026-02-25T11:47:16.652673+0000","last_became_active":"2026-02-25T11:47:12.615165+0000","last_became_peered":"2026-02-25T11:47:12.615165+0000","last_unstale":"2026-02-25T11:47:16.652673+0000","last_undegraded":"2026-02-25T11:47:16.652673+0000","last_fullsized":"2026-02-25T11:47:16.652673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:40:52.452074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627665+0000","last_change":"2026-02-25T11:47:13.631460+0000","last_active":"2026-02-25T11:47:14.627665+0000","last_peered":"2026-02-25T11:47:14.627665+0000","last_clean":"2026-02-25T11:47:14.627665+0000","last_became_active":"2026-02-25T11:47:13.631298+0000","last_became_peered":"2026-02-25T11:47:13.631298+0000","last_unstale":"2026-02-25T11:47:14.627665+0000","last_undegraded":"2026-02-25T11:47:14.627665+0000","last_fullsized":"2026-02-25T11:47:14.627665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:26:03.362106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628432+0000","last_change":"2026-02-25T11:47:12.615779+0000","last_active":"2026-02-25T11:47:14.628432+0000","last_peered":"2026-02-25T11:47:14.628432+0000","last_clean":"2026-02-25T11:47:14.628432+0000","last_became_active":"2026-02-25T11:47:12.615073+0000","last_became_peered":"2026-02-25T11:47:12.615073+0000","last_unstale":"2026-02-25T11:47:14.628432+0000","last_undegraded":"2026-02-25T11:47:14.628432+0000","last_fullsized":"2026-02-25T11:47:14.628432+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:28:51.468199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635380+0000","last_change":"2026-02-25T11:47:13.632351+0000","last_active":"2026-02-25T11:47:14.635380+0000","last_peered":"2026-02-25T11:47:14.635380+0000","last_clean":"2026-02-25T11:47:14.635380+0000","last_became_active":"2026-02-25T11:47:13.632081+0000","last_became_peered":"2026-02-25T11:47:13.632081+0000","last_unstale":"2026-02-25T11:47:14.635380+0000","last_undegraded":"2026-02-25T11:47:14.635380+0000","last_fullsized":"2026-02-25T11:47:14.635380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:38:59.143253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650353+0000","last_change":"2026-02-25T11:47:12.616555+0000","last_active":"2026-02-25T11:47:16.650353+0000","last_peered":"2026-02-25T11:47:16.650353+0000","last_clean":"2026-02-25T11:47:16.650353+0000","last_became_active":"2026-02-25T11:47:12.616174+0000","last_became_peered":"2026-02-25T11:47:12.616174+0000","last_unstale":"2026-02-25T11:47:16.650353+0000","last_undegraded":"2026-02-25T11:47:16.650353+0000","last_fullsized":"2026-02-25T11:47:16.650353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:01:17.494129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628458+0000","last_change":"2026-02-25T11:47:13.627400+0000","last_active":"2026-02-25T11:47:14.628458+0000","last_peered":"2026-02-25T11:47:14.628458+0000","last_clean":"2026-02-25T11:47:14.628458+0000","last_became_active":"2026-02-25T11:47:13.626782+0000","last_became_peered":"2026-02-25T11:47:13.626782+0000","last_unstale":"2026-02-25T11:47:14.628458+0000","last_undegraded":"2026-02-25T11:47:14.628458+0000","last_fullsized":"2026-02-25T11:47:14.628458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:42:39.996935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750027+0000","last_change":"2026-02-25T11:47:13.634030+0000","last_active":"2026-02-25T11:47:14.750027+0000","last_peered":"2026-02-25T11:47:14.750027+0000","last_clean":"2026-02-25T11:47:14.750027+0000","last_became_active":"2026-02-25T11:47:13.632844+0000","last_became_peered":"2026-02-25T11:47:13.632844+0000","last_unstale":"2026-02-25T11:47:14.750027+0000","last_undegraded":"2026-02-25T11:47:14.750027+0000","last_fullsized":"2026-02-25T11:47:14.750027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:52:16.013596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628805+0000","last_change":"2026-02-25T11:47:12.613666+0000","last_active":"2026-02-25T11:47:14.628805+0000","last_peered":"2026-02-25T11:47:14.628805+0000","last_clean":"2026-02-25T11:47:14.628805+0000","last_became_active":"2026-02-25T11:47:12.613570+0000","last_became_peered":"2026-02-25T11:47:12.613570+0000","last_unstale":"2026-02-25T11:47:14.628805+0000","last_undegraded":"2026-02-25T11:47:14.628805+0000","last_fullsized":"2026-02-25T11:47:14.628805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:28:39.750737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750924+0000","last_change":"2026-02-25T11:47:13.631515+0000","last_active":"2026-02-25T11:47:14.750924+0000","last_peered":"2026-02-25T11:47:14.750924+0000","last_clean":"2026-02-25T11:47:14.750924+0000","last_became_active":"2026-02-25T11:47:13.631161+0000","last_became_peered":"2026-02-25T11:47:13.631161+0000","last_unstale":"2026-02-25T11:47:14.750924+0000","last_undegraded":"2026-02-25T11:47:14.750924+0000","last_fullsized":"2026-02-25T11:47:14.750924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:13:30.496521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656891+0000","last_change":"2026-02-25T11:47:12.615904+0000","last_active":"2026-02-25T11:47:17.656891+0000","last_peered":"2026-02-25T11:47:17.656891+0000","last_clean":"2026-02-25T11:47:17.656891+0000","last_became_active":"2026-02-25T11:47:12.615784+0000","last_became_peered":"2026-02-25T11:47:12.615784+0000","last_unstale":"2026-02-25T11:47:17.656891+0000","last_undegraded":"2026-02-25T11:47:17.656891+0000","last_fullsized":"2026-02-25T11:47:17.656891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:32:26.802870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627754+0000","last_change":"2026-02-25T11:47:13.630065+0000","last_active":"2026-02-25T11:47:14.627754+0000","last_peered":"2026-02-25T11:47:14.627754+0000","last_clean":"2026-02-25T11:47:14.627754+0000","last_became_active":"2026-02-25T11:47:13.629214+0000","last_became_peered":"2026-02-25T11:47:13.629214+0000","last_unstale":"2026-02-25T11:47:14.627754+0000","last_undegraded":"2026-02-25T11:47:14.627754+0000","last_fullsized":"2026-02-25T11:47:14.627754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:53:45.639526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.310632+0000","last_change":"2026-02-25T11:47:12.612956+0000","last_active":"2026-02-25T11:47:16.310632+0000","last_peered":"2026-02-25T11:47:16.310632+0000","last_clean":"2026-02-25T11:47:16.310632+0000","last_became_active":"2026-02-25T11:47:12.612843+0000","last_became_peered":"2026-02-25T11:47:12.612843+0000","last_unstale":"2026-02-25T11:47:16.310632+0000","last_undegraded":"2026-02-25T11:47:16.310632+0000","last_fullsized":"2026-02-25T11:47:16.310632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:41:30.867465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635780+0000","last_change":"2026-02-25T11:47:13.637712+0000","last_active":"2026-02-25T11:47:14.635780+0000","last_peered":"2026-02-25T11:47:14.635780+0000","last_clean":"2026-02-25T11:47:14.635780+0000","last_became_active":"2026-02-25T11:47:13.635030+0000","last_became_peered":"2026-02-25T11:47:13.635030+0000","last_unstale":"2026-02-25T11:47:14.635780+0000","last_undegraded":"2026-02-25T11:47:14.635780+0000","last_fullsized":"2026-02-25T11:47:14.635780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:01:22.896129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656736+0000","last_change":"2026-02-25T11:47:12.614149+0000","last_active":"2026-02-25T11:47:17.656736+0000","last_peered":"2026-02-25T11:47:17.656736+0000","last_clean":"2026-02-25T11:47:17.656736+0000","last_became_active":"2026-02-25T11:47:12.613137+0000","last_became_peered":"2026-02-25T11:47:12.613137+0000","last_unstale":"2026-02-25T11:47:17.656736+0000","last_undegraded":"2026-02-25T11:47:17.656736+0000","last_fullsized":"2026-02-25T11:47:17.656736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:49:49.451435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66174,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":89,"num_read_kb":112,"num_write":158,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":227293,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":262884,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350987,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28132,"kb_used_data":912,"kb_used_omap":9,"kb_used_meta":27190,"kb_avail":733975068,"statfs":{"total":751619276800,"available":751590469632,"internally_reserved":0,"allocated":933888,"data_stored":455512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10109,"internal_metadata":27842691},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612621,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28272,"kb_used_data":1052,"kb_used_omap":28,"kb_used_meta":27171,"kb_avail":733974928,"statfs":{"total":751619276800,"available":751590326272,"internally_reserved":0,"allocated":1077248,"data_stored":1101119,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":29123,"internal_metadata":27823677},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776144,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28148,"kb_used_data":928,"kb_used_omap":27,"kb_used_meta":27172,"kb_avail":733975052,"statfs":{"total":751619276800,"available":751590453248,"internally_reserved":0,"allocated":950272,"data_stored":455748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28025,"internal_metadata":27824775},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037779,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28148,"kb_used_data":928,"kb_used_omap":35,"kb_used_meta":27164,"kb_avail":733975052,"statfs":{"total":751619276800,"available":751590453248,"internally_reserved":0,"allocated":950272,"data_stored":455668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36257,"internal_metadata":27816543},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299414,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28196,"kb_used_data":976,"kb_used_omap":35,"kb_used_meta":27164,"kb_avail":733975004,"statfs":{"total":751619276800,"available":751590404096,"internally_reserved":0,"allocated":999424,"data_stored":1047617,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36168,"internal_metadata":27816632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462937,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28136,"kb_used_data":916,"kb_used_omap":31,"kb_used_meta":27168,"kb_avail":733975064,"statfs":{"total":751619276800,"available":751590465536,"internally_reserved":0,"allocated":937984,"data_stored":455115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32619,"internal_metadata":27820181},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724571,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28208,"kb_used_data":988,"kb_used_omap":36,"kb_used_meta":27163,"kb_avail":733974992,"statfs":{"total":751619276800,"available":751590391808,"internally_reserved":0,"allocated":1011712,"data_stored":1046891,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36952,"internal_metadata":27815848},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888094,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28140,"kb_used_data":920,"kb_used_omap":45,"kb_used_meta":27154,"kb_avail":733975060,"statfs":{"total":751619276800,"available":751590461440,"internally_reserved":0,"allocated":942080,"data_stored":455962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":46332,"internal_metadata":27806468},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149728,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28152,"kb_used_data":932,"kb_used_omap":51,"kb_used_meta":27148,"kb_avail":733975048,"statfs":{"total":751619276800,"available":751590449152,"internally_reserved":0,"allocated":954368,"data_stored":455426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52913,"internal_metadata":27799887},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378659,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28220,"kb_used_data":1000,"kb_used_omap":64,"kb_used_meta":27135,"kb_avail":733974980,"statfs":{"total":751619276800,"available":751590379520,"internally_reserved":0,"allocated":1024000,"data_stored":518368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":66234,"internal_metadata":27786566},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542182,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28196,"kb_used_data":976,"kb_used_omap":43,"kb_used_meta":27156,"kb_avail":733975004,"statfs":{"total":751619276800,"available":751590404096,"internally_reserved":0,"allocated":999424,"data_stored":516996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44488,"internal_metadata":27808312},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705705,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28156,"kb_used_data":936,"kb_used_omap":54,"kb_used_meta":27145,"kb_avail":733975044,"statfs":{"total":751619276800,"available":751590445056,"internally_reserved":0,"allocated":958464,"data_stored":456430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56100,"internal_metadata":27796700},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8853,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1407,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56835,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49702,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6449,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3478,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14040,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9858,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25902,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22214,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21693,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6862,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T11:47:32.145 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2026-02-25T11:47:32.146 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-25T11:47:32.146 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2026-02-25T11:47:32.146 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph health --format=json 2026-02-25T11:47:32.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:32 trial028 ceph-mon[39224]: pgmap v203: 97 pgs: 97 active+clean; 641 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 9.0 KiB/s rd, 8.2 KiB/s wr, 13 op/s 2026-02-25T11:47:32.329 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:32.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:32 trial167 ceph-mon[43392]: pgmap v203: 97 pgs: 97 active+clean; 641 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 9.0 KiB/s rd, 8.2 KiB/s wr, 13 op/s 2026-02-25T11:47:32.500 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.497+0000 7f3d6708e640 1 -- 10.20.193.28:0/1709375247 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d6010df70 msgr2=0x7f3d60112fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:32.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.497+0000 7f3d6708e640 1 --2- 10.20.193.28:0/1709375247 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d6010df70 0x7f3d60112fc0 secure :-1 s=READY pgs=288 gs=1 cs=0 l=1 c_cookie=15973097924950174708 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d48009950 tx=0x7f3d4802f190 comp rx=0 tx=0).stop 2026-02-25T11:47:32.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.497+0000 7f3d6708e640 1 -- 10.20.193.28:0/1709375247 shutdown_connections 2026-02-25T11:47:32.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.497+0000 7f3d6708e640 1 --2- 10.20.193.28:0/1709375247 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3d60113770 0x7f3d60174140 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.497+0000 7f3d6708e640 1 --2- 10.20.193.28:0/1709375247 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d6010df70 0x7f3d60112fc0 unknown :-1 s=CLOSED pgs=288 gs=1 cs=0 l=1 c_cookie=15973097924950174708 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.497+0000 7f3d6708e640 1 --2- 10.20.193.28:0/1709375247 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3d6010d580 0x7f3d6010d950 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.497+0000 7f3d6708e640 1 -- 10.20.193.28:0/1709375247 >> 10.20.193.28:0/1709375247 conn(0x7f3d600eae70 msgr2=0x7f3d6010b280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:32.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.497+0000 7f3d6708e640 1 -- 10.20.193.28:0/1709375247 shutdown_connections 2026-02-25T11:47:32.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.498+0000 7f3d6708e640 1 -- 10.20.193.28:0/1709375247 wait complete. 2026-02-25T11:47:32.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.498+0000 7f3d6708e640 1 Processor -- start 2026-02-25T11:47:32.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.498+0000 7f3d6708e640 1 -- start start 2026-02-25T11:47:32.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d6708e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d6010d580 0x7f3d600f5f90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:32.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d6708e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3d6010df70 0x7f3d60174040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:32.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d6708e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3d60113770 0x7f3d600f6690 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:32.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d6708e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d600fb8a0 con 0x7f3d6010d580 2026-02-25T11:47:32.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d6708e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3d600fba10 con 0x7f3d60113770 2026-02-25T11:47:32.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d6708e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d600fbbd0 con 0x7f3d6010df70 2026-02-25T11:47:32.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d64e03640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d6010d580 0x7f3d600f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:32.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d57fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3d6010df70 0x7f3d60174040 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:32.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d57fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3d6010df70 0x7f3d60174040 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52626/0 (socket says 10.20.193.28:52626) 2026-02-25T11:47:32.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d57fff640 1 -- 10.20.193.28:0/1515256490 learned_addr learned my addr 10.20.193.28:0/1515256490 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:32.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d65604640 1 --2- 10.20.193.28:0/1515256490 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3d60113770 0x7f3d600f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:32.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d64e03640 1 -- 10.20.193.28:0/1515256490 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3d6010df70 msgr2=0x7f3d60174040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:32.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d64e03640 1 --2- 10.20.193.28:0/1515256490 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3d6010df70 0x7f3d60174040 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d64e03640 1 -- 10.20.193.28:0/1515256490 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3d60113770 msgr2=0x7f3d600f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:32.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d64e03640 1 --2- 10.20.193.28:0/1515256490 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3d60113770 0x7f3d600f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.499+0000 7f3d64e03640 1 -- 10.20.193.28:0/1515256490 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d48009570 con 0x7f3d6010d580 2026-02-25T11:47:32.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.500+0000 7f3d64e03640 1 --2- 10.20.193.28:0/1515256490 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d6010d580 0x7f3d600f5f90 secure :-1 s=READY pgs=289 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d5000e9c0 tx=0x7f3d5000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:32.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.500+0000 7f3d55ffb640 1 -- 10.20.193.28:0/1515256490 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3d5000ccb0 con 0x7f3d6010d580 2026-02-25T11:47:32.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.500+0000 7f3d55ffb640 1 -- 10.20.193.28:0/1515256490 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3d5000ce10 con 0x7f3d6010d580 2026-02-25T11:47:32.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.500+0000 7f3d55ffb640 1 -- 10.20.193.28:0/1515256490 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3d50010430 con 0x7f3d6010d580 2026-02-25T11:47:32.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.500+0000 7f3d6708e640 1 -- 10.20.193.28:0/1515256490 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d600fbeb0 con 0x7f3d6010d580 2026-02-25T11:47:32.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.500+0000 7f3d6708e640 1 -- 10.20.193.28:0/1515256490 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d600fc330 con 0x7f3d6010d580 2026-02-25T11:47:32.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.501+0000 7f3d6708e640 1 -- 10.20.193.28:0/1515256490 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d60060c10 con 0x7f3d6010d580 2026-02-25T11:47:32.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.503+0000 7f3d55ffb640 1 -- 10.20.193.28:0/1515256490 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f3d5001f060 con 0x7f3d6010d580 2026-02-25T11:47:32.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:32 trial119 ceph-mon[42710]: pgmap v203: 97 pgs: 97 active+clean; 641 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 9.0 KiB/s rd, 8.2 KiB/s wr, 13 op/s 2026-02-25T11:47:32.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.504+0000 7f3d55ffb640 1 --2- 10.20.193.28:0/1515256490 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3d3c0b57e0 0x7f3d3c0d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:32.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.504+0000 7f3d55ffb640 1 -- 10.20.193.28:0/1515256490 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f3d50014070 con 0x7f3d6010d580 2026-02-25T11:47:32.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.505+0000 7f3d57fff640 1 --2- 10.20.193.28:0/1515256490 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3d3c0b57e0 0x7f3d3c0d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:32.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.505+0000 7f3d55ffb640 1 -- 10.20.193.28:0/1515256490 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f3d500a95d0 con 0x7f3d6010d580 2026-02-25T11:47:32.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.505+0000 7f3d57fff640 1 --2- 10.20.193.28:0/1515256490 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3d3c0b57e0 0x7f3d3c0d5c90 secure :-1 s=READY pgs=220 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d600fc930 tx=0x7f3d4803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:32.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.645+0000 7f3d6708e640 1 -- 10.20.193.28:0/1515256490 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f3d600f70f0 con 0x7f3d6010d580 2026-02-25T11:47:32.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.646+0000 7f3d55ffb640 1 -- 10.20.193.28:0/1515256490 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f3d500ac780 con 0x7f3d6010d580 2026-02-25T11:47:32.649 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:32.649 INFO:teuthology.orchestra.run.trial028.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-25T11:47:32.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.647+0000 7f3d6708e640 1 -- 10.20.193.28:0/1515256490 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3d3c0b57e0 msgr2=0x7f3d3c0d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:32.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.647+0000 7f3d6708e640 1 --2- 10.20.193.28:0/1515256490 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3d3c0b57e0 0x7f3d3c0d5c90 secure :-1 s=READY pgs=220 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d600fc930 tx=0x7f3d4803a040 comp rx=0 tx=0).stop 2026-02-25T11:47:32.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.647+0000 7f3d6708e640 1 -- 10.20.193.28:0/1515256490 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d6010d580 msgr2=0x7f3d600f5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:32.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.647+0000 7f3d6708e640 1 --2- 10.20.193.28:0/1515256490 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d6010d580 0x7f3d600f5f90 secure :-1 s=READY pgs=289 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3d5000e9c0 tx=0x7f3d5000ee90 comp rx=0 tx=0).stop 2026-02-25T11:47:32.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.647+0000 7f3d6708e640 1 -- 10.20.193.28:0/1515256490 shutdown_connections 2026-02-25T11:47:32.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.647+0000 7f3d6708e640 1 --2- 10.20.193.28:0/1515256490 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3d3c0b57e0 0x7f3d3c0d5c90 unknown :-1 s=CLOSED pgs=220 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.647+0000 7f3d6708e640 1 --2- 10.20.193.28:0/1515256490 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3d60113770 0x7f3d600f6690 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.647+0000 7f3d6708e640 1 --2- 10.20.193.28:0/1515256490 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3d6010df70 0x7f3d60174040 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.647+0000 7f3d6708e640 1 --2- 10.20.193.28:0/1515256490 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3d6010d580 0x7f3d600f5f90 unknown :-1 s=CLOSED pgs=289 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:32.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.647+0000 7f3d6708e640 1 -- 10.20.193.28:0/1515256490 >> 10.20.193.28:0/1515256490 conn(0x7f3d600eae70 msgr2=0x7f3d6010fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:32.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.647+0000 7f3d6708e640 1 -- 10.20.193.28:0/1515256490 shutdown_connections 2026-02-25T11:47:32.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:32.647+0000 7f3d6708e640 1 -- 10.20.193.28:0/1515256490 wait complete. 2026-02-25T11:47:32.747 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2026-02-25T11:47:32.747 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:47:32.930 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:33.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.094+0000 7fc67eb2a640 1 -- 10.20.193.28:0/1359964985 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc678110230 msgr2=0x7fc678115280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:33.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.094+0000 7fc67eb2a640 1 --2- 10.20.193.28:0/1359964985 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc678110230 0x7fc678115280 secure :-1 s=READY pgs=290 gs=1 cs=0 l=1 c_cookie=15189448816539451531 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc660009950 tx=0x7fc66002f190 comp rx=0 tx=0).stop 2026-02-25T11:47:33.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.094+0000 7fc67eb2a640 1 -- 10.20.193.28:0/1359964985 shutdown_connections 2026-02-25T11:47:33.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.094+0000 7fc67eb2a640 1 --2- 10.20.193.28:0/1359964985 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc678115a30 0x7fc678176400 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.094+0000 7fc67eb2a640 1 --2- 10.20.193.28:0/1359964985 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc678110230 0x7fc678115280 unknown :-1 s=CLOSED pgs=290 gs=1 cs=0 l=1 c_cookie=15189448816539451531 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.094+0000 7fc67eb2a640 1 --2- 10.20.193.28:0/1359964985 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc67810f840 0x7fc67810fc10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.094+0000 7fc67eb2a640 1 -- 10.20.193.28:0/1359964985 >> 10.20.193.28:0/1359964985 conn(0x7fc6780ed130 msgr2=0x7fc67810d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:33.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.094+0000 7fc67eb2a640 1 -- 10.20.193.28:0/1359964985 shutdown_connections 2026-02-25T11:47:33.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.095+0000 7fc67eb2a640 1 -- 10.20.193.28:0/1359964985 wait complete. 2026-02-25T11:47:33.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.096+0000 7fc67eb2a640 1 Processor -- start 2026-02-25T11:47:33.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.096+0000 7fc67eb2a640 1 -- start start 2026-02-25T11:47:33.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.096+0000 7fc67eb2a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc67810f840 0x7fc6780f5b10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:33.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.096+0000 7fc67eb2a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc678110230 0x7fc678176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:33.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.096+0000 7fc67eb2a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc678115a30 0x7fc6780f6210 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:33.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.096+0000 7fc67eb2a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc6780fb420 con 0x7fc67810f840 2026-02-25T11:47:33.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.096+0000 7fc67eb2a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc6780fb590 con 0x7fc678115a30 2026-02-25T11:47:33.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.096+0000 7fc67eb2a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc6780fb750 con 0x7fc678110230 2026-02-25T11:47:33.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.096+0000 7fc67c89f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc67810f840 0x7fc6780f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:33.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.096+0000 7fc67c89f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc67810f840 0x7fc6780f5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41144/0 (socket says 10.20.193.28:41144) 2026-02-25T11:47:33.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.096+0000 7fc67c89f640 1 -- 10.20.193.28:0/2333056347 learned_addr learned my addr 10.20.193.28:0/2333056347 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:33.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.096+0000 7fc67d0a0640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc678115a30 0x7fc6780f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:33.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.097+0000 7fc67c89f640 1 -- 10.20.193.28:0/2333056347 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc678110230 msgr2=0x7fc678176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:33.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.097+0000 7fc66ffff640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc678110230 0x7fc678176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:33.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.097+0000 7fc67c89f640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc678110230 0x7fc678176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.097+0000 7fc67c89f640 1 -- 10.20.193.28:0/2333056347 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc678115a30 msgr2=0x7fc6780f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:33.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.097+0000 7fc67c89f640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc678115a30 0x7fc6780f6210 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.097+0000 7fc67c89f640 1 -- 10.20.193.28:0/2333056347 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc660009570 con 0x7fc67810f840 2026-02-25T11:47:33.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.097+0000 7fc66ffff640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc678110230 0x7fc678176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:33.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.097+0000 7fc67c89f640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc67810f840 0x7fc6780f5b10 secure :-1 s=READY pgs=291 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc6680072a0 tx=0x7fc66800ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:33.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.097+0000 7fc66dffb640 1 -- 10.20.193.28:0/2333056347 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc668004500 con 0x7fc67810f840 2026-02-25T11:47:33.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.097+0000 7fc66dffb640 1 -- 10.20.193.28:0/2333056347 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc66800bd00 con 0x7fc67810f840 2026-02-25T11:47:33.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.097+0000 7fc66dffb640 1 -- 10.20.193.28:0/2333056347 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc6680106e0 con 0x7fc67810f840 2026-02-25T11:47:33.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.097+0000 7fc67eb2a640 1 -- 10.20.193.28:0/2333056347 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6780fba30 con 0x7fc67810f840 2026-02-25T11:47:33.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.097+0000 7fc67eb2a640 1 -- 10.20.193.28:0/2333056347 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6780fbf50 con 0x7fc67810f840 2026-02-25T11:47:33.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.098+0000 7fc67eb2a640 1 -- 10.20.193.28:0/2333056347 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc678110b20 con 0x7fc67810f840 2026-02-25T11:47:33.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.100+0000 7fc66dffb640 1 -- 10.20.193.28:0/2333056347 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fc66800b840 con 0x7fc67810f840 2026-02-25T11:47:33.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.102+0000 7fc66dffb640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc6540b57e0 0x7fc6540d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:33.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.102+0000 7fc66dffb640 1 -- 10.20.193.28:0/2333056347 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fc6680e6c10 con 0x7fc67810f840 2026-02-25T11:47:33.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.102+0000 7fc66dffb640 1 -- 10.20.193.28:0/2333056347 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fc6680e7090 con 0x7fc67810f840 2026-02-25T11:47:33.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.102+0000 7fc66ffff640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc6540b57e0 0x7fc6540d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:33.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.103+0000 7fc66ffff640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc6540b57e0 0x7fc6540d5c90 secure :-1 s=READY pgs=221 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc6600098a0 tx=0x7fc660031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:33.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.245+0000 7fc67eb2a640 1 -- 10.20.193.28:0/2333056347 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc67810fc10 con 0x7fc67810f840 2026-02-25T11:47:33.249 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:33 trial028 ceph-mon[39224]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:33.249 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:33 trial028 ceph-mon[39224]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:33.249 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:33 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1515256490' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-25T11:47:33.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.246+0000 7fc66dffb640 1 -- 10.20.193.28:0/2333056347 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11940 (secure 0 0 0) 0x7fc6680a9560 con 0x7fc67810f840 2026-02-25T11:47:33.250 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:33.250 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":22,"btime":"2026-02-25T11:47:18:567831+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:18.567827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24730,24731,24749]},"id":1}]} 2026-02-25T11:47:33.251 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 22 2026-02-25T11:47:33.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.247+0000 7fc67eb2a640 1 -- 10.20.193.28:0/2333056347 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc6540b57e0 msgr2=0x7fc6540d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:33.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.247+0000 7fc67eb2a640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc6540b57e0 0x7fc6540d5c90 secure :-1 s=READY pgs=221 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc6600098a0 tx=0x7fc660031040 comp rx=0 tx=0).stop 2026-02-25T11:47:33.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.247+0000 7fc67eb2a640 1 -- 10.20.193.28:0/2333056347 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc67810f840 msgr2=0x7fc6780f5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:33.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.247+0000 7fc67eb2a640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc67810f840 0x7fc6780f5b10 secure :-1 s=READY pgs=291 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc6680072a0 tx=0x7fc66800ddf0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.247+0000 7fc67eb2a640 1 -- 10.20.193.28:0/2333056347 shutdown_connections 2026-02-25T11:47:33.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.247+0000 7fc67eb2a640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc6540b57e0 0x7fc6540d5c90 unknown :-1 s=CLOSED pgs=221 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.247+0000 7fc67eb2a640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc678115a30 0x7fc6780f6210 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.247+0000 7fc67eb2a640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc678110230 0x7fc678176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.247+0000 7fc67eb2a640 1 --2- 10.20.193.28:0/2333056347 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc67810f840 0x7fc6780f5b10 unknown :-1 s=CLOSED pgs=291 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.247+0000 7fc67eb2a640 1 -- 10.20.193.28:0/2333056347 >> 10.20.193.28:0/2333056347 conn(0x7fc6780ed130 msgr2=0x7fc67810dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:33.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.248+0000 7fc67eb2a640 1 -- 10.20.193.28:0/2333056347 shutdown_connections 2026-02-25T11:47:33.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.248+0000 7fc67eb2a640 1 -- 10.20.193.28:0/2333056347 wait complete. 2026-02-25T11:47:33.347 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:47:33.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:33 trial167 ceph-mon[43392]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:33.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:33 trial167 ceph-mon[43392]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:33.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:33 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1515256490' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-25T11:47:33.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:33 trial119 ceph-mon[42710]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:33.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:33 trial119 ceph-mon[42710]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:33.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:33 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1515256490' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2026-02-25T11:47:33.528 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:33.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.701+0000 7ff4dc94d640 1 -- 10.20.193.28:0/4159426007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff4d810ddc0 msgr2=0x7ff4d8112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:33.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.701+0000 7ff4dc94d640 1 --2- 10.20.193.28:0/4159426007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff4d810ddc0 0x7ff4d8112e10 secure :-1 s=READY pgs=292 gs=1 cs=0 l=1 c_cookie=5865622732225820944 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff4c8009950 tx=0x7ff4c802f190 comp rx=0 tx=0).stop 2026-02-25T11:47:33.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.701+0000 7ff4dc94d640 1 -- 10.20.193.28:0/4159426007 shutdown_connections 2026-02-25T11:47:33.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.701+0000 7ff4dc94d640 1 --2- 10.20.193.28:0/4159426007 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff4d81135c0 0x7ff4d8173f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.701+0000 7ff4dc94d640 1 --2- 10.20.193.28:0/4159426007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff4d810ddc0 0x7ff4d8112e10 unknown :-1 s=CLOSED pgs=292 gs=1 cs=0 l=1 c_cookie=5865622732225820944 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.701+0000 7ff4dc94d640 1 --2- 10.20.193.28:0/4159426007 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff4d810d3d0 0x7ff4d810d7a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.701+0000 7ff4dc94d640 1 -- 10.20.193.28:0/4159426007 >> 10.20.193.28:0/4159426007 conn(0x7ff4d80eaca0 msgr2=0x7ff4d810b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:33.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.701+0000 7ff4dc94d640 1 -- 10.20.193.28:0/4159426007 shutdown_connections 2026-02-25T11:47:33.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.701+0000 7ff4dc94d640 1 -- 10.20.193.28:0/4159426007 wait complete. 2026-02-25T11:47:33.707 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.702+0000 7ff4dc94d640 1 Processor -- start 2026-02-25T11:47:33.707 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.702+0000 7ff4dc94d640 1 -- start start 2026-02-25T11:47:33.707 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.702+0000 7ff4dc94d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff4d810d3d0 0x7ff4d80f5a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:33.707 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.702+0000 7ff4dc94d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff4d810ddc0 0x7ff4d8173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:33.707 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4dc94d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff4d81135c0 0x7ff4d80f6110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:33.707 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4dc94d640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d80fb2d0 con 0x7ff4d81135c0 2026-02-25T11:47:33.707 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4dc94d640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d80fb440 con 0x7ff4d810d3d0 2026-02-25T11:47:33.707 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4dc94d640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d80fb600 con 0x7ff4d810ddc0 2026-02-25T11:47:33.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d7fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff4d81135c0 0x7ff4d80f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:33.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d7fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff4d81135c0 0x7ff4d80f6110 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41162/0 (socket says 10.20.193.28:41162) 2026-02-25T11:47:33.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d7fff640 1 -- 10.20.193.28:0/163102204 learned_addr learned my addr 10.20.193.28:0/163102204 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:33.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d6ffd640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff4d810ddc0 0x7ff4d8173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:33.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d77fe640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff4d810d3d0 0x7ff4d80f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:33.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d7fff640 1 -- 10.20.193.28:0/163102204 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff4d810ddc0 msgr2=0x7ff4d8173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:33.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d7fff640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff4d810ddc0 0x7ff4d8173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d7fff640 1 -- 10.20.193.28:0/163102204 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff4d810d3d0 msgr2=0x7ff4d80f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:33.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d7fff640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff4d810d3d0 0x7ff4d80f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d7fff640 1 -- 10.20.193.28:0/163102204 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c8009570 con 0x7ff4d81135c0 2026-02-25T11:47:33.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d7fff640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff4d81135c0 0x7ff4d80f6110 secure :-1 s=READY pgs=293 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff4cc00f8d0 tx=0x7ff4cc00fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:33.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d4ff9640 1 -- 10.20.193.28:0/163102204 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff4cc00dce0 con 0x7ff4d81135c0 2026-02-25T11:47:33.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d4ff9640 1 -- 10.20.193.28:0/163102204 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff4cc00de40 con 0x7ff4d81135c0 2026-02-25T11:47:33.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4d4ff9640 1 -- 10.20.193.28:0/163102204 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff4cc0116b0 con 0x7ff4d81135c0 2026-02-25T11:47:33.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4dc94d640 1 -- 10.20.193.28:0/163102204 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d80fb8e0 con 0x7ff4d81135c0 2026-02-25T11:47:33.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.703+0000 7ff4dc94d640 1 -- 10.20.193.28:0/163102204 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d80fbd60 con 0x7ff4d81135c0 2026-02-25T11:47:33.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.704+0000 7ff4dc94d640 1 -- 10.20.193.28:0/163102204 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d810e6b0 con 0x7ff4d81135c0 2026-02-25T11:47:33.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.706+0000 7ff4d4ff9640 1 -- 10.20.193.28:0/163102204 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7ff4cc020060 con 0x7ff4d81135c0 2026-02-25T11:47:33.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.708+0000 7ff4d4ff9640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff4ac0b57e0 0x7ff4ac0d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:33.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.708+0000 7ff4d4ff9640 1 -- 10.20.193.28:0/163102204 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7ff4cc015070 con 0x7ff4d81135c0 2026-02-25T11:47:33.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.708+0000 7ff4d4ff9640 1 -- 10.20.193.28:0/163102204 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7ff4cc120940 con 0x7ff4d81135c0 2026-02-25T11:47:33.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.708+0000 7ff4d77fe640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff4ac0b57e0 0x7ff4ac0d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:33.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.709+0000 7ff4d77fe640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff4ac0b57e0 0x7ff4ac0d5c90 secure :-1 s=READY pgs=222 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff4c4004740 tx=0x7ff4c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:33.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.848+0000 7ff4dc94d640 1 -- 10.20.193.28:0/163102204 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4d810d7a0 con 0x7ff4d81135c0 2026-02-25T11:47:33.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.849+0000 7ff4d4ff9640 1 -- 10.20.193.28:0/163102204 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11940 (secure 0 0 0) 0x7ff4cc0aa6d0 con 0x7ff4d81135c0 2026-02-25T11:47:33.853 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:33.853 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":22,"btime":"2026-02-25T11:47:18:567831+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:18.567827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24730,24731,24749]},"id":1}]} 2026-02-25T11:47:33.853 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 22 2026-02-25T11:47:33.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.850+0000 7ff4dc94d640 1 -- 10.20.193.28:0/163102204 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff4ac0b57e0 msgr2=0x7ff4ac0d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:33.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.850+0000 7ff4dc94d640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff4ac0b57e0 0x7ff4ac0d5c90 secure :-1 s=READY pgs=222 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff4c4004740 tx=0x7ff4c4005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:33.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.850+0000 7ff4dc94d640 1 -- 10.20.193.28:0/163102204 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff4d81135c0 msgr2=0x7ff4d80f6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:33.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.850+0000 7ff4dc94d640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff4d81135c0 0x7ff4d80f6110 secure :-1 s=READY pgs=293 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff4cc00f8d0 tx=0x7ff4cc00fda0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.850+0000 7ff4dc94d640 1 -- 10.20.193.28:0/163102204 shutdown_connections 2026-02-25T11:47:33.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.850+0000 7ff4dc94d640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff4ac0b57e0 0x7ff4ac0d5c90 unknown :-1 s=CLOSED pgs=222 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.850+0000 7ff4dc94d640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff4d81135c0 0x7ff4d80f6110 unknown :-1 s=CLOSED pgs=293 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.850+0000 7ff4dc94d640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff4d810ddc0 0x7ff4d8173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.850+0000 7ff4dc94d640 1 --2- 10.20.193.28:0/163102204 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff4d810d3d0 0x7ff4d80f5a10 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:33.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.850+0000 7ff4dc94d640 1 -- 10.20.193.28:0/163102204 >> 10.20.193.28:0/163102204 conn(0x7ff4d80eaca0 msgr2=0x7ff4d810fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:33.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.851+0000 7ff4dc94d640 1 -- 10.20.193.28:0/163102204 shutdown_connections 2026-02-25T11:47:33.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:33.851+0000 7ff4dc94d640 1 -- 10.20.193.28:0/163102204 wait complete. 2026-02-25T11:47:33.952 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd dump --format=json 2026-02-25T11:47:34.134 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:34.142 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:34 trial028 ceph-mon[39224]: pgmap v204: 97 pgs: 97 active+clean; 641 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 9.0 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2026-02-25T11:47:34.142 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:34 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2333056347' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:34.142 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:34 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/163102204' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:34.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.308+0000 7f558676a640 1 -- 10.20.193.28:0/3802070386 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5580060110 msgr2=0x7f55800604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:34.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.308+0000 7f558676a640 1 --2- 10.20.193.28:0/3802070386 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5580060110 0x7f55800604e0 secure :-1 s=READY pgs=294 gs=1 cs=0 l=1 c_cookie=18180012149975759447 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5570009950 tx=0x7f557002f190 comp rx=0 tx=0).stop 2026-02-25T11:47:34.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.308+0000 7f558676a640 1 -- 10.20.193.28:0/3802070386 shutdown_connections 2026-02-25T11:47:34.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.308+0000 7f558676a640 1 --2- 10.20.193.28:0/3802070386 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f55801135c0 0x7f5580173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.308+0000 7f558676a640 1 --2- 10.20.193.28:0/3802070386 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5580060b00 0x7f5580112e10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.308+0000 7f558676a640 1 --2- 10.20.193.28:0/3802070386 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5580060110 0x7f55800604e0 unknown :-1 s=CLOSED pgs=294 gs=1 cs=0 l=1 c_cookie=18180012149975759447 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.308+0000 7f558676a640 1 -- 10.20.193.28:0/3802070386 >> 10.20.193.28:0/3802070386 conn(0x7f55800e9b40 msgr2=0x7f5580109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:34.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.308+0000 7f558676a640 1 -- 10.20.193.28:0/3802070386 shutdown_connections 2026-02-25T11:47:34.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.308+0000 7f558676a640 1 -- 10.20.193.28:0/3802070386 wait complete. 2026-02-25T11:47:34.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.309+0000 7f558676a640 1 Processor -- start 2026-02-25T11:47:34.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.309+0000 7f558676a640 1 -- start start 2026-02-25T11:47:34.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.309+0000 7f558676a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5580060110 0x7f55800f4880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:34.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.309+0000 7f558676a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5580060b00 0x7f5580173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:34.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.309+0000 7f558676a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f55801135c0 0x7f55800f4f80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:34.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.309+0000 7f558676a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f55800fa140 con 0x7f55801135c0 2026-02-25T11:47:34.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.309+0000 7f558676a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f55800fa2b0 con 0x7f5580060110 2026-02-25T11:47:34.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.309+0000 7f558676a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f55800fa470 con 0x7f5580060b00 2026-02-25T11:47:34.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.309+0000 7f5585768640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5580060110 0x7f55800f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:34.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.309+0000 7f5585768640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5580060110 0x7f55800f4880 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:42220/0 (socket says 10.20.193.28:42220) 2026-02-25T11:47:34.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.309+0000 7f5585768640 1 -- 10.20.193.28:0/3723096641 learned_addr learned my addr 10.20.193.28:0/3723096641 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:34.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.309+0000 7f5584f67640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5580060b00 0x7f5580173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:34.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.309+0000 7f5585f69640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f55801135c0 0x7f55800f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:34.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.310+0000 7f5584f67640 1 -- 10.20.193.28:0/3723096641 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5580060110 msgr2=0x7f55800f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:34.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.310+0000 7f5584f67640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5580060110 0x7f55800f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.310+0000 7f5584f67640 1 -- 10.20.193.28:0/3723096641 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f55801135c0 msgr2=0x7f55800f4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:34.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.310+0000 7f5584f67640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f55801135c0 0x7f55800f4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.310+0000 7f5584f67640 1 -- 10.20.193.28:0/3723096641 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5570009570 con 0x7f5580060b00 2026-02-25T11:47:34.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.310+0000 7f5585f69640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f55801135c0 0x7f55800f4f80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:34.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.310+0000 7f5585768640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5580060110 0x7f55800f4880 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:34.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.310+0000 7f5584f67640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5580060b00 0x7f5580173e90 secure :-1 s=READY pgs=97 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f557400d720 tx=0x7f557400dbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:34.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.310+0000 7f556e7fc640 1 -- 10.20.193.28:0/3723096641 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5574004500 con 0x7f5580060b00 2026-02-25T11:47:34.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.310+0000 7f558676a640 1 -- 10.20.193.28:0/3723096641 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55800fa750 con 0x7f5580060b00 2026-02-25T11:47:34.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.310+0000 7f556e7fc640 1 -- 10.20.193.28:0/3723096641 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5574004d10 con 0x7f5580060b00 2026-02-25T11:47:34.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.310+0000 7f556e7fc640 1 -- 10.20.193.28:0/3723096641 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f55740052a0 con 0x7f5580060b00 2026-02-25T11:47:34.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.310+0000 7f558676a640 1 -- 10.20.193.28:0/3723096641 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55800fac70 con 0x7f5580060b00 2026-02-25T11:47:34.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.311+0000 7f558676a640 1 -- 10.20.193.28:0/3723096641 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558010e6b0 con 0x7f5580060b00 2026-02-25T11:47:34.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.313+0000 7f556e7fc640 1 -- 10.20.193.28:0/3723096641 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f5574005430 con 0x7f5580060b00 2026-02-25T11:47:34.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.315+0000 7f556e7fc640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f555c0b57e0 0x7f555c0d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:34.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.315+0000 7f556e7fc640 1 -- 10.20.193.28:0/3723096641 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f55740e69f0 con 0x7f5580060b00 2026-02-25T11:47:34.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.315+0000 7f5585768640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f555c0b57e0 0x7f555c0d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:34.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.315+0000 7f556e7fc640 1 -- 10.20.193.28:0/3723096641 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f557411f940 con 0x7f5580060b00 2026-02-25T11:47:34.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.316+0000 7f5585768640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f555c0b57e0 0x7f555c0d5c90 secure :-1 s=READY pgs=223 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f55700023b0 tx=0x7f557003a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:34.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.431+0000 7f558676a640 1 -- 10.20.193.28:0/3723096641 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55800604e0 con 0x7f5580060b00 2026-02-25T11:47:34.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.432+0000 7f556e7fc640 1 -- 10.20.193.28:0/3723096641 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22340 (secure 0 0 0) 0x7f55740a93f0 con 0x7f5580060b00 2026-02-25T11:47:34.435 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:34.435 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":86,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","created":"2026-02-25T11:42:51.962844+0000","modified":"2026-02-25T11:47:14.624486+0000","last_up_change":"2026-02-25T11:46:45.156629+0000","last_in_change":"2026-02-25T11:46:31.585002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":38,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T11:44:43.266336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-25T11:47:11.236575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-25T11:47:12.196775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf596df7-d1fb-4d10-8d4c-14a35210bab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6803","nonce":668944205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6805","nonce":668944205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6809","nonce":668944205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6807","nonce":668944205}]},"public_addr":"10.20.193.28:6803/668944205","cluster_addr":"10.20.193.28:6805/668944205","heartbeat_back_addr":"10.20.193.28:6809/668944205","heartbeat_front_addr":"10.20.193.28:6807/668944205","state":["exists","up"]},{"osd":1,"uuid":"185719c0-7c2d-431f-9925-88bbee86a98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6800","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6801","nonce":3486566242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6802","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6803","nonce":3486566242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6806","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6807","nonce":3486566242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6804","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6805","nonce":3486566242}]},"public_addr":"10.20.193.119:6801/3486566242","cluster_addr":"10.20.193.119:6803/3486566242","heartbeat_back_addr":"10.20.193.119:6807/3486566242","heartbeat_front_addr":"10.20.193.119:6805/3486566242","state":["exists","up"]},{"osd":2,"uuid":"4de13b90-1c3c-4614-9fef-2d7315626433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6800","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6801","nonce":484781559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6802","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6803","nonce":484781559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6806","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6807","nonce":484781559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6804","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6805","nonce":484781559}]},"public_addr":"10.20.193.167:6801/484781559","cluster_addr":"10.20.193.167:6803/484781559","heartbeat_back_addr":"10.20.193.167:6807/484781559","heartbeat_front_addr":"10.20.193.167:6805/484781559","state":["exists","up"]},{"osd":3,"uuid":"45d9ea8e-5a77-4738-96da-ada98515fc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6811","nonce":2452541436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6813","nonce":2452541436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6817","nonce":2452541436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6815","nonce":2452541436}]},"public_addr":"10.20.193.28:6811/2452541436","cluster_addr":"10.20.193.28:6813/2452541436","heartbeat_back_addr":"10.20.193.28:6817/2452541436","heartbeat_front_addr":"10.20.193.28:6815/2452541436","state":["exists","up"]},{"osd":4,"uuid":"c9c3f512-665b-44b2-a6f0-f3356a4b661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6808","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6809","nonce":135971996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6810","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6811","nonce":135971996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6814","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6815","nonce":135971996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6812","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6813","nonce":135971996}]},"public_addr":"10.20.193.119:6809/135971996","cluster_addr":"10.20.193.119:6811/135971996","heartbeat_back_addr":"10.20.193.119:6815/135971996","heartbeat_front_addr":"10.20.193.119:6813/135971996","state":["exists","up"]},{"osd":5,"uuid":"777c3819-5c90-4b2b-a545-c024884c2eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6808","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6809","nonce":4203420884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6810","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6811","nonce":4203420884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6814","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6815","nonce":4203420884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6812","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6813","nonce":4203420884}]},"public_addr":"10.20.193.167:6809/4203420884","cluster_addr":"10.20.193.167:6811/4203420884","heartbeat_back_addr":"10.20.193.167:6815/4203420884","heartbeat_front_addr":"10.20.193.167:6813/4203420884","state":["exists","up"]},{"osd":6,"uuid":"bbd9de64-b9fe-499e-b451-c16828be2c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6819","nonce":621484999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6821","nonce":621484999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6825","nonce":621484999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6823","nonce":621484999}]},"public_addr":"10.20.193.28:6819/621484999","cluster_addr":"10.20.193.28:6821/621484999","heartbeat_back_addr":"10.20.193.28:6825/621484999","heartbeat_front_addr":"10.20.193.28:6823/621484999","state":["exists","up"]},{"osd":7,"uuid":"546d9f28-597b-454c-8456-fcc2d46c69f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6816","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6817","nonce":2724541747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6818","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6819","nonce":2724541747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6822","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6823","nonce":2724541747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6820","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6821","nonce":2724541747}]},"public_addr":"10.20.193.119:6817/2724541747","cluster_addr":"10.20.193.119:6819/2724541747","heartbeat_back_addr":"10.20.193.119:6823/2724541747","heartbeat_front_addr":"10.20.193.119:6821/2724541747","state":["exists","up"]},{"osd":8,"uuid":"a1a7cea6-3586-4bb2-aa4c-bf6f7799a501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6816","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6817","nonce":1328734317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6818","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6819","nonce":1328734317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6822","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6823","nonce":1328734317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6820","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6821","nonce":1328734317}]},"public_addr":"10.20.193.167:6817/1328734317","cluster_addr":"10.20.193.167:6819/1328734317","heartbeat_back_addr":"10.20.193.167:6823/1328734317","heartbeat_front_addr":"10.20.193.167:6821/1328734317","state":["exists","up"]},{"osd":9,"uuid":"03fef134-6946-43d5-9c26-dc0e59fffe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6827","nonce":354500625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6829","nonce":354500625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6833","nonce":354500625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6831","nonce":354500625}]},"public_addr":"10.20.193.28:6827/354500625","cluster_addr":"10.20.193.28:6829/354500625","heartbeat_back_addr":"10.20.193.28:6833/354500625","heartbeat_front_addr":"10.20.193.28:6831/354500625","state":["exists","up"]},{"osd":10,"uuid":"b46c29ef-285c-4184-be28-d99b76bc0e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6824","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6825","nonce":2996581116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6826","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6827","nonce":2996581116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6830","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6831","nonce":2996581116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6828","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6829","nonce":2996581116}]},"public_addr":"10.20.193.119:6825/2996581116","cluster_addr":"10.20.193.119:6827/2996581116","heartbeat_back_addr":"10.20.193.119:6831/2996581116","heartbeat_front_addr":"10.20.193.119:6829/2996581116","state":["exists","up"]},{"osd":11,"uuid":"f6696d8c-5de7-4fd9-b693-524f204e4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6824","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6825","nonce":2182906700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6826","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6827","nonce":2182906700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6830","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6831","nonce":2182906700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6828","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6829","nonce":2182906700}]},"public_addr":"10.20.193.167:6825/2182906700","cluster_addr":"10.20.193.167:6827/2182906700","heartbeat_back_addr":"10.20.193.167:6831/2182906700","heartbeat_front_addr":"10.20.193.167:6829/2182906700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:13.410235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:27.254773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:40.956615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:54.529866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:08.306772+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:21.899498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:35.500407+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:49.094965+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:02.790138+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:16.286937+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:29.922722+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:43.396028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.28:0/1467902196":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6800/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/2274270913":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/60087422":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:6800/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/1725332841":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/3082630924":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/2391715078":"2026-02-26T11:43:04.102029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T11:47:34.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.433+0000 7f558676a640 1 -- 10.20.193.28:0/3723096641 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f555c0b57e0 msgr2=0x7f555c0d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:34.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.433+0000 7f558676a640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f555c0b57e0 0x7f555c0d5c90 secure :-1 s=READY pgs=223 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f55700023b0 tx=0x7f557003a040 comp rx=0 tx=0).stop 2026-02-25T11:47:34.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.433+0000 7f558676a640 1 -- 10.20.193.28:0/3723096641 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5580060b00 msgr2=0x7f5580173e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:34.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.433+0000 7f558676a640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5580060b00 0x7f5580173e90 secure :-1 s=READY pgs=97 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f557400d720 tx=0x7f557400dbf0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.433+0000 7f558676a640 1 -- 10.20.193.28:0/3723096641 shutdown_connections 2026-02-25T11:47:34.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.433+0000 7f558676a640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f555c0b57e0 0x7f555c0d5c90 unknown :-1 s=CLOSED pgs=223 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.433+0000 7f558676a640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f55801135c0 0x7f55800f4f80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.433+0000 7f558676a640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5580060b00 0x7f5580173e90 unknown :-1 s=CLOSED pgs=97 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.433+0000 7f558676a640 1 --2- 10.20.193.28:0/3723096641 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5580060110 0x7f55800f4880 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.433+0000 7f558676a640 1 -- 10.20.193.28:0/3723096641 >> 10.20.193.28:0/3723096641 conn(0x7f55800e9b40 msgr2=0x7f55801105c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:34.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.433+0000 7f558676a640 1 -- 10.20.193.28:0/3723096641 shutdown_connections 2026-02-25T11:47:34.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.434+0000 7f558676a640 1 -- 10.20.193.28:0/3723096641 wait complete. 2026-02-25T11:47:34.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:34 trial167 ceph-mon[43392]: pgmap v204: 97 pgs: 97 active+clean; 641 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 9.0 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2026-02-25T11:47:34.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:34 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2333056347' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:34.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:34 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/163102204' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:34.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:34 trial119 ceph-mon[42710]: pgmap v204: 97 pgs: 97 active+clean; 641 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 9.0 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2026-02-25T11:47:34.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:34 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2333056347' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:34.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:34 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/163102204' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:34.538 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:47:34.721 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:34.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.891+0000 7f4b9507b640 1 -- 10.20.193.28:0/2971929309 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4b90115a30 msgr2=0x7f4b90176400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:34.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.891+0000 7f4b9507b640 1 --2- 10.20.193.28:0/2971929309 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4b90115a30 0x7f4b90176400 secure :-1 s=READY pgs=295 gs=3 cs=0 l=1 c_cookie=12000246705602033825 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4b8000aa30 tx=0x7f4b80030270 comp rx=0 tx=0).stop 2026-02-25T11:47:34.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.891+0000 7f4b9507b640 1 -- 10.20.193.28:0/2971929309 shutdown_connections 2026-02-25T11:47:34.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.891+0000 7f4b9507b640 1 --2- 10.20.193.28:0/2971929309 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4b90115a30 0x7f4b90176400 unknown :-1 s=CLOSED pgs=295 gs=3 cs=0 l=1 c_cookie=12000246705602033825 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.891+0000 7f4b9507b640 1 --2- 10.20.193.28:0/2971929309 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4b90110230 0x7f4b90115280 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.891+0000 7f4b9507b640 1 --2- 10.20.193.28:0/2971929309 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4b9010f840 0x7f4b9010fc10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.891+0000 7f4b9507b640 1 -- 10.20.193.28:0/2971929309 >> 10.20.193.28:0/2971929309 conn(0x7f4b900ed130 msgr2=0x7f4b9010d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:34.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.891+0000 7f4b9507b640 1 -- 10.20.193.28:0/2971929309 shutdown_connections 2026-02-25T11:47:34.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.891+0000 7f4b9507b640 1 -- 10.20.193.28:0/2971929309 wait complete. 2026-02-25T11:47:34.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.892+0000 7f4b9507b640 1 Processor -- start 2026-02-25T11:47:34.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.892+0000 7f4b9507b640 1 -- start start 2026-02-25T11:47:34.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.892+0000 7f4b9507b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4b9010f840 0x7f4b900f5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:34.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.892+0000 7f4b9507b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4b90110230 0x7f4b90176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:34.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.892+0000 7f4b9507b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4b90115a30 0x7f4b900f6230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:34.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.892+0000 7f4b9507b640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4b900fb3f0 con 0x7f4b9010f840 2026-02-25T11:47:34.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.892+0000 7f4b9507b640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4b900fb560 con 0x7f4b90110230 2026-02-25T11:47:34.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.892+0000 7f4b9507b640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b900fb720 con 0x7f4b90115a30 2026-02-25T11:47:34.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.892+0000 7f4b8ed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4b9010f840 0x7f4b900f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:34.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.892+0000 7f4b8ed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4b9010f840 0x7f4b900f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41200/0 (socket says 10.20.193.28:41200) 2026-02-25T11:47:34.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.892+0000 7f4b8ed76640 1 -- 10.20.193.28:0/304696380 learned_addr learned my addr 10.20.193.28:0/304696380 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:34.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.892+0000 7f4b8e575640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4b90110230 0x7f4b90176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:34.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.893+0000 7f4b8f577640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4b90115a30 0x7f4b900f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:34.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.893+0000 7f4b8ed76640 1 -- 10.20.193.28:0/304696380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4b90115a30 msgr2=0x7f4b900f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:34.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.893+0000 7f4b8ed76640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4b90115a30 0x7f4b900f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.893+0000 7f4b8ed76640 1 -- 10.20.193.28:0/304696380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4b90110230 msgr2=0x7f4b90176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:34.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.893+0000 7f4b8ed76640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4b90110230 0x7f4b90176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:34.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.893+0000 7f4b8ed76640 1 -- 10.20.193.28:0/304696380 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8000a650 con 0x7f4b9010f840 2026-02-25T11:47:34.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.893+0000 7f4b8e575640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4b90110230 0x7f4b90176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:34.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.893+0000 7f4b8ed76640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4b9010f840 0x7f4b900f5b30 secure :-1 s=READY pgs=296 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4b84009480 tx=0x7f4b8400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:34.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.893+0000 7f4b6ffff640 1 -- 10.20.193.28:0/304696380 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4b8400cd40 con 0x7f4b9010f840 2026-02-25T11:47:34.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.893+0000 7f4b6ffff640 1 -- 10.20.193.28:0/304696380 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4b84004d10 con 0x7f4b9010f840 2026-02-25T11:47:34.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.893+0000 7f4b9507b640 1 -- 10.20.193.28:0/304696380 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b900fba00 con 0x7f4b9010f840 2026-02-25T11:47:34.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.893+0000 7f4b6ffff640 1 -- 10.20.193.28:0/304696380 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4b840106b0 con 0x7f4b9010f840 2026-02-25T11:47:34.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.893+0000 7f4b9507b640 1 -- 10.20.193.28:0/304696380 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b900fbf20 con 0x7f4b9010f840 2026-02-25T11:47:34.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.894+0000 7f4b9507b640 1 -- 10.20.193.28:0/304696380 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b90110b20 con 0x7f4b9010f840 2026-02-25T11:47:34.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.896+0000 7f4b6ffff640 1 -- 10.20.193.28:0/304696380 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f4b8401f060 con 0x7f4b9010f840 2026-02-25T11:47:34.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.897+0000 7f4b6ffff640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4b640b57e0 0x7f4b640d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:34.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.898+0000 7f4b6ffff640 1 -- 10.20.193.28:0/304696380 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f4b84014070 con 0x7f4b9010f840 2026-02-25T11:47:34.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.898+0000 7f4b8e575640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4b640b57e0 0x7f4b640d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:34.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.898+0000 7f4b6ffff640 1 -- 10.20.193.28:0/304696380 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f4b8411f940 con 0x7f4b9010f840 2026-02-25T11:47:34.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:34.898+0000 7f4b8e575640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4b640b57e0 0x7f4b640d5c90 secure :-1 s=READY pgs=224 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4b900fc480 tx=0x7f4b78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:35.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.041+0000 7f4b9507b640 1 -- 10.20.193.28:0/304696380 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b9010fc10 con 0x7f4b9010f840 2026-02-25T11:47:35.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.041+0000 7f4b6ffff640 1 -- 10.20.193.28:0/304696380 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11940 (secure 0 0 0) 0x7f4b840a9630 con 0x7f4b9010f840 2026-02-25T11:47:35.045 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:35.045 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":22,"btime":"2026-02-25T11:47:18:567831+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:18.567827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24730,24731,24749]},"id":1}]} 2026-02-25T11:47:35.046 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 22 2026-02-25T11:47:35.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.043+0000 7f4b9507b640 1 -- 10.20.193.28:0/304696380 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4b640b57e0 msgr2=0x7f4b640d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.043+0000 7f4b9507b640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4b640b57e0 0x7f4b640d5c90 secure :-1 s=READY pgs=224 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4b900fc480 tx=0x7f4b78005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:35.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.043+0000 7f4b9507b640 1 -- 10.20.193.28:0/304696380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4b9010f840 msgr2=0x7f4b900f5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.043+0000 7f4b9507b640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4b9010f840 0x7f4b900f5b30 secure :-1 s=READY pgs=296 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4b84009480 tx=0x7f4b8400ee60 comp rx=0 tx=0).stop 2026-02-25T11:47:35.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.043+0000 7f4b9507b640 1 -- 10.20.193.28:0/304696380 shutdown_connections 2026-02-25T11:47:35.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.043+0000 7f4b9507b640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4b640b57e0 0x7f4b640d5c90 unknown :-1 s=CLOSED pgs=224 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.043+0000 7f4b9507b640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4b90115a30 0x7f4b900f6230 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.043+0000 7f4b9507b640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4b90110230 0x7f4b90176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.043+0000 7f4b9507b640 1 --2- 10.20.193.28:0/304696380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4b9010f840 0x7f4b900f5b30 unknown :-1 s=CLOSED pgs=296 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.043+0000 7f4b9507b640 1 -- 10.20.193.28:0/304696380 >> 10.20.193.28:0/304696380 conn(0x7f4b900ed130 msgr2=0x7f4b9010dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:35.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.043+0000 7f4b9507b640 1 -- 10.20.193.28:0/304696380 shutdown_connections 2026-02-25T11:47:35.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.043+0000 7f4b9507b640 1 -- 10.20.193.28:0/304696380 wait complete. 2026-02-25T11:47:35.146 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False, 'allow_referent_inodes': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2026-02-25T11:47:13.223613+0000', 'modified': '2026-02-25T11:47:18.567827+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24749, 'mds_1': 24731, 'mds_2': 24730}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24706': {'gid': 24706, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '10.20.193.167:6835/3681115733', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '10.20.193.167:6834', 'nonce': 3681115733}, {'type': 'v1', 'addr': '10.20.193.167:6835', 'nonce': 3681115733}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4544132024016699391, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24707': {'gid': 24707, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '10.20.193.167:6837/3206735083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '10.20.193.167:6836', 'nonce': 3206735083}, {'type': 'v1', 'addr': '10.20.193.167:6837', 'nonce': 3206735083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4544132024016699391, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24719': {'gid': 24719, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '10.20.193.119:6837/392187157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '10.20.193.119:6836', 'nonce': 392187157}, {'type': 'v1', 'addr': '10.20.193.119:6837', 'nonce': 392187157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4544132024016699391, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24730': {'gid': 24730, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 5, 'addr': '10.20.193.167:6839/408350258', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '10.20.193.167:6838', 'nonce': 408350258}, {'type': 'v1', 'addr': '10.20.193.167:6839', 'nonce': 408350258}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4544132024016699391, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24731': {'gid': 24731, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '10.20.193.28:6839/3137382094', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '10.20.193.28:6838', 'nonce': 3137382094}, {'type': 'v1', 'addr': '10.20.193.28:6839', 'nonce': 3137382094}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4544132024016699391, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24749': {'gid': 24749, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '10.20.193.119:6839/1063465486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '10.20.193.119:6838', 'nonce': 1063465486}, {'type': 'v1', 'addr': '10.20.193.119:6839', 'nonce': 1063465486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4544132024016699391, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24749, 'qdb_cluster': [24730, 24731, 24749]} 2026-02-25T11:47:35.146 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2026-02-25T11:47:35.147 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.i status 2026-02-25T11:47:35.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:35 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3723096641' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:35.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:35 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/304696380' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:35.331 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:35.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:35 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3723096641' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:35.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:35 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/304696380' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:35.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.505+0000 7ff8cb8ec640 1 -- 10.20.193.28:0/3434835615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8c4115a30 msgr2=0x7ff8c4176400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.505+0000 7ff8cb8ec640 1 --2- 10.20.193.28:0/3434835615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8c4115a30 0x7ff8c4176400 secure :-1 s=READY pgs=297 gs=3 cs=0 l=1 c_cookie=2515110392453771837 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8c000aa30 tx=0x7ff8c0030280 comp rx=0 tx=0).stop 2026-02-25T11:47:35.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.506+0000 7ff8cb8ec640 1 -- 10.20.193.28:0/3434835615 shutdown_connections 2026-02-25T11:47:35.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.506+0000 7ff8cb8ec640 1 --2- 10.20.193.28:0/3434835615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8c4115a30 0x7ff8c4176400 unknown :-1 s=CLOSED pgs=297 gs=3 cs=0 l=1 c_cookie=2515110392453771837 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.506+0000 7ff8cb8ec640 1 --2- 10.20.193.28:0/3434835615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff8c4110230 0x7ff8c4115280 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.506+0000 7ff8cb8ec640 1 --2- 10.20.193.28:0/3434835615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff8c410f840 0x7ff8c410fc10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.506+0000 7ff8cb8ec640 1 -- 10.20.193.28:0/3434835615 >> 10.20.193.28:0/3434835615 conn(0x7ff8c40ed130 msgr2=0x7ff8c410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:35.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.506+0000 7ff8cb8ec640 1 -- 10.20.193.28:0/3434835615 shutdown_connections 2026-02-25T11:47:35.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.506+0000 7ff8cb8ec640 1 -- 10.20.193.28:0/3434835615 wait complete. 2026-02-25T11:47:35.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.506+0000 7ff8cb8ec640 1 Processor -- start 2026-02-25T11:47:35.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:35 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3723096641' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:35.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:35 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/304696380' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:35.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8cb8ec640 1 -- start start 2026-02-25T11:47:35.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8cb8ec640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff8c410f840 0x7ff8c40f5ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8cb8ec640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff8c4110230 0x7ff8c4176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8cb8ec640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8c4115a30 0x7ff8c40f62a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8cb8ec640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c40fb4b0 con 0x7ff8c4115a30 2026-02-25T11:47:35.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8cb8ec640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c40fb620 con 0x7ff8c4110230 2026-02-25T11:47:35.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8cb8ec640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c40fb7e0 con 0x7ff8c410f840 2026-02-25T11:47:35.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8c9e62640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8c4115a30 0x7ff8c40f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8c9e62640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8c4115a30 0x7ff8c40f62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41220/0 (socket says 10.20.193.28:41220) 2026-02-25T11:47:35.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8c9e62640 1 -- 10.20.193.28:0/2630603634 learned_addr learned my addr 10.20.193.28:0/2630603634 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:35.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8c9661640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff8c410f840 0x7ff8c40f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8c8e60640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff8c4110230 0x7ff8c4176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8c9e62640 1 -- 10.20.193.28:0/2630603634 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff8c410f840 msgr2=0x7ff8c40f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff8c410f840 0x7ff8c40f5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8c9e62640 1 -- 10.20.193.28:0/2630603634 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff8c4110230 msgr2=0x7ff8c4176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff8c4110230 0x7ff8c4176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8c9e62640 1 -- 10.20.193.28:0/2630603634 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c000a650 con 0x7ff8c4115a30 2026-02-25T11:47:35.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.507+0000 7ff8c9661640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff8c410f840 0x7ff8c40f5ba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:35.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.508+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8c4115a30 0x7ff8c40f62a0 secure :-1 s=READY pgs=298 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8c0039820 tx=0x7ff8c0039b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:35.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.508+0000 7ff8b27fc640 1 -- 10.20.193.28:0/2630603634 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff8c003e070 con 0x7ff8c4115a30 2026-02-25T11:47:35.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.508+0000 7ff8b27fc640 1 -- 10.20.193.28:0/2630603634 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff8c0003780 con 0x7ff8c4115a30 2026-02-25T11:47:35.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.508+0000 7ff8cb8ec640 1 -- 10.20.193.28:0/2630603634 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c40fba60 con 0x7ff8c4115a30 2026-02-25T11:47:35.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.508+0000 7ff8b27fc640 1 -- 10.20.193.28:0/2630603634 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff8c00428c0 con 0x7ff8c4115a30 2026-02-25T11:47:35.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.508+0000 7ff8cb8ec640 1 -- 10.20.193.28:0/2630603634 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c40fbed0 con 0x7ff8c4115a30 2026-02-25T11:47:35.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.511+0000 7ff8b27fc640 1 -- 10.20.193.28:0/2630603634 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7ff8c00038f0 con 0x7ff8c4115a30 2026-02-25T11:47:35.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.513+0000 7ff8b27fc640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff8a00b57e0 0x7ff8a00d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.513+0000 7ff8b27fc640 1 -- 10.20.193.28:0/2630603634 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7ff8c010c5a0 con 0x7ff8c4115a30 2026-02-25T11:47:35.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.513+0000 7ff8c9661640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff8a00b57e0 0x7ff8a00d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.513+0000 7ff8cd8e7e00 1 Processor -- start 2026-02-25T11:47:35.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.513+0000 7ff8cd8e7e00 1 -- start start 2026-02-25T11:47:35.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.513+0000 7ff8c9661640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff8a00b57e0 0x7ff8a00d5c90 secure :-1 s=READY pgs=225 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8b4004740 tx=0x7ff8b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:35.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.513+0000 7ff8cd8e7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559189558480 0x559189558850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.513+0000 7ff8cd8e7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558d90 0x55918959d4f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.513+0000 7ff8cd8e7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55918959da30 0x5591895bde10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.513+0000 7ff8cd8e7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55918946ea30 con 0x559189558480 2026-02-25T11:47:35.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.513+0000 7ff8cd8e7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55918946a760 con 0x55918959da30 2026-02-25T11:47:35.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.513+0000 7ff8cd8e7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559189464700 con 0x559189558d90 2026-02-25T11:47:35.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.514+0000 7ff8c9661640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559189558480 0x559189558850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.514+0000 7ff8c9661640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559189558480 0x559189558850 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41222/0 (socket says 10.20.193.28:41222) 2026-02-25T11:47:35.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.514+0000 7ff8c9661640 1 -- 10.20.193.28:0/1964206563 learned_addr learned my addr 10.20.193.28:0/1964206563 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:35.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.514+0000 7ff8c8e60640 1 --2- 10.20.193.28:0/1964206563 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558d90 0x55918959d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.514+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/1964206563 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55918959da30 0x5591895bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.514+0000 7ff8c8e60640 1 -- 10.20.193.28:0/1964206563 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55918959da30 msgr2=0x5591895bde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:35.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.514+0000 7ff8c8e60640 1 --2- 10.20.193.28:0/1964206563 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55918959da30 0x5591895bde10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.514+0000 7ff8c8e60640 1 -- 10.20.193.28:0/1964206563 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559189558480 msgr2=0x559189558850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:35.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.514+0000 7ff8c8e60640 1 --2- 10.20.193.28:0/1964206563 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559189558480 0x559189558850 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.514+0000 7ff8c8e60640 1 -- 10.20.193.28:0/1964206563 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55918940fcf0 con 0x559189558d90 2026-02-25T11:47:35.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.514+0000 7ff8c9661640 1 --2- 10.20.193.28:0/1964206563 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559189558480 0x559189558850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:35.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.514+0000 7ff8c8e60640 1 --2- 10.20.193.28:0/1964206563 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558d90 0x55918959d4f0 secure :-1 s=READY pgs=98 gs=1 cs=0 l=1 c_cookie=12198281571615702348 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8b8004740 tx=0x7ff8b80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9490075ca58fd4c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:35.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.514+0000 7ff891ce5640 1 -- 10.20.193.28:0/1964206563 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff8b80040d0 con 0x559189558d90 2026-02-25T11:47:35.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.515+0000 7ff891ce5640 1 -- 10.20.193.28:0/1964206563 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff8b8004d10 con 0x559189558d90 2026-02-25T11:47:35.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.515+0000 7ff891ce5640 1 -- 10.20.193.28:0/1964206563 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff8b8002c80 con 0x559189558d90 2026-02-25T11:47:35.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.515+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/1964206563 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558d90 msgr2=0x55918959d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.515+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/1964206563 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558d90 0x55918959d4f0 secure :-1 s=READY pgs=98 gs=1 cs=0 l=1 c_cookie=12198281571615702348 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8b8004740 tx=0x7ff8b80107a0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.515+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/1964206563 shutdown_connections 2026-02-25T11:47:35.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.515+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/1964206563 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55918959da30 0x5591895bde10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.515+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/1964206563 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558d90 0x55918959d4f0 unknown :-1 s=CLOSED pgs=98 gs=1 cs=0 l=1 c_cookie=12198281571615702348 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.515+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/1964206563 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559189558480 0x559189558850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.515+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/1964206563 >> 10.20.193.28:0/1964206563 conn(0x559189478720 msgr2=0x559189556d60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:35.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.515+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/1964206563 shutdown_connections 2026-02-25T11:47:35.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.515+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/1964206563 wait complete. 2026-02-25T11:47:35.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.516+0000 7ff8cd8e7e00 1 -- start start 2026-02-25T11:47:35.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.517+0000 7ff8cd8e7e00 1 Processor -- start 2026-02-25T11:47:35.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.517+0000 7ff8cd8e7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558480 0x559189570b50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.517+0000 7ff8cd8e7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558d90 0x559189571090 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.517+0000 7ff8cd8e7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55918959da30 0x559189576080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.517+0000 7ff8cd8e7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5591895765c0 con 0x55918959da30 2026-02-25T11:47:35.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.517+0000 7ff8cd8e7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559189576730 con 0x559189558480 2026-02-25T11:47:35.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.517+0000 7ff8cd8e7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5591895768f0 con 0x559189558d90 2026-02-25T11:47:35.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.517+0000 7ff8c9e62640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55918959da30 0x559189576080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.517+0000 7ff8c9e62640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55918959da30 0x559189576080 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41232/0 (socket says 10.20.193.28:41232) 2026-02-25T11:47:35.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.517+0000 7ff8c9e62640 1 -- 10.20.193.28:0/675897549 learned_addr learned my addr 10.20.193.28:0/675897549 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:35.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.517+0000 7ff8c8e60640 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558d90 0x559189571090 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.517+0000 7ff8c9661640 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558480 0x559189570b50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff8c9e62640 1 -- 10.20.193.28:0/675897549 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558d90 msgr2=0x559189571090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:35.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558d90 0x559189571090 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff8c9e62640 1 -- 10.20.193.28:0/675897549 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558480 msgr2=0x559189570b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:35.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558480 0x559189570b50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff8c9e62640 1 -- 10.20.193.28:0/675897549 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8b80097e0 con 0x55918959da30 2026-02-25T11:47:35.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff8c8e60640 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558d90 0x559189571090 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:35.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55918959da30 0x559189576080 secure :-1 s=READY pgs=299 gs=1 cs=0 l=1 c_cookie=15672110611598114853 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8c0039c00 tx=0x7ff8c0003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97e85911cef8025 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:35.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff87effd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:47:35.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff87effd640 1 -- 10.20.193.28:0/675897549 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff8c00d3860 con 0x55918959da30 2026-02-25T11:47:35.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff87effd640 1 -- 10.20.193.28:0/675897549 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff8c00d33b0 con 0x55918959da30 2026-02-25T11:47:35.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff87effd640 1 -- 10.20.193.28:0/675897549 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff8c0097d30 con 0x55918959da30 2026-02-25T11:47:35.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/675897549 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8b4002dd0 con 0x55918959da30 2026-02-25T11:47:35.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff87effd640 1 -- 10.20.193.28:0/675897549 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7ff8c00dd8d0 con 0x55918959da30 2026-02-25T11:47:35.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff87effd640 10 client.15186 inactive, discarding osd_map(86..86 src has 1..86) 2026-02-25T11:47:35.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff8cd8e7e00 10 client.15186 _ll_register_callbacks cb 0x559189452a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:35.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/675897549 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5591891bc3b0 con 0x55918959da30 2026-02-25T11:47:35.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff87effd640 1 -- 10.20.193.28:0/675897549 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff8c00d86d0 con 0x55918959da30 2026-02-25T11:47:35.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff8cd8e7e00 10 client.15186 fetch_fsmap learned FSMap version 22 2026-02-25T11:47:35.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.518+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/675897549 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=22}) -- 0x5591893e82c0 con 0x55918959da30 2026-02-25T11:47:35.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.519+0000 7ff87effd640 1 -- 10.20.193.28:0/675897549 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7ff8c00d0990 con 0x55918959da30 2026-02-25T11:47:35.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.519+0000 7ff8cd8e7e00 10 client.15186 fetch_fsmap finished waiting for FSMap version 22 2026-02-25T11:47:35.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.519+0000 7ff8cd8e7e00 10 client.15186 resolve_mds: resolved name 'i' to daemon mds.i 2026-02-25T11:47:35.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.519+0000 7ff8cd8e7e00 20 client.15186 populate_metadata read hostname 'trial028' 2026-02-25T11:47:35.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.519+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55918959bb00 0x55918959cf70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.519+0000 7ff8cd8e7e00 4 client.15186 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:47:35.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.519+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/675897549 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55918945a140 con 0x55918959bb00 2026-02-25T11:47:35.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.519+0000 7ff8c9661640 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55918959bb00 0x55918959cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.520+0000 7ff8c9661640 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55918959bb00 0x55918959cf70 crc :-1 s=READY pgs=17 gs=4 cs=0 l=0 c_cookie=560149834863955409 s_cookie=934404733298297055 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c60d4e980f4dd1 server_cookie=cf7ac277cb048df in_seq=0 out_seq=0 2026-02-25T11:47:35.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.520+0000 7ff87effd640 10 client.15186 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:47:35.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff87effd640 1 -- 10.20.193.28:0/675897549 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55918945a140 con 0x55918959bb00 2026-02-25T11:47:35.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff87effd640 10 client.15186 handle_command_reply: tid=0 2026-02-25T11:47:35.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff8cd8e7e00 1 client.15186 shutdown 2026-02-25T11:47:35.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/675897549 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55918959da30 msgr2=0x559189576080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55918959da30 0x559189576080 secure :-1 s=READY pgs=299 gs=1 cs=0 l=1 c_cookie=15672110611598114853 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8c0039c00 tx=0x7ff8c0003f00 comp rx=0 tx=0).stop 2026-02-25T11:47:35.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/675897549 shutdown_connections 2026-02-25T11:47:35.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55918959bb00 0x55918959cf70 crc :-1 s=READY pgs=17 gs=4 cs=0 l=0 c_cookie=560149834863955409 s_cookie=934404733298297055 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55918959da30 0x559189576080 unknown :-1 s=CLOSED pgs=299 gs=1 cs=0 l=1 c_cookie=15672110611598114853 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558d90 0x559189571090 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/675897549 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558480 0x559189570b50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff87effd640 0 client.15186 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:47:35.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/675897549 >> 10.20.193.28:0/675897549 conn(0x559189478720 msgr2=0x559189557470 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:35.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/675897549 shutdown_connections 2026-02-25T11:47:35.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/675897549 wait complete. 2026-02-25T11:47:35.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.521+0000 7ff8cd8e7e00 20 client.15186 trim_cache size 0 max 16384 2026-02-25T11:47:35.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.530+0000 7ff8cd8e7e00 1 Processor -- start 2026-02-25T11:47:35.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.530+0000 7ff8cd8e7e00 1 -- start start 2026-02-25T11:47:35.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8cd8e7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558480 0x55918956de00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8cd8e7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559189558d90 0x55918956e340 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8cd8e7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55918959da30 0x559189556660 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8cd8e7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559189462330 con 0x559189558d90 2026-02-25T11:47:35.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8cd8e7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559189122610 con 0x559189558480 2026-02-25T11:47:35.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8cd8e7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559189599890 con 0x55918959da30 2026-02-25T11:47:35.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8c8e60640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559189558d90 0x55918956e340 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8c9e62640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55918959da30 0x559189556660 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8c9e62640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55918959da30 0x559189556660 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52756/0 (socket says 10.20.193.28:52756) 2026-02-25T11:47:35.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8c9e62640 1 -- 10.20.193.28:0/1609208990 learned_addr learned my addr 10.20.193.28:0/1609208990 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:35.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8c9661640 1 --2- 10.20.193.28:0/1609208990 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558480 0x55918956de00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8c9e62640 1 -- 10.20.193.28:0/1609208990 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558480 msgr2=0x55918956de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:35.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/1609208990 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558480 0x55918956de00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8c9e62640 1 -- 10.20.193.28:0/1609208990 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559189558d90 msgr2=0x55918956e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:35.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/1609208990 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559189558d90 0x55918956e340 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8c9e62640 1 -- 10.20.193.28:0/1609208990 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591893fa0c0 con 0x55918959da30 2026-02-25T11:47:35.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8c8e60640 1 --2- 10.20.193.28:0/1609208990 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559189558d90 0x55918956e340 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:35.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.531+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/1609208990 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55918959da30 0x559189556660 secure :-1 s=READY pgs=99 gs=2 cs=0 l=1 c_cookie=12664361250797960687 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8c0030fd0 tx=0x7ff8c0099000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afc0d9757d08bdef server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:35.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.532+0000 7ff8914e4640 1 -- 10.20.193.28:0/1609208990 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff8c00d86d0 con 0x55918959da30 2026-02-25T11:47:35.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.532+0000 7ff8914e4640 1 -- 10.20.193.28:0/1609208990 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5591891bc3b0 con 0x55918959da30 2026-02-25T11:47:35.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.532+0000 7ff8914e4640 1 -- 10.20.193.28:0/1609208990 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff8c00ddaf0 con 0x55918959da30 2026-02-25T11:47:35.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.532+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/1609208990 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55918959da30 msgr2=0x559189556660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.532+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/1609208990 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55918959da30 0x559189556660 secure :-1 s=READY pgs=99 gs=2 cs=0 l=1 c_cookie=12664361250797960687 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8c0030fd0 tx=0x7ff8c0099000 comp rx=0 tx=0).stop 2026-02-25T11:47:35.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.532+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/1609208990 shutdown_connections 2026-02-25T11:47:35.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.532+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/1609208990 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55918959da30 0x559189556660 unknown :-1 s=CLOSED pgs=99 gs=2 cs=0 l=1 c_cookie=12664361250797960687 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.532+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/1609208990 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559189558d90 0x55918956e340 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.532+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/1609208990 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558480 0x55918956de00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.532+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/1609208990 >> 10.20.193.28:0/1609208990 conn(0x559189478720 msgr2=0x55918955b360 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:35.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.532+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/1609208990 shutdown_connections 2026-02-25T11:47:35.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.532+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/1609208990 wait complete. 2026-02-25T11:47:35.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.533+0000 7ff8cd8e7e00 1 -- start start 2026-02-25T11:47:35.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.533+0000 7ff8cd8e7e00 1 Processor -- start 2026-02-25T11:47:35.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.533+0000 7ff8cd8e7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558480 0x5591895772f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.533+0000 7ff8cd8e7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558d90 0x559189577830 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.533+0000 7ff8cd8e7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55918959da30 0x55918957c190 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.533+0000 7ff8cd8e7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559189577f00 con 0x55918959da30 2026-02-25T11:47:35.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.533+0000 7ff8cd8e7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55918957c6d0 con 0x559189558d90 2026-02-25T11:47:35.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.533+0000 7ff8cd8e7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55918957c890 con 0x559189558480 2026-02-25T11:47:35.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.533+0000 7ff8c9661640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558480 0x5591895772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.533+0000 7ff8c9661640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558480 0x5591895772f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52760/0 (socket says 10.20.193.28:52760) 2026-02-25T11:47:35.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.533+0000 7ff8c9661640 1 -- 10.20.193.28:0/345073638 learned_addr learned my addr 10.20.193.28:0/345073638 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:35.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.533+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/345073638 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55918959da30 0x55918957c190 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.533+0000 7ff8c8e60640 1 --2- 10.20.193.28:0/345073638 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558d90 0x559189577830 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff8c9e62640 1 -- 10.20.193.28:0/345073638 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558480 msgr2=0x5591895772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:35.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/345073638 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558480 0x5591895772f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff8c9e62640 1 -- 10.20.193.28:0/345073638 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558d90 msgr2=0x559189577830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:35.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/345073638 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558d90 0x559189577830 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff8c9e62640 1 -- 10.20.193.28:0/345073638 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591893f4b40 con 0x55918959da30 2026-02-25T11:47:35.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff8c9e62640 1 --2- 10.20.193.28:0/345073638 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55918959da30 0x55918957c190 secure :-1 s=READY pgs=300 gs=3 cs=0 l=1 c_cookie=10461851950784332555 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8c00d2c20 tx=0x7ff8c0003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912ff6eb833a430b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:35.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff87effd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:47:35.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff87effd640 1 -- 10.20.193.28:0/345073638 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff8c00d86d0 con 0x55918959da30 2026-02-25T11:47:35.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff87effd640 1 -- 10.20.193.28:0/345073638 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff8c00ce690 con 0x55918959da30 2026-02-25T11:47:35.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff87effd640 1 -- 10.20.193.28:0/345073638 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff8c00d3640 con 0x55918959da30 2026-02-25T11:47:35.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/345073638 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c009c070 con 0x55918959da30 2026-02-25T11:47:35.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff87effd640 1 -- 10.20.193.28:0/345073638 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7ff8c00d37a0 con 0x55918959da30 2026-02-25T11:47:35.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff8cd8e7e00 10 client.15192 _ll_register_callbacks cb 0x559189452a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:35.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.534+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/345073638 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8b8004d10 con 0x55918959da30 2026-02-25T11:47:35.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.535+0000 7ff87effd640 1 -- 10.20.193.28:0/345073638 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7ff8c00d33b0 con 0x55918959da30 2026-02-25T11:47:35.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.535+0000 7ff8cd8e7e00 10 client.15192 fetch_fsmap learned FSMap version 22 2026-02-25T11:47:35.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.535+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/345073638 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=22}) -- 0x5591893e85f0 con 0x55918959da30 2026-02-25T11:47:35.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.535+0000 7ff87effd640 1 -- 10.20.193.28:0/345073638 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7ff8c00d43a0 con 0x55918959da30 2026-02-25T11:47:35.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.535+0000 7ff8cd8e7e00 10 client.15192 fetch_fsmap finished waiting for FSMap version 22 2026-02-25T11:47:35.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.535+0000 7ff8cd8e7e00 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2026-02-25T11:47:35.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.535+0000 7ff8cd8e7e00 20 client.15192 populate_metadata read hostname 'trial028' 2026-02-25T11:47:35.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.535+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/345073638 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55918964f5b0 0x55918964e2c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.535+0000 7ff8cd8e7e00 4 client.15192 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "status"}] 2026-02-25T11:47:35.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.535+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/345073638 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "status"}) -- 0x5591891a1070 con 0x55918964f5b0 2026-02-25T11:47:35.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.535+0000 7ff8c9661640 1 --2- 10.20.193.28:0/345073638 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55918964f5b0 0x55918964e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.536+0000 7ff8c9661640 1 --2- 10.20.193.28:0/345073638 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55918964f5b0 0x55918964e2c0 crc :-1 s=READY pgs=18 gs=4 cs=0 l=0 c_cookie=10847995529839729774 s_cookie=4352343295857547439 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=968bd275911f5c6e server_cookie=3c66a12afa2458af in_seq=0 out_seq=0 2026-02-25T11:47:35.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.536+0000 7ff87effd640 10 client.15192 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:47:35.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.536+0000 7ff87effd640 1 -- 10.20.193.28:0/345073638 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1217 (crc 0 0 0) 0x5591891a1070 con 0x55918964f5b0 2026-02-25T11:47:35.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.536+0000 7ff87effd640 10 client.15192 handle_command_reply: tid=0 2026-02-25T11:47:35.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.536+0000 7ff8cd8e7e00 1 client.15192 shutdown 2026-02-25T11:47:35.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.536+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/345073638 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55918959da30 msgr2=0x55918957c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.536+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/345073638 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55918959da30 0x55918957c190 secure :-1 s=READY pgs=300 gs=3 cs=0 l=1 c_cookie=10461851950784332555 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8c00d2c20 tx=0x7ff8c0003c40 comp rx=0 tx=0).stop 2026-02-25T11:47:35.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.537+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/345073638 shutdown_connections 2026-02-25T11:47:35.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.537+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/345073638 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55918964f5b0 0x55918964e2c0 crc :-1 s=READY pgs=18 gs=4 cs=0 l=0 c_cookie=10847995529839729774 s_cookie=4352343295857547439 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.537+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/345073638 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55918959da30 0x55918957c190 unknown :-1 s=CLOSED pgs=300 gs=3 cs=0 l=1 c_cookie=10461851950784332555 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.537+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/345073638 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559189558d90 0x559189577830 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.537+0000 7ff8cd8e7e00 1 --2- 10.20.193.28:0/345073638 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559189558480 0x5591895772f0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.537+0000 7ff87effd640 0 client.15192 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:47:35.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.537+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/345073638 >> 10.20.193.28:0/345073638 conn(0x559189478720 msgr2=0x55918956ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:35.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.537+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/345073638 shutdown_connections 2026-02-25T11:47:35.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.537+0000 7ff8cd8e7e00 1 -- 10.20.193.28:0/345073638 wait complete. 2026-02-25T11:47:35.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.537+0000 7ff8cd8e7e00 20 client.15192 trim_cache size 0 max 16384 2026-02-25T11:47:35.546 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:47:35.546 INFO:teuthology.orchestra.run.trial028.stdout: "cluster_fsid": "123fa7bf-123f-11f1-80da-d404e6e7d460", 2026-02-25T11:47:35.546 INFO:teuthology.orchestra.run.trial028.stdout: "whoami": 2, 2026-02-25T11:47:35.547 INFO:teuthology.orchestra.run.trial028.stdout: "id": 24730, 2026-02-25T11:47:35.547 INFO:teuthology.orchestra.run.trial028.stdout: "want_state": "up:active", 2026-02-25T11:47:35.547 INFO:teuthology.orchestra.run.trial028.stdout: "state": "up:active", 2026-02-25T11:47:35.547 INFO:teuthology.orchestra.run.trial028.stdout: "fs_name": "cephfs", 2026-02-25T11:47:35.547 INFO:teuthology.orchestra.run.trial028.stdout: "rank_uptime": 17.887810193, 2026-02-25T11:47:35.547 INFO:teuthology.orchestra.run.trial028.stdout: "mdsmap_epoch": 22, 2026-02-25T11:47:35.547 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_epoch": 86, 2026-02-25T11:47:35.547 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_epoch_barrier": 86, 2026-02-25T11:47:35.547 INFO:teuthology.orchestra.run.trial028.stdout: "uptime": 32.228200078999997, 2026-02-25T11:47:35.547 INFO:teuthology.orchestra.run.trial028.stdout: "sysinfo": { 2026-02-25T11:47:35.548 INFO:teuthology.orchestra.run.trial028.stdout: "arch": "x86_64", 2026-02-25T11:47:35.548 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_release": "tentacle", 2026-02-25T11:47:35.548 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version": "ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev - RelWithDebInfo)", 2026-02-25T11:47:35.548 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version_short": "20.3.0-5549-g92715cd3", 2026-02-25T11:47:35.548 INFO:teuthology.orchestra.run.trial028.stdout: "container_hostname": "trial167", 2026-02-25T11:47:35.548 INFO:teuthology.orchestra.run.trial028.stdout: "container_image": "quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33", 2026-02-25T11:47:35.548 INFO:teuthology.orchestra.run.trial028.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-25T11:47:35.548 INFO:teuthology.orchestra.run.trial028.stdout: "distro": "centos", 2026-02-25T11:47:35.548 INFO:teuthology.orchestra.run.trial028.stdout: "distro_description": "CentOS Stream 9", 2026-02-25T11:47:35.548 INFO:teuthology.orchestra.run.trial028.stdout: "distro_version": "9", 2026-02-25T11:47:35.548 INFO:teuthology.orchestra.run.trial028.stdout: "hostname": "trial167", 2026-02-25T11:47:35.548 INFO:teuthology.orchestra.run.trial028.stdout: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Tue Feb 24 21:03:19 EST 2026", 2026-02-25T11:47:35.549 INFO:teuthology.orchestra.run.trial028.stdout: "kernel_version": "6.19.0-g712f4a43c125", 2026-02-25T11:47:35.549 INFO:teuthology.orchestra.run.trial028.stdout: "mem_swap_kb": "8388604", 2026-02-25T11:47:35.549 INFO:teuthology.orchestra.run.trial028.stdout: "mem_total_kb": "130448332", 2026-02-25T11:47:35.549 INFO:teuthology.orchestra.run.trial028.stdout: "os": "Linux" 2026-02-25T11:47:35.549 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:47:35.549 INFO:teuthology.orchestra.run.trial028.stdout: "endian": "little" 2026-02-25T11:47:35.549 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:47:35.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.538+0000 7ff8cb8ec640 1 -- 10.20.193.28:0/2630603634 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff8a00b57e0 msgr2=0x7ff8a00d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.538+0000 7ff8cb8ec640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff8a00b57e0 0x7ff8a00d5c90 secure :-1 s=READY pgs=225 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8b4004740 tx=0x7ff8b4005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:35.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.538+0000 7ff8cb8ec640 1 -- 10.20.193.28:0/2630603634 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8c4115a30 msgr2=0x7ff8c40f62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.538+0000 7ff8cb8ec640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8c4115a30 0x7ff8c40f62a0 secure :-1 s=READY pgs=298 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff8c0039820 tx=0x7ff8c0039b90 comp rx=0 tx=0).stop 2026-02-25T11:47:35.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.538+0000 7ff8cb8ec640 1 -- 10.20.193.28:0/2630603634 shutdown_connections 2026-02-25T11:47:35.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.538+0000 7ff8cb8ec640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff8a00b57e0 0x7ff8a00d5c90 unknown :-1 s=CLOSED pgs=225 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.538+0000 7ff8cb8ec640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff8c4115a30 0x7ff8c40f62a0 unknown :-1 s=CLOSED pgs=298 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.538+0000 7ff8cb8ec640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff8c4110230 0x7ff8c4176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.538+0000 7ff8cb8ec640 1 --2- 10.20.193.28:0/2630603634 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff8c410f840 0x7ff8c40f5ba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.538+0000 7ff8cb8ec640 1 -- 10.20.193.28:0/2630603634 >> 10.20.193.28:0/2630603634 conn(0x7ff8c40ed130 msgr2=0x7ff8c4113090 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:35.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.538+0000 7ff8cb8ec640 1 -- 10.20.193.28:0/2630603634 shutdown_connections 2026-02-25T11:47:35.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.538+0000 7ff8cb8ec640 1 -- 10.20.193.28:0/2630603634 wait complete. 2026-02-25T11:47:35.637 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.g status 2026-02-25T11:47:35.823 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:35.994 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.991+0000 7f4dd3fff640 1 -- 10.20.193.28:0/1802910091 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4dd4110080 msgr2=0x7f4dd41150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.994 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.991+0000 7f4dd3fff640 1 --2- 10.20.193.28:0/1802910091 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4dd4110080 0x7f4dd41150d0 secure :-1 s=READY pgs=100 gs=2 cs=0 l=1 c_cookie=9912877953093529835 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4dbc009950 tx=0x7f4dbc02f190 comp rx=0 tx=0).stop 2026-02-25T11:47:35.994 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.991+0000 7f4dd3fff640 1 -- 10.20.193.28:0/1802910091 shutdown_connections 2026-02-25T11:47:35.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.991+0000 7f4dd3fff640 1 --2- 10.20.193.28:0/1802910091 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4dd4115880 0x7f4dd4176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.991+0000 7f4dd3fff640 1 --2- 10.20.193.28:0/1802910091 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4dd4110080 0x7f4dd41150d0 unknown :-1 s=CLOSED pgs=100 gs=2 cs=0 l=1 c_cookie=9912877953093529835 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.991+0000 7f4dd3fff640 1 --2- 10.20.193.28:0/1802910091 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4dd410f690 0x7f4dd410fa60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.991+0000 7f4dd3fff640 1 -- 10.20.193.28:0/1802910091 >> 10.20.193.28:0/1802910091 conn(0x7f4dd40ecf00 msgr2=0x7f4dd410d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:35.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.991+0000 7f4dd3fff640 1 -- 10.20.193.28:0/1802910091 shutdown_connections 2026-02-25T11:47:35.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.991+0000 7f4dd3fff640 1 -- 10.20.193.28:0/1802910091 wait complete. 2026-02-25T11:47:35.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.992+0000 7f4dd3fff640 1 Processor -- start 2026-02-25T11:47:35.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.992+0000 7f4dd3fff640 1 -- start start 2026-02-25T11:47:35.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd3fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4dd410f690 0x7f4dd411e090 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd3fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4dd4110080 0x7f4dd411e5d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4dd4115880 0x7f4dd41239a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:35.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd3fff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd4123fc0 con 0x7f4dd4115880 2026-02-25T11:47:35.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd3fff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd4124130 con 0x7f4dd4110080 2026-02-25T11:47:35.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd3fff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd41242f0 con 0x7f4dd410f690 2026-02-25T11:47:35.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd2ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4dd410f690 0x7f4dd411e090 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd2ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4dd410f690 0x7f4dd411e090 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:45946/0 (socket says 10.20.193.28:45946) 2026-02-25T11:47:35.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd2ffd640 1 -- 10.20.193.28:0/3487842021 learned_addr learned my addr 10.20.193.28:0/3487842021 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:35.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd27fc640 1 --2- 10.20.193.28:0/3487842021 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4dd4110080 0x7f4dd411e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:35.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd2ffd640 1 -- 10.20.193.28:0/3487842021 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4dd4110080 msgr2=0x7f4dd411e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd2ffd640 1 --2- 10.20.193.28:0/3487842021 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4dd4110080 0x7f4dd411e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd2ffd640 1 -- 10.20.193.28:0/3487842021 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4dd4115880 msgr2=0x7f4dd41239a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:35.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd2ffd640 1 --2- 10.20.193.28:0/3487842021 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4dd4115880 0x7f4dd41239a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:35.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd2ffd640 1 -- 10.20.193.28:0/3487842021 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc80094b0 con 0x7f4dd410f690 2026-02-25T11:47:35.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.993+0000 7f4dd2ffd640 1 --2- 10.20.193.28:0/3487842021 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4dd410f690 0x7f4dd411e090 secure :-1 s=READY pgs=101 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4dc8009fd0 tx=0x7f4dc8002bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:35.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.994+0000 7f4db3fff640 1 -- 10.20.193.28:0/3487842021 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4dc800ed60 con 0x7f4dd410f690 2026-02-25T11:47:35.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.994+0000 7f4db3fff640 1 -- 10.20.193.28:0/3487842021 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4dc8002e20 con 0x7f4dd410f690 2026-02-25T11:47:35.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.994+0000 7f4db3fff640 1 -- 10.20.193.28:0/3487842021 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4dc80188b0 con 0x7f4dd410f690 2026-02-25T11:47:35.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.994+0000 7f4dd3fff640 1 -- 10.20.193.28:0/3487842021 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dbc009570 con 0x7f4dd410f690 2026-02-25T11:47:35.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.994+0000 7f4dd3fff640 1 -- 10.20.193.28:0/3487842021 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd4164b30 con 0x7f4dd410f690 2026-02-25T11:47:35.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.997+0000 7f4db3fff640 1 -- 10.20.193.28:0/3487842021 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f4dc8016020 con 0x7f4dd410f690 2026-02-25T11:47:36.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.999+0000 7f4db3fff640 1 --2- 10.20.193.28:0/3487842021 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4da80b57f0 0x7f4da80d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.999+0000 7f4db3fff640 1 -- 10.20.193.28:0/3487842021 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f4dc80e7160 con 0x7f4dd410f690 2026-02-25T11:47:36.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.999+0000 7f4ddb680e00 1 Processor -- start 2026-02-25T11:47:36.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.999+0000 7f4ddb680e00 1 -- start start 2026-02-25T11:47:36.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.999+0000 7f4dd27fc640 1 --2- 10.20.193.28:0/3487842021 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4da80b57f0 0x7f4da80d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.999+0000 7f4ddb680e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838480 0x559fcf838850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:35.999+0000 7f4ddb680e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf838d90 0x559fcf87d4f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4ddb680e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf87da30 0x559fcf89de10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4ddb680e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559fcf74ea30 con 0x559fcf87da30 2026-02-25T11:47:36.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4ddb680e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559fcf74a760 con 0x559fcf838d90 2026-02-25T11:47:36.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4ddb680e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559fcf744700 con 0x559fcf838480 2026-02-25T11:47:36.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4dd27fc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf838d90 0x559fcf87d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4dd27fc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf838d90 0x559fcf87d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43156/0 (socket says 10.20.193.28:43156) 2026-02-25T11:47:36.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4dd27fc640 1 -- 10.20.193.28:0/3858736920 learned_addr learned my addr 10.20.193.28:0/3858736920 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:36.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4dd2ffd640 1 --2- 10.20.193.28:0/3858736920 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838480 0x559fcf838850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4dd27fc640 1 --2- 10.20.193.28:0/3487842021 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4da80b57f0 0x7f4da80d5ca0 secure :-1 s=READY pgs=226 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4dd4124a30 tx=0x7f4dbc002390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:36.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4dd27fc640 1 -- 10.20.193.28:0/3858736920 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838480 msgr2=0x559fcf838850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4dd27fc640 1 --2- 10.20.193.28:0/3858736920 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838480 0x559fcf838850 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4dd37fe640 1 --2- 10.20.193.28:0/3858736920 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf87da30 0x559fcf89de10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4dd27fc640 1 -- 10.20.193.28:0/3858736920 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf87da30 msgr2=0x559fcf89de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4dd27fc640 1 --2- 10.20.193.28:0/3858736920 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf87da30 0x559fcf89de10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4dd27fc640 1 -- 10.20.193.28:0/3858736920 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcf6efcf0 con 0x559fcf838d90 2026-02-25T11:47:36.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4dd37fe640 1 --2- 10.20.193.28:0/3858736920 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf87da30 0x559fcf89de10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:36.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.000+0000 7f4dd27fc640 1 --2- 10.20.193.28:0/3858736920 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf838d90 0x559fcf87d4f0 secure :-1 s=READY pgs=96 gs=1 cs=0 l=1 c_cookie=17287261585311685357 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4dbc005d00 tx=0x7f4dbc048000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efe8b0d1ebc25aed server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:36.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.001+0000 7f4d9f7fe640 1 -- 10.20.193.28:0/3858736920 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4dbc0026a0 con 0x559fcf838d90 2026-02-25T11:47:36.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.001+0000 7f4d9f7fe640 1 -- 10.20.193.28:0/3858736920 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4dbc03e040 con 0x559fcf838d90 2026-02-25T11:47:36.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.001+0000 7f4d9f7fe640 1 -- 10.20.193.28:0/3858736920 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4dbc038cf0 con 0x559fcf838d90 2026-02-25T11:47:36.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.001+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3858736920 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf838d90 msgr2=0x559fcf87d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.001+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/3858736920 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf838d90 0x559fcf87d4f0 secure :-1 s=READY pgs=96 gs=1 cs=0 l=1 c_cookie=17287261585311685357 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4dbc005d00 tx=0x7f4dbc048000 comp rx=0 tx=0).stop 2026-02-25T11:47:36.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.001+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3858736920 shutdown_connections 2026-02-25T11:47:36.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.001+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/3858736920 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf87da30 0x559fcf89de10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.001+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/3858736920 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf838d90 0x559fcf87d4f0 unknown :-1 s=CLOSED pgs=96 gs=1 cs=0 l=1 c_cookie=17287261585311685357 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.001+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/3858736920 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838480 0x559fcf838850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.001+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3858736920 >> 10.20.193.28:0/3858736920 conn(0x559fcf758720 msgr2=0x559fcf836d60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:36.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.001+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3858736920 shutdown_connections 2026-02-25T11:47:36.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.001+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3858736920 wait complete. 2026-02-25T11:47:36.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.002+0000 7f4ddb680e00 1 -- start start 2026-02-25T11:47:36.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.003+0000 7f4ddb680e00 1 Processor -- start 2026-02-25T11:47:36.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.003+0000 7f4ddb680e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838480 0x559fcf850ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.003+0000 7f4ddb680e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838d90 0x559fcf8510e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.003+0000 7f4ddb680e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf87da30 0x559fcf8560d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.003+0000 7f4ddb680e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559fcf856610 con 0x559fcf838480 2026-02-25T11:47:36.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.003+0000 7f4ddb680e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559fcf856780 con 0x559fcf87da30 2026-02-25T11:47:36.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.003+0000 7f4ddb680e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559fcf856940 con 0x559fcf838d90 2026-02-25T11:47:36.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.003+0000 7f4dd2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838480 0x559fcf850ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.003+0000 7f4dd27fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838d90 0x559fcf8510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.003+0000 7f4dd27fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838d90 0x559fcf8510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:45968/0 (socket says 10.20.193.28:45968) 2026-02-25T11:47:36.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.003+0000 7f4dd27fc640 1 -- 10.20.193.28:0/1460393829 learned_addr learned my addr 10.20.193.28:0/1460393829 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:36.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.003+0000 7f4dd37fe640 1 --2- 10.20.193.28:0/1460393829 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf87da30 0x559fcf8560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.004+0000 7f4dd27fc640 1 -- 10.20.193.28:0/1460393829 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf87da30 msgr2=0x559fcf8560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.004+0000 7f4dd27fc640 1 --2- 10.20.193.28:0/1460393829 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf87da30 0x559fcf8560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.004+0000 7f4dd27fc640 1 -- 10.20.193.28:0/1460393829 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838480 msgr2=0x559fcf850ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.004+0000 7f4dd27fc640 1 --2- 10.20.193.28:0/1460393829 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838480 0x559fcf850ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.004+0000 7f4dd27fc640 1 -- 10.20.193.28:0/1460393829 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcf6c82c0 con 0x559fcf838d90 2026-02-25T11:47:36.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.004+0000 7f4dd27fc640 1 --2- 10.20.193.28:0/1460393829 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838d90 0x559fcf8510e0 secure :-1 s=READY pgs=102 gs=1 cs=0 l=1 c_cookie=3439106519623291766 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4dbc018150 tx=0x7f4dbc0092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fba29288ee05376 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:36.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.004+0000 7f4d9cff9640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:47:36.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.004+0000 7f4d9cff9640 1 -- 10.20.193.28:0/1460393829 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4dbc0386a0 con 0x559fcf838d90 2026-02-25T11:47:36.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.004+0000 7f4d9cff9640 1 -- 10.20.193.28:0/1460393829 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4dbc062070 con 0x559fcf838d90 2026-02-25T11:47:36.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.004+0000 7f4ddb680e00 1 -- 10.20.193.28:0/1460393829 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd4176ac0 con 0x559fcf838d90 2026-02-25T11:47:36.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.004+0000 7f4d9cff9640 1 -- 10.20.193.28:0/1460393829 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4dbc04fa80 con 0x559fcf838d90 2026-02-25T11:47:36.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.004+0000 7f4d9cff9640 1 -- 10.20.193.28:0/1460393829 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f4dbc064070 con 0x559fcf838d90 2026-02-25T11:47:36.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.004+0000 7f4ddb680e00 10 client.25066 _ll_register_callbacks cb 0x559fcf732a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:36.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.005+0000 7f4ddb680e00 1 -- 10.20.193.28:0/1460393829 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559fcf49c3b0 con 0x559fcf838d90 2026-02-25T11:47:36.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.005+0000 7f4d9cff9640 1 -- 10.20.193.28:0/1460393829 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4dbc038c60 con 0x559fcf838d90 2026-02-25T11:47:36.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.005+0000 7f4ddb680e00 10 client.25066 fetch_fsmap learned FSMap version 22 2026-02-25T11:47:36.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.005+0000 7f4ddb680e00 1 -- 10.20.193.28:0/1460393829 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=22}) -- 0x559fcf6da0c0 con 0x559fcf838d90 2026-02-25T11:47:36.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.005+0000 7f4d9cff9640 1 -- 10.20.193.28:0/1460393829 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f4dbc072020 con 0x559fcf838d90 2026-02-25T11:47:36.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.005+0000 7f4ddb680e00 10 client.25066 fetch_fsmap finished waiting for FSMap version 22 2026-02-25T11:47:36.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.005+0000 7f4ddb680e00 10 client.25066 resolve_mds: resolved name 'g' to daemon mds.g 2026-02-25T11:47:36.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.005+0000 7f4ddb680e00 20 client.25066 populate_metadata read hostname 'trial028' 2026-02-25T11:47:36.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.005+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/1460393829 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559fcf87ba50 0x559fcf87cec0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.005+0000 7f4ddb680e00 4 client.25066 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:47:36.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.005+0000 7f4ddb680e00 1 -- 10.20.193.28:0/1460393829 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fcf73a140 con 0x559fcf87ba50 2026-02-25T11:47:36.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.005+0000 7f4dd2ffd640 1 --2- 10.20.193.28:0/1460393829 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559fcf87ba50 0x559fcf87cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.006+0000 7f4dd2ffd640 1 --2- 10.20.193.28:0/1460393829 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559fcf87ba50 0x559fcf87cec0 crc :-1 s=READY pgs=17 gs=4 cs=0 l=0 c_cookie=11831508438057556543 s_cookie=14366658694849822566 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a431f63d07751a3f server_cookie=c7609ff713ccff66 in_seq=0 out_seq=0 2026-02-25T11:47:36.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.006+0000 7f4d9cff9640 10 client.25066 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:47:36.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4d9cff9640 1 -- 10.20.193.28:0/1460393829 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x559fcf73a140 con 0x559fcf87ba50 2026-02-25T11:47:36.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4d9cff9640 10 client.25066 handle_command_reply: tid=0 2026-02-25T11:47:36.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4ddb680e00 1 client.25066 shutdown 2026-02-25T11:47:36.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4ddb680e00 1 -- 10.20.193.28:0/1460393829 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838d90 msgr2=0x559fcf8510e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/1460393829 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838d90 0x559fcf8510e0 secure :-1 s=READY pgs=102 gs=1 cs=0 l=1 c_cookie=3439106519623291766 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4dbc018150 tx=0x7f4dbc0092f0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4ddb680e00 1 -- 10.20.193.28:0/1460393829 shutdown_connections 2026-02-25T11:47:36.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/1460393829 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559fcf87ba50 0x559fcf87cec0 crc :-1 s=READY pgs=17 gs=4 cs=0 l=0 c_cookie=11831508438057556543 s_cookie=14366658694849822566 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/1460393829 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf87da30 0x559fcf8560d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/1460393829 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838d90 0x559fcf8510e0 unknown :-1 s=CLOSED pgs=102 gs=1 cs=0 l=1 c_cookie=3439106519623291766 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/1460393829 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838480 0x559fcf850ba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4ddb680e00 1 -- 10.20.193.28:0/1460393829 >> 10.20.193.28:0/1460393829 conn(0x559fcf758720 msgr2=0x559fcf837470 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:36.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4d9cff9640 0 client.25066 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:47:36.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4ddb680e00 1 -- 10.20.193.28:0/1460393829 shutdown_connections 2026-02-25T11:47:36.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4ddb680e00 1 -- 10.20.193.28:0/1460393829 wait complete. 2026-02-25T11:47:36.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.007+0000 7f4ddb680e00 20 client.25066 trim_cache size 0 max 16384 2026-02-25T11:47:36.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.016+0000 7f4ddb680e00 1 Processor -- start 2026-02-25T11:47:36.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.016+0000 7f4ddb680e00 1 -- start start 2026-02-25T11:47:36.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4ddb680e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838480 0x559fcf83b3e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4ddb680e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838d90 0x559fcf84de50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4ddb680e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf87da30 0x559fcf84e390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4ddb680e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559fcf742330 con 0x559fcf838d90 2026-02-25T11:47:36.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4ddb680e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559fcf402610 con 0x559fcf87da30 2026-02-25T11:47:36.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4ddb680e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559fcf849b30 con 0x559fcf838480 2026-02-25T11:47:36.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4dd27fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838d90 0x559fcf84de50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4dd2ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838480 0x559fcf83b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4dd2ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838480 0x559fcf83b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:45984/0 (socket says 10.20.193.28:45984) 2026-02-25T11:47:36.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4dd2ffd640 1 -- 10.20.193.28:0/2039401330 learned_addr learned my addr 10.20.193.28:0/2039401330 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:36.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4dd37fe640 1 --2- 10.20.193.28:0/2039401330 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf87da30 0x559fcf84e390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4dd2ffd640 1 -- 10.20.193.28:0/2039401330 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf87da30 msgr2=0x559fcf84e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4dd2ffd640 1 --2- 10.20.193.28:0/2039401330 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf87da30 0x559fcf84e390 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4dd2ffd640 1 -- 10.20.193.28:0/2039401330 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838d90 msgr2=0x559fcf84de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4dd2ffd640 1 --2- 10.20.193.28:0/2039401330 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838d90 0x559fcf84de50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4dd2ffd640 1 -- 10.20.193.28:0/2039401330 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcf6d4b40 con 0x559fcf838480 2026-02-25T11:47:36.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.017+0000 7f4dd2ffd640 1 --2- 10.20.193.28:0/2039401330 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838480 0x559fcf83b3e0 secure :-1 s=READY pgs=103 gs=1 cs=0 l=1 c_cookie=6356115782826902686 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4dc80a9a30 tx=0x7f4dc8002f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=583575a0daa6c49e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:36.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.018+0000 7f4d9effd640 1 -- 10.20.193.28:0/2039401330 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4dc80b85a0 con 0x559fcf838480 2026-02-25T11:47:36.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.018+0000 7f4d9effd640 1 -- 10.20.193.28:0/2039401330 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x559fcf73a140 con 0x559fcf838480 2026-02-25T11:47:36.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.018+0000 7f4d9effd640 1 -- 10.20.193.28:0/2039401330 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4dc80b8110 con 0x559fcf838480 2026-02-25T11:47:36.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.018+0000 7f4ddb680e00 1 -- 10.20.193.28:0/2039401330 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838480 msgr2=0x559fcf83b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.018+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/2039401330 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838480 0x559fcf83b3e0 secure :-1 s=READY pgs=103 gs=1 cs=0 l=1 c_cookie=6356115782826902686 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4dc80a9a30 tx=0x7f4dc8002f90 comp rx=0 tx=0).stop 2026-02-25T11:47:36.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.018+0000 7f4ddb680e00 1 -- 10.20.193.28:0/2039401330 shutdown_connections 2026-02-25T11:47:36.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.018+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/2039401330 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf87da30 0x559fcf84e390 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.018+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/2039401330 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838d90 0x559fcf84de50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.018+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/2039401330 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf838480 0x559fcf83b3e0 unknown :-1 s=CLOSED pgs=103 gs=1 cs=0 l=1 c_cookie=6356115782826902686 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.018+0000 7f4ddb680e00 1 -- 10.20.193.28:0/2039401330 >> 10.20.193.28:0/2039401330 conn(0x559fcf758720 msgr2=0x559fcf842820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:36.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.018+0000 7f4ddb680e00 1 -- 10.20.193.28:0/2039401330 shutdown_connections 2026-02-25T11:47:36.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.018+0000 7f4ddb680e00 1 -- 10.20.193.28:0/2039401330 wait complete. 2026-02-25T11:47:36.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.019+0000 7f4ddb680e00 1 -- start start 2026-02-25T11:47:36.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.019+0000 7f4ddb680e00 1 Processor -- start 2026-02-25T11:47:36.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.019+0000 7f4ddb680e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838480 0x559fcf8e3f60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.019+0000 7f4ddb680e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf838d90 0x559fcf857160 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.019+0000 7f4ddb680e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf87da30 0x559fcf8576a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.019+0000 7f4ddb680e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559fcf856800 con 0x559fcf838480 2026-02-25T11:47:36.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.019+0000 7f4ddb680e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559fcf857be0 con 0x559fcf838d90 2026-02-25T11:47:36.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.019+0000 7f4ddb680e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559fcf857da0 con 0x559fcf87da30 2026-02-25T11:47:36.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.019+0000 7f4dd2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838480 0x559fcf8e3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.019+0000 7f4dd37fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf87da30 0x559fcf8576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.019+0000 7f4dd2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838480 0x559fcf8e3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35432/0 (socket says 10.20.193.28:35432) 2026-02-25T11:47:36.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.019+0000 7f4dd2ffd640 1 -- 10.20.193.28:0/3807087613 learned_addr learned my addr 10.20.193.28:0/3807087613 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:36.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4dd27fc640 1 --2- 10.20.193.28:0/3807087613 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf838d90 0x559fcf857160 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4dd2ffd640 1 -- 10.20.193.28:0/3807087613 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf87da30 msgr2=0x559fcf8576a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4dd2ffd640 1 --2- 10.20.193.28:0/3807087613 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf87da30 0x559fcf8576a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4dd2ffd640 1 -- 10.20.193.28:0/3807087613 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf838d90 msgr2=0x559fcf857160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4dd2ffd640 1 --2- 10.20.193.28:0/3807087613 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf838d90 0x559fcf857160 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4dd2ffd640 1 -- 10.20.193.28:0/3807087613 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcf6c85f0 con 0x559fcf838480 2026-02-25T11:47:36.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4dd2ffd640 1 --2- 10.20.193.28:0/3807087613 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838480 0x559fcf8e3f60 secure :-1 s=READY pgs=301 gs=1 cs=0 l=1 c_cookie=5893427232347697390 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4dc80ad3f0 tx=0x7f4dc8002390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c9a8d7d5b2b0ee server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:36.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4d9cff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:47:36.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4d9cff9640 1 -- 10.20.193.28:0/3807087613 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4dc80b8110 con 0x559fcf838480 2026-02-25T11:47:36.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3807087613 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd4164b30 con 0x559fcf838480 2026-02-25T11:47:36.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4ddb680e00 10 client.15204 _ll_register_callbacks cb 0x559fcf732a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:36.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3807087613 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dbc03e040 con 0x559fcf838480 2026-02-25T11:47:36.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4d9cff9640 1 -- 10.20.193.28:0/3807087613 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4dc80b85a0 con 0x559fcf838480 2026-02-25T11:47:36.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4d9cff9640 1 -- 10.20.193.28:0/3807087613 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4dc80725f0 con 0x559fcf838480 2026-02-25T11:47:36.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4d9cff9640 1 -- 10.20.193.28:0/3807087613 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f4dc8072780 con 0x559fcf838480 2026-02-25T11:47:36.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4d9cff9640 1 -- 10.20.193.28:0/3807087613 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4dc80cc1b0 con 0x559fcf838480 2026-02-25T11:47:36.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4ddb680e00 10 client.15204 fetch_fsmap learned FSMap version 22 2026-02-25T11:47:36.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.020+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3807087613 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=22}) -- 0x559fcf6da320 con 0x559fcf838480 2026-02-25T11:47:36.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.021+0000 7f4d9cff9640 1 -- 10.20.193.28:0/3807087613 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f4dc80ab440 con 0x559fcf838480 2026-02-25T11:47:36.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.021+0000 7f4ddb680e00 10 client.15204 fetch_fsmap finished waiting for FSMap version 22 2026-02-25T11:47:36.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.021+0000 7f4ddb680e00 10 client.15204 resolve_mds: resolved name 'g' to daemon mds.g 2026-02-25T11:47:36.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.021+0000 7f4ddb680e00 20 client.15204 populate_metadata read hostname 'trial028' 2026-02-25T11:47:36.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.021+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/3807087613 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559fcf92f350 0x559fcf92e060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.021+0000 7f4ddb680e00 4 client.15204 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "status"}] 2026-02-25T11:47:36.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.021+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3807087613 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "status"}) -- 0x559fcf481070 con 0x559fcf92f350 2026-02-25T11:47:36.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.021+0000 7f4dd27fc640 1 --2- 10.20.193.28:0/3807087613 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559fcf92f350 0x559fcf92e060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.021+0000 7f4dd27fc640 1 --2- 10.20.193.28:0/3807087613 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559fcf92f350 0x559fcf92e060 crc :-1 s=READY pgs=18 gs=4 cs=0 l=0 c_cookie=6623043219297205777 s_cookie=16154268873635511035 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5be9c6b7b244fe11 server_cookie=e02f7df39b31bafb in_seq=0 out_seq=0 2026-02-25T11:47:36.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.021+0000 7f4d9cff9640 10 client.15204 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:47:36.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4d9cff9640 1 -- 10.20.193.28:0/3807087613 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1211 (crc 0 0 0) 0x7f4dbc0624d0 con 0x559fcf92f350 2026-02-25T11:47:36.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4d9cff9640 10 client.15204 handle_command_reply: tid=0 2026-02-25T11:47:36.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4ddb680e00 1 client.15204 shutdown 2026-02-25T11:47:36.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3807087613 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838480 msgr2=0x559fcf8e3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/3807087613 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838480 0x559fcf8e3f60 secure :-1 s=READY pgs=301 gs=1 cs=0 l=1 c_cookie=5893427232347697390 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4dc80ad3f0 tx=0x7f4dc8002390 comp rx=0 tx=0).stop 2026-02-25T11:47:36.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3807087613 shutdown_connections 2026-02-25T11:47:36.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/3807087613 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559fcf92f350 0x559fcf92e060 crc :-1 s=READY pgs=18 gs=4 cs=0 l=0 c_cookie=6623043219297205777 s_cookie=16154268873635511035 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/3807087613 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559fcf87da30 0x559fcf8576a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.032 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:47:36.032 INFO:teuthology.orchestra.run.trial028.stdout: "cluster_fsid": "123fa7bf-123f-11f1-80da-d404e6e7d460", 2026-02-25T11:47:36.032 INFO:teuthology.orchestra.run.trial028.stdout: "whoami": 1, 2026-02-25T11:47:36.032 INFO:teuthology.orchestra.run.trial028.stdout: "id": 24731, 2026-02-25T11:47:36.033 INFO:teuthology.orchestra.run.trial028.stdout: "want_state": "up:active", 2026-02-25T11:47:36.033 INFO:teuthology.orchestra.run.trial028.stdout: "state": "up:active", 2026-02-25T11:47:36.033 INFO:teuthology.orchestra.run.trial028.stdout: "fs_name": "cephfs", 2026-02-25T11:47:36.033 INFO:teuthology.orchestra.run.trial028.stdout: "rank_uptime": 19.378084843, 2026-02-25T11:47:36.033 INFO:teuthology.orchestra.run.trial028.stdout: "mdsmap_epoch": 22, 2026-02-25T11:47:36.033 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_epoch": 86, 2026-02-25T11:47:36.033 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_epoch_barrier": 86, 2026-02-25T11:47:36.033 INFO:teuthology.orchestra.run.trial028.stdout: "uptime": 31.437977692, 2026-02-25T11:47:36.033 INFO:teuthology.orchestra.run.trial028.stdout: "sysinfo": { 2026-02-25T11:47:36.033 INFO:teuthology.orchestra.run.trial028.stdout: "arch": "x86_64", 2026-02-25T11:47:36.034 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_release": "tentacle", 2026-02-25T11:47:36.034 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version": "ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev - RelWithDebInfo)", 2026-02-25T11:47:36.034 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version_short": "20.3.0-5549-g92715cd3", 2026-02-25T11:47:36.034 INFO:teuthology.orchestra.run.trial028.stdout: "container_hostname": "trial028", 2026-02-25T11:47:36.034 INFO:teuthology.orchestra.run.trial028.stdout: "container_image": "quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33", 2026-02-25T11:47:36.034 INFO:teuthology.orchestra.run.trial028.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-25T11:47:36.034 INFO:teuthology.orchestra.run.trial028.stdout: "distro": "centos", 2026-02-25T11:47:36.034 INFO:teuthology.orchestra.run.trial028.stdout: "distro_description": "CentOS Stream 9", 2026-02-25T11:47:36.034 INFO:teuthology.orchestra.run.trial028.stdout: "distro_version": "9", 2026-02-25T11:47:36.034 INFO:teuthology.orchestra.run.trial028.stdout: "hostname": "trial028", 2026-02-25T11:47:36.035 INFO:teuthology.orchestra.run.trial028.stdout: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Tue Feb 24 21:03:19 EST 2026", 2026-02-25T11:47:36.035 INFO:teuthology.orchestra.run.trial028.stdout: "kernel_version": "6.19.0-g712f4a43c125", 2026-02-25T11:47:36.035 INFO:teuthology.orchestra.run.trial028.stdout: "mem_swap_kb": "8388604", 2026-02-25T11:47:36.035 INFO:teuthology.orchestra.run.trial028.stdout: "mem_total_kb": "130448336", 2026-02-25T11:47:36.035 INFO:teuthology.orchestra.run.trial028.stdout: "os": "Linux" 2026-02-25T11:47:36.035 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:47:36.035 INFO:teuthology.orchestra.run.trial028.stdout: "endian": "little" 2026-02-25T11:47:36.035 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:47:36.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/3807087613 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559fcf838d90 0x559fcf857160 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4ddb680e00 1 --2- 10.20.193.28:0/3807087613 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559fcf838480 0x559fcf8e3f60 unknown :-1 s=CLOSED pgs=301 gs=1 cs=0 l=1 c_cookie=5893427232347697390 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3807087613 >> 10.20.193.28:0/3807087613 conn(0x559fcf758720 msgr2=0x559fcf83b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:36.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4d9cff9640 0 client.15204 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:47:36.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3807087613 shutdown_connections 2026-02-25T11:47:36.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4ddb680e00 1 -- 10.20.193.28:0/3807087613 wait complete. 2026-02-25T11:47:36.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.022+0000 7f4ddb680e00 20 client.15204 trim_cache size 0 max 16384 2026-02-25T11:47:36.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.023+0000 7f4dd3fff640 1 -- 10.20.193.28:0/3487842021 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4da80b57f0 msgr2=0x7f4da80d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.023+0000 7f4dd3fff640 1 --2- 10.20.193.28:0/3487842021 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4da80b57f0 0x7f4da80d5ca0 secure :-1 s=READY pgs=226 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4dd4124a30 tx=0x7f4dbc002390 comp rx=0 tx=0).stop 2026-02-25T11:47:36.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.023+0000 7f4dd3fff640 1 -- 10.20.193.28:0/3487842021 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4dd410f690 msgr2=0x7f4dd411e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.023+0000 7f4dd3fff640 1 --2- 10.20.193.28:0/3487842021 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4dd410f690 0x7f4dd411e090 secure :-1 s=READY pgs=101 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4dc8009fd0 tx=0x7f4dc8002bc0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.023+0000 7f4dd3fff640 1 -- 10.20.193.28:0/3487842021 shutdown_connections 2026-02-25T11:47:36.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.023+0000 7f4dd3fff640 1 --2- 10.20.193.28:0/3487842021 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4da80b57f0 0x7f4da80d5ca0 unknown :-1 s=CLOSED pgs=226 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.023+0000 7f4dd3fff640 1 --2- 10.20.193.28:0/3487842021 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4dd4115880 0x7f4dd41239a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.023+0000 7f4dd3fff640 1 --2- 10.20.193.28:0/3487842021 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4dd4110080 0x7f4dd411e5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.023+0000 7f4dd3fff640 1 --2- 10.20.193.28:0/3487842021 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4dd410f690 0x7f4dd411e090 unknown :-1 s=CLOSED pgs=101 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.023+0000 7f4dd3fff640 1 -- 10.20.193.28:0/3487842021 >> 10.20.193.28:0/3487842021 conn(0x7f4dd40ecf00 msgr2=0x7f4dd410d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:36.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.023+0000 7f4dd3fff640 1 -- 10.20.193.28:0/3487842021 shutdown_connections 2026-02-25T11:47:36.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.023+0000 7f4dd3fff640 1 -- 10.20.193.28:0/3487842021 wait complete. 2026-02-25T11:47:36.121 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.b status 2026-02-25T11:47:36.147 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:36 trial028 ceph-mon[39224]: pgmap v205: 97 pgs: 97 active+clean; 641 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2026-02-25T11:47:36.303 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:36.481 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.478+0000 7f3921641640 1 -- 10.20.193.28:0/1170975092 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f391c10bf30 msgr2=0x7f391c10c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.481 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.478+0000 7f3921641640 1 --2- 10.20.193.28:0/1170975092 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f391c10bf30 0x7f391c10c300 secure :-1 s=READY pgs=302 gs=1 cs=0 l=1 c_cookie=1362636319029449104 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3908009980 tx=0x7f390802f190 comp rx=0 tx=0).stop 2026-02-25T11:47:36.481 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.478+0000 7f3921641640 1 -- 10.20.193.28:0/1170975092 shutdown_connections 2026-02-25T11:47:36.481 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.478+0000 7f3921641640 1 --2- 10.20.193.28:0/1170975092 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391c111510 0x7f391c1119b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.478+0000 7f3921641640 1 --2- 10.20.193.28:0/1170975092 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f391c10c920 0x7f391c110ef0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.478+0000 7f3921641640 1 --2- 10.20.193.28:0/1170975092 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f391c10bf30 0x7f391c10c300 unknown :-1 s=CLOSED pgs=302 gs=1 cs=0 l=1 c_cookie=1362636319029449104 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.478+0000 7f3921641640 1 -- 10.20.193.28:0/1170975092 >> 10.20.193.28:0/1170975092 conn(0x7f391c0e9b40 msgr2=0x7f391c109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:36.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.478+0000 7f3921641640 1 -- 10.20.193.28:0/1170975092 shutdown_connections 2026-02-25T11:47:36.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.478+0000 7f3921641640 1 -- 10.20.193.28:0/1170975092 wait complete. 2026-02-25T11:47:36.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.479+0000 7f3921641640 1 Processor -- start 2026-02-25T11:47:36.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.479+0000 7f3921641640 1 -- start start 2026-02-25T11:47:36.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.479+0000 7f3921641640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391c10bf30 0x7f391c122a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.479+0000 7f3921641640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f391c10c920 0x7f391c122f50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.479+0000 7f3921641640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f391c111510 0x7f391c128320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.479+0000 7f3921641640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f391c128940 con 0x7f391c10c920 2026-02-25T11:47:36.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.479+0000 7f3921641640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f391c128ab0 con 0x7f391c10bf30 2026-02-25T11:47:36.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.479+0000 7f3921641640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f391c128c70 con 0x7f391c111510 2026-02-25T11:47:36.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.479+0000 7f391b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f391c10c920 0x7f391c122f50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.479+0000 7f391b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f391c10c920 0x7f391c122f50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35462/0 (socket says 10.20.193.28:35462) 2026-02-25T11:47:36.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.479+0000 7f391b7fe640 1 -- 10.20.193.28:0/776014979 learned_addr learned my addr 10.20.193.28:0/776014979 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:36.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f391bfff640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391c10bf30 0x7f391c122a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f3920e40640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f391c111510 0x7f391c128320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f391b7fe640 1 -- 10.20.193.28:0/776014979 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f391c111510 msgr2=0x7f391c128320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f391b7fe640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f391c111510 0x7f391c128320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f391b7fe640 1 -- 10.20.193.28:0/776014979 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391c10bf30 msgr2=0x7f391c122a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f391b7fe640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391c10bf30 0x7f391c122a10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f391b7fe640 1 -- 10.20.193.28:0/776014979 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3908009570 con 0x7f391c10c920 2026-02-25T11:47:36.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f3920e40640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f391c111510 0x7f391c128320 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:36.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f391bfff640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391c10bf30 0x7f391c122a10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:36.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f391b7fe640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f391c10c920 0x7f391c122f50 secure :-1 s=READY pgs=303 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f390c009480 tx=0x7f390c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:36.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f39197fa640 1 -- 10.20.193.28:0/776014979 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f390c00cd40 con 0x7f391c10c920 2026-02-25T11:47:36.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f39197fa640 1 -- 10.20.193.28:0/776014979 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f390c004d10 con 0x7f391c10c920 2026-02-25T11:47:36.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f39197fa640 1 -- 10.20.193.28:0/776014979 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f390c0106b0 con 0x7f391c10c920 2026-02-25T11:47:36.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f3921641640 1 -- 10.20.193.28:0/776014979 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c128f50 con 0x7f391c10c920 2026-02-25T11:47:36.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.480+0000 7f3921641640 1 -- 10.20.193.28:0/776014979 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c169650 con 0x7f391c10c920 2026-02-25T11:47:36.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.483+0000 7f39197fa640 1 -- 10.20.193.28:0/776014979 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f390c01f060 con 0x7f391c10c920 2026-02-25T11:47:36.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.485+0000 7f39197fa640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f38f00b57f0 0x7f38f00d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.485+0000 7f39197fa640 1 -- 10.20.193.28:0/776014979 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f390c014070 con 0x7f391c10c920 2026-02-25T11:47:36.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.485+0000 7f391bfff640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f38f00b57f0 0x7f38f00d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.485+0000 7f391bfff640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f38f00b57f0 0x7f38f00d5ca0 secure :-1 s=READY pgs=227 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39080023d0 tx=0x7f3908031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:36.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f39248c5e00 1 Processor -- start 2026-02-25T11:47:36.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f39248c5e00 1 -- start start 2026-02-25T11:47:36.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f39248c5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d272787850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f39248c5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787d90 0x55d2727cc4f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f39248c5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2727cca30 0x55d2727ece10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f39248c5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d27269da30 con 0x55d2727cca30 2026-02-25T11:47:36.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f39248c5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d272699760 con 0x55d272787d90 2026-02-25T11:47:36.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f39248c5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d272693700 con 0x55d272787480 2026-02-25T11:47:36.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f391bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d272787850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f3920e40640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2727cca30 0x55d2727ece10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f391bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d272787850 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:46028/0 (socket says 10.20.193.28:46028) 2026-02-25T11:47:36.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f391bfff640 1 -- 10.20.193.28:0/372968969 learned_addr learned my addr 10.20.193.28:0/372968969 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:36.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f391b7fe640 1 --2- 10.20.193.28:0/372968969 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787d90 0x55d2727cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.486+0000 7f3920e40640 1 -- 10.20.193.28:0/372968969 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 msgr2=0x55d272787850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f3920e40640 1 --2- 10.20.193.28:0/372968969 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d272787850 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f3920e40640 1 -- 10.20.193.28:0/372968969 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787d90 msgr2=0x55d2727cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f3920e40640 1 --2- 10.20.193.28:0/372968969 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787d90 0x55d2727cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f3920e40640 1 -- 10.20.193.28:0/372968969 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27263ecf0 con 0x55d2727cca30 2026-02-25T11:47:36.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f391bfff640 1 --2- 10.20.193.28:0/372968969 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d272787850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:36.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f3920e40640 1 --2- 10.20.193.28:0/372968969 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2727cca30 0x55d2727ece10 secure :-1 s=READY pgs=304 gs=1 cs=0 l=1 c_cookie=11251142392560407948 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f391000abf0 tx=0x7f3910011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c24166519e6d58c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:36.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f38ecce3640 1 -- 10.20.193.28:0/372968969 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f39100050d0 con 0x55d2727cca30 2026-02-25T11:47:36.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f38ecce3640 1 -- 10.20.193.28:0/372968969 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3910003780 con 0x55d2727cca30 2026-02-25T11:47:36.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f38ecce3640 1 -- 10.20.193.28:0/372968969 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3910013760 con 0x55d2727cca30 2026-02-25T11:47:36.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f39248c5e00 1 -- 10.20.193.28:0/372968969 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2727cca30 msgr2=0x55d2727ece10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f39248c5e00 1 --2- 10.20.193.28:0/372968969 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2727cca30 0x55d2727ece10 secure :-1 s=READY pgs=304 gs=1 cs=0 l=1 c_cookie=11251142392560407948 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f391000abf0 tx=0x7f3910011740 comp rx=0 tx=0).stop 2026-02-25T11:47:36.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f39248c5e00 1 -- 10.20.193.28:0/372968969 shutdown_connections 2026-02-25T11:47:36.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f39248c5e00 1 --2- 10.20.193.28:0/372968969 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2727cca30 0x55d2727ece10 unknown :-1 s=CLOSED pgs=304 gs=1 cs=0 l=1 c_cookie=11251142392560407948 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f39248c5e00 1 --2- 10.20.193.28:0/372968969 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787d90 0x55d2727cc4f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f39248c5e00 1 --2- 10.20.193.28:0/372968969 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d272787850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f39248c5e00 1 -- 10.20.193.28:0/372968969 >> 10.20.193.28:0/372968969 conn(0x55d2726a7720 msgr2=0x55d272785d60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:36.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f39248c5e00 1 -- 10.20.193.28:0/372968969 shutdown_connections 2026-02-25T11:47:36.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.487+0000 7f39248c5e00 1 -- 10.20.193.28:0/372968969 wait complete. 2026-02-25T11:47:36.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.489+0000 7f39248c5e00 1 -- start start 2026-02-25T11:47:36.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.489+0000 7f39248c5e00 1 Processor -- start 2026-02-25T11:47:36.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.489+0000 7f39248c5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d2727e45f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.489+0000 7f39248c5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787d90 0x55d2727e4b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.489+0000 7f39248c5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2727cca30 0x55d2727e9b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.489+0000 7f39248c5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d2727ea060 con 0x55d2727cca30 2026-02-25T11:47:36.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.489+0000 7f39248c5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d2727ea1d0 con 0x55d272787d90 2026-02-25T11:47:36.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.489+0000 7f39248c5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d2727ea390 con 0x55d272787480 2026-02-25T11:47:36.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.489+0000 7f3920e40640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2727cca30 0x55d2727e9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f3920e40640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2727cca30 0x55d2727e9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35478/0 (socket says 10.20.193.28:35478) 2026-02-25T11:47:36.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f3920e40640 1 -- 10.20.193.28:0/883732148 learned_addr learned my addr 10.20.193.28:0/883732148 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:36.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f391bfff640 1 --2- 10.20.193.28:0/883732148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d2727e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f391b7fe640 1 --2- 10.20.193.28:0/883732148 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787d90 0x55d2727e4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f3920e40640 1 -- 10.20.193.28:0/883732148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 msgr2=0x55d2727e45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f3920e40640 1 --2- 10.20.193.28:0/883732148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d2727e45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f3920e40640 1 -- 10.20.193.28:0/883732148 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787d90 msgr2=0x55d2727e4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f3920e40640 1 --2- 10.20.193.28:0/883732148 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787d90 0x55d2727e4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f3920e40640 1 -- 10.20.193.28:0/883732148 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391000a890 con 0x55d2727cca30 2026-02-25T11:47:36.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f3920e40640 1 --2- 10.20.193.28:0/883732148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2727cca30 0x55d2727e9b20 secure :-1 s=READY pgs=305 gs=2 cs=0 l=1 c_cookie=6624000650161719450 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3910003eb0 tx=0x7f39100069d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bed2d7ef0d8889a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:36.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f38ddffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:47:36.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f38ddffb640 1 -- 10.20.193.28:0/883732148 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3910022070 con 0x55d2727cca30 2026-02-25T11:47:36.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f38ddffb640 1 -- 10.20.193.28:0/883732148 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3910005590 con 0x55d2727cca30 2026-02-25T11:47:36.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.490+0000 7f38ddffb640 1 -- 10.20.193.28:0/883732148 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f39100276f0 con 0x55d2727cca30 2026-02-25T11:47:36.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f39248c5e00 1 -- 10.20.193.28:0/883732148 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c17b080 con 0x55d2727cca30 2026-02-25T11:47:36.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f38ddffb640 1 -- 10.20.193.28:0/883732148 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f391001c450 con 0x55d2727cca30 2026-02-25T11:47:36.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f39248c5e00 10 client.15216 _ll_register_callbacks cb 0x55d272681a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:36.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f39248c5e00 1 -- 10.20.193.28:0/883732148 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d2723eb3b0 con 0x55d2727cca30 2026-02-25T11:47:36.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f38ddffb640 1 -- 10.20.193.28:0/883732148 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f39100138f0 con 0x55d2727cca30 2026-02-25T11:47:36.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f39248c5e00 10 client.15216 fetch_fsmap learned FSMap version 22 2026-02-25T11:47:36.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f39248c5e00 1 -- 10.20.193.28:0/883732148 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=22}) -- 0x55d2726172c0 con 0x55d2727cca30 2026-02-25T11:47:36.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f38ddffb640 1 -- 10.20.193.28:0/883732148 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f3910005070 con 0x55d2727cca30 2026-02-25T11:47:36.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f39248c5e00 10 client.15216 fetch_fsmap finished waiting for FSMap version 22 2026-02-25T11:47:36.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f39248c5e00 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2026-02-25T11:47:36.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f39248c5e00 20 client.15216 populate_metadata read hostname 'trial028' 2026-02-25T11:47:36.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f39248c5e00 1 --2- 10.20.193.28:0/883732148 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d2727caa30 0x55d2727cb7d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f39248c5e00 4 client.15216 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:47:36.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.491+0000 7f39248c5e00 1 -- 10.20.193.28:0/883732148 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d272689140 con 0x55d2727caa30 2026-02-25T11:47:36.500 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.492+0000 7f391bfff640 1 --2- 10.20.193.28:0/883732148 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d2727caa30 0x55d2727cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.493+0000 7f391bfff640 1 --2- 10.20.193.28:0/883732148 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d2727caa30 0x55d2727cb7d0 crc :-1 s=READY pgs=15 gs=4 cs=0 l=0 c_cookie=4045349868270670623 s_cookie=7789115342900764312 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3823f845a272531f server_cookie=6c187f22c8251e98 in_seq=0 out_seq=0 2026-02-25T11:47:36.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.493+0000 7f38ddffb640 10 client.15216 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:36.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f38ddffb640 1 -- 10.20.193.28:0/883732148 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55d272689140 con 0x55d2727caa30 2026-02-25T11:47:36.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f38ddffb640 10 client.15216 handle_command_reply: tid=0 2026-02-25T11:47:36.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f39248c5e00 1 client.15216 shutdown 2026-02-25T11:47:36.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f39248c5e00 1 -- 10.20.193.28:0/883732148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2727cca30 msgr2=0x55d2727e9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f39248c5e00 1 --2- 10.20.193.28:0/883732148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2727cca30 0x55d2727e9b20 secure :-1 s=READY pgs=305 gs=2 cs=0 l=1 c_cookie=6624000650161719450 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3910003eb0 tx=0x7f39100069d0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f39248c5e00 1 -- 10.20.193.28:0/883732148 shutdown_connections 2026-02-25T11:47:36.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f39248c5e00 1 --2- 10.20.193.28:0/883732148 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d2727caa30 0x55d2727cb7d0 crc :-1 s=READY pgs=15 gs=4 cs=0 l=0 c_cookie=4045349868270670623 s_cookie=7789115342900764312 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f39248c5e00 1 --2- 10.20.193.28:0/883732148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2727cca30 0x55d2727e9b20 unknown :-1 s=CLOSED pgs=305 gs=2 cs=0 l=1 c_cookie=6624000650161719450 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f39248c5e00 1 --2- 10.20.193.28:0/883732148 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787d90 0x55d2727e4b30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f39248c5e00 1 --2- 10.20.193.28:0/883732148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d2727e45f0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f38ddffb640 0 client.15216 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:36.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f39248c5e00 1 -- 10.20.193.28:0/883732148 >> 10.20.193.28:0/883732148 conn(0x55d2726a7720 msgr2=0x55d272785570 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:36.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f39248c5e00 1 -- 10.20.193.28:0/883732148 shutdown_connections 2026-02-25T11:47:36.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f39248c5e00 1 -- 10.20.193.28:0/883732148 wait complete. 2026-02-25T11:47:36.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.494+0000 7f39248c5e00 20 client.15216 trim_cache size 0 max 16384 2026-02-25T11:47:36.503 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:36 trial167 ceph-mon[43392]: pgmap v205: 97 pgs: 97 active+clean; 641 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2026-02-25T11:47:36.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.503+0000 7f39248c5e00 1 Processor -- start 2026-02-25T11:47:36.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f39248c5e00 1 -- start start 2026-02-25T11:47:36.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f39248c5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d2727d1850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f39248c5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 0x55d2727cee10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f39248c5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2727cca30 0x55d2727cf350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f39248c5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d272691330 con 0x55d272787d90 2026-02-25T11:47:36.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f39248c5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d272351610 con 0x55d2727cca30 2026-02-25T11:47:36.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f39248c5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d2727ca590 con 0x55d272787480 2026-02-25T11:47:36.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f391b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 0x55d2727cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f391b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 0x55d2727cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35484/0 (socket says 10.20.193.28:35484) 2026-02-25T11:47:36.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f391b7fe640 1 -- 10.20.193.28:0/1959446000 learned_addr learned my addr 10.20.193.28:0/1959446000 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:36.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f3920e40640 1 --2- 10.20.193.28:0/1959446000 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2727cca30 0x55d2727cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f391bfff640 1 --2- 10.20.193.28:0/1959446000 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d2727d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f391b7fe640 1 -- 10.20.193.28:0/1959446000 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 msgr2=0x55d2727d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f391b7fe640 1 --2- 10.20.193.28:0/1959446000 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d2727d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f391b7fe640 1 -- 10.20.193.28:0/1959446000 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2727cca30 msgr2=0x55d2727cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f391b7fe640 1 --2- 10.20.193.28:0/1959446000 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2727cca30 0x55d2727cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f391b7fe640 1 -- 10.20.193.28:0/1959446000 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2727d69b0 con 0x55d272787d90 2026-02-25T11:47:36.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f391b7fe640 1 --2- 10.20.193.28:0/1959446000 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 0x55d2727cee10 secure :-1 s=READY pgs=306 gs=1 cs=0 l=1 c_cookie=13250205896371117258 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f390c00c6a0 tx=0x7f390c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e2300f65fc9cca server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:36.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.504+0000 7f38dffff640 1 -- 10.20.193.28:0/1959446000 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f390c0ac970 con 0x55d272787d90 2026-02-25T11:47:36.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.505+0000 7f38dffff640 1 -- 10.20.193.28:0/1959446000 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f390c004e80 con 0x55d272787d90 2026-02-25T11:47:36.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.505+0000 7f38dffff640 1 -- 10.20.193.28:0/1959446000 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f390c0b5500 con 0x55d272787d90 2026-02-25T11:47:36.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.505+0000 7f39248c5e00 1 -- 10.20.193.28:0/1959446000 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 msgr2=0x55d2727cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.505+0000 7f39248c5e00 1 --2- 10.20.193.28:0/1959446000 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 0x55d2727cee10 secure :-1 s=READY pgs=306 gs=1 cs=0 l=1 c_cookie=13250205896371117258 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f390c00c6a0 tx=0x7f390c009260 comp rx=0 tx=0).stop 2026-02-25T11:47:36.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.505+0000 7f39248c5e00 1 -- 10.20.193.28:0/1959446000 shutdown_connections 2026-02-25T11:47:36.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.505+0000 7f39248c5e00 1 --2- 10.20.193.28:0/1959446000 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2727cca30 0x55d2727cf350 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.505+0000 7f39248c5e00 1 --2- 10.20.193.28:0/1959446000 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 0x55d2727cee10 unknown :-1 s=CLOSED pgs=306 gs=1 cs=0 l=1 c_cookie=13250205896371117258 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.505+0000 7f39248c5e00 1 --2- 10.20.193.28:0/1959446000 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d272787480 0x55d2727d1850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.505+0000 7f39248c5e00 1 -- 10.20.193.28:0/1959446000 >> 10.20.193.28:0/1959446000 conn(0x55d2726a7720 msgr2=0x55d2727c9610 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:36.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.505+0000 7f39248c5e00 1 -- 10.20.193.28:0/1959446000 shutdown_connections 2026-02-25T11:47:36.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.505+0000 7f39248c5e00 1 -- 10.20.193.28:0/1959446000 wait complete. 2026-02-25T11:47:36.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:36 trial119 ceph-mon[42710]: pgmap v205: 97 pgs: 97 active+clean; 641 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2026-02-25T11:47:36.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f39248c5e00 1 -- start start 2026-02-25T11:47:36.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f39248c5e00 1 Processor -- start 2026-02-25T11:47:36.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f39248c5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787480 0x55d2727862c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f39248c5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 0x55d272786800 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f39248c5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2727cca30 0x55d272786d60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.515 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f39248c5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d2727f0520 con 0x55d272787d90 2026-02-25T11:47:36.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f39248c5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d2727f0690 con 0x55d272787480 2026-02-25T11:47:36.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f39248c5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d2727f0850 con 0x55d2727cca30 2026-02-25T11:47:36.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f391b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 0x55d272786800 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f391b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 0x55d272786800 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35486/0 (socket says 10.20.193.28:35486) 2026-02-25T11:47:36.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f391b7fe640 1 -- 10.20.193.28:0/1331734377 learned_addr learned my addr 10.20.193.28:0/1331734377 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:36.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f391bfff640 1 --2- 10.20.193.28:0/1331734377 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787480 0x55d2727862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f3920e40640 1 --2- 10.20.193.28:0/1331734377 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2727cca30 0x55d272786d60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f391b7fe640 1 -- 10.20.193.28:0/1331734377 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2727cca30 msgr2=0x55d272786d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f391b7fe640 1 --2- 10.20.193.28:0/1331734377 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2727cca30 0x55d272786d60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f391b7fe640 1 -- 10.20.193.28:0/1331734377 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787480 msgr2=0x55d2727862c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:36.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f391b7fe640 1 --2- 10.20.193.28:0/1331734377 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787480 0x55d2727862c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.506+0000 7f391b7fe640 1 -- 10.20.193.28:0/1331734377 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2726290c0 con 0x55d272787d90 2026-02-25T11:47:36.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f391b7fe640 1 --2- 10.20.193.28:0/1331734377 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 0x55d272786800 secure :-1 s=READY pgs=307 gs=1 cs=0 l=1 c_cookie=8097047322675706090 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f390c0a8a50 tx=0x7f390c0acf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705e7da3d59cf8ea server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:36.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f38ddffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:47:36.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f38ddffb640 1 -- 10.20.193.28:0/1331734377 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f390c0b54e0 con 0x55d272787d90 2026-02-25T11:47:36.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f38ddffb640 1 -- 10.20.193.28:0/1331734377 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f390c0ac970 con 0x55d272787d90 2026-02-25T11:47:36.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f38ddffb640 1 -- 10.20.193.28:0/1331734377 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f390c0ae080 con 0x55d272787d90 2026-02-25T11:47:36.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f39248c5e00 1 -- 10.20.193.28:0/1331734377 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c169650 con 0x55d272787d90 2026-02-25T11:47:36.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f38ddffb640 1 -- 10.20.193.28:0/1331734377 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f390c0b7e20 con 0x55d272787d90 2026-02-25T11:47:36.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f38ddffb640 10 client.15222 inactive, discarding osd_map(86..86 src has 1..86) 2026-02-25T11:47:36.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f39248c5e00 10 client.15222 _ll_register_callbacks cb 0x55d272681a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:36.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f39248c5e00 1 -- 10.20.193.28:0/1331734377 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3910003780 con 0x55d272787d90 2026-02-25T11:47:36.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f38ddffb640 1 -- 10.20.193.28:0/1331734377 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f390c0b2f00 con 0x55d272787d90 2026-02-25T11:47:36.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f39248c5e00 10 client.15222 fetch_fsmap learned FSMap version 22 2026-02-25T11:47:36.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f39248c5e00 1 -- 10.20.193.28:0/1331734377 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=22}) -- 0x55d272623b40 con 0x55d272787d90 2026-02-25T11:47:36.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f38ddffb640 1 -- 10.20.193.28:0/1331734377 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f390c00b550 con 0x55d272787d90 2026-02-25T11:47:36.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f39248c5e00 10 client.15222 fetch_fsmap finished waiting for FSMap version 22 2026-02-25T11:47:36.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f39248c5e00 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2026-02-25T11:47:36.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f39248c5e00 20 client.15222 populate_metadata read hostname 'trial028' 2026-02-25T11:47:36.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f39248c5e00 1 --2- 10.20.193.28:0/1331734377 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d27287eee0 0x55d27287dbf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:36.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f39248c5e00 4 client.15222 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "status"}] 2026-02-25T11:47:36.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.507+0000 7f39248c5e00 1 -- 10.20.193.28:0/1331734377 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "status"}) -- 0x55d2723d0070 con 0x55d27287eee0 2026-02-25T11:47:36.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.508+0000 7f391bfff640 1 --2- 10.20.193.28:0/1331734377 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d27287eee0 0x55d27287dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:36.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f38ddffb640 10 client.15222 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:36.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f391bfff640 1 --2- 10.20.193.28:0/1331734377 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d27287eee0 0x55d27287dbf0 crc :-1 s=READY pgs=16 gs=4 cs=0 l=0 c_cookie=466599339931910704 s_cookie=10016009605049739917 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679b1a0a512c230 server_cookie=8b0003ab2fe3ea8d in_seq=0 out_seq=0 2026-02-25T11:47:36.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f38ddffb640 1 -- 10.20.193.28:0/1331734377 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1223 (crc 0 0 0) 0x55d2723d0070 con 0x55d27287eee0 2026-02-25T11:47:36.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f38ddffb640 10 client.15222 handle_command_reply: tid=0 2026-02-25T11:47:36.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f39248c5e00 1 client.15222 shutdown 2026-02-25T11:47:36.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f39248c5e00 1 -- 10.20.193.28:0/1331734377 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 msgr2=0x55d272786800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f39248c5e00 1 --2- 10.20.193.28:0/1331734377 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 0x55d272786800 secure :-1 s=READY pgs=307 gs=1 cs=0 l=1 c_cookie=8097047322675706090 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f390c0a8a50 tx=0x7f390c0acf20 comp rx=0 tx=0).stop 2026-02-25T11:47:36.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f39248c5e00 1 -- 10.20.193.28:0/1331734377 shutdown_connections 2026-02-25T11:47:36.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f39248c5e00 1 --2- 10.20.193.28:0/1331734377 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d27287eee0 0x55d27287dbf0 crc :-1 s=READY pgs=16 gs=4 cs=0 l=0 c_cookie=466599339931910704 s_cookie=10016009605049739917 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f39248c5e00 1 --2- 10.20.193.28:0/1331734377 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2727cca30 0x55d272786d60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f39248c5e00 1 --2- 10.20.193.28:0/1331734377 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d272787d90 0x55d272786800 unknown :-1 s=CLOSED pgs=307 gs=1 cs=0 l=1 c_cookie=8097047322675706090 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f39248c5e00 1 --2- 10.20.193.28:0/1331734377 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d272787480 0x55d2727862c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f38ddffb640 0 client.15222 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:36.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.509+0000 7f39248c5e00 1 -- 10.20.193.28:0/1331734377 >> 10.20.193.28:0/1331734377 conn(0x55d2726a7720 msgr2=0x55d2727cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:36.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f39248c5e00 1 -- 10.20.193.28:0/1331734377 shutdown_connections 2026-02-25T11:47:36.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f39248c5e00 1 -- 10.20.193.28:0/1331734377 wait complete. 2026-02-25T11:47:36.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f39248c5e00 20 client.15222 trim_cache size 0 max 16384 2026-02-25T11:47:36.521 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:47:36.521 INFO:teuthology.orchestra.run.trial028.stdout: "cluster_fsid": "123fa7bf-123f-11f1-80da-d404e6e7d460", 2026-02-25T11:47:36.521 INFO:teuthology.orchestra.run.trial028.stdout: "whoami": 0, 2026-02-25T11:47:36.521 INFO:teuthology.orchestra.run.trial028.stdout: "id": 24749, 2026-02-25T11:47:36.521 INFO:teuthology.orchestra.run.trial028.stdout: "want_state": "up:active", 2026-02-25T11:47:36.521 INFO:teuthology.orchestra.run.trial028.stdout: "state": "up:active", 2026-02-25T11:47:36.521 INFO:teuthology.orchestra.run.trial028.stdout: "fs_name": "cephfs", 2026-02-25T11:47:36.522 INFO:teuthology.orchestra.run.trial028.stdout: "rank_uptime": 22.883485885999999, 2026-02-25T11:47:36.522 INFO:teuthology.orchestra.run.trial028.stdout: "mdsmap_epoch": 22, 2026-02-25T11:47:36.522 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_epoch": 86, 2026-02-25T11:47:36.522 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_epoch_barrier": 86, 2026-02-25T11:47:36.522 INFO:teuthology.orchestra.run.trial028.stdout: "uptime": 29.360239758999999, 2026-02-25T11:47:36.522 INFO:teuthology.orchestra.run.trial028.stdout: "sysinfo": { 2026-02-25T11:47:36.522 INFO:teuthology.orchestra.run.trial028.stdout: "arch": "x86_64", 2026-02-25T11:47:36.522 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_release": "tentacle", 2026-02-25T11:47:36.522 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version": "ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev - RelWithDebInfo)", 2026-02-25T11:47:36.522 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version_short": "20.3.0-5549-g92715cd3", 2026-02-25T11:47:36.523 INFO:teuthology.orchestra.run.trial028.stdout: "container_hostname": "trial119", 2026-02-25T11:47:36.523 INFO:teuthology.orchestra.run.trial028.stdout: "container_image": "quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33", 2026-02-25T11:47:36.523 INFO:teuthology.orchestra.run.trial028.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-25T11:47:36.523 INFO:teuthology.orchestra.run.trial028.stdout: "distro": "centos", 2026-02-25T11:47:36.523 INFO:teuthology.orchestra.run.trial028.stdout: "distro_description": "CentOS Stream 9", 2026-02-25T11:47:36.523 INFO:teuthology.orchestra.run.trial028.stdout: "distro_version": "9", 2026-02-25T11:47:36.523 INFO:teuthology.orchestra.run.trial028.stdout: "hostname": "trial119", 2026-02-25T11:47:36.523 INFO:teuthology.orchestra.run.trial028.stdout: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Tue Feb 24 21:03:19 EST 2026", 2026-02-25T11:47:36.523 INFO:teuthology.orchestra.run.trial028.stdout: "kernel_version": "6.19.0-g712f4a43c125", 2026-02-25T11:47:36.523 INFO:teuthology.orchestra.run.trial028.stdout: "mem_swap_kb": "8388604", 2026-02-25T11:47:36.524 INFO:teuthology.orchestra.run.trial028.stdout: "mem_total_kb": "130448332", 2026-02-25T11:47:36.524 INFO:teuthology.orchestra.run.trial028.stdout: "os": "Linux" 2026-02-25T11:47:36.524 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:47:36.524 INFO:teuthology.orchestra.run.trial028.stdout: "endian": "little" 2026-02-25T11:47:36.524 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:47:36.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f3921641640 1 -- 10.20.193.28:0/776014979 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f38f00b57f0 msgr2=0x7f38f00d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f3921641640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f38f00b57f0 0x7f38f00d5ca0 secure :-1 s=READY pgs=227 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f39080023d0 tx=0x7f3908031040 comp rx=0 tx=0).stop 2026-02-25T11:47:36.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f3921641640 1 -- 10.20.193.28:0/776014979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f391c10c920 msgr2=0x7f391c122f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:36.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f3921641640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f391c10c920 0x7f391c122f50 secure :-1 s=READY pgs=303 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f390c009480 tx=0x7f390c00ee60 comp rx=0 tx=0).stop 2026-02-25T11:47:36.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f3921641640 1 -- 10.20.193.28:0/776014979 shutdown_connections 2026-02-25T11:47:36.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f3921641640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f38f00b57f0 0x7f38f00d5ca0 unknown :-1 s=CLOSED pgs=227 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f3921641640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f391c111510 0x7f391c128320 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f3921641640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f391c10c920 0x7f391c122f50 unknown :-1 s=CLOSED pgs=303 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f3921641640 1 --2- 10.20.193.28:0/776014979 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f391c10bf30 0x7f391c122a10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:36.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f3921641640 1 -- 10.20.193.28:0/776014979 >> 10.20.193.28:0/776014979 conn(0x7f391c0e9b40 msgr2=0x7f391c109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:36.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f3921641640 1 -- 10.20.193.28:0/776014979 shutdown_connections 2026-02-25T11:47:36.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:36.510+0000 7f3921641640 1 -- 10.20.193.28:0/776014979 wait complete. 2026-02-25T11:47:36.647 INFO:tasks.cephadm:Setup complete, yielding 2026-02-25T11:47:36.648 INFO:teuthology.run_tasks:Running task cephadm.shell... 2026-02-25T11:47:36.662 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@trial028.front.sepia.ceph.com 2026-02-25T11:47:36.662 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- bash -c 'ceph orch status' 2026-02-25T11:47:36.873 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:37.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.050+0000 7f3e529dd640 1 -- 10.20.193.28:0/12721665 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3e4c116d80 msgr2=0x7f3e4c1171f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:37.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.050+0000 7f3e529dd640 1 --2- 10.20.193.28:0/12721665 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3e4c116d80 0x7f3e4c1171f0 secure :-1 s=READY pgs=308 gs=1 cs=0 l=1 c_cookie=4883657845414655961 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3e3c009950 tx=0x7f3e3c02f190 comp rx=0 tx=0).stop 2026-02-25T11:47:37.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.050+0000 7f3e529dd640 1 -- 10.20.193.28:0/12721665 shutdown_connections 2026-02-25T11:47:37.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.050+0000 7f3e529dd640 1 --2- 10.20.193.28:0/12721665 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3e4c117730 0x7f3e4c1780c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.050+0000 7f3e529dd640 1 --2- 10.20.193.28:0/12721665 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3e4c116d80 0x7f3e4c1171f0 unknown :-1 s=CLOSED pgs=308 gs=1 cs=0 l=1 c_cookie=4883657845414655961 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.050+0000 7f3e529dd640 1 --2- 10.20.193.28:0/12721665 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3e4c115bf0 0x7f3e4c115fe0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.050+0000 7f3e529dd640 1 -- 10.20.193.28:0/12721665 >> 10.20.193.28:0/12721665 conn(0x7f3e4c05d960 msgr2=0x7f3e4c05dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:37.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.050+0000 7f3e529dd640 1 -- 10.20.193.28:0/12721665 shutdown_connections 2026-02-25T11:47:37.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.051+0000 7f3e529dd640 1 -- 10.20.193.28:0/12721665 wait complete. 2026-02-25T11:47:37.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.051+0000 7f3e529dd640 1 Processor -- start 2026-02-25T11:47:37.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.051+0000 7f3e529dd640 1 -- start start 2026-02-25T11:47:37.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e529dd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3e4c115bf0 0x7f3e4c1039d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:37.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e529dd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3e4c116d80 0x7f3e4c1780c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:37.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e529dd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3e4c117730 0x7f3e4c104230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:37.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e529dd640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c104e20 con 0x7f3e4c117730 2026-02-25T11:47:37.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e529dd640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c104f90 con 0x7f3e4c115bf0 2026-02-25T11:47:37.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e529dd640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c105150 con 0x7f3e4c116d80 2026-02-25T11:47:37.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e50f53640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3e4c117730 0x7f3e4c104230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:37.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e4b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3e4c116d80 0x7f3e4c1780c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:37.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e50f53640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3e4c117730 0x7f3e4c104230 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35508/0 (socket says 10.20.193.28:35508) 2026-02-25T11:47:37.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e50f53640 1 -- 10.20.193.28:0/531610322 learned_addr learned my addr 10.20.193.28:0/531610322 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:37.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e4bfff640 1 --2- 10.20.193.28:0/531610322 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3e4c115bf0 0x7f3e4c1039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:37.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e50f53640 1 -- 10.20.193.28:0/531610322 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3e4c116d80 msgr2=0x7f3e4c1780c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:37.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e50f53640 1 --2- 10.20.193.28:0/531610322 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3e4c116d80 0x7f3e4c1780c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e50f53640 1 -- 10.20.193.28:0/531610322 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3e4c115bf0 msgr2=0x7f3e4c1039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:37.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e50f53640 1 --2- 10.20.193.28:0/531610322 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3e4c115bf0 0x7f3e4c1039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.052+0000 7f3e50f53640 1 -- 10.20.193.28:0/531610322 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3c009570 con 0x7f3e4c117730 2026-02-25T11:47:37.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.053+0000 7f3e50f53640 1 --2- 10.20.193.28:0/531610322 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3e4c117730 0x7f3e4c104230 secure :-1 s=READY pgs=309 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3e4000f8d0 tx=0x7f3e4000fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:37.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.053+0000 7f3e497fa640 1 -- 10.20.193.28:0/531610322 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3e4000dce0 con 0x7f3e4c117730 2026-02-25T11:47:37.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.053+0000 7f3e497fa640 1 -- 10.20.193.28:0/531610322 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3e4000de40 con 0x7f3e4c117730 2026-02-25T11:47:37.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.053+0000 7f3e529dd640 1 -- 10.20.193.28:0/531610322 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4c105430 con 0x7f3e4c117730 2026-02-25T11:47:37.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.053+0000 7f3e497fa640 1 -- 10.20.193.28:0/531610322 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3e400116b0 con 0x7f3e4c117730 2026-02-25T11:47:37.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.053+0000 7f3e529dd640 1 -- 10.20.193.28:0/531610322 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4c1058b0 con 0x7f3e4c117730 2026-02-25T11:47:37.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.053+0000 7f3e529dd640 1 -- 10.20.193.28:0/531610322 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4c055e30 con 0x7f3e4c117730 2026-02-25T11:47:37.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.055+0000 7f3e497fa640 1 -- 10.20.193.28:0/531610322 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f3e40020060 con 0x7f3e4c117730 2026-02-25T11:47:37.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.057+0000 7f3e497fa640 1 --2- 10.20.193.28:0/531610322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3e200b57f0 0x7f3e200d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:37.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.057+0000 7f3e497fa640 1 -- 10.20.193.28:0/531610322 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f3e40015070 con 0x7f3e4c117730 2026-02-25T11:47:37.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.057+0000 7f3e4bfff640 1 --2- 10.20.193.28:0/531610322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3e200b57f0 0x7f3e200d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:37.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.057+0000 7f3e497fa640 1 -- 10.20.193.28:0/531610322 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f3e400e80f0 con 0x7f3e4c117730 2026-02-25T11:47:37.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.058+0000 7f3e4bfff640 1 --2- 10.20.193.28:0/531610322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3e200b57f0 0x7f3e200d5ca0 secure :-1 s=READY pgs=228 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3e38004740 tx=0x7f3e38004420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:37.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.181+0000 7f3e529dd640 1 -- 10.20.193.28:0/531610322 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3e4c17b610 con 0x7f3e200b57f0 2026-02-25T11:47:37.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.182+0000 7f3e497fa640 1 -- 10.20.193.28:0/531610322 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3e4c17b610 con 0x7f3e200b57f0 2026-02-25T11:47:37.185 INFO:teuthology.orchestra.run.trial028.stdout:Backend: cephadm 2026-02-25T11:47:37.185 INFO:teuthology.orchestra.run.trial028.stdout:Available: Yes 2026-02-25T11:47:37.185 INFO:teuthology.orchestra.run.trial028.stdout:Paused: No 2026-02-25T11:47:37.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.183+0000 7f3e529dd640 1 -- 10.20.193.28:0/531610322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3e200b57f0 msgr2=0x7f3e200d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:37.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.183+0000 7f3e529dd640 1 --2- 10.20.193.28:0/531610322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3e200b57f0 0x7f3e200d5ca0 secure :-1 s=READY pgs=228 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3e38004740 tx=0x7f3e38004420 comp rx=0 tx=0).stop 2026-02-25T11:47:37.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.183+0000 7f3e529dd640 1 -- 10.20.193.28:0/531610322 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3e4c117730 msgr2=0x7f3e4c104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:37.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.183+0000 7f3e529dd640 1 --2- 10.20.193.28:0/531610322 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3e4c117730 0x7f3e4c104230 secure :-1 s=READY pgs=309 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3e4000f8d0 tx=0x7f3e4000fda0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.183+0000 7f3e529dd640 1 -- 10.20.193.28:0/531610322 shutdown_connections 2026-02-25T11:47:37.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.183+0000 7f3e529dd640 1 --2- 10.20.193.28:0/531610322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3e200b57f0 0x7f3e200d5ca0 unknown :-1 s=CLOSED pgs=228 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.183+0000 7f3e529dd640 1 --2- 10.20.193.28:0/531610322 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3e4c117730 0x7f3e4c104230 unknown :-1 s=CLOSED pgs=309 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.183+0000 7f3e529dd640 1 --2- 10.20.193.28:0/531610322 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3e4c116d80 0x7f3e4c1780c0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.183+0000 7f3e529dd640 1 --2- 10.20.193.28:0/531610322 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3e4c115bf0 0x7f3e4c1039d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.183+0000 7f3e529dd640 1 -- 10.20.193.28:0/531610322 >> 10.20.193.28:0/531610322 conn(0x7f3e4c05d960 msgr2=0x7f3e4c0f72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:37.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.183+0000 7f3e529dd640 1 -- 10.20.193.28:0/531610322 shutdown_connections 2026-02-25T11:47:37.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.184+0000 7f3e529dd640 1 -- 10.20.193.28:0/531610322 wait complete. 2026-02-25T11:47:37.322 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- bash -c 'ceph orch ps' 2026-02-25T11:47:37.503 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:37.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.674+0000 7f69e798a640 1 -- 10.20.193.28:0/3061094439 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e014f140 msgr2=0x7f69e016f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:37.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.674+0000 7f69e798a640 1 --2- 10.20.193.28:0/3061094439 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e014f140 0x7f69e016f520 secure :-1 s=READY pgs=310 gs=1 cs=0 l=1 c_cookie=13347404446960411926 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69dc00aa30 tx=0x7f69dc030270 comp rx=0 tx=0).stop 2026-02-25T11:47:37.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.675+0000 7f69e798a640 1 -- 10.20.193.28:0/3061094439 shutdown_connections 2026-02-25T11:47:37.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.675+0000 7f69e798a640 1 --2- 10.20.193.28:0/3061094439 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e014f140 0x7f69e016f520 unknown :-1 s=CLOSED pgs=310 gs=1 cs=0 l=1 c_cookie=13347404446960411926 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.675+0000 7f69e798a640 1 --2- 10.20.193.28:0/3061094439 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69e010b880 0x7f69e014ec00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.675+0000 7f69e798a640 1 --2- 10.20.193.28:0/3061094439 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e010af50 0x7f69e010b340 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.675+0000 7f69e798a640 1 -- 10.20.193.28:0/3061094439 >> 10.20.193.28:0/3061094439 conn(0x7f69e00e8b10 msgr2=0x7f69e0108f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:37.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.675+0000 7f69e798a640 1 -- 10.20.193.28:0/3061094439 shutdown_connections 2026-02-25T11:47:37.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.675+0000 7f69e798a640 1 -- 10.20.193.28:0/3061094439 wait complete. 2026-02-25T11:47:37.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.675+0000 7f69e798a640 1 Processor -- start 2026-02-25T11:47:37.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e798a640 1 -- start start 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e798a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e010af50 0x7f69e015fdf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e798a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69e010b880 0x7f69e0160440 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e798a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e014f140 0x7f69e0161040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e798a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f69e0161780 con 0x7f69e014f140 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e798a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69e01618f0 con 0x7f69e010b880 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e798a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69e0161ab0 con 0x7f69e010af50 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e56ff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e010af50 0x7f69e015fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e56ff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e010af50 0x7f69e015fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:46086/0 (socket says 10.20.193.28:46086) 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e56ff640 1 -- 10.20.193.28:0/3351754322 learned_addr learned my addr 10.20.193.28:0/3351754322 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e5f00640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e014f140 0x7f69e0161040 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e4efe640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69e010b880 0x7f69e0160440 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e5f00640 1 -- 10.20.193.28:0/3351754322 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e010af50 msgr2=0x7f69e015fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e5f00640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e010af50 0x7f69e015fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e5f00640 1 -- 10.20.193.28:0/3351754322 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69e010b880 msgr2=0x7f69e0160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e5f00640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69e010b880 0x7f69e0160440 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e5f00640 1 -- 10.20.193.28:0/3351754322 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc00a650 con 0x7f69e014f140 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.676+0000 7f69e56ff640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e010af50 0x7f69e015fdf0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:37.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.677+0000 7f69e5f00640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e014f140 0x7f69e0161040 secure :-1 s=READY pgs=311 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69dc00aa30 tx=0x7f69dc0039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:37.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.677+0000 7f69ce7fc640 1 -- 10.20.193.28:0/3351754322 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f69dc03e070 con 0x7f69e014f140 2026-02-25T11:47:37.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.677+0000 7f69ce7fc640 1 -- 10.20.193.28:0/3351754322 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f69dc003e30 con 0x7f69e014f140 2026-02-25T11:47:37.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.677+0000 7f69ce7fc640 1 -- 10.20.193.28:0/3351754322 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f69dc042940 con 0x7f69e014f140 2026-02-25T11:47:37.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.677+0000 7f69e798a640 1 -- 10.20.193.28:0/3351754322 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e0161d30 con 0x7f69e014f140 2026-02-25T11:47:37.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.677+0000 7f69e798a640 1 -- 10.20.193.28:0/3351754322 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e014cb10 con 0x7f69e014f140 2026-02-25T11:47:37.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.677+0000 7f69e798a640 1 -- 10.20.193.28:0/3351754322 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a8005300 con 0x7f69e014f140 2026-02-25T11:47:37.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.680+0000 7f69ce7fc640 1 -- 10.20.193.28:0/3351754322 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f69dc0054a0 con 0x7f69e014f140 2026-02-25T11:47:37.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.681+0000 7f69ce7fc640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69bc0b57f0 0x7f69bc0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:37.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.681+0000 7f69ce7fc640 1 -- 10.20.193.28:0/3351754322 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f69dc10c530 con 0x7f69e014f140 2026-02-25T11:47:37.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.682+0000 7f69e56ff640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69bc0b57f0 0x7f69bc0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:37.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.682+0000 7f69ce7fc640 1 -- 10.20.193.28:0/3351754322 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f69dc10c9b0 con 0x7f69e014f140 2026-02-25T11:47:37.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.682+0000 7f69e56ff640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69bc0b57f0 0x7f69bc0d5ca0 secure :-1 s=READY pgs=229 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69d00095b0 tx=0x7f69d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:37.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.800+0000 7f69e798a640 1 -- 10.20.193.28:0/3351754322 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f69a8002bb0 con 0x7f69bc0b57f0 2026-02-25T11:47:37.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.807+0000 7f69ce7fc640 1 -- 10.20.193.28:0/3351754322 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f69a8002bb0 con 0x7f69bc0b57f0 2026-02-25T11:47:37.810 INFO:teuthology.orchestra.run.trial028.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2026-02-25T11:47:37.810 INFO:teuthology.orchestra.run.trial028.stdout:mds.a trial028 running (32s) 28s ago 31s 13.5M - 20.3.0-5549-g92715cd3 8e51ef34fd35 23038f7d2dd3 2026-02-25T11:47:37.810 INFO:teuthology.orchestra.run.trial028.stdout:mds.b trial119 running (30s) 28s ago 30s 13.3M - 20.3.0-5549-g92715cd3 8e51ef34fd35 c566a26541b5 2026-02-25T11:47:37.810 INFO:teuthology.orchestra.run.trial028.stdout:mds.c trial167 running (37s) 28s ago 37s 13.4M - 20.3.0-5549-g92715cd3 8e51ef34fd35 26211a9b34e7 2026-02-25T11:47:37.811 INFO:teuthology.orchestra.run.trial028.stdout:mds.d trial028 running (40s) 28s ago 40s 13.9M - 20.3.0-5549-g92715cd3 8e51ef34fd35 b798644ac443 2026-02-25T11:47:37.811 INFO:teuthology.orchestra.run.trial028.stdout:mds.e trial119 running (39s) 28s ago 39s 13.5M - 20.3.0-5549-g92715cd3 8e51ef34fd35 e1c2f92764ae 2026-02-25T11:47:37.811 INFO:teuthology.orchestra.run.trial028.stdout:mds.f trial167 running (38s) 28s ago 38s 13.8M - 20.3.0-5549-g92715cd3 8e51ef34fd35 2ea3e1280d47 2026-02-25T11:47:37.811 INFO:teuthology.orchestra.run.trial028.stdout:mds.g trial028 running (33s) 28s ago 33s 13.5M - 20.3.0-5549-g92715cd3 8e51ef34fd35 a7134b0cb083 2026-02-25T11:47:37.811 INFO:teuthology.orchestra.run.trial028.stdout:mds.h trial119 running (35s) 28s ago 35s 13.8M - 20.3.0-5549-g92715cd3 8e51ef34fd35 7aae0e577c66 2026-02-25T11:47:37.811 INFO:teuthology.orchestra.run.trial028.stdout:mds.i trial167 running (34s) 28s ago 34s 13.7M - 20.3.0-5549-g92715cd3 8e51ef34fd35 582e7d64a1eb 2026-02-25T11:47:37.811 INFO:teuthology.orchestra.run.trial028.stdout:mds.j trial028 running (42s) 28s ago 42s 13.5M - 20.3.0-5549-g92715cd3 8e51ef34fd35 5dbafe81b84f 2026-02-25T11:47:37.811 INFO:teuthology.orchestra.run.trial028.stdout:mds.k trial119 running (43s) 28s ago 43s 13.4M - 20.3.0-5549-g92715cd3 8e51ef34fd35 6b629be09a36 2026-02-25T11:47:37.811 INFO:teuthology.orchestra.run.trial028.stdout:mds.l trial167 running (44s) 28s ago 44s 14.0M - 20.3.0-5549-g92715cd3 8e51ef34fd35 6352c49e38bb 2026-02-25T11:47:37.811 INFO:teuthology.orchestra.run.trial028.stdout:mgr.x trial028 *:9283,8765 running (4m) 28s ago 4m 568M - 20.3.0-5549-g92715cd3 8e51ef34fd35 d6b9f684dc01 2026-02-25T11:47:37.811 INFO:teuthology.orchestra.run.trial028.stdout:mgr.y trial119 *:8443,8765 running (3m) 28s ago 3m 497M - 20.3.0-5549-g92715cd3 8e51ef34fd35 defaf63b2fbc 2026-02-25T11:47:37.811 INFO:teuthology.orchestra.run.trial028.stdout:mgr.z trial167 *:8443,8765 running (3m) 28s ago 3m 497M - 20.3.0-5549-g92715cd3 8e51ef34fd35 516c7ac1c961 2026-02-25T11:47:37.812 INFO:teuthology.orchestra.run.trial028.stdout:mon.a trial028 running (4m) 28s ago 4m 54.3M 2048M 20.3.0-5549-g92715cd3 8e51ef34fd35 42d9b9b6f58f 2026-02-25T11:47:37.812 INFO:teuthology.orchestra.run.trial028.stdout:mon.b trial119 running (3m) 28s ago 3m 47.2M 2048M 20.3.0-5549-g92715cd3 8e51ef34fd35 a452335ef191 2026-02-25T11:47:37.812 INFO:teuthology.orchestra.run.trial028.stdout:mon.c trial167 running (4m) 28s ago 4m 46.4M 2048M 20.3.0-5549-g92715cd3 8e51ef34fd35 d2a97993071e 2026-02-25T11:47:37.812 INFO:teuthology.orchestra.run.trial028.stdout:osd.0 trial028 running (3m) 28s ago 3m 45.1M 20.5G 20.3.0-5549-g92715cd3 8e51ef34fd35 778bff754f86 2026-02-25T11:47:37.812 INFO:teuthology.orchestra.run.trial028.stdout:osd.1 trial119 running (3m) 28s ago 3m 46.8M 20.5G 20.3.0-5549-g92715cd3 8e51ef34fd35 6ba86dbfd87e 2026-02-25T11:47:37.812 INFO:teuthology.orchestra.run.trial028.stdout:osd.2 trial167 running (2m) 28s ago 3m 45.8M 20.5G 20.3.0-5549-g92715cd3 8e51ef34fd35 6131fde28a67 2026-02-25T11:47:37.812 INFO:teuthology.orchestra.run.trial028.stdout:osd.3 trial028 running (2m) 28s ago 2m 43.4M 20.5G 20.3.0-5549-g92715cd3 8e51ef34fd35 b3fc165ad5f7 2026-02-25T11:47:37.812 INFO:teuthology.orchestra.run.trial028.stdout:osd.4 trial119 running (2m) 28s ago 2m 41.5M 20.5G 20.3.0-5549-g92715cd3 8e51ef34fd35 0e3df6ef3446 2026-02-25T11:47:37.812 INFO:teuthology.orchestra.run.trial028.stdout:osd.5 trial167 running (2m) 28s ago 2m 46.1M 20.5G 20.3.0-5549-g92715cd3 8e51ef34fd35 af9e2da3ad4e 2026-02-25T11:47:37.812 INFO:teuthology.orchestra.run.trial028.stdout:osd.6 trial028 running (2m) 28s ago 2m 42.4M 20.5G 20.3.0-5549-g92715cd3 8e51ef34fd35 96d2c6086b3b 2026-02-25T11:47:37.812 INFO:teuthology.orchestra.run.trial028.stdout:osd.7 trial119 running (110s) 28s ago 114s 45.5M 20.5G 20.3.0-5549-g92715cd3 8e51ef34fd35 d648b0a5eb59 2026-02-25T11:47:37.812 INFO:teuthology.orchestra.run.trial028.stdout:osd.8 trial167 running (97s) 28s ago 100s 43.6M 20.5G 20.3.0-5549-g92715cd3 8e51ef34fd35 c41cc6cd3d15 2026-02-25T11:47:37.813 INFO:teuthology.orchestra.run.trial028.stdout:osd.9 trial028 running (83s) 28s ago 86s 41.7M 20.5G 20.3.0-5549-g92715cd3 8e51ef34fd35 b1e689039db2 2026-02-25T11:47:37.813 INFO:teuthology.orchestra.run.trial028.stdout:osd.10 trial119 running (70s) 28s ago 73s 45.1M 20.5G 20.3.0-5549-g92715cd3 8e51ef34fd35 4b6ef9eae0f3 2026-02-25T11:47:37.813 INFO:teuthology.orchestra.run.trial028.stdout:osd.11 trial167 running (56s) 28s ago 59s 41.4M 20.5G 20.3.0-5549-g92715cd3 8e51ef34fd35 1549eba9d4cb 2026-02-25T11:47:37.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.808+0000 7f69e798a640 1 -- 10.20.193.28:0/3351754322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69bc0b57f0 msgr2=0x7f69bc0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:37.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.808+0000 7f69e798a640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69bc0b57f0 0x7f69bc0d5ca0 secure :-1 s=READY pgs=229 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69d00095b0 tx=0x7f69d0005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:37.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.808+0000 7f69e798a640 1 -- 10.20.193.28:0/3351754322 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e014f140 msgr2=0x7f69e0161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:37.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.808+0000 7f69e798a640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e014f140 0x7f69e0161040 secure :-1 s=READY pgs=311 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69dc00aa30 tx=0x7f69dc0039b0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.809+0000 7f69e798a640 1 -- 10.20.193.28:0/3351754322 shutdown_connections 2026-02-25T11:47:37.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.809+0000 7f69e798a640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69bc0b57f0 0x7f69bc0d5ca0 unknown :-1 s=CLOSED pgs=229 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.809+0000 7f69e798a640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e014f140 0x7f69e0161040 unknown :-1 s=CLOSED pgs=311 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.809+0000 7f69e798a640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69e010b880 0x7f69e0160440 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.809+0000 7f69e798a640 1 --2- 10.20.193.28:0/3351754322 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e010af50 0x7f69e015fdf0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:37.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.809+0000 7f69e798a640 1 -- 10.20.193.28:0/3351754322 >> 10.20.193.28:0/3351754322 conn(0x7f69e00e8b10 msgr2=0x7f69e0108ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:37.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.809+0000 7f69e798a640 1 -- 10.20.193.28:0/3351754322 shutdown_connections 2026-02-25T11:47:37.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:37.809+0000 7f69e798a640 1 -- 10.20.193.28:0/3351754322 wait complete. 2026-02-25T11:47:37.911 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- bash -c 'ceph orch ls' 2026-02-25T11:47:38.098 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:38.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.280+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3982577096 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe1f810b5d0 msgr2=0x7fe1f814e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:38.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.280+0000 7fe1fc99d640 1 --2- 10.20.193.28:0/3982577096 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe1f810b5d0 0x7fe1f814e950 secure :-1 s=READY pgs=312 gs=1 cs=0 l=1 c_cookie=5200967910275892918 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe1e8009950 tx=0x7fe1e802f1d0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.280+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3982577096 shutdown_connections 2026-02-25T11:47:38.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.280+0000 7fe1fc99d640 1 --2- 10.20.193.28:0/3982577096 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe1f814ee90 0x7fe1f816f270 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.280+0000 7fe1fc99d640 1 --2- 10.20.193.28:0/3982577096 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe1f810b5d0 0x7fe1f814e950 unknown :-1 s=CLOSED pgs=312 gs=1 cs=0 l=1 c_cookie=5200967910275892918 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.280+0000 7fe1fc99d640 1 --2- 10.20.193.28:0/3982577096 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe1f810aca0 0x7fe1f810b090 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.280+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3982577096 >> 10.20.193.28:0/3982577096 conn(0x7fe1f80e8860 msgr2=0x7fe1f8108c90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:38.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.280+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3982577096 shutdown_connections 2026-02-25T11:47:38.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.280+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3982577096 wait complete. 2026-02-25T11:47:38.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.281+0000 7fe1fc99d640 1 Processor -- start 2026-02-25T11:47:38.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.281+0000 7fe1fc99d640 1 -- start start 2026-02-25T11:47:38.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.281+0000 7fe1fc99d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe1f810aca0 0x7fe1f815f290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:38.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.281+0000 7fe1fc99d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe1f810b5d0 0x7fe1f815f8e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:38.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.281+0000 7fe1fc99d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe1f814ee90 0x7fe1f81604e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:38.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.281+0000 7fe1fc99d640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8160c20 con 0x7fe1f814ee90 2026-02-25T11:47:38.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.281+0000 7fe1fc99d640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8160d90 con 0x7fe1f810aca0 2026-02-25T11:47:38.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.281+0000 7fe1fc99d640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8160f50 con 0x7fe1f810b5d0 2026-02-25T11:47:38.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.281+0000 7fe1f7fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe1f814ee90 0x7fe1f81604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:38.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.281+0000 7fe1f77fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe1f810aca0 0x7fe1f815f290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:38.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.281+0000 7fe1f6ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe1f810b5d0 0x7fe1f815f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:38.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1f77fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe1f810aca0 0x7fe1f815f290 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43322/0 (socket says 10.20.193.28:43322) 2026-02-25T11:47:38.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1f77fe640 1 -- 10.20.193.28:0/3173940537 learned_addr learned my addr 10.20.193.28:0/3173940537 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:38.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1f77fe640 1 -- 10.20.193.28:0/3173940537 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe1f810b5d0 msgr2=0x7fe1f815f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:38.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1f77fe640 1 --2- 10.20.193.28:0/3173940537 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe1f810b5d0 0x7fe1f815f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1f77fe640 1 -- 10.20.193.28:0/3173940537 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe1f814ee90 msgr2=0x7fe1f81604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:38.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1f77fe640 1 --2- 10.20.193.28:0/3173940537 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe1f814ee90 0x7fe1f81604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1f77fe640 1 -- 10.20.193.28:0/3173940537 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e8009570 con 0x7fe1f810aca0 2026-02-25T11:47:38.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1f7fff640 1 --2- 10.20.193.28:0/3173940537 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe1f814ee90 0x7fe1f81604e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:38.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1f77fe640 1 --2- 10.20.193.28:0/3173940537 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe1f810aca0 0x7fe1f815f290 secure :-1 s=READY pgs=97 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe1e4009480 tx=0x7fe1e400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:38.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1f4ff9640 1 -- 10.20.193.28:0/3173940537 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe1e400cd40 con 0x7fe1f810aca0 2026-02-25T11:47:38.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1f4ff9640 1 -- 10.20.193.28:0/3173940537 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe1e4004d10 con 0x7fe1f810aca0 2026-02-25T11:47:38.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3173940537 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f8161230 con 0x7fe1f810aca0 2026-02-25T11:47:38.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1f4ff9640 1 -- 10.20.193.28:0/3173940537 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe1e40106b0 con 0x7fe1f810aca0 2026-02-25T11:47:38.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.282+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3173940537 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f814c910 con 0x7fe1f810aca0 2026-02-25T11:47:38.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.283+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3173940537 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f8173350 con 0x7fe1f810aca0 2026-02-25T11:47:38.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.285+0000 7fe1f4ff9640 1 -- 10.20.193.28:0/3173940537 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fe1e401f060 con 0x7fe1f810aca0 2026-02-25T11:47:38.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.287+0000 7fe1f4ff9640 1 --2- 10.20.193.28:0/3173940537 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe1cc0b57f0 0x7fe1cc0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:38.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.287+0000 7fe1f4ff9640 1 -- 10.20.193.28:0/3173940537 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fe1e4014070 con 0x7fe1f810aca0 2026-02-25T11:47:38.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.287+0000 7fe1f4ff9640 1 -- 10.20.193.28:0/3173940537 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fe1e411f940 con 0x7fe1f810aca0 2026-02-25T11:47:38.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.287+0000 7fe1f6ffd640 1 --2- 10.20.193.28:0/3173940537 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe1cc0b57f0 0x7fe1cc0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:38.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.288+0000 7fe1f6ffd640 1 --2- 10.20.193.28:0/3173940537 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe1cc0b57f0 0x7fe1cc0d5ca0 secure :-1 s=READY pgs=230 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe1e80098a0 tx=0x7fe1e803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:38.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.406+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3173940537 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fe1f8172b00 con 0x7fe1cc0b57f0 2026-02-25T11:47:38.409 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:38 trial028 ceph-mon[39224]: pgmap v206: 97 pgs: 97 active+clean; 646 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2026-02-25T11:47:38.409 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:38 trial028 ceph-mon[39224]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:38.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.409+0000 7fe1f4ff9640 1 -- 10.20.193.28:0/3173940537 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+940 (secure 0 0 0) 0x7fe1f8172b00 con 0x7fe1cc0b57f0 2026-02-25T11:47:38.412 INFO:teuthology.orchestra.run.trial028.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2026-02-25T11:47:38.412 INFO:teuthology.orchestra.run.trial028.stdout:mds.all 12/12 28s ago 50s trial028=a;trial028=d;trial028=g;trial028=j;trial119=b;trial119=e;trial119=h;trial119=k;trial167=c;trial167=f;trial167=i;trial167=l;count:12 2026-02-25T11:47:38.412 INFO:teuthology.orchestra.run.trial028.stdout:mgr 3/3 28s ago 3m trial028=x;trial119=y;trial167=z;count:3 2026-02-25T11:47:38.412 INFO:teuthology.orchestra.run.trial028.stdout:mon 3/3 28s ago 4m trial028:10.20.193.28=a;trial119:10.20.193.119=b;trial167:10.20.193.167=c;count:3 2026-02-25T11:47:38.412 INFO:teuthology.orchestra.run.trial028.stdout:osd.default 12 28s ago 67s trial167 2026-02-25T11:47:38.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.410+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3173940537 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe1cc0b57f0 msgr2=0x7fe1cc0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:38.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.410+0000 7fe1fc99d640 1 --2- 10.20.193.28:0/3173940537 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe1cc0b57f0 0x7fe1cc0d5ca0 secure :-1 s=READY pgs=230 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe1e80098a0 tx=0x7fe1e803a040 comp rx=0 tx=0).stop 2026-02-25T11:47:38.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.410+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3173940537 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe1f810aca0 msgr2=0x7fe1f815f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:38.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.410+0000 7fe1fc99d640 1 --2- 10.20.193.28:0/3173940537 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe1f810aca0 0x7fe1f815f290 secure :-1 s=READY pgs=97 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe1e4009480 tx=0x7fe1e400ee60 comp rx=0 tx=0).stop 2026-02-25T11:47:38.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.411+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3173940537 shutdown_connections 2026-02-25T11:47:38.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.411+0000 7fe1fc99d640 1 --2- 10.20.193.28:0/3173940537 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe1cc0b57f0 0x7fe1cc0d5ca0 unknown :-1 s=CLOSED pgs=230 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.411+0000 7fe1fc99d640 1 --2- 10.20.193.28:0/3173940537 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe1f814ee90 0x7fe1f81604e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.411+0000 7fe1fc99d640 1 --2- 10.20.193.28:0/3173940537 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe1f810b5d0 0x7fe1f815f8e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.411+0000 7fe1fc99d640 1 --2- 10.20.193.28:0/3173940537 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe1f810aca0 0x7fe1f815f290 unknown :-1 s=CLOSED pgs=97 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.411+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3173940537 >> 10.20.193.28:0/3173940537 conn(0x7fe1f80e8860 msgr2=0x7fe1f8108c40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:38.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.411+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3173940537 shutdown_connections 2026-02-25T11:47:38.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.411+0000 7fe1fc99d640 1 -- 10.20.193.28:0/3173940537 wait complete. 2026-02-25T11:47:38.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:38 trial167 ceph-mon[43392]: pgmap v206: 97 pgs: 97 active+clean; 646 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2026-02-25T11:47:38.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:38 trial167 ceph-mon[43392]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:38.510 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- bash -c 'ceph orch host ls' 2026-02-25T11:47:38.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:38 trial119 ceph-mon[42710]: pgmap v206: 97 pgs: 97 active+clean; 646 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.9 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2026-02-25T11:47:38.511 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:38 trial119 ceph-mon[42710]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:38.695 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:38.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.880+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/628308318 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdf9c057fd0 msgr2=0x7fdf9c058440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:38.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.880+0000 7fdfa0cbd640 1 --2- 10.20.193.28:0/628308318 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdf9c057fd0 0x7fdf9c058440 secure :-1 s=READY pgs=313 gs=1 cs=0 l=1 c_cookie=6837283918305388270 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdf84009950 tx=0x7fdf8402f190 comp rx=0 tx=0).stop 2026-02-25T11:47:38.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.880+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/628308318 shutdown_connections 2026-02-25T11:47:38.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.880+0000 7fdfa0cbd640 1 --2- 10.20.193.28:0/628308318 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdf9c115720 0x7fdf9c115bc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.880+0000 7fdfa0cbd640 1 --2- 10.20.193.28:0/628308318 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdf9c057fd0 0x7fdf9c058440 unknown :-1 s=CLOSED pgs=313 gs=1 cs=0 l=1 c_cookie=6837283918305388270 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.880+0000 7fdfa0cbd640 1 --2- 10.20.193.28:0/628308318 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdf9c059400 0x7fdf9c057a90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.880+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/628308318 >> 10.20.193.28:0/628308318 conn(0x7fdf9c05d960 msgr2=0x7fdf9c05dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:38.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.880+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/628308318 shutdown_connections 2026-02-25T11:47:38.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.880+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/628308318 wait complete. 2026-02-25T11:47:38.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.881+0000 7fdfa0cbd640 1 Processor -- start 2026-02-25T11:47:38.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.881+0000 7fdfa0cbd640 1 -- start start 2026-02-25T11:47:38.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.881+0000 7fdfa0cbd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdf9c057fd0 0x7fdf9c101050 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:38.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.881+0000 7fdfa0cbd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdf9c059400 0x7fdf9c1016a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:38.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.881+0000 7fdfa0cbd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdf9c115720 0x7fdf9c1022a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:38.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.881+0000 7fdfa0cbd640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c1029e0 con 0x7fdf9c057fd0 2026-02-25T11:47:38.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.881+0000 7fdfa0cbd640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c102b50 con 0x7fdf9c115720 2026-02-25T11:47:38.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.881+0000 7fdfa0cbd640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9c102d10 con 0x7fdf9c059400 2026-02-25T11:47:38.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.881+0000 7fdf9a575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdf9c057fd0 0x7fdf9c101050 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:38.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.881+0000 7fdf9a575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdf9c057fd0 0x7fdf9c101050 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35572/0 (socket says 10.20.193.28:35572) 2026-02-25T11:47:38.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.881+0000 7fdf9a575640 1 -- 10.20.193.28:0/3182001447 learned_addr learned my addr 10.20.193.28:0/3182001447 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:38.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.881+0000 7fdf9ad76640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdf9c115720 0x7fdf9c1022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:38.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.881+0000 7fdf99d74640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdf9c059400 0x7fdf9c1016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:38.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.882+0000 7fdf9a575640 1 -- 10.20.193.28:0/3182001447 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdf9c059400 msgr2=0x7fdf9c1016a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:38.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.882+0000 7fdf9a575640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdf9c059400 0x7fdf9c1016a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.882+0000 7fdf9a575640 1 -- 10.20.193.28:0/3182001447 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdf9c115720 msgr2=0x7fdf9c1022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:38.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.882+0000 7fdf9a575640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdf9c115720 0x7fdf9c1022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:38.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.882+0000 7fdf9a575640 1 -- 10.20.193.28:0/3182001447 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf84009570 con 0x7fdf9c057fd0 2026-02-25T11:47:38.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.882+0000 7fdf9a575640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdf9c057fd0 0x7fdf9c101050 secure :-1 s=READY pgs=314 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdf90009480 tx=0x7fdf9000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:38.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.882+0000 7fdf837fe640 1 -- 10.20.193.28:0/3182001447 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdf9000cd40 con 0x7fdf9c057fd0 2026-02-25T11:47:38.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.882+0000 7fdf837fe640 1 -- 10.20.193.28:0/3182001447 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdf90004d10 con 0x7fdf9c057fd0 2026-02-25T11:47:38.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.882+0000 7fdf837fe640 1 -- 10.20.193.28:0/3182001447 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdf900106b0 con 0x7fdf9c057fd0 2026-02-25T11:47:38.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.882+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/3182001447 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9c102ff0 con 0x7fdf9c057fd0 2026-02-25T11:47:38.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.882+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/3182001447 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9c103570 con 0x7fdf9c057fd0 2026-02-25T11:47:38.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.883+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/3182001447 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9c055e30 con 0x7fdf9c057fd0 2026-02-25T11:47:38.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.885+0000 7fdf837fe640 1 -- 10.20.193.28:0/3182001447 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fdf90004770 con 0x7fdf9c057fd0 2026-02-25T11:47:38.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.887+0000 7fdf837fe640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdf740b57f0 0x7fdf740d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:38.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.887+0000 7fdf837fe640 1 -- 10.20.193.28:0/3182001447 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fdf90014070 con 0x7fdf9c057fd0 2026-02-25T11:47:38.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.887+0000 7fdf837fe640 1 -- 10.20.193.28:0/3182001447 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fdf900e73f0 con 0x7fdf9c057fd0 2026-02-25T11:47:38.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.887+0000 7fdf99d74640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdf740b57f0 0x7fdf740d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:38.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:38.887+0000 7fdf99d74640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdf740b57f0 0x7fdf740d5ca0 secure :-1 s=READY pgs=231 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdf840098a0 tx=0x7fdf8403a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:39.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.004+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/3182001447 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fdf9c1193e0 con 0x7fdf740b57f0 2026-02-25T11:47:39.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.006+0000 7fdf837fe640 1 -- 10.20.193.28:0/3182001447 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7fdf9c1193e0 con 0x7fdf740b57f0 2026-02-25T11:47:39.008 INFO:teuthology.orchestra.run.trial028.stdout:HOST ADDR LABELS STATUS 2026-02-25T11:47:39.009 INFO:teuthology.orchestra.run.trial028.stdout:trial028 10.20.193.28 2026-02-25T11:47:39.009 INFO:teuthology.orchestra.run.trial028.stdout:trial119 10.20.193.119 2026-02-25T11:47:39.009 INFO:teuthology.orchestra.run.trial028.stdout:trial167 10.20.193.167 2026-02-25T11:47:39.009 INFO:teuthology.orchestra.run.trial028.stdout:3 hosts in cluster 2026-02-25T11:47:39.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.007+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/3182001447 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdf740b57f0 msgr2=0x7fdf740d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:39.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.007+0000 7fdfa0cbd640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdf740b57f0 0x7fdf740d5ca0 secure :-1 s=READY pgs=231 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdf840098a0 tx=0x7fdf8403a040 comp rx=0 tx=0).stop 2026-02-25T11:47:39.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.007+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/3182001447 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdf9c057fd0 msgr2=0x7fdf9c101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:39.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.007+0000 7fdfa0cbd640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdf9c057fd0 0x7fdf9c101050 secure :-1 s=READY pgs=314 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdf90009480 tx=0x7fdf9000ee60 comp rx=0 tx=0).stop 2026-02-25T11:47:39.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.007+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/3182001447 shutdown_connections 2026-02-25T11:47:39.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.007+0000 7fdfa0cbd640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdf740b57f0 0x7fdf740d5ca0 unknown :-1 s=CLOSED pgs=231 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.007+0000 7fdfa0cbd640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdf9c115720 0x7fdf9c1022a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.007+0000 7fdfa0cbd640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdf9c059400 0x7fdf9c1016a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.007+0000 7fdfa0cbd640 1 --2- 10.20.193.28:0/3182001447 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdf9c057fd0 0x7fdf9c101050 unknown :-1 s=CLOSED pgs=314 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.007+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/3182001447 >> 10.20.193.28:0/3182001447 conn(0x7fdf9c05d960 msgr2=0x7fdf9c0f7670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:39.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.007+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/3182001447 shutdown_connections 2026-02-25T11:47:39.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.007+0000 7fdfa0cbd640 1 -- 10.20.193.28:0/3182001447 wait complete. 2026-02-25T11:47:39.114 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- bash -c 'ceph orch device ls' 2026-02-25T11:47:39.300 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:39.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:39 trial028 ceph-mon[39224]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:39.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.482+0000 7fb472b09640 1 -- 10.20.193.28:0/653635123 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb46c10ed50 msgr2=0x7fb46c16f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:39.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.482+0000 7fb472b09640 1 --2- 10.20.193.28:0/653635123 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb46c10ed50 0x7fb46c16f6e0 secure :-1 s=READY pgs=315 gs=2 cs=0 l=1 c_cookie=16673527617643759746 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb46800a970 tx=0x7fb4680301b0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.482+0000 7fb472b09640 1 -- 10.20.193.28:0/653635123 shutdown_connections 2026-02-25T11:47:39.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.482+0000 7fb472b09640 1 --2- 10.20.193.28:0/653635123 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb46c10ed50 0x7fb46c16f6e0 unknown :-1 s=CLOSED pgs=315 gs=2 cs=0 l=1 c_cookie=16673527617643759746 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.482+0000 7fb472b09640 1 --2- 10.20.193.28:0/653635123 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb46c10e3a0 0x7fb46c10e810 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.482+0000 7fb472b09640 1 --2- 10.20.193.28:0/653635123 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb46c10d210 0x7fb46c10d600 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.482+0000 7fb472b09640 1 -- 10.20.193.28:0/653635123 >> 10.20.193.28:0/653635123 conn(0x7fb46c0ea940 msgr2=0x7fb46c10ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:39.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.482+0000 7fb472b09640 1 -- 10.20.193.28:0/653635123 shutdown_connections 2026-02-25T11:47:39.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.482+0000 7fb472b09640 1 -- 10.20.193.28:0/653635123 wait complete. 2026-02-25T11:47:39.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.482+0000 7fb472b09640 1 Processor -- start 2026-02-25T11:47:39.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb472b09640 1 -- start start 2026-02-25T11:47:39.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb472b09640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb46c10d210 0x7fb46c0f72f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:39.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb472b09640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb46c10e3a0 0x7fb46c16f6e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:39.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb472b09640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb46c10ed50 0x7fb46c0f7b50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:39.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb472b09640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c0f8740 con 0x7fb46c10d210 2026-02-25T11:47:39.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb472b09640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c10a9f0 con 0x7fb46c10e3a0 2026-02-25T11:47:39.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb472b09640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c10abb0 con 0x7fb46c10ed50 2026-02-25T11:47:39.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb47087e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb46c10d210 0x7fb46c0f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:39.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb47087e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb46c10d210 0x7fb46c0f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35594/0 (socket says 10.20.193.28:35594) 2026-02-25T11:47:39.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb47087e640 1 -- 10.20.193.28:0/287613915 learned_addr learned my addr 10.20.193.28:0/287613915 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:39.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb47107f640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb46c10ed50 0x7fb46c0f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:39.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb463fff640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb46c10e3a0 0x7fb46c16f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:39.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb47087e640 1 -- 10.20.193.28:0/287613915 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb46c10ed50 msgr2=0x7fb46c0f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:39.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb47087e640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb46c10ed50 0x7fb46c0f7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb47087e640 1 -- 10.20.193.28:0/287613915 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb46c10e3a0 msgr2=0x7fb46c16f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:39.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb47087e640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb46c10e3a0 0x7fb46c16f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.483+0000 7fb47087e640 1 -- 10.20.193.28:0/287613915 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46800a590 con 0x7fb46c10d210 2026-02-25T11:47:39.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.484+0000 7fb47087e640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb46c10d210 0x7fb46c0f72f0 secure :-1 s=READY pgs=316 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb45c00e9c0 tx=0x7fb45c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:39.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.484+0000 7fb461ffb640 1 -- 10.20.193.28:0/287613915 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb45c00ccf0 con 0x7fb46c10d210 2026-02-25T11:47:39.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.484+0000 7fb461ffb640 1 -- 10.20.193.28:0/287613915 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb45c00ce50 con 0x7fb46c10d210 2026-02-25T11:47:39.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.484+0000 7fb472b09640 1 -- 10.20.193.28:0/287613915 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c10ae90 con 0x7fb46c10d210 2026-02-25T11:47:39.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.484+0000 7fb461ffb640 1 -- 10.20.193.28:0/287613915 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb45c0106b0 con 0x7fb46c10d210 2026-02-25T11:47:39.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.484+0000 7fb472b09640 1 -- 10.20.193.28:0/287613915 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c0f8880 con 0x7fb46c10d210 2026-02-25T11:47:39.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.485+0000 7fb43b7fe640 1 -- 10.20.193.28:0/287613915 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb46c173600 con 0x7fb46c10d210 2026-02-25T11:47:39.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.487+0000 7fb461ffb640 1 -- 10.20.193.28:0/287613915 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fb45c01f060 con 0x7fb46c10d210 2026-02-25T11:47:39.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.488+0000 7fb461ffb640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb4480b5940 0x7fb4480d5df0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:39.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.488+0000 7fb461ffb640 1 -- 10.20.193.28:0/287613915 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fb45c014070 con 0x7fb46c10d210 2026-02-25T11:47:39.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.489+0000 7fb461ffb640 1 -- 10.20.193.28:0/287613915 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fb45c11f940 con 0x7fb46c10d210 2026-02-25T11:47:39.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.489+0000 7fb463fff640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb4480b5940 0x7fb4480d5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:39.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.489+0000 7fb463fff640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb4480b5940 0x7fb4480d5df0 secure :-1 s=READY pgs=232 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb454004740 tx=0x7fb454005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:39.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:39 trial167 ceph-mon[43392]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:39.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:39 trial119 ceph-mon[42710]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:39.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.607+0000 7fb43b7fe640 1 -- 10.20.193.28:0/287613915 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb46c05f560 con 0x7fb4480b5940 2026-02-25T11:47:39.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.609+0000 7fb461ffb640 1 -- 10.20.193.28:0/287613915 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+910 (secure 0 0 0) 0x7fb46c05f560 con 0x7fb4480b5940 2026-02-25T11:47:39.612 INFO:teuthology.orchestra.run.trial028.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2026-02-25T11:47:39.612 INFO:teuthology.orchestra.run.trial028.stdout:trial028 /dev/nvme1n1 ssd MTFDKCC1T9TGP-1BK1JABYY_112510AB50DD 1788G No 67s ago Has a FileSystem, LVM detected 2026-02-25T11:47:39.612 INFO:teuthology.orchestra.run.trial028.stdout:trial028 /dev/nvme2n1 ssd MTFDKCC1T9TGP-1BK1JABYY_112510AB50F4 1788G No 67s ago Has a FileSystem, LVM detected 2026-02-25T11:47:39.612 INFO:teuthology.orchestra.run.trial028.stdout:trial119 /dev/nvme1n1 ssd MTFDKCC1T9TGP-1BK1JABYY_092510AB6E45 1788G No 53s ago Has a FileSystem, LVM detected 2026-02-25T11:47:39.612 INFO:teuthology.orchestra.run.trial028.stdout:trial119 /dev/nvme2n1 ssd MTFDKCC1T9TGP-1BK1JABYY_092510AB6DCB 1788G No 53s ago Has a FileSystem, LVM detected 2026-02-25T11:47:39.612 INFO:teuthology.orchestra.run.trial028.stdout:trial167 /dev/nvme0n1 ssd MTFDKCC1T9TGP-1BK1JABYY_012510B003E8 1788G No 47s ago Has a FileSystem, LVM detected 2026-02-25T11:47:39.612 INFO:teuthology.orchestra.run.trial028.stdout:trial167 /dev/nvme1n1 ssd MTFDKCC1T9TGP-1BK1JABYY_012510B00408 1788G No 47s ago Has a FileSystem, LVM detected 2026-02-25T11:47:39.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.610+0000 7fb43b7fe640 1 -- 10.20.193.28:0/287613915 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb4480b5940 msgr2=0x7fb4480d5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:39.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.610+0000 7fb43b7fe640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb4480b5940 0x7fb4480d5df0 secure :-1 s=READY pgs=232 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb454004740 tx=0x7fb454005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:39.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.610+0000 7fb43b7fe640 1 -- 10.20.193.28:0/287613915 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb46c10d210 msgr2=0x7fb46c0f72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:39.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.610+0000 7fb43b7fe640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb46c10d210 0x7fb46c0f72f0 secure :-1 s=READY pgs=316 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb45c00e9c0 tx=0x7fb45c00ee90 comp rx=0 tx=0).stop 2026-02-25T11:47:39.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.610+0000 7fb43b7fe640 1 -- 10.20.193.28:0/287613915 shutdown_connections 2026-02-25T11:47:39.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.610+0000 7fb43b7fe640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb4480b5940 0x7fb4480d5df0 unknown :-1 s=CLOSED pgs=232 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.610+0000 7fb43b7fe640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb46c10ed50 0x7fb46c0f7b50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.610+0000 7fb43b7fe640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb46c10e3a0 0x7fb46c16f6e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.610+0000 7fb43b7fe640 1 --2- 10.20.193.28:0/287613915 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb46c10d210 0x7fb46c0f72f0 unknown :-1 s=CLOSED pgs=316 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:39.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.610+0000 7fb43b7fe640 1 -- 10.20.193.28:0/287613915 >> 10.20.193.28:0/287613915 conn(0x7fb46c0ea940 msgr2=0x7fb46c0ed910 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:39.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.610+0000 7fb43b7fe640 1 -- 10.20.193.28:0/287613915 shutdown_connections 2026-02-25T11:47:39.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:39.610+0000 7fb43b7fe640 1 -- 10.20.193.28:0/287613915 wait complete. 2026-02-25T11:47:39.710 INFO:teuthology.run_tasks:Running task cephadm.shell... 2026-02-25T11:47:39.730 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@trial028.front.sepia.ceph.com 2026-02-25T11:47:39.730 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- bash -c 'ceph fs dump' 2026-02-25T11:47:39.913 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:40.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.091+0000 7f87865c0640 1 -- 10.20.193.28:0/1929066193 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8780151060 msgr2=0x7f8780171440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:40.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.091+0000 7f87865c0640 1 --2- 10.20.193.28:0/1929066193 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8780151060 0x7f8780171440 secure :-1 s=READY pgs=98 gs=1 cs=0 l=1 c_cookie=15424225403098294403 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f877c00aa30 tx=0x7f877c030270 comp rx=0 tx=0).stop 2026-02-25T11:47:40.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.091+0000 7f87865c0640 1 -- 10.20.193.28:0/1929066193 shutdown_connections 2026-02-25T11:47:40.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.091+0000 7f87865c0640 1 --2- 10.20.193.28:0/1929066193 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8780151060 0x7f8780171440 unknown :-1 s=CLOSED pgs=98 gs=1 cs=0 l=1 c_cookie=15424225403098294403 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.091+0000 7f87865c0640 1 --2- 10.20.193.28:0/1929066193 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8780061610 0x7f8780061a80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.091+0000 7f87865c0640 1 --2- 10.20.193.28:0/1929066193 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8780062a40 0x7f87800610d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.091+0000 7f87865c0640 1 -- 10.20.193.28:0/1929066193 >> 10.20.193.28:0/1929066193 conn(0x7f87800ea6d0 msgr2=0x7f878010aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:40.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.091+0000 7f87865c0640 1 -- 10.20.193.28:0/1929066193 shutdown_connections 2026-02-25T11:47:40.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.091+0000 7f87865c0640 1 -- 10.20.193.28:0/1929066193 wait complete. 2026-02-25T11:47:40.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.092+0000 7f87865c0640 1 Processor -- start 2026-02-25T11:47:40.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.092+0000 7f87865c0640 1 -- start start 2026-02-25T11:47:40.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.092+0000 7f87865c0640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8780061610 0x7f878015c950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:40.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.092+0000 7f87865c0640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8780062a40 0x7f878015cfa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:40.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.092+0000 7f87865c0640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8780151060 0x7f878015dba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:40.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.092+0000 7f87865c0640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f878015e2e0 con 0x7f8780061610 2026-02-25T11:47:40.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.092+0000 7f87865c0640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f878015e450 con 0x7f8780062a40 2026-02-25T11:47:40.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.092+0000 7f87865c0640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f878015e610 con 0x7f8780151060 2026-02-25T11:47:40.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.092+0000 7f87855be640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8780061610 0x7f878015c950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:40.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.092+0000 7f87855be640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8780061610 0x7f878015c950 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35614/0 (socket says 10.20.193.28:35614) 2026-02-25T11:47:40.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.092+0000 7f87855be640 1 -- 10.20.193.28:0/1639988637 learned_addr learned my addr 10.20.193.28:0/1639988637 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:40.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.093+0000 7f8785dbf640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8780151060 0x7f878015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:40.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.093+0000 7f8784dbd640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8780062a40 0x7f878015cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:40.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.093+0000 7f87855be640 1 -- 10.20.193.28:0/1639988637 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8780151060 msgr2=0x7f878015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:40.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.093+0000 7f87855be640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8780151060 0x7f878015dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.093+0000 7f87855be640 1 -- 10.20.193.28:0/1639988637 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8780062a40 msgr2=0x7f878015cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:40.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.093+0000 7f87855be640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8780062a40 0x7f878015cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.093+0000 7f87855be640 1 -- 10.20.193.28:0/1639988637 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c00a650 con 0x7f8780061610 2026-02-25T11:47:40.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.093+0000 7f87855be640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8780061610 0x7f878015c950 secure :-1 s=READY pgs=317 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8770009480 tx=0x7f877000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:40.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.093+0000 7f876e7fc640 1 -- 10.20.193.28:0/1639988637 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f877000ccc0 con 0x7f8780061610 2026-02-25T11:47:40.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.093+0000 7f87865c0640 1 -- 10.20.193.28:0/1639988637 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878015e8f0 con 0x7f8780061610 2026-02-25T11:47:40.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.093+0000 7f876e7fc640 1 -- 10.20.193.28:0/1639988637 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f877000ce20 con 0x7f8780061610 2026-02-25T11:47:40.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.093+0000 7f876e7fc640 1 -- 10.20.193.28:0/1639988637 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f87700106b0 con 0x7f8780061610 2026-02-25T11:47:40.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.093+0000 7f87865c0640 1 -- 10.20.193.28:0/1639988637 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878015ee70 con 0x7f8780061610 2026-02-25T11:47:40.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.094+0000 7f87865c0640 1 -- 10.20.193.28:0/1639988637 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87801755f0 con 0x7f8780061610 2026-02-25T11:47:40.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.096+0000 7f876e7fc640 1 -- 10.20.193.28:0/1639988637 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f877001f060 con 0x7f8780061610 2026-02-25T11:47:40.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.098+0000 7f876e7fc640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f875c0b57f0 0x7f875c0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:40.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.098+0000 7f876e7fc640 1 -- 10.20.193.28:0/1639988637 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f8770014070 con 0x7f8780061610 2026-02-25T11:47:40.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.098+0000 7f8784dbd640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f875c0b57f0 0x7f875c0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:40.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.098+0000 7f876e7fc640 1 -- 10.20.193.28:0/1639988637 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f877011f940 con 0x7f8780061610 2026-02-25T11:47:40.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.099+0000 7f8784dbd640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f875c0b57f0 0x7f875c0d5ca0 secure :-1 s=READY pgs=233 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8774004740 tx=0x7f8774005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:40.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.239+0000 7f87865c0640 1 -- 10.20.193.28:0/1639988637 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8780174cb0 con 0x7f8780061610 2026-02-25T11:47:40.241 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:40 trial028 ceph-mon[39224]: from='client.25130 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:40.241 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:40 trial028 ceph-mon[39224]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:40.241 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:40 trial028 ceph-mon[39224]: pgmap v207: 97 pgs: 97 active+clean; 646 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2026-02-25T11:47:40.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.239+0000 7f876e7fc640 1 -- 10.20.193.28:0/1639988637 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) ==== 76+0+3086 (secure 0 0 0) 0x7f87700a96f0 con 0x7f8780061610 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:e22 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:btime 2026-02-25T11:47:18:567831+0000 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:enable_multiple, ever_enabled_multiple: 1,1 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:legacy client fscid: 1 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:Filesystem 'cephfs' (1) 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:fs_name cephfs 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:epoch 22 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:created 2026-02-25T11:47:13.223613+0000 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:modified 2026-02-25T11:47:18.567827+0000 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:tableserver 0 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:root 0 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:session_timeout 300 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:session_autoclose 300 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:max_file_size 1099511627776 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:max_xattr_size 65536 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:required_client_features {} 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:last_failure 0 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:last_failure_osd_epoch 0 2026-02-25T11:47:40.242 INFO:teuthology.orchestra.run.trial028.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:max_mds 3 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:in 0,1,2 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:up {0=24749,1=24731,2=24730} 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:failed 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:damaged 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:stopped 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:data_pools [3] 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:metadata_pool 2 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:inline_data disabled 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:balancer 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:bal_rank_mask -1 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:standby_count_wanted 0 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:qdb_cluster leader: 24749 members: 24730,24731,24749 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:[mds.b{0:24749} state up:active seq 3 addr [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] compat {c=[1],r=[1],i=[1fff]}] 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:[mds.h{0:24719} state up:standby-replay seq 1 addr [v2:10.20.193.119:6836/392187157,v1:10.20.193.119:6837/392187157] compat {c=[1],r=[1],i=[1fff]}] 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:[mds.g{1:24731} state up:active seq 5 addr [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] compat {c=[1],r=[1],i=[1fff]}] 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:[mds.c{1:24707} state up:standby-replay seq 1 addr [v2:10.20.193.167:6836/3206735083,v1:10.20.193.167:6837/3206735083] compat {c=[1],r=[1],i=[1fff]}] 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:[mds.i{2:24730} state up:active seq 5 addr [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] compat {c=[1],r=[1],i=[1fff]}] 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:[mds.f{2:24706} state up:standby-replay seq 1 addr [v2:10.20.193.167:6834/3681115733,v1:10.20.193.167:6835/3681115733] compat {c=[1],r=[1],i=[1fff]}] 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:Standby daemons: 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:40.243 INFO:teuthology.orchestra.run.trial028.stdout:[mds.j{-1:14787} state up:standby seq 1 addr [v2:10.20.193.28:6834/685551537,v1:10.20.193.28:6835/685551537] compat {c=[1],r=[1],i=[1fff]}] 2026-02-25T11:47:40.244 INFO:teuthology.orchestra.run.trial028.stdout:[mds.d{-1:14799} state up:standby seq 1 addr [v2:10.20.193.28:6836/281937174,v1:10.20.193.28:6837/281937174] compat {c=[1],r=[1],i=[1fff]}] 2026-02-25T11:47:40.244 INFO:teuthology.orchestra.run.trial028.stdout:[mds.a{-1:14850} state up:standby seq 1 addr [v2:10.20.193.28:6840/1424760953,v1:10.20.193.28:6841/1424760953] compat {c=[1],r=[1],i=[1fff]}] 2026-02-25T11:47:40.244 INFO:teuthology.orchestra.run.trial028.stdout:[mds.k{-1:24671} state up:standby seq 1 addr [v2:10.20.193.119:6832/4164243082,v1:10.20.193.119:6833/4164243082] compat {c=[1],r=[1],i=[1fff]}] 2026-02-25T11:47:40.244 INFO:teuthology.orchestra.run.trial028.stdout:[mds.l{-1:24673} state up:standby seq 1 addr [v2:10.20.193.167:6832/478500593,v1:10.20.193.167:6833/478500593] compat {c=[1],r=[1],i=[1fff]}] 2026-02-25T11:47:40.244 INFO:teuthology.orchestra.run.trial028.stdout:[mds.e{-1:24694} state up:standby seq 1 addr [v2:10.20.193.119:6834/3262473702,v1:10.20.193.119:6835/3262473702] compat {c=[1],r=[1],i=[1fff]}] 2026-02-25T11:47:40.245 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 22 2026-02-25T11:47:40.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.240+0000 7f87865c0640 1 -- 10.20.193.28:0/1639988637 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f875c0b57f0 msgr2=0x7f875c0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:40.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.240+0000 7f87865c0640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f875c0b57f0 0x7f875c0d5ca0 secure :-1 s=READY pgs=233 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8774004740 tx=0x7f8774005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:40.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.240+0000 7f87865c0640 1 -- 10.20.193.28:0/1639988637 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8780061610 msgr2=0x7f878015c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:40.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.240+0000 7f87865c0640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8780061610 0x7f878015c950 secure :-1 s=READY pgs=317 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8770009480 tx=0x7f877000ee60 comp rx=0 tx=0).stop 2026-02-25T11:47:40.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.241+0000 7f87865c0640 1 -- 10.20.193.28:0/1639988637 shutdown_connections 2026-02-25T11:47:40.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.241+0000 7f87865c0640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f875c0b57f0 0x7f875c0d5ca0 unknown :-1 s=CLOSED pgs=233 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.241+0000 7f87865c0640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8780151060 0x7f878015dba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.241+0000 7f87865c0640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8780062a40 0x7f878015cfa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.241+0000 7f87865c0640 1 --2- 10.20.193.28:0/1639988637 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8780061610 0x7f878015c950 unknown :-1 s=CLOSED pgs=317 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.241+0000 7f87865c0640 1 -- 10.20.193.28:0/1639988637 >> 10.20.193.28:0/1639988637 conn(0x7f87800ea6d0 msgr2=0x7f878010aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:40.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.241+0000 7f87865c0640 1 -- 10.20.193.28:0/1639988637 shutdown_connections 2026-02-25T11:47:40.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.241+0000 7f87865c0640 1 -- 10.20.193.28:0/1639988637 wait complete. 2026-02-25T11:47:40.346 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- bash -c 'ceph osd dump' 2026-02-25T11:47:40.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:40 trial167 ceph-mon[43392]: from='client.25130 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:40.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:40 trial167 ceph-mon[43392]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:40.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:40 trial167 ceph-mon[43392]: pgmap v207: 97 pgs: 97 active+clean; 646 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2026-02-25T11:47:40.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:40 trial119 ceph-mon[42710]: from='client.25130 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:40.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:40 trial119 ceph-mon[42710]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:40.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:40 trial119 ceph-mon[42710]: pgmap v207: 97 pgs: 97 active+clean; 646 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2026-02-25T11:47:40.531 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:40.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.717+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/4108026363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a88110600 msgr2=0x7f3a88110a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:40.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.717+0000 7f3a8d3cc640 1 --2- 10.20.193.28:0/4108026363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a88110600 0x7f3a88110a70 secure :-1 s=READY pgs=318 gs=1 cs=0 l=1 c_cookie=3127534822935375830 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3a70009950 tx=0x7f3a7002f190 comp rx=0 tx=0).stop 2026-02-25T11:47:40.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.717+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/4108026363 shutdown_connections 2026-02-25T11:47:40.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.717+0000 7f3a8d3cc640 1 --2- 10.20.193.28:0/4108026363 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3a88110fb0 0x7f3a88171940 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.717+0000 7f3a8d3cc640 1 --2- 10.20.193.28:0/4108026363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a88110600 0x7f3a88110a70 unknown :-1 s=CLOSED pgs=318 gs=1 cs=0 l=1 c_cookie=3127534822935375830 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.717+0000 7f3a8d3cc640 1 --2- 10.20.193.28:0/4108026363 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3a8810f470 0x7f3a8810f860 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.717+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/4108026363 >> 10.20.193.28:0/4108026363 conn(0x7f3a880ecba0 msgr2=0x7f3a8810cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:40.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.717+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/4108026363 shutdown_connections 2026-02-25T11:47:40.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.717+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/4108026363 wait complete. 2026-02-25T11:47:40.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.718+0000 7f3a8d3cc640 1 Processor -- start 2026-02-25T11:47:40.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.718+0000 7f3a8d3cc640 1 -- start start 2026-02-25T11:47:40.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.718+0000 7f3a8d3cc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3a8810f470 0x7f3a880f95a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:40.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.718+0000 7f3a8d3cc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a88110600 0x7f3a88171940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:40.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.718+0000 7f3a8d3cc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3a88110fb0 0x7f3a880f9e00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:40.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.718+0000 7f3a8d3cc640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3a880fa9f0 con 0x7f3a88110600 2026-02-25T11:47:40.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.718+0000 7f3a8d3cc640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3a880fab60 con 0x7f3a88110fb0 2026-02-25T11:47:40.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.718+0000 7f3a8d3cc640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3a880fad20 con 0x7f3a8810f470 2026-02-25T11:47:40.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.718+0000 7f3a867fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a88110600 0x7f3a88171940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:40.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.718+0000 7f3a867fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a88110600 0x7f3a88171940 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35638/0 (socket says 10.20.193.28:35638) 2026-02-25T11:47:40.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.718+0000 7f3a867fc640 1 -- 10.20.193.28:0/2719198973 learned_addr learned my addr 10.20.193.28:0/2719198973 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:40.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.718+0000 7f3a877fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3a88110fb0 0x7f3a880f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:40.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.718+0000 7f3a86ffd640 1 --2- 10.20.193.28:0/2719198973 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3a8810f470 0x7f3a880f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:40.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.719+0000 7f3a867fc640 1 -- 10.20.193.28:0/2719198973 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3a8810f470 msgr2=0x7f3a880f95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:40.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.719+0000 7f3a867fc640 1 --2- 10.20.193.28:0/2719198973 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3a8810f470 0x7f3a880f95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.719+0000 7f3a867fc640 1 -- 10.20.193.28:0/2719198973 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3a88110fb0 msgr2=0x7f3a880f9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:40.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.719+0000 7f3a867fc640 1 --2- 10.20.193.28:0/2719198973 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3a88110fb0 0x7f3a880f9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.719+0000 7f3a867fc640 1 -- 10.20.193.28:0/2719198973 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a70009570 con 0x7f3a88110600 2026-02-25T11:47:40.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.719+0000 7f3a867fc640 1 --2- 10.20.193.28:0/2719198973 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a88110600 0x7f3a88171940 secure :-1 s=READY pgs=319 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3a700023b0 tx=0x7f3a700028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:40.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.719+0000 7f3a67fff640 1 -- 10.20.193.28:0/2719198973 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3a7003d070 con 0x7f3a88110600 2026-02-25T11:47:40.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.719+0000 7f3a67fff640 1 -- 10.20.193.28:0/2719198973 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3a70002a40 con 0x7f3a88110600 2026-02-25T11:47:40.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.719+0000 7f3a67fff640 1 -- 10.20.193.28:0/2719198973 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3a700418c0 con 0x7f3a88110600 2026-02-25T11:47:40.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.719+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/2719198973 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a880fafa0 con 0x7f3a88110600 2026-02-25T11:47:40.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.719+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/2719198973 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a880fb390 con 0x7f3a88110600 2026-02-25T11:47:40.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.720+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/2719198973 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a88175810 con 0x7f3a88110600 2026-02-25T11:47:40.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.722+0000 7f3a67fff640 1 -- 10.20.193.28:0/2719198973 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f3a70041a50 con 0x7f3a88110600 2026-02-25T11:47:40.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.723+0000 7f3a67fff640 1 --2- 10.20.193.28:0/2719198973 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3a5c0b57f0 0x7f3a5c0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:40.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.723+0000 7f3a67fff640 1 -- 10.20.193.28:0/2719198973 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f3a7010b6c0 con 0x7f3a88110600 2026-02-25T11:47:40.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.724+0000 7f3a86ffd640 1 --2- 10.20.193.28:0/2719198973 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3a5c0b57f0 0x7f3a5c0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:40.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.724+0000 7f3a67fff640 1 -- 10.20.193.28:0/2719198973 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f3a7010bb40 con 0x7f3a88110600 2026-02-25T11:47:40.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.724+0000 7f3a86ffd640 1 --2- 10.20.193.28:0/2719198973 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3a5c0b57f0 0x7f3a5c0d5ca0 secure :-1 s=READY pgs=234 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3a7c004740 tx=0x7f3a7c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:40.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.840+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/2719198973 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f3a8810f840 con 0x7f3a88110600 2026-02-25T11:47:40.843 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.841+0000 7f3a67fff640 1 -- 10.20.193.28:0/2719198973 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v86) ==== 56+0+5068 (secure 0 0 0) 0x7f3a700ce1d0 con 0x7f3a88110600 2026-02-25T11:47:40.844 INFO:teuthology.orchestra.run.trial028.stdout:epoch 86 2026-02-25T11:47:40.844 INFO:teuthology.orchestra.run.trial028.stdout:fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T11:47:40.844 INFO:teuthology.orchestra.run.trial028.stdout:created 2026-02-25T11:42:51.962844+0000 2026-02-25T11:47:40.844 INFO:teuthology.orchestra.run.trial028.stdout:modified 2026-02-25T11:47:14.624486+0000 2026-02-25T11:47:40.844 INFO:teuthology.orchestra.run.trial028.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2026-02-25T11:47:40.844 INFO:teuthology.orchestra.run.trial028.stdout:crush_version 38 2026-02-25T11:47:40.844 INFO:teuthology.orchestra.run.trial028.stdout:full_ratio 0.95 2026-02-25T11:47:40.845 INFO:teuthology.orchestra.run.trial028.stdout:backfillfull_ratio 0.9 2026-02-25T11:47:40.845 INFO:teuthology.orchestra.run.trial028.stdout:nearfull_ratio 0.85 2026-02-25T11:47:40.845 INFO:teuthology.orchestra.run.trial028.stdout:require_min_compat_client luminous 2026-02-25T11:47:40.845 INFO:teuthology.orchestra.run.trial028.stdout:min_compat_client jewel 2026-02-25T11:47:40.845 INFO:teuthology.orchestra.run.trial028.stdout:require_osd_release tentacle 2026-02-25T11:47:40.845 INFO:teuthology.orchestra.run.trial028.stdout:stretch_mode_enabled false 2026-02-25T11:47:40.845 INFO:teuthology.orchestra.run.trial028.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2026-02-25T11:47:40.845 INFO:teuthology.orchestra.run.trial028.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2026-02-25T11:47:40.845 INFO:teuthology.orchestra.run.trial028.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2026-02-25T11:47:40.845 INFO:teuthology.orchestra.run.trial028.stdout:max_osd 12 2026-02-25T11:47:40.845 INFO:teuthology.orchestra.run.trial028.stdout:osd.0 up in weight 1 up_from 9 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:10.20.193.28:6802/668944205,v1:10.20.193.28:6803/668944205] [v2:10.20.193.28:6804/668944205,v1:10.20.193.28:6805/668944205] exists,up bf596df7-d1fb-4d10-8d4c-14a35210bab2 2026-02-25T11:47:40.845 INFO:teuthology.orchestra.run.trial028.stdout:osd.1 up in weight 1 up_from 14 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:10.20.193.119:6800/3486566242,v1:10.20.193.119:6801/3486566242] [v2:10.20.193.119:6802/3486566242,v1:10.20.193.119:6803/3486566242] exists,up 185719c0-7c2d-431f-9925-88bbee86a98d 2026-02-25T11:47:40.846 INFO:teuthology.orchestra.run.trial028.stdout:osd.2 up in weight 1 up_from 19 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:10.20.193.167:6800/484781559,v1:10.20.193.167:6801/484781559] [v2:10.20.193.167:6802/484781559,v1:10.20.193.167:6803/484781559] exists,up 4de13b90-1c3c-4614-9fef-2d7315626433 2026-02-25T11:47:40.846 INFO:teuthology.orchestra.run.trial028.stdout:osd.3 up in weight 1 up_from 26 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:10.20.193.28:6810/2452541436,v1:10.20.193.28:6811/2452541436] [v2:10.20.193.28:6812/2452541436,v1:10.20.193.28:6813/2452541436] exists,up 45d9ea8e-5a77-4738-96da-ada98515fc02 2026-02-25T11:47:40.846 INFO:teuthology.orchestra.run.trial028.stdout:osd.4 up in weight 1 up_from 34 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:10.20.193.119:6808/135971996,v1:10.20.193.119:6809/135971996] [v2:10.20.193.119:6810/135971996,v1:10.20.193.119:6811/135971996] exists,up c9c3f512-665b-44b2-a6f0-f3356a4b661e 2026-02-25T11:47:40.846 INFO:teuthology.orchestra.run.trial028.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:10.20.193.167:6808/4203420884,v1:10.20.193.167:6809/4203420884] [v2:10.20.193.167:6810/4203420884,v1:10.20.193.167:6811/4203420884] exists,up 777c3819-5c90-4b2b-a545-c024884c2eb4 2026-02-25T11:47:40.846 INFO:teuthology.orchestra.run.trial028.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:10.20.193.28:6818/621484999,v1:10.20.193.28:6819/621484999] [v2:10.20.193.28:6820/621484999,v1:10.20.193.28:6821/621484999] exists,up bbd9de64-b9fe-499e-b451-c16828be2c01 2026-02-25T11:47:40.846 INFO:teuthology.orchestra.run.trial028.stdout:osd.7 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:10.20.193.119:6816/2724541747,v1:10.20.193.119:6817/2724541747] [v2:10.20.193.119:6818/2724541747,v1:10.20.193.119:6819/2724541747] exists,up 546d9f28-597b-454c-8456-fcc2d46c69f3 2026-02-25T11:47:40.846 INFO:teuthology.orchestra.run.trial028.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:10.20.193.167:6816/1328734317,v1:10.20.193.167:6817/1328734317] [v2:10.20.193.167:6818/1328734317,v1:10.20.193.167:6819/1328734317] exists,up a1a7cea6-3586-4bb2-aa4c-bf6f7799a501 2026-02-25T11:47:40.846 INFO:teuthology.orchestra.run.trial028.stdout:osd.9 up in weight 1 up_from 68 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:10.20.193.28:6826/354500625,v1:10.20.193.28:6827/354500625] [v2:10.20.193.28:6828/354500625,v1:10.20.193.28:6829/354500625] exists,up 03fef134-6946-43d5-9c26-dc0e59fffe51 2026-02-25T11:47:40.846 INFO:teuthology.orchestra.run.trial028.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:10.20.193.119:6824/2996581116,v1:10.20.193.119:6825/2996581116] [v2:10.20.193.119:6826/2996581116,v1:10.20.193.119:6827/2996581116] exists,up b46c29ef-285c-4184-be28-d99b76bc0e1d 2026-02-25T11:47:40.846 INFO:teuthology.orchestra.run.trial028.stdout:osd.11 up in weight 1 up_from 81 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:10.20.193.167:6824/2182906700,v1:10.20.193.167:6825/2182906700] [v2:10.20.193.167:6826/2182906700,v1:10.20.193.167:6827/2182906700] exists,up f6696d8c-5de7-4fd9-b693-524f204e4e11 2026-02-25T11:47:40.846 INFO:teuthology.orchestra.run.trial028.stdout:blocklist 10.20.193.28:0/1467902196 expires 2026-02-26T11:43:17.037464+0000 2026-02-25T11:47:40.846 INFO:teuthology.orchestra.run.trial028.stdout:blocklist 10.20.193.28:6801/2907029718 expires 2026-02-26T11:43:17.037464+0000 2026-02-25T11:47:40.847 INFO:teuthology.orchestra.run.trial028.stdout:blocklist 10.20.193.28:6800/2907029718 expires 2026-02-26T11:43:17.037464+0000 2026-02-25T11:47:40.847 INFO:teuthology.orchestra.run.trial028.stdout:blocklist 10.20.193.28:0/2274270913 expires 2026-02-26T11:43:17.037464+0000 2026-02-25T11:47:40.847 INFO:teuthology.orchestra.run.trial028.stdout:blocklist 10.20.193.28:0/60087422 expires 2026-02-26T11:43:17.037464+0000 2026-02-25T11:47:40.847 INFO:teuthology.orchestra.run.trial028.stdout:blocklist 10.20.193.28:6801/3221634678 expires 2026-02-26T11:43:04.102029+0000 2026-02-25T11:47:40.847 INFO:teuthology.orchestra.run.trial028.stdout:blocklist 10.20.193.28:6800/3221634678 expires 2026-02-26T11:43:04.102029+0000 2026-02-25T11:47:40.847 INFO:teuthology.orchestra.run.trial028.stdout:blocklist 10.20.193.28:0/1725332841 expires 2026-02-26T11:43:04.102029+0000 2026-02-25T11:47:40.847 INFO:teuthology.orchestra.run.trial028.stdout:blocklist 10.20.193.28:0/3082630924 expires 2026-02-26T11:43:04.102029+0000 2026-02-25T11:47:40.847 INFO:teuthology.orchestra.run.trial028.stdout:blocklist 10.20.193.28:0/2391715078 expires 2026-02-26T11:43:04.102029+0000 2026-02-25T11:47:40.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.842+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/2719198973 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3a5c0b57f0 msgr2=0x7f3a5c0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:40.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.842+0000 7f3a8d3cc640 1 --2- 10.20.193.28:0/2719198973 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3a5c0b57f0 0x7f3a5c0d5ca0 secure :-1 s=READY pgs=234 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3a7c004740 tx=0x7f3a7c005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:40.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.842+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/2719198973 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a88110600 msgr2=0x7f3a88171940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:40.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.842+0000 7f3a8d3cc640 1 --2- 10.20.193.28:0/2719198973 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a88110600 0x7f3a88171940 secure :-1 s=READY pgs=319 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3a700023b0 tx=0x7f3a700028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.842+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/2719198973 shutdown_connections 2026-02-25T11:47:40.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.842+0000 7f3a8d3cc640 1 --2- 10.20.193.28:0/2719198973 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3a5c0b57f0 0x7f3a5c0d5ca0 unknown :-1 s=CLOSED pgs=234 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.842+0000 7f3a8d3cc640 1 --2- 10.20.193.28:0/2719198973 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3a88110fb0 0x7f3a880f9e00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.842+0000 7f3a8d3cc640 1 --2- 10.20.193.28:0/2719198973 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3a88110600 0x7f3a88171940 unknown :-1 s=CLOSED pgs=319 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.842+0000 7f3a8d3cc640 1 --2- 10.20.193.28:0/2719198973 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3a8810f470 0x7f3a880f95a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:40.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.842+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/2719198973 >> 10.20.193.28:0/2719198973 conn(0x7f3a880ecba0 msgr2=0x7f3a880efc50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:40.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.843+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/2719198973 shutdown_connections 2026-02-25T11:47:40.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:40.843+0000 7f3a8d3cc640 1 -- 10.20.193.28:0/2719198973 wait complete. 2026-02-25T11:47:40.943 INFO:teuthology.run_tasks:Running task fs.ready... 2026-02-25T11:47:40.962 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:47:41.146 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:41.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.326+0000 7fede3577640 1 -- 10.20.193.28:0/3395755844 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fede4110080 msgr2=0x7fede41150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:41.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.326+0000 7fede3577640 1 --2- 10.20.193.28:0/3395755844 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fede4110080 0x7fede41150d0 secure :-1 s=READY pgs=99 gs=2 cs=0 l=1 c_cookie=9388911827962006738 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedcc009890 tx=0x7fedcc02f120 comp rx=0 tx=0).stop 2026-02-25T11:47:41.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.326+0000 7fede3577640 1 -- 10.20.193.28:0/3395755844 shutdown_connections 2026-02-25T11:47:41.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.326+0000 7fede3577640 1 --2- 10.20.193.28:0/3395755844 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fede4115880 0x7fede4176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.326+0000 7fede3577640 1 --2- 10.20.193.28:0/3395755844 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fede4110080 0x7fede41150d0 unknown :-1 s=CLOSED pgs=99 gs=2 cs=0 l=1 c_cookie=9388911827962006738 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.326+0000 7fede3577640 1 --2- 10.20.193.28:0/3395755844 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fede410f690 0x7fede410fa60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.326+0000 7fede3577640 1 -- 10.20.193.28:0/3395755844 >> 10.20.193.28:0/3395755844 conn(0x7fede40ecf00 msgr2=0x7fede410d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:41.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.326+0000 7fede3577640 1 -- 10.20.193.28:0/3395755844 shutdown_connections 2026-02-25T11:47:41.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.326+0000 7fede3577640 1 -- 10.20.193.28:0/3395755844 wait complete. 2026-02-25T11:47:41.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.327+0000 7fede3577640 1 Processor -- start 2026-02-25T11:47:41.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.327+0000 7fede3577640 1 -- start start 2026-02-25T11:47:41.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.327+0000 7fede3577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fede410f690 0x7fede411e0e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:41.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.327+0000 7fede3577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fede4110080 0x7fede411e620 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:41.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.327+0000 7fede3577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fede4115880 0x7fede41239f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:41.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.327+0000 7fede3577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fede4124010 con 0x7fede4110080 2026-02-25T11:47:41.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.327+0000 7fede3577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fede4124180 con 0x7fede4115880 2026-02-25T11:47:41.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.327+0000 7fede3577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fede4124340 con 0x7fede410f690 2026-02-25T11:47:41.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.327+0000 7fede1d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fede4110080 0x7fede411e620 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:41.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.327+0000 7fede2575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fede410f690 0x7fede411e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:41.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.327+0000 7fede2575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fede410f690 0x7fede411e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:46184/0 (socket says 10.20.193.28:46184) 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.327+0000 7fede2575640 1 -- 10.20.193.28:0/3342438933 learned_addr learned my addr 10.20.193.28:0/3342438933 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.327+0000 7fede2d76640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fede4115880 0x7fede41239f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.328+0000 7fede2d76640 1 -- 10.20.193.28:0/3342438933 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fede410f690 msgr2=0x7fede411e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.328+0000 7fede2d76640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fede410f690 0x7fede411e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.328+0000 7fede2d76640 1 -- 10.20.193.28:0/3342438933 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fede4110080 msgr2=0x7fede411e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.328+0000 7fede2d76640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fede4110080 0x7fede411e620 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.328+0000 7fede2d76640 1 -- 10.20.193.28:0/3342438933 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedcc0094b0 con 0x7fede4115880 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.328+0000 7fede2575640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fede410f690 0x7fede411e0e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.328+0000 7fede2d76640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fede4115880 0x7fede41239f0 secure :-1 s=READY pgs=100 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedd400f940 tx=0x7fedd400fe10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.328+0000 7fedcb7fe640 1 -- 10.20.193.28:0/3342438933 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fedd400dce0 con 0x7fede4115880 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.328+0000 7fedcb7fe640 1 -- 10.20.193.28:0/3342438933 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fedd400de40 con 0x7fede4115880 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.328+0000 7fede3577640 1 -- 10.20.193.28:0/3342438933 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede4124620 con 0x7fede4115880 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.328+0000 7fede3577640 1 -- 10.20.193.28:0/3342438933 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede4164d20 con 0x7fede4115880 2026-02-25T11:47:41.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.328+0000 7fedcb7fe640 1 -- 10.20.193.28:0/3342438933 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fedd40116b0 con 0x7fede4115880 2026-02-25T11:47:41.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.329+0000 7fede3577640 1 -- 10.20.193.28:0/3342438933 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede4110970 con 0x7fede4115880 2026-02-25T11:47:41.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.331+0000 7fedcb7fe640 1 -- 10.20.193.28:0/3342438933 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fedd4020060 con 0x7fede4115880 2026-02-25T11:47:41.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.332+0000 7fedcb7fe640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fedbc0b5770 0x7fedbc0d5c20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:41.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.332+0000 7fedcb7fe640 1 -- 10.20.193.28:0/3342438933 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fedd4015070 con 0x7fede4115880 2026-02-25T11:47:41.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.332+0000 7fedcb7fe640 1 -- 10.20.193.28:0/3342438933 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fedd40aa5f0 con 0x7fede4115880 2026-02-25T11:47:41.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.332+0000 7fede2575640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fedbc0b5770 0x7fedbc0d5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:41.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.333+0000 7fede2575640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fedbc0b5770 0x7fedbc0d5c20 secure :-1 s=READY pgs=235 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedd80097c0 tx=0x7fedd8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:41.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.473+0000 7fede3577640 1 -- 10.20.193.28:0/3342438933 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fede411f240 con 0x7fede4115880 2026-02-25T11:47:41.476 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:41 trial028 ceph-mon[39224]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:41.476 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:41 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1639988637' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2026-02-25T11:47:41.476 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:41 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2719198973' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2026-02-25T11:47:41.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.473+0000 7fedcb7fe640 1 -- 10.20.193.28:0/3342438933 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11940 (secure 0 0 0) 0x7fedd40ad7a0 con 0x7fede4115880 2026-02-25T11:47:41.477 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:41.477 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":22,"btime":"2026-02-25T11:47:18:567831+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:18.567827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24730,24731,24749]},"id":1}]} 2026-02-25T11:47:41.478 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 22 2026-02-25T11:47:41.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.474+0000 7fede3577640 1 -- 10.20.193.28:0/3342438933 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fedbc0b5770 msgr2=0x7fedbc0d5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:41.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.474+0000 7fede3577640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fedbc0b5770 0x7fedbc0d5c20 secure :-1 s=READY pgs=235 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedd80097c0 tx=0x7fedd8009310 comp rx=0 tx=0).stop 2026-02-25T11:47:41.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.474+0000 7fede3577640 1 -- 10.20.193.28:0/3342438933 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fede4115880 msgr2=0x7fede41239f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:41.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.474+0000 7fede3577640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fede4115880 0x7fede41239f0 secure :-1 s=READY pgs=100 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fedd400f940 tx=0x7fedd400fe10 comp rx=0 tx=0).stop 2026-02-25T11:47:41.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.474+0000 7fede3577640 1 -- 10.20.193.28:0/3342438933 shutdown_connections 2026-02-25T11:47:41.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.474+0000 7fede3577640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fedbc0b5770 0x7fedbc0d5c20 unknown :-1 s=CLOSED pgs=235 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.474+0000 7fede3577640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fede4115880 0x7fede41239f0 unknown :-1 s=CLOSED pgs=100 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.474+0000 7fede3577640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fede4110080 0x7fede411e620 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.474+0000 7fede3577640 1 --2- 10.20.193.28:0/3342438933 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fede410f690 0x7fede411e0e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.474+0000 7fede3577640 1 -- 10.20.193.28:0/3342438933 >> 10.20.193.28:0/3342438933 conn(0x7fede40ecf00 msgr2=0x7fede410d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:41.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.475+0000 7fede3577640 1 -- 10.20.193.28:0/3342438933 shutdown_connections 2026-02-25T11:47:41.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.475+0000 7fede3577640 1 -- 10.20.193.28:0/3342438933 wait complete. 2026-02-25T11:47:41.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:41 trial167 ceph-mon[43392]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:41.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:41 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1639988637' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2026-02-25T11:47:41.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:41 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2719198973' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2026-02-25T11:47:41.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:41 trial119 ceph-mon[42710]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2026-02-25T11:47:41.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:41 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1639988637' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2026-02-25T11:47:41.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:41 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2719198973' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2026-02-25T11:47:41.576 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:47:41.758 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:41.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.928+0000 7f423c18e640 1 -- 10.20.193.28:0/1252597099 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4234115a30 msgr2=0x7f4234176400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:41.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.928+0000 7f423c18e640 1 --2- 10.20.193.28:0/1252597099 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4234115a30 0x7f4234176400 secure :-1 s=READY pgs=320 gs=1 cs=0 l=1 c_cookie=1104562301072249691 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f423000aa30 tx=0x7f4230030270 comp rx=0 tx=0).stop 2026-02-25T11:47:41.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.928+0000 7f423c18e640 1 -- 10.20.193.28:0/1252597099 shutdown_connections 2026-02-25T11:47:41.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.928+0000 7f423c18e640 1 --2- 10.20.193.28:0/1252597099 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4234115a30 0x7f4234176400 unknown :-1 s=CLOSED pgs=320 gs=1 cs=0 l=1 c_cookie=1104562301072249691 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.928+0000 7f423c18e640 1 --2- 10.20.193.28:0/1252597099 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4234110230 0x7f4234115280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.928+0000 7f423c18e640 1 --2- 10.20.193.28:0/1252597099 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f423410f840 0x7f423410fc10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.928+0000 7f423c18e640 1 -- 10.20.193.28:0/1252597099 >> 10.20.193.28:0/1252597099 conn(0x7f42340ed130 msgr2=0x7f423410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:41.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.928+0000 7f423c18e640 1 -- 10.20.193.28:0/1252597099 shutdown_connections 2026-02-25T11:47:41.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.928+0000 7f423c18e640 1 -- 10.20.193.28:0/1252597099 wait complete. 2026-02-25T11:47:41.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.929+0000 7f423c18e640 1 Processor -- start 2026-02-25T11:47:41.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.929+0000 7f423c18e640 1 -- start start 2026-02-25T11:47:41.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.929+0000 7f423c18e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f423410f840 0x7f42340f5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:41.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.929+0000 7f423c18e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4234110230 0x7f4234176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:41.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.929+0000 7f423c18e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4234115a30 0x7f42340f6290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:41.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.929+0000 7f423c18e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f42340fb4a0 con 0x7f4234110230 2026-02-25T11:47:41.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.929+0000 7f423c18e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f42340fb610 con 0x7f423410f840 2026-02-25T11:47:41.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.929+0000 7f423c18e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42340fb7d0 con 0x7f4234115a30 2026-02-25T11:47:41.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.929+0000 7f4239f03640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f423410f840 0x7f42340f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:41.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.929+0000 7f4239f03640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f423410f840 0x7f42340f5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43416/0 (socket says 10.20.193.28:43416) 2026-02-25T11:47:41.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.929+0000 7f4239f03640 1 -- 10.20.193.28:0/127234732 learned_addr learned my addr 10.20.193.28:0/127234732 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:41.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.929+0000 7f423a704640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4234115a30 0x7f42340f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:41.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.929+0000 7f4239702640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4234110230 0x7f4234176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:41.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.930+0000 7f4239f03640 1 -- 10.20.193.28:0/127234732 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4234115a30 msgr2=0x7f42340f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:41.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.930+0000 7f4239f03640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4234115a30 0x7f42340f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.930+0000 7f4239f03640 1 -- 10.20.193.28:0/127234732 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4234110230 msgr2=0x7f4234176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:41.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.930+0000 7f4239f03640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4234110230 0x7f4234176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:41.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.930+0000 7f4239f03640 1 -- 10.20.193.28:0/127234732 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423000a650 con 0x7f423410f840 2026-02-25T11:47:41.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.930+0000 7f4239702640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4234110230 0x7f4234176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:41.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.930+0000 7f423a704640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4234115a30 0x7f42340f6290 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:41.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.930+0000 7f4239f03640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f423410f840 0x7f42340f5b90 secure :-1 s=READY pgs=101 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4224009480 tx=0x7f422400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:41.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.930+0000 7f4222ffd640 1 -- 10.20.193.28:0/127234732 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f422400cc80 con 0x7f423410f840 2026-02-25T11:47:41.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.930+0000 7f4222ffd640 1 -- 10.20.193.28:0/127234732 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f422400cde0 con 0x7f423410f840 2026-02-25T11:47:41.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.930+0000 7f4222ffd640 1 -- 10.20.193.28:0/127234732 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f42240106b0 con 0x7f423410f840 2026-02-25T11:47:41.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.930+0000 7f423c18e640 1 -- 10.20.193.28:0/127234732 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42340fbab0 con 0x7f423410f840 2026-02-25T11:47:41.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.930+0000 7f423c18e640 1 -- 10.20.193.28:0/127234732 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42340fbf30 con 0x7f423410f840 2026-02-25T11:47:41.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.931+0000 7f4220ff9640 1 -- 10.20.193.28:0/127234732 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4234110b20 con 0x7f423410f840 2026-02-25T11:47:41.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.933+0000 7f4222ffd640 1 -- 10.20.193.28:0/127234732 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f4224004080 con 0x7f423410f840 2026-02-25T11:47:41.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.935+0000 7f4222ffd640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42100b57e0 0x7f42100d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:41.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.935+0000 7f4222ffd640 1 -- 10.20.193.28:0/127234732 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f4224014070 con 0x7f423410f840 2026-02-25T11:47:41.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.935+0000 7f4239702640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42100b57e0 0x7f42100d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:41.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.935+0000 7f4222ffd640 1 -- 10.20.193.28:0/127234732 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f42240e7410 con 0x7f423410f840 2026-02-25T11:47:41.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:41.936+0000 7f4239702640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42100b57e0 0x7f42100d5c90 secure :-1 s=READY pgs=236 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42340fc530 tx=0x7f4228005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:42.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.075+0000 7f4220ff9640 1 -- 10.20.193.28:0/127234732 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f423410fc10 con 0x7f423410f840 2026-02-25T11:47:42.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.075+0000 7f4222ffd640 1 -- 10.20.193.28:0/127234732 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+11940 (secure 0 0 0) 0x7f42240a9aa0 con 0x7f423410f840 2026-02-25T11:47:42.079 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:42.080 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":22,"btime":"2026-02-25T11:47:18:567831+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:18.567827+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24730,24731,24749]},"id":1}]} 2026-02-25T11:47:42.081 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 22 2026-02-25T11:47:42.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.076+0000 7f4220ff9640 1 -- 10.20.193.28:0/127234732 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42100b57e0 msgr2=0x7f42100d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:42.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.076+0000 7f4220ff9640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42100b57e0 0x7f42100d5c90 secure :-1 s=READY pgs=236 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42340fc530 tx=0x7f4228005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:42.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.076+0000 7f4220ff9640 1 -- 10.20.193.28:0/127234732 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f423410f840 msgr2=0x7f42340f5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:42.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.076+0000 7f4220ff9640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f423410f840 0x7f42340f5b90 secure :-1 s=READY pgs=101 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4224009480 tx=0x7f422400ee60 comp rx=0 tx=0).stop 2026-02-25T11:47:42.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.077+0000 7f4220ff9640 1 -- 10.20.193.28:0/127234732 shutdown_connections 2026-02-25T11:47:42.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.077+0000 7f4220ff9640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42100b57e0 0x7f42100d5c90 unknown :-1 s=CLOSED pgs=236 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.077+0000 7f4220ff9640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4234115a30 0x7f42340f6290 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.077+0000 7f4220ff9640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4234110230 0x7f4234176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.077+0000 7f4220ff9640 1 --2- 10.20.193.28:0/127234732 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f423410f840 0x7f42340f5b90 unknown :-1 s=CLOSED pgs=101 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.077+0000 7f4220ff9640 1 -- 10.20.193.28:0/127234732 >> 10.20.193.28:0/127234732 conn(0x7f42340ed130 msgr2=0x7f423410dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:42.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.077+0000 7f4220ff9640 1 -- 10.20.193.28:0/127234732 shutdown_connections 2026-02-25T11:47:42.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.077+0000 7f4220ff9640 1 -- 10.20.193.28:0/127234732 wait complete. 2026-02-25T11:47:42.182 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd dump --format=json 2026-02-25T11:47:42.364 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:42.406 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:42 trial028 ceph-mon[39224]: pgmap v208: 97 pgs: 97 active+clean; 646 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2026-02-25T11:47:42.406 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:42 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3342438933' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:42.406 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:42 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/127234732' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:42.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:42 trial167 ceph-mon[43392]: pgmap v208: 97 pgs: 97 active+clean; 646 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2026-02-25T11:47:42.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:42 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3342438933' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:42.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:42 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/127234732' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:42.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:42 trial119 ceph-mon[42710]: pgmap v208: 97 pgs: 97 active+clean; 646 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2026-02-25T11:47:42.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:42 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3342438933' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:42.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:42 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/127234732' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:42.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.542+0000 7f95ff2bd640 1 -- 10.20.193.28:0/1630937696 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f95f810f840 msgr2=0x7f95f810fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:42.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.542+0000 7f95ff2bd640 1 --2- 10.20.193.28:0/1630937696 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f95f810f840 0x7f95f810fc10 secure :-1 s=READY pgs=321 gs=1 cs=0 l=1 c_cookie=13166767332527734569 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f95e8009950 tx=0x7f95e802f190 comp rx=0 tx=0).stop 2026-02-25T11:47:42.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.542+0000 7f95ff2bd640 1 -- 10.20.193.28:0/1630937696 shutdown_connections 2026-02-25T11:47:42.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.542+0000 7f95ff2bd640 1 --2- 10.20.193.28:0/1630937696 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f95f8115a30 0x7f95f8176400 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.542+0000 7f95ff2bd640 1 --2- 10.20.193.28:0/1630937696 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f95f8110230 0x7f95f8115280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.542+0000 7f95ff2bd640 1 --2- 10.20.193.28:0/1630937696 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f95f810f840 0x7f95f810fc10 unknown :-1 s=CLOSED pgs=321 gs=1 cs=0 l=1 c_cookie=13166767332527734569 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.542+0000 7f95ff2bd640 1 -- 10.20.193.28:0/1630937696 >> 10.20.193.28:0/1630937696 conn(0x7f95f80ed130 msgr2=0x7f95f810d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.542+0000 7f95ff2bd640 1 -- 10.20.193.28:0/1630937696 shutdown_connections 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.542+0000 7f95ff2bd640 1 -- 10.20.193.28:0/1630937696 wait complete. 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.542+0000 7f95ff2bd640 1 Processor -- start 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95ff2bd640 1 -- start start 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95ff2bd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f95f810f840 0x7f95f80f5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95ff2bd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f95f8110230 0x7f95f8176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95ff2bd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f95f8115a30 0x7f95f80f6240 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95ff2bd640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f95f80fb450 con 0x7f95f810f840 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95ff2bd640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f95f80fb5c0 con 0x7f95f8115a30 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95ff2bd640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95f80fb780 con 0x7f95f8110230 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95fd032640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f95f810f840 0x7f95f80f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95fd032640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f95f810f840 0x7f95f80f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35692/0 (socket says 10.20.193.28:35692) 2026-02-25T11:47:42.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95fd032640 1 -- 10.20.193.28:0/2097807841 learned_addr learned my addr 10.20.193.28:0/2097807841 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95fc831640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f95f8110230 0x7f95f8176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95fd833640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f95f8115a30 0x7f95f80f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95fd032640 1 -- 10.20.193.28:0/2097807841 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f95f8110230 msgr2=0x7f95f8176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95fd032640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f95f8110230 0x7f95f8176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95fd032640 1 -- 10.20.193.28:0/2097807841 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f95f8115a30 msgr2=0x7f95f80f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95fd032640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f95f8115a30 0x7f95f80f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95fd032640 1 -- 10.20.193.28:0/2097807841 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e8009570 con 0x7f95f810f840 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.543+0000 7f95fd032640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f95f810f840 0x7f95f80f5b40 secure :-1 s=READY pgs=322 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f95e80098a0 tx=0x7f95e80028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.544+0000 7f95e67fc640 1 -- 10.20.193.28:0/2097807841 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f95e803d070 con 0x7f95f810f840 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.544+0000 7f95ff2bd640 1 -- 10.20.193.28:0/2097807841 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f80fba00 con 0x7f95f810f840 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.544+0000 7f95e67fc640 1 -- 10.20.193.28:0/2097807841 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f95e8002a40 con 0x7f95f810f840 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.544+0000 7f95e67fc640 1 -- 10.20.193.28:0/2097807841 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f95e8041720 con 0x7f95f810f840 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.544+0000 7f95ff2bd640 1 -- 10.20.193.28:0/2097807841 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f80fbec0 con 0x7f95f810f840 2026-02-25T11:47:42.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.544+0000 7f95c7fff640 1 -- 10.20.193.28:0/2097807841 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f8110b20 con 0x7f95f810f840 2026-02-25T11:47:42.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.546+0000 7f95e67fc640 1 -- 10.20.193.28:0/2097807841 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f95e80386a0 con 0x7f95f810f840 2026-02-25T11:47:42.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.548+0000 7f95e67fc640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f95d40b57e0 0x7f95d40d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:42.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.548+0000 7f95e67fc640 1 -- 10.20.193.28:0/2097807841 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f95e810b3a0 con 0x7f95f810f840 2026-02-25T11:47:42.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.548+0000 7f95fc831640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f95d40b57e0 0x7f95d40d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:42.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.548+0000 7f95e67fc640 1 -- 10.20.193.28:0/2097807841 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f95e810b820 con 0x7f95f810f840 2026-02-25T11:47:42.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.549+0000 7f95fc831640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f95d40b57e0 0x7f95d40d5c90 secure :-1 s=READY pgs=237 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f95f80fc4e0 tx=0x7f95ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:42.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.663+0000 7f95c7fff640 1 -- 10.20.193.28:0/2097807841 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95f80f6ca0 con 0x7f95f810f840 2026-02-25T11:47:42.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.664+0000 7f95e67fc640 1 -- 10.20.193.28:0/2097807841 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22340 (secure 0 0 0) 0x7f95e80cde00 con 0x7f95f810f840 2026-02-25T11:47:42.667 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:42.667 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":86,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","created":"2026-02-25T11:42:51.962844+0000","modified":"2026-02-25T11:47:14.624486+0000","last_up_change":"2026-02-25T11:46:45.156629+0000","last_in_change":"2026-02-25T11:46:31.585002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":38,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T11:44:43.266336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-25T11:47:11.236575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-25T11:47:12.196775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf596df7-d1fb-4d10-8d4c-14a35210bab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6803","nonce":668944205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6805","nonce":668944205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6809","nonce":668944205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6807","nonce":668944205}]},"public_addr":"10.20.193.28:6803/668944205","cluster_addr":"10.20.193.28:6805/668944205","heartbeat_back_addr":"10.20.193.28:6809/668944205","heartbeat_front_addr":"10.20.193.28:6807/668944205","state":["exists","up"]},{"osd":1,"uuid":"185719c0-7c2d-431f-9925-88bbee86a98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6800","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6801","nonce":3486566242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6802","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6803","nonce":3486566242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6806","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6807","nonce":3486566242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6804","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6805","nonce":3486566242}]},"public_addr":"10.20.193.119:6801/3486566242","cluster_addr":"10.20.193.119:6803/3486566242","heartbeat_back_addr":"10.20.193.119:6807/3486566242","heartbeat_front_addr":"10.20.193.119:6805/3486566242","state":["exists","up"]},{"osd":2,"uuid":"4de13b90-1c3c-4614-9fef-2d7315626433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6800","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6801","nonce":484781559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6802","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6803","nonce":484781559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6806","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6807","nonce":484781559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6804","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6805","nonce":484781559}]},"public_addr":"10.20.193.167:6801/484781559","cluster_addr":"10.20.193.167:6803/484781559","heartbeat_back_addr":"10.20.193.167:6807/484781559","heartbeat_front_addr":"10.20.193.167:6805/484781559","state":["exists","up"]},{"osd":3,"uuid":"45d9ea8e-5a77-4738-96da-ada98515fc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6811","nonce":2452541436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6813","nonce":2452541436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6817","nonce":2452541436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6815","nonce":2452541436}]},"public_addr":"10.20.193.28:6811/2452541436","cluster_addr":"10.20.193.28:6813/2452541436","heartbeat_back_addr":"10.20.193.28:6817/2452541436","heartbeat_front_addr":"10.20.193.28:6815/2452541436","state":["exists","up"]},{"osd":4,"uuid":"c9c3f512-665b-44b2-a6f0-f3356a4b661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6808","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6809","nonce":135971996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6810","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6811","nonce":135971996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6814","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6815","nonce":135971996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6812","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6813","nonce":135971996}]},"public_addr":"10.20.193.119:6809/135971996","cluster_addr":"10.20.193.119:6811/135971996","heartbeat_back_addr":"10.20.193.119:6815/135971996","heartbeat_front_addr":"10.20.193.119:6813/135971996","state":["exists","up"]},{"osd":5,"uuid":"777c3819-5c90-4b2b-a545-c024884c2eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6808","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6809","nonce":4203420884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6810","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6811","nonce":4203420884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6814","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6815","nonce":4203420884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6812","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6813","nonce":4203420884}]},"public_addr":"10.20.193.167:6809/4203420884","cluster_addr":"10.20.193.167:6811/4203420884","heartbeat_back_addr":"10.20.193.167:6815/4203420884","heartbeat_front_addr":"10.20.193.167:6813/4203420884","state":["exists","up"]},{"osd":6,"uuid":"bbd9de64-b9fe-499e-b451-c16828be2c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6819","nonce":621484999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6821","nonce":621484999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6825","nonce":621484999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6823","nonce":621484999}]},"public_addr":"10.20.193.28:6819/621484999","cluster_addr":"10.20.193.28:6821/621484999","heartbeat_back_addr":"10.20.193.28:6825/621484999","heartbeat_front_addr":"10.20.193.28:6823/621484999","state":["exists","up"]},{"osd":7,"uuid":"546d9f28-597b-454c-8456-fcc2d46c69f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6816","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6817","nonce":2724541747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6818","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6819","nonce":2724541747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6822","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6823","nonce":2724541747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6820","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6821","nonce":2724541747}]},"public_addr":"10.20.193.119:6817/2724541747","cluster_addr":"10.20.193.119:6819/2724541747","heartbeat_back_addr":"10.20.193.119:6823/2724541747","heartbeat_front_addr":"10.20.193.119:6821/2724541747","state":["exists","up"]},{"osd":8,"uuid":"a1a7cea6-3586-4bb2-aa4c-bf6f7799a501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6816","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6817","nonce":1328734317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6818","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6819","nonce":1328734317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6822","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6823","nonce":1328734317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6820","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6821","nonce":1328734317}]},"public_addr":"10.20.193.167:6817/1328734317","cluster_addr":"10.20.193.167:6819/1328734317","heartbeat_back_addr":"10.20.193.167:6823/1328734317","heartbeat_front_addr":"10.20.193.167:6821/1328734317","state":["exists","up"]},{"osd":9,"uuid":"03fef134-6946-43d5-9c26-dc0e59fffe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6827","nonce":354500625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6829","nonce":354500625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6833","nonce":354500625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6831","nonce":354500625}]},"public_addr":"10.20.193.28:6827/354500625","cluster_addr":"10.20.193.28:6829/354500625","heartbeat_back_addr":"10.20.193.28:6833/354500625","heartbeat_front_addr":"10.20.193.28:6831/354500625","state":["exists","up"]},{"osd":10,"uuid":"b46c29ef-285c-4184-be28-d99b76bc0e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6824","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6825","nonce":2996581116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6826","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6827","nonce":2996581116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6830","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6831","nonce":2996581116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6828","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6829","nonce":2996581116}]},"public_addr":"10.20.193.119:6825/2996581116","cluster_addr":"10.20.193.119:6827/2996581116","heartbeat_back_addr":"10.20.193.119:6831/2996581116","heartbeat_front_addr":"10.20.193.119:6829/2996581116","state":["exists","up"]},{"osd":11,"uuid":"f6696d8c-5de7-4fd9-b693-524f204e4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6824","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6825","nonce":2182906700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6826","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6827","nonce":2182906700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6830","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6831","nonce":2182906700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6828","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6829","nonce":2182906700}]},"public_addr":"10.20.193.167:6825/2182906700","cluster_addr":"10.20.193.167:6827/2182906700","heartbeat_back_addr":"10.20.193.167:6831/2182906700","heartbeat_front_addr":"10.20.193.167:6829/2182906700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:13.410235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:27.254773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:40.956615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:54.529866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:08.306772+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:21.899498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:35.500407+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:49.094965+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:02.790138+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:16.286937+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:29.922722+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:43.396028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.28:0/1467902196":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6800/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/2274270913":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/60087422":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:6800/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/1725332841":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/3082630924":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/2391715078":"2026-02-26T11:43:04.102029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T11:47:42.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.665+0000 7f95c7fff640 1 -- 10.20.193.28:0/2097807841 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f95d40b57e0 msgr2=0x7f95d40d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:42.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.665+0000 7f95c7fff640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f95d40b57e0 0x7f95d40d5c90 secure :-1 s=READY pgs=237 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f95f80fc4e0 tx=0x7f95ec005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:42.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.665+0000 7f95c7fff640 1 -- 10.20.193.28:0/2097807841 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f95f810f840 msgr2=0x7f95f80f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:42.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.665+0000 7f95c7fff640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f95f810f840 0x7f95f80f5b40 secure :-1 s=READY pgs=322 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f95e80098a0 tx=0x7f95e80028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.666+0000 7f95c7fff640 1 -- 10.20.193.28:0/2097807841 shutdown_connections 2026-02-25T11:47:42.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.666+0000 7f95c7fff640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f95d40b57e0 0x7f95d40d5c90 unknown :-1 s=CLOSED pgs=237 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.666+0000 7f95c7fff640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f95f8115a30 0x7f95f80f6240 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.666+0000 7f95c7fff640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f95f8110230 0x7f95f8176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.666+0000 7f95c7fff640 1 --2- 10.20.193.28:0/2097807841 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f95f810f840 0x7f95f80f5b40 unknown :-1 s=CLOSED pgs=322 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:42.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.666+0000 7f95c7fff640 1 -- 10.20.193.28:0/2097807841 >> 10.20.193.28:0/2097807841 conn(0x7f95f80ed130 msgr2=0x7f95f8111c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:42.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.666+0000 7f95c7fff640 1 -- 10.20.193.28:0/2097807841 shutdown_connections 2026-02-25T11:47:42.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:42.666+0000 7f95c7fff640 1 -- 10.20.193.28:0/2097807841 wait complete. 2026-02-25T11:47:42.769 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False, 'allow_referent_inodes': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2026-02-25T11:47:13.223613+0000', 'modified': '2026-02-25T11:47:18.567827+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24749, 'mds_1': 24731, 'mds_2': 24730}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24706': {'gid': 24706, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '10.20.193.167:6835/3681115733', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '10.20.193.167:6834', 'nonce': 3681115733}, {'type': 'v1', 'addr': '10.20.193.167:6835', 'nonce': 3681115733}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4544132024016699391, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24707': {'gid': 24707, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '10.20.193.167:6837/3206735083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '10.20.193.167:6836', 'nonce': 3206735083}, {'type': 'v1', 'addr': '10.20.193.167:6837', 'nonce': 3206735083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4544132024016699391, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24719': {'gid': 24719, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '10.20.193.119:6837/392187157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '10.20.193.119:6836', 'nonce': 392187157}, {'type': 'v1', 'addr': '10.20.193.119:6837', 'nonce': 392187157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4544132024016699391, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24730': {'gid': 24730, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 5, 'addr': '10.20.193.167:6839/408350258', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '10.20.193.167:6838', 'nonce': 408350258}, {'type': 'v1', 'addr': '10.20.193.167:6839', 'nonce': 408350258}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4544132024016699391, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24731': {'gid': 24731, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '10.20.193.28:6839/3137382094', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '10.20.193.28:6838', 'nonce': 3137382094}, {'type': 'v1', 'addr': '10.20.193.28:6839', 'nonce': 3137382094}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4544132024016699391, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24749': {'gid': 24749, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '10.20.193.119:6839/1063465486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '10.20.193.119:6838', 'nonce': 1063465486}, {'type': 'v1', 'addr': '10.20.193.119:6839', 'nonce': 1063465486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4544132024016699391, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24749, 'qdb_cluster': [24730, 24731, 24749]} 2026-02-25T11:47:42.769 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2026-02-25T11:47:42.769 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.i status 2026-02-25T11:47:42.954 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:43.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.123+0000 7f596e7ed640 1 -- 10.20.193.28:0/2999885820 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f596810f840 msgr2=0x7f596810fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.123+0000 7f596e7ed640 1 --2- 10.20.193.28:0/2999885820 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f596810f840 0x7f596810fc10 secure :-1 s=READY pgs=323 gs=1 cs=0 l=1 c_cookie=3496282785025215359 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5954009950 tx=0x7f595402f190 comp rx=0 tx=0).stop 2026-02-25T11:47:43.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.123+0000 7f596e7ed640 1 -- 10.20.193.28:0/2999885820 shutdown_connections 2026-02-25T11:47:43.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.123+0000 7f596e7ed640 1 --2- 10.20.193.28:0/2999885820 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5968115a30 0x7f5968176400 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.123+0000 7f596e7ed640 1 --2- 10.20.193.28:0/2999885820 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5968110230 0x7f5968115280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.123+0000 7f596e7ed640 1 --2- 10.20.193.28:0/2999885820 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f596810f840 0x7f596810fc10 unknown :-1 s=CLOSED pgs=323 gs=1 cs=0 l=1 c_cookie=3496282785025215359 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.123+0000 7f596e7ed640 1 -- 10.20.193.28:0/2999885820 >> 10.20.193.28:0/2999885820 conn(0x7f59680ed130 msgr2=0x7f596810d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:43.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.123+0000 7f596e7ed640 1 -- 10.20.193.28:0/2999885820 shutdown_connections 2026-02-25T11:47:43.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.123+0000 7f596e7ed640 1 -- 10.20.193.28:0/2999885820 wait complete. 2026-02-25T11:47:43.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.124+0000 7f596e7ed640 1 Processor -- start 2026-02-25T11:47:43.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.124+0000 7f596e7ed640 1 -- start start 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.124+0000 7f596e7ed640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f596810f840 0x7f59680f5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.124+0000 7f596e7ed640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5968110230 0x7f5968176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.124+0000 7f596e7ed640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5968115a30 0x7f59680f6310 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.124+0000 7f596e7ed640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f59680fb520 con 0x7f596810f840 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.124+0000 7f596e7ed640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f59680fb690 con 0x7f5968110230 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.124+0000 7f596e7ed640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59680fb850 con 0x7f5968115a30 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.124+0000 7f5967fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f596810f840 0x7f59680f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.124+0000 7f5967fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f596810f840 0x7f59680f5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35706/0 (socket says 10.20.193.28:35706) 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.124+0000 7f5967fff640 1 -- 10.20.193.28:0/601669517 learned_addr learned my addr 10.20.193.28:0/601669517 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.125+0000 7f5967fff640 1 -- 10.20.193.28:0/601669517 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5968115a30 msgr2=0x7f59680f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.125+0000 7f5967fff640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5968115a30 0x7f59680f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.125+0000 7f5967fff640 1 -- 10.20.193.28:0/601669517 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5968110230 msgr2=0x7f5968176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.125+0000 7f59677fe640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5968110230 0x7f5968176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.125+0000 7f5967fff640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5968110230 0x7f5968176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.125+0000 7f5967fff640 1 -- 10.20.193.28:0/601669517 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5954009570 con 0x7f596810f840 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.125+0000 7f59677fe640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5968110230 0x7f5968176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.125+0000 7f5967fff640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f596810f840 0x7f59680f5c10 secure :-1 s=READY pgs=324 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5954004820 tx=0x7f59540028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.125+0000 7f59657fa640 1 -- 10.20.193.28:0/601669517 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f595403d070 con 0x7f596810f840 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.125+0000 7f59657fa640 1 -- 10.20.193.28:0/601669517 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f59540043b0 con 0x7f596810f840 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.125+0000 7f596e7ed640 1 -- 10.20.193.28:0/601669517 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59680fbad0 con 0x7f596810f840 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.125+0000 7f59657fa640 1 -- 10.20.193.28:0/601669517 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5954041960 con 0x7f596810f840 2026-02-25T11:47:43.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.125+0000 7f596e7ed640 1 -- 10.20.193.28:0/601669517 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59680fbf90 con 0x7f596810f840 2026-02-25T11:47:43.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.127+0000 7f59657fa640 1 -- 10.20.193.28:0/601669517 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f5954041b70 con 0x7f596810f840 2026-02-25T11:47:43.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.129+0000 7f59657fa640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f593c0b57f0 0x7f593c0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.130+0000 7f59657fa640 1 -- 10.20.193.28:0/601669517 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f595410b2c0 con 0x7f596810f840 2026-02-25T11:47:43.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.130+0000 7f59677fe640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f593c0b57f0 0x7f593c0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.130+0000 7f59677fe640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f593c0b57f0 0x7f593c0d5ca0 secure :-1 s=READY pgs=238 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f59680fc5b0 tx=0x7f5958008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.132+0000 7f59707e8e00 1 Processor -- start 2026-02-25T11:47:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.132+0000 7f59707e8e00 1 -- start start 2026-02-25T11:47:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f59707e8e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x55802946c850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f59707e8e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55802946cd90 0x5580294b14f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f59707e8e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5580294b1a30 0x5580294d1e10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f59707e8e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558029382a30 con 0x55802946c480 2026-02-25T11:47:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f59707e8e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55802937e760 con 0x55802946cd90 2026-02-25T11:47:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f59707e8e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558029378700 con 0x5580294b1a30 2026-02-25T11:47:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f5967fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x55802946c850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f5967fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x55802946c850 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35714/0 (socket says 10.20.193.28:35714) 2026-02-25T11:47:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f5967fff640 1 -- 10.20.193.28:0/87690886 learned_addr learned my addr 10.20.193.28:0/87690886 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f59677fe640 1 --2- 10.20.193.28:0/87690886 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55802946cd90 0x5580294b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f596cd63640 1 --2- 10.20.193.28:0/87690886 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5580294b1a30 0x5580294d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f5967fff640 1 -- 10.20.193.28:0/87690886 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5580294b1a30 msgr2=0x5580294d1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f5967fff640 1 --2- 10.20.193.28:0/87690886 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5580294b1a30 0x5580294d1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f5967fff640 1 -- 10.20.193.28:0/87690886 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55802946cd90 msgr2=0x5580294b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f5967fff640 1 --2- 10.20.193.28:0/87690886 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55802946cd90 0x5580294b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f5967fff640 1 -- 10.20.193.28:0/87690886 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558029323cf0 con 0x55802946c480 2026-02-25T11:47:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.133+0000 7f5967fff640 1 --2- 10.20.193.28:0/87690886 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x55802946c850 secure :-1 s=READY pgs=325 gs=1 cs=0 l=1 c_cookie=3878122803237554830 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f59540d16b0 tx=0x7f59540d10b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d1dc2472510a8e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.134+0000 7f5938ce3640 1 -- 10.20.193.28:0/87690886 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f595402fe20 con 0x55802946c480 2026-02-25T11:47:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.134+0000 7f5938ce3640 1 -- 10.20.193.28:0/87690886 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f59540d73f0 con 0x55802946c480 2026-02-25T11:47:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.134+0000 7f5938ce3640 1 -- 10.20.193.28:0/87690886 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f59540dc380 con 0x55802946c480 2026-02-25T11:47:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.134+0000 7f59707e8e00 1 -- 10.20.193.28:0/87690886 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 msgr2=0x55802946c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.134+0000 7f59707e8e00 1 --2- 10.20.193.28:0/87690886 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x55802946c850 secure :-1 s=READY pgs=325 gs=1 cs=0 l=1 c_cookie=3878122803237554830 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f59540d16b0 tx=0x7f59540d10b0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.134+0000 7f59707e8e00 1 -- 10.20.193.28:0/87690886 shutdown_connections 2026-02-25T11:47:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.134+0000 7f59707e8e00 1 --2- 10.20.193.28:0/87690886 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5580294b1a30 0x5580294d1e10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.134+0000 7f59707e8e00 1 --2- 10.20.193.28:0/87690886 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55802946cd90 0x5580294b14f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.134+0000 7f59707e8e00 1 --2- 10.20.193.28:0/87690886 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x55802946c850 unknown :-1 s=CLOSED pgs=325 gs=1 cs=0 l=1 c_cookie=3878122803237554830 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.134+0000 7f59707e8e00 1 -- 10.20.193.28:0/87690886 >> 10.20.193.28:0/87690886 conn(0x55802938c720 msgr2=0x55802946ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.134+0000 7f59707e8e00 1 -- 10.20.193.28:0/87690886 shutdown_connections 2026-02-25T11:47:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.134+0000 7f59707e8e00 1 -- 10.20.193.28:0/87690886 wait complete. 2026-02-25T11:47:43.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.135+0000 7f59707e8e00 1 -- start start 2026-02-25T11:47:43.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.136+0000 7f59707e8e00 1 Processor -- start 2026-02-25T11:47:43.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.136+0000 7f59707e8e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x558029484830 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.136+0000 7f59707e8e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55802946cd90 0x558029484d70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.136+0000 7f59707e8e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5580294b1a30 0x558029489d60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.136+0000 7f59707e8e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55802948a2a0 con 0x55802946c480 2026-02-25T11:47:43.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.136+0000 7f59707e8e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55802948a410 con 0x5580294b1a30 2026-02-25T11:47:43.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.136+0000 7f59707e8e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55802948a5d0 con 0x55802946cd90 2026-02-25T11:47:43.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.136+0000 7f59677fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55802946cd90 0x558029484d70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.136+0000 7f596cd63640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5580294b1a30 0x558029489d60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.136+0000 7f59677fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55802946cd90 0x558029484d70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:46250/0 (socket says 10.20.193.28:46250) 2026-02-25T11:47:43.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.136+0000 7f59677fe640 1 -- 10.20.193.28:0/840893011 learned_addr learned my addr 10.20.193.28:0/840893011 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:43.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.137+0000 7f5967fff640 1 --2- 10.20.193.28:0/840893011 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x558029484830 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.137+0000 7f59677fe640 1 -- 10.20.193.28:0/840893011 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5580294b1a30 msgr2=0x558029489d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.137+0000 7f59677fe640 1 --2- 10.20.193.28:0/840893011 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5580294b1a30 0x558029489d60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.137+0000 7f59677fe640 1 -- 10.20.193.28:0/840893011 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 msgr2=0x558029484830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.137+0000 7f59677fe640 1 --2- 10.20.193.28:0/840893011 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x558029484830 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.137+0000 7f59677fe640 1 -- 10.20.193.28:0/840893011 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580292fc2c0 con 0x55802946cd90 2026-02-25T11:47:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.137+0000 7f5967fff640 1 --2- 10.20.193.28:0/840893011 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x558029484830 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.137+0000 7f59677fe640 1 --2- 10.20.193.28:0/840893011 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55802946cd90 0x558029484d70 secure :-1 s=READY pgs=104 gs=1 cs=0 l=1 c_cookie=9471008508967244723 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f59580076a0 tx=0x7f5958004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=836fc80f31b1d3b3 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.137+0000 7f5929ffb640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:47:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.137+0000 7f5929ffb640 1 -- 10.20.193.28:0/840893011 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5958011840 con 0x55802946cd90 2026-02-25T11:47:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.138+0000 7f59707e8e00 1 -- 10.20.193.28:0/840893011 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59680fbf90 con 0x55802946cd90 2026-02-25T11:47:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.138+0000 7f5929ffb640 1 -- 10.20.193.28:0/840893011 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5958011e60 con 0x55802946cd90 2026-02-25T11:47:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.138+0000 7f59707e8e00 10 client.25153 _ll_register_callbacks cb 0x558029366a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.138+0000 7f5929ffb640 1 -- 10.20.193.28:0/840893011 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5958010e50 con 0x55802946cd90 2026-02-25T11:47:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.138+0000 7f59707e8e00 1 -- 10.20.193.28:0/840893011 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5580290d03b0 con 0x55802946cd90 2026-02-25T11:47:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.138+0000 7f5929ffb640 1 -- 10.20.193.28:0/840893011 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f59580119a0 con 0x55802946cd90 2026-02-25T11:47:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.138+0000 7f5929ffb640 1 -- 10.20.193.28:0/840893011 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5958011b60 con 0x55802946cd90 2026-02-25T11:47:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.138+0000 7f59707e8e00 10 client.25153 fetch_fsmap learned FSMap version 22 2026-02-25T11:47:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.138+0000 7f59707e8e00 1 -- 10.20.193.28:0/840893011 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=22}) -- 0x55802930e0c0 con 0x55802946cd90 2026-02-25T11:47:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.138+0000 7f5929ffb640 1 -- 10.20.193.28:0/840893011 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f5958010370 con 0x55802946cd90 2026-02-25T11:47:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.138+0000 7f59707e8e00 10 client.25153 fetch_fsmap finished waiting for FSMap version 22 2026-02-25T11:47:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.139+0000 7f59707e8e00 10 client.25153 resolve_mds: resolved name 'i' to daemon mds.i 2026-02-25T11:47:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.139+0000 7f59707e8e00 20 client.25153 populate_metadata read hostname 'trial028' 2026-02-25T11:47:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.139+0000 7f59707e8e00 1 --2- 10.20.193.28:0/840893011 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5580294afc30 0x5580294b10a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.139+0000 7f59707e8e00 4 client.25153 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:47:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.139+0000 7f59707e8e00 1 -- 10.20.193.28:0/840893011 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55802936e140 con 0x5580294afc30 2026-02-25T11:47:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.139+0000 7f5967fff640 1 --2- 10.20.193.28:0/840893011 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5580294afc30 0x5580294b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.140+0000 7f5967fff640 1 --2- 10.20.193.28:0/840893011 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5580294afc30 0x5580294b10a0 crc :-1 s=READY pgs=19 gs=4 cs=0 l=0 c_cookie=13134165118823528745 s_cookie=10042978186325234789 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b645ed9678cd7929 server_cookie=8b5fd37352e99465 in_seq=0 out_seq=0 2026-02-25T11:47:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.140+0000 7f5929ffb640 10 client.25153 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:47:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.140+0000 7f5929ffb640 1 -- 10.20.193.28:0/840893011 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55802936e140 con 0x5580294afc30 2026-02-25T11:47:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.140+0000 7f5929ffb640 10 client.25153 handle_command_reply: tid=0 2026-02-25T11:47:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.140+0000 7f59707e8e00 1 client.25153 shutdown 2026-02-25T11:47:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.140+0000 7f59707e8e00 1 -- 10.20.193.28:0/840893011 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55802946cd90 msgr2=0x558029484d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.140+0000 7f59707e8e00 1 --2- 10.20.193.28:0/840893011 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55802946cd90 0x558029484d70 secure :-1 s=READY pgs=104 gs=1 cs=0 l=1 c_cookie=9471008508967244723 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f59580076a0 tx=0x7f5958004770 comp rx=0 tx=0).stop 2026-02-25T11:47:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.141+0000 7f59707e8e00 1 -- 10.20.193.28:0/840893011 shutdown_connections 2026-02-25T11:47:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.141+0000 7f59707e8e00 1 --2- 10.20.193.28:0/840893011 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5580294afc30 0x5580294b10a0 crc :-1 s=READY pgs=19 gs=4 cs=0 l=0 c_cookie=13134165118823528745 s_cookie=10042978186325234789 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.141+0000 7f59707e8e00 1 --2- 10.20.193.28:0/840893011 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5580294b1a30 0x558029489d60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.141+0000 7f59707e8e00 1 --2- 10.20.193.28:0/840893011 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55802946cd90 0x558029484d70 unknown :-1 s=CLOSED pgs=104 gs=1 cs=0 l=1 c_cookie=9471008508967244723 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.141+0000 7f59707e8e00 1 --2- 10.20.193.28:0/840893011 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x558029484830 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.141+0000 7f59707e8e00 1 -- 10.20.193.28:0/840893011 >> 10.20.193.28:0/840893011 conn(0x55802938c720 msgr2=0x55802946b470 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.141+0000 7f5929ffb640 0 client.25153 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:47:43.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.141+0000 7f59707e8e00 1 -- 10.20.193.28:0/840893011 shutdown_connections 2026-02-25T11:47:43.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.141+0000 7f59707e8e00 1 -- 10.20.193.28:0/840893011 wait complete. 2026-02-25T11:47:43.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.141+0000 7f59707e8e00 20 client.25153 trim_cache size 0 max 16384 2026-02-25T11:47:43.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.150+0000 7f59707e8e00 1 Processor -- start 2026-02-25T11:47:43.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.150+0000 7f59707e8e00 1 -- start start 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.150+0000 7f59707e8e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x55802946f170 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.150+0000 7f59707e8e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55802946cd90 0x558029481ae0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.150+0000 7f59707e8e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5580294b1a30 0x558029482020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.150+0000 7f59707e8e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558029376330 con 0x55802946c480 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.150+0000 7f59707e8e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558029036610 con 0x5580294b1a30 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.150+0000 7f59707e8e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55802937e760 con 0x55802946cd90 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f5967fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x55802946f170 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f5967fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x55802946f170 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35734/0 (socket says 10.20.193.28:35734) 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f5967fff640 1 -- 10.20.193.28:0/2044733862 learned_addr learned my addr 10.20.193.28:0/2044733862 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f596cd63640 1 --2- 10.20.193.28:0/2044733862 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5580294b1a30 0x558029482020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f59677fe640 1 --2- 10.20.193.28:0/2044733862 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55802946cd90 0x558029481ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f5967fff640 1 -- 10.20.193.28:0/2044733862 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55802946cd90 msgr2=0x558029481ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f5967fff640 1 --2- 10.20.193.28:0/2044733862 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55802946cd90 0x558029481ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f5967fff640 1 -- 10.20.193.28:0/2044733862 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5580294b1a30 msgr2=0x558029482020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f5967fff640 1 --2- 10.20.193.28:0/2044733862 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5580294b1a30 0x558029482020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f5967fff640 1 -- 10.20.193.28:0/2044733862 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558029308b40 con 0x55802946c480 2026-02-25T11:47:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f5967fff640 1 --2- 10.20.193.28:0/2044733862 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x55802946f170 secure :-1 s=READY pgs=326 gs=1 cs=0 l=1 c_cookie=15167912964730305361 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5954108ee0 tx=0x7f595402f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27f407a76cabb51 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f592bfff640 1 -- 10.20.193.28:0/2044733862 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f595402fe20 con 0x55802946c480 2026-02-25T11:47:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f592bfff640 1 -- 10.20.193.28:0/2044733862 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f59540d70d0 con 0x55802946c480 2026-02-25T11:47:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f592bfff640 1 -- 10.20.193.28:0/2044733862 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f59540d2360 con 0x55802946c480 2026-02-25T11:47:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f59707e8e00 1 -- 10.20.193.28:0/2044733862 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 msgr2=0x55802946f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.151+0000 7f59707e8e00 1 --2- 10.20.193.28:0/2044733862 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x55802946f170 secure :-1 s=READY pgs=326 gs=1 cs=0 l=1 c_cookie=15167912964730305361 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5954108ee0 tx=0x7f595402f190 comp rx=0 tx=0).stop 2026-02-25T11:47:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.152+0000 7f59707e8e00 1 -- 10.20.193.28:0/2044733862 shutdown_connections 2026-02-25T11:47:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.152+0000 7f59707e8e00 1 --2- 10.20.193.28:0/2044733862 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5580294b1a30 0x558029482020 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.152+0000 7f59707e8e00 1 --2- 10.20.193.28:0/2044733862 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55802946cd90 0x558029481ae0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.152+0000 7f59707e8e00 1 --2- 10.20.193.28:0/2044733862 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946c480 0x55802946f170 unknown :-1 s=CLOSED pgs=326 gs=1 cs=0 l=1 c_cookie=15167912964730305361 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.152+0000 7f59707e8e00 1 -- 10.20.193.28:0/2044733862 >> 10.20.193.28:0/2044733862 conn(0x55802938c720 msgr2=0x558029476420 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.152+0000 7f59707e8e00 1 -- 10.20.193.28:0/2044733862 shutdown_connections 2026-02-25T11:47:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.152+0000 7f59707e8e00 1 -- 10.20.193.28:0/2044733862 wait complete. 2026-02-25T11:47:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.152+0000 7f59707e8e00 1 -- start start 2026-02-25T11:47:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.152+0000 7f59707e8e00 1 Processor -- start 2026-02-25T11:47:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59707e8e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55802946c480 0x558029481d80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59707e8e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946cd90 0x55802948adf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59707e8e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5580294b1a30 0x55802948b330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59707e8e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558029482310 con 0x55802946cd90 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59707e8e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558029482480 con 0x55802946c480 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59707e8e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55802948b870 con 0x5580294b1a30 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59677fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946cd90 0x55802948adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59677fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946cd90 0x55802948adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35740/0 (socket says 10.20.193.28:35740) 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59677fe640 1 -- 10.20.193.28:0/86792804 learned_addr learned my addr 10.20.193.28:0/86792804 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f5967fff640 1 --2- 10.20.193.28:0/86792804 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55802946c480 0x558029481d80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f596cd63640 1 --2- 10.20.193.28:0/86792804 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5580294b1a30 0x55802948b330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59677fe640 1 -- 10.20.193.28:0/86792804 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5580294b1a30 msgr2=0x55802948b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59677fe640 1 --2- 10.20.193.28:0/86792804 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5580294b1a30 0x55802948b330 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59677fe640 1 -- 10.20.193.28:0/86792804 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55802946c480 msgr2=0x558029481d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59677fe640 1 --2- 10.20.193.28:0/86792804 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55802946c480 0x558029481d80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.153+0000 7f59677fe640 1 -- 10.20.193.28:0/86792804 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580292fc5f0 con 0x55802946cd90 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.154+0000 7f59677fe640 1 --2- 10.20.193.28:0/86792804 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946cd90 0x55802948adf0 secure :-1 s=READY pgs=327 gs=1 cs=0 l=1 c_cookie=12892272016413087143 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f595801f480 tx=0x7f595800df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ea8d17be02d5a7 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.154+0000 7f5929ffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.154+0000 7f5929ffb640 1 -- 10.20.193.28:0/86792804 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5958007e90 con 0x55802946cd90 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.154+0000 7f5929ffb640 1 -- 10.20.193.28:0/86792804 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5580290d03b0 con 0x55802946cd90 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.154+0000 7f5929ffb640 1 -- 10.20.193.28:0/86792804 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f595801fd20 con 0x55802946cd90 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.154+0000 7f59707e8e00 1 -- 10.20.193.28:0/86792804 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558029323cf0 con 0x55802946cd90 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.154+0000 7f5929ffb640 1 -- 10.20.193.28:0/86792804 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f5958011620 con 0x55802946cd90 2026-02-25T11:47:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.154+0000 7f59707e8e00 10 client.15300 _ll_register_callbacks cb 0x558029366a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.154+0000 7f59707e8e00 1 -- 10.20.193.28:0/86792804 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59540d73f0 con 0x55802946cd90 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.154+0000 7f5929ffb640 1 -- 10.20.193.28:0/86792804 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5958029cf0 con 0x55802946cd90 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.154+0000 7f59707e8e00 10 client.15300 fetch_fsmap learned FSMap version 22 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.154+0000 7f59707e8e00 1 -- 10.20.193.28:0/86792804 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=22}) -- 0x55802930e320 con 0x55802946cd90 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.154+0000 7f5929ffb640 1 -- 10.20.193.28:0/86792804 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f59580293b0 con 0x55802946cd90 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.155+0000 7f59707e8e00 10 client.15300 fetch_fsmap finished waiting for FSMap version 22 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.155+0000 7f59707e8e00 10 client.15300 resolve_mds: resolved name 'i' to daemon mds.i 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.155+0000 7f59707e8e00 20 client.15300 populate_metadata read hostname 'trial028' 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.155+0000 7f59707e8e00 1 --2- 10.20.193.28:0/86792804 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5580294df0e0 0x5580294dddf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.155+0000 7f59707e8e00 4 client.15300 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "status"}] 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.155+0000 7f59707e8e00 1 -- 10.20.193.28:0/86792804 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "status"}) -- 0x5580290b5070 con 0x5580294df0e0 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.155+0000 7f5967fff640 1 --2- 10.20.193.28:0/86792804 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5580294df0e0 0x5580294dddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.155+0000 7f5967fff640 1 --2- 10.20.193.28:0/86792804 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5580294df0e0 0x5580294dddf0 crc :-1 s=READY pgs=20 gs=4 cs=0 l=0 c_cookie=9387403769391291635 s_cookie=17991618444296413574 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8246c1fdddbb1cf3 server_cookie=f9af11a6c2a76186 in_seq=0 out_seq=0 2026-02-25T11:47:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.155+0000 7f5929ffb640 10 client.15300 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f5929ffb640 1 -- 10.20.193.28:0/86792804 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1210 (crc 0 0 0) 0x5580290b5070 con 0x5580294df0e0 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f5929ffb640 10 client.15300 handle_command_reply: tid=0 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f59707e8e00 1 client.15300 shutdown 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f59707e8e00 1 -- 10.20.193.28:0/86792804 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946cd90 msgr2=0x55802948adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f59707e8e00 1 --2- 10.20.193.28:0/86792804 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946cd90 0x55802948adf0 secure :-1 s=READY pgs=327 gs=1 cs=0 l=1 c_cookie=12892272016413087143 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f595801f480 tx=0x7f595800df40 comp rx=0 tx=0).stop 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f59707e8e00 1 -- 10.20.193.28:0/86792804 shutdown_connections 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f59707e8e00 1 --2- 10.20.193.28:0/86792804 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5580294df0e0 0x5580294dddf0 crc :-1 s=READY pgs=20 gs=4 cs=0 l=0 c_cookie=9387403769391291635 s_cookie=17991618444296413574 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f59707e8e00 1 --2- 10.20.193.28:0/86792804 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5580294b1a30 0x55802948b330 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f59707e8e00 1 --2- 10.20.193.28:0/86792804 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55802946cd90 0x55802948adf0 unknown :-1 s=CLOSED pgs=327 gs=1 cs=0 l=1 c_cookie=12892272016413087143 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f59707e8e00 1 --2- 10.20.193.28:0/86792804 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55802946c480 0x558029481d80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f59707e8e00 1 -- 10.20.193.28:0/86792804 >> 10.20.193.28:0/86792804 conn(0x55802938c720 msgr2=0x55802946f170 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f5929ffb640 0 client.15300 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f59707e8e00 1 -- 10.20.193.28:0/86792804 shutdown_connections 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f59707e8e00 1 -- 10.20.193.28:0/86792804 wait complete. 2026-02-25T11:47:43.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.156+0000 7f59707e8e00 20 client.15300 trim_cache size 0 max 16384 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "cluster_fsid": "123fa7bf-123f-11f1-80da-d404e6e7d460", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "whoami": 2, 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "id": 24730, 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "want_state": "up:active", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "state": "up:active", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "fs_name": "cephfs", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "rank_uptime": 25.507191574, 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "mdsmap_epoch": 22, 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_epoch": 86, 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_epoch_barrier": 86, 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "uptime": 39.84758068, 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "sysinfo": { 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "arch": "x86_64", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_release": "tentacle", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version": "ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev - RelWithDebInfo)", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version_short": "20.3.0-5549-g92715cd3", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "container_hostname": "trial167", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "container_image": "quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "distro": "centos", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "distro_description": "CentOS Stream 9", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "distro_version": "9", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "hostname": "trial167", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Tue Feb 24 21:03:19 EST 2026", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "kernel_version": "6.19.0-g712f4a43c125", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "mem_swap_kb": "8388604", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "mem_total_kb": "130448332", 2026-02-25T11:47:43.159 INFO:teuthology.orchestra.run.trial028.stdout: "os": "Linux" 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stdout: "endian": "little" 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.157+0000 7f596e7ed640 1 -- 10.20.193.28:0/601669517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f593c0b57f0 msgr2=0x7f593c0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.157+0000 7f596e7ed640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f593c0b57f0 0x7f593c0d5ca0 secure :-1 s=READY pgs=238 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f59680fc5b0 tx=0x7f5958008040 comp rx=0 tx=0).stop 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.157+0000 7f596e7ed640 1 -- 10.20.193.28:0/601669517 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f596810f840 msgr2=0x7f59680f5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.157+0000 7f596e7ed640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f596810f840 0x7f59680f5c10 secure :-1 s=READY pgs=324 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5954004820 tx=0x7f59540028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.157+0000 7f596e7ed640 1 -- 10.20.193.28:0/601669517 shutdown_connections 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.157+0000 7f596e7ed640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f593c0b57f0 0x7f593c0d5ca0 unknown :-1 s=CLOSED pgs=238 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.157+0000 7f596e7ed640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5968115a30 0x7f59680f6310 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.157+0000 7f596e7ed640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5968110230 0x7f5968176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.157+0000 7f596e7ed640 1 --2- 10.20.193.28:0/601669517 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f596810f840 0x7f59680f5c10 unknown :-1 s=CLOSED pgs=324 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.157+0000 7f596e7ed640 1 -- 10.20.193.28:0/601669517 >> 10.20.193.28:0/601669517 conn(0x7f59680ed130 msgr2=0x7f596810dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.157+0000 7f596e7ed640 1 -- 10.20.193.28:0/601669517 shutdown_connections 2026-02-25T11:47:43.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.157+0000 7f596e7ed640 1 -- 10.20.193.28:0/601669517 wait complete. 2026-02-25T11:47:43.257 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:43 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2097807841' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:43.260 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.g status 2026-02-25T11:47:43.445 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:43.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:43 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2097807841' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:43.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:43 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2097807841' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:43.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.624+0000 7f4fc7fff640 1 -- 10.20.193.28:0/3123050219 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fc8110080 msgr2=0x7f4fc81150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.624+0000 7f4fc7fff640 1 --2- 10.20.193.28:0/3123050219 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fc8110080 0x7f4fc81150d0 secure :-1 s=READY pgs=328 gs=1 cs=0 l=1 c_cookie=2945397496898638632 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fb0009950 tx=0x7f4fb002f190 comp rx=0 tx=0).stop 2026-02-25T11:47:43.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.624+0000 7f4fc7fff640 1 -- 10.20.193.28:0/3123050219 shutdown_connections 2026-02-25T11:47:43.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.624+0000 7f4fc7fff640 1 --2- 10.20.193.28:0/3123050219 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fc8115880 0x7f4fc8176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.624+0000 7f4fc7fff640 1 --2- 10.20.193.28:0/3123050219 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fc8110080 0x7f4fc81150d0 unknown :-1 s=CLOSED pgs=328 gs=1 cs=0 l=1 c_cookie=2945397496898638632 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.624+0000 7f4fc7fff640 1 --2- 10.20.193.28:0/3123050219 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4fc810f690 0x7f4fc810fa60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.624+0000 7f4fc7fff640 1 -- 10.20.193.28:0/3123050219 >> 10.20.193.28:0/3123050219 conn(0x7f4fc80ecf00 msgr2=0x7f4fc810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:43.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.624+0000 7f4fc7fff640 1 -- 10.20.193.28:0/3123050219 shutdown_connections 2026-02-25T11:47:43.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.625+0000 7f4fc7fff640 1 -- 10.20.193.28:0/3123050219 wait complete. 2026-02-25T11:47:43.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.625+0000 7f4fc7fff640 1 Processor -- start 2026-02-25T11:47:43.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.625+0000 7f4fc7fff640 1 -- start start 2026-02-25T11:47:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.625+0000 7f4fc7fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fc810f690 0x7f4fc811e0b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.625+0000 7f4fc7fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4fc8110080 0x7f4fc811e5f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc7fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fc8115880 0x7f4fc81239c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc7fff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8123fe0 con 0x7f4fc8115880 2026-02-25T11:47:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc7fff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8124150 con 0x7f4fc8110080 2026-02-25T11:47:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc7fff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc8124310 con 0x7f4fc810f690 2026-02-25T11:47:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc6ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fc810f690 0x7f4fc811e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc77fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fc8115880 0x7f4fc81239c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc6ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fc810f690 0x7f4fc811e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:46288/0 (socket says 10.20.193.28:46288) 2026-02-25T11:47:43.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/4110115142 learned_addr learned my addr 10.20.193.28:0/4110115142 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:43.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc67fc640 1 --2- 10.20.193.28:0/4110115142 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4fc8110080 0x7f4fc811e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/4110115142 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4fc8110080 msgr2=0x7f4fc811e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/4110115142 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4fc8110080 0x7f4fc811e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/4110115142 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fc8115880 msgr2=0x7f4fc81239c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/4110115142 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fc8115880 0x7f4fc81239c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/4110115142 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb0009570 con 0x7f4fc810f690 2026-02-25T11:47:43.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/4110115142 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fc810f690 0x7f4fc811e0b0 secure :-1 s=READY pgs=105 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fbc009480 tx=0x7f4fbc00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:43.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.626+0000 7f4fcc83b640 1 -- 10.20.193.28:0/4110115142 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4fbc00cd40 con 0x7f4fc810f690 2026-02-25T11:47:43.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.627+0000 7f4fc7fff640 1 -- 10.20.193.28:0/4110115142 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc81245f0 con 0x7f4fc810f690 2026-02-25T11:47:43.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.627+0000 7f4fcc83b640 1 -- 10.20.193.28:0/4110115142 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4fbc004d10 con 0x7f4fc810f690 2026-02-25T11:47:43.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.627+0000 7f4fc7fff640 1 -- 10.20.193.28:0/4110115142 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc8164cf0 con 0x7f4fc810f690 2026-02-25T11:47:43.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.627+0000 7f4fcc83b640 1 -- 10.20.193.28:0/4110115142 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4fbc0106b0 con 0x7f4fc810f690 2026-02-25T11:47:43.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.629+0000 7f4fcc83b640 1 -- 10.20.193.28:0/4110115142 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f4fbc01f060 con 0x7f4fc810f690 2026-02-25T11:47:43.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.631+0000 7f4fcc83b640 1 --2- 10.20.193.28:0/4110115142 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f9c0b57f0 0x7f4f9c0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.631+0000 7f4fcc83b640 1 -- 10.20.193.28:0/4110115142 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f4fbc014070 con 0x7f4fc810f690 2026-02-25T11:47:43.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.631+0000 7f4fc67fc640 1 --2- 10.20.193.28:0/4110115142 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f9c0b57f0 0x7f4f9c0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.632+0000 7f4fc67fc640 1 --2- 10.20.193.28:0/4110115142 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f9c0b57f0 0x7f4f9c0d5ca0 secure :-1 s=READY pgs=239 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fc8124a50 tx=0x7f4fb003a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:43.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.632+0000 7f4fcf838e00 1 Processor -- start 2026-02-25T11:47:43.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.632+0000 7f4fcf838e00 1 -- start start 2026-02-25T11:47:43.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fcf838e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786480 0x557985786850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fcf838e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786d90 0x5579857cb4f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fcf838e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 0x5579857ebe10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fcf838e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55798569ca30 con 0x557985786d90 2026-02-25T11:47:43.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fcf838e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x557985698760 con 0x557985786480 2026-02-25T11:47:43.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fcf838e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x557985692700 con 0x5579857cba30 2026-02-25T11:47:43.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fc6ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786480 0x557985786850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fc67fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786d90 0x5579857cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fc6ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786480 0x557985786850 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43530/0 (socket says 10.20.193.28:43530) 2026-02-25T11:47:43.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/3700142251 learned_addr learned my addr 10.20.193.28:0/3700142251 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fc77fe640 1 --2- 10.20.193.28:0/3700142251 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 0x5579857ebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/3700142251 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 msgr2=0x5579857ebe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/3700142251 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 0x5579857ebe10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/3700142251 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786d90 msgr2=0x5579857cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/3700142251 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786d90 0x5579857cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/3700142251 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798563dcf0 con 0x557985786480 2026-02-25T11:47:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fc77fe640 1 --2- 10.20.193.28:0/3700142251 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 0x5579857ebe10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:43.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fc67fc640 1 --2- 10.20.193.28:0/3700142251 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786d90 0x5579857cb4f0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:43.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/3700142251 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786480 0x557985786850 secure :-1 s=READY pgs=102 gs=2 cs=0 l=1 c_cookie=14697251105848699795 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fbc0e4910 tx=0x7f4fbc015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbf72002ead42b93 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:43.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.633+0000 7f4f8bfff640 1 -- 10.20.193.28:0/3700142251 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4fbc0ad960 con 0x557985786480 2026-02-25T11:47:43.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.634+0000 7f4f8bfff640 1 -- 10.20.193.28:0/3700142251 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4fbc0d0e90 con 0x557985786480 2026-02-25T11:47:43.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.634+0000 7f4f8bfff640 1 -- 10.20.193.28:0/3700142251 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4fbc015530 con 0x557985786480 2026-02-25T11:47:43.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.634+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3700142251 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786480 msgr2=0x557985786850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.634+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/3700142251 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786480 0x557985786850 secure :-1 s=READY pgs=102 gs=2 cs=0 l=1 c_cookie=14697251105848699795 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fbc0e4910 tx=0x7f4fbc015000 comp rx=0 tx=0).stop 2026-02-25T11:47:43.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.634+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3700142251 shutdown_connections 2026-02-25T11:47:43.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.634+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/3700142251 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 0x5579857ebe10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.634+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/3700142251 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786d90 0x5579857cb4f0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.634+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/3700142251 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786480 0x557985786850 unknown :-1 s=CLOSED pgs=102 gs=2 cs=0 l=1 c_cookie=14697251105848699795 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.634+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3700142251 >> 10.20.193.28:0/3700142251 conn(0x5579856a6720 msgr2=0x557985784d60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:43.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.634+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3700142251 shutdown_connections 2026-02-25T11:47:43.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.634+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3700142251 wait complete. 2026-02-25T11:47:43.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.635+0000 7f4fcf838e00 1 -- start start 2026-02-25T11:47:43.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fcf838e00 1 Processor -- start 2026-02-25T11:47:43.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fcf838e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x557985787620 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fcf838e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786d90 0x557985787b60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fcf838e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 0x55798578c530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fcf838e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55798578ca70 con 0x557985786480 2026-02-25T11:47:43.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fcf838e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55798578cbe0 con 0x557985786d90 2026-02-25T11:47:43.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fcf838e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55798578cda0 con 0x5579857cba30 2026-02-25T11:47:43.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fc6ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x557985787620 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fc6ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x557985787620 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35760/0 (socket says 10.20.193.28:35760) 2026-02-25T11:47:43.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/3151420536 learned_addr learned my addr 10.20.193.28:0/3151420536 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fc77fe640 1 --2- 10.20.193.28:0/3151420536 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 0x55798578c530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fc67fc640 1 --2- 10.20.193.28:0/3151420536 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786d90 0x557985787b60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/3151420536 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 msgr2=0x55798578c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/3151420536 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 0x55798578c530 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/3151420536 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786d90 msgr2=0x557985787b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.636+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/3151420536 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786d90 0x557985787b60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.637+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/3151420536 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc0e4e70 con 0x557985786480 2026-02-25T11:47:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.637+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/3151420536 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x557985787620 secure :-1 s=READY pgs=329 gs=1 cs=0 l=1 c_cookie=15506123796796705086 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fbc0e6f00 tx=0x7f4fbc009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d730d1702f4c893e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.637+0000 7f4f897fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:47:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.637+0000 7f4f897fa640 1 -- 10.20.193.28:0/3151420536 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4fbc0c8020 con 0x557985786480 2026-02-25T11:47:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.637+0000 7f4f897fa640 1 -- 10.20.193.28:0/3151420536 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4fbc009260 con 0x557985786480 2026-02-25T11:47:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.637+0000 7f4f897fa640 1 -- 10.20.193.28:0/3151420536 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4fbc0050b0 con 0x557985786480 2026-02-25T11:47:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.637+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3151420536 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc8176ac0 con 0x557985786480 2026-02-25T11:47:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.637+0000 7f4f897fa640 1 -- 10.20.193.28:0/3151420536 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f4fbc0ad270 con 0x557985786480 2026-02-25T11:47:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.637+0000 7f4f897fa640 10 client.15312 inactive, discarding osd_map(86..86 src has 1..86) 2026-02-25T11:47:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.637+0000 7f4fcf838e00 10 client.15312 _ll_register_callbacks cb 0x557985680a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.637+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3151420536 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5579853ea3b0 con 0x557985786480 2026-02-25T11:47:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.638+0000 7f4f897fa640 1 -- 10.20.193.28:0/3151420536 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4fbc0ad430 con 0x557985786480 2026-02-25T11:47:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.638+0000 7f4fcf838e00 10 client.15312 fetch_fsmap learned FSMap version 22 2026-02-25T11:47:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.638+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3151420536 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=22}) -- 0x5579856162c0 con 0x557985786480 2026-02-25T11:47:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.638+0000 7f4f897fa640 1 -- 10.20.193.28:0/3151420536 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f4fbc08c070 con 0x557985786480 2026-02-25T11:47:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.638+0000 7f4fcf838e00 10 client.15312 fetch_fsmap finished waiting for FSMap version 22 2026-02-25T11:47:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.638+0000 7f4fcf838e00 10 client.15312 resolve_mds: resolved name 'g' to daemon mds.g 2026-02-25T11:47:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.638+0000 7f4fcf838e00 20 client.15312 populate_metadata read hostname 'trial028' 2026-02-25T11:47:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.638+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/3151420536 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5579857b1f60 0x5579857b33d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.638+0000 7f4fcf838e00 4 client.15312 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:47:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.638+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3151420536 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557985688140 con 0x5579857b1f60 2026-02-25T11:47:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.638+0000 7f4fc67fc640 1 --2- 10.20.193.28:0/3151420536 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5579857b1f60 0x5579857b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.639+0000 7f4fc67fc640 1 --2- 10.20.193.28:0/3151420536 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5579857b1f60 0x5579857b33d0 crc :-1 s=READY pgs=19 gs=4 cs=0 l=0 c_cookie=3068629110993520105 s_cookie=525062082660291601 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a95f5eb0c3105e9 server_cookie=749652ea9fe5411 in_seq=0 out_seq=0 2026-02-25T11:47:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.639+0000 7f4f897fa640 10 client.15312 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:47:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.639+0000 7f4f897fa640 1 -- 10.20.193.28:0/3151420536 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x557985688140 con 0x5579857b1f60 2026-02-25T11:47:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.639+0000 7f4f897fa640 10 client.15312 handle_command_reply: tid=0 2026-02-25T11:47:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.639+0000 7f4fcf838e00 1 client.15312 shutdown 2026-02-25T11:47:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.640+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3151420536 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 msgr2=0x557985787620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.640+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/3151420536 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x557985787620 secure :-1 s=READY pgs=329 gs=1 cs=0 l=1 c_cookie=15506123796796705086 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fbc0e6f00 tx=0x7f4fbc009f70 comp rx=0 tx=0).stop 2026-02-25T11:47:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.640+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3151420536 shutdown_connections 2026-02-25T11:47:43.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.640+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/3151420536 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5579857b1f60 0x5579857b33d0 crc :-1 s=READY pgs=19 gs=4 cs=0 l=0 c_cookie=3068629110993520105 s_cookie=525062082660291601 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.640+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/3151420536 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 0x55798578c530 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.640+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/3151420536 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786d90 0x557985787b60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.640+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/3151420536 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x557985787620 unknown :-1 s=CLOSED pgs=329 gs=1 cs=0 l=1 c_cookie=15506123796796705086 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.640+0000 7f4f897fa640 0 client.15312 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:47:43.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.640+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3151420536 >> 10.20.193.28:0/3151420536 conn(0x5579856a6720 msgr2=0x557985784570 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:43.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.640+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3151420536 shutdown_connections 2026-02-25T11:47:43.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.640+0000 7f4fcf838e00 1 -- 10.20.193.28:0/3151420536 wait complete. 2026-02-25T11:47:43.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.640+0000 7f4fcf838e00 20 client.15312 trim_cache size 0 max 16384 2026-02-25T11:47:43.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.650+0000 7f4fcf838e00 1 Processor -- start 2026-02-25T11:47:43.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.650+0000 7f4fcf838e00 1 -- start start 2026-02-25T11:47:43.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.650+0000 7f4fcf838e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x5579857d9290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.650+0000 7f4fcf838e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786d90 0x5579857e9170 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.650+0000 7f4fcf838e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 0x5579857e96b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.650+0000 7f4fcf838e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x557985690330 con 0x557985786480 2026-02-25T11:47:43.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.650+0000 7f4fcf838e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x557985350610 con 0x557985786d90 2026-02-25T11:47:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.650+0000 7f4fcf838e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5579857b3eb0 con 0x5579857cba30 2026-02-25T11:47:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4fc67fc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786d90 0x5579857e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4fc67fc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786d90 0x5579857e9170 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43554/0 (socket says 10.20.193.28:43554) 2026-02-25T11:47:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4fc67fc640 1 -- 10.20.193.28:0/1204792460 learned_addr learned my addr 10.20.193.28:0/1204792460 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/1204792460 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x5579857d9290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4fc67fc640 1 -- 10.20.193.28:0/1204792460 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 msgr2=0x5579857e96b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2026-02-25T11:47:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4fc67fc640 1 --2- 10.20.193.28:0/1204792460 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 0x5579857e96b0 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4fc67fc640 1 -- 10.20.193.28:0/1204792460 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 msgr2=0x5579857d9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4fc67fc640 1 --2- 10.20.193.28:0/1204792460 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x5579857d9290 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4fc67fc640 1 -- 10.20.193.28:0/1204792460 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579856280c0 con 0x557985786d90 2026-02-25T11:47:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/1204792460 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x5579857d9290 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4fc67fc640 1 --2- 10.20.193.28:0/1204792460 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786d90 0x5579857e9170 secure :-1 s=READY pgs=103 gs=2 cs=0 l=1 c_cookie=15379280314574690347 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fb002f6a0 tx=0x7f4fb0040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d56e2df69361d82b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4f8b7fe640 1 -- 10.20.193.28:0/1204792460 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4fb0058070 con 0x557985786d90 2026-02-25T11:47:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4f8b7fe640 1 -- 10.20.193.28:0/1204792460 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4fb003e040 con 0x557985786d90 2026-02-25T11:47:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.651+0000 7f4f8b7fe640 1 -- 10.20.193.28:0/1204792460 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4fb0038bd0 con 0x557985786d90 2026-02-25T11:47:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.652+0000 7f4fcf838e00 1 -- 10.20.193.28:0/1204792460 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786d90 msgr2=0x5579857e9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.652+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/1204792460 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786d90 0x5579857e9170 secure :-1 s=READY pgs=103 gs=2 cs=0 l=1 c_cookie=15379280314574690347 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fb002f6a0 tx=0x7f4fb0040000 comp rx=0 tx=0).stop 2026-02-25T11:47:43.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.652+0000 7f4fcf838e00 1 -- 10.20.193.28:0/1204792460 shutdown_connections 2026-02-25T11:47:43.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.652+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/1204792460 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5579857cba30 0x5579857e96b0 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.652+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/1204792460 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557985786d90 0x5579857e9170 unknown :-1 s=CLOSED pgs=103 gs=2 cs=0 l=1 c_cookie=15379280314574690347 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.652+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/1204792460 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x5579857d9290 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.652+0000 7f4fcf838e00 1 -- 10.20.193.28:0/1204792460 >> 10.20.193.28:0/1204792460 conn(0x5579856a6720 msgr2=0x5579857ddb40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:43.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.652+0000 7f4fcf838e00 1 -- 10.20.193.28:0/1204792460 shutdown_connections 2026-02-25T11:47:43.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.652+0000 7f4fcf838e00 1 -- 10.20.193.28:0/1204792460 wait complete. 2026-02-25T11:47:43.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.652+0000 7f4fcf838e00 1 -- start start 2026-02-25T11:47:43.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.652+0000 7f4fcf838e00 1 Processor -- start 2026-02-25T11:47:43.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fcf838e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x5579857cbef0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fcf838e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557985786d90 0x5579857cc470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fcf838e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5579857cba30 0x5579857c0cd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fcf838e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5579857617d0 con 0x557985786480 2026-02-25T11:47:43.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fcf838e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5579857adcd0 con 0x5579857cba30 2026-02-25T11:47:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fcf838e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5579857ec210 con 0x557985786d90 2026-02-25T11:47:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fc77fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5579857cba30 0x5579857c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fc67fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557985786d90 0x5579857cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fc77fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5579857cba30 0x5579857c0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43570/0 (socket says 10.20.193.28:43570) 2026-02-25T11:47:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fc77fe640 1 -- 10.20.193.28:0/2396589144 learned_addr learned my addr 10.20.193.28:0/2396589144 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/2396589144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x5579857cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/2396589144 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557985786d90 msgr2=0x5579857cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/2396589144 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557985786d90 0x5579857cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/2396589144 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5579857cba30 msgr2=0x5579857c0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/2396589144 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5579857cba30 0x5579857c0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.653+0000 7f4fc6ffd640 1 -- 10.20.193.28:0/2396589144 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557985622b40 con 0x557985786480 2026-02-25T11:47:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4fc6ffd640 1 --2- 10.20.193.28:0/2396589144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x5579857cbef0 secure :-1 s=READY pgs=330 gs=1 cs=0 l=1 c_cookie=16312291494026443064 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fbc0ae3b0 tx=0x7f4fbc004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e260e6b006127938 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4f897fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:47:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4f897fa640 1 -- 10.20.193.28:0/2396589144 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4fbc0c8040 con 0x557985786480 2026-02-25T11:47:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4fcf838e00 1 -- 10.20.193.28:0/2396589144 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb0004150 con 0x557985786480 2026-02-25T11:47:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4fcf838e00 10 client.15315 _ll_register_callbacks cb 0x557985680a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4fcf838e00 1 -- 10.20.193.28:0/2396589144 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fbc0d0e90 con 0x557985786480 2026-02-25T11:47:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4f897fa640 1 -- 10.20.193.28:0/2396589144 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5579853ea3b0 con 0x557985786480 2026-02-25T11:47:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4f897fa640 1 -- 10.20.193.28:0/2396589144 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4fbc0ae9d0 con 0x557985786480 2026-02-25T11:47:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4f897fa640 1 -- 10.20.193.28:0/2396589144 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f4fbc0b6280 con 0x557985786480 2026-02-25T11:47:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4f897fa640 1 -- 10.20.193.28:0/2396589144 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f4fbc0050b0 con 0x557985786480 2026-02-25T11:47:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4fcf838e00 10 client.15315 fetch_fsmap learned FSMap version 22 2026-02-25T11:47:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4fcf838e00 1 -- 10.20.193.28:0/2396589144 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=22}) -- 0x5579856165f0 con 0x557985786480 2026-02-25T11:47:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4f897fa640 1 -- 10.20.193.28:0/2396589144 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f4fbc004850 con 0x557985786480 2026-02-25T11:47:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4fcf838e00 10 client.15315 fetch_fsmap finished waiting for FSMap version 22 2026-02-25T11:47:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4fcf838e00 10 client.15315 resolve_mds: resolved name 'g' to daemon mds.g 2026-02-25T11:47:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4fcf838e00 20 client.15315 populate_metadata read hostname 'trial028' 2026-02-25T11:47:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/2396589144 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x557985886270 0x557985884f80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4fcf838e00 4 client.15315 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "status"}] 2026-02-25T11:47:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.654+0000 7f4fcf838e00 1 -- 10.20.193.28:0/2396589144 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "status"}) -- 0x5579853cf070 con 0x557985886270 2026-02-25T11:47:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.655+0000 7f4fc67fc640 1 --2- 10.20.193.28:0/2396589144 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x557985886270 0x557985884f80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.655+0000 7f4f897fa640 10 client.15315 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:47:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.655+0000 7f4fc67fc640 1 --2- 10.20.193.28:0/2396589144 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x557985886270 0x557985884f80 crc :-1 s=READY pgs=20 gs=4 cs=0 l=0 c_cookie=13931703351772438216 s_cookie=3984798906839160343 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1575a629683dec8 server_cookie=374cd97e4f835617 in_seq=0 out_seq=0 2026-02-25T11:47:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.655+0000 7f4f897fa640 1 -- 10.20.193.28:0/2396589144 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1223 (crc 0 0 0) 0x5579853cf070 con 0x557985886270 2026-02-25T11:47:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.655+0000 7f4f897fa640 10 client.15315 handle_command_reply: tid=0 2026-02-25T11:47:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.655+0000 7f4fcf838e00 1 client.15315 shutdown 2026-02-25T11:47:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fcf838e00 1 -- 10.20.193.28:0/2396589144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 msgr2=0x5579857cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/2396589144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x5579857cbef0 secure :-1 s=READY pgs=330 gs=1 cs=0 l=1 c_cookie=16312291494026443064 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fbc0ae3b0 tx=0x7f4fbc004770 comp rx=0 tx=0).stop 2026-02-25T11:47:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fcf838e00 1 -- 10.20.193.28:0/2396589144 shutdown_connections 2026-02-25T11:47:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/2396589144 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x557985886270 0x557985884f80 crc :-1 s=READY pgs=20 gs=4 cs=0 l=0 c_cookie=13931703351772438216 s_cookie=3984798906839160343 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/2396589144 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5579857cba30 0x5579857c0cd0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/2396589144 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557985786d90 0x5579857cc470 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fcf838e00 1 --2- 10.20.193.28:0/2396589144 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557985786480 0x5579857cbef0 unknown :-1 s=CLOSED pgs=330 gs=1 cs=0 l=1 c_cookie=16312291494026443064 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4f897fa640 0 client.15315 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:47:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fcf838e00 1 -- 10.20.193.28:0/2396589144 >> 10.20.193.28:0/2396589144 conn(0x5579856a6720 msgr2=0x557985787430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fcf838e00 1 -- 10.20.193.28:0/2396589144 shutdown_connections 2026-02-25T11:47:43.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fcf838e00 1 -- 10.20.193.28:0/2396589144 wait complete. 2026-02-25T11:47:43.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fcf838e00 20 client.15315 trim_cache size 0 max 16384 2026-02-25T11:47:43.665 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:47:43.665 INFO:teuthology.orchestra.run.trial028.stdout: "cluster_fsid": "123fa7bf-123f-11f1-80da-d404e6e7d460", 2026-02-25T11:47:43.665 INFO:teuthology.orchestra.run.trial028.stdout: "whoami": 1, 2026-02-25T11:47:43.665 INFO:teuthology.orchestra.run.trial028.stdout: "id": 24731, 2026-02-25T11:47:43.665 INFO:teuthology.orchestra.run.trial028.stdout: "want_state": "up:active", 2026-02-25T11:47:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "state": "up:active", 2026-02-25T11:47:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "fs_name": "cephfs", 2026-02-25T11:47:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "rank_uptime": 27.011899928999998, 2026-02-25T11:47:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "mdsmap_epoch": 22, 2026-02-25T11:47:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_epoch": 86, 2026-02-25T11:47:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_epoch_barrier": 86, 2026-02-25T11:47:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "uptime": 39.071793790000001, 2026-02-25T11:47:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "sysinfo": { 2026-02-25T11:47:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "arch": "x86_64", 2026-02-25T11:47:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_release": "tentacle", 2026-02-25T11:47:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version": "ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev - RelWithDebInfo)", 2026-02-25T11:47:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version_short": "20.3.0-5549-g92715cd3", 2026-02-25T11:47:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "container_hostname": "trial028", 2026-02-25T11:47:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "container_image": "quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33", 2026-02-25T11:47:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-25T11:47:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "distro": "centos", 2026-02-25T11:47:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "distro_description": "CentOS Stream 9", 2026-02-25T11:47:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "distro_version": "9", 2026-02-25T11:47:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "hostname": "trial028", 2026-02-25T11:47:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Tue Feb 24 21:03:19 EST 2026", 2026-02-25T11:47:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "kernel_version": "6.19.0-g712f4a43c125", 2026-02-25T11:47:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "mem_swap_kb": "8388604", 2026-02-25T11:47:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "mem_total_kb": "130448336", 2026-02-25T11:47:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "os": "Linux" 2026-02-25T11:47:43.667 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:47:43.668 INFO:teuthology.orchestra.run.trial028.stdout: "endian": "little" 2026-02-25T11:47:43.668 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:47:43.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fc7fff640 1 -- 10.20.193.28:0/4110115142 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f9c0b57f0 msgr2=0x7f4f9c0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fc7fff640 1 --2- 10.20.193.28:0/4110115142 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f9c0b57f0 0x7f4f9c0d5ca0 secure :-1 s=READY pgs=239 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fc8124a50 tx=0x7f4fb003a040 comp rx=0 tx=0).stop 2026-02-25T11:47:43.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fc7fff640 1 -- 10.20.193.28:0/4110115142 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fc810f690 msgr2=0x7f4fc811e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:43.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.656+0000 7f4fc7fff640 1 --2- 10.20.193.28:0/4110115142 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fc810f690 0x7f4fc811e0b0 secure :-1 s=READY pgs=105 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fbc009480 tx=0x7f4fbc00ee60 comp rx=0 tx=0).stop 2026-02-25T11:47:43.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.657+0000 7f4fc7fff640 1 -- 10.20.193.28:0/4110115142 shutdown_connections 2026-02-25T11:47:43.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.657+0000 7f4fc7fff640 1 --2- 10.20.193.28:0/4110115142 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f9c0b57f0 0x7f4f9c0d5ca0 unknown :-1 s=CLOSED pgs=239 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.657+0000 7f4fc7fff640 1 --2- 10.20.193.28:0/4110115142 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fc8115880 0x7f4fc81239c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.657+0000 7f4fc7fff640 1 --2- 10.20.193.28:0/4110115142 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4fc8110080 0x7f4fc811e5f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.657+0000 7f4fc7fff640 1 --2- 10.20.193.28:0/4110115142 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fc810f690 0x7f4fc811e0b0 unknown :-1 s=CLOSED pgs=105 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:43.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.657+0000 7f4fc7fff640 1 -- 10.20.193.28:0/4110115142 >> 10.20.193.28:0/4110115142 conn(0x7f4fc80ecf00 msgr2=0x7f4fc810d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:43.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.657+0000 7f4fc7fff640 1 -- 10.20.193.28:0/4110115142 shutdown_connections 2026-02-25T11:47:43.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:43.657+0000 7f4fc7fff640 1 -- 10.20.193.28:0/4110115142 wait complete. 2026-02-25T11:47:43.792 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.b status 2026-02-25T11:47:43.975 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:44.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.145+0000 7f2009d58640 1 -- 10.20.193.28:0/1573183374 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f200410ddc0 msgr2=0x7f2004112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.145+0000 7f2009d58640 1 --2- 10.20.193.28:0/1573183374 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f200410ddc0 0x7f2004112e10 secure :-1 s=READY pgs=331 gs=1 cs=0 l=1 c_cookie=14663561870365127207 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1fec009950 tx=0x7f1fec02f190 comp rx=0 tx=0).stop 2026-02-25T11:47:44.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.145+0000 7f2009d58640 1 -- 10.20.193.28:0/1573183374 shutdown_connections 2026-02-25T11:47:44.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.145+0000 7f2009d58640 1 --2- 10.20.193.28:0/1573183374 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f20041135c0 0x7f2004173f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.145+0000 7f2009d58640 1 --2- 10.20.193.28:0/1573183374 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f200410ddc0 0x7f2004112e10 unknown :-1 s=CLOSED pgs=331 gs=1 cs=0 l=1 c_cookie=14663561870365127207 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.145+0000 7f2009d58640 1 --2- 10.20.193.28:0/1573183374 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f200410d3d0 0x7f200410d7a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.145+0000 7f2009d58640 1 -- 10.20.193.28:0/1573183374 >> 10.20.193.28:0/1573183374 conn(0x7f20040eaca0 msgr2=0x7f200410b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:44.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.145+0000 7f2009d58640 1 -- 10.20.193.28:0/1573183374 shutdown_connections 2026-02-25T11:47:44.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.145+0000 7f2009d58640 1 -- 10.20.193.28:0/1573183374 wait complete. 2026-02-25T11:47:44.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.146+0000 7f2009d58640 1 Processor -- start 2026-02-25T11:47:44.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.146+0000 7f2009d58640 1 -- start start 2026-02-25T11:47:44.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.146+0000 7f2009d58640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f200410d3d0 0x7f20040f5a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.146+0000 7f2009d58640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f200410ddc0 0x7f2004173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.146+0000 7f2009d58640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f20041135c0 0x7f20040f6110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.146+0000 7f2009d58640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f20040fb2d0 con 0x7f200410ddc0 2026-02-25T11:47:44.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.146+0000 7f2009d58640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f20040fb440 con 0x7f200410d3d0 2026-02-25T11:47:44.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.146+0000 7f2009d58640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f20040fb600 con 0x7f20041135c0 2026-02-25T11:47:44.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f1ffbfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f200410ddc0 0x7f2004173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f1ffbfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f200410ddc0 0x7f2004173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35788/0 (socket says 10.20.193.28:35788) 2026-02-25T11:47:44.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f1ffbfff640 1 -- 10.20.193.28:0/392476948 learned_addr learned my addr 10.20.193.28:0/392476948 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:44.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f2008d56640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f200410d3d0 0x7f20040f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f2009557640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f20041135c0 0x7f20040f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f1ffbfff640 1 -- 10.20.193.28:0/392476948 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f20041135c0 msgr2=0x7f20040f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f1ffbfff640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f20041135c0 0x7f20040f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f1ffbfff640 1 -- 10.20.193.28:0/392476948 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f200410d3d0 msgr2=0x7f20040f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f1ffbfff640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f200410d3d0 0x7f20040f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f1ffbfff640 1 -- 10.20.193.28:0/392476948 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec009570 con 0x7f200410ddc0 2026-02-25T11:47:44.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f1ffbfff640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f200410ddc0 0x7f2004173e90 secure :-1 s=READY pgs=332 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1fec0098a0 tx=0x7f1fec0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:44.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f1ff9ffb640 1 -- 10.20.193.28:0/392476948 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1fec03d070 con 0x7f200410ddc0 2026-02-25T11:47:44.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f1ff9ffb640 1 -- 10.20.193.28:0/392476948 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1fec002a40 con 0x7f200410ddc0 2026-02-25T11:47:44.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f1ff9ffb640 1 -- 10.20.193.28:0/392476948 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1fec041820 con 0x7f200410ddc0 2026-02-25T11:47:44.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f2009d58640 1 -- 10.20.193.28:0/392476948 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20040fb880 con 0x7f200410ddc0 2026-02-25T11:47:44.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.147+0000 7f2009d58640 1 -- 10.20.193.28:0/392476948 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20040fbca0 con 0x7f200410ddc0 2026-02-25T11:47:44.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.150+0000 7f1ff9ffb640 1 -- 10.20.193.28:0/392476948 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f1fec0419c0 con 0x7f200410ddc0 2026-02-25T11:47:44.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.152+0000 7f1ff9ffb640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1fe00b57e0 0x7f1fe00d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.152+0000 7f1ff9ffb640 1 -- 10.20.193.28:0/392476948 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f1fec10b090 con 0x7f200410ddc0 2026-02-25T11:47:44.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.152+0000 7f2008d56640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1fe00b57e0 0x7f1fe00d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.152+0000 7f2008d56640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1fe00b57e0 0x7f1fe00d5c90 secure :-1 s=READY pgs=240 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ff4004740 tx=0x7f1ff40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:44.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.153+0000 7f200cfdce00 1 Processor -- start 2026-02-25T11:47:44.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.153+0000 7f200cfdce00 1 -- start start 2026-02-25T11:47:44.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.153+0000 7f200cfdce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bebf5260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.153+0000 7f200cfdce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bebf57a0 0x5613bec39f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f200cfdce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bec3a440 0x5613bec5a820 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f200cfdce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5613beb02a30 con 0x5613bec3a440 2026-02-25T11:47:44.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f200cfdce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5613beafe760 con 0x5613bebf57a0 2026-02-25T11:47:44.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f200cfdce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5613beaf8700 con 0x5613bebf4e90 2026-02-25T11:47:44.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f2009557640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bec3a440 0x5613bec5a820 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f2008d56640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bebf5260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f2008d56640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bebf5260 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:46330/0 (socket says 10.20.193.28:46330) 2026-02-25T11:47:44.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f2008d56640 1 -- 10.20.193.28:0/2069985350 learned_addr learned my addr 10.20.193.28:0/2069985350 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:44.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f1ffbfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bebf57a0 0x5613bec39f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f2008d56640 1 -- 10.20.193.28:0/2069985350 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bebf57a0 msgr2=0x5613bec39f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:44.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f2008d56640 1 --2- 10.20.193.28:0/2069985350 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bebf57a0 0x5613bec39f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f2008d56640 1 -- 10.20.193.28:0/2069985350 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bec3a440 msgr2=0x5613bec5a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:44.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f2008d56640 1 --2- 10.20.193.28:0/2069985350 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bec3a440 0x5613bec5a820 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f2008d56640 1 -- 10.20.193.28:0/2069985350 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613beaa3cf0 con 0x5613bebf4e90 2026-02-25T11:47:44.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.154+0000 7f2008d56640 1 --2- 10.20.193.28:0/2069985350 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bebf5260 secure :-1 s=READY pgs=106 gs=2 cs=0 l=1 c_cookie=7539769902812851134 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ff4007a00 tx=0x7f1ff4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68a2a4c779576bbe server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:44.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.155+0000 7f1fd14e4640 1 -- 10.20.193.28:0/2069985350 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ff4011dd0 con 0x5613bebf4e90 2026-02-25T11:47:44.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.155+0000 7f1fd14e4640 1 -- 10.20.193.28:0/2069985350 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1ff4015070 con 0x5613bebf4e90 2026-02-25T11:47:44.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.155+0000 7f1fd14e4640 1 -- 10.20.193.28:0/2069985350 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ff401d960 con 0x5613bebf4e90 2026-02-25T11:47:44.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.155+0000 7f200cfdce00 1 -- 10.20.193.28:0/2069985350 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 msgr2=0x5613bebf5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.155+0000 7f200cfdce00 1 --2- 10.20.193.28:0/2069985350 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bebf5260 secure :-1 s=READY pgs=106 gs=2 cs=0 l=1 c_cookie=7539769902812851134 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ff4007a00 tx=0x7f1ff4013000 comp rx=0 tx=0).stop 2026-02-25T11:47:44.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.155+0000 7f200cfdce00 1 -- 10.20.193.28:0/2069985350 shutdown_connections 2026-02-25T11:47:44.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.155+0000 7f200cfdce00 1 --2- 10.20.193.28:0/2069985350 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bec3a440 0x5613bec5a820 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.155+0000 7f200cfdce00 1 --2- 10.20.193.28:0/2069985350 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bebf57a0 0x5613bec39f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.155+0000 7f200cfdce00 1 --2- 10.20.193.28:0/2069985350 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bebf5260 unknown :-1 s=CLOSED pgs=106 gs=2 cs=0 l=1 c_cookie=7539769902812851134 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.155+0000 7f200cfdce00 1 -- 10.20.193.28:0/2069985350 >> 10.20.193.28:0/2069985350 conn(0x5613beb0c720 msgr2=0x5613bebead60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:44.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.155+0000 7f200cfdce00 1 -- 10.20.193.28:0/2069985350 shutdown_connections 2026-02-25T11:47:44.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.155+0000 7f200cfdce00 1 -- 10.20.193.28:0/2069985350 wait complete. 2026-02-25T11:47:44.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.156+0000 7f200cfdce00 1 -- start start 2026-02-25T11:47:44.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.156+0000 7f200cfdce00 1 Processor -- start 2026-02-25T11:47:44.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.157+0000 7f200cfdce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bec51ef0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.157+0000 7f200cfdce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf57a0 0x5613bec52430 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.157+0000 7f2008d56640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bec51ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.157+0000 7f2008d56640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bec51ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:46332/0 (socket says 10.20.193.28:46332) 2026-02-25T11:47:44.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.157+0000 7f1ffbfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf57a0 0x5613bec52430 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.157+0000 7f1ffbfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf57a0 0x5613bec52430 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35810/0 (socket says 10.20.193.28:35810) 2026-02-25T11:47:44.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.157+0000 7f200cfdce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bec3a440 0x5613bec57420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.157+0000 7f1ffbfff640 1 -- 10.20.193.28:0/3848541297 learned_addr learned my addr 10.20.193.28:0/3848541297 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:44.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.157+0000 7f200cfdce00 1 -- 10.20.193.28:0/3848541297 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5613bec57960 con 0x5613bebf57a0 2026-02-25T11:47:44.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.157+0000 7f200cfdce00 1 -- 10.20.193.28:0/3848541297 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5613bec57ad0 con 0x5613bec3a440 2026-02-25T11:47:44.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.157+0000 7f200cfdce00 1 -- 10.20.193.28:0/3848541297 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5613bec57c90 con 0x5613bebf4e90 2026-02-25T11:47:44.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.157+0000 7f2009557640 1 --2- 10.20.193.28:0/3848541297 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bec3a440 0x5613bec57420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.158+0000 7f2008d56640 1 -- 10.20.193.28:0/3848541297 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bec3a440 msgr2=0x5613bec57420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:44.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.158+0000 7f2008d56640 1 --2- 10.20.193.28:0/3848541297 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bec3a440 0x5613bec57420 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.158+0000 7f2008d56640 1 -- 10.20.193.28:0/3848541297 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf57a0 msgr2=0x5613bec52430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:44.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.158+0000 7f2008d56640 1 --2- 10.20.193.28:0/3848541297 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf57a0 0x5613bec52430 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.158+0000 7f2008d56640 1 -- 10.20.193.28:0/3848541297 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613bea7c2c0 con 0x5613bebf4e90 2026-02-25T11:47:44.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.158+0000 7f1ffbfff640 1 --2- 10.20.193.28:0/3848541297 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf57a0 0x5613bec52430 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:44.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.158+0000 7f2008d56640 1 --2- 10.20.193.28:0/3848541297 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bec51ef0 secure :-1 s=READY pgs=107 gs=1 cs=0 l=1 c_cookie=5407117063801717061 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ff401d5e0 tx=0x7f1ff4005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b09f05246dac145 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:44.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.158+0000 7f1fc27fc640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:47:44.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.158+0000 7f1fc27fc640 1 -- 10.20.193.28:0/3848541297 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ff402c070 con 0x5613bebf4e90 2026-02-25T11:47:44.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.158+0000 7f1fc27fc640 1 -- 10.20.193.28:0/3848541297 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1ff401cea0 con 0x5613bebf4e90 2026-02-25T11:47:44.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.158+0000 7f1fc27fc640 1 -- 10.20.193.28:0/3848541297 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ff401cb50 con 0x5613bebf4e90 2026-02-25T11:47:44.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.158+0000 7f200cfdce00 1 -- 10.20.193.28:0/3848541297 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff4007cf0 con 0x5613bebf4e90 2026-02-25T11:47:44.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.158+0000 7f1fc27fc640 1 -- 10.20.193.28:0/3848541297 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f1ff4026420 con 0x5613bebf4e90 2026-02-25T11:47:44.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f1fc27fc640 10 client.25183 inactive, discarding osd_map(86..86 src has 1..86) 2026-02-25T11:47:44.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f200cfdce00 10 client.25183 _ll_register_callbacks cb 0x5613beae6a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:44.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f200cfdce00 1 -- 10.20.193.28:0/3848541297 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5613be8503b0 con 0x5613bebf4e90 2026-02-25T11:47:44.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f1fc27fc640 1 -- 10.20.193.28:0/3848541297 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f1ff401cce0 con 0x5613bebf4e90 2026-02-25T11:47:44.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f200cfdce00 10 client.25183 fetch_fsmap learned FSMap version 22 2026-02-25T11:47:44.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f200cfdce00 1 -- 10.20.193.28:0/3848541297 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=22}) -- 0x5613bea8e0c0 con 0x5613bebf4e90 2026-02-25T11:47:44.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f1fc27fc640 1 -- 10.20.193.28:0/3848541297 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f1ff401c410 con 0x5613bebf4e90 2026-02-25T11:47:44.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f200cfdce00 10 client.25183 fetch_fsmap finished waiting for FSMap version 22 2026-02-25T11:47:44.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f200cfdce00 10 client.25183 resolve_mds: resolved name 'b' to daemon mds.b 2026-02-25T11:47:44.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f200cfdce00 20 client.25183 populate_metadata read hostname 'trial028' 2026-02-25T11:47:44.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f200cfdce00 1 --2- 10.20.193.28:0/3848541297 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5613bec383a0 0x5613bec39810 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f200cfdce00 4 client.25183 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:47:44.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f200cfdce00 1 -- 10.20.193.28:0/3848541297 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613beaee140 con 0x5613bec383a0 2026-02-25T11:47:44.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.159+0000 7f1ffbfff640 1 --2- 10.20.193.28:0/3848541297 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5613bec383a0 0x5613bec39810 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.160+0000 7f1ffbfff640 1 --2- 10.20.193.28:0/3848541297 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5613bec383a0 0x5613bec39810 crc :-1 s=READY pgs=17 gs=4 cs=0 l=0 c_cookie=1843192992408900674 s_cookie=15006369306891416960 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19945642cd059442 server_cookie=d0415560a1967580 in_seq=0 out_seq=0 2026-02-25T11:47:44.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.160+0000 7f1fc27fc640 10 client.25183 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:44.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f1fc27fc640 1 -- 10.20.193.28:0/3848541297 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5613beaee140 con 0x5613bec383a0 2026-02-25T11:47:44.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f1fc27fc640 10 client.25183 handle_command_reply: tid=0 2026-02-25T11:47:44.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f200cfdce00 1 client.25183 shutdown 2026-02-25T11:47:44.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f200cfdce00 1 -- 10.20.193.28:0/3848541297 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 msgr2=0x5613bec51ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f200cfdce00 1 --2- 10.20.193.28:0/3848541297 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bec51ef0 secure :-1 s=READY pgs=107 gs=1 cs=0 l=1 c_cookie=5407117063801717061 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ff401d5e0 tx=0x7f1ff4005780 comp rx=0 tx=0).stop 2026-02-25T11:47:44.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f200cfdce00 1 -- 10.20.193.28:0/3848541297 shutdown_connections 2026-02-25T11:47:44.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f200cfdce00 1 --2- 10.20.193.28:0/3848541297 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5613bec383a0 0x5613bec39810 crc :-1 s=READY pgs=17 gs=4 cs=0 l=0 c_cookie=1843192992408900674 s_cookie=15006369306891416960 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f200cfdce00 1 --2- 10.20.193.28:0/3848541297 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bec3a440 0x5613bec57420 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f200cfdce00 1 --2- 10.20.193.28:0/3848541297 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf57a0 0x5613bec52430 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f200cfdce00 1 --2- 10.20.193.28:0/3848541297 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bec51ef0 unknown :-1 s=CLOSED pgs=107 gs=1 cs=0 l=1 c_cookie=5407117063801717061 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f200cfdce00 1 -- 10.20.193.28:0/3848541297 >> 10.20.193.28:0/3848541297 conn(0x5613beb0c720 msgr2=0x5613bebea570 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:44.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f1fc27fc640 0 client.25183 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:44.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f200cfdce00 1 -- 10.20.193.28:0/3848541297 shutdown_connections 2026-02-25T11:47:44.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f200cfdce00 1 -- 10.20.193.28:0/3848541297 wait complete. 2026-02-25T11:47:44.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.161+0000 7f200cfdce00 20 client.25183 trim_cache size 0 max 16384 2026-02-25T11:47:44.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.171+0000 7f200cfdce00 1 Processor -- start 2026-02-25T11:47:44.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.171+0000 7f200cfdce00 1 -- start start 2026-02-25T11:47:44.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.171+0000 7f200cfdce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf4e90 0x5613bec3cf30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.171+0000 7f200cfdce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf57a0 0x5613bec3f290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.171+0000 7f200cfdce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bec3a440 0x5613bec7a210 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.171+0000 7f200cfdce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5613beaf6330 con 0x5613bebf4e90 2026-02-25T11:47:44.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.171+0000 7f200cfdce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5613be7b6610 con 0x5613bec3a440 2026-02-25T11:47:44.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.171+0000 7f200cfdce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5613bebc77d0 con 0x5613bebf57a0 2026-02-25T11:47:44.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.171+0000 7f2008d56640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf4e90 0x5613bec3cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.171+0000 7f2008d56640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf4e90 0x5613bec3cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35826/0 (socket says 10.20.193.28:35826) 2026-02-25T11:47:44.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.171+0000 7f2008d56640 1 -- 10.20.193.28:0/3733250586 learned_addr learned my addr 10.20.193.28:0/3733250586 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:44.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.171+0000 7f1ffbfff640 1 --2- 10.20.193.28:0/3733250586 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf57a0 0x5613bec3f290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.171+0000 7f2009557640 1 --2- 10.20.193.28:0/3733250586 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bec3a440 0x5613bec7a210 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f2008d56640 1 -- 10.20.193.28:0/3733250586 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf57a0 msgr2=0x5613bec3f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:44.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f2008d56640 1 --2- 10.20.193.28:0/3733250586 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf57a0 0x5613bec3f290 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f2008d56640 1 -- 10.20.193.28:0/3733250586 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bec3a440 msgr2=0x5613bec7a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:44.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f2008d56640 1 --2- 10.20.193.28:0/3733250586 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bec3a440 0x5613bec7a210 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f2008d56640 1 -- 10.20.193.28:0/3733250586 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613bea88b40 con 0x5613bebf4e90 2026-02-25T11:47:44.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f2008d56640 1 --2- 10.20.193.28:0/3733250586 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf4e90 0x5613bec3cf30 secure :-1 s=READY pgs=333 gs=1 cs=0 l=1 c_cookie=17929246711720723690 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ff4011460 tx=0x7f1ff4013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d17ae41e7444ea server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:44.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f1fd0ce3640 1 -- 10.20.193.28:0/3733250586 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ff402c070 con 0x5613bebf4e90 2026-02-25T11:47:44.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f1fd0ce3640 1 -- 10.20.193.28:0/3733250586 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5613be8503b0 con 0x5613bebf4e90 2026-02-25T11:47:44.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f1fd0ce3640 1 -- 10.20.193.28:0/3733250586 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ff401c8d0 con 0x5613bebf4e90 2026-02-25T11:47:44.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f200cfdce00 1 -- 10.20.193.28:0/3733250586 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf4e90 msgr2=0x5613bec3cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f200cfdce00 1 --2- 10.20.193.28:0/3733250586 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf4e90 0x5613bec3cf30 secure :-1 s=READY pgs=333 gs=1 cs=0 l=1 c_cookie=17929246711720723690 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ff4011460 tx=0x7f1ff4013070 comp rx=0 tx=0).stop 2026-02-25T11:47:44.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f200cfdce00 1 -- 10.20.193.28:0/3733250586 shutdown_connections 2026-02-25T11:47:44.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f200cfdce00 1 --2- 10.20.193.28:0/3733250586 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bec3a440 0x5613bec7a210 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f200cfdce00 1 --2- 10.20.193.28:0/3733250586 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf57a0 0x5613bec3f290 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f200cfdce00 1 --2- 10.20.193.28:0/3733250586 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bebf4e90 0x5613bec3cf30 unknown :-1 s=CLOSED pgs=333 gs=1 cs=0 l=1 c_cookie=17929246711720723690 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.172+0000 7f200cfdce00 1 -- 10.20.193.28:0/3733250586 >> 10.20.193.28:0/3733250586 conn(0x5613beb0c720 msgr2=0x5613bec3c820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:44.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.173+0000 7f200cfdce00 1 -- 10.20.193.28:0/3733250586 shutdown_connections 2026-02-25T11:47:44.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.173+0000 7f200cfdce00 1 -- 10.20.193.28:0/3733250586 wait complete. 2026-02-25T11:47:44.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.173+0000 7f200cfdce00 1 -- start start 2026-02-25T11:47:44.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.173+0000 7f200cfdce00 1 Processor -- start 2026-02-25T11:47:44.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f200cfdce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bec50b60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f200cfdce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bebf57a0 0x5613bebea660 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f200cfdce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bec3a440 0x5613bebeaba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f200cfdce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5613bec39f60 con 0x5613bec3a440 2026-02-25T11:47:44.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f200cfdce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5613beaf6330 con 0x5613bebf57a0 2026-02-25T11:47:44.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f200cfdce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5613bec4f940 con 0x5613bebf4e90 2026-02-25T11:47:44.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f2009557640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bec3a440 0x5613bebeaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f2009557640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bec3a440 0x5613bebeaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35842/0 (socket says 10.20.193.28:35842) 2026-02-25T11:47:44.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f2009557640 1 -- 10.20.193.28:0/1770565969 learned_addr learned my addr 10.20.193.28:0/1770565969 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:44.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f2008d56640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bec50b60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f1ffbfff640 1 --2- 10.20.193.28:0/1770565969 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bebf57a0 0x5613bebea660 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f2009557640 1 -- 10.20.193.28:0/1770565969 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 msgr2=0x5613bec50b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:44.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f2009557640 1 --2- 10.20.193.28:0/1770565969 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bec50b60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f2009557640 1 -- 10.20.193.28:0/1770565969 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bebf57a0 msgr2=0x5613bebea660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:44.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f2009557640 1 --2- 10.20.193.28:0/1770565969 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bebf57a0 0x5613bebea660 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f2009557640 1 -- 10.20.193.28:0/1770565969 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613bea7c5f0 con 0x5613bec3a440 2026-02-25T11:47:44.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f2009557640 1 --2- 10.20.193.28:0/1770565969 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bec3a440 0x5613bebeaba0 secure :-1 s=READY pgs=334 gs=1 cs=0 l=1 c_cookie=12393764427548407984 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2000006fd0 tx=0x7f200000aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abff7f14f5e538b0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:44.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.174+0000 7f1fc27fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:47:44.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f1fc27fc640 1 -- 10.20.193.28:0/1770565969 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f20000119e0 con 0x5613bec3a440 2026-02-25T11:47:44.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f200cfdce00 1 -- 10.20.193.28:0/1770565969 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613bea8e0c0 con 0x5613bec3a440 2026-02-25T11:47:44.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f200cfdce00 10 client.15333 _ll_register_callbacks cb 0x5613beae6a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:44.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f200cfdce00 1 -- 10.20.193.28:0/1770565969 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff4015070 con 0x5613bec3a440 2026-02-25T11:47:44.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f1fc27fc640 1 -- 10.20.193.28:0/1770565969 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f200001d3f0 con 0x5613bec3a440 2026-02-25T11:47:44.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f1fc27fc640 1 -- 10.20.193.28:0/1770565969 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f20000119e0 con 0x5613bec3a440 2026-02-25T11:47:44.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f1fc27fc640 1 -- 10.20.193.28:0/1770565969 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f200001d560 con 0x5613bec3a440 2026-02-25T11:47:44.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f1fc27fc640 1 -- 10.20.193.28:0/1770565969 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f200001d760 con 0x5613bec3a440 2026-02-25T11:47:44.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f200cfdce00 10 client.15333 fetch_fsmap learned FSMap version 22 2026-02-25T11:47:44.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f200cfdce00 1 -- 10.20.193.28:0/1770565969 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=22}) -- 0x5613bea8e320 con 0x5613bec3a440 2026-02-25T11:47:44.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f1fc27fc640 1 -- 10.20.193.28:0/1770565969 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 22) ==== 8346+0+0 (secure 0 0 0) 0x7f200001c070 con 0x5613bec3a440 2026-02-25T11:47:44.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f200cfdce00 10 client.15333 fetch_fsmap finished waiting for FSMap version 22 2026-02-25T11:47:44.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f200cfdce00 10 client.15333 resolve_mds: resolved name 'b' to daemon mds.b 2026-02-25T11:47:44.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f200cfdce00 20 client.15333 populate_metadata read hostname 'trial028' 2026-02-25T11:47:44.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f200cfdce00 1 --2- 10.20.193.28:0/1770565969 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5613bec1a370 0x5613bec19080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f200cfdce00 4 client.15333 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "status"}] 2026-02-25T11:47:44.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.175+0000 7f200cfdce00 1 -- 10.20.193.28:0/1770565969 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "status"}) -- 0x5613be835070 con 0x5613bec1a370 2026-02-25T11:47:44.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.176+0000 7f2008d56640 1 --2- 10.20.193.28:0/1770565969 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5613bec1a370 0x5613bec19080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.176+0000 7f2008d56640 1 --2- 10.20.193.28:0/1770565969 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5613bec1a370 0x5613bec19080 crc :-1 s=READY pgs=18 gs=4 cs=0 l=0 c_cookie=8459491767487993594 s_cookie=16549580400562876477 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=756626f1357696fa server_cookie=e5abebb0dd51303d in_seq=0 out_seq=0 2026-02-25T11:47:44.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.176+0000 7f1fc27fc640 10 client.15333 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:44.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f1fc27fc640 1 -- 10.20.193.28:0/1770565969 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1217 (crc 0 0 0) 0x5613be835070 con 0x5613bec1a370 2026-02-25T11:47:44.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f1fc27fc640 10 client.15333 handle_command_reply: tid=0 2026-02-25T11:47:44.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f200cfdce00 1 client.15333 shutdown 2026-02-25T11:47:44.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f200cfdce00 1 -- 10.20.193.28:0/1770565969 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bec3a440 msgr2=0x5613bebeaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f200cfdce00 1 --2- 10.20.193.28:0/1770565969 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bec3a440 0x5613bebeaba0 secure :-1 s=READY pgs=334 gs=1 cs=0 l=1 c_cookie=12393764427548407984 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2000006fd0 tx=0x7f200000aab0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f200cfdce00 1 -- 10.20.193.28:0/1770565969 shutdown_connections 2026-02-25T11:47:44.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f200cfdce00 1 --2- 10.20.193.28:0/1770565969 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5613bec1a370 0x5613bec19080 crc :-1 s=READY pgs=18 gs=4 cs=0 l=0 c_cookie=8459491767487993594 s_cookie=16549580400562876477 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f200cfdce00 1 --2- 10.20.193.28:0/1770565969 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5613bec3a440 0x5613bebeaba0 unknown :-1 s=CLOSED pgs=334 gs=1 cs=0 l=1 c_cookie=12393764427548407984 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f200cfdce00 1 --2- 10.20.193.28:0/1770565969 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5613bebf57a0 0x5613bebea660 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f200cfdce00 1 --2- 10.20.193.28:0/1770565969 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5613bebf4e90 0x5613bec50b60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f1fc27fc640 0 client.15333 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:44.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f200cfdce00 1 -- 10.20.193.28:0/1770565969 >> 10.20.193.28:0/1770565969 conn(0x5613beb0c720 msgr2=0x5613bec56280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:44.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f200cfdce00 1 -- 10.20.193.28:0/1770565969 shutdown_connections 2026-02-25T11:47:44.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f200cfdce00 1 -- 10.20.193.28:0/1770565969 wait complete. 2026-02-25T11:47:44.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f200cfdce00 20 client.15333 trim_cache size 0 max 16384 2026-02-25T11:47:44.187 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:47:44.187 INFO:teuthology.orchestra.run.trial028.stdout: "cluster_fsid": "123fa7bf-123f-11f1-80da-d404e6e7d460", 2026-02-25T11:47:44.187 INFO:teuthology.orchestra.run.trial028.stdout: "whoami": 0, 2026-02-25T11:47:44.187 INFO:teuthology.orchestra.run.trial028.stdout: "id": 24749, 2026-02-25T11:47:44.187 INFO:teuthology.orchestra.run.trial028.stdout: "want_state": "up:active", 2026-02-25T11:47:44.187 INFO:teuthology.orchestra.run.trial028.stdout: "state": "up:active", 2026-02-25T11:47:44.188 INFO:teuthology.orchestra.run.trial028.stdout: "fs_name": "cephfs", 2026-02-25T11:47:44.188 INFO:teuthology.orchestra.run.trial028.stdout: "rank_uptime": 30.551071955000001, 2026-02-25T11:47:44.188 INFO:teuthology.orchestra.run.trial028.stdout: "mdsmap_epoch": 22, 2026-02-25T11:47:44.188 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_epoch": 86, 2026-02-25T11:47:44.188 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_epoch_barrier": 86, 2026-02-25T11:47:44.188 INFO:teuthology.orchestra.run.trial028.stdout: "uptime": 37.027825647, 2026-02-25T11:47:44.188 INFO:teuthology.orchestra.run.trial028.stdout: "sysinfo": { 2026-02-25T11:47:44.188 INFO:teuthology.orchestra.run.trial028.stdout: "arch": "x86_64", 2026-02-25T11:47:44.188 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_release": "tentacle", 2026-02-25T11:47:44.188 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version": "ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev - RelWithDebInfo)", 2026-02-25T11:47:44.188 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version_short": "20.3.0-5549-g92715cd3", 2026-02-25T11:47:44.188 INFO:teuthology.orchestra.run.trial028.stdout: "container_hostname": "trial119", 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout: "container_image": "quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33", 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout: "distro": "centos", 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout: "distro_description": "CentOS Stream 9", 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout: "distro_version": "9", 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout: "hostname": "trial119", 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Tue Feb 24 21:03:19 EST 2026", 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout: "kernel_version": "6.19.0-g712f4a43c125", 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout: "mem_swap_kb": "8388604", 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout: "mem_total_kb": "130448332", 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout: "os": "Linux" 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout: "endian": "little" 2026-02-25T11:47:44.189 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:47:44.190 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f2009d58640 1 -- 10.20.193.28:0/392476948 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1fe00b57e0 msgr2=0x7f1fe00d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.190 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f2009d58640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1fe00b57e0 0x7f1fe00d5c90 secure :-1 s=READY pgs=240 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ff4004740 tx=0x7f1ff40073d0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.190 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f2009d58640 1 -- 10.20.193.28:0/392476948 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f200410ddc0 msgr2=0x7f2004173e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.190 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.177+0000 7f2009d58640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f200410ddc0 0x7f2004173e90 secure :-1 s=READY pgs=332 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1fec0098a0 tx=0x7f1fec0028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.178+0000 7f2009d58640 1 -- 10.20.193.28:0/392476948 shutdown_connections 2026-02-25T11:47:44.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.178+0000 7f2009d58640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1fe00b57e0 0x7f1fe00d5c90 unknown :-1 s=CLOSED pgs=240 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.178+0000 7f2009d58640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f20041135c0 0x7f20040f6110 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.178+0000 7f2009d58640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f200410ddc0 0x7f2004173e90 unknown :-1 s=CLOSED pgs=332 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.178+0000 7f2009d58640 1 --2- 10.20.193.28:0/392476948 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f200410d3d0 0x7f20040f5a10 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.178+0000 7f2009d58640 1 -- 10.20.193.28:0/392476948 >> 10.20.193.28:0/392476948 conn(0x7f20040eaca0 msgr2=0x7f200410fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:44.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.178+0000 7f2009d58640 1 -- 10.20.193.28:0/392476948 shutdown_connections 2026-02-25T11:47:44.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.178+0000 7f2009d58640 1 -- 10.20.193.28:0/392476948 wait complete. 2026-02-25T11:47:44.281 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:44 trial028 ceph-mon[39224]: pgmap v209: 97 pgs: 97 active+clean; 648 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 6 op/s 2026-02-25T11:47:44.282 INFO:teuthology.run_tasks:Running task ceph-fuse... 2026-02-25T11:47:44.325 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2026-02-25T11:47:44.325 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2026-02-25T11:47:44.325 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2026-02-25T11:47:44.325 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2026-02-25T11:47:44.325 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2026-02-25T11:47:44.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:47:44.325 DEBUG:teuthology.orchestra.run.trial028:> ip netns list 2026-02-25T11:47:44.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:47:44.366 DEBUG:teuthology.orchestra.run.trial028:> sudo ip link delete ceph-brx 2026-02-25T11:47:44.466 INFO:teuthology.orchestra.run.trial028.stderr:Cannot find device "ceph-brx" 2026-02-25T11:47:44.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T11:47:44.470 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2026-02-25T11:47:44.470 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2026-02-25T11:47:44.470 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs ls 2026-02-25T11:47:44.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:44 trial167 ceph-mon[43392]: pgmap v209: 97 pgs: 97 active+clean; 648 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 6 op/s 2026-02-25T11:47:44.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:44 trial119 ceph-mon[42710]: pgmap v209: 97 pgs: 97 active+clean; 648 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.2 KiB/s rd, 853 B/s wr, 6 op/s 2026-02-25T11:47:44.655 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:44.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.828+0000 7fb0d9cad640 1 -- 10.20.193.28:0/3045721973 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb0d4111510 msgr2=0x7fb0d41119b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.828+0000 7fb0d9cad640 1 --2- 10.20.193.28:0/3045721973 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb0d4111510 0x7fb0d41119b0 secure :-1 s=READY pgs=335 gs=1 cs=0 l=1 c_cookie=1743616415782827014 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb0d000aa60 tx=0x7fb0d00302b0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.828+0000 7fb0d9cad640 1 -- 10.20.193.28:0/3045721973 shutdown_connections 2026-02-25T11:47:44.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.828+0000 7fb0d9cad640 1 --2- 10.20.193.28:0/3045721973 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb0d4111510 0x7fb0d41119b0 unknown :-1 s=CLOSED pgs=335 gs=1 cs=0 l=1 c_cookie=1743616415782827014 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.828+0000 7fb0d9cad640 1 --2- 10.20.193.28:0/3045721973 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb0d410c920 0x7fb0d4110ef0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.828+0000 7fb0d9cad640 1 --2- 10.20.193.28:0/3045721973 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb0d410bf30 0x7fb0d410c300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.828+0000 7fb0d9cad640 1 -- 10.20.193.28:0/3045721973 >> 10.20.193.28:0/3045721973 conn(0x7fb0d40e9b40 msgr2=0x7fb0d4109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:44.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.829+0000 7fb0d9cad640 1 -- 10.20.193.28:0/3045721973 shutdown_connections 2026-02-25T11:47:44.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.829+0000 7fb0d9cad640 1 -- 10.20.193.28:0/3045721973 wait complete. 2026-02-25T11:47:44.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0d9cad640 1 Processor -- start 2026-02-25T11:47:44.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0d9cad640 1 -- start start 2026-02-25T11:47:44.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0d9cad640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb0d410bf30 0x7fb0d4122950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0d9cad640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb0d410c920 0x7fb0d4122e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0d9cad640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb0d4111510 0x7fb0d4128260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0d9cad640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4128880 con 0x7fb0d410c920 2026-02-25T11:47:44.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0d9cad640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d41289f0 con 0x7fb0d4111510 2026-02-25T11:47:44.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0d9cad640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4128bb0 con 0x7fb0d410bf30 2026-02-25T11:47:44.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0cbfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb0d410c920 0x7fb0d4122e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0cbfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb0d410c920 0x7fb0d4122e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35856/0 (socket says 10.20.193.28:35856) 2026-02-25T11:47:44.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0cbfff640 1 -- 10.20.193.28:0/375119499 learned_addr learned my addr 10.20.193.28:0/375119499 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:44.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0d94ac640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb0d4111510 0x7fb0d4128260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0d8cab640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb0d410bf30 0x7fb0d4122950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0cbfff640 1 -- 10.20.193.28:0/375119499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb0d410bf30 msgr2=0x7fb0d4122950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.836 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0cbfff640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb0d410bf30 0x7fb0d4122950 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.836 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0cbfff640 1 -- 10.20.193.28:0/375119499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb0d4111510 msgr2=0x7fb0d4128260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.836 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0cbfff640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb0d4111510 0x7fb0d4128260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.836 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.830+0000 7fb0cbfff640 1 -- 10.20.193.28:0/375119499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d000a650 con 0x7fb0d410c920 2026-02-25T11:47:44.836 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.831+0000 7fb0cbfff640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb0d410c920 0x7fb0d4122e90 secure :-1 s=READY pgs=336 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb0bc00d950 tx=0x7fb0bc00de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:44.836 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.831+0000 7fb0c9ffb640 1 -- 10.20.193.28:0/375119499 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb0bc004540 con 0x7fb0d410c920 2026-02-25T11:47:44.836 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.831+0000 7fb0c9ffb640 1 -- 10.20.193.28:0/375119499 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb0bc009ce0 con 0x7fb0d410c920 2026-02-25T11:47:44.836 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.831+0000 7fb0c9ffb640 1 -- 10.20.193.28:0/375119499 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb0bc0106e0 con 0x7fb0d410c920 2026-02-25T11:47:44.836 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.831+0000 7fb0d9cad640 1 -- 10.20.193.28:0/375119499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d4128e90 con 0x7fb0d410c920 2026-02-25T11:47:44.836 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.831+0000 7fb0d9cad640 1 -- 10.20.193.28:0/375119499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d4169590 con 0x7fb0d410c920 2026-02-25T11:47:44.836 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.832+0000 7fb0d9cad640 1 -- 10.20.193.28:0/375119499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d41290b0 con 0x7fb0d410c920 2026-02-25T11:47:44.837 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.834+0000 7fb0c9ffb640 1 -- 10.20.193.28:0/375119499 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fb0bc0097c0 con 0x7fb0d410c920 2026-02-25T11:47:44.838 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.836+0000 7fb0c9ffb640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb0b00b57f0 0x7fb0b00d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:44.839 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.836+0000 7fb0c9ffb640 1 -- 10.20.193.28:0/375119499 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fb0bc0e6ea0 con 0x7fb0d410c920 2026-02-25T11:47:44.839 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.836+0000 7fb0d8cab640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb0b00b57f0 0x7fb0b00d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:44.839 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.836+0000 7fb0c9ffb640 1 -- 10.20.193.28:0/375119499 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fb0bc0e7320 con 0x7fb0d410c920 2026-02-25T11:47:44.839 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.836+0000 7fb0d8cab640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb0b00b57f0 0x7fb0b00d5ca0 secure :-1 s=READY pgs=241 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb0c4004740 tx=0x7fb0c4004420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:44.983 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.980+0000 7fb0d9cad640 1 -- 10.20.193.28:0/375119499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb0d4123ab0 con 0x7fb0d410c920 2026-02-25T11:47:44.984 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.981+0000 7fb0c9ffb640 1 -- 10.20.193.28:0/375119499 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) ==== 53+0+73 (secure 0 0 0) 0x7fb0bc0a9900 con 0x7fb0d410c920 2026-02-25T11:47:44.984 INFO:teuthology.orchestra.run.trial028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2026-02-25T11:47:44.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.982+0000 7fb0d9cad640 1 -- 10.20.193.28:0/375119499 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb0b00b57f0 msgr2=0x7fb0b00d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.982+0000 7fb0d9cad640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb0b00b57f0 0x7fb0b00d5ca0 secure :-1 s=READY pgs=241 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb0c4004740 tx=0x7fb0c4004420 comp rx=0 tx=0).stop 2026-02-25T11:47:44.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.982+0000 7fb0d9cad640 1 -- 10.20.193.28:0/375119499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb0d410c920 msgr2=0x7fb0d4122e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:44.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.982+0000 7fb0d9cad640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb0d410c920 0x7fb0d4122e90 secure :-1 s=READY pgs=336 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb0bc00d950 tx=0x7fb0bc00de20 comp rx=0 tx=0).stop 2026-02-25T11:47:44.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.982+0000 7fb0d9cad640 1 -- 10.20.193.28:0/375119499 shutdown_connections 2026-02-25T11:47:44.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.982+0000 7fb0d9cad640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb0b00b57f0 0x7fb0b00d5ca0 unknown :-1 s=CLOSED pgs=241 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.982+0000 7fb0d9cad640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb0d4111510 0x7fb0d4128260 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.982+0000 7fb0d9cad640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb0d410c920 0x7fb0d4122e90 unknown :-1 s=CLOSED pgs=336 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.982+0000 7fb0d9cad640 1 --2- 10.20.193.28:0/375119499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb0d410bf30 0x7fb0d4122950 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:44.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.982+0000 7fb0d9cad640 1 -- 10.20.193.28:0/375119499 >> 10.20.193.28:0/375119499 conn(0x7fb0d40e9b40 msgr2=0x7fb0d4109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:44.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.984+0000 7fb0d9cad640 1 -- 10.20.193.28:0/375119499 shutdown_connections 2026-02-25T11:47:44.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:44.984+0000 7fb0d9cad640 1 -- 10.20.193.28:0/375119499 wait complete. 2026-02-25T11:47:45.086 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2026-02-25T11:47:45.086 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2026-02-25T11:47:45.086 INFO:tasks.cephfs.mount:self.client_remote.hostname = trial028.front.sepia.ceph.com 2026-02-25T11:47:45.086 INFO:tasks.cephfs.mount:self.client.name = client.0 2026-02-25T11:47:45.086 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2026-02-25T11:47:45.086 INFO:tasks.cephfs.mount:self.cephfs_name = None 2026-02-25T11:47:45.086 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2026-02-25T11:47:45.086 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2026-02-25T11:47:45.086 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2026-02-25T11:47:45.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:47:45.086 DEBUG:teuthology.orchestra.run.trial028:> ip addr 2026-02-25T11:47:45.125 INFO:teuthology.orchestra.run.trial028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2026-02-25T11:47:45.125 INFO:teuthology.orchestra.run.trial028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2026-02-25T11:47:45.125 INFO:teuthology.orchestra.run.trial028.stdout: inet 127.0.0.1/8 scope host lo 2026-02-25T11:47:45.125 INFO:teuthology.orchestra.run.trial028.stdout: valid_lft forever preferred_lft forever 2026-02-25T11:47:45.126 INFO:teuthology.orchestra.run.trial028.stdout: inet6 ::1/128 scope host 2026-02-25T11:47:45.126 INFO:teuthology.orchestra.run.trial028.stdout: valid_lft forever preferred_lft forever 2026-02-25T11:47:45.126 INFO:teuthology.orchestra.run.trial028.stdout:2: enp1s0f0np0: mtu 1500 qdisc fq_codel state UP group default qlen 1000 2026-02-25T11:47:45.126 INFO:teuthology.orchestra.run.trial028.stdout: link/ether 90:5a:08:77:63:0c brd ff:ff:ff:ff:ff:ff 2026-02-25T11:47:45.126 INFO:teuthology.orchestra.run.trial028.stdout: inet 10.20.193.28/20 brd 10.20.207.255 scope global dynamic noprefixroute enp1s0f0np0 2026-02-25T11:47:45.126 INFO:teuthology.orchestra.run.trial028.stdout: valid_lft 688sec preferred_lft 688sec 2026-02-25T11:47:45.126 INFO:teuthology.orchestra.run.trial028.stdout: inet6 fe80::925a:8ff:fe77:630c/64 scope link proto kernel_ll 2026-02-25T11:47:45.126 INFO:teuthology.orchestra.run.trial028.stdout: valid_lft forever preferred_lft forever 2026-02-25T11:47:45.126 INFO:teuthology.orchestra.run.trial028.stdout:3: enp1s0f1np1: mtu 1500 qdisc fq_codel state DOWN group default qlen 1000 2026-02-25T11:47:45.126 INFO:teuthology.orchestra.run.trial028.stdout: link/ether 90:5a:08:77:63:0d brd ff:ff:ff:ff:ff:ff 2026-02-25T11:47:45.127 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.159.254/20 2026-02-25T11:47:45.127 DEBUG:teuthology.orchestra.run.trial028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2026-02-25T11:47:45.127 DEBUG:teuthology.orchestra.run.trial028:> set -e 2026-02-25T11:47:45.128 DEBUG:teuthology.orchestra.run.trial028:> sudo ip link add name ceph-brx type bridge 2026-02-25T11:47:45.128 DEBUG:teuthology.orchestra.run.trial028:> sudo ip addr flush dev ceph-brx 2026-02-25T11:47:45.128 DEBUG:teuthology.orchestra.run.trial028:> sudo ip link set ceph-brx up 2026-02-25T11:47:45.128 DEBUG:teuthology.orchestra.run.trial028:> sudo ip addr add 192.168.159.254/20 brd 192.168.159.255 dev ceph-brx 2026-02-25T11:47:45.128 DEBUG:teuthology.orchestra.run.trial028:> ') 2026-02-25T11:47:45.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2026-02-25T11:47:45.245 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:45 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/375119499' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2026-02-25T11:47:45.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2026-02-25T11:47:45.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:47:45.389 DEBUG:teuthology.orchestra.run.trial028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2026-02-25T11:47:45.445 INFO:teuthology.orchestra.run.trial028.stdout:1 2026-02-25T11:47:45.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:47:45.448 DEBUG:teuthology.orchestra.run.trial028:> ip r 2026-02-25T11:47:45.488 INFO:teuthology.orchestra.run.trial028.stdout:default via 10.20.192.1 dev enp1s0f0np0 proto dhcp src 10.20.193.28 metric 100 2026-02-25T11:47:45.488 INFO:teuthology.orchestra.run.trial028.stdout:10.20.192.0/20 dev enp1s0f0np0 proto kernel scope link src 10.20.193.28 metric 100 2026-02-25T11:47:45.488 INFO:teuthology.orchestra.run.trial028.stdout:192.168.144.0/20 dev ceph-brx proto kernel scope link src 192.168.159.254 2026-02-25T11:47:45.489 DEBUG:teuthology.orchestra.run.trial028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2026-02-25T11:47:45.490 DEBUG:teuthology.orchestra.run.trial028:> set -e 2026-02-25T11:47:45.490 DEBUG:teuthology.orchestra.run.trial028:> sudo iptables -A FORWARD -o enp1s0f0np0 -i ceph-brx -j ACCEPT 2026-02-25T11:47:45.490 DEBUG:teuthology.orchestra.run.trial028:> sudo iptables -A FORWARD -i enp1s0f0np0 -o ceph-brx -j ACCEPT 2026-02-25T11:47:45.490 DEBUG:teuthology.orchestra.run.trial028:> sudo iptables -t nat -A POSTROUTING -s 192.168.159.254/20 -o enp1s0f0np0 -j MASQUERADE 2026-02-25T11:47:45.490 DEBUG:teuthology.orchestra.run.trial028:> ') 2026-02-25T11:47:45.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:45 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/375119499' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2026-02-25T11:47:45.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:45 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/375119499' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2026-02-25T11:47:45.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2026-02-25T11:47:45.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2026-02-25T11:47:45.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:47:45.786 DEBUG:teuthology.orchestra.run.trial028:> ip netns list 2026-02-25T11:47:45.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:47:45.827 DEBUG:teuthology.orchestra.run.trial028:> ip netns list-id 2026-02-25T11:47:45.905 DEBUG:teuthology.orchestra.run.trial028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2026-02-25T11:47:45.905 DEBUG:teuthology.orchestra.run.trial028:> set -e 2026-02-25T11:47:45.905 DEBUG:teuthology.orchestra.run.trial028:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2026-02-25T11:47:45.905 DEBUG:teuthology.orchestra.run.trial028:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2026-02-25T11:47:45.906 DEBUG:teuthology.orchestra.run.trial028:> ') 2026-02-25T11:47:46.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2026-02-25T11:47:46.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2026-02-25T11:47:46.062 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.144.1/20 2026-02-25T11:47:46.063 DEBUG:teuthology.orchestra.run.trial028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2026-02-25T11:47:46.063 DEBUG:teuthology.orchestra.run.trial028:> set -e 2026-02-25T11:47:46.063 DEBUG:teuthology.orchestra.run.trial028:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2026-02-25T11:47:46.063 DEBUG:teuthology.orchestra.run.trial028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.144.1/20 brd 192.168.159.255 dev veth0 2026-02-25T11:47:46.063 DEBUG:teuthology.orchestra.run.trial028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2026-02-25T11:47:46.063 DEBUG:teuthology.orchestra.run.trial028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2026-02-25T11:47:46.063 DEBUG:teuthology.orchestra.run.trial028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.159.254 2026-02-25T11:47:46.064 DEBUG:teuthology.orchestra.run.trial028:> ') 2026-02-25T11:47:46.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2026-02-25T11:47:46.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2026-02-25T11:47:46.263 DEBUG:teuthology.orchestra.run.trial028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2026-02-25T11:47:46.263 DEBUG:teuthology.orchestra.run.trial028:> set -e 2026-02-25T11:47:46.263 DEBUG:teuthology.orchestra.run.trial028:> sudo ip link set brx.0 up 2026-02-25T11:47:46.263 DEBUG:teuthology.orchestra.run.trial028:> sudo ip link set dev brx.0 master ceph-brx 2026-02-25T11:47:46.263 DEBUG:teuthology.orchestra.run.trial028:> ') 2026-02-25T11:47:46.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:46 trial028 ceph-mon[39224]: pgmap v210: 97 pgs: 97 active+clean; 648 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2026-02-25T11:47:46.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2026-02-25T11:47:46.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2026-02-25T11:47:46.414 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2026-02-25T11:47:46.414 INFO:teuthology.orchestra.run:Running command with timeout 60 2026-02-25T11:47:46.414 DEBUG:teuthology.orchestra.run.trial028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2026-02-25T11:47:46.453 INFO:teuthology.orchestra.run.trial028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2026-02-25T11:47:46.453 INFO:teuthology.orchestra.run:Running command with timeout 60 2026-02-25T11:47:46.454 DEBUG:teuthology.orchestra.run.trial028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2026-02-25T11:47:46.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:46 trial167 ceph-mon[43392]: pgmap v210: 97 pgs: 97 active+clean; 648 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2026-02-25T11:47:46.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:46 trial119 ceph-mon[42710]: pgmap v210: 97 pgs: 97 active+clean; 648 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2026-02-25T11:47:46.531 DEBUG:teuthology.orchestra.run.trial028:> sudo modprobe fuse 2026-02-25T11:47:46.629 DEBUG:teuthology.orchestra.run.trial028:> cat /proc/self/mounts | awk '{print $2}' 2026-02-25T11:47:46.667 INFO:teuthology.orchestra.run.trial028.stdout:/proc 2026-02-25T11:47:46.667 INFO:teuthology.orchestra.run.trial028.stdout:/sys 2026-02-25T11:47:46.667 INFO:teuthology.orchestra.run.trial028.stdout:/dev 2026-02-25T11:47:46.667 INFO:teuthology.orchestra.run.trial028.stdout:/sys/kernel/security 2026-02-25T11:47:46.667 INFO:teuthology.orchestra.run.trial028.stdout:/dev/shm 2026-02-25T11:47:46.667 INFO:teuthology.orchestra.run.trial028.stdout:/dev/pts 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/run 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/sys/fs/cgroup 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/sys/fs/pstore 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/sys/firmware/efi/efivars 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/sys/fs/bpf 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/sys/kernel/config 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/ 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/sys/fs/selinux 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/proc/sys/fs/binfmt_misc 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/dev/hugepages 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/dev/mqueue 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/sys/kernel/debug 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/sys/kernel/tracing 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/sys/fs/fuse/connections 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/run/credentials/systemd-sysctl.service 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/boot/efi 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/run/credentials/systemd-tmpfiles-setup.service 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/nfs/rpc_pipefs 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/run/user/1000 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/ceph 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/85b20f27d563a78905a04255c8f9a7fde7354fe5809a069cbded78290bd91a82/merged 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/61c572d30cc7a37b94f8c07a72c476851a4e7cf3a0b9f901f4f9292cf81d25d3/merged 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/run/user/0 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/proc/sys/fs/binfmt_misc 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/6344dddc3cdec15e321832c3a1bbd4737350889bf935a09d77527871df2fa062/merged 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/6f9d32ec0007f28dd2992febc174b9c9611e01796e0aa54352077937a0e8b99f/merged 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/ade9fbaea8ee7876cbbbc71a9da4cf46b61d9b4ddbb3f58a97a93cf1fb05f78c/merged 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/03f861a23b1da140cd9ce62b7ea3d38a898014c10fe01cfb90e122c5a99c9a96/merged 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/d053680983d11a3b1b6e2b2d36eeb59fc11b8c84afcabc5f5871c5884106b1b9/merged 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/c1e2db6273f193b06ee8b03c700b4cc2dee701d7e77d812714f7d8340c9874ca/merged 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/95e7ac679e173e47727b2d1b1809d932f6f5909fb2aa4d96eff143de56927983/merged 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/ac4c6ea905bcc85cc23cfc8ab926d6f5b90c16818f82d62789168744b5cf65f0/merged 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/run/netns 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2026-02-25T11:47:46.668 INFO:teuthology.orchestra.run.trial028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2026-02-25T11:47:46.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:47:46.669 DEBUG:teuthology.orchestra.run.trial028:> ls /sys/fs/fuse/connections 2026-02-25T11:47:46.747 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2026-02-25T11:47:46.747 DEBUG:teuthology.orchestra.run.trial028:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/d31804c4-30cc-4124-85d4-92e1f982c86a --client_fs=cephfs) 2026-02-25T11:47:46.791 DEBUG:teuthology.orchestra.run.trial028:> sudo modprobe fuse 2026-02-25T11:47:46.846 DEBUG:teuthology.orchestra.run.trial028:> cat /proc/self/mounts | awk '{print $2}' 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/proc 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/sys 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/dev 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/sys/kernel/security 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/dev/shm 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/dev/pts 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/run 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/sys/fs/cgroup 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/sys/fs/pstore 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/sys/firmware/efi/efivars 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/sys/fs/bpf 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/sys/kernel/config 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/ 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/sys/fs/selinux 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/proc/sys/fs/binfmt_misc 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/dev/hugepages 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/dev/mqueue 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/sys/kernel/debug 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/sys/kernel/tracing 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/sys/fs/fuse/connections 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/run/credentials/systemd-sysctl.service 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/boot/efi 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/run/credentials/systemd-tmpfiles-setup.service 2026-02-25T11:47:46.884 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/nfs/rpc_pipefs 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/run/user/1000 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/ceph 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/85b20f27d563a78905a04255c8f9a7fde7354fe5809a069cbded78290bd91a82/merged 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/61c572d30cc7a37b94f8c07a72c476851a4e7cf3a0b9f901f4f9292cf81d25d3/merged 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/run/user/0 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/proc/sys/fs/binfmt_misc 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/6344dddc3cdec15e321832c3a1bbd4737350889bf935a09d77527871df2fa062/merged 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/6f9d32ec0007f28dd2992febc174b9c9611e01796e0aa54352077937a0e8b99f/merged 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/ade9fbaea8ee7876cbbbc71a9da4cf46b61d9b4ddbb3f58a97a93cf1fb05f78c/merged 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/03f861a23b1da140cd9ce62b7ea3d38a898014c10fe01cfb90e122c5a99c9a96/merged 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/d053680983d11a3b1b6e2b2d36eeb59fc11b8c84afcabc5f5871c5884106b1b9/merged 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/c1e2db6273f193b06ee8b03c700b4cc2dee701d7e77d812714f7d8340c9874ca/merged 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/95e7ac679e173e47727b2d1b1809d932f6f5909fb2aa4d96eff143de56927983/merged 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/var/lib/containers/storage/overlay/ac4c6ea905bcc85cc23cfc8ab926d6f5b90c16818f82d62789168744b5cf65f0/merged 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/run/netns 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run.trial028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2026-02-25T11:47:46.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:47:46.885 DEBUG:teuthology.orchestra.run.trial028:> ls /sys/fs/fuse/connections 2026-02-25T11:47:46.931 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.trial028.stderr:2026-02-25T11:47:46.928+0000 7feff2835400 -1 init, newargv = 0x79ed130 newargc=13 2026-02-25T11:47:46.932 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.trial028.stderr:2026-02-25T11:47:46.928+0000 7feff2835400 -1 init, args.argv = 0x79e8c60 args.argc=4 2026-02-25T11:47:46.932 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.trial028.stderr:ceph-fuse[113767]: starting ceph client 2026-02-25T11:47:46.956 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.trial028.stderr:ceph-fuse[113767]: starting fuse 2026-02-25T11:47:46.967 INFO:teuthology.orchestra.run.trial028.stdout:43 2026-02-25T11:47:46.967 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2026-02-25T11:47:46.967 DEBUG:teuthology.orchestra.run.trial028:> sudo stdin-killer -- python3 -c ' 2026-02-25T11:47:46.968 DEBUG:teuthology.orchestra.run.trial028:> import glob 2026-02-25T11:47:46.968 DEBUG:teuthology.orchestra.run.trial028:> import re 2026-02-25T11:47:46.968 DEBUG:teuthology.orchestra.run.trial028:> import os 2026-02-25T11:47:46.968 DEBUG:teuthology.orchestra.run.trial028:> import subprocess 2026-02-25T11:47:46.968 DEBUG:teuthology.orchestra.run.trial028:> 2026-02-25T11:47:46.968 DEBUG:teuthology.orchestra.run.trial028:> def _find_admin_socket(client_name): 2026-02-25T11:47:46.968 DEBUG:teuthology.orchestra.run.trial028:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2026-02-25T11:47:46.968 DEBUG:teuthology.orchestra.run.trial028:> files = glob.glob(asok_path) 2026-02-25T11:47:46.968 DEBUG:teuthology.orchestra.run.trial028:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2026-02-25T11:47:46.968 DEBUG:teuthology.orchestra.run.trial028:> 2026-02-25T11:47:46.968 DEBUG:teuthology.orchestra.run.trial028:> # Given a non-glob path, it better be there 2026-02-25T11:47:46.969 DEBUG:teuthology.orchestra.run.trial028:> if "*" not in asok_path: 2026-02-25T11:47:46.969 DEBUG:teuthology.orchestra.run.trial028:> assert(len(files) == 1) 2026-02-25T11:47:46.969 DEBUG:teuthology.orchestra.run.trial028:> return files[0] 2026-02-25T11:47:46.969 DEBUG:teuthology.orchestra.run.trial028:> 2026-02-25T11:47:46.969 DEBUG:teuthology.orchestra.run.trial028:> for f in files: 2026-02-25T11:47:46.969 DEBUG:teuthology.orchestra.run.trial028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2026-02-25T11:47:46.969 DEBUG:teuthology.orchestra.run.trial028:> if os.path.exists("/proc/{0}".format(pid)): 2026-02-25T11:47:46.969 DEBUG:teuthology.orchestra.run.trial028:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2026-02-25T11:47:46.969 DEBUG:teuthology.orchestra.run.trial028:> contents = proc_f.read() 2026-02-25T11:47:46.969 DEBUG:teuthology.orchestra.run.trial028:> if mountpoint in contents: 2026-02-25T11:47:46.969 DEBUG:teuthology.orchestra.run.trial028:> return f 2026-02-25T11:47:46.969 DEBUG:teuthology.orchestra.run.trial028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2026-02-25T11:47:46.969 DEBUG:teuthology.orchestra.run.trial028:> 2026-02-25T11:47:46.970 DEBUG:teuthology.orchestra.run.trial028:> print(_find_admin_socket("client.0")) 2026-02-25T11:47:46.970 DEBUG:teuthology.orchestra.run.trial028:> ' 2026-02-25T11:47:47.106 INFO:teuthology.orchestra.run.trial028.stdout:/var/run/ceph/ceph-client.0.113767.asok 2026-02-25T11:47:47.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2026-02-25T11:47:47.116 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.113767.asok 2026-02-25T11:47:47.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:47:47.116 DEBUG:teuthology.orchestra.run.trial028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.113767.asok status 2026-02-25T11:47:47.263 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:47:47.263 INFO:teuthology.orchestra.run.trial028.stdout: "metadata": { 2026-02-25T11:47:47.263 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_sha1": "92715cd3c42198664811edeb975bd585312154aa", 2026-02-25T11:47:47.264 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_version": "ceph version 20.3.0-5549-g92715cd3 (92715cd3c42198664811edeb975bd585312154aa) tentacle (dev - RelWithDebInfo)", 2026-02-25T11:47:47.264 INFO:teuthology.orchestra.run.trial028.stdout: "entity_id": "0", 2026-02-25T11:47:47.264 INFO:teuthology.orchestra.run.trial028.stdout: "hostname": "trial028", 2026-02-25T11:47:47.264 INFO:teuthology.orchestra.run.trial028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2026-02-25T11:47:47.264 INFO:teuthology.orchestra.run.trial028.stdout: "pid": "113767", 2026-02-25T11:47:47.264 INFO:teuthology.orchestra.run.trial028.stdout: "root": "/volumes/qa/sv_0/d31804c4-30cc-4124-85d4-92e1f982c86a" 2026-02-25T11:47:47.264 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:47:47.264 INFO:teuthology.orchestra.run.trial028.stdout: "dentry_count": 0, 2026-02-25T11:47:47.264 INFO:teuthology.orchestra.run.trial028.stdout: "dentry_pinned_count": 0, 2026-02-25T11:47:47.264 INFO:teuthology.orchestra.run.trial028.stdout: "id": 15345, 2026-02-25T11:47:47.264 INFO:teuthology.orchestra.run.trial028.stdout: "inst": { 2026-02-25T11:47:47.265 INFO:teuthology.orchestra.run.trial028.stdout: "name": { 2026-02-25T11:47:47.265 INFO:teuthology.orchestra.run.trial028.stdout: "type": "client", 2026-02-25T11:47:47.265 INFO:teuthology.orchestra.run.trial028.stdout: "num": 15345 2026-02-25T11:47:47.265 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:47:47.265 INFO:teuthology.orchestra.run.trial028.stdout: "addr": { 2026-02-25T11:47:47.265 INFO:teuthology.orchestra.run.trial028.stdout: "type": "v1", 2026-02-25T11:47:47.265 INFO:teuthology.orchestra.run.trial028.stdout: "addr": "192.168.144.1:0", 2026-02-25T11:47:47.265 INFO:teuthology.orchestra.run.trial028.stdout: "nonce": 3601514065 2026-02-25T11:47:47.265 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:47:47.265 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:47:47.265 INFO:teuthology.orchestra.run.trial028.stdout: "addr": { 2026-02-25T11:47:47.265 INFO:teuthology.orchestra.run.trial028.stdout: "type": "v1", 2026-02-25T11:47:47.266 INFO:teuthology.orchestra.run.trial028.stdout: "addr": "192.168.144.1:0", 2026-02-25T11:47:47.266 INFO:teuthology.orchestra.run.trial028.stdout: "nonce": 3601514065 2026-02-25T11:47:47.266 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:47:47.266 INFO:teuthology.orchestra.run.trial028.stdout: "inst_str": "client.15345 192.168.144.1:0/3601514065", 2026-02-25T11:47:47.266 INFO:teuthology.orchestra.run.trial028.stdout: "addr_str": "192.168.144.1:0/3601514065", 2026-02-25T11:47:47.266 INFO:teuthology.orchestra.run.trial028.stdout: "inode_count": 5, 2026-02-25T11:47:47.266 INFO:teuthology.orchestra.run.trial028.stdout: "mds_epoch": 22, 2026-02-25T11:47:47.266 INFO:teuthology.orchestra.run.trial028.stdout: "osd_epoch": 86, 2026-02-25T11:47:47.266 INFO:teuthology.orchestra.run.trial028.stdout: "osd_epoch_barrier": 86, 2026-02-25T11:47:47.266 INFO:teuthology.orchestra.run.trial028.stdout: "blocklisted": false, 2026-02-25T11:47:47.266 INFO:teuthology.orchestra.run.trial028.stdout: "fs_name": "cephfs" 2026-02-25T11:47:47.266 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:47:47.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:47:47.271 DEBUG:teuthology.orchestra.run.trial028:> stat --file-system '--printf=%T 2026-02-25T11:47:47.271 DEBUG:teuthology.orchestra.run.trial028:> ' -- /home/ubuntu/cephtest/mnt.0 2026-02-25T11:47:47.311 INFO:teuthology.orchestra.run.trial028.stdout:fuseblk 2026-02-25T11:47:47.311 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2026-02-25T11:47:47.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:47:47.312 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2026-02-25T11:47:47.410 INFO:teuthology.run_tasks:Running task exec... 2026-02-25T11:47:47.428 INFO:teuthology.task.exec:Executing custom commands... 2026-02-25T11:47:47.428 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@trial028.front.sepia.ceph.com 2026-02-25T11:47:47.428 DEBUG:teuthology.orchestra.run.trial028:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2026-02-25T11:47:47.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.542+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/3003586627 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7b4113e70 msgr2=0x7fd7b4134250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:47.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.542+0000 7fd7b8bd2640 1 --2- 10.20.193.28:0/3003586627 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7b4113e70 0x7fd7b4134250 secure :-1 s=READY pgs=339 gs=1 cs=0 l=1 c_cookie=406292998129518007 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd7a0009950 tx=0x7fd7a002f190 comp rx=0 tx=0).stop 2026-02-25T11:47:47.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.542+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/3003586627 shutdown_connections 2026-02-25T11:47:47.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.542+0000 7fd7b8bd2640 1 --2- 10.20.193.28:0/3003586627 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7b41553a0 0x7fd7b4175850 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:47.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.542+0000 7fd7b8bd2640 1 --2- 10.20.193.28:0/3003586627 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7b4134790 0x7fd7b4154c10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:47.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.542+0000 7fd7b8bd2640 1 --2- 10.20.193.28:0/3003586627 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7b4113e70 0x7fd7b4134250 unknown :-1 s=CLOSED pgs=339 gs=1 cs=0 l=1 c_cookie=406292998129518007 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:47.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.542+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/3003586627 >> 10.20.193.28:0/3003586627 conn(0x7fd7b40645b0 msgr2=0x7fd7b40649d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:47.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.542+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/3003586627 shutdown_connections 2026-02-25T11:47:47.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.542+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/3003586627 wait complete. 2026-02-25T11:47:47.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.542+0000 7fd7b8bd2640 1 Processor -- start 2026-02-25T11:47:47.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b8bd2640 1 -- start start 2026-02-25T11:47:47.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b8bd2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7b4113e70 0x7fd7b4175850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:47.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b8bd2640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7b4134790 0x7fd7b4154bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:47.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b8bd2640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7b41553a0 0x7fd7b40ffc00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:47.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b8bd2640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b4100390 con 0x7fd7b4113e70 2026-02-25T11:47:47.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b8bd2640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b4100500 con 0x7fd7b41553a0 2026-02-25T11:47:47.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b8bd2640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b4100b80 con 0x7fd7b4134790 2026-02-25T11:47:47.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b37fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7b4113e70 0x7fd7b4175850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:47.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b37fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7b4113e70 0x7fd7b4175850 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59408/0 (socket says 10.20.193.28:59408) 2026-02-25T11:47:47.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b37fe640 1 -- 10.20.193.28:0/1649722977 learned_addr learned my addr 10.20.193.28:0/1649722977 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:47.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b2ffd640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7b4134790 0x7fd7b4154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:47.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b3fff640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7b41553a0 0x7fd7b40ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:47.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b37fe640 1 -- 10.20.193.28:0/1649722977 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7b4134790 msgr2=0x7fd7b4154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:47.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b37fe640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7b4134790 0x7fd7b4154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:47.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b37fe640 1 -- 10.20.193.28:0/1649722977 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7b41553a0 msgr2=0x7fd7b40ffc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:47.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b37fe640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7b41553a0 0x7fd7b40ffc00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:47.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b37fe640 1 -- 10.20.193.28:0/1649722977 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a0009570 con 0x7fd7b4113e70 2026-02-25T11:47:47.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b2ffd640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7b4134790 0x7fd7b4154bc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:47.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b3fff640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7b41553a0 0x7fd7b40ffc00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:47.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.543+0000 7fd7b37fe640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7b4113e70 0x7fd7b4175850 secure :-1 s=READY pgs=340 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd7a00023b0 tx=0x7fd7a00028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:47.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.544+0000 7fd7b0ff9640 1 -- 10.20.193.28:0/1649722977 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd7a003d070 con 0x7fd7b4113e70 2026-02-25T11:47:47.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.544+0000 7fd7b0ff9640 1 -- 10.20.193.28:0/1649722977 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd7a002fe20 con 0x7fd7b4113e70 2026-02-25T11:47:47.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.544+0000 7fd7b0ff9640 1 -- 10.20.193.28:0/1649722977 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd7a0041a40 con 0x7fd7b4113e70 2026-02-25T11:47:47.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.544+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/1649722977 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b4100da0 con 0x7fd7b4113e70 2026-02-25T11:47:47.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.544+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/1649722977 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b41011c0 con 0x7fd7b4113e70 2026-02-25T11:47:47.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.544+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/1649722977 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b405f4f0 con 0x7fd7b4113e70 2026-02-25T11:47:47.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.546+0000 7fd7b0ff9640 1 -- 10.20.193.28:0/1649722977 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fd7a0041bd0 con 0x7fd7b4113e70 2026-02-25T11:47:47.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.548+0000 7fd7b0ff9640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd7880b5770 0x7fd7880d5c20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:47.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.548+0000 7fd7b0ff9640 1 -- 10.20.193.28:0/1649722977 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fd7a010b9f0 con 0x7fd7b4113e70 2026-02-25T11:47:47.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.548+0000 7fd7b0ff9640 1 -- 10.20.193.28:0/1649722977 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fd7a0144940 con 0x7fd7b4113e70 2026-02-25T11:47:47.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.548+0000 7fd7b2ffd640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd7880b5770 0x7fd7880d5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:47.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.549+0000 7fd7b2ffd640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd7880b5770 0x7fd7880d5c20 secure :-1 s=READY pgs=242 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd7a4005970 tx=0x7fd7a4005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:47.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:47.691+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/1649722977 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fd7b4179bf0 con 0x7fd7b4113e70 2026-02-25T11:47:48.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.171+0000 7fd7b0ff9640 1 -- 10.20.193.28:0/1649722977 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v23) ==== 117+0+0 (secure 0 0 0) 0x7fd7a00ce450 con 0x7fd7b4113e70 2026-02-25T11:47:48.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.174+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/1649722977 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd7880b5770 msgr2=0x7fd7880d5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.174+0000 7fd7b8bd2640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd7880b5770 0x7fd7880d5c20 secure :-1 s=READY pgs=242 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd7a4005970 tx=0x7fd7a4005900 comp rx=0 tx=0).stop 2026-02-25T11:47:48.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.174+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/1649722977 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7b4113e70 msgr2=0x7fd7b4175850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.174+0000 7fd7b8bd2640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7b4113e70 0x7fd7b4175850 secure :-1 s=READY pgs=340 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd7a00023b0 tx=0x7fd7a00028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.174+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/1649722977 shutdown_connections 2026-02-25T11:47:48.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.174+0000 7fd7b8bd2640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd7880b5770 0x7fd7880d5c20 unknown :-1 s=CLOSED pgs=242 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.174+0000 7fd7b8bd2640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7b41553a0 0x7fd7b40ffc00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.174+0000 7fd7b8bd2640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7b4134790 0x7fd7b4154bc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.174+0000 7fd7b8bd2640 1 --2- 10.20.193.28:0/1649722977 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7b4113e70 0x7fd7b4175850 unknown :-1 s=CLOSED pgs=340 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.174+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/1649722977 >> 10.20.193.28:0/1649722977 conn(0x7fd7b40645b0 msgr2=0x7fd7b40f64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:48.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.174+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/1649722977 shutdown_connections 2026-02-25T11:47:48.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.174+0000 7fd7b8bd2640 1 -- 10.20.193.28:0/1649722977 wait complete. 2026-02-25T11:47:48.190 INFO:teuthology.run_tasks:Running task check-counter... 2026-02-25T11:47:48.206 INFO:teuthology.run_tasks:Running task fwd_scrub... 2026-02-25T11:47:48.218 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T11:47:48.337 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.335+0000 7faebebad640 1 -- 10.20.193.28:0/2657608048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faeb810e9d0 msgr2=0x7faeb816f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.335+0000 7faebebad640 1 --2- 10.20.193.28:0/2657608048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faeb810e9d0 0x7faeb816f360 secure :-1 s=READY pgs=341 gs=3 cs=0 l=1 c_cookie=10076019542397091389 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faeb400a970 tx=0x7faeb40301b0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.335+0000 7faebebad640 1 -- 10.20.193.28:0/2657608048 shutdown_connections 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.335+0000 7faebebad640 1 --2- 10.20.193.28:0/2657608048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faeb810e9d0 0x7faeb816f360 unknown :-1 s=CLOSED pgs=341 gs=3 cs=0 l=1 c_cookie=10076019542397091389 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.335+0000 7faebebad640 1 --2- 10.20.193.28:0/2657608048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7faeb810e020 0x7faeb810e490 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.335+0000 7faebebad640 1 --2- 10.20.193.28:0/2657608048 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faeb810ce90 0x7faeb810d280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.335+0000 7faebebad640 1 -- 10.20.193.28:0/2657608048 >> 10.20.193.28:0/2657608048 conn(0x7faeb80ea5c0 msgr2=0x7faeb810a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.335+0000 7faebebad640 1 -- 10.20.193.28:0/2657608048 shutdown_connections 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.335+0000 7faebebad640 1 -- 10.20.193.28:0/2657608048 wait complete. 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.336+0000 7faebebad640 1 Processor -- start 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.336+0000 7faebebad640 1 -- start start 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.336+0000 7faebebad640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faeb810ce90 0x7faeb8061df0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.336+0000 7faebebad640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7faeb810e020 0x7faeb8062440 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.336+0000 7faebebad640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faeb810e9d0 0x7faeb8060700 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.336+0000 7faebebad640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7faeb8060c40 con 0x7faeb810ce90 2026-02-25T11:47:48.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.336+0000 7faebebad640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7faeb8060db0 con 0x7faeb810e020 2026-02-25T11:47:48.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.336+0000 7faebebad640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7faeb8060f70 con 0x7faeb810e9d0 2026-02-25T11:47:48.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.336+0000 7faebdbab640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faeb810ce90 0x7faeb8061df0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:48.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.336+0000 7faebdbab640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faeb810ce90 0x7faeb8061df0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59422/0 (socket says 10.20.193.28:59422) 2026-02-25T11:47:48.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.336+0000 7faebdbab640 1 -- 10.20.193.28:0/2164803104 learned_addr learned my addr 10.20.193.28:0/2164803104 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:48.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.336+0000 7faebe3ac640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faeb810e9d0 0x7faeb8060700 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:48.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.336+0000 7faebd3aa640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7faeb810e020 0x7faeb8062440 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:48.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.337+0000 7faebdbab640 1 -- 10.20.193.28:0/2164803104 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faeb810e9d0 msgr2=0x7faeb8060700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.337+0000 7faebdbab640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faeb810e9d0 0x7faeb8060700 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.337+0000 7faebdbab640 1 -- 10.20.193.28:0/2164803104 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7faeb810e020 msgr2=0x7faeb8062440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.337+0000 7faebdbab640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7faeb810e020 0x7faeb8062440 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.337+0000 7faebdbab640 1 -- 10.20.193.28:0/2164803104 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac009570 con 0x7faeb810ce90 2026-02-25T11:47:48.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.337+0000 7faebe3ac640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faeb810e9d0 0x7faeb8060700 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:48.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.337+0000 7faebdbab640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faeb810ce90 0x7faeb8061df0 secure :-1 s=READY pgs=342 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faea8009480 tx=0x7faea800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:48.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.337+0000 7faea6ffd640 1 -- 10.20.193.28:0/2164803104 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7faea800cd40 con 0x7faeb810ce90 2026-02-25T11:47:48.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.337+0000 7faea6ffd640 1 -- 10.20.193.28:0/2164803104 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7faea8004d10 con 0x7faeb810ce90 2026-02-25T11:47:48.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.337+0000 7faebebad640 1 -- 10.20.193.28:0/2164803104 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb400a590 con 0x7faeb810ce90 2026-02-25T11:47:48.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.337+0000 7faea6ffd640 1 -- 10.20.193.28:0/2164803104 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7faea80106b0 con 0x7faeb810ce90 2026-02-25T11:47:48.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.337+0000 7faebebad640 1 -- 10.20.193.28:0/2164803104 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb8061540 con 0x7faeb810ce90 2026-02-25T11:47:48.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.338+0000 7faebebad640 1 -- 10.20.193.28:0/2164803104 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb8173280 con 0x7faeb810ce90 2026-02-25T11:47:48.342 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.340+0000 7faea6ffd640 1 -- 10.20.193.28:0/2164803104 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7faea801f060 con 0x7faeb810ce90 2026-02-25T11:47:48.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.341+0000 7faea6ffd640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fae940b57f0 0x7fae940d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:48.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.342+0000 7faea6ffd640 1 -- 10.20.193.28:0/2164803104 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7faea8014070 con 0x7faeb810ce90 2026-02-25T11:47:48.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.342+0000 7faebd3aa640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fae940b57f0 0x7fae940d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:48.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.342+0000 7faea6ffd640 1 -- 10.20.193.28:0/2164803104 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7faea80e7290 con 0x7faeb810ce90 2026-02-25T11:47:48.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.342+0000 7faebd3aa640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fae940b57f0 0x7fae940d5ca0 secure :-1 s=READY pgs=243 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faeac005b30 tx=0x7faeac009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:48.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.456+0000 7faebebad640 1 -- 10.20.193.28:0/2164803104 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faeb805ed00 con 0x7faeb810ce90 2026-02-25T11:47:48.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.457+0000 7faea6ffd640 1 -- 10.20.193.28:0/2164803104 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22340 (secure 0 0 0) 0x7faea80a9920 con 0x7faeb810ce90 2026-02-25T11:47:48.460 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:48.460 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":86,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","created":"2026-02-25T11:42:51.962844+0000","modified":"2026-02-25T11:47:14.624486+0000","last_up_change":"2026-02-25T11:46:45.156629+0000","last_in_change":"2026-02-25T11:46:31.585002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":38,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T11:44:43.266336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-25T11:47:11.236575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-25T11:47:12.196775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf596df7-d1fb-4d10-8d4c-14a35210bab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6803","nonce":668944205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6805","nonce":668944205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6809","nonce":668944205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6807","nonce":668944205}]},"public_addr":"10.20.193.28:6803/668944205","cluster_addr":"10.20.193.28:6805/668944205","heartbeat_back_addr":"10.20.193.28:6809/668944205","heartbeat_front_addr":"10.20.193.28:6807/668944205","state":["exists","up"]},{"osd":1,"uuid":"185719c0-7c2d-431f-9925-88bbee86a98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6800","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6801","nonce":3486566242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6802","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6803","nonce":3486566242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6806","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6807","nonce":3486566242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6804","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6805","nonce":3486566242}]},"public_addr":"10.20.193.119:6801/3486566242","cluster_addr":"10.20.193.119:6803/3486566242","heartbeat_back_addr":"10.20.193.119:6807/3486566242","heartbeat_front_addr":"10.20.193.119:6805/3486566242","state":["exists","up"]},{"osd":2,"uuid":"4de13b90-1c3c-4614-9fef-2d7315626433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6800","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6801","nonce":484781559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6802","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6803","nonce":484781559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6806","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6807","nonce":484781559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6804","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6805","nonce":484781559}]},"public_addr":"10.20.193.167:6801/484781559","cluster_addr":"10.20.193.167:6803/484781559","heartbeat_back_addr":"10.20.193.167:6807/484781559","heartbeat_front_addr":"10.20.193.167:6805/484781559","state":["exists","up"]},{"osd":3,"uuid":"45d9ea8e-5a77-4738-96da-ada98515fc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6811","nonce":2452541436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6813","nonce":2452541436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6817","nonce":2452541436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6815","nonce":2452541436}]},"public_addr":"10.20.193.28:6811/2452541436","cluster_addr":"10.20.193.28:6813/2452541436","heartbeat_back_addr":"10.20.193.28:6817/2452541436","heartbeat_front_addr":"10.20.193.28:6815/2452541436","state":["exists","up"]},{"osd":4,"uuid":"c9c3f512-665b-44b2-a6f0-f3356a4b661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6808","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6809","nonce":135971996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6810","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6811","nonce":135971996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6814","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6815","nonce":135971996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6812","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6813","nonce":135971996}]},"public_addr":"10.20.193.119:6809/135971996","cluster_addr":"10.20.193.119:6811/135971996","heartbeat_back_addr":"10.20.193.119:6815/135971996","heartbeat_front_addr":"10.20.193.119:6813/135971996","state":["exists","up"]},{"osd":5,"uuid":"777c3819-5c90-4b2b-a545-c024884c2eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6808","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6809","nonce":4203420884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6810","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6811","nonce":4203420884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6814","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6815","nonce":4203420884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6812","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6813","nonce":4203420884}]},"public_addr":"10.20.193.167:6809/4203420884","cluster_addr":"10.20.193.167:6811/4203420884","heartbeat_back_addr":"10.20.193.167:6815/4203420884","heartbeat_front_addr":"10.20.193.167:6813/4203420884","state":["exists","up"]},{"osd":6,"uuid":"bbd9de64-b9fe-499e-b451-c16828be2c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6819","nonce":621484999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6821","nonce":621484999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6825","nonce":621484999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6823","nonce":621484999}]},"public_addr":"10.20.193.28:6819/621484999","cluster_addr":"10.20.193.28:6821/621484999","heartbeat_back_addr":"10.20.193.28:6825/621484999","heartbeat_front_addr":"10.20.193.28:6823/621484999","state":["exists","up"]},{"osd":7,"uuid":"546d9f28-597b-454c-8456-fcc2d46c69f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6816","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6817","nonce":2724541747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6818","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6819","nonce":2724541747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6822","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6823","nonce":2724541747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6820","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6821","nonce":2724541747}]},"public_addr":"10.20.193.119:6817/2724541747","cluster_addr":"10.20.193.119:6819/2724541747","heartbeat_back_addr":"10.20.193.119:6823/2724541747","heartbeat_front_addr":"10.20.193.119:6821/2724541747","state":["exists","up"]},{"osd":8,"uuid":"a1a7cea6-3586-4bb2-aa4c-bf6f7799a501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6816","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6817","nonce":1328734317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6818","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6819","nonce":1328734317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6822","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6823","nonce":1328734317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6820","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6821","nonce":1328734317}]},"public_addr":"10.20.193.167:6817/1328734317","cluster_addr":"10.20.193.167:6819/1328734317","heartbeat_back_addr":"10.20.193.167:6823/1328734317","heartbeat_front_addr":"10.20.193.167:6821/1328734317","state":["exists","up"]},{"osd":9,"uuid":"03fef134-6946-43d5-9c26-dc0e59fffe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6827","nonce":354500625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6829","nonce":354500625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6833","nonce":354500625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6831","nonce":354500625}]},"public_addr":"10.20.193.28:6827/354500625","cluster_addr":"10.20.193.28:6829/354500625","heartbeat_back_addr":"10.20.193.28:6833/354500625","heartbeat_front_addr":"10.20.193.28:6831/354500625","state":["exists","up"]},{"osd":10,"uuid":"b46c29ef-285c-4184-be28-d99b76bc0e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6824","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6825","nonce":2996581116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6826","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6827","nonce":2996581116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6830","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6831","nonce":2996581116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6828","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6829","nonce":2996581116}]},"public_addr":"10.20.193.119:6825/2996581116","cluster_addr":"10.20.193.119:6827/2996581116","heartbeat_back_addr":"10.20.193.119:6831/2996581116","heartbeat_front_addr":"10.20.193.119:6829/2996581116","state":["exists","up"]},{"osd":11,"uuid":"f6696d8c-5de7-4fd9-b693-524f204e4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6824","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6825","nonce":2182906700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6826","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6827","nonce":2182906700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6830","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6831","nonce":2182906700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6828","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6829","nonce":2182906700}]},"public_addr":"10.20.193.167:6825/2182906700","cluster_addr":"10.20.193.167:6827/2182906700","heartbeat_back_addr":"10.20.193.167:6831/2182906700","heartbeat_front_addr":"10.20.193.167:6829/2182906700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:13.410235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:27.254773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:40.956615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:54.529866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:08.306772+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:21.899498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:35.500407+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:49.094965+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:02.790138+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:16.286937+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:29.922722+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:43.396028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.28:0/1467902196":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6800/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/2274270913":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/60087422":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:6800/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/1725332841":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/3082630924":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/2391715078":"2026-02-26T11:43:04.102029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T11:47:48.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.458+0000 7faebebad640 1 -- 10.20.193.28:0/2164803104 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fae940b57f0 msgr2=0x7fae940d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.458+0000 7faebebad640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fae940b57f0 0x7fae940d5ca0 secure :-1 s=READY pgs=243 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faeac005b30 tx=0x7faeac009310 comp rx=0 tx=0).stop 2026-02-25T11:47:48.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.458+0000 7faebebad640 1 -- 10.20.193.28:0/2164803104 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faeb810ce90 msgr2=0x7faeb8061df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.458+0000 7faebebad640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faeb810ce90 0x7faeb8061df0 secure :-1 s=READY pgs=342 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7faea8009480 tx=0x7faea800ee60 comp rx=0 tx=0).stop 2026-02-25T11:47:48.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.458+0000 7faebebad640 1 -- 10.20.193.28:0/2164803104 shutdown_connections 2026-02-25T11:47:48.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.458+0000 7faebebad640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fae940b57f0 0x7fae940d5ca0 unknown :-1 s=CLOSED pgs=243 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.458+0000 7faebebad640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7faeb810e9d0 0x7faeb8060700 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.458+0000 7faebebad640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7faeb810e020 0x7faeb8062440 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.458+0000 7faebebad640 1 --2- 10.20.193.28:0/2164803104 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7faeb810ce90 0x7faeb8061df0 unknown :-1 s=CLOSED pgs=342 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.458+0000 7faebebad640 1 -- 10.20.193.28:0/2164803104 >> 10.20.193.28:0/2164803104 conn(0x7faeb80ea5c0 msgr2=0x7faeb80ecb90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:48.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.458+0000 7faebebad640 1 -- 10.20.193.28:0/2164803104 shutdown_connections 2026-02-25T11:47:48.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.458+0000 7faebebad640 1 -- 10.20.193.28:0/2164803104 wait complete. 2026-02-25T11:47:48.470 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:48 trial028 ceph-mon[39224]: pgmap v211: 97 pgs: 97 active+clean; 648 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 7 op/s 2026-02-25T11:47:48.470 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:48 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1649722977' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2026-02-25T11:47:48.473 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2026-02-25T11:44:43.266336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2026-02-25T11:47:11.236575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2026-02-25T11:47:12.196775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2026-02-25T11:47:48.473 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2026-02-25T11:47:48.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:48 trial167 ceph-mon[43392]: pgmap v211: 97 pgs: 97 active+clean; 648 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 7 op/s 2026-02-25T11:47:48.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:48 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1649722977' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2026-02-25T11:47:48.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:48 trial119 ceph-mon[42710]: pgmap v211: 97 pgs: 97 active+clean; 648 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.0 KiB/s rd, 853 B/s wr, 7 op/s 2026-02-25T11:47:48.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:48 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1649722977' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2026-02-25T11:47:48.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.591+0000 7f90c7fff640 1 -- 10.20.193.28:0/462064561 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90c8115180 msgr2=0x7f90c8175b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.591+0000 7f90c7fff640 1 --2- 10.20.193.28:0/462064561 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90c8115180 0x7f90c8175b10 secure :-1 s=READY pgs=108 gs=2 cs=0 l=1 c_cookie=11274377746163152812 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f90b800aa30 tx=0x7f90b8030270 comp rx=0 tx=0).stop 2026-02-25T11:47:48.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.591+0000 7f90c7fff640 1 -- 10.20.193.28:0/462064561 shutdown_connections 2026-02-25T11:47:48.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.591+0000 7f90c7fff640 1 --2- 10.20.193.28:0/462064561 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90c8115180 0x7f90c8175b10 unknown :-1 s=CLOSED pgs=108 gs=2 cs=0 l=1 c_cookie=11274377746163152812 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.591+0000 7f90c7fff640 1 --2- 10.20.193.28:0/462064561 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90c81147d0 0x7f90c8114c40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.591+0000 7f90c7fff640 1 --2- 10.20.193.28:0/462064561 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f90c81138f0 0x7f90c8113ce0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.591+0000 7f90c7fff640 1 -- 10.20.193.28:0/462064561 >> 10.20.193.28:0/462064561 conn(0x7f90c8064570 msgr2=0x7f90c8064970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:48.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.591+0000 7f90c7fff640 1 -- 10.20.193.28:0/462064561 shutdown_connections 2026-02-25T11:47:48.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.591+0000 7f90c7fff640 1 -- 10.20.193.28:0/462064561 wait complete. 2026-02-25T11:47:48.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c7fff640 1 Processor -- start 2026-02-25T11:47:48.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c7fff640 1 -- start start 2026-02-25T11:47:48.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c7fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f90c81138f0 0x7f90c80feb40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:48.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c7fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90c81147d0 0x7f90c8175b10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:48.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c7fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90c8115180 0x7f90c80ff3a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:48.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c7fff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f90c80fff40 con 0x7f90c8115180 2026-02-25T11:47:48.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c7fff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f90c81000b0 con 0x7f90c81138f0 2026-02-25T11:47:48.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c7fff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f90c8100270 con 0x7f90c81147d0 2026-02-25T11:47:48.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c77fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90c8115180 0x7f90c80ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:48.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c77fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90c8115180 0x7f90c80ff3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59444/0 (socket says 10.20.193.28:59444) 2026-02-25T11:47:48.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c77fe640 1 -- 10.20.193.28:0/13315615 learned_addr learned my addr 10.20.193.28:0/13315615 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:48.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c6ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f90c81138f0 0x7f90c80feb40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:48.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c67fc640 1 --2- 10.20.193.28:0/13315615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90c81147d0 0x7f90c8175b10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:48.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c77fe640 1 -- 10.20.193.28:0/13315615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90c81147d0 msgr2=0x7f90c8175b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c77fe640 1 --2- 10.20.193.28:0/13315615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90c81147d0 0x7f90c8175b10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c77fe640 1 -- 10.20.193.28:0/13315615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f90c81138f0 msgr2=0x7f90c80feb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c77fe640 1 --2- 10.20.193.28:0/13315615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f90c81138f0 0x7f90c80feb40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.592+0000 7f90c77fe640 1 -- 10.20.193.28:0/13315615 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc0094b0 con 0x7f90c8115180 2026-02-25T11:47:48.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.593+0000 7f90c77fe640 1 --2- 10.20.193.28:0/13315615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90c8115180 0x7f90c80ff3a0 secure :-1 s=READY pgs=343 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f90b8032fe0 tx=0x7f90b8030d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:48.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.593+0000 7f90a7fff640 1 -- 10.20.193.28:0/13315615 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f90b803e070 con 0x7f90c8115180 2026-02-25T11:47:48.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.593+0000 7f90a7fff640 1 -- 10.20.193.28:0/13315615 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f90b80050c0 con 0x7f90c8115180 2026-02-25T11:47:48.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.593+0000 7f90c7fff640 1 -- 10.20.193.28:0/13315615 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b800a650 con 0x7f90c8115180 2026-02-25T11:47:48.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.593+0000 7f90a7fff640 1 -- 10.20.193.28:0/13315615 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f90b8042a30 con 0x7f90c8115180 2026-02-25T11:47:48.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.593+0000 7f90c7fff640 1 -- 10.20.193.28:0/13315615 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c8100780 con 0x7f90c8115180 2026-02-25T11:47:48.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.593+0000 7f90c7fff640 1 -- 10.20.193.28:0/13315615 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90c8179970 con 0x7f90c8115180 2026-02-25T11:47:48.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.596+0000 7f90a7fff640 1 -- 10.20.193.28:0/13315615 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f90b8005230 con 0x7f90c8115180 2026-02-25T11:47:48.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.597+0000 7f90a7fff640 1 --2- 10.20.193.28:0/13315615 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f909c0b5770 0x7f909c0d5c20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:48.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.598+0000 7f90a7fff640 1 -- 10.20.193.28:0/13315615 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f90b810cb00 con 0x7f90c8115180 2026-02-25T11:47:48.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.598+0000 7f90a7fff640 1 -- 10.20.193.28:0/13315615 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f90b8145940 con 0x7f90c8115180 2026-02-25T11:47:48.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.598+0000 7f90c6ffd640 1 --2- 10.20.193.28:0/13315615 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f909c0b5770 0x7f909c0d5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:48.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.598+0000 7f90c6ffd640 1 --2- 10.20.193.28:0/13315615 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f909c0b5770 0x7f909c0d5c20 secure :-1 s=READY pgs=244 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f90bc004740 tx=0x7f90bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:48.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.711+0000 7f90c7fff640 1 -- 10.20.193.28:0/13315615 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f90c8100430 con 0x7f90c8115180 2026-02-25T11:47:48.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.711+0000 7f90a7fff640 1 -- 10.20.193.28:0/13315615 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) ==== 93+0+10 (secure 0 0 0) 0x7f90b80cf560 con 0x7f90c8115180 2026-02-25T11:47:48.714 INFO:teuthology.orchestra.run.trial028.stdout:pg_num: 1 2026-02-25T11:47:48.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.712+0000 7f90c7fff640 1 -- 10.20.193.28:0/13315615 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f909c0b5770 msgr2=0x7f909c0d5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.712+0000 7f90c7fff640 1 --2- 10.20.193.28:0/13315615 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f909c0b5770 0x7f909c0d5c20 secure :-1 s=READY pgs=244 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f90bc004740 tx=0x7f90bc005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:48.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.712+0000 7f90c7fff640 1 -- 10.20.193.28:0/13315615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90c8115180 msgr2=0x7f90c80ff3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.712+0000 7f90c7fff640 1 --2- 10.20.193.28:0/13315615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90c8115180 0x7f90c80ff3a0 secure :-1 s=READY pgs=343 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f90b8032fe0 tx=0x7f90b8030d70 comp rx=0 tx=0).stop 2026-02-25T11:47:48.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.712+0000 7f90c7fff640 1 -- 10.20.193.28:0/13315615 shutdown_connections 2026-02-25T11:47:48.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.712+0000 7f90c7fff640 1 --2- 10.20.193.28:0/13315615 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f909c0b5770 0x7f909c0d5c20 unknown :-1 s=CLOSED pgs=244 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.712+0000 7f90c7fff640 1 --2- 10.20.193.28:0/13315615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90c8115180 0x7f90c80ff3a0 unknown :-1 s=CLOSED pgs=343 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.712+0000 7f90c7fff640 1 --2- 10.20.193.28:0/13315615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90c81147d0 0x7f90c8175b10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.712+0000 7f90c7fff640 1 --2- 10.20.193.28:0/13315615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f90c81138f0 0x7f90c80feb40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.712+0000 7f90c7fff640 1 -- 10.20.193.28:0/13315615 >> 10.20.193.28:0/13315615 conn(0x7f90c8064570 msgr2=0x7f90c80f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:48.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.712+0000 7f90c7fff640 1 -- 10.20.193.28:0/13315615 shutdown_connections 2026-02-25T11:47:48.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.712+0000 7f90c7fff640 1 -- 10.20.193.28:0/13315615 wait complete. 2026-02-25T11:47:48.726 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2026-02-25T11:47:48.847 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.844+0000 7f42d3577640 1 -- 10.20.193.28:0/3539074620 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42d410bde0 msgr2=0x7f42d410c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.844+0000 7f42d3577640 1 --2- 10.20.193.28:0/3539074620 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42d410bde0 0x7f42d410c250 secure :-1 s=READY pgs=344 gs=1 cs=0 l=1 c_cookie=1098429591685728380 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42bc009950 tx=0x7f42bc02f190 comp rx=0 tx=0).stop 2026-02-25T11:47:48.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.844+0000 7f42d3577640 1 -- 10.20.193.28:0/3539074620 shutdown_connections 2026-02-25T11:47:48.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.844+0000 7f42d3577640 1 --2- 10.20.193.28:0/3539074620 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f42d410c790 0x7f42d410cc30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.844+0000 7f42d3577640 1 --2- 10.20.193.28:0/3539074620 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42d410bde0 0x7f42d410c250 unknown :-1 s=CLOSED pgs=344 gs=1 cs=0 l=1 c_cookie=1098429591685728380 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.844+0000 7f42d3577640 1 --2- 10.20.193.28:0/3539074620 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f42d4060020 0x7f42d410b8a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.844+0000 7f42d3577640 1 -- 10.20.193.28:0/3539074620 >> 10.20.193.28:0/3539074620 conn(0x7f42d40e94a0 msgr2=0x7f42d41098b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:48.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.844+0000 7f42d3577640 1 -- 10.20.193.28:0/3539074620 shutdown_connections 2026-02-25T11:47:48.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.844+0000 7f42d3577640 1 -- 10.20.193.28:0/3539074620 wait complete. 2026-02-25T11:47:48.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.845+0000 7f42d3577640 1 Processor -- start 2026-02-25T11:47:48.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.845+0000 7f42d3577640 1 -- start start 2026-02-25T11:47:48.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.845+0000 7f42d3577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42d4060020 0x7f42d40f4f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:48.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.845+0000 7f42d3577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f42d410bde0 0x7f42d40f5570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:48.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.845+0000 7f42d3577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f42d410c790 0x7f42d40f6170 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:48.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.845+0000 7f42d3577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f42d40f68b0 con 0x7f42d4060020 2026-02-25T11:47:48.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.845+0000 7f42d3577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f42d40f6a20 con 0x7f42d410bde0 2026-02-25T11:47:48.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.845+0000 7f42d3577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42d40f6be0 con 0x7f42d410c790 2026-02-25T11:47:48.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42d2575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42d4060020 0x7f42d40f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:48.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42d2575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42d4060020 0x7f42d40f4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59452/0 (socket says 10.20.193.28:59452) 2026-02-25T11:47:48.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42d2575640 1 -- 10.20.193.28:0/3210366713 learned_addr learned my addr 10.20.193.28:0/3210366713 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:48.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42d1d74640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f42d410bde0 0x7f42d40f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:48.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42d2d76640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f42d410c790 0x7f42d40f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:48.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42d2575640 1 -- 10.20.193.28:0/3210366713 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f42d410c790 msgr2=0x7f42d40f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42d2575640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f42d410c790 0x7f42d40f6170 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42d2575640 1 -- 10.20.193.28:0/3210366713 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f42d410bde0 msgr2=0x7f42d40f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42d2575640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f42d410bde0 0x7f42d40f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42d2575640 1 -- 10.20.193.28:0/3210366713 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc009570 con 0x7f42d4060020 2026-02-25T11:47:48.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42d2575640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42d4060020 0x7f42d40f4f20 secure :-1 s=READY pgs=345 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42c800d950 tx=0x7f42c800de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:48.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42bb7fe640 1 -- 10.20.193.28:0/3210366713 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f42c8004540 con 0x7f42d4060020 2026-02-25T11:47:48.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42bb7fe640 1 -- 10.20.193.28:0/3210366713 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f42c8009ce0 con 0x7f42d4060020 2026-02-25T11:47:48.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42d3577640 1 -- 10.20.193.28:0/3210366713 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d40f6ec0 con 0x7f42d4060020 2026-02-25T11:47:48.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42d3577640 1 -- 10.20.193.28:0/3210366713 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d40f7430 con 0x7f42d4060020 2026-02-25T11:47:48.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.846+0000 7f42bb7fe640 1 -- 10.20.193.28:0/3210366713 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f42c80106e0 con 0x7f42d4060020 2026-02-25T11:47:48.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.847+0000 7f42d3577640 1 -- 10.20.193.28:0/3210366713 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d41754f0 con 0x7f42d4060020 2026-02-25T11:47:48.853 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.849+0000 7f42bb7fe640 1 -- 10.20.193.28:0/3210366713 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f42c80097c0 con 0x7f42d4060020 2026-02-25T11:47:48.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.851+0000 7f42bb7fe640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42ac0b5770 0x7f42ac0d5c20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:48.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.851+0000 7f42bb7fe640 1 -- 10.20.193.28:0/3210366713 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f42c80e71b0 con 0x7f42d4060020 2026-02-25T11:47:48.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.851+0000 7f42bb7fe640 1 -- 10.20.193.28:0/3210366713 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f42c80e7630 con 0x7f42d4060020 2026-02-25T11:47:48.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.851+0000 7f42d1d74640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42ac0b5770 0x7f42ac0d5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:48.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.852+0000 7f42d1d74640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42ac0b5770 0x7f42ac0d5c20 secure :-1 s=READY pgs=245 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42bc0098a0 tx=0x7f42bc03a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:48.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.965+0000 7f42d3577640 1 -- 10.20.193.28:0/3210366713 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f42d410b8a0 con 0x7f42d4060020 2026-02-25T11:47:48.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.965+0000 7f42bb7fe640 1 -- 10.20.193.28:0/3210366713 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) ==== 104+0+11 (secure 0 0 0) 0x7f42c80a9c10 con 0x7f42d4060020 2026-02-25T11:47:48.967 INFO:teuthology.orchestra.run.trial028.stdout:pg_num: 32 2026-02-25T11:47:48.968 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.966+0000 7f42d3577640 1 -- 10.20.193.28:0/3210366713 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42ac0b5770 msgr2=0x7f42ac0d5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.968 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.966+0000 7f42d3577640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42ac0b5770 0x7f42ac0d5c20 secure :-1 s=READY pgs=245 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42bc0098a0 tx=0x7f42bc03a040 comp rx=0 tx=0).stop 2026-02-25T11:47:48.968 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.966+0000 7f42d3577640 1 -- 10.20.193.28:0/3210366713 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42d4060020 msgr2=0x7f42d40f4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:48.968 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.966+0000 7f42d3577640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42d4060020 0x7f42d40f4f20 secure :-1 s=READY pgs=345 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42c800d950 tx=0x7f42c800de20 comp rx=0 tx=0).stop 2026-02-25T11:47:48.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.966+0000 7f42d3577640 1 -- 10.20.193.28:0/3210366713 shutdown_connections 2026-02-25T11:47:48.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.966+0000 7f42d3577640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42ac0b5770 0x7f42ac0d5c20 unknown :-1 s=CLOSED pgs=245 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.966+0000 7f42d3577640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f42d410c790 0x7f42d40f6170 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.966+0000 7f42d3577640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f42d410bde0 0x7f42d40f5570 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.966+0000 7f42d3577640 1 --2- 10.20.193.28:0/3210366713 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42d4060020 0x7f42d40f4f20 unknown :-1 s=CLOSED pgs=345 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:48.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.966+0000 7f42d3577640 1 -- 10.20.193.28:0/3210366713 >> 10.20.193.28:0/3210366713 conn(0x7f42d40e94a0 msgr2=0x7f42d40eb150 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:48.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.966+0000 7f42d3577640 1 -- 10.20.193.28:0/3210366713 shutdown_connections 2026-02-25T11:47:48.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:48.966+0000 7f42d3577640 1 -- 10.20.193.28:0/3210366713 wait complete. 2026-02-25T11:47:48.980 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2026-02-25T11:47:49.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.098+0000 7f42dbfff640 1 -- 10.20.193.28:0/4271322866 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42dc110c40 msgr2=0x7f42dc1715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:49.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.098+0000 7f42dbfff640 1 --2- 10.20.193.28:0/4271322866 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42dc110c40 0x7f42dc1715d0 secure :-1 s=READY pgs=346 gs=2 cs=0 l=1 c_cookie=842840437912844709 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42cc00aa30 tx=0x7f42cc030270 comp rx=0 tx=0).stop 2026-02-25T11:47:49.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.098+0000 7f42dbfff640 1 -- 10.20.193.28:0/4271322866 shutdown_connections 2026-02-25T11:47:49.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.098+0000 7f42dbfff640 1 --2- 10.20.193.28:0/4271322866 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42dc110c40 0x7f42dc1715d0 unknown :-1 s=CLOSED pgs=346 gs=2 cs=0 l=1 c_cookie=842840437912844709 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.098+0000 7f42dbfff640 1 --2- 10.20.193.28:0/4271322866 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f42dc110290 0x7f42dc110700 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.098+0000 7f42dbfff640 1 --2- 10.20.193.28:0/4271322866 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f42dc10f100 0x7f42dc10f4f0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.098+0000 7f42dbfff640 1 -- 10.20.193.28:0/4271322866 >> 10.20.193.28:0/4271322866 conn(0x7f42dc0ec830 msgr2=0x7f42dc10cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:49.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.098+0000 7f42dbfff640 1 -- 10.20.193.28:0/4271322866 shutdown_connections 2026-02-25T11:47:49.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.098+0000 7f42dbfff640 1 -- 10.20.193.28:0/4271322866 wait complete. 2026-02-25T11:47:49.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.098+0000 7f42dbfff640 1 Processor -- start 2026-02-25T11:47:49.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42dbfff640 1 -- start start 2026-02-25T11:47:49.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42dbfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f42dc10f100 0x7f42dc0f8800 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:49.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42dbfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42dc110290 0x7f42dc1715d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:49.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42dbfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f42dc110c40 0x7f42dc0f9060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:49.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42dbfff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc0f9c50 con 0x7f42dc110290 2026-02-25T11:47:49.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42dbfff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc0f9dc0 con 0x7f42dc110c40 2026-02-25T11:47:49.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42dbfff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f42dc0f9f80 con 0x7f42dc10f100 2026-02-25T11:47:49.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42da7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42dc110290 0x7f42dc1715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:49.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42daffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f42dc10f100 0x7f42dc0f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:49.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42db7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f42dc110c40 0x7f42dc0f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:49.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42daffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f42dc10f100 0x7f42dc0f8800 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:55816/0 (socket says 10.20.193.28:55816) 2026-02-25T11:47:49.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42daffd640 1 -- 10.20.193.28:0/2353914324 learned_addr learned my addr 10.20.193.28:0/2353914324 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:49.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42da7fc640 1 -- 10.20.193.28:0/2353914324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f42dc10f100 msgr2=0x7f42dc0f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:49.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42da7fc640 1 --2- 10.20.193.28:0/2353914324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f42dc10f100 0x7f42dc0f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42da7fc640 1 -- 10.20.193.28:0/2353914324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f42dc110c40 msgr2=0x7f42dc0f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:49.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42da7fc640 1 --2- 10.20.193.28:0/2353914324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f42dc110c40 0x7f42dc0f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.099+0000 7f42da7fc640 1 -- 10.20.193.28:0/2353914324 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42cc00a650 con 0x7f42dc110290 2026-02-25T11:47:49.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.100+0000 7f42da7fc640 1 --2- 10.20.193.28:0/2353914324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42dc110290 0x7f42dc1715d0 secure :-1 s=READY pgs=347 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42c400e7b0 tx=0x7f42c400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:49.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.100+0000 7f42bbfff640 1 -- 10.20.193.28:0/2353914324 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f42c400cce0 con 0x7f42dc110290 2026-02-25T11:47:49.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.100+0000 7f42bbfff640 1 -- 10.20.193.28:0/2353914324 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f42c400ce40 con 0x7f42dc110290 2026-02-25T11:47:49.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.100+0000 7f42bbfff640 1 -- 10.20.193.28:0/2353914324 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f42c40052a0 con 0x7f42dc110290 2026-02-25T11:47:49.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.100+0000 7f42dbfff640 1 -- 10.20.193.28:0/2353914324 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42dc0fa260 con 0x7f42dc110290 2026-02-25T11:47:49.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.100+0000 7f42dbfff640 1 -- 10.20.193.28:0/2353914324 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42dc0fa6e0 con 0x7f42dc110290 2026-02-25T11:47:49.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.101+0000 7f42dbfff640 1 -- 10.20.193.28:0/2353914324 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42dc1754f0 con 0x7f42dc110290 2026-02-25T11:47:49.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.103+0000 7f42bbfff640 1 -- 10.20.193.28:0/2353914324 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f42c4005430 con 0x7f42dc110290 2026-02-25T11:47:49.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.104+0000 7f42bbfff640 1 --2- 10.20.193.28:0/2353914324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42b00b57f0 0x7f42b00d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:49.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.105+0000 7f42bbfff640 1 -- 10.20.193.28:0/2353914324 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f42c4014070 con 0x7f42dc110290 2026-02-25T11:47:49.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.105+0000 7f42bbfff640 1 -- 10.20.193.28:0/2353914324 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f42c411f940 con 0x7f42dc110290 2026-02-25T11:47:49.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.105+0000 7f42daffd640 1 --2- 10.20.193.28:0/2353914324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42b00b57f0 0x7f42b00d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:49.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.105+0000 7f42daffd640 1 --2- 10.20.193.28:0/2353914324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42b00b57f0 0x7f42b00d5ca0 secure :-1 s=READY pgs=246 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42d00095d0 tx=0x7f42d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:49.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.219+0000 7f42dbfff640 1 -- 10.20.193.28:0/2353914324 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f42dc174bb0 con 0x7f42dc110290 2026-02-25T11:47:49.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.219+0000 7f42bbfff640 1 -- 10.20.193.28:0/2353914324 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) ==== 100+0+11 (secure 0 0 0) 0x7f42c40a9480 con 0x7f42dc110290 2026-02-25T11:47:49.222 INFO:teuthology.orchestra.run.trial028.stdout:pg_num: 64 2026-02-25T11:47:49.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.221+0000 7f42dbfff640 1 -- 10.20.193.28:0/2353914324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42b00b57f0 msgr2=0x7f42b00d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:49.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.221+0000 7f42dbfff640 1 --2- 10.20.193.28:0/2353914324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42b00b57f0 0x7f42b00d5ca0 secure :-1 s=READY pgs=246 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42d00095d0 tx=0x7f42d0005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:49.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.221+0000 7f42dbfff640 1 -- 10.20.193.28:0/2353914324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42dc110290 msgr2=0x7f42dc1715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:49.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.221+0000 7f42dbfff640 1 --2- 10.20.193.28:0/2353914324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42dc110290 0x7f42dc1715d0 secure :-1 s=READY pgs=347 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f42c400e7b0 tx=0x7f42c400ec80 comp rx=0 tx=0).stop 2026-02-25T11:47:49.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.221+0000 7f42dbfff640 1 -- 10.20.193.28:0/2353914324 shutdown_connections 2026-02-25T11:47:49.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.221+0000 7f42dbfff640 1 --2- 10.20.193.28:0/2353914324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f42b00b57f0 0x7f42b00d5ca0 unknown :-1 s=CLOSED pgs=246 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.221+0000 7f42dbfff640 1 --2- 10.20.193.28:0/2353914324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f42dc110c40 0x7f42dc0f9060 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.221+0000 7f42dbfff640 1 --2- 10.20.193.28:0/2353914324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f42dc110290 0x7f42dc1715d0 unknown :-1 s=CLOSED pgs=347 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.221+0000 7f42dbfff640 1 --2- 10.20.193.28:0/2353914324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f42dc10f100 0x7f42dc0f8800 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.221+0000 7f42dbfff640 1 -- 10.20.193.28:0/2353914324 >> 10.20.193.28:0/2353914324 conn(0x7f42dc0ec830 msgr2=0x7f42dc0eee20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:49.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.221+0000 7f42dbfff640 1 -- 10.20.193.28:0/2353914324 shutdown_connections 2026-02-25T11:47:49.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.221+0000 7f42dbfff640 1 -- 10.20.193.28:0/2353914324 wait complete. 2026-02-25T11:47:49.234 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:49 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1649722977' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2026-02-25T11:47:49.234 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:49 trial028 ceph-mon[39224]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2026-02-25T11:47:49.234 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:49 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2164803104' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:49.234 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:49 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/13315615' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2026-02-25T11:47:49.234 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:49 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3210366713' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2026-02-25T11:47:49.236 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2026-02-25T11:47:49.236 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:47:49.421 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:49.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:49 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1649722977' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2026-02-25T11:47:49.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:49 trial167 ceph-mon[43392]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2026-02-25T11:47:49.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:49 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2164803104' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:49.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:49 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/13315615' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2026-02-25T11:47:49.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:49 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3210366713' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2026-02-25T11:47:49.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:49 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1649722977' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2026-02-25T11:47:49.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:49 trial119 ceph-mon[42710]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2026-02-25T11:47:49.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:49 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2164803104' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:49.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:49 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/13315615' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2026-02-25T11:47:49.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:49 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3210366713' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2026-02-25T11:47:49.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.606+0000 7fdddeeaa640 1 -- 10.20.193.28:0/2380955275 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fddd810c1e0 msgr2=0x7fddd8113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:49.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.606+0000 7fdddeeaa640 1 --2- 10.20.193.28:0/2380955275 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fddd810c1e0 0x7fddd8113f80 secure :-1 s=READY pgs=348 gs=1 cs=0 l=1 c_cookie=352337858852269812 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fddcc009950 tx=0x7fddcc02f190 comp rx=0 tx=0).stop 2026-02-25T11:47:49.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.606+0000 7fdddeeaa640 1 -- 10.20.193.28:0/2380955275 shutdown_connections 2026-02-25T11:47:49.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.606+0000 7fdddeeaa640 1 --2- 10.20.193.28:0/2380955275 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fddd8114700 0x7fddd8176230 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.606+0000 7fdddeeaa640 1 --2- 10.20.193.28:0/2380955275 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fddd810c1e0 0x7fddd8113f80 unknown :-1 s=CLOSED pgs=348 gs=1 cs=0 l=1 c_cookie=352337858852269812 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.606+0000 7fdddeeaa640 1 --2- 10.20.193.28:0/2380955275 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fddd810b840 0x7fddd810bc10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.606+0000 7fdddeeaa640 1 -- 10.20.193.28:0/2380955275 >> 10.20.193.28:0/2380955275 conn(0x7fddd80e9290 msgr2=0x7fddd81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:49.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.606+0000 7fdddeeaa640 1 -- 10.20.193.28:0/2380955275 shutdown_connections 2026-02-25T11:47:49.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.606+0000 7fdddeeaa640 1 -- 10.20.193.28:0/2380955275 wait complete. 2026-02-25T11:47:49.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddeeaa640 1 Processor -- start 2026-02-25T11:47:49.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddeeaa640 1 -- start start 2026-02-25T11:47:49.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddeeaa640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fddd810b840 0x7fddd811e180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:49.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddeeaa640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fddd810c1e0 0x7fddd811e6c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:49.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddeeaa640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fddd8114700 0x7fddd8123a40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:49.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddeeaa640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8124060 con 0x7fddd810c1e0 2026-02-25T11:47:49.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddeeaa640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fddd81241d0 con 0x7fddd810b840 2026-02-25T11:47:49.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddeeaa640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8124390 con 0x7fddd8114700 2026-02-25T11:47:49.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddd6a7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fddd810c1e0 0x7fddd811e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:49.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddd6a7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fddd810c1e0 0x7fddd811e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59478/0 (socket says 10.20.193.28:59478) 2026-02-25T11:47:49.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddd6a7640 1 -- 10.20.193.28:0/782240480 learned_addr learned my addr 10.20.193.28:0/782240480 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:49.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fddddea8640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fddd810b840 0x7fddd811e180 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:49.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fddde6a9640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fddd8114700 0x7fddd8123a40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:49.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddd6a7640 1 -- 10.20.193.28:0/782240480 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fddd8114700 msgr2=0x7fddd8123a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:49.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddd6a7640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fddd8114700 0x7fddd8123a40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddd6a7640 1 -- 10.20.193.28:0/782240480 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fddd810b840 msgr2=0x7fddd811e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:49.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddd6a7640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fddd810b840 0x7fddd811e180 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fdddd6a7640 1 -- 10.20.193.28:0/782240480 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddcc009570 con 0x7fddd810c1e0 2026-02-25T11:47:49.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.607+0000 7fddde6a9640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fddd8114700 0x7fddd8123a40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:49.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.608+0000 7fdddd6a7640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fddd810c1e0 0x7fddd811e6c0 secure :-1 s=READY pgs=349 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fddcc0098a0 tx=0x7fddcc0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:49.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.608+0000 7fddc6ffd640 1 -- 10.20.193.28:0/782240480 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fddcc03d070 con 0x7fddd810c1e0 2026-02-25T11:47:49.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.608+0000 7fdddeeaa640 1 -- 10.20.193.28:0/782240480 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd8124610 con 0x7fddd810c1e0 2026-02-25T11:47:49.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.608+0000 7fdddeeaa640 1 -- 10.20.193.28:0/782240480 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd8164d70 con 0x7fddd810c1e0 2026-02-25T11:47:49.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.608+0000 7fddc6ffd640 1 -- 10.20.193.28:0/782240480 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fddcc002a70 con 0x7fddd810c1e0 2026-02-25T11:47:49.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.608+0000 7fddc6ffd640 1 -- 10.20.193.28:0/782240480 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fddcc03d070 con 0x7fddd810c1e0 2026-02-25T11:47:49.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.608+0000 7fdddeeaa640 1 -- 10.20.193.28:0/782240480 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd805fb90 con 0x7fddd810c1e0 2026-02-25T11:47:49.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.611+0000 7fddc6ffd640 1 -- 10.20.193.28:0/782240480 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fddcc0386a0 con 0x7fddd810c1e0 2026-02-25T11:47:49.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.613+0000 7fddc6ffd640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fddb40b5850 0x7fddb40d5d00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:49.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.613+0000 7fddc6ffd640 1 -- 10.20.193.28:0/782240480 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fddcc10af80 con 0x7fddd810c1e0 2026-02-25T11:47:49.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.613+0000 7fddddea8640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fddb40b5850 0x7fddb40d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:49.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.613+0000 7fddc6ffd640 1 -- 10.20.193.28:0/782240480 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fddcc10b470 con 0x7fddd810c1e0 2026-02-25T11:47:49.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.613+0000 7fddddea8640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fddb40b5850 0x7fddb40d5d00 secure :-1 s=READY pgs=247 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fddc8004740 tx=0x7fddc8004340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:49.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.757+0000 7fdddeeaa640 1 -- 10.20.193.28:0/782240480 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddd810bc10 con 0x7fddd810c1e0 2026-02-25T11:47:49.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.757+0000 7fddc6ffd640 1 -- 10.20.193.28:0/782240480 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7fddcc0cda20 con 0x7fddd810c1e0 2026-02-25T11:47:49.760 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:49.760 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:47:49.760 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:47:49.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.758+0000 7fdddeeaa640 1 -- 10.20.193.28:0/782240480 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fddb40b5850 msgr2=0x7fddb40d5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:49.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.758+0000 7fdddeeaa640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fddb40b5850 0x7fddb40d5d00 secure :-1 s=READY pgs=247 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fddc8004740 tx=0x7fddc8004340 comp rx=0 tx=0).stop 2026-02-25T11:47:49.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.758+0000 7fdddeeaa640 1 -- 10.20.193.28:0/782240480 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fddd810c1e0 msgr2=0x7fddd811e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:49.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.758+0000 7fdddeeaa640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fddd810c1e0 0x7fddd811e6c0 secure :-1 s=READY pgs=349 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fddcc0098a0 tx=0x7fddcc0028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.759+0000 7fdddeeaa640 1 -- 10.20.193.28:0/782240480 shutdown_connections 2026-02-25T11:47:49.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.759+0000 7fdddeeaa640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fddb40b5850 0x7fddb40d5d00 unknown :-1 s=CLOSED pgs=247 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.759+0000 7fdddeeaa640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fddd8114700 0x7fddd8123a40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.759+0000 7fdddeeaa640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fddd810c1e0 0x7fddd811e6c0 unknown :-1 s=CLOSED pgs=349 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.759+0000 7fdddeeaa640 1 --2- 10.20.193.28:0/782240480 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fddd810b840 0x7fddd811e180 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.759+0000 7fdddeeaa640 1 -- 10.20.193.28:0/782240480 >> 10.20.193.28:0/782240480 conn(0x7fddd80e9290 msgr2=0x7fddd8109670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:49.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.759+0000 7fdddeeaa640 1 -- 10.20.193.28:0/782240480 shutdown_connections 2026-02-25T11:47:49.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.759+0000 7fdddeeaa640 1 -- 10.20.193.28:0/782240480 wait complete. 2026-02-25T11:47:49.860 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2026-02-25T11:47:49.860 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2026-02-25T11:47:49.861 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T11:47:49.983 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.978+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/1246016753 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd98110ea0 msgr2=0x7fbd98171830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:49.984 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.978+0000 7fbd9e5d4640 1 --2- 10.20.193.28:0/1246016753 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd98110ea0 0x7fbd98171830 secure :-1 s=READY pgs=350 gs=1 cs=0 l=1 c_cookie=11572096020507947192 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbd8c00aa30 tx=0x7fbd8c030270 comp rx=0 tx=0).stop 2026-02-25T11:47:49.984 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.978+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/1246016753 shutdown_connections 2026-02-25T11:47:49.984 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.978+0000 7fbd9e5d4640 1 --2- 10.20.193.28:0/1246016753 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd98110ea0 0x7fbd98171830 unknown :-1 s=CLOSED pgs=350 gs=1 cs=0 l=1 c_cookie=11572096020507947192 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.984 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.978+0000 7fbd9e5d4640 1 --2- 10.20.193.28:0/1246016753 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbd981104f0 0x7fbd98110960 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.984 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.978+0000 7fbd9e5d4640 1 --2- 10.20.193.28:0/1246016753 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd9810f360 0x7fbd9810f750 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.984 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.978+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/1246016753 >> 10.20.193.28:0/1246016753 conn(0x7fbd980ecab0 msgr2=0x7fbd9810cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:49.984 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.978+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/1246016753 shutdown_connections 2026-02-25T11:47:49.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.978+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/1246016753 wait complete. 2026-02-25T11:47:49.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.979+0000 7fbd9e5d4640 1 Processor -- start 2026-02-25T11:47:49.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.979+0000 7fbd9e5d4640 1 -- start start 2026-02-25T11:47:49.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.979+0000 7fbd9e5d4640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbd9810f360 0x7fbd980f8a50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:49.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.979+0000 7fbd9e5d4640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd981104f0 0x7fbd98171830 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:49.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.979+0000 7fbd9e5d4640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd98110ea0 0x7fbd980f92b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:49.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.979+0000 7fbd9e5d4640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd980f9ea0 con 0x7fbd98110ea0 2026-02-25T11:47:49.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.979+0000 7fbd9e5d4640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbd980fa010 con 0x7fbd9810f360 2026-02-25T11:47:49.985 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.979+0000 7fbd9e5d4640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd980fa1d0 con 0x7fbd981104f0 2026-02-25T11:47:49.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd977fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd981104f0 0x7fbd98171830 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:49.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd977fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd981104f0 0x7fbd98171830 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:55856/0 (socket says 10.20.193.28:55856) 2026-02-25T11:47:49.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd977fe640 1 -- 10.20.193.28:0/891111378 learned_addr learned my addr 10.20.193.28:0/891111378 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:49.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd97fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbd9810f360 0x7fbd980f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:49.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd9cb4a640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd98110ea0 0x7fbd980f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:49.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd97fff640 1 -- 10.20.193.28:0/891111378 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd981104f0 msgr2=0x7fbd98171830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:49.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd97fff640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd981104f0 0x7fbd98171830 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd97fff640 1 -- 10.20.193.28:0/891111378 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd98110ea0 msgr2=0x7fbd980f92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:49.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd97fff640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd98110ea0 0x7fbd980f92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:49.986 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd97fff640 1 -- 10.20.193.28:0/891111378 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd8c00a650 con 0x7fbd9810f360 2026-02-25T11:47:49.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd977fe640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd981104f0 0x7fbd98171830 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:49.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd97fff640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbd9810f360 0x7fbd980f8a50 secure :-1 s=READY pgs=104 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbd8400d720 tx=0x7fbd8400dbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:49.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd957fa640 1 -- 10.20.193.28:0/891111378 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbd84004500 con 0x7fbd9810f360 2026-02-25T11:47:49.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd957fa640 1 -- 10.20.193.28:0/891111378 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fbd84004d10 con 0x7fbd9810f360 2026-02-25T11:47:49.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.980+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/891111378 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd980fa4b0 con 0x7fbd9810f360 2026-02-25T11:47:49.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.981+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/891111378 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd980fa980 con 0x7fbd9810f360 2026-02-25T11:47:49.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.981+0000 7fbd957fa640 1 -- 10.20.193.28:0/891111378 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbd840052a0 con 0x7fbd9810f360 2026-02-25T11:47:49.987 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.981+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/891111378 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd98060010 con 0x7fbd9810f360 2026-02-25T11:47:49.988 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.983+0000 7fbd957fa640 1 -- 10.20.193.28:0/891111378 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fbd84005430 con 0x7fbd9810f360 2026-02-25T11:47:49.988 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.985+0000 7fbd957fa640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbd6c0b5770 0x7fbd6c0d5c20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:49.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.985+0000 7fbd957fa640 1 -- 10.20.193.28:0/891111378 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fbd840e6850 con 0x7fbd9810f360 2026-02-25T11:47:49.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.985+0000 7fbd977fe640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbd6c0b5770 0x7fbd6c0d5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:49.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.986+0000 7fbd957fa640 1 -- 10.20.193.28:0/891111378 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fbd840a9290 con 0x7fbd9810f360 2026-02-25T11:47:49.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:49.986+0000 7fbd977fe640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbd6c0b5770 0x7fbd6c0d5c20 secure :-1 s=READY pgs=248 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbd880095d0 tx=0x7fbd88005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:50.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.098+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/891111378 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd98174d80 con 0x7fbd6c0b5770 2026-02-25T11:47:50.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.101+0000 7fbd957fa640 1 -- 10.20.193.28:0/891111378 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258968 (secure 0 0 0) 0x7fbd98174d80 con 0x7fbd6c0b5770 2026-02-25T11:47:50.107 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:50.109 INFO:teuthology.orchestra.run.trial028.stderr:dumped all 2026-02-25T11:47:50.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.104+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/891111378 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbd6c0b5770 msgr2=0x7fbd6c0d5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:50.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.104+0000 7fbd9e5d4640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbd6c0b5770 0x7fbd6c0d5c20 secure :-1 s=READY pgs=248 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbd880095d0 tx=0x7fbd88005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:50.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.104+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/891111378 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbd9810f360 msgr2=0x7fbd980f8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:50.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.104+0000 7fbd9e5d4640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbd9810f360 0x7fbd980f8a50 secure :-1 s=READY pgs=104 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbd8400d720 tx=0x7fbd8400dbf0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.104+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/891111378 shutdown_connections 2026-02-25T11:47:50.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.104+0000 7fbd9e5d4640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbd6c0b5770 0x7fbd6c0d5c20 unknown :-1 s=CLOSED pgs=248 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.104+0000 7fbd9e5d4640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbd98110ea0 0x7fbd980f92b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.104+0000 7fbd9e5d4640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbd981104f0 0x7fbd98171830 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.104+0000 7fbd9e5d4640 1 --2- 10.20.193.28:0/891111378 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbd9810f360 0x7fbd980f8a50 unknown :-1 s=CLOSED pgs=104 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.104+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/891111378 >> 10.20.193.28:0/891111378 conn(0x7fbd980ecab0 msgr2=0x7fbd980ef050 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:50.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.104+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/891111378 shutdown_connections 2026-02-25T11:47:50.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.104+0000 7fbd9e5d4640 1 -- 10.20.193.28:0/891111378 wait complete. 2026-02-25T11:47:50.119 INFO:teuthology.orchestra.run.trial028.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2026-02-25T11:47:49.080274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664241,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":309,"num_read_kb":269,"num_write":359,"num_write_kb":3847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":297,"ondisk_log_size":297,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":8808038400,"kb_used":338144,"kb_used_data":11504,"kb_used_omap":464,"kb_used_meta":325935,"kb_avail":8807700256,"statfs":{"total":9019431321600,"available":9019085062144,"internally_reserved":0,"allocated":11780096,"data_stored":7459523,"data_compressed":114024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":475426,"internal_metadata":333758174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3098,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":39,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002180"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627807+0000","last_change":"2026-02-25T11:47:13.630492+0000","last_active":"2026-02-25T11:47:14.627807+0000","last_peered":"2026-02-25T11:47:14.627807+0000","last_clean":"2026-02-25T11:47:14.627807+0000","last_became_active":"2026-02-25T11:47:13.630388+0000","last_became_peered":"2026-02-25T11:47:13.630388+0000","last_unstale":"2026-02-25T11:47:14.627807+0000","last_undegraded":"2026-02-25T11:47:14.627807+0000","last_fullsized":"2026-02-25T11:47:14.627807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:01:17.024058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627657+0000","last_change":"2026-02-25T11:47:13.629507+0000","last_active":"2026-02-25T11:47:14.627657+0000","last_peered":"2026-02-25T11:47:14.627657+0000","last_clean":"2026-02-25T11:47:14.627657+0000","last_became_active":"2026-02-25T11:47:13.628937+0000","last_became_peered":"2026-02-25T11:47:13.628937+0000","last_unstale":"2026-02-25T11:47:14.627657+0000","last_undegraded":"2026-02-25T11:47:14.627657+0000","last_fullsized":"2026-02-25T11:47:14.627657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:47:20.441522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627713+0000","last_change":"2026-02-25T11:47:13.630415+0000","last_active":"2026-02-25T11:47:14.627713+0000","last_peered":"2026-02-25T11:47:14.627713+0000","last_clean":"2026-02-25T11:47:14.627713+0000","last_became_active":"2026-02-25T11:47:13.630319+0000","last_became_peered":"2026-02-25T11:47:13.630319+0000","last_unstale":"2026-02-25T11:47:14.627713+0000","last_undegraded":"2026-02-25T11:47:14.627713+0000","last_fullsized":"2026-02-25T11:47:14.627713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:18:57.675760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627800+0000","last_change":"2026-02-25T11:47:13.633017+0000","last_active":"2026-02-25T11:47:14.627800+0000","last_peered":"2026-02-25T11:47:14.627800+0000","last_clean":"2026-02-25T11:47:14.627800+0000","last_became_active":"2026-02-25T11:47:13.632934+0000","last_became_peered":"2026-02-25T11:47:13.632934+0000","last_unstale":"2026-02-25T11:47:14.627800+0000","last_undegraded":"2026-02-25T11:47:14.627800+0000","last_fullsized":"2026-02-25T11:47:14.627800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:08:54.195980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628667+0000","last_change":"2026-02-25T11:47:13.638057+0000","last_active":"2026-02-25T11:47:14.628667+0000","last_peered":"2026-02-25T11:47:14.628667+0000","last_clean":"2026-02-25T11:47:14.628667+0000","last_became_active":"2026-02-25T11:47:13.636387+0000","last_became_peered":"2026-02-25T11:47:13.636387+0000","last_unstale":"2026-02-25T11:47:14.628667+0000","last_undegraded":"2026-02-25T11:47:14.628667+0000","last_fullsized":"2026-02-25T11:47:14.628667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:34:29.570622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750630+0000","last_change":"2026-02-25T11:47:13.631824+0000","last_active":"2026-02-25T11:47:14.750630+0000","last_peered":"2026-02-25T11:47:14.750630+0000","last_clean":"2026-02-25T11:47:14.750630+0000","last_became_active":"2026-02-25T11:47:13.631656+0000","last_became_peered":"2026-02-25T11:47:13.631656+0000","last_unstale":"2026-02-25T11:47:14.750630+0000","last_undegraded":"2026-02-25T11:47:14.750630+0000","last_fullsized":"2026-02-25T11:47:14.750630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:15:05.237015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628186+0000","last_change":"2026-02-25T11:47:13.626877+0000","last_active":"2026-02-25T11:47:14.628186+0000","last_peered":"2026-02-25T11:47:14.628186+0000","last_clean":"2026-02-25T11:47:14.628186+0000","last_became_active":"2026-02-25T11:47:13.626543+0000","last_became_peered":"2026-02-25T11:47:13.626543+0000","last_unstale":"2026-02-25T11:47:14.628186+0000","last_undegraded":"2026-02-25T11:47:14.628186+0000","last_fullsized":"2026-02-25T11:47:14.628186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:56:58.194911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635224+0000","last_change":"2026-02-25T11:47:13.637661+0000","last_active":"2026-02-25T11:47:14.635224+0000","last_peered":"2026-02-25T11:47:14.635224+0000","last_clean":"2026-02-25T11:47:14.635224+0000","last_became_active":"2026-02-25T11:47:13.637230+0000","last_became_peered":"2026-02-25T11:47:13.637230+0000","last_unstale":"2026-02-25T11:47:14.635224+0000","last_undegraded":"2026-02-25T11:47:14.635224+0000","last_fullsized":"2026-02-25T11:47:14.635224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:44:52.310360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751287+0000","last_change":"2026-02-25T11:47:13.625160+0000","last_active":"2026-02-25T11:47:14.751287+0000","last_peered":"2026-02-25T11:47:14.751287+0000","last_clean":"2026-02-25T11:47:14.751287+0000","last_became_active":"2026-02-25T11:47:13.625020+0000","last_became_peered":"2026-02-25T11:47:13.625020+0000","last_unstale":"2026-02-25T11:47:14.751287+0000","last_undegraded":"2026-02-25T11:47:14.751287+0000","last_fullsized":"2026-02-25T11:47:14.751287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:35:44.509551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750832+0000","last_change":"2026-02-25T11:47:13.637252+0000","last_active":"2026-02-25T11:47:14.750832+0000","last_peered":"2026-02-25T11:47:14.750832+0000","last_clean":"2026-02-25T11:47:14.750832+0000","last_became_active":"2026-02-25T11:47:13.636670+0000","last_became_peered":"2026-02-25T11:47:13.636670+0000","last_unstale":"2026-02-25T11:47:14.750832+0000","last_undegraded":"2026-02-25T11:47:14.750832+0000","last_fullsized":"2026-02-25T11:47:14.750832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:01:53.119101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628443+0000","last_change":"2026-02-25T11:47:13.628252+0000","last_active":"2026-02-25T11:47:14.628443+0000","last_peered":"2026-02-25T11:47:14.628443+0000","last_clean":"2026-02-25T11:47:14.628443+0000","last_became_active":"2026-02-25T11:47:13.628183+0000","last_became_peered":"2026-02-25T11:47:13.628183+0000","last_unstale":"2026-02-25T11:47:14.628443+0000","last_undegraded":"2026-02-25T11:47:14.628443+0000","last_fullsized":"2026-02-25T11:47:14.628443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:12:00.576747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.632358+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.631487+0000","last_became_peered":"2026-02-25T11:47:13.631487+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:41:17.150851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749735+0000","last_change":"2026-02-25T11:47:13.629675+0000","last_active":"2026-02-25T11:47:14.749735+0000","last_peered":"2026-02-25T11:47:14.749735+0000","last_clean":"2026-02-25T11:47:14.749735+0000","last_became_active":"2026-02-25T11:47:13.628686+0000","last_became_peered":"2026-02-25T11:47:13.628686+0000","last_unstale":"2026-02-25T11:47:14.749735+0000","last_undegraded":"2026-02-25T11:47:14.749735+0000","last_fullsized":"2026-02-25T11:47:14.749735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:00:54.430287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627615+0000","last_change":"2026-02-25T11:47:13.628842+0000","last_active":"2026-02-25T11:47:14.627615+0000","last_peered":"2026-02-25T11:47:14.627615+0000","last_clean":"2026-02-25T11:47:14.627615+0000","last_became_active":"2026-02-25T11:47:13.628695+0000","last_became_peered":"2026-02-25T11:47:13.628695+0000","last_unstale":"2026-02-25T11:47:14.627615+0000","last_undegraded":"2026-02-25T11:47:14.627615+0000","last_fullsized":"2026-02-25T11:47:14.627615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:10:05.404740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.635652+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.635505+0000","last_became_peered":"2026-02-25T11:47:13.635505+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:46:03.598577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749928+0000","last_change":"2026-02-25T11:47:13.628651+0000","last_active":"2026-02-25T11:47:14.749928+0000","last_peered":"2026-02-25T11:47:14.749928+0000","last_clean":"2026-02-25T11:47:14.749928+0000","last_became_active":"2026-02-25T11:47:13.628280+0000","last_became_peered":"2026-02-25T11:47:13.628280+0000","last_unstale":"2026-02-25T11:47:14.749928+0000","last_undegraded":"2026-02-25T11:47:14.749928+0000","last_fullsized":"2026-02-25T11:47:14.749928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:18:56.238078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628351+0000","last_change":"2026-02-25T11:47:13.630656+0000","last_active":"2026-02-25T11:47:14.628351+0000","last_peered":"2026-02-25T11:47:14.628351+0000","last_clean":"2026-02-25T11:47:14.628351+0000","last_became_active":"2026-02-25T11:47:13.630513+0000","last_became_peered":"2026-02-25T11:47:13.630513+0000","last_unstale":"2026-02-25T11:47:14.628351+0000","last_undegraded":"2026-02-25T11:47:14.628351+0000","last_fullsized":"2026-02-25T11:47:14.628351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:03:33.580576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.634036+0000","last_change":"2026-02-25T11:47:13.629202+0000","last_active":"2026-02-25T11:47:14.634036+0000","last_peered":"2026-02-25T11:47:14.634036+0000","last_clean":"2026-02-25T11:47:14.634036+0000","last_became_active":"2026-02-25T11:47:13.628860+0000","last_became_peered":"2026-02-25T11:47:13.628860+0000","last_unstale":"2026-02-25T11:47:14.634036+0000","last_undegraded":"2026-02-25T11:47:14.634036+0000","last_fullsized":"2026-02-25T11:47:14.634036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:47:14.635796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628235+0000","last_change":"2026-02-25T11:47:13.638872+0000","last_active":"2026-02-25T11:47:14.628235+0000","last_peered":"2026-02-25T11:47:14.628235+0000","last_clean":"2026-02-25T11:47:14.628235+0000","last_became_active":"2026-02-25T11:47:13.638443+0000","last_became_peered":"2026-02-25T11:47:13.638443+0000","last_unstale":"2026-02-25T11:47:14.628235+0000","last_undegraded":"2026-02-25T11:47:14.628235+0000","last_fullsized":"2026-02-25T11:47:14.628235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:51.699838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628595+0000","last_change":"2026-02-25T11:47:13.635894+0000","last_active":"2026-02-25T11:47:14.628595+0000","last_peered":"2026-02-25T11:47:14.628595+0000","last_clean":"2026-02-25T11:47:14.628595+0000","last_became_active":"2026-02-25T11:47:13.635725+0000","last_became_peered":"2026-02-25T11:47:13.635725+0000","last_unstale":"2026-02-25T11:47:14.628595+0000","last_undegraded":"2026-02-25T11:47:14.628595+0000","last_fullsized":"2026-02-25T11:47:14.628595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:28:20.134454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749794+0000","last_change":"2026-02-25T11:47:13.628117+0000","last_active":"2026-02-25T11:47:14.749794+0000","last_peered":"2026-02-25T11:47:14.749794+0000","last_clean":"2026-02-25T11:47:14.749794+0000","last_became_active":"2026-02-25T11:47:13.628037+0000","last_became_peered":"2026-02-25T11:47:13.628037+0000","last_unstale":"2026-02-25T11:47:14.749794+0000","last_undegraded":"2026-02-25T11:47:14.749794+0000","last_fullsized":"2026-02-25T11:47:14.749794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:28:22.976133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750830+0000","last_change":"2026-02-25T11:47:13.634652+0000","last_active":"2026-02-25T11:47:14.750830+0000","last_peered":"2026-02-25T11:47:14.750830+0000","last_clean":"2026-02-25T11:47:14.750830+0000","last_became_active":"2026-02-25T11:47:13.634433+0000","last_became_peered":"2026-02-25T11:47:13.634433+0000","last_unstale":"2026-02-25T11:47:14.750830+0000","last_undegraded":"2026-02-25T11:47:14.750830+0000","last_fullsized":"2026-02-25T11:47:14.750830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:01:29.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635319+0000","last_change":"2026-02-25T11:47:13.632721+0000","last_active":"2026-02-25T11:47:14.635319+0000","last_peered":"2026-02-25T11:47:14.635319+0000","last_clean":"2026-02-25T11:47:14.635319+0000","last_became_active":"2026-02-25T11:47:13.632626+0000","last_became_peered":"2026-02-25T11:47:13.632626+0000","last_unstale":"2026-02-25T11:47:14.635319+0000","last_undegraded":"2026-02-25T11:47:14.635319+0000","last_fullsized":"2026-02-25T11:47:14.635319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:28:35.201966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749844+0000","last_change":"2026-02-25T11:47:13.637504+0000","last_active":"2026-02-25T11:47:14.749844+0000","last_peered":"2026-02-25T11:47:14.749844+0000","last_clean":"2026-02-25T11:47:14.749844+0000","last_became_active":"2026-02-25T11:47:13.636824+0000","last_became_peered":"2026-02-25T11:47:13.636824+0000","last_unstale":"2026-02-25T11:47:14.749844+0000","last_undegraded":"2026-02-25T11:47:14.749844+0000","last_fullsized":"2026-02-25T11:47:14.749844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:20:31.416302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635724+0000","last_change":"2026-02-25T11:47:13.633135+0000","last_active":"2026-02-25T11:47:14.635724+0000","last_peered":"2026-02-25T11:47:14.635724+0000","last_clean":"2026-02-25T11:47:14.635724+0000","last_became_active":"2026-02-25T11:47:13.632966+0000","last_became_peered":"2026-02-25T11:47:13.632966+0000","last_unstale":"2026-02-25T11:47:14.635724+0000","last_undegraded":"2026-02-25T11:47:14.635724+0000","last_fullsized":"2026-02-25T11:47:14.635724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:26:17.052924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628538+0000","last_change":"2026-02-25T11:47:13.633210+0000","last_active":"2026-02-25T11:47:14.628538+0000","last_peered":"2026-02-25T11:47:14.628538+0000","last_clean":"2026-02-25T11:47:14.628538+0000","last_became_active":"2026-02-25T11:47:13.633099+0000","last_became_peered":"2026-02-25T11:47:13.633099+0000","last_unstale":"2026-02-25T11:47:14.628538+0000","last_undegraded":"2026-02-25T11:47:14.628538+0000","last_fullsized":"2026-02-25T11:47:14.628538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:07:46.884629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628656+0000","last_change":"2026-02-25T11:47:13.634757+0000","last_active":"2026-02-25T11:47:14.628656+0000","last_peered":"2026-02-25T11:47:14.628656+0000","last_clean":"2026-02-25T11:47:14.628656+0000","last_became_active":"2026-02-25T11:47:13.634643+0000","last_became_peered":"2026-02-25T11:47:13.634643+0000","last_unstale":"2026-02-25T11:47:14.628656+0000","last_undegraded":"2026-02-25T11:47:14.628656+0000","last_fullsized":"2026-02-25T11:47:14.628656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:20:23.383359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749674+0000","last_change":"2026-02-25T11:47:13.630515+0000","last_active":"2026-02-25T11:47:14.749674+0000","last_peered":"2026-02-25T11:47:14.749674+0000","last_clean":"2026-02-25T11:47:14.749674+0000","last_became_active":"2026-02-25T11:47:13.630202+0000","last_became_peered":"2026-02-25T11:47:13.630202+0000","last_unstale":"2026-02-25T11:47:14.749674+0000","last_undegraded":"2026-02-25T11:47:14.749674+0000","last_fullsized":"2026-02-25T11:47:14.749674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:51:24.646614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627367+0000","last_change":"2026-02-25T11:47:13.635749+0000","last_active":"2026-02-25T11:47:14.627367+0000","last_peered":"2026-02-25T11:47:14.627367+0000","last_clean":"2026-02-25T11:47:14.627367+0000","last_became_active":"2026-02-25T11:47:13.635284+0000","last_became_peered":"2026-02-25T11:47:13.635284+0000","last_unstale":"2026-02-25T11:47:14.627367+0000","last_undegraded":"2026-02-25T11:47:14.627367+0000","last_fullsized":"2026-02-25T11:47:14.627367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:23:45.042142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628732+0000","last_change":"2026-02-25T11:47:13.635448+0000","last_active":"2026-02-25T11:47:14.628732+0000","last_peered":"2026-02-25T11:47:14.628732+0000","last_clean":"2026-02-25T11:47:14.628732+0000","last_became_active":"2026-02-25T11:47:13.635232+0000","last_became_peered":"2026-02-25T11:47:13.635232+0000","last_unstale":"2026-02-25T11:47:14.628732+0000","last_undegraded":"2026-02-25T11:47:14.628732+0000","last_fullsized":"2026-02-25T11:47:14.628732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:33:38.020605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628332+0000","last_change":"2026-02-25T11:47:13.634237+0000","last_active":"2026-02-25T11:47:14.628332+0000","last_peered":"2026-02-25T11:47:14.628332+0000","last_clean":"2026-02-25T11:47:14.628332+0000","last_became_active":"2026-02-25T11:47:13.634054+0000","last_became_peered":"2026-02-25T11:47:13.634054+0000","last_unstale":"2026-02-25T11:47:14.628332+0000","last_undegraded":"2026-02-25T11:47:14.628332+0000","last_fullsized":"2026-02-25T11:47:14.628332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:38:34.603077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627786+0000","last_change":"2026-02-25T11:47:13.638347+0000","last_active":"2026-02-25T11:47:14.627786+0000","last_peered":"2026-02-25T11:47:14.627786+0000","last_clean":"2026-02-25T11:47:14.627786+0000","last_became_active":"2026-02-25T11:47:13.637984+0000","last_became_peered":"2026-02-25T11:47:13.637984+0000","last_unstale":"2026-02-25T11:47:14.627786+0000","last_undegraded":"2026-02-25T11:47:14.627786+0000","last_fullsized":"2026-02-25T11:47:14.627786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:36:11.273078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628202+0000","last_change":"2026-02-25T11:47:13.634523+0000","last_active":"2026-02-25T11:47:14.628202+0000","last_peered":"2026-02-25T11:47:14.628202+0000","last_clean":"2026-02-25T11:47:14.628202+0000","last_became_active":"2026-02-25T11:47:13.634184+0000","last_became_peered":"2026-02-25T11:47:13.634184+0000","last_unstale":"2026-02-25T11:47:14.628202+0000","last_undegraded":"2026-02-25T11:47:14.628202+0000","last_fullsized":"2026-02-25T11:47:14.628202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:52:23.917090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:47.599052+0000","last_change":"2026-02-25T11:47:12.613380+0000","last_active":"2026-02-25T11:47:47.599052+0000","last_peered":"2026-02-25T11:47:47.599052+0000","last_clean":"2026-02-25T11:47:47.599052+0000","last_became_active":"2026-02-25T11:47:12.613224+0000","last_became_peered":"2026-02-25T11:47:12.613224+0000","last_unstale":"2026-02-25T11:47:47.599052+0000","last_undegraded":"2026-02-25T11:47:47.599052+0000","last_fullsized":"2026-02-25T11:47:47.599052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:59:39.894067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1135,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749915+0000","last_change":"2026-02-25T11:47:13.636227+0000","last_active":"2026-02-25T11:47:14.749915+0000","last_peered":"2026-02-25T11:47:14.749915+0000","last_clean":"2026-02-25T11:47:14.749915+0000","last_became_active":"2026-02-25T11:47:13.635212+0000","last_became_peered":"2026-02-25T11:47:13.635212+0000","last_unstale":"2026-02-25T11:47:14.749915+0000","last_undegraded":"2026-02-25T11:47:14.749915+0000","last_fullsized":"2026-02-25T11:47:14.749915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:51:07.635399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657308+0000","last_change":"2026-02-25T11:47:12.613789+0000","last_active":"2026-02-25T11:47:17.657308+0000","last_peered":"2026-02-25T11:47:17.657308+0000","last_clean":"2026-02-25T11:47:17.657308+0000","last_became_active":"2026-02-25T11:47:12.613635+0000","last_became_peered":"2026-02-25T11:47:12.613635+0000","last_unstale":"2026-02-25T11:47:17.657308+0000","last_undegraded":"2026-02-25T11:47:17.657308+0000","last_fullsized":"2026-02-25T11:47:17.657308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:13:32.216593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657439+0000","last_change":"2026-02-25T11:47:12.612012+0000","last_active":"2026-02-25T11:47:17.657439+0000","last_peered":"2026-02-25T11:47:17.657439+0000","last_clean":"2026-02-25T11:47:17.657439+0000","last_became_active":"2026-02-25T11:47:12.611862+0000","last_became_peered":"2026-02-25T11:47:12.611862+0000","last_unstale":"2026-02-25T11:47:17.657439+0000","last_undegraded":"2026-02-25T11:47:17.657439+0000","last_fullsized":"2026-02-25T11:47:17.657439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:08:17.490884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635279+0000","last_change":"2026-02-25T11:47:13.627129+0000","last_active":"2026-02-25T11:47:14.635279+0000","last_peered":"2026-02-25T11:47:14.635279+0000","last_clean":"2026-02-25T11:47:14.635279+0000","last_became_active":"2026-02-25T11:47:13.626987+0000","last_became_peered":"2026-02-25T11:47:13.626987+0000","last_unstale":"2026-02-25T11:47:14.635279+0000","last_undegraded":"2026-02-25T11:47:14.635279+0000","last_fullsized":"2026-02-25T11:47:14.635279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:53:45.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749940+0000","last_change":"2026-02-25T11:47:12.611992+0000","last_active":"2026-02-25T11:47:14.749940+0000","last_peered":"2026-02-25T11:47:14.749940+0000","last_clean":"2026-02-25T11:47:14.749940+0000","last_became_active":"2026-02-25T11:47:12.611502+0000","last_became_peered":"2026-02-25T11:47:12.611502+0000","last_unstale":"2026-02-25T11:47:14.749940+0000","last_undegraded":"2026-02-25T11:47:14.749940+0000","last_fullsized":"2026-02-25T11:47:14.749940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:15:45.718628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633835+0000","last_change":"2026-02-25T11:47:13.635344+0000","last_active":"2026-02-25T11:47:14.633835+0000","last_peered":"2026-02-25T11:47:14.633835+0000","last_clean":"2026-02-25T11:47:14.633835+0000","last_became_active":"2026-02-25T11:47:13.634327+0000","last_became_peered":"2026-02-25T11:47:13.634327+0000","last_unstale":"2026-02-25T11:47:14.633835+0000","last_undegraded":"2026-02-25T11:47:14.633835+0000","last_fullsized":"2026-02-25T11:47:14.633835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:30:06.778398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:45.596468+0000","last_change":"2026-02-25T11:47:12.611130+0000","last_active":"2026-02-25T11:47:45.596468+0000","last_peered":"2026-02-25T11:47:45.596468+0000","last_clean":"2026-02-25T11:47:45.596468+0000","last_became_active":"2026-02-25T11:47:12.610838+0000","last_became_peered":"2026-02-25T11:47:12.610838+0000","last_unstale":"2026-02-25T11:47:45.596468+0000","last_undegraded":"2026-02-25T11:47:45.596468+0000","last_fullsized":"2026-02-25T11:47:45.596468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:34:19.366424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751244+0000","last_change":"2026-02-25T11:47:13.629263+0000","last_active":"2026-02-25T11:47:14.751244+0000","last_peered":"2026-02-25T11:47:14.751244+0000","last_clean":"2026-02-25T11:47:14.751244+0000","last_became_active":"2026-02-25T11:47:13.628765+0000","last_became_peered":"2026-02-25T11:47:13.628765+0000","last_unstale":"2026-02-25T11:47:14.751244+0000","last_undegraded":"2026-02-25T11:47:14.751244+0000","last_fullsized":"2026-02-25T11:47:14.751244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:49:04.773052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.649872+0000","last_change":"2026-02-25T11:47:12.613455+0000","last_active":"2026-02-25T11:47:16.649872+0000","last_peered":"2026-02-25T11:47:16.649872+0000","last_clean":"2026-02-25T11:47:16.649872+0000","last_became_active":"2026-02-25T11:47:12.612904+0000","last_became_peered":"2026-02-25T11:47:12.612904+0000","last_unstale":"2026-02-25T11:47:16.649872+0000","last_undegraded":"2026-02-25T11:47:16.649872+0000","last_fullsized":"2026-02-25T11:47:16.649872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:16:13.149517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627673+0000","last_change":"2026-02-25T11:47:13.635178+0000","last_active":"2026-02-25T11:47:14.627673+0000","last_peered":"2026-02-25T11:47:14.627673+0000","last_clean":"2026-02-25T11:47:14.627673+0000","last_became_active":"2026-02-25T11:47:13.634985+0000","last_became_peered":"2026-02-25T11:47:13.634985+0000","last_unstale":"2026-02-25T11:47:14.627673+0000","last_undegraded":"2026-02-25T11:47:14.627673+0000","last_fullsized":"2026-02-25T11:47:14.627673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:08.068457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653282+0000","last_change":"2026-02-25T11:47:12.610853+0000","last_active":"2026-02-25T11:47:16.653282+0000","last_peered":"2026-02-25T11:47:16.653282+0000","last_clean":"2026-02-25T11:47:16.653282+0000","last_became_active":"2026-02-25T11:47:12.610639+0000","last_became_peered":"2026-02-25T11:47:12.610639+0000","last_unstale":"2026-02-25T11:47:16.653282+0000","last_undegraded":"2026-02-25T11:47:16.653282+0000","last_fullsized":"2026-02-25T11:47:16.653282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:00:06.042790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":694,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750953+0000","last_change":"2026-02-25T11:47:13.630063+0000","last_active":"2026-02-25T11:47:14.750953+0000","last_peered":"2026-02-25T11:47:14.750953+0000","last_clean":"2026-02-25T11:47:14.750953+0000","last_became_active":"2026-02-25T11:47:13.629906+0000","last_became_peered":"2026-02-25T11:47:13.629906+0000","last_unstale":"2026-02-25T11:47:14.750953+0000","last_undegraded":"2026-02-25T11:47:14.750953+0000","last_fullsized":"2026-02-25T11:47:14.750953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:09:13.614948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":113,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:46.600201+0000","last_change":"2026-02-25T11:47:12.615044+0000","last_active":"2026-02-25T11:47:46.600201+0000","last_peered":"2026-02-25T11:47:46.600201+0000","last_clean":"2026-02-25T11:47:46.600201+0000","last_became_active":"2026-02-25T11:47:12.614698+0000","last_became_peered":"2026-02-25T11:47:12.614698+0000","last_unstale":"2026-02-25T11:47:46.600201+0000","last_undegraded":"2026-02-25T11:47:46.600201+0000","last_fullsized":"2026-02-25T11:47:46.600201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:52:32.148801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69040,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":67,"num_read_kb":100,"num_write":31,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628510+0000","last_change":"2026-02-25T11:47:13.630945+0000","last_active":"2026-02-25T11:47:14.628510+0000","last_peered":"2026-02-25T11:47:14.628510+0000","last_clean":"2026-02-25T11:47:14.628510+0000","last_became_active":"2026-02-25T11:47:13.630693+0000","last_became_peered":"2026-02-25T11:47:13.630693+0000","last_unstale":"2026-02-25T11:47:14.628510+0000","last_undegraded":"2026-02-25T11:47:14.628510+0000","last_fullsized":"2026-02-25T11:47:14.628510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:39:52.744366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656758+0000","last_change":"2026-02-25T11:47:12.614950+0000","last_active":"2026-02-25T11:47:17.656758+0000","last_peered":"2026-02-25T11:47:17.656758+0000","last_clean":"2026-02-25T11:47:17.656758+0000","last_became_active":"2026-02-25T11:47:12.614272+0000","last_became_peered":"2026-02-25T11:47:12.614272+0000","last_unstale":"2026-02-25T11:47:17.656758+0000","last_undegraded":"2026-02-25T11:47:17.656758+0000","last_fullsized":"2026-02-25T11:47:17.656758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:45:50.928166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749596+0000","last_change":"2026-02-25T11:47:13.633128+0000","last_active":"2026-02-25T11:47:14.749596+0000","last_peered":"2026-02-25T11:47:14.749596+0000","last_clean":"2026-02-25T11:47:14.749596+0000","last_became_active":"2026-02-25T11:47:13.633047+0000","last_became_peered":"2026-02-25T11:47:13.633047+0000","last_unstale":"2026-02-25T11:47:14.749596+0000","last_undegraded":"2026-02-25T11:47:14.749596+0000","last_fullsized":"2026-02-25T11:47:14.749596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:56:33.191193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653783+0000","last_change":"2026-02-25T11:47:12.616898+0000","last_active":"2026-02-25T11:47:16.653783+0000","last_peered":"2026-02-25T11:47:16.653783+0000","last_clean":"2026-02-25T11:47:16.653783+0000","last_became_active":"2026-02-25T11:47:12.616774+0000","last_became_peered":"2026-02-25T11:47:12.616774+0000","last_unstale":"2026-02-25T11:47:16.653783+0000","last_undegraded":"2026-02-25T11:47:16.653783+0000","last_fullsized":"2026-02-25T11:47:16.653783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:59:33.930544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628464+0000","last_change":"2026-02-25T11:47:13.630377+0000","last_active":"2026-02-25T11:47:14.628464+0000","last_peered":"2026-02-25T11:47:14.628464+0000","last_clean":"2026-02-25T11:47:14.628464+0000","last_became_active":"2026-02-25T11:47:13.630124+0000","last_became_peered":"2026-02-25T11:47:13.630124+0000","last_unstale":"2026-02-25T11:47:14.628464+0000","last_undegraded":"2026-02-25T11:47:14.628464+0000","last_fullsized":"2026-02-25T11:47:14.628464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:42:12.028258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:43.604899+0000","last_change":"2026-02-25T11:47:12.613208+0000","last_active":"2026-02-25T11:47:43.604899+0000","last_peered":"2026-02-25T11:47:43.604899+0000","last_clean":"2026-02-25T11:47:43.604899+0000","last_became_active":"2026-02-25T11:47:12.612460+0000","last_became_peered":"2026-02-25T11:47:12.612460+0000","last_unstale":"2026-02-25T11:47:43.604899+0000","last_undegraded":"2026-02-25T11:47:43.604899+0000","last_fullsized":"2026-02-25T11:47:43.604899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:15:59.810632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":998,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628121+0000","last_change":"2026-02-25T11:47:13.631123+0000","last_active":"2026-02-25T11:47:14.628121+0000","last_peered":"2026-02-25T11:47:14.628121+0000","last_clean":"2026-02-25T11:47:14.628121+0000","last_became_active":"2026-02-25T11:47:13.631031+0000","last_became_peered":"2026-02-25T11:47:13.631031+0000","last_unstale":"2026-02-25T11:47:14.628121+0000","last_undegraded":"2026-02-25T11:47:14.628121+0000","last_fullsized":"2026-02-25T11:47:14.628121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:47:54.073226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657937+0000","last_change":"2026-02-25T11:47:12.612153+0000","last_active":"2026-02-25T11:47:17.657937+0000","last_peered":"2026-02-25T11:47:17.657937+0000","last_clean":"2026-02-25T11:47:17.657937+0000","last_became_active":"2026-02-25T11:47:12.611107+0000","last_became_peered":"2026-02-25T11:47:12.611107+0000","last_unstale":"2026-02-25T11:47:17.657937+0000","last_undegraded":"2026-02-25T11:47:17.657937+0000","last_fullsized":"2026-02-25T11:47:17.657937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:37:21.791728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749858+0000","last_change":"2026-02-25T11:47:13.635800+0000","last_active":"2026-02-25T11:47:14.749858+0000","last_peered":"2026-02-25T11:47:14.749858+0000","last_clean":"2026-02-25T11:47:14.749858+0000","last_became_active":"2026-02-25T11:47:13.635590+0000","last_became_peered":"2026-02-25T11:47:13.635590+0000","last_unstale":"2026-02-25T11:47:14.749858+0000","last_undegraded":"2026-02-25T11:47:14.749858+0000","last_fullsized":"2026-02-25T11:47:14.749858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:46:36.324518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575342+0000","last_change":"2026-02-25T11:47:12.616473+0000","last_active":"2026-02-25T11:47:18.575342+0000","last_peered":"2026-02-25T11:47:18.575342+0000","last_clean":"2026-02-25T11:47:18.575342+0000","last_became_active":"2026-02-25T11:47:12.616039+0000","last_became_peered":"2026-02-25T11:47:12.616039+0000","last_unstale":"2026-02-25T11:47:18.575342+0000","last_undegraded":"2026-02-25T11:47:18.575342+0000","last_fullsized":"2026-02-25T11:47:18.575342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:28.324889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628413+0000","last_change":"2026-02-25T11:47:13.634771+0000","last_active":"2026-02-25T11:47:14.628413+0000","last_peered":"2026-02-25T11:47:14.628413+0000","last_clean":"2026-02-25T11:47:14.628413+0000","last_became_active":"2026-02-25T11:47:13.634672+0000","last_became_peered":"2026-02-25T11:47:13.634672+0000","last_unstale":"2026-02-25T11:47:14.628413+0000","last_undegraded":"2026-02-25T11:47:14.628413+0000","last_fullsized":"2026-02-25T11:47:14.628413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:19:03.692423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749863+0000","last_change":"2026-02-25T11:47:12.616685+0000","last_active":"2026-02-25T11:47:14.749863+0000","last_peered":"2026-02-25T11:47:14.749863+0000","last_clean":"2026-02-25T11:47:14.749863+0000","last_became_active":"2026-02-25T11:47:12.616518+0000","last_became_peered":"2026-02-25T11:47:12.616518+0000","last_unstale":"2026-02-25T11:47:14.749863+0000","last_undegraded":"2026-02-25T11:47:14.749863+0000","last_fullsized":"2026-02-25T11:47:14.749863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:28:50.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628462+0000","last_change":"2026-02-25T11:47:13.627734+0000","last_active":"2026-02-25T11:47:14.628462+0000","last_peered":"2026-02-25T11:47:14.628462+0000","last_clean":"2026-02-25T11:47:14.628462+0000","last_became_active":"2026-02-25T11:47:13.627632+0000","last_became_peered":"2026-02-25T11:47:13.627632+0000","last_unstale":"2026-02-25T11:47:14.628462+0000","last_undegraded":"2026-02-25T11:47:14.628462+0000","last_fullsized":"2026-02-25T11:47:14.628462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:01:20.907699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750104+0000","last_change":"2026-02-25T11:47:12.612830+0000","last_active":"2026-02-25T11:47:14.750104+0000","last_peered":"2026-02-25T11:47:14.750104+0000","last_clean":"2026-02-25T11:47:14.750104+0000","last_became_active":"2026-02-25T11:47:12.612661+0000","last_became_peered":"2026-02-25T11:47:12.612661+0000","last_unstale":"2026-02-25T11:47:14.750104+0000","last_undegraded":"2026-02-25T11:47:14.750104+0000","last_fullsized":"2026-02-25T11:47:14.750104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:05:15.367058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:15.914932+0000","last_change":"2026-02-25T11:47:13.633998+0000","last_active":"2026-02-25T11:47:15.914932+0000","last_peered":"2026-02-25T11:47:15.914932+0000","last_clean":"2026-02-25T11:47:15.914932+0000","last_became_active":"2026-02-25T11:47:13.633820+0000","last_became_peered":"2026-02-25T11:47:13.633820+0000","last_unstale":"2026-02-25T11:47:15.914932+0000","last_undegraded":"2026-02-25T11:47:15.914932+0000","last_fullsized":"2026-02-25T11:47:15.914932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:02:38.693106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.658417+0000","last_change":"2026-02-25T11:47:12.614091+0000","last_active":"2026-02-25T11:47:17.658417+0000","last_peered":"2026-02-25T11:47:17.658417+0000","last_clean":"2026-02-25T11:47:17.658417+0000","last_became_active":"2026-02-25T11:47:12.612962+0000","last_became_peered":"2026-02-25T11:47:12.612962+0000","last_unstale":"2026-02-25T11:47:17.658417+0000","last_undegraded":"2026-02-25T11:47:17.658417+0000","last_fullsized":"2026-02-25T11:47:17.658417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:11:40.214146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628504+0000","last_change":"2026-02-25T11:47:13.627288+0000","last_active":"2026-02-25T11:47:14.628504+0000","last_peered":"2026-02-25T11:47:14.628504+0000","last_clean":"2026-02-25T11:47:14.628504+0000","last_became_active":"2026-02-25T11:47:13.626902+0000","last_became_peered":"2026-02-25T11:47:13.626902+0000","last_unstale":"2026-02-25T11:47:14.628504+0000","last_undegraded":"2026-02-25T11:47:14.628504+0000","last_fullsized":"2026-02-25T11:47:14.628504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:38:19.137242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'152","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628095+0000","last_change":"2026-02-25T11:46:35.865282+0000","last_active":"2026-02-25T11:47:14.628095+0000","last_peered":"2026-02-25T11:47:14.628095+0000","last_clean":"2026-02-25T11:47:14.628095+0000","last_became_active":"2026-02-25T11:46:35.865139+0000","last_became_peered":"2026-02-25T11:46:35.865139+0000","last_unstale":"2026-02-25T11:47:14.628095+0000","last_undegraded":"2026-02-25T11:47:14.628095+0000","last_fullsized":"2026-02-25T11:47:14.628095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_clean_scrub_stamp":"2026-02-25T11:44:43.739673+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:29:57.573039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575170+0000","last_change":"2026-02-25T11:47:12.612834+0000","last_active":"2026-02-25T11:47:18.575170+0000","last_peered":"2026-02-25T11:47:18.575170+0000","last_clean":"2026-02-25T11:47:18.575170+0000","last_became_active":"2026-02-25T11:47:12.611827+0000","last_became_peered":"2026-02-25T11:47:12.611827+0000","last_unstale":"2026-02-25T11:47:18.575170+0000","last_undegraded":"2026-02-25T11:47:18.575170+0000","last_fullsized":"2026-02-25T11:47:18.575170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:05:32.112908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635504+0000","last_change":"2026-02-25T11:47:13.634534+0000","last_active":"2026-02-25T11:47:14.635504+0000","last_peered":"2026-02-25T11:47:14.635504+0000","last_clean":"2026-02-25T11:47:14.635504+0000","last_became_active":"2026-02-25T11:47:13.633769+0000","last_became_peered":"2026-02-25T11:47:13.633769+0000","last_unstale":"2026-02-25T11:47:14.635504+0000","last_undegraded":"2026-02-25T11:47:14.635504+0000","last_fullsized":"2026-02-25T11:47:14.635504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:16:09.255508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656541+0000","last_change":"2026-02-25T11:47:12.612621+0000","last_active":"2026-02-25T11:47:17.656541+0000","last_peered":"2026-02-25T11:47:17.656541+0000","last_clean":"2026-02-25T11:47:17.656541+0000","last_became_active":"2026-02-25T11:47:12.612284+0000","last_became_peered":"2026-02-25T11:47:12.612284+0000","last_unstale":"2026-02-25T11:47:17.656541+0000","last_undegraded":"2026-02-25T11:47:17.656541+0000","last_fullsized":"2026-02-25T11:47:17.656541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:49:09.299914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749514+0000","last_change":"2026-02-25T11:47:13.629107+0000","last_active":"2026-02-25T11:47:14.749514+0000","last_peered":"2026-02-25T11:47:14.749514+0000","last_clean":"2026-02-25T11:47:14.749514+0000","last_became_active":"2026-02-25T11:47:13.629031+0000","last_became_peered":"2026-02-25T11:47:13.629031+0000","last_unstale":"2026-02-25T11:47:14.749514+0000","last_undegraded":"2026-02-25T11:47:14.749514+0000","last_fullsized":"2026-02-25T11:47:14.749514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:07:58.915416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:46.606658+0000","last_change":"2026-02-25T11:47:12.616543+0000","last_active":"2026-02-25T11:47:46.606658+0000","last_peered":"2026-02-25T11:47:46.606658+0000","last_clean":"2026-02-25T11:47:46.606658+0000","last_became_active":"2026-02-25T11:47:12.616345+0000","last_became_peered":"2026-02-25T11:47:12.616345+0000","last_unstale":"2026-02-25T11:47:46.606658+0000","last_undegraded":"2026-02-25T11:47:46.606658+0000","last_fullsized":"2026-02-25T11:47:46.606658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:27:30.862168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750014+0000","last_change":"2026-02-25T11:47:13.637385+0000","last_active":"2026-02-25T11:47:14.750014+0000","last_peered":"2026-02-25T11:47:14.750014+0000","last_clean":"2026-02-25T11:47:14.750014+0000","last_became_active":"2026-02-25T11:47:13.634653+0000","last_became_peered":"2026-02-25T11:47:13.634653+0000","last_unstale":"2026-02-25T11:47:14.750014+0000","last_undegraded":"2026-02-25T11:47:14.750014+0000","last_fullsized":"2026-02-25T11:47:14.750014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:05:08.662208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:15.914921+0000","last_change":"2026-02-25T11:47:12.614640+0000","last_active":"2026-02-25T11:47:15.914921+0000","last_peered":"2026-02-25T11:47:15.914921+0000","last_clean":"2026-02-25T11:47:15.914921+0000","last_became_active":"2026-02-25T11:47:12.614486+0000","last_became_peered":"2026-02-25T11:47:12.614486+0000","last_unstale":"2026-02-25T11:47:15.914921+0000","last_undegraded":"2026-02-25T11:47:15.914921+0000","last_fullsized":"2026-02-25T11:47:15.914921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:16:06.314709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750327+0000","last_change":"2026-02-25T11:47:13.631828+0000","last_active":"2026-02-25T11:47:14.750327+0000","last_peered":"2026-02-25T11:47:14.750327+0000","last_clean":"2026-02-25T11:47:14.750327+0000","last_became_active":"2026-02-25T11:47:13.631590+0000","last_became_peered":"2026-02-25T11:47:13.631590+0000","last_unstale":"2026-02-25T11:47:14.750327+0000","last_undegraded":"2026-02-25T11:47:14.750327+0000","last_fullsized":"2026-02-25T11:47:14.750327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:25.518794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633984+0000","last_change":"2026-02-25T11:47:12.614032+0000","last_active":"2026-02-25T11:47:14.633984+0000","last_peered":"2026-02-25T11:47:14.633984+0000","last_clean":"2026-02-25T11:47:14.633984+0000","last_became_active":"2026-02-25T11:47:12.613936+0000","last_became_peered":"2026-02-25T11:47:12.613936+0000","last_unstale":"2026-02-25T11:47:14.633984+0000","last_undegraded":"2026-02-25T11:47:14.633984+0000","last_fullsized":"2026-02-25T11:47:14.633984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:16:46.397485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.859010+0000","last_change":"2026-02-25T11:47:13.636173+0000","last_active":"2026-02-25T11:47:16.859010+0000","last_peered":"2026-02-25T11:47:16.859010+0000","last_clean":"2026-02-25T11:47:16.859010+0000","last_became_active":"2026-02-25T11:47:13.635745+0000","last_became_peered":"2026-02-25T11:47:13.635745+0000","last_unstale":"2026-02-25T11:47:16.859010+0000","last_undegraded":"2026-02-25T11:47:16.859010+0000","last_fullsized":"2026-02-25T11:47:16.859010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:03:33.671318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651871+0000","last_change":"2026-02-25T11:47:12.614003+0000","last_active":"2026-02-25T11:47:16.651871+0000","last_peered":"2026-02-25T11:47:16.651871+0000","last_clean":"2026-02-25T11:47:16.651871+0000","last_became_active":"2026-02-25T11:47:12.613893+0000","last_became_peered":"2026-02-25T11:47:12.613893+0000","last_unstale":"2026-02-25T11:47:16.651871+0000","last_undegraded":"2026-02-25T11:47:16.651871+0000","last_fullsized":"2026-02-25T11:47:16.651871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:33:21.133364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749746+0000","last_change":"2026-02-25T11:47:13.632632+0000","last_active":"2026-02-25T11:47:14.749746+0000","last_peered":"2026-02-25T11:47:14.749746+0000","last_clean":"2026-02-25T11:47:14.749746+0000","last_became_active":"2026-02-25T11:47:13.632408+0000","last_became_peered":"2026-02-25T11:47:13.632408+0000","last_unstale":"2026-02-25T11:47:14.749746+0000","last_undegraded":"2026-02-25T11:47:14.749746+0000","last_fullsized":"2026-02-25T11:47:14.749746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:38:45.448637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:27.638764+0000","last_change":"2026-02-25T11:47:12.613816+0000","last_active":"2026-02-25T11:47:27.638764+0000","last_peered":"2026-02-25T11:47:27.638764+0000","last_clean":"2026-02-25T11:47:27.638764+0000","last_became_active":"2026-02-25T11:47:12.613196+0000","last_became_peered":"2026-02-25T11:47:12.613196+0000","last_unstale":"2026-02-25T11:47:27.638764+0000","last_undegraded":"2026-02-25T11:47:27.638764+0000","last_fullsized":"2026-02-25T11:47:27.638764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:23:52.372808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635752+0000","last_change":"2026-02-25T11:47:13.632431+0000","last_active":"2026-02-25T11:47:14.635752+0000","last_peered":"2026-02-25T11:47:14.635752+0000","last_clean":"2026-02-25T11:47:14.635752+0000","last_became_active":"2026-02-25T11:47:13.631972+0000","last_became_peered":"2026-02-25T11:47:13.631972+0000","last_unstale":"2026-02-25T11:47:14.635752+0000","last_undegraded":"2026-02-25T11:47:14.635752+0000","last_fullsized":"2026-02-25T11:47:14.635752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:59:34.655028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651131+0000","last_change":"2026-02-25T11:47:12.612805+0000","last_active":"2026-02-25T11:47:16.651131+0000","last_peered":"2026-02-25T11:47:16.651131+0000","last_clean":"2026-02-25T11:47:16.651131+0000","last_became_active":"2026-02-25T11:47:12.612155+0000","last_became_peered":"2026-02-25T11:47:12.612155+0000","last_unstale":"2026-02-25T11:47:16.651131+0000","last_undegraded":"2026-02-25T11:47:16.651131+0000","last_fullsized":"2026-02-25T11:47:16.651131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:58:00.429639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627853+0000","last_change":"2026-02-25T11:47:13.636089+0000","last_active":"2026-02-25T11:47:14.627853+0000","last_peered":"2026-02-25T11:47:14.627853+0000","last_clean":"2026-02-25T11:47:14.627853+0000","last_became_active":"2026-02-25T11:47:13.635963+0000","last_became_peered":"2026-02-25T11:47:13.635963+0000","last_unstale":"2026-02-25T11:47:14.627853+0000","last_undegraded":"2026-02-25T11:47:14.627853+0000","last_fullsized":"2026-02-25T11:47:14.627853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:50:28.012023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650081+0000","last_change":"2026-02-25T11:47:12.609749+0000","last_active":"2026-02-25T11:47:16.650081+0000","last_peered":"2026-02-25T11:47:16.650081+0000","last_clean":"2026-02-25T11:47:16.650081+0000","last_became_active":"2026-02-25T11:47:12.609190+0000","last_became_peered":"2026-02-25T11:47:12.609190+0000","last_unstale":"2026-02-25T11:47:16.650081+0000","last_undegraded":"2026-02-25T11:47:16.650081+0000","last_fullsized":"2026-02-25T11:47:16.650081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:12:42.204122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628201+0000","last_change":"2026-02-25T11:47:13.629339+0000","last_active":"2026-02-25T11:47:14.628201+0000","last_peered":"2026-02-25T11:47:14.628201+0000","last_clean":"2026-02-25T11:47:14.628201+0000","last_became_active":"2026-02-25T11:47:13.629107+0000","last_became_peered":"2026-02-25T11:47:13.629107+0000","last_unstale":"2026-02-25T11:47:14.628201+0000","last_undegraded":"2026-02-25T11:47:14.628201+0000","last_fullsized":"2026-02-25T11:47:14.628201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:19:50.087771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.652673+0000","last_change":"2026-02-25T11:47:12.615249+0000","last_active":"2026-02-25T11:47:16.652673+0000","last_peered":"2026-02-25T11:47:16.652673+0000","last_clean":"2026-02-25T11:47:16.652673+0000","last_became_active":"2026-02-25T11:47:12.615165+0000","last_became_peered":"2026-02-25T11:47:12.615165+0000","last_unstale":"2026-02-25T11:47:16.652673+0000","last_undegraded":"2026-02-25T11:47:16.652673+0000","last_fullsized":"2026-02-25T11:47:16.652673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:40:52.452074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627665+0000","last_change":"2026-02-25T11:47:13.631460+0000","last_active":"2026-02-25T11:47:14.627665+0000","last_peered":"2026-02-25T11:47:14.627665+0000","last_clean":"2026-02-25T11:47:14.627665+0000","last_became_active":"2026-02-25T11:47:13.631298+0000","last_became_peered":"2026-02-25T11:47:13.631298+0000","last_unstale":"2026-02-25T11:47:14.627665+0000","last_undegraded":"2026-02-25T11:47:14.627665+0000","last_fullsized":"2026-02-25T11:47:14.627665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:26:03.362106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.648950+0000","last_change":"2026-02-25T11:47:12.615779+0000","last_active":"2026-02-25T11:47:17.648950+0000","last_peered":"2026-02-25T11:47:17.648950+0000","last_clean":"2026-02-25T11:47:17.648950+0000","last_became_active":"2026-02-25T11:47:12.615073+0000","last_became_peered":"2026-02-25T11:47:12.615073+0000","last_unstale":"2026-02-25T11:47:17.648950+0000","last_undegraded":"2026-02-25T11:47:17.648950+0000","last_fullsized":"2026-02-25T11:47:17.648950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:28:51.468199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635380+0000","last_change":"2026-02-25T11:47:13.632351+0000","last_active":"2026-02-25T11:47:14.635380+0000","last_peered":"2026-02-25T11:47:14.635380+0000","last_clean":"2026-02-25T11:47:14.635380+0000","last_became_active":"2026-02-25T11:47:13.632081+0000","last_became_peered":"2026-02-25T11:47:13.632081+0000","last_unstale":"2026-02-25T11:47:14.635380+0000","last_undegraded":"2026-02-25T11:47:14.635380+0000","last_fullsized":"2026-02-25T11:47:14.635380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:38:59.143253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650353+0000","last_change":"2026-02-25T11:47:12.616555+0000","last_active":"2026-02-25T11:47:16.650353+0000","last_peered":"2026-02-25T11:47:16.650353+0000","last_clean":"2026-02-25T11:47:16.650353+0000","last_became_active":"2026-02-25T11:47:12.616174+0000","last_became_peered":"2026-02-25T11:47:12.616174+0000","last_unstale":"2026-02-25T11:47:16.650353+0000","last_undegraded":"2026-02-25T11:47:16.650353+0000","last_fullsized":"2026-02-25T11:47:16.650353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:01:17.494129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628458+0000","last_change":"2026-02-25T11:47:13.627400+0000","last_active":"2026-02-25T11:47:14.628458+0000","last_peered":"2026-02-25T11:47:14.628458+0000","last_clean":"2026-02-25T11:47:14.628458+0000","last_became_active":"2026-02-25T11:47:13.626782+0000","last_became_peered":"2026-02-25T11:47:13.626782+0000","last_unstale":"2026-02-25T11:47:14.628458+0000","last_undegraded":"2026-02-25T11:47:14.628458+0000","last_fullsized":"2026-02-25T11:47:14.628458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:42:39.996935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750027+0000","last_change":"2026-02-25T11:47:13.634030+0000","last_active":"2026-02-25T11:47:14.750027+0000","last_peered":"2026-02-25T11:47:14.750027+0000","last_clean":"2026-02-25T11:47:14.750027+0000","last_became_active":"2026-02-25T11:47:13.632844+0000","last_became_peered":"2026-02-25T11:47:13.632844+0000","last_unstale":"2026-02-25T11:47:14.750027+0000","last_undegraded":"2026-02-25T11:47:14.750027+0000","last_fullsized":"2026-02-25T11:47:14.750027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:52:16.013596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628805+0000","last_change":"2026-02-25T11:47:12.613666+0000","last_active":"2026-02-25T11:47:14.628805+0000","last_peered":"2026-02-25T11:47:14.628805+0000","last_clean":"2026-02-25T11:47:14.628805+0000","last_became_active":"2026-02-25T11:47:12.613570+0000","last_became_peered":"2026-02-25T11:47:12.613570+0000","last_unstale":"2026-02-25T11:47:14.628805+0000","last_undegraded":"2026-02-25T11:47:14.628805+0000","last_fullsized":"2026-02-25T11:47:14.628805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:28:39.750737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750924+0000","last_change":"2026-02-25T11:47:13.631515+0000","last_active":"2026-02-25T11:47:14.750924+0000","last_peered":"2026-02-25T11:47:14.750924+0000","last_clean":"2026-02-25T11:47:14.750924+0000","last_became_active":"2026-02-25T11:47:13.631161+0000","last_became_peered":"2026-02-25T11:47:13.631161+0000","last_unstale":"2026-02-25T11:47:14.750924+0000","last_undegraded":"2026-02-25T11:47:14.750924+0000","last_fullsized":"2026-02-25T11:47:14.750924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:13:30.496521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656891+0000","last_change":"2026-02-25T11:47:12.615904+0000","last_active":"2026-02-25T11:47:17.656891+0000","last_peered":"2026-02-25T11:47:17.656891+0000","last_clean":"2026-02-25T11:47:17.656891+0000","last_became_active":"2026-02-25T11:47:12.615784+0000","last_became_peered":"2026-02-25T11:47:12.615784+0000","last_unstale":"2026-02-25T11:47:17.656891+0000","last_undegraded":"2026-02-25T11:47:17.656891+0000","last_fullsized":"2026-02-25T11:47:17.656891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:32:26.802870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627754+0000","last_change":"2026-02-25T11:47:13.630065+0000","last_active":"2026-02-25T11:47:14.627754+0000","last_peered":"2026-02-25T11:47:14.627754+0000","last_clean":"2026-02-25T11:47:14.627754+0000","last_became_active":"2026-02-25T11:47:13.629214+0000","last_became_peered":"2026-02-25T11:47:13.629214+0000","last_unstale":"2026-02-25T11:47:14.627754+0000","last_undegraded":"2026-02-25T11:47:14.627754+0000","last_fullsized":"2026-02-25T11:47:14.627754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:53:45.639526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.310632+0000","last_change":"2026-02-25T11:47:12.612956+0000","last_active":"2026-02-25T11:47:16.310632+0000","last_peered":"2026-02-25T11:47:16.310632+0000","last_clean":"2026-02-25T11:47:16.310632+0000","last_became_active":"2026-02-25T11:47:12.612843+0000","last_became_peered":"2026-02-25T11:47:12.612843+0000","last_unstale":"2026-02-25T11:47:16.310632+0000","last_undegraded":"2026-02-25T11:47:16.310632+0000","last_fullsized":"2026-02-25T11:47:16.310632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:41:30.867465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635780+0000","last_change":"2026-02-25T11:47:13.637712+0000","last_active":"2026-02-25T11:47:14.635780+0000","last_peered":"2026-02-25T11:47:14.635780+0000","last_clean":"2026-02-25T11:47:14.635780+0000","last_became_active":"2026-02-25T11:47:13.635030+0000","last_became_peered":"2026-02-25T11:47:13.635030+0000","last_unstale":"2026-02-25T11:47:14.635780+0000","last_undegraded":"2026-02-25T11:47:14.635780+0000","last_fullsized":"2026-02-25T11:47:14.635780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:01:22.896129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656736+0000","last_change":"2026-02-25T11:47:12.614149+0000","last_active":"2026-02-25T11:47:17.656736+0000","last_peered":"2026-02-25T11:47:17.656736+0000","last_clean":"2026-02-25T11:47:17.656736+0000","last_became_active":"2026-02-25T11:47:12.613137+0000","last_became_peered":"2026-02-25T11:47:12.613137+0000","last_unstale":"2026-02-25T11:47:17.656736+0000","last_undegraded":"2026-02-25T11:47:17.656736+0000","last_fullsized":"2026-02-25T11:47:17.656736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:49:49.451435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73639,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":203,"num_read_kb":178,"num_write":168,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":450560,"data_stored":228821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":227295,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":262884,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28132,"kb_used_data":912,"kb_used_omap":9,"kb_used_meta":27190,"kb_avail":733975068,"statfs":{"total":751619276800,"available":751590469632,"internally_reserved":0,"allocated":933888,"data_stored":455512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10109,"internal_metadata":27842691},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612625,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28288,"kb_used_data":1068,"kb_used_omap":28,"kb_used_meta":27171,"kb_avail":733974912,"statfs":{"total":751619276800,"available":751590309888,"internally_reserved":0,"allocated":1093632,"data_stored":1116443,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":29123,"internal_metadata":27823677},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776148,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28148,"kb_used_data":928,"kb_used_omap":27,"kb_used_meta":27172,"kb_avail":733975052,"statfs":{"total":751619276800,"available":751590453248,"internally_reserved":0,"allocated":950272,"data_stored":455748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28131,"internal_metadata":27824669},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037782,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28148,"kb_used_data":928,"kb_used_omap":35,"kb_used_meta":27164,"kb_avail":733975052,"statfs":{"total":751619276800,"available":751590453248,"internally_reserved":0,"allocated":950272,"data_stored":455668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36257,"internal_metadata":27816543},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299417,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28196,"kb_used_data":976,"kb_used_omap":35,"kb_used_meta":27164,"kb_avail":733975004,"statfs":{"total":751619276800,"available":751590404096,"internally_reserved":0,"allocated":999424,"data_stored":1047617,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36168,"internal_metadata":27816632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28136,"kb_used_data":916,"kb_used_omap":31,"kb_used_meta":27168,"kb_avail":733975064,"statfs":{"total":751619276800,"available":751590465536,"internally_reserved":0,"allocated":937984,"data_stored":455115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32619,"internal_metadata":27820181},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724575,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28208,"kb_used_data":988,"kb_used_omap":36,"kb_used_meta":27163,"kb_avail":733974992,"statfs":{"total":751619276800,"available":751590391808,"internally_reserved":0,"allocated":1011712,"data_stored":1047404,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36952,"internal_metadata":27815848},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888097,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28140,"kb_used_data":920,"kb_used_omap":45,"kb_used_meta":27154,"kb_avail":733975060,"statfs":{"total":751619276800,"available":751590461440,"internally_reserved":0,"allocated":942080,"data_stored":455962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":46332,"internal_metadata":27806468},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149732,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28152,"kb_used_data":932,"kb_used_omap":51,"kb_used_meta":27148,"kb_avail":733975048,"statfs":{"total":751619276800,"available":751590449152,"internally_reserved":0,"allocated":954368,"data_stored":455426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52913,"internal_metadata":27799887},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378663,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28236,"kb_used_data":1016,"kb_used_omap":64,"kb_used_meta":27135,"kb_avail":733974964,"statfs":{"total":751619276800,"available":751590363136,"internally_reserved":0,"allocated":1040384,"data_stored":534250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":66234,"internal_metadata":27786566},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28204,"kb_used_data":984,"kb_used_omap":43,"kb_used_meta":27156,"kb_avail":733974996,"statfs":{"total":751619276800,"available":751590395904,"internally_reserved":0,"allocated":1007616,"data_stored":523948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44488,"internal_metadata":27808312},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28156,"kb_used_data":936,"kb_used_omap":54,"kb_used_meta":27145,"kb_avail":733975044,"statfs":{"total":751619276800,"available":751590445056,"internally_reserved":0,"allocated":958464,"data_stored":456430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56100,"internal_metadata":27796700},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8853,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1407,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":79342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56705,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49702,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6449,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3478,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14040,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9858,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25902,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22346,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21693,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6862,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T11:47:50.127 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T11:47:50.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.275+0000 7fe29a8c3640 1 -- 10.20.193.28:0/2433207501 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe2941147d0 msgr2=0x7fe294114c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:50.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.275+0000 7fe29a8c3640 1 --2- 10.20.193.28:0/2433207501 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe2941147d0 0x7fe294114c40 secure :-1 s=READY pgs=351 gs=1 cs=0 l=1 c_cookie=6730589831534989335 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe288009950 tx=0x7fe28802f190 comp rx=0 tx=0).stop 2026-02-25T11:47:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.276+0000 7fe29a8c3640 1 -- 10.20.193.28:0/2433207501 shutdown_connections 2026-02-25T11:47:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.276+0000 7fe29a8c3640 1 --2- 10.20.193.28:0/2433207501 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe294115180 0x7fe294175af0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.276+0000 7fe29a8c3640 1 --2- 10.20.193.28:0/2433207501 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe2941147d0 0x7fe294114c40 unknown :-1 s=CLOSED pgs=351 gs=1 cs=0 l=1 c_cookie=6730589831534989335 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.276+0000 7fe29a8c3640 1 --2- 10.20.193.28:0/2433207501 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe294113910 0x7fe294113d00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.276+0000 7fe29a8c3640 1 -- 10.20.193.28:0/2433207501 >> 10.20.193.28:0/2433207501 conn(0x7fe294064570 msgr2=0x7fe294064990 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.276+0000 7fe29a8c3640 1 -- 10.20.193.28:0/2433207501 shutdown_connections 2026-02-25T11:47:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.276+0000 7fe29a8c3640 1 -- 10.20.193.28:0/2433207501 wait complete. 2026-02-25T11:47:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.276+0000 7fe29a8c3640 1 Processor -- start 2026-02-25T11:47:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.276+0000 7fe29a8c3640 1 -- start start 2026-02-25T11:47:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a8c3640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe294113910 0x7fe2940feb40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a8c3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe2941147d0 0x7fe294175af0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a8c3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe294115180 0x7fe2940ff3a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a8c3640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe2940fff40 con 0x7fe294115180 2026-02-25T11:47:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a8c3640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe2941000b0 con 0x7fe2941147d0 2026-02-25T11:47:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a8c3640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe294100270 con 0x7fe294113910 2026-02-25T11:47:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a0c2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe294115180 0x7fe2940ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe2990c0640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe2941147d0 0x7fe294175af0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a0c2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe294115180 0x7fe2940ff3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59496/0 (socket says 10.20.193.28:59496) 2026-02-25T11:47:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a0c2640 1 -- 10.20.193.28:0/3159462517 learned_addr learned my addr 10.20.193.28:0/3159462517 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe2998c1640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe294113910 0x7fe2940feb40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a0c2640 1 -- 10.20.193.28:0/3159462517 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe294113910 msgr2=0x7fe2940feb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a0c2640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe294113910 0x7fe2940feb40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a0c2640 1 -- 10.20.193.28:0/3159462517 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe2941147d0 msgr2=0x7fe294175af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a0c2640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe2941147d0 0x7fe294175af0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a0c2640 1 -- 10.20.193.28:0/3159462517 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe288009570 con 0x7fe294115180 2026-02-25T11:47:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe2998c1640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe294113910 0x7fe2940feb40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe29a0c2640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe294115180 0x7fe2940ff3a0 secure :-1 s=READY pgs=352 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe29000f8d0 tx=0x7fe29000fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.277+0000 7fe282ffd640 1 -- 10.20.193.28:0/3159462517 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe29000dce0 con 0x7fe294115180 2026-02-25T11:47:50.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.278+0000 7fe282ffd640 1 -- 10.20.193.28:0/3159462517 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe29000de40 con 0x7fe294115180 2026-02-25T11:47:50.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.278+0000 7fe282ffd640 1 -- 10.20.193.28:0/3159462517 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe2900116b0 con 0x7fe294115180 2026-02-25T11:47:50.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.278+0000 7fe29a8c3640 1 -- 10.20.193.28:0/3159462517 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe294100550 con 0x7fe294115180 2026-02-25T11:47:50.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.278+0000 7fe29a8c3640 1 -- 10.20.193.28:0/3159462517 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2941009d0 con 0x7fe294115180 2026-02-25T11:47:50.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.278+0000 7fe29a8c3640 1 -- 10.20.193.28:0/3159462517 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2941799a0 con 0x7fe294115180 2026-02-25T11:47:50.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.280+0000 7fe282ffd640 1 -- 10.20.193.28:0/3159462517 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fe290020060 con 0x7fe294115180 2026-02-25T11:47:50.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.282+0000 7fe282ffd640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe2700b57f0 0x7fe2700d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:50.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.282+0000 7fe282ffd640 1 -- 10.20.193.28:0/3159462517 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fe290015070 con 0x7fe294115180 2026-02-25T11:47:50.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.282+0000 7fe282ffd640 1 -- 10.20.193.28:0/3159462517 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fe290120960 con 0x7fe294115180 2026-02-25T11:47:50.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.282+0000 7fe2998c1640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe2700b57f0 0x7fe2700d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:50.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.282+0000 7fe2998c1640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe2700b57f0 0x7fe2700d5ca0 secure :-1 s=READY pgs=249 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe284005970 tx=0x7fe284005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:50.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.394+0000 7fe29a8c3640 1 -- 10.20.193.28:0/3159462517 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe294179120 con 0x7fe2700b57f0 2026-02-25T11:47:50.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.397+0000 7fe282ffd640 1 -- 10.20.193.28:0/3159462517 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258968 (secure 0 0 0) 0x7fe294179120 con 0x7fe2700b57f0 2026-02-25T11:47:50.402 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:50.404 INFO:teuthology.orchestra.run.trial028.stderr:dumped all 2026-02-25T11:47:50.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.399+0000 7fe29a8c3640 1 -- 10.20.193.28:0/3159462517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe2700b57f0 msgr2=0x7fe2700d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:50.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.400+0000 7fe29a8c3640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe2700b57f0 0x7fe2700d5ca0 secure :-1 s=READY pgs=249 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe284005970 tx=0x7fe284005900 comp rx=0 tx=0).stop 2026-02-25T11:47:50.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.400+0000 7fe29a8c3640 1 -- 10.20.193.28:0/3159462517 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe294115180 msgr2=0x7fe2940ff3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:50.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.400+0000 7fe29a8c3640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe294115180 0x7fe2940ff3a0 secure :-1 s=READY pgs=352 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe29000f8d0 tx=0x7fe29000fda0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.400+0000 7fe29a8c3640 1 -- 10.20.193.28:0/3159462517 shutdown_connections 2026-02-25T11:47:50.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.400+0000 7fe29a8c3640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe2700b57f0 0x7fe2700d5ca0 unknown :-1 s=CLOSED pgs=249 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.400+0000 7fe29a8c3640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe294115180 0x7fe2940ff3a0 unknown :-1 s=CLOSED pgs=352 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.400+0000 7fe29a8c3640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe2941147d0 0x7fe294175af0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.400+0000 7fe29a8c3640 1 --2- 10.20.193.28:0/3159462517 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe294113910 0x7fe2940feb40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.400+0000 7fe29a8c3640 1 -- 10.20.193.28:0/3159462517 >> 10.20.193.28:0/3159462517 conn(0x7fe294064570 msgr2=0x7fe2940f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:50.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.400+0000 7fe29a8c3640 1 -- 10.20.193.28:0/3159462517 shutdown_connections 2026-02-25T11:47:50.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.400+0000 7fe29a8c3640 1 -- 10.20.193.28:0/3159462517 wait complete. 2026-02-25T11:47:50.413 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:50 trial028 ceph-mon[39224]: pgmap v212: 97 pgs: 97 active+clean; 649 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2026-02-25T11:47:50.413 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:50 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2353914324' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2026-02-25T11:47:50.414 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:50 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/782240480' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:50.415 INFO:teuthology.orchestra.run.trial028.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2026-02-25T11:47:49.080274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664241,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":309,"num_read_kb":269,"num_write":359,"num_write_kb":3847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":297,"ondisk_log_size":297,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":8808038400,"kb_used":338144,"kb_used_data":11504,"kb_used_omap":464,"kb_used_meta":325935,"kb_avail":8807700256,"statfs":{"total":9019431321600,"available":9019085062144,"internally_reserved":0,"allocated":11780096,"data_stored":7459523,"data_compressed":114024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":475426,"internal_metadata":333758174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3098,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":39,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002180"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627807+0000","last_change":"2026-02-25T11:47:13.630492+0000","last_active":"2026-02-25T11:47:14.627807+0000","last_peered":"2026-02-25T11:47:14.627807+0000","last_clean":"2026-02-25T11:47:14.627807+0000","last_became_active":"2026-02-25T11:47:13.630388+0000","last_became_peered":"2026-02-25T11:47:13.630388+0000","last_unstale":"2026-02-25T11:47:14.627807+0000","last_undegraded":"2026-02-25T11:47:14.627807+0000","last_fullsized":"2026-02-25T11:47:14.627807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:01:17.024058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627657+0000","last_change":"2026-02-25T11:47:13.629507+0000","last_active":"2026-02-25T11:47:14.627657+0000","last_peered":"2026-02-25T11:47:14.627657+0000","last_clean":"2026-02-25T11:47:14.627657+0000","last_became_active":"2026-02-25T11:47:13.628937+0000","last_became_peered":"2026-02-25T11:47:13.628937+0000","last_unstale":"2026-02-25T11:47:14.627657+0000","last_undegraded":"2026-02-25T11:47:14.627657+0000","last_fullsized":"2026-02-25T11:47:14.627657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:47:20.441522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627713+0000","last_change":"2026-02-25T11:47:13.630415+0000","last_active":"2026-02-25T11:47:14.627713+0000","last_peered":"2026-02-25T11:47:14.627713+0000","last_clean":"2026-02-25T11:47:14.627713+0000","last_became_active":"2026-02-25T11:47:13.630319+0000","last_became_peered":"2026-02-25T11:47:13.630319+0000","last_unstale":"2026-02-25T11:47:14.627713+0000","last_undegraded":"2026-02-25T11:47:14.627713+0000","last_fullsized":"2026-02-25T11:47:14.627713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:18:57.675760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627800+0000","last_change":"2026-02-25T11:47:13.633017+0000","last_active":"2026-02-25T11:47:14.627800+0000","last_peered":"2026-02-25T11:47:14.627800+0000","last_clean":"2026-02-25T11:47:14.627800+0000","last_became_active":"2026-02-25T11:47:13.632934+0000","last_became_peered":"2026-02-25T11:47:13.632934+0000","last_unstale":"2026-02-25T11:47:14.627800+0000","last_undegraded":"2026-02-25T11:47:14.627800+0000","last_fullsized":"2026-02-25T11:47:14.627800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:08:54.195980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628667+0000","last_change":"2026-02-25T11:47:13.638057+0000","last_active":"2026-02-25T11:47:14.628667+0000","last_peered":"2026-02-25T11:47:14.628667+0000","last_clean":"2026-02-25T11:47:14.628667+0000","last_became_active":"2026-02-25T11:47:13.636387+0000","last_became_peered":"2026-02-25T11:47:13.636387+0000","last_unstale":"2026-02-25T11:47:14.628667+0000","last_undegraded":"2026-02-25T11:47:14.628667+0000","last_fullsized":"2026-02-25T11:47:14.628667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:34:29.570622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750630+0000","last_change":"2026-02-25T11:47:13.631824+0000","last_active":"2026-02-25T11:47:14.750630+0000","last_peered":"2026-02-25T11:47:14.750630+0000","last_clean":"2026-02-25T11:47:14.750630+0000","last_became_active":"2026-02-25T11:47:13.631656+0000","last_became_peered":"2026-02-25T11:47:13.631656+0000","last_unstale":"2026-02-25T11:47:14.750630+0000","last_undegraded":"2026-02-25T11:47:14.750630+0000","last_fullsized":"2026-02-25T11:47:14.750630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:15:05.237015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628186+0000","last_change":"2026-02-25T11:47:13.626877+0000","last_active":"2026-02-25T11:47:14.628186+0000","last_peered":"2026-02-25T11:47:14.628186+0000","last_clean":"2026-02-25T11:47:14.628186+0000","last_became_active":"2026-02-25T11:47:13.626543+0000","last_became_peered":"2026-02-25T11:47:13.626543+0000","last_unstale":"2026-02-25T11:47:14.628186+0000","last_undegraded":"2026-02-25T11:47:14.628186+0000","last_fullsized":"2026-02-25T11:47:14.628186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:56:58.194911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635224+0000","last_change":"2026-02-25T11:47:13.637661+0000","last_active":"2026-02-25T11:47:14.635224+0000","last_peered":"2026-02-25T11:47:14.635224+0000","last_clean":"2026-02-25T11:47:14.635224+0000","last_became_active":"2026-02-25T11:47:13.637230+0000","last_became_peered":"2026-02-25T11:47:13.637230+0000","last_unstale":"2026-02-25T11:47:14.635224+0000","last_undegraded":"2026-02-25T11:47:14.635224+0000","last_fullsized":"2026-02-25T11:47:14.635224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:44:52.310360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751287+0000","last_change":"2026-02-25T11:47:13.625160+0000","last_active":"2026-02-25T11:47:14.751287+0000","last_peered":"2026-02-25T11:47:14.751287+0000","last_clean":"2026-02-25T11:47:14.751287+0000","last_became_active":"2026-02-25T11:47:13.625020+0000","last_became_peered":"2026-02-25T11:47:13.625020+0000","last_unstale":"2026-02-25T11:47:14.751287+0000","last_undegraded":"2026-02-25T11:47:14.751287+0000","last_fullsized":"2026-02-25T11:47:14.751287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:35:44.509551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750832+0000","last_change":"2026-02-25T11:47:13.637252+0000","last_active":"2026-02-25T11:47:14.750832+0000","last_peered":"2026-02-25T11:47:14.750832+0000","last_clean":"2026-02-25T11:47:14.750832+0000","last_became_active":"2026-02-25T11:47:13.636670+0000","last_became_peered":"2026-02-25T11:47:13.636670+0000","last_unstale":"2026-02-25T11:47:14.750832+0000","last_undegraded":"2026-02-25T11:47:14.750832+0000","last_fullsized":"2026-02-25T11:47:14.750832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:01:53.119101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628443+0000","last_change":"2026-02-25T11:47:13.628252+0000","last_active":"2026-02-25T11:47:14.628443+0000","last_peered":"2026-02-25T11:47:14.628443+0000","last_clean":"2026-02-25T11:47:14.628443+0000","last_became_active":"2026-02-25T11:47:13.628183+0000","last_became_peered":"2026-02-25T11:47:13.628183+0000","last_unstale":"2026-02-25T11:47:14.628443+0000","last_undegraded":"2026-02-25T11:47:14.628443+0000","last_fullsized":"2026-02-25T11:47:14.628443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:12:00.576747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.632358+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.631487+0000","last_became_peered":"2026-02-25T11:47:13.631487+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:41:17.150851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749735+0000","last_change":"2026-02-25T11:47:13.629675+0000","last_active":"2026-02-25T11:47:14.749735+0000","last_peered":"2026-02-25T11:47:14.749735+0000","last_clean":"2026-02-25T11:47:14.749735+0000","last_became_active":"2026-02-25T11:47:13.628686+0000","last_became_peered":"2026-02-25T11:47:13.628686+0000","last_unstale":"2026-02-25T11:47:14.749735+0000","last_undegraded":"2026-02-25T11:47:14.749735+0000","last_fullsized":"2026-02-25T11:47:14.749735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:00:54.430287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627615+0000","last_change":"2026-02-25T11:47:13.628842+0000","last_active":"2026-02-25T11:47:14.627615+0000","last_peered":"2026-02-25T11:47:14.627615+0000","last_clean":"2026-02-25T11:47:14.627615+0000","last_became_active":"2026-02-25T11:47:13.628695+0000","last_became_peered":"2026-02-25T11:47:13.628695+0000","last_unstale":"2026-02-25T11:47:14.627615+0000","last_undegraded":"2026-02-25T11:47:14.627615+0000","last_fullsized":"2026-02-25T11:47:14.627615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:10:05.404740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.635652+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.635505+0000","last_became_peered":"2026-02-25T11:47:13.635505+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:46:03.598577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749928+0000","last_change":"2026-02-25T11:47:13.628651+0000","last_active":"2026-02-25T11:47:14.749928+0000","last_peered":"2026-02-25T11:47:14.749928+0000","last_clean":"2026-02-25T11:47:14.749928+0000","last_became_active":"2026-02-25T11:47:13.628280+0000","last_became_peered":"2026-02-25T11:47:13.628280+0000","last_unstale":"2026-02-25T11:47:14.749928+0000","last_undegraded":"2026-02-25T11:47:14.749928+0000","last_fullsized":"2026-02-25T11:47:14.749928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:18:56.238078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628351+0000","last_change":"2026-02-25T11:47:13.630656+0000","last_active":"2026-02-25T11:47:14.628351+0000","last_peered":"2026-02-25T11:47:14.628351+0000","last_clean":"2026-02-25T11:47:14.628351+0000","last_became_active":"2026-02-25T11:47:13.630513+0000","last_became_peered":"2026-02-25T11:47:13.630513+0000","last_unstale":"2026-02-25T11:47:14.628351+0000","last_undegraded":"2026-02-25T11:47:14.628351+0000","last_fullsized":"2026-02-25T11:47:14.628351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:03:33.580576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.634036+0000","last_change":"2026-02-25T11:47:13.629202+0000","last_active":"2026-02-25T11:47:14.634036+0000","last_peered":"2026-02-25T11:47:14.634036+0000","last_clean":"2026-02-25T11:47:14.634036+0000","last_became_active":"2026-02-25T11:47:13.628860+0000","last_became_peered":"2026-02-25T11:47:13.628860+0000","last_unstale":"2026-02-25T11:47:14.634036+0000","last_undegraded":"2026-02-25T11:47:14.634036+0000","last_fullsized":"2026-02-25T11:47:14.634036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:47:14.635796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628235+0000","last_change":"2026-02-25T11:47:13.638872+0000","last_active":"2026-02-25T11:47:14.628235+0000","last_peered":"2026-02-25T11:47:14.628235+0000","last_clean":"2026-02-25T11:47:14.628235+0000","last_became_active":"2026-02-25T11:47:13.638443+0000","last_became_peered":"2026-02-25T11:47:13.638443+0000","last_unstale":"2026-02-25T11:47:14.628235+0000","last_undegraded":"2026-02-25T11:47:14.628235+0000","last_fullsized":"2026-02-25T11:47:14.628235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:51.699838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628595+0000","last_change":"2026-02-25T11:47:13.635894+0000","last_active":"2026-02-25T11:47:14.628595+0000","last_peered":"2026-02-25T11:47:14.628595+0000","last_clean":"2026-02-25T11:47:14.628595+0000","last_became_active":"2026-02-25T11:47:13.635725+0000","last_became_peered":"2026-02-25T11:47:13.635725+0000","last_unstale":"2026-02-25T11:47:14.628595+0000","last_undegraded":"2026-02-25T11:47:14.628595+0000","last_fullsized":"2026-02-25T11:47:14.628595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:28:20.134454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749794+0000","last_change":"2026-02-25T11:47:13.628117+0000","last_active":"2026-02-25T11:47:14.749794+0000","last_peered":"2026-02-25T11:47:14.749794+0000","last_clean":"2026-02-25T11:47:14.749794+0000","last_became_active":"2026-02-25T11:47:13.628037+0000","last_became_peered":"2026-02-25T11:47:13.628037+0000","last_unstale":"2026-02-25T11:47:14.749794+0000","last_undegraded":"2026-02-25T11:47:14.749794+0000","last_fullsized":"2026-02-25T11:47:14.749794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:28:22.976133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750830+0000","last_change":"2026-02-25T11:47:13.634652+0000","last_active":"2026-02-25T11:47:14.750830+0000","last_peered":"2026-02-25T11:47:14.750830+0000","last_clean":"2026-02-25T11:47:14.750830+0000","last_became_active":"2026-02-25T11:47:13.634433+0000","last_became_peered":"2026-02-25T11:47:13.634433+0000","last_unstale":"2026-02-25T11:47:14.750830+0000","last_undegraded":"2026-02-25T11:47:14.750830+0000","last_fullsized":"2026-02-25T11:47:14.750830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:01:29.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635319+0000","last_change":"2026-02-25T11:47:13.632721+0000","last_active":"2026-02-25T11:47:14.635319+0000","last_peered":"2026-02-25T11:47:14.635319+0000","last_clean":"2026-02-25T11:47:14.635319+0000","last_became_active":"2026-02-25T11:47:13.632626+0000","last_became_peered":"2026-02-25T11:47:13.632626+0000","last_unstale":"2026-02-25T11:47:14.635319+0000","last_undegraded":"2026-02-25T11:47:14.635319+0000","last_fullsized":"2026-02-25T11:47:14.635319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:28:35.201966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749844+0000","last_change":"2026-02-25T11:47:13.637504+0000","last_active":"2026-02-25T11:47:14.749844+0000","last_peered":"2026-02-25T11:47:14.749844+0000","last_clean":"2026-02-25T11:47:14.749844+0000","last_became_active":"2026-02-25T11:47:13.636824+0000","last_became_peered":"2026-02-25T11:47:13.636824+0000","last_unstale":"2026-02-25T11:47:14.749844+0000","last_undegraded":"2026-02-25T11:47:14.749844+0000","last_fullsized":"2026-02-25T11:47:14.749844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:20:31.416302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635724+0000","last_change":"2026-02-25T11:47:13.633135+0000","last_active":"2026-02-25T11:47:14.635724+0000","last_peered":"2026-02-25T11:47:14.635724+0000","last_clean":"2026-02-25T11:47:14.635724+0000","last_became_active":"2026-02-25T11:47:13.632966+0000","last_became_peered":"2026-02-25T11:47:13.632966+0000","last_unstale":"2026-02-25T11:47:14.635724+0000","last_undegraded":"2026-02-25T11:47:14.635724+0000","last_fullsized":"2026-02-25T11:47:14.635724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:26:17.052924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628538+0000","last_change":"2026-02-25T11:47:13.633210+0000","last_active":"2026-02-25T11:47:14.628538+0000","last_peered":"2026-02-25T11:47:14.628538+0000","last_clean":"2026-02-25T11:47:14.628538+0000","last_became_active":"2026-02-25T11:47:13.633099+0000","last_became_peered":"2026-02-25T11:47:13.633099+0000","last_unstale":"2026-02-25T11:47:14.628538+0000","last_undegraded":"2026-02-25T11:47:14.628538+0000","last_fullsized":"2026-02-25T11:47:14.628538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:07:46.884629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628656+0000","last_change":"2026-02-25T11:47:13.634757+0000","last_active":"2026-02-25T11:47:14.628656+0000","last_peered":"2026-02-25T11:47:14.628656+0000","last_clean":"2026-02-25T11:47:14.628656+0000","last_became_active":"2026-02-25T11:47:13.634643+0000","last_became_peered":"2026-02-25T11:47:13.634643+0000","last_unstale":"2026-02-25T11:47:14.628656+0000","last_undegraded":"2026-02-25T11:47:14.628656+0000","last_fullsized":"2026-02-25T11:47:14.628656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:20:23.383359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749674+0000","last_change":"2026-02-25T11:47:13.630515+0000","last_active":"2026-02-25T11:47:14.749674+0000","last_peered":"2026-02-25T11:47:14.749674+0000","last_clean":"2026-02-25T11:47:14.749674+0000","last_became_active":"2026-02-25T11:47:13.630202+0000","last_became_peered":"2026-02-25T11:47:13.630202+0000","last_unstale":"2026-02-25T11:47:14.749674+0000","last_undegraded":"2026-02-25T11:47:14.749674+0000","last_fullsized":"2026-02-25T11:47:14.749674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:51:24.646614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627367+0000","last_change":"2026-02-25T11:47:13.635749+0000","last_active":"2026-02-25T11:47:14.627367+0000","last_peered":"2026-02-25T11:47:14.627367+0000","last_clean":"2026-02-25T11:47:14.627367+0000","last_became_active":"2026-02-25T11:47:13.635284+0000","last_became_peered":"2026-02-25T11:47:13.635284+0000","last_unstale":"2026-02-25T11:47:14.627367+0000","last_undegraded":"2026-02-25T11:47:14.627367+0000","last_fullsized":"2026-02-25T11:47:14.627367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:23:45.042142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628732+0000","last_change":"2026-02-25T11:47:13.635448+0000","last_active":"2026-02-25T11:47:14.628732+0000","last_peered":"2026-02-25T11:47:14.628732+0000","last_clean":"2026-02-25T11:47:14.628732+0000","last_became_active":"2026-02-25T11:47:13.635232+0000","last_became_peered":"2026-02-25T11:47:13.635232+0000","last_unstale":"2026-02-25T11:47:14.628732+0000","last_undegraded":"2026-02-25T11:47:14.628732+0000","last_fullsized":"2026-02-25T11:47:14.628732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:33:38.020605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628332+0000","last_change":"2026-02-25T11:47:13.634237+0000","last_active":"2026-02-25T11:47:14.628332+0000","last_peered":"2026-02-25T11:47:14.628332+0000","last_clean":"2026-02-25T11:47:14.628332+0000","last_became_active":"2026-02-25T11:47:13.634054+0000","last_became_peered":"2026-02-25T11:47:13.634054+0000","last_unstale":"2026-02-25T11:47:14.628332+0000","last_undegraded":"2026-02-25T11:47:14.628332+0000","last_fullsized":"2026-02-25T11:47:14.628332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:38:34.603077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627786+0000","last_change":"2026-02-25T11:47:13.638347+0000","last_active":"2026-02-25T11:47:14.627786+0000","last_peered":"2026-02-25T11:47:14.627786+0000","last_clean":"2026-02-25T11:47:14.627786+0000","last_became_active":"2026-02-25T11:47:13.637984+0000","last_became_peered":"2026-02-25T11:47:13.637984+0000","last_unstale":"2026-02-25T11:47:14.627786+0000","last_undegraded":"2026-02-25T11:47:14.627786+0000","last_fullsized":"2026-02-25T11:47:14.627786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:36:11.273078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628202+0000","last_change":"2026-02-25T11:47:13.634523+0000","last_active":"2026-02-25T11:47:14.628202+0000","last_peered":"2026-02-25T11:47:14.628202+0000","last_clean":"2026-02-25T11:47:14.628202+0000","last_became_active":"2026-02-25T11:47:13.634184+0000","last_became_peered":"2026-02-25T11:47:13.634184+0000","last_unstale":"2026-02-25T11:47:14.628202+0000","last_undegraded":"2026-02-25T11:47:14.628202+0000","last_fullsized":"2026-02-25T11:47:14.628202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:52:23.917090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:47.599052+0000","last_change":"2026-02-25T11:47:12.613380+0000","last_active":"2026-02-25T11:47:47.599052+0000","last_peered":"2026-02-25T11:47:47.599052+0000","last_clean":"2026-02-25T11:47:47.599052+0000","last_became_active":"2026-02-25T11:47:12.613224+0000","last_became_peered":"2026-02-25T11:47:12.613224+0000","last_unstale":"2026-02-25T11:47:47.599052+0000","last_undegraded":"2026-02-25T11:47:47.599052+0000","last_fullsized":"2026-02-25T11:47:47.599052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:59:39.894067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1135,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749915+0000","last_change":"2026-02-25T11:47:13.636227+0000","last_active":"2026-02-25T11:47:14.749915+0000","last_peered":"2026-02-25T11:47:14.749915+0000","last_clean":"2026-02-25T11:47:14.749915+0000","last_became_active":"2026-02-25T11:47:13.635212+0000","last_became_peered":"2026-02-25T11:47:13.635212+0000","last_unstale":"2026-02-25T11:47:14.749915+0000","last_undegraded":"2026-02-25T11:47:14.749915+0000","last_fullsized":"2026-02-25T11:47:14.749915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:51:07.635399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657308+0000","last_change":"2026-02-25T11:47:12.613789+0000","last_active":"2026-02-25T11:47:17.657308+0000","last_peered":"2026-02-25T11:47:17.657308+0000","last_clean":"2026-02-25T11:47:17.657308+0000","last_became_active":"2026-02-25T11:47:12.613635+0000","last_became_peered":"2026-02-25T11:47:12.613635+0000","last_unstale":"2026-02-25T11:47:17.657308+0000","last_undegraded":"2026-02-25T11:47:17.657308+0000","last_fullsized":"2026-02-25T11:47:17.657308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:13:32.216593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657439+0000","last_change":"2026-02-25T11:47:12.612012+0000","last_active":"2026-02-25T11:47:17.657439+0000","last_peered":"2026-02-25T11:47:17.657439+0000","last_clean":"2026-02-25T11:47:17.657439+0000","last_became_active":"2026-02-25T11:47:12.611862+0000","last_became_peered":"2026-02-25T11:47:12.611862+0000","last_unstale":"2026-02-25T11:47:17.657439+0000","last_undegraded":"2026-02-25T11:47:17.657439+0000","last_fullsized":"2026-02-25T11:47:17.657439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:08:17.490884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635279+0000","last_change":"2026-02-25T11:47:13.627129+0000","last_active":"2026-02-25T11:47:14.635279+0000","last_peered":"2026-02-25T11:47:14.635279+0000","last_clean":"2026-02-25T11:47:14.635279+0000","last_became_active":"2026-02-25T11:47:13.626987+0000","last_became_peered":"2026-02-25T11:47:13.626987+0000","last_unstale":"2026-02-25T11:47:14.635279+0000","last_undegraded":"2026-02-25T11:47:14.635279+0000","last_fullsized":"2026-02-25T11:47:14.635279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:53:45.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749940+0000","last_change":"2026-02-25T11:47:12.611992+0000","last_active":"2026-02-25T11:47:14.749940+0000","last_peered":"2026-02-25T11:47:14.749940+0000","last_clean":"2026-02-25T11:47:14.749940+0000","last_became_active":"2026-02-25T11:47:12.611502+0000","last_became_peered":"2026-02-25T11:47:12.611502+0000","last_unstale":"2026-02-25T11:47:14.749940+0000","last_undegraded":"2026-02-25T11:47:14.749940+0000","last_fullsized":"2026-02-25T11:47:14.749940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:15:45.718628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633835+0000","last_change":"2026-02-25T11:47:13.635344+0000","last_active":"2026-02-25T11:47:14.633835+0000","last_peered":"2026-02-25T11:47:14.633835+0000","last_clean":"2026-02-25T11:47:14.633835+0000","last_became_active":"2026-02-25T11:47:13.634327+0000","last_became_peered":"2026-02-25T11:47:13.634327+0000","last_unstale":"2026-02-25T11:47:14.633835+0000","last_undegraded":"2026-02-25T11:47:14.633835+0000","last_fullsized":"2026-02-25T11:47:14.633835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:30:06.778398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:45.596468+0000","last_change":"2026-02-25T11:47:12.611130+0000","last_active":"2026-02-25T11:47:45.596468+0000","last_peered":"2026-02-25T11:47:45.596468+0000","last_clean":"2026-02-25T11:47:45.596468+0000","last_became_active":"2026-02-25T11:47:12.610838+0000","last_became_peered":"2026-02-25T11:47:12.610838+0000","last_unstale":"2026-02-25T11:47:45.596468+0000","last_undegraded":"2026-02-25T11:47:45.596468+0000","last_fullsized":"2026-02-25T11:47:45.596468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:34:19.366424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751244+0000","last_change":"2026-02-25T11:47:13.629263+0000","last_active":"2026-02-25T11:47:14.751244+0000","last_peered":"2026-02-25T11:47:14.751244+0000","last_clean":"2026-02-25T11:47:14.751244+0000","last_became_active":"2026-02-25T11:47:13.628765+0000","last_became_peered":"2026-02-25T11:47:13.628765+0000","last_unstale":"2026-02-25T11:47:14.751244+0000","last_undegraded":"2026-02-25T11:47:14.751244+0000","last_fullsized":"2026-02-25T11:47:14.751244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:49:04.773052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.649872+0000","last_change":"2026-02-25T11:47:12.613455+0000","last_active":"2026-02-25T11:47:16.649872+0000","last_peered":"2026-02-25T11:47:16.649872+0000","last_clean":"2026-02-25T11:47:16.649872+0000","last_became_active":"2026-02-25T11:47:12.612904+0000","last_became_peered":"2026-02-25T11:47:12.612904+0000","last_unstale":"2026-02-25T11:47:16.649872+0000","last_undegraded":"2026-02-25T11:47:16.649872+0000","last_fullsized":"2026-02-25T11:47:16.649872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:16:13.149517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627673+0000","last_change":"2026-02-25T11:47:13.635178+0000","last_active":"2026-02-25T11:47:14.627673+0000","last_peered":"2026-02-25T11:47:14.627673+0000","last_clean":"2026-02-25T11:47:14.627673+0000","last_became_active":"2026-02-25T11:47:13.634985+0000","last_became_peered":"2026-02-25T11:47:13.634985+0000","last_unstale":"2026-02-25T11:47:14.627673+0000","last_undegraded":"2026-02-25T11:47:14.627673+0000","last_fullsized":"2026-02-25T11:47:14.627673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:08.068457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653282+0000","last_change":"2026-02-25T11:47:12.610853+0000","last_active":"2026-02-25T11:47:16.653282+0000","last_peered":"2026-02-25T11:47:16.653282+0000","last_clean":"2026-02-25T11:47:16.653282+0000","last_became_active":"2026-02-25T11:47:12.610639+0000","last_became_peered":"2026-02-25T11:47:12.610639+0000","last_unstale":"2026-02-25T11:47:16.653282+0000","last_undegraded":"2026-02-25T11:47:16.653282+0000","last_fullsized":"2026-02-25T11:47:16.653282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:00:06.042790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":694,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750953+0000","last_change":"2026-02-25T11:47:13.630063+0000","last_active":"2026-02-25T11:47:14.750953+0000","last_peered":"2026-02-25T11:47:14.750953+0000","last_clean":"2026-02-25T11:47:14.750953+0000","last_became_active":"2026-02-25T11:47:13.629906+0000","last_became_peered":"2026-02-25T11:47:13.629906+0000","last_unstale":"2026-02-25T11:47:14.750953+0000","last_undegraded":"2026-02-25T11:47:14.750953+0000","last_fullsized":"2026-02-25T11:47:14.750953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:09:13.614948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":113,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:46.600201+0000","last_change":"2026-02-25T11:47:12.615044+0000","last_active":"2026-02-25T11:47:46.600201+0000","last_peered":"2026-02-25T11:47:46.600201+0000","last_clean":"2026-02-25T11:47:46.600201+0000","last_became_active":"2026-02-25T11:47:12.614698+0000","last_became_peered":"2026-02-25T11:47:12.614698+0000","last_unstale":"2026-02-25T11:47:46.600201+0000","last_undegraded":"2026-02-25T11:47:46.600201+0000","last_fullsized":"2026-02-25T11:47:46.600201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:52:32.148801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69040,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":67,"num_read_kb":100,"num_write":31,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628510+0000","last_change":"2026-02-25T11:47:13.630945+0000","last_active":"2026-02-25T11:47:14.628510+0000","last_peered":"2026-02-25T11:47:14.628510+0000","last_clean":"2026-02-25T11:47:14.628510+0000","last_became_active":"2026-02-25T11:47:13.630693+0000","last_became_peered":"2026-02-25T11:47:13.630693+0000","last_unstale":"2026-02-25T11:47:14.628510+0000","last_undegraded":"2026-02-25T11:47:14.628510+0000","last_fullsized":"2026-02-25T11:47:14.628510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:39:52.744366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656758+0000","last_change":"2026-02-25T11:47:12.614950+0000","last_active":"2026-02-25T11:47:17.656758+0000","last_peered":"2026-02-25T11:47:17.656758+0000","last_clean":"2026-02-25T11:47:17.656758+0000","last_became_active":"2026-02-25T11:47:12.614272+0000","last_became_peered":"2026-02-25T11:47:12.614272+0000","last_unstale":"2026-02-25T11:47:17.656758+0000","last_undegraded":"2026-02-25T11:47:17.656758+0000","last_fullsized":"2026-02-25T11:47:17.656758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:45:50.928166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749596+0000","last_change":"2026-02-25T11:47:13.633128+0000","last_active":"2026-02-25T11:47:14.749596+0000","last_peered":"2026-02-25T11:47:14.749596+0000","last_clean":"2026-02-25T11:47:14.749596+0000","last_became_active":"2026-02-25T11:47:13.633047+0000","last_became_peered":"2026-02-25T11:47:13.633047+0000","last_unstale":"2026-02-25T11:47:14.749596+0000","last_undegraded":"2026-02-25T11:47:14.749596+0000","last_fullsized":"2026-02-25T11:47:14.749596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:56:33.191193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653783+0000","last_change":"2026-02-25T11:47:12.616898+0000","last_active":"2026-02-25T11:47:16.653783+0000","last_peered":"2026-02-25T11:47:16.653783+0000","last_clean":"2026-02-25T11:47:16.653783+0000","last_became_active":"2026-02-25T11:47:12.616774+0000","last_became_peered":"2026-02-25T11:47:12.616774+0000","last_unstale":"2026-02-25T11:47:16.653783+0000","last_undegraded":"2026-02-25T11:47:16.653783+0000","last_fullsized":"2026-02-25T11:47:16.653783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:59:33.930544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628464+0000","last_change":"2026-02-25T11:47:13.630377+0000","last_active":"2026-02-25T11:47:14.628464+0000","last_peered":"2026-02-25T11:47:14.628464+0000","last_clean":"2026-02-25T11:47:14.628464+0000","last_became_active":"2026-02-25T11:47:13.630124+0000","last_became_peered":"2026-02-25T11:47:13.630124+0000","last_unstale":"2026-02-25T11:47:14.628464+0000","last_undegraded":"2026-02-25T11:47:14.628464+0000","last_fullsized":"2026-02-25T11:47:14.628464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:42:12.028258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:43.604899+0000","last_change":"2026-02-25T11:47:12.613208+0000","last_active":"2026-02-25T11:47:43.604899+0000","last_peered":"2026-02-25T11:47:43.604899+0000","last_clean":"2026-02-25T11:47:43.604899+0000","last_became_active":"2026-02-25T11:47:12.612460+0000","last_became_peered":"2026-02-25T11:47:12.612460+0000","last_unstale":"2026-02-25T11:47:43.604899+0000","last_undegraded":"2026-02-25T11:47:43.604899+0000","last_fullsized":"2026-02-25T11:47:43.604899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:15:59.810632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":998,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628121+0000","last_change":"2026-02-25T11:47:13.631123+0000","last_active":"2026-02-25T11:47:14.628121+0000","last_peered":"2026-02-25T11:47:14.628121+0000","last_clean":"2026-02-25T11:47:14.628121+0000","last_became_active":"2026-02-25T11:47:13.631031+0000","last_became_peered":"2026-02-25T11:47:13.631031+0000","last_unstale":"2026-02-25T11:47:14.628121+0000","last_undegraded":"2026-02-25T11:47:14.628121+0000","last_fullsized":"2026-02-25T11:47:14.628121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:47:54.073226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657937+0000","last_change":"2026-02-25T11:47:12.612153+0000","last_active":"2026-02-25T11:47:17.657937+0000","last_peered":"2026-02-25T11:47:17.657937+0000","last_clean":"2026-02-25T11:47:17.657937+0000","last_became_active":"2026-02-25T11:47:12.611107+0000","last_became_peered":"2026-02-25T11:47:12.611107+0000","last_unstale":"2026-02-25T11:47:17.657937+0000","last_undegraded":"2026-02-25T11:47:17.657937+0000","last_fullsized":"2026-02-25T11:47:17.657937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:37:21.791728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749858+0000","last_change":"2026-02-25T11:47:13.635800+0000","last_active":"2026-02-25T11:47:14.749858+0000","last_peered":"2026-02-25T11:47:14.749858+0000","last_clean":"2026-02-25T11:47:14.749858+0000","last_became_active":"2026-02-25T11:47:13.635590+0000","last_became_peered":"2026-02-25T11:47:13.635590+0000","last_unstale":"2026-02-25T11:47:14.749858+0000","last_undegraded":"2026-02-25T11:47:14.749858+0000","last_fullsized":"2026-02-25T11:47:14.749858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:46:36.324518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575342+0000","last_change":"2026-02-25T11:47:12.616473+0000","last_active":"2026-02-25T11:47:18.575342+0000","last_peered":"2026-02-25T11:47:18.575342+0000","last_clean":"2026-02-25T11:47:18.575342+0000","last_became_active":"2026-02-25T11:47:12.616039+0000","last_became_peered":"2026-02-25T11:47:12.616039+0000","last_unstale":"2026-02-25T11:47:18.575342+0000","last_undegraded":"2026-02-25T11:47:18.575342+0000","last_fullsized":"2026-02-25T11:47:18.575342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:28.324889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628413+0000","last_change":"2026-02-25T11:47:13.634771+0000","last_active":"2026-02-25T11:47:14.628413+0000","last_peered":"2026-02-25T11:47:14.628413+0000","last_clean":"2026-02-25T11:47:14.628413+0000","last_became_active":"2026-02-25T11:47:13.634672+0000","last_became_peered":"2026-02-25T11:47:13.634672+0000","last_unstale":"2026-02-25T11:47:14.628413+0000","last_undegraded":"2026-02-25T11:47:14.628413+0000","last_fullsized":"2026-02-25T11:47:14.628413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:19:03.692423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749863+0000","last_change":"2026-02-25T11:47:12.616685+0000","last_active":"2026-02-25T11:47:14.749863+0000","last_peered":"2026-02-25T11:47:14.749863+0000","last_clean":"2026-02-25T11:47:14.749863+0000","last_became_active":"2026-02-25T11:47:12.616518+0000","last_became_peered":"2026-02-25T11:47:12.616518+0000","last_unstale":"2026-02-25T11:47:14.749863+0000","last_undegraded":"2026-02-25T11:47:14.749863+0000","last_fullsized":"2026-02-25T11:47:14.749863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:28:50.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628462+0000","last_change":"2026-02-25T11:47:13.627734+0000","last_active":"2026-02-25T11:47:14.628462+0000","last_peered":"2026-02-25T11:47:14.628462+0000","last_clean":"2026-02-25T11:47:14.628462+0000","last_became_active":"2026-02-25T11:47:13.627632+0000","last_became_peered":"2026-02-25T11:47:13.627632+0000","last_unstale":"2026-02-25T11:47:14.628462+0000","last_undegraded":"2026-02-25T11:47:14.628462+0000","last_fullsized":"2026-02-25T11:47:14.628462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:01:20.907699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750104+0000","last_change":"2026-02-25T11:47:12.612830+0000","last_active":"2026-02-25T11:47:14.750104+0000","last_peered":"2026-02-25T11:47:14.750104+0000","last_clean":"2026-02-25T11:47:14.750104+0000","last_became_active":"2026-02-25T11:47:12.612661+0000","last_became_peered":"2026-02-25T11:47:12.612661+0000","last_unstale":"2026-02-25T11:47:14.750104+0000","last_undegraded":"2026-02-25T11:47:14.750104+0000","last_fullsized":"2026-02-25T11:47:14.750104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:05:15.367058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:15.914932+0000","last_change":"2026-02-25T11:47:13.633998+0000","last_active":"2026-02-25T11:47:15.914932+0000","last_peered":"2026-02-25T11:47:15.914932+0000","last_clean":"2026-02-25T11:47:15.914932+0000","last_became_active":"2026-02-25T11:47:13.633820+0000","last_became_peered":"2026-02-25T11:47:13.633820+0000","last_unstale":"2026-02-25T11:47:15.914932+0000","last_undegraded":"2026-02-25T11:47:15.914932+0000","last_fullsized":"2026-02-25T11:47:15.914932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:02:38.693106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.658417+0000","last_change":"2026-02-25T11:47:12.614091+0000","last_active":"2026-02-25T11:47:17.658417+0000","last_peered":"2026-02-25T11:47:17.658417+0000","last_clean":"2026-02-25T11:47:17.658417+0000","last_became_active":"2026-02-25T11:47:12.612962+0000","last_became_peered":"2026-02-25T11:47:12.612962+0000","last_unstale":"2026-02-25T11:47:17.658417+0000","last_undegraded":"2026-02-25T11:47:17.658417+0000","last_fullsized":"2026-02-25T11:47:17.658417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:11:40.214146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628504+0000","last_change":"2026-02-25T11:47:13.627288+0000","last_active":"2026-02-25T11:47:14.628504+0000","last_peered":"2026-02-25T11:47:14.628504+0000","last_clean":"2026-02-25T11:47:14.628504+0000","last_became_active":"2026-02-25T11:47:13.626902+0000","last_became_peered":"2026-02-25T11:47:13.626902+0000","last_unstale":"2026-02-25T11:47:14.628504+0000","last_undegraded":"2026-02-25T11:47:14.628504+0000","last_fullsized":"2026-02-25T11:47:14.628504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:38:19.137242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'152","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628095+0000","last_change":"2026-02-25T11:46:35.865282+0000","last_active":"2026-02-25T11:47:14.628095+0000","last_peered":"2026-02-25T11:47:14.628095+0000","last_clean":"2026-02-25T11:47:14.628095+0000","last_became_active":"2026-02-25T11:46:35.865139+0000","last_became_peered":"2026-02-25T11:46:35.865139+0000","last_unstale":"2026-02-25T11:47:14.628095+0000","last_undegraded":"2026-02-25T11:47:14.628095+0000","last_fullsized":"2026-02-25T11:47:14.628095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_clean_scrub_stamp":"2026-02-25T11:44:43.739673+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:29:57.573039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575170+0000","last_change":"2026-02-25T11:47:12.612834+0000","last_active":"2026-02-25T11:47:18.575170+0000","last_peered":"2026-02-25T11:47:18.575170+0000","last_clean":"2026-02-25T11:47:18.575170+0000","last_became_active":"2026-02-25T11:47:12.611827+0000","last_became_peered":"2026-02-25T11:47:12.611827+0000","last_unstale":"2026-02-25T11:47:18.575170+0000","last_undegraded":"2026-02-25T11:47:18.575170+0000","last_fullsized":"2026-02-25T11:47:18.575170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:05:32.112908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635504+0000","last_change":"2026-02-25T11:47:13.634534+0000","last_active":"2026-02-25T11:47:14.635504+0000","last_peered":"2026-02-25T11:47:14.635504+0000","last_clean":"2026-02-25T11:47:14.635504+0000","last_became_active":"2026-02-25T11:47:13.633769+0000","last_became_peered":"2026-02-25T11:47:13.633769+0000","last_unstale":"2026-02-25T11:47:14.635504+0000","last_undegraded":"2026-02-25T11:47:14.635504+0000","last_fullsized":"2026-02-25T11:47:14.635504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:16:09.255508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656541+0000","last_change":"2026-02-25T11:47:12.612621+0000","last_active":"2026-02-25T11:47:17.656541+0000","last_peered":"2026-02-25T11:47:17.656541+0000","last_clean":"2026-02-25T11:47:17.656541+0000","last_became_active":"2026-02-25T11:47:12.612284+0000","last_became_peered":"2026-02-25T11:47:12.612284+0000","last_unstale":"2026-02-25T11:47:17.656541+0000","last_undegraded":"2026-02-25T11:47:17.656541+0000","last_fullsized":"2026-02-25T11:47:17.656541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:49:09.299914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749514+0000","last_change":"2026-02-25T11:47:13.629107+0000","last_active":"2026-02-25T11:47:14.749514+0000","last_peered":"2026-02-25T11:47:14.749514+0000","last_clean":"2026-02-25T11:47:14.749514+0000","last_became_active":"2026-02-25T11:47:13.629031+0000","last_became_peered":"2026-02-25T11:47:13.629031+0000","last_unstale":"2026-02-25T11:47:14.749514+0000","last_undegraded":"2026-02-25T11:47:14.749514+0000","last_fullsized":"2026-02-25T11:47:14.749514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:07:58.915416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:46.606658+0000","last_change":"2026-02-25T11:47:12.616543+0000","last_active":"2026-02-25T11:47:46.606658+0000","last_peered":"2026-02-25T11:47:46.606658+0000","last_clean":"2026-02-25T11:47:46.606658+0000","last_became_active":"2026-02-25T11:47:12.616345+0000","last_became_peered":"2026-02-25T11:47:12.616345+0000","last_unstale":"2026-02-25T11:47:46.606658+0000","last_undegraded":"2026-02-25T11:47:46.606658+0000","last_fullsized":"2026-02-25T11:47:46.606658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:27:30.862168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750014+0000","last_change":"2026-02-25T11:47:13.637385+0000","last_active":"2026-02-25T11:47:14.750014+0000","last_peered":"2026-02-25T11:47:14.750014+0000","last_clean":"2026-02-25T11:47:14.750014+0000","last_became_active":"2026-02-25T11:47:13.634653+0000","last_became_peered":"2026-02-25T11:47:13.634653+0000","last_unstale":"2026-02-25T11:47:14.750014+0000","last_undegraded":"2026-02-25T11:47:14.750014+0000","last_fullsized":"2026-02-25T11:47:14.750014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:05:08.662208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:15.914921+0000","last_change":"2026-02-25T11:47:12.614640+0000","last_active":"2026-02-25T11:47:15.914921+0000","last_peered":"2026-02-25T11:47:15.914921+0000","last_clean":"2026-02-25T11:47:15.914921+0000","last_became_active":"2026-02-25T11:47:12.614486+0000","last_became_peered":"2026-02-25T11:47:12.614486+0000","last_unstale":"2026-02-25T11:47:15.914921+0000","last_undegraded":"2026-02-25T11:47:15.914921+0000","last_fullsized":"2026-02-25T11:47:15.914921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:16:06.314709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750327+0000","last_change":"2026-02-25T11:47:13.631828+0000","last_active":"2026-02-25T11:47:14.750327+0000","last_peered":"2026-02-25T11:47:14.750327+0000","last_clean":"2026-02-25T11:47:14.750327+0000","last_became_active":"2026-02-25T11:47:13.631590+0000","last_became_peered":"2026-02-25T11:47:13.631590+0000","last_unstale":"2026-02-25T11:47:14.750327+0000","last_undegraded":"2026-02-25T11:47:14.750327+0000","last_fullsized":"2026-02-25T11:47:14.750327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:25.518794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633984+0000","last_change":"2026-02-25T11:47:12.614032+0000","last_active":"2026-02-25T11:47:14.633984+0000","last_peered":"2026-02-25T11:47:14.633984+0000","last_clean":"2026-02-25T11:47:14.633984+0000","last_became_active":"2026-02-25T11:47:12.613936+0000","last_became_peered":"2026-02-25T11:47:12.613936+0000","last_unstale":"2026-02-25T11:47:14.633984+0000","last_undegraded":"2026-02-25T11:47:14.633984+0000","last_fullsized":"2026-02-25T11:47:14.633984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:16:46.397485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.859010+0000","last_change":"2026-02-25T11:47:13.636173+0000","last_active":"2026-02-25T11:47:16.859010+0000","last_peered":"2026-02-25T11:47:16.859010+0000","last_clean":"2026-02-25T11:47:16.859010+0000","last_became_active":"2026-02-25T11:47:13.635745+0000","last_became_peered":"2026-02-25T11:47:13.635745+0000","last_unstale":"2026-02-25T11:47:16.859010+0000","last_undegraded":"2026-02-25T11:47:16.859010+0000","last_fullsized":"2026-02-25T11:47:16.859010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:03:33.671318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651871+0000","last_change":"2026-02-25T11:47:12.614003+0000","last_active":"2026-02-25T11:47:16.651871+0000","last_peered":"2026-02-25T11:47:16.651871+0000","last_clean":"2026-02-25T11:47:16.651871+0000","last_became_active":"2026-02-25T11:47:12.613893+0000","last_became_peered":"2026-02-25T11:47:12.613893+0000","last_unstale":"2026-02-25T11:47:16.651871+0000","last_undegraded":"2026-02-25T11:47:16.651871+0000","last_fullsized":"2026-02-25T11:47:16.651871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:33:21.133364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749746+0000","last_change":"2026-02-25T11:47:13.632632+0000","last_active":"2026-02-25T11:47:14.749746+0000","last_peered":"2026-02-25T11:47:14.749746+0000","last_clean":"2026-02-25T11:47:14.749746+0000","last_became_active":"2026-02-25T11:47:13.632408+0000","last_became_peered":"2026-02-25T11:47:13.632408+0000","last_unstale":"2026-02-25T11:47:14.749746+0000","last_undegraded":"2026-02-25T11:47:14.749746+0000","last_fullsized":"2026-02-25T11:47:14.749746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:38:45.448637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:27.638764+0000","last_change":"2026-02-25T11:47:12.613816+0000","last_active":"2026-02-25T11:47:27.638764+0000","last_peered":"2026-02-25T11:47:27.638764+0000","last_clean":"2026-02-25T11:47:27.638764+0000","last_became_active":"2026-02-25T11:47:12.613196+0000","last_became_peered":"2026-02-25T11:47:12.613196+0000","last_unstale":"2026-02-25T11:47:27.638764+0000","last_undegraded":"2026-02-25T11:47:27.638764+0000","last_fullsized":"2026-02-25T11:47:27.638764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:23:52.372808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635752+0000","last_change":"2026-02-25T11:47:13.632431+0000","last_active":"2026-02-25T11:47:14.635752+0000","last_peered":"2026-02-25T11:47:14.635752+0000","last_clean":"2026-02-25T11:47:14.635752+0000","last_became_active":"2026-02-25T11:47:13.631972+0000","last_became_peered":"2026-02-25T11:47:13.631972+0000","last_unstale":"2026-02-25T11:47:14.635752+0000","last_undegraded":"2026-02-25T11:47:14.635752+0000","last_fullsized":"2026-02-25T11:47:14.635752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:59:34.655028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651131+0000","last_change":"2026-02-25T11:47:12.612805+0000","last_active":"2026-02-25T11:47:16.651131+0000","last_peered":"2026-02-25T11:47:16.651131+0000","last_clean":"2026-02-25T11:47:16.651131+0000","last_became_active":"2026-02-25T11:47:12.612155+0000","last_became_peered":"2026-02-25T11:47:12.612155+0000","last_unstale":"2026-02-25T11:47:16.651131+0000","last_undegraded":"2026-02-25T11:47:16.651131+0000","last_fullsized":"2026-02-25T11:47:16.651131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:58:00.429639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627853+0000","last_change":"2026-02-25T11:47:13.636089+0000","last_active":"2026-02-25T11:47:14.627853+0000","last_peered":"2026-02-25T11:47:14.627853+0000","last_clean":"2026-02-25T11:47:14.627853+0000","last_became_active":"2026-02-25T11:47:13.635963+0000","last_became_peered":"2026-02-25T11:47:13.635963+0000","last_unstale":"2026-02-25T11:47:14.627853+0000","last_undegraded":"2026-02-25T11:47:14.627853+0000","last_fullsized":"2026-02-25T11:47:14.627853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:50:28.012023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650081+0000","last_change":"2026-02-25T11:47:12.609749+0000","last_active":"2026-02-25T11:47:16.650081+0000","last_peered":"2026-02-25T11:47:16.650081+0000","last_clean":"2026-02-25T11:47:16.650081+0000","last_became_active":"2026-02-25T11:47:12.609190+0000","last_became_peered":"2026-02-25T11:47:12.609190+0000","last_unstale":"2026-02-25T11:47:16.650081+0000","last_undegraded":"2026-02-25T11:47:16.650081+0000","last_fullsized":"2026-02-25T11:47:16.650081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:12:42.204122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628201+0000","last_change":"2026-02-25T11:47:13.629339+0000","last_active":"2026-02-25T11:47:14.628201+0000","last_peered":"2026-02-25T11:47:14.628201+0000","last_clean":"2026-02-25T11:47:14.628201+0000","last_became_active":"2026-02-25T11:47:13.629107+0000","last_became_peered":"2026-02-25T11:47:13.629107+0000","last_unstale":"2026-02-25T11:47:14.628201+0000","last_undegraded":"2026-02-25T11:47:14.628201+0000","last_fullsized":"2026-02-25T11:47:14.628201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:19:50.087771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.652673+0000","last_change":"2026-02-25T11:47:12.615249+0000","last_active":"2026-02-25T11:47:16.652673+0000","last_peered":"2026-02-25T11:47:16.652673+0000","last_clean":"2026-02-25T11:47:16.652673+0000","last_became_active":"2026-02-25T11:47:12.615165+0000","last_became_peered":"2026-02-25T11:47:12.615165+0000","last_unstale":"2026-02-25T11:47:16.652673+0000","last_undegraded":"2026-02-25T11:47:16.652673+0000","last_fullsized":"2026-02-25T11:47:16.652673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:40:52.452074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627665+0000","last_change":"2026-02-25T11:47:13.631460+0000","last_active":"2026-02-25T11:47:14.627665+0000","last_peered":"2026-02-25T11:47:14.627665+0000","last_clean":"2026-02-25T11:47:14.627665+0000","last_became_active":"2026-02-25T11:47:13.631298+0000","last_became_peered":"2026-02-25T11:47:13.631298+0000","last_unstale":"2026-02-25T11:47:14.627665+0000","last_undegraded":"2026-02-25T11:47:14.627665+0000","last_fullsized":"2026-02-25T11:47:14.627665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:26:03.362106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.648950+0000","last_change":"2026-02-25T11:47:12.615779+0000","last_active":"2026-02-25T11:47:17.648950+0000","last_peered":"2026-02-25T11:47:17.648950+0000","last_clean":"2026-02-25T11:47:17.648950+0000","last_became_active":"2026-02-25T11:47:12.615073+0000","last_became_peered":"2026-02-25T11:47:12.615073+0000","last_unstale":"2026-02-25T11:47:17.648950+0000","last_undegraded":"2026-02-25T11:47:17.648950+0000","last_fullsized":"2026-02-25T11:47:17.648950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:28:51.468199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635380+0000","last_change":"2026-02-25T11:47:13.632351+0000","last_active":"2026-02-25T11:47:14.635380+0000","last_peered":"2026-02-25T11:47:14.635380+0000","last_clean":"2026-02-25T11:47:14.635380+0000","last_became_active":"2026-02-25T11:47:13.632081+0000","last_became_peered":"2026-02-25T11:47:13.632081+0000","last_unstale":"2026-02-25T11:47:14.635380+0000","last_undegraded":"2026-02-25T11:47:14.635380+0000","last_fullsized":"2026-02-25T11:47:14.635380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:38:59.143253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650353+0000","last_change":"2026-02-25T11:47:12.616555+0000","last_active":"2026-02-25T11:47:16.650353+0000","last_peered":"2026-02-25T11:47:16.650353+0000","last_clean":"2026-02-25T11:47:16.650353+0000","last_became_active":"2026-02-25T11:47:12.616174+0000","last_became_peered":"2026-02-25T11:47:12.616174+0000","last_unstale":"2026-02-25T11:47:16.650353+0000","last_undegraded":"2026-02-25T11:47:16.650353+0000","last_fullsized":"2026-02-25T11:47:16.650353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:01:17.494129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628458+0000","last_change":"2026-02-25T11:47:13.627400+0000","last_active":"2026-02-25T11:47:14.628458+0000","last_peered":"2026-02-25T11:47:14.628458+0000","last_clean":"2026-02-25T11:47:14.628458+0000","last_became_active":"2026-02-25T11:47:13.626782+0000","last_became_peered":"2026-02-25T11:47:13.626782+0000","last_unstale":"2026-02-25T11:47:14.628458+0000","last_undegraded":"2026-02-25T11:47:14.628458+0000","last_fullsized":"2026-02-25T11:47:14.628458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:42:39.996935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750027+0000","last_change":"2026-02-25T11:47:13.634030+0000","last_active":"2026-02-25T11:47:14.750027+0000","last_peered":"2026-02-25T11:47:14.750027+0000","last_clean":"2026-02-25T11:47:14.750027+0000","last_became_active":"2026-02-25T11:47:13.632844+0000","last_became_peered":"2026-02-25T11:47:13.632844+0000","last_unstale":"2026-02-25T11:47:14.750027+0000","last_undegraded":"2026-02-25T11:47:14.750027+0000","last_fullsized":"2026-02-25T11:47:14.750027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:52:16.013596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628805+0000","last_change":"2026-02-25T11:47:12.613666+0000","last_active":"2026-02-25T11:47:14.628805+0000","last_peered":"2026-02-25T11:47:14.628805+0000","last_clean":"2026-02-25T11:47:14.628805+0000","last_became_active":"2026-02-25T11:47:12.613570+0000","last_became_peered":"2026-02-25T11:47:12.613570+0000","last_unstale":"2026-02-25T11:47:14.628805+0000","last_undegraded":"2026-02-25T11:47:14.628805+0000","last_fullsized":"2026-02-25T11:47:14.628805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:28:39.750737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750924+0000","last_change":"2026-02-25T11:47:13.631515+0000","last_active":"2026-02-25T11:47:14.750924+0000","last_peered":"2026-02-25T11:47:14.750924+0000","last_clean":"2026-02-25T11:47:14.750924+0000","last_became_active":"2026-02-25T11:47:13.631161+0000","last_became_peered":"2026-02-25T11:47:13.631161+0000","last_unstale":"2026-02-25T11:47:14.750924+0000","last_undegraded":"2026-02-25T11:47:14.750924+0000","last_fullsized":"2026-02-25T11:47:14.750924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:13:30.496521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656891+0000","last_change":"2026-02-25T11:47:12.615904+0000","last_active":"2026-02-25T11:47:17.656891+0000","last_peered":"2026-02-25T11:47:17.656891+0000","last_clean":"2026-02-25T11:47:17.656891+0000","last_became_active":"2026-02-25T11:47:12.615784+0000","last_became_peered":"2026-02-25T11:47:12.615784+0000","last_unstale":"2026-02-25T11:47:17.656891+0000","last_undegraded":"2026-02-25T11:47:17.656891+0000","last_fullsized":"2026-02-25T11:47:17.656891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:32:26.802870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627754+0000","last_change":"2026-02-25T11:47:13.630065+0000","last_active":"2026-02-25T11:47:14.627754+0000","last_peered":"2026-02-25T11:47:14.627754+0000","last_clean":"2026-02-25T11:47:14.627754+0000","last_became_active":"2026-02-25T11:47:13.629214+0000","last_became_peered":"2026-02-25T11:47:13.629214+0000","last_unstale":"2026-02-25T11:47:14.627754+0000","last_undegraded":"2026-02-25T11:47:14.627754+0000","last_fullsized":"2026-02-25T11:47:14.627754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:53:45.639526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.310632+0000","last_change":"2026-02-25T11:47:12.612956+0000","last_active":"2026-02-25T11:47:16.310632+0000","last_peered":"2026-02-25T11:47:16.310632+0000","last_clean":"2026-02-25T11:47:16.310632+0000","last_became_active":"2026-02-25T11:47:12.612843+0000","last_became_peered":"2026-02-25T11:47:12.612843+0000","last_unstale":"2026-02-25T11:47:16.310632+0000","last_undegraded":"2026-02-25T11:47:16.310632+0000","last_fullsized":"2026-02-25T11:47:16.310632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:41:30.867465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635780+0000","last_change":"2026-02-25T11:47:13.637712+0000","last_active":"2026-02-25T11:47:14.635780+0000","last_peered":"2026-02-25T11:47:14.635780+0000","last_clean":"2026-02-25T11:47:14.635780+0000","last_became_active":"2026-02-25T11:47:13.635030+0000","last_became_peered":"2026-02-25T11:47:13.635030+0000","last_unstale":"2026-02-25T11:47:14.635780+0000","last_undegraded":"2026-02-25T11:47:14.635780+0000","last_fullsized":"2026-02-25T11:47:14.635780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:01:22.896129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656736+0000","last_change":"2026-02-25T11:47:12.614149+0000","last_active":"2026-02-25T11:47:17.656736+0000","last_peered":"2026-02-25T11:47:17.656736+0000","last_clean":"2026-02-25T11:47:17.656736+0000","last_became_active":"2026-02-25T11:47:12.613137+0000","last_became_peered":"2026-02-25T11:47:12.613137+0000","last_unstale":"2026-02-25T11:47:17.656736+0000","last_undegraded":"2026-02-25T11:47:17.656736+0000","last_fullsized":"2026-02-25T11:47:17.656736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:49:49.451435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73639,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":203,"num_read_kb":178,"num_write":168,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":450560,"data_stored":228821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":227295,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":262884,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28132,"kb_used_data":912,"kb_used_omap":9,"kb_used_meta":27190,"kb_avail":733975068,"statfs":{"total":751619276800,"available":751590469632,"internally_reserved":0,"allocated":933888,"data_stored":455512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10109,"internal_metadata":27842691},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612625,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28288,"kb_used_data":1068,"kb_used_omap":28,"kb_used_meta":27171,"kb_avail":733974912,"statfs":{"total":751619276800,"available":751590309888,"internally_reserved":0,"allocated":1093632,"data_stored":1116443,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":29123,"internal_metadata":27823677},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776148,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28148,"kb_used_data":928,"kb_used_omap":27,"kb_used_meta":27172,"kb_avail":733975052,"statfs":{"total":751619276800,"available":751590453248,"internally_reserved":0,"allocated":950272,"data_stored":455748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28131,"internal_metadata":27824669},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037782,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28148,"kb_used_data":928,"kb_used_omap":35,"kb_used_meta":27164,"kb_avail":733975052,"statfs":{"total":751619276800,"available":751590453248,"internally_reserved":0,"allocated":950272,"data_stored":455668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36257,"internal_metadata":27816543},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299417,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28196,"kb_used_data":976,"kb_used_omap":35,"kb_used_meta":27164,"kb_avail":733975004,"statfs":{"total":751619276800,"available":751590404096,"internally_reserved":0,"allocated":999424,"data_stored":1047617,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36168,"internal_metadata":27816632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28136,"kb_used_data":916,"kb_used_omap":31,"kb_used_meta":27168,"kb_avail":733975064,"statfs":{"total":751619276800,"available":751590465536,"internally_reserved":0,"allocated":937984,"data_stored":455115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32619,"internal_metadata":27820181},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724575,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28208,"kb_used_data":988,"kb_used_omap":36,"kb_used_meta":27163,"kb_avail":733974992,"statfs":{"total":751619276800,"available":751590391808,"internally_reserved":0,"allocated":1011712,"data_stored":1047404,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36952,"internal_metadata":27815848},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888097,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28140,"kb_used_data":920,"kb_used_omap":45,"kb_used_meta":27154,"kb_avail":733975060,"statfs":{"total":751619276800,"available":751590461440,"internally_reserved":0,"allocated":942080,"data_stored":455962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":46332,"internal_metadata":27806468},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149732,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28152,"kb_used_data":932,"kb_used_omap":51,"kb_used_meta":27148,"kb_avail":733975048,"statfs":{"total":751619276800,"available":751590449152,"internally_reserved":0,"allocated":954368,"data_stored":455426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52913,"internal_metadata":27799887},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378663,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28236,"kb_used_data":1016,"kb_used_omap":64,"kb_used_meta":27135,"kb_avail":733974964,"statfs":{"total":751619276800,"available":751590363136,"internally_reserved":0,"allocated":1040384,"data_stored":534250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":66234,"internal_metadata":27786566},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28204,"kb_used_data":984,"kb_used_omap":43,"kb_used_meta":27156,"kb_avail":733974996,"statfs":{"total":751619276800,"available":751590395904,"internally_reserved":0,"allocated":1007616,"data_stored":523948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44488,"internal_metadata":27808312},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28156,"kb_used_data":936,"kb_used_omap":54,"kb_used_meta":27145,"kb_avail":733975044,"statfs":{"total":751619276800,"available":751590445056,"internally_reserved":0,"allocated":958464,"data_stored":456430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56100,"internal_metadata":27796700},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8853,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1407,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":79342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56705,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49702,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6449,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3478,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14040,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9858,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25902,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22346,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21693,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6862,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T11:47:50.423 INFO:tasks.fwd_scrub.ceph_manager:clean! 2026-02-25T11:47:50.423 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T11:47:50.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:50 trial167 ceph-mon[43392]: pgmap v212: 97 pgs: 97 active+clean; 649 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2026-02-25T11:47:50.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:50 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2353914324' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2026-02-25T11:47:50.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:50 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/782240480' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:50.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:50 trial119 ceph-mon[42710]: pgmap v212: 97 pgs: 97 active+clean; 649 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2026-02-25T11:47:50.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:50 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2353914324' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2026-02-25T11:47:50.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:50 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/782240480' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.571+0000 7fa728d39640 1 -- 10.20.193.28:0/2860763744 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa724062de0 msgr2=0x7fa7240631d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.571+0000 7fa728d39640 1 --2- 10.20.193.28:0/2860763744 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa724062de0 0x7fa7240631d0 secure :-1 s=READY pgs=109 gs=1 cs=0 l=1 c_cookie=11058143909034414317 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa710009950 tx=0x7fa71002f190 comp rx=0 tx=0).stop 2026-02-25T11:47:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.572+0000 7fa728d39640 1 -- 10.20.193.28:0/2860763744 shutdown_connections 2026-02-25T11:47:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.572+0000 7fa728d39640 1 --2- 10.20.193.28:0/2860763744 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa724064920 0x7fa724065280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.572+0000 7fa728d39640 1 --2- 10.20.193.28:0/2860763744 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa724063f70 0x7fa7240643e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.572+0000 7fa728d39640 1 --2- 10.20.193.28:0/2860763744 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa724062de0 0x7fa7240631d0 unknown :-1 s=CLOSED pgs=109 gs=1 cs=0 l=1 c_cookie=11058143909034414317 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.572+0000 7fa728d39640 1 -- 10.20.193.28:0/2860763744 >> 10.20.193.28:0/2860763744 conn(0x7fa7240ec860 msgr2=0x7fa72410cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.572+0000 7fa728d39640 1 -- 10.20.193.28:0/2860763744 shutdown_connections 2026-02-25T11:47:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.572+0000 7fa728d39640 1 -- 10.20.193.28:0/2860763744 wait complete. 2026-02-25T11:47:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.572+0000 7fa728d39640 1 Processor -- start 2026-02-25T11:47:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.572+0000 7fa728d39640 1 -- start start 2026-02-25T11:47:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.572+0000 7fa728d39640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa724062de0 0x7fa7240f8290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.572+0000 7fa728d39640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa724063f70 0x7fa7240f88e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa728d39640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa724064920 0x7fa7240f94e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa728d39640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa7240f9c20 con 0x7fa724062de0 2026-02-25T11:47:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa728d39640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa7240f9d90 con 0x7fa724063f70 2026-02-25T11:47:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa728d39640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa7240f9f50 con 0x7fa724064920 2026-02-25T11:47:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa7237fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa724062de0 0x7fa7240f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa7237fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa724062de0 0x7fa7240f8290 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59506/0 (socket says 10.20.193.28:59506) 2026-02-25T11:47:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa7237fe640 1 -- 10.20.193.28:0/996748042 learned_addr learned my addr 10.20.193.28:0/996748042 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa723fff640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa724064920 0x7fa7240f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa722ffd640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa724063f70 0x7fa7240f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa7237fe640 1 -- 10.20.193.28:0/996748042 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa724064920 msgr2=0x7fa7240f94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa7237fe640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa724064920 0x7fa7240f94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa7237fe640 1 -- 10.20.193.28:0/996748042 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa724063f70 msgr2=0x7fa7240f88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa7237fe640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa724063f70 0x7fa7240f88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa7237fe640 1 -- 10.20.193.28:0/996748042 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa710009570 con 0x7fa724062de0 2026-02-25T11:47:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa722ffd640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa724063f70 0x7fa7240f88e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa7237fe640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa724062de0 0x7fa7240f8290 secure :-1 s=READY pgs=353 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa710005b30 tx=0x7fa7100028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa720ff9640 1 -- 10.20.193.28:0/996748042 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa71003d070 con 0x7fa724062de0 2026-02-25T11:47:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa728d39640 1 -- 10.20.193.28:0/996748042 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7240fa1d0 con 0x7fa724062de0 2026-02-25T11:47:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa720ff9640 1 -- 10.20.193.28:0/996748042 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa710005dd0 con 0x7fa724062de0 2026-02-25T11:47:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa720ff9640 1 -- 10.20.193.28:0/996748042 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa710041960 con 0x7fa724062de0 2026-02-25T11:47:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.573+0000 7fa728d39640 1 -- 10.20.193.28:0/996748042 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7240fa6f0 con 0x7fa724062de0 2026-02-25T11:47:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.574+0000 7fa728d39640 1 -- 10.20.193.28:0/996748042 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa724179a00 con 0x7fa724062de0 2026-02-25T11:47:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.576+0000 7fa720ff9640 1 -- 10.20.193.28:0/996748042 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fa710041af0 con 0x7fa724062de0 2026-02-25T11:47:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.578+0000 7fa720ff9640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa6f80b57f0 0x7fa6f80d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.578+0000 7fa720ff9640 1 -- 10.20.193.28:0/996748042 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fa71010b440 con 0x7fa724062de0 2026-02-25T11:47:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.578+0000 7fa720ff9640 1 -- 10.20.193.28:0/996748042 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fa71010b8c0 con 0x7fa724062de0 2026-02-25T11:47:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.578+0000 7fa722ffd640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa6f80b57f0 0x7fa6f80d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:50.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.581+0000 7fa722ffd640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa6f80b57f0 0x7fa6f80d5ca0 secure :-1 s=READY pgs=250 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa714004740 tx=0x7fa714005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:50.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.691+0000 7fa728d39640 1 -- 10.20.193.28:0/996748042 --> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa7241790c0 con 0x7fa6f80b57f0 2026-02-25T11:47:50.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.694+0000 7fa720ff9640 1 -- 10.20.193.28:0/996748042 <== mgr.14152 v2:10.20.193.28:6800/3140694734 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258968 (secure 0 0 0) 0x7fa7241790c0 con 0x7fa6f80b57f0 2026-02-25T11:47:50.699 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:50.701 INFO:teuthology.orchestra.run.trial028.stderr:dumped all 2026-02-25T11:47:50.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.696+0000 7fa728d39640 1 -- 10.20.193.28:0/996748042 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa6f80b57f0 msgr2=0x7fa6f80d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:50.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.696+0000 7fa728d39640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa6f80b57f0 0x7fa6f80d5ca0 secure :-1 s=READY pgs=250 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa714004740 tx=0x7fa714005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:50.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.696+0000 7fa728d39640 1 -- 10.20.193.28:0/996748042 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa724062de0 msgr2=0x7fa7240f8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:50.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.696+0000 7fa728d39640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa724062de0 0x7fa7240f8290 secure :-1 s=READY pgs=353 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa710005b30 tx=0x7fa7100028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.696+0000 7fa728d39640 1 -- 10.20.193.28:0/996748042 shutdown_connections 2026-02-25T11:47:50.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.696+0000 7fa728d39640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa6f80b57f0 0x7fa6f80d5ca0 unknown :-1 s=CLOSED pgs=250 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.696+0000 7fa728d39640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa724064920 0x7fa7240f94e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.696+0000 7fa728d39640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa724063f70 0x7fa7240f88e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.696+0000 7fa728d39640 1 --2- 10.20.193.28:0/996748042 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa724062de0 0x7fa7240f8290 unknown :-1 s=CLOSED pgs=353 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:50.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.696+0000 7fa728d39640 1 -- 10.20.193.28:0/996748042 >> 10.20.193.28:0/996748042 conn(0x7fa7240ec860 msgr2=0x7fa7240ee510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:50.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.697+0000 7fa728d39640 1 -- 10.20.193.28:0/996748042 shutdown_connections 2026-02-25T11:47:50.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:50.697+0000 7fa728d39640 1 -- 10.20.193.28:0/996748042 wait complete. 2026-02-25T11:47:50.711 INFO:teuthology.orchestra.run.trial028.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2026-02-25T11:47:49.080274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664241,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":309,"num_read_kb":269,"num_write":359,"num_write_kb":3847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":297,"ondisk_log_size":297,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":8808038400,"kb_used":338144,"kb_used_data":11504,"kb_used_omap":464,"kb_used_meta":325935,"kb_avail":8807700256,"statfs":{"total":9019431321600,"available":9019085062144,"internally_reserved":0,"allocated":11780096,"data_stored":7459523,"data_compressed":114024,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":475426,"internal_metadata":333758174},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3098,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":39,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002180"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627807+0000","last_change":"2026-02-25T11:47:13.630492+0000","last_active":"2026-02-25T11:47:14.627807+0000","last_peered":"2026-02-25T11:47:14.627807+0000","last_clean":"2026-02-25T11:47:14.627807+0000","last_became_active":"2026-02-25T11:47:13.630388+0000","last_became_peered":"2026-02-25T11:47:13.630388+0000","last_unstale":"2026-02-25T11:47:14.627807+0000","last_undegraded":"2026-02-25T11:47:14.627807+0000","last_fullsized":"2026-02-25T11:47:14.627807+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:01:17.024058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627657+0000","last_change":"2026-02-25T11:47:13.629507+0000","last_active":"2026-02-25T11:47:14.627657+0000","last_peered":"2026-02-25T11:47:14.627657+0000","last_clean":"2026-02-25T11:47:14.627657+0000","last_became_active":"2026-02-25T11:47:13.628937+0000","last_became_peered":"2026-02-25T11:47:13.628937+0000","last_unstale":"2026-02-25T11:47:14.627657+0000","last_undegraded":"2026-02-25T11:47:14.627657+0000","last_fullsized":"2026-02-25T11:47:14.627657+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:47:20.441522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627713+0000","last_change":"2026-02-25T11:47:13.630415+0000","last_active":"2026-02-25T11:47:14.627713+0000","last_peered":"2026-02-25T11:47:14.627713+0000","last_clean":"2026-02-25T11:47:14.627713+0000","last_became_active":"2026-02-25T11:47:13.630319+0000","last_became_peered":"2026-02-25T11:47:13.630319+0000","last_unstale":"2026-02-25T11:47:14.627713+0000","last_undegraded":"2026-02-25T11:47:14.627713+0000","last_fullsized":"2026-02-25T11:47:14.627713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:18:57.675760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627800+0000","last_change":"2026-02-25T11:47:13.633017+0000","last_active":"2026-02-25T11:47:14.627800+0000","last_peered":"2026-02-25T11:47:14.627800+0000","last_clean":"2026-02-25T11:47:14.627800+0000","last_became_active":"2026-02-25T11:47:13.632934+0000","last_became_peered":"2026-02-25T11:47:13.632934+0000","last_unstale":"2026-02-25T11:47:14.627800+0000","last_undegraded":"2026-02-25T11:47:14.627800+0000","last_fullsized":"2026-02-25T11:47:14.627800+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:08:54.195980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628667+0000","last_change":"2026-02-25T11:47:13.638057+0000","last_active":"2026-02-25T11:47:14.628667+0000","last_peered":"2026-02-25T11:47:14.628667+0000","last_clean":"2026-02-25T11:47:14.628667+0000","last_became_active":"2026-02-25T11:47:13.636387+0000","last_became_peered":"2026-02-25T11:47:13.636387+0000","last_unstale":"2026-02-25T11:47:14.628667+0000","last_undegraded":"2026-02-25T11:47:14.628667+0000","last_fullsized":"2026-02-25T11:47:14.628667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:34:29.570622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750630+0000","last_change":"2026-02-25T11:47:13.631824+0000","last_active":"2026-02-25T11:47:14.750630+0000","last_peered":"2026-02-25T11:47:14.750630+0000","last_clean":"2026-02-25T11:47:14.750630+0000","last_became_active":"2026-02-25T11:47:13.631656+0000","last_became_peered":"2026-02-25T11:47:13.631656+0000","last_unstale":"2026-02-25T11:47:14.750630+0000","last_undegraded":"2026-02-25T11:47:14.750630+0000","last_fullsized":"2026-02-25T11:47:14.750630+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:15:05.237015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628186+0000","last_change":"2026-02-25T11:47:13.626877+0000","last_active":"2026-02-25T11:47:14.628186+0000","last_peered":"2026-02-25T11:47:14.628186+0000","last_clean":"2026-02-25T11:47:14.628186+0000","last_became_active":"2026-02-25T11:47:13.626543+0000","last_became_peered":"2026-02-25T11:47:13.626543+0000","last_unstale":"2026-02-25T11:47:14.628186+0000","last_undegraded":"2026-02-25T11:47:14.628186+0000","last_fullsized":"2026-02-25T11:47:14.628186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:56:58.194911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635224+0000","last_change":"2026-02-25T11:47:13.637661+0000","last_active":"2026-02-25T11:47:14.635224+0000","last_peered":"2026-02-25T11:47:14.635224+0000","last_clean":"2026-02-25T11:47:14.635224+0000","last_became_active":"2026-02-25T11:47:13.637230+0000","last_became_peered":"2026-02-25T11:47:13.637230+0000","last_unstale":"2026-02-25T11:47:14.635224+0000","last_undegraded":"2026-02-25T11:47:14.635224+0000","last_fullsized":"2026-02-25T11:47:14.635224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:44:52.310360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751287+0000","last_change":"2026-02-25T11:47:13.625160+0000","last_active":"2026-02-25T11:47:14.751287+0000","last_peered":"2026-02-25T11:47:14.751287+0000","last_clean":"2026-02-25T11:47:14.751287+0000","last_became_active":"2026-02-25T11:47:13.625020+0000","last_became_peered":"2026-02-25T11:47:13.625020+0000","last_unstale":"2026-02-25T11:47:14.751287+0000","last_undegraded":"2026-02-25T11:47:14.751287+0000","last_fullsized":"2026-02-25T11:47:14.751287+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:35:44.509551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750832+0000","last_change":"2026-02-25T11:47:13.637252+0000","last_active":"2026-02-25T11:47:14.750832+0000","last_peered":"2026-02-25T11:47:14.750832+0000","last_clean":"2026-02-25T11:47:14.750832+0000","last_became_active":"2026-02-25T11:47:13.636670+0000","last_became_peered":"2026-02-25T11:47:13.636670+0000","last_unstale":"2026-02-25T11:47:14.750832+0000","last_undegraded":"2026-02-25T11:47:14.750832+0000","last_fullsized":"2026-02-25T11:47:14.750832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:01:53.119101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628443+0000","last_change":"2026-02-25T11:47:13.628252+0000","last_active":"2026-02-25T11:47:14.628443+0000","last_peered":"2026-02-25T11:47:14.628443+0000","last_clean":"2026-02-25T11:47:14.628443+0000","last_became_active":"2026-02-25T11:47:13.628183+0000","last_became_peered":"2026-02-25T11:47:13.628183+0000","last_unstale":"2026-02-25T11:47:14.628443+0000","last_undegraded":"2026-02-25T11:47:14.628443+0000","last_fullsized":"2026-02-25T11:47:14.628443+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:12:00.576747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.632358+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.631487+0000","last_became_peered":"2026-02-25T11:47:13.631487+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:41:17.150851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749735+0000","last_change":"2026-02-25T11:47:13.629675+0000","last_active":"2026-02-25T11:47:14.749735+0000","last_peered":"2026-02-25T11:47:14.749735+0000","last_clean":"2026-02-25T11:47:14.749735+0000","last_became_active":"2026-02-25T11:47:13.628686+0000","last_became_peered":"2026-02-25T11:47:13.628686+0000","last_unstale":"2026-02-25T11:47:14.749735+0000","last_undegraded":"2026-02-25T11:47:14.749735+0000","last_fullsized":"2026-02-25T11:47:14.749735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:00:54.430287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627615+0000","last_change":"2026-02-25T11:47:13.628842+0000","last_active":"2026-02-25T11:47:14.627615+0000","last_peered":"2026-02-25T11:47:14.627615+0000","last_clean":"2026-02-25T11:47:14.627615+0000","last_became_active":"2026-02-25T11:47:13.628695+0000","last_became_peered":"2026-02-25T11:47:13.628695+0000","last_unstale":"2026-02-25T11:47:14.627615+0000","last_undegraded":"2026-02-25T11:47:14.627615+0000","last_fullsized":"2026-02-25T11:47:14.627615+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:10:05.404740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627836+0000","last_change":"2026-02-25T11:47:13.635652+0000","last_active":"2026-02-25T11:47:14.627836+0000","last_peered":"2026-02-25T11:47:14.627836+0000","last_clean":"2026-02-25T11:47:14.627836+0000","last_became_active":"2026-02-25T11:47:13.635505+0000","last_became_peered":"2026-02-25T11:47:13.635505+0000","last_unstale":"2026-02-25T11:47:14.627836+0000","last_undegraded":"2026-02-25T11:47:14.627836+0000","last_fullsized":"2026-02-25T11:47:14.627836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:46:03.598577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749928+0000","last_change":"2026-02-25T11:47:13.628651+0000","last_active":"2026-02-25T11:47:14.749928+0000","last_peered":"2026-02-25T11:47:14.749928+0000","last_clean":"2026-02-25T11:47:14.749928+0000","last_became_active":"2026-02-25T11:47:13.628280+0000","last_became_peered":"2026-02-25T11:47:13.628280+0000","last_unstale":"2026-02-25T11:47:14.749928+0000","last_undegraded":"2026-02-25T11:47:14.749928+0000","last_fullsized":"2026-02-25T11:47:14.749928+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:18:56.238078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628351+0000","last_change":"2026-02-25T11:47:13.630656+0000","last_active":"2026-02-25T11:47:14.628351+0000","last_peered":"2026-02-25T11:47:14.628351+0000","last_clean":"2026-02-25T11:47:14.628351+0000","last_became_active":"2026-02-25T11:47:13.630513+0000","last_became_peered":"2026-02-25T11:47:13.630513+0000","last_unstale":"2026-02-25T11:47:14.628351+0000","last_undegraded":"2026-02-25T11:47:14.628351+0000","last_fullsized":"2026-02-25T11:47:14.628351+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:03:33.580576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.634036+0000","last_change":"2026-02-25T11:47:13.629202+0000","last_active":"2026-02-25T11:47:14.634036+0000","last_peered":"2026-02-25T11:47:14.634036+0000","last_clean":"2026-02-25T11:47:14.634036+0000","last_became_active":"2026-02-25T11:47:13.628860+0000","last_became_peered":"2026-02-25T11:47:13.628860+0000","last_unstale":"2026-02-25T11:47:14.634036+0000","last_undegraded":"2026-02-25T11:47:14.634036+0000","last_fullsized":"2026-02-25T11:47:14.634036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:47:14.635796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628235+0000","last_change":"2026-02-25T11:47:13.638872+0000","last_active":"2026-02-25T11:47:14.628235+0000","last_peered":"2026-02-25T11:47:14.628235+0000","last_clean":"2026-02-25T11:47:14.628235+0000","last_became_active":"2026-02-25T11:47:13.638443+0000","last_became_peered":"2026-02-25T11:47:13.638443+0000","last_unstale":"2026-02-25T11:47:14.628235+0000","last_undegraded":"2026-02-25T11:47:14.628235+0000","last_fullsized":"2026-02-25T11:47:14.628235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:51.699838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628595+0000","last_change":"2026-02-25T11:47:13.635894+0000","last_active":"2026-02-25T11:47:14.628595+0000","last_peered":"2026-02-25T11:47:14.628595+0000","last_clean":"2026-02-25T11:47:14.628595+0000","last_became_active":"2026-02-25T11:47:13.635725+0000","last_became_peered":"2026-02-25T11:47:13.635725+0000","last_unstale":"2026-02-25T11:47:14.628595+0000","last_undegraded":"2026-02-25T11:47:14.628595+0000","last_fullsized":"2026-02-25T11:47:14.628595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:28:20.134454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749794+0000","last_change":"2026-02-25T11:47:13.628117+0000","last_active":"2026-02-25T11:47:14.749794+0000","last_peered":"2026-02-25T11:47:14.749794+0000","last_clean":"2026-02-25T11:47:14.749794+0000","last_became_active":"2026-02-25T11:47:13.628037+0000","last_became_peered":"2026-02-25T11:47:13.628037+0000","last_unstale":"2026-02-25T11:47:14.749794+0000","last_undegraded":"2026-02-25T11:47:14.749794+0000","last_fullsized":"2026-02-25T11:47:14.749794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:28:22.976133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750830+0000","last_change":"2026-02-25T11:47:13.634652+0000","last_active":"2026-02-25T11:47:14.750830+0000","last_peered":"2026-02-25T11:47:14.750830+0000","last_clean":"2026-02-25T11:47:14.750830+0000","last_became_active":"2026-02-25T11:47:13.634433+0000","last_became_peered":"2026-02-25T11:47:13.634433+0000","last_unstale":"2026-02-25T11:47:14.750830+0000","last_undegraded":"2026-02-25T11:47:14.750830+0000","last_fullsized":"2026-02-25T11:47:14.750830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:01:29.689085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635319+0000","last_change":"2026-02-25T11:47:13.632721+0000","last_active":"2026-02-25T11:47:14.635319+0000","last_peered":"2026-02-25T11:47:14.635319+0000","last_clean":"2026-02-25T11:47:14.635319+0000","last_became_active":"2026-02-25T11:47:13.632626+0000","last_became_peered":"2026-02-25T11:47:13.632626+0000","last_unstale":"2026-02-25T11:47:14.635319+0000","last_undegraded":"2026-02-25T11:47:14.635319+0000","last_fullsized":"2026-02-25T11:47:14.635319+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:28:35.201966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749844+0000","last_change":"2026-02-25T11:47:13.637504+0000","last_active":"2026-02-25T11:47:14.749844+0000","last_peered":"2026-02-25T11:47:14.749844+0000","last_clean":"2026-02-25T11:47:14.749844+0000","last_became_active":"2026-02-25T11:47:13.636824+0000","last_became_peered":"2026-02-25T11:47:13.636824+0000","last_unstale":"2026-02-25T11:47:14.749844+0000","last_undegraded":"2026-02-25T11:47:14.749844+0000","last_fullsized":"2026-02-25T11:47:14.749844+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:20:31.416302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635724+0000","last_change":"2026-02-25T11:47:13.633135+0000","last_active":"2026-02-25T11:47:14.635724+0000","last_peered":"2026-02-25T11:47:14.635724+0000","last_clean":"2026-02-25T11:47:14.635724+0000","last_became_active":"2026-02-25T11:47:13.632966+0000","last_became_peered":"2026-02-25T11:47:13.632966+0000","last_unstale":"2026-02-25T11:47:14.635724+0000","last_undegraded":"2026-02-25T11:47:14.635724+0000","last_fullsized":"2026-02-25T11:47:14.635724+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:26:17.052924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628538+0000","last_change":"2026-02-25T11:47:13.633210+0000","last_active":"2026-02-25T11:47:14.628538+0000","last_peered":"2026-02-25T11:47:14.628538+0000","last_clean":"2026-02-25T11:47:14.628538+0000","last_became_active":"2026-02-25T11:47:13.633099+0000","last_became_peered":"2026-02-25T11:47:13.633099+0000","last_unstale":"2026-02-25T11:47:14.628538+0000","last_undegraded":"2026-02-25T11:47:14.628538+0000","last_fullsized":"2026-02-25T11:47:14.628538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:07:46.884629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628656+0000","last_change":"2026-02-25T11:47:13.634757+0000","last_active":"2026-02-25T11:47:14.628656+0000","last_peered":"2026-02-25T11:47:14.628656+0000","last_clean":"2026-02-25T11:47:14.628656+0000","last_became_active":"2026-02-25T11:47:13.634643+0000","last_became_peered":"2026-02-25T11:47:13.634643+0000","last_unstale":"2026-02-25T11:47:14.628656+0000","last_undegraded":"2026-02-25T11:47:14.628656+0000","last_fullsized":"2026-02-25T11:47:14.628656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:20:23.383359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749674+0000","last_change":"2026-02-25T11:47:13.630515+0000","last_active":"2026-02-25T11:47:14.749674+0000","last_peered":"2026-02-25T11:47:14.749674+0000","last_clean":"2026-02-25T11:47:14.749674+0000","last_became_active":"2026-02-25T11:47:13.630202+0000","last_became_peered":"2026-02-25T11:47:13.630202+0000","last_unstale":"2026-02-25T11:47:14.749674+0000","last_undegraded":"2026-02-25T11:47:14.749674+0000","last_fullsized":"2026-02-25T11:47:14.749674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:51:24.646614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627367+0000","last_change":"2026-02-25T11:47:13.635749+0000","last_active":"2026-02-25T11:47:14.627367+0000","last_peered":"2026-02-25T11:47:14.627367+0000","last_clean":"2026-02-25T11:47:14.627367+0000","last_became_active":"2026-02-25T11:47:13.635284+0000","last_became_peered":"2026-02-25T11:47:13.635284+0000","last_unstale":"2026-02-25T11:47:14.627367+0000","last_undegraded":"2026-02-25T11:47:14.627367+0000","last_fullsized":"2026-02-25T11:47:14.627367+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:23:45.042142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628732+0000","last_change":"2026-02-25T11:47:13.635448+0000","last_active":"2026-02-25T11:47:14.628732+0000","last_peered":"2026-02-25T11:47:14.628732+0000","last_clean":"2026-02-25T11:47:14.628732+0000","last_became_active":"2026-02-25T11:47:13.635232+0000","last_became_peered":"2026-02-25T11:47:13.635232+0000","last_unstale":"2026-02-25T11:47:14.628732+0000","last_undegraded":"2026-02-25T11:47:14.628732+0000","last_fullsized":"2026-02-25T11:47:14.628732+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:33:38.020605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628332+0000","last_change":"2026-02-25T11:47:13.634237+0000","last_active":"2026-02-25T11:47:14.628332+0000","last_peered":"2026-02-25T11:47:14.628332+0000","last_clean":"2026-02-25T11:47:14.628332+0000","last_became_active":"2026-02-25T11:47:13.634054+0000","last_became_peered":"2026-02-25T11:47:13.634054+0000","last_unstale":"2026-02-25T11:47:14.628332+0000","last_undegraded":"2026-02-25T11:47:14.628332+0000","last_fullsized":"2026-02-25T11:47:14.628332+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:38:34.603077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627786+0000","last_change":"2026-02-25T11:47:13.638347+0000","last_active":"2026-02-25T11:47:14.627786+0000","last_peered":"2026-02-25T11:47:14.627786+0000","last_clean":"2026-02-25T11:47:14.627786+0000","last_became_active":"2026-02-25T11:47:13.637984+0000","last_became_peered":"2026-02-25T11:47:13.637984+0000","last_unstale":"2026-02-25T11:47:14.627786+0000","last_undegraded":"2026-02-25T11:47:14.627786+0000","last_fullsized":"2026-02-25T11:47:14.627786+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:36:11.273078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628202+0000","last_change":"2026-02-25T11:47:13.634523+0000","last_active":"2026-02-25T11:47:14.628202+0000","last_peered":"2026-02-25T11:47:14.628202+0000","last_clean":"2026-02-25T11:47:14.628202+0000","last_became_active":"2026-02-25T11:47:13.634184+0000","last_became_peered":"2026-02-25T11:47:13.634184+0000","last_unstale":"2026-02-25T11:47:14.628202+0000","last_undegraded":"2026-02-25T11:47:14.628202+0000","last_fullsized":"2026-02-25T11:47:14.628202+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:52:23.917090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":56,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:47.599052+0000","last_change":"2026-02-25T11:47:12.613380+0000","last_active":"2026-02-25T11:47:47.599052+0000","last_peered":"2026-02-25T11:47:47.599052+0000","last_clean":"2026-02-25T11:47:47.599052+0000","last_became_active":"2026-02-25T11:47:12.613224+0000","last_became_peered":"2026-02-25T11:47:12.613224+0000","last_unstale":"2026-02-25T11:47:47.599052+0000","last_undegraded":"2026-02-25T11:47:47.599052+0000","last_fullsized":"2026-02-25T11:47:47.599052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:59:39.894067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1135,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749915+0000","last_change":"2026-02-25T11:47:13.636227+0000","last_active":"2026-02-25T11:47:14.749915+0000","last_peered":"2026-02-25T11:47:14.749915+0000","last_clean":"2026-02-25T11:47:14.749915+0000","last_became_active":"2026-02-25T11:47:13.635212+0000","last_became_peered":"2026-02-25T11:47:13.635212+0000","last_unstale":"2026-02-25T11:47:14.749915+0000","last_undegraded":"2026-02-25T11:47:14.749915+0000","last_fullsized":"2026-02-25T11:47:14.749915+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:51:07.635399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657308+0000","last_change":"2026-02-25T11:47:12.613789+0000","last_active":"2026-02-25T11:47:17.657308+0000","last_peered":"2026-02-25T11:47:17.657308+0000","last_clean":"2026-02-25T11:47:17.657308+0000","last_became_active":"2026-02-25T11:47:12.613635+0000","last_became_peered":"2026-02-25T11:47:12.613635+0000","last_unstale":"2026-02-25T11:47:17.657308+0000","last_undegraded":"2026-02-25T11:47:17.657308+0000","last_fullsized":"2026-02-25T11:47:17.657308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:13:32.216593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657439+0000","last_change":"2026-02-25T11:47:12.612012+0000","last_active":"2026-02-25T11:47:17.657439+0000","last_peered":"2026-02-25T11:47:17.657439+0000","last_clean":"2026-02-25T11:47:17.657439+0000","last_became_active":"2026-02-25T11:47:12.611862+0000","last_became_peered":"2026-02-25T11:47:12.611862+0000","last_unstale":"2026-02-25T11:47:17.657439+0000","last_undegraded":"2026-02-25T11:47:17.657439+0000","last_fullsized":"2026-02-25T11:47:17.657439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:08:17.490884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635279+0000","last_change":"2026-02-25T11:47:13.627129+0000","last_active":"2026-02-25T11:47:14.635279+0000","last_peered":"2026-02-25T11:47:14.635279+0000","last_clean":"2026-02-25T11:47:14.635279+0000","last_became_active":"2026-02-25T11:47:13.626987+0000","last_became_peered":"2026-02-25T11:47:13.626987+0000","last_unstale":"2026-02-25T11:47:14.635279+0000","last_undegraded":"2026-02-25T11:47:14.635279+0000","last_fullsized":"2026-02-25T11:47:14.635279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:53:45.973855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749940+0000","last_change":"2026-02-25T11:47:12.611992+0000","last_active":"2026-02-25T11:47:14.749940+0000","last_peered":"2026-02-25T11:47:14.749940+0000","last_clean":"2026-02-25T11:47:14.749940+0000","last_became_active":"2026-02-25T11:47:12.611502+0000","last_became_peered":"2026-02-25T11:47:12.611502+0000","last_unstale":"2026-02-25T11:47:14.749940+0000","last_undegraded":"2026-02-25T11:47:14.749940+0000","last_fullsized":"2026-02-25T11:47:14.749940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:15:45.718628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633835+0000","last_change":"2026-02-25T11:47:13.635344+0000","last_active":"2026-02-25T11:47:14.633835+0000","last_peered":"2026-02-25T11:47:14.633835+0000","last_clean":"2026-02-25T11:47:14.633835+0000","last_became_active":"2026-02-25T11:47:13.634327+0000","last_became_peered":"2026-02-25T11:47:13.634327+0000","last_unstale":"2026-02-25T11:47:14.633835+0000","last_undegraded":"2026-02-25T11:47:14.633835+0000","last_fullsized":"2026-02-25T11:47:14.633835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:30:06.778398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:45.596468+0000","last_change":"2026-02-25T11:47:12.611130+0000","last_active":"2026-02-25T11:47:45.596468+0000","last_peered":"2026-02-25T11:47:45.596468+0000","last_clean":"2026-02-25T11:47:45.596468+0000","last_became_active":"2026-02-25T11:47:12.610838+0000","last_became_peered":"2026-02-25T11:47:12.610838+0000","last_unstale":"2026-02-25T11:47:45.596468+0000","last_undegraded":"2026-02-25T11:47:45.596468+0000","last_fullsized":"2026-02-25T11:47:45.596468+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:34:19.366424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.751244+0000","last_change":"2026-02-25T11:47:13.629263+0000","last_active":"2026-02-25T11:47:14.751244+0000","last_peered":"2026-02-25T11:47:14.751244+0000","last_clean":"2026-02-25T11:47:14.751244+0000","last_became_active":"2026-02-25T11:47:13.628765+0000","last_became_peered":"2026-02-25T11:47:13.628765+0000","last_unstale":"2026-02-25T11:47:14.751244+0000","last_undegraded":"2026-02-25T11:47:14.751244+0000","last_fullsized":"2026-02-25T11:47:14.751244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:49:04.773052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.649872+0000","last_change":"2026-02-25T11:47:12.613455+0000","last_active":"2026-02-25T11:47:16.649872+0000","last_peered":"2026-02-25T11:47:16.649872+0000","last_clean":"2026-02-25T11:47:16.649872+0000","last_became_active":"2026-02-25T11:47:12.612904+0000","last_became_peered":"2026-02-25T11:47:12.612904+0000","last_unstale":"2026-02-25T11:47:16.649872+0000","last_undegraded":"2026-02-25T11:47:16.649872+0000","last_fullsized":"2026-02-25T11:47:16.649872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:16:13.149517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627673+0000","last_change":"2026-02-25T11:47:13.635178+0000","last_active":"2026-02-25T11:47:14.627673+0000","last_peered":"2026-02-25T11:47:14.627673+0000","last_clean":"2026-02-25T11:47:14.627673+0000","last_became_active":"2026-02-25T11:47:13.634985+0000","last_became_peered":"2026-02-25T11:47:13.634985+0000","last_unstale":"2026-02-25T11:47:14.627673+0000","last_undegraded":"2026-02-25T11:47:14.627673+0000","last_fullsized":"2026-02-25T11:47:14.627673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:08.068457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653282+0000","last_change":"2026-02-25T11:47:12.610853+0000","last_active":"2026-02-25T11:47:16.653282+0000","last_peered":"2026-02-25T11:47:16.653282+0000","last_clean":"2026-02-25T11:47:16.653282+0000","last_became_active":"2026-02-25T11:47:12.610639+0000","last_became_peered":"2026-02-25T11:47:12.610639+0000","last_unstale":"2026-02-25T11:47:16.653282+0000","last_undegraded":"2026-02-25T11:47:16.653282+0000","last_fullsized":"2026-02-25T11:47:16.653282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:00:06.042790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":694,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750953+0000","last_change":"2026-02-25T11:47:13.630063+0000","last_active":"2026-02-25T11:47:14.750953+0000","last_peered":"2026-02-25T11:47:14.750953+0000","last_clean":"2026-02-25T11:47:14.750953+0000","last_became_active":"2026-02-25T11:47:13.629906+0000","last_became_peered":"2026-02-25T11:47:13.629906+0000","last_unstale":"2026-02-25T11:47:14.750953+0000","last_undegraded":"2026-02-25T11:47:14.750953+0000","last_fullsized":"2026-02-25T11:47:14.750953+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:09:13.614948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":113,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:46.600201+0000","last_change":"2026-02-25T11:47:12.615044+0000","last_active":"2026-02-25T11:47:46.600201+0000","last_peered":"2026-02-25T11:47:46.600201+0000","last_clean":"2026-02-25T11:47:46.600201+0000","last_became_active":"2026-02-25T11:47:12.614698+0000","last_became_peered":"2026-02-25T11:47:12.614698+0000","last_unstale":"2026-02-25T11:47:46.600201+0000","last_undegraded":"2026-02-25T11:47:46.600201+0000","last_fullsized":"2026-02-25T11:47:46.600201+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:52:32.148801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69040,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":67,"num_read_kb":100,"num_write":31,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628510+0000","last_change":"2026-02-25T11:47:13.630945+0000","last_active":"2026-02-25T11:47:14.628510+0000","last_peered":"2026-02-25T11:47:14.628510+0000","last_clean":"2026-02-25T11:47:14.628510+0000","last_became_active":"2026-02-25T11:47:13.630693+0000","last_became_peered":"2026-02-25T11:47:13.630693+0000","last_unstale":"2026-02-25T11:47:14.628510+0000","last_undegraded":"2026-02-25T11:47:14.628510+0000","last_fullsized":"2026-02-25T11:47:14.628510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:39:52.744366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656758+0000","last_change":"2026-02-25T11:47:12.614950+0000","last_active":"2026-02-25T11:47:17.656758+0000","last_peered":"2026-02-25T11:47:17.656758+0000","last_clean":"2026-02-25T11:47:17.656758+0000","last_became_active":"2026-02-25T11:47:12.614272+0000","last_became_peered":"2026-02-25T11:47:12.614272+0000","last_unstale":"2026-02-25T11:47:17.656758+0000","last_undegraded":"2026-02-25T11:47:17.656758+0000","last_fullsized":"2026-02-25T11:47:17.656758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:45:50.928166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749596+0000","last_change":"2026-02-25T11:47:13.633128+0000","last_active":"2026-02-25T11:47:14.749596+0000","last_peered":"2026-02-25T11:47:14.749596+0000","last_clean":"2026-02-25T11:47:14.749596+0000","last_became_active":"2026-02-25T11:47:13.633047+0000","last_became_peered":"2026-02-25T11:47:13.633047+0000","last_unstale":"2026-02-25T11:47:14.749596+0000","last_undegraded":"2026-02-25T11:47:14.749596+0000","last_fullsized":"2026-02-25T11:47:14.749596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:56:33.191193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.653783+0000","last_change":"2026-02-25T11:47:12.616898+0000","last_active":"2026-02-25T11:47:16.653783+0000","last_peered":"2026-02-25T11:47:16.653783+0000","last_clean":"2026-02-25T11:47:16.653783+0000","last_became_active":"2026-02-25T11:47:12.616774+0000","last_became_peered":"2026-02-25T11:47:12.616774+0000","last_unstale":"2026-02-25T11:47:16.653783+0000","last_undegraded":"2026-02-25T11:47:16.653783+0000","last_fullsized":"2026-02-25T11:47:16.653783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:59:33.930544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628464+0000","last_change":"2026-02-25T11:47:13.630377+0000","last_active":"2026-02-25T11:47:14.628464+0000","last_peered":"2026-02-25T11:47:14.628464+0000","last_clean":"2026-02-25T11:47:14.628464+0000","last_became_active":"2026-02-25T11:47:13.630124+0000","last_became_peered":"2026-02-25T11:47:13.630124+0000","last_unstale":"2026-02-25T11:47:14.628464+0000","last_undegraded":"2026-02-25T11:47:14.628464+0000","last_fullsized":"2026-02-25T11:47:14.628464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:42:12.028258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:43.604899+0000","last_change":"2026-02-25T11:47:12.613208+0000","last_active":"2026-02-25T11:47:43.604899+0000","last_peered":"2026-02-25T11:47:43.604899+0000","last_clean":"2026-02-25T11:47:43.604899+0000","last_became_active":"2026-02-25T11:47:12.612460+0000","last_became_peered":"2026-02-25T11:47:12.612460+0000","last_unstale":"2026-02-25T11:47:43.604899+0000","last_undegraded":"2026-02-25T11:47:43.604899+0000","last_fullsized":"2026-02-25T11:47:43.604899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:15:59.810632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":998,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628121+0000","last_change":"2026-02-25T11:47:13.631123+0000","last_active":"2026-02-25T11:47:14.628121+0000","last_peered":"2026-02-25T11:47:14.628121+0000","last_clean":"2026-02-25T11:47:14.628121+0000","last_became_active":"2026-02-25T11:47:13.631031+0000","last_became_peered":"2026-02-25T11:47:13.631031+0000","last_unstale":"2026-02-25T11:47:14.628121+0000","last_undegraded":"2026-02-25T11:47:14.628121+0000","last_fullsized":"2026-02-25T11:47:14.628121+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:47:54.073226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.657937+0000","last_change":"2026-02-25T11:47:12.612153+0000","last_active":"2026-02-25T11:47:17.657937+0000","last_peered":"2026-02-25T11:47:17.657937+0000","last_clean":"2026-02-25T11:47:17.657937+0000","last_became_active":"2026-02-25T11:47:12.611107+0000","last_became_peered":"2026-02-25T11:47:12.611107+0000","last_unstale":"2026-02-25T11:47:17.657937+0000","last_undegraded":"2026-02-25T11:47:17.657937+0000","last_fullsized":"2026-02-25T11:47:17.657937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:37:21.791728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749858+0000","last_change":"2026-02-25T11:47:13.635800+0000","last_active":"2026-02-25T11:47:14.749858+0000","last_peered":"2026-02-25T11:47:14.749858+0000","last_clean":"2026-02-25T11:47:14.749858+0000","last_became_active":"2026-02-25T11:47:13.635590+0000","last_became_peered":"2026-02-25T11:47:13.635590+0000","last_unstale":"2026-02-25T11:47:14.749858+0000","last_undegraded":"2026-02-25T11:47:14.749858+0000","last_fullsized":"2026-02-25T11:47:14.749858+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:46:36.324518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575342+0000","last_change":"2026-02-25T11:47:12.616473+0000","last_active":"2026-02-25T11:47:18.575342+0000","last_peered":"2026-02-25T11:47:18.575342+0000","last_clean":"2026-02-25T11:47:18.575342+0000","last_became_active":"2026-02-25T11:47:12.616039+0000","last_became_peered":"2026-02-25T11:47:12.616039+0000","last_unstale":"2026-02-25T11:47:18.575342+0000","last_undegraded":"2026-02-25T11:47:18.575342+0000","last_fullsized":"2026-02-25T11:47:18.575342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:03:28.324889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628413+0000","last_change":"2026-02-25T11:47:13.634771+0000","last_active":"2026-02-25T11:47:14.628413+0000","last_peered":"2026-02-25T11:47:14.628413+0000","last_clean":"2026-02-25T11:47:14.628413+0000","last_became_active":"2026-02-25T11:47:13.634672+0000","last_became_peered":"2026-02-25T11:47:13.634672+0000","last_unstale":"2026-02-25T11:47:14.628413+0000","last_undegraded":"2026-02-25T11:47:14.628413+0000","last_fullsized":"2026-02-25T11:47:14.628413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:19:03.692423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749863+0000","last_change":"2026-02-25T11:47:12.616685+0000","last_active":"2026-02-25T11:47:14.749863+0000","last_peered":"2026-02-25T11:47:14.749863+0000","last_clean":"2026-02-25T11:47:14.749863+0000","last_became_active":"2026-02-25T11:47:12.616518+0000","last_became_peered":"2026-02-25T11:47:12.616518+0000","last_unstale":"2026-02-25T11:47:14.749863+0000","last_undegraded":"2026-02-25T11:47:14.749863+0000","last_fullsized":"2026-02-25T11:47:14.749863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:28:50.684568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628462+0000","last_change":"2026-02-25T11:47:13.627734+0000","last_active":"2026-02-25T11:47:14.628462+0000","last_peered":"2026-02-25T11:47:14.628462+0000","last_clean":"2026-02-25T11:47:14.628462+0000","last_became_active":"2026-02-25T11:47:13.627632+0000","last_became_peered":"2026-02-25T11:47:13.627632+0000","last_unstale":"2026-02-25T11:47:14.628462+0000","last_undegraded":"2026-02-25T11:47:14.628462+0000","last_fullsized":"2026-02-25T11:47:14.628462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:01:20.907699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750104+0000","last_change":"2026-02-25T11:47:12.612830+0000","last_active":"2026-02-25T11:47:14.750104+0000","last_peered":"2026-02-25T11:47:14.750104+0000","last_clean":"2026-02-25T11:47:14.750104+0000","last_became_active":"2026-02-25T11:47:12.612661+0000","last_became_peered":"2026-02-25T11:47:12.612661+0000","last_unstale":"2026-02-25T11:47:14.750104+0000","last_undegraded":"2026-02-25T11:47:14.750104+0000","last_fullsized":"2026-02-25T11:47:14.750104+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:05:15.367058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:15.914932+0000","last_change":"2026-02-25T11:47:13.633998+0000","last_active":"2026-02-25T11:47:15.914932+0000","last_peered":"2026-02-25T11:47:15.914932+0000","last_clean":"2026-02-25T11:47:15.914932+0000","last_became_active":"2026-02-25T11:47:13.633820+0000","last_became_peered":"2026-02-25T11:47:13.633820+0000","last_unstale":"2026-02-25T11:47:15.914932+0000","last_undegraded":"2026-02-25T11:47:15.914932+0000","last_fullsized":"2026-02-25T11:47:15.914932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:02:38.693106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.658417+0000","last_change":"2026-02-25T11:47:12.614091+0000","last_active":"2026-02-25T11:47:17.658417+0000","last_peered":"2026-02-25T11:47:17.658417+0000","last_clean":"2026-02-25T11:47:17.658417+0000","last_became_active":"2026-02-25T11:47:12.612962+0000","last_became_peered":"2026-02-25T11:47:12.612962+0000","last_unstale":"2026-02-25T11:47:17.658417+0000","last_undegraded":"2026-02-25T11:47:17.658417+0000","last_fullsized":"2026-02-25T11:47:17.658417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:11:40.214146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628504+0000","last_change":"2026-02-25T11:47:13.627288+0000","last_active":"2026-02-25T11:47:14.628504+0000","last_peered":"2026-02-25T11:47:14.628504+0000","last_clean":"2026-02-25T11:47:14.628504+0000","last_became_active":"2026-02-25T11:47:13.626902+0000","last_became_peered":"2026-02-25T11:47:13.626902+0000","last_unstale":"2026-02-25T11:47:14.628504+0000","last_undegraded":"2026-02-25T11:47:14.628504+0000","last_fullsized":"2026-02-25T11:47:14.628504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:38:19.137242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"21'152","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628095+0000","last_change":"2026-02-25T11:46:35.865282+0000","last_active":"2026-02-25T11:47:14.628095+0000","last_peered":"2026-02-25T11:47:14.628095+0000","last_clean":"2026-02-25T11:47:14.628095+0000","last_became_active":"2026-02-25T11:46:35.865139+0000","last_became_peered":"2026-02-25T11:46:35.865139+0000","last_unstale":"2026-02-25T11:47:14.628095+0000","last_undegraded":"2026-02-25T11:47:14.628095+0000","last_fullsized":"2026-02-25T11:47:14.628095+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:44:43.739673+0000","last_clean_scrub_stamp":"2026-02-25T11:44:43.739673+0000","objects_scrubbed":0,"log_size":152,"log_dups_size":0,"ondisk_log_size":152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:29:57.573039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:18.575170+0000","last_change":"2026-02-25T11:47:12.612834+0000","last_active":"2026-02-25T11:47:18.575170+0000","last_peered":"2026-02-25T11:47:18.575170+0000","last_clean":"2026-02-25T11:47:18.575170+0000","last_became_active":"2026-02-25T11:47:12.611827+0000","last_became_peered":"2026-02-25T11:47:12.611827+0000","last_unstale":"2026-02-25T11:47:18.575170+0000","last_undegraded":"2026-02-25T11:47:18.575170+0000","last_fullsized":"2026-02-25T11:47:18.575170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:05:32.112908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635504+0000","last_change":"2026-02-25T11:47:13.634534+0000","last_active":"2026-02-25T11:47:14.635504+0000","last_peered":"2026-02-25T11:47:14.635504+0000","last_clean":"2026-02-25T11:47:14.635504+0000","last_became_active":"2026-02-25T11:47:13.633769+0000","last_became_peered":"2026-02-25T11:47:13.633769+0000","last_unstale":"2026-02-25T11:47:14.635504+0000","last_undegraded":"2026-02-25T11:47:14.635504+0000","last_fullsized":"2026-02-25T11:47:14.635504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:16:09.255508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656541+0000","last_change":"2026-02-25T11:47:12.612621+0000","last_active":"2026-02-25T11:47:17.656541+0000","last_peered":"2026-02-25T11:47:17.656541+0000","last_clean":"2026-02-25T11:47:17.656541+0000","last_became_active":"2026-02-25T11:47:12.612284+0000","last_became_peered":"2026-02-25T11:47:12.612284+0000","last_unstale":"2026-02-25T11:47:17.656541+0000","last_undegraded":"2026-02-25T11:47:17.656541+0000","last_fullsized":"2026-02-25T11:47:17.656541+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:49:09.299914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749514+0000","last_change":"2026-02-25T11:47:13.629107+0000","last_active":"2026-02-25T11:47:14.749514+0000","last_peered":"2026-02-25T11:47:14.749514+0000","last_clean":"2026-02-25T11:47:14.749514+0000","last_became_active":"2026-02-25T11:47:13.629031+0000","last_became_peered":"2026-02-25T11:47:13.629031+0000","last_unstale":"2026-02-25T11:47:14.749514+0000","last_undegraded":"2026-02-25T11:47:14.749514+0000","last_fullsized":"2026-02-25T11:47:14.749514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:07:58.915416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":50,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:46.606658+0000","last_change":"2026-02-25T11:47:12.616543+0000","last_active":"2026-02-25T11:47:46.606658+0000","last_peered":"2026-02-25T11:47:46.606658+0000","last_clean":"2026-02-25T11:47:46.606658+0000","last_became_active":"2026-02-25T11:47:12.616345+0000","last_became_peered":"2026-02-25T11:47:12.616345+0000","last_unstale":"2026-02-25T11:47:46.606658+0000","last_undegraded":"2026-02-25T11:47:46.606658+0000","last_fullsized":"2026-02-25T11:47:46.606658+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:27:30.862168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750014+0000","last_change":"2026-02-25T11:47:13.637385+0000","last_active":"2026-02-25T11:47:14.750014+0000","last_peered":"2026-02-25T11:47:14.750014+0000","last_clean":"2026-02-25T11:47:14.750014+0000","last_became_active":"2026-02-25T11:47:13.634653+0000","last_became_peered":"2026-02-25T11:47:13.634653+0000","last_unstale":"2026-02-25T11:47:14.750014+0000","last_undegraded":"2026-02-25T11:47:14.750014+0000","last_fullsized":"2026-02-25T11:47:14.750014+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:05:08.662208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:15.914921+0000","last_change":"2026-02-25T11:47:12.614640+0000","last_active":"2026-02-25T11:47:15.914921+0000","last_peered":"2026-02-25T11:47:15.914921+0000","last_clean":"2026-02-25T11:47:15.914921+0000","last_became_active":"2026-02-25T11:47:12.614486+0000","last_became_peered":"2026-02-25T11:47:12.614486+0000","last_unstale":"2026-02-25T11:47:15.914921+0000","last_undegraded":"2026-02-25T11:47:15.914921+0000","last_fullsized":"2026-02-25T11:47:15.914921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:16:06.314709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750327+0000","last_change":"2026-02-25T11:47:13.631828+0000","last_active":"2026-02-25T11:47:14.750327+0000","last_peered":"2026-02-25T11:47:14.750327+0000","last_clean":"2026-02-25T11:47:14.750327+0000","last_became_active":"2026-02-25T11:47:13.631590+0000","last_became_peered":"2026-02-25T11:47:13.631590+0000","last_unstale":"2026-02-25T11:47:14.750327+0000","last_undegraded":"2026-02-25T11:47:14.750327+0000","last_fullsized":"2026-02-25T11:47:14.750327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:29:25.518794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.633984+0000","last_change":"2026-02-25T11:47:12.614032+0000","last_active":"2026-02-25T11:47:14.633984+0000","last_peered":"2026-02-25T11:47:14.633984+0000","last_clean":"2026-02-25T11:47:14.633984+0000","last_became_active":"2026-02-25T11:47:12.613936+0000","last_became_peered":"2026-02-25T11:47:12.613936+0000","last_unstale":"2026-02-25T11:47:14.633984+0000","last_undegraded":"2026-02-25T11:47:14.633984+0000","last_fullsized":"2026-02-25T11:47:14.633984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:16:46.397485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.859010+0000","last_change":"2026-02-25T11:47:13.636173+0000","last_active":"2026-02-25T11:47:16.859010+0000","last_peered":"2026-02-25T11:47:16.859010+0000","last_clean":"2026-02-25T11:47:16.859010+0000","last_became_active":"2026-02-25T11:47:13.635745+0000","last_became_peered":"2026-02-25T11:47:13.635745+0000","last_unstale":"2026-02-25T11:47:16.859010+0000","last_undegraded":"2026-02-25T11:47:16.859010+0000","last_fullsized":"2026-02-25T11:47:16.859010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:03:33.671318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651871+0000","last_change":"2026-02-25T11:47:12.614003+0000","last_active":"2026-02-25T11:47:16.651871+0000","last_peered":"2026-02-25T11:47:16.651871+0000","last_clean":"2026-02-25T11:47:16.651871+0000","last_became_active":"2026-02-25T11:47:12.613893+0000","last_became_peered":"2026-02-25T11:47:12.613893+0000","last_unstale":"2026-02-25T11:47:16.651871+0000","last_undegraded":"2026-02-25T11:47:16.651871+0000","last_fullsized":"2026-02-25T11:47:16.651871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:33:21.133364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.749746+0000","last_change":"2026-02-25T11:47:13.632632+0000","last_active":"2026-02-25T11:47:14.749746+0000","last_peered":"2026-02-25T11:47:14.749746+0000","last_clean":"2026-02-25T11:47:14.749746+0000","last_became_active":"2026-02-25T11:47:13.632408+0000","last_became_peered":"2026-02-25T11:47:13.632408+0000","last_unstale":"2026-02-25T11:47:14.749746+0000","last_undegraded":"2026-02-25T11:47:14.749746+0000","last_fullsized":"2026-02-25T11:47:14.749746+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T13:38:45.448637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'11","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:27.638764+0000","last_change":"2026-02-25T11:47:12.613816+0000","last_active":"2026-02-25T11:47:27.638764+0000","last_peered":"2026-02-25T11:47:27.638764+0000","last_clean":"2026-02-25T11:47:27.638764+0000","last_became_active":"2026-02-25T11:47:12.613196+0000","last_became_peered":"2026-02-25T11:47:12.613196+0000","last_unstale":"2026-02-25T11:47:27.638764+0000","last_undegraded":"2026-02-25T11:47:27.638764+0000","last_fullsized":"2026-02-25T11:47:27.638764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:23:52.372808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635752+0000","last_change":"2026-02-25T11:47:13.632431+0000","last_active":"2026-02-25T11:47:14.635752+0000","last_peered":"2026-02-25T11:47:14.635752+0000","last_clean":"2026-02-25T11:47:14.635752+0000","last_became_active":"2026-02-25T11:47:13.631972+0000","last_became_peered":"2026-02-25T11:47:13.631972+0000","last_unstale":"2026-02-25T11:47:14.635752+0000","last_undegraded":"2026-02-25T11:47:14.635752+0000","last_fullsized":"2026-02-25T11:47:14.635752+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T19:59:34.655028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.651131+0000","last_change":"2026-02-25T11:47:12.612805+0000","last_active":"2026-02-25T11:47:16.651131+0000","last_peered":"2026-02-25T11:47:16.651131+0000","last_clean":"2026-02-25T11:47:16.651131+0000","last_became_active":"2026-02-25T11:47:12.612155+0000","last_became_peered":"2026-02-25T11:47:12.612155+0000","last_unstale":"2026-02-25T11:47:16.651131+0000","last_undegraded":"2026-02-25T11:47:16.651131+0000","last_fullsized":"2026-02-25T11:47:16.651131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:58:00.429639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627853+0000","last_change":"2026-02-25T11:47:13.636089+0000","last_active":"2026-02-25T11:47:14.627853+0000","last_peered":"2026-02-25T11:47:14.627853+0000","last_clean":"2026-02-25T11:47:14.627853+0000","last_became_active":"2026-02-25T11:47:13.635963+0000","last_became_peered":"2026-02-25T11:47:13.635963+0000","last_unstale":"2026-02-25T11:47:14.627853+0000","last_undegraded":"2026-02-25T11:47:14.627853+0000","last_fullsized":"2026-02-25T11:47:14.627853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:50:28.012023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650081+0000","last_change":"2026-02-25T11:47:12.609749+0000","last_active":"2026-02-25T11:47:16.650081+0000","last_peered":"2026-02-25T11:47:16.650081+0000","last_clean":"2026-02-25T11:47:16.650081+0000","last_became_active":"2026-02-25T11:47:12.609190+0000","last_became_peered":"2026-02-25T11:47:12.609190+0000","last_unstale":"2026-02-25T11:47:16.650081+0000","last_undegraded":"2026-02-25T11:47:16.650081+0000","last_fullsized":"2026-02-25T11:47:16.650081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T22:12:42.204122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628201+0000","last_change":"2026-02-25T11:47:13.629339+0000","last_active":"2026-02-25T11:47:14.628201+0000","last_peered":"2026-02-25T11:47:14.628201+0000","last_clean":"2026-02-25T11:47:14.628201+0000","last_became_active":"2026-02-25T11:47:13.629107+0000","last_became_peered":"2026-02-25T11:47:13.629107+0000","last_unstale":"2026-02-25T11:47:14.628201+0000","last_undegraded":"2026-02-25T11:47:14.628201+0000","last_fullsized":"2026-02-25T11:47:14.628201+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:19:50.087771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.652673+0000","last_change":"2026-02-25T11:47:12.615249+0000","last_active":"2026-02-25T11:47:16.652673+0000","last_peered":"2026-02-25T11:47:16.652673+0000","last_clean":"2026-02-25T11:47:16.652673+0000","last_became_active":"2026-02-25T11:47:12.615165+0000","last_became_peered":"2026-02-25T11:47:12.615165+0000","last_unstale":"2026-02-25T11:47:16.652673+0000","last_undegraded":"2026-02-25T11:47:16.652673+0000","last_fullsized":"2026-02-25T11:47:16.652673+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:40:52.452074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627665+0000","last_change":"2026-02-25T11:47:13.631460+0000","last_active":"2026-02-25T11:47:14.627665+0000","last_peered":"2026-02-25T11:47:14.627665+0000","last_clean":"2026-02-25T11:47:14.627665+0000","last_became_active":"2026-02-25T11:47:13.631298+0000","last_became_peered":"2026-02-25T11:47:13.631298+0000","last_unstale":"2026-02-25T11:47:14.627665+0000","last_undegraded":"2026-02-25T11:47:14.627665+0000","last_fullsized":"2026-02-25T11:47:14.627665+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:26:03.362106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.648950+0000","last_change":"2026-02-25T11:47:12.615779+0000","last_active":"2026-02-25T11:47:17.648950+0000","last_peered":"2026-02-25T11:47:17.648950+0000","last_clean":"2026-02-25T11:47:17.648950+0000","last_became_active":"2026-02-25T11:47:12.615073+0000","last_became_peered":"2026-02-25T11:47:12.615073+0000","last_unstale":"2026-02-25T11:47:17.648950+0000","last_undegraded":"2026-02-25T11:47:17.648950+0000","last_fullsized":"2026-02-25T11:47:17.648950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T21:28:51.468199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635380+0000","last_change":"2026-02-25T11:47:13.632351+0000","last_active":"2026-02-25T11:47:14.635380+0000","last_peered":"2026-02-25T11:47:14.635380+0000","last_clean":"2026-02-25T11:47:14.635380+0000","last_became_active":"2026-02-25T11:47:13.632081+0000","last_became_peered":"2026-02-25T11:47:13.632081+0000","last_unstale":"2026-02-25T11:47:14.635380+0000","last_undegraded":"2026-02-25T11:47:14.635380+0000","last_fullsized":"2026-02-25T11:47:14.635380+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T18:38:59.143253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.650353+0000","last_change":"2026-02-25T11:47:12.616555+0000","last_active":"2026-02-25T11:47:16.650353+0000","last_peered":"2026-02-25T11:47:16.650353+0000","last_clean":"2026-02-25T11:47:16.650353+0000","last_became_active":"2026-02-25T11:47:12.616174+0000","last_became_peered":"2026-02-25T11:47:12.616174+0000","last_unstale":"2026-02-25T11:47:16.650353+0000","last_undegraded":"2026-02-25T11:47:16.650353+0000","last_fullsized":"2026-02-25T11:47:16.650353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T14:01:17.494129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628458+0000","last_change":"2026-02-25T11:47:13.627400+0000","last_active":"2026-02-25T11:47:14.628458+0000","last_peered":"2026-02-25T11:47:14.628458+0000","last_clean":"2026-02-25T11:47:14.628458+0000","last_became_active":"2026-02-25T11:47:13.626782+0000","last_became_peered":"2026-02-25T11:47:13.626782+0000","last_unstale":"2026-02-25T11:47:14.628458+0000","last_undegraded":"2026-02-25T11:47:14.628458+0000","last_fullsized":"2026-02-25T11:47:14.628458+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:42:39.996935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750027+0000","last_change":"2026-02-25T11:47:13.634030+0000","last_active":"2026-02-25T11:47:14.750027+0000","last_peered":"2026-02-25T11:47:14.750027+0000","last_clean":"2026-02-25T11:47:14.750027+0000","last_became_active":"2026-02-25T11:47:13.632844+0000","last_became_peered":"2026-02-25T11:47:13.632844+0000","last_unstale":"2026-02-25T11:47:14.750027+0000","last_undegraded":"2026-02-25T11:47:14.750027+0000","last_fullsized":"2026-02-25T11:47:14.750027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T17:52:16.013596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.628805+0000","last_change":"2026-02-25T11:47:12.613666+0000","last_active":"2026-02-25T11:47:14.628805+0000","last_peered":"2026-02-25T11:47:14.628805+0000","last_clean":"2026-02-25T11:47:14.628805+0000","last_became_active":"2026-02-25T11:47:12.613570+0000","last_became_peered":"2026-02-25T11:47:12.613570+0000","last_unstale":"2026-02-25T11:47:14.628805+0000","last_undegraded":"2026-02-25T11:47:14.628805+0000","last_fullsized":"2026-02-25T11:47:14.628805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T23:28:39.750737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.750924+0000","last_change":"2026-02-25T11:47:13.631515+0000","last_active":"2026-02-25T11:47:14.750924+0000","last_peered":"2026-02-25T11:47:14.750924+0000","last_clean":"2026-02-25T11:47:14.750924+0000","last_became_active":"2026-02-25T11:47:13.631161+0000","last_became_peered":"2026-02-25T11:47:13.631161+0000","last_unstale":"2026-02-25T11:47:14.750924+0000","last_undegraded":"2026-02-25T11:47:14.750924+0000","last_fullsized":"2026-02-25T11:47:14.750924+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:13:30.496521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656891+0000","last_change":"2026-02-25T11:47:12.615904+0000","last_active":"2026-02-25T11:47:17.656891+0000","last_peered":"2026-02-25T11:47:17.656891+0000","last_clean":"2026-02-25T11:47:17.656891+0000","last_became_active":"2026-02-25T11:47:12.615784+0000","last_became_peered":"2026-02-25T11:47:12.615784+0000","last_unstale":"2026-02-25T11:47:17.656891+0000","last_undegraded":"2026-02-25T11:47:17.656891+0000","last_fullsized":"2026-02-25T11:47:17.656891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T20:32:26.802870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.627754+0000","last_change":"2026-02-25T11:47:13.630065+0000","last_active":"2026-02-25T11:47:14.627754+0000","last_peered":"2026-02-25T11:47:14.627754+0000","last_clean":"2026-02-25T11:47:14.627754+0000","last_became_active":"2026-02-25T11:47:13.629214+0000","last_became_peered":"2026-02-25T11:47:13.629214+0000","last_unstale":"2026-02-25T11:47:14.627754+0000","last_undegraded":"2026-02-25T11:47:14.627754+0000","last_fullsized":"2026-02-25T11:47:14.627754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T15:53:45.639526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:16.310632+0000","last_change":"2026-02-25T11:47:12.612956+0000","last_active":"2026-02-25T11:47:16.310632+0000","last_peered":"2026-02-25T11:47:16.310632+0000","last_clean":"2026-02-25T11:47:16.310632+0000","last_became_active":"2026-02-25T11:47:12.612843+0000","last_became_peered":"2026-02-25T11:47:12.612843+0000","last_unstale":"2026-02-25T11:47:16.310632+0000","last_undegraded":"2026-02-25T11:47:16.310632+0000","last_fullsized":"2026-02-25T11:47:16.310632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T12:41:30.867465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:14.635780+0000","last_change":"2026-02-25T11:47:13.637712+0000","last_active":"2026-02-25T11:47:14.635780+0000","last_peered":"2026-02-25T11:47:14.635780+0000","last_clean":"2026-02-25T11:47:14.635780+0000","last_became_active":"2026-02-25T11:47:13.635030+0000","last_became_peered":"2026-02-25T11:47:13.635030+0000","last_unstale":"2026-02-25T11:47:14.635780+0000","last_undegraded":"2026-02-25T11:47:14.635780+0000","last_fullsized":"2026-02-25T11:47:14.635780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:12.598575+0000","last_clean_scrub_stamp":"2026-02-25T11:47:12.598575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T16:01:22.896129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2026-02-25T11:47:17.656736+0000","last_change":"2026-02-25T11:47:12.614149+0000","last_active":"2026-02-25T11:47:17.656736+0000","last_peered":"2026-02-25T11:47:17.656736+0000","last_clean":"2026-02-25T11:47:17.656736+0000","last_became_active":"2026-02-25T11:47:12.613137+0000","last_became_peered":"2026-02-25T11:47:12.613137+0000","last_unstale":"2026-02-25T11:47:17.656736+0000","last_undegraded":"2026-02-25T11:47:17.656736+0000","last_fullsized":"2026-02-25T11:47:17.656736+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T11:47:11.591176+0000","last_clean_scrub_stamp":"2026-02-25T11:47:11.591176+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T11:49:49.451435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":604,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73639,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":203,"num_read_kb":178,"num_write":168,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":450560,"data_stored":228821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":227295,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":189,"num_write_kb":3716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":262884,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28132,"kb_used_data":912,"kb_used_omap":9,"kb_used_meta":27190,"kb_avail":733975068,"statfs":{"total":751619276800,"available":751590469632,"internally_reserved":0,"allocated":933888,"data_stored":455512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10109,"internal_metadata":27842691},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612625,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28288,"kb_used_data":1068,"kb_used_omap":28,"kb_used_meta":27171,"kb_avail":733974912,"statfs":{"total":751619276800,"available":751590309888,"internally_reserved":0,"allocated":1093632,"data_stored":1116443,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":29123,"internal_metadata":27823677},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776148,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28148,"kb_used_data":928,"kb_used_omap":27,"kb_used_meta":27172,"kb_avail":733975052,"statfs":{"total":751619276800,"available":751590453248,"internally_reserved":0,"allocated":950272,"data_stored":455748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":28131,"internal_metadata":27824669},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037782,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28148,"kb_used_data":928,"kb_used_omap":35,"kb_used_meta":27164,"kb_avail":733975052,"statfs":{"total":751619276800,"available":751590453248,"internally_reserved":0,"allocated":950272,"data_stored":455668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36257,"internal_metadata":27816543},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299417,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28196,"kb_used_data":976,"kb_used_omap":35,"kb_used_meta":27164,"kb_avail":733975004,"statfs":{"total":751619276800,"available":751590404096,"internally_reserved":0,"allocated":999424,"data_stored":1047617,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36168,"internal_metadata":27816632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462941,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28136,"kb_used_data":916,"kb_used_omap":31,"kb_used_meta":27168,"kb_avail":733975064,"statfs":{"total":751619276800,"available":751590465536,"internally_reserved":0,"allocated":937984,"data_stored":455115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32619,"internal_metadata":27820181},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724575,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28208,"kb_used_data":988,"kb_used_omap":36,"kb_used_meta":27163,"kb_avail":733974992,"statfs":{"total":751619276800,"available":751590391808,"internally_reserved":0,"allocated":1011712,"data_stored":1047404,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":36952,"internal_metadata":27815848},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888097,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28140,"kb_used_data":920,"kb_used_omap":45,"kb_used_meta":27154,"kb_avail":733975060,"statfs":{"total":751619276800,"available":751590461440,"internally_reserved":0,"allocated":942080,"data_stored":455962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":46332,"internal_metadata":27806468},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149732,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28152,"kb_used_data":932,"kb_used_omap":51,"kb_used_meta":27148,"kb_avail":733975048,"statfs":{"total":751619276800,"available":751590449152,"internally_reserved":0,"allocated":954368,"data_stored":455426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":52913,"internal_metadata":27799887},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378663,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28236,"kb_used_data":1016,"kb_used_omap":64,"kb_used_meta":27135,"kb_avail":733974964,"statfs":{"total":751619276800,"available":751590363136,"internally_reserved":0,"allocated":1040384,"data_stored":534250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":66234,"internal_metadata":27786566},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542185,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28204,"kb_used_data":984,"kb_used_omap":43,"kb_used_meta":27156,"kb_avail":733974996,"statfs":{"total":751619276800,"available":751590395904,"internally_reserved":0,"allocated":1007616,"data_stored":523948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44488,"internal_metadata":27808312},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":734003200,"kb_used":28156,"kb_used_data":936,"kb_used_omap":54,"kb_used_meta":27145,"kb_avail":733975044,"statfs":{"total":751619276800,"available":751590445056,"internally_reserved":0,"allocated":958464,"data_stored":456430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56100,"internal_metadata":27796700},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":36422,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":38008,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8853,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1407,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":79342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":56705,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":49702,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6449,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3478,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14040,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9858,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25902,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":22346,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":71050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21693,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6862,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T11:47:50.719 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:47:50.935 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:51.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.108+0000 7f797c27e640 1 -- 10.20.193.28:0/2586806456 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f797410c280 msgr2=0x7f797410ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:51.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.108+0000 7f797c27e640 1 --2- 10.20.193.28:0/2586806456 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f797410c280 0x7f797410ec60 secure :-1 s=READY pgs=354 gs=2 cs=0 l=1 c_cookie=10428912845799045097 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7968009950 tx=0x7f796802f190 comp rx=0 tx=0).stop 2026-02-25T11:47:51.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.108+0000 7f797c27e640 1 -- 10.20.193.28:0/2586806456 shutdown_connections 2026-02-25T11:47:51.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.108+0000 7f797c27e640 1 --2- 10.20.193.28:0/2586806456 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f797410f2b0 0x7f797410f6f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.108+0000 7f797c27e640 1 --2- 10.20.193.28:0/2586806456 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f797410c280 0x7f797410ec60 unknown :-1 s=CLOSED pgs=354 gs=2 cs=0 l=1 c_cookie=10428912845799045097 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.108+0000 7f797c27e640 1 --2- 10.20.193.28:0/2586806456 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f797410b970 0x7f797410bd40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.108+0000 7f797c27e640 1 -- 10.20.193.28:0/2586806456 >> 10.20.193.28:0/2586806456 conn(0x7f79740e94c0 msgr2=0x7f79741098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:51.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.108+0000 7f797c27e640 1 -- 10.20.193.28:0/2586806456 shutdown_connections 2026-02-25T11:47:51.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.108+0000 7f797c27e640 1 -- 10.20.193.28:0/2586806456 wait complete. 2026-02-25T11:47:51.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.109+0000 7f797c27e640 1 Processor -- start 2026-02-25T11:47:51.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.109+0000 7f797c27e640 1 -- start start 2026-02-25T11:47:51.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.109+0000 7f797c27e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f797410b970 0x7f79740fae40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:51.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.109+0000 7f797c27e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f797410c280 0x7f79740fb380 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:51.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797c27e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f797410f2b0 0x7f79740ffd60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:51.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797c27e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7974100330 con 0x7f797410f2b0 2026-02-25T11:47:51.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797c27e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f79741004a0 con 0x7f797410c280 2026-02-25T11:47:51.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797c27e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7974100660 con 0x7f797410b970 2026-02-25T11:47:51.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797a7f4640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f797410f2b0 0x7f79740ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:51.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797a7f4640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f797410f2b0 0x7f79740ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59528/0 (socket says 10.20.193.28:59528) 2026-02-25T11:47:51.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797a7f4640 1 -- 10.20.193.28:0/1319333814 learned_addr learned my addr 10.20.193.28:0/1319333814 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:51.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f79797f2640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f797410c280 0x7f79740fb380 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:51.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f7979ff3640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f797410b970 0x7f79740fae40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:51.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797a7f4640 1 -- 10.20.193.28:0/1319333814 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f797410b970 msgr2=0x7f79740fae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:51.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797a7f4640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f797410b970 0x7f79740fae40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797a7f4640 1 -- 10.20.193.28:0/1319333814 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f797410c280 msgr2=0x7f79740fb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:51.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797a7f4640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f797410c280 0x7f79740fb380 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797a7f4640 1 -- 10.20.193.28:0/1319333814 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7968009570 con 0x7f797410f2b0 2026-02-25T11:47:51.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797a7f4640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f797410f2b0 0x7f79740ffd60 secure :-1 s=READY pgs=355 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7970006e90 tx=0x7f797000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:51.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f7962ffd640 1 -- 10.20.193.28:0/1319333814 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7970005500 con 0x7f797410f2b0 2026-02-25T11:47:51.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f7962ffd640 1 -- 10.20.193.28:0/1319333814 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7970005df0 con 0x7f797410f2b0 2026-02-25T11:47:51.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f797c27e640 1 -- 10.20.193.28:0/1319333814 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7974100940 con 0x7f797410f2b0 2026-02-25T11:47:51.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.110+0000 7f7962ffd640 1 -- 10.20.193.28:0/1319333814 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7970003a00 con 0x7f797410f2b0 2026-02-25T11:47:51.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.111+0000 7f797c27e640 1 -- 10.20.193.28:0/1319333814 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7974100fe0 con 0x7f797410f2b0 2026-02-25T11:47:51.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.111+0000 7f797c27e640 1 -- 10.20.193.28:0/1319333814 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797410c750 con 0x7f797410f2b0 2026-02-25T11:47:51.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.113+0000 7f7962ffd640 1 -- 10.20.193.28:0/1319333814 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f7970003b60 con 0x7f797410f2b0 2026-02-25T11:47:51.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.115+0000 7f7962ffd640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f79500b5770 0x7f79500d5c20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:51.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.115+0000 7f7962ffd640 1 -- 10.20.193.28:0/1319333814 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f79700e76e0 con 0x7f797410f2b0 2026-02-25T11:47:51.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.116+0000 7f7979ff3640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f79500b5770 0x7f79500d5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:51.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.116+0000 7f7979ff3640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f79500b5770 0x7f79500d5c20 secure :-1 s=READY pgs=251 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7964006fd0 tx=0x7f7964008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:51.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.116+0000 7f7962ffd640 1 -- 10.20.193.28:0/1319333814 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f79700aa030 con 0x7f797410f2b0 2026-02-25T11:47:51.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.259+0000 7f797c27e640 1 -- 10.20.193.28:0/1319333814 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7974100b30 con 0x7f797410f2b0 2026-02-25T11:47:51.262 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:51 trial028 ceph-mon[39224]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:51.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.259+0000 7f7962ffd640 1 -- 10.20.193.28:0/1319333814 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f79700ad1e0 con 0x7f797410f2b0 2026-02-25T11:47:51.263 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:51.263 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:47:51.264 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:47:51.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.260+0000 7f797c27e640 1 -- 10.20.193.28:0/1319333814 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f79500b5770 msgr2=0x7f79500d5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:51.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.260+0000 7f797c27e640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f79500b5770 0x7f79500d5c20 secure :-1 s=READY pgs=251 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7964006fd0 tx=0x7f7964008040 comp rx=0 tx=0).stop 2026-02-25T11:47:51.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.260+0000 7f797c27e640 1 -- 10.20.193.28:0/1319333814 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f797410f2b0 msgr2=0x7f79740ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:51.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.260+0000 7f797c27e640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f797410f2b0 0x7f79740ffd60 secure :-1 s=READY pgs=355 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7970006e90 tx=0x7f797000edf0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.261+0000 7f797c27e640 1 -- 10.20.193.28:0/1319333814 shutdown_connections 2026-02-25T11:47:51.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.261+0000 7f797c27e640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f79500b5770 0x7f79500d5c20 unknown :-1 s=CLOSED pgs=251 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.261+0000 7f797c27e640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f797410f2b0 0x7f79740ffd60 unknown :-1 s=CLOSED pgs=355 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.261+0000 7f797c27e640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f797410c280 0x7f79740fb380 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.261+0000 7f797c27e640 1 --2- 10.20.193.28:0/1319333814 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f797410b970 0x7f79740fae40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.261+0000 7f797c27e640 1 -- 10.20.193.28:0/1319333814 >> 10.20.193.28:0/1319333814 conn(0x7f79740e94c0 msgr2=0x7f797410d970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:51.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.262+0000 7f797c27e640 1 -- 10.20.193.28:0/1319333814 shutdown_connections 2026-02-25T11:47:51.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.262+0000 7f797c27e640 1 -- 10.20.193.28:0/1319333814 wait complete. 2026-02-25T11:47:51.363 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd dump --format=json 2026-02-25T11:47:51.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:51 trial167 ceph-mon[43392]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:51.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:51 trial119 ceph-mon[42710]: from='client.25271 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:51.546 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:51.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.722+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4281947694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1fac10f840 msgr2=0x7f1fac10fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:51.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.722+0000 7f1fb2a9e640 1 --2- 10.20.193.28:0/4281947694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1fac10f840 0x7f1fac10fc10 secure :-1 s=READY pgs=356 gs=1 cs=0 l=1 c_cookie=6114006798638405547 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1f9c009950 tx=0x7f1f9c02f190 comp rx=0 tx=0).stop 2026-02-25T11:47:51.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.723+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4281947694 shutdown_connections 2026-02-25T11:47:51.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.723+0000 7f1fb2a9e640 1 --2- 10.20.193.28:0/4281947694 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1fac115a30 0x7f1fac176400 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.723+0000 7f1fb2a9e640 1 --2- 10.20.193.28:0/4281947694 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1fac110230 0x7f1fac115280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.723+0000 7f1fb2a9e640 1 --2- 10.20.193.28:0/4281947694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1fac10f840 0x7f1fac10fc10 unknown :-1 s=CLOSED pgs=356 gs=1 cs=0 l=1 c_cookie=6114006798638405547 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.723+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4281947694 >> 10.20.193.28:0/4281947694 conn(0x7f1fac0ed130 msgr2=0x7f1fac10d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:51.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.723+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4281947694 shutdown_connections 2026-02-25T11:47:51.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.723+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4281947694 wait complete. 2026-02-25T11:47:51.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.724+0000 7f1fb2a9e640 1 Processor -- start 2026-02-25T11:47:51.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.724+0000 7f1fb2a9e640 1 -- start start 2026-02-25T11:47:51.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.724+0000 7f1fb2a9e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1fac10f840 0x7f1fac11e190 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:51.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.724+0000 7f1fb2a9e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1fac110230 0x7f1fac11e6d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:51.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.724+0000 7f1fb2a9e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1fac115a30 0x7f1fac123aa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:51.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.724+0000 7f1fb2a9e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac124030 con 0x7f1fac110230 2026-02-25T11:47:51.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.724+0000 7f1fb2a9e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac1241a0 con 0x7f1fac10f840 2026-02-25T11:47:51.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.724+0000 7f1fb2a9e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1fac124360 con 0x7f1fac115a30 2026-02-25T11:47:51.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.724+0000 7f1fa3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1fac110230 0x7f1fac11e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:51.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.724+0000 7f1fa3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1fac110230 0x7f1fac11e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59544/0 (socket says 10.20.193.28:59544) 2026-02-25T11:47:51.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.724+0000 7f1fa3fff640 1 -- 10.20.193.28:0/4179900536 learned_addr learned my addr 10.20.193.28:0/4179900536 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:51.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.724+0000 7f1fb0813640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1fac10f840 0x7f1fac11e190 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:51.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.724+0000 7f1fb1014640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1fac115a30 0x7f1fac123aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:51.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.725+0000 7f1fa3fff640 1 -- 10.20.193.28:0/4179900536 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1fac115a30 msgr2=0x7f1fac123aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:51.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.725+0000 7f1fa3fff640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1fac115a30 0x7f1fac123aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.725+0000 7f1fa3fff640 1 -- 10.20.193.28:0/4179900536 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1fac10f840 msgr2=0x7f1fac11e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:51.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.725+0000 7f1fa3fff640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1fac10f840 0x7f1fac11e190 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.725+0000 7f1fa3fff640 1 -- 10.20.193.28:0/4179900536 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c009570 con 0x7f1fac110230 2026-02-25T11:47:51.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.725+0000 7f1fa3fff640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1fac110230 0x7f1fac11e6d0 secure :-1 s=READY pgs=357 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1f9400c9b0 tx=0x7f1f9400ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:51.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.725+0000 7f1fa1ffb640 1 -- 10.20.193.28:0/4179900536 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1f940044a0 con 0x7f1fac110230 2026-02-25T11:47:51.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.725+0000 7f1fa1ffb640 1 -- 10.20.193.28:0/4179900536 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1f94007d80 con 0x7f1fac110230 2026-02-25T11:47:51.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.725+0000 7f1fa1ffb640 1 -- 10.20.193.28:0/4179900536 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1f9400f6d0 con 0x7f1fac110230 2026-02-25T11:47:51.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.725+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4179900536 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fac124640 con 0x7f1fac110230 2026-02-25T11:47:51.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.725+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4179900536 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fac124920 con 0x7f1fac110230 2026-02-25T11:47:51.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.726+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4179900536 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f74005300 con 0x7f1fac110230 2026-02-25T11:47:51.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.728+0000 7f1fa1ffb640 1 -- 10.20.193.28:0/4179900536 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f1f940078c0 con 0x7f1fac110230 2026-02-25T11:47:51.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.730+0000 7f1fa1ffb640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1f880b5770 0x7f1f880d5c20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:51.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.730+0000 7f1fa1ffb640 1 -- 10.20.193.28:0/4179900536 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f1f940e6fa0 con 0x7f1fac110230 2026-02-25T11:47:51.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.730+0000 7f1fb0813640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1f880b5770 0x7f1f880d5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:51.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.730+0000 7f1fa1ffb640 1 -- 10.20.193.28:0/4179900536 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f1f940e7420 con 0x7f1fac110230 2026-02-25T11:47:51.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.730+0000 7f1fb0813640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1f880b5770 0x7f1f880d5c20 secure :-1 s=READY pgs=252 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1f9c0023d0 tx=0x7f1f9c031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:51.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.845+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4179900536 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f74005880 con 0x7f1fac110230 2026-02-25T11:47:51.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.846+0000 7f1fa1ffb640 1 -- 10.20.193.28:0/4179900536 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22340 (secure 0 0 0) 0x7f1f940a98f0 con 0x7f1fac110230 2026-02-25T11:47:51.850 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:47:51.850 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":86,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","created":"2026-02-25T11:42:51.962844+0000","modified":"2026-02-25T11:47:14.624486+0000","last_up_change":"2026-02-25T11:46:45.156629+0000","last_in_change":"2026-02-25T11:46:31.585002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":38,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T11:44:43.266336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-25T11:47:11.236575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-25T11:47:12.196775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf596df7-d1fb-4d10-8d4c-14a35210bab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6803","nonce":668944205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6805","nonce":668944205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6809","nonce":668944205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6807","nonce":668944205}]},"public_addr":"10.20.193.28:6803/668944205","cluster_addr":"10.20.193.28:6805/668944205","heartbeat_back_addr":"10.20.193.28:6809/668944205","heartbeat_front_addr":"10.20.193.28:6807/668944205","state":["exists","up"]},{"osd":1,"uuid":"185719c0-7c2d-431f-9925-88bbee86a98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6800","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6801","nonce":3486566242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6802","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6803","nonce":3486566242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6806","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6807","nonce":3486566242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6804","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6805","nonce":3486566242}]},"public_addr":"10.20.193.119:6801/3486566242","cluster_addr":"10.20.193.119:6803/3486566242","heartbeat_back_addr":"10.20.193.119:6807/3486566242","heartbeat_front_addr":"10.20.193.119:6805/3486566242","state":["exists","up"]},{"osd":2,"uuid":"4de13b90-1c3c-4614-9fef-2d7315626433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6800","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6801","nonce":484781559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6802","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6803","nonce":484781559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6806","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6807","nonce":484781559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6804","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6805","nonce":484781559}]},"public_addr":"10.20.193.167:6801/484781559","cluster_addr":"10.20.193.167:6803/484781559","heartbeat_back_addr":"10.20.193.167:6807/484781559","heartbeat_front_addr":"10.20.193.167:6805/484781559","state":["exists","up"]},{"osd":3,"uuid":"45d9ea8e-5a77-4738-96da-ada98515fc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6811","nonce":2452541436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6813","nonce":2452541436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6817","nonce":2452541436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6815","nonce":2452541436}]},"public_addr":"10.20.193.28:6811/2452541436","cluster_addr":"10.20.193.28:6813/2452541436","heartbeat_back_addr":"10.20.193.28:6817/2452541436","heartbeat_front_addr":"10.20.193.28:6815/2452541436","state":["exists","up"]},{"osd":4,"uuid":"c9c3f512-665b-44b2-a6f0-f3356a4b661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6808","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6809","nonce":135971996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6810","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6811","nonce":135971996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6814","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6815","nonce":135971996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6812","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6813","nonce":135971996}]},"public_addr":"10.20.193.119:6809/135971996","cluster_addr":"10.20.193.119:6811/135971996","heartbeat_back_addr":"10.20.193.119:6815/135971996","heartbeat_front_addr":"10.20.193.119:6813/135971996","state":["exists","up"]},{"osd":5,"uuid":"777c3819-5c90-4b2b-a545-c024884c2eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6808","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6809","nonce":4203420884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6810","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6811","nonce":4203420884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6814","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6815","nonce":4203420884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6812","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6813","nonce":4203420884}]},"public_addr":"10.20.193.167:6809/4203420884","cluster_addr":"10.20.193.167:6811/4203420884","heartbeat_back_addr":"10.20.193.167:6815/4203420884","heartbeat_front_addr":"10.20.193.167:6813/4203420884","state":["exists","up"]},{"osd":6,"uuid":"bbd9de64-b9fe-499e-b451-c16828be2c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6819","nonce":621484999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6821","nonce":621484999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6825","nonce":621484999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6823","nonce":621484999}]},"public_addr":"10.20.193.28:6819/621484999","cluster_addr":"10.20.193.28:6821/621484999","heartbeat_back_addr":"10.20.193.28:6825/621484999","heartbeat_front_addr":"10.20.193.28:6823/621484999","state":["exists","up"]},{"osd":7,"uuid":"546d9f28-597b-454c-8456-fcc2d46c69f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6816","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6817","nonce":2724541747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6818","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6819","nonce":2724541747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6822","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6823","nonce":2724541747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6820","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6821","nonce":2724541747}]},"public_addr":"10.20.193.119:6817/2724541747","cluster_addr":"10.20.193.119:6819/2724541747","heartbeat_back_addr":"10.20.193.119:6823/2724541747","heartbeat_front_addr":"10.20.193.119:6821/2724541747","state":["exists","up"]},{"osd":8,"uuid":"a1a7cea6-3586-4bb2-aa4c-bf6f7799a501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6816","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6817","nonce":1328734317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6818","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6819","nonce":1328734317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6822","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6823","nonce":1328734317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6820","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6821","nonce":1328734317}]},"public_addr":"10.20.193.167:6817/1328734317","cluster_addr":"10.20.193.167:6819/1328734317","heartbeat_back_addr":"10.20.193.167:6823/1328734317","heartbeat_front_addr":"10.20.193.167:6821/1328734317","state":["exists","up"]},{"osd":9,"uuid":"03fef134-6946-43d5-9c26-dc0e59fffe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6827","nonce":354500625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6829","nonce":354500625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6833","nonce":354500625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6831","nonce":354500625}]},"public_addr":"10.20.193.28:6827/354500625","cluster_addr":"10.20.193.28:6829/354500625","heartbeat_back_addr":"10.20.193.28:6833/354500625","heartbeat_front_addr":"10.20.193.28:6831/354500625","state":["exists","up"]},{"osd":10,"uuid":"b46c29ef-285c-4184-be28-d99b76bc0e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6824","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6825","nonce":2996581116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6826","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6827","nonce":2996581116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6830","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6831","nonce":2996581116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6828","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6829","nonce":2996581116}]},"public_addr":"10.20.193.119:6825/2996581116","cluster_addr":"10.20.193.119:6827/2996581116","heartbeat_back_addr":"10.20.193.119:6831/2996581116","heartbeat_front_addr":"10.20.193.119:6829/2996581116","state":["exists","up"]},{"osd":11,"uuid":"f6696d8c-5de7-4fd9-b693-524f204e4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6824","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6825","nonce":2182906700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6826","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6827","nonce":2182906700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6830","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6831","nonce":2182906700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6828","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6829","nonce":2182906700}]},"public_addr":"10.20.193.167:6825/2182906700","cluster_addr":"10.20.193.167:6827/2182906700","heartbeat_back_addr":"10.20.193.167:6831/2182906700","heartbeat_front_addr":"10.20.193.167:6829/2182906700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:13.410235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:27.254773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:40.956615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:54.529866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:08.306772+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:21.899498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:35.500407+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:49.094965+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:02.790138+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:16.286937+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:29.922722+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:43.396028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.28:0/1467902196":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6800/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/2274270913":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/60087422":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:6800/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/1725332841":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/3082630924":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/2391715078":"2026-02-26T11:43:04.102029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T11:47:51.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.847+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4179900536 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1f880b5770 msgr2=0x7f1f880d5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:51.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.847+0000 7f1fb2a9e640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1f880b5770 0x7f1f880d5c20 secure :-1 s=READY pgs=252 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1f9c0023d0 tx=0x7f1f9c031040 comp rx=0 tx=0).stop 2026-02-25T11:47:51.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.847+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4179900536 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1fac110230 msgr2=0x7f1fac11e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:51.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.847+0000 7f1fb2a9e640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1fac110230 0x7f1fac11e6d0 secure :-1 s=READY pgs=357 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1f9400c9b0 tx=0x7f1f9400ce80 comp rx=0 tx=0).stop 2026-02-25T11:47:51.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.848+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4179900536 shutdown_connections 2026-02-25T11:47:51.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.848+0000 7f1fb2a9e640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1f880b5770 0x7f1f880d5c20 unknown :-1 s=CLOSED pgs=252 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.848+0000 7f1fb2a9e640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1fac115a30 0x7f1fac123aa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.848+0000 7f1fb2a9e640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1fac110230 0x7f1fac11e6d0 unknown :-1 s=CLOSED pgs=357 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.848+0000 7f1fb2a9e640 1 --2- 10.20.193.28:0/4179900536 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1fac10f840 0x7f1fac11e190 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:51.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.848+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4179900536 >> 10.20.193.28:0/4179900536 conn(0x7f1fac0ed130 msgr2=0x7f1fac10d510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:51.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.848+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4179900536 shutdown_connections 2026-02-25T11:47:51.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:51.848+0000 7f1fb2a9e640 1 -- 10.20.193.28:0/4179900536 wait complete. 2026-02-25T11:47:51.948 DEBUG:tasks.fwd_scrub:Yielding 2026-02-25T11:47:51.949 INFO:teuthology.run_tasks:Running task workunit... 2026-02-25T11:47:51.974 INFO:tasks.workunit:Pulling workunits from ref 7c9c9f709b02b16b3e2005e450ecf519f2bb54de 2026-02-25T11:47:51.975 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-25T11:47:51.975 INFO:tasks.workunit:timeout=3h 2026-02-25T11:47:51.975 INFO:tasks.workunit:cleanup=True 2026-02-25T11:47:51.975 DEBUG:teuthology.orchestra.run.trial028:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-25T11:47:51.976 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2026-02-25T11:47:51.976 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:47:51.976 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:47:52.016 INFO:teuthology.orchestra.run.trial028.stdout: File: /home/ubuntu/cephtest/mnt.0 2026-02-25T11:47:52.016 INFO:teuthology.orchestra.run.trial028.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2026-02-25T11:47:52.016 INFO:teuthology.orchestra.run.trial028.stdout:Device: 2bh/43d Inode: 1 Links: 2 2026-02-25T11:47:52.017 INFO:teuthology.orchestra.run.trial028.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T11:47:52.017 INFO:teuthology.orchestra.run.trial028.stdout:Context: system_u:object_r:fusefs_t:s0 2026-02-25T11:47:52.017 INFO:teuthology.orchestra.run.trial028.stdout:Access: 2026-02-25 11:47:19.810671017 +0000 2026-02-25T11:47:52.017 INFO:teuthology.orchestra.run.trial028.stdout:Modify: 2026-02-25 11:47:46.952390002 +0000 2026-02-25T11:47:52.017 INFO:teuthology.orchestra.run.trial028.stdout:Change: 2026-02-25 11:47:47.405963942 +0000 2026-02-25T11:47:52.017 INFO:teuthology.orchestra.run.trial028.stdout: Birth: - 2026-02-25T11:47:52.017 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2026-02-25T11:47:52.018 DEBUG:teuthology.orchestra.run.trial028:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2026-02-25T11:47:52.123 DEBUG:teuthology.orchestra.run.trial028:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone http://github.com/vshankar/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7c9c9f709b02b16b3e2005e450ecf519f2bb54de 2026-02-25T11:47:52.160 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:52.176 INFO:tasks.workunit.client.0.trial028.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-25T11:47:52.287 INFO:tasks.workunit.client.0.trial028.stderr:warning: redirecting to https://github.com/vshankar/ceph.git/ 2026-02-25T11:47:52.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.348+0000 7f466755b640 1 -- 10.20.193.28:0/3115383778 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f466010f810 msgr2=0x7f466010fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.348+0000 7f466755b640 1 --2- 10.20.193.28:0/3115383778 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f466010f810 0x7f466010fbe0 secure :-1 s=READY pgs=358 gs=2 cs=0 l=1 c_cookie=15010030189568889498 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4650009950 tx=0x7f465002f190 comp rx=0 tx=0).stop 2026-02-25T11:47:52.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.348+0000 7f466755b640 1 -- 10.20.193.28:0/3115383778 shutdown_connections 2026-02-25T11:47:52.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.348+0000 7f466755b640 1 --2- 10.20.193.28:0/3115383778 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4660115a00 0x7f46601763d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.348+0000 7f466755b640 1 --2- 10.20.193.28:0/3115383778 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4660110200 0x7f4660115250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.348+0000 7f466755b640 1 --2- 10.20.193.28:0/3115383778 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f466010f810 0x7f466010fbe0 unknown :-1 s=CLOSED pgs=358 gs=2 cs=0 l=1 c_cookie=15010030189568889498 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.348+0000 7f466755b640 1 -- 10.20.193.28:0/3115383778 >> 10.20.193.28:0/3115383778 conn(0x7f46600ed100 msgr2=0x7f466010d510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:52.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.348+0000 7f466755b640 1 -- 10.20.193.28:0/3115383778 shutdown_connections 2026-02-25T11:47:52.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.348+0000 7f466755b640 1 -- 10.20.193.28:0/3115383778 wait complete. 2026-02-25T11:47:52.351 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.349+0000 7f466755b640 1 Processor -- start 2026-02-25T11:47:52.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.350+0000 7f466755b640 1 -- start start 2026-02-25T11:47:52.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.350+0000 7f466755b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f466010f810 0x7f466011e640 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.350+0000 7f466755b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4660110200 0x7f466011eb80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.350+0000 7f466755b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4660115a00 0x7f4660123eb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.350+0000 7f466755b640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f46601244d0 con 0x7f4660115a00 2026-02-25T11:47:52.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.350+0000 7f466755b640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4660124640 con 0x7f4660110200 2026-02-25T11:47:52.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.350+0000 7f466755b640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4660124800 con 0x7f466010f810 2026-02-25T11:47:52.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.350+0000 7f4665ad1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4660115a00 0x7f4660123eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.350+0000 7f4665ad1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4660115a00 0x7f4660123eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59566/0 (socket says 10.20.193.28:59566) 2026-02-25T11:47:52.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.351+0000 7f4665ad1640 1 -- 10.20.193.28:0/2919441513 learned_addr learned my addr 10.20.193.28:0/2919441513 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:52.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.351+0000 7f46652d0640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f466010f810 0x7f466011e640 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.351+0000 7f4664acf640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4660110200 0x7f466011eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.351+0000 7f46652d0640 1 -- 10.20.193.28:0/2919441513 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4660110200 msgr2=0x7f466011eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.351+0000 7f46652d0640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4660110200 0x7f466011eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.351+0000 7f46652d0640 1 -- 10.20.193.28:0/2919441513 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4660115a00 msgr2=0x7f4660123eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.351+0000 7f46652d0640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4660115a00 0x7f4660123eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.351+0000 7f46652d0640 1 -- 10.20.193.28:0/2919441513 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4650009570 con 0x7f466010f810 2026-02-25T11:47:52.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.351+0000 7f46652d0640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f466010f810 0x7f466011e640 secure :-1 s=READY pgs=110 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4650009950 tx=0x7f46500028d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:52.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.352+0000 7f464e7fc640 1 -- 10.20.193.28:0/2919441513 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f465003d070 con 0x7f466010f810 2026-02-25T11:47:52.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.352+0000 7f464e7fc640 1 -- 10.20.193.28:0/2919441513 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f465002fe20 con 0x7f466010f810 2026-02-25T11:47:52.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.352+0000 7f464e7fc640 1 -- 10.20.193.28:0/2919441513 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f46500418a0 con 0x7f466010f810 2026-02-25T11:47:52.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.352+0000 7f466755b640 1 -- 10.20.193.28:0/2919441513 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4660124a80 con 0x7f466010f810 2026-02-25T11:47:52.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.352+0000 7f466755b640 1 -- 10.20.193.28:0/2919441513 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4660165040 con 0x7f466010f810 2026-02-25T11:47:52.357 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.355+0000 7f464e7fc640 1 -- 10.20.193.28:0/2919441513 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f4650038bf0 con 0x7f466010f810 2026-02-25T11:47:52.359 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:52 trial028 ceph-mon[39224]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:52.359 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:52 trial028 ceph-mon[39224]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:52.359 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:52 trial028 ceph-mon[39224]: pgmap v213: 97 pgs: 97 active+clean; 649 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2026-02-25T11:47:52.359 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:52 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1319333814' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:52.359 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:52 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/4179900536' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.357+0000 7f464e7fc640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f463c0b57f0 0x7f463c0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.357+0000 7f464e7fc640 1 -- 10.20.193.28:0/2919441513 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f465010afe0 con 0x7f466010f810 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.357+0000 7f4664acf640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f463c0b57f0 0x7f463c0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.357+0000 7f4669556e00 1 Processor -- start 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.357+0000 7f4664acf640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f463c0b57f0 0x7f463c0d5ca0 secure :-1 s=READY pgs=253 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4660124f40 tx=0x7f465400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.357+0000 7f4669556e00 1 -- start start 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f4669556e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc61717970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f4669556e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc6175c460 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f4669556e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc6177cd80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f4669556e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55bc6162dd80 con 0x55bc6175c9a0 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f4669556e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55bc61629ab0 con 0x55bc61717eb0 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f4669556e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc61623a50 con 0x55bc617175a0 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f46652d0640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc61717970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f4665ad1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc6177cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f4664acf640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc6175c460 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f46652d0640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc61717970 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:55952/0 (socket says 10.20.193.28:55952) 2026-02-25T11:47:52.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f46652d0640 1 -- 10.20.193.28:0/1875439796 learned_addr learned my addr 10.20.193.28:0/1875439796 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:52.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f4664acf640 1 -- 10.20.193.28:0/1875439796 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 msgr2=0x55bc61717970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f4664acf640 1 --2- 10.20.193.28:0/1875439796 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc61717970 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f4664acf640 1 -- 10.20.193.28:0/1875439796 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 msgr2=0x55bc6177cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f4664acf640 1 --2- 10.20.193.28:0/1875439796 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc6177cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.358+0000 7f4664acf640 1 -- 10.20.193.28:0/1875439796 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc615cf120 con 0x55bc61717eb0 2026-02-25T11:47:52.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f4665ad1640 1 --2- 10.20.193.28:0/1875439796 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc6177cd80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:52.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f4664acf640 1 --2- 10.20.193.28:0/1875439796 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc6175c460 secure :-1 s=READY pgs=105 gs=2 cs=0 l=1 c_cookie=3862705641931972787 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f465400b6c0 tx=0x7f4654013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=359b1650db40a0b3 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:52.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f462dce5640 1 -- 10.20.193.28:0/1875439796 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f465401e640 con 0x55bc61717eb0 2026-02-25T11:47:52.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f462dce5640 1 -- 10.20.193.28:0/1875439796 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4654015070 con 0x55bc61717eb0 2026-02-25T11:47:52.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f462dce5640 1 -- 10.20.193.28:0/1875439796 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4654026ca0 con 0x55bc61717eb0 2026-02-25T11:47:52.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f4669556e00 1 -- 10.20.193.28:0/1875439796 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 msgr2=0x55bc6175c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f4669556e00 1 --2- 10.20.193.28:0/1875439796 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc6175c460 secure :-1 s=READY pgs=105 gs=2 cs=0 l=1 c_cookie=3862705641931972787 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f465400b6c0 tx=0x7f4654013000 comp rx=0 tx=0).stop 2026-02-25T11:47:52.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f4669556e00 1 -- 10.20.193.28:0/1875439796 shutdown_connections 2026-02-25T11:47:52.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f4669556e00 1 --2- 10.20.193.28:0/1875439796 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc6177cd80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f4669556e00 1 --2- 10.20.193.28:0/1875439796 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc6175c460 unknown :-1 s=CLOSED pgs=105 gs=2 cs=0 l=1 c_cookie=3862705641931972787 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f4669556e00 1 --2- 10.20.193.28:0/1875439796 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc61717970 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f4669556e00 1 -- 10.20.193.28:0/1875439796 >> 10.20.193.28:0/1875439796 conn(0x55bc616f58f0 msgr2=0x55bc61715cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:52.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f4669556e00 1 -- 10.20.193.28:0/1875439796 shutdown_connections 2026-02-25T11:47:52.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.359+0000 7f4669556e00 1 -- 10.20.193.28:0/1875439796 wait complete. 2026-02-25T11:47:52.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.361+0000 7f4669556e00 1 -- start start 2026-02-25T11:47:52.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.361+0000 7f4669556e00 1 Processor -- start 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.361+0000 7f4669556e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc617743a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.361+0000 7f4669556e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc617748e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.361+0000 7f4669556e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc617798d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.361+0000 7f4669556e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55bc61779e10 con 0x55bc6175c9a0 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.361+0000 7f4669556e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55bc61779f80 con 0x55bc61717eb0 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.361+0000 7f4669556e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc6177a140 con 0x55bc617175a0 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f4665ad1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc617798d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f4665ad1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc617798d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59586/0 (socket says 10.20.193.28:59586) 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f4665ad1640 1 -- 10.20.193.28:0/633637191 learned_addr learned my addr 10.20.193.28:0/633637191 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f4664acf640 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc617748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f46652d0640 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc617743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f4665ad1640 1 -- 10.20.193.28:0/633637191 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 msgr2=0x55bc617743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f4665ad1640 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc617743a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f4665ad1640 1 -- 10.20.193.28:0/633637191 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 msgr2=0x55bc617748e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f4665ad1640 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc617748e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f4665ad1640 1 -- 10.20.193.28:0/633637191 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465400bcf0 con 0x55bc6175c9a0 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f46652d0640 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc617743a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f4665ad1640 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc617798d0 secure :-1 s=READY pgs=359 gs=1 cs=0 l=1 c_cookie=1071647338287054859 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f465c0126c0 tx=0x7f465c008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf41938b17c00b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:52.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f461affd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f461affd640 1 -- 10.20.193.28:0/633637191 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f465c017070 con 0x55bc6175c9a0 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f461affd640 1 -- 10.20.193.28:0/633637191 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f465c005600 con 0x55bc6175c9a0 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f4669556e00 1 -- 10.20.193.28:0/633637191 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f465c0088b0 con 0x55bc6175c9a0 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.362+0000 7f461affd640 1 -- 10.20.193.28:0/633637191 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f465c003ea0 con 0x55bc6175c9a0 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.363+0000 7f4669556e00 10 client.15423 _ll_register_callbacks cb 0x55bc615a4e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.363+0000 7f461affd640 1 -- 10.20.193.28:0/633637191 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f465c003780 con 0x55bc6175c9a0 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.363+0000 7f4669556e00 1 -- 10.20.193.28:0/633637191 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc6137b530 con 0x55bc6175c9a0 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.363+0000 7f461affd640 1 -- 10.20.193.28:0/633637191 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f465c003980 con 0x55bc6175c9a0 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.363+0000 7f4669556e00 10 client.15423 fetch_fsmap learned FSMap version 23 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.363+0000 7f4669556e00 1 -- 10.20.193.28:0/633637191 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55bc615a7810 con 0x55bc6175c9a0 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.363+0000 7f461affd640 1 -- 10.20.193.28:0/633637191 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f465c011930 con 0x55bc6175c9a0 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.363+0000 7f4669556e00 10 client.15423 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.363+0000 7f4669556e00 10 client.15423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.363+0000 7f4669556e00 20 client.15423 populate_metadata read hostname 'trial028' 2026-02-25T11:47:52.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.363+0000 7f4669556e00 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bc6177c910 0x55bc6177cdb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.366 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.363+0000 7f4669556e00 4 client.15423 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:47:52.366 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.363+0000 7f4669556e00 1 -- 10.20.193.28:0/633637191 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc615d0b50 con 0x55bc6177c910 2026-02-25T11:47:52.366 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.364+0000 7f46652d0640 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bc6177c910 0x55bc6177cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.364+0000 7f46652d0640 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bc6177c910 0x55bc6177cdb0 crc :-1 s=READY pgs=20 gs=4 cs=0 l=0 c_cookie=5339679570932138439 s_cookie=7876124905790159224 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1a5a4767d201c7 server_cookie=6d4d9ddf503ab978 in_seq=0 out_seq=0 2026-02-25T11:47:52.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.364+0000 7f461affd640 10 client.15423 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:52.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.365+0000 7f461affd640 1 -- 10.20.193.28:0/633637191 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55bc615d0b50 con 0x55bc6177c910 2026-02-25T11:47:52.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.365+0000 7f461affd640 10 client.15423 handle_command_reply: tid=0 2026-02-25T11:47:52.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.365+0000 7f4669556e00 1 client.15423 shutdown 2026-02-25T11:47:52.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.365+0000 7f4669556e00 1 -- 10.20.193.28:0/633637191 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 msgr2=0x55bc617798d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.365+0000 7f4669556e00 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc617798d0 secure :-1 s=READY pgs=359 gs=1 cs=0 l=1 c_cookie=1071647338287054859 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f465c0126c0 tx=0x7f465c008f90 comp rx=0 tx=0).stop 2026-02-25T11:47:52.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.365+0000 7f4669556e00 1 -- 10.20.193.28:0/633637191 shutdown_connections 2026-02-25T11:47:52.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.365+0000 7f4669556e00 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bc6177c910 0x55bc6177cdb0 crc :-1 s=READY pgs=20 gs=4 cs=0 l=0 c_cookie=5339679570932138439 s_cookie=7876124905790159224 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.365+0000 7f4669556e00 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc617798d0 unknown :-1 s=CLOSED pgs=359 gs=1 cs=0 l=1 c_cookie=1071647338287054859 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.365+0000 7f4669556e00 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc617748e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.365+0000 7f4669556e00 1 --2- 10.20.193.28:0/633637191 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc617743a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.365+0000 7f461affd640 0 client.15423 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:52.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.365+0000 7f4669556e00 1 -- 10.20.193.28:0/633637191 >> 10.20.193.28:0/633637191 conn(0x55bc616f58f0 msgr2=0x55bc61715cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:52.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.366+0000 7f4669556e00 1 -- 10.20.193.28:0/633637191 shutdown_connections 2026-02-25T11:47:52.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.366+0000 7f4669556e00 1 -- 10.20.193.28:0/633637191 wait complete. 2026-02-25T11:47:52.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.366+0000 7f4669556e00 20 client.15423 trim_cache size 0 max 16384 2026-02-25T11:47:52.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.375+0000 7f4669556e00 1 Processor -- start 2026-02-25T11:47:52.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.375+0000 7f4669556e00 1 -- start start 2026-02-25T11:47:52.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4669556e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc61771d60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4669556e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc617565f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4669556e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc61756b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4669556e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55bc61621680 con 0x55bc6175c9a0 2026-02-25T11:47:52.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4669556e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55bc612e1790 con 0x55bc61717eb0 2026-02-25T11:47:52.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4669556e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc6168b020 con 0x55bc617175a0 2026-02-25T11:47:52.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4665ad1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc61756b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4664acf640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc617565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f46652d0640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc61771d60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4665ad1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc61756b30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59592/0 (socket says 10.20.193.28:59592) 2026-02-25T11:47:52.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4665ad1640 1 -- 10.20.193.28:0/605238843 learned_addr learned my addr 10.20.193.28:0/605238843 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:52.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4664acf640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc617565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:44940/0 (socket says 10.20.193.28:44940) 2026-02-25T11:47:52.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4665ad1640 1 -- 10.20.193.28:0/605238843 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 msgr2=0x55bc61771d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4665ad1640 1 --2- 10.20.193.28:0/605238843 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc61771d60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4665ad1640 1 -- 10.20.193.28:0/605238843 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 msgr2=0x55bc617565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4665ad1640 1 --2- 10.20.193.28:0/605238843 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc617565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4665ad1640 1 -- 10.20.193.28:0/605238843 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc615b94f0 con 0x55bc6175c9a0 2026-02-25T11:47:52.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f4664acf640 1 --2- 10.20.193.28:0/605238843 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc617565f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:52.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.376+0000 7f46652d0640 1 --2- 10.20.193.28:0/605238843 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc61771d60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:52.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.377+0000 7f4665ad1640 1 --2- 10.20.193.28:0/605238843 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc61756b30 secure :-1 s=READY pgs=360 gs=3 cs=0 l=1 c_cookie=4106409682082036512 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f465c0273f0 tx=0x7f465c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fce5d962b97b20 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:52.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.377+0000 7f462d4e4640 1 -- 10.20.193.28:0/605238843 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f465c017040 con 0x55bc6175c9a0 2026-02-25T11:47:52.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.377+0000 7f462d4e4640 1 -- 10.20.193.28:0/605238843 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55bc6137b530 con 0x55bc6175c9a0 2026-02-25T11:47:52.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.377+0000 7f462d4e4640 1 -- 10.20.193.28:0/605238843 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f465c011930 con 0x55bc6175c9a0 2026-02-25T11:47:52.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.377+0000 7f4669556e00 1 -- 10.20.193.28:0/605238843 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 msgr2=0x55bc61756b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.377+0000 7f4669556e00 1 --2- 10.20.193.28:0/605238843 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc61756b30 secure :-1 s=READY pgs=360 gs=3 cs=0 l=1 c_cookie=4106409682082036512 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f465c0273f0 tx=0x7f465c020000 comp rx=0 tx=0).stop 2026-02-25T11:47:52.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.377+0000 7f4669556e00 1 -- 10.20.193.28:0/605238843 shutdown_connections 2026-02-25T11:47:52.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.377+0000 7f4669556e00 1 --2- 10.20.193.28:0/605238843 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc6175c9a0 0x55bc61756b30 unknown :-1 s=CLOSED pgs=360 gs=3 cs=0 l=1 c_cookie=4106409682082036512 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.377+0000 7f4669556e00 1 --2- 10.20.193.28:0/605238843 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc61717eb0 0x55bc617565f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.377+0000 7f4669556e00 1 --2- 10.20.193.28:0/605238843 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc61771d60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.377+0000 7f4669556e00 1 -- 10.20.193.28:0/605238843 >> 10.20.193.28:0/605238843 conn(0x55bc616f58f0 msgr2=0x55bc61770640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:52.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.377+0000 7f4669556e00 1 -- 10.20.193.28:0/605238843 shutdown_connections 2026-02-25T11:47:52.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.377+0000 7f4669556e00 1 -- 10.20.193.28:0/605238843 wait complete. 2026-02-25T11:47:52.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.378+0000 7f4669556e00 1 -- start start 2026-02-25T11:47:52.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.378+0000 7f4669556e00 1 Processor -- start 2026-02-25T11:47:52.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.378+0000 7f4669556e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc616fb620 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.378+0000 7f4669556e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc61717eb0 0x55bc616f7b10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.378+0000 7f4669556e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc6175c9a0 0x55bc616f8050 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.378+0000 7f4669556e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55bc616f2370 con 0x55bc61717eb0 2026-02-25T11:47:52.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.378+0000 7f4669556e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55bc616fbc60 con 0x55bc6175c9a0 2026-02-25T11:47:52.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.378+0000 7f4669556e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc616fbe20 con 0x55bc617175a0 2026-02-25T11:47:52.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f4664acf640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc61717eb0 0x55bc616f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f4664acf640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc61717eb0 0x55bc616f7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59598/0 (socket says 10.20.193.28:59598) 2026-02-25T11:47:52.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f4664acf640 1 -- 10.20.193.28:0/1509999186 learned_addr learned my addr 10.20.193.28:0/1509999186 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:52.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f46652d0640 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc616fb620 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f4665ad1640 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc6175c9a0 0x55bc616f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f4664acf640 1 -- 10.20.193.28:0/1509999186 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 msgr2=0x55bc616fb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f4664acf640 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc616fb620 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f4664acf640 1 -- 10.20.193.28:0/1509999186 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc6175c9a0 msgr2=0x55bc616f8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f4664acf640 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc6175c9a0 0x55bc616f8050 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f4664acf640 1 -- 10.20.193.28:0/1509999186 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc615cf120 con 0x55bc61717eb0 2026-02-25T11:47:52.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f46652d0640 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc616fb620 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:52.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f4665ad1640 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc6175c9a0 0x55bc616f8050 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:52.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f4664acf640 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc61717eb0 0x55bc616f7b10 secure :-1 s=READY pgs=361 gs=1 cs=0 l=1 c_cookie=15776326942410680670 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f465400b6c0 tx=0x7f4654031eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf0c5c47ed0295e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:52.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f461affd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:47:52.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f461affd640 1 -- 10.20.193.28:0/1509999186 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4654038040 con 0x55bc61717eb0 2026-02-25T11:47:52.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f461affd640 1 -- 10.20.193.28:0/1509999186 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f46540054f0 con 0x55bc61717eb0 2026-02-25T11:47:52.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f461affd640 1 -- 10.20.193.28:0/1509999186 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f46540269e0 con 0x55bc61717eb0 2026-02-25T11:47:52.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.379+0000 7f4669556e00 1 -- 10.20.193.28:0/1509999186 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4660165040 con 0x55bc61717eb0 2026-02-25T11:47:52.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f461affd640 1 -- 10.20.193.28:0/1509999186 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7f4654017070 con 0x55bc61717eb0 2026-02-25T11:47:52.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f4669556e00 10 client.15429 _ll_register_callbacks cb 0x55bc615a4e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:52.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f4669556e00 1 -- 10.20.193.28:0/1509999186 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4654015070 con 0x55bc61717eb0 2026-02-25T11:47:52.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f461affd640 1 -- 10.20.193.28:0/1509999186 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4654004c30 con 0x55bc61717eb0 2026-02-25T11:47:52.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f4669556e00 10 client.15429 fetch_fsmap learned FSMap version 23 2026-02-25T11:47:52.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f4669556e00 1 -- 10.20.193.28:0/1509999186 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55bc615a7810 con 0x55bc61717eb0 2026-02-25T11:47:52.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f461affd640 1 -- 10.20.193.28:0/1509999186 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f4654044060 con 0x55bc61717eb0 2026-02-25T11:47:52.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f4669556e00 10 client.15429 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:47:52.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f4669556e00 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:47:52.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f4669556e00 20 client.15429 populate_metadata read hostname 'trial028' 2026-02-25T11:47:52.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f4669556e00 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bc6180b440 0x55bc6182b8f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f4669556e00 4 client.15429 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:47:52.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f4669556e00 1 -- 10.20.193.28:0/1509999186 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bc61484690 con 0x55bc6180b440 2026-02-25T11:47:52.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.380+0000 7f46652d0640 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bc6180b440 0x55bc6182b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.381+0000 7f46652d0640 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bc6180b440 0x55bc6182b8f0 crc :-1 s=READY pgs=21 gs=4 cs=0 l=0 c_cookie=14802166538253786556 s_cookie=13547878295811234652 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd6bdc0a92ca5dbc server_cookie=bc03bb87c20d9b5c in_seq=0 out_seq=0 2026-02-25T11:47:52.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.381+0000 7f461affd640 10 client.15429 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:52.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f461affd640 1 -- 10.20.193.28:0/1509999186 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bc61484690 con 0x55bc6180b440 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f461affd640 10 client.15429 handle_command_reply: tid=0 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f4669556e00 1 client.15429 shutdown 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f4669556e00 1 -- 10.20.193.28:0/1509999186 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc61717eb0 msgr2=0x55bc616f7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f4669556e00 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc61717eb0 0x55bc616f7b10 secure :-1 s=READY pgs=361 gs=1 cs=0 l=1 c_cookie=15776326942410680670 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f465400b6c0 tx=0x7f4654031eb0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f4669556e00 1 -- 10.20.193.28:0/1509999186 shutdown_connections 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f4669556e00 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bc6180b440 0x55bc6182b8f0 crc :-1 s=READY pgs=21 gs=4 cs=0 l=0 c_cookie=14802166538253786556 s_cookie=13547878295811234652 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f4669556e00 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bc6175c9a0 0x55bc616f8050 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f4669556e00 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bc61717eb0 0x55bc616f7b10 unknown :-1 s=CLOSED pgs=361 gs=1 cs=0 l=1 c_cookie=15776326942410680670 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f4669556e00 1 --2- 10.20.193.28:0/1509999186 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bc617175a0 0x55bc616fb620 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f4669556e00 1 -- 10.20.193.28:0/1509999186 >> 10.20.193.28:0/1509999186 conn(0x55bc616f58f0 msgr2=0x55bc61770e30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f461affd640 0 client.15429 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f4669556e00 1 -- 10.20.193.28:0/1509999186 shutdown_connections 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f4669556e00 1 -- 10.20.193.28:0/1509999186 wait complete. 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.382+0000 7f4669556e00 20 client.15429 trim_cache size 0 max 16384 2026-02-25T11:47:52.385 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:47:52.386 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:47:52.386 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "3b1cc8e2-c9db-4e9d-9750-434809f320a2", 2026-02-25T11:47:52.386 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:47:52.386 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:47:52.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.383+0000 7f466755b640 1 -- 10.20.193.28:0/2919441513 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f463c0b57f0 msgr2=0x7f463c0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.383+0000 7f466755b640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f463c0b57f0 0x7f463c0d5ca0 secure :-1 s=READY pgs=253 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4660124f40 tx=0x7f465400b410 comp rx=0 tx=0).stop 2026-02-25T11:47:52.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.383+0000 7f466755b640 1 -- 10.20.193.28:0/2919441513 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f466010f810 msgr2=0x7f466011e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.383+0000 7f466755b640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f466010f810 0x7f466011e640 secure :-1 s=READY pgs=110 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4650009950 tx=0x7f46500028d0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.383+0000 7f466755b640 1 -- 10.20.193.28:0/2919441513 shutdown_connections 2026-02-25T11:47:52.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.383+0000 7f466755b640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f463c0b57f0 0x7f463c0d5ca0 unknown :-1 s=CLOSED pgs=253 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.383+0000 7f466755b640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4660115a00 0x7f4660123eb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.383+0000 7f466755b640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4660110200 0x7f466011eb80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.383+0000 7f466755b640 1 --2- 10.20.193.28:0/2919441513 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f466010f810 0x7f466011e640 unknown :-1 s=CLOSED pgs=110 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.383+0000 7f466755b640 1 -- 10.20.193.28:0/2919441513 >> 10.20.193.28:0/2919441513 conn(0x7f46600ed100 msgr2=0x7f466010d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:52.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.383+0000 7f466755b640 1 -- 10.20.193.28:0/2919441513 shutdown_connections 2026-02-25T11:47:52.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.383+0000 7f466755b640 1 -- 10.20.193.28:0/2919441513 wait complete. 2026-02-25T11:47:52.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:52 trial167 ceph-mon[43392]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:52.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:52 trial167 ceph-mon[43392]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:52.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:52 trial167 ceph-mon[43392]: pgmap v213: 97 pgs: 97 active+clean; 649 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2026-02-25T11:47:52.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:52 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1319333814' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:52.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:52 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/4179900536' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:52.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:52 trial119 ceph-mon[42710]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:52.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:52 trial119 ceph-mon[42710]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2026-02-25T11:47:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:52 trial119 ceph-mon[42710]: pgmap v213: 97 pgs: 97 active+clean; 649 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 2.9 KiB/s rd, 682 B/s wr, 6 op/s 2026-02-25T11:47:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:52 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1319333814' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:47:52.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:52 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/4179900536' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:47:52.521 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:47:52.746 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:47:52.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.928+0000 7fdd3a58a640 1 -- 10.20.193.28:0/1986068130 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdd34110130 msgr2=0x7fdd34115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.928+0000 7fdd3a58a640 1 --2- 10.20.193.28:0/1986068130 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdd34110130 0x7fdd34115180 secure :-1 s=READY pgs=111 gs=2 cs=0 l=1 c_cookie=1967908881506963469 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd24009950 tx=0x7fdd2402f190 comp rx=0 tx=0).stop 2026-02-25T11:47:52.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.928+0000 7fdd3a58a640 1 -- 10.20.193.28:0/1986068130 shutdown_connections 2026-02-25T11:47:52.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.928+0000 7fdd3a58a640 1 --2- 10.20.193.28:0/1986068130 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdd34115930 0x7fdd34176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.928+0000 7fdd3a58a640 1 --2- 10.20.193.28:0/1986068130 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdd34110130 0x7fdd34115180 unknown :-1 s=CLOSED pgs=111 gs=2 cs=0 l=1 c_cookie=1967908881506963469 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.928+0000 7fdd3a58a640 1 --2- 10.20.193.28:0/1986068130 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdd3410f740 0x7fdd3410fb10 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.928+0000 7fdd3a58a640 1 -- 10.20.193.28:0/1986068130 >> 10.20.193.28:0/1986068130 conn(0x7fdd340ecff0 msgr2=0x7fdd3410d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:52.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.928+0000 7fdd3a58a640 1 -- 10.20.193.28:0/1986068130 shutdown_connections 2026-02-25T11:47:52.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.928+0000 7fdd3a58a640 1 -- 10.20.193.28:0/1986068130 wait complete. 2026-02-25T11:47:52.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.929+0000 7fdd3a58a640 1 Processor -- start 2026-02-25T11:47:52.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.929+0000 7fdd3a58a640 1 -- start start 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.929+0000 7fdd3a58a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdd3410f740 0x7fdd340f5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.929+0000 7fdd3a58a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdd34110130 0x7fdd34176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd3a58a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdd34115930 0x7fdd340f61d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd3a58a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdd340fb450 con 0x7fdd34115930 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd3a58a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdd340fb5c0 con 0x7fdd3410f740 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd3a58a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd340fb730 con 0x7fdd34110130 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd38b00640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdd34115930 0x7fdd340f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd337fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdd34110130 0x7fdd34176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd38b00640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdd34115930 0x7fdd340f61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59612/0 (socket says 10.20.193.28:59612) 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd38b00640 1 -- 10.20.193.28:0/3680537692 learned_addr learned my addr 10.20.193.28:0/3680537692 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd33fff640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdd3410f740 0x7fdd340f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd38b00640 1 -- 10.20.193.28:0/3680537692 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdd34110130 msgr2=0x7fdd34176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd38b00640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdd34110130 0x7fdd34176200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd38b00640 1 -- 10.20.193.28:0/3680537692 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdd3410f740 msgr2=0x7fdd340f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd38b00640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdd3410f740 0x7fdd340f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd38b00640 1 -- 10.20.193.28:0/3680537692 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd24009570 con 0x7fdd34115930 2026-02-25T11:47:52.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd337fe640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdd34110130 0x7fdd34176200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:47:52.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd38b00640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdd34115930 0x7fdd340f61d0 secure :-1 s=READY pgs=362 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd2800f8d0 tx=0x7fdd2800fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:52.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.930+0000 7fdd317fa640 1 -- 10.20.193.28:0/3680537692 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd2800dce0 con 0x7fdd34115930 2026-02-25T11:47:52.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.931+0000 7fdd317fa640 1 -- 10.20.193.28:0/3680537692 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdd2800de40 con 0x7fdd34115930 2026-02-25T11:47:52.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.931+0000 7fdd317fa640 1 -- 10.20.193.28:0/3680537692 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd280116b0 con 0x7fdd34115930 2026-02-25T11:47:52.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.931+0000 7fdd3a58a640 1 -- 10.20.193.28:0/3680537692 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd340fba10 con 0x7fdd34115930 2026-02-25T11:47:52.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.931+0000 7fdd3a58a640 1 -- 10.20.193.28:0/3680537692 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd340fbe90 con 0x7fdd34115930 2026-02-25T11:47:52.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.934+0000 7fdd317fa640 1 -- 10.20.193.28:0/3680537692 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fdd28020060 con 0x7fdd34115930 2026-02-25T11:47:52.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.935+0000 7fdd317fa640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdd080b57f0 0x7fdd080d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.935+0000 7fdd317fa640 1 -- 10.20.193.28:0/3680537692 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fdd28015070 con 0x7fdd34115930 2026-02-25T11:47:52.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.935+0000 7fdd33fff640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdd080b57f0 0x7fdd080d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.936+0000 7fdd33fff640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdd080b57f0 0x7fdd080d5ca0 secure :-1 s=READY pgs=254 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd20004740 tx=0x7fdd20005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:52.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.936+0000 7fdd3c585e00 1 Processor -- start 2026-02-25T11:47:52.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.936+0000 7fdd3c585e00 1 -- start start 2026-02-25T11:47:52.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.936+0000 7fdd3c585e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179d140 0x55bcb179d510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.936+0000 7fdd3c585e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb179da50 0x55bcb17e21b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd3c585e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb17e26f0 0x55bcb1802ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd3c585e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55bcb16b39b0 con 0x55bcb17e26f0 2026-02-25T11:47:52.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd3c585e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55bcb16af6e0 con 0x55bcb179da50 2026-02-25T11:47:52.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd3c585e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55bcb16a9840 con 0x55bcb179d140 2026-02-25T11:47:52.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd38b00640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb17e26f0 0x55bcb1802ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd33fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179d140 0x55bcb179d510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd337fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb179da50 0x55bcb17e21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd38b00640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb17e26f0 0x55bcb1802ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59624/0 (socket says 10.20.193.28:59624) 2026-02-25T11:47:52.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd38b00640 1 -- 10.20.193.28:0/2418964754 learned_addr learned my addr 10.20.193.28:0/2418964754 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:52.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd38b00640 1 -- 10.20.193.28:0/2418964754 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179d140 msgr2=0x55bcb179d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd38b00640 1 --2- 10.20.193.28:0/2418964754 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179d140 0x55bcb179d510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd38b00640 1 -- 10.20.193.28:0/2418964754 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb179da50 msgr2=0x55bcb17e21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd38b00640 1 --2- 10.20.193.28:0/2418964754 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb179da50 0x55bcb17e21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd38b00640 1 -- 10.20.193.28:0/2418964754 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb1654d80 con 0x55bcb17e26f0 2026-02-25T11:47:52.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd38b00640 1 --2- 10.20.193.28:0/2418964754 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb17e26f0 0x55bcb1802ad0 secure :-1 s=READY pgs=363 gs=3 cs=0 l=1 c_cookie=6930822696474944601 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd280aa610 tx=0x7fdd280adaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602f3a85789aa459 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:52.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd04ce3640 1 -- 10.20.193.28:0/2418964754 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd280162d0 con 0x55bcb17e26f0 2026-02-25T11:47:52.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd04ce3640 1 -- 10.20.193.28:0/2418964754 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdd280aea40 con 0x55bcb17e26f0 2026-02-25T11:47:52.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.937+0000 7fdd04ce3640 1 -- 10.20.193.28:0/2418964754 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd280ade60 con 0x55bcb17e26f0 2026-02-25T11:47:52.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.938+0000 7fdd3c585e00 1 -- 10.20.193.28:0/2418964754 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb17e26f0 msgr2=0x55bcb1802ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.938+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/2418964754 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb17e26f0 0x55bcb1802ad0 secure :-1 s=READY pgs=363 gs=3 cs=0 l=1 c_cookie=6930822696474944601 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd280aa610 tx=0x7fdd280adaf0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.938+0000 7fdd3c585e00 1 -- 10.20.193.28:0/2418964754 shutdown_connections 2026-02-25T11:47:52.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.938+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/2418964754 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb17e26f0 0x55bcb1802ad0 unknown :-1 s=CLOSED pgs=363 gs=3 cs=0 l=1 c_cookie=6930822696474944601 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.938+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/2418964754 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb179da50 0x55bcb17e21b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.938+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/2418964754 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179d140 0x55bcb179d510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.938+0000 7fdd3c585e00 1 -- 10.20.193.28:0/2418964754 >> 10.20.193.28:0/2418964754 conn(0x55bcb16bcd40 msgr2=0x55bcb179ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:52.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.938+0000 7fdd3c585e00 1 -- 10.20.193.28:0/2418964754 shutdown_connections 2026-02-25T11:47:52.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.938+0000 7fdd3c585e00 1 -- 10.20.193.28:0/2418964754 wait complete. 2026-02-25T11:47:52.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.939+0000 7fdd3c585e00 1 -- start start 2026-02-25T11:47:52.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.939+0000 7fdd3c585e00 1 Processor -- start 2026-02-25T11:47:52.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd3c585e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179d140 0x55bcb17fa3a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd3c585e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179da50 0x55bcb17fa8e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd3c585e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 0x55bcb17ff8d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd3c585e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55bcb17ffe10 con 0x55bcb179d140 2026-02-25T11:47:52.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd3c585e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55bcb17fff80 con 0x55bcb17e26f0 2026-02-25T11:47:52.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd3c585e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55bcb1800140 con 0x55bcb179da50 2026-02-25T11:47:52.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd33fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179d140 0x55bcb17fa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd337fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179da50 0x55bcb17fa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd337fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179da50 0x55bcb17fa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56012/0 (socket says 10.20.193.28:56012) 2026-02-25T11:47:52.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd337fe640 1 -- 10.20.193.28:0/3894222466 learned_addr learned my addr 10.20.193.28:0/3894222466 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:52.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd38b00640 1 --2- 10.20.193.28:0/3894222466 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 0x55bcb17ff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd33fff640 1 -- 10.20.193.28:0/3894222466 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179da50 msgr2=0x55bcb17fa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd33fff640 1 --2- 10.20.193.28:0/3894222466 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179da50 0x55bcb17fa8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd33fff640 1 -- 10.20.193.28:0/3894222466 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 msgr2=0x55bcb17ff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd33fff640 1 --2- 10.20.193.28:0/3894222466 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 0x55bcb17ff8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.940+0000 7fdd33fff640 1 -- 10.20.193.28:0/3894222466 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb162d350 con 0x55bcb179d140 2026-02-25T11:47:52.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.941+0000 7fdd33fff640 1 --2- 10.20.193.28:0/3894222466 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179d140 0x55bcb17fa3a0 secure :-1 s=READY pgs=364 gs=3 cs=0 l=1 c_cookie=5885562147230626055 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd20009ea0 tx=0x7fdd20013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51adb797012ab107 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:52.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.941+0000 7fdcf5ffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:47:52.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.941+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/3894222466 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd20022040 con 0x55bcb179d140 2026-02-25T11:47:52.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.941+0000 7fdd3c585e00 1 -- 10.20.193.28:0/3894222466 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd340fbe90 con 0x55bcb179d140 2026-02-25T11:47:52.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.941+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/3894222466 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdd20011e60 con 0x55bcb179d140 2026-02-25T11:47:52.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.941+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/3894222466 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd20027e40 con 0x55bcb179d140 2026-02-25T11:47:52.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.941+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/3894222466 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fdd20018400 con 0x55bcb179d140 2026-02-25T11:47:52.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.941+0000 7fdd3c585e00 10 client.15441 _ll_register_callbacks cb 0x55bcb162a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:52.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.941+0000 7fdd3c585e00 1 -- 10.20.193.28:0/3894222466 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bcb133f6f0 con 0x55bcb179d140 2026-02-25T11:47:52.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.941+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/3894222466 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdd20027430 con 0x55bcb179d140 2026-02-25T11:47:52.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.941+0000 7fdd3c585e00 10 client.15441 fetch_fsmap learned FSMap version 23 2026-02-25T11:47:52.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.941+0000 7fdd3c585e00 1 -- 10.20.193.28:0/3894222466 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55bcb163f150 con 0x55bcb179d140 2026-02-25T11:47:52.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.942+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/3894222466 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fdd200275d0 con 0x55bcb179d140 2026-02-25T11:47:52.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.942+0000 7fdd3c585e00 10 client.15441 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:47:52.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.942+0000 7fdd3c585e00 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:47:52.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.942+0000 7fdd3c585e00 20 client.15441 populate_metadata read hostname 'trial028' 2026-02-25T11:47:52.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.942+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/3894222466 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bcb18028c0 0x55bcb17e0470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.942+0000 7fdd3c585e00 4 client.15441 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:47:52.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.942+0000 7fdd3c585e00 1 -- 10.20.193.28:0/3894222466 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcb167b910 con 0x55bcb18028c0 2026-02-25T11:47:52.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.942+0000 7fdd337fe640 1 --2- 10.20.193.28:0/3894222466 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bcb18028c0 0x55bcb17e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.943+0000 7fdd337fe640 1 --2- 10.20.193.28:0/3894222466 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bcb18028c0 0x55bcb17e0470 crc :-1 s=READY pgs=23 gs=4 cs=0 l=0 c_cookie=10216758400531077492 s_cookie=16063587454341345561 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dc937a273a49974 server_cookie=deed53ae626ff119 in_seq=0 out_seq=0 2026-02-25T11:47:52.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.943+0000 7fdcf5ffb640 10 client.15441 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:52.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.943+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/3894222466 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55bcb167b910 con 0x55bcb18028c0 2026-02-25T11:47:52.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.943+0000 7fdcf5ffb640 10 client.15441 handle_command_reply: tid=0 2026-02-25T11:47:52.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.943+0000 7fdd3c585e00 1 client.15441 shutdown 2026-02-25T11:47:52.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.943+0000 7fdd3c585e00 1 -- 10.20.193.28:0/3894222466 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179d140 msgr2=0x55bcb17fa3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.943+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/3894222466 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179d140 0x55bcb17fa3a0 secure :-1 s=READY pgs=364 gs=3 cs=0 l=1 c_cookie=5885562147230626055 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd20009ea0 tx=0x7fdd20013070 comp rx=0 tx=0).stop 2026-02-25T11:47:52.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.944+0000 7fdd3c585e00 1 -- 10.20.193.28:0/3894222466 shutdown_connections 2026-02-25T11:47:52.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.944+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/3894222466 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bcb18028c0 0x55bcb17e0470 crc :-1 s=READY pgs=23 gs=4 cs=0 l=0 c_cookie=10216758400531077492 s_cookie=16063587454341345561 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.944+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/3894222466 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 0x55bcb17ff8d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.944+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/3894222466 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179da50 0x55bcb17fa8e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.944+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/3894222466 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179d140 0x55bcb17fa3a0 unknown :-1 s=CLOSED pgs=364 gs=3 cs=0 l=1 c_cookie=5885562147230626055 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.944+0000 7fdcf5ffb640 0 client.15441 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:52.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.944+0000 7fdd3c585e00 1 -- 10.20.193.28:0/3894222466 >> 10.20.193.28:0/3894222466 conn(0x55bcb16bcd40 msgr2=0x55bcb179b230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:52.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.944+0000 7fdd3c585e00 1 -- 10.20.193.28:0/3894222466 shutdown_connections 2026-02-25T11:47:52.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.944+0000 7fdd3c585e00 1 -- 10.20.193.28:0/3894222466 wait complete. 2026-02-25T11:47:52.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.944+0000 7fdd3c585e00 20 client.15441 trim_cache size 0 max 16384 2026-02-25T11:47:52.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.953+0000 7fdd3c585e00 1 Processor -- start 2026-02-25T11:47:52.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.953+0000 7fdd3c585e00 1 -- start start 2026-02-25T11:47:52.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.953+0000 7fdd3c585e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179d140 0x55bcb17fee40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.953+0000 7fdd3c585e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179da50 0x55bcb17f7570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.953+0000 7fdd3c585e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 0x55bcb17f7ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.953+0000 7fdd3c585e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55bcb16a7470 con 0x55bcb179d140 2026-02-25T11:47:52.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.953+0000 7fdd3c585e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55bcb1367700 con 0x55bcb17e26f0 2026-02-25T11:47:52.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.953+0000 7fdd3c585e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55bcb17116e0 con 0x55bcb179da50 2026-02-25T11:47:52.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.953+0000 7fdd33fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179d140 0x55bcb17fee40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.953+0000 7fdd337fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179da50 0x55bcb17f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.953+0000 7fdd337fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179da50 0x55bcb17f7570 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56016/0 (socket says 10.20.193.28:56016) 2026-02-25T11:47:52.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.953+0000 7fdd337fe640 1 -- 10.20.193.28:0/1834489768 learned_addr learned my addr 10.20.193.28:0/1834489768 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:52.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.953+0000 7fdd38b00640 1 --2- 10.20.193.28:0/1834489768 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 0x55bcb17f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.954+0000 7fdd337fe640 1 -- 10.20.193.28:0/1834489768 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 msgr2=0x55bcb17f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.954+0000 7fdd337fe640 1 --2- 10.20.193.28:0/1834489768 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 0x55bcb17f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.954+0000 7fdd337fe640 1 -- 10.20.193.28:0/1834489768 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179d140 msgr2=0x55bcb17fee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.954+0000 7fdd337fe640 1 --2- 10.20.193.28:0/1834489768 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179d140 0x55bcb17fee40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.954+0000 7fdd337fe640 1 -- 10.20.193.28:0/1834489768 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb17ec610 con 0x55bcb179da50 2026-02-25T11:47:52.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.954+0000 7fdd337fe640 1 --2- 10.20.193.28:0/1834489768 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179da50 0x55bcb17f7570 secure :-1 s=READY pgs=112 gs=2 cs=0 l=1 c_cookie=12936151903909410562 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd240023d0 tx=0x7fdd240046a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b386719e31037702 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:52.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.954+0000 7fdcf7fff640 1 -- 10.20.193.28:0/1834489768 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd24002d50 con 0x55bcb179da50 2026-02-25T11:47:52.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.954+0000 7fdcf7fff640 1 -- 10.20.193.28:0/1834489768 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdd240040d0 con 0x55bcb179da50 2026-02-25T11:47:52.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.954+0000 7fdcf7fff640 1 -- 10.20.193.28:0/1834489768 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd2404c970 con 0x55bcb179da50 2026-02-25T11:47:52.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.954+0000 7fdd3c585e00 1 -- 10.20.193.28:0/1834489768 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179da50 msgr2=0x55bcb17f7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.954+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/1834489768 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179da50 0x55bcb17f7570 secure :-1 s=READY pgs=112 gs=2 cs=0 l=1 c_cookie=12936151903909410562 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd240023d0 tx=0x7fdd240046a0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.955+0000 7fdd3c585e00 1 -- 10.20.193.28:0/1834489768 shutdown_connections 2026-02-25T11:47:52.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.955+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/1834489768 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 0x55bcb17f7ab0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.955+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/1834489768 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179da50 0x55bcb17f7570 unknown :-1 s=CLOSED pgs=112 gs=2 cs=0 l=1 c_cookie=12936151903909410562 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.955+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/1834489768 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179d140 0x55bcb17fee40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.955+0000 7fdd3c585e00 1 -- 10.20.193.28:0/1834489768 >> 10.20.193.28:0/1834489768 conn(0x55bcb16bcd40 msgr2=0x55bcb17fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:52.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.955+0000 7fdd3c585e00 1 -- 10.20.193.28:0/1834489768 shutdown_connections 2026-02-25T11:47:52.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.955+0000 7fdd3c585e00 1 -- 10.20.193.28:0/1834489768 wait complete. 2026-02-25T11:47:52.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.955+0000 7fdd3c585e00 1 -- start start 2026-02-25T11:47:52.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.955+0000 7fdd3c585e00 1 Processor -- start 2026-02-25T11:47:52.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd3c585e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179d140 0x55bcb177b220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd3c585e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179da50 0x55bcb177b760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd3c585e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 0x55bcb177bca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd3c585e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55bcb1849590 con 0x55bcb179da50 2026-02-25T11:47:52.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd3c585e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55bcb1780270 con 0x55bcb17e26f0 2026-02-25T11:47:52.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd3c585e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55bcb1780430 con 0x55bcb179d140 2026-02-25T11:47:52.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd33fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179d140 0x55bcb177b220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd33fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179d140 0x55bcb177b220 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:56030/0 (socket says 10.20.193.28:56030) 2026-02-25T11:47:52.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd33fff640 1 -- 10.20.193.28:0/4035233288 learned_addr learned my addr 10.20.193.28:0/4035233288 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:47:52.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd337fe640 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179da50 0x55bcb177b760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd33fff640 1 -- 10.20.193.28:0/4035233288 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 msgr2=0x55bcb177bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd38b00640 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 0x55bcb177bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd33fff640 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 0x55bcb177bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd33fff640 1 -- 10.20.193.28:0/4035233288 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179da50 msgr2=0x55bcb177b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:47:52.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd33fff640 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179da50 0x55bcb177b760 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd33fff640 1 -- 10.20.193.28:0/4035233288 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcb1639bd0 con 0x55bcb179d140 2026-02-25T11:47:52.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd337fe640 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179da50 0x55bcb177b760 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:47:52.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.956+0000 7fdd38b00640 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 0x55bcb177bca0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:47:52.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.957+0000 7fdd33fff640 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179d140 0x55bcb177b220 secure :-1 s=READY pgs=113 gs=1 cs=0 l=1 c_cookie=6026034386740854043 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd2001a6e0 tx=0x7fdd2001da50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53a0c658f1fa3d1b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:47:52.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.957+0000 7fdcf5ffb640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:47:52.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.957+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/4035233288 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd20022030 con 0x55bcb179d140 2026-02-25T11:47:52.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.957+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/4035233288 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdd20039070 con 0x55bcb179d140 2026-02-25T11:47:52.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.957+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/4035233288 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd20027430 con 0x55bcb179d140 2026-02-25T11:47:52.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.957+0000 7fdd3c585e00 1 -- 10.20.193.28:0/4035233288 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd34176a30 con 0x55bcb179d140 2026-02-25T11:47:52.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.957+0000 7fdd3c585e00 10 client.25297 _ll_register_callbacks cb 0x55bcb162a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:47:52.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.957+0000 7fdd3c585e00 1 -- 10.20.193.28:0/4035233288 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd280aea40 con 0x55bcb179d140 2026-02-25T11:47:52.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.957+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/4035233288 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(86..86 src has 1..86) ==== 7977+0+0 (secure 0 0 0) 0x7fdd200279f0 con 0x55bcb179d140 2026-02-25T11:47:52.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.957+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/4035233288 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdd20027bb0 con 0x55bcb179d140 2026-02-25T11:47:52.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.957+0000 7fdd3c585e00 10 client.25297 fetch_fsmap learned FSMap version 23 2026-02-25T11:47:52.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.957+0000 7fdd3c585e00 1 -- 10.20.193.28:0/4035233288 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55bcb162d680 con 0x55bcb179d140 2026-02-25T11:47:52.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.958+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/4035233288 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fdd20045030 con 0x55bcb179d140 2026-02-25T11:47:52.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.958+0000 7fdd3c585e00 10 client.25297 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:47:52.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.958+0000 7fdd3c585e00 10 client.25297 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:47:52.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.958+0000 7fdd3c585e00 20 client.25297 populate_metadata read hostname 'trial028' 2026-02-25T11:47:52.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.958+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bcb18905c0 0x55bcb18b0a70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:47:52.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.958+0000 7fdd3c585e00 4 client.25297 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:47:52.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.958+0000 7fdd3c585e00 1 -- 10.20.193.28:0/4035233288 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bcb133cf40 con 0x55bcb18905c0 2026-02-25T11:47:52.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.958+0000 7fdd337fe640 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bcb18905c0 0x55bcb18b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:47:52.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.958+0000 7fdd337fe640 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bcb18905c0 0x55bcb18b0a70 crc :-1 s=READY pgs=24 gs=4 cs=0 l=0 c_cookie=14332636047035902204 s_cookie=17954427015681359689 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e7c08c1688a8fc server_cookie=f92af03e7f5d3f49 in_seq=0 out_seq=0 2026-02-25T11:47:52.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.958+0000 7fdcf5ffb640 10 client.25297 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:52.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdcf5ffb640 1 -- 10.20.193.28:0/4035233288 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x55bcb133cf40 con 0x55bcb18905c0 2026-02-25T11:47:52.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdcf5ffb640 10 client.25297 handle_command_reply: tid=0 2026-02-25T11:47:52.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3c585e00 1 client.25297 shutdown 2026-02-25T11:47:52.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3c585e00 1 -- 10.20.193.28:0/4035233288 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179d140 msgr2=0x55bcb177b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179d140 0x55bcb177b220 secure :-1 s=READY pgs=113 gs=1 cs=0 l=1 c_cookie=6026034386740854043 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd2001a6e0 tx=0x7fdd2001da50 comp rx=0 tx=0).stop 2026-02-25T11:47:52.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3c585e00 1 -- 10.20.193.28:0/4035233288 shutdown_connections 2026-02-25T11:47:52.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bcb18905c0 0x55bcb18b0a70 crc :-1 s=READY pgs=24 gs=4 cs=0 l=0 c_cookie=14332636047035902204 s_cookie=17954427015681359689 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bcb17e26f0 0x55bcb177bca0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bcb179da50 0x55bcb177b760 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3c585e00 1 --2- 10.20.193.28:0/4035233288 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bcb179d140 0x55bcb177b220 unknown :-1 s=CLOSED pgs=113 gs=1 cs=0 l=1 c_cookie=6026034386740854043 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3c585e00 1 -- 10.20.193.28:0/4035233288 >> 10.20.193.28:0/4035233288 conn(0x55bcb16bcd40 msgr2=0x55bcb17fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:52.967 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdcf5ffb640 0 client.25297 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:47:52.968 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3c585e00 1 -- 10.20.193.28:0/4035233288 shutdown_connections 2026-02-25T11:47:52.968 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3c585e00 1 -- 10.20.193.28:0/4035233288 wait complete. 2026-02-25T11:47:52.968 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3c585e00 20 client.25297 trim_cache size 0 max 16384 2026-02-25T11:47:52.968 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:47:52.968 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:47:52.969 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:47:52.969 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:47:52.969 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:47:52.969 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "3b1cc8e2-c9db-4e9d-9750-434809f320a2", 2026-02-25T11:47:52.969 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:47:52.969 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:47:52.969 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:47:52.969 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:47:52.970 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:47:52.970 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:47:52.970 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:47:52.970 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:47:52.970 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:47:52.970 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:47:52.970 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:47:52.971 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:47:52.971 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:47:52.971 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:47:52.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3a58a640 1 -- 10.20.193.28:0/3680537692 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdd080b57f0 msgr2=0x7fdd080d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3a58a640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdd080b57f0 0x7fdd080d5ca0 secure :-1 s=READY pgs=254 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd20004740 tx=0x7fdd20005f90 comp rx=0 tx=0).stop 2026-02-25T11:47:52.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3a58a640 1 -- 10.20.193.28:0/3680537692 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdd34115930 msgr2=0x7fdd340f61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:47:52.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.959+0000 7fdd3a58a640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdd34115930 0x7fdd340f61d0 secure :-1 s=READY pgs=362 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd2800f8d0 tx=0x7fdd2800fda0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.960+0000 7fdd3a58a640 1 -- 10.20.193.28:0/3680537692 shutdown_connections 2026-02-25T11:47:52.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.960+0000 7fdd3a58a640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdd080b57f0 0x7fdd080d5ca0 unknown :-1 s=CLOSED pgs=254 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.960+0000 7fdd3a58a640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdd34115930 0x7fdd340f61d0 unknown :-1 s=CLOSED pgs=362 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.960+0000 7fdd3a58a640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdd34110130 0x7fdd34176200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.960+0000 7fdd3a58a640 1 --2- 10.20.193.28:0/3680537692 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdd3410f740 0x7fdd340f5b40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:47:52.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.960+0000 7fdd3a58a640 1 -- 10.20.193.28:0/3680537692 >> 10.20.193.28:0/3680537692 conn(0x7fdd340ecff0 msgr2=0x7fdd34111f40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:47:52.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.960+0000 7fdd3a58a640 1 -- 10.20.193.28:0/3680537692 shutdown_connections 2026-02-25T11:47:52.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:47:52.960+0000 7fdd3a58a640 1 -- 10.20.193.28:0/3680537692 wait complete. 2026-02-25T11:47:53.095 INFO:tasks.cephfs.filesystem:scrub status for tag:3b1cc8e2-c9db-4e9d-9750-434809f320a2 - {'path': '/', 'tag': '3b1cc8e2-c9db-4e9d-9750-434809f320a2', 'options': 'recursive,force'} 2026-02-25T11:47:53.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:53 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:47:53.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:53 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:47:53.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:53 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:47:53.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:53 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:47:53.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:53 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:47:53.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:53 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:47:53.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:53 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:47:53.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:53 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:47:53.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:53 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:47:53.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:53 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:47:53.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:53 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:47:53.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:53 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:47:54.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:54 trial167 ceph-mon[43392]: pgmap v214: 97 pgs: 97 active+clean; 657 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2026-02-25T11:47:54.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:54 trial119 ceph-mon[42710]: pgmap v214: 97 pgs: 97 active+clean; 657 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2026-02-25T11:47:54.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:54 trial028 ceph-mon[39224]: pgmap v214: 97 pgs: 97 active+clean; 657 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2026-02-25T11:47:56.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:56 trial167 ceph-mon[43392]: pgmap v215: 97 pgs: 97 active+clean; 657 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2026-02-25T11:47:56.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:56 trial119 ceph-mon[42710]: pgmap v215: 97 pgs: 97 active+clean; 657 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2026-02-25T11:47:56.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:56 trial028 ceph-mon[39224]: pgmap v215: 97 pgs: 97 active+clean; 657 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2026-02-25T11:47:57.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:57 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:47:57.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:57 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:47:57.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:57 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:47:58.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:47:58 trial167 ceph-mon[43392]: pgmap v216: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 5.4 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2026-02-25T11:47:58.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:47:58 trial119 ceph-mon[42710]: pgmap v216: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 5.4 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2026-02-25T11:47:58.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:47:58 trial028 ceph-mon[39224]: pgmap v216: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 5.4 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2026-02-25T11:48:00.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:00 trial167 ceph-mon[43392]: pgmap v217: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2026-02-25T11:48:00.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:00 trial119 ceph-mon[42710]: pgmap v217: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2026-02-25T11:48:00.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:00 trial028 ceph-mon[39224]: pgmap v217: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 5.0 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2026-02-25T11:48:02.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:02 trial167 ceph-mon[43392]: pgmap v218: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2026-02-25T11:48:02.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:02 trial119 ceph-mon[42710]: pgmap v218: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2026-02-25T11:48:02.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:02 trial028 ceph-mon[39224]: pgmap v218: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2026-02-25T11:48:04.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:04 trial167 ceph-mon[43392]: pgmap v219: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2026-02-25T11:48:04.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:04 trial119 ceph-mon[42710]: pgmap v219: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2026-02-25T11:48:04.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:04 trial028 ceph-mon[39224]: pgmap v219: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2026-02-25T11:48:06.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:06 trial167 ceph-mon[43392]: pgmap v220: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2026-02-25T11:48:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:06 trial119 ceph-mon[42710]: pgmap v220: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2026-02-25T11:48:06.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:06 trial028 ceph-mon[39224]: pgmap v220: 97 pgs: 97 active+clean; 669 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2026-02-25T11:48:08.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:08 trial167 ceph-mon[43392]: pgmap v221: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2026-02-25T11:48:08.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:08 trial119 ceph-mon[42710]: pgmap v221: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2026-02-25T11:48:08.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:08 trial028 ceph-mon[39224]: pgmap v221: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.9 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2026-02-25T11:48:10.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:10 trial167 ceph-mon[43392]: pgmap v222: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2026-02-25T11:48:10.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:10 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:48:10.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:10 trial119 ceph-mon[42710]: pgmap v222: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2026-02-25T11:48:10.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:10 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:48:10.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:10 trial028 ceph-mon[39224]: pgmap v222: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2026-02-25T11:48:10.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:10 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:48:12.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:11 trial028 ceph-mon[39224]: pgmap v223: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2026-02-25T11:48:12.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:48:12.061 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:48:12.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.062 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:11 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:48:12.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:11 trial167 ceph-mon[43392]: pgmap v223: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2026-02-25T11:48:12.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:48:12.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:48:12.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.251 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:11 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:48:12.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:11 trial119 ceph-mon[42710]: pgmap v223: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2026-02-25T11:48:12.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:48:12.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:48:12.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:48:12.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:11 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:48:14.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:14 trial167 ceph-mon[43392]: pgmap v224: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2026-02-25T11:48:14.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:14 trial119 ceph-mon[42710]: pgmap v224: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2026-02-25T11:48:14.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:14 trial028 ceph-mon[39224]: pgmap v224: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2026-02-25T11:48:16.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:16 trial167 ceph-mon[43392]: pgmap v225: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2026-02-25T11:48:16.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:16 trial119 ceph-mon[42710]: pgmap v225: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2026-02-25T11:48:16.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:16 trial028 ceph-mon[39224]: pgmap v225: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2026-02-25T11:48:17.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]} : dispatch 2026-02-25T11:48:17.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]} : dispatch 2026-02-25T11:48:17.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]} : dispatch 2026-02-25T11:48:17.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]} : dispatch 2026-02-25T11:48:17.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2026-02-25T11:48:17.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2026-02-25T11:48:17.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2026-02-25T11:48:17.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]} : dispatch 2026-02-25T11:48:17.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]} : dispatch 2026-02-25T11:48:17.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]} : dispatch 2026-02-25T11:48:17.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]} : dispatch 2026-02-25T11:48:17.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2026-02-25T11:48:17.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2026-02-25T11:48:17.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2026-02-25T11:48:17.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]} : dispatch 2026-02-25T11:48:17.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]} : dispatch 2026-02-25T11:48:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]} : dispatch 2026-02-25T11:48:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]} : dispatch 2026-02-25T11:48:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2026-02-25T11:48:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2026-02-25T11:48:17.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]} : dispatch 2026-02-25T11:48:18.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:18 trial167 ceph-mon[43392]: pgmap v226: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2026-02-25T11:48:18.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]': finished 2026-02-25T11:48:18.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]': finished 2026-02-25T11:48:18.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2026-02-25T11:48:18.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2026-02-25T11:48:18.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2026-02-25T11:48:18.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2026-02-25T11:48:18.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2026-02-25T11:48:18.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:18 trial167 ceph-mon[43392]: osdmap e87: 12 total, 12 up, 12 in 2026-02-25T11:48:18.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:18 trial119 ceph-mon[42710]: pgmap v226: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2026-02-25T11:48:18.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]': finished 2026-02-25T11:48:18.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]': finished 2026-02-25T11:48:18.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2026-02-25T11:48:18.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2026-02-25T11:48:18.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2026-02-25T11:48:18.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2026-02-25T11:48:18.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2026-02-25T11:48:18.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:18 trial119 ceph-mon[42710]: osdmap e87: 12 total, 12 up, 12 in 2026-02-25T11:48:18.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:18 trial028 ceph-mon[39224]: pgmap v226: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2026-02-25T11:48:18.598 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [7, 11]}]': finished 2026-02-25T11:48:18.598 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]': finished 2026-02-25T11:48:18.598 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2026-02-25T11:48:18.598 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 1]}]': finished 2026-02-25T11:48:18.598 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2026-02-25T11:48:18.598 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2026-02-25T11:48:18.598 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2026-02-25T11:48:18.598 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:18 trial028 ceph-mon[39224]: osdmap e87: 12 total, 12 up, 12 in 2026-02-25T11:48:19.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:19 trial167 ceph-mon[43392]: osdmap e88: 12 total, 12 up, 12 in 2026-02-25T11:48:19.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:19 trial119 ceph-mon[42710]: osdmap e88: 12 total, 12 up, 12 in 2026-02-25T11:48:19.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:19 trial028 ceph-mon[39224]: osdmap e88: 12 total, 12 up, 12 in 2026-02-25T11:48:20.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:20 trial167 ceph-mon[43392]: pgmap v229: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.7 KiB/s rd, 6 op/s; 141 B/s, 0 objects/s recovering 2026-02-25T11:48:20.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:20 trial119 ceph-mon[42710]: pgmap v229: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.7 KiB/s rd, 6 op/s; 141 B/s, 0 objects/s recovering 2026-02-25T11:48:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:20 trial028 ceph-mon[39224]: pgmap v229: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 3.7 KiB/s rd, 6 op/s; 141 B/s, 0 objects/s recovering 2026-02-25T11:48:21.871 INFO:tasks.workunit.client.0.trial028.stderr:Updating files: 79% (9022/11369) Updating files: 80% (9096/11369) Updating files: 81% (9209/11369) Updating files: 82% (9323/11369) Updating files: 83% (9437/11369) Updating files: 84% (9550/11369) Updating files: 85% (9664/11369) Updating files: 86% (9778/11369) Updating files: 87% (9892/11369) Updating files: 88% (10005/11369) Updating files: 89% (10119/11369) Updating files: 90% (10233/11369) Updating files: 91% (10346/11369) Updating files: 92% (10460/11369) Updating files: 93% (10574/11369) Updating files: 94% (10687/11369) Updating files: 95% (10801/11369) Updating files: 96% (10915/11369) Updating files: 97% (11028/11369) Updating files: 98% (11142/11369) Updating files: 99% (11256/11369) Updating files: 100% (11369/11369) Updating files: 100% (11369/11369), done. 2026-02-25T11:48:22.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:22 trial167 ceph-mon[43392]: pgmap v230: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 1.9 KiB/s rd, 4 op/s; 141 B/s, 0 objects/s recovering 2026-02-25T11:48:22.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:22 trial119 ceph-mon[42710]: pgmap v230: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 1.9 KiB/s rd, 4 op/s; 141 B/s, 0 objects/s recovering 2026-02-25T11:48:22.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:22 trial028 ceph-mon[39224]: pgmap v230: 97 pgs: 97 active+clean; 674 KiB data, 330 MiB used, 8.2 TiB / 8.2 TiB avail; 1.9 KiB/s rd, 4 op/s; 141 B/s, 0 objects/s recovering 2026-02-25T11:48:23.096 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:48:23.284 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr:Note: switching to '7c9c9f709b02b16b3e2005e450ecf519f2bb54de'. 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr: 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr:state without impacting any branches by switching back to a branch. 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr: 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr: 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr: git switch -c 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr: 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr:Or undo this operation with: 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr: 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr: git switch - 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr: 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr: 2026-02-25T11:48:23.412 INFO:tasks.workunit.client.0.trial028.stderr:HEAD is now at 7c9c9f709b0 Merge PR #66572 into wip-vshankar-testing-20260225.073815 2026-02-25T11:48:23.423 DEBUG:teuthology.orchestra.run.trial028:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-25T11:48:23.472 INFO:tasks.workunit.client.0.trial028.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-25T11:48:23.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.471+0000 7fa871b08640 1 -- 10.20.193.28:0/2368808206 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa86c10f740 msgr2=0x7fa86c10fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:23.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.471+0000 7fa871b08640 1 --2- 10.20.193.28:0/2368808206 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa86c10f740 0x7fa86c10fb10 secure :-1 s=READY pgs=365 gs=1 cs=0 l=1 c_cookie=5277305065933346926 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa858009950 tx=0x7fa85802f190 comp rx=0 tx=0).stop 2026-02-25T11:48:23.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.471+0000 7fa871b08640 1 -- 10.20.193.28:0/2368808206 shutdown_connections 2026-02-25T11:48:23.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.471+0000 7fa871b08640 1 --2- 10.20.193.28:0/2368808206 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa86c115930 0x7fa86c176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.471+0000 7fa871b08640 1 --2- 10.20.193.28:0/2368808206 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa86c110130 0x7fa86c115180 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.471+0000 7fa871b08640 1 --2- 10.20.193.28:0/2368808206 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa86c10f740 0x7fa86c10fb10 unknown :-1 s=CLOSED pgs=365 gs=1 cs=0 l=1 c_cookie=5277305065933346926 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.471+0000 7fa871b08640 1 -- 10.20.193.28:0/2368808206 >> 10.20.193.28:0/2368808206 conn(0x7fa86c0ecff0 msgr2=0x7fa86c10d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:23.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.471+0000 7fa871b08640 1 -- 10.20.193.28:0/2368808206 shutdown_connections 2026-02-25T11:48:23.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.471+0000 7fa871b08640 1 -- 10.20.193.28:0/2368808206 wait complete. 2026-02-25T11:48:23.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.472+0000 7fa871b08640 1 Processor -- start 2026-02-25T11:48:23.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.472+0000 7fa871b08640 1 -- start start 2026-02-25T11:48:23.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.472+0000 7fa871b08640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa86c10f740 0x7fa86c11e0f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.472+0000 7fa871b08640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa86c110130 0x7fa86c11e630 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa871b08640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa86c115930 0x7fa86c123a00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa871b08640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c124020 con 0x7fa86c115930 2026-02-25T11:48:23.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa871b08640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c124190 con 0x7fa86c10f740 2026-02-25T11:48:23.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa871b08640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa86c124350 con 0x7fa86c110130 2026-02-25T11:48:23.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa86bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa86c115930 0x7fa86c123a00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa86b7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa86c10f740 0x7fa86c11e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa86affd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa86c110130 0x7fa86c11e630 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa86b7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa86c10f740 0x7fa86c11e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41916/0 (socket says 10.20.193.28:41916) 2026-02-25T11:48:23.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa86b7fe640 1 -- 10.20.193.28:0/3009075775 learned_addr learned my addr 10.20.193.28:0/3009075775 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:23.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa86b7fe640 1 -- 10.20.193.28:0/3009075775 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa86c110130 msgr2=0x7fa86c11e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:23.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa86b7fe640 1 --2- 10.20.193.28:0/3009075775 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa86c110130 0x7fa86c11e630 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa86b7fe640 1 -- 10.20.193.28:0/3009075775 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa86c115930 msgr2=0x7fa86c123a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:23.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa86b7fe640 1 --2- 10.20.193.28:0/3009075775 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa86c115930 0x7fa86c123a00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.473+0000 7fa86b7fe640 1 -- 10.20.193.28:0/3009075775 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa858009570 con 0x7fa86c10f740 2026-02-25T11:48:23.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.474+0000 7fa86b7fe640 1 --2- 10.20.193.28:0/3009075775 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa86c10f740 0x7fa86c11e0f0 secure :-1 s=READY pgs=106 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa8580098a0 tx=0x7fa8580028d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:23.478 INFO:tasks.workunit.client.0.trial028.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-25T11:48:23.478 INFO:tasks.workunit.client.0.trial028.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-25T11:48:23.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.474+0000 7fa868ff9640 1 -- 10.20.193.28:0/3009075775 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa85803d070 con 0x7fa86c10f740 2026-02-25T11:48:23.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.474+0000 7fa868ff9640 1 -- 10.20.193.28:0/3009075775 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa858002a70 con 0x7fa86c10f740 2026-02-25T11:48:23.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.474+0000 7fa868ff9640 1 -- 10.20.193.28:0/3009075775 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa858041890 con 0x7fa86c10f740 2026-02-25T11:48:23.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.474+0000 7fa871b08640 1 -- 10.20.193.28:0/3009075775 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86c1245d0 con 0x7fa86c10f740 2026-02-25T11:48:23.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.474+0000 7fa871b08640 1 -- 10.20.193.28:0/3009075775 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86c164b90 con 0x7fa86c10f740 2026-02-25T11:48:23.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.477+0000 7fa868ff9640 1 -- 10.20.193.28:0/3009075775 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fa858041a20 con 0x7fa86c10f740 2026-02-25T11:48:23.481 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.479+0000 7fa868ff9640 1 --2- 10.20.193.28:0/3009075775 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa8400b57f0 0x7fa8400d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.481 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.479+0000 7fa868ff9640 1 -- 10.20.193.28:0/3009075775 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa85810b0e0 con 0x7fa86c10f740 2026-02-25T11:48:23.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.479+0000 7fa86affd640 1 --2- 10.20.193.28:0/3009075775 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa8400b57f0 0x7fa8400d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa873b03e00 1 Processor -- start 2026-02-25T11:48:23.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa873b03e00 1 -- start start 2026-02-25T11:48:23.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa86affd640 1 --2- 10.20.193.28:0/3009075775 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa8400b57f0 0x7fa8400d5ca0 secure :-1 s=READY pgs=255 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa86c124a90 tx=0x7fa85c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:23.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa873b03e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x561145228f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa873b03e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x56114526dbc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa873b03e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56114526e100 0x56114528e4e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa873b03e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5611451369b0 con 0x561145228b50 2026-02-25T11:48:23.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa873b03e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5611451326e0 con 0x561145229460 2026-02-25T11:48:23.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa873b03e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56114512c840 con 0x56114526e100 2026-02-25T11:48:23.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa86bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56114526e100 0x56114528e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa86bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56114526e100 0x56114528e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:51710/0 (socket says 10.20.193.28:51710) 2026-02-25T11:48:23.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa86bfff640 1 -- 10.20.193.28:0/2805752781 learned_addr learned my addr 10.20.193.28:0/2805752781 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:23.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa86affd640 1 --2- 10.20.193.28:0/2805752781 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x56114526dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.480+0000 7fa86b7fe640 1 --2- 10.20.193.28:0/2805752781 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x561145228f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa86b7fe640 1 -- 10.20.193.28:0/2805752781 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56114526e100 msgr2=0x56114528e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:23.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa86b7fe640 1 --2- 10.20.193.28:0/2805752781 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56114526e100 0x56114528e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa86b7fe640 1 -- 10.20.193.28:0/2805752781 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 msgr2=0x56114526dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:23.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa86b7fe640 1 --2- 10.20.193.28:0/2805752781 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x56114526dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa86b7fe640 1 -- 10.20.193.28:0/2805752781 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611450d7d80 con 0x561145228b50 2026-02-25T11:48:23.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa86b7fe640 1 --2- 10.20.193.28:0/2805752781 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x561145228f20 secure :-1 s=READY pgs=366 gs=1 cs=0 l=1 c_cookie=12814817048114754451 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa858032040 tx=0x7fa858004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d760359e7ff393 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:23.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa82ffff640 1 -- 10.20.193.28:0/2805752781 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa858041d00 con 0x561145228b50 2026-02-25T11:48:23.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa82ffff640 1 -- 10.20.193.28:0/2805752781 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa8580d7130 con 0x561145228b50 2026-02-25T11:48:23.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa82ffff640 1 -- 10.20.193.28:0/2805752781 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa8580cf4a0 con 0x561145228b50 2026-02-25T11:48:23.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa873b03e00 1 -- 10.20.193.28:0/2805752781 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 msgr2=0x561145228f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:23.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa873b03e00 1 --2- 10.20.193.28:0/2805752781 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x561145228f20 secure :-1 s=READY pgs=366 gs=1 cs=0 l=1 c_cookie=12814817048114754451 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa858032040 tx=0x7fa858004490 comp rx=0 tx=0).stop 2026-02-25T11:48:23.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa873b03e00 1 -- 10.20.193.28:0/2805752781 shutdown_connections 2026-02-25T11:48:23.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa873b03e00 1 --2- 10.20.193.28:0/2805752781 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56114526e100 0x56114528e4e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa873b03e00 1 --2- 10.20.193.28:0/2805752781 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x56114526dbc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa873b03e00 1 --2- 10.20.193.28:0/2805752781 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x561145228f20 unknown :-1 s=CLOSED pgs=366 gs=1 cs=0 l=1 c_cookie=12814817048114754451 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa873b03e00 1 -- 10.20.193.28:0/2805752781 >> 10.20.193.28:0/2805752781 conn(0x56114513fd40 msgr2=0x561145227430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:23.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa873b03e00 1 -- 10.20.193.28:0/2805752781 shutdown_connections 2026-02-25T11:48:23.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.481+0000 7fa873b03e00 1 -- 10.20.193.28:0/2805752781 wait complete. 2026-02-25T11:48:23.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.483+0000 7fa873b03e00 1 -- start start 2026-02-25T11:48:23.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.483+0000 7fa873b03e00 1 Processor -- start 2026-02-25T11:48:23.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.483+0000 7fa873b03e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561145228b50 0x561145285e20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.483+0000 7fa873b03e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x561145286360 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.483+0000 7fa873b03e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56114526e100 0x56114528b350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.483+0000 7fa873b03e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56114528b890 con 0x56114526e100 2026-02-25T11:48:23.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.483+0000 7fa873b03e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56114528ba00 con 0x561145229460 2026-02-25T11:48:23.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.483+0000 7fa873b03e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56114528bbc0 con 0x561145228b50 2026-02-25T11:48:23.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa86b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561145228b50 0x561145285e20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa86affd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x561145286360 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa86affd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x561145286360 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41934/0 (socket says 10.20.193.28:41934) 2026-02-25T11:48:23.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa86affd640 1 -- 10.20.193.28:0/3812764794 learned_addr learned my addr 10.20.193.28:0/3812764794 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:23.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa86bfff640 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56114526e100 0x56114528b350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa86affd640 1 -- 10.20.193.28:0/3812764794 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561145228b50 msgr2=0x561145285e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:23.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa86affd640 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561145228b50 0x561145285e20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa86affd640 1 -- 10.20.193.28:0/3812764794 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56114526e100 msgr2=0x56114528b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:23.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa86affd640 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56114526e100 0x56114528b350 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa86affd640 1 -- 10.20.193.28:0/3812764794 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611450b0350 con 0x561145229460 2026-02-25T11:48:23.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa86b7fe640 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561145228b50 0x561145285e20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:48:23.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa86bfff640 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56114526e100 0x56114528b350 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:48:23.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa86affd640 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x561145286360 secure :-1 s=READY pgs=107 gs=2 cs=0 l=1 c_cookie=17243550792001031158 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa85c011790 tx=0x7fa85c0047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef4d6615c7a08bf6 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:23.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.484+0000 7fa82d7fa640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:48:23.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.485+0000 7fa82d7fa640 1 -- 10.20.193.28:0/3812764794 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa85c01f9b0 con 0x561145229460 2026-02-25T11:48:23.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.485+0000 7fa82d7fa640 1 -- 10.20.193.28:0/3812764794 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa85c015070 con 0x561145229460 2026-02-25T11:48:23.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.485+0000 7fa873b03e00 1 -- 10.20.193.28:0/3812764794 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86c164b90 con 0x561145229460 2026-02-25T11:48:23.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.485+0000 7fa873b03e00 10 client.25334 _ll_register_callbacks cb 0x5611450ad9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:23.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.485+0000 7fa873b03e00 1 -- 10.20.193.28:0/3812764794 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561144dc26f0 con 0x561145229460 2026-02-25T11:48:23.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.485+0000 7fa82d7fa640 1 -- 10.20.193.28:0/3812764794 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa85c01a3a0 con 0x561145229460 2026-02-25T11:48:23.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.485+0000 7fa82d7fa640 1 -- 10.20.193.28:0/3812764794 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa85c017070 con 0x561145229460 2026-02-25T11:48:23.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.485+0000 7fa82d7fa640 1 -- 10.20.193.28:0/3812764794 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa85c01fb10 con 0x561145229460 2026-02-25T11:48:23.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.485+0000 7fa873b03e00 10 client.25334 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:23.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.485+0000 7fa873b03e00 1 -- 10.20.193.28:0/3812764794 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5611450c2150 con 0x561145229460 2026-02-25T11:48:23.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.485+0000 7fa82d7fa640 1 -- 10.20.193.28:0/3812764794 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa85c02f060 con 0x561145229460 2026-02-25T11:48:23.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.485+0000 7fa873b03e00 10 client.25334 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:23.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.485+0000 7fa873b03e00 10 client.25334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:23.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.486+0000 7fa873b03e00 20 client.25334 populate_metadata read hostname 'trial028' 2026-02-25T11:48:23.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.486+0000 7fa873b03e00 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56114528e410 0x56114526be30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.486+0000 7fa873b03e00 4 client.25334 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:48:23.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.486+0000 7fa873b03e00 1 -- 10.20.193.28:0/3812764794 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611450fe910 con 0x56114528e410 2026-02-25T11:48:23.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.486+0000 7fa86b7fe640 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56114528e410 0x56114526be30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.486+0000 7fa86b7fe640 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56114528e410 0x56114526be30 crc :-1 s=READY pgs=25 gs=4 cs=0 l=0 c_cookie=5367665165472215469 s_cookie=843608764182468108 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a7dc7076535b1ad server_cookie=bb519b38e80960c in_seq=0 out_seq=0 2026-02-25T11:48:23.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.486+0000 7fa82d7fa640 10 client.25334 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:23.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa82d7fa640 1 -- 10.20.193.28:0/3812764794 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5611450fe910 con 0x56114528e410 2026-02-25T11:48:23.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa82d7fa640 10 client.25334 handle_command_reply: tid=0 2026-02-25T11:48:23.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa873b03e00 1 client.25334 shutdown 2026-02-25T11:48:23.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa873b03e00 1 -- 10.20.193.28:0/3812764794 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 msgr2=0x561145286360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:23.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa873b03e00 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x561145286360 secure :-1 s=READY pgs=107 gs=2 cs=0 l=1 c_cookie=17243550792001031158 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa85c011790 tx=0x7fa85c0047e0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa873b03e00 1 -- 10.20.193.28:0/3812764794 shutdown_connections 2026-02-25T11:48:23.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa873b03e00 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56114528e410 0x56114526be30 crc :-1 s=READY pgs=25 gs=4 cs=0 l=0 c_cookie=5367665165472215469 s_cookie=843608764182468108 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa873b03e00 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56114526e100 0x56114528b350 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa873b03e00 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x561145286360 unknown :-1 s=CLOSED pgs=107 gs=2 cs=0 l=1 c_cookie=17243550792001031158 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa873b03e00 1 --2- 10.20.193.28:0/3812764794 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561145228b50 0x561145285e20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa873b03e00 1 -- 10.20.193.28:0/3812764794 >> 10.20.193.28:0/3812764794 conn(0x56114513fd40 msgr2=0x561145227430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:23.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa82d7fa640 0 client.25334 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:23.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa873b03e00 1 -- 10.20.193.28:0/3812764794 shutdown_connections 2026-02-25T11:48:23.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa873b03e00 1 -- 10.20.193.28:0/3812764794 wait complete. 2026-02-25T11:48:23.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.487+0000 7fa873b03e00 20 client.25334 trim_cache size 0 max 16384 2026-02-25T11:48:23.499 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa873b03e00 1 Processor -- start 2026-02-25T11:48:23.499 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa873b03e00 1 -- start start 2026-02-25T11:48:23.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa873b03e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x561145282ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa873b03e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561145229460 0x561145283530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa873b03e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56114526e100 0x561145268d10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa873b03e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56114512a470 con 0x561145228b50 2026-02-25T11:48:23.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa873b03e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561144dea700 con 0x56114526e100 2026-02-25T11:48:23.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa873b03e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5611451946e0 con 0x561145229460 2026-02-25T11:48:23.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa86b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x561145282ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa86b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x561145282ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:43838/0 (socket says 10.20.193.28:43838) 2026-02-25T11:48:23.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa86b7fe640 1 -- 10.20.193.28:0/1196117521 learned_addr learned my addr 10.20.193.28:0/1196117521 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:23.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa86affd640 1 --2- 10.20.193.28:0/1196117521 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561145229460 0x561145283530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa86bfff640 1 --2- 10.20.193.28:0/1196117521 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56114526e100 0x561145268d10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa86affd640 1 -- 10.20.193.28:0/1196117521 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56114526e100 msgr2=0x561145268d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:23.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa86affd640 1 --2- 10.20.193.28:0/1196117521 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56114526e100 0x561145268d10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa86affd640 1 -- 10.20.193.28:0/1196117521 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 msgr2=0x561145282ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:23.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa86affd640 1 --2- 10.20.193.28:0/1196117521 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x561145282ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.497+0000 7fa86affd640 1 -- 10.20.193.28:0/1196117521 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611450bcbd0 con 0x561145229460 2026-02-25T11:48:23.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.498+0000 7fa86affd640 1 --2- 10.20.193.28:0/1196117521 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561145229460 0x561145283530 secure :-1 s=READY pgs=114 gs=2 cs=0 l=1 c_cookie=17903479163023662158 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa85c031d40 tx=0x7fa85c02d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f875ef712ba21c4e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:23.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.498+0000 7fa82f7fe640 1 -- 10.20.193.28:0/1196117521 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa85c026410 con 0x561145229460 2026-02-25T11:48:23.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.498+0000 7fa82f7fe640 1 -- 10.20.193.28:0/1196117521 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa85c015040 con 0x561145229460 2026-02-25T11:48:23.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.498+0000 7fa82f7fe640 1 -- 10.20.193.28:0/1196117521 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa85c01fb30 con 0x561145229460 2026-02-25T11:48:23.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.498+0000 7fa873b03e00 1 -- 10.20.193.28:0/1196117521 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561145229460 msgr2=0x561145283530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:23.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.498+0000 7fa873b03e00 1 --2- 10.20.193.28:0/1196117521 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561145229460 0x561145283530 secure :-1 s=READY pgs=114 gs=2 cs=0 l=1 c_cookie=17903479163023662158 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa85c031d40 tx=0x7fa85c02d000 comp rx=0 tx=0).stop 2026-02-25T11:48:23.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.498+0000 7fa873b03e00 1 -- 10.20.193.28:0/1196117521 shutdown_connections 2026-02-25T11:48:23.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.498+0000 7fa873b03e00 1 --2- 10.20.193.28:0/1196117521 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56114526e100 0x561145268d10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.498+0000 7fa873b03e00 1 --2- 10.20.193.28:0/1196117521 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561145229460 0x561145283530 unknown :-1 s=CLOSED pgs=114 gs=2 cs=0 l=1 c_cookie=17903479163023662158 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.498+0000 7fa873b03e00 1 --2- 10.20.193.28:0/1196117521 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x561145282ff0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.498+0000 7fa873b03e00 1 -- 10.20.193.28:0/1196117521 >> 10.20.193.28:0/1196117521 conn(0x56114513fd40 msgr2=0x5611452778c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:23.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.498+0000 7fa873b03e00 1 -- 10.20.193.28:0/1196117521 shutdown_connections 2026-02-25T11:48:23.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.498+0000 7fa873b03e00 1 -- 10.20.193.28:0/1196117521 wait complete. 2026-02-25T11:48:23.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.499+0000 7fa873b03e00 1 -- start start 2026-02-25T11:48:23.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.499+0000 7fa873b03e00 1 Processor -- start 2026-02-25T11:48:23.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa873b03e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x56114520d140 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa873b03e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x561145208750 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa873b03e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56114526e100 0x561145208e20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa873b03e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5611451326e0 con 0x561145228b50 2026-02-25T11:48:23.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa873b03e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561145209520 con 0x561145229460 2026-02-25T11:48:23.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa873b03e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5611452d3c50 con 0x56114526e100 2026-02-25T11:48:23.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa86b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x56114520d140 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa86b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x56114520d140 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:43844/0 (socket says 10.20.193.28:43844) 2026-02-25T11:48:23.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa86b7fe640 1 -- 10.20.193.28:0/702482232 learned_addr learned my addr 10.20.193.28:0/702482232 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:23.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa86bfff640 1 --2- 10.20.193.28:0/702482232 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56114526e100 0x561145208e20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa86affd640 1 --2- 10.20.193.28:0/702482232 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x561145208750 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa86b7fe640 1 -- 10.20.193.28:0/702482232 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56114526e100 msgr2=0x561145208e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:23.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa86b7fe640 1 --2- 10.20.193.28:0/702482232 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56114526e100 0x561145208e20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa86b7fe640 1 -- 10.20.193.28:0/702482232 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 msgr2=0x561145208750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:23.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa86b7fe640 1 --2- 10.20.193.28:0/702482232 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x561145208750 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa86b7fe640 1 -- 10.20.193.28:0/702482232 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611450b0680 con 0x561145228b50 2026-02-25T11:48:23.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa86b7fe640 1 --2- 10.20.193.28:0/702482232 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x56114520d140 secure :-1 s=READY pgs=367 gs=1 cs=0 l=1 c_cookie=2888036610882753823 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa858002940 tx=0x7fa858005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28145dff01fd551f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:23.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.500+0000 7fa82d7fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:23.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa82d7fa640 1 -- 10.20.193.28:0/702482232 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa8580dc550 con 0x561145228b50 2026-02-25T11:48:23.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa82d7fa640 1 -- 10.20.193.28:0/702482232 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa8580d9710 con 0x561145228b50 2026-02-25T11:48:23.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa873b03e00 1 -- 10.20.193.28:0/702482232 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611450c2150 con 0x561145228b50 2026-02-25T11:48:23.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa82d7fa640 1 -- 10.20.193.28:0/702482232 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa8580d3100 con 0x561145228b50 2026-02-25T11:48:23.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa82d7fa640 1 -- 10.20.193.28:0/702482232 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa8580cce10 con 0x561145228b50 2026-02-25T11:48:23.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa873b03e00 10 client.15465 _ll_register_callbacks cb 0x5611450ad9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:23.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa873b03e00 1 -- 10.20.193.28:0/702482232 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8580d7130 con 0x561145228b50 2026-02-25T11:48:23.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa82d7fa640 1 -- 10.20.193.28:0/702482232 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa8580f00c0 con 0x561145228b50 2026-02-25T11:48:23.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa873b03e00 10 client.15465 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:23.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa873b03e00 1 -- 10.20.193.28:0/702482232 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5611450c23b0 con 0x561145228b50 2026-02-25T11:48:23.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa82d7fa640 1 -- 10.20.193.28:0/702482232 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa8580cf4a0 con 0x561145228b50 2026-02-25T11:48:23.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa873b03e00 10 client.15465 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:23.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa873b03e00 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:23.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa873b03e00 20 client.15465 populate_metadata read hostname 'trial028' 2026-02-25T11:48:23.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa873b03e00 1 --2- 10.20.193.28:0/702482232 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56114531b810 0x56114533bc90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa873b03e00 4 client.15465 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:48:23.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.501+0000 7fa873b03e00 1 -- 10.20.193.28:0/702482232 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561144dbff40 con 0x56114531b810 2026-02-25T11:48:23.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.502+0000 7fa86affd640 1 --2- 10.20.193.28:0/702482232 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56114531b810 0x56114533bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:23.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.502+0000 7fa86affd640 1 --2- 10.20.193.28:0/702482232 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56114531b810 0x56114533bc90 crc :-1 s=READY pgs=26 gs=4 cs=0 l=0 c_cookie=16230838399579498714 s_cookie=165605781648516719 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e13f85879db4a8da server_cookie=24c5994bbe9266f in_seq=0 out_seq=0 2026-02-25T11:48:23.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.502+0000 7fa82d7fa640 10 client.15465 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:23.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.502+0000 7fa82d7fa640 1 -- 10.20.193.28:0/702482232 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x561144dbff40 con 0x56114531b810 2026-02-25T11:48:23.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.502+0000 7fa82d7fa640 10 client.15465 handle_command_reply: tid=0 2026-02-25T11:48:23.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.502+0000 7fa873b03e00 1 client.15465 shutdown 2026-02-25T11:48:23.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.502+0000 7fa873b03e00 1 -- 10.20.193.28:0/702482232 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 msgr2=0x56114520d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:23.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.502+0000 7fa873b03e00 1 --2- 10.20.193.28:0/702482232 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x56114520d140 secure :-1 s=READY pgs=367 gs=1 cs=0 l=1 c_cookie=2888036610882753823 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa858002940 tx=0x7fa858005ab0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa873b03e00 1 -- 10.20.193.28:0/702482232 shutdown_connections 2026-02-25T11:48:23.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa873b03e00 1 --2- 10.20.193.28:0/702482232 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56114531b810 0x56114533bc90 crc :-1 s=READY pgs=26 gs=4 cs=0 l=0 c_cookie=16230838399579498714 s_cookie=165605781648516719 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa873b03e00 1 --2- 10.20.193.28:0/702482232 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56114526e100 0x561145208e20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa873b03e00 1 --2- 10.20.193.28:0/702482232 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561145229460 0x561145208750 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa873b03e00 1 --2- 10.20.193.28:0/702482232 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561145228b50 0x56114520d140 unknown :-1 s=CLOSED pgs=367 gs=1 cs=0 l=1 c_cookie=2888036610882753823 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa82d7fa640 0 client.15465 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:23.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa873b03e00 1 -- 10.20.193.28:0/702482232 >> 10.20.193.28:0/702482232 conn(0x56114513fd40 msgr2=0x56114526c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:23.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa873b03e00 1 -- 10.20.193.28:0/702482232 shutdown_connections 2026-02-25T11:48:23.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa873b03e00 1 -- 10.20.193.28:0/702482232 wait complete. 2026-02-25T11:48:23.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa873b03e00 20 client.15465 trim_cache size 0 max 16384 2026-02-25T11:48:23.514 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:48:23.514 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:48:23.514 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:48:23.514 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:48:23.515 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:48:23.515 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:48:23.515 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:48:23.515 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:48:23.515 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:48:23.515 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:48:23.515 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:48:23.515 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:48:23.515 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:48:23.515 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:48:23.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa871b08640 1 -- 10.20.193.28:0/3009075775 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa8400b57f0 msgr2=0x7fa8400d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:23.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa871b08640 1 --2- 10.20.193.28:0/3009075775 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa8400b57f0 0x7fa8400d5ca0 secure :-1 s=READY pgs=255 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa86c124a90 tx=0x7fa85c00b410 comp rx=0 tx=0).stop 2026-02-25T11:48:23.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa871b08640 1 -- 10.20.193.28:0/3009075775 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa86c10f740 msgr2=0x7fa86c11e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:23.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.503+0000 7fa871b08640 1 --2- 10.20.193.28:0/3009075775 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa86c10f740 0x7fa86c11e0f0 secure :-1 s=READY pgs=106 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa8580098a0 tx=0x7fa8580028d0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.504+0000 7fa871b08640 1 -- 10.20.193.28:0/3009075775 shutdown_connections 2026-02-25T11:48:23.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.504+0000 7fa871b08640 1 --2- 10.20.193.28:0/3009075775 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa8400b57f0 0x7fa8400d5ca0 unknown :-1 s=CLOSED pgs=255 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.504+0000 7fa871b08640 1 --2- 10.20.193.28:0/3009075775 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa86c115930 0x7fa86c123a00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.504+0000 7fa871b08640 1 --2- 10.20.193.28:0/3009075775 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa86c110130 0x7fa86c11e630 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.504+0000 7fa871b08640 1 --2- 10.20.193.28:0/3009075775 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa86c10f740 0x7fa86c11e0f0 unknown :-1 s=CLOSED pgs=106 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.504+0000 7fa871b08640 1 -- 10.20.193.28:0/3009075775 >> 10.20.193.28:0/3009075775 conn(0x7fa86c0ecff0 msgr2=0x7fa86c10d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:23.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.504+0000 7fa871b08640 1 -- 10.20.193.28:0/3009075775 shutdown_connections 2026-02-25T11:48:23.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.504+0000 7fa871b08640 1 -- 10.20.193.28:0/3009075775 wait complete. 2026-02-25T11:48:23.603 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:48:23.604 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:48:23.634 INFO:tasks.workunit.client.0.trial028.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-25T11:48:23.698 INFO:tasks.workunit.client.0.trial028.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-25T11:48:23.751 INFO:tasks.workunit.client.0.trial028.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-25T11:48:23.755 INFO:tasks.workunit.client.0.trial028.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-25T11:48:23.755 INFO:tasks.workunit.client.0.trial028.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-25T11:48:23.808 INFO:tasks.workunit.client.0.trial028.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-25T11:48:23.815 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:48:23.815 DEBUG:teuthology.orchestra.run.trial028:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-25T11:48:23.821 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:48:23.855 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2026-02-25T11:48:23.858 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2026-02-25T11:48:23.858 DEBUG:teuthology.orchestra.run.trial028:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7c9c9f709b02b16b3e2005e450ecf519f2bb54de TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2026-02-25T11:48:23.956 INFO:tasks.workunit.client.0.trial028.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2026-02-25T11:48:23.958 INFO:tasks.workunit.client.0.trial028.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2026-02-25T11:48:23.958 INFO:tasks.workunit.client.0.trial028.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2026-02-25T11:48:23.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.992+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/2385087093 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a6810f840 msgr2=0x7f4a6810fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:23.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.992+0000 7f4a6e5d6640 1 --2- 10.20.193.28:0/2385087093 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a6810f840 0x7f4a6810fc10 secure :-1 s=READY pgs=368 gs=1 cs=0 l=1 c_cookie=14029062685720665800 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4a54009950 tx=0x7f4a5402f190 comp rx=0 tx=0).stop 2026-02-25T11:48:23.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.993+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/2385087093 shutdown_connections 2026-02-25T11:48:23.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.993+0000 7f4a6e5d6640 1 --2- 10.20.193.28:0/2385087093 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4a68115a30 0x7f4a68176400 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.993+0000 7f4a6e5d6640 1 --2- 10.20.193.28:0/2385087093 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4a68110230 0x7f4a68115280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.993+0000 7f4a6e5d6640 1 --2- 10.20.193.28:0/2385087093 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a6810f840 0x7f4a6810fc10 unknown :-1 s=CLOSED pgs=368 gs=1 cs=0 l=1 c_cookie=14029062685720665800 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:23.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.993+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/2385087093 >> 10.20.193.28:0/2385087093 conn(0x7f4a680ed130 msgr2=0x7f4a6810d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:23.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.993+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/2385087093 shutdown_connections 2026-02-25T11:48:23.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.993+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/2385087093 wait complete. 2026-02-25T11:48:23.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.993+0000 7f4a6e5d6640 1 Processor -- start 2026-02-25T11:48:23.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.993+0000 7f4a6e5d6640 1 -- start start 2026-02-25T11:48:23.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a6e5d6640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4a6810f840 0x7f4a680f5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a6e5d6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a68110230 0x7f4a68176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a6e5d6640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4a68115a30 0x7f4a680f6240 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:23.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a6e5d6640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4a680fb450 con 0x7f4a68110230 2026-02-25T11:48:23.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a6e5d6640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4a680fb5c0 con 0x7f4a6810f840 2026-02-25T11:48:23.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a6e5d6640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a680fb780 con 0x7f4a68115a30 2026-02-25T11:48:23.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a677fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a68110230 0x7f4a68176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a677fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a68110230 0x7f4a68176300 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:43852/0 (socket says 10.20.193.28:43852) 2026-02-25T11:48:24.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a677fe640 1 -- 10.20.193.28:0/745303305 learned_addr learned my addr 10.20.193.28:0/745303305 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:24.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a67fff640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4a6810f840 0x7f4a680f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a677fe640 1 -- 10.20.193.28:0/745303305 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4a68115a30 msgr2=0x7f4a680f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:24.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a6cb4c640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4a68115a30 0x7f4a680f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a677fe640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4a68115a30 0x7f4a680f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a677fe640 1 -- 10.20.193.28:0/745303305 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4a6810f840 msgr2=0x7f4a680f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:24.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a677fe640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4a6810f840 0x7f4a680f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a677fe640 1 -- 10.20.193.28:0/745303305 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a54009570 con 0x7f4a68110230 2026-02-25T11:48:24.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a6cb4c640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4a68115a30 0x7f4a680f6240 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:48:24.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a67fff640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4a6810f840 0x7f4a680f5b40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:48:24.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.994+0000 7f4a677fe640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a68110230 0x7f4a68176300 secure :-1 s=READY pgs=369 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4a58009480 tx=0x7f4a5800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:24.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.995+0000 7f4a657fa640 1 -- 10.20.193.28:0/745303305 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4a5800cd40 con 0x7f4a68110230 2026-02-25T11:48:24.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.995+0000 7f4a657fa640 1 -- 10.20.193.28:0/745303305 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4a58004d10 con 0x7f4a68110230 2026-02-25T11:48:24.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.995+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/745303305 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a680fba60 con 0x7f4a68110230 2026-02-25T11:48:24.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.995+0000 7f4a657fa640 1 -- 10.20.193.28:0/745303305 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4a580106b0 con 0x7f4a68110230 2026-02-25T11:48:24.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.995+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/745303305 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a680fbf30 con 0x7f4a68110230 2026-02-25T11:48:24.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.995+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/745303305 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a68110b20 con 0x7f4a68110230 2026-02-25T11:48:24.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.998+0000 7f4a657fa640 1 -- 10.20.193.28:0/745303305 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f4a5801f060 con 0x7f4a68110230 2026-02-25T11:48:24.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.999+0000 7f4a657fa640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4a3c0b57e0 0x7f4a3c0d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.999+0000 7f4a657fa640 1 -- 10.20.193.28:0/745303305 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4a58014070 con 0x7f4a68110230 2026-02-25T11:48:24.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:23.999+0000 7f4a657fa640 1 -- 10.20.193.28:0/745303305 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f4a580e73b0 con 0x7f4a68110230 2026-02-25T11:48:24.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.000+0000 7f4a67fff640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4a3c0b57e0 0x7f4a3c0d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.000+0000 7f4a67fff640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4a3c0b57e0 0x7f4a3c0d5c90 secure :-1 s=READY pgs=256 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4a540023d0 tx=0x7f4a54031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:24.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.142+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/745303305 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a6810fc10 con 0x7f4a68110230 2026-02-25T11:48:24.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.142+0000 7f4a657fa640 1 -- 10.20.193.28:0/745303305 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f4a58076ec0 con 0x7f4a68110230 2026-02-25T11:48:24.145 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:48:24.145 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:48:24.146 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:48:24.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.143+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/745303305 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4a3c0b57e0 msgr2=0x7f4a3c0d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:24.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.143+0000 7f4a6e5d6640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4a3c0b57e0 0x7f4a3c0d5c90 secure :-1 s=READY pgs=256 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4a540023d0 tx=0x7f4a54031040 comp rx=0 tx=0).stop 2026-02-25T11:48:24.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.143+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/745303305 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a68110230 msgr2=0x7f4a68176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:24.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.143+0000 7f4a6e5d6640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a68110230 0x7f4a68176300 secure :-1 s=READY pgs=369 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4a58009480 tx=0x7f4a5800ee60 comp rx=0 tx=0).stop 2026-02-25T11:48:24.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.143+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/745303305 shutdown_connections 2026-02-25T11:48:24.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.143+0000 7f4a6e5d6640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4a3c0b57e0 0x7f4a3c0d5c90 unknown :-1 s=CLOSED pgs=256 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.143+0000 7f4a6e5d6640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4a68115a30 0x7f4a680f6240 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.143+0000 7f4a6e5d6640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4a68110230 0x7f4a68176300 unknown :-1 s=CLOSED pgs=369 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.143+0000 7f4a6e5d6640 1 --2- 10.20.193.28:0/745303305 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4a6810f840 0x7f4a680f5b40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.143+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/745303305 >> 10.20.193.28:0/745303305 conn(0x7f4a680ed130 msgr2=0x7f4a68111c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:24.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.143+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/745303305 shutdown_connections 2026-02-25T11:48:24.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.143+0000 7f4a6e5d6640 1 -- 10.20.193.28:0/745303305 wait complete. 2026-02-25T11:48:24.217 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:24 trial028 ceph-mon[39224]: pgmap v231: 97 pgs: 97 active+clean; 674 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 3.5 KiB/s rd, 6 op/s; 141 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:24.217 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:24 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/745303305' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:48:24.243 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:48:24.428 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:48:24.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:24 trial167 ceph-mon[43392]: pgmap v231: 97 pgs: 97 active+clean; 674 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 3.5 KiB/s rd, 6 op/s; 141 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:24.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:24 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/745303305' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:48:24.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:24 trial119 ceph-mon[42710]: pgmap v231: 97 pgs: 97 active+clean; 674 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 3.5 KiB/s rd, 6 op/s; 141 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:24.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:24 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/745303305' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:48:24.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.598+0000 7fbc04feb640 1 -- 10.20.193.28:0/1279332288 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbc0010bd50 msgr2=0x7fbc00110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:24.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.598+0000 7fbc04feb640 1 --2- 10.20.193.28:0/1279332288 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbc0010bd50 0x7fbc00110be0 secure :-1 s=READY pgs=370 gs=1 cs=0 l=1 c_cookie=7325220245061979178 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbbf0009950 tx=0x7fbbf002f190 comp rx=0 tx=0).stop 2026-02-25T11:48:24.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.598+0000 7fbc04feb640 1 -- 10.20.193.28:0/1279332288 shutdown_connections 2026-02-25T11:48:24.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.598+0000 7fbc04feb640 1 --2- 10.20.193.28:0/1279332288 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbc00111370 0x7fbc00111d50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.598+0000 7fbc04feb640 1 --2- 10.20.193.28:0/1279332288 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbc0010bd50 0x7fbc00110be0 unknown :-1 s=CLOSED pgs=370 gs=1 cs=0 l=1 c_cookie=7325220245061979178 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.598+0000 7fbc04feb640 1 --2- 10.20.193.28:0/1279332288 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbc0010b360 0x7fbc0010b730 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.598+0000 7fbc04feb640 1 -- 10.20.193.28:0/1279332288 >> 10.20.193.28:0/1279332288 conn(0x7fbc000e8e70 msgr2=0x7fbc00109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:24.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.598+0000 7fbc04feb640 1 -- 10.20.193.28:0/1279332288 shutdown_connections 2026-02-25T11:48:24.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.599+0000 7fbc04feb640 1 -- 10.20.193.28:0/1279332288 wait complete. 2026-02-25T11:48:24.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.599+0000 7fbc04feb640 1 Processor -- start 2026-02-25T11:48:24.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.599+0000 7fbc04feb640 1 -- start start 2026-02-25T11:48:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.599+0000 7fbc04feb640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbc0010b360 0x7fbc00123650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbc04feb640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbc0010bd50 0x7fbc00123b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbc04feb640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbc00111370 0x7fbc00128f60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbc04feb640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00129580 con 0x7fbc0010bd50 2026-02-25T11:48:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbc04feb640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001296f0 con 0x7fbc0010b360 2026-02-25T11:48:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbc04feb640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbc001298b0 con 0x7fbc00111370 2026-02-25T11:48:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbbfeffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbc0010bd50 0x7fbc00123b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbbff7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbc0010b360 0x7fbc00123650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbbff7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbc0010b360 0x7fbc00123650 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41988/0 (socket says 10.20.193.28:41988) 2026-02-25T11:48:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbbfeffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbc0010bd50 0x7fbc00123b90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:43872/0 (socket says 10.20.193.28:43872) 2026-02-25T11:48:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbbff7fe640 1 -- 10.20.193.28:0/193863091 learned_addr learned my addr 10.20.193.28:0/193863091 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbbfffff640 1 --2- 10.20.193.28:0/193863091 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbc00111370 0x7fbc00128f60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbbfeffd640 1 -- 10.20.193.28:0/193863091 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbc00111370 msgr2=0x7fbc00128f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/193863091 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbc00111370 0x7fbc00128f60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbbfeffd640 1 -- 10.20.193.28:0/193863091 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbc0010b360 msgr2=0x7fbc00123650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/193863091 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbc0010b360 0x7fbc00123650 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbbfeffd640 1 -- 10.20.193.28:0/193863091 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf0009570 con 0x7fbc0010bd50 2026-02-25T11:48:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.600+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/193863091 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbc0010bd50 0x7fbc00123b90 secure :-1 s=READY pgs=371 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbc00111c70 tx=0x7fbbf00028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.601+0000 7fbbfcff9640 1 -- 10.20.193.28:0/193863091 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbbf003d070 con 0x7fbc0010bd50 2026-02-25T11:48:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.601+0000 7fbbfcff9640 1 -- 10.20.193.28:0/193863091 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fbbf00043b0 con 0x7fbc0010bd50 2026-02-25T11:48:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.601+0000 7fbbfcff9640 1 -- 10.20.193.28:0/193863091 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbbf0041690 con 0x7fbc0010bd50 2026-02-25T11:48:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.601+0000 7fbc04feb640 1 -- 10.20.193.28:0/193863091 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc00129b30 con 0x7fbc0010bd50 2026-02-25T11:48:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.601+0000 7fbc04feb640 1 -- 10.20.193.28:0/193863091 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0016a0f0 con 0x7fbc0010bd50 2026-02-25T11:48:24.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.604+0000 7fbbfcff9640 1 -- 10.20.193.28:0/193863091 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fbbf00386a0 con 0x7fbc0010bd50 2026-02-25T11:48:24.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.605+0000 7fbbfcff9640 1 --2- 10.20.193.28:0/193863091 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbbd40b5870 0x7fbbd40d5d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.605+0000 7fbbfcff9640 1 -- 10.20.193.28:0/193863091 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fbbf010af60 con 0x7fbc0010bd50 2026-02-25T11:48:24.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.605+0000 7fbbff7fe640 1 --2- 10.20.193.28:0/193863091 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbbd40b5870 0x7fbbd40d5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.606+0000 7fbbff7fe640 1 --2- 10.20.193.28:0/193863091 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbbd40b5870 0x7fbbd40d5d20 secure :-1 s=READY pgs=257 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbbec0095b0 tx=0x7fbbec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.607+0000 7fbc0826fe00 1 Processor -- start 2026-02-25T11:48:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.607+0000 7fbc0826fe00 1 -- start start 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.607+0000 7fbc0826fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c2b30 0x55b3af2c2f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.607+0000 7fbc0826fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c3440 0x55b3af307ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.607+0000 7fbc0826fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af3080e0 0x55b3af3284c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.607+0000 7fbc0826fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b3af1d88a0 con 0x55b3af2c3440 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.607+0000 7fbc0826fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b3af1d45d0 con 0x55b3af3080e0 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.607+0000 7fbc0826fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b3af1ce570 con 0x55b3af2c2b30 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbfeffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c3440 0x55b3af307ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbfeffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c3440 0x55b3af307ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:43876/0 (socket says 10.20.193.28:43876) 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbfeffd640 1 -- 10.20.193.28:0/4131497846 learned_addr learned my addr 10.20.193.28:0/4131497846 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbfffff640 1 --2- 10.20.193.28:0/4131497846 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af3080e0 0x55b3af3284c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbff7fe640 1 --2- 10.20.193.28:0/4131497846 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c2b30 0x55b3af2c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbfeffd640 1 -- 10.20.193.28:0/4131497846 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c2b30 msgr2=0x55b3af2c2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/4131497846 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c2b30 0x55b3af2c2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbfeffd640 1 -- 10.20.193.28:0/4131497846 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af3080e0 msgr2=0x55b3af3284c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/4131497846 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af3080e0 0x55b3af3284c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbfeffd640 1 -- 10.20.193.28:0/4131497846 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3af179d10 con 0x55b3af2c3440 2026-02-25T11:48:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/4131497846 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c3440 0x55b3af307ba0 secure :-1 s=READY pgs=372 gs=1 cs=0 l=1 c_cookie=7497373729822073998 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbbf00d1270 tx=0x7fbbf00cd9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680c05af3a63e48e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbc3fff640 1 -- 10.20.193.28:0/4131497846 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbbf00d0a90 con 0x55b3af2c3440 2026-02-25T11:48:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbc3fff640 1 -- 10.20.193.28:0/4131497846 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fbbf002fe20 con 0x55b3af2c3440 2026-02-25T11:48:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.608+0000 7fbbc3fff640 1 -- 10.20.193.28:0/4131497846 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbbf00dbf40 con 0x55b3af2c3440 2026-02-25T11:48:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.609+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4131497846 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c3440 msgr2=0x55b3af307ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.609+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/4131497846 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c3440 0x55b3af307ba0 secure :-1 s=READY pgs=372 gs=1 cs=0 l=1 c_cookie=7497373729822073998 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbbf00d1270 tx=0x7fbbf00cd9b0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.609+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4131497846 shutdown_connections 2026-02-25T11:48:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.609+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/4131497846 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af3080e0 0x55b3af3284c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.609+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/4131497846 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c3440 0x55b3af307ba0 unknown :-1 s=CLOSED pgs=372 gs=1 cs=0 l=1 c_cookie=7497373729822073998 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.609+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/4131497846 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c2b30 0x55b3af2c2f00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.609+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4131497846 >> 10.20.193.28:0/4131497846 conn(0x55b3af1e17a0 msgr2=0x55b3af2c1410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.609+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4131497846 shutdown_connections 2026-02-25T11:48:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.609+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4131497846 wait complete. 2026-02-25T11:48:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.610+0000 7fbc0826fe00 1 -- start start 2026-02-25T11:48:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.610+0000 7fbc0826fe00 1 Processor -- start 2026-02-25T11:48:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbc0826fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c2b30 0x55b3af2dad20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbc0826fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af2c3440 0x55b3af2db260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbc0826fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af3080e0 0x55b3af2e0250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbc0826fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b3af2e0790 con 0x55b3af3080e0 2026-02-25T11:48:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbc0826fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b3af2e0900 con 0x55b3af2c3440 2026-02-25T11:48:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbc0826fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b3af2e0ac0 con 0x55b3af2c2b30 2026-02-25T11:48:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbbff7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c2b30 0x55b3af2dad20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbbff7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c2b30 0x55b3af2dad20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:51794/0 (socket says 10.20.193.28:51794) 2026-02-25T11:48:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbbff7fe640 1 -- 10.20.193.28:0/406832268 learned_addr learned my addr 10.20.193.28:0/406832268 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/406832268 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af2c3440 0x55b3af2db260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbbfffff640 1 --2- 10.20.193.28:0/406832268 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af3080e0 0x55b3af2e0250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbbff7fe640 1 -- 10.20.193.28:0/406832268 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af2c3440 msgr2=0x55b3af2db260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbbff7fe640 1 --2- 10.20.193.28:0/406832268 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af2c3440 0x55b3af2db260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbbff7fe640 1 -- 10.20.193.28:0/406832268 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af3080e0 msgr2=0x55b3af2e0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbbff7fe640 1 --2- 10.20.193.28:0/406832268 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af3080e0 0x55b3af2e0250 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.611+0000 7fbbff7fe640 1 -- 10.20.193.28:0/406832268 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3af1523c0 con 0x55b3af2c2b30 2026-02-25T11:48:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.612+0000 7fbbff7fe640 1 --2- 10.20.193.28:0/406832268 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c2b30 0x55b3af2dad20 secure :-1 s=READY pgs=115 gs=1 cs=0 l=1 c_cookie=10235780499979814164 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbbec009230 tx=0x7fbbec0047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e0ccc221c2ac514 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.612+0000 7fbbc17fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:48:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.612+0000 7fbbc17fa640 1 -- 10.20.193.28:0/406832268 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbbec01c620 con 0x55b3af2c2b30 2026-02-25T11:48:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.612+0000 7fbc0826fe00 1 -- 10.20.193.28:0/406832268 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0016a0f0 con 0x55b3af2c2b30 2026-02-25T11:48:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.612+0000 7fbbc17fa640 1 -- 10.20.193.28:0/406832268 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fbbec01cc40 con 0x55b3af2c2b30 2026-02-25T11:48:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.612+0000 7fbbc17fa640 1 -- 10.20.193.28:0/406832268 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbbec019cc0 con 0x55b3af2c2b30 2026-02-25T11:48:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.612+0000 7fbc0826fe00 10 client.25330 _ll_register_callbacks cb 0x55b3af14fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.612+0000 7fbc0826fe00 1 -- 10.20.193.28:0/406832268 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b3aee646f0 con 0x55b3af2c2b30 2026-02-25T11:48:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.612+0000 7fbbc17fa640 1 -- 10.20.193.28:0/406832268 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fbbec01cdb0 con 0x55b3af2c2b30 2026-02-25T11:48:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.612+0000 7fbbc17fa640 1 -- 10.20.193.28:0/406832268 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbbec019e50 con 0x55b3af2c2b30 2026-02-25T11:48:24.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.612+0000 7fbc0826fe00 10 client.25330 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:24.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.612+0000 7fbc0826fe00 1 -- 10.20.193.28:0/406832268 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55b3af1640e0 con 0x55b3af2c2b30 2026-02-25T11:48:24.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.613+0000 7fbbc17fa640 1 -- 10.20.193.28:0/406832268 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fbbec01c780 con 0x55b3af2c2b30 2026-02-25T11:48:24.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.613+0000 7fbc0826fe00 10 client.25330 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:24.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.613+0000 7fbc0826fe00 10 client.25330 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:48:24.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.613+0000 7fbc0826fe00 20 client.25330 populate_metadata read hostname 'trial028' 2026-02-25T11:48:24.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.613+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/406832268 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55b3af3056c0 0x55b3af305b60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.613+0000 7fbc0826fe00 4 client.25330 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:48:24.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.613+0000 7fbc0826fe00 1 -- 10.20.193.28:0/406832268 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3af1a0fc0 con 0x55b3af3056c0 2026-02-25T11:48:24.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.613+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/406832268 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55b3af3056c0 0x55b3af305b60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.614+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/406832268 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55b3af3056c0 0x55b3af305b60 crc :-1 s=READY pgs=23 gs=4 cs=0 l=0 c_cookie=8540205228418867078 s_cookie=10607808064631681087 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7684e76844d71f86 server_cookie=9336813b71e5303f in_seq=0 out_seq=0 2026-02-25T11:48:24.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.614+0000 7fbbc17fa640 10 client.25330 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:48:24.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbbc17fa640 1 -- 10.20.193.28:0/406832268 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55b3af1a0fc0 con 0x55b3af3056c0 2026-02-25T11:48:24.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbbc17fa640 10 client.25330 handle_command_reply: tid=0 2026-02-25T11:48:24.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbc0826fe00 1 client.25330 shutdown 2026-02-25T11:48:24.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbc0826fe00 1 -- 10.20.193.28:0/406832268 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c2b30 msgr2=0x55b3af2dad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:24.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/406832268 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c2b30 0x55b3af2dad20 secure :-1 s=READY pgs=115 gs=1 cs=0 l=1 c_cookie=10235780499979814164 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbbec009230 tx=0x7fbbec0047e0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbc0826fe00 1 -- 10.20.193.28:0/406832268 shutdown_connections 2026-02-25T11:48:24.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/406832268 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55b3af3056c0 0x55b3af305b60 crc :-1 s=READY pgs=23 gs=4 cs=0 l=0 c_cookie=8540205228418867078 s_cookie=10607808064631681087 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/406832268 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af3080e0 0x55b3af2e0250 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/406832268 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af2c3440 0x55b3af2db260 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/406832268 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c2b30 0x55b3af2dad20 unknown :-1 s=CLOSED pgs=115 gs=1 cs=0 l=1 c_cookie=10235780499979814164 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbbc17fa640 0 client.25330 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:48:24.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbc0826fe00 1 -- 10.20.193.28:0/406832268 >> 10.20.193.28:0/406832268 conn(0x55b3af1e17a0 msgr2=0x55b3af2c1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:24.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbc0826fe00 1 -- 10.20.193.28:0/406832268 shutdown_connections 2026-02-25T11:48:24.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbc0826fe00 1 -- 10.20.193.28:0/406832268 wait complete. 2026-02-25T11:48:24.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.615+0000 7fbc0826fe00 20 client.25330 trim_cache size 0 max 16384 2026-02-25T11:48:24.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.623+0000 7fbc0826fe00 1 Processor -- start 2026-02-25T11:48:24.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.623+0000 7fbc0826fe00 1 -- start start 2026-02-25T11:48:24.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.623+0000 7fbc0826fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 0x55b3af2c5820 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.623+0000 7fbc0826fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c3440 0x55b3af2c1d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.623+0000 7fbc0826fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af3080e0 0x55b3af2c2260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.623+0000 7fbc0826fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b3af1cc1a0 con 0x55b3af2c2b30 2026-02-25T11:48:24.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.623+0000 7fbc0826fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b3aee8c700 con 0x55b3af3080e0 2026-02-25T11:48:24.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.623+0000 7fbc0826fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b3af1d45d0 con 0x55b3af2c3440 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.623+0000 7fbbff7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 0x55b3af2c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.623+0000 7fbbff7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 0x55b3af2c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:43894/0 (socket says 10.20.193.28:43894) 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.623+0000 7fbbff7fe640 1 -- 10.20.193.28:0/3071471701 learned_addr learned my addr 10.20.193.28:0/3071471701 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.623+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/3071471701 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c3440 0x55b3af2c1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.623+0000 7fbbfffff640 1 --2- 10.20.193.28:0/3071471701 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af3080e0 0x55b3af2c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbbff7fe640 1 -- 10.20.193.28:0/3071471701 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c3440 msgr2=0x55b3af2c1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbbff7fe640 1 --2- 10.20.193.28:0/3071471701 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c3440 0x55b3af2c1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbbff7fe640 1 -- 10.20.193.28:0/3071471701 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af3080e0 msgr2=0x55b3af2c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbbff7fe640 1 --2- 10.20.193.28:0/3071471701 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af3080e0 0x55b3af2c2260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbbff7fe640 1 -- 10.20.193.28:0/3071471701 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3af15eb60 con 0x55b3af2c2b30 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/3071471701 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c3440 0x55b3af2c1d20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbbfffff640 1 --2- 10.20.193.28:0/3071471701 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af3080e0 0x55b3af2c2260 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbbff7fe640 1 --2- 10.20.193.28:0/3071471701 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 0x55b3af2c5820 secure :-1 s=READY pgs=373 gs=1 cs=0 l=1 c_cookie=11660615845303505179 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbbec01cae0 tx=0x7fbbec004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d2d454a99b511b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbbc37fe640 1 -- 10.20.193.28:0/3071471701 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbbec019e50 con 0x55b3af2c2b30 2026-02-25T11:48:24.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbbc37fe640 1 -- 10.20.193.28:0/3071471701 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55b3aee646f0 con 0x55b3af2c2b30 2026-02-25T11:48:24.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbbc37fe640 1 -- 10.20.193.28:0/3071471701 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbbec004ad0 con 0x55b3af2c2b30 2026-02-25T11:48:24.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbc0826fe00 1 -- 10.20.193.28:0/3071471701 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 msgr2=0x55b3af2c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:24.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/3071471701 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 0x55b3af2c5820 secure :-1 s=READY pgs=373 gs=1 cs=0 l=1 c_cookie=11660615845303505179 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbbec01cae0 tx=0x7fbbec004770 comp rx=0 tx=0).stop 2026-02-25T11:48:24.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbc0826fe00 1 -- 10.20.193.28:0/3071471701 shutdown_connections 2026-02-25T11:48:24.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/3071471701 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af3080e0 0x55b3af2c2260 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/3071471701 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af2c3440 0x55b3af2c1d20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/3071471701 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 0x55b3af2c5820 unknown :-1 s=CLOSED pgs=373 gs=1 cs=0 l=1 c_cookie=11660615845303505179 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbc0826fe00 1 -- 10.20.193.28:0/3071471701 >> 10.20.193.28:0/3071471701 conn(0x55b3af1e17a0 msgr2=0x55b3af2cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:24.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbc0826fe00 1 -- 10.20.193.28:0/3071471701 shutdown_connections 2026-02-25T11:48:24.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.624+0000 7fbc0826fe00 1 -- 10.20.193.28:0/3071471701 wait complete. 2026-02-25T11:48:24.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.625+0000 7fbc0826fe00 1 -- start start 2026-02-25T11:48:24.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.625+0000 7fbc0826fe00 1 Processor -- start 2026-02-25T11:48:24.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbc0826fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 0x55b3af2e6d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbc0826fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af2c3440 0x55b3af307020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbc0826fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af3080e0 0x55b3af307560 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbc0826fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b3af301ee0 con 0x55b3af2c2b30 2026-02-25T11:48:24.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbc0826fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b3af2e7450 con 0x55b3af2c3440 2026-02-25T11:48:24.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbc0826fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b3af307aa0 con 0x55b3af3080e0 2026-02-25T11:48:24.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbff7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 0x55b3af2e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbff7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 0x55b3af2e6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:43902/0 (socket says 10.20.193.28:43902) 2026-02-25T11:48:24.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbff7fe640 1 -- 10.20.193.28:0/4156182336 learned_addr learned my addr 10.20.193.28:0/4156182336 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:24.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/4156182336 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af2c3440 0x55b3af307020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbfffff640 1 --2- 10.20.193.28:0/4156182336 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af3080e0 0x55b3af307560 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbff7fe640 1 -- 10.20.193.28:0/4156182336 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af3080e0 msgr2=0x55b3af307560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbff7fe640 1 --2- 10.20.193.28:0/4156182336 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af3080e0 0x55b3af307560 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbff7fe640 1 -- 10.20.193.28:0/4156182336 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af2c3440 msgr2=0x55b3af307020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbff7fe640 1 --2- 10.20.193.28:0/4156182336 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af2c3440 0x55b3af307020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbff7fe640 1 -- 10.20.193.28:0/4156182336 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3af1526f0 con 0x55b3af2c2b30 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbff7fe640 1 --2- 10.20.193.28:0/4156182336 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 0x55b3af2e6d20 secure :-1 s=READY pgs=374 gs=1 cs=0 l=1 c_cookie=4264519175308793750 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbbec02f620 tx=0x7fbbec01bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2e9d98761f5796 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbc17fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbc17fa640 1 -- 10.20.193.28:0/4156182336 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbbec03e070 con 0x55b3af2c2b30 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbbc17fa640 1 -- 10.20.193.28:0/4156182336 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fbbec01b8b0 con 0x55b3af2c2b30 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4156182336 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbec016040 con 0x55b3af2c2b30 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.626+0000 7fbc0826fe00 10 client.15489 _ll_register_callbacks cb 0x55b3af14fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.627+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4156182336 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbf002fe20 con 0x55b3af2c2b30 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.627+0000 7fbbc17fa640 1 -- 10.20.193.28:0/4156182336 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbbec019600 con 0x55b3af2c2b30 2026-02-25T11:48:24.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.627+0000 7fbbc17fa640 1 -- 10.20.193.28:0/4156182336 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fbbec019790 con 0x55b3af2c2b30 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.627+0000 7fbbc17fa640 1 -- 10.20.193.28:0/4156182336 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbbec0199d0 con 0x55b3af2c2b30 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.627+0000 7fbc0826fe00 10 client.15489 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.627+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4156182336 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55b3af164340 con 0x55b3af2c2b30 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.627+0000 7fbbc17fa640 1 -- 10.20.193.28:0/4156182336 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fbbec036070 con 0x55b3af2c2b30 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.627+0000 7fbc0826fe00 10 client.15489 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.627+0000 7fbc0826fe00 10 client.15489 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.627+0000 7fbc0826fe00 20 client.15489 populate_metadata read hostname 'trial028' 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.627+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/4156182336 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55b3af3b4ee0 0x55b3af3d5390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.627+0000 7fbc0826fe00 4 client.15489 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.627+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4156182336 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b3aee61f40 con 0x55b3af3b4ee0 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.628+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/4156182336 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55b3af3b4ee0 0x55b3af3d5390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.628+0000 7fbbfeffd640 1 --2- 10.20.193.28:0/4156182336 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55b3af3b4ee0 0x55b3af3d5390 crc :-1 s=READY pgs=24 gs=4 cs=0 l=0 c_cookie=18400394152667292963 s_cookie=13803195915331806485 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff5b54fe0ce8a523 server_cookie=bf8ecd8d7279a515 in_seq=0 out_seq=0 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.628+0000 7fbbc17fa640 10 client.15489 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:48:24.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.628+0000 7fbbc17fa640 1 -- 10.20.193.28:0/4156182336 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b3aee61f40 con 0x55b3af3b4ee0 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.628+0000 7fbbc17fa640 10 client.15489 handle_command_reply: tid=0 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.628+0000 7fbc0826fe00 1 client.15489 shutdown 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4156182336 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 msgr2=0x55b3af2e6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/4156182336 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 0x55b3af2e6d20 secure :-1 s=READY pgs=374 gs=1 cs=0 l=1 c_cookie=4264519175308793750 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbbec02f620 tx=0x7fbbec01bf50 comp rx=0 tx=0).stop 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4156182336 shutdown_connections 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/4156182336 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55b3af3b4ee0 0x55b3af3d5390 crc :-1 s=READY pgs=24 gs=4 cs=0 l=0 c_cookie=18400394152667292963 s_cookie=13803195915331806485 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/4156182336 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b3af3080e0 0x55b3af307560 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/4156182336 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b3af2c3440 0x55b3af307020 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc0826fe00 1 --2- 10.20.193.28:0/4156182336 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b3af2c2b30 0x55b3af2e6d20 unknown :-1 s=CLOSED pgs=374 gs=1 cs=0 l=1 c_cookie=4264519175308793750 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbbc17fa640 0 client.15489 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4156182336 >> 10.20.193.28:0/4156182336 conn(0x55b3af1e17a0 msgr2=0x55b3af2c5820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4156182336 shutdown_connections 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc0826fe00 1 -- 10.20.193.28:0/4156182336 wait complete. 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc0826fe00 20 client.15489 trim_cache size 0 max 16384 2026-02-25T11:48:24.631 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:48:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc04feb640 1 -- 10.20.193.28:0/193863091 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbbd40b5870 msgr2=0x7fbbd40d5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc04feb640 1 --2- 10.20.193.28:0/193863091 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbbd40b5870 0x7fbbd40d5d20 secure :-1 s=READY pgs=257 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbbec0095b0 tx=0x7fbbec005f90 comp rx=0 tx=0).stop 2026-02-25T11:48:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc04feb640 1 -- 10.20.193.28:0/193863091 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbc0010bd50 msgr2=0x7fbc00123b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc04feb640 1 --2- 10.20.193.28:0/193863091 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbc0010bd50 0x7fbc00123b90 secure :-1 s=READY pgs=371 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbc00111c70 tx=0x7fbbf00028d0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc04feb640 1 -- 10.20.193.28:0/193863091 shutdown_connections 2026-02-25T11:48:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc04feb640 1 --2- 10.20.193.28:0/193863091 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbbd40b5870 0x7fbbd40d5d20 unknown :-1 s=CLOSED pgs=257 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc04feb640 1 --2- 10.20.193.28:0/193863091 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbc00111370 0x7fbc00128f60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc04feb640 1 --2- 10.20.193.28:0/193863091 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbc0010bd50 0x7fbc00123b90 unknown :-1 s=CLOSED pgs=371 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc04feb640 1 --2- 10.20.193.28:0/193863091 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbc0010b360 0x7fbc00123650 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc04feb640 1 -- 10.20.193.28:0/193863091 >> 10.20.193.28:0/193863091 conn(0x7fbc000e8e70 msgr2=0x7fbc00109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc04feb640 1 -- 10.20.193.28:0/193863091 shutdown_connections 2026-02-25T11:48:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:24.629+0000 7fbc04feb640 1 -- 10.20.193.28:0/193863091 wait complete. 2026-02-25T11:48:24.730 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:48:24.916 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:48:25.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.104+0000 7fa340dff640 1 -- 10.20.193.28:0/3734957352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33c10c290 msgr2=0x7fa33c10c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.104+0000 7fa340dff640 1 --2- 10.20.193.28:0/3734957352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33c10c290 0x7fa33c10c660 secure :-1 s=READY pgs=375 gs=1 cs=0 l=1 c_cookie=4944408084150702376 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa328009950 tx=0x7fa32802f190 comp rx=0 tx=0).stop 2026-02-25T11:48:25.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.104+0000 7fa340dff640 1 -- 10.20.193.28:0/3734957352 shutdown_connections 2026-02-25T11:48:25.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.104+0000 7fa340dff640 1 --2- 10.20.193.28:0/3734957352 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa33c112480 0x7fa33c172e50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.104+0000 7fa340dff640 1 --2- 10.20.193.28:0/3734957352 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa33c10cc80 0x7fa33c111cd0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.104+0000 7fa340dff640 1 --2- 10.20.193.28:0/3734957352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33c10c290 0x7fa33c10c660 unknown :-1 s=CLOSED pgs=375 gs=1 cs=0 l=1 c_cookie=4944408084150702376 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.104+0000 7fa340dff640 1 -- 10.20.193.28:0/3734957352 >> 10.20.193.28:0/3734957352 conn(0x7fa33c0e9b40 msgr2=0x7fa33c109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:25.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.104+0000 7fa340dff640 1 -- 10.20.193.28:0/3734957352 shutdown_connections 2026-02-25T11:48:25.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.104+0000 7fa340dff640 1 -- 10.20.193.28:0/3734957352 wait complete. 2026-02-25T11:48:25.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.105+0000 7fa340dff640 1 Processor -- start 2026-02-25T11:48:25.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.105+0000 7fa340dff640 1 -- start start 2026-02-25T11:48:25.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.105+0000 7fa340dff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa33c10c290 0x7fa33c0f4840 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.105+0000 7fa340dff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33c10cc80 0x7fa33c172d50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.105+0000 7fa340dff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa33c112480 0x7fa33c0f4f40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.105+0000 7fa340dff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c0fa100 con 0x7fa33c10cc80 2026-02-25T11:48:25.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.105+0000 7fa340dff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c0fa270 con 0x7fa33c10c290 2026-02-25T11:48:25.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.105+0000 7fa340dff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c0fa430 con 0x7fa33c112480 2026-02-25T11:48:25.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa33affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33c10cc80 0x7fa33c172d50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa33affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33c10cc80 0x7fa33c172d50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:43918/0 (socket says 10.20.193.28:43918) 2026-02-25T11:48:25.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa33affd640 1 -- 10.20.193.28:0/1898598973 learned_addr learned my addr 10.20.193.28:0/1898598973 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:25.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa33c10c290 0x7fa33c0f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa33bfff640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa33c112480 0x7fa33c0f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa33affd640 1 -- 10.20.193.28:0/1898598973 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa33c112480 msgr2=0x7fa33c0f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa33affd640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa33c112480 0x7fa33c0f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa33affd640 1 -- 10.20.193.28:0/1898598973 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa33c10c290 msgr2=0x7fa33c0f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa33affd640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa33c10c290 0x7fa33c0f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa33affd640 1 -- 10.20.193.28:0/1898598973 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa328009570 con 0x7fa33c10cc80 2026-02-25T11:48:25.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa33affd640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33c10cc80 0x7fa33c172d50 secure :-1 s=READY pgs=376 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa32c005db0 tx=0x7fa32c00ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:25.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa338ff9640 1 -- 10.20.193.28:0/1898598973 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa32c004500 con 0x7fa33c10cc80 2026-02-25T11:48:25.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa338ff9640 1 -- 10.20.193.28:0/1898598973 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa32c004d10 con 0x7fa33c10cc80 2026-02-25T11:48:25.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa340dff640 1 -- 10.20.193.28:0/1898598973 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33c0fa710 con 0x7fa33c10cc80 2026-02-25T11:48:25.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa338ff9640 1 -- 10.20.193.28:0/1898598973 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa32c005280 con 0x7fa33c10cc80 2026-02-25T11:48:25.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.106+0000 7fa340dff640 1 -- 10.20.193.28:0/1898598973 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa33c0fabe0 con 0x7fa33c10cc80 2026-02-25T11:48:25.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.109+0000 7fa338ff9640 1 -- 10.20.193.28:0/1898598973 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fa32c005410 con 0x7fa33c10cc80 2026-02-25T11:48:25.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.111+0000 7fa338ff9640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa3100b57e0 0x7fa3100d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.111+0000 7fa338ff9640 1 -- 10.20.193.28:0/1898598973 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa32c0e6de0 con 0x7fa33c10cc80 2026-02-25T11:48:25.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.111+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa3100b57e0 0x7fa3100d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.112+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa3100b57e0 0x7fa3100d5c90 secure :-1 s=READY pgs=258 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa3280098a0 tx=0x7fa32803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:25.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.112+0000 7fa344083e00 1 Processor -- start 2026-02-25T11:48:25.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.112+0000 7fa344083e00 1 -- start start 2026-02-25T11:48:25.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.112+0000 7fa344083e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689cb30 0x55eee689cf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.112+0000 7fa344083e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee689d440 0x55eee68e1ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.112+0000 7fa344083e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee68e20e0 0x55eee69024c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.112+0000 7fa344083e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55eee67b28a0 con 0x55eee689d440 2026-02-25T11:48:25.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.112+0000 7fa344083e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55eee67ae5d0 con 0x55eee689cb30 2026-02-25T11:48:25.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.112+0000 7fa344083e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55eee67a8570 con 0x55eee68e20e0 2026-02-25T11:48:25.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa33affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee689d440 0x55eee68e1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa33bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee68e20e0 0x55eee69024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa33bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee68e20e0 0x55eee69024c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:51842/0 (socket says 10.20.193.28:51842) 2026-02-25T11:48:25.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa33bfff640 1 -- 10.20.193.28:0/3759195749 learned_addr learned my addr 10.20.193.28:0/3759195749 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:25.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/3759195749 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689cb30 0x55eee689cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa33affd640 1 -- 10.20.193.28:0/3759195749 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee68e20e0 msgr2=0x55eee69024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa33affd640 1 --2- 10.20.193.28:0/3759195749 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee68e20e0 0x55eee69024c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa33affd640 1 -- 10.20.193.28:0/3759195749 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689cb30 msgr2=0x55eee689cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa33affd640 1 --2- 10.20.193.28:0/3759195749 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689cb30 0x55eee689cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa33affd640 1 -- 10.20.193.28:0/3759195749 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee6753d10 con 0x55eee689d440 2026-02-25T11:48:25.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/3759195749 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689cb30 0x55eee689cf00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:48:25.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa33affd640 1 --2- 10.20.193.28:0/3759195749 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee689d440 0x55eee68e1ba0 secure :-1 s=READY pgs=377 gs=2 cs=0 l=1 c_cookie=6952325899834720618 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa32c0040a0 tx=0x7fa32c0a9380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607b9f922665816a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:25.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa2fffff640 1 -- 10.20.193.28:0/3759195749 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa32c097040 con 0x55eee689d440 2026-02-25T11:48:25.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa2fffff640 1 -- 10.20.193.28:0/3759195749 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa32c0acce0 con 0x55eee689d440 2026-02-25T11:48:25.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.113+0000 7fa2fffff640 1 -- 10.20.193.28:0/3759195749 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa32c0b8620 con 0x55eee689d440 2026-02-25T11:48:25.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.114+0000 7fa344083e00 1 -- 10.20.193.28:0/3759195749 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee689d440 msgr2=0x55eee68e1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.114+0000 7fa344083e00 1 --2- 10.20.193.28:0/3759195749 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee689d440 0x55eee68e1ba0 secure :-1 s=READY pgs=377 gs=2 cs=0 l=1 c_cookie=6952325899834720618 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa32c0040a0 tx=0x7fa32c0a9380 comp rx=0 tx=0).stop 2026-02-25T11:48:25.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.114+0000 7fa344083e00 1 -- 10.20.193.28:0/3759195749 shutdown_connections 2026-02-25T11:48:25.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.114+0000 7fa344083e00 1 --2- 10.20.193.28:0/3759195749 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee68e20e0 0x55eee69024c0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.114+0000 7fa344083e00 1 --2- 10.20.193.28:0/3759195749 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee689d440 0x55eee68e1ba0 unknown :-1 s=CLOSED pgs=377 gs=2 cs=0 l=1 c_cookie=6952325899834720618 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.114+0000 7fa344083e00 1 --2- 10.20.193.28:0/3759195749 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689cb30 0x55eee689cf00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.114+0000 7fa344083e00 1 -- 10.20.193.28:0/3759195749 >> 10.20.193.28:0/3759195749 conn(0x55eee67bb7a0 msgr2=0x55eee689b410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:25.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.114+0000 7fa344083e00 1 -- 10.20.193.28:0/3759195749 shutdown_connections 2026-02-25T11:48:25.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.114+0000 7fa344083e00 1 -- 10.20.193.28:0/3759195749 wait complete. 2026-02-25T11:48:25.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.115+0000 7fa344083e00 1 -- start start 2026-02-25T11:48:25.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.115+0000 7fa344083e00 1 Processor -- start 2026-02-25T11:48:25.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.115+0000 7fa344083e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee689cb30 0x55eee68c0d90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.115+0000 7fa344083e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 0x55eee68c12d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.115+0000 7fa344083e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee68e20e0 0x55eee68bfb90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.115+0000 7fa344083e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55eee68c00d0 con 0x55eee689cb30 2026-02-25T11:48:25.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.115+0000 7fa344083e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55eee68c0240 con 0x55eee689d440 2026-02-25T11:48:25.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.115+0000 7fa344083e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55eee68c0400 con 0x55eee68e20e0 2026-02-25T11:48:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.116+0000 7fa33b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee689cb30 0x55eee68c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.116+0000 7fa33b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee689cb30 0x55eee68c0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:43924/0 (socket says 10.20.193.28:43924) 2026-02-25T11:48:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.116+0000 7fa33b7fe640 1 -- 10.20.193.28:0/3877680546 learned_addr learned my addr 10.20.193.28:0/3877680546 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.116+0000 7fa33affd640 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 0x55eee68c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.116+0000 7fa33bfff640 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee68e20e0 0x55eee68bfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.116+0000 7fa33b7fe640 1 -- 10.20.193.28:0/3877680546 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee68e20e0 msgr2=0x55eee68bfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.116+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee68e20e0 0x55eee68bfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.116+0000 7fa33b7fe640 1 -- 10.20.193.28:0/3877680546 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 msgr2=0x55eee68c12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.116+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 0x55eee68c12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.116+0000 7fa33b7fe640 1 -- 10.20.193.28:0/3877680546 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee672c3c0 con 0x55eee689cb30 2026-02-25T11:48:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.116+0000 7fa33affd640 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 0x55eee68c12d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:48:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.116+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee689cb30 0x55eee68c0d90 secure :-1 s=READY pgs=378 gs=1 cs=0 l=1 c_cookie=6191338775290760411 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa3280047a0 tx=0x7fa32802f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ec0dd0214eccdb server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:25.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.116+0000 7fa2fd7fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:25.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3877680546 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa328038b90 con 0x55eee689cb30 2026-02-25T11:48:25.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3877680546 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa32804a050 con 0x55eee689cb30 2026-02-25T11:48:25.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3877680546 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa328044690 con 0x55eee689cb30 2026-02-25T11:48:25.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa344083e00 1 -- 10.20.193.28:0/3877680546 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa33c0fabe0 con 0x55eee689cb30 2026-02-25T11:48:25.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3877680546 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa328044820 con 0x55eee689cb30 2026-02-25T11:48:25.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa344083e00 10 client.15501 _ll_register_callbacks cb 0x55eee6729a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:25.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa344083e00 1 -- 10.20.193.28:0/3877680546 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eee643e6f0 con 0x55eee689cb30 2026-02-25T11:48:25.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3877680546 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa328038cf0 con 0x55eee689cb30 2026-02-25T11:48:25.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa344083e00 10 client.15501 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:25.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa344083e00 1 -- 10.20.193.28:0/3877680546 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55eee673e0e0 con 0x55eee689cb30 2026-02-25T11:48:25.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3877680546 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa32804f410 con 0x55eee689cb30 2026-02-25T11:48:25.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa344083e00 10 client.15501 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:25.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa344083e00 10 client.15501 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:48:25.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa344083e00 20 client.15501 populate_metadata read hostname 'trial028' 2026-02-25T11:48:25.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa344083e00 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55eee68c80c0 0x55eee68c8560 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa344083e00 4 client.15501 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:48:25.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.117+0000 7fa344083e00 1 -- 10.20.193.28:0/3877680546 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eee677afc0 con 0x55eee68c80c0 2026-02-25T11:48:25.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.118+0000 7fa33affd640 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55eee68c80c0 0x55eee68c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.118+0000 7fa33affd640 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55eee68c80c0 0x55eee68c8560 crc :-1 s=READY pgs=24 gs=4 cs=0 l=0 c_cookie=8580578425917777424 s_cookie=14892158543329974569 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7714569dd435a610 server_cookie=ceab934b09fb5929 in_seq=0 out_seq=0 2026-02-25T11:48:25.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.118+0000 7fa2fd7fa640 10 client.15501 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:48:25.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3877680546 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55eee677afc0 con 0x55eee68c80c0 2026-02-25T11:48:25.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa2fd7fa640 10 client.15501 handle_command_reply: tid=0 2026-02-25T11:48:25.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa344083e00 1 client.15501 shutdown 2026-02-25T11:48:25.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa344083e00 1 -- 10.20.193.28:0/3877680546 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee689cb30 msgr2=0x55eee68c0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa344083e00 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee689cb30 0x55eee68c0d90 secure :-1 s=READY pgs=378 gs=1 cs=0 l=1 c_cookie=6191338775290760411 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa3280047a0 tx=0x7fa32802f190 comp rx=0 tx=0).stop 2026-02-25T11:48:25.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa344083e00 1 -- 10.20.193.28:0/3877680546 shutdown_connections 2026-02-25T11:48:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa344083e00 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55eee68c80c0 0x55eee68c8560 crc :-1 s=READY pgs=24 gs=4 cs=0 l=0 c_cookie=8580578425917777424 s_cookie=14892158543329974569 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa344083e00 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee68e20e0 0x55eee68bfb90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa344083e00 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 0x55eee68c12d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa344083e00 1 --2- 10.20.193.28:0/3877680546 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee689cb30 0x55eee68c0d90 unknown :-1 s=CLOSED pgs=378 gs=1 cs=0 l=1 c_cookie=6191338775290760411 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa344083e00 1 -- 10.20.193.28:0/3877680546 >> 10.20.193.28:0/3877680546 conn(0x55eee67bb7a0 msgr2=0x55eee6884520 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa2fd7fa640 0 client.15501 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:48:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa344083e00 1 -- 10.20.193.28:0/3877680546 shutdown_connections 2026-02-25T11:48:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa344083e00 1 -- 10.20.193.28:0/3877680546 wait complete. 2026-02-25T11:48:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.119+0000 7fa344083e00 20 client.15501 trim_cache size 0 max 16384 2026-02-25T11:48:25.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.126+0000 7fa344083e00 1 Processor -- start 2026-02-25T11:48:25.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.126+0000 7fa344083e00 1 -- start start 2026-02-25T11:48:25.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa344083e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 0x55eee68bfb20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa344083e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 0x55eee68c1210 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa344083e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee68e20e0 0x55eee687b320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa344083e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55eee67a61a0 con 0x55eee68e20e0 2026-02-25T11:48:25.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa344083e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55eee6466700 con 0x55eee689d440 2026-02-25T11:48:25.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa344083e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55eee6895cf0 con 0x55eee689cb30 2026-02-25T11:48:25.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa33affd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 0x55eee68c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa33b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 0x55eee68bfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa33b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 0x55eee68bfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:51870/0 (socket says 10.20.193.28:51870) 2026-02-25T11:48:25.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa33b7fe640 1 -- 10.20.193.28:0/1631307395 learned_addr learned my addr 10.20.193.28:0/1631307395 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:25.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa33bfff640 1 --2- 10.20.193.28:0/1631307395 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee68e20e0 0x55eee687b320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa33b7fe640 1 -- 10.20.193.28:0/1631307395 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 msgr2=0x55eee68c1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/1631307395 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 0x55eee68c1210 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa33b7fe640 1 -- 10.20.193.28:0/1631307395 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee68e20e0 msgr2=0x55eee687b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/1631307395 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee68e20e0 0x55eee687b320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa33b7fe640 1 -- 10.20.193.28:0/1631307395 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee6738b60 con 0x55eee689cb30 2026-02-25T11:48:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.127+0000 7fa33bfff640 1 --2- 10.20.193.28:0/1631307395 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee68e20e0 0x55eee687b320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:48:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.128+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/1631307395 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 0x55eee68bfb20 secure :-1 s=READY pgs=116 gs=1 cs=0 l=1 c_cookie=12673253210501694356 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa32802f6a0 tx=0x7fa328059000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afe070a630884394 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.128+0000 7fa2ff7fe640 1 -- 10.20.193.28:0/1631307395 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa328057360 con 0x55eee689cb30 2026-02-25T11:48:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.128+0000 7fa2ff7fe640 1 -- 10.20.193.28:0/1631307395 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa32804a040 con 0x55eee689cb30 2026-02-25T11:48:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.128+0000 7fa2ff7fe640 1 -- 10.20.193.28:0/1631307395 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa328044410 con 0x55eee689cb30 2026-02-25T11:48:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.128+0000 7fa344083e00 1 -- 10.20.193.28:0/1631307395 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 msgr2=0x55eee68bfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.128+0000 7fa344083e00 1 --2- 10.20.193.28:0/1631307395 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 0x55eee68bfb20 secure :-1 s=READY pgs=116 gs=1 cs=0 l=1 c_cookie=12673253210501694356 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa32802f6a0 tx=0x7fa328059000 comp rx=0 tx=0).stop 2026-02-25T11:48:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.128+0000 7fa344083e00 1 -- 10.20.193.28:0/1631307395 shutdown_connections 2026-02-25T11:48:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.128+0000 7fa344083e00 1 --2- 10.20.193.28:0/1631307395 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee68e20e0 0x55eee687b320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.128+0000 7fa344083e00 1 --2- 10.20.193.28:0/1631307395 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 0x55eee68c1210 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.128+0000 7fa344083e00 1 --2- 10.20.193.28:0/1631307395 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 0x55eee68bfb20 unknown :-1 s=CLOSED pgs=116 gs=1 cs=0 l=1 c_cookie=12673253210501694356 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.128+0000 7fa344083e00 1 -- 10.20.193.28:0/1631307395 >> 10.20.193.28:0/1631307395 conn(0x55eee67bb7a0 msgr2=0x55eee687ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.128+0000 7fa344083e00 1 -- 10.20.193.28:0/1631307395 shutdown_connections 2026-02-25T11:48:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.128+0000 7fa344083e00 1 -- 10.20.193.28:0/1631307395 wait complete. 2026-02-25T11:48:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.129+0000 7fa344083e00 1 -- start start 2026-02-25T11:48:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.129+0000 7fa344083e00 1 Processor -- start 2026-02-25T11:48:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.129+0000 7fa344083e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 0x55eee68bf6c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.129+0000 7fa344083e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 0x55eee68bdd00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.129+0000 7fa344083e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee68e20e0 0x55eee68be240 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.129+0000 7fa344083e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55eee68be780 con 0x55eee68e20e0 2026-02-25T11:48:25.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.129+0000 7fa344083e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55eee68be8f0 con 0x55eee689d440 2026-02-25T11:48:25.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.129+0000 7fa344083e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55eee68beab0 con 0x55eee689cb30 2026-02-25T11:48:25.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.130+0000 7fa33b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 0x55eee68bf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.130+0000 7fa33bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee68e20e0 0x55eee68be240 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.130+0000 7fa33affd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 0x55eee68bdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.130+0000 7fa33b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 0x55eee68bf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:51878/0 (socket says 10.20.193.28:51878) 2026-02-25T11:48:25.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.130+0000 7fa33b7fe640 1 -- 10.20.193.28:0/3288440362 learned_addr learned my addr 10.20.193.28:0/3288440362 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:25.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.130+0000 7fa33b7fe640 1 -- 10.20.193.28:0/3288440362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 msgr2=0x55eee68bdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.130+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/3288440362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 0x55eee68bdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.130+0000 7fa33b7fe640 1 -- 10.20.193.28:0/3288440362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee68e20e0 msgr2=0x55eee68be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.130+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/3288440362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee68e20e0 0x55eee68be240 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.130+0000 7fa33b7fe640 1 -- 10.20.193.28:0/3288440362 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee673e0e0 con 0x55eee689cb30 2026-02-25T11:48:25.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.130+0000 7fa33bfff640 1 --2- 10.20.193.28:0/3288440362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee68e20e0 0x55eee68be240 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:48:25.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.130+0000 7fa33b7fe640 1 --2- 10.20.193.28:0/3288440362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 0x55eee68bf6c0 secure :-1 s=READY pgs=117 gs=1 cs=0 l=1 c_cookie=7622145095299355948 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa3280047a0 tx=0x7fa3280047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69c74c948fcdd52c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.130+0000 7fa2fd7fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:48:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.131+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3288440362 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa328058ea0 con 0x55eee689cb30 2026-02-25T11:48:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.131+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3288440362 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55eee643e6f0 con 0x55eee689cb30 2026-02-25T11:48:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.131+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3288440362 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa3280583c0 con 0x55eee689cb30 2026-02-25T11:48:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.131+0000 7fa344083e00 1 -- 10.20.193.28:0/3288440362 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa330003860 con 0x55eee689cb30 2026-02-25T11:48:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.131+0000 7fa344083e00 10 client.25351 _ll_register_callbacks cb 0x55eee6729a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.131+0000 7fa344083e00 1 -- 10.20.193.28:0/3288440362 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa32c0acce0 con 0x55eee689cb30 2026-02-25T11:48:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.131+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3288440362 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa32804f470 con 0x55eee689cb30 2026-02-25T11:48:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.131+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3288440362 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa3280579f0 con 0x55eee689cb30 2026-02-25T11:48:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.131+0000 7fa344083e00 10 client.25351 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.131+0000 7fa344083e00 1 -- 10.20.193.28:0/3288440362 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55eee672c6f0 con 0x55eee689cb30 2026-02-25T11:48:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.131+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3288440362 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa328073060 con 0x55eee689cb30 2026-02-25T11:48:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.132+0000 7fa344083e00 10 client.25351 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.132+0000 7fa344083e00 10 client.25351 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:48:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.132+0000 7fa344083e00 20 client.25351 populate_metadata read hostname 'trial028' 2026-02-25T11:48:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.132+0000 7fa344083e00 1 --2- 10.20.193.28:0/3288440362 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55eee68ce140 0x55eee68ce5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.132+0000 7fa344083e00 4 client.25351 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:48:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.132+0000 7fa344083e00 1 -- 10.20.193.28:0/3288440362 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eee643bf40 con 0x55eee68ce140 2026-02-25T11:48:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.132+0000 7fa33affd640 1 --2- 10.20.193.28:0/3288440362 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55eee68ce140 0x55eee68ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.132+0000 7fa33affd640 1 --2- 10.20.193.28:0/3288440362 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55eee68ce140 0x55eee68ce5e0 crc :-1 s=READY pgs=25 gs=4 cs=0 l=0 c_cookie=11346984766548542744 s_cookie=16195750922255892890 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d789686b9e50518 server_cookie=e0c2dda7afc4819a in_seq=0 out_seq=0 2026-02-25T11:48:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.132+0000 7fa2fd7fa640 10 client.25351 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:48:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.132+0000 7fa2fd7fa640 1 -- 10.20.193.28:0/3288440362 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eee643bf40 con 0x55eee68ce140 2026-02-25T11:48:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.132+0000 7fa2fd7fa640 10 client.25351 handle_command_reply: tid=0 2026-02-25T11:48:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.132+0000 7fa344083e00 1 client.25351 shutdown 2026-02-25T11:48:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa344083e00 1 -- 10.20.193.28:0/3288440362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 msgr2=0x55eee68bf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa344083e00 1 --2- 10.20.193.28:0/3288440362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 0x55eee68bf6c0 secure :-1 s=READY pgs=117 gs=1 cs=0 l=1 c_cookie=7622145095299355948 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa3280047a0 tx=0x7fa3280047d0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa344083e00 1 -- 10.20.193.28:0/3288440362 shutdown_connections 2026-02-25T11:48:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa344083e00 1 --2- 10.20.193.28:0/3288440362 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55eee68ce140 0x55eee68ce5e0 crc :-1 s=READY pgs=25 gs=4 cs=0 l=0 c_cookie=11346984766548542744 s_cookie=16195750922255892890 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa344083e00 1 --2- 10.20.193.28:0/3288440362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eee68e20e0 0x55eee68be240 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa2fd7fa640 0 client.25351 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:48:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa344083e00 1 --2- 10.20.193.28:0/3288440362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eee689d440 0x55eee68bdd00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa344083e00 1 --2- 10.20.193.28:0/3288440362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eee689cb30 0x55eee68bf6c0 unknown :-1 s=CLOSED pgs=117 gs=1 cs=0 l=1 c_cookie=7622145095299355948 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa344083e00 1 -- 10.20.193.28:0/3288440362 >> 10.20.193.28:0/3288440362 conn(0x55eee67bb7a0 msgr2=0x55eee687b400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:25.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa344083e00 1 -- 10.20.193.28:0/3288440362 shutdown_connections 2026-02-25T11:48:25.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa344083e00 1 -- 10.20.193.28:0/3288440362 wait complete. 2026-02-25T11:48:25.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa344083e00 20 client.25351 trim_cache size 0 max 16384 2026-02-25T11:48:25.156 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:48:25.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa340dff640 1 -- 10.20.193.28:0/1898598973 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa3100b57e0 msgr2=0x7fa3100d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa340dff640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa3100b57e0 0x7fa3100d5c90 secure :-1 s=READY pgs=258 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa3280098a0 tx=0x7fa32803a040 comp rx=0 tx=0).stop 2026-02-25T11:48:25.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa340dff640 1 -- 10.20.193.28:0/1898598973 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33c10cc80 msgr2=0x7fa33c172d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa340dff640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33c10cc80 0x7fa33c172d50 secure :-1 s=READY pgs=376 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa32c005db0 tx=0x7fa32c00ddf0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa340dff640 1 -- 10.20.193.28:0/1898598973 shutdown_connections 2026-02-25T11:48:25.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa340dff640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa3100b57e0 0x7fa3100d5c90 unknown :-1 s=CLOSED pgs=258 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa340dff640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa33c112480 0x7fa33c0f4f40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa340dff640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa33c10cc80 0x7fa33c172d50 unknown :-1 s=CLOSED pgs=376 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa340dff640 1 --2- 10.20.193.28:0/1898598973 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa33c10c290 0x7fa33c0f4840 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa340dff640 1 -- 10.20.193.28:0/1898598973 >> 10.20.193.28:0/1898598973 conn(0x7fa33c0e9b40 msgr2=0x7fa33c10a700 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:25.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa340dff640 1 -- 10.20.193.28:0/1898598973 shutdown_connections 2026-02-25T11:48:25.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.133+0000 7fa340dff640 1 -- 10.20.193.28:0/1898598973 wait complete. 2026-02-25T11:48:25.233 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:48:25.417 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:48:25.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.593+0000 7f0764ffc640 1 -- 10.20.193.28:0/1776420690 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f076010b360 msgr2=0x7f076010b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.593+0000 7f0764ffc640 1 --2- 10.20.193.28:0/1776420690 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f076010b360 0x7f076010b730 secure :-1 s=READY pgs=379 gs=1 cs=0 l=1 c_cookie=762574543993915926 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f074c009950 tx=0x7f074c02f190 comp rx=0 tx=0).stop 2026-02-25T11:48:25.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.593+0000 7f0764ffc640 1 -- 10.20.193.28:0/1776420690 shutdown_connections 2026-02-25T11:48:25.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.593+0000 7f0764ffc640 1 --2- 10.20.193.28:0/1776420690 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0760111370 0x7f0760111d50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.593+0000 7f0764ffc640 1 --2- 10.20.193.28:0/1776420690 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f076010bd50 0x7f0760110be0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.593+0000 7f0764ffc640 1 --2- 10.20.193.28:0/1776420690 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f076010b360 0x7f076010b730 unknown :-1 s=CLOSED pgs=379 gs=1 cs=0 l=1 c_cookie=762574543993915926 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.593+0000 7f0764ffc640 1 -- 10.20.193.28:0/1776420690 >> 10.20.193.28:0/1776420690 conn(0x7f07600e8e70 msgr2=0x7f0760109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:25.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.593+0000 7f0764ffc640 1 -- 10.20.193.28:0/1776420690 shutdown_connections 2026-02-25T11:48:25.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.593+0000 7f0764ffc640 1 -- 10.20.193.28:0/1776420690 wait complete. 2026-02-25T11:48:25.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.594+0000 7f0764ffc640 1 Processor -- start 2026-02-25T11:48:25.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.594+0000 7f0764ffc640 1 -- start start 2026-02-25T11:48:25.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.594+0000 7f0764ffc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f076010b360 0x7f07601236a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.594+0000 7f0764ffc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f076010bd50 0x7f0760123be0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.594+0000 7f0764ffc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0760111370 0x7f0760128fb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.594+0000 7f0764ffc640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f07601295d0 con 0x7f076010b360 2026-02-25T11:48:25.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.594+0000 7f0764ffc640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0760129740 con 0x7f076010bd50 2026-02-25T11:48:25.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.594+0000 7f0764ffc640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0760129900 con 0x7f0760111370 2026-02-25T11:48:25.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.594+0000 7f075f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f076010b360 0x7f07601236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.594+0000 7f075f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f076010b360 0x7f07601236a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:43966/0 (socket says 10.20.193.28:43966) 2026-02-25T11:48:25.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.594+0000 7f075f7fe640 1 -- 10.20.193.28:0/569770411 learned_addr learned my addr 10.20.193.28:0/569770411 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:25.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.594+0000 7f075effd640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f076010bd50 0x7f0760123be0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.595+0000 7f075ffff640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0760111370 0x7f0760128fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.595+0000 7f075f7fe640 1 -- 10.20.193.28:0/569770411 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0760111370 msgr2=0x7f0760128fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.595+0000 7f075f7fe640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0760111370 0x7f0760128fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.595+0000 7f075f7fe640 1 -- 10.20.193.28:0/569770411 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f076010bd50 msgr2=0x7f0760123be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.595+0000 7f075f7fe640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f076010bd50 0x7f0760123be0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.595+0000 7f075f7fe640 1 -- 10.20.193.28:0/569770411 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074c009570 con 0x7f076010b360 2026-02-25T11:48:25.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.595+0000 7f075f7fe640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f076010b360 0x7f07601236a0 secure :-1 s=READY pgs=380 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f074c0098a0 tx=0x7f074c0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:25.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.595+0000 7f075cff9640 1 -- 10.20.193.28:0/569770411 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f074c03d070 con 0x7f076010b360 2026-02-25T11:48:25.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.595+0000 7f075cff9640 1 -- 10.20.193.28:0/569770411 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f074c002a40 con 0x7f076010b360 2026-02-25T11:48:25.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.595+0000 7f075cff9640 1 -- 10.20.193.28:0/569770411 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f074c041820 con 0x7f076010b360 2026-02-25T11:48:25.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.595+0000 7f0764ffc640 1 -- 10.20.193.28:0/569770411 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0760129b80 con 0x7f076010b360 2026-02-25T11:48:25.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.595+0000 7f0764ffc640 1 -- 10.20.193.28:0/569770411 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f076016a140 con 0x7f076010b360 2026-02-25T11:48:25.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.599+0000 7f075cff9640 1 -- 10.20.193.28:0/569770411 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f074c0419c0 con 0x7f076010b360 2026-02-25T11:48:25.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.600+0000 7f075cff9640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07340b57f0 0x7f07340d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.600+0000 7f075cff9640 1 -- 10.20.193.28:0/569770411 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f074c10af80 con 0x7f076010b360 2026-02-25T11:48:25.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.600+0000 7f075effd640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07340b57f0 0x7f07340d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f0768280e00 1 Processor -- start 2026-02-25T11:48:25.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f0768280e00 1 -- start start 2026-02-25T11:48:25.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f0768280e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 0x564cbc974f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f0768280e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc975440 0x564cbc9b9ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f0768280e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc9ba0e0 0x564cbc9da4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f0768280e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564cbc88a8a0 con 0x564cbc974b30 2026-02-25T11:48:25.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f0768280e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564cbc8865d0 con 0x564cbc975440 2026-02-25T11:48:25.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f0768280e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564cbc880570 con 0x564cbc9ba0e0 2026-02-25T11:48:25.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f075effd640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07340b57f0 0x7f07340d5ca0 secure :-1 s=READY pgs=259 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f076012a040 tx=0x7f07500073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:25.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f075f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 0x564cbc974f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f075f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 0x564cbc974f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:43978/0 (socket says 10.20.193.28:43978) 2026-02-25T11:48:25.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f075f7fe640 1 -- 10.20.193.28:0/2218875063 learned_addr learned my addr 10.20.193.28:0/2218875063 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:25.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f075ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc9ba0e0 0x564cbc9da4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f075effd640 1 --2- 10.20.193.28:0/2218875063 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc975440 0x564cbc9b9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f075f7fe640 1 -- 10.20.193.28:0/2218875063 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc9ba0e0 msgr2=0x564cbc9da4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f075f7fe640 1 --2- 10.20.193.28:0/2218875063 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc9ba0e0 0x564cbc9da4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f075f7fe640 1 -- 10.20.193.28:0/2218875063 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc975440 msgr2=0x564cbc9b9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f075f7fe640 1 --2- 10.20.193.28:0/2218875063 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc975440 0x564cbc9b9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.601+0000 7f075f7fe640 1 -- 10.20.193.28:0/2218875063 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cbc82bd10 con 0x564cbc974b30 2026-02-25T11:48:25.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.602+0000 7f075f7fe640 1 --2- 10.20.193.28:0/2218875063 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 0x564cbc974f00 secure :-1 s=READY pgs=381 gs=1 cs=0 l=1 c_cookie=5300792411534281328 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f074c032040 tx=0x7f074c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4990329d0d964a70 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:25.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.602+0000 7f0723fff640 1 -- 10.20.193.28:0/2218875063 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f074c0d0ab0 con 0x564cbc974b30 2026-02-25T11:48:25.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.602+0000 7f0723fff640 1 -- 10.20.193.28:0/2218875063 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f074c048e80 con 0x564cbc974b30 2026-02-25T11:48:25.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.602+0000 7f0723fff640 1 -- 10.20.193.28:0/2218875063 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f074c0dc1e0 con 0x564cbc974b30 2026-02-25T11:48:25.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.602+0000 7f0768280e00 1 -- 10.20.193.28:0/2218875063 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 msgr2=0x564cbc974f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.602+0000 7f0768280e00 1 --2- 10.20.193.28:0/2218875063 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 0x564cbc974f00 secure :-1 s=READY pgs=381 gs=1 cs=0 l=1 c_cookie=5300792411534281328 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f074c032040 tx=0x7f074c009310 comp rx=0 tx=0).stop 2026-02-25T11:48:25.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.602+0000 7f0768280e00 1 -- 10.20.193.28:0/2218875063 shutdown_connections 2026-02-25T11:48:25.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.602+0000 7f0768280e00 1 --2- 10.20.193.28:0/2218875063 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc9ba0e0 0x564cbc9da4c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.602+0000 7f0768280e00 1 --2- 10.20.193.28:0/2218875063 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc975440 0x564cbc9b9ba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.602+0000 7f0768280e00 1 --2- 10.20.193.28:0/2218875063 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 0x564cbc974f00 unknown :-1 s=CLOSED pgs=381 gs=1 cs=0 l=1 c_cookie=5300792411534281328 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.602+0000 7f0768280e00 1 -- 10.20.193.28:0/2218875063 >> 10.20.193.28:0/2218875063 conn(0x564cbc8937a0 msgr2=0x564cbc973410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:25.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.602+0000 7f0768280e00 1 -- 10.20.193.28:0/2218875063 shutdown_connections 2026-02-25T11:48:25.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.602+0000 7f0768280e00 1 -- 10.20.193.28:0/2218875063 wait complete. 2026-02-25T11:48:25.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.603+0000 7f0768280e00 1 -- start start 2026-02-25T11:48:25.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.604+0000 7f0768280e00 1 Processor -- start 2026-02-25T11:48:25.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.604+0000 7f0768280e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc974b30 0x564cbc98cd40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.604+0000 7f0768280e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc975440 0x564cbc98d280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.604+0000 7f0768280e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc9ba0e0 0x564cbc992270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.604+0000 7f0768280e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564cbc9927b0 con 0x564cbc9ba0e0 2026-02-25T11:48:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.604+0000 7f0768280e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564cbc992920 con 0x564cbc974b30 2026-02-25T11:48:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.604+0000 7f0768280e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564cbc992ae0 con 0x564cbc975440 2026-02-25T11:48:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.604+0000 7f075ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc9ba0e0 0x564cbc992270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.604+0000 7f075ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc9ba0e0 0x564cbc992270 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:43994/0 (socket says 10.20.193.28:43994) 2026-02-25T11:48:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.604+0000 7f075ffff640 1 -- 10.20.193.28:0/2813558215 learned_addr learned my addr 10.20.193.28:0/2813558215 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.604+0000 7f075effd640 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc975440 0x564cbc98d280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.604+0000 7f075f7fe640 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc974b30 0x564cbc98cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f075ffff640 1 -- 10.20.193.28:0/2813558215 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc975440 msgr2=0x564cbc98d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f075ffff640 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc975440 0x564cbc98d280 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f075ffff640 1 -- 10.20.193.28:0/2813558215 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc974b30 msgr2=0x564cbc98cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f075ffff640 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc974b30 0x564cbc98cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f075ffff640 1 -- 10.20.193.28:0/2813558215 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cbc8043c0 con 0x564cbc9ba0e0 2026-02-25T11:48:25.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f075effd640 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc975440 0x564cbc98d280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:48:25.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f075f7fe640 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc974b30 0x564cbc98cd40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:48:25.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f075ffff640 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc9ba0e0 0x564cbc992270 secure :-1 s=READY pgs=382 gs=3 cs=0 l=1 c_cookie=11808366910897086896 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0754006ec0 tx=0x7f075400aa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3dfbf245d7f61b0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:25.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f07217fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:25.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f07217fa640 1 -- 10.20.193.28:0/2813558215 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0754017070 con 0x564cbc9ba0e0 2026-02-25T11:48:25.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f07217fa640 1 -- 10.20.193.28:0/2813558215 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0754012d30 con 0x564cbc9ba0e0 2026-02-25T11:48:25.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f07217fa640 1 -- 10.20.193.28:0/2813558215 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0754011440 con 0x564cbc9ba0e0 2026-02-25T11:48:25.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f0768280e00 1 -- 10.20.193.28:0/2813558215 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f076016a140 con 0x564cbc9ba0e0 2026-02-25T11:48:25.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f0768280e00 10 client.15519 _ll_register_callbacks cb 0x564cbc801a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:25.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f0768280e00 1 -- 10.20.193.28:0/2813558215 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564cbc5166f0 con 0x564cbc9ba0e0 2026-02-25T11:48:25.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f07217fa640 1 -- 10.20.193.28:0/2813558215 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f07540115d0 con 0x564cbc9ba0e0 2026-02-25T11:48:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f07217fa640 1 -- 10.20.193.28:0/2813558215 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f07540117b0 con 0x564cbc9ba0e0 2026-02-25T11:48:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f0768280e00 10 client.15519 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.605+0000 7f0768280e00 1 -- 10.20.193.28:0/2813558215 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x564cbc8160e0 con 0x564cbc9ba0e0 2026-02-25T11:48:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.606+0000 7f07217fa640 1 -- 10.20.193.28:0/2813558215 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f0754012870 con 0x564cbc9ba0e0 2026-02-25T11:48:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.606+0000 7f0768280e00 10 client.15519 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.606+0000 7f0768280e00 10 client.15519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.606+0000 7f0768280e00 20 client.15519 populate_metadata read hostname 'trial028' 2026-02-25T11:48:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.606+0000 7f0768280e00 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564cbc9b7650 0x564cbc9b7af0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.606+0000 7f0768280e00 4 client.15519 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:48:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.606+0000 7f0768280e00 1 -- 10.20.193.28:0/2813558215 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cbc852fc0 con 0x564cbc9b7650 2026-02-25T11:48:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.606+0000 7f075f7fe640 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564cbc9b7650 0x564cbc9b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.607+0000 7f075f7fe640 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564cbc9b7650 0x564cbc9b7af0 crc :-1 s=READY pgs=27 gs=4 cs=0 l=0 c_cookie=218859218710212193 s_cookie=16872624643824067458 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3098b4cc30aae61 server_cookie=ea279ab7f3a2a382 in_seq=0 out_seq=0 2026-02-25T11:48:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.607+0000 7f07217fa640 10 client.15519 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.607+0000 7f07217fa640 1 -- 10.20.193.28:0/2813558215 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x564cbc852fc0 con 0x564cbc9b7650 2026-02-25T11:48:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.607+0000 7f07217fa640 10 client.15519 handle_command_reply: tid=0 2026-02-25T11:48:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.607+0000 7f0768280e00 1 client.15519 shutdown 2026-02-25T11:48:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.608+0000 7f0768280e00 1 -- 10.20.193.28:0/2813558215 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc9ba0e0 msgr2=0x564cbc992270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.608+0000 7f0768280e00 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc9ba0e0 0x564cbc992270 secure :-1 s=READY pgs=382 gs=3 cs=0 l=1 c_cookie=11808366910897086896 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0754006ec0 tx=0x7f075400aa00 comp rx=0 tx=0).stop 2026-02-25T11:48:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.608+0000 7f0768280e00 1 -- 10.20.193.28:0/2813558215 shutdown_connections 2026-02-25T11:48:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.608+0000 7f0768280e00 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564cbc9b7650 0x564cbc9b7af0 crc :-1 s=READY pgs=27 gs=4 cs=0 l=0 c_cookie=218859218710212193 s_cookie=16872624643824067458 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.608+0000 7f0768280e00 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc9ba0e0 0x564cbc992270 unknown :-1 s=CLOSED pgs=382 gs=3 cs=0 l=1 c_cookie=11808366910897086896 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.608+0000 7f0768280e00 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc975440 0x564cbc98d280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.608+0000 7f0768280e00 1 --2- 10.20.193.28:0/2813558215 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc974b30 0x564cbc98cd40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.608+0000 7f07217fa640 0 client.15519 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.608+0000 7f0768280e00 1 -- 10.20.193.28:0/2813558215 >> 10.20.193.28:0/2813558215 conn(0x564cbc8937a0 msgr2=0x564cbc973b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:25.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.608+0000 7f0768280e00 1 -- 10.20.193.28:0/2813558215 shutdown_connections 2026-02-25T11:48:25.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.608+0000 7f0768280e00 1 -- 10.20.193.28:0/2813558215 wait complete. 2026-02-25T11:48:25.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.608+0000 7f0768280e00 20 client.15519 trim_cache size 0 max 16384 2026-02-25T11:48:25.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.615+0000 7f0768280e00 1 Processor -- start 2026-02-25T11:48:25.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.615+0000 7f0768280e00 1 -- start start 2026-02-25T11:48:25.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.615+0000 7f0768280e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc974b30 0x564cbc973c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.615+0000 7f0768280e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc975440 0x564cbc974150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.615+0000 7f0768280e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc9ba0e0 0x564cbc9b2750 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.615+0000 7f0768280e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564cbc87e1a0 con 0x564cbc975440 2026-02-25T11:48:25.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.615+0000 7f0768280e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564cbc53e700 con 0x564cbc9ba0e0 2026-02-25T11:48:25.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.615+0000 7f0768280e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564cbc8865d0 con 0x564cbc974b30 2026-02-25T11:48:25.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f075effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc975440 0x564cbc974150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f075effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc975440 0x564cbc974150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44004/0 (socket says 10.20.193.28:44004) 2026-02-25T11:48:25.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f075effd640 1 -- 10.20.193.28:0/628657812 learned_addr learned my addr 10.20.193.28:0/628657812 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:25.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f075f7fe640 1 --2- 10.20.193.28:0/628657812 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc974b30 0x564cbc973c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f075ffff640 1 --2- 10.20.193.28:0/628657812 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc9ba0e0 0x564cbc9b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f075effd640 1 -- 10.20.193.28:0/628657812 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc974b30 msgr2=0x564cbc973c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f075effd640 1 --2- 10.20.193.28:0/628657812 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc974b30 0x564cbc973c10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f075effd640 1 -- 10.20.193.28:0/628657812 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc9ba0e0 msgr2=0x564cbc9b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f075effd640 1 --2- 10.20.193.28:0/628657812 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc9ba0e0 0x564cbc9b2750 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f075effd640 1 -- 10.20.193.28:0/628657812 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cbc97f660 con 0x564cbc975440 2026-02-25T11:48:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f075effd640 1 --2- 10.20.193.28:0/628657812 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc975440 0x564cbc974150 secure :-1 s=READY pgs=383 gs=1 cs=0 l=1 c_cookie=2672831151590994762 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0750007720 tx=0x7f075000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2517cdc5672df34a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f07237fe640 1 -- 10.20.193.28:0/628657812 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0750022070 con 0x564cbc975440 2026-02-25T11:48:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f07237fe640 1 -- 10.20.193.28:0/628657812 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0750023040 con 0x564cbc975440 2026-02-25T11:48:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.616+0000 7f07237fe640 1 -- 10.20.193.28:0/628657812 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f075001da60 con 0x564cbc975440 2026-02-25T11:48:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.617+0000 7f0768280e00 1 -- 10.20.193.28:0/628657812 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc975440 msgr2=0x564cbc974150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.617+0000 7f0768280e00 1 --2- 10.20.193.28:0/628657812 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc975440 0x564cbc974150 secure :-1 s=READY pgs=383 gs=1 cs=0 l=1 c_cookie=2672831151590994762 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0750007720 tx=0x7f075000b000 comp rx=0 tx=0).stop 2026-02-25T11:48:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.617+0000 7f0768280e00 1 -- 10.20.193.28:0/628657812 shutdown_connections 2026-02-25T11:48:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.617+0000 7f0768280e00 1 --2- 10.20.193.28:0/628657812 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc9ba0e0 0x564cbc9b2750 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.617+0000 7f0768280e00 1 --2- 10.20.193.28:0/628657812 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc975440 0x564cbc974150 unknown :-1 s=CLOSED pgs=383 gs=1 cs=0 l=1 c_cookie=2672831151590994762 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.617+0000 7f0768280e00 1 --2- 10.20.193.28:0/628657812 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc974b30 0x564cbc973c10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.617+0000 7f0768280e00 1 -- 10.20.193.28:0/628657812 >> 10.20.193.28:0/628657812 conn(0x564cbc8937a0 msgr2=0x564cbc973500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.617+0000 7f0768280e00 1 -- 10.20.193.28:0/628657812 shutdown_connections 2026-02-25T11:48:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.617+0000 7f0768280e00 1 -- 10.20.193.28:0/628657812 wait complete. 2026-02-25T11:48:25.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.617+0000 7f0768280e00 1 -- start start 2026-02-25T11:48:25.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.617+0000 7f0768280e00 1 Processor -- start 2026-02-25T11:48:25.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f0768280e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 0x564cbc996370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f0768280e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc975440 0x564cbc9968b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f0768280e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc9ba0e0 0x564cbca1f600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f0768280e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564cbca1fb40 con 0x564cbc974b30 2026-02-25T11:48:25.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f0768280e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564cbca1fcb0 con 0x564cbc9ba0e0 2026-02-25T11:48:25.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f0768280e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564cbca1fe70 con 0x564cbc975440 2026-02-25T11:48:25.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f075f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 0x564cbc996370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f075f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 0x564cbc996370 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44016/0 (socket says 10.20.193.28:44016) 2026-02-25T11:48:25.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f075f7fe640 1 -- 10.20.193.28:0/4256127285 learned_addr learned my addr 10.20.193.28:0/4256127285 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:25.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f075ffff640 1 --2- 10.20.193.28:0/4256127285 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc9ba0e0 0x564cbca1f600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f075effd640 1 --2- 10.20.193.28:0/4256127285 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc975440 0x564cbc9968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f075f7fe640 1 -- 10.20.193.28:0/4256127285 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc975440 msgr2=0x564cbc9968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f075f7fe640 1 --2- 10.20.193.28:0/4256127285 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc975440 0x564cbc9968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f075f7fe640 1 -- 10.20.193.28:0/4256127285 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc9ba0e0 msgr2=0x564cbca1f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:25.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f075f7fe640 1 --2- 10.20.193.28:0/4256127285 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc9ba0e0 0x564cbca1f600 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f075f7fe640 1 -- 10.20.193.28:0/4256127285 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cbc810b60 con 0x564cbc974b30 2026-02-25T11:48:25.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f075f7fe640 1 --2- 10.20.193.28:0/4256127285 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 0x564cbc996370 secure :-1 s=READY pgs=384 gs=1 cs=0 l=1 c_cookie=12621511991719091446 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f074c0cd880 tx=0x7f074c0cda30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af289e48eabfdcf6 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:25.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.618+0000 7f07217fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:25.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f07217fa640 1 -- 10.20.193.28:0/4256127285 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f074c0dc180 con 0x564cbc974b30 2026-02-25T11:48:25.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f07217fa640 1 -- 10.20.193.28:0/4256127285 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f074c0d7050 con 0x564cbc974b30 2026-02-25T11:48:25.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f07217fa640 1 -- 10.20.193.28:0/4256127285 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f074c0cf290 con 0x564cbc974b30 2026-02-25T11:48:25.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f0768280e00 1 -- 10.20.193.28:0/4256127285 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0750011440 con 0x564cbc974b30 2026-02-25T11:48:25.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f07217fa640 1 -- 10.20.193.28:0/4256127285 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f074c0cf3f0 con 0x564cbc974b30 2026-02-25T11:48:25.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f07217fa640 10 client.15525 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:48:25.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f0768280e00 10 client.15525 _ll_register_callbacks cb 0x564cbc801a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:25.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f0768280e00 1 -- 10.20.193.28:0/4256127285 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f074c048e80 con 0x564cbc974b30 2026-02-25T11:48:25.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f07217fa640 1 -- 10.20.193.28:0/4256127285 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f074c0d1cb0 con 0x564cbc974b30 2026-02-25T11:48:25.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f0768280e00 10 client.15525 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:25.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f0768280e00 1 -- 10.20.193.28:0/4256127285 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x564cbc8046f0 con 0x564cbc974b30 2026-02-25T11:48:25.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f07217fa640 1 -- 10.20.193.28:0/4256127285 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f074c0d2ca0 con 0x564cbc974b30 2026-02-25T11:48:25.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f0768280e00 10 client.15525 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:25.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f0768280e00 10 client.15525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:25.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f0768280e00 20 client.15525 populate_metadata read hostname 'trial028' 2026-02-25T11:48:25.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f0768280e00 1 --2- 10.20.193.28:0/4256127285 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564cbca67670 0x564cbca87b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:25.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f0768280e00 4 client.15525 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:48:25.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.619+0000 7f0768280e00 1 -- 10.20.193.28:0/4256127285 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564cbc513f40 con 0x564cbca67670 2026-02-25T11:48:25.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.620+0000 7f075effd640 1 --2- 10.20.193.28:0/4256127285 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564cbca67670 0x564cbca87b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:25.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.620+0000 7f075effd640 1 --2- 10.20.193.28:0/4256127285 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564cbca67670 0x564cbca87b20 crc :-1 s=READY pgs=28 gs=4 cs=0 l=0 c_cookie=15214987108825653048 s_cookie=2936214789256040134 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3267e29bbb35f38 server_cookie=28bf87cb49dacec6 in_seq=0 out_seq=0 2026-02-25T11:48:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.620+0000 7f07217fa640 10 client.15525 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.620+0000 7f07217fa640 1 -- 10.20.193.28:0/4256127285 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564cbc513f40 con 0x564cbca67670 2026-02-25T11:48:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.620+0000 7f07217fa640 10 client.15525 handle_command_reply: tid=0 2026-02-25T11:48:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.620+0000 7f0768280e00 1 client.15525 shutdown 2026-02-25T11:48:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0768280e00 1 -- 10.20.193.28:0/4256127285 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 msgr2=0x564cbc996370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0768280e00 1 --2- 10.20.193.28:0/4256127285 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 0x564cbc996370 secure :-1 s=READY pgs=384 gs=1 cs=0 l=1 c_cookie=12621511991719091446 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f074c0cd880 tx=0x7f074c0cda30 comp rx=0 tx=0).stop 2026-02-25T11:48:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0768280e00 1 -- 10.20.193.28:0/4256127285 shutdown_connections 2026-02-25T11:48:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0768280e00 1 --2- 10.20.193.28:0/4256127285 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564cbca67670 0x564cbca87b20 crc :-1 s=READY pgs=28 gs=4 cs=0 l=0 c_cookie=15214987108825653048 s_cookie=2936214789256040134 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0768280e00 1 --2- 10.20.193.28:0/4256127285 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564cbc9ba0e0 0x564cbca1f600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0768280e00 1 --2- 10.20.193.28:0/4256127285 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564cbc975440 0x564cbc9968b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0768280e00 1 --2- 10.20.193.28:0/4256127285 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564cbc974b30 0x564cbc996370 unknown :-1 s=CLOSED pgs=384 gs=1 cs=0 l=1 c_cookie=12621511991719091446 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f07217fa640 0 client.15525 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0768280e00 1 -- 10.20.193.28:0/4256127285 >> 10.20.193.28:0/4256127285 conn(0x564cbc8937a0 msgr2=0x564cbc973c10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0768280e00 1 -- 10.20.193.28:0/4256127285 shutdown_connections 2026-02-25T11:48:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0768280e00 1 -- 10.20.193.28:0/4256127285 wait complete. 2026-02-25T11:48:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0768280e00 20 client.15525 trim_cache size 0 max 16384 2026-02-25T11:48:25.630 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:48:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0764ffc640 1 -- 10.20.193.28:0/569770411 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07340b57f0 msgr2=0x7f07340d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0764ffc640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07340b57f0 0x7f07340d5ca0 secure :-1 s=READY pgs=259 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f076012a040 tx=0x7f07500073d0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0764ffc640 1 -- 10.20.193.28:0/569770411 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f076010b360 msgr2=0x7f07601236a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.621+0000 7f0764ffc640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f076010b360 0x7f07601236a0 secure :-1 s=READY pgs=380 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f074c0098a0 tx=0x7f074c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.622+0000 7f0764ffc640 1 -- 10.20.193.28:0/569770411 shutdown_connections 2026-02-25T11:48:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.622+0000 7f0764ffc640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07340b57f0 0x7f07340d5ca0 unknown :-1 s=CLOSED pgs=259 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.622+0000 7f0764ffc640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0760111370 0x7f0760128fb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.622+0000 7f0764ffc640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f076010bd50 0x7f0760123be0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.622+0000 7f0764ffc640 1 --2- 10.20.193.28:0/569770411 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f076010b360 0x7f07601236a0 unknown :-1 s=CLOSED pgs=380 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:25.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.622+0000 7f0764ffc640 1 -- 10.20.193.28:0/569770411 >> 10.20.193.28:0/569770411 conn(0x7f07600e8e70 msgr2=0x7f0760109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:25.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.622+0000 7f0764ffc640 1 -- 10.20.193.28:0/569770411 shutdown_connections 2026-02-25T11:48:25.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:25.622+0000 7f0764ffc640 1 -- 10.20.193.28:0/569770411 wait complete. 2026-02-25T11:48:26.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:26 trial167 ceph-mon[43392]: pgmap v232: 97 pgs: 97 active+clean; 674 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 2.2 KiB/s rd, 4 op/s; 141 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:26.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:26 trial119 ceph-mon[42710]: pgmap v232: 97 pgs: 97 active+clean; 674 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 2.2 KiB/s rd, 4 op/s; 141 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:26.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:26 trial028 ceph-mon[39224]: pgmap v232: 97 pgs: 97 active+clean; 674 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 2.2 KiB/s rd, 4 op/s; 141 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:26.726 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:48:26.726 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:48:26.912 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:48:27.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.091+0000 7feba9c29640 1 -- 10.20.193.28:0/2189590919 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feba41135c0 msgr2=0x7feba4173f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.091+0000 7feba9c29640 1 --2- 10.20.193.28:0/2189590919 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feba41135c0 0x7feba4173f90 secure :-1 s=READY pgs=385 gs=1 cs=0 l=1 c_cookie=18431875341069942415 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feba000aa30 tx=0x7feba0030270 comp rx=0 tx=0).stop 2026-02-25T11:48:27.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.091+0000 7feba9c29640 1 -- 10.20.193.28:0/2189590919 shutdown_connections 2026-02-25T11:48:27.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.091+0000 7feba9c29640 1 --2- 10.20.193.28:0/2189590919 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feba41135c0 0x7feba4173f90 unknown :-1 s=CLOSED pgs=385 gs=1 cs=0 l=1 c_cookie=18431875341069942415 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.091+0000 7feba9c29640 1 --2- 10.20.193.28:0/2189590919 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7feba410ddc0 0x7feba4112e10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.091+0000 7feba9c29640 1 --2- 10.20.193.28:0/2189590919 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7feba410d3d0 0x7feba410d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.091+0000 7feba9c29640 1 -- 10.20.193.28:0/2189590919 >> 10.20.193.28:0/2189590919 conn(0x7feba40eaca0 msgr2=0x7feba410b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:27.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.091+0000 7feba9c29640 1 -- 10.20.193.28:0/2189590919 shutdown_connections 2026-02-25T11:48:27.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.091+0000 7feba9c29640 1 -- 10.20.193.28:0/2189590919 wait complete. 2026-02-25T11:48:27.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.092+0000 7feba9c29640 1 Processor -- start 2026-02-25T11:48:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.092+0000 7feba9c29640 1 -- start start 2026-02-25T11:48:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feba9c29640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7feba410d3d0 0x7feba40f5960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feba9c29640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feba410ddc0 0x7feba4173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feba9c29640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7feba41135c0 0x7feba40f6060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feba9c29640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7feba40fb270 con 0x7feba410ddc0 2026-02-25T11:48:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feba9c29640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7feba40fb3e0 con 0x7feba41135c0 2026-02-25T11:48:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feba9c29640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7feba40fb5a0 con 0x7feba410d3d0 2026-02-25T11:48:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feb9bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feba410ddc0 0x7feba4173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feb9bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feba410ddc0 0x7feba4173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58106/0 (socket says 10.20.193.28:58106) 2026-02-25T11:48:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feb9bfff640 1 -- 10.20.193.28:0/1153363324 learned_addr learned my addr 10.20.193.28:0/1153363324 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feba9428640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7feba41135c0 0x7feba40f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feba8c27640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7feba410d3d0 0x7feba40f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feb9bfff640 1 -- 10.20.193.28:0/1153363324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7feba410d3d0 msgr2=0x7feba40f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feb9bfff640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7feba410d3d0 0x7feba40f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feb9bfff640 1 -- 10.20.193.28:0/1153363324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7feba41135c0 msgr2=0x7feba40f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feb9bfff640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7feba41135c0 0x7feba40f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feb9bfff640 1 -- 10.20.193.28:0/1153363324 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba000a650 con 0x7feba410ddc0 2026-02-25T11:48:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.093+0000 7feb9bfff640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feba410ddc0 0x7feba4173e90 secure :-1 s=READY pgs=386 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feb8c005db0 tx=0x7feb8c00ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.094+0000 7feb99ffb640 1 -- 10.20.193.28:0/1153363324 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feb8c004500 con 0x7feba410ddc0 2026-02-25T11:48:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.094+0000 7feb99ffb640 1 -- 10.20.193.28:0/1153363324 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7feb8c004d10 con 0x7feba410ddc0 2026-02-25T11:48:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.094+0000 7feb99ffb640 1 -- 10.20.193.28:0/1153363324 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feb8c005280 con 0x7feba410ddc0 2026-02-25T11:48:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.094+0000 7feba9c29640 1 -- 10.20.193.28:0/1153363324 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba40fb880 con 0x7feba410ddc0 2026-02-25T11:48:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.094+0000 7feba9c29640 1 -- 10.20.193.28:0/1153363324 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba40fbd00 con 0x7feba410ddc0 2026-02-25T11:48:27.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.096+0000 7feb99ffb640 1 -- 10.20.193.28:0/1153363324 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7feb8c005410 con 0x7feba410ddc0 2026-02-25T11:48:27.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.098+0000 7feb99ffb640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7feb800b57e0 0x7feb800d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.098+0000 7feb99ffb640 1 -- 10.20.193.28:0/1153363324 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7feb8c0e6e10 con 0x7feba410ddc0 2026-02-25T11:48:27.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.098+0000 7feba8c27640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7feb800b57e0 0x7feb800d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.099+0000 7feba8c27640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7feb800b57e0 0x7feb800d5c90 secure :-1 s=READY pgs=260 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feb94004740 tx=0x7feb94004420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:27.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.099+0000 7febaceade00 1 Processor -- start 2026-02-25T11:48:27.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.099+0000 7febaceade00 1 -- start start 2026-02-25T11:48:27.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7febaceade00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2414750 0x55afd2414b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7febaceade00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2415060 0x55afd2459610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7febaceade00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 0x55afd2479f30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7febaceade00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55afd232ad80 con 0x55afd2459b50 2026-02-25T11:48:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7febaceade00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55afd2326ab0 con 0x55afd2414750 2026-02-25T11:48:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7febaceade00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55afd2320a50 con 0x55afd2415060 2026-02-25T11:48:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7feba9428640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 0x55afd2479f30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7feba9428640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 0x55afd2479f30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58120/0 (socket says 10.20.193.28:58120) 2026-02-25T11:48:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7feba9428640 1 -- 10.20.193.28:0/1547854457 learned_addr learned my addr 10.20.193.28:0/1547854457 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7feba8c27640 1 --2- 10.20.193.28:0/1547854457 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2414750 0x55afd2414b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7feba9428640 1 -- 10.20.193.28:0/1547854457 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2415060 msgr2=0x55afd2459610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7feba9428640 1 --2- 10.20.193.28:0/1547854457 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2415060 0x55afd2459610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7feba9428640 1 -- 10.20.193.28:0/1547854457 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2414750 msgr2=0x55afd2414b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7feba9428640 1 --2- 10.20.193.28:0/1547854457 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2414750 0x55afd2414b20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7feba9428640 1 -- 10.20.193.28:0/1547854457 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd22cc120 con 0x55afd2459b50 2026-02-25T11:48:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7feba8c27640 1 --2- 10.20.193.28:0/1547854457 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2414750 0x55afd2414b20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:48:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.100+0000 7feba9428640 1 --2- 10.20.193.28:0/1547854457 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 0x55afd2479f30 secure :-1 s=READY pgs=387 gs=1 cs=0 l=1 c_cookie=11106650405866990692 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feba000a420 tx=0x7feba0006d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a22bfb2d6266c64 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.101+0000 7feb714e4640 1 -- 10.20.193.28:0/1547854457 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feba003e040 con 0x55afd2459b50 2026-02-25T11:48:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.101+0000 7feb714e4640 1 -- 10.20.193.28:0/1547854457 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7feba0030ad0 con 0x55afd2459b50 2026-02-25T11:48:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.101+0000 7feb714e4640 1 -- 10.20.193.28:0/1547854457 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feba00050c0 con 0x55afd2459b50 2026-02-25T11:48:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.101+0000 7febaceade00 1 -- 10.20.193.28:0/1547854457 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 msgr2=0x55afd2479f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.101+0000 7febaceade00 1 --2- 10.20.193.28:0/1547854457 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 0x55afd2479f30 secure :-1 s=READY pgs=387 gs=1 cs=0 l=1 c_cookie=11106650405866990692 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feba000a420 tx=0x7feba0006d60 comp rx=0 tx=0).stop 2026-02-25T11:48:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.101+0000 7febaceade00 1 -- 10.20.193.28:0/1547854457 shutdown_connections 2026-02-25T11:48:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.101+0000 7febaceade00 1 --2- 10.20.193.28:0/1547854457 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 0x55afd2479f30 unknown :-1 s=CLOSED pgs=387 gs=1 cs=0 l=1 c_cookie=11106650405866990692 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.101+0000 7febaceade00 1 --2- 10.20.193.28:0/1547854457 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2415060 0x55afd2459610 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.101+0000 7febaceade00 1 --2- 10.20.193.28:0/1547854457 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2414750 0x55afd2414b20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.101+0000 7febaceade00 1 -- 10.20.193.28:0/1547854457 >> 10.20.193.28:0/1547854457 conn(0x55afd23f28f0 msgr2=0x55afd2412cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.101+0000 7febaceade00 1 -- 10.20.193.28:0/1547854457 shutdown_connections 2026-02-25T11:48:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.101+0000 7febaceade00 1 -- 10.20.193.28:0/1547854457 wait complete. 2026-02-25T11:48:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.103+0000 7febaceade00 1 -- start start 2026-02-25T11:48:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.103+0000 7febaceade00 1 Processor -- start 2026-02-25T11:48:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.103+0000 7febaceade00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2414750 0x55afd24714f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7febaceade00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2415060 0x55afd2471a30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7febaceade00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2459b50 0x55afd2476a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7febaceade00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55afd2476f60 con 0x55afd2414750 2026-02-25T11:48:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7febaceade00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55afd24770d0 con 0x55afd2459b50 2026-02-25T11:48:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7febaceade00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55afd2477290 con 0x55afd2415060 2026-02-25T11:48:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7feba8c27640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2414750 0x55afd24714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7feb9bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2415060 0x55afd2471a30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7feba8c27640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2414750 0x55afd24714f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58122/0 (socket says 10.20.193.28:58122) 2026-02-25T11:48:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7feba8c27640 1 -- 10.20.193.28:0/521049577 learned_addr learned my addr 10.20.193.28:0/521049577 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7feba9428640 1 --2- 10.20.193.28:0/521049577 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2459b50 0x55afd2476a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7feba8c27640 1 -- 10.20.193.28:0/521049577 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2415060 msgr2=0x55afd2471a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7feba8c27640 1 --2- 10.20.193.28:0/521049577 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2415060 0x55afd2471a30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7feba8c27640 1 -- 10.20.193.28:0/521049577 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2459b50 msgr2=0x55afd2476a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7feba8c27640 1 --2- 10.20.193.28:0/521049577 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2459b50 0x55afd2476a20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.104+0000 7feba8c27640 1 -- 10.20.193.28:0/521049577 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd22a4810 con 0x55afd2414750 2026-02-25T11:48:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.105+0000 7feba8c27640 1 --2- 10.20.193.28:0/521049577 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2414750 0x55afd24714f0 secure :-1 s=READY pgs=388 gs=1 cs=0 l=1 c_cookie=17083373210740390959 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feb940040e0 tx=0x7feb94004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed14556c6f9afc2f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.105+0000 7feb627fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.105+0000 7feb627fc640 1 -- 10.20.193.28:0/521049577 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feb9401e860 con 0x55afd2414750 2026-02-25T11:48:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.105+0000 7feb627fc640 1 -- 10.20.193.28:0/521049577 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7feb9401ee80 con 0x55afd2414750 2026-02-25T11:48:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.105+0000 7feb627fc640 1 -- 10.20.193.28:0/521049577 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feb9401d3c0 con 0x55afd2414750 2026-02-25T11:48:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.105+0000 7febaceade00 1 -- 10.20.193.28:0/521049577 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba4174800 con 0x55afd2414750 2026-02-25T11:48:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.105+0000 7feb627fc640 1 -- 10.20.193.28:0/521049577 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7feb9401d550 con 0x55afd2414750 2026-02-25T11:48:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.105+0000 7febaceade00 10 client.15537 _ll_register_callbacks cb 0x55afd22a1e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.105+0000 7febaceade00 1 -- 10.20.193.28:0/521049577 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55afd2078530 con 0x55afd2414750 2026-02-25T11:48:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.105+0000 7feb627fc640 1 -- 10.20.193.28:0/521049577 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7feb9401e9c0 con 0x55afd2414750 2026-02-25T11:48:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.105+0000 7febaceade00 10 client.15537 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.105+0000 7febaceade00 1 -- 10.20.193.28:0/521049577 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55afd22b64f0 con 0x55afd2414750 2026-02-25T11:48:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.106+0000 7feb627fc640 1 -- 10.20.193.28:0/521049577 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7feb94019070 con 0x55afd2414750 2026-02-25T11:48:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.106+0000 7febaceade00 10 client.15537 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:27.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.106+0000 7febaceade00 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:27.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.106+0000 7febaceade00 20 client.15537 populate_metadata read hostname 'trial028' 2026-02-25T11:48:27.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.106+0000 7febaceade00 1 --2- 10.20.193.28:0/521049577 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55afd2456e50 0x55afd24572f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.106+0000 7febaceade00 4 client.15537 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:48:27.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.106+0000 7febaceade00 1 -- 10.20.193.28:0/521049577 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afd22cdb50 con 0x55afd2456e50 2026-02-25T11:48:27.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.106+0000 7feb9bfff640 1 --2- 10.20.193.28:0/521049577 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55afd2456e50 0x55afd24572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.107+0000 7feb9bfff640 1 --2- 10.20.193.28:0/521049577 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55afd2456e50 0x55afd24572f0 crc :-1 s=READY pgs=29 gs=4 cs=0 l=0 c_cookie=7836152607970832630 s_cookie=3113476763126553839 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cbf9b477bda30f6 server_cookie=2b354a9ec2975cef in_seq=0 out_seq=0 2026-02-25T11:48:27.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.107+0000 7feb627fc640 10 client.15537 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:27.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.108+0000 7feb627fc640 1 -- 10.20.193.28:0/521049577 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x7feb8c0c4e90 con 0x55afd2456e50 2026-02-25T11:48:27.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.108+0000 7feb627fc640 10 client.15537 handle_command_reply: tid=0 2026-02-25T11:48:27.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.108+0000 7febaceade00 1 client.15537 shutdown 2026-02-25T11:48:27.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.108+0000 7febaceade00 1 -- 10.20.193.28:0/521049577 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2414750 msgr2=0x55afd24714f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.108+0000 7febaceade00 1 --2- 10.20.193.28:0/521049577 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2414750 0x55afd24714f0 secure :-1 s=READY pgs=388 gs=1 cs=0 l=1 c_cookie=17083373210740390959 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feb940040e0 tx=0x7feb94004770 comp rx=0 tx=0).stop 2026-02-25T11:48:27.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.108+0000 7febaceade00 1 -- 10.20.193.28:0/521049577 shutdown_connections 2026-02-25T11:48:27.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.108+0000 7febaceade00 1 --2- 10.20.193.28:0/521049577 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55afd2456e50 0x55afd24572f0 crc :-1 s=READY pgs=29 gs=4 cs=0 l=0 c_cookie=7836152607970832630 s_cookie=3113476763126553839 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.108+0000 7febaceade00 1 --2- 10.20.193.28:0/521049577 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2459b50 0x55afd2476a20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.108+0000 7febaceade00 1 --2- 10.20.193.28:0/521049577 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2415060 0x55afd2471a30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.108+0000 7febaceade00 1 --2- 10.20.193.28:0/521049577 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2414750 0x55afd24714f0 unknown :-1 s=CLOSED pgs=388 gs=1 cs=0 l=1 c_cookie=17083373210740390959 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.108+0000 7feb627fc640 0 client.15537 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:27.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.108+0000 7febaceade00 1 -- 10.20.193.28:0/521049577 >> 10.20.193.28:0/521049577 conn(0x55afd23f28f0 msgr2=0x55afd2412cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:27.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.108+0000 7febaceade00 1 -- 10.20.193.28:0/521049577 shutdown_connections 2026-02-25T11:48:27.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.109+0000 7febaceade00 1 -- 10.20.193.28:0/521049577 wait complete. 2026-02-25T11:48:27.116 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.109+0000 7febaceade00 20 client.15537 trim_cache size 0 max 16384 2026-02-25T11:48:27.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.118+0000 7febaceade00 1 Processor -- start 2026-02-25T11:48:27.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.118+0000 7febaceade00 1 -- start start 2026-02-25T11:48:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.118+0000 7febaceade00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2414750 0x55afd246e7a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.118+0000 7febaceade00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2415060 0x55afd246ece0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.118+0000 7febaceade00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 0x55afd2454740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.118+0000 7febaceade00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55afd231e680 con 0x55afd2459b50 2026-02-25T11:48:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.118+0000 7febaceade00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55afd1fde790 con 0x55afd2415060 2026-02-25T11:48:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.118+0000 7febaceade00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55afd246a650 con 0x55afd2414750 2026-02-25T11:48:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.118+0000 7feba9428640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 0x55afd2454740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.118+0000 7feba9428640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 0x55afd2454740 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58124/0 (socket says 10.20.193.28:58124) 2026-02-25T11:48:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.118+0000 7feba9428640 1 -- 10.20.193.28:0/4193214267 learned_addr learned my addr 10.20.193.28:0/4193214267 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.119+0000 7feb9bfff640 1 --2- 10.20.193.28:0/4193214267 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2415060 0x55afd246ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.119+0000 7feba8c27640 1 --2- 10.20.193.28:0/4193214267 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2414750 0x55afd246e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.119+0000 7feba9428640 1 -- 10.20.193.28:0/4193214267 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2414750 msgr2=0x55afd246e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.119+0000 7feba9428640 1 --2- 10.20.193.28:0/4193214267 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2414750 0x55afd246e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.119+0000 7feba9428640 1 -- 10.20.193.28:0/4193214267 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2415060 msgr2=0x55afd246ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.119+0000 7feba9428640 1 --2- 10.20.193.28:0/4193214267 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2415060 0x55afd246ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.119+0000 7feba9428640 1 -- 10.20.193.28:0/4193214267 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd23f2760 con 0x55afd2459b50 2026-02-25T11:48:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.119+0000 7feba8c27640 1 --2- 10.20.193.28:0/4193214267 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2414750 0x55afd246e7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:48:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.119+0000 7feba9428640 1 --2- 10.20.193.28:0/4193214267 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 0x55afd2454740 secure :-1 s=READY pgs=389 gs=1 cs=0 l=1 c_cookie=11320721875396958620 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feba000a980 tx=0x7feba0030d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1b4890e007999c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.119+0000 7feb70ce3640 1 -- 10.20.193.28:0/4193214267 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feba003e040 con 0x55afd2459b50 2026-02-25T11:48:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.119+0000 7feb70ce3640 1 -- 10.20.193.28:0/4193214267 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7feba0039690 con 0x55afd2459b50 2026-02-25T11:48:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.119+0000 7feb70ce3640 1 -- 10.20.193.28:0/4193214267 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feba0005570 con 0x55afd2459b50 2026-02-25T11:48:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.120+0000 7febaceade00 1 -- 10.20.193.28:0/4193214267 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 msgr2=0x55afd2454740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.120+0000 7febaceade00 1 --2- 10.20.193.28:0/4193214267 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 0x55afd2454740 secure :-1 s=READY pgs=389 gs=1 cs=0 l=1 c_cookie=11320721875396958620 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feba000a980 tx=0x7feba0030d10 comp rx=0 tx=0).stop 2026-02-25T11:48:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.120+0000 7febaceade00 1 -- 10.20.193.28:0/4193214267 shutdown_connections 2026-02-25T11:48:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.120+0000 7febaceade00 1 --2- 10.20.193.28:0/4193214267 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2459b50 0x55afd2454740 unknown :-1 s=CLOSED pgs=389 gs=1 cs=0 l=1 c_cookie=11320721875396958620 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.120+0000 7febaceade00 1 --2- 10.20.193.28:0/4193214267 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2415060 0x55afd246ece0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.120+0000 7febaceade00 1 --2- 10.20.193.28:0/4193214267 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2414750 0x55afd246e7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.120+0000 7febaceade00 1 -- 10.20.193.28:0/4193214267 >> 10.20.193.28:0/4193214267 conn(0x55afd23f28f0 msgr2=0x55afd245bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:27.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.120+0000 7febaceade00 1 -- 10.20.193.28:0/4193214267 shutdown_connections 2026-02-25T11:48:27.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.120+0000 7febaceade00 1 -- 10.20.193.28:0/4193214267 wait complete. 2026-02-25T11:48:27.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.120+0000 7febaceade00 1 -- start start 2026-02-25T11:48:27.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7febaceade00 1 Processor -- start 2026-02-25T11:48:27.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7febaceade00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2414750 0x55afd23f2cc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7febaceade00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2415060 0x55afd23f3200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7febaceade00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2459b50 0x55afd23f3740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7febaceade00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55afd231e680 con 0x55afd2415060 2026-02-25T11:48:27.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7febaceade00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55afd24c94e0 con 0x55afd2459b50 2026-02-25T11:48:27.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7febaceade00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55afd23f7d10 con 0x55afd2414750 2026-02-25T11:48:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7feb9bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2415060 0x55afd23f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7feb9bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2415060 0x55afd23f3200 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58140/0 (socket says 10.20.193.28:58140) 2026-02-25T11:48:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7feb9bfff640 1 -- 10.20.193.28:0/3813534081 learned_addr learned my addr 10.20.193.28:0/3813534081 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7feba8c27640 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2414750 0x55afd23f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7feba9428640 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2459b50 0x55afd23f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7feb9bfff640 1 -- 10.20.193.28:0/3813534081 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2414750 msgr2=0x55afd23f2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.121+0000 7feb9bfff640 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2414750 0x55afd23f2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7feb9bfff640 1 -- 10.20.193.28:0/3813534081 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2459b50 msgr2=0x55afd23f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7feb9bfff640 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2459b50 0x55afd23f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7feb9bfff640 1 -- 10.20.193.28:0/3813534081 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afd22b0f70 con 0x55afd2415060 2026-02-25T11:48:27.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7feba8c27640 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2414750 0x55afd23f2cc0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:48:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7feb9bfff640 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2415060 0x55afd23f3200 secure :-1 s=READY pgs=390 gs=2 cs=0 l=1 c_cookie=3709297731577266835 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feb8c0aa2a0 tx=0x7feb8c0e4c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337a12a258d34293 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7feb627fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7feb627fc640 1 -- 10.20.193.28:0/3813534081 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feb8c009940 con 0x55afd2415060 2026-02-25T11:48:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7feb627fc640 1 -- 10.20.193.28:0/3813534081 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7feb8c072e90 con 0x55afd2415060 2026-02-25T11:48:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7feb627fc640 1 -- 10.20.193.28:0/3813534081 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feb8c0ace50 con 0x55afd2415060 2026-02-25T11:48:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7febaceade00 1 -- 10.20.193.28:0/3813534081 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afd22cc120 con 0x55afd2415060 2026-02-25T11:48:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7feb627fc640 1 -- 10.20.193.28:0/3813534081 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7feb8c072130 con 0x55afd2415060 2026-02-25T11:48:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7feb627fc640 10 client.15543 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:48:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7febaceade00 10 client.15543 _ll_register_callbacks cb 0x55afd22a1e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7febaceade00 1 -- 10.20.193.28:0/3813534081 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feba0030ad0 con 0x55afd2415060 2026-02-25T11:48:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7feb627fc640 1 -- 10.20.193.28:0/3813534081 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7feb8c0ada30 con 0x55afd2415060 2026-02-25T11:48:27.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7febaceade00 10 client.15543 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.122+0000 7febaceade00 1 -- 10.20.193.28:0/3813534081 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55afd22a4b40 con 0x55afd2415060 2026-02-25T11:48:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.123+0000 7feb627fc640 1 -- 10.20.193.28:0/3813534081 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7feb8c0adcc0 con 0x55afd2415060 2026-02-25T11:48:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.123+0000 7febaceade00 10 client.15543 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.123+0000 7febaceade00 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.123+0000 7febaceade00 20 client.15543 populate_metadata read hostname 'trial028' 2026-02-25T11:48:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.123+0000 7febaceade00 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55afd25106a0 0x55afd2530b50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.123+0000 7febaceade00 4 client.15543 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:48:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.123+0000 7febaceade00 1 -- 10.20.193.28:0/3813534081 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55afd2181690 con 0x55afd25106a0 2026-02-25T11:48:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.123+0000 7feba8c27640 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55afd25106a0 0x55afd2530b50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.124+0000 7feba8c27640 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55afd25106a0 0x55afd2530b50 crc :-1 s=READY pgs=30 gs=4 cs=0 l=0 c_cookie=1047303472867893254 s_cookie=13184819725999368220 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e88c4f5b052e006 server_cookie=b6f9e3af43c1681c in_seq=0 out_seq=0 2026-02-25T11:48:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.124+0000 7feb627fc640 10 client.15543 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:27.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7feb627fc640 1 -- 10.20.193.28:0/3813534081 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55afd2181690 con 0x55afd25106a0 2026-02-25T11:48:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7feb627fc640 10 client.15543 handle_command_reply: tid=0 2026-02-25T11:48:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7febaceade00 1 client.15543 shutdown 2026-02-25T11:48:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7febaceade00 1 -- 10.20.193.28:0/3813534081 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2415060 msgr2=0x55afd23f3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7febaceade00 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2415060 0x55afd23f3200 secure :-1 s=READY pgs=390 gs=2 cs=0 l=1 c_cookie=3709297731577266835 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feb8c0aa2a0 tx=0x7feb8c0e4c90 comp rx=0 tx=0).stop 2026-02-25T11:48:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7febaceade00 1 -- 10.20.193.28:0/3813534081 shutdown_connections 2026-02-25T11:48:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7febaceade00 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55afd25106a0 0x55afd2530b50 crc :-1 s=READY pgs=30 gs=4 cs=0 l=0 c_cookie=1047303472867893254 s_cookie=13184819725999368220 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7febaceade00 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55afd2459b50 0x55afd23f3740 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7febaceade00 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55afd2415060 0x55afd23f3200 unknown :-1 s=CLOSED pgs=390 gs=2 cs=0 l=1 c_cookie=3709297731577266835 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7febaceade00 1 --2- 10.20.193.28:0/3813534081 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55afd2414750 0x55afd23f2cc0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7feb627fc640 0 client.15543 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7febaceade00 1 -- 10.20.193.28:0/3813534081 >> 10.20.193.28:0/3813534081 conn(0x55afd23f28f0 msgr2=0x55afd246d790 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:27.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7febaceade00 1 -- 10.20.193.28:0/3813534081 shutdown_connections 2026-02-25T11:48:27.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7febaceade00 1 -- 10.20.193.28:0/3813534081 wait complete. 2026-02-25T11:48:27.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.125+0000 7febaceade00 20 client.15543 trim_cache size 0 max 16384 2026-02-25T11:48:27.134 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:48:27.134 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:48:27.134 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "feaa75b0-9031-483c-96ae-588afa74d2df", 2026-02-25T11:48:27.134 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:48:27.134 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:48:27.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.126+0000 7feba9c29640 1 -- 10.20.193.28:0/1153363324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7feb800b57e0 msgr2=0x7feb800d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.126+0000 7feba9c29640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7feb800b57e0 0x7feb800d5c90 secure :-1 s=READY pgs=260 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feb94004740 tx=0x7feb94004420 comp rx=0 tx=0).stop 2026-02-25T11:48:27.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.126+0000 7feba9c29640 1 -- 10.20.193.28:0/1153363324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feba410ddc0 msgr2=0x7feba4173e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.126+0000 7feba9c29640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feba410ddc0 0x7feba4173e90 secure :-1 s=READY pgs=386 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feb8c005db0 tx=0x7feb8c00ddf0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.126+0000 7feba9c29640 1 -- 10.20.193.28:0/1153363324 shutdown_connections 2026-02-25T11:48:27.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.126+0000 7feba9c29640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7feb800b57e0 0x7feb800d5c90 unknown :-1 s=CLOSED pgs=260 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.126+0000 7feba9c29640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7feba41135c0 0x7feba40f6060 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.126+0000 7feba9c29640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7feba410ddc0 0x7feba4173e90 unknown :-1 s=CLOSED pgs=386 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.126+0000 7feba9c29640 1 --2- 10.20.193.28:0/1153363324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7feba410d3d0 0x7feba40f5960 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.126+0000 7feba9c29640 1 -- 10.20.193.28:0/1153363324 >> 10.20.193.28:0/1153363324 conn(0x7feba40eaca0 msgr2=0x7feba410b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:27.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.126+0000 7feba9c29640 1 -- 10.20.193.28:0/1153363324 shutdown_connections 2026-02-25T11:48:27.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.126+0000 7feba9c29640 1 -- 10.20.193.28:0/1153363324 wait complete. 2026-02-25T11:48:27.228 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:48:27.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:27 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:48:27.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:27 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:48:27.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:27 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:48:27.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:27 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:48:27.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:27 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:48:27.414 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:48:27.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:27 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:48:27.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:27 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:48:27.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:27 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:48:27.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:27 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:48:27.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:27 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:48:27.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:27 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:48:27.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:27 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:48:27.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:27 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:48:27.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:27 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:48:27.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:27 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:48:27.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.587+0000 7f8c688c7640 1 -- 10.20.193.28:0/2340137978 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c601140a0 msgr2=0x7f8c60114470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.587+0000 7f8c688c7640 1 --2- 10.20.193.28:0/2340137978 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c601140a0 0x7f8c60114470 secure :-1 s=READY pgs=391 gs=1 cs=0 l=1 c_cookie=3194988074163278566 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c58009950 tx=0x7f8c5802f1d0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.587+0000 7f8c688c7640 1 -- 10.20.193.28:0/2340137978 shutdown_connections 2026-02-25T11:48:27.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.587+0000 7f8c688c7640 1 --2- 10.20.193.28:0/2340137978 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8c60119e40 0x7f8c6017a810 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.587+0000 7f8c688c7640 1 --2- 10.20.193.28:0/2340137978 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8c60114a90 0x7f8c60119690 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.587+0000 7f8c688c7640 1 --2- 10.20.193.28:0/2340137978 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c601140a0 0x7f8c60114470 unknown :-1 s=CLOSED pgs=391 gs=1 cs=0 l=1 c_cookie=3194988074163278566 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.587+0000 7f8c688c7640 1 -- 10.20.193.28:0/2340137978 >> 10.20.193.28:0/2340137978 conn(0x7f8c60064d70 msgr2=0x7f8c60065170 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:27.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.587+0000 7f8c688c7640 1 -- 10.20.193.28:0/2340137978 shutdown_connections 2026-02-25T11:48:27.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.587+0000 7f8c688c7640 1 -- 10.20.193.28:0/2340137978 wait complete. 2026-02-25T11:48:27.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.588+0000 7f8c688c7640 1 Processor -- start 2026-02-25T11:48:27.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.588+0000 7f8c688c7640 1 -- start start 2026-02-25T11:48:27.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.588+0000 7f8c688c7640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8c601140a0 0x7f8c60122670 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.588+0000 7f8c688c7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c60114a90 0x7f8c60122bb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.588+0000 7f8c688c7640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8c60119e40 0x7f8c60127f80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.588+0000 7f8c688c7640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601285a0 con 0x7f8c60114a90 2026-02-25T11:48:27.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.588+0000 7f8c688c7640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8c60128710 con 0x7f8c601140a0 2026-02-25T11:48:27.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.588+0000 7f8c688c7640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601288d0 con 0x7f8c60119e40 2026-02-25T11:48:27.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.588+0000 7f8c65e3b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c60114a90 0x7f8c60122bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.588+0000 7f8c65e3b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c60114a90 0x7f8c60122bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58172/0 (socket says 10.20.193.28:58172) 2026-02-25T11:48:27.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c65e3b640 1 -- 10.20.193.28:0/2725262121 learned_addr learned my addr 10.20.193.28:0/2725262121 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:27.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c6663c640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8c601140a0 0x7f8c60122670 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c66e3d640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8c60119e40 0x7f8c60127f80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c65e3b640 1 -- 10.20.193.28:0/2725262121 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8c60119e40 msgr2=0x7f8c60127f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8c60119e40 0x7f8c60127f80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c65e3b640 1 -- 10.20.193.28:0/2725262121 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8c601140a0 msgr2=0x7f8c60122670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8c601140a0 0x7f8c60122670 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c65e3b640 1 -- 10.20.193.28:0/2725262121 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c58009570 con 0x7f8c60114a90 2026-02-25T11:48:27.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c6663c640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8c601140a0 0x7f8c60122670 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:48:27.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c60114a90 0x7f8c60122bb0 secure :-1 s=READY pgs=392 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c4c009480 tx=0x7f8c4c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:27.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c4b7fe640 1 -- 10.20.193.28:0/2725262121 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c4c00cd40 con 0x7f8c60114a90 2026-02-25T11:48:27.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c4b7fe640 1 -- 10.20.193.28:0/2725262121 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8c4c004d10 con 0x7f8c60114a90 2026-02-25T11:48:27.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c4b7fe640 1 -- 10.20.193.28:0/2725262121 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c4c0106b0 con 0x7f8c60114a90 2026-02-25T11:48:27.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c688c7640 1 -- 10.20.193.28:0/2725262121 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c60128bb0 con 0x7f8c60114a90 2026-02-25T11:48:27.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.589+0000 7f8c688c7640 1 -- 10.20.193.28:0/2725262121 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c601692b0 con 0x7f8c60114a90 2026-02-25T11:48:27.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.592+0000 7f8c4b7fe640 1 -- 10.20.193.28:0/2725262121 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f8c4c01f060 con 0x7f8c60114a90 2026-02-25T11:48:27.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.594+0000 7f8c4b7fe640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8c3c0b57f0 0x7f8c3c0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.594+0000 7f8c4b7fe640 1 -- 10.20.193.28:0/2725262121 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8c4c014070 con 0x7f8c60114a90 2026-02-25T11:48:27.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.594+0000 7f8c6663c640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8c3c0b57f0 0x7f8c3c0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.594+0000 7f8c6a8c2e00 1 Processor -- start 2026-02-25T11:48:27.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c6a8c2e00 1 -- start start 2026-02-25T11:48:27.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c6a8c2e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc175140 0x55d4bc175510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c6a8c2e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1ba1b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c6a8c2e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc1daad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c6a8c2e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d4bc08b9b0 con 0x55d4bc175a50 2026-02-25T11:48:27.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c6a8c2e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d4bc0876e0 con 0x55d4bc175140 2026-02-25T11:48:27.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c6a8c2e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d4bc081840 con 0x55d4bc1ba6f0 2026-02-25T11:48:27.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c6663c640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8c3c0b57f0 0x7f8c3c0d5ca0 secure :-1 s=READY pgs=261 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c580023d0 tx=0x7f8c58031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:27.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c6663c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc175140 0x55d4bc175510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c65e3b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1ba1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c66e3d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc1daad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c66e3d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc1daad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:39130/0 (socket says 10.20.193.28:39130) 2026-02-25T11:48:27.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c66e3d640 1 -- 10.20.193.28:0/2407375244 learned_addr learned my addr 10.20.193.28:0/2407375244 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:27.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c65e3b640 1 -- 10.20.193.28:0/2407375244 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc1ba6f0 msgr2=0x55d4bc1daad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/2407375244 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc1daad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c65e3b640 1 -- 10.20.193.28:0/2407375244 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc175140 msgr2=0x55d4bc175510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/2407375244 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc175140 0x55d4bc175510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.595+0000 7f8c65e3b640 1 -- 10.20.193.28:0/2407375244 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4bc02cd80 con 0x55d4bc175a50 2026-02-25T11:48:27.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.596+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/2407375244 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1ba1b0 secure :-1 s=READY pgs=393 gs=1 cs=0 l=1 c_cookie=2886612006327159559 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c4c0e48e0 tx=0x7f8c4c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280f4e5369820b07 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:27.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.596+0000 7f8c48cf3640 1 -- 10.20.193.28:0/2407375244 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c4c0e4ea0 con 0x55d4bc175a50 2026-02-25T11:48:27.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.596+0000 7f8c48cf3640 1 -- 10.20.193.28:0/2407375244 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8c4c0ad1e0 con 0x55d4bc175a50 2026-02-25T11:48:27.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.596+0000 7f8c48cf3640 1 -- 10.20.193.28:0/2407375244 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c4c0b2e50 con 0x55d4bc175a50 2026-02-25T11:48:27.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.596+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/2407375244 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 msgr2=0x55d4bc1ba1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.596+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/2407375244 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1ba1b0 secure :-1 s=READY pgs=393 gs=1 cs=0 l=1 c_cookie=2886612006327159559 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c4c0e48e0 tx=0x7f8c4c00cf90 comp rx=0 tx=0).stop 2026-02-25T11:48:27.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.596+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/2407375244 shutdown_connections 2026-02-25T11:48:27.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.596+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/2407375244 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc1daad0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.596+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/2407375244 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1ba1b0 unknown :-1 s=CLOSED pgs=393 gs=1 cs=0 l=1 c_cookie=2886612006327159559 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.596+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/2407375244 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc175140 0x55d4bc175510 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.596+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/2407375244 >> 10.20.193.28:0/2407375244 conn(0x55d4bc094d40 msgr2=0x55d4bc173a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.597+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/2407375244 shutdown_connections 2026-02-25T11:48:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.597+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/2407375244 wait complete. 2026-02-25T11:48:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.598+0000 7f8c6a8c2e00 1 -- start start 2026-02-25T11:48:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.598+0000 7f8c6a8c2e00 1 Processor -- start 2026-02-25T11:48:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.598+0000 7f8c6a8c2e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175140 0x55d4bc1d23c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.598+0000 7f8c6a8c2e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1d2900 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.598+0000 7f8c6a8c2e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc1d78f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.598+0000 7f8c6a8c2e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d4bc1d7e30 con 0x55d4bc175a50 2026-02-25T11:48:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.598+0000 7f8c6a8c2e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d4bc1d7fa0 con 0x55d4bc1ba6f0 2026-02-25T11:48:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.598+0000 7f8c6a8c2e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d4bc1d8160 con 0x55d4bc175140 2026-02-25T11:48:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c65e3b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c65e3b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1d2900 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58186/0 (socket says 10.20.193.28:58186) 2026-02-25T11:48:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c65e3b640 1 -- 10.20.193.28:0/1331763635 learned_addr learned my addr 10.20.193.28:0/1331763635 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c66e3d640 1 --2- 10.20.193.28:0/1331763635 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc1d78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c6663c640 1 --2- 10.20.193.28:0/1331763635 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175140 0x55d4bc1d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c65e3b640 1 -- 10.20.193.28:0/1331763635 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175140 msgr2=0x55d4bc1d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/1331763635 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175140 0x55d4bc1d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c65e3b640 1 -- 10.20.193.28:0/1331763635 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 msgr2=0x55d4bc1d78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/1331763635 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc1d78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c65e3b640 1 -- 10.20.193.28:0/1331763635 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4bc005350 con 0x55d4bc175a50 2026-02-25T11:48:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/1331763635 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1d2900 secure :-1 s=READY pgs=394 gs=1 cs=0 l=1 c_cookie=6393184541921063499 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c4c0e48b0 tx=0x7f8c4c0e4d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b92777e4b9324b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c0ffff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c0ffff640 1 -- 10.20.193.28:0/1331763635 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c4c0c8070 con 0x55d4bc175a50 2026-02-25T11:48:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c0ffff640 1 -- 10.20.193.28:0/1331763635 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8c4c00b2c0 con 0x55d4bc175a50 2026-02-25T11:48:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c0ffff640 1 -- 10.20.193.28:0/1331763635 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c4c0e4ea0 con 0x55d4bc175a50 2026-02-25T11:48:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/1331763635 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c601692b0 con 0x55d4bc175a50 2026-02-25T11:48:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c0ffff640 1 -- 10.20.193.28:0/1331763635 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8c4c0acee0 con 0x55d4bc175a50 2026-02-25T11:48:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c0ffff640 10 client.15555 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:48:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.599+0000 7f8c6a8c2e00 10 client.15555 _ll_register_callbacks cb 0x55d4bc0029a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.600+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/1331763635 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d4bbd176f0 con 0x55d4bc175a50 2026-02-25T11:48:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.600+0000 7f8c0ffff640 1 -- 10.20.193.28:0/1331763635 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8c4c0adb30 con 0x55d4bc175a50 2026-02-25T11:48:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.600+0000 7f8c6a8c2e00 10 client.15555 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.600+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/1331763635 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55d4bc017150 con 0x55d4bc175a50 2026-02-25T11:48:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.600+0000 7f8c0ffff640 1 -- 10.20.193.28:0/1331763635 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f8c4c0b7de0 con 0x55d4bc175a50 2026-02-25T11:48:27.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.600+0000 7f8c6a8c2e00 10 client.15555 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:27.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.600+0000 7f8c6a8c2e00 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:27.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.600+0000 7f8c6a8c2e00 20 client.15555 populate_metadata read hostname 'trial028' 2026-02-25T11:48:27.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.600+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/1331763635 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d4bc1b79a0 0x55d4bc1b7e40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.600+0000 7f8c6a8c2e00 4 client.15555 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:48:27.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.600+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/1331763635 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4bc053910 con 0x55d4bc1b79a0 2026-02-25T11:48:27.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.600+0000 7f8c6663c640 1 --2- 10.20.193.28:0/1331763635 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d4bc1b79a0 0x55d4bc1b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.601+0000 7f8c6663c640 1 --2- 10.20.193.28:0/1331763635 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d4bc1b79a0 0x55d4bc1b7e40 crc :-1 s=READY pgs=31 gs=4 cs=0 l=0 c_cookie=4905915781366171081 s_cookie=2189057591252762016 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44155069165045c9 server_cookie=1e611847f77e3da0 in_seq=0 out_seq=0 2026-02-25T11:48:27.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.601+0000 7f8c0ffff640 10 client.15555 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:27.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.601+0000 7f8c0ffff640 1 -- 10.20.193.28:0/1331763635 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55d4bc053910 con 0x55d4bc1b79a0 2026-02-25T11:48:27.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.601+0000 7f8c0ffff640 10 client.15555 handle_command_reply: tid=0 2026-02-25T11:48:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.602+0000 7f8c6a8c2e00 1 client.15555 shutdown 2026-02-25T11:48:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.602+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/1331763635 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 msgr2=0x55d4bc1d2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.602+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/1331763635 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1d2900 secure :-1 s=READY pgs=394 gs=1 cs=0 l=1 c_cookie=6393184541921063499 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c4c0e48b0 tx=0x7f8c4c0e4d90 comp rx=0 tx=0).stop 2026-02-25T11:48:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.602+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/1331763635 shutdown_connections 2026-02-25T11:48:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.602+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/1331763635 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d4bc1b79a0 0x55d4bc1b7e40 crc :-1 s=READY pgs=31 gs=4 cs=0 l=0 c_cookie=4905915781366171081 s_cookie=2189057591252762016 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.602+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/1331763635 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc1d78f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.602+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/1331763635 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1d2900 unknown :-1 s=CLOSED pgs=394 gs=1 cs=0 l=1 c_cookie=6393184541921063499 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.602+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/1331763635 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175140 0x55d4bc1d23c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.602+0000 7f8c0ffff640 0 client.15555 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.602+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/1331763635 >> 10.20.193.28:0/1331763635 conn(0x55d4bc094d40 msgr2=0x55d4bc173230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.602+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/1331763635 shutdown_connections 2026-02-25T11:48:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.602+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/1331763635 wait complete. 2026-02-25T11:48:27.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.602+0000 7f8c6a8c2e00 20 client.15555 trim_cache size 0 max 16384 2026-02-25T11:48:27.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.611+0000 7f8c6a8c2e00 1 Processor -- start 2026-02-25T11:48:27.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.611+0000 7f8c6a8c2e00 1 -- start start 2026-02-25T11:48:27.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.611+0000 7f8c6a8c2e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175140 0x55d4bc1cfca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.611+0000 7f8c6a8c2e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1b8720 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.611+0000 7f8c6a8c2e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc1b8c60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.611+0000 7f8c6a8c2e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d4bc07f470 con 0x55d4bc175a50 2026-02-25T11:48:27.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.611+0000 7f8c6a8c2e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d4bbd3f700 con 0x55d4bc1ba6f0 2026-02-25T11:48:27.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.611+0000 7f8c6a8c2e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d4bc0e49d0 con 0x55d4bc175140 2026-02-25T11:48:27.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.612+0000 7f8c6663c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175140 0x55d4bc1cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.612+0000 7f8c66e3d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc1b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.612+0000 7f8c6663c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175140 0x55d4bc1cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:39140/0 (socket says 10.20.193.28:39140) 2026-02-25T11:48:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.612+0000 7f8c6663c640 1 -- 10.20.193.28:0/4199942025 learned_addr learned my addr 10.20.193.28:0/4199942025 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.612+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/4199942025 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.612+0000 7f8c6663c640 1 -- 10.20.193.28:0/4199942025 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 msgr2=0x55d4bc1b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.612+0000 7f8c6663c640 1 --2- 10.20.193.28:0/4199942025 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc1b8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.612+0000 7f8c6663c640 1 -- 10.20.193.28:0/4199942025 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 msgr2=0x55d4bc1b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.612+0000 7f8c6663c640 1 --2- 10.20.193.28:0/4199942025 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.612+0000 7f8c6663c640 1 -- 10.20.193.28:0/4199942025 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4bc1c4630 con 0x55d4bc175140 2026-02-25T11:48:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.612+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/4199942025 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1b8720 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:48:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.612+0000 7f8c6663c640 1 --2- 10.20.193.28:0/4199942025 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175140 0x55d4bc1cfca0 secure :-1 s=READY pgs=118 gs=1 cs=0 l=1 c_cookie=4362933110222049157 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c58004820 tx=0x7f8c58040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c8c408c45934f85 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.613+0000 7f8c2e4e6640 1 -- 10.20.193.28:0/4199942025 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c580028c0 con 0x55d4bc175140 2026-02-25T11:48:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.613+0000 7f8c2e4e6640 1 -- 10.20.193.28:0/4199942025 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8c580561b0 con 0x55d4bc175140 2026-02-25T11:48:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.613+0000 7f8c2e4e6640 1 -- 10.20.193.28:0/4199942025 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c58056610 con 0x55d4bc175140 2026-02-25T11:48:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.613+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/4199942025 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175140 msgr2=0x55d4bc1cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.613+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/4199942025 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175140 0x55d4bc1cfca0 secure :-1 s=READY pgs=118 gs=1 cs=0 l=1 c_cookie=4362933110222049157 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c58004820 tx=0x7f8c58040000 comp rx=0 tx=0).stop 2026-02-25T11:48:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.613+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/4199942025 shutdown_connections 2026-02-25T11:48:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.613+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/4199942025 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc1b8c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.613+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/4199942025 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175a50 0x55d4bc1b8720 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.613+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/4199942025 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175140 0x55d4bc1cfca0 unknown :-1 s=CLOSED pgs=118 gs=1 cs=0 l=1 c_cookie=4362933110222049157 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.613+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/4199942025 >> 10.20.193.28:0/4199942025 conn(0x55d4bc094d40 msgr2=0x55d4bc1ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.613+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/4199942025 shutdown_connections 2026-02-25T11:48:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.613+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/4199942025 wait complete. 2026-02-25T11:48:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.614+0000 7f8c6a8c2e00 1 -- start start 2026-02-25T11:48:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.614+0000 7f8c6a8c2e00 1 Processor -- start 2026-02-25T11:48:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.614+0000 7f8c6a8c2e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175140 0x55d4bc158dc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.614+0000 7f8c6a8c2e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175a50 0x55d4bc156400 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.614+0000 7f8c6a8c2e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc156940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.614+0000 7f8c6a8c2e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d4bc156fa0 con 0x55d4bc175140 2026-02-25T11:48:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.614+0000 7f8c6a8c2e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d4bc157110 con 0x55d4bc1ba6f0 2026-02-25T11:48:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.614+0000 7f8c6a8c2e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d4bc1572d0 con 0x55d4bc175a50 2026-02-25T11:48:27.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.614+0000 7f8c6663c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175140 0x55d4bc158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.614+0000 7f8c65e3b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175a50 0x55d4bc156400 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.614+0000 7f8c66e3d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc156940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.614+0000 7f8c6663c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175140 0x55d4bc158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58204/0 (socket says 10.20.193.28:58204) 2026-02-25T11:48:27.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.614+0000 7f8c6663c640 1 -- 10.20.193.28:0/474918363 learned_addr learned my addr 10.20.193.28:0/474918363 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:27.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c6663c640 1 -- 10.20.193.28:0/474918363 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175a50 msgr2=0x55d4bc156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c6663c640 1 --2- 10.20.193.28:0/474918363 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175a50 0x55d4bc156400 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c6663c640 1 -- 10.20.193.28:0/474918363 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 msgr2=0x55d4bc156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:27.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c6663c640 1 --2- 10.20.193.28:0/474918363 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc156940 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c6663c640 1 -- 10.20.193.28:0/474918363 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5804c040 con 0x55d4bc175140 2026-02-25T11:48:27.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/474918363 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175a50 0x55d4bc156400 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:48:27.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c66e3d640 1 --2- 10.20.193.28:0/474918363 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc156940 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:48:27.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c6663c640 1 --2- 10.20.193.28:0/474918363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175140 0x55d4bc158dc0 secure :-1 s=READY pgs=395 gs=1 cs=0 l=1 c_cookie=5124491341036513797 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c5802f6e0 tx=0x7f8c580387e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471dd9b9550bea05 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:27.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c0ffff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:27.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c0ffff640 1 -- 10.20.193.28:0/474918363 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c58056610 con 0x55d4bc175140 2026-02-25T11:48:27.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c0ffff640 1 -- 10.20.193.28:0/474918363 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8c580388f0 con 0x55d4bc175140 2026-02-25T11:48:27.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c0ffff640 1 -- 10.20.193.28:0/474918363 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8c58054430 con 0x55d4bc175140 2026-02-25T11:48:27.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/474918363 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4bc017150 con 0x55d4bc175140 2026-02-25T11:48:27.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.615+0000 7f8c0ffff640 1 -- 10.20.193.28:0/474918363 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8c58054600 con 0x55d4bc175140 2026-02-25T11:48:27.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c0ffff640 10 client.15561 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:48:27.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c6a8c2e00 10 client.15561 _ll_register_callbacks cb 0x55d4bc0029a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:27.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/474918363 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c0ad1e0 con 0x55d4bc175140 2026-02-25T11:48:27.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c0ffff640 1 -- 10.20.193.28:0/474918363 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8c5802f9f0 con 0x55d4bc175140 2026-02-25T11:48:27.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c6a8c2e00 10 client.15561 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:27.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/474918363 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55d4bc011bd0 con 0x55d4bc175140 2026-02-25T11:48:27.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c0ffff640 1 -- 10.20.193.28:0/474918363 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f8c5802fb90 con 0x55d4bc175140 2026-02-25T11:48:27.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c6a8c2e00 10 client.15561 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:27.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c6a8c2e00 10 client.15561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:27.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c6a8c2e00 20 client.15561 populate_metadata read hostname 'trial028' 2026-02-25T11:48:27.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/474918363 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d4bc267db0 0x55d4bc288260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:27.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c6a8c2e00 4 client.15561 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:48:27.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/474918363 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d4bbd14f40 con 0x55d4bc267db0 2026-02-25T11:48:27.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.616+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/474918363 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d4bc267db0 0x55d4bc288260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:27.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c65e3b640 1 --2- 10.20.193.28:0/474918363 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d4bc267db0 0x55d4bc288260 crc :-1 s=READY pgs=32 gs=4 cs=0 l=0 c_cookie=7478141095871409759 s_cookie=16097612719958318130 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67c7b1b4afaa6e5f server_cookie=df66357ae0ecdc32 in_seq=0 out_seq=0 2026-02-25T11:48:27.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c0ffff640 10 client.15561 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:27.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c0ffff640 1 -- 10.20.193.28:0/474918363 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x55d4bbd14f40 con 0x55d4bc267db0 2026-02-25T11:48:27.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c0ffff640 10 client.15561 handle_command_reply: tid=0 2026-02-25T11:48:27.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c6a8c2e00 1 client.15561 shutdown 2026-02-25T11:48:27.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/474918363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175140 msgr2=0x55d4bc158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/474918363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175140 0x55d4bc158dc0 secure :-1 s=READY pgs=395 gs=1 cs=0 l=1 c_cookie=5124491341036513797 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c5802f6e0 tx=0x7f8c580387e0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/474918363 shutdown_connections 2026-02-25T11:48:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/474918363 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d4bc267db0 0x55d4bc288260 crc :-1 s=READY pgs=32 gs=4 cs=0 l=0 c_cookie=7478141095871409759 s_cookie=16097612719958318130 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/474918363 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d4bc1ba6f0 0x55d4bc156940 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c0ffff640 0 client.15561 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/474918363 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d4bc175a50 0x55d4bc156400 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c6a8c2e00 1 --2- 10.20.193.28:0/474918363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d4bc175140 0x55d4bc158dc0 unknown :-1 s=CLOSED pgs=395 gs=1 cs=0 l=1 c_cookie=5124491341036513797 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.617+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/474918363 >> 10.20.193.28:0/474918363 conn(0x55d4bc094d40 msgr2=0x55d4bc1ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/474918363 shutdown_connections 2026-02-25T11:48:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c6a8c2e00 1 -- 10.20.193.28:0/474918363 wait complete. 2026-02-25T11:48:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c6a8c2e00 20 client.15561 trim_cache size 0 max 16384 2026-02-25T11:48:27.628 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:48:27.628 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:48:27.628 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:48:27.628 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:48:27.628 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:48:27.628 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:48:27.628 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:48:27.628 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:48:27.629 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:48:27.629 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:48:27.629 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:48:27.629 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:48:27.629 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:48:27.629 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:48:27.629 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:48:27.629 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:48:27.629 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:48:27.629 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:48:27.630 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:48:27.630 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:48:27.630 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:48:27.630 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:48:27.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c688c7640 1 -- 10.20.193.28:0/2725262121 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8c3c0b57f0 msgr2=0x7f8c3c0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c688c7640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8c3c0b57f0 0x7f8c3c0d5ca0 secure :-1 s=READY pgs=261 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c580023d0 tx=0x7f8c58031040 comp rx=0 tx=0).stop 2026-02-25T11:48:27.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c688c7640 1 -- 10.20.193.28:0/2725262121 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c60114a90 msgr2=0x7f8c60122bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:27.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c688c7640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c60114a90 0x7f8c60122bb0 secure :-1 s=READY pgs=392 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8c4c009480 tx=0x7f8c4c00ee60 comp rx=0 tx=0).stop 2026-02-25T11:48:27.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c688c7640 1 -- 10.20.193.28:0/2725262121 shutdown_connections 2026-02-25T11:48:27.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c688c7640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8c3c0b57f0 0x7f8c3c0d5ca0 unknown :-1 s=CLOSED pgs=261 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c688c7640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8c60119e40 0x7f8c60127f80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c688c7640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8c60114a90 0x7f8c60122bb0 unknown :-1 s=CLOSED pgs=392 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c688c7640 1 --2- 10.20.193.28:0/2725262121 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8c601140a0 0x7f8c60122670 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:27.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c688c7640 1 -- 10.20.193.28:0/2725262121 >> 10.20.193.28:0/2725262121 conn(0x7f8c60064d70 msgr2=0x7f8c60117c90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:27.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c688c7640 1 -- 10.20.193.28:0/2725262121 shutdown_connections 2026-02-25T11:48:27.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:27.618+0000 7f8c688c7640 1 -- 10.20.193.28:0/2725262121 wait complete. 2026-02-25T11:48:27.720 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:48:27.721 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:48:27.906 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:48:28.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.085+0000 7f993d9e7640 1 -- 10.20.193.28:0/1913340297 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99381140a0 msgr2=0x7f9938114470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.085+0000 7f993d9e7640 1 --2- 10.20.193.28:0/1913340297 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99381140a0 0x7f9938114470 secure :-1 s=READY pgs=396 gs=1 cs=0 l=1 c_cookie=15755713495733659743 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9928009950 tx=0x7f992802f190 comp rx=0 tx=0).stop 2026-02-25T11:48:28.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.085+0000 7f993d9e7640 1 -- 10.20.193.28:0/1913340297 shutdown_connections 2026-02-25T11:48:28.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.085+0000 7f993d9e7640 1 --2- 10.20.193.28:0/1913340297 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9938119d30 0x7f993817a700 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.085+0000 7f993d9e7640 1 --2- 10.20.193.28:0/1913340297 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9938114a90 0x7f9938119580 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.085+0000 7f993d9e7640 1 --2- 10.20.193.28:0/1913340297 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f99381140a0 0x7f9938114470 unknown :-1 s=CLOSED pgs=396 gs=1 cs=0 l=1 c_cookie=15755713495733659743 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.085+0000 7f993d9e7640 1 -- 10.20.193.28:0/1913340297 >> 10.20.193.28:0/1913340297 conn(0x7f9938064c40 msgr2=0x7f9938065040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:28.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.085+0000 7f993d9e7640 1 -- 10.20.193.28:0/1913340297 shutdown_connections 2026-02-25T11:48:28.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.085+0000 7f993d9e7640 1 -- 10.20.193.28:0/1913340297 wait complete. 2026-02-25T11:48:28.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f993d9e7640 1 Processor -- start 2026-02-25T11:48:28.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f993d9e7640 1 -- start start 2026-02-25T11:48:28.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f993d9e7640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99381140a0 0x7f9938122550 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f993d9e7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9938114a90 0x7f9938122a90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f993d9e7640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9938119d30 0x7f9938127e60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f993d9e7640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f99381283f0 con 0x7f9938114a90 2026-02-25T11:48:28.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f993d9e7640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9938128560 con 0x7f99381140a0 2026-02-25T11:48:28.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f993d9e7640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9938128720 con 0x7f9938119d30 2026-02-25T11:48:28.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f992ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9938114a90 0x7f9938122a90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f992ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9938114a90 0x7f9938122a90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58222/0 (socket says 10.20.193.28:58222) 2026-02-25T11:48:28.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f992ffff640 1 -- 10.20.193.28:0/2158115965 learned_addr learned my addr 10.20.193.28:0/2158115965 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:28.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f993d1e6640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9938119d30 0x7f9938127e60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f993c9e5640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99381140a0 0x7f9938122550 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f992ffff640 1 -- 10.20.193.28:0/2158115965 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9938119d30 msgr2=0x7f9938127e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f992ffff640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9938119d30 0x7f9938127e60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.086+0000 7f992ffff640 1 -- 10.20.193.28:0/2158115965 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99381140a0 msgr2=0x7f9938122550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.087+0000 7f992ffff640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99381140a0 0x7f9938122550 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.087+0000 7f992ffff640 1 -- 10.20.193.28:0/2158115965 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9928009570 con 0x7f9938114a90 2026-02-25T11:48:28.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.087+0000 7f992ffff640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9938114a90 0x7f9938122a90 secure :-1 s=READY pgs=397 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9920005db0 tx=0x7f992000ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:28.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.087+0000 7f992dffb640 1 -- 10.20.193.28:0/2158115965 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9920004500 con 0x7f9938114a90 2026-02-25T11:48:28.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.087+0000 7f992dffb640 1 -- 10.20.193.28:0/2158115965 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9920004d10 con 0x7f9938114a90 2026-02-25T11:48:28.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.087+0000 7f992dffb640 1 -- 10.20.193.28:0/2158115965 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9920005280 con 0x7f9938114a90 2026-02-25T11:48:28.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.087+0000 7f993d9e7640 1 -- 10.20.193.28:0/2158115965 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9938128a00 con 0x7f9938114a90 2026-02-25T11:48:28.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.087+0000 7f993d9e7640 1 -- 10.20.193.28:0/2158115965 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9938128ce0 con 0x7f9938114a90 2026-02-25T11:48:28.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.088+0000 7f993d9e7640 1 -- 10.20.193.28:0/2158115965 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9938114e80 con 0x7f9938114a90 2026-02-25T11:48:28.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.089+0000 7f992dffb640 1 -- 10.20.193.28:0/2158115965 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f9920005410 con 0x7f9938114a90 2026-02-25T11:48:28.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.091+0000 7f992dffb640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f99140b57f0 0x7f99140d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.091+0000 7f992dffb640 1 -- 10.20.193.28:0/2158115965 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f99200e7160 con 0x7f9938114a90 2026-02-25T11:48:28.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.091+0000 7f992dffb640 1 -- 10.20.193.28:0/2158115965 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f99200e75e0 con 0x7f9938114a90 2026-02-25T11:48:28.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.092+0000 7f993c9e5640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f99140b57f0 0x7f99140d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.092+0000 7f993c9e5640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f99140b57f0 0x7f99140d5ca0 secure :-1 s=READY pgs=262 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f99280023d0 tx=0x7f9928031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:28.234 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.232+0000 7f993d9e7640 1 -- 10.20.193.28:0/2158115965 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99381236b0 con 0x7f9938114a90 2026-02-25T11:48:28.235 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:28 trial028 ceph-mon[39224]: pgmap v233: 97 pgs: 97 active+clean; 706 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 6.1 KiB/s rd, 3.8 KiB/s wr, 6 op/s; 114 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:28.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.233+0000 7f992dffb640 1 -- 10.20.193.28:0/2158115965 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f99200a99d0 con 0x7f9938114a90 2026-02-25T11:48:28.236 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:48:28.236 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:48:28.237 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:48:28.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.234+0000 7f993d9e7640 1 -- 10.20.193.28:0/2158115965 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f99140b57f0 msgr2=0x7f99140d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.234+0000 7f993d9e7640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f99140b57f0 0x7f99140d5ca0 secure :-1 s=READY pgs=262 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f99280023d0 tx=0x7f9928031040 comp rx=0 tx=0).stop 2026-02-25T11:48:28.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.234+0000 7f993d9e7640 1 -- 10.20.193.28:0/2158115965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9938114a90 msgr2=0x7f9938122a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.234+0000 7f993d9e7640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9938114a90 0x7f9938122a90 secure :-1 s=READY pgs=397 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9920005db0 tx=0x7f992000ddf0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.234+0000 7f993d9e7640 1 -- 10.20.193.28:0/2158115965 shutdown_connections 2026-02-25T11:48:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.234+0000 7f993d9e7640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f99140b57f0 0x7f99140d5ca0 unknown :-1 s=CLOSED pgs=262 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.234+0000 7f993d9e7640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9938119d30 0x7f9938127e60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.234+0000 7f993d9e7640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9938114a90 0x7f9938122a90 unknown :-1 s=CLOSED pgs=397 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.234+0000 7f993d9e7640 1 --2- 10.20.193.28:0/2158115965 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f99381140a0 0x7f9938122550 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.234+0000 7f993d9e7640 1 -- 10.20.193.28:0/2158115965 >> 10.20.193.28:0/2158115965 conn(0x7f9938064c40 msgr2=0x7f9938115b00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.234+0000 7f993d9e7640 1 -- 10.20.193.28:0/2158115965 shutdown_connections 2026-02-25T11:48:28.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.234+0000 7f993d9e7640 1 -- 10.20.193.28:0/2158115965 wait complete. 2026-02-25T11:48:28.334 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:48:28.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:28 trial167 ceph-mon[43392]: pgmap v233: 97 pgs: 97 active+clean; 706 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 6.1 KiB/s rd, 3.8 KiB/s wr, 6 op/s; 114 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:28.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:28 trial119 ceph-mon[42710]: pgmap v233: 97 pgs: 97 active+clean; 706 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 6.1 KiB/s rd, 3.8 KiB/s wr, 6 op/s; 114 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:28.518 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:48:28.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.698+0000 7ff7f72e4640 1 -- 10.20.193.28:0/1781364148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff7f0110040 msgr2=0x7ff7f0115090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.698+0000 7ff7f72e4640 1 --2- 10.20.193.28:0/1781364148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff7f0110040 0x7ff7f0115090 secure :-1 s=READY pgs=398 gs=1 cs=0 l=1 c_cookie=4115877360887264463 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff7e4009950 tx=0x7ff7e402f190 comp rx=0 tx=0).stop 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.698+0000 7ff7f72e4640 1 -- 10.20.193.28:0/1781364148 shutdown_connections 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.698+0000 7ff7f72e4640 1 --2- 10.20.193.28:0/1781364148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff7f0115840 0x7ff7f0176210 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.698+0000 7ff7f72e4640 1 --2- 10.20.193.28:0/1781364148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff7f0110040 0x7ff7f0115090 unknown :-1 s=CLOSED pgs=398 gs=1 cs=0 l=1 c_cookie=4115877360887264463 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.698+0000 7ff7f72e4640 1 --2- 10.20.193.28:0/1781364148 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff7f010f650 0x7ff7f010fa20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.698+0000 7ff7f72e4640 1 -- 10.20.193.28:0/1781364148 >> 10.20.193.28:0/1781364148 conn(0x7ff7f00ecf00 msgr2=0x7ff7f010d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.698+0000 7ff7f72e4640 1 -- 10.20.193.28:0/1781364148 shutdown_connections 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.698+0000 7ff7f72e4640 1 -- 10.20.193.28:0/1781364148 wait complete. 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.699+0000 7ff7f72e4640 1 Processor -- start 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.699+0000 7ff7f72e4640 1 -- start start 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.699+0000 7ff7f72e4640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff7f010f650 0x7ff7f00f5990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.699+0000 7ff7f72e4640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff7f0110040 0x7ff7f0176110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.699+0000 7ff7f72e4640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff7f0115840 0x7ff7f00f6090 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.699+0000 7ff7f72e4640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f00fb2a0 con 0x7ff7f010f650 2026-02-25T11:48:28.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.699+0000 7ff7f72e4640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f00fb410 con 0x7ff7f0110040 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.699+0000 7ff7f72e4640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f00fb5d0 con 0x7ff7f0115840 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.699+0000 7ff7f62e2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff7f010f650 0x7ff7f00f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7f62e2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff7f010f650 0x7ff7f00f5990 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58236/0 (socket says 10.20.193.28:58236) 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7f62e2640 1 -- 10.20.193.28:0/993402145 learned_addr learned my addr 10.20.193.28:0/993402145 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7f5ae1640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff7f0110040 0x7ff7f0176110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7f6ae3640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff7f0115840 0x7ff7f00f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7f62e2640 1 -- 10.20.193.28:0/993402145 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff7f0115840 msgr2=0x7ff7f00f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff7f0115840 0x7ff7f00f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7f62e2640 1 -- 10.20.193.28:0/993402145 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff7f0110040 msgr2=0x7ff7f0176110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff7f0110040 0x7ff7f0176110 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7f62e2640 1 -- 10.20.193.28:0/993402145 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e4009570 con 0x7ff7f010f650 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7f5ae1640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff7f0110040 0x7ff7f0176110 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff7f010f650 0x7ff7f00f5990 secure :-1 s=READY pgs=399 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff7e000d950 tx=0x7ff7e000de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7d77fe640 1 -- 10.20.193.28:0/993402145 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff7e0004540 con 0x7ff7f010f650 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7d77fe640 1 -- 10.20.193.28:0/993402145 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff7e0009ce0 con 0x7ff7f010f650 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7f72e4640 1 -- 10.20.193.28:0/993402145 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f00fb8b0 con 0x7ff7f010f650 2026-02-25T11:48:28.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7d77fe640 1 -- 10.20.193.28:0/993402145 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff7e00106e0 con 0x7ff7f010f650 2026-02-25T11:48:28.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.700+0000 7ff7f72e4640 1 -- 10.20.193.28:0/993402145 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f00fbdd0 con 0x7ff7f010f650 2026-02-25T11:48:28.706 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.703+0000 7ff7d77fe640 1 -- 10.20.193.28:0/993402145 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7ff7e0010870 con 0x7ff7f010f650 2026-02-25T11:48:28.707 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.705+0000 7ff7d77fe640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff7cc0b57f0 0x7ff7cc0d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.707 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.705+0000 7ff7d77fe640 1 -- 10.20.193.28:0/993402145 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ff7e00e6f40 con 0x7ff7f010f650 2026-02-25T11:48:28.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7fa568e00 1 Processor -- start 2026-02-25T11:48:28.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7f5ae1640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff7cc0b57f0 0x7ff7cc0d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.708 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7fa568e00 1 -- start start 2026-02-25T11:48:28.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7fa568e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab539f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7fa568e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab53a440 0x5632ab57eba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7fa568e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab57f0e0 0x5632ab59f4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7fa568e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5632ab44f8a0 con 0x5632ab539b30 2026-02-25T11:48:28.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7fa568e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5632ab44b5d0 con 0x5632ab53a440 2026-02-25T11:48:28.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7fa568e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5632ab445570 con 0x5632ab57f0e0 2026-02-25T11:48:28.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7f62e2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab539f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7f62e2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab539f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58242/0 (socket says 10.20.193.28:58242) 2026-02-25T11:48:28.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7f62e2640 1 -- 10.20.193.28:0/2796268019 learned_addr learned my addr 10.20.193.28:0/2796268019 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:28.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7f5ae1640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff7cc0b57f0 0x7ff7cc0d5ca0 secure :-1 s=READY pgs=263 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff7f00fc330 tx=0x7ff7e4031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:28.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7f6ae3640 1 --2- 10.20.193.28:0/2796268019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab57f0e0 0x5632ab59f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7f62e2640 1 -- 10.20.193.28:0/2796268019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab57f0e0 msgr2=0x5632ab59f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:28.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/2796268019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab57f0e0 0x5632ab59f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7f62e2640 1 -- 10.20.193.28:0/2796268019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab53a440 msgr2=0x5632ab57eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:28.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/2796268019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab53a440 0x5632ab57eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.706+0000 7ff7f62e2640 1 -- 10.20.193.28:0/2796268019 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ab3f0d10 con 0x5632ab539b30 2026-02-25T11:48:28.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.707+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/2796268019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab539f00 secure :-1 s=READY pgs=400 gs=1 cs=0 l=1 c_cookie=10327210387606897340 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff7e0005830 tx=0x7ff7e00aca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f519f21b20882bc server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:28.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.707+0000 7ff7d4b0a640 1 -- 10.20.193.28:0/2796268019 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff7e00b5460 con 0x5632ab539b30 2026-02-25T11:48:28.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.707+0000 7ff7d4b0a640 1 -- 10.20.193.28:0/2796268019 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff7e00b82a0 con 0x5632ab539b30 2026-02-25T11:48:28.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.707+0000 7ff7d4b0a640 1 -- 10.20.193.28:0/2796268019 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff7e00735c0 con 0x5632ab539b30 2026-02-25T11:48:28.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.707+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2796268019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 msgr2=0x5632ab539f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.717 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.707+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/2796268019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab539f00 secure :-1 s=READY pgs=400 gs=1 cs=0 l=1 c_cookie=10327210387606897340 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff7e0005830 tx=0x7ff7e00aca50 comp rx=0 tx=0).stop 2026-02-25T11:48:28.717 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.707+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2796268019 shutdown_connections 2026-02-25T11:48:28.717 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.707+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/2796268019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab57f0e0 0x5632ab59f4c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.717 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.707+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/2796268019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab53a440 0x5632ab57eba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.717 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.707+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/2796268019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab539f00 unknown :-1 s=CLOSED pgs=400 gs=1 cs=0 l=1 c_cookie=10327210387606897340 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.717 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.707+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2796268019 >> 10.20.193.28:0/2796268019 conn(0x5632ab4587a0 msgr2=0x5632ab538410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:28.717 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.708+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2796268019 shutdown_connections 2026-02-25T11:48:28.717 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.708+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2796268019 wait complete. 2026-02-25T11:48:28.717 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.709+0000 7ff7fa568e00 1 -- start start 2026-02-25T11:48:28.717 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.709+0000 7ff7fa568e00 1 Processor -- start 2026-02-25T11:48:28.717 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.709+0000 7ff7fa568e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab539b30 0x5632ab596ac0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.709+0000 7ff7fa568e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab53a440 0x5632ab597000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.709+0000 7ff7fa568e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab57f0e0 0x5632ab59bff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.709+0000 7ff7fa568e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5632ab59c530 con 0x5632ab53a440 2026-02-25T11:48:28.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.709+0000 7ff7fa568e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5632ab59c6a0 con 0x5632ab539b30 2026-02-25T11:48:28.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.709+0000 7ff7fa568e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5632ab59c860 con 0x5632ab57f0e0 2026-02-25T11:48:28.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7f5ae1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab53a440 0x5632ab597000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7f5ae1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab53a440 0x5632ab597000 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58250/0 (socket says 10.20.193.28:58250) 2026-02-25T11:48:28.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7f5ae1640 1 -- 10.20.193.28:0/2948276925 learned_addr learned my addr 10.20.193.28:0/2948276925 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:28.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7f6ae3640 1 --2- 10.20.193.28:0/2948276925 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab57f0e0 0x5632ab59bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/2948276925 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab539b30 0x5632ab596ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7f5ae1640 1 -- 10.20.193.28:0/2948276925 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab57f0e0 msgr2=0x5632ab59bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:28.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7f5ae1640 1 --2- 10.20.193.28:0/2948276925 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab57f0e0 0x5632ab59bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7f5ae1640 1 -- 10.20.193.28:0/2948276925 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab539b30 msgr2=0x5632ab596ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:28.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7f5ae1640 1 --2- 10.20.193.28:0/2948276925 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab539b30 0x5632ab596ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7f5ae1640 1 -- 10.20.193.28:0/2948276925 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ab3c93c0 con 0x5632ab53a440 2026-02-25T11:48:28.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7f5ae1640 1 --2- 10.20.193.28:0/2948276925 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab53a440 0x5632ab597000 secure :-1 s=READY pgs=401 gs=1 cs=0 l=1 c_cookie=4406579564419131476 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff7e4002fd0 tx=0x7ff7e4002a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2750c44bb2a854 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:28.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7a3fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:28.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7a3fff640 1 -- 10.20.193.28:0/2948276925 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff7e404b070 con 0x5632ab53a440 2026-02-25T11:48:28.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7a3fff640 1 -- 10.20.193.28:0/2948276925 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff7e402fe40 con 0x5632ab53a440 2026-02-25T11:48:28.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7a3fff640 1 -- 10.20.193.28:0/2948276925 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff7e403f070 con 0x5632ab53a440 2026-02-25T11:48:28.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2948276925 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f00fbdd0 con 0x5632ab53a440 2026-02-25T11:48:28.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7fa568e00 10 client.15579 _ll_register_callbacks cb 0x5632ab3c6a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:28.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.710+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2948276925 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5632ab0db6f0 con 0x5632ab53a440 2026-02-25T11:48:28.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.711+0000 7ff7a3fff640 1 -- 10.20.193.28:0/2948276925 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ff7e4039ae0 con 0x5632ab53a440 2026-02-25T11:48:28.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.711+0000 7ff7a3fff640 1 -- 10.20.193.28:0/2948276925 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff7e4039ca0 con 0x5632ab53a440 2026-02-25T11:48:28.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.711+0000 7ff7fa568e00 10 client.15579 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:28.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.711+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2948276925 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5632ab3db0e0 con 0x5632ab53a440 2026-02-25T11:48:28.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.711+0000 7ff7a3fff640 1 -- 10.20.193.28:0/2948276925 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7ff7e4057060 con 0x5632ab53a440 2026-02-25T11:48:28.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.711+0000 7ff7fa568e00 10 client.15579 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:28.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.711+0000 7ff7fa568e00 10 client.15579 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:48:28.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.711+0000 7ff7fa568e00 20 client.15579 populate_metadata read hostname 'trial028' 2026-02-25T11:48:28.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.711+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/2948276925 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5632ab57c140 0x5632ab57c5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.711+0000 7ff7fa568e00 4 client.15579 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:48:28.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.711+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2948276925 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632ab417fc0 con 0x5632ab57c140 2026-02-25T11:48:28.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.712+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/2948276925 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5632ab57c140 0x5632ab57c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.712+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/2948276925 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5632ab57c140 0x5632ab57c5e0 crc :-1 s=READY pgs=25 gs=4 cs=0 l=0 c_cookie=7123183816898940585 s_cookie=26787282844734448 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62daa1f0f4128ea9 server_cookie=5f2ae44e7553f0 in_seq=0 out_seq=0 2026-02-25T11:48:28.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.712+0000 7ff7a3fff640 10 client.15579 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:48:28.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.713+0000 7ff7a3fff640 1 -- 10.20.193.28:0/2948276925 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5632ab417fc0 con 0x5632ab57c140 2026-02-25T11:48:28.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.713+0000 7ff7a3fff640 10 client.15579 handle_command_reply: tid=0 2026-02-25T11:48:28.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.713+0000 7ff7fa568e00 1 client.15579 shutdown 2026-02-25T11:48:28.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.713+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2948276925 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab53a440 msgr2=0x5632ab597000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.713+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/2948276925 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab53a440 0x5632ab597000 secure :-1 s=READY pgs=401 gs=1 cs=0 l=1 c_cookie=4406579564419131476 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff7e4002fd0 tx=0x7ff7e4002a70 comp rx=0 tx=0).stop 2026-02-25T11:48:28.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.713+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2948276925 shutdown_connections 2026-02-25T11:48:28.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.713+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/2948276925 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5632ab57c140 0x5632ab57c5e0 crc :-1 s=READY pgs=25 gs=4 cs=0 l=0 c_cookie=7123183816898940585 s_cookie=26787282844734448 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.713+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/2948276925 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab57f0e0 0x5632ab59bff0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.713+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/2948276925 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab53a440 0x5632ab597000 unknown :-1 s=CLOSED pgs=401 gs=1 cs=0 l=1 c_cookie=4406579564419131476 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.713+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/2948276925 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab539b30 0x5632ab596ac0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.713+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2948276925 >> 10.20.193.28:0/2948276925 conn(0x5632ab4587a0 msgr2=0x5632ab537c20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:28.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.713+0000 7ff7a3fff640 0 client.15579 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:48:28.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.713+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2948276925 shutdown_connections 2026-02-25T11:48:28.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.714+0000 7ff7fa568e00 1 -- 10.20.193.28:0/2948276925 wait complete. 2026-02-25T11:48:28.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.714+0000 7ff7fa568e00 20 client.15579 trim_cache size 0 max 16384 2026-02-25T11:48:28.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.721+0000 7ff7fa568e00 1 Processor -- start 2026-02-25T11:48:28.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.721+0000 7ff7fa568e00 1 -- start start 2026-02-25T11:48:28.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.721+0000 7ff7fa568e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab578b10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.721+0000 7ff7fa568e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab53a440 0x5632ab579050 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.721+0000 7ff7fa568e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab57f0e0 0x5632ab579590 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.721+0000 7ff7fa568e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5632ab4431a0 con 0x5632ab539b30 2026-02-25T11:48:28.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.721+0000 7ff7fa568e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5632ab103700 con 0x5632ab57f0e0 2026-02-25T11:48:28.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.721+0000 7ff7fa568e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5632ab59c9a0 con 0x5632ab53a440 2026-02-25T11:48:28.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.722+0000 7ff7f62e2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab578b10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.722+0000 7ff7f62e2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab578b10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58256/0 (socket says 10.20.193.28:58256) 2026-02-25T11:48:28.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.722+0000 7ff7f62e2640 1 -- 10.20.193.28:0/3367234705 learned_addr learned my addr 10.20.193.28:0/3367234705 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:28.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.722+0000 7ff7f62e2640 1 -- 10.20.193.28:0/3367234705 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab53a440 msgr2=0x5632ab579050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:28.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.722+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/3367234705 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab53a440 0x5632ab579050 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.722+0000 7ff7f62e2640 1 -- 10.20.193.28:0/3367234705 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab57f0e0 msgr2=0x5632ab579590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:28.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.722+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/3367234705 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab57f0e0 0x5632ab579590 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.722+0000 7ff7f62e2640 1 -- 10.20.193.28:0/3367234705 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ab5893c0 con 0x5632ab539b30 2026-02-25T11:48:28.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.722+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/3367234705 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab578b10 secure :-1 s=READY pgs=402 gs=1 cs=0 l=1 c_cookie=13212558637162392334 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff7e000b680 tx=0x7ff7e00e4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75c7013ce156b0e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:28.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.722+0000 7ff7be4e6640 1 -- 10.20.193.28:0/3367234705 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff7e00b7e10 con 0x5632ab539b30 2026-02-25T11:48:28.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.722+0000 7ff7be4e6640 1 -- 10.20.193.28:0/3367234705 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff7e00b7f70 con 0x5632ab539b30 2026-02-25T11:48:28.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.722+0000 7ff7be4e6640 1 -- 10.20.193.28:0/3367234705 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff7e00aedd0 con 0x5632ab539b30 2026-02-25T11:48:28.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.723+0000 7ff7fa568e00 1 -- 10.20.193.28:0/3367234705 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 msgr2=0x5632ab578b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.723+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/3367234705 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab578b10 secure :-1 s=READY pgs=402 gs=1 cs=0 l=1 c_cookie=13212558637162392334 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff7e000b680 tx=0x7ff7e00e4040 comp rx=0 tx=0).stop 2026-02-25T11:48:28.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.723+0000 7ff7fa568e00 1 -- 10.20.193.28:0/3367234705 shutdown_connections 2026-02-25T11:48:28.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.723+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/3367234705 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab57f0e0 0x5632ab579590 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.723+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/3367234705 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab53a440 0x5632ab579050 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.723+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/3367234705 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab578b10 unknown :-1 s=CLOSED pgs=402 gs=1 cs=0 l=1 c_cookie=13212558637162392334 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.723+0000 7ff7fa568e00 1 -- 10.20.193.28:0/3367234705 >> 10.20.193.28:0/3367234705 conn(0x5632ab4587a0 msgr2=0x5632ab538030 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:28.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.723+0000 7ff7fa568e00 1 -- 10.20.193.28:0/3367234705 shutdown_connections 2026-02-25T11:48:28.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.723+0000 7ff7fa568e00 1 -- 10.20.193.28:0/3367234705 wait complete. 2026-02-25T11:48:28.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.723+0000 7ff7fa568e00 1 -- start start 2026-02-25T11:48:28.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.723+0000 7ff7fa568e00 1 Processor -- start 2026-02-25T11:48:28.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.724+0000 7ff7fa568e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab51d4e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.724+0000 7ff7fa568e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab53a440 0x5632ab518f40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.724+0000 7ff7fa568e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab57f0e0 0x5632ab519480 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.724+0000 7ff7fa568e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5632ab5199c0 con 0x5632ab539b30 2026-02-25T11:48:28.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.724+0000 7ff7fa568e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5632ab519b30 con 0x5632ab57f0e0 2026-02-25T11:48:28.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.724+0000 7ff7fa568e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5632ab519cf0 con 0x5632ab53a440 2026-02-25T11:48:28.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.724+0000 7ff7f62e2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab51d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.724+0000 7ff7f62e2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab51d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58268/0 (socket says 10.20.193.28:58268) 2026-02-25T11:48:28.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.724+0000 7ff7f62e2640 1 -- 10.20.193.28:0/557679082 learned_addr learned my addr 10.20.193.28:0/557679082 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:28.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.724+0000 7ff7f62e2640 1 -- 10.20.193.28:0/557679082 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab53a440 msgr2=0x5632ab518f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:28.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.724+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/557679082 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab53a440 0x5632ab518f40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.724+0000 7ff7f62e2640 1 -- 10.20.193.28:0/557679082 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab57f0e0 msgr2=0x5632ab519480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:28.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.724+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/557679082 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab57f0e0 0x5632ab519480 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7f62e2640 1 -- 10.20.193.28:0/557679082 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ab3d5b60 con 0x5632ab539b30 2026-02-25T11:48:28.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7f62e2640 1 --2- 10.20.193.28:0/557679082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab51d4e0 secure :-1 s=READY pgs=403 gs=1 cs=0 l=1 c_cookie=16168499886688316287 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff7e00aef80 tx=0x7ff7e00adb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0620cfb89002b7f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:28.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7a3fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:28.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7a3fff640 1 -- 10.20.193.28:0/557679082 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff7e00e4b90 con 0x5632ab539b30 2026-02-25T11:48:28.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7a3fff640 1 -- 10.20.193.28:0/557679082 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff7e00aeb50 con 0x5632ab539b30 2026-02-25T11:48:28.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7a3fff640 1 -- 10.20.193.28:0/557679082 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff7e00d2350 con 0x5632ab539b30 2026-02-25T11:48:28.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7fa568e00 1 -- 10.20.193.28:0/557679082 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ab3f0d10 con 0x5632ab539b30 2026-02-25T11:48:28.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7a3fff640 1 -- 10.20.193.28:0/557679082 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ff7e00acb60 con 0x5632ab539b30 2026-02-25T11:48:28.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7fa568e00 10 client.15585 _ll_register_callbacks cb 0x5632ab3c6a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:28.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7fa568e00 1 -- 10.20.193.28:0/557679082 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e00b82a0 con 0x5632ab539b30 2026-02-25T11:48:28.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7a3fff640 1 -- 10.20.193.28:0/557679082 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff7e00d24e0 con 0x5632ab539b30 2026-02-25T11:48:28.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7fa568e00 10 client.15585 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:28.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7fa568e00 1 -- 10.20.193.28:0/557679082 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5632ab3c96f0 con 0x5632ab539b30 2026-02-25T11:48:28.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.725+0000 7ff7a3fff640 1 -- 10.20.193.28:0/557679082 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7ff7e00acd20 con 0x5632ab539b30 2026-02-25T11:48:28.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.726+0000 7ff7fa568e00 10 client.15585 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:28.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.726+0000 7ff7fa568e00 10 client.15585 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:48:28.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.726+0000 7ff7fa568e00 20 client.15585 populate_metadata read hostname 'trial028' 2026-02-25T11:48:28.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.726+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/557679082 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5632ab634ab0 0x5632ab654e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:28.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.726+0000 7ff7fa568e00 4 client.15585 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:48:28.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.726+0000 7ff7fa568e00 1 -- 10.20.193.28:0/557679082 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632ab0d8f40 con 0x5632ab634ab0 2026-02-25T11:48:28.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.726+0000 7ff7f5ae1640 1 --2- 10.20.193.28:0/557679082 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5632ab634ab0 0x5632ab654e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:28.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7f5ae1640 1 --2- 10.20.193.28:0/557679082 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5632ab634ab0 0x5632ab654e90 crc :-1 s=READY pgs=26 gs=4 cs=0 l=0 c_cookie=13545674026183625149 s_cookie=1953518160992969060 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bbfbe6c23de909bd server_cookie=1b1c4a6ad042a164 in_seq=0 out_seq=0 2026-02-25T11:48:28.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7a3fff640 10 client.15585 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:48:28.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7a3fff640 1 -- 10.20.193.28:0/557679082 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632ab0d8f40 con 0x5632ab634ab0 2026-02-25T11:48:28.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7a3fff640 10 client.15585 handle_command_reply: tid=0 2026-02-25T11:48:28.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7fa568e00 1 client.15585 shutdown 2026-02-25T11:48:28.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7fa568e00 1 -- 10.20.193.28:0/557679082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 msgr2=0x5632ab51d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/557679082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab51d4e0 secure :-1 s=READY pgs=403 gs=1 cs=0 l=1 c_cookie=16168499886688316287 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff7e00aef80 tx=0x7ff7e00adb60 comp rx=0 tx=0).stop 2026-02-25T11:48:28.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7fa568e00 1 -- 10.20.193.28:0/557679082 shutdown_connections 2026-02-25T11:48:28.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/557679082 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5632ab634ab0 0x5632ab654e90 crc :-1 s=READY pgs=26 gs=4 cs=0 l=0 c_cookie=13545674026183625149 s_cookie=1953518160992969060 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/557679082 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5632ab57f0e0 0x5632ab519480 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/557679082 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5632ab53a440 0x5632ab518f40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7fa568e00 1 --2- 10.20.193.28:0/557679082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5632ab539b30 0x5632ab51d4e0 unknown :-1 s=CLOSED pgs=403 gs=1 cs=0 l=1 c_cookie=16168499886688316287 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7a3fff640 0 client.15585 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:48:28.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7fa568e00 1 -- 10.20.193.28:0/557679082 >> 10.20.193.28:0/557679082 conn(0x5632ab4587a0 msgr2=0x5632ab538820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:28.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7fa568e00 1 -- 10.20.193.28:0/557679082 shutdown_connections 2026-02-25T11:48:28.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.727+0000 7ff7fa568e00 1 -- 10.20.193.28:0/557679082 wait complete. 2026-02-25T11:48:28.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.728+0000 7ff7fa568e00 20 client.15585 trim_cache size 0 max 16384 2026-02-25T11:48:28.736 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:48:28.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.728+0000 7ff7f72e4640 1 -- 10.20.193.28:0/993402145 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff7cc0b57f0 msgr2=0x7ff7cc0d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.728+0000 7ff7f72e4640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff7cc0b57f0 0x7ff7cc0d5ca0 secure :-1 s=READY pgs=263 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff7f00fc330 tx=0x7ff7e4031040 comp rx=0 tx=0).stop 2026-02-25T11:48:28.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.728+0000 7ff7f72e4640 1 -- 10.20.193.28:0/993402145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff7f010f650 msgr2=0x7ff7f00f5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:28.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.728+0000 7ff7f72e4640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff7f010f650 0x7ff7f00f5990 secure :-1 s=READY pgs=399 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff7e000d950 tx=0x7ff7e000de20 comp rx=0 tx=0).stop 2026-02-25T11:48:28.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.728+0000 7ff7f72e4640 1 -- 10.20.193.28:0/993402145 shutdown_connections 2026-02-25T11:48:28.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.728+0000 7ff7f72e4640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff7cc0b57f0 0x7ff7cc0d5ca0 unknown :-1 s=CLOSED pgs=263 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.728+0000 7ff7f72e4640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff7f0115840 0x7ff7f00f6090 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.728+0000 7ff7f72e4640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff7f0110040 0x7ff7f0176110 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.728+0000 7ff7f72e4640 1 --2- 10.20.193.28:0/993402145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff7f010f650 0x7ff7f00f5990 unknown :-1 s=CLOSED pgs=399 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:28.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.728+0000 7ff7f72e4640 1 -- 10.20.193.28:0/993402145 >> 10.20.193.28:0/993402145 conn(0x7ff7f00ecf00 msgr2=0x7ff7f010dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:28.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.728+0000 7ff7f72e4640 1 -- 10.20.193.28:0/993402145 shutdown_connections 2026-02-25T11:48:28.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:28.728+0000 7ff7f72e4640 1 -- 10.20.193.28:0/993402145 wait complete. 2026-02-25T11:48:28.831 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:48:29.015 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:48:29.190 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.187+0000 7fdc6354b640 1 -- 10.20.193.28:0/2700015817 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdc5c10d3d0 msgr2=0x7fdc5c10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.187+0000 7fdc6354b640 1 --2- 10.20.193.28:0/2700015817 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdc5c10d3d0 0x7fdc5c10d7a0 secure :-1 s=READY pgs=108 gs=1 cs=0 l=1 c_cookie=1660596030407672915 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc4c009950 tx=0x7fdc4c02f190 comp rx=0 tx=0).stop 2026-02-25T11:48:29.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.187+0000 7fdc6354b640 1 -- 10.20.193.28:0/2700015817 shutdown_connections 2026-02-25T11:48:29.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.187+0000 7fdc6354b640 1 --2- 10.20.193.28:0/2700015817 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdc5c1135c0 0x7fdc5c173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.187+0000 7fdc6354b640 1 --2- 10.20.193.28:0/2700015817 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdc5c10ddc0 0x7fdc5c112e10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.187+0000 7fdc6354b640 1 --2- 10.20.193.28:0/2700015817 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdc5c10d3d0 0x7fdc5c10d7a0 unknown :-1 s=CLOSED pgs=108 gs=1 cs=0 l=1 c_cookie=1660596030407672915 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.187+0000 7fdc6354b640 1 -- 10.20.193.28:0/2700015817 >> 10.20.193.28:0/2700015817 conn(0x7fdc5c0eaca0 msgr2=0x7fdc5c10b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:29.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.188+0000 7fdc6354b640 1 -- 10.20.193.28:0/2700015817 shutdown_connections 2026-02-25T11:48:29.191 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.188+0000 7fdc6354b640 1 -- 10.20.193.28:0/2700015817 wait complete. 2026-02-25T11:48:29.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.188+0000 7fdc6354b640 1 Processor -- start 2026-02-25T11:48:29.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.188+0000 7fdc6354b640 1 -- start start 2026-02-25T11:48:29.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc6354b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdc5c10d3d0 0x7fdc5c0f5920 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc6354b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdc5c10ddc0 0x7fdc5c173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc6354b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdc5c1135c0 0x7fdc5c0f6020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc6354b640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c0fb230 con 0x7fdc5c1135c0 2026-02-25T11:48:29.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc6354b640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c0fb3a0 con 0x7fdc5c10d3d0 2026-02-25T11:48:29.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc6354b640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c0fb560 con 0x7fdc5c10ddc0 2026-02-25T11:48:29.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc62549640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdc5c10d3d0 0x7fdc5c0f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc62549640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdc5c10d3d0 0x7fdc5c0f5920 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:46258/0 (socket says 10.20.193.28:46258) 2026-02-25T11:48:29.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc62549640 1 -- 10.20.193.28:0/2199693981 learned_addr learned my addr 10.20.193.28:0/2199693981 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:29.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc61d48640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdc5c10ddc0 0x7fdc5c173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc62d4a640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdc5c1135c0 0x7fdc5c0f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc62549640 1 -- 10.20.193.28:0/2199693981 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdc5c10ddc0 msgr2=0x7fdc5c173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc62549640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdc5c10ddc0 0x7fdc5c173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.194 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc62549640 1 -- 10.20.193.28:0/2199693981 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdc5c1135c0 msgr2=0x7fdc5c0f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.195 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc62549640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdc5c1135c0 0x7fdc5c0f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.195 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.189+0000 7fdc62549640 1 -- 10.20.193.28:0/2199693981 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc4c009570 con 0x7fdc5c10d3d0 2026-02-25T11:48:29.195 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.190+0000 7fdc62549640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdc5c10d3d0 0x7fdc5c0f5920 secure :-1 s=READY pgs=109 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc4c0098a0 tx=0x7fdc4c0028d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:29.195 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.190+0000 7fdc437fe640 1 -- 10.20.193.28:0/2199693981 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdc4c03d070 con 0x7fdc5c10d3d0 2026-02-25T11:48:29.195 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.190+0000 7fdc437fe640 1 -- 10.20.193.28:0/2199693981 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdc4c005dd0 con 0x7fdc5c10d3d0 2026-02-25T11:48:29.195 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.190+0000 7fdc6354b640 1 -- 10.20.193.28:0/2199693981 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5c0fb7e0 con 0x7fdc5c10d3d0 2026-02-25T11:48:29.195 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.190+0000 7fdc437fe640 1 -- 10.20.193.28:0/2199693981 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdc4c0417b0 con 0x7fdc5c10d3d0 2026-02-25T11:48:29.195 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.190+0000 7fdc6354b640 1 -- 10.20.193.28:0/2199693981 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5c0fbc50 con 0x7fdc5c10d3d0 2026-02-25T11:48:29.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.193+0000 7fdc437fe640 1 -- 10.20.193.28:0/2199693981 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fdc4c0386a0 con 0x7fdc5c10d3d0 2026-02-25T11:48:29.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.194+0000 7fdc437fe640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdc380b5760 0x7fdc380d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.194+0000 7fdc437fe640 1 -- 10.20.193.28:0/2199693981 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fdc4c10b280 con 0x7fdc5c10d3d0 2026-02-25T11:48:29.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.195+0000 7fdc61d48640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdc380b5760 0x7fdc380d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.195+0000 7fdc61d48640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdc380b5760 0x7fdc380d5c10 secure :-1 s=READY pgs=264 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc5c0fc2c0 tx=0x7fdc50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:29.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.196+0000 7fdc667cfe00 1 Processor -- start 2026-02-25T11:48:29.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.196+0000 7fdc667cfe00 1 -- start start 2026-02-25T11:48:29.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.196+0000 7fdc667cfe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef61b30 0x55826ef61f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.196+0000 7fdc667cfe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef62440 0x55826efa6ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.196+0000 7fdc667cfe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 0x55826efc74c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.196+0000 7fdc667cfe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55826ee778a0 con 0x55826ef61b30 2026-02-25T11:48:29.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.196+0000 7fdc667cfe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55826ee735d0 con 0x55826efa70e0 2026-02-25T11:48:29.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.196+0000 7fdc667cfe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55826ee6d570 con 0x55826ef62440 2026-02-25T11:48:29.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.196+0000 7fdc62549640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef61b30 0x55826ef61f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.196+0000 7fdc62549640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef61b30 0x55826ef61f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58300/0 (socket says 10.20.193.28:58300) 2026-02-25T11:48:29.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.196+0000 7fdc62549640 1 -- 10.20.193.28:0/4055659690 learned_addr learned my addr 10.20.193.28:0/4055659690 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:29.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.197+0000 7fdc61d48640 1 --2- 10.20.193.28:0/4055659690 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef62440 0x55826efa6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.197+0000 7fdc62d4a640 1 --2- 10.20.193.28:0/4055659690 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 0x55826efc74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.197+0000 7fdc62549640 1 -- 10.20.193.28:0/4055659690 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef62440 msgr2=0x55826efa6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.197+0000 7fdc62549640 1 --2- 10.20.193.28:0/4055659690 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef62440 0x55826efa6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.197+0000 7fdc62549640 1 -- 10.20.193.28:0/4055659690 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 msgr2=0x55826efc74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.197+0000 7fdc62549640 1 --2- 10.20.193.28:0/4055659690 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 0x55826efc74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.197+0000 7fdc62549640 1 -- 10.20.193.28:0/4055659690 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826ee18d10 con 0x55826ef61b30 2026-02-25T11:48:29.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.197+0000 7fdc62549640 1 --2- 10.20.193.28:0/4055659690 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef61b30 0x55826ef61f00 secure :-1 s=READY pgs=404 gs=1 cs=0 l=1 c_cookie=12232797992061314159 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc4c004030 tx=0x7fdc4c0059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c3a0f62ded246f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:29.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.197+0000 7fdc40cf3640 1 -- 10.20.193.28:0/4055659690 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdc4c0d0f90 con 0x55826ef61b30 2026-02-25T11:48:29.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.197+0000 7fdc40cf3640 1 -- 10.20.193.28:0/4055659690 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdc4c0d0db0 con 0x55826ef61b30 2026-02-25T11:48:29.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.197+0000 7fdc40cf3640 1 -- 10.20.193.28:0/4055659690 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdc4c0dc260 con 0x55826ef61b30 2026-02-25T11:48:29.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.198+0000 7fdc667cfe00 1 -- 10.20.193.28:0/4055659690 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef61b30 msgr2=0x55826ef61f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.198+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/4055659690 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef61b30 0x55826ef61f00 secure :-1 s=READY pgs=404 gs=1 cs=0 l=1 c_cookie=12232797992061314159 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc4c004030 tx=0x7fdc4c0059b0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.198+0000 7fdc667cfe00 1 -- 10.20.193.28:0/4055659690 shutdown_connections 2026-02-25T11:48:29.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.198+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/4055659690 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 0x55826efc74c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.198+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/4055659690 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef62440 0x55826efa6ba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.198+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/4055659690 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef61b30 0x55826ef61f00 unknown :-1 s=CLOSED pgs=404 gs=1 cs=0 l=1 c_cookie=12232797992061314159 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.198+0000 7fdc667cfe00 1 -- 10.20.193.28:0/4055659690 >> 10.20.193.28:0/4055659690 conn(0x55826ee807a0 msgr2=0x55826ef60410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:29.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.198+0000 7fdc667cfe00 1 -- 10.20.193.28:0/4055659690 shutdown_connections 2026-02-25T11:48:29.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.198+0000 7fdc667cfe00 1 -- 10.20.193.28:0/4055659690 wait complete. 2026-02-25T11:48:29.207 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.199+0000 7fdc667cfe00 1 -- start start 2026-02-25T11:48:29.207 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.199+0000 7fdc667cfe00 1 Processor -- start 2026-02-25T11:48:29.207 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc667cfe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef61b30 0x55826ef79ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc667cfe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef62440 0x55826ef7a220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc667cfe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 0x55826ef7f210 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc667cfe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55826ef7f750 con 0x55826ef62440 2026-02-25T11:48:29.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc667cfe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55826ef7f8c0 con 0x55826efa70e0 2026-02-25T11:48:29.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc667cfe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55826ef7fa80 con 0x55826ef61b30 2026-02-25T11:48:29.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc62549640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef61b30 0x55826ef79ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc61d48640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef62440 0x55826ef7a220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc62549640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef61b30 0x55826ef79ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:39270/0 (socket says 10.20.193.28:39270) 2026-02-25T11:48:29.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc62549640 1 -- 10.20.193.28:0/1142435061 learned_addr learned my addr 10.20.193.28:0/1142435061 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:29.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc62d4a640 1 --2- 10.20.193.28:0/1142435061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 0x55826ef7f210 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc61d48640 1 -- 10.20.193.28:0/1142435061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef61b30 msgr2=0x55826ef79ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc61d48640 1 --2- 10.20.193.28:0/1142435061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef61b30 0x55826ef79ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc61d48640 1 -- 10.20.193.28:0/1142435061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 msgr2=0x55826ef7f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc61d48640 1 --2- 10.20.193.28:0/1142435061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 0x55826ef7f210 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc61d48640 1 -- 10.20.193.28:0/1142435061 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826edf13c0 con 0x55826ef62440 2026-02-25T11:48:29.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.200+0000 7fdc62d4a640 1 --2- 10.20.193.28:0/1142435061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 0x55826ef7f210 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:48:29.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.201+0000 7fdc61d48640 1 --2- 10.20.193.28:0/1142435061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef62440 0x55826ef7a220 secure :-1 s=READY pgs=405 gs=1 cs=0 l=1 c_cookie=15189296296762278565 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc50004740 tx=0x7fdc500047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cb38816cac02a5 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:29.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.201+0000 7fdc0ffff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:29.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.201+0000 7fdc0ffff640 1 -- 10.20.193.28:0/1142435061 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdc50022070 con 0x55826ef62440 2026-02-25T11:48:29.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.201+0000 7fdc0ffff640 1 -- 10.20.193.28:0/1142435061 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdc5001db40 con 0x55826ef62440 2026-02-25T11:48:29.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.201+0000 7fdc667cfe00 1 -- 10.20.193.28:0/1142435061 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5c0fbc50 con 0x55826ef62440 2026-02-25T11:48:29.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.201+0000 7fdc0ffff640 1 -- 10.20.193.28:0/1142435061 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdc50027e70 con 0x55826ef62440 2026-02-25T11:48:29.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.201+0000 7fdc667cfe00 10 client.15597 _ll_register_callbacks cb 0x55826edeea10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:29.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.201+0000 7fdc667cfe00 1 -- 10.20.193.28:0/1142435061 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55826eb036f0 con 0x55826ef62440 2026-02-25T11:48:29.210 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.201+0000 7fdc0ffff640 1 -- 10.20.193.28:0/1142435061 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fdc5001b670 con 0x55826ef62440 2026-02-25T11:48:29.210 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.201+0000 7fdc0ffff640 1 -- 10.20.193.28:0/1142435061 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdc5001b830 con 0x55826ef62440 2026-02-25T11:48:29.210 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.201+0000 7fdc667cfe00 10 client.15597 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:29.210 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.201+0000 7fdc667cfe00 1 -- 10.20.193.28:0/1142435061 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55826ee030e0 con 0x55826ef62440 2026-02-25T11:48:29.210 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.202+0000 7fdc0ffff640 1 -- 10.20.193.28:0/1142435061 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fdc5001d680 con 0x55826ef62440 2026-02-25T11:48:29.210 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.202+0000 7fdc667cfe00 10 client.15597 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:29.210 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.202+0000 7fdc667cfe00 10 client.15597 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:48:29.210 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.202+0000 7fdc667cfe00 20 client.15597 populate_metadata read hostname 'trial028' 2026-02-25T11:48:29.210 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.202+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/1142435061 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55826efa4580 0x55826efa4a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.202+0000 7fdc667cfe00 4 client.15597 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:48:29.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.202+0000 7fdc667cfe00 1 -- 10.20.193.28:0/1142435061 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55826eb036f0 con 0x55826efa4580 2026-02-25T11:48:29.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.202+0000 7fdc62549640 1 --2- 10.20.193.28:0/1142435061 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55826efa4580 0x55826efa4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.203+0000 7fdc62549640 1 --2- 10.20.193.28:0/1142435061 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55826efa4580 0x55826efa4a20 crc :-1 s=READY pgs=26 gs=4 cs=0 l=0 c_cookie=13470980992578087871 s_cookie=5280206062017976355 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=baf289d726a7ebbf server_cookie=49470f700b721823 in_seq=0 out_seq=0 2026-02-25T11:48:29.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.203+0000 7fdc0ffff640 10 client.15597 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:48:29.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.203+0000 7fdc0ffff640 1 -- 10.20.193.28:0/1142435061 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55826eb036f0 con 0x55826efa4580 2026-02-25T11:48:29.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.203+0000 7fdc0ffff640 10 client.15597 handle_command_reply: tid=0 2026-02-25T11:48:29.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.203+0000 7fdc667cfe00 1 client.15597 shutdown 2026-02-25T11:48:29.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.203+0000 7fdc667cfe00 1 -- 10.20.193.28:0/1142435061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef62440 msgr2=0x55826ef7a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.203+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/1142435061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef62440 0x55826ef7a220 secure :-1 s=READY pgs=405 gs=1 cs=0 l=1 c_cookie=15189296296762278565 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc50004740 tx=0x7fdc500047e0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.204+0000 7fdc667cfe00 1 -- 10.20.193.28:0/1142435061 shutdown_connections 2026-02-25T11:48:29.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.204+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/1142435061 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55826efa4580 0x55826efa4a20 crc :-1 s=READY pgs=26 gs=4 cs=0 l=0 c_cookie=13470980992578087871 s_cookie=5280206062017976355 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.204+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/1142435061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 0x55826ef7f210 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.204+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/1142435061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef62440 0x55826ef7a220 unknown :-1 s=CLOSED pgs=405 gs=1 cs=0 l=1 c_cookie=15189296296762278565 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.204+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/1142435061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef61b30 0x55826ef79ce0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.204+0000 7fdc667cfe00 1 -- 10.20.193.28:0/1142435061 >> 10.20.193.28:0/1142435061 conn(0x55826ee807a0 msgr2=0x55826ef60b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:29.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.204+0000 7fdc0ffff640 0 client.15597 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:48:29.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.204+0000 7fdc667cfe00 1 -- 10.20.193.28:0/1142435061 shutdown_connections 2026-02-25T11:48:29.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.204+0000 7fdc667cfe00 1 -- 10.20.193.28:0/1142435061 wait complete. 2026-02-25T11:48:29.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.204+0000 7fdc667cfe00 20 client.15597 trim_cache size 0 max 16384 2026-02-25T11:48:29.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.211+0000 7fdc667cfe00 1 Processor -- start 2026-02-25T11:48:29.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.211+0000 7fdc667cfe00 1 -- start start 2026-02-25T11:48:29.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.211+0000 7fdc667cfe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef61b30 0x55826ef772e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.211+0000 7fdc667cfe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef62440 0x55826efa60f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.211+0000 7fdc667cfe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 0x55826efa6630 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.211+0000 7fdc667cfe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55826ee6b1a0 con 0x55826ef61b30 2026-02-25T11:48:29.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.211+0000 7fdc667cfe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55826eb2b700 con 0x55826efa70e0 2026-02-25T11:48:29.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.211+0000 7fdc667cfe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55826ee735d0 con 0x55826ef62440 2026-02-25T11:48:29.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.212+0000 7fdc61d48640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef62440 0x55826efa60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.212+0000 7fdc62549640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef61b30 0x55826ef772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.212+0000 7fdc61d48640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef62440 0x55826efa60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:39274/0 (socket says 10.20.193.28:39274) 2026-02-25T11:48:29.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.212+0000 7fdc61d48640 1 -- 10.20.193.28:0/2189848365 learned_addr learned my addr 10.20.193.28:0/2189848365 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:29.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.212+0000 7fdc62d4a640 1 --2- 10.20.193.28:0/2189848365 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 0x55826efa6630 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.212+0000 7fdc61d48640 1 -- 10.20.193.28:0/2189848365 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 msgr2=0x55826efa6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.212+0000 7fdc61d48640 1 --2- 10.20.193.28:0/2189848365 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 0x55826efa6630 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.212+0000 7fdc61d48640 1 -- 10.20.193.28:0/2189848365 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef61b30 msgr2=0x55826ef772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.212+0000 7fdc61d48640 1 --2- 10.20.193.28:0/2189848365 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef61b30 0x55826ef772e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.212+0000 7fdc61d48640 1 -- 10.20.193.28:0/2189848365 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826edfdb60 con 0x55826ef62440 2026-02-25T11:48:29.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.212+0000 7fdc62549640 1 --2- 10.20.193.28:0/2189848365 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef61b30 0x55826ef772e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:48:29.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.212+0000 7fdc61d48640 1 --2- 10.20.193.28:0/2189848365 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef62440 0x55826efa60f0 secure :-1 s=READY pgs=119 gs=2 cs=0 l=1 c_cookie=3762036494385361811 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc50009230 tx=0x7fdc5002b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=343570426fe53393 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:29.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.213+0000 7fdc2a4e6640 1 -- 10.20.193.28:0/2189848365 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdc50022040 con 0x55826ef62440 2026-02-25T11:48:29.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.213+0000 7fdc2a4e6640 1 -- 10.20.193.28:0/2189848365 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdc50034020 con 0x55826ef62440 2026-02-25T11:48:29.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.213+0000 7fdc2a4e6640 1 -- 10.20.193.28:0/2189848365 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdc50027460 con 0x55826ef62440 2026-02-25T11:48:29.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.213+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2189848365 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef62440 msgr2=0x55826efa60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.213+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/2189848365 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef62440 0x55826efa60f0 secure :-1 s=READY pgs=119 gs=2 cs=0 l=1 c_cookie=3762036494385361811 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc50009230 tx=0x7fdc5002b000 comp rx=0 tx=0).stop 2026-02-25T11:48:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.213+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2189848365 shutdown_connections 2026-02-25T11:48:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.213+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/2189848365 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826efa70e0 0x55826efa6630 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.213+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/2189848365 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826ef62440 0x55826efa60f0 unknown :-1 s=CLOSED pgs=119 gs=2 cs=0 l=1 c_cookie=3762036494385361811 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.213+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/2189848365 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef61b30 0x55826ef772e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.213+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2189848365 >> 10.20.193.28:0/2189848365 conn(0x55826ee807a0 msgr2=0x55826ef5fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.213+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2189848365 shutdown_connections 2026-02-25T11:48:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.213+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2189848365 wait complete. 2026-02-25T11:48:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.214+0000 7fdc667cfe00 1 -- start start 2026-02-25T11:48:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.214+0000 7fdc667cfe00 1 Processor -- start 2026-02-25T11:48:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.214+0000 7fdc667cfe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826ef61b30 0x55826ef86010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.222 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.214+0000 7fdc667cfe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef62440 0x55826ef81530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.214+0000 7fdc667cfe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826efa70e0 0x55826ef81cc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.214+0000 7fdc667cfe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55826ef823c0 con 0x55826ef62440 2026-02-25T11:48:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.214+0000 7fdc667cfe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55826eff5000 con 0x55826ef61b30 2026-02-25T11:48:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.214+0000 7fdc667cfe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55826eff51c0 con 0x55826efa70e0 2026-02-25T11:48:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.214+0000 7fdc62d4a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826efa70e0 0x55826ef81cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.214+0000 7fdc62d4a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826efa70e0 0x55826ef81cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:39288/0 (socket says 10.20.193.28:39288) 2026-02-25T11:48:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.214+0000 7fdc62d4a640 1 -- 10.20.193.28:0/2372656909 learned_addr learned my addr 10.20.193.28:0/2372656909 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.214+0000 7fdc62549640 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826ef61b30 0x55826ef86010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.215+0000 7fdc61d48640 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef62440 0x55826ef81530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.215+0000 7fdc62d4a640 1 -- 10.20.193.28:0/2372656909 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826ef61b30 msgr2=0x55826ef86010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.215+0000 7fdc62d4a640 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826ef61b30 0x55826ef86010 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.223 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.215+0000 7fdc62d4a640 1 -- 10.20.193.28:0/2372656909 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef62440 msgr2=0x55826ef81530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.215+0000 7fdc62d4a640 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef62440 0x55826ef81530 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.215+0000 7fdc62d4a640 1 -- 10.20.193.28:0/2372656909 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826ee030e0 con 0x55826efa70e0 2026-02-25T11:48:29.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.215+0000 7fdc61d48640 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef62440 0x55826ef81530 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:48:29.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.215+0000 7fdc62d4a640 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826efa70e0 0x55826ef81cc0 secure :-1 s=READY pgs=120 gs=1 cs=0 l=1 c_cookie=2862773593794653270 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc58004ee0 tx=0x7fdc58004fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ba9d6a7a7fac56 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:29.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.215+0000 7fdc0ffff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:48:29.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.215+0000 7fdc0ffff640 1 -- 10.20.193.28:0/2372656909 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdc58011a60 con 0x55826efa70e0 2026-02-25T11:48:29.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.216+0000 7fdc0ffff640 1 -- 10.20.193.28:0/2372656909 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdc58019070 con 0x55826efa70e0 2026-02-25T11:48:29.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.216+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2372656909 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55826ee18d10 con 0x55826efa70e0 2026-02-25T11:48:29.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.216+0000 7fdc0ffff640 1 -- 10.20.193.28:0/2372656909 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdc5800f640 con 0x55826efa70e0 2026-02-25T11:48:29.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.216+0000 7fdc667cfe00 10 client.25429 _ll_register_callbacks cb 0x55826edeea10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:29.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.216+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2372656909 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc4c0d0db0 con 0x55826efa70e0 2026-02-25T11:48:29.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.216+0000 7fdc0ffff640 1 -- 10.20.193.28:0/2372656909 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fdc58011bc0 con 0x55826efa70e0 2026-02-25T11:48:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.216+0000 7fdc0ffff640 1 -- 10.20.193.28:0/2372656909 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdc58011dc0 con 0x55826efa70e0 2026-02-25T11:48:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.216+0000 7fdc667cfe00 10 client.25429 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.216+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2372656909 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55826edf16f0 con 0x55826efa70e0 2026-02-25T11:48:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc0ffff640 1 -- 10.20.193.28:0/2372656909 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fdc5801e410 con 0x55826efa70e0 2026-02-25T11:48:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc667cfe00 10 client.25429 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc667cfe00 10 client.25429 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:48:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc667cfe00 20 client.25429 populate_metadata read hostname 'trial028' 2026-02-25T11:48:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55826eff7030 0x55826eff74d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc667cfe00 4 client.25429 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:48:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2372656909 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55826ee3ffc0 con 0x55826eff7030 2026-02-25T11:48:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc62549640 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55826eff7030 0x55826eff74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc0ffff640 10 client.25429 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:48:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc62549640 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55826eff7030 0x55826eff74d0 crc :-1 s=READY pgs=27 gs=4 cs=0 l=0 c_cookie=7343972023596176763 s_cookie=15958823850218285900 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65eb07a53adb557b server_cookie=dd7921bd014aeb4c in_seq=0 out_seq=0 2026-02-25T11:48:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc0ffff640 1 -- 10.20.193.28:0/2372656909 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55826ee3ffc0 con 0x55826eff7030 2026-02-25T11:48:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc0ffff640 10 client.25429 handle_command_reply: tid=0 2026-02-25T11:48:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc667cfe00 1 client.25429 shutdown 2026-02-25T11:48:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2372656909 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826efa70e0 msgr2=0x55826ef81cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.217+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826efa70e0 0x55826ef81cc0 secure :-1 s=READY pgs=120 gs=1 cs=0 l=1 c_cookie=2862773593794653270 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc58004ee0 tx=0x7fdc58004fe0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2372656909 shutdown_connections 2026-02-25T11:48:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55826eff7030 0x55826eff74d0 crc :-1 s=READY pgs=27 gs=4 cs=0 l=0 c_cookie=7343972023596176763 s_cookie=15958823850218285900 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55826efa70e0 0x55826ef81cc0 unknown :-1 s=CLOSED pgs=120 gs=1 cs=0 l=1 c_cookie=2862773593794653270 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55826ef62440 0x55826ef81530 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc667cfe00 1 --2- 10.20.193.28:0/2372656909 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55826ef61b30 0x55826ef86010 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc0ffff640 0 client.25429 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:48:29.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2372656909 >> 10.20.193.28:0/2372656909 conn(0x55826ee807a0 msgr2=0x55826ef60500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2372656909 shutdown_connections 2026-02-25T11:48:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc667cfe00 1 -- 10.20.193.28:0/2372656909 wait complete. 2026-02-25T11:48:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc667cfe00 20 client.25429 trim_cache size 0 max 16384 2026-02-25T11:48:29.227 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:48:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc6354b640 1 -- 10.20.193.28:0/2199693981 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdc380b5760 msgr2=0x7fdc380d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc6354b640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdc380b5760 0x7fdc380d5c10 secure :-1 s=READY pgs=264 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc5c0fc2c0 tx=0x7fdc50005f90 comp rx=0 tx=0).stop 2026-02-25T11:48:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc6354b640 1 -- 10.20.193.28:0/2199693981 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdc5c10d3d0 msgr2=0x7fdc5c0f5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc6354b640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdc5c10d3d0 0x7fdc5c0f5920 secure :-1 s=READY pgs=109 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdc4c0098a0 tx=0x7fdc4c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc6354b640 1 -- 10.20.193.28:0/2199693981 shutdown_connections 2026-02-25T11:48:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc6354b640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdc380b5760 0x7fdc380d5c10 unknown :-1 s=CLOSED pgs=264 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc6354b640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdc5c1135c0 0x7fdc5c0f6020 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc6354b640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdc5c10ddc0 0x7fdc5c173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc6354b640 1 --2- 10.20.193.28:0/2199693981 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdc5c10d3d0 0x7fdc5c0f5920 unknown :-1 s=CLOSED pgs=109 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.218+0000 7fdc6354b640 1 -- 10.20.193.28:0/2199693981 >> 10.20.193.28:0/2199693981 conn(0x7fdc5c0eaca0 msgr2=0x7fdc5c110580 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.219+0000 7fdc6354b640 1 -- 10.20.193.28:0/2199693981 shutdown_connections 2026-02-25T11:48:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.219+0000 7fdc6354b640 1 -- 10.20.193.28:0/2199693981 wait complete. 2026-02-25T11:48:29.320 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:48:29.373 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:29 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2158115965' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:48:29.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:29 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2158115965' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:48:29.506 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:48:29.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:29 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2158115965' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:48:29.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.674+0000 7f2e528bc640 1 -- 10.20.193.28:0/3181883845 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e4c110130 msgr2=0x7f2e4c115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.674+0000 7f2e528bc640 1 --2- 10.20.193.28:0/3181883845 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e4c110130 0x7f2e4c115180 secure :-1 s=READY pgs=406 gs=1 cs=0 l=1 c_cookie=1092758092381512873 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e3c009950 tx=0x7f2e3c02f190 comp rx=0 tx=0).stop 2026-02-25T11:48:29.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.674+0000 7f2e528bc640 1 -- 10.20.193.28:0/3181883845 shutdown_connections 2026-02-25T11:48:29.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.674+0000 7f2e528bc640 1 --2- 10.20.193.28:0/3181883845 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2e4c115930 0x7f2e4c176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.674+0000 7f2e528bc640 1 --2- 10.20.193.28:0/3181883845 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e4c110130 0x7f2e4c115180 unknown :-1 s=CLOSED pgs=406 gs=1 cs=0 l=1 c_cookie=1092758092381512873 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.674+0000 7f2e528bc640 1 --2- 10.20.193.28:0/3181883845 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2e4c10f740 0x7f2e4c10fb10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.674+0000 7f2e528bc640 1 -- 10.20.193.28:0/3181883845 >> 10.20.193.28:0/3181883845 conn(0x7f2e4c0ecff0 msgr2=0x7f2e4c10d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:29.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.674+0000 7f2e528bc640 1 -- 10.20.193.28:0/3181883845 shutdown_connections 2026-02-25T11:48:29.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.674+0000 7f2e528bc640 1 -- 10.20.193.28:0/3181883845 wait complete. 2026-02-25T11:48:29.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e528bc640 1 Processor -- start 2026-02-25T11:48:29.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e528bc640 1 -- start start 2026-02-25T11:48:29.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e528bc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2e4c10f740 0x7f2e4c10b360 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e528bc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e4c110130 0x7f2e4c176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e528bc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2e4c115930 0x7f2e4c10ba20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e528bc640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c0f04d0 con 0x7f2e4c110130 2026-02-25T11:48:29.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e528bc640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c0f0640 con 0x7f2e4c10f740 2026-02-25T11:48:29.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e528bc640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4c0f0800 con 0x7f2e4c115930 2026-02-25T11:48:29.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e4b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e4c110130 0x7f2e4c176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e4bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2e4c10f740 0x7f2e4c10b360 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e4bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2e4c10f740 0x7f2e4c10b360 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:46294/0 (socket says 10.20.193.28:46294) 2026-02-25T11:48:29.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e4bfff640 1 -- 10.20.193.28:0/1334833322 learned_addr learned my addr 10.20.193.28:0/1334833322 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:29.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e50e32640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2e4c115930 0x7f2e4c10ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e4b7fe640 1 -- 10.20.193.28:0/1334833322 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2e4c115930 msgr2=0x7f2e4c10ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e4b7fe640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2e4c115930 0x7f2e4c10ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e4b7fe640 1 -- 10.20.193.28:0/1334833322 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2e4c10f740 msgr2=0x7f2e4c10b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e4b7fe640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2e4c10f740 0x7f2e4c10b360 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e4b7fe640 1 -- 10.20.193.28:0/1334833322 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c009570 con 0x7f2e4c110130 2026-02-25T11:48:29.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.675+0000 7f2e50e32640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2e4c115930 0x7f2e4c10ba20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:48:29.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.676+0000 7f2e4b7fe640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e4c110130 0x7f2e4c176200 secure :-1 s=READY pgs=407 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e3c009950 tx=0x7f2e3c0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:29.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.676+0000 7f2e497fa640 1 -- 10.20.193.28:0/1334833322 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e3c03d070 con 0x7f2e4c110130 2026-02-25T11:48:29.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.676+0000 7f2e497fa640 1 -- 10.20.193.28:0/1334833322 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2e3c02fe20 con 0x7f2e4c110130 2026-02-25T11:48:29.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.676+0000 7f2e497fa640 1 -- 10.20.193.28:0/1334833322 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e3c0418c0 con 0x7f2e4c110130 2026-02-25T11:48:29.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.676+0000 7f2e528bc640 1 -- 10.20.193.28:0/1334833322 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4c0f0a80 con 0x7f2e4c110130 2026-02-25T11:48:29.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.676+0000 7f2e528bc640 1 -- 10.20.193.28:0/1334833322 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4c0f0ea0 con 0x7f2e4c110130 2026-02-25T11:48:29.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.679+0000 7f2e497fa640 1 -- 10.20.193.28:0/1334833322 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f2e3c041a50 con 0x7f2e4c110130 2026-02-25T11:48:29.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.680+0000 7f2e497fa640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2e200b57f0 0x7f2e200d5ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.680+0000 7f2e497fa640 1 -- 10.20.193.28:0/1334833322 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f2e3c10b150 con 0x7f2e4c110130 2026-02-25T11:48:29.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.681+0000 7f2e4bfff640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2e200b57f0 0x7f2e200d5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.681+0000 7f2e4bfff640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2e200b57f0 0x7f2e200d5ca0 secure :-1 s=READY pgs=265 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e3800b2c0 tx=0x7f2e3800b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:29.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.682+0000 7f2e548b7e00 1 Processor -- start 2026-02-25T11:48:29.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.682+0000 7f2e548b7e00 1 -- start start 2026-02-25T11:48:29.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.682+0000 7f2e548b7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 0x55b674938f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.682+0000 7f2e548b7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674939440 0x55b67497dba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.682+0000 7f2e548b7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b67497e0e0 0x55b67499e4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.682+0000 7f2e548b7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b67484e8a0 con 0x55b674938b30 2026-02-25T11:48:29.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.682+0000 7f2e548b7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b67484a5d0 con 0x55b67497e0e0 2026-02-25T11:48:29.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.682+0000 7f2e548b7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b674844570 con 0x55b674939440 2026-02-25T11:48:29.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.682+0000 7f2e4bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 0x55b674938f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.683+0000 7f2e4bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 0x55b674938f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58370/0 (socket says 10.20.193.28:58370) 2026-02-25T11:48:29.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.683+0000 7f2e4bfff640 1 -- 10.20.193.28:0/2263580145 learned_addr learned my addr 10.20.193.28:0/2263580145 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:29.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.683+0000 7f2e50e32640 1 --2- 10.20.193.28:0/2263580145 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b67497e0e0 0x55b67499e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.683+0000 7f2e4b7fe640 1 --2- 10.20.193.28:0/2263580145 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674939440 0x55b67497dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.683+0000 7f2e4bfff640 1 -- 10.20.193.28:0/2263580145 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674939440 msgr2=0x55b67497dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.683+0000 7f2e4bfff640 1 --2- 10.20.193.28:0/2263580145 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674939440 0x55b67497dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.683+0000 7f2e4bfff640 1 -- 10.20.193.28:0/2263580145 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b67497e0e0 msgr2=0x55b67499e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.683+0000 7f2e4bfff640 1 --2- 10.20.193.28:0/2263580145 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b67497e0e0 0x55b67499e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.683+0000 7f2e4bfff640 1 -- 10.20.193.28:0/2263580145 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6747efd10 con 0x55b674938b30 2026-02-25T11:48:29.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.683+0000 7f2e4bfff640 1 --2- 10.20.193.28:0/2263580145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 0x55b674938f00 secure :-1 s=READY pgs=408 gs=1 cs=0 l=1 c_cookie=12826451773922240056 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e38011560 tx=0x7f2e38013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b200b5ee51506638 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:29.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.683+0000 7f2e1cce3640 1 -- 10.20.193.28:0/2263580145 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e3801e640 con 0x55b674938b30 2026-02-25T11:48:29.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.683+0000 7f2e1cce3640 1 -- 10.20.193.28:0/2263580145 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2e38015070 con 0x55b674938b30 2026-02-25T11:48:29.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.683+0000 7f2e1cce3640 1 -- 10.20.193.28:0/2263580145 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e3800dcb0 con 0x55b674938b30 2026-02-25T11:48:29.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.684+0000 7f2e548b7e00 1 -- 10.20.193.28:0/2263580145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 msgr2=0x55b674938f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.684+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/2263580145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 0x55b674938f00 secure :-1 s=READY pgs=408 gs=1 cs=0 l=1 c_cookie=12826451773922240056 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e38011560 tx=0x7f2e38013000 comp rx=0 tx=0).stop 2026-02-25T11:48:29.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.684+0000 7f2e548b7e00 1 -- 10.20.193.28:0/2263580145 shutdown_connections 2026-02-25T11:48:29.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.684+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/2263580145 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b67497e0e0 0x55b67499e4c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.684+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/2263580145 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674939440 0x55b67497dba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.684+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/2263580145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 0x55b674938f00 unknown :-1 s=CLOSED pgs=408 gs=1 cs=0 l=1 c_cookie=12826451773922240056 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.684+0000 7f2e548b7e00 1 -- 10.20.193.28:0/2263580145 >> 10.20.193.28:0/2263580145 conn(0x55b6748577a0 msgr2=0x55b674937410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:29.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.684+0000 7f2e548b7e00 1 -- 10.20.193.28:0/2263580145 shutdown_connections 2026-02-25T11:48:29.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.684+0000 7f2e548b7e00 1 -- 10.20.193.28:0/2263580145 wait complete. 2026-02-25T11:48:29.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.685+0000 7f2e548b7e00 1 -- start start 2026-02-25T11:48:29.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.685+0000 7f2e548b7e00 1 Processor -- start 2026-02-25T11:48:29.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e548b7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 0x55b674950dd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e548b7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674939440 0x55b674951310 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e548b7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b67497e0e0 0x55b674956300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e548b7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b674956840 con 0x55b674938b30 2026-02-25T11:48:29.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e548b7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b6749569b0 con 0x55b67497e0e0 2026-02-25T11:48:29.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e548b7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b674956b70 con 0x55b674939440 2026-02-25T11:48:29.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e4bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 0x55b674950dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e4bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 0x55b674950dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58384/0 (socket says 10.20.193.28:58384) 2026-02-25T11:48:29.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e4bfff640 1 -- 10.20.193.28:0/999323972 learned_addr learned my addr 10.20.193.28:0/999323972 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:29.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e50e32640 1 --2- 10.20.193.28:0/999323972 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b67497e0e0 0x55b674956300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e4b7fe640 1 --2- 10.20.193.28:0/999323972 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674939440 0x55b674951310 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e4bfff640 1 -- 10.20.193.28:0/999323972 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674939440 msgr2=0x55b674951310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e4bfff640 1 --2- 10.20.193.28:0/999323972 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674939440 0x55b674951310 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e4bfff640 1 -- 10.20.193.28:0/999323972 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b67497e0e0 msgr2=0x55b674956300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e4bfff640 1 --2- 10.20.193.28:0/999323972 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b67497e0e0 0x55b674956300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e4bfff640 1 -- 10.20.193.28:0/999323972 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6747c83c0 con 0x55b674938b30 2026-02-25T11:48:29.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e4bfff640 1 --2- 10.20.193.28:0/999323972 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 0x55b674950dd0 secure :-1 s=READY pgs=409 gs=1 cs=0 l=1 c_cookie=2028685300421049213 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e38010c40 tx=0x7f2e380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c27568844a73b7d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:29.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.686+0000 7f2e0dffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:29.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e0dffb640 1 -- 10.20.193.28:0/999323972 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e3802c070 con 0x55b674938b30 2026-02-25T11:48:29.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e0dffb640 1 -- 10.20.193.28:0/999323972 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2e3800d660 con 0x55b674938b30 2026-02-25T11:48:29.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e548b7e00 1 -- 10.20.193.28:0/999323972 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e3800bde0 con 0x55b674938b30 2026-02-25T11:48:29.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e548b7e00 10 client.15615 _ll_register_callbacks cb 0x55b6747c5a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:29.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e548b7e00 1 -- 10.20.193.28:0/999323972 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6744da6f0 con 0x55b674938b30 2026-02-25T11:48:29.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e0dffb640 1 -- 10.20.193.28:0/999323972 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e3801e420 con 0x55b674938b30 2026-02-25T11:48:29.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e0dffb640 1 -- 10.20.193.28:0/999323972 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f2e38017070 con 0x55b674938b30 2026-02-25T11:48:29.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e0dffb640 1 -- 10.20.193.28:0/999323972 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2e3801e610 con 0x55b674938b30 2026-02-25T11:48:29.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e548b7e00 10 client.15615 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:29.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e548b7e00 1 -- 10.20.193.28:0/999323972 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55b6747da0e0 con 0x55b674938b30 2026-02-25T11:48:29.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e0dffb640 1 -- 10.20.193.28:0/999323972 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f2e38038060 con 0x55b674938b30 2026-02-25T11:48:29.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e548b7e00 10 client.15615 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:29.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e548b7e00 10 client.15615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:29.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e548b7e00 20 client.15615 populate_metadata read hostname 'trial028' 2026-02-25T11:48:29.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/999323972 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b67497b430 0x55b67497b8d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e548b7e00 4 client.15615 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:48:29.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.687+0000 7f2e548b7e00 1 -- 10.20.193.28:0/999323972 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b674816fc0 con 0x55b67497b430 2026-02-25T11:48:29.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.688+0000 7f2e4b7fe640 1 --2- 10.20.193.28:0/999323972 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b67497b430 0x55b67497b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.689+0000 7f2e4b7fe640 1 --2- 10.20.193.28:0/999323972 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b67497b430 0x55b67497b8d0 crc :-1 s=READY pgs=33 gs=4 cs=0 l=0 c_cookie=6330841901534010406 s_cookie=8134225962925545554 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57dbab2ace17dc26 server_cookie=70e2936a92efd852 in_seq=0 out_seq=0 2026-02-25T11:48:29.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.689+0000 7f2e0dffb640 10 client.15615 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:29.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.689+0000 7f2e0dffb640 1 -- 10.20.193.28:0/999323972 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55b674816fc0 con 0x55b67497b430 2026-02-25T11:48:29.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.689+0000 7f2e0dffb640 10 client.15615 handle_command_reply: tid=0 2026-02-25T11:48:29.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.689+0000 7f2e548b7e00 1 client.15615 shutdown 2026-02-25T11:48:29.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.689+0000 7f2e548b7e00 1 -- 10.20.193.28:0/999323972 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 msgr2=0x55b674950dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.689+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/999323972 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 0x55b674950dd0 secure :-1 s=READY pgs=409 gs=1 cs=0 l=1 c_cookie=2028685300421049213 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e38010c40 tx=0x7f2e380057f0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.690+0000 7f2e548b7e00 1 -- 10.20.193.28:0/999323972 shutdown_connections 2026-02-25T11:48:29.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.690+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/999323972 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b67497b430 0x55b67497b8d0 crc :-1 s=READY pgs=33 gs=4 cs=0 l=0 c_cookie=6330841901534010406 s_cookie=8134225962925545554 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.690+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/999323972 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b67497e0e0 0x55b674956300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.690+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/999323972 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674939440 0x55b674951310 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.690+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/999323972 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b674938b30 0x55b674950dd0 unknown :-1 s=CLOSED pgs=409 gs=1 cs=0 l=1 c_cookie=2028685300421049213 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.690+0000 7f2e548b7e00 1 -- 10.20.193.28:0/999323972 >> 10.20.193.28:0/999323972 conn(0x55b6748577a0 msgr2=0x55b674937b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:29.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.690+0000 7f2e0dffb640 0 client.15615 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:29.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.690+0000 7f2e548b7e00 1 -- 10.20.193.28:0/999323972 shutdown_connections 2026-02-25T11:48:29.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.690+0000 7f2e548b7e00 1 -- 10.20.193.28:0/999323972 wait complete. 2026-02-25T11:48:29.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.690+0000 7f2e548b7e00 20 client.15615 trim_cache size 0 max 16384 2026-02-25T11:48:29.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.697+0000 7f2e548b7e00 1 Processor -- start 2026-02-25T11:48:29.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e548b7e00 1 -- start start 2026-02-25T11:48:29.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e548b7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674938b30 0x55b674955870 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e548b7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b674939440 0x55b6749767e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e548b7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b67497e0e0 0x55b674976d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e548b7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b6748421a0 con 0x55b67497e0e0 2026-02-25T11:48:29.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e548b7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b674502700 con 0x55b674939440 2026-02-25T11:48:29.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e548b7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b674977e40 con 0x55b674938b30 2026-02-25T11:48:29.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e50e32640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b67497e0e0 0x55b674976d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e4bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674938b30 0x55b674955870 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e4bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674938b30 0x55b674955870 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:39334/0 (socket says 10.20.193.28:39334) 2026-02-25T11:48:29.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e50e32640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b67497e0e0 0x55b674976d20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58390/0 (socket says 10.20.193.28:58390) 2026-02-25T11:48:29.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e4bfff640 1 -- 10.20.193.28:0/706671845 learned_addr learned my addr 10.20.193.28:0/706671845 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:29.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e4b7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b674939440 0x55b6749767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e50e32640 1 -- 10.20.193.28:0/706671845 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674938b30 msgr2=0x55b674955870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e50e32640 1 --2- 10.20.193.28:0/706671845 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674938b30 0x55b674955870 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e50e32640 1 -- 10.20.193.28:0/706671845 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b674939440 msgr2=0x55b6749767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e50e32640 1 --2- 10.20.193.28:0/706671845 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b674939440 0x55b6749767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.698+0000 7f2e50e32640 1 -- 10.20.193.28:0/706671845 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6749436d0 con 0x55b67497e0e0 2026-02-25T11:48:29.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.699+0000 7f2e50e32640 1 --2- 10.20.193.28:0/706671845 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b67497e0e0 0x55b674976d20 secure :-1 s=READY pgs=410 gs=3 cs=0 l=1 c_cookie=12434530931165300274 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e40003bc0 tx=0x7f2e40013820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9054003cb70632 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:29.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.699+0000 7f2e0ffff640 1 -- 10.20.193.28:0/706671845 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e40018070 con 0x55b67497e0e0 2026-02-25T11:48:29.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.699+0000 7f2e0ffff640 1 -- 10.20.193.28:0/706671845 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2e4001d930 con 0x55b67497e0e0 2026-02-25T11:48:29.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.699+0000 7f2e0ffff640 1 -- 10.20.193.28:0/706671845 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e40011a90 con 0x55b67497e0e0 2026-02-25T11:48:29.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.699+0000 7f2e548b7e00 1 -- 10.20.193.28:0/706671845 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b67497e0e0 msgr2=0x55b674976d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.699+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/706671845 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b67497e0e0 0x55b674976d20 secure :-1 s=READY pgs=410 gs=3 cs=0 l=1 c_cookie=12434530931165300274 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e40003bc0 tx=0x7f2e40013820 comp rx=0 tx=0).stop 2026-02-25T11:48:29.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.699+0000 7f2e548b7e00 1 -- 10.20.193.28:0/706671845 shutdown_connections 2026-02-25T11:48:29.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.699+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/706671845 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b67497e0e0 0x55b674976d20 unknown :-1 s=CLOSED pgs=410 gs=3 cs=0 l=1 c_cookie=12434530931165300274 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.699+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/706671845 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b674939440 0x55b6749767e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.699+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/706671845 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674938b30 0x55b674955870 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.699+0000 7f2e548b7e00 1 -- 10.20.193.28:0/706671845 >> 10.20.193.28:0/706671845 conn(0x55b6748577a0 msgr2=0x55b674955160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:29.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.699+0000 7f2e548b7e00 1 -- 10.20.193.28:0/706671845 shutdown_connections 2026-02-25T11:48:29.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.699+0000 7f2e548b7e00 1 -- 10.20.193.28:0/706671845 wait complete. 2026-02-25T11:48:29.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.700+0000 7f2e548b7e00 1 -- start start 2026-02-25T11:48:29.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.700+0000 7f2e548b7e00 1 Processor -- start 2026-02-25T11:48:29.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.700+0000 7f2e548b7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674938b30 0x55b67497daa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.705 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.700+0000 7f2e548b7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b674939440 0x55b6749572f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.706 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.700+0000 7f2e548b7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b67497e0e0 0x55b674957830 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.706 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.700+0000 7f2e548b7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b674957e70 con 0x55b67497e0e0 2026-02-25T11:48:29.706 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.700+0000 7f2e548b7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b674957fe0 con 0x55b674939440 2026-02-25T11:48:29.706 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.700+0000 7f2e548b7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b6749581a0 con 0x55b674938b30 2026-02-25T11:48:29.706 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.701+0000 7f2e4bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674938b30 0x55b67497daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.706 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.701+0000 7f2e4bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674938b30 0x55b67497daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:39348/0 (socket says 10.20.193.28:39348) 2026-02-25T11:48:29.706 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.701+0000 7f2e4bfff640 1 -- 10.20.193.28:0/750238380 learned_addr learned my addr 10.20.193.28:0/750238380 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:29.706 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.701+0000 7f2e4b7fe640 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b674939440 0x55b6749572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.706 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.701+0000 7f2e50e32640 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b67497e0e0 0x55b674957830 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.701+0000 7f2e4bfff640 1 -- 10.20.193.28:0/750238380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b674939440 msgr2=0x55b6749572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.701+0000 7f2e4bfff640 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b674939440 0x55b6749572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.701+0000 7f2e4bfff640 1 -- 10.20.193.28:0/750238380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b67497e0e0 msgr2=0x55b674957830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:29.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.701+0000 7f2e4bfff640 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b67497e0e0 0x55b674957830 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.701+0000 7f2e4bfff640 1 -- 10.20.193.28:0/750238380 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6747d4b60 con 0x55b674938b30 2026-02-25T11:48:29.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.701+0000 7f2e4b7fe640 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b674939440 0x55b6749572f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:48:29.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.702+0000 7f2e4bfff640 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674938b30 0x55b67497daa0 secure :-1 s=READY pgs=121 gs=1 cs=0 l=1 c_cookie=14396989752877292943 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e3800baf0 tx=0x7f2e38036000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7cc61e6d7309d8f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:29.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.702+0000 7f2e0dffb640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:48:29.709 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.702+0000 7f2e0dffb640 1 -- 10.20.193.28:0/750238380 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e3803e070 con 0x55b674938b30 2026-02-25T11:48:29.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.702+0000 7f2e0dffb640 1 -- 10.20.193.28:0/750238380 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55b6744da6f0 con 0x55b674938b30 2026-02-25T11:48:29.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.702+0000 7f2e548b7e00 1 -- 10.20.193.28:0/750238380 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e40003860 con 0x55b674938b30 2026-02-25T11:48:29.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.702+0000 7f2e548b7e00 10 client.25447 _ll_register_callbacks cb 0x55b6747c5a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:29.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.702+0000 7f2e548b7e00 1 -- 10.20.193.28:0/750238380 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e38015070 con 0x55b674938b30 2026-02-25T11:48:29.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.702+0000 7f2e0dffb640 1 -- 10.20.193.28:0/750238380 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2e3802c030 con 0x55b674938b30 2026-02-25T11:48:29.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.702+0000 7f2e0dffb640 1 -- 10.20.193.28:0/750238380 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f2e38017040 con 0x55b674938b30 2026-02-25T11:48:29.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.702+0000 7f2e0dffb640 1 -- 10.20.193.28:0/750238380 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2e380045d0 con 0x55b674938b30 2026-02-25T11:48:29.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.702+0000 7f2e548b7e00 10 client.25447 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:29.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.702+0000 7f2e548b7e00 1 -- 10.20.193.28:0/750238380 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55b6747c86f0 con 0x55b674938b30 2026-02-25T11:48:29.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.703+0000 7f2e0dffb640 1 -- 10.20.193.28:0/750238380 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f2e3804a020 con 0x55b674938b30 2026-02-25T11:48:29.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.703+0000 7f2e548b7e00 10 client.25447 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:29.710 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.703+0000 7f2e548b7e00 10 client.25447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:29.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.703+0000 7f2e548b7e00 20 client.25447 populate_metadata read hostname 'trial028' 2026-02-25T11:48:29.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.703+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b674a2afb0 0x55b674a4b3d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:29.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.703+0000 7f2e548b7e00 4 client.25447 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:48:29.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.703+0000 7f2e548b7e00 1 -- 10.20.193.28:0/750238380 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6744d7f40 con 0x55b674a2afb0 2026-02-25T11:48:29.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.703+0000 7f2e4b7fe640 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b674a2afb0 0x55b674a4b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:29.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.704+0000 7f2e4b7fe640 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b674a2afb0 0x55b674a4b3d0 crc :-1 s=READY pgs=34 gs=4 cs=0 l=0 c_cookie=3867693847394501883 s_cookie=3635248490690066680 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35accf0ff1e9c4fb server_cookie=3272ff3deddb38f8 in_seq=0 out_seq=0 2026-02-25T11:48:29.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.704+0000 7f2e0dffb640 10 client.25447 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:29.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.704+0000 7f2e0dffb640 1 -- 10.20.193.28:0/750238380 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6744d7f40 con 0x55b674a2afb0 2026-02-25T11:48:29.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.704+0000 7f2e0dffb640 10 client.25447 handle_command_reply: tid=0 2026-02-25T11:48:29.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.704+0000 7f2e548b7e00 1 client.25447 shutdown 2026-02-25T11:48:29.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.704+0000 7f2e548b7e00 1 -- 10.20.193.28:0/750238380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674938b30 msgr2=0x55b67497daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.704+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674938b30 0x55b67497daa0 secure :-1 s=READY pgs=121 gs=1 cs=0 l=1 c_cookie=14396989752877292943 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e3800baf0 tx=0x7f2e38036000 comp rx=0 tx=0).stop 2026-02-25T11:48:29.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e548b7e00 1 -- 10.20.193.28:0/750238380 shutdown_connections 2026-02-25T11:48:29.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b674a2afb0 0x55b674a4b3d0 crc :-1 s=READY pgs=34 gs=4 cs=0 l=0 c_cookie=3867693847394501883 s_cookie=3635248490690066680 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b67497e0e0 0x55b674957830 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b674939440 0x55b6749572f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e548b7e00 1 --2- 10.20.193.28:0/750238380 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b674938b30 0x55b67497daa0 unknown :-1 s=CLOSED pgs=121 gs=1 cs=0 l=1 c_cookie=14396989752877292943 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e548b7e00 1 -- 10.20.193.28:0/750238380 >> 10.20.193.28:0/750238380 conn(0x55b6748577a0 msgr2=0x55b674955950 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:29.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e0dffb640 0 client.25447 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:29.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e548b7e00 1 -- 10.20.193.28:0/750238380 shutdown_connections 2026-02-25T11:48:29.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e548b7e00 1 -- 10.20.193.28:0/750238380 wait complete. 2026-02-25T11:48:29.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e548b7e00 20 client.25447 trim_cache size 0 max 16384 2026-02-25T11:48:29.713 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:48:29.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e528bc640 1 -- 10.20.193.28:0/1334833322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2e200b57f0 msgr2=0x7f2e200d5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e528bc640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2e200b57f0 0x7f2e200d5ca0 secure :-1 s=READY pgs=265 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e3800b2c0 tx=0x7f2e3800b4a0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e528bc640 1 -- 10.20.193.28:0/1334833322 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e4c110130 msgr2=0x7f2e4c176200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:29.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e528bc640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e4c110130 0x7f2e4c176200 secure :-1 s=READY pgs=407 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2e3c009950 tx=0x7f2e3c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e528bc640 1 -- 10.20.193.28:0/1334833322 shutdown_connections 2026-02-25T11:48:29.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e528bc640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f2e200b57f0 0x7f2e200d5ca0 unknown :-1 s=CLOSED pgs=265 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e528bc640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2e4c115930 0x7f2e4c10ba20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e528bc640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2e4c110130 0x7f2e4c176200 unknown :-1 s=CLOSED pgs=407 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e528bc640 1 --2- 10.20.193.28:0/1334833322 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2e4c10f740 0x7f2e4c10b360 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:29.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e528bc640 1 -- 10.20.193.28:0/1334833322 >> 10.20.193.28:0/1334833322 conn(0x7f2e4c0ecff0 msgr2=0x7f2e4c112f90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:29.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e528bc640 1 -- 10.20.193.28:0/1334833322 shutdown_connections 2026-02-25T11:48:29.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:29.705+0000 7f2e528bc640 1 -- 10.20.193.28:0/1334833322 wait complete. 2026-02-25T11:48:30.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:30 trial167 ceph-mon[43392]: pgmap v234: 97 pgs: 97 active+clean; 706 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 5.7 KiB/s rd, 4.8 KiB/s wr, 7 op/s; 103 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:30.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:30 trial119 ceph-mon[42710]: pgmap v234: 97 pgs: 97 active+clean; 706 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 5.7 KiB/s rd, 4.8 KiB/s wr, 7 op/s; 103 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:30 trial028 ceph-mon[39224]: pgmap v234: 97 pgs: 97 active+clean; 706 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 5.7 KiB/s rd, 4.8 KiB/s wr, 7 op/s; 103 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:30.808 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:48:30.809 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:48:30.995 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:48:31.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.163+0000 7f4faf248640 1 -- 10.20.193.28:0/2356339074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fa810ddc0 msgr2=0x7f4fa8112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.163+0000 7f4faf248640 1 --2- 10.20.193.28:0/2356339074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fa810ddc0 0x7f4fa8112e10 secure :-1 s=READY pgs=411 gs=1 cs=0 l=1 c_cookie=8757345542668870915 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f9c009950 tx=0x7f4f9c02f190 comp rx=0 tx=0).stop 2026-02-25T11:48:31.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.163+0000 7f4faf248640 1 -- 10.20.193.28:0/2356339074 shutdown_connections 2026-02-25T11:48:31.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.163+0000 7f4faf248640 1 --2- 10.20.193.28:0/2356339074 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fa81135c0 0x7f4fa8173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.163+0000 7f4faf248640 1 --2- 10.20.193.28:0/2356339074 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fa810ddc0 0x7f4fa8112e10 unknown :-1 s=CLOSED pgs=411 gs=1 cs=0 l=1 c_cookie=8757345542668870915 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.163+0000 7f4faf248640 1 --2- 10.20.193.28:0/2356339074 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4fa810d3d0 0x7f4fa810d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.163+0000 7f4faf248640 1 -- 10.20.193.28:0/2356339074 >> 10.20.193.28:0/2356339074 conn(0x7f4fa80eaca0 msgr2=0x7f4fa810b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:31.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.163+0000 7f4faf248640 1 -- 10.20.193.28:0/2356339074 shutdown_connections 2026-02-25T11:48:31.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.163+0000 7f4faf248640 1 -- 10.20.193.28:0/2356339074 wait complete. 2026-02-25T11:48:31.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.164+0000 7f4faf248640 1 Processor -- start 2026-02-25T11:48:31.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.164+0000 7f4faf248640 1 -- start start 2026-02-25T11:48:31.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.164+0000 7f4faf248640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fa810d3d0 0x7f4fa80f36d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.164+0000 7f4faf248640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4fa810ddc0 0x7f4fa8173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.164+0000 7f4faf248640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fa81135c0 0x7f4fa80f3dd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.164+0000 7f4faf248640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa80f8fe0 con 0x7f4fa81135c0 2026-02-25T11:48:31.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.164+0000 7f4faf248640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa80f9150 con 0x7f4fa810ddc0 2026-02-25T11:48:31.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.164+0000 7f4faf248640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa80f9310 con 0x7f4fa810d3d0 2026-02-25T11:48:31.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4faea47640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fa81135c0 0x7f4fa80f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4fada45640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4fa810ddc0 0x7f4fa8173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4faea47640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fa81135c0 0x7f4fa80f3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58418/0 (socket says 10.20.193.28:58418) 2026-02-25T11:48:31.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4faea47640 1 -- 10.20.193.28:0/1453597991 learned_addr learned my addr 10.20.193.28:0/1453597991 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:31.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4fae246640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fa810d3d0 0x7f4fa80f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4faea47640 1 -- 10.20.193.28:0/1453597991 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fa810d3d0 msgr2=0x7f4fa80f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4faea47640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fa810d3d0 0x7f4fa80f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4faea47640 1 -- 10.20.193.28:0/1453597991 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4fa810ddc0 msgr2=0x7f4fa8173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4faea47640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4fa810ddc0 0x7f4fa8173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4faea47640 1 -- 10.20.193.28:0/1453597991 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9c009570 con 0x7f4fa81135c0 2026-02-25T11:48:31.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4fae246640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fa810d3d0 0x7f4fa80f36d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:48:31.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4faea47640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fa81135c0 0x7f4fa80f3dd0 secure :-1 s=READY pgs=412 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fa4006e90 tx=0x7f4fa400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:31.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4f8f7fe640 1 -- 10.20.193.28:0/1453597991 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4fa4005500 con 0x7f4fa81135c0 2026-02-25T11:48:31.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4faf248640 1 -- 10.20.193.28:0/1453597991 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa80f95f0 con 0x7f4fa81135c0 2026-02-25T11:48:31.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4f8f7fe640 1 -- 10.20.193.28:0/1453597991 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4fa4005df0 con 0x7f4fa81135c0 2026-02-25T11:48:31.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.165+0000 7f4f8f7fe640 1 -- 10.20.193.28:0/1453597991 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4fa4005500 con 0x7f4fa81135c0 2026-02-25T11:48:31.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.166+0000 7f4faf248640 1 -- 10.20.193.28:0/1453597991 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa80f9ac0 con 0x7f4fa81135c0 2026-02-25T11:48:31.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.168+0000 7f4f8f7fe640 1 -- 10.20.193.28:0/1453597991 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7f4fa4005850 con 0x7f4fa81135c0 2026-02-25T11:48:31.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.170+0000 7f4f8f7fe640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f840b57b0 0x7f4f840d5c60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.170+0000 7f4f8f7fe640 1 -- 10.20.193.28:0/1453597991 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4fa40e76c0 con 0x7f4fa81135c0 2026-02-25T11:48:31.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.170+0000 7f4fae246640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f840b57b0 0x7f4f840d5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.170+0000 7f4fae246640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f840b57b0 0x7f4f840d5c60 secure :-1 s=READY pgs=266 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f98005970 tx=0x7f4f98005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:31.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fb24cce00 1 Processor -- start 2026-02-25T11:48:31.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fb24cce00 1 -- start start 2026-02-25T11:48:31.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fb24cce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518cd750 0x55ff518cdb20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fb24cce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 0x55ff51912610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fb24cce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff51912b50 0x55ff51932f30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fb24cce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ff517e3d80 con 0x55ff518ce060 2026-02-25T11:48:31.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fb24cce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ff517dfab0 con 0x55ff51912b50 2026-02-25T11:48:31.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fb24cce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ff517d9a50 con 0x55ff518cd750 2026-02-25T11:48:31.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fada45640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 0x55ff51912610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fada45640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 0x55ff51912610 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58426/0 (socket says 10.20.193.28:58426) 2026-02-25T11:48:31.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fada45640 1 -- 10.20.193.28:0/1189454713 learned_addr learned my addr 10.20.193.28:0/1189454713 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:31.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4faea47640 1 --2- 10.20.193.28:0/1189454713 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff51912b50 0x55ff51932f30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fae246640 1 --2- 10.20.193.28:0/1189454713 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518cd750 0x55ff518cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fada45640 1 -- 10.20.193.28:0/1189454713 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518cd750 msgr2=0x55ff518cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fada45640 1 --2- 10.20.193.28:0/1189454713 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518cd750 0x55ff518cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fada45640 1 -- 10.20.193.28:0/1189454713 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff51912b50 msgr2=0x55ff51932f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fada45640 1 --2- 10.20.193.28:0/1189454713 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff51912b50 0x55ff51932f30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.172+0000 7f4fada45640 1 -- 10.20.193.28:0/1189454713 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff51785120 con 0x55ff518ce060 2026-02-25T11:48:31.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.173+0000 7f4fada45640 1 --2- 10.20.193.28:0/1189454713 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 0x55ff51912610 secure :-1 s=READY pgs=413 gs=3 cs=0 l=1 c_cookie=526460537102869684 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f9c005bb0 tx=0x7f4f9c002390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e5d11b64724b4 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:31.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.173+0000 7f4f8ca6d640 1 -- 10.20.193.28:0/1189454713 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f9c03d070 con 0x55ff518ce060 2026-02-25T11:48:31.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.173+0000 7f4f8ca6d640 1 -- 10.20.193.28:0/1189454713 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4f9c03e040 con 0x55ff518ce060 2026-02-25T11:48:31.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.173+0000 7f4f8ca6d640 1 -- 10.20.193.28:0/1189454713 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f9c02fea0 con 0x55ff518ce060 2026-02-25T11:48:31.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.173+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1189454713 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 msgr2=0x55ff51912610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.173+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1189454713 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 0x55ff51912610 secure :-1 s=READY pgs=413 gs=3 cs=0 l=1 c_cookie=526460537102869684 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f9c005bb0 tx=0x7f4f9c002390 comp rx=0 tx=0).stop 2026-02-25T11:48:31.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.173+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1189454713 shutdown_connections 2026-02-25T11:48:31.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.173+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1189454713 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff51912b50 0x55ff51932f30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.173+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1189454713 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 0x55ff51912610 unknown :-1 s=CLOSED pgs=413 gs=3 cs=0 l=1 c_cookie=526460537102869684 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.173+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1189454713 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518cd750 0x55ff518cdb20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.173+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1189454713 >> 10.20.193.28:0/1189454713 conn(0x55ff518ab8f0 msgr2=0x55ff518cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:31.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.173+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1189454713 shutdown_connections 2026-02-25T11:48:31.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.174+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1189454713 wait complete. 2026-02-25T11:48:31.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.175+0000 7f4fb24cce00 1 -- start start 2026-02-25T11:48:31.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.175+0000 7f4fb24cce00 1 Processor -- start 2026-02-25T11:48:31.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.175+0000 7f4fb24cce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518cd750 0x55ff5192a530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.175+0000 7f4fb24cce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 0x55ff5192aa70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.175+0000 7f4fb24cce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff51912b50 0x55ff5192fa60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.175+0000 7f4fb24cce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ff5192ffa0 con 0x55ff518ce060 2026-02-25T11:48:31.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.175+0000 7f4fb24cce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ff51930110 con 0x55ff51912b50 2026-02-25T11:48:31.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.175+0000 7f4fb24cce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ff519302d0 con 0x55ff518cd750 2026-02-25T11:48:31.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.175+0000 7f4fada45640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 0x55ff5192aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4fada45640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 0x55ff5192aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58442/0 (socket says 10.20.193.28:58442) 2026-02-25T11:48:31.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4fada45640 1 -- 10.20.193.28:0/116383305 learned_addr learned my addr 10.20.193.28:0/116383305 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:31.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4fae246640 1 --2- 10.20.193.28:0/116383305 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518cd750 0x55ff5192a530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4fada45640 1 -- 10.20.193.28:0/116383305 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518cd750 msgr2=0x55ff5192a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4fada45640 1 --2- 10.20.193.28:0/116383305 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518cd750 0x55ff5192a530 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4fada45640 1 -- 10.20.193.28:0/116383305 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff51912b50 msgr2=0x55ff5192fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4fada45640 1 --2- 10.20.193.28:0/116383305 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff51912b50 0x55ff5192fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4fada45640 1 -- 10.20.193.28:0/116383305 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff5175d810 con 0x55ff518ce060 2026-02-25T11:48:31.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4fae246640 1 --2- 10.20.193.28:0/116383305 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518cd750 0x55ff5192a530 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:48:31.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4fada45640 1 --2- 10.20.193.28:0/116383305 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 0x55ff5192aa70 secure :-1 s=READY pgs=414 gs=1 cs=0 l=1 c_cookie=6376976754701154105 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f9c003fd0 tx=0x7f4f9c004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587f92925ce2b739 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:31.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4f57fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:31.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4f57fff640 1 -- 10.20.193.28:0/116383305 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f9c03d070 con 0x55ff518ce060 2026-02-25T11:48:31.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4f57fff640 1 -- 10.20.193.28:0/116383305 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4f9c005900 con 0x55ff518ce060 2026-02-25T11:48:31.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4f57fff640 1 -- 10.20.193.28:0/116383305 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f9c042680 con 0x55ff518ce060 2026-02-25T11:48:31.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4fb24cce00 1 -- 10.20.193.28:0/116383305 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa8174800 con 0x55ff518ce060 2026-02-25T11:48:31.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4fb24cce00 10 client.15633 _ll_register_callbacks cb 0x55ff5175ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:31.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.176+0000 7f4fb24cce00 1 -- 10.20.193.28:0/116383305 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ff51531530 con 0x55ff518ce060 2026-02-25T11:48:31.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.177+0000 7f4f57fff640 1 -- 10.20.193.28:0/116383305 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4f9c0386a0 con 0x55ff518ce060 2026-02-25T11:48:31.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.177+0000 7f4f57fff640 1 -- 10.20.193.28:0/116383305 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4f9c038880 con 0x55ff518ce060 2026-02-25T11:48:31.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.177+0000 7f4fb24cce00 10 client.15633 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:31.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.177+0000 7f4fb24cce00 1 -- 10.20.193.28:0/116383305 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55ff5176f4f0 con 0x55ff518ce060 2026-02-25T11:48:31.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.177+0000 7f4f57fff640 1 -- 10.20.193.28:0/116383305 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f4f9c042810 con 0x55ff518ce060 2026-02-25T11:48:31.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.177+0000 7f4fb24cce00 10 client.15633 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:31.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.177+0000 7f4fb24cce00 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:31.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.177+0000 7f4fb24cce00 20 client.15633 populate_metadata read hostname 'trial028' 2026-02-25T11:48:31.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.177+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/116383305 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ff5190fba0 0x55ff51910040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.177+0000 7f4fb24cce00 4 client.15633 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:48:31.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.177+0000 7f4fb24cce00 1 -- 10.20.193.28:0/116383305 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff51786b50 con 0x55ff5190fba0 2026-02-25T11:48:31.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.177+0000 7f4fae246640 1 --2- 10.20.193.28:0/116383305 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ff5190fba0 0x55ff51910040 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.178+0000 7f4f57fff640 10 client.15633 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:31.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.178+0000 7f4fae246640 1 --2- 10.20.193.28:0/116383305 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ff5190fba0 0x55ff51910040 crc :-1 s=READY pgs=35 gs=4 cs=0 l=0 c_cookie=12262712093548313703 s_cookie=9448644769963394393 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa2de7adaed57c67 server_cookie=8320545b72854559 in_seq=0 out_seq=0 2026-02-25T11:48:31.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4f57fff640 1 -- 10.20.193.28:0/116383305 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55ff51786b50 con 0x55ff5190fba0 2026-02-25T11:48:31.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4f57fff640 10 client.15633 handle_command_reply: tid=0 2026-02-25T11:48:31.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4fb24cce00 1 client.15633 shutdown 2026-02-25T11:48:31.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4fb24cce00 1 -- 10.20.193.28:0/116383305 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 msgr2=0x55ff5192aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/116383305 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 0x55ff5192aa70 secure :-1 s=READY pgs=414 gs=1 cs=0 l=1 c_cookie=6376976754701154105 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f9c003fd0 tx=0x7f4f9c004280 comp rx=0 tx=0).stop 2026-02-25T11:48:31.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4fb24cce00 1 -- 10.20.193.28:0/116383305 shutdown_connections 2026-02-25T11:48:31.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/116383305 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ff5190fba0 0x55ff51910040 crc :-1 s=READY pgs=35 gs=4 cs=0 l=0 c_cookie=12262712093548313703 s_cookie=9448644769963394393 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/116383305 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff51912b50 0x55ff5192fa60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/116383305 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518ce060 0x55ff5192aa70 unknown :-1 s=CLOSED pgs=414 gs=1 cs=0 l=1 c_cookie=6376976754701154105 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/116383305 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518cd750 0x55ff5192a530 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4f57fff640 0 client.15633 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:31.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4fb24cce00 1 -- 10.20.193.28:0/116383305 >> 10.20.193.28:0/116383305 conn(0x55ff518ab8f0 msgr2=0x55ff518cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:31.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4fb24cce00 1 -- 10.20.193.28:0/116383305 shutdown_connections 2026-02-25T11:48:31.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4fb24cce00 1 -- 10.20.193.28:0/116383305 wait complete. 2026-02-25T11:48:31.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.179+0000 7f4fb24cce00 20 client.15633 trim_cache size 0 max 16384 2026-02-25T11:48:31.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.189+0000 7f4fb24cce00 1 Processor -- start 2026-02-25T11:48:31.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.189+0000 7f4fb24cce00 1 -- start start 2026-02-25T11:48:31.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.189+0000 7f4fb24cce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518cd750 0x55ff519276d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.189+0000 7f4fb24cce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff518ce060 0x55ff51927c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.189+0000 7f4fb24cce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff51912b50 0x55ff5190d740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.189+0000 7f4fb24cce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ff517d7680 con 0x55ff518cd750 2026-02-25T11:48:31.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.189+0000 7f4fb24cce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ff51497790 con 0x55ff518ce060 2026-02-25T11:48:31.192 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.189+0000 7f4fb24cce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ff51930410 con 0x55ff51912b50 2026-02-25T11:48:31.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.189+0000 7f4fae246640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518cd750 0x55ff519276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.189+0000 7f4fae246640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518cd750 0x55ff519276d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58448/0 (socket says 10.20.193.28:58448) 2026-02-25T11:48:31.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.189+0000 7f4fae246640 1 -- 10.20.193.28:0/1873035079 learned_addr learned my addr 10.20.193.28:0/1873035079 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:31.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.189+0000 7f4faea47640 1 --2- 10.20.193.28:0/1873035079 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff51912b50 0x55ff5190d740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.189+0000 7f4fada45640 1 --2- 10.20.193.28:0/1873035079 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff518ce060 0x55ff51927c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4fae246640 1 -- 10.20.193.28:0/1873035079 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff51912b50 msgr2=0x55ff5190d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4fae246640 1 --2- 10.20.193.28:0/1873035079 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff51912b50 0x55ff5190d740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4fae246640 1 -- 10.20.193.28:0/1873035079 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff518ce060 msgr2=0x55ff51927c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4fae246640 1 --2- 10.20.193.28:0/1873035079 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff518ce060 0x55ff51927c10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4fae246640 1 -- 10.20.193.28:0/1873035079 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff5191cac0 con 0x55ff518cd750 2026-02-25T11:48:31.193 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4faea47640 1 --2- 10.20.193.28:0/1873035079 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff51912b50 0x55ff5190d740 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:48:31.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4fae246640 1 --2- 10.20.193.28:0/1873035079 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518cd750 0x55ff519276d0 secure :-1 s=READY pgs=415 gs=1 cs=0 l=1 c_cookie=936742890989149296 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f98014040 tx=0x7f4f98009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfffab23f47dc70 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:31.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4f764e6640 1 -- 10.20.193.28:0/1873035079 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f98024040 con 0x55ff518cd750 2026-02-25T11:48:31.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4f764e6640 1 -- 10.20.193.28:0/1873035079 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4f98011e60 con 0x55ff518cd750 2026-02-25T11:48:31.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4f764e6640 1 -- 10.20.193.28:0/1873035079 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f98028780 con 0x55ff518cd750 2026-02-25T11:48:31.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1873035079 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518cd750 msgr2=0x55ff519276d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1873035079 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518cd750 0x55ff519276d0 secure :-1 s=READY pgs=415 gs=1 cs=0 l=1 c_cookie=936742890989149296 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f98014040 tx=0x7f4f98009000 comp rx=0 tx=0).stop 2026-02-25T11:48:31.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1873035079 shutdown_connections 2026-02-25T11:48:31.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1873035079 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff51912b50 0x55ff5190d740 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.197 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1873035079 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff518ce060 0x55ff51927c10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.190+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1873035079 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff518cd750 0x55ff519276d0 unknown :-1 s=CLOSED pgs=415 gs=1 cs=0 l=1 c_cookie=936742890989149296 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.191+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1873035079 >> 10.20.193.28:0/1873035079 conn(0x55ff518ab8f0 msgr2=0x55ff51926fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:31.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.191+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1873035079 shutdown_connections 2026-02-25T11:48:31.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.191+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1873035079 wait complete. 2026-02-25T11:48:31.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.191+0000 7f4fb24cce00 1 -- start start 2026-02-25T11:48:31.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.191+0000 7f4fb24cce00 1 Processor -- start 2026-02-25T11:48:31.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.191+0000 7f4fb24cce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff518cd750 0x55ff518ad5a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.191+0000 7f4fb24cce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518ce060 0x55ff518adae0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.191+0000 7f4fb24cce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff51912b50 0x55ff51961330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.191+0000 7f4fb24cce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ff5192adf0 con 0x55ff51912b50 2026-02-25T11:48:31.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.191+0000 7f4fb24cce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ff518ae2b0 con 0x55ff518cd750 2026-02-25T11:48:31.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.191+0000 7f4fb24cce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ff518ae470 con 0x55ff518ce060 2026-02-25T11:48:31.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.192+0000 7f4fada45640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518ce060 0x55ff518adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.192+0000 7f4fada45640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518ce060 0x55ff518adae0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:39404/0 (socket says 10.20.193.28:39404) 2026-02-25T11:48:31.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.192+0000 7f4fada45640 1 -- 10.20.193.28:0/1705834949 learned_addr learned my addr 10.20.193.28:0/1705834949 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:31.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.192+0000 7f4fae246640 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff518cd750 0x55ff518ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.192+0000 7f4faea47640 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff51912b50 0x55ff51961330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.192+0000 7f4fada45640 1 -- 10.20.193.28:0/1705834949 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff518cd750 msgr2=0x55ff518ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.192+0000 7f4fada45640 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff518cd750 0x55ff518ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.192+0000 7f4fada45640 1 -- 10.20.193.28:0/1705834949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff51912b50 msgr2=0x55ff51961330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.192+0000 7f4fada45640 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff51912b50 0x55ff51961330 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.192+0000 7f4fada45640 1 -- 10.20.193.28:0/1705834949 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff5175db40 con 0x55ff518ce060 2026-02-25T11:48:31.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.192+0000 7f4faea47640 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff51912b50 0x55ff51961330 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:48:31.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.192+0000 7f4fada45640 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518ce060 0x55ff518adae0 secure :-1 s=READY pgs=122 gs=2 cs=0 l=1 c_cookie=5141033043101893773 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f9c004500 tx=0x7f4f9c02f9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47589e5078a2448d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:31.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.192+0000 7f4f57fff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:48:31.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4f57fff640 1 -- 10.20.193.28:0/1705834949 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f9c03d040 con 0x55ff518ce060 2026-02-25T11:48:31.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4f57fff640 1 -- 10.20.193.28:0/1705834949 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4f9c05a030 con 0x55ff518ce060 2026-02-25T11:48:31.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1705834949 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff51769f70 con 0x55ff518ce060 2026-02-25T11:48:31.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4f57fff640 1 -- 10.20.193.28:0/1705834949 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f9c038900 con 0x55ff518ce060 2026-02-25T11:48:31.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4fb24cce00 10 client.25459 _ll_register_callbacks cb 0x55ff5175ae60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:31.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1705834949 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f9c03e040 con 0x55ff518ce060 2026-02-25T11:48:31.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4f57fff640 1 -- 10.20.193.28:0/1705834949 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4f9c038a60 con 0x55ff518ce060 2026-02-25T11:48:31.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4f57fff640 1 -- 10.20.193.28:0/1705834949 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4f9c038c60 con 0x55ff518ce060 2026-02-25T11:48:31.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4fb24cce00 10 client.25459 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:31.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1705834949 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55ff5176f750 con 0x55ff518ce060 2026-02-25T11:48:31.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4f57fff640 1 -- 10.20.193.28:0/1705834949 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f4f9c037890 con 0x55ff518ce060 2026-02-25T11:48:31.200 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4fb24cce00 10 client.25459 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:31.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4fb24cce00 10 client.25459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:31.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4fb24cce00 20 client.25459 populate_metadata read hostname 'trial028' 2026-02-25T11:48:31.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ff51962fe0 0x55ff51963480 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4fb24cce00 4 client.25459 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:48:31.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.193+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1705834949 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ff5163a690 con 0x55ff51962fe0 2026-02-25T11:48:31.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.194+0000 7f4fae246640 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ff51962fe0 0x55ff51963480 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.195+0000 7f4fae246640 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ff51962fe0 0x55ff51963480 crc :-1 s=READY pgs=36 gs=4 cs=0 l=0 c_cookie=2831289738196424386 s_cookie=124107014737108259 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=274ac3041d6816c2 server_cookie=1b8eaac1f8a1d23 in_seq=0 out_seq=0 2026-02-25T11:48:31.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.195+0000 7f4f57fff640 10 client.25459 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:31.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.195+0000 7f4f57fff640 1 -- 10.20.193.28:0/1705834949 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ff5163a690 con 0x55ff51962fe0 2026-02-25T11:48:31.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.195+0000 7f4f57fff640 10 client.25459 handle_command_reply: tid=0 2026-02-25T11:48:31.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.195+0000 7f4fb24cce00 1 client.25459 shutdown 2026-02-25T11:48:31.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.195+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1705834949 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518ce060 msgr2=0x55ff518adae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.195+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518ce060 0x55ff518adae0 secure :-1 s=READY pgs=122 gs=2 cs=0 l=1 c_cookie=5141033043101893773 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f9c004500 tx=0x7f4f9c02f9f0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1705834949 shutdown_connections 2026-02-25T11:48:31.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ff51962fe0 0x55ff51963480 crc :-1 s=READY pgs=36 gs=4 cs=0 l=0 c_cookie=2831289738196424386 s_cookie=124107014737108259 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ff51912b50 0x55ff51961330 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ff518ce060 0x55ff518adae0 unknown :-1 s=CLOSED pgs=122 gs=2 cs=0 l=1 c_cookie=5141033043101893773 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4fb24cce00 1 --2- 10.20.193.28:0/1705834949 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ff518cd750 0x55ff518ad5a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4f57fff640 0 client.25459 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:31.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1705834949 >> 10.20.193.28:0/1705834949 conn(0x55ff518ab8f0 msgr2=0x55ff519276d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:31.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1705834949 shutdown_connections 2026-02-25T11:48:31.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4fb24cce00 1 -- 10.20.193.28:0/1705834949 wait complete. 2026-02-25T11:48:31.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4fb24cce00 20 client.25459 trim_cache size 0 max 16384 2026-02-25T11:48:31.204 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:48:31.204 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:48:31.204 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "df724da3-1c68-4141-ad99-f8332cf2f6c6", 2026-02-25T11:48:31.204 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:48:31.204 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:48:31.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4faf248640 1 -- 10.20.193.28:0/1453597991 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f840b57b0 msgr2=0x7f4f840d5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4faf248640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f840b57b0 0x7f4f840d5c60 secure :-1 s=READY pgs=266 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f98005970 tx=0x7f4f98005900 comp rx=0 tx=0).stop 2026-02-25T11:48:31.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4faf248640 1 -- 10.20.193.28:0/1453597991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fa81135c0 msgr2=0x7f4fa80f3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4faf248640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fa81135c0 0x7f4fa80f3dd0 secure :-1 s=READY pgs=412 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4fa4006e90 tx=0x7f4fa400edf0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4faf248640 1 -- 10.20.193.28:0/1453597991 shutdown_connections 2026-02-25T11:48:31.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4faf248640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4f840b57b0 0x7f4f840d5c60 unknown :-1 s=CLOSED pgs=266 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4faf248640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4fa81135c0 0x7f4fa80f3dd0 unknown :-1 s=CLOSED pgs=412 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4faf248640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4fa810ddc0 0x7f4fa8173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4faf248640 1 --2- 10.20.193.28:0/1453597991 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4fa810d3d0 0x7f4fa80f36d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.196+0000 7f4faf248640 1 -- 10.20.193.28:0/1453597991 >> 10.20.193.28:0/1453597991 conn(0x7f4fa80eaca0 msgr2=0x7f4fa810b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:31.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.197+0000 7f4faf248640 1 -- 10.20.193.28:0/1453597991 shutdown_connections 2026-02-25T11:48:31.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.197+0000 7f4faf248640 1 -- 10.20.193.28:0/1453597991 wait complete. 2026-02-25T11:48:31.299 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:48:31.484 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:48:31.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.667+0000 7fbdf570f640 1 -- 10.20.193.28:0/1809790150 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbdf0110080 msgr2=0x7fbdf01150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.667+0000 7fbdf570f640 1 --2- 10.20.193.28:0/1809790150 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbdf0110080 0x7fbdf01150d0 secure :-1 s=READY pgs=416 gs=1 cs=0 l=1 c_cookie=16017133624341899154 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbde0009950 tx=0x7fbde002f190 comp rx=0 tx=0).stop 2026-02-25T11:48:31.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.667+0000 7fbdf570f640 1 -- 10.20.193.28:0/1809790150 shutdown_connections 2026-02-25T11:48:31.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.667+0000 7fbdf570f640 1 --2- 10.20.193.28:0/1809790150 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbdf0115880 0x7fbdf0176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.667+0000 7fbdf570f640 1 --2- 10.20.193.28:0/1809790150 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbdf0110080 0x7fbdf01150d0 unknown :-1 s=CLOSED pgs=416 gs=1 cs=0 l=1 c_cookie=16017133624341899154 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.667+0000 7fbdf570f640 1 --2- 10.20.193.28:0/1809790150 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbdf010f690 0x7fbdf010fa60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.667+0000 7fbdf570f640 1 -- 10.20.193.28:0/1809790150 >> 10.20.193.28:0/1809790150 conn(0x7fbdf00ecf00 msgr2=0x7fbdf010d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:31.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.667+0000 7fbdf570f640 1 -- 10.20.193.28:0/1809790150 shutdown_connections 2026-02-25T11:48:31.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.667+0000 7fbdf570f640 1 -- 10.20.193.28:0/1809790150 wait complete. 2026-02-25T11:48:31.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.668+0000 7fbdf570f640 1 Processor -- start 2026-02-25T11:48:31.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.668+0000 7fbdf570f640 1 -- start start 2026-02-25T11:48:31.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.668+0000 7fbdf570f640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbdf010f690 0x7fbdf00f5a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.668+0000 7fbdf570f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbdf0110080 0x7fbdf0176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.668+0000 7fbdf570f640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbdf0115880 0x7fbdf00f6120 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.668+0000 7fbdf570f640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf00fb330 con 0x7fbdf0110080 2026-02-25T11:48:31.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.668+0000 7fbdf570f640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf00fb4a0 con 0x7fbdf0115880 2026-02-25T11:48:31.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.668+0000 7fbdf570f640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf00fb660 con 0x7fbdf010f690 2026-02-25T11:48:31.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbdef7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbdf0110080 0x7fbdf0176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbdef7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbdf0110080 0x7fbdf0176150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58470/0 (socket says 10.20.193.28:58470) 2026-02-25T11:48:31.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbdef7fe640 1 -- 10.20.193.28:0/229588038 learned_addr learned my addr 10.20.193.28:0/229588038 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:31.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbdeffff640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbdf010f690 0x7fbdf00f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbdf4f0e640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbdf0115880 0x7fbdf00f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbdef7fe640 1 -- 10.20.193.28:0/229588038 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbdf010f690 msgr2=0x7fbdf00f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbdef7fe640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbdf010f690 0x7fbdf00f5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbdef7fe640 1 -- 10.20.193.28:0/229588038 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbdf0115880 msgr2=0x7fbdf00f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbdef7fe640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbdf0115880 0x7fbdf00f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbdef7fe640 1 -- 10.20.193.28:0/229588038 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde0009570 con 0x7fbdf0110080 2026-02-25T11:48:31.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbdef7fe640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbdf0110080 0x7fbdf0176150 secure :-1 s=READY pgs=417 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbde00098a0 tx=0x7fbde0005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:31.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbded7fa640 1 -- 10.20.193.28:0/229588038 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbde0043cf0 con 0x7fbdf0110080 2026-02-25T11:48:31.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbded7fa640 1 -- 10.20.193.28:0/229588038 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fbde0043e50 con 0x7fbdf0110080 2026-02-25T11:48:31.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbded7fa640 1 -- 10.20.193.28:0/229588038 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbde004b690 con 0x7fbdf0110080 2026-02-25T11:48:31.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbdf570f640 1 -- 10.20.193.28:0/229588038 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf00fb8e0 con 0x7fbdf0110080 2026-02-25T11:48:31.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.669+0000 7fbdf570f640 1 -- 10.20.193.28:0/229588038 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf00fbd40 con 0x7fbdf0110080 2026-02-25T11:48:31.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.672+0000 7fbded7fa640 1 -- 10.20.193.28:0/229588038 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 16) ==== 157173+0+0 (secure 0 0 0) 0x7fbde002f9f0 con 0x7fbdf0110080 2026-02-25T11:48:31.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.674+0000 7fbded7fa640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbdc40b57e0 0x7fbdc40d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.674+0000 7fbded7fa640 1 -- 10.20.193.28:0/229588038 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fbde01142b0 con 0x7fbdf0110080 2026-02-25T11:48:31.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.674+0000 7fbdeffff640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbdc40b57e0 0x7fbdc40d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.674+0000 7fbdeffff640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbdc40b57e0 0x7fbdc40d5c90 secure :-1 s=READY pgs=267 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbddc005970 tx=0x7fbddc005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:31.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.675+0000 7fbdf8993e00 1 Processor -- start 2026-02-25T11:48:31.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.675+0000 7fbdf8993e00 1 -- start start 2026-02-25T11:48:31.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.675+0000 7fbdf8993e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71140 0x55b7abd71510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.675+0000 7fbdf8993e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71a50 0x55b7abdb61b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.675+0000 7fbdf8993e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdd6ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.675+0000 7fbdf8993e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b7abc879b0 con 0x55b7abd71140 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.675+0000 7fbdf8993e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b7abc836e0 con 0x55b7abdb66f0 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.675+0000 7fbdf8993e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b7abc7d840 con 0x55b7abd71a50 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.675+0000 7fbdeffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71140 0x55b7abd71510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.675+0000 7fbdf4f0e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdd6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.675+0000 7fbdf4f0e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdd6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:46408/0 (socket says 10.20.193.28:46408) 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.675+0000 7fbdf4f0e640 1 -- 10.20.193.28:0/14883130 learned_addr learned my addr 10.20.193.28:0/14883130 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.676+0000 7fbdef7fe640 1 --2- 10.20.193.28:0/14883130 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71a50 0x55b7abdb61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.676+0000 7fbdf4f0e640 1 -- 10.20.193.28:0/14883130 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71a50 msgr2=0x55b7abdb61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.676+0000 7fbdf4f0e640 1 --2- 10.20.193.28:0/14883130 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71a50 0x55b7abdb61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.676+0000 7fbdf4f0e640 1 -- 10.20.193.28:0/14883130 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71140 msgr2=0x55b7abd71510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.676+0000 7fbdf4f0e640 1 --2- 10.20.193.28:0/14883130 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71140 0x55b7abd71510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.676+0000 7fbdf4f0e640 1 -- 10.20.193.28:0/14883130 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7abc28d80 con 0x55b7abdb66f0 2026-02-25T11:48:31.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.676+0000 7fbdeffff640 1 --2- 10.20.193.28:0/14883130 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71140 0x55b7abd71510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:48:31.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.676+0000 7fbdf4f0e640 1 --2- 10.20.193.28:0/14883130 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdd6ad0 secure :-1 s=READY pgs=110 gs=3 cs=0 l=1 c_cookie=2406800204217366916 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbde4006a50 tx=0x7fbde4003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2166ac08b1b6b984 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:31.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.676+0000 7fbdc0ce3640 1 -- 10.20.193.28:0/14883130 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbde4010c20 con 0x55b7abdb66f0 2026-02-25T11:48:31.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.676+0000 7fbdc0ce3640 1 -- 10.20.193.28:0/14883130 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fbde4010d80 con 0x55b7abdb66f0 2026-02-25T11:48:31.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.676+0000 7fbdc0ce3640 1 -- 10.20.193.28:0/14883130 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbde401a810 con 0x55b7abdb66f0 2026-02-25T11:48:31.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.677+0000 7fbdf8993e00 1 -- 10.20.193.28:0/14883130 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 msgr2=0x55b7abdd6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.677+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/14883130 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdd6ad0 secure :-1 s=READY pgs=110 gs=3 cs=0 l=1 c_cookie=2406800204217366916 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbde4006a50 tx=0x7fbde4003e70 comp rx=0 tx=0).stop 2026-02-25T11:48:31.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.677+0000 7fbdf8993e00 1 -- 10.20.193.28:0/14883130 shutdown_connections 2026-02-25T11:48:31.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.677+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/14883130 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdd6ad0 unknown :-1 s=CLOSED pgs=110 gs=3 cs=0 l=1 c_cookie=2406800204217366916 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.677+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/14883130 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71a50 0x55b7abdb61b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.677+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/14883130 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71140 0x55b7abd71510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.677+0000 7fbdf8993e00 1 -- 10.20.193.28:0/14883130 >> 10.20.193.28:0/14883130 conn(0x55b7abc90d40 msgr2=0x55b7abd6fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:31.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.677+0000 7fbdf8993e00 1 -- 10.20.193.28:0/14883130 shutdown_connections 2026-02-25T11:48:31.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.677+0000 7fbdf8993e00 1 -- 10.20.193.28:0/14883130 wait complete. 2026-02-25T11:48:31.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.678+0000 7fbdf8993e00 1 -- start start 2026-02-25T11:48:31.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.678+0000 7fbdf8993e00 1 Processor -- start 2026-02-25T11:48:31.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdf8993e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71140 0x55b7abdce400 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdf8993e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abdce940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdf8993e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdd3930 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdf8993e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b7abdd3e70 con 0x55b7abd71a50 2026-02-25T11:48:31.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdf8993e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b7abdd3fe0 con 0x55b7abdb66f0 2026-02-25T11:48:31.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdf8993e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b7abdd41a0 con 0x55b7abd71140 2026-02-25T11:48:31.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdef7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abdce940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdef7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abdce940 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58484/0 (socket says 10.20.193.28:58484) 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdef7fe640 1 -- 10.20.193.28:0/1514595813 learned_addr learned my addr 10.20.193.28:0/1514595813 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdf4f0e640 1 --2- 10.20.193.28:0/1514595813 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdd3930 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdeffff640 1 --2- 10.20.193.28:0/1514595813 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71140 0x55b7abdce400 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdef7fe640 1 -- 10.20.193.28:0/1514595813 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71140 msgr2=0x55b7abdce400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdef7fe640 1 --2- 10.20.193.28:0/1514595813 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71140 0x55b7abdce400 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdef7fe640 1 -- 10.20.193.28:0/1514595813 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 msgr2=0x55b7abdd3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdef7fe640 1 --2- 10.20.193.28:0/1514595813 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdd3930 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdef7fe640 1 -- 10.20.193.28:0/1514595813 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde40055b0 con 0x55b7abd71a50 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdef7fe640 1 --2- 10.20.193.28:0/1514595813 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abdce940 secure :-1 s=READY pgs=418 gs=2 cs=0 l=1 c_cookie=11769881806534074057 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbde00d7850 tx=0x7fbde00d6ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3570524becbdec9 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.679+0000 7fbdb1ffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/1514595813 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbde009f900 con 0x55b7abd71a50 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/1514595813 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fbde00d9fc0 con 0x55b7abd71a50 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/1514595813 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbde00db4f0 con 0x55b7abd71a50 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1514595813 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbddc00f9b0 con 0x55b7abd71a50 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdf8993e00 10 client.15651 _ll_register_callbacks cb 0x55b7abbfe9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1514595813 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b7ab9136f0 con 0x55b7abd71a50 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/1514595813 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fbde00e0300 con 0x55b7abd71a50 2026-02-25T11:48:31.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/1514595813 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbde00d9de0 con 0x55b7abd71a50 2026-02-25T11:48:31.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdf8993e00 10 client.15651 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:31.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1514595813 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55b7abc01350 con 0x55b7abd71a50 2026-02-25T11:48:31.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/1514595813 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fbde00e5290 con 0x55b7abd71a50 2026-02-25T11:48:31.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdf8993e00 10 client.15651 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:31.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdf8993e00 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:31.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdf8993e00 20 client.15651 populate_metadata read hostname 'trial028' 2026-02-25T11:48:31.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/1514595813 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b7abdd6690 0x55b7abdb4420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdf8993e00 4 client.15651 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:48:31.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.680+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1514595813 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7abc4f910 con 0x55b7abdd6690 2026-02-25T11:48:31.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.681+0000 7fbdeffff640 1 --2- 10.20.193.28:0/1514595813 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b7abdd6690 0x55b7abdb4420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.682+0000 7fbdeffff640 1 --2- 10.20.193.28:0/1514595813 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b7abdd6690 0x55b7abdb4420 crc :-1 s=READY pgs=37 gs=4 cs=0 l=0 c_cookie=9354581226435608917 s_cookie=16619670939114781148 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d22610103b8155 server_cookie=e6a4eeaa11d1b9dc in_seq=0 out_seq=0 2026-02-25T11:48:31.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.682+0000 7fbdb1ffb640 10 client.15651 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:31.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.682+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/1514595813 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55b7abc4f910 con 0x55b7abdd6690 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.682+0000 7fbdb1ffb640 10 client.15651 handle_command_reply: tid=0 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.682+0000 7fbdf8993e00 1 client.15651 shutdown 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.683+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1514595813 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 msgr2=0x55b7abdce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.683+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/1514595813 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abdce940 secure :-1 s=READY pgs=418 gs=2 cs=0 l=1 c_cookie=11769881806534074057 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbde00d7850 tx=0x7fbde00d6ad0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.683+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1514595813 shutdown_connections 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.683+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/1514595813 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b7abdd6690 0x55b7abdb4420 crc :-1 s=READY pgs=37 gs=4 cs=0 l=0 c_cookie=9354581226435608917 s_cookie=16619670939114781148 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.683+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/1514595813 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdd3930 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.683+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/1514595813 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abdce940 unknown :-1 s=CLOSED pgs=418 gs=2 cs=0 l=1 c_cookie=11769881806534074057 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.683+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/1514595813 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71140 0x55b7abdce400 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.683+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1514595813 >> 10.20.193.28:0/1514595813 conn(0x55b7abc90d40 msgr2=0x55b7abd6f230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.683+0000 7fbdb1ffb640 0 client.15651 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.683+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1514595813 shutdown_connections 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.683+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1514595813 wait complete. 2026-02-25T11:48:31.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.683+0000 7fbdf8993e00 20 client.15651 trim_cache size 0 max 16384 2026-02-25T11:48:31.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.692+0000 7fbdf8993e00 1 Processor -- start 2026-02-25T11:48:31.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.692+0000 7fbdf8993e00 1 -- start start 2026-02-25T11:48:31.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.692+0000 7fbdf8993e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71140 0x55b7abdb91e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.692+0000 7fbdf8993e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abdcb5d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.692+0000 7fbdf8993e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdcbb10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.692+0000 7fbdf8993e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b7abc7b470 con 0x55b7abd71a50 2026-02-25T11:48:31.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.692+0000 7fbdf8993e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b7ab93b700 con 0x55b7abdb66f0 2026-02-25T11:48:31.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.692+0000 7fbdf8993e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b7abd4c490 con 0x55b7abd71140 2026-02-25T11:48:31.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.692+0000 7fbdef7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abdcb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.692+0000 7fbdef7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abdcb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58486/0 (socket says 10.20.193.28:58486) 2026-02-25T11:48:31.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.692+0000 7fbdef7fe640 1 -- 10.20.193.28:0/1548464717 learned_addr learned my addr 10.20.193.28:0/1548464717 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:31.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.693+0000 7fbdeffff640 1 --2- 10.20.193.28:0/1548464717 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71140 0x55b7abdb91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.693+0000 7fbdf4f0e640 1 --2- 10.20.193.28:0/1548464717 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdcbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.693+0000 7fbdef7fe640 1 -- 10.20.193.28:0/1548464717 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71140 msgr2=0x55b7abdb91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.693+0000 7fbdef7fe640 1 --2- 10.20.193.28:0/1548464717 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71140 0x55b7abdb91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.693+0000 7fbdef7fe640 1 -- 10.20.193.28:0/1548464717 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 msgr2=0x55b7abdcbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.693+0000 7fbdef7fe640 1 --2- 10.20.193.28:0/1548464717 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdcbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.693+0000 7fbdef7fe640 1 -- 10.20.193.28:0/1548464717 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7abc13150 con 0x55b7abd71a50 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.693+0000 7fbdef7fe640 1 --2- 10.20.193.28:0/1548464717 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abdcb5d0 secure :-1 s=READY pgs=419 gs=2 cs=0 l=1 c_cookie=14313223087472182749 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbde00d6bb0 tx=0x7fbde0111000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a2c89037b305dd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.693+0000 7fbdb3fff640 1 -- 10.20.193.28:0/1548464717 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbde00d9fc0 con 0x55b7abd71a50 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.693+0000 7fbdb3fff640 1 -- 10.20.193.28:0/1548464717 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55b7ab9136f0 con 0x55b7abd71a50 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.693+0000 7fbdb3fff640 1 -- 10.20.193.28:0/1548464717 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbde00e5720 con 0x55b7abd71a50 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.693+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1548464717 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 msgr2=0x55b7abdcb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.693+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/1548464717 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abdcb5d0 secure :-1 s=READY pgs=419 gs=2 cs=0 l=1 c_cookie=14313223087472182749 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbde00d6bb0 tx=0x7fbde0111000 comp rx=0 tx=0).stop 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1548464717 shutdown_connections 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/1548464717 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abdb66f0 0x55b7abdcbb10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/1548464717 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abdcb5d0 unknown :-1 s=CLOSED pgs=419 gs=2 cs=0 l=1 c_cookie=14313223087472182749 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/1548464717 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abd71140 0x55b7abdb91e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1548464717 >> 10.20.193.28:0/1548464717 conn(0x55b7abc90d40 msgr2=0x55b7abdb8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1548464717 shutdown_connections 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 -- 10.20.193.28:0/1548464717 wait complete. 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 -- start start 2026-02-25T11:48:31.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 Processor -- start 2026-02-25T11:48:31.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abd71140 0x55b7abd4fe80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abd503c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abdb66f0 0x55b7abe2c960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b7abe2cea0 con 0x55b7abd71a50 2026-02-25T11:48:31.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b7abe2d010 con 0x55b7abd71140 2026-02-25T11:48:31.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.694+0000 7fbdf8993e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b7abe2d1d0 con 0x55b7abdb66f0 2026-02-25T11:48:31.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdef7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abd503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdef7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abd503c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58500/0 (socket says 10.20.193.28:58500) 2026-02-25T11:48:31.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdef7fe640 1 -- 10.20.193.28:0/3523635470 learned_addr learned my addr 10.20.193.28:0/3523635470 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdf4f0e640 1 --2- 10.20.193.28:0/3523635470 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abdb66f0 0x55b7abe2c960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdeffff640 1 --2- 10.20.193.28:0/3523635470 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abd71140 0x55b7abd4fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdef7fe640 1 -- 10.20.193.28:0/3523635470 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abdb66f0 msgr2=0x55b7abe2c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdef7fe640 1 --2- 10.20.193.28:0/3523635470 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abdb66f0 0x55b7abe2c960 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdef7fe640 1 -- 10.20.193.28:0/3523635470 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abd71140 msgr2=0x55b7abd4fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdef7fe640 1 --2- 10.20.193.28:0/3523635470 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abd71140 0x55b7abd4fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdef7fe640 1 -- 10.20.193.28:0/3523635470 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7abc0dbd0 con 0x55b7abd71a50 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdef7fe640 1 --2- 10.20.193.28:0/3523635470 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abd503c0 secure :-1 s=READY pgs=420 gs=1 cs=0 l=1 c_cookie=5814725826868870385 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbde0111df0 tx=0x7fbde00e00d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b20e54f0cefcf1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdb1ffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/3523635470 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbde00d9fc0 con 0x55b7abd71a50 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/3523635470 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fbde00d9de0 con 0x55b7abd71a50 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/3523635470 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fbde009f6e0 con 0x55b7abd71a50 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdf8993e00 1 -- 10.20.193.28:0/3523635470 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7abc01350 con 0x55b7abd71a50 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.695+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/3523635470 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fbde009f840 con 0x55b7abd71a50 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.696+0000 7fbdf8993e00 10 client.15657 _ll_register_callbacks cb 0x55b7abbfe9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.696+0000 7fbdf8993e00 1 -- 10.20.193.28:0/3523635470 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbde4010d80 con 0x55b7abd71a50 2026-02-25T11:48:31.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.696+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/3523635470 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fbde00e5720 con 0x55b7abd71a50 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.696+0000 7fbdf8993e00 10 client.15657 fetch_fsmap learned FSMap version 23 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.696+0000 7fbdf8993e00 1 -- 10.20.193.28:0/3523635470 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55b7abc01680 con 0x55b7abd71a50 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.696+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/3523635470 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fbde00d85c0 con 0x55b7abd71a50 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.696+0000 7fbdf8993e00 10 client.15657 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.696+0000 7fbdf8993e00 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.696+0000 7fbdf8993e00 20 client.15657 populate_metadata read hostname 'trial028' 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.696+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/3523635470 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b7abe2e510 0x55b7abe95050 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.696+0000 7fbdf8993e00 4 client.15657 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.696+0000 7fbdf8993e00 1 -- 10.20.193.28:0/3523635470 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7ab910f40 con 0x55b7abe2e510 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.696+0000 7fbdeffff640 1 --2- 10.20.193.28:0/3523635470 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b7abe2e510 0x55b7abe95050 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.697+0000 7fbdeffff640 1 --2- 10.20.193.28:0/3523635470 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b7abe2e510 0x55b7abe95050 crc :-1 s=READY pgs=38 gs=4 cs=0 l=0 c_cookie=16627226949524121752 s_cookie=12675212946400221202 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6bfc6d0d62fe898 server_cookie=afe76704c29e1812 in_seq=0 out_seq=0 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.697+0000 7fbdb1ffb640 10 client.15657 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdb1ffb640 1 -- 10.20.193.28:0/3523635470 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x55b7ab910f40 con 0x55b7abe2e510 2026-02-25T11:48:31.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdb1ffb640 10 client.15657 handle_command_reply: tid=0 2026-02-25T11:48:31.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf8993e00 1 client.15657 shutdown 2026-02-25T11:48:31.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf8993e00 1 -- 10.20.193.28:0/3523635470 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 msgr2=0x55b7abd503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/3523635470 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abd503c0 secure :-1 s=READY pgs=420 gs=1 cs=0 l=1 c_cookie=5814725826868870385 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbde0111df0 tx=0x7fbde00e00d0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf8993e00 1 -- 10.20.193.28:0/3523635470 shutdown_connections 2026-02-25T11:48:31.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/3523635470 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b7abe2e510 0x55b7abe95050 crc :-1 s=READY pgs=38 gs=4 cs=0 l=0 c_cookie=16627226949524121752 s_cookie=12675212946400221202 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/3523635470 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b7abdb66f0 0x55b7abe2c960 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/3523635470 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b7abd71a50 0x55b7abd503c0 unknown :-1 s=CLOSED pgs=420 gs=1 cs=0 l=1 c_cookie=5814725826868870385 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf8993e00 1 --2- 10.20.193.28:0/3523635470 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b7abd71140 0x55b7abd4fe80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf8993e00 1 -- 10.20.193.28:0/3523635470 >> 10.20.193.28:0/3523635470 conn(0x55b7abc90d40 msgr2=0x55b7abdca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:31.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdb1ffb640 0 client.15657 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:48:31.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf8993e00 1 -- 10.20.193.28:0/3523635470 shutdown_connections 2026-02-25T11:48:31.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf8993e00 1 -- 10.20.193.28:0/3523635470 wait complete. 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf8993e00 20 client.15657 trim_cache size 0 max 16384 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "df724da3-1c68-4141-ad99-f8332cf2f6c6", 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:48:31.701 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf570f640 1 -- 10.20.193.28:0/229588038 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbdc40b57e0 msgr2=0x7fbdc40d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf570f640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbdc40b57e0 0x7fbdc40d5c90 secure :-1 s=READY pgs=267 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbddc005970 tx=0x7fbddc005900 comp rx=0 tx=0).stop 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf570f640 1 -- 10.20.193.28:0/229588038 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbdf0110080 msgr2=0x7fbdf0176150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:48:31.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.698+0000 7fbdf570f640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbdf0110080 0x7fbdf0176150 secure :-1 s=READY pgs=417 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fbde00098a0 tx=0x7fbde0005f10 comp rx=0 tx=0).stop 2026-02-25T11:48:31.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.699+0000 7fbdf570f640 1 -- 10.20.193.28:0/229588038 shutdown_connections 2026-02-25T11:48:31.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.699+0000 7fbdf570f640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fbdc40b57e0 0x7fbdc40d5c90 unknown :-1 s=CLOSED pgs=267 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.699+0000 7fbdf570f640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fbdf0115880 0x7fbdf00f6120 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.699+0000 7fbdf570f640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fbdf0110080 0x7fbdf0176150 unknown :-1 s=CLOSED pgs=417 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.699+0000 7fbdf570f640 1 --2- 10.20.193.28:0/229588038 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fbdf010f690 0x7fbdf00f5a20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:48:31.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.699+0000 7fbdf570f640 1 -- 10.20.193.28:0/229588038 >> 10.20.193.28:0/229588038 conn(0x7fbdf00ecf00 msgr2=0x7fbdf0111e90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:48:31.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.700+0000 7fbdf570f640 1 -- 10.20.193.28:0/229588038 shutdown_connections 2026-02-25T11:48:31.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:48:31.700+0000 7fbdf570f640 1 -- 10.20.193.28:0/229588038 wait complete. 2026-02-25T11:48:31.801 INFO:tasks.cephfs.filesystem:scrub status for tag:df724da3-1c68-4141-ad99-f8332cf2f6c6 - {'path': '/', 'tag': 'df724da3-1c68-4141-ad99-f8332cf2f6c6', 'options': 'recursive,force'} 2026-02-25T11:48:32.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:32 trial167 ceph-mon[43392]: pgmap v235: 97 pgs: 97 active+clean; 706 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 5.2 KiB/s rd, 4.3 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:32.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:32 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:48:32.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:32 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:48:32.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:32 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:48:32.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:32 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:48:32.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:32 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:48:32.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:32 trial119 ceph-mon[42710]: pgmap v235: 97 pgs: 97 active+clean; 706 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 5.2 KiB/s rd, 4.3 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:32.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:32 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:48:32.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:32 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:48:32.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:32 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:48:32.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:32 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:48:32.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:32 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:48:32.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:32 trial028 ceph-mon[39224]: pgmap v235: 97 pgs: 97 active+clean; 706 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 5.2 KiB/s rd, 4.3 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:32.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:32 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:48:32.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:32 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:48:32.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:32 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:48:32.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:32 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:48:32.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:32 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:48:34.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:34 trial167 ceph-mon[43392]: pgmap v236: 97 pgs: 97 active+clean; 768 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:34.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:34 trial119 ceph-mon[42710]: pgmap v236: 97 pgs: 97 active+clean; 768 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:34.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:34 trial028 ceph-mon[39224]: pgmap v236: 97 pgs: 97 active+clean; 768 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2026-02-25T11:48:36.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:36 trial167 ceph-mon[43392]: pgmap v237: 97 pgs: 97 active+clean; 768 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2026-02-25T11:48:36.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:36 trial119 ceph-mon[42710]: pgmap v237: 97 pgs: 97 active+clean; 768 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2026-02-25T11:48:36.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:36 trial028 ceph-mon[39224]: pgmap v237: 97 pgs: 97 active+clean; 768 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 10 op/s 2026-02-25T11:48:38.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:38 trial167 ceph-mon[43392]: pgmap v238: 97 pgs: 97 active+clean; 824 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2026-02-25T11:48:38.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:38 trial119 ceph-mon[42710]: pgmap v238: 97 pgs: 97 active+clean; 824 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2026-02-25T11:48:38.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:38 trial028 ceph-mon[39224]: pgmap v238: 97 pgs: 97 active+clean; 824 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 17 KiB/s rd, 18 KiB/s wr, 13 op/s 2026-02-25T11:48:40.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:40 trial167 ceph-mon[43392]: pgmap v239: 97 pgs: 97 active+clean; 824 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:48:40.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:40 trial119 ceph-mon[42710]: pgmap v239: 97 pgs: 97 active+clean; 824 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:48:40.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:40 trial028 ceph-mon[39224]: pgmap v239: 97 pgs: 97 active+clean; 824 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:48:42.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:42 trial167 ceph-mon[43392]: pgmap v240: 97 pgs: 97 active+clean; 824 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:48:42.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:42 trial119 ceph-mon[42710]: pgmap v240: 97 pgs: 97 active+clean; 824 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:48:42.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:42 trial028 ceph-mon[39224]: pgmap v240: 97 pgs: 97 active+clean; 824 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:48:44.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:44 trial167 ceph-mon[43392]: pgmap v241: 97 pgs: 97 active+clean; 882 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:48:44.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:44 trial119 ceph-mon[42710]: pgmap v241: 97 pgs: 97 active+clean; 882 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:48:44.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:44 trial028 ceph-mon[39224]: pgmap v241: 97 pgs: 97 active+clean; 882 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:48:46.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:46 trial167 ceph-mon[43392]: pgmap v242: 97 pgs: 97 active+clean; 882 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:48:46.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:46 trial119 ceph-mon[42710]: pgmap v242: 97 pgs: 97 active+clean; 882 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:48:46.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:46 trial028 ceph-mon[39224]: pgmap v242: 97 pgs: 97 active+clean; 882 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:48:48.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:48 trial167 ceph-mon[43392]: pgmap v243: 97 pgs: 97 active+clean; 946 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2026-02-25T11:48:48.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:48 trial119 ceph-mon[42710]: pgmap v243: 97 pgs: 97 active+clean; 946 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2026-02-25T11:48:48.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:48 trial028 ceph-mon[39224]: pgmap v243: 97 pgs: 97 active+clean; 946 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2026-02-25T11:48:50.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:50 trial167 ceph-mon[43392]: pgmap v244: 97 pgs: 97 active+clean; 946 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:48:50.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:50 trial167 ceph-mon[43392]: mgrmap e17: x(active, since 5m), standbys: z, y 2026-02-25T11:48:50.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:50 trial119 ceph-mon[42710]: pgmap v244: 97 pgs: 97 active+clean; 946 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:48:50.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:50 trial119 ceph-mon[42710]: mgrmap e17: x(active, since 5m), standbys: z, y 2026-02-25T11:48:50.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:50 trial028 ceph-mon[39224]: pgmap v244: 97 pgs: 97 active+clean; 946 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:48:50.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:50 trial028 ceph-mon[39224]: mgrmap e17: x(active, since 5m), standbys: z, y 2026-02-25T11:48:52.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:52 trial167 ceph-mon[43392]: pgmap v245: 97 pgs: 97 active+clean; 946 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:48:52.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:52 trial119 ceph-mon[42710]: pgmap v245: 97 pgs: 97 active+clean; 946 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:48:52.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:52 trial028 ceph-mon[39224]: pgmap v245: 97 pgs: 97 active+clean; 946 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:48:54.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:54 trial167 ceph-mon[43392]: pgmap v246: 97 pgs: 97 active+clean; 1004 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 15 op/s 2026-02-25T11:48:54.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:54 trial119 ceph-mon[42710]: pgmap v246: 97 pgs: 97 active+clean; 1004 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 15 op/s 2026-02-25T11:48:54.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:54 trial028 ceph-mon[39224]: pgmap v246: 97 pgs: 97 active+clean; 1004 KiB data, 331 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 15 op/s 2026-02-25T11:48:56.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:56 trial167 ceph-mon[43392]: pgmap v247: 97 pgs: 97 active+clean; 1004 KiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:48:56.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:56 trial119 ceph-mon[42710]: pgmap v247: 97 pgs: 97 active+clean; 1004 KiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:48:56.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:56 trial028 ceph-mon[39224]: pgmap v247: 97 pgs: 97 active+clean; 1004 KiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:48:58.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:48:58 trial167 ceph-mon[43392]: pgmap v248: 97 pgs: 97 active+clean; 1.0 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:48:58.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:48:58 trial119 ceph-mon[42710]: pgmap v248: 97 pgs: 97 active+clean; 1.0 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:48:58.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:48:58 trial028 ceph-mon[39224]: pgmap v248: 97 pgs: 97 active+clean; 1.0 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:00.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:00 trial167 ceph-mon[43392]: pgmap v249: 97 pgs: 97 active+clean; 1.0 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:49:00.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:00 trial119 ceph-mon[42710]: pgmap v249: 97 pgs: 97 active+clean; 1.0 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:49:00.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:00 trial028 ceph-mon[39224]: pgmap v249: 97 pgs: 97 active+clean; 1.0 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:49:01.802 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:49:02.035 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:02.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.223+0000 7fa838b73640 1 -- 10.20.193.28:0/3612917464 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa83410c7f0 msgr2=0x7fa8341556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.223+0000 7fa838b73640 1 --2- 10.20.193.28:0/3612917464 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa83410c7f0 0x7fa8341556d0 secure :-1 s=READY pgs=421 gs=1 cs=0 l=1 c_cookie=5066173597538736266 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa824009950 tx=0x7fa82402f190 comp rx=0 tx=0).stop 2026-02-25T11:49:02.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.224+0000 7fa838b73640 1 -- 10.20.193.28:0/3612917464 shutdown_connections 2026-02-25T11:49:02.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.224+0000 7fa838b73640 1 --2- 10.20.193.28:0/3612917464 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa834155c10 0x7fa834175ff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.224+0000 7fa838b73640 1 --2- 10.20.193.28:0/3612917464 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa83410c7f0 0x7fa8341556d0 unknown :-1 s=CLOSED pgs=421 gs=1 cs=0 l=1 c_cookie=5066173597538736266 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.224+0000 7fa838b73640 1 --2- 10.20.193.28:0/3612917464 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa83410bee0 0x7fa83410c2b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.224+0000 7fa838b73640 1 -- 10.20.193.28:0/3612917464 >> 10.20.193.28:0/3612917464 conn(0x7fa8340e9b40 msgr2=0x7fa834109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:02.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.224+0000 7fa838b73640 1 -- 10.20.193.28:0/3612917464 shutdown_connections 2026-02-25T11:49:02.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.224+0000 7fa838b73640 1 -- 10.20.193.28:0/3612917464 wait complete. 2026-02-25T11:49:02.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.224+0000 7fa838b73640 1 Processor -- start 2026-02-25T11:49:02.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa838b73640 1 -- start start 2026-02-25T11:49:02.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa838b73640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa83410bee0 0x7fa8341613c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa838b73640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa83410c7f0 0x7fa834161900 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa838b73640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa834155c10 0x7fa834166c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa838b73640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa8341672a0 con 0x7fa83410bee0 2026-02-25T11:49:02.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa838b73640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa834167410 con 0x7fa834155c10 2026-02-25T11:49:02.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa838b73640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa8341675d0 con 0x7fa83410c7f0 2026-02-25T11:49:02.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa8337fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa83410bee0 0x7fa8341613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa833fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa834155c10 0x7fa834166c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa8337fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa83410bee0 0x7fa8341613c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:54804/0 (socket says 10.20.193.28:54804) 2026-02-25T11:49:02.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa8337fe640 1 -- 10.20.193.28:0/1792545458 learned_addr learned my addr 10.20.193.28:0/1792545458 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:02.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa832ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa83410c7f0 0x7fa834161900 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa8337fe640 1 -- 10.20.193.28:0/1792545458 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa83410c7f0 msgr2=0x7fa834161900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa8337fe640 1 --2- 10.20.193.28:0/1792545458 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa83410c7f0 0x7fa834161900 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa8337fe640 1 -- 10.20.193.28:0/1792545458 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa834155c10 msgr2=0x7fa834166c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa8337fe640 1 --2- 10.20.193.28:0/1792545458 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa834155c10 0x7fa834166c80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.225+0000 7fa8337fe640 1 -- 10.20.193.28:0/1792545458 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa824009570 con 0x7fa83410bee0 2026-02-25T11:49:02.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.226+0000 7fa8337fe640 1 --2- 10.20.193.28:0/1792545458 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa83410bee0 0x7fa8341613c0 secure :-1 s=READY pgs=422 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa820009480 tx=0x7fa82000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:02.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.226+0000 7fa830ff9640 1 -- 10.20.193.28:0/1792545458 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa82000cd40 con 0x7fa83410bee0 2026-02-25T11:49:02.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.226+0000 7fa830ff9640 1 -- 10.20.193.28:0/1792545458 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa820004d10 con 0x7fa83410bee0 2026-02-25T11:49:02.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.226+0000 7fa830ff9640 1 -- 10.20.193.28:0/1792545458 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa8200106b0 con 0x7fa83410bee0 2026-02-25T11:49:02.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.226+0000 7fa838b73640 1 -- 10.20.193.28:0/1792545458 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8341678b0 con 0x7fa83410bee0 2026-02-25T11:49:02.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.226+0000 7fa838b73640 1 -- 10.20.193.28:0/1792545458 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa834152840 con 0x7fa83410bee0 2026-02-25T11:49:02.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.228+0000 7fa830ff9640 1 -- 10.20.193.28:0/1792545458 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fa82001f060 con 0x7fa83410bee0 2026-02-25T11:49:02.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.230+0000 7fa830ff9640 1 --2- 10.20.193.28:0/1792545458 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa8080b5760 0x7fa8080d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.230+0000 7fa830ff9640 1 -- 10.20.193.28:0/1792545458 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa820014070 con 0x7fa83410bee0 2026-02-25T11:49:02.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.230+0000 7fa832ffd640 1 --2- 10.20.193.28:0/1792545458 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa8080b5760 0x7fa8080d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.231+0000 7fa832ffd640 1 --2- 10.20.193.28:0/1792545458 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa8080b5760 0x7fa8080d5c10 secure :-1 s=READY pgs=268 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa834167d10 tx=0x7fa82403a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:02.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.231+0000 7fa83bdf7e00 1 Processor -- start 2026-02-25T11:49:02.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.231+0000 7fa83bdf7e00 1 -- start start 2026-02-25T11:49:02.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa83bdf7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7140 0x5568d36b7510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa83bdf7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7a50 0x5568d36fc1b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa83bdf7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36fc6f0 0x5568d371cad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa83bdf7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5568d35cd9b0 con 0x5568d36fc6f0 2026-02-25T11:49:02.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa83bdf7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5568d35c96e0 con 0x5568d36b7a50 2026-02-25T11:49:02.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa83bdf7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5568d35c3840 con 0x5568d36b7140 2026-02-25T11:49:02.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa833fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36fc6f0 0x5568d371cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa833fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36fc6f0 0x5568d371cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:54820/0 (socket says 10.20.193.28:54820) 2026-02-25T11:49:02.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa833fff640 1 -- 10.20.193.28:0/4261058450 learned_addr learned my addr 10.20.193.28:0/4261058450 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:02.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa832ffd640 1 --2- 10.20.193.28:0/4261058450 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7a50 0x5568d36fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa8337fe640 1 --2- 10.20.193.28:0/4261058450 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7140 0x5568d36b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa833fff640 1 -- 10.20.193.28:0/4261058450 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7140 msgr2=0x5568d36b7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:02.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa833fff640 1 --2- 10.20.193.28:0/4261058450 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7140 0x5568d36b7510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa833fff640 1 -- 10.20.193.28:0/4261058450 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7a50 msgr2=0x5568d36fc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:02.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa833fff640 1 --2- 10.20.193.28:0/4261058450 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7a50 0x5568d36fc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa833fff640 1 -- 10.20.193.28:0/4261058450 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d356ed80 con 0x5568d36fc6f0 2026-02-25T11:49:02.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.232+0000 7fa833fff640 1 --2- 10.20.193.28:0/4261058450 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36fc6f0 0x5568d371cad0 secure :-1 s=READY pgs=423 gs=3 cs=0 l=1 c_cookie=3577559683502492418 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa828008cd0 tx=0x7fa8280117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a60b93d1eccb02 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:02.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.233+0000 7fa7f7fff640 1 -- 10.20.193.28:0/4261058450 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa8280050d0 con 0x5568d36fc6f0 2026-02-25T11:49:02.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.233+0000 7fa7f7fff640 1 -- 10.20.193.28:0/4261058450 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa828003780 con 0x5568d36fc6f0 2026-02-25T11:49:02.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.233+0000 7fa7f7fff640 1 -- 10.20.193.28:0/4261058450 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa8280127c0 con 0x5568d36fc6f0 2026-02-25T11:49:02.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.233+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/4261058450 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36fc6f0 msgr2=0x5568d371cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.233+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/4261058450 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36fc6f0 0x5568d371cad0 secure :-1 s=READY pgs=423 gs=3 cs=0 l=1 c_cookie=3577559683502492418 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa828008cd0 tx=0x7fa8280117a0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.233+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/4261058450 shutdown_connections 2026-02-25T11:49:02.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.233+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/4261058450 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36fc6f0 0x5568d371cad0 unknown :-1 s=CLOSED pgs=423 gs=3 cs=0 l=1 c_cookie=3577559683502492418 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.233+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/4261058450 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7a50 0x5568d36fc1b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.233+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/4261058450 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7140 0x5568d36b7510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.233+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/4261058450 >> 10.20.193.28:0/4261058450 conn(0x5568d35d6d40 msgr2=0x5568d36b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:02.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.233+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/4261058450 shutdown_connections 2026-02-25T11:49:02.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.233+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/4261058450 wait complete. 2026-02-25T11:49:02.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.235+0000 7fa83bdf7e00 1 -- start start 2026-02-25T11:49:02.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.235+0000 7fa83bdf7e00 1 Processor -- start 2026-02-25T11:49:02.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.235+0000 7fa83bdf7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7140 0x5568d3714380 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.235+0000 7fa83bdf7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36b7a50 0x5568d37148c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.235+0000 7fa83bdf7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36fc6f0 0x5568d37198b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.235+0000 7fa83bdf7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5568d3719df0 con 0x5568d36b7a50 2026-02-25T11:49:02.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.235+0000 7fa83bdf7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5568d3719f60 con 0x5568d36fc6f0 2026-02-25T11:49:02.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.235+0000 7fa83bdf7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5568d371a120 con 0x5568d36b7140 2026-02-25T11:49:02.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa8337fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7140 0x5568d3714380 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa8337fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7140 0x5568d3714380 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:57396/0 (socket says 10.20.193.28:57396) 2026-02-25T11:49:02.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa8337fe640 1 -- 10.20.193.28:0/2060030188 learned_addr learned my addr 10.20.193.28:0/2060030188 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:02.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa833fff640 1 --2- 10.20.193.28:0/2060030188 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36fc6f0 0x5568d37198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa832ffd640 1 --2- 10.20.193.28:0/2060030188 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36b7a50 0x5568d37148c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa8337fe640 1 -- 10.20.193.28:0/2060030188 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36fc6f0 msgr2=0x5568d37198b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:02.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa8337fe640 1 --2- 10.20.193.28:0/2060030188 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36fc6f0 0x5568d37198b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa8337fe640 1 -- 10.20.193.28:0/2060030188 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36b7a50 msgr2=0x5568d37148c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:02.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa8337fe640 1 --2- 10.20.193.28:0/2060030188 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36b7a50 0x5568d37148c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa8337fe640 1 -- 10.20.193.28:0/2060030188 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d3547350 con 0x5568d36b7140 2026-02-25T11:49:02.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa8337fe640 1 --2- 10.20.193.28:0/2060030188 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7140 0x5568d3714380 secure :-1 s=READY pgs=123 gs=1 cs=0 l=1 c_cookie=970298332899499078 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa8200acf20 tx=0x7fa8200acff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d77313162a0dc46 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:02.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa7f57fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:49:02.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa7f57fa640 1 -- 10.20.193.28:0/2060030188 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa8200b7e10 con 0x5568d36b7140 2026-02-25T11:49:02.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2060030188 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8280088f0 con 0x5568d36b7140 2026-02-25T11:49:02.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa7f57fa640 1 -- 10.20.193.28:0/2060030188 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa8200d22e0 con 0x5568d36b7140 2026-02-25T11:49:02.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa7f57fa640 1 -- 10.20.193.28:0/2060030188 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa8200ade50 con 0x5568d36b7140 2026-02-25T11:49:02.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa83bdf7e00 10 client.25489 _ll_register_callbacks cb 0x5568d35449a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:02.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2060030188 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5568d32596f0 con 0x5568d36b7140 2026-02-25T11:49:02.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.236+0000 7fa7f57fa640 1 -- 10.20.193.28:0/2060030188 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa8200aaf20 con 0x5568d36b7140 2026-02-25T11:49:02.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.237+0000 7fa7f57fa640 1 -- 10.20.193.28:0/2060030188 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa820072350 con 0x5568d36b7140 2026-02-25T11:49:02.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.237+0000 7fa83bdf7e00 10 client.25489 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:02.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.237+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2060030188 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5568d3559150 con 0x5568d36b7140 2026-02-25T11:49:02.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.237+0000 7fa7f57fa640 1 -- 10.20.193.28:0/2060030188 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa8200ae930 con 0x5568d36b7140 2026-02-25T11:49:02.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.237+0000 7fa83bdf7e00 10 client.25489 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:02.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.237+0000 7fa83bdf7e00 10 client.25489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:02.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.237+0000 7fa83bdf7e00 20 client.25489 populate_metadata read hostname 'trial028' 2026-02-25T11:49:02.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.237+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/2060030188 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568d371c950 0x5568d36fa420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.237+0000 7fa83bdf7e00 4 client.25489 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:49:02.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.237+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2060030188 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568d3595910 con 0x5568d371c950 2026-02-25T11:49:02.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.237+0000 7fa832ffd640 1 --2- 10.20.193.28:0/2060030188 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568d371c950 0x5568d36fa420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.238+0000 7fa832ffd640 1 --2- 10.20.193.28:0/2060030188 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568d371c950 0x5568d36fa420 crc :-1 s=READY pgs=42 gs=4 cs=0 l=0 c_cookie=11301039286413845944 s_cookie=8765250111729271108 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cd55b5b036e65b8 server_cookie=79a46c894d138144 in_seq=0 out_seq=0 2026-02-25T11:49:02.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.238+0000 7fa7f57fa640 10 client.25489 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:02.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa7f57fa640 1 -- 10.20.193.28:0/2060030188 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5568d3595910 con 0x5568d371c950 2026-02-25T11:49:02.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa7f57fa640 10 client.25489 handle_command_reply: tid=0 2026-02-25T11:49:02.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa83bdf7e00 1 client.25489 shutdown 2026-02-25T11:49:02.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2060030188 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7140 msgr2=0x5568d3714380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/2060030188 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7140 0x5568d3714380 secure :-1 s=READY pgs=123 gs=1 cs=0 l=1 c_cookie=970298332899499078 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa8200acf20 tx=0x7fa8200acff0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2060030188 shutdown_connections 2026-02-25T11:49:02.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/2060030188 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568d371c950 0x5568d36fa420 crc :-1 s=READY pgs=42 gs=4 cs=0 l=0 c_cookie=11301039286413845944 s_cookie=8765250111729271108 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/2060030188 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36fc6f0 0x5568d37198b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/2060030188 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36b7a50 0x5568d37148c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/2060030188 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7140 0x5568d3714380 unknown :-1 s=CLOSED pgs=123 gs=1 cs=0 l=1 c_cookie=970298332899499078 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa7f57fa640 0 client.25489 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2060030188 >> 10.20.193.28:0/2060030188 conn(0x5568d35d6d40 msgr2=0x5568d36b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2060030188 shutdown_connections 2026-02-25T11:49:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2060030188 wait complete. 2026-02-25T11:49:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.239+0000 7fa83bdf7e00 20 client.25489 trim_cache size 0 max 16384 2026-02-25T11:49:02.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.249+0000 7fa83bdf7e00 1 Processor -- start 2026-02-25T11:49:02.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.249+0000 7fa83bdf7e00 1 -- start start 2026-02-25T11:49:02.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.249+0000 7fa83bdf7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7140 0x5568d36ff1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.249+0000 7fa83bdf7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7a50 0x5568d3711550 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.249+0000 7fa83bdf7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36fc6f0 0x5568d3711a90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.249+0000 7fa83bdf7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5568d35c1470 con 0x5568d36fc6f0 2026-02-25T11:49:02.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.249+0000 7fa83bdf7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5568d3281700 con 0x5568d36b7140 2026-02-25T11:49:02.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.249+0000 7fa83bdf7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5568d3701aa0 con 0x5568d36b7a50 2026-02-25T11:49:02.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.249+0000 7fa833fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36fc6f0 0x5568d3711a90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.249+0000 7fa8337fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7140 0x5568d36ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.249+0000 7fa833fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36fc6f0 0x5568d3711a90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:54836/0 (socket says 10.20.193.28:54836) 2026-02-25T11:49:02.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.249+0000 7fa833fff640 1 -- 10.20.193.28:0/2913620702 learned_addr learned my addr 10.20.193.28:0/2913620702 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:02.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.249+0000 7fa832ffd640 1 --2- 10.20.193.28:0/2913620702 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7a50 0x5568d3711550 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa8337fe640 1 -- 10.20.193.28:0/2913620702 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7a50 msgr2=0x5568d3711550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:02.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa8337fe640 1 --2- 10.20.193.28:0/2913620702 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7a50 0x5568d3711550 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa8337fe640 1 -- 10.20.193.28:0/2913620702 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36fc6f0 msgr2=0x5568d3711a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:02.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa8337fe640 1 --2- 10.20.193.28:0/2913620702 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36fc6f0 0x5568d3711a90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa8337fe640 1 -- 10.20.193.28:0/2913620702 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d3553bd0 con 0x5568d36b7140 2026-02-25T11:49:02.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa8337fe640 1 --2- 10.20.193.28:0/2913620702 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7140 0x5568d36ff1e0 secure :-1 s=READY pgs=111 gs=1 cs=0 l=1 c_cookie=2819761805438982824 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa8200e42e0 tx=0x7fa820071000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2721ce6c1c679aa8 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:02.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa7f77fe640 1 -- 10.20.193.28:0/2913620702 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa8200d22e0 con 0x5568d36b7140 2026-02-25T11:49:02.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa7f77fe640 1 -- 10.20.193.28:0/2913620702 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5568d32596f0 con 0x5568d36b7140 2026-02-25T11:49:02.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa7f77fe640 1 -- 10.20.193.28:0/2913620702 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa8200b5240 con 0x5568d36b7140 2026-02-25T11:49:02.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2913620702 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7140 msgr2=0x5568d36ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/2913620702 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7140 0x5568d36ff1e0 secure :-1 s=READY pgs=111 gs=1 cs=0 l=1 c_cookie=2819761805438982824 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa8200e42e0 tx=0x7fa820071000 comp rx=0 tx=0).stop 2026-02-25T11:49:02.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2913620702 shutdown_connections 2026-02-25T11:49:02.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/2913620702 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36fc6f0 0x5568d3711a90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/2913620702 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36b7a50 0x5568d3711550 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/2913620702 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7140 0x5568d36ff1e0 unknown :-1 s=CLOSED pgs=111 gs=1 cs=0 l=1 c_cookie=2819761805438982824 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2913620702 >> 10.20.193.28:0/2913620702 conn(0x5568d35d6d40 msgr2=0x5568d36fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:02.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.250+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2913620702 shutdown_connections 2026-02-25T11:49:02.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.251+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/2913620702 wait complete. 2026-02-25T11:49:02.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.251+0000 7fa83bdf7e00 1 -- start start 2026-02-25T11:49:02.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.251+0000 7fa83bdf7e00 1 Processor -- start 2026-02-25T11:49:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa83bdf7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36b7140 0x5568d36953d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa83bdf7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7a50 0x5568d3695910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa83bdf7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36fc6f0 0x5568d369a270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa83bdf7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5568d36b7f30 con 0x5568d36b7140 2026-02-25T11:49:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa83bdf7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5568d3281700 con 0x5568d36b7a50 2026-02-25T11:49:02.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa83bdf7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5568d3695e80 con 0x5568d36fc6f0 2026-02-25T11:49:02.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa833fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36fc6f0 0x5568d369a270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa833fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36fc6f0 0x5568d369a270 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:57408/0 (socket says 10.20.193.28:57408) 2026-02-25T11:49:02.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa833fff640 1 -- 10.20.193.28:0/3672958076 learned_addr learned my addr 10.20.193.28:0/3672958076 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:02.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa8337fe640 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36b7140 0x5568d36953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa832ffd640 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7a50 0x5568d3695910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa833fff640 1 -- 10.20.193.28:0/3672958076 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7a50 msgr2=0x5568d3695910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:02.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa833fff640 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7a50 0x5568d3695910 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa833fff640 1 -- 10.20.193.28:0/3672958076 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36b7140 msgr2=0x5568d36953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:02.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa833fff640 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36b7140 0x5568d36953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa833fff640 1 -- 10.20.193.28:0/3672958076 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d3547680 con 0x5568d36fc6f0 2026-02-25T11:49:02.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa8337fe640 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36b7140 0x5568d36953d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:02.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa832ffd640 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7a50 0x5568d3695910 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:49:02.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa833fff640 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36fc6f0 0x5568d369a270 secure :-1 s=READY pgs=124 gs=2 cs=0 l=1 c_cookie=12159233358273872049 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa828011cb0 tx=0x7fa828008f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8be465124fe80b1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:02.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.252+0000 7fa7f57fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:49:02.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.253+0000 7fa7f57fa640 1 -- 10.20.193.28:0/3672958076 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa828005580 con 0x5568d36fc6f0 2026-02-25T11:49:02.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.253+0000 7fa7f57fa640 1 -- 10.20.193.28:0/3672958076 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa828023040 con 0x5568d36fc6f0 2026-02-25T11:49:02.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.253+0000 7fa7f57fa640 1 -- 10.20.193.28:0/3672958076 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa828029d90 con 0x5568d36fc6f0 2026-02-25T11:49:02.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.253+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/3672958076 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568d3559150 con 0x5568d36fc6f0 2026-02-25T11:49:02.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.253+0000 7fa83bdf7e00 10 client.25495 _ll_register_callbacks cb 0x5568d35449a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:02.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.253+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/3672958076 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa828003780 con 0x5568d36fc6f0 2026-02-25T11:49:02.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.253+0000 7fa7f57fa640 1 -- 10.20.193.28:0/3672958076 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa828029440 con 0x5568d36fc6f0 2026-02-25T11:49:02.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.253+0000 7fa7f57fa640 1 -- 10.20.193.28:0/3672958076 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa8280038f0 con 0x5568d36fc6f0 2026-02-25T11:49:02.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.253+0000 7fa83bdf7e00 10 client.25495 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:02.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.253+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/3672958076 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5568d35593b0 con 0x5568d36fc6f0 2026-02-25T11:49:02.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.254+0000 7fa7f57fa640 1 -- 10.20.193.28:0/3672958076 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa828003a50 con 0x5568d36fc6f0 2026-02-25T11:49:02.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.254+0000 7fa83bdf7e00 10 client.25495 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:02.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.254+0000 7fa83bdf7e00 10 client.25495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:02.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.254+0000 7fa83bdf7e00 20 client.25495 populate_metadata read hostname 'trial028' 2026-02-25T11:49:02.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.254+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568d37aa7f0 0x5568d37caca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.254+0000 7fa83bdf7e00 4 client.25495 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:49:02.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.254+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/3672958076 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5568d3256f40 con 0x5568d37aa7f0 2026-02-25T11:49:02.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.254+0000 7fa8337fe640 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568d37aa7f0 0x5568d37caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa8337fe640 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568d37aa7f0 0x5568d37caca0 crc :-1 s=READY pgs=43 gs=4 cs=0 l=0 c_cookie=12670235843515017310 s_cookie=5281580423652972231 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afd5b85eb122685e server_cookie=494bf1698c7e16c7 in_seq=0 out_seq=0 2026-02-25T11:49:02.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa7f57fa640 10 client.25495 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:02.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa7f57fa640 1 -- 10.20.193.28:0/3672958076 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x5568d3256f40 con 0x5568d37aa7f0 2026-02-25T11:49:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa7f57fa640 10 client.25495 handle_command_reply: tid=0 2026-02-25T11:49:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa83bdf7e00 1 client.25495 shutdown 2026-02-25T11:49:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/3672958076 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36fc6f0 msgr2=0x5568d369a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36fc6f0 0x5568d369a270 secure :-1 s=READY pgs=124 gs=2 cs=0 l=1 c_cookie=12159233358273872049 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa828011cb0 tx=0x7fa828008f30 comp rx=0 tx=0).stop 2026-02-25T11:49:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/3672958076 shutdown_connections 2026-02-25T11:49:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568d37aa7f0 0x5568d37caca0 crc :-1 s=READY pgs=43 gs=4 cs=0 l=0 c_cookie=12670235843515017310 s_cookie=5281580423652972231 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568d36fc6f0 0x5568d369a270 unknown :-1 s=CLOSED pgs=124 gs=2 cs=0 l=1 c_cookie=12159233358273872049 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568d36b7a50 0x5568d3695910 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa83bdf7e00 1 --2- 10.20.193.28:0/3672958076 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568d36b7140 0x5568d36953d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa7f57fa640 0 client.25495 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/3672958076 >> 10.20.193.28:0/3672958076 conn(0x5568d35d6d40 msgr2=0x5568d3710540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/3672958076 shutdown_connections 2026-02-25T11:49:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa83bdf7e00 1 -- 10.20.193.28:0/3672958076 wait complete. 2026-02-25T11:49:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.255+0000 7fa83bdf7e00 20 client.25495 trim_cache size 0 max 16384 2026-02-25T11:49:02.263 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:49:02.263 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:49:02.264 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:49:02.264 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:49:02.264 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:49:02.264 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:49:02.264 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:02.264 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:02.264 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:02.264 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:02.264 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:02.264 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:02.264 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:49:02.264 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:49:02.265 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:02.265 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:02.265 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:02.265 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:02.265 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:02.265 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:02.265 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:49:02.265 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:49:02.265 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:02.265 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:02.266 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:02.266 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:02.266 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:02.266 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:49:02.266 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:49:02.266 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:49:02.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.256+0000 7fa838b73640 1 -- 10.20.193.28:0/1792545458 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa8080b5760 msgr2=0x7fa8080d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.256+0000 7fa838b73640 1 --2- 10.20.193.28:0/1792545458 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa8080b5760 0x7fa8080d5c10 secure :-1 s=READY pgs=268 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa834167d10 tx=0x7fa82403a040 comp rx=0 tx=0).stop 2026-02-25T11:49:02.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.256+0000 7fa838b73640 1 -- 10.20.193.28:0/1792545458 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa83410bee0 msgr2=0x7fa8341613c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.256+0000 7fa838b73640 1 --2- 10.20.193.28:0/1792545458 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa83410bee0 0x7fa8341613c0 secure :-1 s=READY pgs=422 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa820009480 tx=0x7fa82000ee60 comp rx=0 tx=0).stop 2026-02-25T11:49:02.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.256+0000 7fa838b73640 1 -- 10.20.193.28:0/1792545458 shutdown_connections 2026-02-25T11:49:02.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.256+0000 7fa838b73640 1 --2- 10.20.193.28:0/1792545458 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa8080b5760 0x7fa8080d5c10 unknown :-1 s=CLOSED pgs=268 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.256+0000 7fa838b73640 1 --2- 10.20.193.28:0/1792545458 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa834155c10 0x7fa834166c80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.256+0000 7fa838b73640 1 --2- 10.20.193.28:0/1792545458 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa83410c7f0 0x7fa834161900 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.256+0000 7fa838b73640 1 --2- 10.20.193.28:0/1792545458 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa83410bee0 0x7fa8341613c0 unknown :-1 s=CLOSED pgs=422 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.256+0000 7fa838b73640 1 -- 10.20.193.28:0/1792545458 >> 10.20.193.28:0/1792545458 conn(0x7fa8340e9b40 msgr2=0x7fa834109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:02.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.256+0000 7fa838b73640 1 -- 10.20.193.28:0/1792545458 shutdown_connections 2026-02-25T11:49:02.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.256+0000 7fa838b73640 1 -- 10.20.193.28:0/1792545458 wait complete. 2026-02-25T11:49:02.358 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:49:02.358 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:49:02.409 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:02 trial028 ceph-mon[39224]: pgmap v250: 97 pgs: 97 active+clean; 1.0 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:49:02.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:02 trial167 ceph-mon[43392]: pgmap v250: 97 pgs: 97 active+clean; 1.0 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:49:02.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:02 trial119 ceph-mon[42710]: pgmap v250: 97 pgs: 97 active+clean; 1.0 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:49:02.543 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:02.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.715+0000 7f8bdaebe640 1 -- 10.20.193.28:0/3720272934 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8bd4111370 msgr2=0x7f8bd4111d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.715+0000 7f8bdaebe640 1 --2- 10.20.193.28:0/3720272934 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8bd4111370 0x7f8bd4111d50 secure :-1 s=READY pgs=424 gs=1 cs=0 l=1 c_cookie=1105290295135865297 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8bd000aa30 tx=0x7f8bd0030280 comp rx=0 tx=0).stop 2026-02-25T11:49:02.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.715+0000 7f8bdaebe640 1 -- 10.20.193.28:0/3720272934 shutdown_connections 2026-02-25T11:49:02.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.715+0000 7f8bdaebe640 1 --2- 10.20.193.28:0/3720272934 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8bd4111370 0x7f8bd4111d50 unknown :-1 s=CLOSED pgs=424 gs=1 cs=0 l=1 c_cookie=1105290295135865297 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.715+0000 7f8bdaebe640 1 --2- 10.20.193.28:0/3720272934 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8bd410bd50 0x7f8bd4110be0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.715+0000 7f8bdaebe640 1 --2- 10.20.193.28:0/3720272934 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8bd410b360 0x7f8bd410b730 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.715+0000 7f8bdaebe640 1 -- 10.20.193.28:0/3720272934 >> 10.20.193.28:0/3720272934 conn(0x7f8bd40e8e70 msgr2=0x7f8bd4109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:02.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.715+0000 7f8bdaebe640 1 -- 10.20.193.28:0/3720272934 shutdown_connections 2026-02-25T11:49:02.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.715+0000 7f8bdaebe640 1 -- 10.20.193.28:0/3720272934 wait complete. 2026-02-25T11:49:02.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.716+0000 7f8bdaebe640 1 Processor -- start 2026-02-25T11:49:02.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.716+0000 7f8bdaebe640 1 -- start start 2026-02-25T11:49:02.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.716+0000 7f8bdaebe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8bd410b360 0x7f8bd41235f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.716+0000 7f8bdaebe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8bd410bd50 0x7f8bd4123b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.716+0000 7f8bdaebe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8bd4111370 0x7f8bd4128f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.716+0000 7f8bdaebe640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4129520 con 0x7f8bd410b360 2026-02-25T11:49:02.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.716+0000 7f8bdaebe640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4129690 con 0x7f8bd4111370 2026-02-25T11:49:02.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.716+0000 7f8bdaebe640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4129850 con 0x7f8bd410bd50 2026-02-25T11:49:02.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bd9ebc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8bd410b360 0x7f8bd41235f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bd9ebc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8bd410b360 0x7f8bd41235f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:54858/0 (socket says 10.20.193.28:54858) 2026-02-25T11:49:02.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bd9ebc640 1 -- 10.20.193.28:0/1019919006 learned_addr learned my addr 10.20.193.28:0/1019919006 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:02.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bd96bb640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8bd410bd50 0x7f8bd4123b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bda6bd640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8bd4111370 0x7f8bd4128f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bd9ebc640 1 -- 10.20.193.28:0/1019919006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8bd410bd50 msgr2=0x7f8bd4123b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bd9ebc640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8bd410bd50 0x7f8bd4123b30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bd9ebc640 1 -- 10.20.193.28:0/1019919006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8bd4111370 msgr2=0x7f8bd4128f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bd9ebc640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8bd4111370 0x7f8bd4128f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bd9ebc640 1 -- 10.20.193.28:0/1019919006 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd000a650 con 0x7f8bd410b360 2026-02-25T11:49:02.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bd9ebc640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8bd410b360 0x7f8bd41235f0 secure :-1 s=READY pgs=425 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8bc4009480 tx=0x7f8bc400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:02.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bc2ffd640 1 -- 10.20.193.28:0/1019919006 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8bc400cc80 con 0x7f8bd410b360 2026-02-25T11:49:02.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bc2ffd640 1 -- 10.20.193.28:0/1019919006 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8bc400cde0 con 0x7f8bd410b360 2026-02-25T11:49:02.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bc2ffd640 1 -- 10.20.193.28:0/1019919006 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8bc40106b0 con 0x7f8bd410b360 2026-02-25T11:49:02.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.717+0000 7f8bdaebe640 1 -- 10.20.193.28:0/1019919006 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd4129b30 con 0x7f8bd410b360 2026-02-25T11:49:02.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.718+0000 7f8bdaebe640 1 -- 10.20.193.28:0/1019919006 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd416a230 con 0x7f8bd410b360 2026-02-25T11:49:02.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.718+0000 7f8bdaebe640 1 -- 10.20.193.28:0/1019919006 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd4129d50 con 0x7f8bd410b360 2026-02-25T11:49:02.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.720+0000 7f8bc2ffd640 1 -- 10.20.193.28:0/1019919006 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f8bc40040d0 con 0x7f8bd410b360 2026-02-25T11:49:02.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.722+0000 7f8bc2ffd640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8bb00b5760 0x7f8bb00d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:02.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.722+0000 7f8bc2ffd640 1 -- 10.20.193.28:0/1019919006 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8bc4014070 con 0x7f8bd410b360 2026-02-25T11:49:02.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.722+0000 7f8bc2ffd640 1 -- 10.20.193.28:0/1019919006 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f8bc40e7190 con 0x7f8bd410b360 2026-02-25T11:49:02.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.722+0000 7f8bd96bb640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8bb00b5760 0x7f8bb00d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:02.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.724+0000 7f8bd96bb640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8bb00b5760 0x7f8bb00d5c10 secure :-1 s=READY pgs=269 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8bd4129f90 tx=0x7f8bc8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:02.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.865+0000 7f8bdaebe640 1 -- 10.20.193.28:0/1019919006 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bd4124750 con 0x7f8bd410b360 2026-02-25T11:49:02.868 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.865+0000 7f8bc2ffd640 1 -- 10.20.193.28:0/1019919006 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f8bc40a98a0 con 0x7f8bd410b360 2026-02-25T11:49:02.868 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:49:02.868 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:49:02.869 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:49:02.869 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.866+0000 7f8bdaebe640 1 -- 10.20.193.28:0/1019919006 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8bb00b5760 msgr2=0x7f8bb00d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.869 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.866+0000 7f8bdaebe640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8bb00b5760 0x7f8bb00d5c10 secure :-1 s=READY pgs=269 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8bd4129f90 tx=0x7f8bc8005f90 comp rx=0 tx=0).stop 2026-02-25T11:49:02.869 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.866+0000 7f8bdaebe640 1 -- 10.20.193.28:0/1019919006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8bd410b360 msgr2=0x7f8bd41235f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:02.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.866+0000 7f8bdaebe640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8bd410b360 0x7f8bd41235f0 secure :-1 s=READY pgs=425 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8bc4009480 tx=0x7f8bc400ee60 comp rx=0 tx=0).stop 2026-02-25T11:49:02.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.867+0000 7f8bdaebe640 1 -- 10.20.193.28:0/1019919006 shutdown_connections 2026-02-25T11:49:02.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.867+0000 7f8bdaebe640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8bb00b5760 0x7f8bb00d5c10 unknown :-1 s=CLOSED pgs=269 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.867+0000 7f8bdaebe640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8bd4111370 0x7f8bd4128f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.867+0000 7f8bdaebe640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8bd410bd50 0x7f8bd4123b30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.867+0000 7f8bdaebe640 1 --2- 10.20.193.28:0/1019919006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8bd410b360 0x7f8bd41235f0 unknown :-1 s=CLOSED pgs=425 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:02.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.867+0000 7f8bdaebe640 1 -- 10.20.193.28:0/1019919006 >> 10.20.193.28:0/1019919006 conn(0x7f8bd40e8e70 msgr2=0x7f8bd4109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:02.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.867+0000 7f8bdaebe640 1 -- 10.20.193.28:0/1019919006 shutdown_connections 2026-02-25T11:49:02.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:02.867+0000 7f8bdaebe640 1 -- 10.20.193.28:0/1019919006 wait complete. 2026-02-25T11:49:03.005 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:49:03.190 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:03.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.379+0000 7fb040bca640 1 -- 10.20.193.28:0/1930812227 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb03c10f740 msgr2=0x7fb03c10fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.379+0000 7fb040bca640 1 --2- 10.20.193.28:0/1930812227 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb03c10f740 0x7fb03c10fb10 secure :-1 s=READY pgs=426 gs=1 cs=0 l=1 c_cookie=10723540738141760431 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb030009950 tx=0x7fb03002f190 comp rx=0 tx=0).stop 2026-02-25T11:49:03.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.379+0000 7fb040bca640 1 -- 10.20.193.28:0/1930812227 shutdown_connections 2026-02-25T11:49:03.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.379+0000 7fb040bca640 1 --2- 10.20.193.28:0/1930812227 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb03c115930 0x7fb03c176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.379+0000 7fb040bca640 1 --2- 10.20.193.28:0/1930812227 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb03c110130 0x7fb03c115180 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.379+0000 7fb040bca640 1 --2- 10.20.193.28:0/1930812227 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb03c10f740 0x7fb03c10fb10 unknown :-1 s=CLOSED pgs=426 gs=1 cs=0 l=1 c_cookie=10723540738141760431 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.379+0000 7fb040bca640 1 -- 10.20.193.28:0/1930812227 >> 10.20.193.28:0/1930812227 conn(0x7fb03c0ecff0 msgr2=0x7fb03c10d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:03.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.379+0000 7fb040bca640 1 -- 10.20.193.28:0/1930812227 shutdown_connections 2026-02-25T11:49:03.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.379+0000 7fb040bca640 1 -- 10.20.193.28:0/1930812227 wait complete. 2026-02-25T11:49:03.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.380+0000 7fb040bca640 1 Processor -- start 2026-02-25T11:49:03.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.380+0000 7fb040bca640 1 -- start start 2026-02-25T11:49:03.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.380+0000 7fb040bca640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb03c10f740 0x7fb03c0f5a30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.380+0000 7fb040bca640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb03c110130 0x7fb03c176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.380+0000 7fb040bca640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb03c115930 0x7fb03c0f60c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.380+0000 7fb040bca640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c0fb340 con 0x7fb03c110130 2026-02-25T11:49:03.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.380+0000 7fb040bca640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c0fb4b0 con 0x7fb03c10f740 2026-02-25T11:49:03.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.380+0000 7fb040bca640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb03c0fb620 con 0x7fb03c115930 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.381+0000 7fb03a575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb03c10f740 0x7fb03c0f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.381+0000 7fb03a575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb03c10f740 0x7fb03c0f5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:49002/0 (socket says 10.20.193.28:49002) 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.381+0000 7fb039d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb03c110130 0x7fb03c176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.381+0000 7fb03a575640 1 -- 10.20.193.28:0/3368942488 learned_addr learned my addr 10.20.193.28:0/3368942488 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.381+0000 7fb03ad76640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb03c115930 0x7fb03c0f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.381+0000 7fb03a575640 1 -- 10.20.193.28:0/3368942488 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb03c115930 msgr2=0x7fb03c0f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.381+0000 7fb03a575640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb03c115930 0x7fb03c0f60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.381+0000 7fb03a575640 1 -- 10.20.193.28:0/3368942488 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb03c110130 msgr2=0x7fb03c176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.381+0000 7fb03a575640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb03c110130 0x7fb03c176200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.381+0000 7fb03a575640 1 -- 10.20.193.28:0/3368942488 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb030009570 con 0x7fb03c10f740 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.381+0000 7fb039d74640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb03c110130 0x7fb03c176200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.381+0000 7fb03a575640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb03c10f740 0x7fb03c0f5a30 secure :-1 s=READY pgs=112 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb0300098a0 tx=0x7fb0300028d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.381+0000 7fb0237fe640 1 -- 10.20.193.28:0/3368942488 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb03003d070 con 0x7fb03c10f740 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.382+0000 7fb040bca640 1 -- 10.20.193.28:0/3368942488 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03c0fb8a0 con 0x7fb03c10f740 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.382+0000 7fb0237fe640 1 -- 10.20.193.28:0/3368942488 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb030002a40 con 0x7fb03c10f740 2026-02-25T11:49:03.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.382+0000 7fb0237fe640 1 -- 10.20.193.28:0/3368942488 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb030041720 con 0x7fb03c10f740 2026-02-25T11:49:03.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.382+0000 7fb040bca640 1 -- 10.20.193.28:0/3368942488 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03c0fbd60 con 0x7fb03c10f740 2026-02-25T11:49:03.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.384+0000 7fb0237fe640 1 -- 10.20.193.28:0/3368942488 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fb0300386a0 con 0x7fb03c10f740 2026-02-25T11:49:03.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.386+0000 7fb0237fe640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb0140b5760 0x7fb0140d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.386+0000 7fb0237fe640 1 -- 10.20.193.28:0/3368942488 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb03010aff0 con 0x7fb03c10f740 2026-02-25T11:49:03.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.387+0000 7fb039d74640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb0140b5760 0x7fb0140d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.387+0000 7fb039d74640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb0140b5760 0x7fb0140d5c10 secure :-1 s=READY pgs=270 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb03c0fc3d0 tx=0x7fb024005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:03.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.387+0000 7fb042bc5e00 1 Processor -- start 2026-02-25T11:49:03.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.387+0000 7fb042bc5e00 1 -- start start 2026-02-25T11:49:03.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb042bc5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7e3f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb042bc5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e4440 0x55e2cb828ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb042bc5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb8290e0 0x55e2cb8494c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb042bc5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e2cb6f98a0 con 0x55e2cb8290e0 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb042bc5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e2cb6f55d0 con 0x55e2cb7e3b30 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb042bc5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e2cb6ef570 con 0x55e2cb7e4440 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb03ad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb8290e0 0x55e2cb8494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb03ad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb8290e0 0x55e2cb8494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:54886/0 (socket says 10.20.193.28:54886) 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb03ad76640 1 -- 10.20.193.28:0/2308080900 learned_addr learned my addr 10.20.193.28:0/2308080900 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb039d74640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e4440 0x55e2cb828ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb03a575640 1 --2- 10.20.193.28:0/2308080900 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7e3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb03ad76640 1 -- 10.20.193.28:0/2308080900 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e4440 msgr2=0x55e2cb828ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb03ad76640 1 --2- 10.20.193.28:0/2308080900 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e4440 0x55e2cb828ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb03ad76640 1 -- 10.20.193.28:0/2308080900 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb7e3b30 msgr2=0x55e2cb7e3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb03ad76640 1 --2- 10.20.193.28:0/2308080900 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7e3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb03ad76640 1 -- 10.20.193.28:0/2308080900 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2cb69ad10 con 0x55e2cb8290e0 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.388+0000 7fb03a575640 1 --2- 10.20.193.28:0/2308080900 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7e3f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:03.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.389+0000 7fb03ad76640 1 --2- 10.20.193.28:0/2308080900 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb8290e0 0x55e2cb8494c0 secure :-1 s=READY pgs=427 gs=3 cs=0 l=1 c_cookie=17003695083400464176 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb02c0056b0 tx=0x7fb02c003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf94296c3240730 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:03.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.389+0000 7fb020ff9640 1 -- 10.20.193.28:0/2308080900 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb02c00ae30 con 0x55e2cb8290e0 2026-02-25T11:49:03.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.389+0000 7fb020ff9640 1 -- 10.20.193.28:0/2308080900 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb02c003e90 con 0x55e2cb8290e0 2026-02-25T11:49:03.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.389+0000 7fb020ff9640 1 -- 10.20.193.28:0/2308080900 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb02c011910 con 0x55e2cb8290e0 2026-02-25T11:49:03.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.389+0000 7fb042bc5e00 1 -- 10.20.193.28:0/2308080900 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb8290e0 msgr2=0x55e2cb8494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.389+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/2308080900 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb8290e0 0x55e2cb8494c0 secure :-1 s=READY pgs=427 gs=3 cs=0 l=1 c_cookie=17003695083400464176 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb02c0056b0 tx=0x7fb02c003c30 comp rx=0 tx=0).stop 2026-02-25T11:49:03.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.389+0000 7fb042bc5e00 1 -- 10.20.193.28:0/2308080900 shutdown_connections 2026-02-25T11:49:03.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.389+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/2308080900 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb8290e0 0x55e2cb8494c0 unknown :-1 s=CLOSED pgs=427 gs=3 cs=0 l=1 c_cookie=17003695083400464176 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.389+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/2308080900 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e4440 0x55e2cb828ba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.389+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/2308080900 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7e3f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.389+0000 7fb042bc5e00 1 -- 10.20.193.28:0/2308080900 >> 10.20.193.28:0/2308080900 conn(0x55e2cb7027a0 msgr2=0x55e2cb7e2410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:03.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.389+0000 7fb042bc5e00 1 -- 10.20.193.28:0/2308080900 shutdown_connections 2026-02-25T11:49:03.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.389+0000 7fb042bc5e00 1 -- 10.20.193.28:0/2308080900 wait complete. 2026-02-25T11:49:03.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.390+0000 7fb042bc5e00 1 -- start start 2026-02-25T11:49:03.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.390+0000 7fb042bc5e00 1 Processor -- start 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.391+0000 7fb042bc5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7fbd70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.391+0000 7fb042bc5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb7e4440 0x55e2cb7fc2b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.391+0000 7fb042bc5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb8290e0 0x55e2cb8012a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.391+0000 7fb042bc5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e2cb8017e0 con 0x55e2cb7e4440 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.391+0000 7fb042bc5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e2cb801950 con 0x55e2cb8290e0 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.391+0000 7fb042bc5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e2cb801b10 con 0x55e2cb7e3b30 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.391+0000 7fb039d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb7e4440 0x55e2cb7fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.391+0000 7fb03a575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.391+0000 7fb03a575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7fbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:57438/0 (socket says 10.20.193.28:57438) 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.391+0000 7fb03a575640 1 -- 10.20.193.28:0/1782167265 learned_addr learned my addr 10.20.193.28:0/1782167265 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.391+0000 7fb03ad76640 1 --2- 10.20.193.28:0/1782167265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb8290e0 0x55e2cb8012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.391+0000 7fb03a575640 1 -- 10.20.193.28:0/1782167265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb8290e0 msgr2=0x55e2cb8012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fb03a575640 1 --2- 10.20.193.28:0/1782167265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb8290e0 0x55e2cb8012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fb03a575640 1 -- 10.20.193.28:0/1782167265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb7e4440 msgr2=0x55e2cb7fc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fb03a575640 1 --2- 10.20.193.28:0/1782167265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb7e4440 0x55e2cb7fc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fb03a575640 1 -- 10.20.193.28:0/1782167265 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2cb6733c0 con 0x55e2cb7e3b30 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fb039d74640 1 --2- 10.20.193.28:0/1782167265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb7e4440 0x55e2cb7fc2b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fb03a575640 1 --2- 10.20.193.28:0/1782167265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7fbd70 secure :-1 s=READY pgs=125 gs=1 cs=0 l=1 c_cookie=6986207506261372493 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb030004030 tx=0x7fb0300d3530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60f3feb661e06e4d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fafebfff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fafebfff640 1 -- 10.20.193.28:0/1782167265 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb0300d0b70 con 0x55e2cb7e3b30 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fb042bc5e00 1 -- 10.20.193.28:0/1782167265 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02c005350 con 0x55e2cb7e3b30 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fafebfff640 1 -- 10.20.193.28:0/1782167265 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb030048e80 con 0x55e2cb7e3b30 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fafebfff640 1 -- 10.20.193.28:0/1782167265 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb0300cf350 con 0x55e2cb7e3b30 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fb042bc5e00 10 client.25513 _ll_register_callbacks cb 0x55e2cb670a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fb042bc5e00 1 -- 10.20.193.28:0/1782167265 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e2cb3856f0 con 0x55e2cb7e3b30 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fafebfff640 1 -- 10.20.193.28:0/1782167265 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb0300cf4b0 con 0x55e2cb7e3b30 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.392+0000 7fafebfff640 1 -- 10.20.193.28:0/1782167265 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb0300d2d60 con 0x55e2cb7e3b30 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.393+0000 7fb042bc5e00 10 client.25513 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:03.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.393+0000 7fb042bc5e00 1 -- 10.20.193.28:0/1782167265 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55e2cb6850e0 con 0x55e2cb7e3b30 2026-02-25T11:49:03.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.393+0000 7fafebfff640 1 -- 10.20.193.28:0/1782167265 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb0300ec030 con 0x55e2cb7e3b30 2026-02-25T11:49:03.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.393+0000 7fb042bc5e00 10 client.25513 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:03.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.393+0000 7fb042bc5e00 10 client.25513 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:49:03.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.393+0000 7fb042bc5e00 20 client.25513 populate_metadata read hostname 'trial028' 2026-02-25T11:49:03.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.393+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/1782167265 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55e2cb826310 0x55e2cb8267b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.393+0000 7fb042bc5e00 4 client.25513 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:49:03.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.393+0000 7fb042bc5e00 1 -- 10.20.193.28:0/1782167265 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2cb6c1fc0 con 0x55e2cb826310 2026-02-25T11:49:03.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.393+0000 7fb039d74640 1 --2- 10.20.193.28:0/1782167265 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55e2cb826310 0x55e2cb8267b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.394+0000 7fb039d74640 1 --2- 10.20.193.28:0/1782167265 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55e2cb826310 0x55e2cb8267b0 crc :-1 s=READY pgs=27 gs=4 cs=0 l=0 c_cookie=2328677822843214554 s_cookie=15194593218299945935 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=205120245ae65ada server_cookie=d2de0a071353e3cf in_seq=0 out_seq=0 2026-02-25T11:49:03.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.394+0000 7fafebfff640 10 client.25513 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:49:03.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.394+0000 7fafebfff640 1 -- 10.20.193.28:0/1782167265 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55e2cb6c1fc0 con 0x55e2cb826310 2026-02-25T11:49:03.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.394+0000 7fafebfff640 10 client.25513 handle_command_reply: tid=0 2026-02-25T11:49:03.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.394+0000 7fb042bc5e00 1 client.25513 shutdown 2026-02-25T11:49:03.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.395+0000 7fb042bc5e00 1 -- 10.20.193.28:0/1782167265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 msgr2=0x55e2cb7fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.395+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/1782167265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7fbd70 secure :-1 s=READY pgs=125 gs=1 cs=0 l=1 c_cookie=6986207506261372493 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb030004030 tx=0x7fb0300d3530 comp rx=0 tx=0).stop 2026-02-25T11:49:03.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.395+0000 7fb042bc5e00 1 -- 10.20.193.28:0/1782167265 shutdown_connections 2026-02-25T11:49:03.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.395+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/1782167265 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55e2cb826310 0x55e2cb8267b0 crc :-1 s=READY pgs=27 gs=4 cs=0 l=0 c_cookie=2328677822843214554 s_cookie=15194593218299945935 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.395+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/1782167265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb8290e0 0x55e2cb8012a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.395+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/1782167265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb7e4440 0x55e2cb7fc2b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.395+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/1782167265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7fbd70 unknown :-1 s=CLOSED pgs=125 gs=1 cs=0 l=1 c_cookie=6986207506261372493 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.395+0000 7fb042bc5e00 1 -- 10.20.193.28:0/1782167265 >> 10.20.193.28:0/1782167265 conn(0x55e2cb7027a0 msgr2=0x55e2cb7e2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:03.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.395+0000 7fafebfff640 0 client.25513 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:49:03.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.395+0000 7fb042bc5e00 1 -- 10.20.193.28:0/1782167265 shutdown_connections 2026-02-25T11:49:03.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.395+0000 7fb042bc5e00 1 -- 10.20.193.28:0/1782167265 wait complete. 2026-02-25T11:49:03.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.395+0000 7fb042bc5e00 20 client.25513 trim_cache size 0 max 16384 2026-02-25T11:49:03.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.402+0000 7fb042bc5e00 1 Processor -- start 2026-02-25T11:49:03.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.402+0000 7fb042bc5e00 1 -- start start 2026-02-25T11:49:03.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb042bc5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7edfa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb042bc5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb7e4440 0x55e2cb7f91d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb042bc5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb8290e0 0x55e2cb7f9710 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb042bc5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e2cb6ed1a0 con 0x55e2cb8290e0 2026-02-25T11:49:03.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb042bc5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e2cb3ad700 con 0x55e2cb7e4440 2026-02-25T11:49:03.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb042bc5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e2cb7f5150 con 0x55e2cb7e3b30 2026-02-25T11:49:03.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb03ad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb8290e0 0x55e2cb7f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb03a575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb03a575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7edfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:57444/0 (socket says 10.20.193.28:57444) 2026-02-25T11:49:03.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb03a575640 1 -- 10.20.193.28:0/3545668673 learned_addr learned my addr 10.20.193.28:0/3545668673 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:03.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb039d74640 1 --2- 10.20.193.28:0/3545668673 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb7e4440 0x55e2cb7f91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb03ad76640 1 -- 10.20.193.28:0/3545668673 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 msgr2=0x55e2cb7edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb03ad76640 1 --2- 10.20.193.28:0/3545668673 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7edfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb03ad76640 1 -- 10.20.193.28:0/3545668673 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb7e4440 msgr2=0x55e2cb7f91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb03ad76640 1 --2- 10.20.193.28:0/3545668673 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb7e4440 0x55e2cb7f91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb03ad76640 1 -- 10.20.193.28:0/3545668673 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2cb67fb60 con 0x55e2cb8290e0 2026-02-25T11:49:03.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb03a575640 1 --2- 10.20.193.28:0/3545668673 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7edfa0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:03.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.403+0000 7fb03ad76640 1 --2- 10.20.193.28:0/3545668673 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb8290e0 0x55e2cb7f9710 secure :-1 s=READY pgs=428 gs=3 cs=0 l=1 c_cookie=564882805602627743 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb02c012d10 tx=0x7fb02c006f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6ddeb37abb09f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:03.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.404+0000 7fb0064e6640 1 -- 10.20.193.28:0/3545668673 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb02c0057c0 con 0x55e2cb8290e0 2026-02-25T11:49:03.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.404+0000 7fb0064e6640 1 -- 10.20.193.28:0/3545668673 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb02c023040 con 0x55e2cb8290e0 2026-02-25T11:49:03.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.404+0000 7fb0064e6640 1 -- 10.20.193.28:0/3545668673 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb02c0062c0 con 0x55e2cb8290e0 2026-02-25T11:49:03.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.404+0000 7fb042bc5e00 1 -- 10.20.193.28:0/3545668673 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb8290e0 msgr2=0x55e2cb7f9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.404+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/3545668673 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb8290e0 0x55e2cb7f9710 secure :-1 s=READY pgs=428 gs=3 cs=0 l=1 c_cookie=564882805602627743 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb02c012d10 tx=0x7fb02c006f70 comp rx=0 tx=0).stop 2026-02-25T11:49:03.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.404+0000 7fb042bc5e00 1 -- 10.20.193.28:0/3545668673 shutdown_connections 2026-02-25T11:49:03.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.404+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/3545668673 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb8290e0 0x55e2cb7f9710 unknown :-1 s=CLOSED pgs=428 gs=3 cs=0 l=1 c_cookie=564882805602627743 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.404+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/3545668673 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb7e4440 0x55e2cb7f91d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.404+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/3545668673 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb7edfa0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.404+0000 7fb042bc5e00 1 -- 10.20.193.28:0/3545668673 >> 10.20.193.28:0/3545668673 conn(0x55e2cb7027a0 msgr2=0x55e2cb7e6820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:03.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.404+0000 7fb042bc5e00 1 -- 10.20.193.28:0/3545668673 shutdown_connections 2026-02-25T11:49:03.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.404+0000 7fb042bc5e00 1 -- 10.20.193.28:0/3545668673 wait complete. 2026-02-25T11:49:03.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.405+0000 7fb042bc5e00 1 -- start start 2026-02-25T11:49:03.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.405+0000 7fb042bc5e00 1 Processor -- start 2026-02-25T11:49:03.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.405+0000 7fb042bc5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb807900 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.405+0000 7fb042bc5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb7e4440 0x55e2cb807e60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.405+0000 7fb042bc5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb8290e0 0x55e2cb802f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.405+0000 7fb042bc5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e2cb8035f0 con 0x55e2cb7e4440 2026-02-25T11:49:03.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.405+0000 7fb042bc5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e2cb803760 con 0x55e2cb8290e0 2026-02-25T11:49:03.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.405+0000 7fb042bc5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e2cb803920 con 0x55e2cb7e3b30 2026-02-25T11:49:03.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.405+0000 7fb039d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb7e4440 0x55e2cb807e60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.405+0000 7fb03a575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb807900 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.405+0000 7fb03a575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb807900 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:57450/0 (socket says 10.20.193.28:57450) 2026-02-25T11:49:03.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.405+0000 7fb03a575640 1 -- 10.20.193.28:0/30404789 learned_addr learned my addr 10.20.193.28:0/30404789 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:03.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.405+0000 7fb03ad76640 1 --2- 10.20.193.28:0/30404789 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb8290e0 0x55e2cb802f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fb03a575640 1 -- 10.20.193.28:0/30404789 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb8290e0 msgr2=0x55e2cb802f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fb03a575640 1 --2- 10.20.193.28:0/30404789 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb8290e0 0x55e2cb802f20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fb03a575640 1 -- 10.20.193.28:0/30404789 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb7e4440 msgr2=0x55e2cb807e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fb03a575640 1 --2- 10.20.193.28:0/30404789 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb7e4440 0x55e2cb807e60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fb03a575640 1 -- 10.20.193.28:0/30404789 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2cb69ad10 con 0x55e2cb7e3b30 2026-02-25T11:49:03.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fb03a575640 1 --2- 10.20.193.28:0/30404789 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb807900 secure :-1 s=READY pgs=126 gs=1 cs=0 l=1 c_cookie=12798434001235375274 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb0300cd8f0 tx=0x7fb0300d34b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b19d2bea44ea74aa server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:03.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fafebfff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:49:03.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fafebfff640 1 -- 10.20.193.28:0/30404789 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb0300d8460 con 0x55e2cb7e3b30 2026-02-25T11:49:03.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fafebfff640 1 -- 10.20.193.28:0/30404789 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55e2cb3856f0 con 0x55e2cb7e3b30 2026-02-25T11:49:03.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fb042bc5e00 1 -- 10.20.193.28:0/30404789 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2cb6850e0 con 0x55e2cb7e3b30 2026-02-25T11:49:03.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fafebfff640 1 -- 10.20.193.28:0/30404789 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb0300cf350 con 0x55e2cb7e3b30 2026-02-25T11:49:03.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fafebfff640 1 -- 10.20.193.28:0/30404789 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb0300cf4b0 con 0x55e2cb7e3b30 2026-02-25T11:49:03.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fafebfff640 10 client.25519 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:49:03.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fb042bc5e00 10 client.25519 _ll_register_callbacks cb 0x55e2cb670a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:03.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fb042bc5e00 1 -- 10.20.193.28:0/30404789 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb02c003e90 con 0x55e2cb7e3b30 2026-02-25T11:49:03.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fafebfff640 1 -- 10.20.193.28:0/30404789 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb030048e80 con 0x55e2cb7e3b30 2026-02-25T11:49:03.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fb042bc5e00 10 client.25519 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:03.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.406+0000 7fb042bc5e00 1 -- 10.20.193.28:0/30404789 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55e2cb6736f0 con 0x55e2cb7e3b30 2026-02-25T11:49:03.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.407+0000 7fafebfff640 1 -- 10.20.193.28:0/30404789 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb0300ec030 con 0x55e2cb7e3b30 2026-02-25T11:49:03.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.407+0000 7fb042bc5e00 10 client.25519 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:03.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.407+0000 7fb042bc5e00 10 client.25519 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:49:03.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.407+0000 7fb042bc5e00 20 client.25519 populate_metadata read hostname 'trial028' 2026-02-25T11:49:03.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.407+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/30404789 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55e2cb8781d0 0x55e2cb878670 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.407+0000 7fb042bc5e00 4 client.25519 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:49:03.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.407+0000 7fb042bc5e00 1 -- 10.20.193.28:0/30404789 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2cb382f40 con 0x55e2cb8781d0 2026-02-25T11:49:03.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.407+0000 7fb039d74640 1 --2- 10.20.193.28:0/30404789 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55e2cb8781d0 0x55e2cb878670 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.407+0000 7fb039d74640 1 --2- 10.20.193.28:0/30404789 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55e2cb8781d0 0x55e2cb878670 crc :-1 s=READY pgs=28 gs=4 cs=0 l=0 c_cookie=1818138235619747961 s_cookie=8853990989483958650 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=193b5317a6aba879 server_cookie=7adfb1e4fcb3617a in_seq=0 out_seq=0 2026-02-25T11:49:03.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.407+0000 7fafebfff640 10 client.25519 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:49:03.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fafebfff640 1 -- 10.20.193.28:0/30404789 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2cb382f40 con 0x55e2cb8781d0 2026-02-25T11:49:03.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fafebfff640 10 client.25519 handle_command_reply: tid=0 2026-02-25T11:49:03.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb042bc5e00 1 client.25519 shutdown 2026-02-25T11:49:03.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb042bc5e00 1 -- 10.20.193.28:0/30404789 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 msgr2=0x55e2cb807900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/30404789 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb807900 secure :-1 s=READY pgs=126 gs=1 cs=0 l=1 c_cookie=12798434001235375274 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb0300cd8f0 tx=0x7fb0300d34b0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb042bc5e00 1 -- 10.20.193.28:0/30404789 shutdown_connections 2026-02-25T11:49:03.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/30404789 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55e2cb8781d0 0x55e2cb878670 crc :-1 s=READY pgs=28 gs=4 cs=0 l=0 c_cookie=1818138235619747961 s_cookie=8853990989483958650 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/30404789 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e2cb8290e0 0x55e2cb802f20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/30404789 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e2cb7e4440 0x55e2cb807e60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb042bc5e00 1 --2- 10.20.193.28:0/30404789 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e2cb7e3b30 0x55e2cb807900 unknown :-1 s=CLOSED pgs=126 gs=1 cs=0 l=1 c_cookie=12798434001235375274 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb042bc5e00 1 -- 10.20.193.28:0/30404789 >> 10.20.193.28:0/30404789 conn(0x55e2cb7027a0 msgr2=0x55e2cb800100 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:03.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fafebfff640 0 client.25519 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:49:03.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb042bc5e00 1 -- 10.20.193.28:0/30404789 shutdown_connections 2026-02-25T11:49:03.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb042bc5e00 1 -- 10.20.193.28:0/30404789 wait complete. 2026-02-25T11:49:03.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb042bc5e00 20 client.25519 trim_cache size 0 max 16384 2026-02-25T11:49:03.426 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:49:03.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb040bca640 1 -- 10.20.193.28:0/3368942488 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb0140b5760 msgr2=0x7fb0140d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb040bca640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb0140b5760 0x7fb0140d5c10 secure :-1 s=READY pgs=270 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb03c0fc3d0 tx=0x7fb024005f90 comp rx=0 tx=0).stop 2026-02-25T11:49:03.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb040bca640 1 -- 10.20.193.28:0/3368942488 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb03c10f740 msgr2=0x7fb03c0f5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb040bca640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb03c10f740 0x7fb03c0f5a30 secure :-1 s=READY pgs=112 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb0300098a0 tx=0x7fb0300028d0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb040bca640 1 -- 10.20.193.28:0/3368942488 shutdown_connections 2026-02-25T11:49:03.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb040bca640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb0140b5760 0x7fb0140d5c10 unknown :-1 s=CLOSED pgs=270 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb040bca640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb03c115930 0x7fb03c0f60c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb040bca640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb03c110130 0x7fb03c176200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb040bca640 1 --2- 10.20.193.28:0/3368942488 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb03c10f740 0x7fb03c0f5a30 unknown :-1 s=CLOSED pgs=112 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.408+0000 7fb040bca640 1 -- 10.20.193.28:0/3368942488 >> 10.20.193.28:0/3368942488 conn(0x7fb03c0ecff0 msgr2=0x7fb03c111c40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:03.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.409+0000 7fb040bca640 1 -- 10.20.193.28:0/3368942488 shutdown_connections 2026-02-25T11:49:03.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.409+0000 7fb040bca640 1 -- 10.20.193.28:0/3368942488 wait complete. 2026-02-25T11:49:03.479 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:03 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1019919006' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:49:03.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:03 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1019919006' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:49:03.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:03 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1019919006' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:49:03.519 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:49:03.703 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:03.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.881+0000 7f6b11b8e640 1 -- 10.20.193.28:0/484505288 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b0c10e190 msgr2=0x7f6b0c10e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.881+0000 7f6b11b8e640 1 --2- 10.20.193.28:0/484505288 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b0c10e190 0x7f6b0c10e560 secure :-1 s=READY pgs=113 gs=2 cs=0 l=1 c_cookie=276704183540743609 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6afc009890 tx=0x7f6afc02f0e0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.881+0000 7f6b11b8e640 1 -- 10.20.193.28:0/484505288 shutdown_connections 2026-02-25T11:49:03.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.881+0000 7f6b11b8e640 1 --2- 10.20.193.28:0/484505288 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b0c153740 0x7f6b0c173b60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.881+0000 7f6b11b8e640 1 --2- 10.20.193.28:0/484505288 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b0c10eb80 0x7f6b0c153200 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.881+0000 7f6b11b8e640 1 --2- 10.20.193.28:0/484505288 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b0c10e190 0x7f6b0c10e560 unknown :-1 s=CLOSED pgs=113 gs=2 cs=0 l=1 c_cookie=276704183540743609 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.881+0000 7f6b11b8e640 1 -- 10.20.193.28:0/484505288 >> 10.20.193.28:0/484505288 conn(0x7f6b0c0e9b40 msgr2=0x7f6b0c109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:03.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.881+0000 7f6b11b8e640 1 -- 10.20.193.28:0/484505288 shutdown_connections 2026-02-25T11:49:03.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.881+0000 7f6b11b8e640 1 -- 10.20.193.28:0/484505288 wait complete. 2026-02-25T11:49:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.881+0000 7f6b11b8e640 1 Processor -- start 2026-02-25T11:49:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b11b8e640 1 -- start start 2026-02-25T11:49:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b11b8e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b0c10e190 0x7f6b0c162170 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b11b8e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b0c10eb80 0x7f6b0c1626b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b11b8e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b0c153740 0x7f6b0c167a80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b11b8e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1680a0 con 0x7f6b0c10e190 2026-02-25T11:49:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b11b8e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c168210 con 0x7f6b0c153740 2026-02-25T11:49:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b11b8e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1683d0 con 0x7f6b0c10eb80 2026-02-25T11:49:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b03fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b0c10eb80 0x7f6b0c1626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b10b8c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b0c10e190 0x7f6b0c162170 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b03fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b0c10eb80 0x7f6b0c1626b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:57468/0 (socket says 10.20.193.28:57468) 2026-02-25T11:49:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b03fff640 1 -- 10.20.193.28:0/3673926693 learned_addr learned my addr 10.20.193.28:0/3673926693 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b1138d640 1 --2- 10.20.193.28:0/3673926693 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b0c153740 0x7f6b0c167a80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b03fff640 1 -- 10.20.193.28:0/3673926693 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b0c153740 msgr2=0x7f6b0c167a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b03fff640 1 --2- 10.20.193.28:0/3673926693 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b0c153740 0x7f6b0c167a80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b03fff640 1 -- 10.20.193.28:0/3673926693 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b0c10e190 msgr2=0x7f6b0c162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b03fff640 1 --2- 10.20.193.28:0/3673926693 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b0c10e190 0x7f6b0c162170 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.882+0000 7f6b03fff640 1 -- 10.20.193.28:0/3673926693 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6afc0094b0 con 0x7f6b0c10eb80 2026-02-25T11:49:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.883+0000 7f6b03fff640 1 --2- 10.20.193.28:0/3673926693 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b0c10eb80 0x7f6b0c1626b0 secure :-1 s=READY pgs=127 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6af4009480 tx=0x7f6af400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.883+0000 7f6b01ffb640 1 -- 10.20.193.28:0/3673926693 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6af400cd40 con 0x7f6b0c10eb80 2026-02-25T11:49:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.883+0000 7f6b01ffb640 1 -- 10.20.193.28:0/3673926693 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6af4004d10 con 0x7f6b0c10eb80 2026-02-25T11:49:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.883+0000 7f6b01ffb640 1 -- 10.20.193.28:0/3673926693 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6af40106b0 con 0x7f6b0c10eb80 2026-02-25T11:49:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.883+0000 7f6b11b8e640 1 -- 10.20.193.28:0/3673926693 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b0c1686b0 con 0x7f6b0c10eb80 2026-02-25T11:49:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.883+0000 7f6b11b8e640 1 -- 10.20.193.28:0/3673926693 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b0c14f360 con 0x7f6b0c10eb80 2026-02-25T11:49:03.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.886+0000 7f6b01ffb640 1 -- 10.20.193.28:0/3673926693 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f6af401f060 con 0x7f6b0c10eb80 2026-02-25T11:49:03.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.887+0000 7f6b01ffb640 1 --2- 10.20.193.28:0/3673926693 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ae80b5760 0x7f6ae80d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.887+0000 7f6b01ffb640 1 -- 10.20.193.28:0/3673926693 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6af4014070 con 0x7f6b0c10eb80 2026-02-25T11:49:03.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.888+0000 7f6b10b8c640 1 --2- 10.20.193.28:0/3673926693 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ae80b5760 0x7f6ae80d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.888+0000 7f6b10b8c640 1 --2- 10.20.193.28:0/3673926693 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ae80b5760 0x7f6ae80d5c10 secure :-1 s=READY pgs=271 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6afc0097e0 tx=0x7f6afc03a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:03.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.888+0000 7f6b14e12e00 1 Processor -- start 2026-02-25T11:49:03.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b14e12e00 1 -- start start 2026-02-25T11:49:03.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b14e12e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d2f0f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b14e12e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d335ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b14e12e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 0x561c4d3564c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b14e12e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x561c4d2068a0 con 0x561c4d2f0b30 2026-02-25T11:49:03.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b14e12e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561c4d2025d0 con 0x561c4d2f1440 2026-02-25T11:49:03.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b14e12e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x561c4d1fc570 con 0x561c4d3360e0 2026-02-25T11:49:03.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b10b8c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d2f0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b10b8c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d2f0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:54954/0 (socket says 10.20.193.28:54954) 2026-02-25T11:49:03.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b10b8c640 1 -- 10.20.193.28:0/2957779054 learned_addr learned my addr 10.20.193.28:0/2957779054 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b03fff640 1 --2- 10.20.193.28:0/2957779054 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d335ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b1138d640 1 --2- 10.20.193.28:0/2957779054 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 0x561c4d3564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b10b8c640 1 -- 10.20.193.28:0/2957779054 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 msgr2=0x561c4d3564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b10b8c640 1 --2- 10.20.193.28:0/2957779054 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 0x561c4d3564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b10b8c640 1 -- 10.20.193.28:0/2957779054 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 msgr2=0x561c4d335ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b10b8c640 1 --2- 10.20.193.28:0/2957779054 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d335ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.889+0000 7f6b10b8c640 1 -- 10.20.193.28:0/2957779054 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c4d1a7d10 con 0x561c4d2f0b30 2026-02-25T11:49:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.890+0000 7f6b10b8c640 1 --2- 10.20.193.28:0/2957779054 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d2f0f00 secure :-1 s=READY pgs=429 gs=1 cs=0 l=1 c_cookie=18223597333837417223 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6afc009830 tx=0x7f6afc02f0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce73938ed228b07 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:03.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.890+0000 7f6ad94e4640 1 -- 10.20.193.28:0/2957779054 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6afc02fc70 con 0x561c4d2f0b30 2026-02-25T11:49:03.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.890+0000 7f6ad94e4640 1 -- 10.20.193.28:0/2957779054 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6afc04a040 con 0x561c4d2f0b30 2026-02-25T11:49:03.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.890+0000 7f6ad94e4640 1 -- 10.20.193.28:0/2957779054 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6afc04e700 con 0x561c4d2f0b30 2026-02-25T11:49:03.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.890+0000 7f6b14e12e00 1 -- 10.20.193.28:0/2957779054 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 msgr2=0x561c4d2f0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.890+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/2957779054 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d2f0f00 secure :-1 s=READY pgs=429 gs=1 cs=0 l=1 c_cookie=18223597333837417223 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6afc009830 tx=0x7f6afc02f0e0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.890+0000 7f6b14e12e00 1 -- 10.20.193.28:0/2957779054 shutdown_connections 2026-02-25T11:49:03.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.890+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/2957779054 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 0x561c4d3564c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.890+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/2957779054 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d335ba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.890+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/2957779054 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d2f0f00 unknown :-1 s=CLOSED pgs=429 gs=1 cs=0 l=1 c_cookie=18223597333837417223 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.890+0000 7f6b14e12e00 1 -- 10.20.193.28:0/2957779054 >> 10.20.193.28:0/2957779054 conn(0x561c4d20f7a0 msgr2=0x561c4d2ef410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:03.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.890+0000 7f6b14e12e00 1 -- 10.20.193.28:0/2957779054 shutdown_connections 2026-02-25T11:49:03.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.890+0000 7f6b14e12e00 1 -- 10.20.193.28:0/2957779054 wait complete. 2026-02-25T11:49:03.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.891+0000 7f6b14e12e00 1 -- start start 2026-02-25T11:49:03.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.892+0000 7f6b14e12e00 1 Processor -- start 2026-02-25T11:49:03.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.892+0000 7f6b14e12e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d308f10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.892+0000 7f6b14e12e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d309450 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.892+0000 7f6b14e12e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 0x561c4d30e440 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.892+0000 7f6b14e12e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x561c4d30e980 con 0x561c4d2f0b30 2026-02-25T11:49:03.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.892+0000 7f6b14e12e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561c4d30eaf0 con 0x561c4d2f1440 2026-02-25T11:49:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.892+0000 7f6b14e12e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x561c4d30ecb0 con 0x561c4d3360e0 2026-02-25T11:49:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.892+0000 7f6b10b8c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d308f10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.892+0000 7f6b03fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d309450 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.892+0000 7f6b10b8c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d308f10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:54964/0 (socket says 10.20.193.28:54964) 2026-02-25T11:49:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.892+0000 7f6b10b8c640 1 -- 10.20.193.28:0/1696841363 learned_addr learned my addr 10.20.193.28:0/1696841363 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.892+0000 7f6b1138d640 1 --2- 10.20.193.28:0/1696841363 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 0x561c4d30e440 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6b03fff640 1 -- 10.20.193.28:0/1696841363 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 msgr2=0x561c4d30e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6b03fff640 1 --2- 10.20.193.28:0/1696841363 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 0x561c4d30e440 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6b03fff640 1 -- 10.20.193.28:0/1696841363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 msgr2=0x561c4d308f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6b03fff640 1 --2- 10.20.193.28:0/1696841363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d308f10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6b03fff640 1 -- 10.20.193.28:0/1696841363 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c4d1803c0 con 0x561c4d2f1440 2026-02-25T11:49:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6b10b8c640 1 --2- 10.20.193.28:0/1696841363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d308f10 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6b03fff640 1 --2- 10.20.193.28:0/1696841363 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d309450 secure :-1 s=READY pgs=114 gs=2 cs=0 l=1 c_cookie=4019579538560522456 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6af40e4760 tx=0x7f6af40ad0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37c86a4b2e5418d8 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6aca7fc640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:49:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6aca7fc640 1 -- 10.20.193.28:0/1696841363 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6af40b7e10 con 0x561c4d2f1440 2026-02-25T11:49:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6aca7fc640 1 -- 10.20.193.28:0/1696841363 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6af40d22e0 con 0x561c4d2f1440 2026-02-25T11:49:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1696841363 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b0c1747e0 con 0x561c4d2f1440 2026-02-25T11:49:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6aca7fc640 1 -- 10.20.193.28:0/1696841363 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6af40ade50 con 0x561c4d2f1440 2026-02-25T11:49:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6b14e12e00 10 client.25556 _ll_register_callbacks cb 0x561c4d17da10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.893+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1696841363 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c4ce926f0 con 0x561c4d2f1440 2026-02-25T11:49:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.894+0000 7f6aca7fc640 1 -- 10.20.193.28:0/1696841363 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6af40b5240 con 0x561c4d2f1440 2026-02-25T11:49:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.894+0000 7f6aca7fc640 1 -- 10.20.193.28:0/1696841363 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6af40b2c60 con 0x561c4d2f1440 2026-02-25T11:49:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.894+0000 7f6b14e12e00 10 client.25556 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.894+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1696841363 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x561c4d1920e0 con 0x561c4d2f1440 2026-02-25T11:49:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.894+0000 7f6aca7fc640 1 -- 10.20.193.28:0/1696841363 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6af40aaf20 con 0x561c4d2f1440 2026-02-25T11:49:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.894+0000 7f6b14e12e00 10 client.25556 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.894+0000 7f6b14e12e00 10 client.25556 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:49:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.894+0000 7f6b14e12e00 20 client.25556 populate_metadata read hostname 'trial028' 2026-02-25T11:49:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.894+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/1696841363 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x561c4d3337f0 0x561c4d333c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.894+0000 7f6b14e12e00 4 client.25556 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:49:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.894+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1696841363 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c4d1cefc0 con 0x561c4d3337f0 2026-02-25T11:49:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.895+0000 7f6b10b8c640 1 --2- 10.20.193.28:0/1696841363 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x561c4d3337f0 0x561c4d333c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.895+0000 7f6b10b8c640 1 --2- 10.20.193.28:0/1696841363 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x561c4d3337f0 0x561c4d333c90 crc :-1 s=READY pgs=28 gs=4 cs=0 l=0 c_cookie=6934155234923138445 s_cookie=12061375376867924706 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=603b11728b024d8d server_cookie=a7629d0058db1ae2 in_seq=0 out_seq=0 2026-02-25T11:49:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.895+0000 7f6aca7fc640 10 client.25556 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:49:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6aca7fc640 1 -- 10.20.193.28:0/1696841363 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x561c4d1cefc0 con 0x561c4d3337f0 2026-02-25T11:49:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6aca7fc640 10 client.25556 handle_command_reply: tid=0 2026-02-25T11:49:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6b14e12e00 1 client.25556 shutdown 2026-02-25T11:49:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1696841363 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 msgr2=0x561c4d309450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/1696841363 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d309450 secure :-1 s=READY pgs=114 gs=2 cs=0 l=1 c_cookie=4019579538560522456 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6af40e4760 tx=0x7f6af40ad0c0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1696841363 shutdown_connections 2026-02-25T11:49:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/1696841363 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x561c4d3337f0 0x561c4d333c90 crc :-1 s=READY pgs=28 gs=4 cs=0 l=0 c_cookie=6934155234923138445 s_cookie=12061375376867924706 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/1696841363 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 0x561c4d30e440 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/1696841363 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d309450 unknown :-1 s=CLOSED pgs=114 gs=2 cs=0 l=1 c_cookie=4019579538560522456 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/1696841363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d308f10 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1696841363 >> 10.20.193.28:0/1696841363 conn(0x561c4d20f7a0 msgr2=0x561c4d2efb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6aca7fc640 0 client.25556 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:49:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1696841363 shutdown_connections 2026-02-25T11:49:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1696841363 wait complete. 2026-02-25T11:49:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.896+0000 7f6b14e12e00 20 client.25556 trim_cache size 0 max 16384 2026-02-25T11:49:03.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.903+0000 7f6b14e12e00 1 Processor -- start 2026-02-25T11:49:03.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.903+0000 7f6b14e12e00 1 -- start start 2026-02-25T11:49:03.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.903+0000 7f6b14e12e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d2efd20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.903+0000 7f6b14e12e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d2f0260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.903+0000 7f6b14e12e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 0x561c4d32e640 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.903+0000 7f6b14e12e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x561c4d1fa1a0 con 0x561c4d2f0b30 2026-02-25T11:49:03.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.903+0000 7f6b14e12e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561c4ceba700 con 0x561c4d2f1440 2026-02-25T11:49:03.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.903+0000 7f6b14e12e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x561c4d2cbe80 con 0x561c4d3360e0 2026-02-25T11:49:03.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6b03fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d2f0260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6b03fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d2f0260 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:49084/0 (socket says 10.20.193.28:49084) 2026-02-25T11:49:03.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6b03fff640 1 -- 10.20.193.28:0/1008017019 learned_addr learned my addr 10.20.193.28:0/1008017019 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:03.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6b1138d640 1 --2- 10.20.193.28:0/1008017019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 0x561c4d32e640 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6b03fff640 1 -- 10.20.193.28:0/1008017019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 msgr2=0x561c4d32e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6b03fff640 1 --2- 10.20.193.28:0/1008017019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 0x561c4d32e640 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6b03fff640 1 -- 10.20.193.28:0/1008017019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 msgr2=0x561c4d2efd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6b10b8c640 1 --2- 10.20.193.28:0/1008017019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d2efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6b03fff640 1 --2- 10.20.193.28:0/1008017019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d2efd20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6b03fff640 1 -- 10.20.193.28:0/1008017019 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c4d18cb60 con 0x561c4d2f1440 2026-02-25T11:49:03.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6b10b8c640 1 --2- 10.20.193.28:0/1008017019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d2efd20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:49:03.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6b03fff640 1 --2- 10.20.193.28:0/1008017019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d2f0260 secure :-1 s=READY pgs=115 gs=1 cs=0 l=1 c_cookie=12040941764026355258 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6af40a94c0 tx=0x7f6af406e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a71a04bd1df4ce3a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:03.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6ad8ce3640 1 -- 10.20.193.28:0/1008017019 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6af40b5240 con 0x561c4d2f1440 2026-02-25T11:49:03.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6ad8ce3640 1 -- 10.20.193.28:0/1008017019 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6af40c8070 con 0x561c4d2f1440 2026-02-25T11:49:03.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.904+0000 7f6ad8ce3640 1 -- 10.20.193.28:0/1008017019 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6af40aaf20 con 0x561c4d2f1440 2026-02-25T11:49:03.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.905+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1008017019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 msgr2=0x561c4d2f0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.905+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/1008017019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d2f0260 secure :-1 s=READY pgs=115 gs=1 cs=0 l=1 c_cookie=12040941764026355258 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6af40a94c0 tx=0x7f6af406e000 comp rx=0 tx=0).stop 2026-02-25T11:49:03.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.905+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1008017019 shutdown_connections 2026-02-25T11:49:03.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.905+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/1008017019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d3360e0 0x561c4d32e640 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.905+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/1008017019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f1440 0x561c4d2f0260 unknown :-1 s=CLOSED pgs=115 gs=1 cs=0 l=1 c_cookie=12040941764026355258 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.905+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/1008017019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d2f0b30 0x561c4d2efd20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.905+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1008017019 >> 10.20.193.28:0/1008017019 conn(0x561c4d20f7a0 msgr2=0x561c4d2f3930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:03.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.905+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1008017019 shutdown_connections 2026-02-25T11:49:03.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.905+0000 7f6b14e12e00 1 -- 10.20.193.28:0/1008017019 wait complete. 2026-02-25T11:49:03.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.905+0000 7f6b14e12e00 1 -- start start 2026-02-25T11:49:03.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.906+0000 7f6b14e12e00 1 Processor -- start 2026-02-25T11:49:03.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.906+0000 7f6b14e12e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f0b30 0x561c4d3a4920 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.906+0000 7f6b14e12e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d2f1440 0x561c4d30f220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.906+0000 7f6b14e12e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d3360e0 0x561c4d30f760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.906+0000 7f6b14e12e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x561c4d331e90 con 0x561c4d3360e0 2026-02-25T11:49:03.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.906+0000 7f6b14e12e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561c4ceba700 con 0x561c4d2f0b30 2026-02-25T11:49:03.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.906+0000 7f6b14e12e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x561c4d30fcf0 con 0x561c4d2f1440 2026-02-25T11:49:03.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.906+0000 7f6b1138d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d3360e0 0x561c4d30f760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.906+0000 7f6b1138d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d3360e0 0x561c4d30f760 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:54978/0 (socket says 10.20.193.28:54978) 2026-02-25T11:49:03.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.906+0000 7f6b1138d640 1 -- 10.20.193.28:0/602319063 learned_addr learned my addr 10.20.193.28:0/602319063 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:03.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.906+0000 7f6b10b8c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f0b30 0x561c4d3a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.906+0000 7f6b03fff640 1 --2- 10.20.193.28:0/602319063 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d2f1440 0x561c4d30f220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6b1138d640 1 -- 10.20.193.28:0/602319063 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d2f1440 msgr2=0x561c4d30f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6b1138d640 1 --2- 10.20.193.28:0/602319063 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d2f1440 0x561c4d30f220 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6b1138d640 1 -- 10.20.193.28:0/602319063 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f0b30 msgr2=0x561c4d3a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:03.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6b1138d640 1 --2- 10.20.193.28:0/602319063 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f0b30 0x561c4d3a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6b1138d640 1 -- 10.20.193.28:0/602319063 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c4d1806f0 con 0x561c4d3360e0 2026-02-25T11:49:03.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6b1138d640 1 --2- 10.20.193.28:0/602319063 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d3360e0 0x561c4d30f760 secure :-1 s=READY pgs=430 gs=1 cs=0 l=1 c_cookie=7385857044234096321 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6b08006db0 tx=0x7f6b08004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667fd5d98b63cec1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:03.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6aca7fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:03.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6aca7fc640 1 -- 10.20.193.28:0/602319063 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6b08018040 con 0x561c4d3360e0 2026-02-25T11:49:03.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6b14e12e00 1 -- 10.20.193.28:0/602319063 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c4d1920e0 con 0x561c4d3360e0 2026-02-25T11:49:03.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6aca7fc640 1 -- 10.20.193.28:0/602319063 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6b08019040 con 0x561c4d3360e0 2026-02-25T11:49:03.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6aca7fc640 1 -- 10.20.193.28:0/602319063 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6b08018040 con 0x561c4d3360e0 2026-02-25T11:49:03.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6b14e12e00 10 client.15714 _ll_register_callbacks cb 0x561c4d17da10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:03.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6b14e12e00 1 -- 10.20.193.28:0/602319063 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6afc04a040 con 0x561c4d3360e0 2026-02-25T11:49:03.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6aca7fc640 1 -- 10.20.193.28:0/602319063 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6b08029440 con 0x561c4d3360e0 2026-02-25T11:49:03.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.907+0000 7f6aca7fc640 1 -- 10.20.193.28:0/602319063 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6b08029620 con 0x561c4d3360e0 2026-02-25T11:49:03.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.908+0000 7f6b14e12e00 10 client.15714 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:03.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.908+0000 7f6b14e12e00 1 -- 10.20.193.28:0/602319063 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x561c4d192340 con 0x561c4d3360e0 2026-02-25T11:49:03.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.908+0000 7f6aca7fc640 1 -- 10.20.193.28:0/602319063 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6b0801fbd0 con 0x561c4d3360e0 2026-02-25T11:49:03.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.908+0000 7f6b14e12e00 10 client.15714 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:03.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.908+0000 7f6b14e12e00 10 client.15714 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:49:03.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.908+0000 7f6b14e12e00 20 client.15714 populate_metadata read hostname 'trial028' 2026-02-25T11:49:03.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.908+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/602319063 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x561c4d3ebdb0 0x561c4d40c210 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:03.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.908+0000 7f6b14e12e00 4 client.15714 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:49:03.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.908+0000 7f6b14e12e00 1 -- 10.20.193.28:0/602319063 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c4ce8ff40 con 0x561c4d3ebdb0 2026-02-25T11:49:03.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.908+0000 7f6b10b8c640 1 --2- 10.20.193.28:0/602319063 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x561c4d3ebdb0 0x561c4d40c210 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:03.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6b10b8c640 1 --2- 10.20.193.28:0/602319063 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x561c4d3ebdb0 0x561c4d40c210 crc :-1 s=READY pgs=29 gs=4 cs=0 l=0 c_cookie=16511827512249369842 s_cookie=2336223802277956001 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e525cba3942544f2 server_cookie=206bef2b9a9709a1 in_seq=0 out_seq=0 2026-02-25T11:49:03.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6aca7fc640 10 client.15714 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:49:03.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6aca7fc640 1 -- 10.20.193.28:0/602319063 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c4ce8ff40 con 0x561c4d3ebdb0 2026-02-25T11:49:03.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6aca7fc640 10 client.15714 handle_command_reply: tid=0 2026-02-25T11:49:03.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6b14e12e00 1 client.15714 shutdown 2026-02-25T11:49:03.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6b14e12e00 1 -- 10.20.193.28:0/602319063 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d3360e0 msgr2=0x561c4d30f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/602319063 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d3360e0 0x561c4d30f760 secure :-1 s=READY pgs=430 gs=1 cs=0 l=1 c_cookie=7385857044234096321 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6b08006db0 tx=0x7f6b08004b60 comp rx=0 tx=0).stop 2026-02-25T11:49:03.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6b14e12e00 1 -- 10.20.193.28:0/602319063 shutdown_connections 2026-02-25T11:49:03.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/602319063 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x561c4d3ebdb0 0x561c4d40c210 crc :-1 s=READY pgs=29 gs=4 cs=0 l=0 c_cookie=16511827512249369842 s_cookie=2336223802277956001 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/602319063 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561c4d3360e0 0x561c4d30f760 unknown :-1 s=CLOSED pgs=430 gs=1 cs=0 l=1 c_cookie=7385857044234096321 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/602319063 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561c4d2f1440 0x561c4d30f220 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6b14e12e00 1 --2- 10.20.193.28:0/602319063 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561c4d2f0b30 0x561c4d3a4920 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6b14e12e00 1 -- 10.20.193.28:0/602319063 >> 10.20.193.28:0/602319063 conn(0x561c4d20f7a0 msgr2=0x561c4d2eed10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:03.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6aca7fc640 0 client.15714 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:49:03.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6b14e12e00 1 -- 10.20.193.28:0/602319063 shutdown_connections 2026-02-25T11:49:03.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6b14e12e00 1 -- 10.20.193.28:0/602319063 wait complete. 2026-02-25T11:49:03.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.909+0000 7f6b14e12e00 20 client.15714 trim_cache size 0 max 16384 2026-02-25T11:49:03.929 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:49:03.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.910+0000 7f6b11b8e640 1 -- 10.20.193.28:0/3673926693 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ae80b5760 msgr2=0x7f6ae80d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.910+0000 7f6b11b8e640 1 --2- 10.20.193.28:0/3673926693 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ae80b5760 0x7f6ae80d5c10 secure :-1 s=READY pgs=271 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6afc0097e0 tx=0x7f6afc03a040 comp rx=0 tx=0).stop 2026-02-25T11:49:03.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.910+0000 7f6b11b8e640 1 -- 10.20.193.28:0/3673926693 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b0c10eb80 msgr2=0x7f6b0c1626b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:03.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.910+0000 7f6b11b8e640 1 --2- 10.20.193.28:0/3673926693 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b0c10eb80 0x7f6b0c1626b0 secure :-1 s=READY pgs=127 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6af4009480 tx=0x7f6af400ee60 comp rx=0 tx=0).stop 2026-02-25T11:49:03.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.910+0000 7f6b11b8e640 1 -- 10.20.193.28:0/3673926693 shutdown_connections 2026-02-25T11:49:03.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.910+0000 7f6b11b8e640 1 --2- 10.20.193.28:0/3673926693 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ae80b5760 0x7f6ae80d5c10 unknown :-1 s=CLOSED pgs=271 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.910+0000 7f6b11b8e640 1 --2- 10.20.193.28:0/3673926693 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b0c153740 0x7f6b0c167a80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.910+0000 7f6b11b8e640 1 --2- 10.20.193.28:0/3673926693 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b0c10eb80 0x7f6b0c1626b0 unknown :-1 s=CLOSED pgs=127 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.910+0000 7f6b11b8e640 1 --2- 10.20.193.28:0/3673926693 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b0c10e190 0x7f6b0c162170 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:03.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.910+0000 7f6b11b8e640 1 -- 10.20.193.28:0/3673926693 >> 10.20.193.28:0/3673926693 conn(0x7f6b0c0e9b40 msgr2=0x7f6b0c109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:03.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.910+0000 7f6b11b8e640 1 -- 10.20.193.28:0/3673926693 shutdown_connections 2026-02-25T11:49:03.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:03.910+0000 7f6b11b8e640 1 -- 10.20.193.28:0/3673926693 wait complete. 2026-02-25T11:49:04.045 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:49:04.227 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:04.270 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:04 trial028 ceph-mon[39224]: pgmap v251: 97 pgs: 97 active+clean; 1.1 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.390+0000 7f9312f92640 1 -- 10.20.193.28:0/2979585679 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f930c1114d0 msgr2=0x7f930c111970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.390+0000 7f9312f92640 1 --2- 10.20.193.28:0/2979585679 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f930c1114d0 0x7f930c111970 secure :-1 s=READY pgs=431 gs=3 cs=0 l=1 c_cookie=6643438907280775245 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f930800aa60 tx=0x7f9308030280 comp rx=0 tx=0).stop 2026-02-25T11:49:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.390+0000 7f9312f92640 1 -- 10.20.193.28:0/2979585679 shutdown_connections 2026-02-25T11:49:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.390+0000 7f9312f92640 1 --2- 10.20.193.28:0/2979585679 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f930c1114d0 0x7f930c111970 unknown :-1 s=CLOSED pgs=431 gs=3 cs=0 l=1 c_cookie=6643438907280775245 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.390+0000 7f9312f92640 1 --2- 10.20.193.28:0/2979585679 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f930c10c8e0 0x7f930c110eb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.390+0000 7f9312f92640 1 --2- 10.20.193.28:0/2979585679 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f930c10bef0 0x7f930c10c2c0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.390+0000 7f9312f92640 1 -- 10.20.193.28:0/2979585679 >> 10.20.193.28:0/2979585679 conn(0x7f930c0e9b40 msgr2=0x7f930c109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.391+0000 7f9312f92640 1 -- 10.20.193.28:0/2979585679 shutdown_connections 2026-02-25T11:49:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.391+0000 7f9312f92640 1 -- 10.20.193.28:0/2979585679 wait complete. 2026-02-25T11:49:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.391+0000 7f9312f92640 1 Processor -- start 2026-02-25T11:49:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f9312f92640 1 -- start start 2026-02-25T11:49:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f9312f92640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f930c10bef0 0x7f930c1228a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f9312f92640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f930c10c8e0 0x7f930c122de0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f9312f92640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f930c1114d0 0x7f930c1281b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f9312f92640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f930c1287d0 con 0x7f930c10c8e0 2026-02-25T11:49:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f9312f92640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f930c128940 con 0x7f930c1114d0 2026-02-25T11:49:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f9312f92640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f930c128b00 con 0x7f930c10bef0 2026-02-25T11:49:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f931178f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f930c10c8e0 0x7f930c122de0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f931178f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f930c10c8e0 0x7f930c122de0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:54994/0 (socket says 10.20.193.28:54994) 2026-02-25T11:49:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f931178f640 1 -- 10.20.193.28:0/1549808640 learned_addr learned my addr 10.20.193.28:0/1549808640 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f9312791640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f930c1114d0 0x7f930c1281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f9311f90640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f930c10bef0 0x7f930c1228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f931178f640 1 -- 10.20.193.28:0/1549808640 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f930c10bef0 msgr2=0x7f930c1228a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f931178f640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f930c10bef0 0x7f930c1228a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f931178f640 1 -- 10.20.193.28:0/1549808640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f930c1114d0 msgr2=0x7f930c1281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f931178f640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f930c1114d0 0x7f930c1281b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f931178f640 1 -- 10.20.193.28:0/1549808640 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930800a650 con 0x7f930c10c8e0 2026-02-25T11:49:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.392+0000 7f9312791640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f930c1114d0 0x7f930c1281b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.393+0000 7f931178f640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f930c10c8e0 0x7f930c122de0 secure :-1 s=READY pgs=432 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9300009480 tx=0x7f930000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.393+0000 7f92faffd640 1 -- 10.20.193.28:0/1549808640 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f930000cd40 con 0x7f930c10c8e0 2026-02-25T11:49:04.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.393+0000 7f9312f92640 1 -- 10.20.193.28:0/1549808640 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930c128de0 con 0x7f930c10c8e0 2026-02-25T11:49:04.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.393+0000 7f92faffd640 1 -- 10.20.193.28:0/1549808640 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9300004d10 con 0x7f930c10c8e0 2026-02-25T11:49:04.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.393+0000 7f92faffd640 1 -- 10.20.193.28:0/1549808640 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f93000106b0 con 0x7f930c10c8e0 2026-02-25T11:49:04.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.393+0000 7f9312f92640 1 -- 10.20.193.28:0/1549808640 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f930c1694e0 con 0x7f930c10c8e0 2026-02-25T11:49:04.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.396+0000 7f92faffd640 1 -- 10.20.193.28:0/1549808640 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f930001f060 con 0x7f930c10c8e0 2026-02-25T11:49:04.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.397+0000 7f92faffd640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f92e80b5760 0x7f92e80d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.397+0000 7f92faffd640 1 -- 10.20.193.28:0/1549808640 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9300014070 con 0x7f930c10c8e0 2026-02-25T11:49:04.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.398+0000 7f9311f90640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f92e80b5760 0x7f92e80d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.398+0000 7f9311f90640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f92e80b5760 0x7f92e80d5c10 secure :-1 s=READY pgs=272 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f92fc004740 tx=0x7f92fc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:04.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.398+0000 7f9316216e00 1 Processor -- start 2026-02-25T11:49:04.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.398+0000 7f9316216e00 1 -- start start 2026-02-25T11:49:04.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9316216e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525418b30 0x560525418f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9316216e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525419440 0x56052545dba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9316216e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x56052547e4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9316216e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56052532e8a0 con 0x560525419440 2026-02-25T11:49:04.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9316216e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56052532a5d0 con 0x560525418b30 2026-02-25T11:49:04.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9316216e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560525324570 con 0x56052545e0e0 2026-02-25T11:49:04.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9311f90640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525418b30 0x560525418f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9311f90640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525418b30 0x560525418f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:49110/0 (socket says 10.20.193.28:49110) 2026-02-25T11:49:04.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9311f90640 1 -- 10.20.193.28:0/1475196422 learned_addr learned my addr 10.20.193.28:0/1475196422 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:04.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f931178f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525419440 0x56052545dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9312791640 1 --2- 10.20.193.28:0/1475196422 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x56052547e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9311f90640 1 -- 10.20.193.28:0/1475196422 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 msgr2=0x56052547e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:04.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9311f90640 1 --2- 10.20.193.28:0/1475196422 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x56052547e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9311f90640 1 -- 10.20.193.28:0/1475196422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525419440 msgr2=0x56052545dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:04.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9311f90640 1 --2- 10.20.193.28:0/1475196422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525419440 0x56052545dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9311f90640 1 -- 10.20.193.28:0/1475196422 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605252cfd10 con 0x560525418b30 2026-02-25T11:49:04.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f931178f640 1 --2- 10.20.193.28:0/1475196422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525419440 0x56052545dba0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:04.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.399+0000 7f9311f90640 1 --2- 10.20.193.28:0/1475196422 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525418b30 0x560525418f00 secure :-1 s=READY pgs=116 gs=2 cs=0 l=1 c_cookie=14749563675716858779 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f92fc009c20 tx=0x7f92fc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccb0fa043b1bb39b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:04.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.400+0000 7f92da4e6640 1 -- 10.20.193.28:0/1475196422 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f92fc01d460 con 0x560525418b30 2026-02-25T11:49:04.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.400+0000 7f92da4e6640 1 -- 10.20.193.28:0/1475196422 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f92fc01da80 con 0x560525418b30 2026-02-25T11:49:04.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.400+0000 7f92da4e6640 1 -- 10.20.193.28:0/1475196422 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f92fc01ab30 con 0x560525418b30 2026-02-25T11:49:04.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.400+0000 7f9316216e00 1 -- 10.20.193.28:0/1475196422 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525418b30 msgr2=0x560525418f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:04.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.400+0000 7f9316216e00 1 --2- 10.20.193.28:0/1475196422 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525418b30 0x560525418f00 secure :-1 s=READY pgs=116 gs=2 cs=0 l=1 c_cookie=14749563675716858779 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f92fc009c20 tx=0x7f92fc013000 comp rx=0 tx=0).stop 2026-02-25T11:49:04.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.400+0000 7f9316216e00 1 -- 10.20.193.28:0/1475196422 shutdown_connections 2026-02-25T11:49:04.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.400+0000 7f9316216e00 1 --2- 10.20.193.28:0/1475196422 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x56052547e4c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.400+0000 7f9316216e00 1 --2- 10.20.193.28:0/1475196422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525419440 0x56052545dba0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.400+0000 7f9316216e00 1 --2- 10.20.193.28:0/1475196422 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525418b30 0x560525418f00 unknown :-1 s=CLOSED pgs=116 gs=2 cs=0 l=1 c_cookie=14749563675716858779 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.400+0000 7f9316216e00 1 -- 10.20.193.28:0/1475196422 >> 10.20.193.28:0/1475196422 conn(0x5605253377a0 msgr2=0x560525417410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:04.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.400+0000 7f9316216e00 1 -- 10.20.193.28:0/1475196422 shutdown_connections 2026-02-25T11:49:04.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.400+0000 7f9316216e00 1 -- 10.20.193.28:0/1475196422 wait complete. 2026-02-25T11:49:04.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.401+0000 7f9316216e00 1 -- start start 2026-02-25T11:49:04.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9316216e00 1 Processor -- start 2026-02-25T11:49:04.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9316216e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525418b30 0x560525430f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9316216e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525419440 0x560525431460 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9316216e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x560525436450 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9316216e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x560525436990 con 0x560525418b30 2026-02-25T11:49:04.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9316216e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560525436b00 con 0x560525419440 2026-02-25T11:49:04.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9316216e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560525436cc0 con 0x56052545e0e0 2026-02-25T11:49:04.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f931178f640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525419440 0x560525431460 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9312791640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x560525436450 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9312791640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x560525436450 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:57540/0 (socket says 10.20.193.28:57540) 2026-02-25T11:49:04.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9312791640 1 -- 10.20.193.28:0/408487857 learned_addr learned my addr 10.20.193.28:0/408487857 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:04.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f931178f640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525419440 0x560525431460 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:49112/0 (socket says 10.20.193.28:49112) 2026-02-25T11:49:04.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9312791640 1 -- 10.20.193.28:0/408487857 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525419440 msgr2=0x560525431460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:04.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9312791640 1 --2- 10.20.193.28:0/408487857 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525419440 0x560525431460 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9312791640 1 -- 10.20.193.28:0/408487857 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525418b30 msgr2=0x560525430f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:04.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.402+0000 7f9311f90640 1 --2- 10.20.193.28:0/408487857 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525418b30 0x560525430f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.403+0000 7f9312791640 1 --2- 10.20.193.28:0/408487857 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525418b30 0x560525430f20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.403+0000 7f9312791640 1 -- 10.20.193.28:0/408487857 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605252a83c0 con 0x56052545e0e0 2026-02-25T11:49:04.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.403+0000 7f9311f90640 1 --2- 10.20.193.28:0/408487857 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525418b30 0x560525430f20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:49:04.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.403+0000 7f9312791640 1 --2- 10.20.193.28:0/408487857 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x560525436450 secure :-1 s=READY pgs=128 gs=2 cs=0 l=1 c_cookie=13187899903079294610 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f930800aa30 tx=0x7f93080396a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b704d516e0732692 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:04.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.403+0000 7f92c77fe640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:49:04.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.403+0000 7f92c77fe640 1 -- 10.20.193.28:0/408487857 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f930803e040 con 0x56052545e0e0 2026-02-25T11:49:04.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.403+0000 7f9316216e00 1 -- 10.20.193.28:0/408487857 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92fc0098c0 con 0x56052545e0e0 2026-02-25T11:49:04.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.403+0000 7f9316216e00 10 client.25549 _ll_register_callbacks cb 0x5605252a5a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:04.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.403+0000 7f9316216e00 1 -- 10.20.193.28:0/408487857 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560524fba6f0 con 0x56052545e0e0 2026-02-25T11:49:04.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.403+0000 7f92c77fe640 1 -- 10.20.193.28:0/408487857 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9308039dd0 con 0x56052545e0e0 2026-02-25T11:49:04.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.403+0000 7f92c77fe640 1 -- 10.20.193.28:0/408487857 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9308044470 con 0x56052545e0e0 2026-02-25T11:49:04.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.403+0000 7f92c77fe640 1 -- 10.20.193.28:0/408487857 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9308044600 con 0x56052545e0e0 2026-02-25T11:49:04.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.404+0000 7f92c77fe640 1 -- 10.20.193.28:0/408487857 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9308044820 con 0x56052545e0e0 2026-02-25T11:49:04.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.404+0000 7f9316216e00 10 client.25549 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:04.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.404+0000 7f9316216e00 1 -- 10.20.193.28:0/408487857 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5605252ba0e0 con 0x56052545e0e0 2026-02-25T11:49:04.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.404+0000 7f92c77fe640 1 -- 10.20.193.28:0/408487857 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9308054020 con 0x56052545e0e0 2026-02-25T11:49:04.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.404+0000 7f9316216e00 10 client.25549 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:04.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.404+0000 7f9316216e00 10 client.25549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:04.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.404+0000 7f9316216e00 20 client.25549 populate_metadata read hostname 'trial028' 2026-02-25T11:49:04.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.404+0000 7f9316216e00 1 --2- 10.20.193.28:0/408487857 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56052545b890 0x56052545bd30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.404+0000 7f9316216e00 4 client.25549 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:49:04.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.404+0000 7f9316216e00 1 -- 10.20.193.28:0/408487857 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605252f6fc0 con 0x56052545b890 2026-02-25T11:49:04.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.404+0000 7f9311f90640 1 --2- 10.20.193.28:0/408487857 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56052545b890 0x56052545bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.405+0000 7f9311f90640 1 --2- 10.20.193.28:0/408487857 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56052545b890 0x56052545bd30 crc :-1 s=READY pgs=44 gs=4 cs=0 l=0 c_cookie=7017150936928068956 s_cookie=202240584588620751 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6161ed9942e5a55c server_cookie=2ce80bd7b9273cf in_seq=0 out_seq=0 2026-02-25T11:49:04.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.405+0000 7f92c77fe640 10 client.25549 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:04.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.405+0000 7f92c77fe640 1 -- 10.20.193.28:0/408487857 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5605252f6fc0 con 0x56052545b890 2026-02-25T11:49:04.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.405+0000 7f92c77fe640 10 client.25549 handle_command_reply: tid=0 2026-02-25T11:49:04.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.406+0000 7f9316216e00 1 client.25549 shutdown 2026-02-25T11:49:04.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.406+0000 7f9316216e00 1 -- 10.20.193.28:0/408487857 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 msgr2=0x560525436450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:04.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.406+0000 7f9316216e00 1 --2- 10.20.193.28:0/408487857 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x560525436450 secure :-1 s=READY pgs=128 gs=2 cs=0 l=1 c_cookie=13187899903079294610 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f930800aa30 tx=0x7f93080396a0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.406+0000 7f9316216e00 1 -- 10.20.193.28:0/408487857 shutdown_connections 2026-02-25T11:49:04.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.406+0000 7f9316216e00 1 --2- 10.20.193.28:0/408487857 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56052545b890 0x56052545bd30 crc :-1 s=READY pgs=44 gs=4 cs=0 l=0 c_cookie=7017150936928068956 s_cookie=202240584588620751 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.406+0000 7f9316216e00 1 --2- 10.20.193.28:0/408487857 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x560525436450 unknown :-1 s=CLOSED pgs=128 gs=2 cs=0 l=1 c_cookie=13187899903079294610 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.406+0000 7f9316216e00 1 --2- 10.20.193.28:0/408487857 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525419440 0x560525431460 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.406+0000 7f9316216e00 1 --2- 10.20.193.28:0/408487857 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525418b30 0x560525430f20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.406+0000 7f9316216e00 1 -- 10.20.193.28:0/408487857 >> 10.20.193.28:0/408487857 conn(0x5605253377a0 msgr2=0x560525417b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:04.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.406+0000 7f92c77fe640 0 client.25549 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:04.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.406+0000 7f9316216e00 1 -- 10.20.193.28:0/408487857 shutdown_connections 2026-02-25T11:49:04.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.406+0000 7f9316216e00 1 -- 10.20.193.28:0/408487857 wait complete. 2026-02-25T11:49:04.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.406+0000 7f9316216e00 20 client.25549 trim_cache size 0 max 16384 2026-02-25T11:49:04.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.413+0000 7f9316216e00 1 Processor -- start 2026-02-25T11:49:04.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.413+0000 7f9316216e00 1 -- start start 2026-02-25T11:49:04.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.413+0000 7f9316216e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525418b30 0x560525431340 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.413+0000 7f9316216e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525419440 0x560525417d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.413+0000 7f9316216e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x560525418260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.413+0000 7f9316216e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5605253221a0 con 0x560525419440 2026-02-25T11:49:04.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.413+0000 7f9316216e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560524fe2700 con 0x560525418b30 2026-02-25T11:49:04.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.413+0000 7f9316216e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5605253eda80 con 0x56052545e0e0 2026-02-25T11:49:04.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.413+0000 7f9312791640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x560525418260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.413+0000 7f9311f90640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525418b30 0x560525431340 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.413+0000 7f9312791640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x560525418260 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:57548/0 (socket says 10.20.193.28:57548) 2026-02-25T11:49:04.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.413+0000 7f9312791640 1 -- 10.20.193.28:0/98007583 learned_addr learned my addr 10.20.193.28:0/98007583 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:04.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.413+0000 7f931178f640 1 --2- 10.20.193.28:0/98007583 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525419440 0x560525417d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.414+0000 7f9312791640 1 -- 10.20.193.28:0/98007583 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525418b30 msgr2=0x560525431340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:04.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.414+0000 7f9312791640 1 --2- 10.20.193.28:0/98007583 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525418b30 0x560525431340 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.414+0000 7f9312791640 1 -- 10.20.193.28:0/98007583 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525419440 msgr2=0x560525417d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:04.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.414+0000 7f9312791640 1 --2- 10.20.193.28:0/98007583 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525419440 0x560525417d20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.414+0000 7f9312791640 1 -- 10.20.193.28:0/98007583 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605252b4b60 con 0x56052545e0e0 2026-02-25T11:49:04.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.414+0000 7f9311f90640 1 --2- 10.20.193.28:0/98007583 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525418b30 0x560525431340 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:04.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.414+0000 7f931178f640 1 --2- 10.20.193.28:0/98007583 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525419440 0x560525417d20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:04.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.414+0000 7f9312791640 1 --2- 10.20.193.28:0/98007583 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x560525418260 secure :-1 s=READY pgs=129 gs=2 cs=0 l=1 c_cookie=17415173333044093552 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f930800a230 tx=0x7f930804f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1af1fe079f2e270 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:04.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.414+0000 7f92d9ce5640 1 -- 10.20.193.28:0/98007583 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f930803e040 con 0x56052545e0e0 2026-02-25T11:49:04.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.414+0000 7f92d9ce5640 1 -- 10.20.193.28:0/98007583 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x560524fba6f0 con 0x56052545e0e0 2026-02-25T11:49:04.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.414+0000 7f92d9ce5640 1 -- 10.20.193.28:0/98007583 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9308005950 con 0x56052545e0e0 2026-02-25T11:49:04.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.414+0000 7f9316216e00 1 -- 10.20.193.28:0/98007583 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 msgr2=0x560525418260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:04.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.414+0000 7f9316216e00 1 --2- 10.20.193.28:0/98007583 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x560525418260 secure :-1 s=READY pgs=129 gs=2 cs=0 l=1 c_cookie=17415173333044093552 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f930800a230 tx=0x7f930804f000 comp rx=0 tx=0).stop 2026-02-25T11:49:04.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 -- 10.20.193.28:0/98007583 shutdown_connections 2026-02-25T11:49:04.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 --2- 10.20.193.28:0/98007583 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56052545e0e0 0x560525418260 unknown :-1 s=CLOSED pgs=129 gs=2 cs=0 l=1 c_cookie=17415173333044093552 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 --2- 10.20.193.28:0/98007583 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525419440 0x560525417d20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 --2- 10.20.193.28:0/98007583 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560525418b30 0x560525431340 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 -- 10.20.193.28:0/98007583 >> 10.20.193.28:0/98007583 conn(0x5605253377a0 msgr2=0x56052541b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:04.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 -- 10.20.193.28:0/98007583 shutdown_connections 2026-02-25T11:49:04.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 -- 10.20.193.28:0/98007583 wait complete. 2026-02-25T11:49:04.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 -- start start 2026-02-25T11:49:04.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 Processor -- start 2026-02-25T11:49:04.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525418b30 0x56052543cd80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560525419440 0x56052543d300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56052545e0e0 0x56052543ba00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56052543c100 con 0x560525418b30 2026-02-25T11:49:04.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56052543c270 con 0x56052545e0e0 2026-02-25T11:49:04.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.415+0000 7f9316216e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56052543c430 con 0x560525419440 2026-02-25T11:49:04.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f9311f90640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525418b30 0x56052543cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f931178f640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560525419440 0x56052543d300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f9311f90640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525418b30 0x56052543cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:55036/0 (socket says 10.20.193.28:55036) 2026-02-25T11:49:04.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f9311f90640 1 -- 10.20.193.28:0/1234086147 learned_addr learned my addr 10.20.193.28:0/1234086147 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:04.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f9312791640 1 --2- 10.20.193.28:0/1234086147 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56052545e0e0 0x56052543ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f9311f90640 1 -- 10.20.193.28:0/1234086147 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560525419440 msgr2=0x56052543d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:04.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f9311f90640 1 --2- 10.20.193.28:0/1234086147 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560525419440 0x56052543d300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f9311f90640 1 -- 10.20.193.28:0/1234086147 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56052545e0e0 msgr2=0x56052543ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:04.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f9311f90640 1 --2- 10.20.193.28:0/1234086147 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56052545e0e0 0x56052543ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f9311f90640 1 -- 10.20.193.28:0/1234086147 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930c1694e0 con 0x560525418b30 2026-02-25T11:49:04.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f9311f90640 1 --2- 10.20.193.28:0/1234086147 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525418b30 0x56052543cd80 secure :-1 s=READY pgs=433 gs=1 cs=0 l=1 c_cookie=9296005437135864841 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f92fc01c010 tx=0x7f92fc0041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81020bb156954409 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:04.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f92c77fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:04.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f92c77fe640 1 -- 10.20.193.28:0/1234086147 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f92fc02e040 con 0x560525418b30 2026-02-25T11:49:04.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f92c77fe640 1 -- 10.20.193.28:0/1234086147 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f92fc02f040 con 0x560525418b30 2026-02-25T11:49:04.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f9316216e00 1 -- 10.20.193.28:0/1234086147 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605252cfd10 con 0x560525418b30 2026-02-25T11:49:04.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.416+0000 7f92c77fe640 1 -- 10.20.193.28:0/1234086147 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f92fc034320 con 0x560525418b30 2026-02-25T11:49:04.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f92c77fe640 1 -- 10.20.193.28:0/1234086147 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f92fc034730 con 0x560525418b30 2026-02-25T11:49:04.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f92c77fe640 10 client.15729 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:49:04.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f9316216e00 10 client.15729 _ll_register_callbacks cb 0x5605252a5a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:04.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f9316216e00 1 -- 10.20.193.28:0/1234086147 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92fc01da80 con 0x560525418b30 2026-02-25T11:49:04.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f92c77fe640 1 -- 10.20.193.28:0/1234086147 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f92fc01a6f0 con 0x560525418b30 2026-02-25T11:49:04.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f9316216e00 10 client.15729 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:04.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f9316216e00 1 -- 10.20.193.28:0/1234086147 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5605252ba0e0 con 0x560525418b30 2026-02-25T11:49:04.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f92c77fe640 1 -- 10.20.193.28:0/1234086147 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f92fc034040 con 0x560525418b30 2026-02-25T11:49:04.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f9316216e00 10 client.15729 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:04.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f9316216e00 10 client.15729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:04.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f9316216e00 20 client.15729 populate_metadata read hostname 'trial028' 2026-02-25T11:49:04.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f9316216e00 1 --2- 10.20.193.28:0/1234086147 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56052550b1a0 0x56052552b650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:04.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f9316216e00 4 client.15729 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:49:04.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f9316216e00 1 -- 10.20.193.28:0/1234086147 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560524fb7f40 con 0x56052550b1a0 2026-02-25T11:49:04.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.417+0000 7f931178f640 1 --2- 10.20.193.28:0/1234086147 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56052550b1a0 0x56052552b650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:04.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f931178f640 1 --2- 10.20.193.28:0/1234086147 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56052550b1a0 0x56052552b650 crc :-1 s=READY pgs=45 gs=4 cs=0 l=0 c_cookie=15062271775957075020 s_cookie=16824235016113643796 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d107f0607c50444c server_cookie=e97bb09bc9246114 in_seq=0 out_seq=0 2026-02-25T11:49:04.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f92c77fe640 10 client.15729 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:04.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f92c77fe640 1 -- 10.20.193.28:0/1234086147 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560524fb7f40 con 0x56052550b1a0 2026-02-25T11:49:04.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f92c77fe640 10 client.15729 handle_command_reply: tid=0 2026-02-25T11:49:04.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f9316216e00 1 client.15729 shutdown 2026-02-25T11:49:04.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f9316216e00 1 -- 10.20.193.28:0/1234086147 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525418b30 msgr2=0x56052543cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:04.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f9316216e00 1 --2- 10.20.193.28:0/1234086147 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525418b30 0x56052543cd80 secure :-1 s=READY pgs=433 gs=1 cs=0 l=1 c_cookie=9296005437135864841 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f92fc01c010 tx=0x7f92fc0041f0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f9316216e00 1 -- 10.20.193.28:0/1234086147 shutdown_connections 2026-02-25T11:49:04.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f9316216e00 1 --2- 10.20.193.28:0/1234086147 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56052550b1a0 0x56052552b650 crc :-1 s=READY pgs=45 gs=4 cs=0 l=0 c_cookie=15062271775957075020 s_cookie=16824235016113643796 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f9316216e00 1 --2- 10.20.193.28:0/1234086147 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56052545e0e0 0x56052543ba00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f9316216e00 1 --2- 10.20.193.28:0/1234086147 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560525419440 0x56052543d300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f9316216e00 1 --2- 10.20.193.28:0/1234086147 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560525418b30 0x56052543cd80 unknown :-1 s=CLOSED pgs=433 gs=1 cs=0 l=1 c_cookie=9296005437135864841 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f92c77fe640 0 client.15729 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:04.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f9316216e00 1 -- 10.20.193.28:0/1234086147 >> 10.20.193.28:0/1234086147 conn(0x5605253377a0 msgr2=0x560525416d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:04.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.418+0000 7f9316216e00 1 -- 10.20.193.28:0/1234086147 shutdown_connections 2026-02-25T11:49:04.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9316216e00 1 -- 10.20.193.28:0/1234086147 wait complete. 2026-02-25T11:49:04.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9316216e00 20 client.15729 trim_cache size 0 max 16384 2026-02-25T11:49:04.428 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:49:04.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9312f92640 1 -- 10.20.193.28:0/1549808640 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f92e80b5760 msgr2=0x7f92e80d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:04.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9312f92640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f92e80b5760 0x7f92e80d5c10 secure :-1 s=READY pgs=272 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f92fc004740 tx=0x7f92fc005f90 comp rx=0 tx=0).stop 2026-02-25T11:49:04.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9312f92640 1 -- 10.20.193.28:0/1549808640 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f930c10c8e0 msgr2=0x7f930c122de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:04.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9312f92640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f930c10c8e0 0x7f930c122de0 secure :-1 s=READY pgs=432 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9300009480 tx=0x7f930000ee60 comp rx=0 tx=0).stop 2026-02-25T11:49:04.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9312f92640 1 -- 10.20.193.28:0/1549808640 shutdown_connections 2026-02-25T11:49:04.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9312f92640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f92e80b5760 0x7f92e80d5c10 unknown :-1 s=CLOSED pgs=272 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9312f92640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f930c1114d0 0x7f930c1281b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9312f92640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f930c10c8e0 0x7f930c122de0 unknown :-1 s=CLOSED pgs=432 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9312f92640 1 --2- 10.20.193.28:0/1549808640 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f930c10bef0 0x7f930c1228a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:04.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9312f92640 1 -- 10.20.193.28:0/1549808640 >> 10.20.193.28:0/1549808640 conn(0x7f930c0e9b40 msgr2=0x7f930c109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:04.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9312f92640 1 -- 10.20.193.28:0/1549808640 shutdown_connections 2026-02-25T11:49:04.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:04.419+0000 7f9312f92640 1 -- 10.20.193.28:0/1549808640 wait complete. 2026-02-25T11:49:04.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:04 trial167 ceph-mon[43392]: pgmap v251: 97 pgs: 97 active+clean; 1.1 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:04.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:04 trial119 ceph-mon[42710]: pgmap v251: 97 pgs: 97 active+clean; 1.1 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:05.553 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:49:05.553 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:49:05.742 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:05.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.920+0000 7ff0b3d49640 1 -- 10.20.193.28:0/1583786478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff0ac110230 msgr2=0x7ff0ac115280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:05.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.920+0000 7ff0b3d49640 1 --2- 10.20.193.28:0/1583786478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff0ac110230 0x7ff0ac115280 secure :-1 s=READY pgs=434 gs=2 cs=0 l=1 c_cookie=4276653572931698389 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff0a0009890 tx=0x7ff0a002f120 comp rx=0 tx=0).stop 2026-02-25T11:49:05.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.920+0000 7ff0b3d49640 1 -- 10.20.193.28:0/1583786478 shutdown_connections 2026-02-25T11:49:05.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.920+0000 7ff0b3d49640 1 --2- 10.20.193.28:0/1583786478 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff0ac115a30 0x7ff0ac176400 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.920+0000 7ff0b3d49640 1 --2- 10.20.193.28:0/1583786478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff0ac110230 0x7ff0ac115280 unknown :-1 s=CLOSED pgs=434 gs=2 cs=0 l=1 c_cookie=4276653572931698389 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.920+0000 7ff0b3d49640 1 --2- 10.20.193.28:0/1583786478 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff0ac10f840 0x7ff0ac10fc10 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.920+0000 7ff0b3d49640 1 -- 10.20.193.28:0/1583786478 >> 10.20.193.28:0/1583786478 conn(0x7ff0ac0ed130 msgr2=0x7ff0ac10d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:05.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.921+0000 7ff0b3d49640 1 -- 10.20.193.28:0/1583786478 shutdown_connections 2026-02-25T11:49:05.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.921+0000 7ff0b3d49640 1 -- 10.20.193.28:0/1583786478 wait complete. 2026-02-25T11:49:05.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.921+0000 7ff0b3d49640 1 Processor -- start 2026-02-25T11:49:05.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.921+0000 7ff0b3d49640 1 -- start start 2026-02-25T11:49:05.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b3d49640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff0ac10f840 0x7ff0ac0f5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b3d49640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff0ac110230 0x7ff0ac176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b3d49640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff0ac115a30 0x7ff0ac0f6310 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b3d49640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac0fb520 con 0x7ff0ac115a30 2026-02-25T11:49:05.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b3d49640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac0fb690 con 0x7ff0ac110230 2026-02-25T11:49:05.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b3d49640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac0fb850 con 0x7ff0ac10f840 2026-02-25T11:49:05.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b22bf640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff0ac115a30 0x7ff0ac0f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b1abe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff0ac10f840 0x7ff0ac0f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b1abe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff0ac10f840 0x7ff0ac0f5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44238/0 (socket says 10.20.193.28:44238) 2026-02-25T11:49:05.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b1abe640 1 -- 10.20.193.28:0/1086008035 learned_addr learned my addr 10.20.193.28:0/1086008035 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:05.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b12bd640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff0ac110230 0x7ff0ac176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b1abe640 1 -- 10.20.193.28:0/1086008035 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff0ac110230 msgr2=0x7ff0ac176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:05.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b1abe640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff0ac110230 0x7ff0ac176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b1abe640 1 -- 10.20.193.28:0/1086008035 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff0ac115a30 msgr2=0x7ff0ac0f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:05.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b1abe640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff0ac115a30 0x7ff0ac0f6310 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b1abe640 1 -- 10.20.193.28:0/1086008035 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a00094b0 con 0x7ff0ac10f840 2026-02-25T11:49:05.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.922+0000 7ff0b22bf640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff0ac115a30 0x7ff0ac0f6310 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:05.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.923+0000 7ff0b1abe640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff0ac10f840 0x7ff0ac0f5c10 secure :-1 s=READY pgs=130 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff09c005940 tx=0x7ff09c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:05.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.923+0000 7ff09affd640 1 -- 10.20.193.28:0/1086008035 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff09c00cd10 con 0x7ff0ac10f840 2026-02-25T11:49:05.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.923+0000 7ff09affd640 1 -- 10.20.193.28:0/1086008035 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff09c00ce70 con 0x7ff0ac10f840 2026-02-25T11:49:05.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.923+0000 7ff09affd640 1 -- 10.20.193.28:0/1086008035 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff09c005280 con 0x7ff0ac10f840 2026-02-25T11:49:05.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.923+0000 7ff0b3d49640 1 -- 10.20.193.28:0/1086008035 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ac0fbb30 con 0x7ff0ac10f840 2026-02-25T11:49:05.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.923+0000 7ff0b3d49640 1 -- 10.20.193.28:0/1086008035 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ac0fbfb0 con 0x7ff0ac10f840 2026-02-25T11:49:05.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.926+0000 7ff09affd640 1 -- 10.20.193.28:0/1086008035 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7ff09c005410 con 0x7ff0ac10f840 2026-02-25T11:49:05.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.927+0000 7ff09affd640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff0880b5750 0x7ff0880d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.928+0000 7ff09affd640 1 -- 10.20.193.28:0/1086008035 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ff09c014070 con 0x7ff0ac10f840 2026-02-25T11:49:05.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.928+0000 7ff0b12bd640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff0880b5750 0x7ff0880d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.928+0000 7ff0b12bd640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff0880b5750 0x7ff0880d5c00 secure :-1 s=READY pgs=273 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff0ac0fc5b0 tx=0x7ff0a0002390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:05.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.928+0000 7ff0b5d44e00 1 Processor -- start 2026-02-25T11:49:05.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.928+0000 7ff0b5d44e00 1 -- start start 2026-02-25T11:49:05.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b5d44e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03c5a0 0x5647cf03c970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b5d44e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf03ceb0 0x5647cf081460 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b5d44e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf0819a0 0x5647cf0a1d80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b5d44e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5647cef52d80 con 0x5647cf0819a0 2026-02-25T11:49:05.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b5d44e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5647cef4eab0 con 0x5647cf03ceb0 2026-02-25T11:49:05.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b5d44e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5647cef48a50 con 0x5647cf03c5a0 2026-02-25T11:49:05.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b22bf640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf0819a0 0x5647cf0a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b22bf640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf0819a0 0x5647cf0a1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41094/0 (socket says 10.20.193.28:41094) 2026-02-25T11:49:05.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b22bf640 1 -- 10.20.193.28:0/1158296881 learned_addr learned my addr 10.20.193.28:0/1158296881 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:05.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b12bd640 1 --2- 10.20.193.28:0/1158296881 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf03ceb0 0x5647cf081460 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b1abe640 1 --2- 10.20.193.28:0/1158296881 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03c5a0 0x5647cf03c970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b22bf640 1 -- 10.20.193.28:0/1158296881 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03c5a0 msgr2=0x5647cf03c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:05.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b22bf640 1 --2- 10.20.193.28:0/1158296881 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03c5a0 0x5647cf03c970 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b22bf640 1 -- 10.20.193.28:0/1158296881 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf03ceb0 msgr2=0x5647cf081460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:05.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b22bf640 1 --2- 10.20.193.28:0/1158296881 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf03ceb0 0x5647cf081460 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b22bf640 1 -- 10.20.193.28:0/1158296881 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647ceef4120 con 0x5647cf0819a0 2026-02-25T11:49:05.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.929+0000 7ff0b22bf640 1 --2- 10.20.193.28:0/1158296881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf0819a0 0x5647cf0a1d80 secure :-1 s=READY pgs=435 gs=1 cs=0 l=1 c_cookie=5465189841922820882 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff0a80051a0 tx=0x7ff0a8005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd8413495e20b12 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:05.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.930+0000 7ff07a4e6640 1 -- 10.20.193.28:0/1158296881 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff0a8011650 con 0x5647cf0819a0 2026-02-25T11:49:05.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.930+0000 7ff07a4e6640 1 -- 10.20.193.28:0/1158296881 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff0a8011c70 con 0x5647cf0819a0 2026-02-25T11:49:05.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.930+0000 7ff07a4e6640 1 -- 10.20.193.28:0/1158296881 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff0a800fe50 con 0x5647cf0819a0 2026-02-25T11:49:05.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.930+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/1158296881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf0819a0 msgr2=0x5647cf0a1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:05.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.930+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/1158296881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf0819a0 0x5647cf0a1d80 secure :-1 s=READY pgs=435 gs=1 cs=0 l=1 c_cookie=5465189841922820882 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff0a80051a0 tx=0x7ff0a8005670 comp rx=0 tx=0).stop 2026-02-25T11:49:05.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.930+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/1158296881 shutdown_connections 2026-02-25T11:49:05.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.930+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/1158296881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf0819a0 0x5647cf0a1d80 unknown :-1 s=CLOSED pgs=435 gs=1 cs=0 l=1 c_cookie=5465189841922820882 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.930+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/1158296881 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf03ceb0 0x5647cf081460 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.930+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/1158296881 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03c5a0 0x5647cf03c970 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.930+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/1158296881 >> 10.20.193.28:0/1158296881 conn(0x5647cf01a8f0 msgr2=0x5647cf03acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:05.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.930+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/1158296881 shutdown_connections 2026-02-25T11:49:05.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.930+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/1158296881 wait complete. 2026-02-25T11:49:05.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.932+0000 7ff0b5d44e00 1 -- start start 2026-02-25T11:49:05.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.932+0000 7ff0b5d44e00 1 Processor -- start 2026-02-25T11:49:05.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.932+0000 7ff0b5d44e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf03c5a0 0x5647cf099310 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.932+0000 7ff0b5d44e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 0x5647cf099850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.932+0000 7ff0b5d44e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf0819a0 0x5647cf09e840 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.932+0000 7ff0b5d44e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5647cf09ed80 con 0x5647cf03c5a0 2026-02-25T11:49:05.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.932+0000 7ff0b5d44e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5647cf09eef0 con 0x5647cf0819a0 2026-02-25T11:49:05.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.932+0000 7ff0b5d44e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5647cf09f0b0 con 0x5647cf03ceb0 2026-02-25T11:49:05.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.932+0000 7ff0b1abe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf03c5a0 0x5647cf099310 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.933+0000 7ff0b12bd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 0x5647cf099850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.933+0000 7ff0b22bf640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf0819a0 0x5647cf09e840 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.933+0000 7ff0b22bf640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf0819a0 0x5647cf09e840 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:55956/0 (socket says 10.20.193.28:55956) 2026-02-25T11:49:05.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.933+0000 7ff0b22bf640 1 -- 10.20.193.28:0/2978629733 learned_addr learned my addr 10.20.193.28:0/2978629733 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:05.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.933+0000 7ff0b1abe640 1 -- 10.20.193.28:0/2978629733 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 msgr2=0x5647cf099850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:05.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.933+0000 7ff0b1abe640 1 --2- 10.20.193.28:0/2978629733 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 0x5647cf099850 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.933+0000 7ff0b1abe640 1 -- 10.20.193.28:0/2978629733 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf0819a0 msgr2=0x5647cf09e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:05.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.933+0000 7ff0b1abe640 1 --2- 10.20.193.28:0/2978629733 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf0819a0 0x5647cf09e840 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.933+0000 7ff0b1abe640 1 -- 10.20.193.28:0/2978629733 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647ceecc810 con 0x5647cf03c5a0 2026-02-25T11:49:05.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.933+0000 7ff0b22bf640 1 --2- 10.20.193.28:0/2978629733 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf0819a0 0x5647cf09e840 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:05.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.933+0000 7ff0b1abe640 1 --2- 10.20.193.28:0/2978629733 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf03c5a0 0x5647cf099310 secure :-1 s=READY pgs=436 gs=1 cs=0 l=1 c_cookie=9727816034237202450 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff09c0a9470 tx=0x7ff09c0acf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87002524990ec012 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:05.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.933+0000 7ff0677fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:05.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.934+0000 7ff0677fe640 1 -- 10.20.193.28:0/2978629733 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff09c0acc60 con 0x5647cf03c5a0 2026-02-25T11:49:05.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.934+0000 7ff0677fe640 1 -- 10.20.193.28:0/2978629733 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff09c0ad8a0 con 0x5647cf03c5a0 2026-02-25T11:49:05.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.934+0000 7ff0677fe640 1 -- 10.20.193.28:0/2978629733 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff09c0b7db0 con 0x5647cf03c5a0 2026-02-25T11:49:05.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.934+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/2978629733 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a800adc0 con 0x5647cf03c5a0 2026-02-25T11:49:05.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.934+0000 7ff0677fe640 1 -- 10.20.193.28:0/2978629733 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ff09c0048e0 con 0x5647cf03c5a0 2026-02-25T11:49:05.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.934+0000 7ff0b5d44e00 10 client.15741 _ll_register_callbacks cb 0x5647ceec9e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:05.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.934+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/2978629733 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5647ceca0530 con 0x5647cf03c5a0 2026-02-25T11:49:05.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.934+0000 7ff0677fe640 1 -- 10.20.193.28:0/2978629733 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff09c0b51a0 con 0x5647cf03c5a0 2026-02-25T11:49:05.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.934+0000 7ff0b5d44e00 10 client.15741 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:05.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.934+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/2978629733 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5647ceede4f0 con 0x5647cf03c5a0 2026-02-25T11:49:05.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.934+0000 7ff0677fe640 1 -- 10.20.193.28:0/2978629733 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7ff09c0adb30 con 0x5647cf03c5a0 2026-02-25T11:49:05.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.935+0000 7ff0b5d44e00 10 client.15741 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:05.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.935+0000 7ff0b5d44e00 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:05.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.935+0000 7ff0b5d44e00 20 client.15741 populate_metadata read hostname 'trial028' 2026-02-25T11:49:05.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.935+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/2978629733 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5647cf07eca0 0x5647cf07f140 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.935+0000 7ff0b5d44e00 4 client.15741 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:49:05.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.935+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/2978629733 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647ceef5b50 con 0x5647cf07eca0 2026-02-25T11:49:05.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.935+0000 7ff0b12bd640 1 --2- 10.20.193.28:0/2978629733 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5647cf07eca0 0x5647cf07f140 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.936+0000 7ff0b12bd640 1 --2- 10.20.193.28:0/2978629733 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5647cf07eca0 0x5647cf07f140 crc :-1 s=READY pgs=46 gs=4 cs=0 l=0 c_cookie=4172499498157265042 s_cookie=15566607504956511959 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39e7b22fcd28d092 server_cookie=d807b30ce2dcd6d7 in_seq=0 out_seq=0 2026-02-25T11:49:05.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.936+0000 7ff0677fe640 10 client.15741 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:05.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.936+0000 7ff0677fe640 1 -- 10.20.193.28:0/2978629733 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5647ceef5b50 con 0x5647cf07eca0 2026-02-25T11:49:05.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.936+0000 7ff0677fe640 10 client.15741 handle_command_reply: tid=0 2026-02-25T11:49:05.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.936+0000 7ff0b5d44e00 1 client.15741 shutdown 2026-02-25T11:49:05.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.937+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/2978629733 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf03c5a0 msgr2=0x5647cf099310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:05.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.937+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/2978629733 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf03c5a0 0x5647cf099310 secure :-1 s=READY pgs=436 gs=1 cs=0 l=1 c_cookie=9727816034237202450 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff09c0a9470 tx=0x7ff09c0acf50 comp rx=0 tx=0).stop 2026-02-25T11:49:05.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.937+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/2978629733 shutdown_connections 2026-02-25T11:49:05.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.937+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/2978629733 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5647cf07eca0 0x5647cf07f140 crc :-1 s=READY pgs=46 gs=4 cs=0 l=0 c_cookie=4172499498157265042 s_cookie=15566607504956511959 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.937+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/2978629733 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf0819a0 0x5647cf09e840 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.937+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/2978629733 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 0x5647cf099850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.937+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/2978629733 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf03c5a0 0x5647cf099310 unknown :-1 s=CLOSED pgs=436 gs=1 cs=0 l=1 c_cookie=9727816034237202450 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.937+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/2978629733 >> 10.20.193.28:0/2978629733 conn(0x5647cf01a8f0 msgr2=0x5647cf03acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:05.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.937+0000 7ff0677fe640 0 client.15741 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:05.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.937+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/2978629733 shutdown_connections 2026-02-25T11:49:05.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.937+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/2978629733 wait complete. 2026-02-25T11:49:05.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.937+0000 7ff0b5d44e00 20 client.15741 trim_cache size 0 max 16384 2026-02-25T11:49:05.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.946+0000 7ff0b5d44e00 1 Processor -- start 2026-02-25T11:49:05.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.946+0000 7ff0b5d44e00 1 -- start start 2026-02-25T11:49:05.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.946+0000 7ff0b5d44e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf03c5a0 0x5647cf07c600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.946+0000 7ff0b5d44e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 0x5647cf07cb40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.946+0000 7ff0b5d44e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf0819a0 0x5647cf0955b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.946+0000 7ff0b5d44e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5647cef46680 con 0x5647cf0819a0 2026-02-25T11:49:05.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.946+0000 7ff0b5d44e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5647cec06790 con 0x5647cf03c5a0 2026-02-25T11:49:05.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.946+0000 7ff0b5d44e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5647cf099e30 con 0x5647cf03ceb0 2026-02-25T11:49:05.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.946+0000 7ff0b1abe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf03c5a0 0x5647cf07c600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.946+0000 7ff0b22bf640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf0819a0 0x5647cf0955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.946+0000 7ff0b1abe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf03c5a0 0x5647cf07c600 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:55968/0 (socket says 10.20.193.28:55968) 2026-02-25T11:49:05.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.946+0000 7ff0b1abe640 1 -- 10.20.193.28:0/1126776697 learned_addr learned my addr 10.20.193.28:0/1126776697 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:05.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.946+0000 7ff0b12bd640 1 --2- 10.20.193.28:0/1126776697 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 0x5647cf07cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.947+0000 7ff0b1abe640 1 -- 10.20.193.28:0/1126776697 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 msgr2=0x5647cf07cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:05.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.947+0000 7ff0b1abe640 1 --2- 10.20.193.28:0/1126776697 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 0x5647cf07cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.947+0000 7ff0b1abe640 1 -- 10.20.193.28:0/1126776697 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf0819a0 msgr2=0x5647cf0955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:05.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.947+0000 7ff0b1abe640 1 --2- 10.20.193.28:0/1126776697 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf0819a0 0x5647cf0955b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.947+0000 7ff0b1abe640 1 -- 10.20.193.28:0/1126776697 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647cf08b8a0 con 0x5647cf03c5a0 2026-02-25T11:49:05.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.947+0000 7ff0b22bf640 1 --2- 10.20.193.28:0/1126776697 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf0819a0 0x5647cf0955b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:05.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.947+0000 7ff0b1abe640 1 --2- 10.20.193.28:0/1126776697 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf03c5a0 0x5647cf07c600 secure :-1 s=READY pgs=117 gs=2 cs=0 l=1 c_cookie=17600166413773400022 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff09c0ad110 tx=0x7ff09c004070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4405a1a88b967d6 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:05.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.947+0000 7ff079ce5640 1 -- 10.20.193.28:0/1126776697 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff09c004500 con 0x5647cf03c5a0 2026-02-25T11:49:05.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.947+0000 7ff079ce5640 1 -- 10.20.193.28:0/1126776697 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5647ceca0530 con 0x5647cf03c5a0 2026-02-25T11:49:05.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.947+0000 7ff079ce5640 1 -- 10.20.193.28:0/1126776697 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff09c0b5230 con 0x5647cf03c5a0 2026-02-25T11:49:05.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.948+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/1126776697 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf03c5a0 msgr2=0x5647cf07c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:05.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.948+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/1126776697 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf03c5a0 0x5647cf07c600 secure :-1 s=READY pgs=117 gs=2 cs=0 l=1 c_cookie=17600166413773400022 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff09c0ad110 tx=0x7ff09c004070 comp rx=0 tx=0).stop 2026-02-25T11:49:05.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.948+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/1126776697 shutdown_connections 2026-02-25T11:49:05.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.948+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/1126776697 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf0819a0 0x5647cf0955b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.948+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/1126776697 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 0x5647cf07cb40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.948+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/1126776697 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf03c5a0 0x5647cf07c600 unknown :-1 s=CLOSED pgs=117 gs=2 cs=0 l=1 c_cookie=17600166413773400022 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.948+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/1126776697 >> 10.20.193.28:0/1126776697 conn(0x5647cf01a8f0 msgr2=0x5647cf09de90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:05.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.948+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/1126776697 shutdown_connections 2026-02-25T11:49:05.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.948+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/1126776697 wait complete. 2026-02-25T11:49:05.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.948+0000 7ff0b5d44e00 1 -- start start 2026-02-25T11:49:05.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.948+0000 7ff0b5d44e00 1 Processor -- start 2026-02-25T11:49:05.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b5d44e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf03c5a0 0x5647cf01acc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b5d44e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 0x5647cf01b200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b5d44e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf0819a0 0x5647cf01b740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b5d44e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5647cefb0020 con 0x5647cf03c5a0 2026-02-25T11:49:05.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b5d44e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5647cf0e8810 con 0x5647cf0819a0 2026-02-25T11:49:05.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b5d44e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5647cf0e89d0 con 0x5647cf03ceb0 2026-02-25T11:49:05.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b1abe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf03c5a0 0x5647cf01acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b1abe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf03c5a0 0x5647cf01acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41118/0 (socket says 10.20.193.28:41118) 2026-02-25T11:49:05.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b1abe640 1 -- 10.20.193.28:0/3532413886 learned_addr learned my addr 10.20.193.28:0/3532413886 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:05.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b12bd640 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 0x5647cf01b200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b22bf640 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf0819a0 0x5647cf01b740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b1abe640 1 -- 10.20.193.28:0/3532413886 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 msgr2=0x5647cf01b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:05.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b1abe640 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 0x5647cf01b200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b1abe640 1 -- 10.20.193.28:0/3532413886 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf0819a0 msgr2=0x5647cf01b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:05.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b1abe640 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf0819a0 0x5647cf01b740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b1abe640 1 -- 10.20.193.28:0/3532413886 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647ceede4f0 con 0x5647cf03c5a0 2026-02-25T11:49:05.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b12bd640 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 0x5647cf01b200 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:05.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0b1abe640 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf03c5a0 0x5647cf01acc0 secure :-1 s=READY pgs=437 gs=1 cs=0 l=1 c_cookie=17982989914178248509 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff09c0053e0 tx=0x7ff09c0e4920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9906a0c8a06df3d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:05.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.949+0000 7ff0677fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:05.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.950+0000 7ff0677fe640 1 -- 10.20.193.28:0/3532413886 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff09c004500 con 0x5647cf03c5a0 2026-02-25T11:49:05.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.950+0000 7ff0677fe640 1 -- 10.20.193.28:0/3532413886 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff09c0048e0 con 0x5647cf03c5a0 2026-02-25T11:49:05.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.950+0000 7ff0677fe640 1 -- 10.20.193.28:0/3532413886 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff09c0accc0 con 0x5647cf03c5a0 2026-02-25T11:49:05.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.950+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/3532413886 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647ceef4120 con 0x5647cf03c5a0 2026-02-25T11:49:05.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.950+0000 7ff0b5d44e00 10 client.15747 _ll_register_callbacks cb 0x5647ceec9e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:05.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.950+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/3532413886 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a8011c70 con 0x5647cf03c5a0 2026-02-25T11:49:05.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.950+0000 7ff0677fe640 1 -- 10.20.193.28:0/3532413886 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ff09c0b51a0 con 0x5647cf03c5a0 2026-02-25T11:49:05.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.950+0000 7ff0677fe640 1 -- 10.20.193.28:0/3532413886 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff09c0e4110 con 0x5647cf03c5a0 2026-02-25T11:49:05.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.950+0000 7ff0b5d44e00 10 client.15747 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:05.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.950+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/3532413886 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5647ceed8f70 con 0x5647cf03c5a0 2026-02-25T11:49:05.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.951+0000 7ff0677fe640 1 -- 10.20.193.28:0/3532413886 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7ff09c0aae80 con 0x5647cf03c5a0 2026-02-25T11:49:05.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.951+0000 7ff0b5d44e00 10 client.15747 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:05.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.951+0000 7ff0b5d44e00 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:05.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.951+0000 7ff0b5d44e00 20 client.15747 populate_metadata read hostname 'trial028' 2026-02-25T11:49:05.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.951+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5647cf12ef00 0x5647cf14f340 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:05.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.951+0000 7ff0b5d44e00 4 client.15747 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:49:05.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.951+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/3532413886 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5647ceda9690 con 0x5647cf12ef00 2026-02-25T11:49:05.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.951+0000 7ff0b12bd640 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5647cf12ef00 0x5647cf14f340 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:05.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.952+0000 7ff0b12bd640 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5647cf12ef00 0x5647cf14f340 crc :-1 s=READY pgs=47 gs=4 cs=0 l=0 c_cookie=7919250765074892124 s_cookie=3454444850482245085 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6de6d49ce387855c server_cookie=2ff0a74a012beddd in_seq=0 out_seq=0 2026-02-25T11:49:05.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.952+0000 7ff0677fe640 10 client.15747 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:05.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0677fe640 1 -- 10.20.193.28:0/3532413886 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5647ceda9690 con 0x5647cf12ef00 2026-02-25T11:49:05.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0677fe640 10 client.15747 handle_command_reply: tid=0 2026-02-25T11:49:05.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0b5d44e00 1 client.15747 shutdown 2026-02-25T11:49:05.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/3532413886 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf03c5a0 msgr2=0x5647cf01acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:05.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf03c5a0 0x5647cf01acc0 secure :-1 s=READY pgs=437 gs=1 cs=0 l=1 c_cookie=17982989914178248509 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff09c0053e0 tx=0x7ff09c0e4920 comp rx=0 tx=0).stop 2026-02-25T11:49:05.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/3532413886 shutdown_connections 2026-02-25T11:49:05.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5647cf12ef00 0x5647cf14f340 crc :-1 s=READY pgs=47 gs=4 cs=0 l=0 c_cookie=7919250765074892124 s_cookie=3454444850482245085 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5647cf0819a0 0x5647cf01b740 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5647cf03ceb0 0x5647cf01b200 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0b5d44e00 1 --2- 10.20.193.28:0/3532413886 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5647cf03c5a0 0x5647cf01acc0 unknown :-1 s=CLOSED pgs=437 gs=1 cs=0 l=1 c_cookie=17982989914178248509 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/3532413886 >> 10.20.193.28:0/3532413886 conn(0x5647cf01a8f0 msgr2=0x5647cf09c570 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:05.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0677fe640 0 client.15747 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:05.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/3532413886 shutdown_connections 2026-02-25T11:49:05.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0b5d44e00 1 -- 10.20.193.28:0/3532413886 wait complete. 2026-02-25T11:49:05.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.953+0000 7ff0b5d44e00 20 client.15747 trim_cache size 0 max 16384 2026-02-25T11:49:05.963 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:49:05.963 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:49:05.964 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "0134923a-7ba8-4d64-89bf-665ec8547ef9", 2026-02-25T11:49:05.964 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:49:05.964 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:49:05.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.954+0000 7ff0b3d49640 1 -- 10.20.193.28:0/1086008035 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff0880b5750 msgr2=0x7ff0880d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:05.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.954+0000 7ff0b3d49640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff0880b5750 0x7ff0880d5c00 secure :-1 s=READY pgs=273 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff0ac0fc5b0 tx=0x7ff0a0002390 comp rx=0 tx=0).stop 2026-02-25T11:49:05.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.954+0000 7ff0b3d49640 1 -- 10.20.193.28:0/1086008035 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff0ac10f840 msgr2=0x7ff0ac0f5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:05.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.954+0000 7ff0b3d49640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff0ac10f840 0x7ff0ac0f5c10 secure :-1 s=READY pgs=130 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff09c005940 tx=0x7ff09c00ee60 comp rx=0 tx=0).stop 2026-02-25T11:49:05.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.954+0000 7ff0b3d49640 1 -- 10.20.193.28:0/1086008035 shutdown_connections 2026-02-25T11:49:05.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.954+0000 7ff0b3d49640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff0880b5750 0x7ff0880d5c00 unknown :-1 s=CLOSED pgs=273 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.954+0000 7ff0b3d49640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff0ac115a30 0x7ff0ac0f6310 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.954+0000 7ff0b3d49640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff0ac110230 0x7ff0ac176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.954+0000 7ff0b3d49640 1 --2- 10.20.193.28:0/1086008035 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff0ac10f840 0x7ff0ac0f5c10 unknown :-1 s=CLOSED pgs=130 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:05.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.954+0000 7ff0b3d49640 1 -- 10.20.193.28:0/1086008035 >> 10.20.193.28:0/1086008035 conn(0x7ff0ac0ed130 msgr2=0x7ff0ac112040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:05.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.954+0000 7ff0b3d49640 1 -- 10.20.193.28:0/1086008035 shutdown_connections 2026-02-25T11:49:05.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:05.954+0000 7ff0b3d49640 1 -- 10.20.193.28:0/1086008035 wait complete. 2026-02-25T11:49:06.091 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:49:06.277 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:06.410 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:06 trial028 ceph-mon[39224]: pgmap v252: 97 pgs: 97 active+clean; 1.1 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:06.410 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:06 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:49:06.411 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:06 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:49:06.411 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:06 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:49:06.411 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:06 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:49:06.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.450+0000 7f658a907640 1 -- 10.20.193.28:0/1781437579 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f658410b360 msgr2=0x7f658410b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:06.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.450+0000 7f658a907640 1 --2- 10.20.193.28:0/1781437579 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f658410b360 0x7f658410b730 secure :-1 s=READY pgs=438 gs=1 cs=0 l=1 c_cookie=18091748887059354982 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6574009950 tx=0x7f657402f190 comp rx=0 tx=0).stop 2026-02-25T11:49:06.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.450+0000 7f658a907640 1 -- 10.20.193.28:0/1781437579 shutdown_connections 2026-02-25T11:49:06.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.450+0000 7f658a907640 1 --2- 10.20.193.28:0/1781437579 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6584111370 0x7f6584111d50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.450+0000 7f658a907640 1 --2- 10.20.193.28:0/1781437579 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f658410bd50 0x7f6584110be0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.450+0000 7f658a907640 1 --2- 10.20.193.28:0/1781437579 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f658410b360 0x7f658410b730 unknown :-1 s=CLOSED pgs=438 gs=1 cs=0 l=1 c_cookie=18091748887059354982 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.450+0000 7f658a907640 1 -- 10.20.193.28:0/1781437579 >> 10.20.193.28:0/1781437579 conn(0x7f65840e8e70 msgr2=0x7f6584109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:06.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.450+0000 7f658a907640 1 -- 10.20.193.28:0/1781437579 shutdown_connections 2026-02-25T11:49:06.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.450+0000 7f658a907640 1 -- 10.20.193.28:0/1781437579 wait complete. 2026-02-25T11:49:06.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.451+0000 7f658a907640 1 Processor -- start 2026-02-25T11:49:06.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.451+0000 7f658a907640 1 -- start start 2026-02-25T11:49:06.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.451+0000 7f658a907640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f658410b360 0x7f6584123680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.451+0000 7f658a907640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f658410bd50 0x7f6584123bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.451+0000 7f658a907640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6584111370 0x7f6584128f90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.451+0000 7f658a907640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6584129520 con 0x7f658410bd50 2026-02-25T11:49:06.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.451+0000 7f658a907640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6584129690 con 0x7f6584111370 2026-02-25T11:49:06.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.451+0000 7f658a907640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6584129850 con 0x7f658410b360 2026-02-25T11:49:06.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f6589104640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f658410bd50 0x7f6584123bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f6589104640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f658410bd50 0x7f6584123bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41148/0 (socket says 10.20.193.28:41148) 2026-02-25T11:49:06.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f6589104640 1 -- 10.20.193.28:0/2828242462 learned_addr learned my addr 10.20.193.28:0/2828242462 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:06.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f6589905640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f658410b360 0x7f6584123680 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f658a106640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6584111370 0x7f6584128f90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f6589104640 1 -- 10.20.193.28:0/2828242462 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f658410b360 msgr2=0x7f6584123680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:06.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f6589104640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f658410b360 0x7f6584123680 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f6589104640 1 -- 10.20.193.28:0/2828242462 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6584111370 msgr2=0x7f6584128f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:06.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f6589104640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6584111370 0x7f6584128f90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f6589104640 1 -- 10.20.193.28:0/2828242462 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6574009570 con 0x7f658410bd50 2026-02-25T11:49:06.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f658a106640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6584111370 0x7f6584128f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:06.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f6589104640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f658410bd50 0x7f6584123bc0 secure :-1 s=READY pgs=439 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f657800c940 tx=0x7f657800ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:06.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f6572ffd640 1 -- 10.20.193.28:0/2828242462 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f65780044a0 con 0x7f658410bd50 2026-02-25T11:49:06.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f6572ffd640 1 -- 10.20.193.28:0/2828242462 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6578007d20 con 0x7f658410bd50 2026-02-25T11:49:06.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f6572ffd640 1 -- 10.20.193.28:0/2828242462 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f657800f6d0 con 0x7f658410bd50 2026-02-25T11:49:06.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f658a907640 1 -- 10.20.193.28:0/2828242462 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6584129b30 con 0x7f658410bd50 2026-02-25T11:49:06.471 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.452+0000 7f658a907640 1 -- 10.20.193.28:0/2828242462 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6584129e10 con 0x7f658410bd50 2026-02-25T11:49:06.471 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.455+0000 7f6572ffd640 1 -- 10.20.193.28:0/2828242462 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f6578020070 con 0x7f658410bd50 2026-02-25T11:49:06.471 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.457+0000 7f6572ffd640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65600b5760 0x7f65600d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.471 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.457+0000 7f6572ffd640 1 -- 10.20.193.28:0/2828242462 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f65780e6bd0 con 0x7f658410bd50 2026-02-25T11:49:06.471 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.457+0000 7f6589905640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65600b5760 0x7f65600d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.471 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f658db8be00 1 Processor -- start 2026-02-25T11:49:06.471 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f658db8be00 1 -- start start 2026-02-25T11:49:06.471 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f6589905640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65600b5760 0x7f65600d5c10 secure :-1 s=READY pgs=274 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f65740023d0 tx=0x7f6574031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:06.471 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f658db8be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2140 0x5578c41a2510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.471 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f658db8be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2a50 0x5578c41e71b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.471 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f658db8be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41e76f0 0x5578c4207ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f658db8be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5578c40b89b0 con 0x5578c41a2140 2026-02-25T11:49:06.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f658db8be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5578c40b46e0 con 0x5578c41e76f0 2026-02-25T11:49:06.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f658db8be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5578c40ae840 con 0x5578c41a2a50 2026-02-25T11:49:06.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f6589905640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2140 0x5578c41a2510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f6589905640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2140 0x5578c41a2510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41156/0 (socket says 10.20.193.28:41156) 2026-02-25T11:49:06.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f6589905640 1 -- 10.20.193.28:0/2350009025 learned_addr learned my addr 10.20.193.28:0/2350009025 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:06.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f658a106640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41e76f0 0x5578c4207ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f6589104640 1 --2- 10.20.193.28:0/2350009025 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2a50 0x5578c41e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f6589905640 1 -- 10.20.193.28:0/2350009025 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2a50 msgr2=0x5578c41e71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:06.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f6589905640 1 --2- 10.20.193.28:0/2350009025 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2a50 0x5578c41e71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f6589905640 1 -- 10.20.193.28:0/2350009025 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41e76f0 msgr2=0x5578c4207ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:06.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f6589905640 1 --2- 10.20.193.28:0/2350009025 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41e76f0 0x5578c4207ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f6589905640 1 -- 10.20.193.28:0/2350009025 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578c4059d80 con 0x5578c41a2140 2026-02-25T11:49:06.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.458+0000 7f6589104640 1 --2- 10.20.193.28:0/2350009025 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2a50 0x5578c41e71b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:06.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.459+0000 7f6589905640 1 --2- 10.20.193.28:0/2350009025 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2140 0x5578c41a2510 secure :-1 s=READY pgs=440 gs=1 cs=0 l=1 c_cookie=3764474856594868291 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6574005f70 tx=0x7f6574040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343e19efe13a0843 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:06.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.459+0000 7f65524e6640 1 -- 10.20.193.28:0/2350009025 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6574002ea0 con 0x5578c41a2140 2026-02-25T11:49:06.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.459+0000 7f65524e6640 1 -- 10.20.193.28:0/2350009025 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f657402fc10 con 0x5578c41a2140 2026-02-25T11:49:06.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.459+0000 7f65524e6640 1 -- 10.20.193.28:0/2350009025 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f657404f7f0 con 0x5578c41a2140 2026-02-25T11:49:06.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.459+0000 7f658db8be00 1 -- 10.20.193.28:0/2350009025 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2140 msgr2=0x5578c41a2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:06.473 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.459+0000 7f658db8be00 1 --2- 10.20.193.28:0/2350009025 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2140 0x5578c41a2510 secure :-1 s=READY pgs=440 gs=1 cs=0 l=1 c_cookie=3764474856594868291 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6574005f70 tx=0x7f6574040000 comp rx=0 tx=0).stop 2026-02-25T11:49:06.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.459+0000 7f658db8be00 1 -- 10.20.193.28:0/2350009025 shutdown_connections 2026-02-25T11:49:06.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.459+0000 7f658db8be00 1 --2- 10.20.193.28:0/2350009025 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41e76f0 0x5578c4207ad0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.459+0000 7f658db8be00 1 --2- 10.20.193.28:0/2350009025 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2a50 0x5578c41e71b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.459+0000 7f658db8be00 1 --2- 10.20.193.28:0/2350009025 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2140 0x5578c41a2510 unknown :-1 s=CLOSED pgs=440 gs=1 cs=0 l=1 c_cookie=3764474856594868291 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.459+0000 7f658db8be00 1 -- 10.20.193.28:0/2350009025 >> 10.20.193.28:0/2350009025 conn(0x5578c40c1d40 msgr2=0x5578c41a0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:06.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.460+0000 7f658db8be00 1 -- 10.20.193.28:0/2350009025 shutdown_connections 2026-02-25T11:49:06.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.460+0000 7f658db8be00 1 -- 10.20.193.28:0/2350009025 wait complete. 2026-02-25T11:49:06.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.461+0000 7f658db8be00 1 -- start start 2026-02-25T11:49:06.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.461+0000 7f658db8be00 1 Processor -- start 2026-02-25T11:49:06.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.461+0000 7f658db8be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2140 0x5578c41ff400 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.461+0000 7f658db8be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2a50 0x5578c41ff940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.461+0000 7f658db8be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41e76f0 0x5578c4204930 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.461+0000 7f658db8be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5578c4204e70 con 0x5578c41a2a50 2026-02-25T11:49:06.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.461+0000 7f658db8be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5578c4204fe0 con 0x5578c41e76f0 2026-02-25T11:49:06.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.461+0000 7f658db8be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5578c42051a0 con 0x5578c41a2140 2026-02-25T11:49:06.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.462+0000 7f6589905640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2140 0x5578c41ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.462+0000 7f6589905640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2140 0x5578c41ff400 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44314/0 (socket says 10.20.193.28:44314) 2026-02-25T11:49:06.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.462+0000 7f6589905640 1 -- 10.20.193.28:0/2019295360 learned_addr learned my addr 10.20.193.28:0/2019295360 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:06.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.462+0000 7f658a106640 1 --2- 10.20.193.28:0/2019295360 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41e76f0 0x5578c4204930 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.462+0000 7f6589104640 1 --2- 10.20.193.28:0/2019295360 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2a50 0x5578c41ff940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.462+0000 7f6589905640 1 -- 10.20.193.28:0/2019295360 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41e76f0 msgr2=0x5578c4204930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:06.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.462+0000 7f6589905640 1 --2- 10.20.193.28:0/2019295360 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41e76f0 0x5578c4204930 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.462+0000 7f6589905640 1 -- 10.20.193.28:0/2019295360 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2a50 msgr2=0x5578c41ff940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:06.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.462+0000 7f6589905640 1 --2- 10.20.193.28:0/2019295360 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2a50 0x5578c41ff940 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.462+0000 7f6589905640 1 -- 10.20.193.28:0/2019295360 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578c4032350 con 0x5578c41a2140 2026-02-25T11:49:06.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.462+0000 7f6589905640 1 --2- 10.20.193.28:0/2019295360 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2140 0x5578c41ff400 secure :-1 s=READY pgs=131 gs=1 cs=0 l=1 c_cookie=3869120017477926384 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f65740046e0 tx=0x7f6574005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35b1e0280b1ffdf0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:06.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.462+0000 7f653b7fe640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:49:06.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f653b7fe640 1 -- 10.20.193.28:0/2019295360 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6574002ea0 con 0x5578c41a2140 2026-02-25T11:49:06.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f653b7fe640 1 -- 10.20.193.28:0/2019295360 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f65740388c0 con 0x5578c41a2140 2026-02-25T11:49:06.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f653b7fe640 1 -- 10.20.193.28:0/2019295360 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6574043400 con 0x5578c41a2140 2026-02-25T11:49:06.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f658db8be00 1 -- 10.20.193.28:0/2019295360 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6584112610 con 0x5578c41a2140 2026-02-25T11:49:06.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f653b7fe640 1 -- 10.20.193.28:0/2019295360 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6574038a30 con 0x5578c41a2140 2026-02-25T11:49:06.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f658db8be00 10 client.25579 _ll_register_callbacks cb 0x5578c402f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:06.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f658db8be00 1 -- 10.20.193.28:0/2019295360 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5578c3d446f0 con 0x5578c41a2140 2026-02-25T11:49:06.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f653b7fe640 1 -- 10.20.193.28:0/2019295360 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6574059410 con 0x5578c41a2140 2026-02-25T11:49:06.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f658db8be00 10 client.25579 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:06.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f658db8be00 1 -- 10.20.193.28:0/2019295360 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5578c4044150 con 0x5578c41a2140 2026-02-25T11:49:06.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f653b7fe640 1 -- 10.20.193.28:0/2019295360 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6574062070 con 0x5578c41a2140 2026-02-25T11:49:06.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f658db8be00 10 client.25579 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:06.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f658db8be00 10 client.25579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:06.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f658db8be00 20 client.25579 populate_metadata read hostname 'trial028' 2026-02-25T11:49:06.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f658db8be00 1 --2- 10.20.193.28:0/2019295360 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5578c41e4970 0x5578c41e4e10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f658db8be00 4 client.25579 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:49:06.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.463+0000 7f658db8be00 1 -- 10.20.193.28:0/2019295360 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578c4080910 con 0x5578c41e4970 2026-02-25T11:49:06.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.464+0000 7f6589104640 1 --2- 10.20.193.28:0/2019295360 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5578c41e4970 0x5578c41e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.464+0000 7f6589104640 1 --2- 10.20.193.28:0/2019295360 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5578c41e4970 0x5578c41e4e10 crc :-1 s=READY pgs=48 gs=4 cs=0 l=0 c_cookie=11650458402237742734 s_cookie=3076249707479001490 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1aebe30a7d87e8e server_cookie=2ab108cf6eaf6992 in_seq=0 out_seq=0 2026-02-25T11:49:06.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.464+0000 7f653b7fe640 10 client.25579 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:06.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f653b7fe640 1 -- 10.20.193.28:0/2019295360 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5578c4080910 con 0x5578c41e4970 2026-02-25T11:49:06.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f653b7fe640 10 client.25579 handle_command_reply: tid=0 2026-02-25T11:49:06.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f658db8be00 1 client.25579 shutdown 2026-02-25T11:49:06.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f658db8be00 1 -- 10.20.193.28:0/2019295360 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2140 msgr2=0x5578c41ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:06.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f658db8be00 1 --2- 10.20.193.28:0/2019295360 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2140 0x5578c41ff400 secure :-1 s=READY pgs=131 gs=1 cs=0 l=1 c_cookie=3869120017477926384 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f65740046e0 tx=0x7f6574005ab0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f658db8be00 1 -- 10.20.193.28:0/2019295360 shutdown_connections 2026-02-25T11:49:06.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f658db8be00 1 --2- 10.20.193.28:0/2019295360 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5578c41e4970 0x5578c41e4e10 crc :-1 s=READY pgs=48 gs=4 cs=0 l=0 c_cookie=11650458402237742734 s_cookie=3076249707479001490 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f658db8be00 1 --2- 10.20.193.28:0/2019295360 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41e76f0 0x5578c4204930 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f658db8be00 1 --2- 10.20.193.28:0/2019295360 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2a50 0x5578c41ff940 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f658db8be00 1 --2- 10.20.193.28:0/2019295360 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2140 0x5578c41ff400 unknown :-1 s=CLOSED pgs=131 gs=1 cs=0 l=1 c_cookie=3869120017477926384 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f653b7fe640 0 client.25579 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:06.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f658db8be00 1 -- 10.20.193.28:0/2019295360 >> 10.20.193.28:0/2019295360 conn(0x5578c40c1d40 msgr2=0x5578c41a0230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:06.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f658db8be00 1 -- 10.20.193.28:0/2019295360 shutdown_connections 2026-02-25T11:49:06.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f658db8be00 1 -- 10.20.193.28:0/2019295360 wait complete. 2026-02-25T11:49:06.480 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.465+0000 7f658db8be00 20 client.25579 trim_cache size 0 max 16384 2026-02-25T11:49:06.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.475+0000 7f658db8be00 1 Processor -- start 2026-02-25T11:49:06.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.475+0000 7f658db8be00 1 -- start start 2026-02-25T11:49:06.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.475+0000 7f658db8be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41a2140 0x5578c41ea1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.475+0000 7f658db8be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2a50 0x5578c41fc650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.475+0000 7f658db8be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41e76f0 0x5578c41fcb90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.475+0000 7f658db8be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5578c40ac470 con 0x5578c41a2a50 2026-02-25T11:49:06.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.475+0000 7f658db8be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5578c3d6c700 con 0x5578c41a2140 2026-02-25T11:49:06.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.475+0000 7f658db8be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5578c41a2f30 con 0x5578c41e76f0 2026-02-25T11:49:06.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.475+0000 7f6589104640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2a50 0x5578c41fc650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.476+0000 7f658a106640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41e76f0 0x5578c41fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.476+0000 7f658a106640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41e76f0 0x5578c41fcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44328/0 (socket says 10.20.193.28:44328) 2026-02-25T11:49:06.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.476+0000 7f658a106640 1 -- 10.20.193.28:0/3839866274 learned_addr learned my addr 10.20.193.28:0/3839866274 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:06.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.476+0000 7f6589905640 1 --2- 10.20.193.28:0/3839866274 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41a2140 0x5578c41ea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.476+0000 7f658a106640 1 -- 10.20.193.28:0/3839866274 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41a2140 msgr2=0x5578c41ea1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:06.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.476+0000 7f658a106640 1 --2- 10.20.193.28:0/3839866274 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41a2140 0x5578c41ea1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.476+0000 7f658a106640 1 -- 10.20.193.28:0/3839866274 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2a50 msgr2=0x5578c41fc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:06.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.476+0000 7f658a106640 1 --2- 10.20.193.28:0/3839866274 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2a50 0x5578c41fc650 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.476+0000 7f658a106640 1 -- 10.20.193.28:0/3839866274 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578c41ffcf0 con 0x5578c41e76f0 2026-02-25T11:49:06.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.476+0000 7f658a106640 1 --2- 10.20.193.28:0/3839866274 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41e76f0 0x5578c41fcb90 secure :-1 s=READY pgs=132 gs=3 cs=0 l=1 c_cookie=4812928451247955614 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6580003c30 tx=0x7f6580013820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42caf4ed2a015e9e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:06.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.476+0000 7f6551ce5640 1 -- 10.20.193.28:0/3839866274 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6580013ea0 con 0x5578c41e76f0 2026-02-25T11:49:06.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.477+0000 7f6551ce5640 1 -- 10.20.193.28:0/3839866274 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6580003df0 con 0x5578c41e76f0 2026-02-25T11:49:06.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.477+0000 7f6551ce5640 1 -- 10.20.193.28:0/3839866274 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6580018070 con 0x5578c41e76f0 2026-02-25T11:49:06.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.477+0000 7f658db8be00 1 -- 10.20.193.28:0/3839866274 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41e76f0 msgr2=0x5578c41fcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:06.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.477+0000 7f658db8be00 1 --2- 10.20.193.28:0/3839866274 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41e76f0 0x5578c41fcb90 secure :-1 s=READY pgs=132 gs=3 cs=0 l=1 c_cookie=4812928451247955614 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6580003c30 tx=0x7f6580013820 comp rx=0 tx=0).stop 2026-02-25T11:49:06.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.477+0000 7f658db8be00 1 -- 10.20.193.28:0/3839866274 shutdown_connections 2026-02-25T11:49:06.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.477+0000 7f658db8be00 1 --2- 10.20.193.28:0/3839866274 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41e76f0 0x5578c41fcb90 unknown :-1 s=CLOSED pgs=132 gs=3 cs=0 l=1 c_cookie=4812928451247955614 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.477+0000 7f658db8be00 1 --2- 10.20.193.28:0/3839866274 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41a2a50 0x5578c41fc650 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.477+0000 7f658db8be00 1 --2- 10.20.193.28:0/3839866274 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41a2140 0x5578c41ea1e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.487 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.477+0000 7f658db8be00 1 -- 10.20.193.28:0/3839866274 >> 10.20.193.28:0/3839866274 conn(0x5578c40c1d40 msgr2=0x5578c41e9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:06.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.477+0000 7f658db8be00 1 -- 10.20.193.28:0/3839866274 shutdown_connections 2026-02-25T11:49:06.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.477+0000 7f658db8be00 1 -- 10.20.193.28:0/3839866274 wait complete. 2026-02-25T11:49:06.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.477+0000 7f658db8be00 1 -- start start 2026-02-25T11:49:06.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658db8be00 1 Processor -- start 2026-02-25T11:49:06.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658db8be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41a2140 0x5578c41e2b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658db8be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2a50 0x5578c4180220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658db8be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41e76f0 0x5578c4180760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658db8be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5578c425fab0 con 0x5578c41e76f0 2026-02-25T11:49:06.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658db8be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5578c4180ca0 con 0x5578c41a2140 2026-02-25T11:49:06.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658db8be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5578c4180e60 con 0x5578c41a2a50 2026-02-25T11:49:06.488 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658a106640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41e76f0 0x5578c4180760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658a106640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41e76f0 0x5578c4180760 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:41188/0 (socket says 10.20.193.28:41188) 2026-02-25T11:49:06.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658a106640 1 -- 10.20.193.28:0/1230802831 learned_addr learned my addr 10.20.193.28:0/1230802831 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:06.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f6589104640 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2a50 0x5578c4180220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f6589905640 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41a2140 0x5578c41e2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658a106640 1 -- 10.20.193.28:0/1230802831 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2a50 msgr2=0x5578c4180220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:06.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658a106640 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2a50 0x5578c4180220 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658a106640 1 -- 10.20.193.28:0/1230802831 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41a2140 msgr2=0x5578c41e2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:06.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658a106640 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41a2140 0x5578c41e2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658a106640 1 -- 10.20.193.28:0/1230802831 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578c403ebd0 con 0x5578c41e76f0 2026-02-25T11:49:06.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f6589104640 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2a50 0x5578c4180220 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:06.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f653b7fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:06.489 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.478+0000 7f658a106640 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41e76f0 0x5578c4180760 secure :-1 s=READY pgs=441 gs=1 cs=0 l=1 c_cookie=3701072654516740945 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6580005070 tx=0x7f6580004fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335cd9f85c61e351 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:06.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f653b7fe640 1 -- 10.20.193.28:0/1230802831 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6580018040 con 0x5578c41e76f0 2026-02-25T11:49:06.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f653b7fe640 1 -- 10.20.193.28:0/1230802831 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f658001ccc0 con 0x5578c41e76f0 2026-02-25T11:49:06.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f653b7fe640 1 -- 10.20.193.28:0/1230802831 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6580013ea0 con 0x5578c41e76f0 2026-02-25T11:49:06.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f658db8be00 1 -- 10.20.193.28:0/1230802831 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65800038d0 con 0x5578c41e76f0 2026-02-25T11:49:06.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f653b7fe640 1 -- 10.20.193.28:0/1230802831 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6580011950 con 0x5578c41e76f0 2026-02-25T11:49:06.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f653b7fe640 10 client.15765 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:49:06.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f658db8be00 10 client.15765 _ll_register_callbacks cb 0x5578c402f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:06.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f658db8be00 1 -- 10.20.193.28:0/1230802831 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f657402fc10 con 0x5578c41e76f0 2026-02-25T11:49:06.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f653b7fe640 1 -- 10.20.193.28:0/1230802831 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6580011b10 con 0x5578c41e76f0 2026-02-25T11:49:06.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f658db8be00 10 client.15765 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:06.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f658db8be00 1 -- 10.20.193.28:0/1230802831 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5578c4032680 con 0x5578c41e76f0 2026-02-25T11:49:06.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f653b7fe640 1 -- 10.20.193.28:0/1230802831 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f658001c800 con 0x5578c41e76f0 2026-02-25T11:49:06.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f658db8be00 10 client.15765 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:06.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f658db8be00 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:06.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f658db8be00 20 client.15765 populate_metadata read hostname 'trial028' 2026-02-25T11:49:06.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f658db8be00 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5578c42a6850 0x5578c42c6d00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:06.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f658db8be00 4 client.15765 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:49:06.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.479+0000 7f658db8be00 1 -- 10.20.193.28:0/1230802831 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578c3d41f40 con 0x5578c42a6850 2026-02-25T11:49:06.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.480+0000 7f6589905640 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5578c42a6850 0x5578c42c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:06.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.480+0000 7f6589905640 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5578c42a6850 0x5578c42c6d00 crc :-1 s=READY pgs=49 gs=4 cs=0 l=0 c_cookie=8399726960874867519 s_cookie=7195247002122078732 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7491d32ad137cb3f server_cookie=63daa706c0ae4a0c in_seq=0 out_seq=0 2026-02-25T11:49:06.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.480+0000 7f653b7fe640 10 client.15765 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:06.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.480+0000 7f653b7fe640 1 -- 10.20.193.28:0/1230802831 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x5578c3d41f40 con 0x5578c42a6850 2026-02-25T11:49:06.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.480+0000 7f653b7fe640 10 client.15765 handle_command_reply: tid=0 2026-02-25T11:49:06.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.480+0000 7f658db8be00 1 client.15765 shutdown 2026-02-25T11:49:06.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658db8be00 1 -- 10.20.193.28:0/1230802831 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41e76f0 msgr2=0x5578c4180760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:06.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658db8be00 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41e76f0 0x5578c4180760 secure :-1 s=READY pgs=441 gs=1 cs=0 l=1 c_cookie=3701072654516740945 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6580005070 tx=0x7f6580004fe0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658db8be00 1 -- 10.20.193.28:0/1230802831 shutdown_connections 2026-02-25T11:49:06.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658db8be00 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5578c42a6850 0x5578c42c6d00 crc :-1 s=READY pgs=49 gs=4 cs=0 l=0 c_cookie=8399726960874867519 s_cookie=7195247002122078732 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658db8be00 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5578c41e76f0 0x5578c4180760 unknown :-1 s=CLOSED pgs=441 gs=1 cs=0 l=1 c_cookie=3701072654516740945 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658db8be00 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5578c41a2a50 0x5578c4180220 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658db8be00 1 --2- 10.20.193.28:0/1230802831 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5578c41a2140 0x5578c41e2b40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658db8be00 1 -- 10.20.193.28:0/1230802831 >> 10.20.193.28:0/1230802831 conn(0x5578c40c1d40 msgr2=0x5578c4203790 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:06.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f653b7fe640 0 client.15765 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:06.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658db8be00 1 -- 10.20.193.28:0/1230802831 shutdown_connections 2026-02-25T11:49:06.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658db8be00 1 -- 10.20.193.28:0/1230802831 wait complete. 2026-02-25T11:49:06.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658db8be00 20 client.15765 trim_cache size 0 max 16384 2026-02-25T11:49:06.493 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:49:06.493 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:49:06.493 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:49:06.493 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:49:06.493 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:06.493 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "0134923a-7ba8-4d64-89bf-665ec8547ef9", 2026-02-25T11:49:06.494 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:49:06.494 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:49:06.494 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:06.494 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:49:06.494 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:49:06.494 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:49:06.494 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:06.494 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:06.494 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:06.494 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:06.494 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:06.494 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:06.495 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:06.496 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:06.496 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:06.496 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:49:06.496 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:49:06.496 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:06.496 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:06.496 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:06.496 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:06.496 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:06.496 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:49:06.496 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:49:06.496 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:49:06.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658a907640 1 -- 10.20.193.28:0/2828242462 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65600b5760 msgr2=0x7f65600d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:06.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658a907640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65600b5760 0x7f65600d5c10 secure :-1 s=READY pgs=274 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f65740023d0 tx=0x7f6574031040 comp rx=0 tx=0).stop 2026-02-25T11:49:06.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658a907640 1 -- 10.20.193.28:0/2828242462 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f658410bd50 msgr2=0x7f6584123bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:06.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658a907640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f658410bd50 0x7f6584123bc0 secure :-1 s=READY pgs=439 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f657800c940 tx=0x7f657800ce10 comp rx=0 tx=0).stop 2026-02-25T11:49:06.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658a907640 1 -- 10.20.193.28:0/2828242462 shutdown_connections 2026-02-25T11:49:06.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658a907640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65600b5760 0x7f65600d5c10 unknown :-1 s=CLOSED pgs=274 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658a907640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6584111370 0x7f6584128f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658a907640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f658410bd50 0x7f6584123bc0 unknown :-1 s=CLOSED pgs=439 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658a907640 1 --2- 10.20.193.28:0/2828242462 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f658410b360 0x7f6584123680 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:06.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.481+0000 7f658a907640 1 -- 10.20.193.28:0/2828242462 >> 10.20.193.28:0/2828242462 conn(0x7f65840e8e70 msgr2=0x7f6584109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:06.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.482+0000 7f658a907640 1 -- 10.20.193.28:0/2828242462 shutdown_connections 2026-02-25T11:49:06.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:06.482+0000 7f658a907640 1 -- 10.20.193.28:0/2828242462 wait complete. 2026-02-25T11:49:06.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:06 trial167 ceph-mon[43392]: pgmap v252: 97 pgs: 97 active+clean; 1.1 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:06.506 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:06 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:49:06.506 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:06 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:49:06.506 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:06 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:49:06.506 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:06 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:49:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:06 trial119 ceph-mon[42710]: pgmap v252: 97 pgs: 97 active+clean; 1.1 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:06 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:49:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:06 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:49:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:06 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:49:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:06 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:49:06.583 INFO:tasks.cephfs.filesystem:scrub status for tag:0134923a-7ba8-4d64-89bf-665ec8547ef9 - {'path': '/', 'tag': '0134923a-7ba8-4d64-89bf-665ec8547ef9', 'options': 'recursive,force'} 2026-02-25T11:49:07.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:07 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:49:07.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:07 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:49:07.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:07 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:49:08.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:08 trial167 ceph-mon[43392]: pgmap v253: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:08.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:08 trial119 ceph-mon[42710]: pgmap v253: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:08.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:08 trial028 ceph-mon[39224]: pgmap v253: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:10.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:10 trial167 ceph-mon[43392]: pgmap v254: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:49:10.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:10 trial119 ceph-mon[42710]: pgmap v254: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:49:10.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:10 trial028 ceph-mon[39224]: pgmap v254: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:49:12.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:12 trial167 ceph-mon[43392]: pgmap v255: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:12.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:12 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:49:12.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:12 trial119 ceph-mon[42710]: pgmap v255: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:12.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:12 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:49:12.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:12 trial028 ceph-mon[39224]: pgmap v255: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:12.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:12 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:49:14.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.249 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:13 trial167 ceph-mon[43392]: pgmap v256: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:14.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:49:14.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:49:14.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.250 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:13 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:49:14.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:13 trial119 ceph-mon[42710]: pgmap v256: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:14.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:49:14.259 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:49:14.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.260 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:13 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:49:14.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:13 trial028 ceph-mon[39224]: pgmap v256: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:14.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:49:14.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:49:14.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:49:14.311 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:13 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:49:16.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:16 trial167 ceph-mon[43392]: pgmap v257: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:49:16.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:16 trial119 ceph-mon[42710]: pgmap v257: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:49:16.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:16 trial028 ceph-mon[39224]: pgmap v257: 97 pgs: 97 active+clean; 1.2 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:49:18.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:18 trial167 ceph-mon[43392]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:18.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:18 trial119 ceph-mon[42710]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:18.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:18 trial028 ceph-mon[39224]: pgmap v258: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:20.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:20 trial167 ceph-mon[43392]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:49:20.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:20 trial119 ceph-mon[42710]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:49:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:20 trial028 ceph-mon[39224]: pgmap v259: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:49:22.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:22 trial167 ceph-mon[43392]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:49:22.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:22 trial119 ceph-mon[42710]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:49:22.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:22 trial028 ceph-mon[39224]: pgmap v260: 97 pgs: 97 active+clean; 1.3 MiB data, 332 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:49:24.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:24 trial167 ceph-mon[43392]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:24.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:24 trial119 ceph-mon[42710]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:24.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:24 trial028 ceph-mon[39224]: pgmap v261: 97 pgs: 97 active+clean; 1.3 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:26.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:26 trial167 ceph-mon[43392]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 12 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:49:26.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:26 trial119 ceph-mon[42710]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 12 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:49:26.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:26 trial028 ceph-mon[39224]: pgmap v262: 97 pgs: 97 active+clean; 1.3 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 12 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:49:27.267 INFO:tasks.workunit.client.0.trial028.stdout:1/600: open fd = 3 2026-02-25T11:49:27.267 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.268 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.268 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.268 INFO:tasks.workunit.client.0.trial028.stdout:2/600: open fd = 3 2026-02-25T11:49:27.268 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.268 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.268 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.268 INFO:tasks.workunit.client.0.trial028.stdout:3/600: open fd = 3 2026-02-25T11:49:27.268 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.268 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.268 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.268 INFO:tasks.workunit.client.0.trial028.stdout:4/600: open fd = 3 2026-02-25T11:49:27.269 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.269 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.269 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.269 INFO:tasks.workunit.client.0.trial028.stdout:5/600: open fd = 3 2026-02-25T11:49:27.269 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.269 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.269 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.269 INFO:tasks.workunit.client.0.trial028.stdout:6/600: open fd = 3 2026-02-25T11:49:27.269 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.269 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.269 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.270 INFO:tasks.workunit.client.0.trial028.stdout:7/600: open fd = 3 2026-02-25T11:49:27.270 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.270 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.270 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.270 INFO:tasks.workunit.client.0.trial028.stdout:8/600: open fd = 3 2026-02-25T11:49:27.270 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.270 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.270 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.270 INFO:tasks.workunit.client.0.trial028.stdout:9/600: open fd = 3 2026-02-25T11:49:27.270 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.270 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.271 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.271 INFO:tasks.workunit.client.0.trial028.stdout:10/600: open fd = 3 2026-02-25T11:49:27.271 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.271 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.271 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.271 INFO:tasks.workunit.client.0.trial028.stdout:11/600: open fd = 3 2026-02-25T11:49:27.271 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.271 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.271 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.271 INFO:tasks.workunit.client.0.trial028.stdout:12/600: open fd = 3 2026-02-25T11:49:27.271 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.271 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.272 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.272 INFO:tasks.workunit.client.0.trial028.stdout:13/600: open fd = 3 2026-02-25T11:49:27.272 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.272 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.272 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.272 INFO:tasks.workunit.client.0.trial028.stdout:14/600: open fd = 3 2026-02-25T11:49:27.272 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.272 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.272 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.272 INFO:tasks.workunit.client.0.trial028.stdout:15/600: open fd = 3 2026-02-25T11:49:27.272 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.272 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.272 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.273 INFO:tasks.workunit.client.0.trial028.stdout:16/600: open fd = 3 2026-02-25T11:49:27.273 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.273 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.273 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.273 INFO:tasks.workunit.client.0.trial028.stdout:17/600: open fd = 3 2026-02-25T11:49:27.273 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.273 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.273 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.273 INFO:tasks.workunit.client.0.trial028.stdout:18/600: open fd = 3 2026-02-25T11:49:27.273 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.273 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.273 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.273 INFO:tasks.workunit.client.0.trial028.stdout:19/600: open fd = 3 2026-02-25T11:49:27.274 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.274 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.274 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.274 INFO:tasks.workunit.client.0.trial028.stdout:20/600: open fd = 3 2026-02-25T11:49:27.274 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.274 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.274 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.274 INFO:tasks.workunit.client.0.trial028.stdout:21/600: open fd = 3 2026-02-25T11:49:27.274 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.274 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.274 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.274 INFO:tasks.workunit.client.0.trial028.stdout:22/600: open fd = 3 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:23/600: open fd = 3 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:24/600: open fd = 3 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:25/600: open fd = 3 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.275 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.276 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.276 INFO:tasks.workunit.client.0.trial028.stdout:26/600: open fd = 3 2026-02-25T11:49:27.276 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.276 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.276 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.276 INFO:tasks.workunit.client.0.trial028.stdout:27/600: open fd = 3 2026-02-25T11:49:27.276 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.276 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.276 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.276 INFO:tasks.workunit.client.0.trial028.stdout:28/600: open fd = 3 2026-02-25T11:49:27.276 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.276 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.276 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.277 INFO:tasks.workunit.client.0.trial028.stdout:29/600: open fd = 3 2026-02-25T11:49:27.277 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.277 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.277 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.277 INFO:tasks.workunit.client.0.trial028.stdout:30/600: open fd = 3 2026-02-25T11:49:27.277 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.277 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.277 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.277 INFO:tasks.workunit.client.0.trial028.stdout:31/600: open fd = 3 2026-02-25T11:49:27.277 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.277 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.278 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.278 INFO:tasks.workunit.client.0.trial028.stdout:32/600: open fd = 3 2026-02-25T11:49:27.278 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.278 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.278 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.278 INFO:tasks.workunit.client.0.trial028.stdout:33/600: open fd = 3 2026-02-25T11:49:27.278 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.278 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.278 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.278 INFO:tasks.workunit.client.0.trial028.stdout:34/600: open fd = 3 2026-02-25T11:49:27.278 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.278 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.279 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.279 INFO:tasks.workunit.client.0.trial028.stdout:35/600: open fd = 3 2026-02-25T11:49:27.279 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.279 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.279 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.279 INFO:tasks.workunit.client.0.trial028.stdout:36/600: open fd = 3 2026-02-25T11:49:27.279 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.279 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.279 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.279 INFO:tasks.workunit.client.0.trial028.stdout:37/600: open fd = 3 2026-02-25T11:49:27.279 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.279 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.279 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.280 INFO:tasks.workunit.client.0.trial028.stdout:38/600: open fd = 3 2026-02-25T11:49:27.280 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.280 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.280 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.280 INFO:tasks.workunit.client.0.trial028.stdout:39/600: open fd = 3 2026-02-25T11:49:27.280 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.280 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.280 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.280 INFO:tasks.workunit.client.0.trial028.stdout:40/600: open fd = 3 2026-02-25T11:49:27.280 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.280 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.280 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.281 INFO:tasks.workunit.client.0.trial028.stdout:41/600: open fd = 3 2026-02-25T11:49:27.281 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.281 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.281 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.281 INFO:tasks.workunit.client.0.trial028.stdout:42/600: open fd = 3 2026-02-25T11:49:27.281 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.281 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.281 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.281 INFO:tasks.workunit.client.0.trial028.stdout:43/600: open fd = 3 2026-02-25T11:49:27.281 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.281 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.281 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.281 INFO:tasks.workunit.client.0.trial028.stdout:44/600: open fd = 3 2026-02-25T11:49:27.282 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.282 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.282 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.282 INFO:tasks.workunit.client.0.trial028.stdout:45/600: open fd = 3 2026-02-25T11:49:27.282 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.282 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.282 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.282 INFO:tasks.workunit.client.0.trial028.stdout:46/600: open fd = 3 2026-02-25T11:49:27.282 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.282 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.282 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.282 INFO:tasks.workunit.client.0.trial028.stdout:47/600: open fd = 3 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:48/600: open fd = 3 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:49/600: open fd = 3 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:50/600: open fd = 3 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.283 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.284 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.284 INFO:tasks.workunit.client.0.trial028.stdout:51/600: open fd = 3 2026-02-25T11:49:27.284 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.284 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.284 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.284 INFO:tasks.workunit.client.0.trial028.stdout:52/600: open fd = 3 2026-02-25T11:49:27.284 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.284 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.284 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.284 INFO:tasks.workunit.client.0.trial028.stdout:53/600: open fd = 3 2026-02-25T11:49:27.284 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.284 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.284 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:54/600: open fd = 3 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:55/600: open fd = 3 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:56/600: open fd = 3 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:57/600: open fd = 3 2026-02-25T11:49:27.285 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.286 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.286 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.286 INFO:tasks.workunit.client.0.trial028.stdout:58/600: open fd = 3 2026-02-25T11:49:27.286 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.286 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.286 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.286 INFO:tasks.workunit.client.0.trial028.stdout:59/600: open fd = 3 2026-02-25T11:49:27.286 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.286 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.286 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.286 INFO:tasks.workunit.client.0.trial028.stdout:60/600: open fd = 3 2026-02-25T11:49:27.286 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.287 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.287 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.287 INFO:tasks.workunit.client.0.trial028.stdout:61/600: open fd = 3 2026-02-25T11:49:27.287 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.287 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.287 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.287 INFO:tasks.workunit.client.0.trial028.stdout:62/600: open fd = 3 2026-02-25T11:49:27.287 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.287 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.287 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:27.287 INFO:tasks.workunit.client.0.trial028.stdout:63/600: open fd = 3 2026-02-25T11:49:27.288 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.288 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:49:27.288 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:49:28.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:28 trial167 ceph-mon[43392]: pgmap v263: 97 pgs: 97 active+clean; 1.4 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:28.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:28 trial119 ceph-mon[42710]: pgmap v263: 97 pgs: 97 active+clean; 1.4 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:28.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:28 trial028 ceph-mon[39224]: pgmap v263: 97 pgs: 97 active+clean; 1.4 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:30.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:30 trial167 ceph-mon[43392]: pgmap v264: 97 pgs: 97 active+clean; 1.4 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:49:30.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:30 trial119 ceph-mon[42710]: pgmap v264: 97 pgs: 97 active+clean; 1.4 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:49:30.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:30 trial028 ceph-mon[39224]: pgmap v264: 97 pgs: 97 active+clean; 1.4 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:49:32.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:32 trial167 ceph-mon[43392]: pgmap v265: 97 pgs: 97 active+clean; 1.4 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 12 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:32.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:32 trial119 ceph-mon[42710]: pgmap v265: 97 pgs: 97 active+clean; 1.4 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 12 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:32.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:32 trial028 ceph-mon[39224]: pgmap v265: 97 pgs: 97 active+clean; 1.4 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 12 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:34.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:34 trial167 ceph-mon[43392]: pgmap v266: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:34.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:34 trial119 ceph-mon[42710]: pgmap v266: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:34.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:34 trial028 ceph-mon[39224]: pgmap v266: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:36.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:36 trial167 ceph-mon[43392]: pgmap v267: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:36.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:36 trial119 ceph-mon[42710]: pgmap v267: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:36.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:36 trial028 ceph-mon[39224]: pgmap v267: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:36.584 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:49:36.773 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:36.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.960+0000 7f1cef577640 1 -- 10.20.193.28:0/1151229211 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1cf010ddc0 msgr2=0x7f1cf0112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:36.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.960+0000 7f1cef577640 1 --2- 10.20.193.28:0/1151229211 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1cf010ddc0 0x7f1cf0112e10 secure :-1 s=READY pgs=442 gs=1 cs=0 l=1 c_cookie=18050304396074395644 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cd8009950 tx=0x7f1cd802f190 comp rx=0 tx=0).stop 2026-02-25T11:49:36.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.960+0000 7f1cef577640 1 -- 10.20.193.28:0/1151229211 shutdown_connections 2026-02-25T11:49:36.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.960+0000 7f1cef577640 1 --2- 10.20.193.28:0/1151229211 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1cf01135c0 0x7f1cf0173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.960+0000 7f1cef577640 1 --2- 10.20.193.28:0/1151229211 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1cf010ddc0 0x7f1cf0112e10 unknown :-1 s=CLOSED pgs=442 gs=1 cs=0 l=1 c_cookie=18050304396074395644 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.960+0000 7f1cef577640 1 --2- 10.20.193.28:0/1151229211 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1cf010d3d0 0x7f1cf010d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.960+0000 7f1cef577640 1 -- 10.20.193.28:0/1151229211 >> 10.20.193.28:0/1151229211 conn(0x7f1cf00eaca0 msgr2=0x7f1cf010b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:36.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.960+0000 7f1cef577640 1 -- 10.20.193.28:0/1151229211 shutdown_connections 2026-02-25T11:49:36.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.960+0000 7f1cef577640 1 -- 10.20.193.28:0/1151229211 wait complete. 2026-02-25T11:49:36.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.961+0000 7f1cef577640 1 Processor -- start 2026-02-25T11:49:36.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.961+0000 7f1cef577640 1 -- start start 2026-02-25T11:49:36.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.961+0000 7f1cef577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1cf010d3d0 0x7f1cf00f59b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.961+0000 7f1cef577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1cf010ddc0 0x7f1cf0173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.961+0000 7f1cef577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1cf01135c0 0x7f1cf00f60b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.961+0000 7f1cef577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf00fb2c0 con 0x7f1cf010ddc0 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.961+0000 7f1cef577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf00fb430 con 0x7f1cf010d3d0 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.961+0000 7f1cef577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf00fb5f0 con 0x7f1cf01135c0 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.961+0000 7f1cedd74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1cf010ddc0 0x7f1cf0173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.961+0000 7f1cedd74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1cf010ddc0 0x7f1cf0173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51286/0 (socket says 10.20.193.28:51286) 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.961+0000 7f1cedd74640 1 -- 10.20.193.28:0/2929934265 learned_addr learned my addr 10.20.193.28:0/2929934265 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.962+0000 7f1cee575640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1cf010d3d0 0x7f1cf00f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.962+0000 7f1ceed76640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1cf01135c0 0x7f1cf00f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.962+0000 7f1cedd74640 1 -- 10.20.193.28:0/2929934265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1cf01135c0 msgr2=0x7f1cf00f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.962+0000 7f1cedd74640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1cf01135c0 0x7f1cf00f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.962+0000 7f1cedd74640 1 -- 10.20.193.28:0/2929934265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1cf010d3d0 msgr2=0x7f1cf00f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.962+0000 7f1cedd74640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1cf010d3d0 0x7f1cf00f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.962+0000 7f1cedd74640 1 -- 10.20.193.28:0/2929934265 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd8009570 con 0x7f1cf010ddc0 2026-02-25T11:49:36.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.962+0000 7f1cedd74640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1cf010ddc0 0x7f1cf0173e90 secure :-1 s=READY pgs=443 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cd80098a0 tx=0x7f1cd80028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:36.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.962+0000 7f1cd77fe640 1 -- 10.20.193.28:0/2929934265 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cd803d070 con 0x7f1cf010ddc0 2026-02-25T11:49:36.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.962+0000 7f1cd77fe640 1 -- 10.20.193.28:0/2929934265 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1cd8002a70 con 0x7f1cf010ddc0 2026-02-25T11:49:36.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.962+0000 7f1cef577640 1 -- 10.20.193.28:0/2929934265 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf00fb870 con 0x7f1cf010ddc0 2026-02-25T11:49:36.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.963+0000 7f1cef577640 1 -- 10.20.193.28:0/2929934265 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf00fbce0 con 0x7f1cf010ddc0 2026-02-25T11:49:36.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.963+0000 7f1cd77fe640 1 -- 10.20.193.28:0/2929934265 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cd8041890 con 0x7f1cf010ddc0 2026-02-25T11:49:36.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.965+0000 7f1cd77fe640 1 -- 10.20.193.28:0/2929934265 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f1cd8041a20 con 0x7f1cf010ddc0 2026-02-25T11:49:36.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.967+0000 7f1cd77fe640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1cc40b56d0 0x7f1cc40d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.967+0000 7f1cd77fe640 1 -- 10.20.193.28:0/2929934265 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1cd810b140 con 0x7f1cf010ddc0 2026-02-25T11:49:36.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.967+0000 7f1cee575640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1cc40b56d0 0x7f1cc40d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.968+0000 7f1cee575640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1cc40b56d0 0x7f1cc40d5b80 secure :-1 s=READY pgs=275 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ce4004740 tx=0x7f1ce400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:36.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.968+0000 7f1cf6b3ee00 1 Processor -- start 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.968+0000 7f1cf6b3ee00 1 -- start start 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.968+0000 7f1cf6b3ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 0x55b8461f7f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.968+0000 7f1cf6b3ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 0x55b84623cbc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.968+0000 7f1cf6b3ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b84623d100 0x55b84625d4e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.968+0000 7f1cf6b3ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b8461059b0 con 0x55b8461f7b50 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.968+0000 7f1cf6b3ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b8461016e0 con 0x55b84623d100 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.968+0000 7f1cf6b3ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b8460fb840 con 0x55b8461f8460 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.968+0000 7f1cee575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 0x55b8461f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.968+0000 7f1cee575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 0x55b8461f7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51302/0 (socket says 10.20.193.28:51302) 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.968+0000 7f1cee575640 1 -- 10.20.193.28:0/1433769242 learned_addr learned my addr 10.20.193.28:0/1433769242 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.969+0000 7f1ceed76640 1 --2- 10.20.193.28:0/1433769242 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b84623d100 0x55b84625d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.969+0000 7f1cedd74640 1 --2- 10.20.193.28:0/1433769242 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 0x55b84623cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.969+0000 7f1cee575640 1 -- 10.20.193.28:0/1433769242 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 msgr2=0x55b84623cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:36.971 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.969+0000 7f1cee575640 1 --2- 10.20.193.28:0/1433769242 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 0x55b84623cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.969+0000 7f1cee575640 1 -- 10.20.193.28:0/1433769242 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b84623d100 msgr2=0x55b84625d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:36.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.969+0000 7f1cee575640 1 --2- 10.20.193.28:0/1433769242 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b84623d100 0x55b84625d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.969+0000 7f1cee575640 1 -- 10.20.193.28:0/1433769242 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8460a6d80 con 0x55b8461f7b50 2026-02-25T11:49:36.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.969+0000 7f1cedd74640 1 --2- 10.20.193.28:0/1433769242 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 0x55b84623cbc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:49:36.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.969+0000 7f1cee575640 1 --2- 10.20.193.28:0/1433769242 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 0x55b8461f7f20 secure :-1 s=READY pgs=444 gs=1 cs=0 l=1 c_cookie=17864184624937337788 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ce4005db0 tx=0x7f1ce4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ea5544936283bc server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:36.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.969+0000 7f1cd4ff9640 1 -- 10.20.193.28:0/1433769242 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ce401e640 con 0x55b8461f7b50 2026-02-25T11:49:36.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.969+0000 7f1cd4ff9640 1 -- 10.20.193.28:0/1433769242 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1ce4015070 con 0x55b8461f7b50 2026-02-25T11:49:36.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.969+0000 7f1cd4ff9640 1 -- 10.20.193.28:0/1433769242 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ce4026ce0 con 0x55b8461f7b50 2026-02-25T11:49:36.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.970+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/1433769242 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 msgr2=0x55b8461f7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:36.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.970+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/1433769242 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 0x55b8461f7f20 secure :-1 s=READY pgs=444 gs=1 cs=0 l=1 c_cookie=17864184624937337788 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ce4005db0 tx=0x7f1ce4013000 comp rx=0 tx=0).stop 2026-02-25T11:49:36.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.970+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/1433769242 shutdown_connections 2026-02-25T11:49:36.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.970+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/1433769242 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b84623d100 0x55b84625d4e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.970+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/1433769242 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 0x55b84623cbc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.970+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/1433769242 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 0x55b8461f7f20 unknown :-1 s=CLOSED pgs=444 gs=1 cs=0 l=1 c_cookie=17864184624937337788 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.970+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/1433769242 >> 10.20.193.28:0/1433769242 conn(0x55b84610ed40 msgr2=0x55b8461eda20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:36.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.970+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/1433769242 shutdown_connections 2026-02-25T11:49:36.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.970+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/1433769242 wait complete. 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.971+0000 7f1cf6b3ee00 1 -- start start 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.971+0000 7f1cf6b3ee00 1 Processor -- start 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.971+0000 7f1cf6b3ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b8461f7b50 0x55b8461de3d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.971+0000 7f1cf6b3ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 0x55b8461de910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.971+0000 7f1cf6b3ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b84623d100 0x55b8461e06a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.971+0000 7f1cf6b3ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b8461e0c10 con 0x55b84623d100 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.971+0000 7f1cf6b3ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b8461e0d80 con 0x55b8461f7b50 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.971+0000 7f1cf6b3ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b8461e0f40 con 0x55b8461f8460 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1ceed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b84623d100 0x55b8461e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1ceed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b84623d100 0x55b8461e06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51316/0 (socket says 10.20.193.28:51316) 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1ceed76640 1 -- 10.20.193.28:0/3314700982 learned_addr learned my addr 10.20.193.28:0/3314700982 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1ceed76640 1 -- 10.20.193.28:0/3314700982 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 msgr2=0x55b8461de910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1ceed76640 1 --2- 10.20.193.28:0/3314700982 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 0x55b8461de910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1ceed76640 1 -- 10.20.193.28:0/3314700982 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b8461f7b50 msgr2=0x55b8461de3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1ceed76640 1 --2- 10.20.193.28:0/3314700982 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b8461f7b50 0x55b8461de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.974 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1ceed76640 1 -- 10.20.193.28:0/3314700982 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b84607f350 con 0x55b84623d100 2026-02-25T11:49:36.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1ceed76640 1 --2- 10.20.193.28:0/3314700982 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b84623d100 0x55b8461e06a0 secure :-1 s=READY pgs=445 gs=1 cs=0 l=1 c_cookie=17062467716886747944 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ce00038d0 tx=0x7f1ce0003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecca0ffcd89adb28 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:36.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1c9ffff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:36.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3314700982 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ce0012440 con 0x55b84623d100 2026-02-25T11:49:36.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3314700982 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1ce0012a60 con 0x55b84623d100 2026-02-25T11:49:36.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3314700982 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ce0011d20 con 0x55b84623d100 2026-02-25T11:49:36.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3314700982 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce400bd10 con 0x55b84623d100 2026-02-25T11:49:36.975 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3314700982 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1ce00103c0 con 0x55b84623d100 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1cf6b3ee00 10 client.15777 _ll_register_callbacks cb 0x55b84607c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.972+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3314700982 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b845d916f0 con 0x55b84623d100 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.973+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3314700982 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1ce0011eb0 con 0x55b84623d100 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.973+0000 7f1cf6b3ee00 10 client.15777 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.973+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3314700982 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55b846091150 con 0x55b84623d100 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.973+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3314700982 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f1ce0012bd0 con 0x55b84623d100 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.973+0000 7f1cf6b3ee00 10 client.15777 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.973+0000 7f1cf6b3ee00 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.973+0000 7f1cf6b3ee00 20 client.15777 populate_metadata read hostname 'trial028' 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.973+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/3314700982 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b8461e4a70 0x55b8461e4f10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.973+0000 7f1cf6b3ee00 4 client.15777 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.973+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3314700982 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8460cd910 con 0x55b8461e4a70 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.973+0000 7f1cee575640 1 --2- 10.20.193.28:0/3314700982 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b8461e4a70 0x55b8461e4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.976 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.974+0000 7f1cee575640 1 --2- 10.20.193.28:0/3314700982 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b8461e4a70 0x55b8461e4f10 crc :-1 s=READY pgs=50 gs=4 cs=0 l=0 c_cookie=732260998832622958 s_cookie=2266411607702282825 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a29837fae4a8d6e server_cookie=1f73e95967eaa249 in_seq=0 out_seq=0 2026-02-25T11:49:36.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.974+0000 7f1c9ffff640 10 client.15777 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:36.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3314700982 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55b8460cd910 con 0x55b8461e4a70 2026-02-25T11:49:36.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1c9ffff640 10 client.15777 handle_command_reply: tid=0 2026-02-25T11:49:36.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1cf6b3ee00 1 client.15777 shutdown 2026-02-25T11:49:36.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3314700982 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b84623d100 msgr2=0x55b8461e06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:36.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/3314700982 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b84623d100 0x55b8461e06a0 secure :-1 s=READY pgs=445 gs=1 cs=0 l=1 c_cookie=17062467716886747944 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ce00038d0 tx=0x7f1ce0003da0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3314700982 shutdown_connections 2026-02-25T11:49:36.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/3314700982 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b8461e4a70 0x55b8461e4f10 crc :-1 s=READY pgs=50 gs=4 cs=0 l=0 c_cookie=732260998832622958 s_cookie=2266411607702282825 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/3314700982 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b84623d100 0x55b8461e06a0 unknown :-1 s=CLOSED pgs=445 gs=1 cs=0 l=1 c_cookie=17062467716886747944 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/3314700982 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 0x55b8461de910 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/3314700982 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b8461f7b50 0x55b8461de3d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1c9ffff640 0 client.15777 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:36.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3314700982 >> 10.20.193.28:0/3314700982 conn(0x55b84610ed40 msgr2=0x55b8461f0610 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:36.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3314700982 shutdown_connections 2026-02-25T11:49:36.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3314700982 wait complete. 2026-02-25T11:49:36.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.975+0000 7f1cf6b3ee00 20 client.15777 trim_cache size 0 max 16384 2026-02-25T11:49:36.989 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.984+0000 7f1cf6b3ee00 1 Processor -- start 2026-02-25T11:49:36.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.984+0000 7f1cf6b3ee00 1 -- start start 2026-02-25T11:49:36.990 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cf6b3ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b8461f7b50 0x55b8461e0630 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cf6b3ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f8460 0x55b8461ee880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cf6b3ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b84623d100 0x55b8461e3750 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cf6b3ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b8460f9470 con 0x55b8461f8460 2026-02-25T11:49:36.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cf6b3ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b845db9700 con 0x55b8461f7b50 2026-02-25T11:49:36.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cf6b3ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b8461f8940 con 0x55b84623d100 2026-02-25T11:49:36.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cedd74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f8460 0x55b8461ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cedd74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f8460 0x55b8461ee880 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51328/0 (socket says 10.20.193.28:51328) 2026-02-25T11:49:36.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cedd74640 1 -- 10.20.193.28:0/2518234872 learned_addr learned my addr 10.20.193.28:0/2518234872 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:36.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1ceed76640 1 --2- 10.20.193.28:0/2518234872 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b84623d100 0x55b8461e3750 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cee575640 1 --2- 10.20.193.28:0/2518234872 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b8461f7b50 0x55b8461e0630 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.991 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cedd74640 1 -- 10.20.193.28:0/2518234872 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b84623d100 msgr2=0x55b8461e3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:36.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cedd74640 1 --2- 10.20.193.28:0/2518234872 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b84623d100 0x55b8461e3750 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cedd74640 1 -- 10.20.193.28:0/2518234872 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b8461f7b50 msgr2=0x55b8461e0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:36.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cedd74640 1 --2- 10.20.193.28:0/2518234872 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b8461f7b50 0x55b8461e0630 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cedd74640 1 -- 10.20.193.28:0/2518234872 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b84608bbd0 con 0x55b8461f8460 2026-02-25T11:49:36.992 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.985+0000 7f1cedd74640 1 --2- 10.20.193.28:0/2518234872 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f8460 0x55b8461ee880 secure :-1 s=READY pgs=446 gs=1 cs=0 l=1 c_cookie=4109933244990525612 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cd80419f0 tx=0x7f1cd8004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39096a82ce3a90ac server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:36.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.986+0000 7f1cba4e6640 1 -- 10.20.193.28:0/2518234872 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cd80d71e0 con 0x55b8461f8460 2026-02-25T11:49:36.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.986+0000 7f1cba4e6640 1 -- 10.20.193.28:0/2518234872 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1cd80d97c0 con 0x55b8461f8460 2026-02-25T11:49:36.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.986+0000 7f1cba4e6640 1 -- 10.20.193.28:0/2518234872 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cd80cf7d0 con 0x55b8461f8460 2026-02-25T11:49:36.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.986+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/2518234872 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f8460 msgr2=0x55b8461ee880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:36.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.986+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/2518234872 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f8460 0x55b8461ee880 secure :-1 s=READY pgs=446 gs=1 cs=0 l=1 c_cookie=4109933244990525612 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cd80419f0 tx=0x7f1cd8004490 comp rx=0 tx=0).stop 2026-02-25T11:49:36.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.986+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/2518234872 shutdown_connections 2026-02-25T11:49:36.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.986+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/2518234872 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b84623d100 0x55b8461e3750 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.986+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/2518234872 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f8460 0x55b8461ee880 unknown :-1 s=CLOSED pgs=446 gs=1 cs=0 l=1 c_cookie=4109933244990525612 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.986+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/2518234872 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b8461f7b50 0x55b8461e0630 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.986+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/2518234872 >> 10.20.193.28:0/2518234872 conn(0x55b84610ed40 msgr2=0x55b8461e2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:36.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.986+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/2518234872 shutdown_connections 2026-02-25T11:49:36.993 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.986+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/2518234872 wait complete. 2026-02-25T11:49:36.994 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.987+0000 7f1cf6b3ee00 1 -- start start 2026-02-25T11:49:36.994 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.987+0000 7f1cf6b3ee00 1 Processor -- start 2026-02-25T11:49:36.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.987+0000 7f1cf6b3ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 0x55b8462ae630 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.987+0000 7f1cf6b3ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 0x55b8462abbd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.987+0000 7f1cf6b3ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b84623d100 0x55b8462ac110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:36.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.987+0000 7f1cf6b3ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b8461f38e0 con 0x55b8461f7b50 2026-02-25T11:49:36.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.987+0000 7f1cf6b3ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b8462ac7a0 con 0x55b84623d100 2026-02-25T11:49:36.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.987+0000 7f1cf6b3ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b8462ac960 con 0x55b8461f8460 2026-02-25T11:49:36.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.987+0000 7f1cee575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 0x55b8462ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.987+0000 7f1cee575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 0x55b8462ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51336/0 (socket says 10.20.193.28:51336) 2026-02-25T11:49:36.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.987+0000 7f1cee575640 1 -- 10.20.193.28:0/3287992897 learned_addr learned my addr 10.20.193.28:0/3287992897 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:36.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.987+0000 7f1cedd74640 1 --2- 10.20.193.28:0/3287992897 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 0x55b8462abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.987+0000 7f1ceed76640 1 --2- 10.20.193.28:0/3287992897 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b84623d100 0x55b8462ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:36.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.988+0000 7f1cee575640 1 -- 10.20.193.28:0/3287992897 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 msgr2=0x55b8462abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:36.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.988+0000 7f1cee575640 1 --2- 10.20.193.28:0/3287992897 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 0x55b8462abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.988+0000 7f1cee575640 1 -- 10.20.193.28:0/3287992897 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b84623d100 msgr2=0x55b8462ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:36.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.988+0000 7f1cee575640 1 --2- 10.20.193.28:0/3287992897 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b84623d100 0x55b8462ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:36.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.988+0000 7f1cee575640 1 -- 10.20.193.28:0/3287992897 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b84607f680 con 0x55b8461f7b50 2026-02-25T11:49:36.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.988+0000 7f1cee575640 1 --2- 10.20.193.28:0/3287992897 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 0x55b8462ae630 secure :-1 s=READY pgs=447 gs=1 cs=0 l=1 c_cookie=2423882264641748464 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x55b8461dda80 tx=0x7f1ce4026570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a35c13d1573df0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:36.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.988+0000 7f1c9ffff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:36.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.988+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3287992897 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ce402c040 con 0x55b8461f7b50 2026-02-25T11:49:36.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.988+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3287992897 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1ce4005500 con 0x55b8461f7b50 2026-02-25T11:49:36.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.988+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3287992897 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1ce4004d80 con 0x55b8461f7b50 2026-02-25T11:49:36.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.988+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3287992897 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf00fbce0 con 0x55b8461f7b50 2026-02-25T11:49:36.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.988+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3287992897 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1ce4017070 con 0x55b8461f7b50 2026-02-25T11:49:36.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.988+0000 7f1c9ffff640 10 client.15783 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:49:36.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.989+0000 7f1cf6b3ee00 10 client.15783 _ll_register_callbacks cb 0x55b84607c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:36.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.989+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3287992897 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce4015070 con 0x55b8461f7b50 2026-02-25T11:49:36.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.989+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3287992897 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1ce4005670 con 0x55b8461f7b50 2026-02-25T11:49:36.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.989+0000 7f1cf6b3ee00 10 client.15783 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:36.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.989+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3287992897 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55b8460913b0 con 0x55b8461f7b50 2026-02-25T11:49:37.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.989+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3287992897 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f1ce4024020 con 0x55b8461f7b50 2026-02-25T11:49:37.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.989+0000 7f1cf6b3ee00 10 client.15783 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:37.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.989+0000 7f1cf6b3ee00 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:37.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.989+0000 7f1cf6b3ee00 20 client.15783 populate_metadata read hostname 'trial028' 2026-02-25T11:49:37.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.989+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/3287992897 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b8461ea040 0x55b8461ea4b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:37.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.989+0000 7f1cf6b3ee00 4 client.15783 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:49:37.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.989+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3287992897 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b845d8ef40 con 0x55b8461ea040 2026-02-25T11:49:37.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.990+0000 7f1cedd74640 1 --2- 10.20.193.28:0/3287992897 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b8461ea040 0x55b8461ea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:37.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.990+0000 7f1cedd74640 1 --2- 10.20.193.28:0/3287992897 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b8461ea040 0x55b8461ea4b0 crc :-1 s=READY pgs=51 gs=4 cs=0 l=0 c_cookie=13547076432027036752 s_cookie=8152454835836923098 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc00e23d4c09ac50 server_cookie=7123567ac979acda in_seq=0 out_seq=0 2026-02-25T11:49:37.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.990+0000 7f1c9ffff640 10 client.15783 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:37.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1c9ffff640 1 -- 10.20.193.28:0/3287992897 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x55b845d8ef40 con 0x55b8461ea040 2026-02-25T11:49:37.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1c9ffff640 10 client.15783 handle_command_reply: tid=0 2026-02-25T11:49:37.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1cf6b3ee00 1 client.15783 shutdown 2026-02-25T11:49:37.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3287992897 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 msgr2=0x55b8462ae630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:37.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/3287992897 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 0x55b8462ae630 secure :-1 s=READY pgs=447 gs=1 cs=0 l=1 c_cookie=2423882264641748464 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x55b8461dda80 tx=0x7f1ce4026570 comp rx=0 tx=0).stop 2026-02-25T11:49:37.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3287992897 shutdown_connections 2026-02-25T11:49:37.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/3287992897 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b8461ea040 0x55b8461ea4b0 crc :-1 s=READY pgs=51 gs=4 cs=0 l=0 c_cookie=13547076432027036752 s_cookie=8152454835836923098 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/3287992897 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b84623d100 0x55b8462ac110 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/3287992897 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b8461f8460 0x55b8462abbd0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1cf6b3ee00 1 --2- 10.20.193.28:0/3287992897 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b8461f7b50 0x55b8462ae630 unknown :-1 s=CLOSED pgs=447 gs=1 cs=0 l=1 c_cookie=2423882264641748464 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3287992897 >> 10.20.193.28:0/3287992897 conn(0x55b84610ed40 msgr2=0x55b8461e1650 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:37.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1c9ffff640 0 client.15783 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:37.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3287992897 shutdown_connections 2026-02-25T11:49:37.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1cf6b3ee00 1 -- 10.20.193.28:0/3287992897 wait complete. 2026-02-25T11:49:37.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.991+0000 7f1cf6b3ee00 20 client.15783 trim_cache size 0 max 16384 2026-02-25T11:49:37.002 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:49:37.002 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:49:37.002 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:49:37.002 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:49:37.003 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:49:37.003 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:49:37.003 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:37.003 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:37.003 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:37.003 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:37.003 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:37.003 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:37.003 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:49:37.003 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:49:37.003 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:37.004 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:37.004 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:37.004 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:37.004 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:37.004 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:37.004 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:49:37.004 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:49:37.004 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:37.004 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:37.005 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:37.005 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:37.005 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:37.005 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:37.005 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:49:37.005 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:49:37.005 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:37.005 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:37.005 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:37.005 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:37.005 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:37.005 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:49:37.006 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:49:37.006 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:49:37.006 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.992+0000 7f1cef577640 1 -- 10.20.193.28:0/2929934265 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1cc40b56d0 msgr2=0x7f1cc40d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:37.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.992+0000 7f1cef577640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1cc40b56d0 0x7f1cc40d5b80 secure :-1 s=READY pgs=275 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1ce4004740 tx=0x7f1ce400b410 comp rx=0 tx=0).stop 2026-02-25T11:49:37.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.992+0000 7f1cef577640 1 -- 10.20.193.28:0/2929934265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1cf010ddc0 msgr2=0x7f1cf0173e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:37.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.992+0000 7f1cef577640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1cf010ddc0 0x7f1cf0173e90 secure :-1 s=READY pgs=443 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cd80098a0 tx=0x7f1cd80028d0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.992+0000 7f1cef577640 1 -- 10.20.193.28:0/2929934265 shutdown_connections 2026-02-25T11:49:37.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.992+0000 7f1cef577640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1cc40b56d0 0x7f1cc40d5b80 unknown :-1 s=CLOSED pgs=275 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.992+0000 7f1cef577640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1cf01135c0 0x7f1cf00f60b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.992+0000 7f1cef577640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1cf010ddc0 0x7f1cf0173e90 unknown :-1 s=CLOSED pgs=443 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.992+0000 7f1cef577640 1 --2- 10.20.193.28:0/2929934265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1cf010d3d0 0x7f1cf00f59b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.992+0000 7f1cef577640 1 -- 10.20.193.28:0/2929934265 >> 10.20.193.28:0/2929934265 conn(0x7f1cf00eaca0 msgr2=0x7f1cf010fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:37.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.992+0000 7f1cef577640 1 -- 10.20.193.28:0/2929934265 shutdown_connections 2026-02-25T11:49:37.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:36.992+0000 7f1cef577640 1 -- 10.20.193.28:0/2929934265 wait complete. 2026-02-25T11:49:37.095 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:49:37.096 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:49:37.282 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:37.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.460+0000 7f90b4c08640 1 -- 10.20.193.28:0/1985628575 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90b010f690 msgr2=0x7f90b010fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:37.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.460+0000 7f90b4c08640 1 --2- 10.20.193.28:0/1985628575 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90b010f690 0x7f90b010fa60 secure :-1 s=READY pgs=448 gs=1 cs=0 l=1 c_cookie=4799734479048974004 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f909c009950 tx=0x7f909c02f190 comp rx=0 tx=0).stop 2026-02-25T11:49:37.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.460+0000 7f90b4c08640 1 -- 10.20.193.28:0/1985628575 shutdown_connections 2026-02-25T11:49:37.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.460+0000 7f90b4c08640 1 --2- 10.20.193.28:0/1985628575 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f90b0115880 0x7f90b0176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.460+0000 7f90b4c08640 1 --2- 10.20.193.28:0/1985628575 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90b0110080 0x7f90b01150d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.460+0000 7f90b4c08640 1 --2- 10.20.193.28:0/1985628575 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90b010f690 0x7f90b010fa60 unknown :-1 s=CLOSED pgs=448 gs=1 cs=0 l=1 c_cookie=4799734479048974004 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.460+0000 7f90b4c08640 1 -- 10.20.193.28:0/1985628575 >> 10.20.193.28:0/1985628575 conn(0x7f90b00ecf00 msgr2=0x7f90b010d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:37.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.461+0000 7f90b4c08640 1 -- 10.20.193.28:0/1985628575 shutdown_connections 2026-02-25T11:49:37.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.461+0000 7f90b4c08640 1 -- 10.20.193.28:0/1985628575 wait complete. 2026-02-25T11:49:37.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.461+0000 7f90b4c08640 1 Processor -- start 2026-02-25T11:49:37.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90b4c08640 1 -- start start 2026-02-25T11:49:37.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90b4c08640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90b010f690 0x7f90b00f5960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:37.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90b4c08640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90b0110080 0x7f90b0176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:37.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90b4c08640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f90b0115880 0x7f90b00f6060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:37.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90b4c08640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f90b00fb270 con 0x7f90b0110080 2026-02-25T11:49:37.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90b4c08640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f90b00fb3e0 con 0x7f90b0115880 2026-02-25T11:49:37.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90b4c08640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f90b00fb5a0 con 0x7f90b010f690 2026-02-25T11:49:37.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90aeffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90b0110080 0x7f90b0176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:37.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90aeffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90b0110080 0x7f90b0176150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51356/0 (socket says 10.20.193.28:51356) 2026-02-25T11:49:37.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90aeffd640 1 -- 10.20.193.28:0/499953175 learned_addr learned my addr 10.20.193.28:0/499953175 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:37.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90af7fe640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90b010f690 0x7f90b00f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:37.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90affff640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f90b0115880 0x7f90b00f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:37.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90aeffd640 1 -- 10.20.193.28:0/499953175 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90b010f690 msgr2=0x7f90b00f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:37.466 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90aeffd640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90b010f690 0x7f90b00f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.466 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90aeffd640 1 -- 10.20.193.28:0/499953175 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f90b0115880 msgr2=0x7f90b00f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:37.466 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90aeffd640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f90b0115880 0x7f90b00f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.466 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90aeffd640 1 -- 10.20.193.28:0/499953175 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909c009570 con 0x7f90b0110080 2026-02-25T11:49:37.466 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.462+0000 7f90affff640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f90b0115880 0x7f90b00f6060 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:37.466 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.463+0000 7f90aeffd640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90b0110080 0x7f90b0176150 secure :-1 s=READY pgs=449 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f90a0005db0 tx=0x7f90a000ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:37.466 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.463+0000 7f90acff9640 1 -- 10.20.193.28:0/499953175 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f90a0004500 con 0x7f90b0110080 2026-02-25T11:49:37.466 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.463+0000 7f90acff9640 1 -- 10.20.193.28:0/499953175 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f90a0004d10 con 0x7f90b0110080 2026-02-25T11:49:37.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.463+0000 7f90acff9640 1 -- 10.20.193.28:0/499953175 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f90a0005280 con 0x7f90b0110080 2026-02-25T11:49:37.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.463+0000 7f90b4c08640 1 -- 10.20.193.28:0/499953175 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b00fb880 con 0x7f90b0110080 2026-02-25T11:49:37.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.463+0000 7f90b4c08640 1 -- 10.20.193.28:0/499953175 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b00fbd50 con 0x7f90b0110080 2026-02-25T11:49:37.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.464+0000 7f90b4c08640 1 -- 10.20.193.28:0/499953175 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b0110970 con 0x7f90b0110080 2026-02-25T11:49:37.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.466+0000 7f90acff9640 1 -- 10.20.193.28:0/499953175 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f90a0005410 con 0x7f90b0110080 2026-02-25T11:49:37.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.467+0000 7f90acff9640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f90840b5750 0x7f90840d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:37.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.468+0000 7f90acff9640 1 -- 10.20.193.28:0/499953175 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f90a00e6f70 con 0x7f90b0110080 2026-02-25T11:49:37.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.468+0000 7f90af7fe640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f90840b5750 0x7f90840d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:37.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.468+0000 7f90acff9640 1 -- 10.20.193.28:0/499953175 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f90a00e73f0 con 0x7f90b0110080 2026-02-25T11:49:37.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.468+0000 7f90af7fe640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f90840b5750 0x7f90840d5c00 secure :-1 s=READY pgs=276 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f909c0098a0 tx=0x7f909c03a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:37.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.611+0000 7f90b4c08640 1 -- 10.20.193.28:0/499953175 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90b010fa60 con 0x7f90b0110080 2026-02-25T11:49:37.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.611+0000 7f90acff9640 1 -- 10.20.193.28:0/499953175 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f90a00a9940 con 0x7f90b0110080 2026-02-25T11:49:37.614 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:49:37.614 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:49:37.614 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:49:37.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.613+0000 7f90b4c08640 1 -- 10.20.193.28:0/499953175 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f90840b5750 msgr2=0x7f90840d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:37.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.613+0000 7f90b4c08640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f90840b5750 0x7f90840d5c00 secure :-1 s=READY pgs=276 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f909c0098a0 tx=0x7f909c03a040 comp rx=0 tx=0).stop 2026-02-25T11:49:37.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.613+0000 7f90b4c08640 1 -- 10.20.193.28:0/499953175 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90b0110080 msgr2=0x7f90b0176150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:37.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.613+0000 7f90b4c08640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90b0110080 0x7f90b0176150 secure :-1 s=READY pgs=449 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f90a0005db0 tx=0x7f90a000ddf0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.613+0000 7f90b4c08640 1 -- 10.20.193.28:0/499953175 shutdown_connections 2026-02-25T11:49:37.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.613+0000 7f90b4c08640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f90840b5750 0x7f90840d5c00 unknown :-1 s=CLOSED pgs=276 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.613+0000 7f90b4c08640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f90b0115880 0x7f90b00f6060 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.613+0000 7f90b4c08640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f90b0110080 0x7f90b0176150 unknown :-1 s=CLOSED pgs=449 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.613+0000 7f90b4c08640 1 --2- 10.20.193.28:0/499953175 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f90b010f690 0x7f90b00f5960 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:37.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.613+0000 7f90b4c08640 1 -- 10.20.193.28:0/499953175 >> 10.20.193.28:0/499953175 conn(0x7f90b00ecf00 msgr2=0x7f90b0111b30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:37.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.613+0000 7f90b4c08640 1 -- 10.20.193.28:0/499953175 shutdown_connections 2026-02-25T11:49:37.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:37.613+0000 7f90b4c08640 1 -- 10.20.193.28:0/499953175 wait complete. 2026-02-25T11:49:37.717 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:49:37.902 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:38.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.087+0000 7f5e6ddd2640 1 -- 10.20.193.28:0/2165756066 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5e6810bee0 msgr2=0x7f5e6810c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.087+0000 7f5e6ddd2640 1 --2- 10.20.193.28:0/2165756066 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5e6810bee0 0x7f5e6810c2b0 secure :-1 s=READY pgs=450 gs=1 cs=0 l=1 c_cookie=5345365995364485490 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5e58009950 tx=0x7f5e5802f190 comp rx=0 tx=0).stop 2026-02-25T11:49:38.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.087+0000 7f5e6ddd2640 1 -- 10.20.193.28:0/2165756066 shutdown_connections 2026-02-25T11:49:38.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.087+0000 7f5e6ddd2640 1 --2- 10.20.193.28:0/2165756066 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5e68155c10 0x7f5e68175ff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.087+0000 7f5e6ddd2640 1 --2- 10.20.193.28:0/2165756066 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5e6810c7f0 0x7f5e681556d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.087+0000 7f5e6ddd2640 1 --2- 10.20.193.28:0/2165756066 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5e6810bee0 0x7f5e6810c2b0 unknown :-1 s=CLOSED pgs=450 gs=1 cs=0 l=1 c_cookie=5345365995364485490 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.087+0000 7f5e6ddd2640 1 -- 10.20.193.28:0/2165756066 >> 10.20.193.28:0/2165756066 conn(0x7f5e680e9b40 msgr2=0x7f5e68109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:38.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.087+0000 7f5e6ddd2640 1 -- 10.20.193.28:0/2165756066 shutdown_connections 2026-02-25T11:49:38.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.087+0000 7f5e6ddd2640 1 -- 10.20.193.28:0/2165756066 wait complete. 2026-02-25T11:49:38.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.088+0000 7f5e6ddd2640 1 Processor -- start 2026-02-25T11:49:38.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.088+0000 7f5e6ddd2640 1 -- start start 2026-02-25T11:49:38.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.088+0000 7f5e6ddd2640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5e6810bee0 0x7f5e68161340 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.088+0000 7f5e6ddd2640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5e6810c7f0 0x7f5e68161880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.088+0000 7f5e6ddd2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5e68155c10 0x7f5e68166c50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.088+0000 7f5e6ddd2640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5e68167270 con 0x7f5e68155c10 2026-02-25T11:49:38.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.088+0000 7f5e6ddd2640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e681673e0 con 0x7f5e6810bee0 2026-02-25T11:49:38.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.088+0000 7f5e6ddd2640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5e681675a0 con 0x7f5e6810c7f0 2026-02-25T11:49:38.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e6d5d1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5e68155c10 0x7f5e68166c50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e6d5d1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5e68155c10 0x7f5e68166c50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51380/0 (socket says 10.20.193.28:51380) 2026-02-25T11:49:38.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e6d5d1640 1 -- 10.20.193.28:0/1147973905 learned_addr learned my addr 10.20.193.28:0/1147973905 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:38.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5e6810bee0 0x7f5e68161340 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e5ffff640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5e6810c7f0 0x7f5e68161880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e6d5d1640 1 -- 10.20.193.28:0/1147973905 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5e6810c7f0 msgr2=0x7f5e68161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e6d5d1640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5e6810c7f0 0x7f5e68161880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e6d5d1640 1 -- 10.20.193.28:0/1147973905 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5e6810bee0 msgr2=0x7f5e68161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e6d5d1640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5e6810bee0 0x7f5e68161340 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e6d5d1640 1 -- 10.20.193.28:0/1147973905 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e58009570 con 0x7f5e68155c10 2026-02-25T11:49:38.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5e6810bee0 0x7f5e68161340 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:38.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e6d5d1640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5e68155c10 0x7f5e68166c50 secure :-1 s=READY pgs=451 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5e6400e950 tx=0x7f5e6400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:38.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e5dffb640 1 -- 10.20.193.28:0/1147973905 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5e64005500 con 0x7f5e68155c10 2026-02-25T11:49:38.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.089+0000 7f5e6ddd2640 1 -- 10.20.193.28:0/1147973905 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e68167880 con 0x7f5e68155c10 2026-02-25T11:49:38.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.090+0000 7f5e5dffb640 1 -- 10.20.193.28:0/1147973905 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5e6400cd00 con 0x7f5e68155c10 2026-02-25T11:49:38.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.090+0000 7f5e5dffb640 1 -- 10.20.193.28:0/1147973905 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5e640116e0 con 0x7f5e68155c10 2026-02-25T11:49:38.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.090+0000 7f5e6ddd2640 1 -- 10.20.193.28:0/1147973905 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e68152840 con 0x7f5e68155c10 2026-02-25T11:49:38.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.092+0000 7f5e5dffb640 1 -- 10.20.193.28:0/1147973905 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f5e64011870 con 0x7f5e68155c10 2026-02-25T11:49:38.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.094+0000 7f5e5dffb640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5e440b5760 0x7f5e440d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.094+0000 7f5e5dffb640 1 -- 10.20.193.28:0/1147973905 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f5e640e8af0 con 0x7f5e68155c10 2026-02-25T11:49:38.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.094+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5e440b5760 0x7f5e440d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.095+0000 7f5e71056e00 1 Processor -- start 2026-02-25T11:49:38.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.095+0000 7f5e71056e00 1 -- start start 2026-02-25T11:49:38.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.095+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5e440b5760 0x7f5e440d5c10 secure :-1 s=READY pgs=277 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5e58005b30 tx=0x7f5e5803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:38.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.095+0000 7f5e71056e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae79910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.095+0000 7f5e71056e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930ae79e50 0x55930aebe5b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.095+0000 7f5e71056e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930aebeaf0 0x55930aedeed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.095+0000 7f5e71056e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55930ad868a0 con 0x55930ae79540 2026-02-25T11:49:38.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.095+0000 7f5e71056e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55930ad825d0 con 0x55930aebeaf0 2026-02-25T11:49:38.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.095+0000 7f5e71056e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55930ad7c570 con 0x55930ae79e50 2026-02-25T11:49:38.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.095+0000 7f5e6cdd0640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae79910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.095+0000 7f5e5ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930ae79e50 0x55930aebe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.095+0000 7f5e5ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930ae79e50 0x55930aebe5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52900/0 (socket says 10.20.193.28:52900) 2026-02-25T11:49:38.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.095+0000 7f5e5ffff640 1 -- 10.20.193.28:0/2703497050 learned_addr learned my addr 10.20.193.28:0/2703497050 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:38.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.096+0000 7f5e6d5d1640 1 --2- 10.20.193.28:0/2703497050 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930aebeaf0 0x55930aedeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.096+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/2703497050 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930ae79e50 msgr2=0x55930aebe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.096+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/2703497050 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930ae79e50 0x55930aebe5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.096+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/2703497050 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930aebeaf0 msgr2=0x55930aedeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.096+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/2703497050 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930aebeaf0 0x55930aedeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.096+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/2703497050 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55930ad27d10 con 0x55930ae79540 2026-02-25T11:49:38.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.096+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/2703497050 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae79910 secure :-1 s=READY pgs=452 gs=1 cs=0 l=1 c_cookie=5396761657701695717 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5e580098a0 tx=0x7f5e58040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae5262268c200e5 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:38.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.096+0000 7f5e354e4640 1 -- 10.20.193.28:0/2703497050 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5e58005bb0 con 0x55930ae79540 2026-02-25T11:49:38.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.096+0000 7f5e354e4640 1 -- 10.20.193.28:0/2703497050 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5e5803e040 con 0x55930ae79540 2026-02-25T11:49:38.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.096+0000 7f5e354e4640 1 -- 10.20.193.28:0/2703497050 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5e5804fad0 con 0x55930ae79540 2026-02-25T11:49:38.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.096+0000 7f5e71056e00 1 -- 10.20.193.28:0/2703497050 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 msgr2=0x55930ae79910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.096+0000 7f5e71056e00 1 --2- 10.20.193.28:0/2703497050 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae79910 secure :-1 s=READY pgs=452 gs=1 cs=0 l=1 c_cookie=5396761657701695717 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5e580098a0 tx=0x7f5e58040000 comp rx=0 tx=0).stop 2026-02-25T11:49:38.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.097+0000 7f5e71056e00 1 -- 10.20.193.28:0/2703497050 shutdown_connections 2026-02-25T11:49:38.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.097+0000 7f5e71056e00 1 --2- 10.20.193.28:0/2703497050 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930aebeaf0 0x55930aedeed0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.097+0000 7f5e71056e00 1 --2- 10.20.193.28:0/2703497050 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930ae79e50 0x55930aebe5b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.097+0000 7f5e71056e00 1 --2- 10.20.193.28:0/2703497050 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae79910 unknown :-1 s=CLOSED pgs=452 gs=1 cs=0 l=1 c_cookie=5396761657701695717 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.097+0000 7f5e71056e00 1 -- 10.20.193.28:0/2703497050 >> 10.20.193.28:0/2703497050 conn(0x55930ad8f7a0 msgr2=0x55930ae6f410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:38.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.097+0000 7f5e71056e00 1 -- 10.20.193.28:0/2703497050 shutdown_connections 2026-02-25T11:49:38.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.097+0000 7f5e71056e00 1 -- 10.20.193.28:0/2703497050 wait complete. 2026-02-25T11:49:38.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.098+0000 7f5e71056e00 1 -- start start 2026-02-25T11:49:38.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.098+0000 7f5e71056e00 1 Processor -- start 2026-02-25T11:49:38.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e71056e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae91a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e71056e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79e50 0x55930ae91f50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e71056e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930aebeaf0 0x55930ae96f40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e71056e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55930ae97480 con 0x55930ae79540 2026-02-25T11:49:38.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e71056e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55930ae975f0 con 0x55930ae79e50 2026-02-25T11:49:38.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e71056e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55930ae977b0 con 0x55930aebeaf0 2026-02-25T11:49:38.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e6cdd0640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae91a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e5ffff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79e50 0x55930ae91f50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e6cdd0640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae91a10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51392/0 (socket says 10.20.193.28:51392) 2026-02-25T11:49:38.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/2591826599 learned_addr learned my addr 10.20.193.28:0/2591826599 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:38.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e6d5d1640 1 --2- 10.20.193.28:0/2591826599 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930aebeaf0 0x55930ae96f40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/2591826599 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930aebeaf0 msgr2=0x55930ae96f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/2591826599 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930aebeaf0 0x55930ae96f40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/2591826599 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79e50 msgr2=0x55930ae91f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/2591826599 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79e50 0x55930ae91f50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/2591826599 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55930ad003c0 con 0x55930ae79540 2026-02-25T11:49:38.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/2591826599 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae91a10 secure :-1 s=READY pgs=453 gs=1 cs=0 l=1 c_cookie=17451899490653468204 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5e580098d0 tx=0x7f5e58002390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2319a1f6438762c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:38.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.099+0000 7f5e267fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:38.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e267fc640 1 -- 10.20.193.28:0/2591826599 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5e580388c0 con 0x55930ae79540 2026-02-25T11:49:38.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e267fc640 1 -- 10.20.193.28:0/2591826599 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5e58058460 con 0x55930ae79540 2026-02-25T11:49:38.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e267fc640 1 -- 10.20.193.28:0/2591826599 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5e58044670 con 0x55930ae79540 2026-02-25T11:49:38.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e71056e00 1 -- 10.20.193.28:0/2591826599 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e68176a80 con 0x55930ae79540 2026-02-25T11:49:38.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e267fc640 1 -- 10.20.193.28:0/2591826599 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f5e58038a20 con 0x55930ae79540 2026-02-25T11:49:38.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e71056e00 10 client.15801 _ll_register_callbacks cb 0x55930acfda10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:38.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e71056e00 1 -- 10.20.193.28:0/2591826599 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55930aa126f0 con 0x55930ae79540 2026-02-25T11:49:38.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e267fc640 1 -- 10.20.193.28:0/2591826599 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5e58038c20 con 0x55930ae79540 2026-02-25T11:49:38.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e71056e00 10 client.15801 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:38.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e71056e00 1 -- 10.20.193.28:0/2591826599 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55930ad120e0 con 0x55930ae79540 2026-02-25T11:49:38.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e267fc640 1 -- 10.20.193.28:0/2591826599 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f5e580585d0 con 0x55930ae79540 2026-02-25T11:49:38.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e71056e00 10 client.15801 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:38.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e71056e00 10 client.15801 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:49:38.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.100+0000 7f5e71056e00 20 client.15801 populate_metadata read hostname 'trial028' 2026-02-25T11:49:38.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.101+0000 7f5e71056e00 1 --2- 10.20.193.28:0/2591826599 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55930aebbed0 0x55930aebc370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.101+0000 7f5e71056e00 4 client.15801 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:49:38.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.101+0000 7f5e71056e00 1 -- 10.20.193.28:0/2591826599 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55930ad4efc0 con 0x55930aebbed0 2026-02-25T11:49:38.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.101+0000 7f5e5ffff640 1 --2- 10.20.193.28:0/2591826599 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55930aebbed0 0x55930aebc370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.101+0000 7f5e5ffff640 1 --2- 10.20.193.28:0/2591826599 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55930aebbed0 0x55930aebc370 crc :-1 s=READY pgs=29 gs=4 cs=0 l=0 c_cookie=12289218382401257653 s_cookie=10122383744317631388 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa8c1301fa3a40b5 server_cookie=8c79ee62803dab9c in_seq=0 out_seq=0 2026-02-25T11:49:38.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.101+0000 7f5e267fc640 10 client.15801 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:49:38.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.102+0000 7f5e267fc640 1 -- 10.20.193.28:0/2591826599 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55930ad4efc0 con 0x55930aebbed0 2026-02-25T11:49:38.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.102+0000 7f5e267fc640 10 client.15801 handle_command_reply: tid=0 2026-02-25T11:49:38.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.102+0000 7f5e71056e00 1 client.15801 shutdown 2026-02-25T11:49:38.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.102+0000 7f5e71056e00 1 -- 10.20.193.28:0/2591826599 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 msgr2=0x55930ae91a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.102+0000 7f5e71056e00 1 --2- 10.20.193.28:0/2591826599 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae91a10 secure :-1 s=READY pgs=453 gs=1 cs=0 l=1 c_cookie=17451899490653468204 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5e580098d0 tx=0x7f5e58002390 comp rx=0 tx=0).stop 2026-02-25T11:49:38.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.103+0000 7f5e71056e00 1 -- 10.20.193.28:0/2591826599 shutdown_connections 2026-02-25T11:49:38.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.103+0000 7f5e71056e00 1 --2- 10.20.193.28:0/2591826599 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55930aebbed0 0x55930aebc370 crc :-1 s=READY pgs=29 gs=4 cs=0 l=0 c_cookie=12289218382401257653 s_cookie=10122383744317631388 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.103+0000 7f5e71056e00 1 --2- 10.20.193.28:0/2591826599 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930aebeaf0 0x55930ae96f40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.103+0000 7f5e71056e00 1 --2- 10.20.193.28:0/2591826599 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79e50 0x55930ae91f50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.103+0000 7f5e267fc640 0 client.15801 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:49:38.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.103+0000 7f5e71056e00 1 --2- 10.20.193.28:0/2591826599 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae91a10 unknown :-1 s=CLOSED pgs=453 gs=1 cs=0 l=1 c_cookie=17451899490653468204 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.103+0000 7f5e71056e00 1 -- 10.20.193.28:0/2591826599 >> 10.20.193.28:0/2591826599 conn(0x55930ad8f7a0 msgr2=0x55930ae6fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:38.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.103+0000 7f5e71056e00 1 -- 10.20.193.28:0/2591826599 shutdown_connections 2026-02-25T11:49:38.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.103+0000 7f5e71056e00 1 -- 10.20.193.28:0/2591826599 wait complete. 2026-02-25T11:49:38.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.103+0000 7f5e71056e00 20 client.15801 trim_cache size 0 max 16384 2026-02-25T11:49:38.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.110+0000 7f5e71056e00 1 Processor -- start 2026-02-25T11:49:38.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.110+0000 7f5e71056e00 1 -- start start 2026-02-25T11:49:38.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.110+0000 7f5e71056e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79540 0x55930ae8f010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.110+0000 7f5e71056e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930ae79e50 0x55930ae964b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.110+0000 7f5e71056e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930aebeaf0 0x55930aebdc60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.110+0000 7f5e71056e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55930ad7a1a0 con 0x55930aebeaf0 2026-02-25T11:49:38.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.110+0000 7f5e71056e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55930aa3a700 con 0x55930ae79540 2026-02-25T11:49:38.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.110+0000 7f5e71056e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55930ae4be80 con 0x55930ae79e50 2026-02-25T11:49:38.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.111+0000 7f5e6cdd0640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79540 0x55930ae8f010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.111+0000 7f5e6cdd0640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79540 0x55930ae8f010 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:56090/0 (socket says 10.20.193.28:56090) 2026-02-25T11:49:38.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.111+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/4079878645 learned_addr learned my addr 10.20.193.28:0/4079878645 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:38.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.111+0000 7f5e5ffff640 1 --2- 10.20.193.28:0/4079878645 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930ae79e50 0x55930ae964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.111+0000 7f5e6d5d1640 1 --2- 10.20.193.28:0/4079878645 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930aebeaf0 0x55930aebdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.111+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/4079878645 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930ae79e50 msgr2=0x55930ae964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.111+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/4079878645 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930ae79e50 0x55930ae964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.111+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/4079878645 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930aebeaf0 msgr2=0x55930aebdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.111+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/4079878645 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930aebeaf0 0x55930aebdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.111+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/4079878645 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55930ad0cb60 con 0x55930ae79540 2026-02-25T11:49:38.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.111+0000 7f5e6d5d1640 1 --2- 10.20.193.28:0/4079878645 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930aebeaf0 0x55930aebdc60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:38.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.111+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/4079878645 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79540 0x55930ae8f010 secure :-1 s=READY pgs=118 gs=1 cs=0 l=1 c_cookie=1290774878588884395 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5e58004820 tx=0x7f5e58046070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11e9c0e9cb49d5ab server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:38.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.112+0000 7f5e34ce3640 1 -- 10.20.193.28:0/4079878645 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5e5804fb10 con 0x55930ae79540 2026-02-25T11:49:38.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.112+0000 7f5e34ce3640 1 -- 10.20.193.28:0/4079878645 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5e58068070 con 0x55930ae79540 2026-02-25T11:49:38.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.112+0000 7f5e34ce3640 1 -- 10.20.193.28:0/4079878645 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5e580586e0 con 0x55930ae79540 2026-02-25T11:49:38.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.112+0000 7f5e71056e00 1 -- 10.20.193.28:0/4079878645 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79540 msgr2=0x55930ae8f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.112+0000 7f5e71056e00 1 --2- 10.20.193.28:0/4079878645 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79540 0x55930ae8f010 secure :-1 s=READY pgs=118 gs=1 cs=0 l=1 c_cookie=1290774878588884395 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5e58004820 tx=0x7f5e58046070 comp rx=0 tx=0).stop 2026-02-25T11:49:38.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.112+0000 7f5e71056e00 1 -- 10.20.193.28:0/4079878645 shutdown_connections 2026-02-25T11:49:38.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.112+0000 7f5e71056e00 1 --2- 10.20.193.28:0/4079878645 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930aebeaf0 0x55930aebdc60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.112+0000 7f5e71056e00 1 --2- 10.20.193.28:0/4079878645 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930ae79e50 0x55930ae964b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.112+0000 7f5e71056e00 1 --2- 10.20.193.28:0/4079878645 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79540 0x55930ae8f010 unknown :-1 s=CLOSED pgs=118 gs=1 cs=0 l=1 c_cookie=1290774878588884395 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.112+0000 7f5e71056e00 1 -- 10.20.193.28:0/4079878645 >> 10.20.193.28:0/4079878645 conn(0x55930ad8f7a0 msgr2=0x55930ae95da0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:38.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.112+0000 7f5e71056e00 1 -- 10.20.193.28:0/4079878645 shutdown_connections 2026-02-25T11:49:38.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.112+0000 7f5e71056e00 1 -- 10.20.193.28:0/4079878645 wait complete. 2026-02-25T11:49:38.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.113+0000 7f5e71056e00 1 -- start start 2026-02-25T11:49:38.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.113+0000 7f5e71056e00 1 Processor -- start 2026-02-25T11:49:38.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.113+0000 7f5e71056e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae99500 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.113+0000 7f5e71056e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79e50 0x55930ae99a40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.113+0000 7f5e71056e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930aebeaf0 0x55930ae99f80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.113+0000 7f5e71056e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55930ae706c0 con 0x55930ae79540 2026-02-25T11:49:38.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.113+0000 7f5e71056e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55930ae978f0 con 0x55930ae79e50 2026-02-25T11:49:38.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.113+0000 7f5e71056e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55930aa3a700 con 0x55930aebeaf0 2026-02-25T11:49:38.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.113+0000 7f5e6cdd0640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae99500 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.113+0000 7f5e6cdd0640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae99500 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51398/0 (socket says 10.20.193.28:51398) 2026-02-25T11:49:38.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.113+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/1680670812 learned_addr learned my addr 10.20.193.28:0/1680670812 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:38.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.113+0000 7f5e6d5d1640 1 --2- 10.20.193.28:0/1680670812 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930aebeaf0 0x55930ae99f80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e5ffff640 1 --2- 10.20.193.28:0/1680670812 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79e50 0x55930ae99a40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/1680670812 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930aebeaf0 msgr2=0x55930ae99f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/1680670812 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930aebeaf0 0x55930ae99f80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/1680670812 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79e50 msgr2=0x55930ae99a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/1680670812 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79e50 0x55930ae99a40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e6cdd0640 1 -- 10.20.193.28:0/1680670812 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55930ad120e0 con 0x55930ae79540 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e6cdd0640 1 --2- 10.20.193.28:0/1680670812 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae99500 secure :-1 s=READY pgs=454 gs=1 cs=0 l=1 c_cookie=18181453850782476850 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5e58038540 tx=0x7f5e58002b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc517ff2a40c2a32 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e267fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e267fc640 1 -- 10.20.193.28:0/1680670812 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5e5804fb10 con 0x55930ae79540 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e267fc640 1 -- 10.20.193.28:0/1680670812 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55930aa126f0 con 0x55930ae79540 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e267fc640 1 -- 10.20.193.28:0/1680670812 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5e58044c80 con 0x55930ae79540 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e71056e00 1 -- 10.20.193.28:0/1680670812 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e68152840 con 0x55930ae79540 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e267fc640 1 -- 10.20.193.28:0/1680670812 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f5e58044e10 con 0x55930ae79540 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e71056e00 10 client.15807 _ll_register_callbacks cb 0x55930acfda10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.114+0000 7f5e71056e00 1 -- 10.20.193.28:0/1680670812 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e5803e040 con 0x55930ae79540 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.115+0000 7f5e267fc640 1 -- 10.20.193.28:0/1680670812 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5e58058460 con 0x55930ae79540 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.115+0000 7f5e71056e00 10 client.15807 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:38.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.115+0000 7f5e71056e00 1 -- 10.20.193.28:0/1680670812 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55930ad006f0 con 0x55930ae79540 2026-02-25T11:49:38.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.115+0000 7f5e267fc640 1 -- 10.20.193.28:0/1680670812 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f5e58075030 con 0x55930ae79540 2026-02-25T11:49:38.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.115+0000 7f5e71056e00 10 client.15807 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:38.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.115+0000 7f5e71056e00 10 client.15807 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:49:38.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.115+0000 7f5e71056e00 20 client.15807 populate_metadata read hostname 'trial028' 2026-02-25T11:49:38.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.115+0000 7f5e71056e00 1 --2- 10.20.193.28:0/1680670812 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55930af6baa0 0x55930af8bf20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.115+0000 7f5e71056e00 4 client.15807 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:49:38.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.115+0000 7f5e71056e00 1 -- 10.20.193.28:0/1680670812 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55930aa0ff40 con 0x55930af6baa0 2026-02-25T11:49:38.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.115+0000 7f5e5ffff640 1 --2- 10.20.193.28:0/1680670812 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55930af6baa0 0x55930af8bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e5ffff640 1 --2- 10.20.193.28:0/1680670812 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55930af6baa0 0x55930af8bf20 crc :-1 s=READY pgs=30 gs=4 cs=0 l=0 c_cookie=11579658692463709808 s_cookie=10706924674578194424 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a0b3363aa8904e70 server_cookie=9496a34389191ff8 in_seq=0 out_seq=0 2026-02-25T11:49:38.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e267fc640 10 client.15807 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:49:38.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e267fc640 1 -- 10.20.193.28:0/1680670812 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55930aa0ff40 con 0x55930af6baa0 2026-02-25T11:49:38.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e267fc640 10 client.15807 handle_command_reply: tid=0 2026-02-25T11:49:38.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e71056e00 1 client.15807 shutdown 2026-02-25T11:49:38.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e71056e00 1 -- 10.20.193.28:0/1680670812 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 msgr2=0x55930ae99500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e71056e00 1 --2- 10.20.193.28:0/1680670812 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae99500 secure :-1 s=READY pgs=454 gs=1 cs=0 l=1 c_cookie=18181453850782476850 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5e58038540 tx=0x7f5e58002b40 comp rx=0 tx=0).stop 2026-02-25T11:49:38.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e71056e00 1 -- 10.20.193.28:0/1680670812 shutdown_connections 2026-02-25T11:49:38.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e71056e00 1 --2- 10.20.193.28:0/1680670812 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55930af6baa0 0x55930af8bf20 crc :-1 s=READY pgs=30 gs=4 cs=0 l=0 c_cookie=11579658692463709808 s_cookie=10706924674578194424 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e71056e00 1 --2- 10.20.193.28:0/1680670812 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55930aebeaf0 0x55930ae99f80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e71056e00 1 --2- 10.20.193.28:0/1680670812 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55930ae79e50 0x55930ae99a40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e267fc640 0 client.15807 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:49:38.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e71056e00 1 --2- 10.20.193.28:0/1680670812 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55930ae79540 0x55930ae99500 unknown :-1 s=CLOSED pgs=454 gs=1 cs=0 l=1 c_cookie=18181453850782476850 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.116+0000 7f5e71056e00 1 -- 10.20.193.28:0/1680670812 >> 10.20.193.28:0/1680670812 conn(0x55930ad8f7a0 msgr2=0x55930ae6f500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:38.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e71056e00 1 -- 10.20.193.28:0/1680670812 shutdown_connections 2026-02-25T11:49:38.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e71056e00 1 -- 10.20.193.28:0/1680670812 wait complete. 2026-02-25T11:49:38.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e71056e00 20 client.15807 trim_cache size 0 max 16384 2026-02-25T11:49:38.127 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:49:38.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e6ddd2640 1 -- 10.20.193.28:0/1147973905 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5e440b5760 msgr2=0x7f5e440d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e6ddd2640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5e440b5760 0x7f5e440d5c10 secure :-1 s=READY pgs=277 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5e58005b30 tx=0x7f5e5803a040 comp rx=0 tx=0).stop 2026-02-25T11:49:38.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e6ddd2640 1 -- 10.20.193.28:0/1147973905 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5e68155c10 msgr2=0x7f5e68166c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e6ddd2640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5e68155c10 0x7f5e68166c50 secure :-1 s=READY pgs=451 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5e6400e950 tx=0x7f5e6400ee20 comp rx=0 tx=0).stop 2026-02-25T11:49:38.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e6ddd2640 1 -- 10.20.193.28:0/1147973905 shutdown_connections 2026-02-25T11:49:38.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e6ddd2640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5e440b5760 0x7f5e440d5c10 unknown :-1 s=CLOSED pgs=277 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e6ddd2640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5e68155c10 0x7f5e68166c50 unknown :-1 s=CLOSED pgs=451 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e6ddd2640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5e6810c7f0 0x7f5e68161880 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e6ddd2640 1 --2- 10.20.193.28:0/1147973905 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5e6810bee0 0x7f5e68161340 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e6ddd2640 1 -- 10.20.193.28:0/1147973905 >> 10.20.193.28:0/1147973905 conn(0x7f5e680e9b40 msgr2=0x7f5e68109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:38.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e6ddd2640 1 -- 10.20.193.28:0/1147973905 shutdown_connections 2026-02-25T11:49:38.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.117+0000 7f5e6ddd2640 1 -- 10.20.193.28:0/1147973905 wait complete. 2026-02-25T11:49:38.216 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:38 trial028 ceph-mon[39224]: pgmap v268: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:38.217 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:38 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/499953175' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:49:38.218 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:49:38.401 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:38.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:38 trial167 ceph-mon[43392]: pgmap v268: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:38.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:38 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/499953175' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:49:38.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:38 trial119 ceph-mon[42710]: pgmap v268: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:38.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:38 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/499953175' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:49:38.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.587+0000 7f1741787640 1 -- 10.20.193.28:0/2993600661 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f173c10de10 msgr2=0x7f173c112e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.587+0000 7f1741787640 1 --2- 10.20.193.28:0/2993600661 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f173c10de10 0x7f173c112e60 secure :-1 s=READY pgs=455 gs=1 cs=0 l=1 c_cookie=6654316445903487162 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f172c009950 tx=0x7f172c02f190 comp rx=0 tx=0).stop 2026-02-25T11:49:38.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.587+0000 7f1741787640 1 -- 10.20.193.28:0/2993600661 shutdown_connections 2026-02-25T11:49:38.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.587+0000 7f1741787640 1 --2- 10.20.193.28:0/2993600661 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f173c113610 0x7f173c173fe0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.587+0000 7f1741787640 1 --2- 10.20.193.28:0/2993600661 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f173c10de10 0x7f173c112e60 unknown :-1 s=CLOSED pgs=455 gs=1 cs=0 l=1 c_cookie=6654316445903487162 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.587+0000 7f1741787640 1 --2- 10.20.193.28:0/2993600661 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f173c10d420 0x7f173c10d7f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.587+0000 7f1741787640 1 -- 10.20.193.28:0/2993600661 >> 10.20.193.28:0/2993600661 conn(0x7f173c0eacb0 msgr2=0x7f173c10b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:38.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.587+0000 7f1741787640 1 -- 10.20.193.28:0/2993600661 shutdown_connections 2026-02-25T11:49:38.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.587+0000 7f1741787640 1 -- 10.20.193.28:0/2993600661 wait complete. 2026-02-25T11:49:38.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.587+0000 7f1741787640 1 Processor -- start 2026-02-25T11:49:38.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f1741787640 1 -- start start 2026-02-25T11:49:38.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f1741787640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f173c10d420 0x7f173c17d930 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f1741787640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f173c10de10 0x7f173c17de70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f1741787640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f173c113610 0x7f173c154ee0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f1741787640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f173c155420 con 0x7f173c10de10 2026-02-25T11:49:38.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f1741787640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f173c155590 con 0x7f173c10d420 2026-02-25T11:49:38.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f1741787640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f173c155750 con 0x7f173c113610 2026-02-25T11:49:38.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f173b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f173c10de10 0x7f173c17de70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f173bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f173c10d420 0x7f173c17d930 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f173bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f173c10d420 0x7f173c17d930 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:56130/0 (socket says 10.20.193.28:56130) 2026-02-25T11:49:38.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f173bfff640 1 -- 10.20.193.28:0/1479470615 learned_addr learned my addr 10.20.193.28:0/1479470615 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:38.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f1740f86640 1 --2- 10.20.193.28:0/1479470615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f173c113610 0x7f173c154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f173b7fe640 1 -- 10.20.193.28:0/1479470615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f173c113610 msgr2=0x7f173c154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f173b7fe640 1 --2- 10.20.193.28:0/1479470615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f173c113610 0x7f173c154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f173b7fe640 1 -- 10.20.193.28:0/1479470615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f173c10d420 msgr2=0x7f173c17d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f173b7fe640 1 --2- 10.20.193.28:0/1479470615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f173c10d420 0x7f173c17d930 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.588+0000 7f173b7fe640 1 -- 10.20.193.28:0/1479470615 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172c009570 con 0x7f173c10de10 2026-02-25T11:49:38.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.589+0000 7f173b7fe640 1 --2- 10.20.193.28:0/1479470615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f173c10de10 0x7f173c17de70 secure :-1 s=READY pgs=456 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f172c009950 tx=0x7f172c0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:38.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.589+0000 7f17397fa640 1 -- 10.20.193.28:0/1479470615 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f172c03d070 con 0x7f173c10de10 2026-02-25T11:49:38.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.589+0000 7f17397fa640 1 -- 10.20.193.28:0/1479470615 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f172c0043b0 con 0x7f173c10de10 2026-02-25T11:49:38.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.589+0000 7f17397fa640 1 -- 10.20.193.28:0/1479470615 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f172c041690 con 0x7f173c10de10 2026-02-25T11:49:38.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.589+0000 7f1741787640 1 -- 10.20.193.28:0/1479470615 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f173c1559d0 con 0x7f173c10de10 2026-02-25T11:49:38.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.589+0000 7f1741787640 1 -- 10.20.193.28:0/1479470615 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f173c155df0 con 0x7f173c10de10 2026-02-25T11:49:38.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.592+0000 7f17397fa640 1 -- 10.20.193.28:0/1479470615 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f172c0386a0 con 0x7f173c10de10 2026-02-25T11:49:38.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.594+0000 7f17397fa640 1 --2- 10.20.193.28:0/1479470615 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f17100b56e0 0x7f17100d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.594+0000 7f17397fa640 1 -- 10.20.193.28:0/1479470615 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f172c10b1f0 con 0x7f173c10de10 2026-02-25T11:49:38.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.594+0000 7f173bfff640 1 --2- 10.20.193.28:0/1479470615 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f17100b56e0 0x7f17100d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.594+0000 7f173bfff640 1 --2- 10.20.193.28:0/1479470615 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f17100b56e0 0x7f17100d5b90 secure :-1 s=READY pgs=278 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1728005c70 tx=0x7f1728007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:38.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.595+0000 7f1744a0be00 1 Processor -- start 2026-02-25T11:49:38.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.595+0000 7f1744a0be00 1 -- start start 2026-02-25T11:49:38.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.595+0000 7f1744a0be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6ce8cf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.595+0000 7f1744a0be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 0x556b6ced1ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.595+0000 7f1744a0be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 0x556b6cef24c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.595+0000 7f1744a0be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x556b6cda28a0 con 0x556b6ce8d440 2026-02-25T11:49:38.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.595+0000 7f1744a0be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x556b6cd9e5d0 con 0x556b6ce8cb30 2026-02-25T11:49:38.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f1744a0be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x556b6cd98570 con 0x556b6ced20e0 2026-02-25T11:49:38.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f173b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 0x556b6ced1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f173b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 0x556b6ced1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51442/0 (socket says 10.20.193.28:51442) 2026-02-25T11:49:38.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f173bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6ce8cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f173b7fe640 1 -- 10.20.193.28:0/3947069273 learned_addr learned my addr 10.20.193.28:0/3947069273 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:38.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f173b7fe640 1 -- 10.20.193.28:0/3947069273 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 msgr2=0x556b6cef24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f173b7fe640 1 --2- 10.20.193.28:0/3947069273 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 0x556b6cef24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f173b7fe640 1 -- 10.20.193.28:0/3947069273 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 msgr2=0x556b6ce8cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f173b7fe640 1 --2- 10.20.193.28:0/3947069273 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6ce8cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f173b7fe640 1 -- 10.20.193.28:0/3947069273 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6cd43d10 con 0x556b6ce8d440 2026-02-25T11:49:38.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f173b7fe640 1 --2- 10.20.193.28:0/3947069273 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 0x556b6ced1ba0 secure :-1 s=READY pgs=457 gs=2 cs=0 l=1 c_cookie=16813540719409602269 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f172c0cdb40 tx=0x7f172c02f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e955b233ccf99edd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:38.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f170cce3640 1 -- 10.20.193.28:0/3947069273 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f172c0d7070 con 0x556b6ce8d440 2026-02-25T11:49:38.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f170cce3640 1 -- 10.20.193.28:0/3947069273 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f172c02fe20 con 0x556b6ce8d440 2026-02-25T11:49:38.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f170cce3640 1 -- 10.20.193.28:0/3947069273 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f172c0dc220 con 0x556b6ce8d440 2026-02-25T11:49:38.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f1744a0be00 1 -- 10.20.193.28:0/3947069273 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 msgr2=0x556b6ced1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.596+0000 7f1744a0be00 1 --2- 10.20.193.28:0/3947069273 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 0x556b6ced1ba0 secure :-1 s=READY pgs=457 gs=2 cs=0 l=1 c_cookie=16813540719409602269 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f172c0cdb40 tx=0x7f172c02f190 comp rx=0 tx=0).stop 2026-02-25T11:49:38.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.597+0000 7f1744a0be00 1 -- 10.20.193.28:0/3947069273 shutdown_connections 2026-02-25T11:49:38.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.597+0000 7f1744a0be00 1 --2- 10.20.193.28:0/3947069273 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 0x556b6cef24c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.597+0000 7f1744a0be00 1 --2- 10.20.193.28:0/3947069273 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 0x556b6ced1ba0 unknown :-1 s=CLOSED pgs=457 gs=2 cs=0 l=1 c_cookie=16813540719409602269 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.597+0000 7f1744a0be00 1 --2- 10.20.193.28:0/3947069273 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6ce8cf00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.597+0000 7f1744a0be00 1 -- 10.20.193.28:0/3947069273 >> 10.20.193.28:0/3947069273 conn(0x556b6cdab7a0 msgr2=0x556b6ce8b410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:38.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.597+0000 7f1744a0be00 1 -- 10.20.193.28:0/3947069273 shutdown_connections 2026-02-25T11:49:38.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.597+0000 7f1744a0be00 1 -- 10.20.193.28:0/3947069273 wait complete. 2026-02-25T11:49:38.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.598+0000 7f1744a0be00 1 -- start start 2026-02-25T11:49:38.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.598+0000 7f1744a0be00 1 Processor -- start 2026-02-25T11:49:38.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f1744a0be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6cea4d80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f1744a0be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 0x556b6cea52c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f1744a0be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 0x556b6ceaa2b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f1744a0be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x556b6ceaa7f0 con 0x556b6ce8d440 2026-02-25T11:49:38.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f1744a0be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x556b6ceaa960 con 0x556b6ce8cb30 2026-02-25T11:49:38.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f1744a0be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x556b6ceaab20 con 0x556b6ced20e0 2026-02-25T11:49:38.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f173b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 0x556b6cea52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f173b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 0x556b6cea52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51448/0 (socket says 10.20.193.28:51448) 2026-02-25T11:49:38.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f173b7fe640 1 -- 10.20.193.28:0/1565213390 learned_addr learned my addr 10.20.193.28:0/1565213390 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:38.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f173bfff640 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6cea4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f1740f86640 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 0x556b6ceaa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f173b7fe640 1 -- 10.20.193.28:0/1565213390 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 msgr2=0x556b6ceaa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f173b7fe640 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 0x556b6ceaa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f173b7fe640 1 -- 10.20.193.28:0/1565213390 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 msgr2=0x556b6cea4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f173b7fe640 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6cea4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f173b7fe640 1 -- 10.20.193.28:0/1565213390 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6cd1c3c0 con 0x556b6ce8d440 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f1740f86640 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 0x556b6ceaa2b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f173b7fe640 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 0x556b6cea52c0 secure :-1 s=READY pgs=458 gs=1 cs=0 l=1 c_cookie=4012406912817054130 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f172c02ffb0 tx=0x7f172c0d7140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37aeeed41cc841b2 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.599+0000 7f16fdffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f16fdffb640 1 -- 10.20.193.28:0/1565213390 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f172c096070 con 0x556b6ce8d440 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f16fdffb640 1 -- 10.20.193.28:0/1565213390 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f172c0d7290 con 0x556b6ce8d440 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f16fdffb640 1 -- 10.20.193.28:0/1565213390 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f172c0cf7d0 con 0x556b6ce8d440 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f1744a0be00 1 -- 10.20.193.28:0/1565213390 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f173c155df0 con 0x556b6ce8d440 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f1744a0be00 10 client.15819 _ll_register_callbacks cb 0x556b6cd19a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f1744a0be00 1 -- 10.20.193.28:0/1565213390 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b6ca2e6f0 con 0x556b6ce8d440 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f16fdffb640 1 -- 10.20.193.28:0/1565213390 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f172c0d2f60 con 0x556b6ce8d440 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f16fdffb640 1 -- 10.20.193.28:0/1565213390 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f172c0d31a0 con 0x556b6ce8d440 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f1744a0be00 10 client.15819 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f1744a0be00 1 -- 10.20.193.28:0/1565213390 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x556b6cd2e0e0 con 0x556b6ce8d440 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f16fdffb640 1 -- 10.20.193.28:0/1565213390 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f172c0f2070 con 0x556b6ce8d440 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f1744a0be00 10 client.15819 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f1744a0be00 10 client.15819 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.600+0000 7f1744a0be00 20 client.15819 populate_metadata read hostname 'trial028' 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.601+0000 7f1744a0be00 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x556b6cecf140 0x556b6cecf5b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.601+0000 7f1744a0be00 4 client.15819 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.601+0000 7f1744a0be00 1 -- 10.20.193.28:0/1565213390 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b6cd6afc0 con 0x556b6cecf140 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.601+0000 7f173bfff640 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x556b6cecf140 0x556b6cecf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.601+0000 7f173bfff640 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x556b6cecf140 0x556b6cecf5b0 crc :-1 s=READY pgs=30 gs=4 cs=0 l=0 c_cookie=14969655392182124861 s_cookie=9913182479395284306 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cfbee6443d06c13d server_cookie=8992b2f155fb6552 in_seq=0 out_seq=0 2026-02-25T11:49:38.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.601+0000 7f16fdffb640 10 client.15819 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f16fdffb640 1 -- 10.20.193.28:0/1565213390 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x556b6cd6afc0 con 0x556b6cecf140 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f16fdffb640 10 client.15819 handle_command_reply: tid=0 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f1744a0be00 1 client.15819 shutdown 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f1744a0be00 1 -- 10.20.193.28:0/1565213390 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 msgr2=0x556b6cea52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f1744a0be00 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 0x556b6cea52c0 secure :-1 s=READY pgs=458 gs=1 cs=0 l=1 c_cookie=4012406912817054130 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f172c02ffb0 tx=0x7f172c0d7140 comp rx=0 tx=0).stop 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f1744a0be00 1 -- 10.20.193.28:0/1565213390 shutdown_connections 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f1744a0be00 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x556b6cecf140 0x556b6cecf5b0 crc :-1 s=READY pgs=30 gs=4 cs=0 l=0 c_cookie=14969655392182124861 s_cookie=9913182479395284306 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f1744a0be00 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 0x556b6ceaa2b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f1744a0be00 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8d440 0x556b6cea52c0 unknown :-1 s=CLOSED pgs=458 gs=1 cs=0 l=1 c_cookie=4012406912817054130 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f1744a0be00 1 --2- 10.20.193.28:0/1565213390 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6cea4d80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f16fdffb640 0 client.15819 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f1744a0be00 1 -- 10.20.193.28:0/1565213390 >> 10.20.193.28:0/1565213390 conn(0x556b6cdab7a0 msgr2=0x556b6ce8bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f1744a0be00 1 -- 10.20.193.28:0/1565213390 shutdown_connections 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f1744a0be00 1 -- 10.20.193.28:0/1565213390 wait complete. 2026-02-25T11:49:38.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.602+0000 7f1744a0be00 20 client.15819 trim_cache size 0 max 16384 2026-02-25T11:49:38.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f1744a0be00 1 Processor -- start 2026-02-25T11:49:38.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f1744a0be00 1 -- start start 2026-02-25T11:49:38.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f1744a0be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6cea5fd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f1744a0be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ce8d440 0x556b6cea6510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f1744a0be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ced20e0 0x556b6cea6a50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f1744a0be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x556b6cd961a0 con 0x556b6ced20e0 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f1744a0be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x556b6ca56700 con 0x556b6ce8cb30 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f1744a0be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x556b6cd9e5d0 con 0x556b6ce8d440 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f173bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6cea5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f173bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6cea5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:56164/0 (socket says 10.20.193.28:56164) 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f173bfff640 1 -- 10.20.193.28:0/1118517658 learned_addr learned my addr 10.20.193.28:0/1118517658 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f173b7fe640 1 --2- 10.20.193.28:0/1118517658 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ce8d440 0x556b6cea6510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f1740f86640 1 --2- 10.20.193.28:0/1118517658 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ced20e0 0x556b6cea6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f173bfff640 1 -- 10.20.193.28:0/1118517658 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ce8d440 msgr2=0x556b6cea6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.610+0000 7f173bfff640 1 --2- 10.20.193.28:0/1118517658 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ce8d440 0x556b6cea6510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f173bfff640 1 -- 10.20.193.28:0/1118517658 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ced20e0 msgr2=0x556b6cea6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f173bfff640 1 --2- 10.20.193.28:0/1118517658 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ced20e0 0x556b6cea6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f173bfff640 1 -- 10.20.193.28:0/1118517658 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6cd28b60 con 0x556b6ce8cb30 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f1740f86640 1 --2- 10.20.193.28:0/1118517658 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ced20e0 0x556b6cea6a50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f173bfff640 1 --2- 10.20.193.28:0/1118517658 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6cea5fd0 secure :-1 s=READY pgs=119 gs=1 cs=0 l=1 c_cookie=16705552926826238038 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1728007fd0 tx=0x7f1728013070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7d60be08bf71056 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f16fffff640 1 -- 10.20.193.28:0/1118517658 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1728011e10 con 0x556b6ce8cb30 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f16fffff640 1 -- 10.20.193.28:0/1118517658 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1728023040 con 0x556b6ce8cb30 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f16fffff640 1 -- 10.20.193.28:0/1118517658 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f172801da40 con 0x556b6ce8cb30 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f1744a0be00 1 -- 10.20.193.28:0/1118517658 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 msgr2=0x556b6cea5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f1744a0be00 1 --2- 10.20.193.28:0/1118517658 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6cea5fd0 secure :-1 s=READY pgs=119 gs=1 cs=0 l=1 c_cookie=16705552926826238038 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1728007fd0 tx=0x7f1728013070 comp rx=0 tx=0).stop 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f1744a0be00 1 -- 10.20.193.28:0/1118517658 shutdown_connections 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f1744a0be00 1 --2- 10.20.193.28:0/1118517658 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ced20e0 0x556b6cea6a50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f1744a0be00 1 --2- 10.20.193.28:0/1118517658 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ce8d440 0x556b6cea6510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f1744a0be00 1 --2- 10.20.193.28:0/1118517658 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8cb30 0x556b6cea5fd0 unknown :-1 s=CLOSED pgs=119 gs=1 cs=0 l=1 c_cookie=16705552926826238038 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.611+0000 7f1744a0be00 1 -- 10.20.193.28:0/1118517658 >> 10.20.193.28:0/1118517658 conn(0x556b6cdab7a0 msgr2=0x556b6ce8ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:38.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.612+0000 7f1744a0be00 1 -- 10.20.193.28:0/1118517658 shutdown_connections 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.612+0000 7f1744a0be00 1 -- 10.20.193.28:0/1118517658 wait complete. 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.612+0000 7f1744a0be00 1 -- start start 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.612+0000 7f1744a0be00 1 Processor -- start 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f1744a0be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8cb30 0x556b6ceae1d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f1744a0be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8d440 0x556b6ceae730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f1744a0be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 0x556b6cf37940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f1744a0be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x556b6cecf740 con 0x556b6ce8cb30 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f1744a0be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x556b6cf37f00 con 0x556b6ce8d440 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f1744a0be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x556b6cf380c0 con 0x556b6ced20e0 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f173bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8cb30 0x556b6ceae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f173bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8cb30 0x556b6ceae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51462/0 (socket says 10.20.193.28:51462) 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f173bfff640 1 -- 10.20.193.28:0/2190098641 learned_addr learned my addr 10.20.193.28:0/2190098641 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f173b7fe640 1 --2- 10.20.193.28:0/2190098641 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8d440 0x556b6ceae730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f1740f86640 1 --2- 10.20.193.28:0/2190098641 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 0x556b6cf37940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f173bfff640 1 -- 10.20.193.28:0/2190098641 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 msgr2=0x556b6cf37940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f173bfff640 1 --2- 10.20.193.28:0/2190098641 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 0x556b6cf37940 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f173bfff640 1 -- 10.20.193.28:0/2190098641 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8d440 msgr2=0x556b6ceae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f173bfff640 1 --2- 10.20.193.28:0/2190098641 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8d440 0x556b6ceae730 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f173bfff640 1 -- 10.20.193.28:0/2190098641 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b6cd1c6f0 con 0x556b6ce8cb30 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f173bfff640 1 --2- 10.20.193.28:0/2190098641 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8cb30 0x556b6ceae1d0 secure :-1 s=READY pgs=459 gs=1 cs=0 l=1 c_cookie=5152791507885481238 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f17280059d0 tx=0x7f17280057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478264936529dd16 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.613+0000 7f16fdffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f16fdffb640 1 -- 10.20.193.28:0/2190098641 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f172801d7a0 con 0x556b6ce8cb30 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f16fdffb640 1 -- 10.20.193.28:0/2190098641 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1728015070 con 0x556b6ce8cb30 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f1744a0be00 1 -- 10.20.193.28:0/2190098641 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1728011640 con 0x556b6ce8cb30 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f16fdffb640 1 -- 10.20.193.28:0/2190098641 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f172802bcb0 con 0x556b6ce8cb30 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f1744a0be00 10 client.15825 _ll_register_callbacks cb 0x556b6cd19a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f1744a0be00 1 -- 10.20.193.28:0/2190098641 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f172c02fe20 con 0x556b6ce8cb30 2026-02-25T11:49:38.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f16fdffb640 1 -- 10.20.193.28:0/2190098641 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1728034460 con 0x556b6ce8cb30 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f16fdffb640 1 -- 10.20.193.28:0/2190098641 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1728034640 con 0x556b6ce8cb30 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f1744a0be00 10 client.15825 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f1744a0be00 1 -- 10.20.193.28:0/2190098641 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x556b6cd2e340 con 0x556b6ce8cb30 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f16fdffb640 1 -- 10.20.193.28:0/2190098641 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f1728045060 con 0x556b6ce8cb30 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f1744a0be00 10 client.15825 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f1744a0be00 10 client.15825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f1744a0be00 20 client.15825 populate_metadata read hostname 'trial028' 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f1744a0be00 1 --2- 10.20.193.28:0/2190098641 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x556b6cf7f0b0 0x556b6cf9f560 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f1744a0be00 4 client.15825 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f1744a0be00 1 -- 10.20.193.28:0/2190098641 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b6ca2bf40 con 0x556b6cf7f0b0 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.614+0000 7f173b7fe640 1 --2- 10.20.193.28:0/2190098641 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x556b6cf7f0b0 0x556b6cf9f560 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f16fdffb640 10 client.15825 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f173b7fe640 1 --2- 10.20.193.28:0/2190098641 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x556b6cf7f0b0 0x556b6cf9f560 crc :-1 s=READY pgs=31 gs=4 cs=0 l=0 c_cookie=14867809888984802774 s_cookie=17460709688572454804 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce5512522cfaa5d6 server_cookie=f250e6f371952794 in_seq=0 out_seq=0 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f16fdffb640 1 -- 10.20.193.28:0/2190098641 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b6ca2bf40 con 0x556b6cf7f0b0 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f16fdffb640 10 client.15825 handle_command_reply: tid=0 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f1744a0be00 1 client.15825 shutdown 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f1744a0be00 1 -- 10.20.193.28:0/2190098641 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8cb30 msgr2=0x556b6ceae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f1744a0be00 1 --2- 10.20.193.28:0/2190098641 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8cb30 0x556b6ceae1d0 secure :-1 s=READY pgs=459 gs=1 cs=0 l=1 c_cookie=5152791507885481238 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f17280059d0 tx=0x7f17280057a0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f1744a0be00 1 -- 10.20.193.28:0/2190098641 shutdown_connections 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f1744a0be00 1 --2- 10.20.193.28:0/2190098641 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x556b6cf7f0b0 0x556b6cf9f560 crc :-1 s=READY pgs=31 gs=4 cs=0 l=0 c_cookie=14867809888984802774 s_cookie=17460709688572454804 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f1744a0be00 1 --2- 10.20.193.28:0/2190098641 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556b6ced20e0 0x556b6cf37940 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f1744a0be00 1 --2- 10.20.193.28:0/2190098641 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556b6ce8d440 0x556b6ceae730 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f1744a0be00 1 --2- 10.20.193.28:0/2190098641 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556b6ce8cb30 0x556b6ceae1d0 unknown :-1 s=CLOSED pgs=459 gs=1 cs=0 l=1 c_cookie=5152791507885481238 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f1744a0be00 1 -- 10.20.193.28:0/2190098641 >> 10.20.193.28:0/2190098641 conn(0x556b6cdab7a0 msgr2=0x556b6cea6750 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f16fdffb640 0 client.15825 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f1744a0be00 1 -- 10.20.193.28:0/2190098641 shutdown_connections 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.615+0000 7f1744a0be00 1 -- 10.20.193.28:0/2190098641 wait complete. 2026-02-25T11:49:38.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.616+0000 7f1744a0be00 20 client.15825 trim_cache size 0 max 16384 2026-02-25T11:49:38.622 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:49:38.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.616+0000 7f1741787640 1 -- 10.20.193.28:0/1479470615 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f17100b56e0 msgr2=0x7f17100d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.616+0000 7f1741787640 1 --2- 10.20.193.28:0/1479470615 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f17100b56e0 0x7f17100d5b90 secure :-1 s=READY pgs=278 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1728005c70 tx=0x7f1728007480 comp rx=0 tx=0).stop 2026-02-25T11:49:38.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.616+0000 7f1741787640 1 -- 10.20.193.28:0/1479470615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f173c10de10 msgr2=0x7f173c17de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:38.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.616+0000 7f1741787640 1 --2- 10.20.193.28:0/1479470615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f173c10de10 0x7f173c17de70 secure :-1 s=READY pgs=456 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f172c009950 tx=0x7f172c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.616+0000 7f1741787640 1 -- 10.20.193.28:0/1479470615 shutdown_connections 2026-02-25T11:49:38.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.616+0000 7f1741787640 1 --2- 10.20.193.28:0/1479470615 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f17100b56e0 0x7f17100d5b90 unknown :-1 s=CLOSED pgs=278 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.616+0000 7f1741787640 1 --2- 10.20.193.28:0/1479470615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f173c113610 0x7f173c154ee0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.616+0000 7f1741787640 1 --2- 10.20.193.28:0/1479470615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f173c10de10 0x7f173c17de70 unknown :-1 s=CLOSED pgs=456 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.616+0000 7f1741787640 1 --2- 10.20.193.28:0/1479470615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f173c10d420 0x7f173c17d930 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:38.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.616+0000 7f1741787640 1 -- 10.20.193.28:0/1479470615 >> 10.20.193.28:0/1479470615 conn(0x7f173c0eacb0 msgr2=0x7f173c10b090 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:38.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.616+0000 7f1741787640 1 -- 10.20.193.28:0/1479470615 shutdown_connections 2026-02-25T11:49:38.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:38.616+0000 7f1741787640 1 -- 10.20.193.28:0/1479470615 wait complete. 2026-02-25T11:49:38.752 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:49:38.936 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:39.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.105+0000 7f2407fff640 1 -- 10.20.193.28:0/6630590 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f24081140a0 msgr2=0x7f2408114470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:39.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.105+0000 7f2407fff640 1 --2- 10.20.193.28:0/6630590 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f24081140a0 0x7f2408114470 secure :-1 s=READY pgs=460 gs=1 cs=0 l=1 c_cookie=8815588606838395371 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f23fc009950 tx=0x7f23fc02f190 comp rx=0 tx=0).stop 2026-02-25T11:49:39.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.105+0000 7f2407fff640 1 -- 10.20.193.28:0/6630590 shutdown_connections 2026-02-25T11:49:39.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.105+0000 7f2407fff640 1 --2- 10.20.193.28:0/6630590 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2408119d30 0x7f240817a700 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.105+0000 7f2407fff640 1 --2- 10.20.193.28:0/6630590 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2408114a90 0x7f2408119580 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.105+0000 7f2407fff640 1 --2- 10.20.193.28:0/6630590 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f24081140a0 0x7f2408114470 unknown :-1 s=CLOSED pgs=460 gs=1 cs=0 l=1 c_cookie=8815588606838395371 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.105+0000 7f2407fff640 1 -- 10.20.193.28:0/6630590 >> 10.20.193.28:0/6630590 conn(0x7f2408064c40 msgr2=0x7f2408065040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:39.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.105+0000 7f2407fff640 1 -- 10.20.193.28:0/6630590 shutdown_connections 2026-02-25T11:49:39.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.105+0000 7f2407fff640 1 -- 10.20.193.28:0/6630590 wait complete. 2026-02-25T11:49:39.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.106+0000 7f2407fff640 1 Processor -- start 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.106+0000 7f2407fff640 1 -- start start 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.106+0000 7f2407fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f24081140a0 0x7f24081225a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.106+0000 7f2407fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2408114a90 0x7f2408122ae0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.106+0000 7f2407fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2408119d30 0x7f2408127eb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.106+0000 7f2407fff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f24081284d0 con 0x7f24081140a0 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.106+0000 7f2407fff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2408128640 con 0x7f2408119d30 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.106+0000 7f2407fff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2408128800 con 0x7f2408114a90 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f2406ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f24081140a0 0x7f24081225a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f2406ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f24081140a0 0x7f24081225a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51484/0 (socket says 10.20.193.28:51484) 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f2406ffd640 1 -- 10.20.193.28:0/3349347002 learned_addr learned my addr 10.20.193.28:0/3349347002 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f24067fc640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2408114a90 0x7f2408122ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f24077fe640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2408119d30 0x7f2408127eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f2406ffd640 1 -- 10.20.193.28:0/3349347002 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2408114a90 msgr2=0x7f2408122ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:39.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f2406ffd640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2408114a90 0x7f2408122ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f2406ffd640 1 -- 10.20.193.28:0/3349347002 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2408119d30 msgr2=0x7f2408127eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:39.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f2406ffd640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2408119d30 0x7f2408127eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f2406ffd640 1 -- 10.20.193.28:0/3349347002 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc009570 con 0x7f24081140a0 2026-02-25T11:49:39.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f24077fe640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2408119d30 0x7f2408127eb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:39.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f2406ffd640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f24081140a0 0x7f24081225a0 secure :-1 s=READY pgs=461 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f23fc0098a0 tx=0x7f23fc0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:39.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f23e7fff640 1 -- 10.20.193.28:0/3349347002 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f23fc03d070 con 0x7f24081140a0 2026-02-25T11:49:39.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f23e7fff640 1 -- 10.20.193.28:0/3349347002 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f23fc002a70 con 0x7f24081140a0 2026-02-25T11:49:39.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f23e7fff640 1 -- 10.20.193.28:0/3349347002 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f23fc041890 con 0x7f24081140a0 2026-02-25T11:49:39.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.107+0000 7f2407fff640 1 -- 10.20.193.28:0/3349347002 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2408128a80 con 0x7f24081140a0 2026-02-25T11:49:39.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.108+0000 7f2407fff640 1 -- 10.20.193.28:0/3349347002 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2408169040 con 0x7f24081140a0 2026-02-25T11:49:39.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.111+0000 7f23e7fff640 1 -- 10.20.193.28:0/3349347002 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f23fc041a20 con 0x7f24081140a0 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.112+0000 7f23e7fff640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f23dc0b5760 0x7f23dc0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.112+0000 7f23e7fff640 1 -- 10.20.193.28:0/3349347002 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f23fc10b130 con 0x7f24081140a0 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.112+0000 7f24067fc640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f23dc0b5760 0x7f23dc0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.113+0000 7f24067fc640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f23dc0b5760 0x7f23dc0d5c10 secure :-1 s=READY pgs=279 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2408128f40 tx=0x7f23f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.114+0000 7f240f55ee00 1 Processor -- start 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.114+0000 7f240f55ee00 1 -- start start 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f240f55ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa20b30 0x5619aaa20f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f240f55ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa21440 0x5619aaa65ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f240f55ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa660e0 0x5619aaa864c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f240f55ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5619aa9368a0 con 0x5619aaa660e0 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f240f55ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5619aa9325d0 con 0x5619aaa21440 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f240f55ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5619aa92c570 con 0x5619aaa20b30 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f2406ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa20b30 0x5619aaa20f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f2406ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa20b30 0x5619aaa20f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:53016/0 (socket says 10.20.193.28:53016) 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f2406ffd640 1 -- 10.20.193.28:0/3453071520 learned_addr learned my addr 10.20.193.28:0/3453071520 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f24067fc640 1 --2- 10.20.193.28:0/3453071520 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa21440 0x5619aaa65ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f24077fe640 1 --2- 10.20.193.28:0/3453071520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa660e0 0x5619aaa864c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f2406ffd640 1 -- 10.20.193.28:0/3453071520 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa21440 msgr2=0x5619aaa65ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f2406ffd640 1 --2- 10.20.193.28:0/3453071520 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa21440 0x5619aaa65ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f2406ffd640 1 -- 10.20.193.28:0/3453071520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa660e0 msgr2=0x5619aaa864c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f2406ffd640 1 --2- 10.20.193.28:0/3453071520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa660e0 0x5619aaa864c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.115+0000 7f2406ffd640 1 -- 10.20.193.28:0/3453071520 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619aa8d7d10 con 0x5619aaa20b30 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.116+0000 7f2406ffd640 1 --2- 10.20.193.28:0/3453071520 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa20b30 0x5619aaa20f00 secure :-1 s=READY pgs=133 gs=1 cs=0 l=1 c_cookie=17325697784646882027 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f23fc108c20 tx=0x7f23fc005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0713e56f3a652eb server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.116+0000 7f23d37fe640 1 -- 10.20.193.28:0/3453071520 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f23fc041d00 con 0x5619aaa20b30 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.116+0000 7f23d37fe640 1 -- 10.20.193.28:0/3453071520 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f23fc0d1eb0 con 0x5619aaa20b30 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.116+0000 7f23d37fe640 1 -- 10.20.193.28:0/3453071520 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f23fc0dc3e0 con 0x5619aaa20b30 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.116+0000 7f240f55ee00 1 -- 10.20.193.28:0/3453071520 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa20b30 msgr2=0x5619aaa20f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.116+0000 7f240f55ee00 1 --2- 10.20.193.28:0/3453071520 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa20b30 0x5619aaa20f00 secure :-1 s=READY pgs=133 gs=1 cs=0 l=1 c_cookie=17325697784646882027 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f23fc108c20 tx=0x7f23fc005ab0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.116+0000 7f240f55ee00 1 -- 10.20.193.28:0/3453071520 shutdown_connections 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.116+0000 7f240f55ee00 1 --2- 10.20.193.28:0/3453071520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa660e0 0x5619aaa864c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.116+0000 7f240f55ee00 1 --2- 10.20.193.28:0/3453071520 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa21440 0x5619aaa65ba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.116+0000 7f240f55ee00 1 --2- 10.20.193.28:0/3453071520 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa20b30 0x5619aaa20f00 unknown :-1 s=CLOSED pgs=133 gs=1 cs=0 l=1 c_cookie=17325697784646882027 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.116+0000 7f240f55ee00 1 -- 10.20.193.28:0/3453071520 >> 10.20.193.28:0/3453071520 conn(0x5619aa93f7a0 msgr2=0x5619aaa1f410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.116+0000 7f240f55ee00 1 -- 10.20.193.28:0/3453071520 shutdown_connections 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.116+0000 7f240f55ee00 1 -- 10.20.193.28:0/3453071520 wait complete. 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.118+0000 7f240f55ee00 1 -- start start 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.118+0000 7f240f55ee00 1 Processor -- start 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.118+0000 7f240f55ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 0x5619aaa38d10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.118+0000 7f240f55ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa21440 0x5619aaa39250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.118+0000 7f240f55ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa660e0 0x5619aaa3e240 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.118+0000 7f240f55ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5619aaa3e780 con 0x5619aaa20b30 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.118+0000 7f240f55ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5619aaa3e8f0 con 0x5619aaa660e0 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.118+0000 7f240f55ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5619aaa3eab0 con 0x5619aaa21440 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.118+0000 7f2406ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 0x5619aaa38d10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.118+0000 7f2406ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 0x5619aaa38d10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51492/0 (socket says 10.20.193.28:51492) 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.118+0000 7f2406ffd640 1 -- 10.20.193.28:0/1800232781 learned_addr learned my addr 10.20.193.28:0/1800232781 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:39.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f24067fc640 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa21440 0x5619aaa39250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f24077fe640 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa660e0 0x5619aaa3e240 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f2406ffd640 1 -- 10.20.193.28:0/1800232781 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa21440 msgr2=0x5619aaa39250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f2406ffd640 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa21440 0x5619aaa39250 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f2406ffd640 1 -- 10.20.193.28:0/1800232781 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa660e0 msgr2=0x5619aaa3e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f2406ffd640 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa660e0 0x5619aaa3e240 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f2406ffd640 1 -- 10.20.193.28:0/1800232781 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619aa8b03c0 con 0x5619aaa20b30 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f24077fe640 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa660e0 0x5619aaa3e240 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f2406ffd640 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 0x5619aaa38d10 secure :-1 s=READY pgs=462 gs=2 cs=0 l=1 c_cookie=5792793698948598966 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f23fc0d1490 tx=0x7f23fc02f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5064232d7af3ccb6 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f23d0ff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f23d0ff9640 1 -- 10.20.193.28:0/1800232781 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f23fc0d71d0 con 0x5619aaa20b30 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f23d0ff9640 1 -- 10.20.193.28:0/1800232781 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f23fc0d97b0 con 0x5619aaa20b30 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f23d0ff9640 1 -- 10.20.193.28:0/1800232781 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f23fc0d2ea0 con 0x5619aaa20b30 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f240f55ee00 1 -- 10.20.193.28:0/1800232781 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2408169040 con 0x5619aaa20b30 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f23d0ff9640 1 -- 10.20.193.28:0/1800232781 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f23fc0d3000 con 0x5619aaa20b30 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f23d0ff9640 10 client.15837 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f240f55ee00 10 client.15837 _ll_register_callbacks cb 0x5619aa8ada10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.119+0000 7f240f55ee00 1 -- 10.20.193.28:0/1800232781 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619aa5c26f0 con 0x5619aaa20b30 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.120+0000 7f23d0ff9640 1 -- 10.20.193.28:0/1800232781 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f23fc0dc5f0 con 0x5619aaa20b30 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.120+0000 7f240f55ee00 10 client.15837 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.120+0000 7f240f55ee00 1 -- 10.20.193.28:0/1800232781 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5619aa8c20e0 con 0x5619aaa20b30 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.120+0000 7f23d0ff9640 1 -- 10.20.193.28:0/1800232781 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f23fc0d2140 con 0x5619aaa20b30 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.120+0000 7f240f55ee00 10 client.15837 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.120+0000 7f240f55ee00 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.120+0000 7f240f55ee00 20 client.15837 populate_metadata read hostname 'trial028' 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.120+0000 7f240f55ee00 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5619aaa62f40 0x5619aaa633b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.120+0000 7f240f55ee00 4 client.15837 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.120+0000 7f240f55ee00 1 -- 10.20.193.28:0/1800232781 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619aa8fefc0 con 0x5619aaa62f40 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.120+0000 7f24067fc640 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5619aaa62f40 0x5619aaa633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.121+0000 7f24067fc640 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5619aaa62f40 0x5619aaa633b0 crc :-1 s=READY pgs=52 gs=4 cs=0 l=0 c_cookie=7173316014524547856 s_cookie=1662349327087636065 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=638cbce8cb5b4310 server_cookie=1711d9e7cdab0e61 in_seq=0 out_seq=0 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.121+0000 7f23d0ff9640 10 client.15837 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.121+0000 7f23d0ff9640 1 -- 10.20.193.28:0/1800232781 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5619aa8fefc0 con 0x5619aaa62f40 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.121+0000 7f23d0ff9640 10 client.15837 handle_command_reply: tid=0 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.122+0000 7f240f55ee00 1 client.15837 shutdown 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.122+0000 7f240f55ee00 1 -- 10.20.193.28:0/1800232781 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 msgr2=0x5619aaa38d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.122+0000 7f240f55ee00 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 0x5619aaa38d10 secure :-1 s=READY pgs=462 gs=2 cs=0 l=1 c_cookie=5792793698948598966 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f23fc0d1490 tx=0x7f23fc02f190 comp rx=0 tx=0).stop 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.122+0000 7f240f55ee00 1 -- 10.20.193.28:0/1800232781 shutdown_connections 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.122+0000 7f240f55ee00 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5619aaa62f40 0x5619aaa633b0 crc :-1 s=READY pgs=52 gs=4 cs=0 l=0 c_cookie=7173316014524547856 s_cookie=1662349327087636065 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.122+0000 7f240f55ee00 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa660e0 0x5619aaa3e240 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.122+0000 7f240f55ee00 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa21440 0x5619aaa39250 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.122+0000 7f240f55ee00 1 --2- 10.20.193.28:0/1800232781 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 0x5619aaa38d10 unknown :-1 s=CLOSED pgs=462 gs=2 cs=0 l=1 c_cookie=5792793698948598966 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.122+0000 7f240f55ee00 1 -- 10.20.193.28:0/1800232781 >> 10.20.193.28:0/1800232781 conn(0x5619aa93f7a0 msgr2=0x5619aaa1fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.122+0000 7f23d0ff9640 0 client.15837 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.122+0000 7f240f55ee00 1 -- 10.20.193.28:0/1800232781 shutdown_connections 2026-02-25T11:49:39.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.122+0000 7f240f55ee00 1 -- 10.20.193.28:0/1800232781 wait complete. 2026-02-25T11:49:39.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.122+0000 7f240f55ee00 20 client.15837 trim_cache size 0 max 16384 2026-02-25T11:49:39.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f240f55ee00 1 Processor -- start 2026-02-25T11:49:39.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f240f55ee00 1 -- start start 2026-02-25T11:49:39.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f240f55ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa20b30 0x5619aaa2ae40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f240f55ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa21440 0x5619aaa36100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f240f55ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa660e0 0x5619aaa36640 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f240f55ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5619aa92a1a0 con 0x5619aaa21440 2026-02-25T11:49:39.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f240f55ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5619aa5ea700 con 0x5619aaa20b30 2026-02-25T11:49:39.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f240f55ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5619aa92c570 con 0x5619aaa660e0 2026-02-25T11:49:39.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f24067fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa21440 0x5619aaa36100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f24067fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa21440 0x5619aaa36100 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51504/0 (socket says 10.20.193.28:51504) 2026-02-25T11:49:39.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f24067fc640 1 -- 10.20.193.28:0/529891011 learned_addr learned my addr 10.20.193.28:0/529891011 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:39.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f2406ffd640 1 --2- 10.20.193.28:0/529891011 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa20b30 0x5619aaa2ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f24077fe640 1 --2- 10.20.193.28:0/529891011 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa660e0 0x5619aaa36640 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f24067fc640 1 -- 10.20.193.28:0/529891011 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa660e0 msgr2=0x5619aaa36640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:39.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f24067fc640 1 --2- 10.20.193.28:0/529891011 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa660e0 0x5619aaa36640 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f24067fc640 1 -- 10.20.193.28:0/529891011 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa20b30 msgr2=0x5619aaa2ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:39.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f24067fc640 1 --2- 10.20.193.28:0/529891011 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa20b30 0x5619aaa2ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.130+0000 7f24067fc640 1 -- 10.20.193.28:0/529891011 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619aaa2b610 con 0x5619aaa21440 2026-02-25T11:49:39.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.131+0000 7f24067fc640 1 --2- 10.20.193.28:0/529891011 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa21440 0x5619aaa36100 secure :-1 s=READY pgs=463 gs=1 cs=0 l=1 c_cookie=15429846502714378453 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f23f000b7a0 tx=0x7f23f0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d621d3a4b51808d5 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:39.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.131+0000 7f23d2ffd640 1 -- 10.20.193.28:0/529891011 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f23f0024050 con 0x5619aaa21440 2026-02-25T11:49:39.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.131+0000 7f23d2ffd640 1 -- 10.20.193.28:0/529891011 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f23f0025040 con 0x5619aaa21440 2026-02-25T11:49:39.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.131+0000 7f23d2ffd640 1 -- 10.20.193.28:0/529891011 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f23f002a320 con 0x5619aaa21440 2026-02-25T11:49:39.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.131+0000 7f240f55ee00 1 -- 10.20.193.28:0/529891011 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa21440 msgr2=0x5619aaa36100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:39.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.131+0000 7f240f55ee00 1 --2- 10.20.193.28:0/529891011 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa21440 0x5619aaa36100 secure :-1 s=READY pgs=463 gs=1 cs=0 l=1 c_cookie=15429846502714378453 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f23f000b7a0 tx=0x7f23f0013000 comp rx=0 tx=0).stop 2026-02-25T11:49:39.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.131+0000 7f240f55ee00 1 -- 10.20.193.28:0/529891011 shutdown_connections 2026-02-25T11:49:39.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.131+0000 7f240f55ee00 1 --2- 10.20.193.28:0/529891011 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa660e0 0x5619aaa36640 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.131+0000 7f240f55ee00 1 --2- 10.20.193.28:0/529891011 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa21440 0x5619aaa36100 unknown :-1 s=CLOSED pgs=463 gs=1 cs=0 l=1 c_cookie=15429846502714378453 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.131+0000 7f240f55ee00 1 --2- 10.20.193.28:0/529891011 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa20b30 0x5619aaa2ae40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.131+0000 7f240f55ee00 1 -- 10.20.193.28:0/529891011 >> 10.20.193.28:0/529891011 conn(0x5619aa93f7a0 msgr2=0x5619aaa3d890 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:39.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.131+0000 7f240f55ee00 1 -- 10.20.193.28:0/529891011 shutdown_connections 2026-02-25T11:49:39.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.131+0000 7f240f55ee00 1 -- 10.20.193.28:0/529891011 wait complete. 2026-02-25T11:49:39.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.132+0000 7f240f55ee00 1 -- start start 2026-02-25T11:49:39.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.132+0000 7f240f55ee00 1 Processor -- start 2026-02-25T11:49:39.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.132+0000 7f240f55ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 0x5619aaa36570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.132+0000 7f240f55ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa21440 0x5619aaa3f130 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.132+0000 7f240f55ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa660e0 0x5619aaa3f670 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.132+0000 7f240f55ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5619aa9368a0 con 0x5619aaa20b30 2026-02-25T11:49:39.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.132+0000 7f240f55ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5619aaa3fbb0 con 0x5619aaa660e0 2026-02-25T11:49:39.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.132+0000 7f240f55ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5619aaa3fd40 con 0x5619aaa21440 2026-02-25T11:49:39.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f2406ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 0x5619aaa36570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f2406ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 0x5619aaa36570 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51514/0 (socket says 10.20.193.28:51514) 2026-02-25T11:49:39.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f24067fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa21440 0x5619aaa3f130 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f2406ffd640 1 -- 10.20.193.28:0/584196534 learned_addr learned my addr 10.20.193.28:0/584196534 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:39.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f2406ffd640 1 -- 10.20.193.28:0/584196534 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa21440 msgr2=0x5619aaa3f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:39.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f2406ffd640 1 --2- 10.20.193.28:0/584196534 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa21440 0x5619aaa3f130 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f2406ffd640 1 -- 10.20.193.28:0/584196534 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa660e0 msgr2=0x5619aaa3f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:39.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f2406ffd640 1 --2- 10.20.193.28:0/584196534 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa660e0 0x5619aaa3f670 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f2406ffd640 1 -- 10.20.193.28:0/584196534 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619aa8bcb60 con 0x5619aaa20b30 2026-02-25T11:49:39.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f2406ffd640 1 --2- 10.20.193.28:0/584196534 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 0x5619aaa36570 secure :-1 s=READY pgs=464 gs=1 cs=0 l=1 c_cookie=18328930667683190943 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f23fc0d1720 tx=0x7f23fc038f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5d7154d805189f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:39.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f23d0ff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:39.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f23d0ff9640 1 -- 10.20.193.28:0/584196534 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f23fc0d71d0 con 0x5619aaa20b30 2026-02-25T11:49:39.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f23d0ff9640 1 -- 10.20.193.28:0/584196534 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5619aa5c26f0 con 0x5619aaa20b30 2026-02-25T11:49:39.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f240f55ee00 1 -- 10.20.193.28:0/584196534 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f00115b0 con 0x5619aaa20b30 2026-02-25T11:49:39.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f23d0ff9640 1 -- 10.20.193.28:0/584196534 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f23fc0cf710 con 0x5619aaa20b30 2026-02-25T11:49:39.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f240f55ee00 10 client.15843 _ll_register_callbacks cb 0x5619aa8ada10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:39.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f240f55ee00 1 -- 10.20.193.28:0/584196534 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23fc0d1eb0 con 0x5619aaa20b30 2026-02-25T11:49:39.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f23d0ff9640 1 -- 10.20.193.28:0/584196534 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f23fc0d30b0 con 0x5619aaa20b30 2026-02-25T11:49:39.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f23d0ff9640 1 -- 10.20.193.28:0/584196534 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f23fc0d07f0 con 0x5619aaa20b30 2026-02-25T11:49:39.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f240f55ee00 10 client.15843 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:39.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.133+0000 7f240f55ee00 1 -- 10.20.193.28:0/584196534 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5619aa8b06f0 con 0x5619aaa20b30 2026-02-25T11:49:39.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.134+0000 7f23d0ff9640 1 -- 10.20.193.28:0/584196534 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f23fc0d0990 con 0x5619aaa20b30 2026-02-25T11:49:39.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.134+0000 7f240f55ee00 10 client.15843 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:39.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.134+0000 7f240f55ee00 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:39.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.134+0000 7f240f55ee00 20 client.15843 populate_metadata read hostname 'trial028' 2026-02-25T11:49:39.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.134+0000 7f240f55ee00 1 --2- 10.20.193.28:0/584196534 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5619aab12c30 0x5619aab33070 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:39.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.134+0000 7f240f55ee00 4 client.15843 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:49:39.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.134+0000 7f240f55ee00 1 -- 10.20.193.28:0/584196534 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619aa5bff40 con 0x5619aab12c30 2026-02-25T11:49:39.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.134+0000 7f24067fc640 1 --2- 10.20.193.28:0/584196534 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5619aab12c30 0x5619aab33070 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:39.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f24067fc640 1 --2- 10.20.193.28:0/584196534 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5619aab12c30 0x5619aab33070 crc :-1 s=READY pgs=53 gs=4 cs=0 l=0 c_cookie=1552333065706246270 s_cookie=18275832915814352963 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=158afeb2da83907e server_cookie=fda0cd3519818843 in_seq=0 out_seq=0 2026-02-25T11:49:39.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f23d0ff9640 10 client.15843 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:39.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f23d0ff9640 1 -- 10.20.193.28:0/584196534 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619aa5bff40 con 0x5619aab12c30 2026-02-25T11:49:39.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f23d0ff9640 10 client.15843 handle_command_reply: tid=0 2026-02-25T11:49:39.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f240f55ee00 1 client.15843 shutdown 2026-02-25T11:49:39.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f240f55ee00 1 -- 10.20.193.28:0/584196534 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 msgr2=0x5619aaa36570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:39.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f240f55ee00 1 --2- 10.20.193.28:0/584196534 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 0x5619aaa36570 secure :-1 s=READY pgs=464 gs=1 cs=0 l=1 c_cookie=18328930667683190943 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f23fc0d1720 tx=0x7f23fc038f30 comp rx=0 tx=0).stop 2026-02-25T11:49:39.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f240f55ee00 1 -- 10.20.193.28:0/584196534 shutdown_connections 2026-02-25T11:49:39.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f240f55ee00 1 --2- 10.20.193.28:0/584196534 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5619aab12c30 0x5619aab33070 crc :-1 s=READY pgs=53 gs=4 cs=0 l=0 c_cookie=1552333065706246270 s_cookie=18275832915814352963 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f240f55ee00 1 --2- 10.20.193.28:0/584196534 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5619aaa660e0 0x5619aaa3f670 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f240f55ee00 1 --2- 10.20.193.28:0/584196534 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5619aaa21440 0x5619aaa3f130 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f240f55ee00 1 --2- 10.20.193.28:0/584196534 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5619aaa20b30 0x5619aaa36570 unknown :-1 s=CLOSED pgs=464 gs=1 cs=0 l=1 c_cookie=18328930667683190943 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f23d0ff9640 0 client.15843 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:39.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f240f55ee00 1 -- 10.20.193.28:0/584196534 >> 10.20.193.28:0/584196534 conn(0x5619aa93f7a0 msgr2=0x5619aaa3bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:39.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f240f55ee00 1 -- 10.20.193.28:0/584196534 shutdown_connections 2026-02-25T11:49:39.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f240f55ee00 1 -- 10.20.193.28:0/584196534 wait complete. 2026-02-25T11:49:39.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.135+0000 7f240f55ee00 20 client.15843 trim_cache size 0 max 16384 2026-02-25T11:49:39.150 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:49:39.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.136+0000 7f2407fff640 1 -- 10.20.193.28:0/3349347002 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f23dc0b5760 msgr2=0x7f23dc0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:39.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.136+0000 7f2407fff640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f23dc0b5760 0x7f23dc0d5c10 secure :-1 s=READY pgs=279 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2408128f40 tx=0x7f23f000b410 comp rx=0 tx=0).stop 2026-02-25T11:49:39.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.136+0000 7f2407fff640 1 -- 10.20.193.28:0/3349347002 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f24081140a0 msgr2=0x7f24081225a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:39.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.136+0000 7f2407fff640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f24081140a0 0x7f24081225a0 secure :-1 s=READY pgs=461 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f23fc0098a0 tx=0x7f23fc0028d0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.136+0000 7f2407fff640 1 -- 10.20.193.28:0/3349347002 shutdown_connections 2026-02-25T11:49:39.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.136+0000 7f2407fff640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f23dc0b5760 0x7f23dc0d5c10 unknown :-1 s=CLOSED pgs=279 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.136+0000 7f2407fff640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2408119d30 0x7f2408127eb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.136+0000 7f2407fff640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2408114a90 0x7f2408122ae0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.136+0000 7f2407fff640 1 --2- 10.20.193.28:0/3349347002 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f24081140a0 0x7f24081225a0 unknown :-1 s=CLOSED pgs=461 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:39.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.136+0000 7f2407fff640 1 -- 10.20.193.28:0/3349347002 >> 10.20.193.28:0/3349347002 conn(0x7f2408064c40 msgr2=0x7f2408117b80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:39.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.136+0000 7f2407fff640 1 -- 10.20.193.28:0/3349347002 shutdown_connections 2026-02-25T11:49:39.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:39.136+0000 7f2407fff640 1 -- 10.20.193.28:0/3349347002 wait complete. 2026-02-25T11:49:40.274 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:49:40.275 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:49:40.458 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:40.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:40 trial167 ceph-mon[43392]: pgmap v269: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 11 op/s 2026-02-25T11:49:40.502 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:40 trial028 ceph-mon[39224]: pgmap v269: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 11 op/s 2026-02-25T11:49:40.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:40 trial119 ceph-mon[42710]: pgmap v269: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 11 op/s 2026-02-25T11:49:40.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.637+0000 7f93eac83640 1 -- 10.20.193.28:0/406738059 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f93e410bd50 msgr2=0x7f93e4110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:40.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.637+0000 7f93eac83640 1 --2- 10.20.193.28:0/406738059 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f93e410bd50 0x7f93e4110be0 secure :-1 s=READY pgs=465 gs=1 cs=0 l=1 c_cookie=18274858186833673569 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f93d8009950 tx=0x7f93d802f190 comp rx=0 tx=0).stop 2026-02-25T11:49:40.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.637+0000 7f93eac83640 1 -- 10.20.193.28:0/406738059 shutdown_connections 2026-02-25T11:49:40.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.637+0000 7f93eac83640 1 --2- 10.20.193.28:0/406738059 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f93e4111370 0x7f93e4111d50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.637+0000 7f93eac83640 1 --2- 10.20.193.28:0/406738059 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f93e410bd50 0x7f93e4110be0 unknown :-1 s=CLOSED pgs=465 gs=1 cs=0 l=1 c_cookie=18274858186833673569 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.637+0000 7f93eac83640 1 --2- 10.20.193.28:0/406738059 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f93e410b360 0x7f93e410b730 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.637+0000 7f93eac83640 1 -- 10.20.193.28:0/406738059 >> 10.20.193.28:0/406738059 conn(0x7f93e40e8e70 msgr2=0x7f93e4109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:40.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.638+0000 7f93eac83640 1 -- 10.20.193.28:0/406738059 shutdown_connections 2026-02-25T11:49:40.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.638+0000 7f93eac83640 1 -- 10.20.193.28:0/406738059 wait complete. 2026-02-25T11:49:40.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.638+0000 7f93eac83640 1 Processor -- start 2026-02-25T11:49:40.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93eac83640 1 -- start start 2026-02-25T11:49:40.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93eac83640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f93e410b360 0x7f93e4123640 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93eac83640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f93e410bd50 0x7f93e4123b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93eac83640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f93e4111370 0x7f93e4128f50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93eac83640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f93e4129570 con 0x7f93e410b360 2026-02-25T11:49:40.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93eac83640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f93e41296e0 con 0x7f93e4111370 2026-02-25T11:49:40.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93eac83640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f93e41298a0 con 0x7f93e410bd50 2026-02-25T11:49:40.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93e9c81640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f93e410b360 0x7f93e4123640 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93e9c81640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f93e410b360 0x7f93e4123640 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51546/0 (socket says 10.20.193.28:51546) 2026-02-25T11:49:40.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93e9c81640 1 -- 10.20.193.28:0/1210373788 learned_addr learned my addr 10.20.193.28:0/1210373788 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:40.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93e9480640 1 --2- 10.20.193.28:0/1210373788 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f93e410bd50 0x7f93e4123b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93e9c81640 1 -- 10.20.193.28:0/1210373788 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f93e410bd50 msgr2=0x7f93e4123b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:40.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93e9c81640 1 --2- 10.20.193.28:0/1210373788 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f93e410bd50 0x7f93e4123b80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93e9c81640 1 -- 10.20.193.28:0/1210373788 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f93e4111370 msgr2=0x7f93e4128f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:40.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93e9c81640 1 --2- 10.20.193.28:0/1210373788 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f93e4111370 0x7f93e4128f50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.639+0000 7f93e9c81640 1 -- 10.20.193.28:0/1210373788 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d8009570 con 0x7f93e410b360 2026-02-25T11:49:40.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.640+0000 7f93e9c81640 1 --2- 10.20.193.28:0/1210373788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f93e410b360 0x7f93e4123640 secure :-1 s=READY pgs=466 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f93d4009480 tx=0x7f93d400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:40.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.640+0000 7f93d2ffd640 1 -- 10.20.193.28:0/1210373788 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f93d400cd40 con 0x7f93e410b360 2026-02-25T11:49:40.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.640+0000 7f93d2ffd640 1 -- 10.20.193.28:0/1210373788 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f93d4004d10 con 0x7f93e410b360 2026-02-25T11:49:40.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.640+0000 7f93eac83640 1 -- 10.20.193.28:0/1210373788 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e4129b80 con 0x7f93e410b360 2026-02-25T11:49:40.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.640+0000 7f93d2ffd640 1 -- 10.20.193.28:0/1210373788 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f93d40106b0 con 0x7f93e410b360 2026-02-25T11:49:40.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.640+0000 7f93eac83640 1 -- 10.20.193.28:0/1210373788 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e416a280 con 0x7f93e410b360 2026-02-25T11:49:40.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.643+0000 7f93d2ffd640 1 -- 10.20.193.28:0/1210373788 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f93d401f060 con 0x7f93e410b360 2026-02-25T11:49:40.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.644+0000 7f93d2ffd640 1 --2- 10.20.193.28:0/1210373788 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f93c00b56e0 0x7f93c00d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.644+0000 7f93d2ffd640 1 -- 10.20.193.28:0/1210373788 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f93d4014070 con 0x7f93e410b360 2026-02-25T11:49:40.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.645+0000 7f93e9480640 1 --2- 10.20.193.28:0/1210373788 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f93c00b56e0 0x7f93c00d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.645+0000 7f93e9480640 1 --2- 10.20.193.28:0/1210373788 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f93c00b56e0 0x7f93c00d5b90 secure :-1 s=READY pgs=280 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f93e4129fe0 tx=0x7f93d803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:40.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.645+0000 7f93edf07e00 1 Processor -- start 2026-02-25T11:49:40.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.645+0000 7f93edf07e00 1 -- start start 2026-02-25T11:49:40.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.645+0000 7f93edf07e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 0x555a63511b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.645+0000 7f93edf07e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63512060 0x555a63556610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93edf07e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63556b50 0x555a63576f30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93edf07e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555a63427d80 con 0x555a63556b50 2026-02-25T11:49:40.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93edf07e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555a63423ab0 con 0x555a63512060 2026-02-25T11:49:40.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93edf07e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555a6341da50 con 0x555a63511750 2026-02-25T11:49:40.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93e9480640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63512060 0x555a63556610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93e9480640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63512060 0x555a63556610 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:56266/0 (socket says 10.20.193.28:56266) 2026-02-25T11:49:40.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93e9480640 1 -- 10.20.193.28:0/4096325808 learned_addr learned my addr 10.20.193.28:0/4096325808 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:40.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93e9c81640 1 --2- 10.20.193.28:0/4096325808 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 0x555a63511b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93ea482640 1 --2- 10.20.193.28:0/4096325808 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63556b50 0x555a63576f30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93ea482640 1 -- 10.20.193.28:0/4096325808 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 msgr2=0x555a63511b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:40.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93ea482640 1 --2- 10.20.193.28:0/4096325808 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 0x555a63511b20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93ea482640 1 -- 10.20.193.28:0/4096325808 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63512060 msgr2=0x555a63556610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:40.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93ea482640 1 --2- 10.20.193.28:0/4096325808 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63512060 0x555a63556610 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93ea482640 1 -- 10.20.193.28:0/4096325808 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a633c9120 con 0x555a63556b50 2026-02-25T11:49:40.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93ea482640 1 --2- 10.20.193.28:0/4096325808 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63556b50 0x555a63576f30 secure :-1 s=READY pgs=467 gs=2 cs=0 l=1 c_cookie=5155799664242698561 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f93e000ab70 tx=0x7f93e000f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478d147a6043a941 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:40.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93b24e6640 1 -- 10.20.193.28:0/4096325808 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f93e000ae90 con 0x555a63556b50 2026-02-25T11:49:40.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93b24e6640 1 -- 10.20.193.28:0/4096325808 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f93e000fe80 con 0x555a63556b50 2026-02-25T11:49:40.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.646+0000 7f93b24e6640 1 -- 10.20.193.28:0/4096325808 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f93e00116b0 con 0x555a63556b50 2026-02-25T11:49:40.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.647+0000 7f93edf07e00 1 -- 10.20.193.28:0/4096325808 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63556b50 msgr2=0x555a63576f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:40.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.647+0000 7f93edf07e00 1 --2- 10.20.193.28:0/4096325808 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63556b50 0x555a63576f30 secure :-1 s=READY pgs=467 gs=2 cs=0 l=1 c_cookie=5155799664242698561 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f93e000ab70 tx=0x7f93e000f720 comp rx=0 tx=0).stop 2026-02-25T11:49:40.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.647+0000 7f93edf07e00 1 -- 10.20.193.28:0/4096325808 shutdown_connections 2026-02-25T11:49:40.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.647+0000 7f93edf07e00 1 --2- 10.20.193.28:0/4096325808 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63556b50 0x555a63576f30 unknown :-1 s=CLOSED pgs=467 gs=2 cs=0 l=1 c_cookie=5155799664242698561 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.647+0000 7f93edf07e00 1 --2- 10.20.193.28:0/4096325808 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63512060 0x555a63556610 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.647+0000 7f93edf07e00 1 --2- 10.20.193.28:0/4096325808 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 0x555a63511b20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.647+0000 7f93edf07e00 1 -- 10.20.193.28:0/4096325808 >> 10.20.193.28:0/4096325808 conn(0x555a634ef8f0 msgr2=0x555a6350fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:40.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.647+0000 7f93edf07e00 1 -- 10.20.193.28:0/4096325808 shutdown_connections 2026-02-25T11:49:40.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.647+0000 7f93edf07e00 1 -- 10.20.193.28:0/4096325808 wait complete. 2026-02-25T11:49:40.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.648+0000 7f93edf07e00 1 -- start start 2026-02-25T11:49:40.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.648+0000 7f93edf07e00 1 Processor -- start 2026-02-25T11:49:40.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.648+0000 7f93edf07e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 0x555a6356e510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.648+0000 7f93edf07e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63512060 0x555a6356ea50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.648+0000 7f93edf07e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63556b50 0x555a63573a40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.648+0000 7f93edf07e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555a63573f80 con 0x555a63512060 2026-02-25T11:49:40.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.648+0000 7f93edf07e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555a635740f0 con 0x555a63556b50 2026-02-25T11:49:40.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.648+0000 7f93edf07e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555a635742b0 con 0x555a63511750 2026-02-25T11:49:40.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.649+0000 7f93e9480640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63512060 0x555a6356ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.649+0000 7f93ea482640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63556b50 0x555a63573a40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.649+0000 7f93e9480640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63512060 0x555a6356ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51568/0 (socket says 10.20.193.28:51568) 2026-02-25T11:49:40.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.649+0000 7f93e9480640 1 -- 10.20.193.28:0/4026800415 learned_addr learned my addr 10.20.193.28:0/4026800415 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:40.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.649+0000 7f93e9c81640 1 --2- 10.20.193.28:0/4026800415 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 0x555a6356e510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.649+0000 7f93ea482640 1 -- 10.20.193.28:0/4026800415 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 msgr2=0x555a6356e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:40.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.649+0000 7f93ea482640 1 --2- 10.20.193.28:0/4026800415 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 0x555a6356e510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.649+0000 7f93ea482640 1 -- 10.20.193.28:0/4026800415 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63512060 msgr2=0x555a6356ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:40.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.649+0000 7f93ea482640 1 --2- 10.20.193.28:0/4026800415 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63512060 0x555a6356ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.649+0000 7f93ea482640 1 -- 10.20.193.28:0/4026800415 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a633a1810 con 0x555a63556b50 2026-02-25T11:49:40.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.649+0000 7f93e9480640 1 --2- 10.20.193.28:0/4026800415 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63512060 0x555a6356ea50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:40.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.649+0000 7f93ea482640 1 --2- 10.20.193.28:0/4026800415 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63556b50 0x555a63573a40 secure :-1 s=READY pgs=120 gs=2 cs=0 l=1 c_cookie=16082777312230486493 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f93e000ab70 tx=0x7f93e0006250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3180c15935b1dd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:40.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.649+0000 7f939b7fe640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:49:40.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f939b7fe640 1 -- 10.20.193.28:0/4026800415 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f93e0006740 con 0x555a63556b50 2026-02-25T11:49:40.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f93edf07e00 1 -- 10.20.193.28:0/4026800415 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e000a810 con 0x555a63556b50 2026-02-25T11:49:40.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f939b7fe640 1 -- 10.20.193.28:0/4026800415 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f93e0005df0 con 0x555a63556b50 2026-02-25T11:49:40.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f939b7fe640 1 -- 10.20.193.28:0/4026800415 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f93e0011840 con 0x555a63556b50 2026-02-25T11:49:40.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f93edf07e00 10 client.25694 _ll_register_callbacks cb 0x555a6339ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:40.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f93edf07e00 1 -- 10.20.193.28:0/4026800415 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555a630b1220 con 0x555a63556b50 2026-02-25T11:49:40.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f939b7fe640 1 -- 10.20.193.28:0/4026800415 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f93e0015070 con 0x555a63556b50 2026-02-25T11:49:40.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f939b7fe640 1 -- 10.20.193.28:0/4026800415 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f93e00119d0 con 0x555a63556b50 2026-02-25T11:49:40.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f93edf07e00 10 client.25694 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:40.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f93edf07e00 1 -- 10.20.193.28:0/4026800415 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x555a633b34f0 con 0x555a63556b50 2026-02-25T11:49:40.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f939b7fe640 1 -- 10.20.193.28:0/4026800415 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f93e0011b70 con 0x555a63556b50 2026-02-25T11:49:40.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f93edf07e00 10 client.25694 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:40.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f93edf07e00 10 client.25694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:40.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.650+0000 7f93edf07e00 20 client.25694 populate_metadata read hostname 'trial028' 2026-02-25T11:49:40.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.651+0000 7f93edf07e00 1 --2- 10.20.193.28:0/4026800415 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555a63553e20 0x555a635542c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.651+0000 7f93edf07e00 4 client.25694 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:49:40.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.651+0000 7f93edf07e00 1 -- 10.20.193.28:0/4026800415 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a630b39d0 con 0x555a63553e20 2026-02-25T11:49:40.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.651+0000 7f93e9c81640 1 --2- 10.20.193.28:0/4026800415 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555a63553e20 0x555a635542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.651+0000 7f93e9c81640 1 --2- 10.20.193.28:0/4026800415 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555a63553e20 0x555a635542c0 crc :-1 s=READY pgs=54 gs=4 cs=0 l=0 c_cookie=6115541870617302131 s_cookie=6630186306171926282 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54dec4ee072c1473 server_cookie=5c0327513609770a in_seq=0 out_seq=0 2026-02-25T11:49:40.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.651+0000 7f939b7fe640 10 client.25694 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:40.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f939b7fe640 1 -- 10.20.193.28:0/4026800415 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x555a630b39d0 con 0x555a63553e20 2026-02-25T11:49:40.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f939b7fe640 10 client.25694 handle_command_reply: tid=0 2026-02-25T11:49:40.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f93edf07e00 1 client.25694 shutdown 2026-02-25T11:49:40.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f93edf07e00 1 -- 10.20.193.28:0/4026800415 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63556b50 msgr2=0x555a63573a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:40.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f93edf07e00 1 --2- 10.20.193.28:0/4026800415 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63556b50 0x555a63573a40 secure :-1 s=READY pgs=120 gs=2 cs=0 l=1 c_cookie=16082777312230486493 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f93e000ab70 tx=0x7f93e0006250 comp rx=0 tx=0).stop 2026-02-25T11:49:40.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f93edf07e00 1 -- 10.20.193.28:0/4026800415 shutdown_connections 2026-02-25T11:49:40.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f93edf07e00 1 --2- 10.20.193.28:0/4026800415 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555a63553e20 0x555a635542c0 crc :-1 s=READY pgs=54 gs=4 cs=0 l=0 c_cookie=6115541870617302131 s_cookie=6630186306171926282 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f93edf07e00 1 --2- 10.20.193.28:0/4026800415 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63556b50 0x555a63573a40 unknown :-1 s=CLOSED pgs=120 gs=2 cs=0 l=1 c_cookie=16082777312230486493 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f93edf07e00 1 --2- 10.20.193.28:0/4026800415 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63512060 0x555a6356ea50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f93edf07e00 1 --2- 10.20.193.28:0/4026800415 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 0x555a6356e510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f939b7fe640 0 client.25694 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:40.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f93edf07e00 1 -- 10.20.193.28:0/4026800415 >> 10.20.193.28:0/4026800415 conn(0x555a634ef8f0 msgr2=0x555a6350fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:40.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f93edf07e00 1 -- 10.20.193.28:0/4026800415 shutdown_connections 2026-02-25T11:49:40.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.652+0000 7f93edf07e00 1 -- 10.20.193.28:0/4026800415 wait complete. 2026-02-25T11:49:40.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.653+0000 7f93edf07e00 20 client.25694 trim_cache size 0 max 16384 2026-02-25T11:49:40.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.662+0000 7f93edf07e00 1 Processor -- start 2026-02-25T11:49:40.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.662+0000 7f93edf07e00 1 -- start start 2026-02-25T11:49:40.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.662+0000 7f93edf07e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 0x555a6356bed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.662+0000 7f93edf07e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63512060 0x555a63550730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.662+0000 7f93edf07e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63556b50 0x555a63550c70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.662+0000 7f93edf07e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555a6341b680 con 0x555a63512060 2026-02-25T11:49:40.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.662+0000 7f93edf07e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555a630db790 con 0x555a63556b50 2026-02-25T11:49:40.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.662+0000 7f93edf07e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555a6356ebc0 con 0x555a63511750 2026-02-25T11:49:40.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93ea482640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63556b50 0x555a63550c70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93e9c81640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 0x555a6356bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93ea482640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63556b50 0x555a63550c70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:56296/0 (socket says 10.20.193.28:56296) 2026-02-25T11:49:40.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93ea482640 1 -- 10.20.193.28:0/3654869541 learned_addr learned my addr 10.20.193.28:0/3654869541 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:40.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93e9480640 1 --2- 10.20.193.28:0/3654869541 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63512060 0x555a63550730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93e9c81640 1 -- 10.20.193.28:0/3654869541 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63556b50 msgr2=0x555a63550c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93e9c81640 1 --2- 10.20.193.28:0/3654869541 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63556b50 0x555a63550c70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93e9c81640 1 -- 10.20.193.28:0/3654869541 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63512060 msgr2=0x555a63550730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93e9c81640 1 --2- 10.20.193.28:0/3654869541 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63512060 0x555a63550730 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93e9c81640 1 -- 10.20.193.28:0/3654869541 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a633adf70 con 0x555a63511750 2026-02-25T11:49:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93ea482640 1 --2- 10.20.193.28:0/3654869541 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63556b50 0x555a63550c70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93e9c81640 1 --2- 10.20.193.28:0/3654869541 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 0x555a6356bed0 secure :-1 s=READY pgs=134 gs=2 cs=0 l=1 c_cookie=14862265349907385433 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f93d40ad3b0 tx=0x7f93d4005540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce415f9792d38c59 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93b1ce5640 1 -- 10.20.193.28:0/3654869541 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f93d4005140 con 0x555a63511750 2026-02-25T11:49:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93b1ce5640 1 -- 10.20.193.28:0/3654869541 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f93d40adb40 con 0x555a63511750 2026-02-25T11:49:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.663+0000 7f93b1ce5640 1 -- 10.20.193.28:0/3654869541 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f93d40acce0 con 0x555a63511750 2026-02-25T11:49:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.664+0000 7f93edf07e00 1 -- 10.20.193.28:0/3654869541 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 msgr2=0x555a6356bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.664+0000 7f93edf07e00 1 --2- 10.20.193.28:0/3654869541 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 0x555a6356bed0 secure :-1 s=READY pgs=134 gs=2 cs=0 l=1 c_cookie=14862265349907385433 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f93d40ad3b0 tx=0x7f93d4005540 comp rx=0 tx=0).stop 2026-02-25T11:49:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.664+0000 7f93edf07e00 1 -- 10.20.193.28:0/3654869541 shutdown_connections 2026-02-25T11:49:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.664+0000 7f93edf07e00 1 --2- 10.20.193.28:0/3654869541 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63556b50 0x555a63550c70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.664+0000 7f93edf07e00 1 --2- 10.20.193.28:0/3654869541 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63512060 0x555a63550730 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.664+0000 7f93edf07e00 1 --2- 10.20.193.28:0/3654869541 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63511750 0x555a6356bed0 unknown :-1 s=CLOSED pgs=134 gs=2 cs=0 l=1 c_cookie=14862265349907385433 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.664+0000 7f93edf07e00 1 -- 10.20.193.28:0/3654869541 >> 10.20.193.28:0/3654869541 conn(0x555a634ef8f0 msgr2=0x555a6356a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.664+0000 7f93edf07e00 1 -- 10.20.193.28:0/3654869541 shutdown_connections 2026-02-25T11:49:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.664+0000 7f93edf07e00 1 -- 10.20.193.28:0/3654869541 wait complete. 2026-02-25T11:49:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.665+0000 7f93edf07e00 1 -- start start 2026-02-25T11:49:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.665+0000 7f93edf07e00 1 Processor -- start 2026-02-25T11:49:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.665+0000 7f93edf07e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63511750 0x555a634f17b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.665+0000 7f93edf07e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63512060 0x555a634f1cf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.665+0000 7f93edf07e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63556b50 0x555a635bc990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.665+0000 7f93edf07e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555a634ec370 con 0x555a63556b50 2026-02-25T11:49:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.665+0000 7f93edf07e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555a635bced0 con 0x555a63511750 2026-02-25T11:49:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.665+0000 7f93edf07e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555a635bd090 con 0x555a63512060 2026-02-25T11:49:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.665+0000 7f93ea482640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63556b50 0x555a635bc990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.665+0000 7f93e9480640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63512060 0x555a634f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.665+0000 7f93ea482640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63556b50 0x555a635bc990 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51592/0 (socket says 10.20.193.28:51592) 2026-02-25T11:49:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.665+0000 7f93ea482640 1 -- 10.20.193.28:0/1806682807 learned_addr learned my addr 10.20.193.28:0/1806682807 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.665+0000 7f93e9c81640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63511750 0x555a634f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f93ea482640 1 -- 10.20.193.28:0/1806682807 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63512060 msgr2=0x555a634f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f93ea482640 1 --2- 10.20.193.28:0/1806682807 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63512060 0x555a634f1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f93ea482640 1 -- 10.20.193.28:0/1806682807 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63511750 msgr2=0x555a634f17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:40.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f93ea482640 1 --2- 10.20.193.28:0/1806682807 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63511750 0x555a634f17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f93ea482640 1 -- 10.20.193.28:0/1806682807 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e416a280 con 0x555a63556b50 2026-02-25T11:49:40.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f93e9c81640 1 --2- 10.20.193.28:0/1806682807 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63511750 0x555a634f17b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:40.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f93e9480640 1 --2- 10.20.193.28:0/1806682807 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63512060 0x555a634f1cf0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:40.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f93ea482640 1 --2- 10.20.193.28:0/1806682807 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63556b50 0x555a635bc990 secure :-1 s=READY pgs=468 gs=1 cs=0 l=1 c_cookie=14096814765931621198 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f93e0019c10 tx=0x7f93e000af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a1f2576bd5eb4e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:40.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f939b7fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:40.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f939b7fe640 1 -- 10.20.193.28:0/1806682807 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f93e0018d90 con 0x555a63556b50 2026-02-25T11:49:40.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f939b7fe640 1 -- 10.20.193.28:0/1806682807 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x555a630b1220 con 0x555a63556b50 2026-02-25T11:49:40.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f939b7fe640 1 -- 10.20.193.28:0/1806682807 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f93e00036f0 con 0x555a63556b50 2026-02-25T11:49:40.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f93edf07e00 1 -- 10.20.193.28:0/1806682807 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a633b34f0 con 0x555a63556b50 2026-02-25T11:49:40.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f93edf07e00 10 client.15861 _ll_register_callbacks cb 0x555a6339ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:40.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.666+0000 7f93edf07e00 1 -- 10.20.193.28:0/1806682807 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555a633cab50 con 0x555a63556b50 2026-02-25T11:49:40.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.667+0000 7f939b7fe640 1 -- 10.20.193.28:0/1806682807 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f93e0015040 con 0x555a63556b50 2026-02-25T11:49:40.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.667+0000 7f939b7fe640 1 -- 10.20.193.28:0/1806682807 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f93e0003880 con 0x555a63556b50 2026-02-25T11:49:40.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.667+0000 7f93edf07e00 10 client.15861 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.667+0000 7f93edf07e00 1 -- 10.20.193.28:0/1806682807 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x555a633a1b40 con 0x555a63556b50 2026-02-25T11:49:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.667+0000 7f939b7fe640 1 -- 10.20.193.28:0/1806682807 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f93e0019430 con 0x555a63556b50 2026-02-25T11:49:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.667+0000 7f93edf07e00 10 client.15861 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.667+0000 7f93edf07e00 10 client.15861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.667+0000 7f93edf07e00 20 client.15861 populate_metadata read hostname 'trial028' 2026-02-25T11:49:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.667+0000 7f93edf07e00 1 --2- 10.20.193.28:0/1806682807 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555a63604d50 0x555a63625200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.667+0000 7f93edf07e00 4 client.15861 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:49:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.667+0000 7f93edf07e00 1 -- 10.20.193.28:0/1806682807 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555a63625a20 con 0x555a63604d50 2026-02-25T11:49:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.667+0000 7f93e9c81640 1 --2- 10.20.193.28:0/1806682807 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555a63604d50 0x555a63625200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.668+0000 7f939b7fe640 10 client.15861 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.668+0000 7f93e9c81640 1 --2- 10.20.193.28:0/1806682807 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555a63604d50 0x555a63625200 crc :-1 s=READY pgs=55 gs=4 cs=0 l=0 c_cookie=1680493330781094243 s_cookie=11580886482897770722 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17524fc7dc9ec163 server_cookie=a0b792e5e53e98e2 in_seq=0 out_seq=0 2026-02-25T11:49:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f939b7fe640 1 -- 10.20.193.28:0/1806682807 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555a63625a20 con 0x555a63604d50 2026-02-25T11:49:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f939b7fe640 10 client.15861 handle_command_reply: tid=0 2026-02-25T11:49:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f93edf07e00 1 client.15861 shutdown 2026-02-25T11:49:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f93edf07e00 1 -- 10.20.193.28:0/1806682807 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63556b50 msgr2=0x555a635bc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f93edf07e00 1 --2- 10.20.193.28:0/1806682807 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63556b50 0x555a635bc990 secure :-1 s=READY pgs=468 gs=1 cs=0 l=1 c_cookie=14096814765931621198 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f93e0019c10 tx=0x7f93e000af80 comp rx=0 tx=0).stop 2026-02-25T11:49:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f93edf07e00 1 -- 10.20.193.28:0/1806682807 shutdown_connections 2026-02-25T11:49:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f93edf07e00 1 --2- 10.20.193.28:0/1806682807 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555a63604d50 0x555a63625200 crc :-1 s=READY pgs=55 gs=4 cs=0 l=0 c_cookie=1680493330781094243 s_cookie=11580886482897770722 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f93edf07e00 1 --2- 10.20.193.28:0/1806682807 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a63556b50 0x555a635bc990 unknown :-1 s=CLOSED pgs=468 gs=1 cs=0 l=1 c_cookie=14096814765931621198 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f93edf07e00 1 --2- 10.20.193.28:0/1806682807 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a63512060 0x555a634f1cf0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f93edf07e00 1 --2- 10.20.193.28:0/1806682807 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a63511750 0x555a634f17b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f93edf07e00 1 -- 10.20.193.28:0/1806682807 >> 10.20.193.28:0/1806682807 conn(0x555a634ef8f0 msgr2=0x555a6356afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f939b7fe640 0 client.15861 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f93edf07e00 1 -- 10.20.193.28:0/1806682807 shutdown_connections 2026-02-25T11:49:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f93edf07e00 1 -- 10.20.193.28:0/1806682807 wait complete. 2026-02-25T11:49:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.669+0000 7f93edf07e00 20 client.15861 trim_cache size 0 max 16384 2026-02-25T11:49:40.677 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:49:40.677 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:49:40.677 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "5a89f4dc-b298-4cb1-a5e4-ef808ea49090", 2026-02-25T11:49:40.677 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:49:40.678 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:49:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.670+0000 7f93eac83640 1 -- 10.20.193.28:0/1210373788 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f93c00b56e0 msgr2=0x7f93c00d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.670+0000 7f93eac83640 1 --2- 10.20.193.28:0/1210373788 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f93c00b56e0 0x7f93c00d5b90 secure :-1 s=READY pgs=280 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f93e4129fe0 tx=0x7f93d803a040 comp rx=0 tx=0).stop 2026-02-25T11:49:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.670+0000 7f93eac83640 1 -- 10.20.193.28:0/1210373788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f93e410b360 msgr2=0x7f93e4123640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.670+0000 7f93eac83640 1 --2- 10.20.193.28:0/1210373788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f93e410b360 0x7f93e4123640 secure :-1 s=READY pgs=466 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f93d4009480 tx=0x7f93d400ee60 comp rx=0 tx=0).stop 2026-02-25T11:49:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.670+0000 7f93eac83640 1 -- 10.20.193.28:0/1210373788 shutdown_connections 2026-02-25T11:49:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.670+0000 7f93eac83640 1 --2- 10.20.193.28:0/1210373788 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f93c00b56e0 0x7f93c00d5b90 unknown :-1 s=CLOSED pgs=280 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.670+0000 7f93eac83640 1 --2- 10.20.193.28:0/1210373788 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f93e4111370 0x7f93e4128f50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.670+0000 7f93eac83640 1 --2- 10.20.193.28:0/1210373788 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f93e410bd50 0x7f93e4123b80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.670+0000 7f93eac83640 1 --2- 10.20.193.28:0/1210373788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f93e410b360 0x7f93e4123640 unknown :-1 s=CLOSED pgs=466 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.670+0000 7f93eac83640 1 -- 10.20.193.28:0/1210373788 >> 10.20.193.28:0/1210373788 conn(0x7f93e40e8e70 msgr2=0x7f93e4109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.670+0000 7f93eac83640 1 -- 10.20.193.28:0/1210373788 shutdown_connections 2026-02-25T11:49:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:40.670+0000 7f93eac83640 1 -- 10.20.193.28:0/1210373788 wait complete. 2026-02-25T11:49:40.770 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:49:40.954 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:49:41.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.134+0000 7fc51f050640 1 -- 10.20.193.28:0/2068583800 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc51810eb80 msgr2=0x7fc518153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:41.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.134+0000 7fc51f050640 1 --2- 10.20.193.28:0/2068583800 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc51810eb80 0x7fc518153200 secure :-1 s=READY pgs=469 gs=1 cs=0 l=1 c_cookie=18305136371252490633 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc500009950 tx=0x7fc50002f190 comp rx=0 tx=0).stop 2026-02-25T11:49:41.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.134+0000 7fc51f050640 1 -- 10.20.193.28:0/2068583800 shutdown_connections 2026-02-25T11:49:41.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.134+0000 7fc51f050640 1 --2- 10.20.193.28:0/2068583800 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc518153740 0x7fc518173b60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.134+0000 7fc51f050640 1 --2- 10.20.193.28:0/2068583800 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc51810eb80 0x7fc518153200 unknown :-1 s=CLOSED pgs=469 gs=1 cs=0 l=1 c_cookie=18305136371252490633 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.134+0000 7fc51f050640 1 --2- 10.20.193.28:0/2068583800 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc51810e190 0x7fc51810e560 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.134+0000 7fc51f050640 1 -- 10.20.193.28:0/2068583800 >> 10.20.193.28:0/2068583800 conn(0x7fc5180e9b40 msgr2=0x7fc518109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:41.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.134+0000 7fc51f050640 1 -- 10.20.193.28:0/2068583800 shutdown_connections 2026-02-25T11:49:41.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.134+0000 7fc51f050640 1 -- 10.20.193.28:0/2068583800 wait complete. 2026-02-25T11:49:41.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.135+0000 7fc51f050640 1 Processor -- start 2026-02-25T11:49:41.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.135+0000 7fc51f050640 1 -- start start 2026-02-25T11:49:41.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.135+0000 7fc51f050640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc51810e190 0x7fc518162230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.135+0000 7fc51f050640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc51810eb80 0x7fc518162770 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.135+0000 7fc51f050640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc518153740 0x7fc518167b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.135+0000 7fc51f050640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc518168160 con 0x7fc51810e190 2026-02-25T11:49:41.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.135+0000 7fc51f050640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc5181682d0 con 0x7fc51810eb80 2026-02-25T11:49:41.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.135+0000 7fc51f050640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc518168490 con 0x7fc518153740 2026-02-25T11:49:41.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.135+0000 7fc51cdc5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc51810e190 0x7fc518162230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.135+0000 7fc51cdc5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc51810e190 0x7fc518162230 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51612/0 (socket says 10.20.193.28:51612) 2026-02-25T11:49:41.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.135+0000 7fc51cdc5640 1 -- 10.20.193.28:0/756658008 learned_addr learned my addr 10.20.193.28:0/756658008 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:41.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.135+0000 7fc50ffff640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc51810eb80 0x7fc518162770 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.135+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc518153740 0x7fc518167b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.136+0000 7fc51cdc5640 1 -- 10.20.193.28:0/756658008 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc518153740 msgr2=0x7fc518167b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:41.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.136+0000 7fc51cdc5640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc518153740 0x7fc518167b40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.136+0000 7fc51cdc5640 1 -- 10.20.193.28:0/756658008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc51810eb80 msgr2=0x7fc518162770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:41.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.136+0000 7fc51cdc5640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc51810eb80 0x7fc518162770 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.136+0000 7fc51cdc5640 1 -- 10.20.193.28:0/756658008 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc500009570 con 0x7fc51810e190 2026-02-25T11:49:41.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.136+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc518153740 0x7fc518167b40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:49:41.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.136+0000 7fc50ffff640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc51810eb80 0x7fc518162770 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:49:41.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.136+0000 7fc51cdc5640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc51810e190 0x7fc518162230 secure :-1 s=READY pgs=470 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc508009480 tx=0x7fc50800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:41.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.136+0000 7fc50dffb640 1 -- 10.20.193.28:0/756658008 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc50800cd40 con 0x7fc51810e190 2026-02-25T11:49:41.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.136+0000 7fc51f050640 1 -- 10.20.193.28:0/756658008 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc518168770 con 0x7fc51810e190 2026-02-25T11:49:41.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.136+0000 7fc51f050640 1 -- 10.20.193.28:0/756658008 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51814f360 con 0x7fc51810e190 2026-02-25T11:49:41.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.136+0000 7fc50dffb640 1 -- 10.20.193.28:0/756658008 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc508004d10 con 0x7fc51810e190 2026-02-25T11:49:41.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.136+0000 7fc50dffb640 1 -- 10.20.193.28:0/756658008 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc5080106b0 con 0x7fc51810e190 2026-02-25T11:49:41.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.140+0000 7fc50dffb640 1 -- 10.20.193.28:0/756658008 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fc50801f060 con 0x7fc51810e190 2026-02-25T11:49:41.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.142+0000 7fc50dffb640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4f40b5760 0x7fc4f40d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.142+0000 7fc52104be00 1 Processor -- start 2026-02-25T11:49:41.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.142+0000 7fc50dffb640 1 -- 10.20.193.28:0/756658008 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc508014070 con 0x7fc51810e190 2026-02-25T11:49:41.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.142+0000 7fc52104be00 1 -- start start 2026-02-25T11:49:41.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.142+0000 7fc50ffff640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4f40b5760 0x7fc4f40d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.142+0000 7fc52104be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 0x5571d1fe2510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.142+0000 7fc52104be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 0x5571d20271b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.142+0000 7fc52104be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d2047ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.142+0000 7fc52104be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5571d1ef89b0 con 0x5571d20276f0 2026-02-25T11:49:41.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.142+0000 7fc52104be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5571d1ef46e0 con 0x5571d1fe2a50 2026-02-25T11:49:41.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.142+0000 7fc52104be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5571d1eee840 con 0x5571d1fe2140 2026-02-25T11:49:41.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc51d5c6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d2047ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc51d5c6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d2047ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51620/0 (socket says 10.20.193.28:51620) 2026-02-25T11:49:41.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc51d5c6640 1 -- 10.20.193.28:0/3587825789 learned_addr learned my addr 10.20.193.28:0/3587825789 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:41.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc50ffff640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4f40b5760 0x7fc4f40d5c10 secure :-1 s=READY pgs=281 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc518168bd0 tx=0x7fc50003a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:41.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc51cdc5640 1 --2- 10.20.193.28:0/3587825789 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 0x5571d1fe2510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc51d5c6640 1 -- 10.20.193.28:0/3587825789 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 msgr2=0x5571d1fe2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:41.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/3587825789 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 0x5571d1fe2510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc51d5c6640 1 -- 10.20.193.28:0/3587825789 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 msgr2=0x5571d20271b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:41.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/3587825789 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 0x5571d20271b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc51d5c6640 1 -- 10.20.193.28:0/3587825789 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d1e99d80 con 0x5571d20276f0 2026-02-25T11:49:41.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/3587825789 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d2047ad0 secure :-1 s=READY pgs=471 gs=1 cs=0 l=1 c_cookie=10336640263961651132 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc5140056b0 tx=0x7fc514003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f731f8de06f27bc server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:41.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc4e54e4640 1 -- 10.20.193.28:0/3587825789 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc51400ae30 con 0x5571d20276f0 2026-02-25T11:49:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc4e54e4640 1 -- 10.20.193.28:0/3587825789 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc514003e90 con 0x5571d20276f0 2026-02-25T11:49:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.143+0000 7fc4e54e4640 1 -- 10.20.193.28:0/3587825789 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc514011990 con 0x5571d20276f0 2026-02-25T11:49:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.144+0000 7fc52104be00 1 -- 10.20.193.28:0/3587825789 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 msgr2=0x5571d2047ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.144+0000 7fc52104be00 1 --2- 10.20.193.28:0/3587825789 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d2047ad0 secure :-1 s=READY pgs=471 gs=1 cs=0 l=1 c_cookie=10336640263961651132 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc5140056b0 tx=0x7fc514003c30 comp rx=0 tx=0).stop 2026-02-25T11:49:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.144+0000 7fc52104be00 1 -- 10.20.193.28:0/3587825789 shutdown_connections 2026-02-25T11:49:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.144+0000 7fc52104be00 1 --2- 10.20.193.28:0/3587825789 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d2047ad0 unknown :-1 s=CLOSED pgs=471 gs=1 cs=0 l=1 c_cookie=10336640263961651132 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.144+0000 7fc52104be00 1 --2- 10.20.193.28:0/3587825789 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 0x5571d20271b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.144+0000 7fc52104be00 1 --2- 10.20.193.28:0/3587825789 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 0x5571d1fe2510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.144+0000 7fc52104be00 1 -- 10.20.193.28:0/3587825789 >> 10.20.193.28:0/3587825789 conn(0x5571d1f01d40 msgr2=0x5571d1fe0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.144+0000 7fc52104be00 1 -- 10.20.193.28:0/3587825789 shutdown_connections 2026-02-25T11:49:41.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.144+0000 7fc52104be00 1 -- 10.20.193.28:0/3587825789 wait complete. 2026-02-25T11:49:41.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.145+0000 7fc52104be00 1 -- start start 2026-02-25T11:49:41.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.145+0000 7fc52104be00 1 Processor -- start 2026-02-25T11:49:41.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc52104be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 0x5571d1fe32b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc52104be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 0x5571d1fe37f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc52104be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d1fe5190 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc52104be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5571d1fe56d0 con 0x5571d20276f0 2026-02-25T11:49:41.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc52104be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5571d1fe5840 con 0x5571d1fe2a50 2026-02-25T11:49:41.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc52104be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5571d1fe5a00 con 0x5571d1fe2140 2026-02-25T11:49:41.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc51d5c6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d1fe5190 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc51d5c6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d1fe5190 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51632/0 (socket says 10.20.193.28:51632) 2026-02-25T11:49:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc51d5c6640 1 -- 10.20.193.28:0/4144982303 learned_addr learned my addr 10.20.193.28:0/4144982303 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc51cdc5640 1 --2- 10.20.193.28:0/4144982303 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 0x5571d1fe32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc50ffff640 1 --2- 10.20.193.28:0/4144982303 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 0x5571d1fe37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc51d5c6640 1 -- 10.20.193.28:0/4144982303 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 msgr2=0x5571d1fe32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/4144982303 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 0x5571d1fe32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc51d5c6640 1 -- 10.20.193.28:0/4144982303 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 msgr2=0x5571d1fe37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/4144982303 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 0x5571d1fe37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc51d5c6640 1 -- 10.20.193.28:0/4144982303 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d1e72350 con 0x5571d20276f0 2026-02-25T11:49:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/4144982303 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d1fe5190 secure :-1 s=READY pgs=472 gs=1 cs=0 l=1 c_cookie=11647555640907740939 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc514005050 tx=0x7fc5140068a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a46e24dde7370b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc4d67fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc4d67fc640 1 -- 10.20.193.28:0/4144982303 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc51400aea0 con 0x5571d20276f0 2026-02-25T11:49:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.146+0000 7fc4d67fc640 1 -- 10.20.193.28:0/4144982303 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc514022040 con 0x5571d20276f0 2026-02-25T11:49:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc4d67fc640 1 -- 10.20.193.28:0/4144982303 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc51401c430 con 0x5571d20276f0 2026-02-25T11:49:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc52104be00 1 -- 10.20.193.28:0/4144982303 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5140052d0 con 0x5571d20276f0 2026-02-25T11:49:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc4d67fc640 1 -- 10.20.193.28:0/4144982303 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc51401c5d0 con 0x5571d20276f0 2026-02-25T11:49:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc4d67fc640 10 client.15873 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:49:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc52104be00 10 client.15873 _ll_register_callbacks cb 0x5571d1e6f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc52104be00 1 -- 10.20.193.28:0/4144982303 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5571d1b846f0 con 0x5571d20276f0 2026-02-25T11:49:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc4d67fc640 1 -- 10.20.193.28:0/4144982303 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc5140115a0 con 0x5571d20276f0 2026-02-25T11:49:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc52104be00 10 client.15873 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc52104be00 1 -- 10.20.193.28:0/4144982303 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5571d1e84150 con 0x5571d20276f0 2026-02-25T11:49:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc4d67fc640 1 -- 10.20.193.28:0/4144982303 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fc514011700 con 0x5571d20276f0 2026-02-25T11:49:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc52104be00 10 client.15873 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc52104be00 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc52104be00 20 client.15873 populate_metadata read hostname 'trial028' 2026-02-25T11:49:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc52104be00 1 --2- 10.20.193.28:0/4144982303 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571d200d5a0 0x5571d200da40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc52104be00 4 client.15873 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:49:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.147+0000 7fc52104be00 1 -- 10.20.193.28:0/4144982303 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571d1ec0910 con 0x5571d200d5a0 2026-02-25T11:49:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.148+0000 7fc51cdc5640 1 --2- 10.20.193.28:0/4144982303 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571d200d5a0 0x5571d200da40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc51cdc5640 1 --2- 10.20.193.28:0/4144982303 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571d200d5a0 0x5571d200da40 crc :-1 s=READY pgs=56 gs=4 cs=0 l=0 c_cookie=13526204321067935622 s_cookie=11577408373983886626 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb6bb2a449ba786 server_cookie=a0ab37938ef13d22 in_seq=0 out_seq=0 2026-02-25T11:49:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc4d67fc640 10 client.15873 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:41.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc4d67fc640 1 -- 10.20.193.28:0/4144982303 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5571d1ec0910 con 0x5571d200d5a0 2026-02-25T11:49:41.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc4d67fc640 10 client.15873 handle_command_reply: tid=0 2026-02-25T11:49:41.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc52104be00 1 client.15873 shutdown 2026-02-25T11:49:41.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc52104be00 1 -- 10.20.193.28:0/4144982303 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 msgr2=0x5571d1fe5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:41.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc52104be00 1 --2- 10.20.193.28:0/4144982303 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d1fe5190 secure :-1 s=READY pgs=472 gs=1 cs=0 l=1 c_cookie=11647555640907740939 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc514005050 tx=0x7fc5140068a0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc52104be00 1 -- 10.20.193.28:0/4144982303 shutdown_connections 2026-02-25T11:49:41.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc52104be00 1 --2- 10.20.193.28:0/4144982303 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571d200d5a0 0x5571d200da40 crc :-1 s=READY pgs=56 gs=4 cs=0 l=0 c_cookie=13526204321067935622 s_cookie=11577408373983886626 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc52104be00 1 --2- 10.20.193.28:0/4144982303 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d1fe5190 unknown :-1 s=CLOSED pgs=472 gs=1 cs=0 l=1 c_cookie=11647555640907740939 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc52104be00 1 --2- 10.20.193.28:0/4144982303 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 0x5571d1fe37f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc52104be00 1 --2- 10.20.193.28:0/4144982303 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 0x5571d1fe32b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc4d67fc640 0 client.15873 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:41.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.149+0000 7fc52104be00 1 -- 10.20.193.28:0/4144982303 >> 10.20.193.28:0/4144982303 conn(0x5571d1f01d40 msgr2=0x5571d1fe0230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:41.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.150+0000 7fc52104be00 1 -- 10.20.193.28:0/4144982303 shutdown_connections 2026-02-25T11:49:41.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.150+0000 7fc52104be00 1 -- 10.20.193.28:0/4144982303 wait complete. 2026-02-25T11:49:41.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.150+0000 7fc52104be00 20 client.15873 trim_cache size 0 max 16384 2026-02-25T11:49:41.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc52104be00 1 Processor -- start 2026-02-25T11:49:41.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc52104be00 1 -- start start 2026-02-25T11:49:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc52104be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2140 0x5571d1fe3770 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc52104be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2a50 0x5571d20270b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc52104be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d20281d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc52104be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5571d1eec470 con 0x5571d20276f0 2026-02-25T11:49:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc52104be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5571d1bac700 con 0x5571d1fe2140 2026-02-25T11:49:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc52104be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5571d1fe8100 con 0x5571d1fe2a50 2026-02-25T11:49:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc51d5c6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d20281d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc51d5c6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d20281d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51646/0 (socket says 10.20.193.28:51646) 2026-02-25T11:49:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc51d5c6640 1 -- 10.20.193.28:0/514037358 learned_addr learned my addr 10.20.193.28:0/514037358 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc51cdc5640 1 --2- 10.20.193.28:0/514037358 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2140 0x5571d1fe3770 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc50ffff640 1 --2- 10.20.193.28:0/514037358 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2a50 0x5571d20270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc51d5c6640 1 -- 10.20.193.28:0/514037358 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2a50 msgr2=0x5571d20270b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/514037358 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2a50 0x5571d20270b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc51d5c6640 1 -- 10.20.193.28:0/514037358 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2140 msgr2=0x5571d1fe3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/514037358 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2140 0x5571d1fe3770 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc51d5c6640 1 -- 10.20.193.28:0/514037358 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d203a0a0 con 0x5571d20276f0 2026-02-25T11:49:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc50ffff640 1 --2- 10.20.193.28:0/514037358 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2a50 0x5571d20270b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:49:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.159+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/514037358 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d20281d0 secure :-1 s=READY pgs=473 gs=2 cs=0 l=1 c_cookie=8402640171942056806 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc514012dd0 tx=0x7fc514037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749c2cb7a0089366 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.160+0000 7fc4e4ce3640 1 -- 10.20.193.28:0/514037358 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc51403e070 con 0x5571d20276f0 2026-02-25T11:49:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.160+0000 7fc4e4ce3640 1 -- 10.20.193.28:0/514037358 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc514022050 con 0x5571d20276f0 2026-02-25T11:49:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.160+0000 7fc4e4ce3640 1 -- 10.20.193.28:0/514037358 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc514026a30 con 0x5571d20276f0 2026-02-25T11:49:41.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.160+0000 7fc52104be00 1 -- 10.20.193.28:0/514037358 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 msgr2=0x5571d20281d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:41.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.160+0000 7fc52104be00 1 --2- 10.20.193.28:0/514037358 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d20281d0 secure :-1 s=READY pgs=473 gs=2 cs=0 l=1 c_cookie=8402640171942056806 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc514012dd0 tx=0x7fc514037000 comp rx=0 tx=0).stop 2026-02-25T11:49:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.160+0000 7fc52104be00 1 -- 10.20.193.28:0/514037358 shutdown_connections 2026-02-25T11:49:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.160+0000 7fc52104be00 1 --2- 10.20.193.28:0/514037358 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d20281d0 unknown :-1 s=CLOSED pgs=473 gs=2 cs=0 l=1 c_cookie=8402640171942056806 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.160+0000 7fc52104be00 1 --2- 10.20.193.28:0/514037358 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2a50 0x5571d20270b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.160+0000 7fc52104be00 1 --2- 10.20.193.28:0/514037358 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2140 0x5571d1fe3770 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.160+0000 7fc52104be00 1 -- 10.20.193.28:0/514037358 >> 10.20.193.28:0/514037358 conn(0x5571d1f01d40 msgr2=0x5571d2027ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.160+0000 7fc52104be00 1 -- 10.20.193.28:0/514037358 shutdown_connections 2026-02-25T11:49:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.160+0000 7fc52104be00 1 -- 10.20.193.28:0/514037358 wait complete. 2026-02-25T11:49:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.161+0000 7fc52104be00 1 -- start start 2026-02-25T11:49:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.161+0000 7fc52104be00 1 Processor -- start 2026-02-25T11:49:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.161+0000 7fc52104be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 0x5571d1fe1550 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.161+0000 7fc52104be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 0x5571d1fe1a90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.161+0000 7fc52104be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d2008680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.161+0000 7fc52104be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5571d2008d80 con 0x5571d20276f0 2026-02-25T11:49:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.161+0000 7fc52104be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5571d2008ef0 con 0x5571d1fe2a50 2026-02-25T11:49:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.161+0000 7fc52104be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5571d20090b0 con 0x5571d1fe2140 2026-02-25T11:49:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.161+0000 7fc51d5c6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d2008680 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.161+0000 7fc51d5c6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d2008680 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51660/0 (socket says 10.20.193.28:51660) 2026-02-25T11:49:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.161+0000 7fc51d5c6640 1 -- 10.20.193.28:0/2148907432 learned_addr learned my addr 10.20.193.28:0/2148907432 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:49:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.161+0000 7fc50ffff640 1 --2- 10.20.193.28:0/2148907432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 0x5571d1fe1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.161+0000 7fc51cdc5640 1 --2- 10.20.193.28:0/2148907432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 0x5571d1fe1550 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc51d5c6640 1 -- 10.20.193.28:0/2148907432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 msgr2=0x5571d1fe1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/2148907432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 0x5571d1fe1550 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc51d5c6640 1 -- 10.20.193.28:0/2148907432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 msgr2=0x5571d1fe1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:49:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/2148907432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 0x5571d1fe1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc51d5c6640 1 -- 10.20.193.28:0/2148907432 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571d1e7ebd0 con 0x5571d20276f0 2026-02-25T11:49:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc51d5c6640 1 --2- 10.20.193.28:0/2148907432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d2008680 secure :-1 s=READY pgs=474 gs=1 cs=0 l=1 c_cookie=4472779330370053153 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc51400a4c0 tx=0x7fc514003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e12811a696aa421 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:49:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc4d67fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:49:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc4d67fc640 1 -- 10.20.193.28:0/2148907432 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc51403e070 con 0x5571d20276f0 2026-02-25T11:49:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc4d67fc640 1 -- 10.20.193.28:0/2148907432 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc514035020 con 0x5571d20276f0 2026-02-25T11:49:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc52104be00 1 -- 10.20.193.28:0/2148907432 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571d1e84150 con 0x5571d20276f0 2026-02-25T11:49:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc4d67fc640 1 -- 10.20.193.28:0/2148907432 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc514012950 con 0x5571d20276f0 2026-02-25T11:49:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc52104be00 10 client.15879 _ll_register_callbacks cb 0x5571d1e6f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:49:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc52104be00 1 -- 10.20.193.28:0/2148907432 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc514003e90 con 0x5571d20276f0 2026-02-25T11:49:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc4d67fc640 1 -- 10.20.193.28:0/2148907432 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc514036030 con 0x5571d20276f0 2026-02-25T11:49:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc4d67fc640 1 -- 10.20.193.28:0/2148907432 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc514012ab0 con 0x5571d20276f0 2026-02-25T11:49:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc52104be00 10 client.15879 fetch_fsmap learned FSMap version 23 2026-02-25T11:49:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.162+0000 7fc52104be00 1 -- 10.20.193.28:0/2148907432 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5571d1e72680 con 0x5571d20276f0 2026-02-25T11:49:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.163+0000 7fc4d67fc640 1 -- 10.20.193.28:0/2148907432 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fc51404a020 con 0x5571d20276f0 2026-02-25T11:49:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.163+0000 7fc52104be00 10 client.15879 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:49:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.163+0000 7fc52104be00 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:49:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.163+0000 7fc52104be00 20 client.15879 populate_metadata read hostname 'trial028' 2026-02-25T11:49:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.163+0000 7fc52104be00 1 --2- 10.20.193.28:0/2148907432 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571d20111e0 0x5571d2011650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:49:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.163+0000 7fc52104be00 4 client.15879 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:49:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.163+0000 7fc52104be00 1 -- 10.20.193.28:0/2148907432 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5571d1b81f40 con 0x5571d20111e0 2026-02-25T11:49:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.163+0000 7fc51cdc5640 1 --2- 10.20.193.28:0/2148907432 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571d20111e0 0x5571d2011650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:49:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc51cdc5640 1 --2- 10.20.193.28:0/2148907432 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571d20111e0 0x5571d2011650 crc :-1 s=READY pgs=57 gs=4 cs=0 l=0 c_cookie=8051259134064205628 s_cookie=4697079432605834089 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fbbd18668841f3c server_cookie=412f60db96439b69 in_seq=0 out_seq=0 2026-02-25T11:49:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc4d67fc640 10 client.15879 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:41.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc4d67fc640 1 -- 10.20.193.28:0/2148907432 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x5571d1b81f40 con 0x5571d20111e0 2026-02-25T11:49:41.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc4d67fc640 10 client.15879 handle_command_reply: tid=0 2026-02-25T11:49:41.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc52104be00 1 client.15879 shutdown 2026-02-25T11:49:41.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc52104be00 1 -- 10.20.193.28:0/2148907432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 msgr2=0x5571d2008680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:41.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc52104be00 1 --2- 10.20.193.28:0/2148907432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d2008680 secure :-1 s=READY pgs=474 gs=1 cs=0 l=1 c_cookie=4472779330370053153 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc51400a4c0 tx=0x7fc514003710 comp rx=0 tx=0).stop 2026-02-25T11:49:41.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc52104be00 1 -- 10.20.193.28:0/2148907432 shutdown_connections 2026-02-25T11:49:41.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc52104be00 1 --2- 10.20.193.28:0/2148907432 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571d20111e0 0x5571d2011650 crc :-1 s=READY pgs=57 gs=4 cs=0 l=0 c_cookie=8051259134064205628 s_cookie=4697079432605834089 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc52104be00 1 --2- 10.20.193.28:0/2148907432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571d20276f0 0x5571d2008680 unknown :-1 s=CLOSED pgs=474 gs=1 cs=0 l=1 c_cookie=4472779330370053153 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc52104be00 1 --2- 10.20.193.28:0/2148907432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571d1fe2a50 0x5571d1fe1a90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc52104be00 1 --2- 10.20.193.28:0/2148907432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571d1fe2140 0x5571d1fe1550 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc4d67fc640 0 client.15879 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:49:41.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc52104be00 1 -- 10.20.193.28:0/2148907432 >> 10.20.193.28:0/2148907432 conn(0x5571d1f01d40 msgr2=0x5571d20282b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:41.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc52104be00 1 -- 10.20.193.28:0/2148907432 shutdown_connections 2026-02-25T11:49:41.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc52104be00 1 -- 10.20.193.28:0/2148907432 wait complete. 2026-02-25T11:49:41.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.164+0000 7fc52104be00 20 client.15879 trim_cache size 0 max 16384 2026-02-25T11:49:41.174 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:49:41.174 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:49:41.175 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:49:41.175 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:49:41.175 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:41.175 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "5a89f4dc-b298-4cb1-a5e4-ef808ea49090", 2026-02-25T11:49:41.175 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:49:41.175 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:49:41.175 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:41.175 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:49:41.175 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:49:41.176 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:49:41.176 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:41.176 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:41.176 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:41.176 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:41.176 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:41.176 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:41.176 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:49:41.176 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:49:41.176 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:41.176 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:41.177 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:41.177 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:41.177 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:41.177 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:41.177 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:49:41.177 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:49:41.177 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:41.177 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:41.178 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:41.178 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:41.178 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:41.178 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:41.178 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:49:41.178 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:49:41.178 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:41.178 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:41.178 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:41.178 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:41.179 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:41.179 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:49:41.179 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:49:41.179 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:49:41.179 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:49:41.179 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:49:41.179 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:49:41.179 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:49:41.179 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:49:41.179 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:49:41.179 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:49:41.180 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:49:41.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.165+0000 7fc51f050640 1 -- 10.20.193.28:0/756658008 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4f40b5760 msgr2=0x7fc4f40d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.165+0000 7fc51f050640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4f40b5760 0x7fc4f40d5c10 secure :-1 s=READY pgs=281 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc518168bd0 tx=0x7fc50003a040 comp rx=0 tx=0).stop 2026-02-25T11:49:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.165+0000 7fc51f050640 1 -- 10.20.193.28:0/756658008 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc51810e190 msgr2=0x7fc518162230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:49:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.165+0000 7fc51f050640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc51810e190 0x7fc518162230 secure :-1 s=READY pgs=470 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc508009480 tx=0x7fc50800ee60 comp rx=0 tx=0).stop 2026-02-25T11:49:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.165+0000 7fc51f050640 1 -- 10.20.193.28:0/756658008 shutdown_connections 2026-02-25T11:49:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.165+0000 7fc51f050640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4f40b5760 0x7fc4f40d5c10 unknown :-1 s=CLOSED pgs=281 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.165+0000 7fc51f050640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc518153740 0x7fc518167b40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.165+0000 7fc51f050640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc51810eb80 0x7fc518162770 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.165+0000 7fc51f050640 1 --2- 10.20.193.28:0/756658008 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc51810e190 0x7fc518162230 unknown :-1 s=CLOSED pgs=470 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:49:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.165+0000 7fc51f050640 1 -- 10.20.193.28:0/756658008 >> 10.20.193.28:0/756658008 conn(0x7fc5180e9b40 msgr2=0x7fc518109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:49:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.166+0000 7fc51f050640 1 -- 10.20.193.28:0/756658008 shutdown_connections 2026-02-25T11:49:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:49:41.166+0000 7fc51f050640 1 -- 10.20.193.28:0/756658008 wait complete. 2026-02-25T11:49:41.270 INFO:tasks.cephfs.filesystem:scrub status for tag:5a89f4dc-b298-4cb1-a5e4-ef808ea49090 - {'path': '/', 'tag': '5a89f4dc-b298-4cb1-a5e4-ef808ea49090', 'options': 'recursive,force'} 2026-02-25T11:49:41.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:41 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:49:41.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:41 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:49:41.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:41 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:49:41.310 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:41 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:49:41.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:41 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:49:41.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:41 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:49:41.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:41 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:49:41.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:41 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:49:41.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:41 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:49:41.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:41 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:49:41.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:41 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:49:41.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:41 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:49:42.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:42 trial167 ceph-mon[43392]: pgmap v270: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:49:42.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:42 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:49:42.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:42 trial119 ceph-mon[42710]: pgmap v270: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:49:42.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:42 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:49:42.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:42 trial028 ceph-mon[39224]: pgmap v270: 97 pgs: 97 active+clean; 1.5 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:49:42.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:42 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:49:44.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:44 trial167 ceph-mon[43392]: pgmap v271: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:44.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:44 trial119 ceph-mon[42710]: pgmap v271: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:44.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:44 trial028 ceph-mon[39224]: pgmap v271: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:49:46.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:46 trial167 ceph-mon[43392]: pgmap v272: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:46.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:46 trial119 ceph-mon[42710]: pgmap v272: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:46.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:46 trial028 ceph-mon[39224]: pgmap v272: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:48.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:48 trial167 ceph-mon[43392]: pgmap v273: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:48.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:48 trial119 ceph-mon[42710]: pgmap v273: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:48.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:48 trial028 ceph-mon[39224]: pgmap v273: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:50.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:50 trial167 ceph-mon[43392]: pgmap v274: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:49:50.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:50 trial119 ceph-mon[42710]: pgmap v274: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:49:50.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:50 trial028 ceph-mon[39224]: pgmap v274: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:49:52.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:52 trial167 ceph-mon[43392]: pgmap v275: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:49:52.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:52 trial119 ceph-mon[42710]: pgmap v275: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:49:52.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:52 trial028 ceph-mon[39224]: pgmap v275: 97 pgs: 97 active+clean; 1.6 MiB data, 333 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:49:54.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:54 trial167 ceph-mon[43392]: pgmap v276: 97 pgs: 97 active+clean; 1.7 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:49:54.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:54 trial119 ceph-mon[42710]: pgmap v276: 97 pgs: 97 active+clean; 1.7 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:49:54.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:54 trial028 ceph-mon[39224]: pgmap v276: 97 pgs: 97 active+clean; 1.7 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:49:56.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:56 trial167 ceph-mon[43392]: pgmap v277: 97 pgs: 97 active+clean; 1.7 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:56.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:56 trial119 ceph-mon[42710]: pgmap v277: 97 pgs: 97 active+clean; 1.7 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:56.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:56 trial028 ceph-mon[39224]: pgmap v277: 97 pgs: 97 active+clean; 1.7 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:49:58.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:49:58 trial167 ceph-mon[43392]: pgmap v278: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:58.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:49:58 trial119 ceph-mon[42710]: pgmap v278: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:49:58.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:49:58 trial028 ceph-mon[39224]: pgmap v278: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:50:00.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:00 trial167 ceph-mon[43392]: pgmap v279: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:50:00.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:00 trial167 ceph-mon[43392]: overall HEALTH_OK 2026-02-25T11:50:00.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:00 trial119 ceph-mon[42710]: pgmap v279: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:50:00.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:00 trial119 ceph-mon[42710]: overall HEALTH_OK 2026-02-25T11:50:00.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:00 trial028 ceph-mon[39224]: pgmap v279: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:50:00.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:00 trial028 ceph-mon[39224]: overall HEALTH_OK 2026-02-25T11:50:02.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:02 trial167 ceph-mon[43392]: pgmap v280: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:02.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:02 trial119 ceph-mon[42710]: pgmap v280: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:02.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:02 trial028 ceph-mon[39224]: pgmap v280: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:04.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:04 trial167 ceph-mon[43392]: pgmap v281: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:04.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:04 trial119 ceph-mon[42710]: pgmap v281: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:04.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:04 trial028 ceph-mon[39224]: pgmap v281: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:06.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:06 trial167 ceph-mon[43392]: pgmap v282: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:06.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:06 trial119 ceph-mon[42710]: pgmap v282: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:06.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:06 trial028 ceph-mon[39224]: pgmap v282: 97 pgs: 97 active+clean; 1.8 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:08.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:08 trial167 ceph-mon[43392]: pgmap v283: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:50:08.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:08 trial119 ceph-mon[42710]: pgmap v283: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:50:08.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:08 trial028 ceph-mon[39224]: pgmap v283: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:50:10.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:10 trial167 ceph-mon[43392]: pgmap v284: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:50:10.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:10 trial119 ceph-mon[42710]: pgmap v284: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:50:10.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:10 trial028 ceph-mon[39224]: pgmap v284: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:50:11.271 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:50:11.460 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:11.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.638+0000 7f43a6506640 1 -- 10.20.193.28:0/3195560060 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43a010f690 msgr2=0x7f43a010fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:11.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.638+0000 7f43a6506640 1 --2- 10.20.193.28:0/3195560060 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43a010f690 0x7f43a010fa60 secure :-1 s=READY pgs=475 gs=1 cs=0 l=1 c_cookie=4879728585310554301 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4390009950 tx=0x7f439002f190 comp rx=0 tx=0).stop 2026-02-25T11:50:11.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.638+0000 7f43a6506640 1 -- 10.20.193.28:0/3195560060 shutdown_connections 2026-02-25T11:50:11.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.638+0000 7f43a6506640 1 --2- 10.20.193.28:0/3195560060 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f43a0115880 0x7f43a0176250 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.638+0000 7f43a6506640 1 --2- 10.20.193.28:0/3195560060 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f43a0110080 0x7f43a01150d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.638+0000 7f43a6506640 1 --2- 10.20.193.28:0/3195560060 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43a010f690 0x7f43a010fa60 unknown :-1 s=CLOSED pgs=475 gs=1 cs=0 l=1 c_cookie=4879728585310554301 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.638+0000 7f43a6506640 1 -- 10.20.193.28:0/3195560060 >> 10.20.193.28:0/3195560060 conn(0x7f43a00ecf00 msgr2=0x7f43a010d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:11.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.638+0000 7f43a6506640 1 -- 10.20.193.28:0/3195560060 shutdown_connections 2026-02-25T11:50:11.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.639+0000 7f43a6506640 1 -- 10.20.193.28:0/3195560060 wait complete. 2026-02-25T11:50:11.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.640+0000 7f43a6506640 1 Processor -- start 2026-02-25T11:50:11.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.640+0000 7f43a6506640 1 -- start start 2026-02-25T11:50:11.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.640+0000 7f43a6506640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f43a010f690 0x7f43a00f5960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.640+0000 7f43a6506640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f43a0110080 0x7f43a0176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.640+0000 7f43a6506640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43a0115880 0x7f43a00f6060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.640+0000 7f43a6506640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f43a00fb270 con 0x7f43a0115880 2026-02-25T11:50:11.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.640+0000 7f43a6506640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f43a00fb3e0 con 0x7f43a010f690 2026-02-25T11:50:11.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.640+0000 7f43a6506640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f43a00fb5a0 con 0x7f43a0110080 2026-02-25T11:50:11.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.640+0000 7f43a5d05640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43a0115880 0x7f43a00f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.640+0000 7f43a5d05640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43a0115880 0x7f43a00f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42266/0 (socket says 10.20.193.28:42266) 2026-02-25T11:50:11.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.640+0000 7f43a5d05640 1 -- 10.20.193.28:0/3570440349 learned_addr learned my addr 10.20.193.28:0/3570440349 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:11.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.640+0000 7f43a5504640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f43a010f690 0x7f43a00f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.641+0000 7f43a4d03640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f43a0110080 0x7f43a0176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.641+0000 7f43a5d05640 1 -- 10.20.193.28:0/3570440349 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f43a0110080 msgr2=0x7f43a0176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:11.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.641+0000 7f43a5d05640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f43a0110080 0x7f43a0176150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.641+0000 7f43a5d05640 1 -- 10.20.193.28:0/3570440349 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f43a010f690 msgr2=0x7f43a00f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:11.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.641+0000 7f43a5d05640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f43a010f690 0x7f43a00f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.641+0000 7f43a5d05640 1 -- 10.20.193.28:0/3570440349 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4390009570 con 0x7f43a0115880 2026-02-25T11:50:11.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.641+0000 7f43a4d03640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f43a0110080 0x7f43a0176150 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:11.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.641+0000 7f43a5d05640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43a0115880 0x7f43a00f6060 secure :-1 s=READY pgs=476 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f439c006e90 tx=0x7f439c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:11.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.641+0000 7f438e7fc640 1 -- 10.20.193.28:0/3570440349 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f439c005500 con 0x7f43a0115880 2026-02-25T11:50:11.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.641+0000 7f438e7fc640 1 -- 10.20.193.28:0/3570440349 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f439c005df0 con 0x7f43a0115880 2026-02-25T11:50:11.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.641+0000 7f438e7fc640 1 -- 10.20.193.28:0/3570440349 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f439c003a00 con 0x7f43a0115880 2026-02-25T11:50:11.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.641+0000 7f43a6506640 1 -- 10.20.193.28:0/3570440349 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a00fb880 con 0x7f43a0115880 2026-02-25T11:50:11.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.641+0000 7f43a6506640 1 -- 10.20.193.28:0/3570440349 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a00fbd00 con 0x7f43a0115880 2026-02-25T11:50:11.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.645+0000 7f438e7fc640 1 -- 10.20.193.28:0/3570440349 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f439c003b60 con 0x7f43a0115880 2026-02-25T11:50:11.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.646+0000 7f438e7fc640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f437c0b5820 0x7f437c0d5cd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.646+0000 7f438e7fc640 1 -- 10.20.193.28:0/3570440349 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f439c0e7470 con 0x7f43a0115880 2026-02-25T11:50:11.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.646+0000 7f43a5504640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f437c0b5820 0x7f437c0d5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.647+0000 7f43a5504640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f437c0b5820 0x7f437c0d5cd0 secure :-1 s=READY pgs=282 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f43900098a0 tx=0x7f439003a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:11.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.647+0000 7f43a978ae00 1 Processor -- start 2026-02-25T11:50:11.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.647+0000 7f43a978ae00 1 -- start start 2026-02-25T11:50:11.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.647+0000 7f43a978ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 0x563f75816510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.647+0000 7f43a978ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816a50 0x563f7585b1b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.647+0000 7f43a978ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f7585b6f0 0x563f7587bad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a978ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x563f7572c9b0 con 0x563f75816140 2026-02-25T11:50:11.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a978ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x563f757286e0 con 0x563f7585b6f0 2026-02-25T11:50:11.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a978ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x563f75722840 con 0x563f75816a50 2026-02-25T11:50:11.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a5504640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 0x563f75816510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a5504640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 0x563f75816510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42276/0 (socket says 10.20.193.28:42276) 2026-02-25T11:50:11.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a5504640 1 -- 10.20.193.28:0/4291386799 learned_addr learned my addr 10.20.193.28:0/4291386799 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:11.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a4d03640 1 --2- 10.20.193.28:0/4291386799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816a50 0x563f7585b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a5d05640 1 --2- 10.20.193.28:0/4291386799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f7585b6f0 0x563f7587bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a5504640 1 -- 10.20.193.28:0/4291386799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816a50 msgr2=0x563f7585b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:11.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a5504640 1 --2- 10.20.193.28:0/4291386799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816a50 0x563f7585b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a5504640 1 -- 10.20.193.28:0/4291386799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f7585b6f0 msgr2=0x563f7587bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:11.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a5504640 1 --2- 10.20.193.28:0/4291386799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f7585b6f0 0x563f7587bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a5504640 1 -- 10.20.193.28:0/4291386799 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f756cdd80 con 0x563f75816140 2026-02-25T11:50:11.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f43a5504640 1 --2- 10.20.193.28:0/4291386799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 0x563f75816510 secure :-1 s=READY pgs=477 gs=1 cs=0 l=1 c_cookie=13498958216161399107 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4390005b00 tx=0x7f4390040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb55eefa20241143 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:11.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f436dce5640 1 -- 10.20.193.28:0/4291386799 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f439002fc10 con 0x563f75816140 2026-02-25T11:50:11.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f436dce5640 1 -- 10.20.193.28:0/4291386799 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f439003e040 con 0x563f75816140 2026-02-25T11:50:11.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.648+0000 7f436dce5640 1 -- 10.20.193.28:0/4291386799 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f439004f7e0 con 0x563f75816140 2026-02-25T11:50:11.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.649+0000 7f43a978ae00 1 -- 10.20.193.28:0/4291386799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 msgr2=0x563f75816510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:11.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.649+0000 7f43a978ae00 1 --2- 10.20.193.28:0/4291386799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 0x563f75816510 secure :-1 s=READY pgs=477 gs=1 cs=0 l=1 c_cookie=13498958216161399107 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4390005b00 tx=0x7f4390040000 comp rx=0 tx=0).stop 2026-02-25T11:50:11.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.649+0000 7f43a978ae00 1 -- 10.20.193.28:0/4291386799 shutdown_connections 2026-02-25T11:50:11.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.649+0000 7f43a978ae00 1 --2- 10.20.193.28:0/4291386799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f7585b6f0 0x563f7587bad0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.649+0000 7f43a978ae00 1 --2- 10.20.193.28:0/4291386799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816a50 0x563f7585b1b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.649+0000 7f43a978ae00 1 --2- 10.20.193.28:0/4291386799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 0x563f75816510 unknown :-1 s=CLOSED pgs=477 gs=1 cs=0 l=1 c_cookie=13498958216161399107 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.649+0000 7f43a978ae00 1 -- 10.20.193.28:0/4291386799 >> 10.20.193.28:0/4291386799 conn(0x563f75735d40 msgr2=0x563f75814a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:11.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.649+0000 7f43a978ae00 1 -- 10.20.193.28:0/4291386799 shutdown_connections 2026-02-25T11:50:11.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.649+0000 7f43a978ae00 1 -- 10.20.193.28:0/4291386799 wait complete. 2026-02-25T11:50:11.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.650+0000 7f43a978ae00 1 -- start start 2026-02-25T11:50:11.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.650+0000 7f43a978ae00 1 Processor -- start 2026-02-25T11:50:11.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.650+0000 7f43a978ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 0x563f758172b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.650+0000 7f43a978ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816a50 0x563f758177f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a978ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f7585b6f0 0x563f7581c460 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a978ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x563f7581c9a0 con 0x563f75816140 2026-02-25T11:50:11.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a978ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x563f7581cb10 con 0x563f75816a50 2026-02-25T11:50:11.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a978ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x563f7581ccd0 con 0x563f7585b6f0 2026-02-25T11:50:11.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a5504640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 0x563f758172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a5504640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 0x563f758172b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42286/0 (socket says 10.20.193.28:42286) 2026-02-25T11:50:11.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a5504640 1 -- 10.20.193.28:0/1310499214 learned_addr learned my addr 10.20.193.28:0/1310499214 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:11.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a5d05640 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f7585b6f0 0x563f7581c460 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a4d03640 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816a50 0x563f758177f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a5504640 1 -- 10.20.193.28:0/1310499214 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f7585b6f0 msgr2=0x563f7581c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:11.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a5504640 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f7585b6f0 0x563f7581c460 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a5504640 1 -- 10.20.193.28:0/1310499214 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816a50 msgr2=0x563f758177f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:11.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a5504640 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816a50 0x563f758177f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a5504640 1 -- 10.20.193.28:0/1310499214 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f756a6350 con 0x563f75816140 2026-02-25T11:50:11.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a5d05640 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f7585b6f0 0x563f7581c460 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:11.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a5504640 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 0x563f758172b0 secure :-1 s=READY pgs=478 gs=1 cs=0 l=1 c_cookie=2423856157478741343 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4390005f50 tx=0x7f43900388d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a344554529e55f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:11.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f435affd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:11.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f435affd640 1 -- 10.20.193.28:0/1310499214 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4390038a70 con 0x563f75816140 2026-02-25T11:50:11.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f43a978ae00 1 -- 10.20.193.28:0/1310499214 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a0176ac0 con 0x563f75816140 2026-02-25T11:50:11.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.651+0000 7f435affd640 1 -- 10.20.193.28:0/1310499214 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4390002e20 con 0x563f75816140 2026-02-25T11:50:11.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f435affd640 1 -- 10.20.193.28:0/1310499214 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4390038a70 con 0x563f75816140 2026-02-25T11:50:11.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f435affd640 1 -- 10.20.193.28:0/1310499214 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f43900448d0 con 0x563f75816140 2026-02-25T11:50:11.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f43a978ae00 10 client.15891 _ll_register_callbacks cb 0x563f756a39a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:11.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f43a978ae00 1 -- 10.20.193.28:0/1310499214 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563f753b86f0 con 0x563f75816140 2026-02-25T11:50:11.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f435affd640 1 -- 10.20.193.28:0/1310499214 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f439002fc10 con 0x563f75816140 2026-02-25T11:50:11.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f43a978ae00 10 client.15891 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:11.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f43a978ae00 1 -- 10.20.193.28:0/1310499214 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x563f756b8150 con 0x563f75816140 2026-02-25T11:50:11.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f435affd640 1 -- 10.20.193.28:0/1310499214 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f4390044410 con 0x563f75816140 2026-02-25T11:50:11.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f43a978ae00 10 client.15891 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:11.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f43a978ae00 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:11.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f43a978ae00 20 client.15891 populate_metadata read hostname 'trial028' 2026-02-25T11:50:11.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f43a978ae00 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563f75841400 0x563f758418a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f43a978ae00 4 client.15891 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:50:11.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.652+0000 7f43a978ae00 1 -- 10.20.193.28:0/1310499214 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f756f4910 con 0x563f75841400 2026-02-25T11:50:11.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.653+0000 7f43a4d03640 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563f75841400 0x563f758418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.654+0000 7f43a4d03640 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563f75841400 0x563f758418a0 crc :-1 s=READY pgs=58 gs=4 cs=0 l=0 c_cookie=3861351022268641340 s_cookie=6924606599645833300 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3596464be369f03c server_cookie=60192503a5372854 in_seq=0 out_seq=0 2026-02-25T11:50:11.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.654+0000 7f435affd640 10 client.15891 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:11.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.654+0000 7f435affd640 1 -- 10.20.193.28:0/1310499214 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x563f756f4910 con 0x563f75841400 2026-02-25T11:50:11.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.654+0000 7f435affd640 10 client.15891 handle_command_reply: tid=0 2026-02-25T11:50:11.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.654+0000 7f43a978ae00 1 client.15891 shutdown 2026-02-25T11:50:11.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.654+0000 7f43a978ae00 1 -- 10.20.193.28:0/1310499214 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 msgr2=0x563f758172b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:11.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.654+0000 7f43a978ae00 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 0x563f758172b0 secure :-1 s=READY pgs=478 gs=1 cs=0 l=1 c_cookie=2423856157478741343 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4390005f50 tx=0x7f43900388d0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.655+0000 7f43a978ae00 1 -- 10.20.193.28:0/1310499214 shutdown_connections 2026-02-25T11:50:11.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.655+0000 7f43a978ae00 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563f75841400 0x563f758418a0 crc :-1 s=READY pgs=58 gs=4 cs=0 l=0 c_cookie=3861351022268641340 s_cookie=6924606599645833300 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.655+0000 7f43a978ae00 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f7585b6f0 0x563f7581c460 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.655+0000 7f43a978ae00 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816a50 0x563f758177f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.655+0000 7f43a978ae00 1 --2- 10.20.193.28:0/1310499214 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f75816140 0x563f758172b0 unknown :-1 s=CLOSED pgs=478 gs=1 cs=0 l=1 c_cookie=2423856157478741343 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.655+0000 7f435affd640 0 client.15891 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:11.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.655+0000 7f43a978ae00 1 -- 10.20.193.28:0/1310499214 >> 10.20.193.28:0/1310499214 conn(0x563f75735d40 msgr2=0x563f75814230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:11.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.655+0000 7f43a978ae00 1 -- 10.20.193.28:0/1310499214 shutdown_connections 2026-02-25T11:50:11.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.655+0000 7f43a978ae00 1 -- 10.20.193.28:0/1310499214 wait complete. 2026-02-25T11:50:11.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.655+0000 7f43a978ae00 20 client.15891 trim_cache size 0 max 16384 2026-02-25T11:50:11.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a978ae00 1 Processor -- start 2026-02-25T11:50:11.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a978ae00 1 -- start start 2026-02-25T11:50:11.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a978ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816140 0x563f7585e1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a978ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816a50 0x563f75815330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a978ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 0x563f75815870 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a978ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x563f75720470 con 0x563f7585b6f0 2026-02-25T11:50:11.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a978ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x563f753e0700 con 0x563f75816a50 2026-02-25T11:50:11.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a978ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x563f758657c0 con 0x563f75816140 2026-02-25T11:50:11.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a5d05640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 0x563f75815870 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a5d05640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 0x563f75815870 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42292/0 (socket says 10.20.193.28:42292) 2026-02-25T11:50:11.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a5d05640 1 -- 10.20.193.28:0/2833813316 learned_addr learned my addr 10.20.193.28:0/2833813316 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:11.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a4d03640 1 --2- 10.20.193.28:0/2833813316 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816a50 0x563f75815330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a5d05640 1 -- 10.20.193.28:0/2833813316 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816140 msgr2=0x563f7585e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:11.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a5504640 1 --2- 10.20.193.28:0/2833813316 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816140 0x563f7585e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a5d05640 1 --2- 10.20.193.28:0/2833813316 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816140 0x563f7585e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a5d05640 1 -- 10.20.193.28:0/2833813316 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816a50 msgr2=0x563f75815330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:11.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a5d05640 1 --2- 10.20.193.28:0/2833813316 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816a50 0x563f75815330 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a5d05640 1 -- 10.20.193.28:0/2833813316 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f756b2bd0 con 0x563f7585b6f0 2026-02-25T11:50:11.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a4d03640 1 --2- 10.20.193.28:0/2833813316 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816a50 0x563f75815330 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:11.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.665+0000 7f43a5504640 1 --2- 10.20.193.28:0/2833813316 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816140 0x563f7585e1e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:11.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.666+0000 7f43a5d05640 1 --2- 10.20.193.28:0/2833813316 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 0x563f75815870 secure :-1 s=READY pgs=479 gs=1 cs=0 l=1 c_cookie=15940768591981064753 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f439c00c880 tx=0x7f439c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd38fc939bbc3231 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:11.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.666+0000 7f436d4e4640 1 -- 10.20.193.28:0/2833813316 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f439c0e4070 con 0x563f7585b6f0 2026-02-25T11:50:11.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.666+0000 7f436d4e4640 1 -- 10.20.193.28:0/2833813316 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f439c003e40 con 0x563f7585b6f0 2026-02-25T11:50:11.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.666+0000 7f436d4e4640 1 -- 10.20.193.28:0/2833813316 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f439c00c8d0 con 0x563f7585b6f0 2026-02-25T11:50:11.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.666+0000 7f43a978ae00 1 -- 10.20.193.28:0/2833813316 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 msgr2=0x563f75815870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:11.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.666+0000 7f43a978ae00 1 --2- 10.20.193.28:0/2833813316 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 0x563f75815870 secure :-1 s=READY pgs=479 gs=1 cs=0 l=1 c_cookie=15940768591981064753 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f439c00c880 tx=0x7f439c01f000 comp rx=0 tx=0).stop 2026-02-25T11:50:11.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.666+0000 7f43a978ae00 1 -- 10.20.193.28:0/2833813316 shutdown_connections 2026-02-25T11:50:11.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.666+0000 7f43a978ae00 1 --2- 10.20.193.28:0/2833813316 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 0x563f75815870 unknown :-1 s=CLOSED pgs=479 gs=1 cs=0 l=1 c_cookie=15940768591981064753 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.666+0000 7f43a978ae00 1 --2- 10.20.193.28:0/2833813316 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816a50 0x563f75815330 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.666+0000 7f43a978ae00 1 --2- 10.20.193.28:0/2833813316 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816140 0x563f7585e1e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.666+0000 7f43a978ae00 1 -- 10.20.193.28:0/2833813316 >> 10.20.193.28:0/2833813316 conn(0x563f75735d40 msgr2=0x563f7585dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:11.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.666+0000 7f43a978ae00 1 -- 10.20.193.28:0/2833813316 shutdown_connections 2026-02-25T11:50:11.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.666+0000 7f43a978ae00 1 -- 10.20.193.28:0/2833813316 wait complete. 2026-02-25T11:50:11.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.667+0000 7f43a978ae00 1 -- start start 2026-02-25T11:50:11.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.667+0000 7f43a978ae00 1 Processor -- start 2026-02-25T11:50:11.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.667+0000 7f43a978ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816140 0x563f75854370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.667+0000 7f43a978ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816a50 0x563f7584e860 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.667+0000 7f43a978ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 0x563f7584eda0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.667+0000 7f43a978ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x563f7584f4a0 con 0x563f7585b6f0 2026-02-25T11:50:11.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.667+0000 7f43a978ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x563f7584f610 con 0x563f75816140 2026-02-25T11:50:11.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.667+0000 7f43a978ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x563f75910590 con 0x563f75816a50 2026-02-25T11:50:11.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.667+0000 7f43a5d05640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 0x563f7584eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.667+0000 7f43a5d05640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 0x563f7584eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42308/0 (socket says 10.20.193.28:42308) 2026-02-25T11:50:11.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.667+0000 7f43a5d05640 1 -- 10.20.193.28:0/307355538 learned_addr learned my addr 10.20.193.28:0/307355538 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:11.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f43a5d05640 1 -- 10.20.193.28:0/307355538 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816a50 msgr2=0x563f7584e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:11.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f43a5d05640 1 --2- 10.20.193.28:0/307355538 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816a50 0x563f7584e860 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f43a5d05640 1 -- 10.20.193.28:0/307355538 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816140 msgr2=0x563f75854370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:11.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f43a5d05640 1 --2- 10.20.193.28:0/307355538 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816140 0x563f75854370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f43a5d05640 1 -- 10.20.193.28:0/307355538 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f756a6680 con 0x563f7585b6f0 2026-02-25T11:50:11.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f43a5d05640 1 --2- 10.20.193.28:0/307355538 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 0x563f7584eda0 secure :-1 s=READY pgs=480 gs=1 cs=0 l=1 c_cookie=3721646280092861893 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f439c0a90d0 tx=0x7f439c0a9c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a5f192d896c5c5 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:11.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f435affd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:11.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f435affd640 1 -- 10.20.193.28:0/307355538 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f439c0e4070 con 0x563f7585b6f0 2026-02-25T11:50:11.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f435affd640 1 -- 10.20.193.28:0/307355538 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f439c0b3510 con 0x563f7585b6f0 2026-02-25T11:50:11.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f435affd640 1 -- 10.20.193.28:0/307355538 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f439c005ad0 con 0x563f7585b6f0 2026-02-25T11:50:11.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f43a978ae00 1 -- 10.20.193.28:0/307355538 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a00fbd00 con 0x563f7585b6f0 2026-02-25T11:50:11.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f435affd640 1 -- 10.20.193.28:0/307355538 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f439c0ae480 con 0x563f7585b6f0 2026-02-25T11:50:11.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f435affd640 10 client.15897 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:50:11.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f43a978ae00 10 client.15897 _ll_register_callbacks cb 0x563f756a39a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:11.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f43a978ae00 1 -- 10.20.193.28:0/307355538 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f439003e040 con 0x563f7585b6f0 2026-02-25T11:50:11.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.668+0000 7f435affd640 1 -- 10.20.193.28:0/307355538 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f439c0ae1f0 con 0x563f7585b6f0 2026-02-25T11:50:11.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.669+0000 7f43a978ae00 10 client.15897 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:11.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.669+0000 7f43a978ae00 1 -- 10.20.193.28:0/307355538 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x563f756b83b0 con 0x563f7585b6f0 2026-02-25T11:50:11.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.669+0000 7f435affd640 1 -- 10.20.193.28:0/307355538 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f439c0b30c0 con 0x563f7585b6f0 2026-02-25T11:50:11.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.669+0000 7f43a978ae00 10 client.15897 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:11.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.669+0000 7f43a978ae00 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:11.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.669+0000 7f43a978ae00 20 client.15897 populate_metadata read hostname 'trial028' 2026-02-25T11:50:11.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.669+0000 7f43a978ae00 1 --2- 10.20.193.28:0/307355538 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563f75911e50 0x563f75932300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:11.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.669+0000 7f43a978ae00 4 client.15897 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:50:11.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.669+0000 7f43a978ae00 1 -- 10.20.193.28:0/307355538 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f753b5f40 con 0x563f75911e50 2026-02-25T11:50:11.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.669+0000 7f43a5504640 1 --2- 10.20.193.28:0/307355538 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563f75911e50 0x563f75932300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:11.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.670+0000 7f43a5504640 1 --2- 10.20.193.28:0/307355538 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563f75911e50 0x563f75932300 crc :-1 s=READY pgs=59 gs=4 cs=0 l=0 c_cookie=16731836850597258844 s_cookie=1880019275079395998 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8336cf76fa3e65c server_cookie=1a172b91e5feaa9e in_seq=0 out_seq=0 2026-02-25T11:50:11.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.670+0000 7f435affd640 10 client.15897 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:11.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.670+0000 7f435affd640 1 -- 10.20.193.28:0/307355538 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x563f753b5f40 con 0x563f75911e50 2026-02-25T11:50:11.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.670+0000 7f435affd640 10 client.15897 handle_command_reply: tid=0 2026-02-25T11:50:11.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.670+0000 7f43a978ae00 1 client.15897 shutdown 2026-02-25T11:50:11.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.670+0000 7f43a978ae00 1 -- 10.20.193.28:0/307355538 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 msgr2=0x563f7584eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:11.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.670+0000 7f43a978ae00 1 --2- 10.20.193.28:0/307355538 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 0x563f7584eda0 secure :-1 s=READY pgs=480 gs=1 cs=0 l=1 c_cookie=3721646280092861893 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f439c0a90d0 tx=0x7f439c0a9c90 comp rx=0 tx=0).stop 2026-02-25T11:50:11.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.670+0000 7f43a978ae00 1 -- 10.20.193.28:0/307355538 shutdown_connections 2026-02-25T11:50:11.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.670+0000 7f43a978ae00 1 --2- 10.20.193.28:0/307355538 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563f75911e50 0x563f75932300 crc :-1 s=READY pgs=59 gs=4 cs=0 l=0 c_cookie=16731836850597258844 s_cookie=1880019275079395998 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a978ae00 1 --2- 10.20.193.28:0/307355538 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563f7585b6f0 0x563f7584eda0 unknown :-1 s=CLOSED pgs=480 gs=1 cs=0 l=1 c_cookie=3721646280092861893 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a978ae00 1 --2- 10.20.193.28:0/307355538 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563f75816a50 0x563f7584e860 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a978ae00 1 --2- 10.20.193.28:0/307355538 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563f75816140 0x563f75854370 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a978ae00 1 -- 10.20.193.28:0/307355538 >> 10.20.193.28:0/307355538 conn(0x563f75735d40 msgr2=0x563f7581b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:11.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f435affd640 0 client.15897 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:11.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a978ae00 1 -- 10.20.193.28:0/307355538 shutdown_connections 2026-02-25T11:50:11.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a978ae00 1 -- 10.20.193.28:0/307355538 wait complete. 2026-02-25T11:50:11.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a978ae00 20 client.15897 trim_cache size 0 max 16384 2026-02-25T11:50:11.692 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:50:11.692 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:11.693 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:11.694 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:11.695 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:50:11.695 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:50:11.695 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:11.695 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:11.695 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:11.695 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:11.695 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:11.695 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:50:11.695 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:50:11.695 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:50:11.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a6506640 1 -- 10.20.193.28:0/3570440349 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f437c0b5820 msgr2=0x7f437c0d5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:11.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a6506640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f437c0b5820 0x7f437c0d5cd0 secure :-1 s=READY pgs=282 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f43900098a0 tx=0x7f439003a040 comp rx=0 tx=0).stop 2026-02-25T11:50:11.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a6506640 1 -- 10.20.193.28:0/3570440349 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43a0115880 msgr2=0x7f43a00f6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:11.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a6506640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43a0115880 0x7f43a00f6060 secure :-1 s=READY pgs=476 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f439c006e90 tx=0x7f439c00edf0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a6506640 1 -- 10.20.193.28:0/3570440349 shutdown_connections 2026-02-25T11:50:11.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a6506640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f437c0b5820 0x7f437c0d5cd0 unknown :-1 s=CLOSED pgs=282 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a6506640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f43a0115880 0x7f43a00f6060 unknown :-1 s=CLOSED pgs=476 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a6506640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f43a0110080 0x7f43a0176150 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a6506640 1 --2- 10.20.193.28:0/3570440349 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f43a010f690 0x7f43a00f5960 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:11.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.671+0000 7f43a6506640 1 -- 10.20.193.28:0/3570440349 >> 10.20.193.28:0/3570440349 conn(0x7f43a00ecf00 msgr2=0x7f43a0111b30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:11.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.672+0000 7f43a6506640 1 -- 10.20.193.28:0/3570440349 shutdown_connections 2026-02-25T11:50:11.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:11.672+0000 7f43a6506640 1 -- 10.20.193.28:0/3570440349 wait complete. 2026-02-25T11:50:11.777 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:50:11.777 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:50:11.962 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:12.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.141+0000 7fc550d81640 1 -- 10.20.193.28:0/2093823020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc54c1135c0 msgr2=0x7fc54c173f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.141+0000 7fc550d81640 1 --2- 10.20.193.28:0/2093823020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc54c1135c0 0x7fc54c173f90 secure :-1 s=READY pgs=481 gs=1 cs=0 l=1 c_cookie=4542230903784622239 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc54000aa30 tx=0x7fc540030270 comp rx=0 tx=0).stop 2026-02-25T11:50:12.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.141+0000 7fc550d81640 1 -- 10.20.193.28:0/2093823020 shutdown_connections 2026-02-25T11:50:12.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.141+0000 7fc550d81640 1 --2- 10.20.193.28:0/2093823020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc54c1135c0 0x7fc54c173f90 unknown :-1 s=CLOSED pgs=481 gs=1 cs=0 l=1 c_cookie=4542230903784622239 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.141+0000 7fc550d81640 1 --2- 10.20.193.28:0/2093823020 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc54c10ddc0 0x7fc54c112e10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.141+0000 7fc550d81640 1 --2- 10.20.193.28:0/2093823020 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc54c10d3d0 0x7fc54c10d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.141+0000 7fc550d81640 1 -- 10.20.193.28:0/2093823020 >> 10.20.193.28:0/2093823020 conn(0x7fc54c0eaca0 msgr2=0x7fc54c10b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:12.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.141+0000 7fc550d81640 1 -- 10.20.193.28:0/2093823020 shutdown_connections 2026-02-25T11:50:12.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.141+0000 7fc550d81640 1 -- 10.20.193.28:0/2093823020 wait complete. 2026-02-25T11:50:12.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc550d81640 1 Processor -- start 2026-02-25T11:50:12.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc550d81640 1 -- start start 2026-02-25T11:50:12.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc550d81640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc54c10d3d0 0x7fc54c0f5990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc550d81640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc54c10ddc0 0x7fc54c173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc550d81640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc54c1135c0 0x7fc54c0f6090 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc550d81640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0fb250 con 0x7fc54c10d3d0 2026-02-25T11:50:12.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc550d81640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0fb3c0 con 0x7fc54c10ddc0 2026-02-25T11:50:12.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc550d81640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0fb580 con 0x7fc54c1135c0 2026-02-25T11:50:12.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc54b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc54c10d3d0 0x7fc54c0f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc54b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc54c10d3d0 0x7fc54c0f5990 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42326/0 (socket says 10.20.193.28:42326) 2026-02-25T11:50:12.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc54b7fe640 1 -- 10.20.193.28:0/670868896 learned_addr learned my addr 10.20.193.28:0/670868896 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:12.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc54affd640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc54c10ddc0 0x7fc54c173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc54bfff640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc54c1135c0 0x7fc54c0f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc54b7fe640 1 -- 10.20.193.28:0/670868896 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc54c1135c0 msgr2=0x7fc54c0f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc54b7fe640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc54c1135c0 0x7fc54c0f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc54b7fe640 1 -- 10.20.193.28:0/670868896 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc54c10ddc0 msgr2=0x7fc54c173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc54b7fe640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc54c10ddc0 0x7fc54c173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc54b7fe640 1 -- 10.20.193.28:0/670868896 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54000a650 con 0x7fc54c10d3d0 2026-02-25T11:50:12.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc54bfff640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc54c1135c0 0x7fc54c0f6090 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:12.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.142+0000 7fc54affd640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc54c10ddc0 0x7fc54c173e90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:12.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.143+0000 7fc54b7fe640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc54c10d3d0 0x7fc54c0f5990 secure :-1 s=READY pgs=482 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc53800d950 tx=0x7fc53800de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:12.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.143+0000 7fc548ff9640 1 -- 10.20.193.28:0/670868896 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc538004540 con 0x7fc54c10d3d0 2026-02-25T11:50:12.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.143+0000 7fc548ff9640 1 -- 10.20.193.28:0/670868896 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc538009ce0 con 0x7fc54c10d3d0 2026-02-25T11:50:12.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.143+0000 7fc548ff9640 1 -- 10.20.193.28:0/670868896 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc5380106e0 con 0x7fc54c10d3d0 2026-02-25T11:50:12.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.143+0000 7fc550d81640 1 -- 10.20.193.28:0/670868896 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c0fb860 con 0x7fc54c10d3d0 2026-02-25T11:50:12.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.143+0000 7fc550d81640 1 -- 10.20.193.28:0/670868896 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c0fbce0 con 0x7fc54c10d3d0 2026-02-25T11:50:12.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.143+0000 7fc550d81640 1 -- 10.20.193.28:0/670868896 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54c10e6b0 con 0x7fc54c10d3d0 2026-02-25T11:50:12.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.146+0000 7fc548ff9640 1 -- 10.20.193.28:0/670868896 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fc5380097c0 con 0x7fc54c10d3d0 2026-02-25T11:50:12.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.147+0000 7fc548ff9640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc5200b5750 0x7fc5200d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.147+0000 7fc548ff9640 1 -- 10.20.193.28:0/670868896 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc5380e6fb0 con 0x7fc54c10d3d0 2026-02-25T11:50:12.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.147+0000 7fc54affd640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc5200b5750 0x7fc5200d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.148+0000 7fc548ff9640 1 -- 10.20.193.28:0/670868896 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fc5381208a0 con 0x7fc54c10d3d0 2026-02-25T11:50:12.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.148+0000 7fc54affd640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc5200b5750 0x7fc5200d5c00 secure :-1 s=READY pgs=283 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc54c0fc2e0 tx=0x7fc53c005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:12.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.288+0000 7fc550d81640 1 -- 10.20.193.28:0/670868896 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc54c10d7a0 con 0x7fc54c10d3d0 2026-02-25T11:50:12.291 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:12 trial028 ceph-mon[39224]: pgmap v285: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:12.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.289+0000 7fc548ff9640 1 -- 10.20.193.28:0/670868896 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7fc5380a9980 con 0x7fc54c10d3d0 2026-02-25T11:50:12.292 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:50:12.292 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:50:12.293 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:50:12.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.290+0000 7fc550d81640 1 -- 10.20.193.28:0/670868896 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc5200b5750 msgr2=0x7fc5200d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.290+0000 7fc550d81640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc5200b5750 0x7fc5200d5c00 secure :-1 s=READY pgs=283 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc54c0fc2e0 tx=0x7fc53c005f00 comp rx=0 tx=0).stop 2026-02-25T11:50:12.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.290+0000 7fc550d81640 1 -- 10.20.193.28:0/670868896 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc54c10d3d0 msgr2=0x7fc54c0f5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.290+0000 7fc550d81640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc54c10d3d0 0x7fc54c0f5990 secure :-1 s=READY pgs=482 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc53800d950 tx=0x7fc53800de20 comp rx=0 tx=0).stop 2026-02-25T11:50:12.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.290+0000 7fc550d81640 1 -- 10.20.193.28:0/670868896 shutdown_connections 2026-02-25T11:50:12.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.290+0000 7fc550d81640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc5200b5750 0x7fc5200d5c00 unknown :-1 s=CLOSED pgs=283 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.290+0000 7fc550d81640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc54c1135c0 0x7fc54c0f6090 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.290+0000 7fc550d81640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc54c10ddc0 0x7fc54c173e90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.290+0000 7fc550d81640 1 --2- 10.20.193.28:0/670868896 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc54c10d3d0 0x7fc54c0f5990 unknown :-1 s=CLOSED pgs=482 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.290+0000 7fc550d81640 1 -- 10.20.193.28:0/670868896 >> 10.20.193.28:0/670868896 conn(0x7fc54c0eaca0 msgr2=0x7fc54c10b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:12.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.290+0000 7fc550d81640 1 -- 10.20.193.28:0/670868896 shutdown_connections 2026-02-25T11:50:12.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.291+0000 7fc550d81640 1 -- 10.20.193.28:0/670868896 wait complete. 2026-02-25T11:50:12.391 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:50:12.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:12 trial167 ceph-mon[43392]: pgmap v285: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:12.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:12 trial119 ceph-mon[42710]: pgmap v285: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:12.576 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:12.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.749+0000 7fd74213a640 1 -- 10.20.193.28:0/3113081349 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd73c110150 msgr2=0x7fd73c1151a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.749+0000 7fd74213a640 1 --2- 10.20.193.28:0/3113081349 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd73c110150 0x7fd73c1151a0 secure :-1 s=READY pgs=483 gs=1 cs=0 l=1 c_cookie=13151411615505113762 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd72c009950 tx=0x7fd72c02f190 comp rx=0 tx=0).stop 2026-02-25T11:50:12.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.749+0000 7fd74213a640 1 -- 10.20.193.28:0/3113081349 shutdown_connections 2026-02-25T11:50:12.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.749+0000 7fd74213a640 1 --2- 10.20.193.28:0/3113081349 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd73c115950 0x7fd73c176320 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.749+0000 7fd74213a640 1 --2- 10.20.193.28:0/3113081349 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd73c110150 0x7fd73c1151a0 unknown :-1 s=CLOSED pgs=483 gs=1 cs=0 l=1 c_cookie=13151411615505113762 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.749+0000 7fd74213a640 1 --2- 10.20.193.28:0/3113081349 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd73c10f760 0x7fd73c10fb30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.749+0000 7fd74213a640 1 -- 10.20.193.28:0/3113081349 >> 10.20.193.28:0/3113081349 conn(0x7fd73c0ed050 msgr2=0x7fd73c10d460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:12.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.750+0000 7fd74213a640 1 -- 10.20.193.28:0/3113081349 shutdown_connections 2026-02-25T11:50:12.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.750+0000 7fd74213a640 1 -- 10.20.193.28:0/3113081349 wait complete. 2026-02-25T11:50:12.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.751+0000 7fd74213a640 1 Processor -- start 2026-02-25T11:50:12.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.751+0000 7fd74213a640 1 -- start start 2026-02-25T11:50:12.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.751+0000 7fd74213a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd73c10f760 0x7fd73c0f5b70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.751+0000 7fd74213a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd73c110150 0x7fd73c176220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.751+0000 7fd74213a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd73c115950 0x7fd73c0f6270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.751+0000 7fd74213a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c0fb480 con 0x7fd73c10f760 2026-02-25T11:50:12.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.751+0000 7fd74213a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c0fb5f0 con 0x7fd73c110150 2026-02-25T11:50:12.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.751+0000 7fd74213a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c0fb7b0 con 0x7fd73c115950 2026-02-25T11:50:12.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.751+0000 7fd73b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd73c10f760 0x7fd73c0f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.751+0000 7fd73b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd73c10f760 0x7fd73c0f5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42338/0 (socket says 10.20.193.28:42338) 2026-02-25T11:50:12.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.751+0000 7fd73b7fe640 1 -- 10.20.193.28:0/331937551 learned_addr learned my addr 10.20.193.28:0/331937551 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:12.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.752+0000 7fd73affd640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd73c110150 0x7fd73c176220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.752+0000 7fd73bfff640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd73c115950 0x7fd73c0f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.752+0000 7fd73b7fe640 1 -- 10.20.193.28:0/331937551 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd73c115950 msgr2=0x7fd73c0f6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.752+0000 7fd73b7fe640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd73c115950 0x7fd73c0f6270 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.752+0000 7fd73b7fe640 1 -- 10.20.193.28:0/331937551 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd73c110150 msgr2=0x7fd73c176220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.752+0000 7fd73b7fe640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd73c110150 0x7fd73c176220 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.752+0000 7fd73b7fe640 1 -- 10.20.193.28:0/331937551 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c009570 con 0x7fd73c10f760 2026-02-25T11:50:12.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.752+0000 7fd73b7fe640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd73c10f760 0x7fd73c0f5b70 secure :-1 s=READY pgs=484 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd728005db0 tx=0x7fd72800ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:12.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.752+0000 7fd738ff9640 1 -- 10.20.193.28:0/331937551 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd728004500 con 0x7fd73c10f760 2026-02-25T11:50:12.757 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.752+0000 7fd738ff9640 1 -- 10.20.193.28:0/331937551 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd728004d10 con 0x7fd73c10f760 2026-02-25T11:50:12.757 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.752+0000 7fd738ff9640 1 -- 10.20.193.28:0/331937551 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd728005280 con 0x7fd73c10f760 2026-02-25T11:50:12.757 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.752+0000 7fd74213a640 1 -- 10.20.193.28:0/331937551 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c0fba90 con 0x7fd73c10f760 2026-02-25T11:50:12.757 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.752+0000 7fd74213a640 1 -- 10.20.193.28:0/331937551 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c0fbf10 con 0x7fd73c10f760 2026-02-25T11:50:12.757 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.755+0000 7fd738ff9640 1 -- 10.20.193.28:0/331937551 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fd728020020 con 0x7fd73c10f760 2026-02-25T11:50:12.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.757+0000 7fd738ff9640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd7100b5750 0x7fd7100d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.757+0000 7fd738ff9640 1 -- 10.20.193.28:0/331937551 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd7280e66b0 con 0x7fd73c10f760 2026-02-25T11:50:12.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.758+0000 7fd73affd640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd7100b5750 0x7fd7100d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.758+0000 7fd744135e00 1 Processor -- start 2026-02-25T11:50:12.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.758+0000 7fd744135e00 1 -- start start 2026-02-25T11:50:12.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.758+0000 7fd744135e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dcb30 0x55ccec2dcf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.758+0000 7fd73affd640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd7100b5750 0x7fd7100d5c00 secure :-1 s=READY pgs=284 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd72c0023d0 tx=0x7fd72c031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:12.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.758+0000 7fd744135e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec321ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.758+0000 7fd744135e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec3220e0 0x55ccec3424c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.758+0000 7fd744135e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ccec1f28a0 con 0x55ccec2dd440 2026-02-25T11:50:12.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.758+0000 7fd744135e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ccec1ee5d0 con 0x55ccec2dcb30 2026-02-25T11:50:12.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.758+0000 7fd744135e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ccec1e8570 con 0x55ccec3220e0 2026-02-25T11:50:12.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.758+0000 7fd73affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec321ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.758+0000 7fd73affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec321ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42340/0 (socket says 10.20.193.28:42340) 2026-02-25T11:50:12.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.758+0000 7fd73affd640 1 -- 10.20.193.28:0/1010813374 learned_addr learned my addr 10.20.193.28:0/1010813374 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:12.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.759+0000 7fd73b7fe640 1 --2- 10.20.193.28:0/1010813374 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dcb30 0x55ccec2dcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.759+0000 7fd73bfff640 1 --2- 10.20.193.28:0/1010813374 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec3220e0 0x55ccec3424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.759+0000 7fd73affd640 1 -- 10.20.193.28:0/1010813374 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec3220e0 msgr2=0x55ccec3424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:12.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.759+0000 7fd73affd640 1 --2- 10.20.193.28:0/1010813374 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec3220e0 0x55ccec3424c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.759+0000 7fd73affd640 1 -- 10.20.193.28:0/1010813374 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dcb30 msgr2=0x55ccec2dcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:12.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.759+0000 7fd73affd640 1 --2- 10.20.193.28:0/1010813374 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dcb30 0x55ccec2dcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.759+0000 7fd73affd640 1 -- 10.20.193.28:0/1010813374 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccec193d10 con 0x55ccec2dd440 2026-02-25T11:50:12.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.759+0000 7fd73affd640 1 --2- 10.20.193.28:0/1010813374 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec321ba0 secure :-1 s=READY pgs=485 gs=2 cs=0 l=1 c_cookie=5567949456605535532 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd72c009950 tx=0x7fd72c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d455487d04d5d2c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:12.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.759+0000 7fd6fffff640 1 -- 10.20.193.28:0/1010813374 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd72c02fc10 con 0x55ccec2dd440 2026-02-25T11:50:12.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.759+0000 7fd6fffff640 1 -- 10.20.193.28:0/1010813374 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd72c0029d0 con 0x55ccec2dd440 2026-02-25T11:50:12.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.759+0000 7fd6fffff640 1 -- 10.20.193.28:0/1010813374 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd72c04f810 con 0x55ccec2dd440 2026-02-25T11:50:12.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.760+0000 7fd744135e00 1 -- 10.20.193.28:0/1010813374 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 msgr2=0x55ccec321ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.760+0000 7fd744135e00 1 --2- 10.20.193.28:0/1010813374 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec321ba0 secure :-1 s=READY pgs=485 gs=2 cs=0 l=1 c_cookie=5567949456605535532 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd72c009950 tx=0x7fd72c040000 comp rx=0 tx=0).stop 2026-02-25T11:50:12.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.760+0000 7fd744135e00 1 -- 10.20.193.28:0/1010813374 shutdown_connections 2026-02-25T11:50:12.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.760+0000 7fd744135e00 1 --2- 10.20.193.28:0/1010813374 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec3220e0 0x55ccec3424c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.760+0000 7fd744135e00 1 --2- 10.20.193.28:0/1010813374 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec321ba0 unknown :-1 s=CLOSED pgs=485 gs=2 cs=0 l=1 c_cookie=5567949456605535532 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.760+0000 7fd744135e00 1 --2- 10.20.193.28:0/1010813374 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dcb30 0x55ccec2dcf00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.760+0000 7fd744135e00 1 -- 10.20.193.28:0/1010813374 >> 10.20.193.28:0/1010813374 conn(0x55ccec1fb7a0 msgr2=0x55ccec2db410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:12.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.760+0000 7fd744135e00 1 -- 10.20.193.28:0/1010813374 shutdown_connections 2026-02-25T11:50:12.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.760+0000 7fd744135e00 1 -- 10.20.193.28:0/1010813374 wait complete. 2026-02-25T11:50:12.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.761+0000 7fd744135e00 1 -- start start 2026-02-25T11:50:12.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.761+0000 7fd744135e00 1 Processor -- start 2026-02-25T11:50:12.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd744135e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec2dcb30 0x55ccec2f4d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd744135e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dd440 0x55ccec2f5260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd744135e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec3220e0 0x55ccec2fa250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd744135e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ccec2fa790 con 0x55ccec3220e0 2026-02-25T11:50:12.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd744135e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ccec2fa900 con 0x55ccec2dd440 2026-02-25T11:50:12.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd744135e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ccec2faac0 con 0x55ccec2dcb30 2026-02-25T11:50:12.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd73bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec3220e0 0x55ccec2fa250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd73bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec3220e0 0x55ccec2fa250 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42344/0 (socket says 10.20.193.28:42344) 2026-02-25T11:50:12.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd73bfff640 1 -- 10.20.193.28:0/3173081950 learned_addr learned my addr 10.20.193.28:0/3173081950 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:12.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd73b7fe640 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec2dcb30 0x55ccec2f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd73affd640 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dd440 0x55ccec2f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd73bfff640 1 -- 10.20.193.28:0/3173081950 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec2dcb30 msgr2=0x55ccec2f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:12.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd73bfff640 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec2dcb30 0x55ccec2f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd73bfff640 1 -- 10.20.193.28:0/3173081950 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dd440 msgr2=0x55ccec2f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:12.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd73bfff640 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dd440 0x55ccec2f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd73bfff640 1 -- 10.20.193.28:0/3173081950 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccec16c3c0 con 0x55ccec3220e0 2026-02-25T11:50:12.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd73b7fe640 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec2dcb30 0x55ccec2f4d20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:12.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd73affd640 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dd440 0x55ccec2f5260 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:12.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd73bfff640 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec3220e0 0x55ccec2fa250 secure :-1 s=READY pgs=486 gs=1 cs=0 l=1 c_cookie=9749186743261512920 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd730003bc0 tx=0x7fd7300127b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874c11b0890cc4d8 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:12.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.762+0000 7fd6fd7fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:12.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.763+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/3173081950 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd73001c690 con 0x55ccec3220e0 2026-02-25T11:50:12.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.763+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/3173081950 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd73001ccb0 con 0x55ccec3220e0 2026-02-25T11:50:12.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.763+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/3173081950 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd730013e60 con 0x55ccec3220e0 2026-02-25T11:50:12.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.763+0000 7fd744135e00 1 -- 10.20.193.28:0/3173081950 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c176b90 con 0x55ccec3220e0 2026-02-25T11:50:12.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.763+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/3173081950 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd7300103d0 con 0x55ccec3220e0 2026-02-25T11:50:12.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.763+0000 7fd744135e00 10 client.15915 _ll_register_callbacks cb 0x55ccec169a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:12.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.763+0000 7fd744135e00 1 -- 10.20.193.28:0/3173081950 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ccebe7e6f0 con 0x55ccec3220e0 2026-02-25T11:50:12.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.763+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/3173081950 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd73001c7f0 con 0x55ccec3220e0 2026-02-25T11:50:12.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.763+0000 7fd744135e00 10 client.15915 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:12.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.763+0000 7fd744135e00 1 -- 10.20.193.28:0/3173081950 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55ccec17e0e0 con 0x55ccec3220e0 2026-02-25T11:50:12.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.763+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/3173081950 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd73001c950 con 0x55ccec3220e0 2026-02-25T11:50:12.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.763+0000 7fd744135e00 10 client.15915 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:12.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.763+0000 7fd744135e00 10 client.15915 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:50:12.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.764+0000 7fd744135e00 20 client.15915 populate_metadata read hostname 'trial028' 2026-02-25T11:50:12.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.764+0000 7fd744135e00 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55ccec31f5e0 0x55ccec31fa80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.764+0000 7fd744135e00 4 client.15915 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:50:12.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.764+0000 7fd744135e00 1 -- 10.20.193.28:0/3173081950 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccec1bafc0 con 0x55ccec31f5e0 2026-02-25T11:50:12.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.764+0000 7fd73b7fe640 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55ccec31f5e0 0x55ccec31fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.765+0000 7fd73b7fe640 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55ccec31f5e0 0x55ccec31fa80 crc :-1 s=READY pgs=31 gs=4 cs=0 l=0 c_cookie=2816694687309311127 s_cookie=3561025313003658984 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2716e8e505cd2c97 server_cookie=316b4da7b1c1dee8 in_seq=0 out_seq=0 2026-02-25T11:50:12.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.765+0000 7fd6fd7fa640 10 client.15915 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:50:12.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.765+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/3173081950 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55ccec1bafc0 con 0x55ccec31f5e0 2026-02-25T11:50:12.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.765+0000 7fd6fd7fa640 10 client.15915 handle_command_reply: tid=0 2026-02-25T11:50:12.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.766+0000 7fd744135e00 1 client.15915 shutdown 2026-02-25T11:50:12.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.766+0000 7fd744135e00 1 -- 10.20.193.28:0/3173081950 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec3220e0 msgr2=0x55ccec2fa250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.766+0000 7fd744135e00 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec3220e0 0x55ccec2fa250 secure :-1 s=READY pgs=486 gs=1 cs=0 l=1 c_cookie=9749186743261512920 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd730003bc0 tx=0x7fd7300127b0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.766+0000 7fd744135e00 1 -- 10.20.193.28:0/3173081950 shutdown_connections 2026-02-25T11:50:12.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.766+0000 7fd744135e00 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55ccec31f5e0 0x55ccec31fa80 crc :-1 s=READY pgs=31 gs=4 cs=0 l=0 c_cookie=2816694687309311127 s_cookie=3561025313003658984 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.766+0000 7fd744135e00 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec3220e0 0x55ccec2fa250 unknown :-1 s=CLOSED pgs=486 gs=1 cs=0 l=1 c_cookie=9749186743261512920 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.766+0000 7fd744135e00 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dd440 0x55ccec2f5260 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.766+0000 7fd744135e00 1 --2- 10.20.193.28:0/3173081950 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec2dcb30 0x55ccec2f4d20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.766+0000 7fd6fd7fa640 0 client.15915 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:50:12.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.766+0000 7fd744135e00 1 -- 10.20.193.28:0/3173081950 >> 10.20.193.28:0/3173081950 conn(0x55ccec1fb7a0 msgr2=0x55ccec2dbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:12.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.766+0000 7fd744135e00 1 -- 10.20.193.28:0/3173081950 shutdown_connections 2026-02-25T11:50:12.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.766+0000 7fd744135e00 1 -- 10.20.193.28:0/3173081950 wait complete. 2026-02-25T11:50:12.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.766+0000 7fd744135e00 20 client.15915 trim_cache size 0 max 16384 2026-02-25T11:50:12.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.773+0000 7fd744135e00 1 Processor -- start 2026-02-25T11:50:12.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.773+0000 7fd744135e00 1 -- start start 2026-02-25T11:50:12.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd744135e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dcb30 0x55ccec2f2320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd744135e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec321150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd744135e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec3220e0 0x55ccec321690 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd744135e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ccec1e61a0 con 0x55ccec2dd440 2026-02-25T11:50:12.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd744135e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ccebea6700 con 0x55ccec2dcb30 2026-02-25T11:50:12.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd744135e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ccec2b7e80 con 0x55ccec3220e0 2026-02-25T11:50:12.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd73affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec321150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd73affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec321150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42352/0 (socket says 10.20.193.28:42352) 2026-02-25T11:50:12.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd73affd640 1 -- 10.20.193.28:0/340532987 learned_addr learned my addr 10.20.193.28:0/340532987 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:12.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd73affd640 1 -- 10.20.193.28:0/340532987 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec3220e0 msgr2=0x55ccec321690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:12.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd73affd640 1 --2- 10.20.193.28:0/340532987 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec3220e0 0x55ccec321690 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd73affd640 1 -- 10.20.193.28:0/340532987 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dcb30 msgr2=0x55ccec2f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:12.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd73affd640 1 --2- 10.20.193.28:0/340532987 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dcb30 0x55ccec2f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd73affd640 1 -- 10.20.193.28:0/340532987 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccec178b60 con 0x55ccec2dd440 2026-02-25T11:50:12.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.774+0000 7fd73affd640 1 --2- 10.20.193.28:0/340532987 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec321150 secure :-1 s=READY pgs=487 gs=1 cs=0 l=1 c_cookie=886889857684963372 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd72c052040 tx=0x7fd72c0046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4edda06a35802c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:12.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.775+0000 7fd6ff7fe640 1 -- 10.20.193.28:0/340532987 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd72c02fcd0 con 0x55ccec2dd440 2026-02-25T11:50:12.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.775+0000 7fd6ff7fe640 1 -- 10.20.193.28:0/340532987 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd72c02fe30 con 0x55ccec2dd440 2026-02-25T11:50:12.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.775+0000 7fd6ff7fe640 1 -- 10.20.193.28:0/340532987 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd72c059a60 con 0x55ccec2dd440 2026-02-25T11:50:12.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.775+0000 7fd744135e00 1 -- 10.20.193.28:0/340532987 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 msgr2=0x55ccec321150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.775+0000 7fd744135e00 1 --2- 10.20.193.28:0/340532987 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec321150 secure :-1 s=READY pgs=487 gs=1 cs=0 l=1 c_cookie=886889857684963372 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd72c052040 tx=0x7fd72c0046a0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.775+0000 7fd744135e00 1 -- 10.20.193.28:0/340532987 shutdown_connections 2026-02-25T11:50:12.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.775+0000 7fd744135e00 1 --2- 10.20.193.28:0/340532987 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec3220e0 0x55ccec321690 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.775+0000 7fd744135e00 1 --2- 10.20.193.28:0/340532987 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec321150 unknown :-1 s=CLOSED pgs=487 gs=1 cs=0 l=1 c_cookie=886889857684963372 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.775+0000 7fd744135e00 1 --2- 10.20.193.28:0/340532987 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec2dcb30 0x55ccec2f2320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.775+0000 7fd744135e00 1 -- 10.20.193.28:0/340532987 >> 10.20.193.28:0/340532987 conn(0x55ccec1fb7a0 msgr2=0x55ccec2dad10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:12.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.775+0000 7fd744135e00 1 -- 10.20.193.28:0/340532987 shutdown_connections 2026-02-25T11:50:12.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.775+0000 7fd744135e00 1 -- 10.20.193.28:0/340532987 wait complete. 2026-02-25T11:50:12.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.776+0000 7fd744135e00 1 -- start start 2026-02-25T11:50:12.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.776+0000 7fd744135e00 1 Processor -- start 2026-02-25T11:50:12.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.776+0000 7fd744135e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec2dcb30 0x55ccec2fc740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.776+0000 7fd744135e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec2fcc80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.776+0000 7fd744135e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec3220e0 0x55ccec2fd1c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.776+0000 7fd744135e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ccec2dc6c0 con 0x55ccec2dd440 2026-02-25T11:50:12.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.776+0000 7fd744135e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ccec1e61a0 con 0x55ccec3220e0 2026-02-25T11:50:12.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.776+0000 7fd744135e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ccec370030 con 0x55ccec2dcb30 2026-02-25T11:50:12.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd73affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec2fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd73affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec2fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42356/0 (socket says 10.20.193.28:42356) 2026-02-25T11:50:12.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd73affd640 1 -- 10.20.193.28:0/104181991 learned_addr learned my addr 10.20.193.28:0/104181991 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:12.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd73bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec3220e0 0x55ccec2fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd73b7fe640 1 --2- 10.20.193.28:0/104181991 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec2dcb30 0x55ccec2fc740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd73affd640 1 -- 10.20.193.28:0/104181991 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec2dcb30 msgr2=0x55ccec2fc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:12.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd73affd640 1 --2- 10.20.193.28:0/104181991 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec2dcb30 0x55ccec2fc740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd73affd640 1 -- 10.20.193.28:0/104181991 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec3220e0 msgr2=0x55ccec2fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:12.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd73affd640 1 --2- 10.20.193.28:0/104181991 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec3220e0 0x55ccec2fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd73affd640 1 -- 10.20.193.28:0/104181991 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccec16c6f0 con 0x55ccec2dd440 2026-02-25T11:50:12.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd73affd640 1 --2- 10.20.193.28:0/104181991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec2fcc80 secure :-1 s=READY pgs=488 gs=1 cs=0 l=1 c_cookie=6255544947851030050 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd72c02fe00 tx=0x7fd72c0599f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d028fc9d266e22 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:12.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd6fd7fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:12.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/104181991 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd72c038ad0 con 0x55ccec2dd440 2026-02-25T11:50:12.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/104181991 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd72c038c30 con 0x55ccec2dd440 2026-02-25T11:50:12.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/104181991 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd72c04f400 con 0x55ccec2dd440 2026-02-25T11:50:12.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.777+0000 7fd744135e00 1 -- 10.20.193.28:0/104181991 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccec193d10 con 0x55ccec2dd440 2026-02-25T11:50:12.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/104181991 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd72c04f590 con 0x55ccec2dd440 2026-02-25T11:50:12.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd6fd7fa640 10 client.15921 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:50:12.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd744135e00 10 client.15921 _ll_register_callbacks cb 0x55ccec169a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:12.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd744135e00 1 -- 10.20.193.28:0/104181991 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd72c0029d0 con 0x55ccec2dd440 2026-02-25T11:50:12.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/104181991 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd72c038da0 con 0x55ccec2dd440 2026-02-25T11:50:12.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd744135e00 10 client.15921 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:12.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd744135e00 1 -- 10.20.193.28:0/104181991 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55ccec17e340 con 0x55ccec2dd440 2026-02-25T11:50:12.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/104181991 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd72c062040 con 0x55ccec2dd440 2026-02-25T11:50:12.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd744135e00 10 client.15921 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:12.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd744135e00 10 client.15921 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:50:12.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd744135e00 20 client.15921 populate_metadata read hostname 'trial028' 2026-02-25T11:50:12.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd744135e00 1 --2- 10.20.193.28:0/104181991 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55ccec371740 0x55ccec371be0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:12.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd744135e00 4 client.15921 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:50:12.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.778+0000 7fd744135e00 1 -- 10.20.193.28:0/104181991 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccebe7bf40 con 0x55ccec371740 2026-02-25T11:50:12.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.779+0000 7fd73b7fe640 1 --2- 10.20.193.28:0/104181991 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55ccec371740 0x55ccec371be0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:12.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.779+0000 7fd73b7fe640 1 --2- 10.20.193.28:0/104181991 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55ccec371740 0x55ccec371be0 crc :-1 s=READY pgs=32 gs=4 cs=0 l=0 c_cookie=6978671948646346261 s_cookie=6119527906338380205 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60d93929a70c6e15 server_cookie=54ecee354a6399ad in_seq=0 out_seq=0 2026-02-25T11:50:12.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.779+0000 7fd6fd7fa640 10 client.15921 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:50:12.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd6fd7fa640 1 -- 10.20.193.28:0/104181991 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccebe7bf40 con 0x55ccec371740 2026-02-25T11:50:12.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd6fd7fa640 10 client.15921 handle_command_reply: tid=0 2026-02-25T11:50:12.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd744135e00 1 client.15921 shutdown 2026-02-25T11:50:12.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd744135e00 1 -- 10.20.193.28:0/104181991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 msgr2=0x55ccec2fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd744135e00 1 --2- 10.20.193.28:0/104181991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec2fcc80 secure :-1 s=READY pgs=488 gs=1 cs=0 l=1 c_cookie=6255544947851030050 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd72c02fe00 tx=0x7fd72c0599f0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd744135e00 1 -- 10.20.193.28:0/104181991 shutdown_connections 2026-02-25T11:50:12.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd744135e00 1 --2- 10.20.193.28:0/104181991 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55ccec371740 0x55ccec371be0 crc :-1 s=READY pgs=32 gs=4 cs=0 l=0 c_cookie=6978671948646346261 s_cookie=6119527906338380205 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd744135e00 1 --2- 10.20.193.28:0/104181991 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ccec3220e0 0x55ccec2fd1c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd744135e00 1 --2- 10.20.193.28:0/104181991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ccec2dd440 0x55ccec2fcc80 unknown :-1 s=CLOSED pgs=488 gs=1 cs=0 l=1 c_cookie=6255544947851030050 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd744135e00 1 --2- 10.20.193.28:0/104181991 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ccec2dcb30 0x55ccec2fc740 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd6fd7fa640 0 client.15921 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:50:12.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd744135e00 1 -- 10.20.193.28:0/104181991 >> 10.20.193.28:0/104181991 conn(0x55ccec1fb7a0 msgr2=0x55ccec2db500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:12.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd744135e00 1 -- 10.20.193.28:0/104181991 shutdown_connections 2026-02-25T11:50:12.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd744135e00 1 -- 10.20.193.28:0/104181991 wait complete. 2026-02-25T11:50:12.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.780+0000 7fd744135e00 20 client.15921 trim_cache size 0 max 16384 2026-02-25T11:50:12.788 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:50:12.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.781+0000 7fd74213a640 1 -- 10.20.193.28:0/331937551 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd7100b5750 msgr2=0x7fd7100d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.781+0000 7fd74213a640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd7100b5750 0x7fd7100d5c00 secure :-1 s=READY pgs=284 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd72c0023d0 tx=0x7fd72c031040 comp rx=0 tx=0).stop 2026-02-25T11:50:12.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.781+0000 7fd74213a640 1 -- 10.20.193.28:0/331937551 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd73c10f760 msgr2=0x7fd73c0f5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:12.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.781+0000 7fd74213a640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd73c10f760 0x7fd73c0f5b70 secure :-1 s=READY pgs=484 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd728005db0 tx=0x7fd72800ddf0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.781+0000 7fd74213a640 1 -- 10.20.193.28:0/331937551 shutdown_connections 2026-02-25T11:50:12.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.781+0000 7fd74213a640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd7100b5750 0x7fd7100d5c00 unknown :-1 s=CLOSED pgs=284 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.781+0000 7fd74213a640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd73c115950 0x7fd73c0f6270 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.781+0000 7fd74213a640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd73c110150 0x7fd73c176220 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.781+0000 7fd74213a640 1 --2- 10.20.193.28:0/331937551 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd73c10f760 0x7fd73c0f5b70 unknown :-1 s=CLOSED pgs=484 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:12.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.781+0000 7fd74213a640 1 -- 10.20.193.28:0/331937551 >> 10.20.193.28:0/331937551 conn(0x7fd73c0ed050 msgr2=0x7fd73c111f60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:12.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.781+0000 7fd74213a640 1 -- 10.20.193.28:0/331937551 shutdown_connections 2026-02-25T11:50:12.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:12.781+0000 7fd74213a640 1 -- 10.20.193.28:0/331937551 wait complete. 2026-02-25T11:50:12.880 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:50:13.065 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:13.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.233+0000 7fe47586a640 1 -- 10.20.193.28:0/2373700549 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe47010ddc0 msgr2=0x7fe470112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.233+0000 7fe47586a640 1 --2- 10.20.193.28:0/2373700549 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe47010ddc0 0x7fe470112e10 secure :-1 s=READY pgs=489 gs=1 cs=0 l=1 c_cookie=15243961577931876577 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe458009950 tx=0x7fe45802f190 comp rx=0 tx=0).stop 2026-02-25T11:50:13.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.234+0000 7fe47586a640 1 -- 10.20.193.28:0/2373700549 shutdown_connections 2026-02-25T11:50:13.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.234+0000 7fe47586a640 1 --2- 10.20.193.28:0/2373700549 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe4701135c0 0x7fe470173f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.234+0000 7fe47586a640 1 --2- 10.20.193.28:0/2373700549 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe47010ddc0 0x7fe470112e10 unknown :-1 s=CLOSED pgs=489 gs=1 cs=0 l=1 c_cookie=15243961577931876577 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.234+0000 7fe47586a640 1 --2- 10.20.193.28:0/2373700549 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe47010d3d0 0x7fe47010d7a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.234+0000 7fe47586a640 1 -- 10.20.193.28:0/2373700549 >> 10.20.193.28:0/2373700549 conn(0x7fe4700eaca0 msgr2=0x7fe47010b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:13.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.234+0000 7fe47586a640 1 -- 10.20.193.28:0/2373700549 shutdown_connections 2026-02-25T11:50:13.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.234+0000 7fe47586a640 1 -- 10.20.193.28:0/2373700549 wait complete. 2026-02-25T11:50:13.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.234+0000 7fe47586a640 1 Processor -- start 2026-02-25T11:50:13.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe47586a640 1 -- start start 2026-02-25T11:50:13.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe47586a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe47010d3d0 0x7fe4700f3060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe47586a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe47010ddc0 0x7fe470173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe47586a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe4701135c0 0x7fe4700ef330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe47586a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe4700efa30 con 0x7fe47010ddc0 2026-02-25T11:50:13.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe47586a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe4700efba0 con 0x7fe4701135c0 2026-02-25T11:50:13.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe47586a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4700efd60 con 0x7fe47010d3d0 2026-02-25T11:50:13.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe467fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe47010ddc0 0x7fe470173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe467fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe47010ddc0 0x7fe470173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42374/0 (socket says 10.20.193.28:42374) 2026-02-25T11:50:13.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe467fff640 1 -- 10.20.193.28:0/1166508277 learned_addr learned my addr 10.20.193.28:0/1166508277 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:13.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe475069640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe4701135c0 0x7fe4700ef330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe474868640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe47010d3d0 0x7fe4700f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe467fff640 1 -- 10.20.193.28:0/1166508277 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe47010d3d0 msgr2=0x7fe4700f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe467fff640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe47010d3d0 0x7fe4700f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe467fff640 1 -- 10.20.193.28:0/1166508277 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe4701135c0 msgr2=0x7fe4700ef330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.235+0000 7fe467fff640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe4701135c0 0x7fe4700ef330 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.236+0000 7fe467fff640 1 -- 10.20.193.28:0/1166508277 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe458009570 con 0x7fe47010ddc0 2026-02-25T11:50:13.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.236+0000 7fe467fff640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe47010ddc0 0x7fe470173e90 secure :-1 s=READY pgs=490 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe458004820 tx=0x7fe4580028b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:13.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.236+0000 7fe465ffb640 1 -- 10.20.193.28:0/1166508277 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe45803d070 con 0x7fe47010ddc0 2026-02-25T11:50:13.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.236+0000 7fe465ffb640 1 -- 10.20.193.28:0/1166508277 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe45802fe20 con 0x7fe47010ddc0 2026-02-25T11:50:13.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.236+0000 7fe465ffb640 1 -- 10.20.193.28:0/1166508277 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe458041960 con 0x7fe47010ddc0 2026-02-25T11:50:13.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.236+0000 7fe47586a640 1 -- 10.20.193.28:0/1166508277 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4700effe0 con 0x7fe47010ddc0 2026-02-25T11:50:13.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.236+0000 7fe47586a640 1 -- 10.20.193.28:0/1166508277 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4700f5a20 con 0x7fe47010ddc0 2026-02-25T11:50:13.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.239+0000 7fe465ffb640 1 -- 10.20.193.28:0/1166508277 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fe458041b50 con 0x7fe47010ddc0 2026-02-25T11:50:13.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.240+0000 7fe465ffb640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe44c0b5760 0x7fe44c0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.240+0000 7fe465ffb640 1 -- 10.20.193.28:0/1166508277 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fe45810b090 con 0x7fe47010ddc0 2026-02-25T11:50:13.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.241+0000 7fe474868640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe44c0b5760 0x7fe44c0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.241+0000 7fe474868640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe44c0b5760 0x7fe44c0d5c10 secure :-1 s=READY pgs=285 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe460004740 tx=0x7fe460005f70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:13.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe478aeee00 1 Processor -- start 2026-02-25T11:50:13.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe478aeee00 1 -- start start 2026-02-25T11:50:13.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe478aeee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 0x55c59c66af00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe478aeee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66b440 0x55c59c6afba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe478aeee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c6b00e0 0x55c59c6d04c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe478aeee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55c59c5808a0 con 0x55c59c66ab30 2026-02-25T11:50:13.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe478aeee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55c59c57c5d0 con 0x55c59c66b440 2026-02-25T11:50:13.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe478aeee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55c59c576570 con 0x55c59c6b00e0 2026-02-25T11:50:13.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe474868640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 0x55c59c66af00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe474868640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 0x55c59c66af00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42376/0 (socket says 10.20.193.28:42376) 2026-02-25T11:50:13.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe474868640 1 -- 10.20.193.28:0/2478426008 learned_addr learned my addr 10.20.193.28:0/2478426008 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:13.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe474868640 1 -- 10.20.193.28:0/2478426008 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c6b00e0 msgr2=0x55c59c6d04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe474868640 1 --2- 10.20.193.28:0/2478426008 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c6b00e0 0x55c59c6d04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe467fff640 1 --2- 10.20.193.28:0/2478426008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66b440 0x55c59c6afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe474868640 1 -- 10.20.193.28:0/2478426008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66b440 msgr2=0x55c59c6afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe474868640 1 --2- 10.20.193.28:0/2478426008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66b440 0x55c59c6afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe474868640 1 -- 10.20.193.28:0/2478426008 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c59c521d10 con 0x55c59c66ab30 2026-02-25T11:50:13.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.242+0000 7fe467fff640 1 --2- 10.20.193.28:0/2478426008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66b440 0x55c59c6afba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:13.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.243+0000 7fe474868640 1 --2- 10.20.193.28:0/2478426008 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 0x55c59c66af00 secure :-1 s=READY pgs=491 gs=1 cs=0 l=1 c_cookie=7047099379094812423 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe460005e40 tx=0x7fe460011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cc538c53d27707 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:13.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.243+0000 7fe43d4e4640 1 -- 10.20.193.28:0/2478426008 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe46001d680 con 0x55c59c66ab30 2026-02-25T11:50:13.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.243+0000 7fe43d4e4640 1 -- 10.20.193.28:0/2478426008 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe460002e00 con 0x55c59c66ab30 2026-02-25T11:50:13.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.243+0000 7fe43d4e4640 1 -- 10.20.193.28:0/2478426008 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe460010c40 con 0x55c59c66ab30 2026-02-25T11:50:13.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.243+0000 7fe478aeee00 1 -- 10.20.193.28:0/2478426008 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 msgr2=0x55c59c66af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.243+0000 7fe478aeee00 1 --2- 10.20.193.28:0/2478426008 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 0x55c59c66af00 secure :-1 s=READY pgs=491 gs=1 cs=0 l=1 c_cookie=7047099379094812423 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe460005e40 tx=0x7fe460011000 comp rx=0 tx=0).stop 2026-02-25T11:50:13.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.243+0000 7fe478aeee00 1 -- 10.20.193.28:0/2478426008 shutdown_connections 2026-02-25T11:50:13.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.243+0000 7fe478aeee00 1 --2- 10.20.193.28:0/2478426008 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c6b00e0 0x55c59c6d04c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.243+0000 7fe478aeee00 1 --2- 10.20.193.28:0/2478426008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66b440 0x55c59c6afba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.243+0000 7fe478aeee00 1 --2- 10.20.193.28:0/2478426008 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 0x55c59c66af00 unknown :-1 s=CLOSED pgs=491 gs=1 cs=0 l=1 c_cookie=7047099379094812423 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.243+0000 7fe478aeee00 1 -- 10.20.193.28:0/2478426008 >> 10.20.193.28:0/2478426008 conn(0x55c59c5897a0 msgr2=0x55c59c669410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:13.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.243+0000 7fe478aeee00 1 -- 10.20.193.28:0/2478426008 shutdown_connections 2026-02-25T11:50:13.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.243+0000 7fe478aeee00 1 -- 10.20.193.28:0/2478426008 wait complete. 2026-02-25T11:50:13.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.245+0000 7fe478aeee00 1 -- start start 2026-02-25T11:50:13.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.245+0000 7fe478aeee00 1 Processor -- start 2026-02-25T11:50:13.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.245+0000 7fe478aeee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66ab30 0x55c59c682ed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.245+0000 7fe478aeee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 0x55c59c683410 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.245+0000 7fe478aeee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c6b00e0 0x55c59c688400 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.245+0000 7fe478aeee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55c59c688940 con 0x55c59c66b440 2026-02-25T11:50:13.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.245+0000 7fe478aeee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55c59c688ab0 con 0x55c59c66ab30 2026-02-25T11:50:13.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.245+0000 7fe478aeee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55c59c688c70 con 0x55c59c6b00e0 2026-02-25T11:50:13.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.245+0000 7fe467fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 0x55c59c683410 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.245+0000 7fe474868640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66ab30 0x55c59c682ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.245+0000 7fe467fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 0x55c59c683410 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42390/0 (socket says 10.20.193.28:42390) 2026-02-25T11:50:13.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.245+0000 7fe467fff640 1 -- 10.20.193.28:0/3721407688 learned_addr learned my addr 10.20.193.28:0/3721407688 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:13.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe475069640 1 --2- 10.20.193.28:0/3721407688 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c6b00e0 0x55c59c688400 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe467fff640 1 -- 10.20.193.28:0/3721407688 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c6b00e0 msgr2=0x55c59c688400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe467fff640 1 --2- 10.20.193.28:0/3721407688 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c6b00e0 0x55c59c688400 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe467fff640 1 -- 10.20.193.28:0/3721407688 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66ab30 msgr2=0x55c59c682ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe467fff640 1 --2- 10.20.193.28:0/3721407688 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66ab30 0x55c59c682ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe467fff640 1 -- 10.20.193.28:0/3721407688 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c59c4fa3c0 con 0x55c59c66b440 2026-02-25T11:50:13.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe474868640 1 --2- 10.20.193.28:0/3721407688 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66ab30 0x55c59c682ed0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:13.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe467fff640 1 --2- 10.20.193.28:0/3721407688 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 0x55c59c683410 secure :-1 s=READY pgs=492 gs=3 cs=0 l=1 c_cookie=11211662838720055816 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe458004030 tx=0x7fe45802f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b97d3f32b5b4a08 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:13.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe42e7fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:13.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe42e7fc640 1 -- 10.20.193.28:0/3721407688 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe4580d7170 con 0x55c59c66b440 2026-02-25T11:50:13.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe42e7fc640 1 -- 10.20.193.28:0/3721407688 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe4580d9750 con 0x55c59c66b440 2026-02-25T11:50:13.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe42e7fc640 1 -- 10.20.193.28:0/3721407688 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe458035320 con 0x55c59c66b440 2026-02-25T11:50:13.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe478aeee00 1 -- 10.20.193.28:0/3721407688 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46000ac60 con 0x55c59c66b440 2026-02-25T11:50:13.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.246+0000 7fe42e7fc640 1 -- 10.20.193.28:0/3721407688 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fe458035480 con 0x55c59c66b440 2026-02-25T11:50:13.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.247+0000 7fe478aeee00 10 client.15933 _ll_register_callbacks cb 0x55c59c4f7a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:13.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.247+0000 7fe478aeee00 1 -- 10.20.193.28:0/3721407688 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c59c20c6f0 con 0x55c59c66b440 2026-02-25T11:50:13.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.247+0000 7fe42e7fc640 1 -- 10.20.193.28:0/3721407688 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe4580dc590 con 0x55c59c66b440 2026-02-25T11:50:13.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.247+0000 7fe478aeee00 10 client.15933 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:13.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.247+0000 7fe478aeee00 1 -- 10.20.193.28:0/3721407688 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55c59c50c0e0 con 0x55c59c66b440 2026-02-25T11:50:13.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.247+0000 7fe42e7fc640 1 -- 10.20.193.28:0/3721407688 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe4580ea050 con 0x55c59c66b440 2026-02-25T11:50:13.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.247+0000 7fe478aeee00 10 client.15933 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:13.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.247+0000 7fe478aeee00 10 client.15933 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:50:13.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.247+0000 7fe478aeee00 20 client.15933 populate_metadata read hostname 'trial028' 2026-02-25T11:50:13.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.247+0000 7fe478aeee00 1 --2- 10.20.193.28:0/3721407688 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c59c6ad340 0x55c59c6ad7e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.247+0000 7fe478aeee00 4 client.15933 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:50:13.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.247+0000 7fe478aeee00 1 -- 10.20.193.28:0/3721407688 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c59c548fc0 con 0x55c59c6ad340 2026-02-25T11:50:13.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.247+0000 7fe474868640 1 --2- 10.20.193.28:0/3721407688 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c59c6ad340 0x55c59c6ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.248+0000 7fe474868640 1 --2- 10.20.193.28:0/3721407688 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c59c6ad340 0x55c59c6ad7e0 crc :-1 s=READY pgs=32 gs=4 cs=0 l=0 c_cookie=16364293921879162982 s_cookie=8693161674048306134 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e319a69ec5b10066 server_cookie=78a4507bf4b07bd6 in_seq=0 out_seq=0 2026-02-25T11:50:13.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.248+0000 7fe42e7fc640 10 client.15933 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:50:13.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.248+0000 7fe42e7fc640 1 -- 10.20.193.28:0/3721407688 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55c59c548fc0 con 0x55c59c6ad340 2026-02-25T11:50:13.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.248+0000 7fe42e7fc640 10 client.15933 handle_command_reply: tid=0 2026-02-25T11:50:13.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.248+0000 7fe478aeee00 1 client.15933 shutdown 2026-02-25T11:50:13.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.248+0000 7fe478aeee00 1 -- 10.20.193.28:0/3721407688 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 msgr2=0x55c59c683410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.248+0000 7fe478aeee00 1 --2- 10.20.193.28:0/3721407688 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 0x55c59c683410 secure :-1 s=READY pgs=492 gs=3 cs=0 l=1 c_cookie=11211662838720055816 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe458004030 tx=0x7fe45802f190 comp rx=0 tx=0).stop 2026-02-25T11:50:13.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.249+0000 7fe478aeee00 1 -- 10.20.193.28:0/3721407688 shutdown_connections 2026-02-25T11:50:13.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.249+0000 7fe478aeee00 1 --2- 10.20.193.28:0/3721407688 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c59c6ad340 0x55c59c6ad7e0 crc :-1 s=READY pgs=32 gs=4 cs=0 l=0 c_cookie=16364293921879162982 s_cookie=8693161674048306134 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.249+0000 7fe478aeee00 1 --2- 10.20.193.28:0/3721407688 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c6b00e0 0x55c59c688400 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.249+0000 7fe478aeee00 1 --2- 10.20.193.28:0/3721407688 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 0x55c59c683410 unknown :-1 s=CLOSED pgs=492 gs=3 cs=0 l=1 c_cookie=11211662838720055816 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.249+0000 7fe478aeee00 1 --2- 10.20.193.28:0/3721407688 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66ab30 0x55c59c682ed0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.249+0000 7fe478aeee00 1 -- 10.20.193.28:0/3721407688 >> 10.20.193.28:0/3721407688 conn(0x55c59c5897a0 msgr2=0x55c59c669b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:13.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.249+0000 7fe42e7fc640 0 client.15933 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:50:13.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.249+0000 7fe478aeee00 1 -- 10.20.193.28:0/3721407688 shutdown_connections 2026-02-25T11:50:13.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.249+0000 7fe478aeee00 1 -- 10.20.193.28:0/3721407688 wait complete. 2026-02-25T11:50:13.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.249+0000 7fe478aeee00 20 client.15933 trim_cache size 0 max 16384 2026-02-25T11:50:13.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe478aeee00 1 Processor -- start 2026-02-25T11:50:13.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe478aeee00 1 -- start start 2026-02-25T11:50:13.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe478aeee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 0x55c59c6809b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe478aeee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66b440 0x55c59c6ae0c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe478aeee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c6b00e0 0x55c59c6ae600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe478aeee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55c59c5741a0 con 0x55c59c66ab30 2026-02-25T11:50:13.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe478aeee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55c59c234700 con 0x55c59c66b440 2026-02-25T11:50:13.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe478aeee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55c59c67c220 con 0x55c59c6b00e0 2026-02-25T11:50:13.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe474868640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 0x55c59c6809b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe474868640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 0x55c59c6809b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42394/0 (socket says 10.20.193.28:42394) 2026-02-25T11:50:13.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe474868640 1 -- 10.20.193.28:0/2191783084 learned_addr learned my addr 10.20.193.28:0/2191783084 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:13.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe474868640 1 -- 10.20.193.28:0/2191783084 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c6b00e0 msgr2=0x55c59c6ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe474868640 1 --2- 10.20.193.28:0/2191783084 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c6b00e0 0x55c59c6ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe474868640 1 -- 10.20.193.28:0/2191783084 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66b440 msgr2=0x55c59c6ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe474868640 1 --2- 10.20.193.28:0/2191783084 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66b440 0x55c59c6ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.256+0000 7fe474868640 1 -- 10.20.193.28:0/2191783084 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c59c506b60 con 0x55c59c66ab30 2026-02-25T11:50:13.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.257+0000 7fe474868640 1 --2- 10.20.193.28:0/2191783084 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 0x55c59c6809b0 secure :-1 s=READY pgs=493 gs=1 cs=0 l=1 c_cookie=5585481618092379515 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe460005e40 tx=0x7fe460004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d839df041decd7b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:13.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.257+0000 7fe43cce3640 1 -- 10.20.193.28:0/2191783084 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe46002e040 con 0x55c59c66ab30 2026-02-25T11:50:13.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.257+0000 7fe43cce3640 1 -- 10.20.193.28:0/2191783084 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe4600354f0 con 0x55c59c66ab30 2026-02-25T11:50:13.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.257+0000 7fe43cce3640 1 -- 10.20.193.28:0/2191783084 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe46001d6e0 con 0x55c59c66ab30 2026-02-25T11:50:13.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.257+0000 7fe478aeee00 1 -- 10.20.193.28:0/2191783084 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 msgr2=0x55c59c6809b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.257+0000 7fe478aeee00 1 --2- 10.20.193.28:0/2191783084 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 0x55c59c6809b0 secure :-1 s=READY pgs=493 gs=1 cs=0 l=1 c_cookie=5585481618092379515 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe460005e40 tx=0x7fe460004640 comp rx=0 tx=0).stop 2026-02-25T11:50:13.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.257+0000 7fe478aeee00 1 -- 10.20.193.28:0/2191783084 shutdown_connections 2026-02-25T11:50:13.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.257+0000 7fe478aeee00 1 --2- 10.20.193.28:0/2191783084 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c6b00e0 0x55c59c6ae600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.257+0000 7fe478aeee00 1 --2- 10.20.193.28:0/2191783084 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c66b440 0x55c59c6ae0c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.257+0000 7fe478aeee00 1 --2- 10.20.193.28:0/2191783084 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66ab30 0x55c59c6809b0 unknown :-1 s=CLOSED pgs=493 gs=1 cs=0 l=1 c_cookie=5585481618092379515 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.257+0000 7fe478aeee00 1 -- 10.20.193.28:0/2191783084 >> 10.20.193.28:0/2191783084 conn(0x55c59c5897a0 msgr2=0x55c59c67f290 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:13.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.257+0000 7fe478aeee00 1 -- 10.20.193.28:0/2191783084 shutdown_connections 2026-02-25T11:50:13.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.257+0000 7fe478aeee00 1 -- 10.20.193.28:0/2191783084 wait complete. 2026-02-25T11:50:13.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.258+0000 7fe478aeee00 1 -- start start 2026-02-25T11:50:13.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.258+0000 7fe478aeee00 1 Processor -- start 2026-02-25T11:50:13.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.258+0000 7fe478aeee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c66ab30 0x55c59c68ed40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.258+0000 7fe478aeee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 0x55c59c68c280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe478aeee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c6b00e0 0x55c59c68c7c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe478aeee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55c59c68cd80 con 0x55c59c66b440 2026-02-25T11:50:13.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe478aeee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55c59c68cef0 con 0x55c59c6b00e0 2026-02-25T11:50:13.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe478aeee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55c59c68d0b0 con 0x55c59c66ab30 2026-02-25T11:50:13.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe467fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 0x55c59c68c280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe467fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 0x55c59c68c280 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42400/0 (socket says 10.20.193.28:42400) 2026-02-25T11:50:13.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe467fff640 1 -- 10.20.193.28:0/1838746230 learned_addr learned my addr 10.20.193.28:0/1838746230 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:13.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe474868640 1 --2- 10.20.193.28:0/1838746230 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c66ab30 0x55c59c68ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe467fff640 1 -- 10.20.193.28:0/1838746230 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c66ab30 msgr2=0x55c59c68ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe467fff640 1 --2- 10.20.193.28:0/1838746230 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c66ab30 0x55c59c68ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe467fff640 1 -- 10.20.193.28:0/1838746230 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c6b00e0 msgr2=0x55c59c68c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe467fff640 1 --2- 10.20.193.28:0/1838746230 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c6b00e0 0x55c59c68c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe467fff640 1 -- 10.20.193.28:0/1838746230 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c59c4fa6f0 con 0x55c59c66b440 2026-02-25T11:50:13.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe467fff640 1 --2- 10.20.193.28:0/1838746230 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 0x55c59c68c280 secure :-1 s=READY pgs=494 gs=1 cs=0 l=1 c_cookie=3961331919542520381 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe458038fa0 tx=0x7fe458005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f97a648dbf8a3d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:13.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe42e7fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:13.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe42e7fc640 1 -- 10.20.193.28:0/1838746230 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe4580d7170 con 0x55c59c66b440 2026-02-25T11:50:13.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.259+0000 7fe478aeee00 1 -- 10.20.193.28:0/1838746230 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c59c521d10 con 0x55c59c66b440 2026-02-25T11:50:13.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe478aeee00 10 client.15939 _ll_register_callbacks cb 0x55c59c4f7a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:13.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe478aeee00 1 -- 10.20.193.28:0/1838746230 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe460002e00 con 0x55c59c66b440 2026-02-25T11:50:13.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe42e7fc640 1 -- 10.20.193.28:0/1838746230 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55c59c20c6f0 con 0x55c59c66b440 2026-02-25T11:50:13.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe42e7fc640 1 -- 10.20.193.28:0/1838746230 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe4580d0560 con 0x55c59c66b440 2026-02-25T11:50:13.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe42e7fc640 1 -- 10.20.193.28:0/1838746230 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fe4580dc120 con 0x55c59c66b440 2026-02-25T11:50:13.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe42e7fc640 1 -- 10.20.193.28:0/1838746230 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe4580dc3a0 con 0x55c59c66b440 2026-02-25T11:50:13.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe478aeee00 10 client.15939 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:13.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe478aeee00 1 -- 10.20.193.28:0/1838746230 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55c59c50c340 con 0x55c59c66b440 2026-02-25T11:50:13.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe42e7fc640 1 -- 10.20.193.28:0/1838746230 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe4580ea070 con 0x55c59c66b440 2026-02-25T11:50:13.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe478aeee00 10 client.15939 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:13.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe478aeee00 10 client.15939 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:50:13.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe478aeee00 20 client.15939 populate_metadata read hostname 'trial028' 2026-02-25T11:50:13.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe478aeee00 1 --2- 10.20.193.28:0/1838746230 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c59c75cb60 0x55c59c77cf40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe478aeee00 4 client.15939 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:50:13.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe478aeee00 1 -- 10.20.193.28:0/1838746230 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c59c209f40 con 0x55c59c75cb60 2026-02-25T11:50:13.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.260+0000 7fe474868640 1 --2- 10.20.193.28:0/1838746230 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c59c75cb60 0x55c59c77cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe474868640 1 --2- 10.20.193.28:0/1838746230 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c59c75cb60 0x55c59c77cf40 crc :-1 s=READY pgs=33 gs=4 cs=0 l=0 c_cookie=2259745225125653325 s_cookie=9147676616610359397 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f5c3a4f427feb4d server_cookie=7ef3136a95352865 in_seq=0 out_seq=0 2026-02-25T11:50:13.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe42e7fc640 10 client.15939 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:50:13.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe42e7fc640 1 -- 10.20.193.28:0/1838746230 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c59c209f40 con 0x55c59c75cb60 2026-02-25T11:50:13.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe42e7fc640 10 client.15939 handle_command_reply: tid=0 2026-02-25T11:50:13.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe478aeee00 1 client.15939 shutdown 2026-02-25T11:50:13.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe478aeee00 1 -- 10.20.193.28:0/1838746230 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 msgr2=0x55c59c68c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe478aeee00 1 --2- 10.20.193.28:0/1838746230 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 0x55c59c68c280 secure :-1 s=READY pgs=494 gs=1 cs=0 l=1 c_cookie=3961331919542520381 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe458038fa0 tx=0x7fe458005ab0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe478aeee00 1 -- 10.20.193.28:0/1838746230 shutdown_connections 2026-02-25T11:50:13.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe478aeee00 1 --2- 10.20.193.28:0/1838746230 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c59c75cb60 0x55c59c77cf40 crc :-1 s=READY pgs=33 gs=4 cs=0 l=0 c_cookie=2259745225125653325 s_cookie=9147676616610359397 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe478aeee00 1 --2- 10.20.193.28:0/1838746230 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c59c6b00e0 0x55c59c68c7c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe478aeee00 1 --2- 10.20.193.28:0/1838746230 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c59c66b440 0x55c59c68c280 unknown :-1 s=CLOSED pgs=494 gs=1 cs=0 l=1 c_cookie=3961331919542520381 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe478aeee00 1 --2- 10.20.193.28:0/1838746230 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c59c66ab30 0x55c59c68ed40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe478aeee00 1 -- 10.20.193.28:0/1838746230 >> 10.20.193.28:0/1838746230 conn(0x55c59c5897a0 msgr2=0x55c59c67fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:13.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe42e7fc640 0 client.15939 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:50:13.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe478aeee00 1 -- 10.20.193.28:0/1838746230 shutdown_connections 2026-02-25T11:50:13.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe478aeee00 1 -- 10.20.193.28:0/1838746230 wait complete. 2026-02-25T11:50:13.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.261+0000 7fe478aeee00 20 client.15939 trim_cache size 0 max 16384 2026-02-25T11:50:13.270 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:50:13.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.262+0000 7fe47586a640 1 -- 10.20.193.28:0/1166508277 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe44c0b5760 msgr2=0x7fe44c0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.262+0000 7fe47586a640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe44c0b5760 0x7fe44c0d5c10 secure :-1 s=READY pgs=285 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe460004740 tx=0x7fe460005f70 comp rx=0 tx=0).stop 2026-02-25T11:50:13.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.262+0000 7fe47586a640 1 -- 10.20.193.28:0/1166508277 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe47010ddc0 msgr2=0x7fe470173e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.262+0000 7fe47586a640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe47010ddc0 0x7fe470173e90 secure :-1 s=READY pgs=490 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe458004820 tx=0x7fe4580028b0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.262+0000 7fe47586a640 1 -- 10.20.193.28:0/1166508277 shutdown_connections 2026-02-25T11:50:13.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.262+0000 7fe47586a640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe44c0b5760 0x7fe44c0d5c10 unknown :-1 s=CLOSED pgs=285 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.262+0000 7fe47586a640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe4701135c0 0x7fe4700ef330 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.262+0000 7fe47586a640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe47010ddc0 0x7fe470173e90 unknown :-1 s=CLOSED pgs=490 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.262+0000 7fe47586a640 1 --2- 10.20.193.28:0/1166508277 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe47010d3d0 0x7fe4700f3060 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.262+0000 7fe47586a640 1 -- 10.20.193.28:0/1166508277 >> 10.20.193.28:0/1166508277 conn(0x7fe4700eaca0 msgr2=0x7fe47010fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:13.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.262+0000 7fe47586a640 1 -- 10.20.193.28:0/1166508277 shutdown_connections 2026-02-25T11:50:13.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.262+0000 7fe47586a640 1 -- 10.20.193.28:0/1166508277 wait complete. 2026-02-25T11:50:13.362 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:50:13.411 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:13 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/670868896' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:50:13.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:13 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/670868896' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:50:13.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:13 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/670868896' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:50:13.544 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:13.711 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.709+0000 7f9129d48640 1 -- 10.20.193.28:0/33236051 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f912410f690 msgr2=0x7f912410fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.709+0000 7f9129d48640 1 --2- 10.20.193.28:0/33236051 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f912410f690 0x7f912410fa60 secure :-1 s=READY pgs=495 gs=1 cs=0 l=1 c_cookie=9733877422986179178 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9114009950 tx=0x7f911402f190 comp rx=0 tx=0).stop 2026-02-25T11:50:13.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.709+0000 7f9129d48640 1 -- 10.20.193.28:0/33236051 shutdown_connections 2026-02-25T11:50:13.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.709+0000 7f9129d48640 1 --2- 10.20.193.28:0/33236051 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9124115880 0x7f9124176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.709+0000 7f9129d48640 1 --2- 10.20.193.28:0/33236051 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9124110080 0x7f91241150d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.709+0000 7f9129d48640 1 --2- 10.20.193.28:0/33236051 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f912410f690 0x7f912410fa60 unknown :-1 s=CLOSED pgs=495 gs=1 cs=0 l=1 c_cookie=9733877422986179178 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.709+0000 7f9129d48640 1 -- 10.20.193.28:0/33236051 >> 10.20.193.28:0/33236051 conn(0x7f91240ecf00 msgr2=0x7f912410d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:13.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.709+0000 7f9129d48640 1 -- 10.20.193.28:0/33236051 shutdown_connections 2026-02-25T11:50:13.712 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.709+0000 7f9129d48640 1 -- 10.20.193.28:0/33236051 wait complete. 2026-02-25T11:50:13.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.710+0000 7f9129d48640 1 Processor -- start 2026-02-25T11:50:13.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.710+0000 7f9129d48640 1 -- start start 2026-02-25T11:50:13.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.710+0000 7f9129d48640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f912410f690 0x7f91240f59b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.714 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.710+0000 7f9129d48640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9124110080 0x7f9124176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.710+0000 7f9129d48640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9124115880 0x7f91240f60b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.710+0000 7f9129d48640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f91240fb2c0 con 0x7f9124115880 2026-02-25T11:50:13.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.710+0000 7f9129d48640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f91240fb430 con 0x7f9124110080 2026-02-25T11:50:13.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.710+0000 7f9129d48640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91240fb5f0 con 0x7f912410f690 2026-02-25T11:50:13.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.711+0000 7f911bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9124110080 0x7f9124176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.711+0000 7f9128d46640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f912410f690 0x7f91240f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.711+0000 7f911bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9124110080 0x7f9124176150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:55844/0 (socket says 10.20.193.28:55844) 2026-02-25T11:50:13.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.711+0000 7f911bfff640 1 -- 10.20.193.28:0/4031162976 learned_addr learned my addr 10.20.193.28:0/4031162976 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:13.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.711+0000 7f9129547640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9124115880 0x7f91240f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.711+0000 7f9129547640 1 -- 10.20.193.28:0/4031162976 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f912410f690 msgr2=0x7f91240f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.711+0000 7f9129547640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f912410f690 0x7f91240f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.711+0000 7f9129547640 1 -- 10.20.193.28:0/4031162976 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9124110080 msgr2=0x7f9124176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.711+0000 7f9129547640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9124110080 0x7f9124176150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.711+0000 7f9129547640 1 -- 10.20.193.28:0/4031162976 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9114009570 con 0x7f9124115880 2026-02-25T11:50:13.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.711+0000 7f911bfff640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9124110080 0x7f9124176150 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:13.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.711+0000 7f9129547640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9124115880 0x7f91240f60b0 secure :-1 s=READY pgs=496 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f912000e5e0 tx=0x7f912000eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:13.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.712+0000 7f9119ffb640 1 -- 10.20.193.28:0/4031162976 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f912000a960 con 0x7f9124115880 2026-02-25T11:50:13.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.712+0000 7f9129d48640 1 -- 10.20.193.28:0/4031162976 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91240fb8d0 con 0x7f9124115880 2026-02-25T11:50:13.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.712+0000 7f9119ffb640 1 -- 10.20.193.28:0/4031162976 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9120011460 con 0x7f9124115880 2026-02-25T11:50:13.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.712+0000 7f9119ffb640 1 -- 10.20.193.28:0/4031162976 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9120019680 con 0x7f9124115880 2026-02-25T11:50:13.717 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.712+0000 7f9129d48640 1 -- 10.20.193.28:0/4031162976 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91240fbdf0 con 0x7f9124115880 2026-02-25T11:50:13.717 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.715+0000 7f9119ffb640 1 -- 10.20.193.28:0/4031162976 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f9120017020 con 0x7f9124115880 2026-02-25T11:50:13.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.716+0000 7f9119ffb640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f91000b5750 0x7f91000d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.716+0000 7f9119ffb640 1 -- 10.20.193.28:0/4031162976 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f91200eb020 con 0x7f9124115880 2026-02-25T11:50:13.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.717+0000 7f9128d46640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f91000b5750 0x7f91000d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.717+0000 7f9128d46640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f91000b5750 0x7f91000d5c00 secure :-1 s=READY pgs=286 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f91140098a0 tx=0x7f911403a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:13.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.718+0000 7f912cfcce00 1 Processor -- start 2026-02-25T11:50:13.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.718+0000 7f912cfcce00 1 -- start start 2026-02-25T11:50:13.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.718+0000 7f912cfcce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6d90f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.718+0000 7f912cfcce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d91440 0x5566f6dd5ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.718+0000 7f912cfcce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6dd60e0 0x5566f6df64c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.718+0000 7f912cfcce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5566f6ca68a0 con 0x5566f6d91440 2026-02-25T11:50:13.721 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.718+0000 7f912cfcce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5566f6ca25d0 con 0x5566f6dd60e0 2026-02-25T11:50:13.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.718+0000 7f912cfcce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5566f6c9c570 con 0x5566f6d90b30 2026-02-25T11:50:13.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.719+0000 7f9128d46640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6d90f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.719+0000 7f9128d46640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6d90f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44442/0 (socket says 10.20.193.28:44442) 2026-02-25T11:50:13.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.719+0000 7f9128d46640 1 -- 10.20.193.28:0/2481032745 learned_addr learned my addr 10.20.193.28:0/2481032745 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:13.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.719+0000 7f9129547640 1 --2- 10.20.193.28:0/2481032745 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6dd60e0 0x5566f6df64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.719+0000 7f911bfff640 1 --2- 10.20.193.28:0/2481032745 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d91440 0x5566f6dd5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.719+0000 7f911bfff640 1 -- 10.20.193.28:0/2481032745 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 msgr2=0x5566f6d90f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.719+0000 7f911bfff640 1 --2- 10.20.193.28:0/2481032745 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6d90f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.719+0000 7f911bfff640 1 -- 10.20.193.28:0/2481032745 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6dd60e0 msgr2=0x5566f6df64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.719+0000 7f911bfff640 1 --2- 10.20.193.28:0/2481032745 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6dd60e0 0x5566f6df64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.719+0000 7f911bfff640 1 -- 10.20.193.28:0/2481032745 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566f6c47d10 con 0x5566f6d91440 2026-02-25T11:50:13.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.719+0000 7f911bfff640 1 --2- 10.20.193.28:0/2481032745 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d91440 0x5566f6dd5ba0 secure :-1 s=READY pgs=497 gs=2 cs=0 l=1 c_cookie=8087697522317860568 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f910c00b2c0 tx=0x7f910c010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703d460bcc901ed8 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:13.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.720+0000 7f90f14e4640 1 -- 10.20.193.28:0/2481032745 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f910c0043b0 con 0x5566f6d91440 2026-02-25T11:50:13.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.720+0000 7f90f14e4640 1 -- 10.20.193.28:0/2481032745 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f910c004510 con 0x5566f6d91440 2026-02-25T11:50:13.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.720+0000 7f90f14e4640 1 -- 10.20.193.28:0/2481032745 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f910c011bb0 con 0x5566f6d91440 2026-02-25T11:50:13.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.720+0000 7f912cfcce00 1 -- 10.20.193.28:0/2481032745 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d91440 msgr2=0x5566f6dd5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.720+0000 7f912cfcce00 1 --2- 10.20.193.28:0/2481032745 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d91440 0x5566f6dd5ba0 secure :-1 s=READY pgs=497 gs=2 cs=0 l=1 c_cookie=8087697522317860568 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f910c00b2c0 tx=0x7f910c010940 comp rx=0 tx=0).stop 2026-02-25T11:50:13.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.721+0000 7f912cfcce00 1 -- 10.20.193.28:0/2481032745 shutdown_connections 2026-02-25T11:50:13.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.721+0000 7f912cfcce00 1 --2- 10.20.193.28:0/2481032745 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6dd60e0 0x5566f6df64c0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.721+0000 7f912cfcce00 1 --2- 10.20.193.28:0/2481032745 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d91440 0x5566f6dd5ba0 unknown :-1 s=CLOSED pgs=497 gs=2 cs=0 l=1 c_cookie=8087697522317860568 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.721+0000 7f912cfcce00 1 --2- 10.20.193.28:0/2481032745 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6d90f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.721+0000 7f912cfcce00 1 -- 10.20.193.28:0/2481032745 >> 10.20.193.28:0/2481032745 conn(0x5566f6caf7a0 msgr2=0x5566f6d8f410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:13.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.721+0000 7f912cfcce00 1 -- 10.20.193.28:0/2481032745 shutdown_connections 2026-02-25T11:50:13.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.721+0000 7f912cfcce00 1 -- 10.20.193.28:0/2481032745 wait complete. 2026-02-25T11:50:13.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.722+0000 7f912cfcce00 1 -- start start 2026-02-25T11:50:13.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.723+0000 7f912cfcce00 1 Processor -- start 2026-02-25T11:50:13.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.723+0000 7f912cfcce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6da8e60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.723+0000 7f912cfcce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d91440 0x5566f6da93a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.723+0000 7f912cfcce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6dd60e0 0x5566f6dae470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.723+0000 7f912cfcce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5566f6dae9b0 con 0x5566f6d91440 2026-02-25T11:50:13.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.723+0000 7f912cfcce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5566f6daeb20 con 0x5566f6dd60e0 2026-02-25T11:50:13.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.723+0000 7f912cfcce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5566f6daece0 con 0x5566f6d90b30 2026-02-25T11:50:13.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.723+0000 7f9128d46640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6da8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.724+0000 7f9128d46640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6da8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44448/0 (socket says 10.20.193.28:44448) 2026-02-25T11:50:13.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.724+0000 7f9128d46640 1 -- 10.20.193.28:0/2900164151 learned_addr learned my addr 10.20.193.28:0/2900164151 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:13.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.724+0000 7f9129547640 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6dd60e0 0x5566f6dae470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.724+0000 7f911bfff640 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d91440 0x5566f6da93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.724+0000 7f9128d46640 1 -- 10.20.193.28:0/2900164151 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6dd60e0 msgr2=0x5566f6dae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.724+0000 7f9128d46640 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6dd60e0 0x5566f6dae470 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.724+0000 7f9128d46640 1 -- 10.20.193.28:0/2900164151 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d91440 msgr2=0x5566f6da93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.724+0000 7f9128d46640 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d91440 0x5566f6da93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.724+0000 7f9128d46640 1 -- 10.20.193.28:0/2900164151 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566f6c203c0 con 0x5566f6d90b30 2026-02-25T11:50:13.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.724+0000 7f911bfff640 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d91440 0x5566f6da93a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:13.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.725+0000 7f9128d46640 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6da8e60 secure :-1 s=READY pgs=135 gs=2 cs=0 l=1 c_cookie=11659389197339668455 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9114002920 tx=0x7f9114009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ce78b36d752fe7 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:13.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.725+0000 7f90e27fc640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:50:13.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.725+0000 7f90e27fc640 1 -- 10.20.193.28:0/2900164151 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9114038e40 con 0x5566f6d90b30 2026-02-25T11:50:13.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.725+0000 7f912cfcce00 1 -- 10.20.193.28:0/2900164151 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f910c00bcc0 con 0x5566f6d90b30 2026-02-25T11:50:13.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.725+0000 7f90e27fc640 1 -- 10.20.193.28:0/2900164151 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f911403e040 con 0x5566f6d90b30 2026-02-25T11:50:13.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.725+0000 7f90e27fc640 1 -- 10.20.193.28:0/2900164151 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9114050700 con 0x5566f6d90b30 2026-02-25T11:50:13.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.725+0000 7f912cfcce00 10 client.25735 _ll_register_callbacks cb 0x5566f6c1da10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:13.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.725+0000 7f912cfcce00 1 -- 10.20.193.28:0/2900164151 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5566f69326f0 con 0x5566f6d90b30 2026-02-25T11:50:13.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.725+0000 7f90e27fc640 1 -- 10.20.193.28:0/2900164151 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9114050890 con 0x5566f6d90b30 2026-02-25T11:50:13.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.725+0000 7f90e27fc640 1 -- 10.20.193.28:0/2900164151 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9114050a50 con 0x5566f6d90b30 2026-02-25T11:50:13.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.725+0000 7f912cfcce00 10 client.25735 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:13.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.725+0000 7f912cfcce00 1 -- 10.20.193.28:0/2900164151 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5566f6c320e0 con 0x5566f6d90b30 2026-02-25T11:50:13.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.726+0000 7f90e27fc640 1 -- 10.20.193.28:0/2900164151 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f911404f070 con 0x5566f6d90b30 2026-02-25T11:50:13.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.726+0000 7f912cfcce00 10 client.25735 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:13.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.726+0000 7f912cfcce00 10 client.25735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:13.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.726+0000 7f912cfcce00 20 client.25735 populate_metadata read hostname 'trial028' 2026-02-25T11:50:13.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.726+0000 7f912cfcce00 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5566f6dd37e0 0x5566f6dd3c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.726+0000 7f912cfcce00 4 client.25735 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:50:13.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.726+0000 7f912cfcce00 1 -- 10.20.193.28:0/2900164151 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566f6c6efc0 con 0x5566f6dd37e0 2026-02-25T11:50:13.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.726+0000 7f911bfff640 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5566f6dd37e0 0x5566f6dd3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.727+0000 7f911bfff640 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5566f6dd37e0 0x5566f6dd3c80 crc :-1 s=READY pgs=60 gs=4 cs=0 l=0 c_cookie=7507117937965650659 s_cookie=15589964587474951481 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=682ea3fd7b67d2e3 server_cookie=d85aae31715d1d39 in_seq=0 out_seq=0 2026-02-25T11:50:13.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.727+0000 7f90e27fc640 10 client.25735 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:13.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.727+0000 7f90e27fc640 1 -- 10.20.193.28:0/2900164151 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5566f6c6efc0 con 0x5566f6dd37e0 2026-02-25T11:50:13.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.727+0000 7f90e27fc640 10 client.25735 handle_command_reply: tid=0 2026-02-25T11:50:13.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.727+0000 7f912cfcce00 1 client.25735 shutdown 2026-02-25T11:50:13.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.728+0000 7f912cfcce00 1 -- 10.20.193.28:0/2900164151 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 msgr2=0x5566f6da8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.728+0000 7f912cfcce00 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6da8e60 secure :-1 s=READY pgs=135 gs=2 cs=0 l=1 c_cookie=11659389197339668455 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9114002920 tx=0x7f9114009310 comp rx=0 tx=0).stop 2026-02-25T11:50:13.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.728+0000 7f912cfcce00 1 -- 10.20.193.28:0/2900164151 shutdown_connections 2026-02-25T11:50:13.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.728+0000 7f912cfcce00 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5566f6dd37e0 0x5566f6dd3c80 crc :-1 s=READY pgs=60 gs=4 cs=0 l=0 c_cookie=7507117937965650659 s_cookie=15589964587474951481 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.728+0000 7f912cfcce00 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6dd60e0 0x5566f6dae470 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.728+0000 7f912cfcce00 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d91440 0x5566f6da93a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.728+0000 7f912cfcce00 1 --2- 10.20.193.28:0/2900164151 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6da8e60 unknown :-1 s=CLOSED pgs=135 gs=2 cs=0 l=1 c_cookie=11659389197339668455 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.728+0000 7f912cfcce00 1 -- 10.20.193.28:0/2900164151 >> 10.20.193.28:0/2900164151 conn(0x5566f6caf7a0 msgr2=0x5566f6d8fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:13.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.728+0000 7f90e27fc640 0 client.25735 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:13.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.728+0000 7f912cfcce00 1 -- 10.20.193.28:0/2900164151 shutdown_connections 2026-02-25T11:50:13.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.728+0000 7f912cfcce00 1 -- 10.20.193.28:0/2900164151 wait complete. 2026-02-25T11:50:13.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.728+0000 7f912cfcce00 20 client.25735 trim_cache size 0 max 16384 2026-02-25T11:50:13.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.736+0000 7f912cfcce00 1 Processor -- start 2026-02-25T11:50:13.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.736+0000 7f912cfcce00 1 -- start start 2026-02-25T11:50:13.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.736+0000 7f912cfcce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6d939c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.736+0000 7f912cfcce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6d91440 0x5566f6d93f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.736+0000 7f912cfcce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6dd60e0 0x5566f6d8fd20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.736+0000 7f912cfcce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5566f6c9a1a0 con 0x5566f6dd60e0 2026-02-25T11:50:13.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.736+0000 7f912cfcce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5566f695a700 con 0x5566f6d91440 2026-02-25T11:50:13.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.736+0000 7f912cfcce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5566f6d6be80 con 0x5566f6d90b30 2026-02-25T11:50:13.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f9128d46640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6d939c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f9128d46640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6d939c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44458/0 (socket says 10.20.193.28:44458) 2026-02-25T11:50:13.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f9128d46640 1 -- 10.20.193.28:0/1090795615 learned_addr learned my addr 10.20.193.28:0/1090795615 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:13.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f9129547640 1 --2- 10.20.193.28:0/1090795615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6dd60e0 0x5566f6d8fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f911bfff640 1 --2- 10.20.193.28:0/1090795615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6d91440 0x5566f6d93f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f9129547640 1 -- 10.20.193.28:0/1090795615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 msgr2=0x5566f6d939c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f9129547640 1 --2- 10.20.193.28:0/1090795615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6d939c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f9129547640 1 -- 10.20.193.28:0/1090795615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6d91440 msgr2=0x5566f6d93f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f9129547640 1 --2- 10.20.193.28:0/1090795615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6d91440 0x5566f6d93f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f9129547640 1 -- 10.20.193.28:0/1090795615 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566f6c2cb60 con 0x5566f6dd60e0 2026-02-25T11:50:13.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f911bfff640 1 --2- 10.20.193.28:0/1090795615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6d91440 0x5566f6d93f00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:13.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f9129547640 1 --2- 10.20.193.28:0/1090795615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6dd60e0 0x5566f6d8fd20 secure :-1 s=READY pgs=498 gs=3 cs=0 l=1 c_cookie=11969253068522246675 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f91200e9000 tx=0x7f91200e4a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61b54400005d213 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:13.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f90f0ce3640 1 -- 10.20.193.28:0/1090795615 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f91200addc0 con 0x5566f6dd60e0 2026-02-25T11:50:13.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f90f0ce3640 1 -- 10.20.193.28:0/1090795615 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f91200b30e0 con 0x5566f6dd60e0 2026-02-25T11:50:13.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.737+0000 7f90f0ce3640 1 -- 10.20.193.28:0/1090795615 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f91200e4d30 con 0x5566f6dd60e0 2026-02-25T11:50:13.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.738+0000 7f912cfcce00 1 -- 10.20.193.28:0/1090795615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6dd60e0 msgr2=0x5566f6d8fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.738+0000 7f912cfcce00 1 --2- 10.20.193.28:0/1090795615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6dd60e0 0x5566f6d8fd20 secure :-1 s=READY pgs=498 gs=3 cs=0 l=1 c_cookie=11969253068522246675 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f91200e9000 tx=0x7f91200e4a90 comp rx=0 tx=0).stop 2026-02-25T11:50:13.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.738+0000 7f912cfcce00 1 -- 10.20.193.28:0/1090795615 shutdown_connections 2026-02-25T11:50:13.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.738+0000 7f912cfcce00 1 --2- 10.20.193.28:0/1090795615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6dd60e0 0x5566f6d8fd20 unknown :-1 s=CLOSED pgs=498 gs=3 cs=0 l=1 c_cookie=11969253068522246675 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.738+0000 7f912cfcce00 1 --2- 10.20.193.28:0/1090795615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6d91440 0x5566f6d93f00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.738+0000 7f912cfcce00 1 --2- 10.20.193.28:0/1090795615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6d90b30 0x5566f6d939c0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.738+0000 7f912cfcce00 1 -- 10.20.193.28:0/1090795615 >> 10.20.193.28:0/1090795615 conn(0x5566f6caf7a0 msgr2=0x5566f6dd37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:13.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.738+0000 7f912cfcce00 1 -- 10.20.193.28:0/1090795615 shutdown_connections 2026-02-25T11:50:13.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.738+0000 7f912cfcce00 1 -- 10.20.193.28:0/1090795615 wait complete. 2026-02-25T11:50:13.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.738+0000 7f912cfcce00 1 -- start start 2026-02-25T11:50:13.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.739+0000 7f912cfcce00 1 Processor -- start 2026-02-25T11:50:13.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.739+0000 7f912cfcce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d90b30 0x5566f6dd59d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.739+0000 7f912cfcce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6d91440 0x5566f6daf530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.739+0000 7f912cfcce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6dd60e0 0x5566f6dafa70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.739+0000 7f912cfcce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5566f6db0170 con 0x5566f6d90b30 2026-02-25T11:50:13.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.739+0000 7f912cfcce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5566f6db02e0 con 0x5566f6d91440 2026-02-25T11:50:13.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.739+0000 7f912cfcce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5566f6db4580 con 0x5566f6dd60e0 2026-02-25T11:50:13.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.739+0000 7f9128d46640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d90b30 0x5566f6dd59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.739+0000 7f9128d46640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d90b30 0x5566f6dd59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42432/0 (socket says 10.20.193.28:42432) 2026-02-25T11:50:13.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.739+0000 7f9128d46640 1 -- 10.20.193.28:0/1738057045 learned_addr learned my addr 10.20.193.28:0/1738057045 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:13.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.739+0000 7f911bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6d91440 0x5566f6daf530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.739+0000 7f9129547640 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6dd60e0 0x5566f6dafa70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.739+0000 7f9128d46640 1 -- 10.20.193.28:0/1738057045 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6dd60e0 msgr2=0x5566f6dafa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f9128d46640 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6dd60e0 0x5566f6dafa70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f9128d46640 1 -- 10.20.193.28:0/1738057045 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6d91440 msgr2=0x5566f6daf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:13.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f9128d46640 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6d91440 0x5566f6daf530 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f9128d46640 1 -- 10.20.193.28:0/1738057045 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91240fbdf0 con 0x5566f6d90b30 2026-02-25T11:50:13.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f9129547640 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6dd60e0 0x5566f6dafa70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:13.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f911bfff640 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6d91440 0x5566f6daf530 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:13.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f9128d46640 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d90b30 0x5566f6dd59d0 secure :-1 s=READY pgs=499 gs=3 cs=0 l=1 c_cookie=18361830649505225647 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f911402f1d0 tx=0x7f911405c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed253b0c9a39baf server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:13.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f90e27fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:13.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f90e27fc640 1 -- 10.20.193.28:0/1738057045 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f911406a070 con 0x5566f6d90b30 2026-02-25T11:50:13.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f912cfcce00 1 -- 10.20.193.28:0/1738057045 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566f6c47d10 con 0x5566f6d90b30 2026-02-25T11:50:13.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f912cfcce00 10 client.15957 _ll_register_callbacks cb 0x5566f6c1da10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:13.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f912cfcce00 1 -- 10.20.193.28:0/1738057045 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f910c004510 con 0x5566f6d90b30 2026-02-25T11:50:13.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f90e27fc640 1 -- 10.20.193.28:0/1738057045 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f910c004510 con 0x5566f6d90b30 2026-02-25T11:50:13.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f90e27fc640 1 -- 10.20.193.28:0/1738057045 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f911405c110 con 0x5566f6d90b30 2026-02-25T11:50:13.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.740+0000 7f90e27fc640 1 -- 10.20.193.28:0/1738057045 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f911405c270 con 0x5566f6d90b30 2026-02-25T11:50:13.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.741+0000 7f90e27fc640 1 -- 10.20.193.28:0/1738057045 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f91140653d0 con 0x5566f6d90b30 2026-02-25T11:50:13.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.741+0000 7f912cfcce00 10 client.15957 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:13.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.741+0000 7f912cfcce00 1 -- 10.20.193.28:0/1738057045 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5566f6c206f0 con 0x5566f6d90b30 2026-02-25T11:50:13.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.741+0000 7f90e27fc640 1 -- 10.20.193.28:0/1738057045 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f911404f040 con 0x5566f6d90b30 2026-02-25T11:50:13.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.741+0000 7f912cfcce00 10 client.15957 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:13.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.741+0000 7f912cfcce00 10 client.15957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:13.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.741+0000 7f912cfcce00 20 client.15957 populate_metadata read hostname 'trial028' 2026-02-25T11:50:13.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.741+0000 7f912cfcce00 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5566f6e94620 0x5566f6eb4ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:13.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.741+0000 7f912cfcce00 4 client.15957 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:50:13.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.741+0000 7f912cfcce00 1 -- 10.20.193.28:0/1738057045 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566f692ff40 con 0x5566f6e94620 2026-02-25T11:50:13.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.741+0000 7f911bfff640 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5566f6e94620 0x5566f6eb4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:13.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f911bfff640 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5566f6e94620 0x5566f6eb4ad0 crc :-1 s=READY pgs=61 gs=4 cs=0 l=0 c_cookie=938330916034232760 s_cookie=12563427387312992954 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d059eff23fcc1b8 server_cookie=ae5a42a5163472ba in_seq=0 out_seq=0 2026-02-25T11:50:13.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f90e27fc640 10 client.15957 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:13.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f90e27fc640 1 -- 10.20.193.28:0/1738057045 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566f692ff40 con 0x5566f6e94620 2026-02-25T11:50:13.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f90e27fc640 10 client.15957 handle_command_reply: tid=0 2026-02-25T11:50:13.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f912cfcce00 1 client.15957 shutdown 2026-02-25T11:50:13.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f912cfcce00 1 -- 10.20.193.28:0/1738057045 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d90b30 msgr2=0x5566f6dd59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f912cfcce00 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d90b30 0x5566f6dd59d0 secure :-1 s=READY pgs=499 gs=3 cs=0 l=1 c_cookie=18361830649505225647 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f911402f1d0 tx=0x7f911405c5e0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f912cfcce00 1 -- 10.20.193.28:0/1738057045 shutdown_connections 2026-02-25T11:50:13.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f912cfcce00 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5566f6e94620 0x5566f6eb4ad0 crc :-1 s=READY pgs=61 gs=4 cs=0 l=0 c_cookie=938330916034232760 s_cookie=12563427387312992954 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f912cfcce00 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5566f6dd60e0 0x5566f6dafa70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f912cfcce00 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5566f6d91440 0x5566f6daf530 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f912cfcce00 1 --2- 10.20.193.28:0/1738057045 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5566f6d90b30 0x5566f6dd59d0 unknown :-1 s=CLOSED pgs=499 gs=3 cs=0 l=1 c_cookie=18361830649505225647 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f90e27fc640 0 client.15957 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:13.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.742+0000 7f912cfcce00 1 -- 10.20.193.28:0/1738057045 >> 10.20.193.28:0/1738057045 conn(0x5566f6caf7a0 msgr2=0x5566f6d939c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:13.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.743+0000 7f912cfcce00 1 -- 10.20.193.28:0/1738057045 shutdown_connections 2026-02-25T11:50:13.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.743+0000 7f912cfcce00 1 -- 10.20.193.28:0/1738057045 wait complete. 2026-02-25T11:50:13.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.743+0000 7f912cfcce00 20 client.15957 trim_cache size 0 max 16384 2026-02-25T11:50:13.750 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:50:13.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.743+0000 7f9129d48640 1 -- 10.20.193.28:0/4031162976 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f91000b5750 msgr2=0x7f91000d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.743+0000 7f9129d48640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f91000b5750 0x7f91000d5c00 secure :-1 s=READY pgs=286 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f91140098a0 tx=0x7f911403a040 comp rx=0 tx=0).stop 2026-02-25T11:50:13.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.743+0000 7f9129d48640 1 -- 10.20.193.28:0/4031162976 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9124115880 msgr2=0x7f91240f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:13.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.743+0000 7f9129d48640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9124115880 0x7f91240f60b0 secure :-1 s=READY pgs=496 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f912000e5e0 tx=0x7f912000eab0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.743+0000 7f9129d48640 1 -- 10.20.193.28:0/4031162976 shutdown_connections 2026-02-25T11:50:13.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.743+0000 7f9129d48640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f91000b5750 0x7f91000d5c00 unknown :-1 s=CLOSED pgs=286 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.743+0000 7f9129d48640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9124115880 0x7f91240f60b0 unknown :-1 s=CLOSED pgs=496 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.743+0000 7f9129d48640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9124110080 0x7f9124176150 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.743+0000 7f9129d48640 1 --2- 10.20.193.28:0/4031162976 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f912410f690 0x7f91240f59b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:13.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.743+0000 7f9129d48640 1 -- 10.20.193.28:0/4031162976 >> 10.20.193.28:0/4031162976 conn(0x7f91240ecf00 msgr2=0x7f9124112d60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:13.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.744+0000 7f9129d48640 1 -- 10.20.193.28:0/4031162976 shutdown_connections 2026-02-25T11:50:13.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:13.744+0000 7f9129d48640 1 -- 10.20.193.28:0/4031162976 wait complete. 2026-02-25T11:50:14.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:14 trial167 ceph-mon[43392]: pgmap v286: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:14.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:14 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:50:14.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:14 trial119 ceph-mon[42710]: pgmap v286: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:14.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:14 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:50:14.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:14 trial028 ceph-mon[39224]: pgmap v286: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:14.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:14 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:50:14.883 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:50:14.883 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:50:15.069 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:15.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.250+0000 7f3ce35a7640 1 -- 10.20.193.28:0/66026854 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3cdc115880 msgr2=0x7f3cdc176250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.250+0000 7f3ce35a7640 1 --2- 10.20.193.28:0/66026854 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3cdc115880 0x7f3cdc176250 secure :-1 s=READY pgs=500 gs=1 cs=0 l=1 c_cookie=8882446706233238240 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3cd000aa30 tx=0x7f3cd0030270 comp rx=0 tx=0).stop 2026-02-25T11:50:15.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.250+0000 7f3ce35a7640 1 -- 10.20.193.28:0/66026854 shutdown_connections 2026-02-25T11:50:15.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.250+0000 7f3ce35a7640 1 --2- 10.20.193.28:0/66026854 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3cdc115880 0x7f3cdc176250 unknown :-1 s=CLOSED pgs=500 gs=1 cs=0 l=1 c_cookie=8882446706233238240 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.250+0000 7f3ce35a7640 1 --2- 10.20.193.28:0/66026854 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3cdc110080 0x7f3cdc1150d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.250+0000 7f3ce35a7640 1 --2- 10.20.193.28:0/66026854 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3cdc10f690 0x7f3cdc10fa60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.250+0000 7f3ce35a7640 1 -- 10.20.193.28:0/66026854 >> 10.20.193.28:0/66026854 conn(0x7f3cdc0ecf00 msgr2=0x7f3cdc10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:15.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.250+0000 7f3ce35a7640 1 -- 10.20.193.28:0/66026854 shutdown_connections 2026-02-25T11:50:15.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.251+0000 7f3ce35a7640 1 -- 10.20.193.28:0/66026854 wait complete. 2026-02-25T11:50:15.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.251+0000 7f3ce35a7640 1 Processor -- start 2026-02-25T11:50:15.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.251+0000 7f3ce35a7640 1 -- start start 2026-02-25T11:50:15.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce35a7640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3cdc10f690 0x7f3cdc11e030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce35a7640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3cdc110080 0x7f3cdc11e570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce35a7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3cdc115880 0x7f3cdc123940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce35a7640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc123f60 con 0x7f3cdc115880 2026-02-25T11:50:15.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce35a7640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc1240d0 con 0x7f3cdc110080 2026-02-25T11:50:15.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce35a7640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3cdc124290 con 0x7f3cdc10f690 2026-02-25T11:50:15.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce25a5640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3cdc10f690 0x7f3cdc11e030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce25a5640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3cdc10f690 0x7f3cdc11e030 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44482/0 (socket says 10.20.193.28:44482) 2026-02-25T11:50:15.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce25a5640 1 -- 10.20.193.28:0/1099872005 learned_addr learned my addr 10.20.193.28:0/1099872005 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:15.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce2da6640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3cdc115880 0x7f3cdc123940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce1da4640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3cdc110080 0x7f3cdc11e570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce2da6640 1 -- 10.20.193.28:0/1099872005 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3cdc10f690 msgr2=0x7f3cdc11e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce2da6640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3cdc10f690 0x7f3cdc11e030 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce2da6640 1 -- 10.20.193.28:0/1099872005 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3cdc110080 msgr2=0x7f3cdc11e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce2da6640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3cdc110080 0x7f3cdc11e570 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce2da6640 1 -- 10.20.193.28:0/1099872005 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd000a650 con 0x7f3cdc115880 2026-02-25T11:50:15.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.252+0000 7f3ce2da6640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3cdc115880 0x7f3cdc123940 secure :-1 s=READY pgs=501 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3cd0033040 tx=0x7f3cd0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:15.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.253+0000 7f3cc77fe640 1 -- 10.20.193.28:0/1099872005 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3cd003e070 con 0x7f3cdc115880 2026-02-25T11:50:15.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.253+0000 7f3cc77fe640 1 -- 10.20.193.28:0/1099872005 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3cd0039de0 con 0x7f3cdc115880 2026-02-25T11:50:15.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.253+0000 7f3ce35a7640 1 -- 10.20.193.28:0/1099872005 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cdc124510 con 0x7f3cdc115880 2026-02-25T11:50:15.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.253+0000 7f3ce35a7640 1 -- 10.20.193.28:0/1099872005 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cdc164ad0 con 0x7f3cdc115880 2026-02-25T11:50:15.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.253+0000 7f3cc77fe640 1 -- 10.20.193.28:0/1099872005 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3cd0042820 con 0x7f3cdc115880 2026-02-25T11:50:15.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.255+0000 7f3cc77fe640 1 -- 10.20.193.28:0/1099872005 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f3cd00053a0 con 0x7f3cdc115880 2026-02-25T11:50:15.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.257+0000 7f3cc77fe640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3cb80b56e0 0x7f3cb80d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.257+0000 7f3cc77fe640 1 -- 10.20.193.28:0/1099872005 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f3cd010c170 con 0x7f3cdc115880 2026-02-25T11:50:15.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.257+0000 7f3ce25a5640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3cb80b56e0 0x7f3cb80d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.258+0000 7f3ce25a5640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3cb80b56e0 0x7f3cb80d5b90 secure :-1 s=READY pgs=287 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3cd40095d0 tx=0x7f3cd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:15.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.258+0000 7f3ce682be00 1 Processor -- start 2026-02-25T11:50:15.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce682be00 1 -- start start 2026-02-25T11:50:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce682be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea7555750 0x562ea7555b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce682be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea759a610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce682be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea759ab50 0x562ea75baf30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce682be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562ea746bd80 con 0x562ea7556060 2026-02-25T11:50:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce682be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562ea7467ab0 con 0x562ea759ab50 2026-02-25T11:50:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce682be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562ea7461a50 con 0x562ea7555750 2026-02-25T11:50:15.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce2da6640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea759ab50 0x562ea75baf30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce2da6640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea759ab50 0x562ea75baf30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:55916/0 (socket says 10.20.193.28:55916) 2026-02-25T11:50:15.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce2da6640 1 -- 10.20.193.28:0/187986876 learned_addr learned my addr 10.20.193.28:0/187986876 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:15.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce25a5640 1 --2- 10.20.193.28:0/187986876 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea7555750 0x562ea7555b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce2da6640 1 -- 10.20.193.28:0/187986876 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea7555750 msgr2=0x562ea7555b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce2da6640 1 --2- 10.20.193.28:0/187986876 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea7555750 0x562ea7555b20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce2da6640 1 -- 10.20.193.28:0/187986876 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 msgr2=0x562ea759a610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2026-02-25T11:50:15.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.259+0000 7f3ce1da4640 1 --2- 10.20.193.28:0/187986876 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea759a610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3ce2da6640 1 --2- 10.20.193.28:0/187986876 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea759a610 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3ce2da6640 1 -- 10.20.193.28:0/187986876 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea740d120 con 0x562ea759ab50 2026-02-25T11:50:15.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3ce1da4640 1 --2- 10.20.193.28:0/187986876 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea759a610 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:15.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3ce2da6640 1 --2- 10.20.193.28:0/187986876 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea759ab50 0x562ea75baf30 secure :-1 s=READY pgs=121 gs=1 cs=0 l=1 c_cookie=10693116470004611236 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3cd010c380 tx=0x7f3cd004a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=946594c66080f8a4 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3cc4cf3640 1 -- 10.20.193.28:0/187986876 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3cd00d1da0 con 0x562ea759ab50 2026-02-25T11:50:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3cc4cf3640 1 -- 10.20.193.28:0/187986876 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3cd00d8210 con 0x562ea759ab50 2026-02-25T11:50:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3cc4cf3640 1 -- 10.20.193.28:0/187986876 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3cd00d1da0 con 0x562ea759ab50 2026-02-25T11:50:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3ce682be00 1 -- 10.20.193.28:0/187986876 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea759ab50 msgr2=0x562ea75baf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3ce682be00 1 --2- 10.20.193.28:0/187986876 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea759ab50 0x562ea75baf30 secure :-1 s=READY pgs=121 gs=1 cs=0 l=1 c_cookie=10693116470004611236 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3cd010c380 tx=0x7f3cd004a000 comp rx=0 tx=0).stop 2026-02-25T11:50:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3ce682be00 1 -- 10.20.193.28:0/187986876 shutdown_connections 2026-02-25T11:50:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3ce682be00 1 --2- 10.20.193.28:0/187986876 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea759ab50 0x562ea75baf30 unknown :-1 s=CLOSED pgs=121 gs=1 cs=0 l=1 c_cookie=10693116470004611236 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3ce682be00 1 --2- 10.20.193.28:0/187986876 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea759a610 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3ce682be00 1 --2- 10.20.193.28:0/187986876 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea7555750 0x562ea7555b20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3ce682be00 1 -- 10.20.193.28:0/187986876 >> 10.20.193.28:0/187986876 conn(0x562ea75338f0 msgr2=0x562ea7553cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:15.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3ce682be00 1 -- 10.20.193.28:0/187986876 shutdown_connections 2026-02-25T11:50:15.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.260+0000 7f3ce682be00 1 -- 10.20.193.28:0/187986876 wait complete. 2026-02-25T11:50:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.262+0000 7f3ce682be00 1 -- start start 2026-02-25T11:50:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.262+0000 7f3ce682be00 1 Processor -- start 2026-02-25T11:50:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.262+0000 7f3ce682be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea7555750 0x562ea75b2430 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.262+0000 7f3ce682be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea75b2970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.262+0000 7f3ce682be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea759ab50 0x562ea75b7960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.262+0000 7f3ce682be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562ea75b7ea0 con 0x562ea7556060 2026-02-25T11:50:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.262+0000 7f3ce682be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562ea75b8010 con 0x562ea759ab50 2026-02-25T11:50:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.262+0000 7f3ce682be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562ea75b81d0 con 0x562ea7555750 2026-02-25T11:50:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.262+0000 7f3ce1da4640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea75b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.262+0000 7f3ce1da4640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea75b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42464/0 (socket says 10.20.193.28:42464) 2026-02-25T11:50:15.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.262+0000 7f3ce1da4640 1 -- 10.20.193.28:0/3215227811 learned_addr learned my addr 10.20.193.28:0/3215227811 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:15.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.262+0000 7f3ce25a5640 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea7555750 0x562ea75b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3ce2da6640 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea759ab50 0x562ea75b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3ce25a5640 1 -- 10.20.193.28:0/3215227811 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea759ab50 msgr2=0x562ea75b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3ce25a5640 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea759ab50 0x562ea75b7960 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3ce25a5640 1 -- 10.20.193.28:0/3215227811 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 msgr2=0x562ea75b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3ce25a5640 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea75b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3ce25a5640 1 -- 10.20.193.28:0/3215227811 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea73e5810 con 0x562ea7555750 2026-02-25T11:50:15.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3ce1da4640 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea75b2970 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:15.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3ce2da6640 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea759ab50 0x562ea75b7960 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:15.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3ce25a5640 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea7555750 0x562ea75b2430 secure :-1 s=READY pgs=136 gs=2 cs=0 l=1 c_cookie=5505362578844428684 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3cd4011790 tx=0x7f3cd4013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c66fa18c7c0298c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:15.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3c8bfff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:50:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3c8bfff640 1 -- 10.20.193.28:0/3215227811 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3cd4021040 con 0x562ea7555750 2026-02-25T11:50:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3c8bfff640 1 -- 10.20.193.28:0/3215227811 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3cd4004d10 con 0x562ea7555750 2026-02-25T11:50:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3c8bfff640 1 -- 10.20.193.28:0/3215227811 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3cd4005570 con 0x562ea7555750 2026-02-25T11:50:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3ce682be00 1 -- 10.20.193.28:0/3215227811 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cdc164ad0 con 0x562ea7555750 2026-02-25T11:50:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3ce682be00 10 client.25753 _ll_register_callbacks cb 0x562ea73e2e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.263+0000 7f3ce682be00 1 -- 10.20.193.28:0/3215227811 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562ea71b9530 con 0x562ea7555750 2026-02-25T11:50:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.264+0000 7f3c8bfff640 1 -- 10.20.193.28:0/3215227811 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f3cd4005700 con 0x562ea7555750 2026-02-25T11:50:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.264+0000 7f3c8bfff640 1 -- 10.20.193.28:0/3215227811 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3cd401a460 con 0x562ea7555750 2026-02-25T11:50:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.264+0000 7f3ce682be00 10 client.25753 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.264+0000 7f3ce682be00 1 -- 10.20.193.28:0/3215227811 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x562ea73f74f0 con 0x562ea7555750 2026-02-25T11:50:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.264+0000 7f3c8bfff640 1 -- 10.20.193.28:0/3215227811 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f3cd4004770 con 0x562ea7555750 2026-02-25T11:50:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.264+0000 7f3ce682be00 10 client.25753 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.264+0000 7f3ce682be00 10 client.25753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.264+0000 7f3ce682be00 20 client.25753 populate_metadata read hostname 'trial028' 2026-02-25T11:50:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.264+0000 7f3ce682be00 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562ea7597ec0 0x562ea7598360 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.264+0000 7f3ce682be00 4 client.25753 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:50:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.264+0000 7f3ce682be00 1 -- 10.20.193.28:0/3215227811 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ea740eb50 con 0x562ea7597ec0 2026-02-25T11:50:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.264+0000 7f3ce1da4640 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562ea7597ec0 0x562ea7598360 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.265+0000 7f3ce1da4640 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562ea7597ec0 0x562ea7598360 crc :-1 s=READY pgs=62 gs=4 cs=0 l=0 c_cookie=16124692735032319172 s_cookie=5960888753599861435 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc66a9c37ec5cc4 server_cookie=52b954bd3ff0d2bb in_seq=0 out_seq=0 2026-02-25T11:50:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.265+0000 7f3c8bfff640 10 client.25753 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.265+0000 7f3c8bfff640 1 -- 10.20.193.28:0/3215227811 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x562ea740eb50 con 0x562ea7597ec0 2026-02-25T11:50:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.265+0000 7f3c8bfff640 10 client.25753 handle_command_reply: tid=0 2026-02-25T11:50:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.265+0000 7f3ce682be00 1 client.25753 shutdown 2026-02-25T11:50:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.266+0000 7f3ce682be00 1 -- 10.20.193.28:0/3215227811 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea7555750 msgr2=0x562ea75b2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.266+0000 7f3ce682be00 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea7555750 0x562ea75b2430 secure :-1 s=READY pgs=136 gs=2 cs=0 l=1 c_cookie=5505362578844428684 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3cd4011790 tx=0x7f3cd4013040 comp rx=0 tx=0).stop 2026-02-25T11:50:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.266+0000 7f3ce682be00 1 -- 10.20.193.28:0/3215227811 shutdown_connections 2026-02-25T11:50:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.266+0000 7f3ce682be00 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562ea7597ec0 0x562ea7598360 crc :-1 s=READY pgs=62 gs=4 cs=0 l=0 c_cookie=16124692735032319172 s_cookie=5960888753599861435 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.266+0000 7f3ce682be00 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea759ab50 0x562ea75b7960 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.266+0000 7f3ce682be00 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea75b2970 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.266+0000 7f3ce682be00 1 --2- 10.20.193.28:0/3215227811 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea7555750 0x562ea75b2430 unknown :-1 s=CLOSED pgs=136 gs=2 cs=0 l=1 c_cookie=5505362578844428684 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.266+0000 7f3c8bfff640 0 client.25753 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.266+0000 7f3ce682be00 1 -- 10.20.193.28:0/3215227811 >> 10.20.193.28:0/3215227811 conn(0x562ea75338f0 msgr2=0x562ea7553cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.266+0000 7f3ce682be00 1 -- 10.20.193.28:0/3215227811 shutdown_connections 2026-02-25T11:50:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.266+0000 7f3ce682be00 1 -- 10.20.193.28:0/3215227811 wait complete. 2026-02-25T11:50:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.266+0000 7f3ce682be00 20 client.25753 trim_cache size 0 max 16384 2026-02-25T11:50:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce682be00 1 Processor -- start 2026-02-25T11:50:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce682be00 1 -- start start 2026-02-25T11:50:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce682be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea7555750 0x562ea75b6ed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce682be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea7595790 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce682be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea759ab50 0x562ea7595cd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce682be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562ea745f680 con 0x562ea7556060 2026-02-25T11:50:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce682be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562ea711f790 con 0x562ea7555750 2026-02-25T11:50:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce682be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562ea75b2f50 con 0x562ea759ab50 2026-02-25T11:50:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce1da4640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea7595790 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce25a5640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea7555750 0x562ea75b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce1da4640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea7595790 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42468/0 (socket says 10.20.193.28:42468) 2026-02-25T11:50:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce1da4640 1 -- 10.20.193.28:0/2086068294 learned_addr learned my addr 10.20.193.28:0/2086068294 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce2da6640 1 --2- 10.20.193.28:0/2086068294 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea759ab50 0x562ea7595cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce1da4640 1 -- 10.20.193.28:0/2086068294 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea759ab50 msgr2=0x562ea7595cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce1da4640 1 --2- 10.20.193.28:0/2086068294 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea759ab50 0x562ea7595cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce1da4640 1 -- 10.20.193.28:0/2086068294 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea7555750 msgr2=0x562ea75b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce1da4640 1 --2- 10.20.193.28:0/2086068294 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea7555750 0x562ea75b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.276+0000 7f3ce1da4640 1 -- 10.20.193.28:0/2086068294 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea73f1f70 con 0x562ea7556060 2026-02-25T11:50:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.277+0000 7f3ce1da4640 1 --2- 10.20.193.28:0/2086068294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea7595790 secure :-1 s=READY pgs=502 gs=2 cs=0 l=1 c_cookie=4323090414055077919 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3cc8004070 tx=0x7f3cc8002c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfeb3d3ada6381f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.277+0000 7f3caa4e6640 1 -- 10.20.193.28:0/2086068294 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3cc8017040 con 0x562ea7556060 2026-02-25T11:50:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.277+0000 7f3caa4e6640 1 -- 10.20.193.28:0/2086068294 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3cc800bcd0 con 0x562ea7556060 2026-02-25T11:50:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.277+0000 7f3caa4e6640 1 -- 10.20.193.28:0/2086068294 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3cc8012900 con 0x562ea7556060 2026-02-25T11:50:15.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.277+0000 7f3ce682be00 1 -- 10.20.193.28:0/2086068294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 msgr2=0x562ea7595790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.277+0000 7f3ce682be00 1 --2- 10.20.193.28:0/2086068294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea7595790 secure :-1 s=READY pgs=502 gs=2 cs=0 l=1 c_cookie=4323090414055077919 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3cc8004070 tx=0x7f3cc8002c30 comp rx=0 tx=0).stop 2026-02-25T11:50:15.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.277+0000 7f3ce682be00 1 -- 10.20.193.28:0/2086068294 shutdown_connections 2026-02-25T11:50:15.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.277+0000 7f3ce682be00 1 --2- 10.20.193.28:0/2086068294 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea759ab50 0x562ea7595cd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.277+0000 7f3ce682be00 1 --2- 10.20.193.28:0/2086068294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7556060 0x562ea7595790 unknown :-1 s=CLOSED pgs=502 gs=2 cs=0 l=1 c_cookie=4323090414055077919 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.277+0000 7f3ce682be00 1 --2- 10.20.193.28:0/2086068294 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea7555750 0x562ea75b6ed0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.277+0000 7f3ce682be00 1 -- 10.20.193.28:0/2086068294 >> 10.20.193.28:0/2086068294 conn(0x562ea75338f0 msgr2=0x562ea75b67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:15.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.277+0000 7f3ce682be00 1 -- 10.20.193.28:0/2086068294 shutdown_connections 2026-02-25T11:50:15.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.278+0000 7f3ce682be00 1 -- 10.20.193.28:0/2086068294 wait complete. 2026-02-25T11:50:15.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.278+0000 7f3ce682be00 1 -- start start 2026-02-25T11:50:15.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.278+0000 7f3ce682be00 1 Processor -- start 2026-02-25T11:50:15.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.278+0000 7f3ce682be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7555750 0x562ea7533cc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.278+0000 7f3ce682be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea7556060 0x562ea7534200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.278+0000 7f3ce682be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea759ab50 0x562ea7534740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.278+0000 7f3ce682be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562ea7530370 con 0x562ea7555750 2026-02-25T11:50:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.278+0000 7f3ce682be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562ea75b95c0 con 0x562ea7556060 2026-02-25T11:50:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.278+0000 7f3ce682be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562ea746bd80 con 0x562ea759ab50 2026-02-25T11:50:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce25a5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7555750 0x562ea7533cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce25a5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7555750 0x562ea7533cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42482/0 (socket says 10.20.193.28:42482) 2026-02-25T11:50:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce2da6640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea759ab50 0x562ea7534740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce25a5640 1 -- 10.20.193.28:0/1545811231 learned_addr learned my addr 10.20.193.28:0/1545811231 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce1da4640 1 --2- 10.20.193.28:0/1545811231 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea7556060 0x562ea7534200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce25a5640 1 -- 10.20.193.28:0/1545811231 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea759ab50 msgr2=0x562ea7534740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce25a5640 1 --2- 10.20.193.28:0/1545811231 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea759ab50 0x562ea7534740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce25a5640 1 -- 10.20.193.28:0/1545811231 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea7556060 msgr2=0x562ea7534200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce25a5640 1 --2- 10.20.193.28:0/1545811231 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea7556060 0x562ea7534200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce25a5640 1 -- 10.20.193.28:0/1545811231 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea73e5b40 con 0x562ea7555750 2026-02-25T11:50:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce1da4640 1 --2- 10.20.193.28:0/1545811231 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea7556060 0x562ea7534200 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce25a5640 1 --2- 10.20.193.28:0/1545811231 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7555750 0x562ea7533cc0 secure :-1 s=READY pgs=503 gs=2 cs=0 l=1 c_cookie=17273596206698230293 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3cd401b7c0 tx=0x7f3cd4031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb8243b05c87215 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3c8bfff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3c8bfff640 1 -- 10.20.193.28:0/1545811231 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3cd4021030 con 0x562ea7555750 2026-02-25T11:50:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3c8bfff640 1 -- 10.20.193.28:0/1545811231 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3cd403c070 con 0x562ea7555750 2026-02-25T11:50:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce682be00 1 -- 10.20.193.28:0/1545811231 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc8002900 con 0x562ea7555750 2026-02-25T11:50:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3c8bfff640 1 -- 10.20.193.28:0/1545811231 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3cd401bc20 con 0x562ea7555750 2026-02-25T11:50:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce682be00 10 client.15972 _ll_register_callbacks cb 0x562ea73e2e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.279+0000 7f3ce682be00 1 -- 10.20.193.28:0/1545811231 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cd00d8210 con 0x562ea7555750 2026-02-25T11:50:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.280+0000 7f3c8bfff640 1 -- 10.20.193.28:0/1545811231 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f3cd403e070 con 0x562ea7555750 2026-02-25T11:50:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.280+0000 7f3c8bfff640 1 -- 10.20.193.28:0/1545811231 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3cd400f670 con 0x562ea7555750 2026-02-25T11:50:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.280+0000 7f3ce682be00 10 client.15972 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.280+0000 7f3ce682be00 1 -- 10.20.193.28:0/1545811231 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x562ea73f7750 con 0x562ea7555750 2026-02-25T11:50:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.280+0000 7f3c8bfff640 1 -- 10.20.193.28:0/1545811231 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f3cd400f7d0 con 0x562ea7555750 2026-02-25T11:50:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.280+0000 7f3ce682be00 10 client.15972 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.280+0000 7f3ce682be00 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.280+0000 7f3ce682be00 20 client.15972 populate_metadata read hostname 'trial028' 2026-02-25T11:50:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.280+0000 7f3ce682be00 1 --2- 10.20.193.28:0/1545811231 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562ea7648b40 0x562ea7668ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.280+0000 7f3ce682be00 4 client.15972 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:50:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.280+0000 7f3ce682be00 1 -- 10.20.193.28:0/1545811231 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562ea72c2690 con 0x562ea7648b40 2026-02-25T11:50:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.281+0000 7f3ce1da4640 1 --2- 10.20.193.28:0/1545811231 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562ea7648b40 0x562ea7668ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.281+0000 7f3ce1da4640 1 --2- 10.20.193.28:0/1545811231 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562ea7648b40 0x562ea7668ff0 crc :-1 s=READY pgs=63 gs=4 cs=0 l=0 c_cookie=10831625020067995211 s_cookie=13261573239044666674 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9651a9913d62de4b server_cookie=b80a92991c157532 in_seq=0 out_seq=0 2026-02-25T11:50:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.281+0000 7f3c8bfff640 10 client.15972 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.282+0000 7f3c8bfff640 1 -- 10.20.193.28:0/1545811231 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562ea72c2690 con 0x562ea7648b40 2026-02-25T11:50:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.282+0000 7f3c8bfff640 10 client.15972 handle_command_reply: tid=0 2026-02-25T11:50:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.282+0000 7f3ce682be00 1 client.15972 shutdown 2026-02-25T11:50:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.282+0000 7f3ce682be00 1 -- 10.20.193.28:0/1545811231 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7555750 msgr2=0x562ea7533cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.282+0000 7f3ce682be00 1 --2- 10.20.193.28:0/1545811231 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7555750 0x562ea7533cc0 secure :-1 s=READY pgs=503 gs=2 cs=0 l=1 c_cookie=17273596206698230293 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3cd401b7c0 tx=0x7f3cd4031000 comp rx=0 tx=0).stop 2026-02-25T11:50:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.283+0000 7f3ce682be00 1 -- 10.20.193.28:0/1545811231 shutdown_connections 2026-02-25T11:50:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.283+0000 7f3ce682be00 1 --2- 10.20.193.28:0/1545811231 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562ea7648b40 0x562ea7668ff0 crc :-1 s=READY pgs=63 gs=4 cs=0 l=0 c_cookie=10831625020067995211 s_cookie=13261573239044666674 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.283+0000 7f3ce682be00 1 --2- 10.20.193.28:0/1545811231 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562ea759ab50 0x562ea7534740 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.283+0000 7f3ce682be00 1 --2- 10.20.193.28:0/1545811231 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562ea7556060 0x562ea7534200 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.283+0000 7f3ce682be00 1 --2- 10.20.193.28:0/1545811231 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562ea7555750 0x562ea7533cc0 unknown :-1 s=CLOSED pgs=503 gs=2 cs=0 l=1 c_cookie=17273596206698230293 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.283+0000 7f3c8bfff640 0 client.15972 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.283+0000 7f3ce682be00 1 -- 10.20.193.28:0/1545811231 >> 10.20.193.28:0/1545811231 conn(0x562ea75338f0 msgr2=0x562ea75aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.283+0000 7f3ce682be00 1 -- 10.20.193.28:0/1545811231 shutdown_connections 2026-02-25T11:50:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.283+0000 7f3ce682be00 1 -- 10.20.193.28:0/1545811231 wait complete. 2026-02-25T11:50:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.283+0000 7f3ce682be00 20 client.15972 trim_cache size 0 max 16384 2026-02-25T11:50:15.290 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:50:15.290 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:50:15.290 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "92296b1b-2ea3-4dd6-a647-6be0790774fe", 2026-02-25T11:50:15.290 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:50:15.290 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:50:15.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.284+0000 7f3ce35a7640 1 -- 10.20.193.28:0/1099872005 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3cb80b56e0 msgr2=0x7f3cb80d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.284+0000 7f3ce35a7640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3cb80b56e0 0x7f3cb80d5b90 secure :-1 s=READY pgs=287 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3cd40095d0 tx=0x7f3cd4005f90 comp rx=0 tx=0).stop 2026-02-25T11:50:15.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.284+0000 7f3ce35a7640 1 -- 10.20.193.28:0/1099872005 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3cdc115880 msgr2=0x7f3cdc123940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.284+0000 7f3ce35a7640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3cdc115880 0x7f3cdc123940 secure :-1 s=READY pgs=501 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3cd0033040 tx=0x7f3cd0003f60 comp rx=0 tx=0).stop 2026-02-25T11:50:15.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.284+0000 7f3ce35a7640 1 -- 10.20.193.28:0/1099872005 shutdown_connections 2026-02-25T11:50:15.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.284+0000 7f3ce35a7640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3cb80b56e0 0x7f3cb80d5b90 unknown :-1 s=CLOSED pgs=287 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.284+0000 7f3ce35a7640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3cdc115880 0x7f3cdc123940 unknown :-1 s=CLOSED pgs=501 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.284+0000 7f3ce35a7640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3cdc110080 0x7f3cdc11e570 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.284+0000 7f3ce35a7640 1 --2- 10.20.193.28:0/1099872005 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3cdc10f690 0x7f3cdc11e030 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.284+0000 7f3ce35a7640 1 -- 10.20.193.28:0/1099872005 >> 10.20.193.28:0/1099872005 conn(0x7f3cdc0ecf00 msgr2=0x7f3cdc10d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:15.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.284+0000 7f3ce35a7640 1 -- 10.20.193.28:0/1099872005 shutdown_connections 2026-02-25T11:50:15.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.284+0000 7f3ce35a7640 1 -- 10.20.193.28:0/1099872005 wait complete. 2026-02-25T11:50:15.383 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:50:15.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:50:15.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:50:15.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:50:15.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:50:15.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:50:15.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:50:15.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:50:15.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:50:15.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:50:15.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:50:15.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:50:15.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:50:15.567 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:15.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.746+0000 7f14f3577640 1 -- 10.20.193.28:0/4198066480 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f14f4114a90 msgr2=0x7f14f4119580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.746+0000 7f14f3577640 1 --2- 10.20.193.28:0/4198066480 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f14f4114a90 0x7f14f4119580 secure :-1 s=READY pgs=137 gs=2 cs=0 l=1 c_cookie=6110971160717874377 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14dc009950 tx=0x7f14dc02f190 comp rx=0 tx=0).stop 2026-02-25T11:50:15.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.747+0000 7f14f3577640 1 -- 10.20.193.28:0/4198066480 shutdown_connections 2026-02-25T11:50:15.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.747+0000 7f14f3577640 1 --2- 10.20.193.28:0/4198066480 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14f4119d30 0x7f14f417a700 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.747+0000 7f14f3577640 1 --2- 10.20.193.28:0/4198066480 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f14f4114a90 0x7f14f4119580 unknown :-1 s=CLOSED pgs=137 gs=2 cs=0 l=1 c_cookie=6110971160717874377 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.747+0000 7f14f3577640 1 --2- 10.20.193.28:0/4198066480 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f14f41140a0 0x7f14f4114470 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.747+0000 7f14f3577640 1 -- 10.20.193.28:0/4198066480 >> 10.20.193.28:0/4198066480 conn(0x7f14f4064c40 msgr2=0x7f14f4065040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:15.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.747+0000 7f14f3577640 1 -- 10.20.193.28:0/4198066480 shutdown_connections 2026-02-25T11:50:15.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.747+0000 7f14f3577640 1 -- 10.20.193.28:0/4198066480 wait complete. 2026-02-25T11:50:15.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.747+0000 7f14f3577640 1 Processor -- start 2026-02-25T11:50:15.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f3577640 1 -- start start 2026-02-25T11:50:15.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f3577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f14f41140a0 0x7f14f4122570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f3577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14f4114a90 0x7f14f4122ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f3577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f14f4119d30 0x7f14f4127e30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f3577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f14f4128450 con 0x7f14f4114a90 2026-02-25T11:50:15.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f3577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f14f41285c0 con 0x7f14f4119d30 2026-02-25T11:50:15.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f3577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f14f4128780 con 0x7f14f41140a0 2026-02-25T11:50:15.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f1d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14f4114a90 0x7f14f4122ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f1d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14f4114a90 0x7f14f4122ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42506/0 (socket says 10.20.193.28:42506) 2026-02-25T11:50:15.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f2575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f14f41140a0 0x7f14f4122570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f1d74640 1 -- 10.20.193.28:0/2419964661 learned_addr learned my addr 10.20.193.28:0/2419964661 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:15.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f2d76640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f14f4119d30 0x7f14f4127e30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f2575640 1 -- 10.20.193.28:0/2419964661 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f14f4119d30 msgr2=0x7f14f4127e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f2575640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f14f4119d30 0x7f14f4127e30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f2575640 1 -- 10.20.193.28:0/2419964661 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14f4114a90 msgr2=0x7f14f4122ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f2575640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14f4114a90 0x7f14f4122ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f2575640 1 -- 10.20.193.28:0/2419964661 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc009570 con 0x7f14f41140a0 2026-02-25T11:50:15.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.748+0000 7f14f1d74640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14f4114a90 0x7f14f4122ab0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:15.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.749+0000 7f14f2575640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f14f41140a0 0x7f14f4122570 secure :-1 s=READY pgs=138 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14e800e7b0 tx=0x7f14e800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:15.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.749+0000 7f14db7fe640 1 -- 10.20.193.28:0/2419964661 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f14e800cce0 con 0x7f14f41140a0 2026-02-25T11:50:15.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.749+0000 7f14f3577640 1 -- 10.20.193.28:0/2419964661 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f4128a60 con 0x7f14f41140a0 2026-02-25T11:50:15.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.749+0000 7f14db7fe640 1 -- 10.20.193.28:0/2419964661 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f14e800ce40 con 0x7f14f41140a0 2026-02-25T11:50:15.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.749+0000 7f14db7fe640 1 -- 10.20.193.28:0/2419964661 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f14e80052a0 con 0x7f14f41140a0 2026-02-25T11:50:15.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.749+0000 7f14f3577640 1 -- 10.20.193.28:0/2419964661 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f4169160 con 0x7f14f41140a0 2026-02-25T11:50:15.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.752+0000 7f14db7fe640 1 -- 10.20.193.28:0/2419964661 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f14e8005430 con 0x7f14f41140a0 2026-02-25T11:50:15.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.754+0000 7f14db7fe640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f14cc0b5760 0x7f14cc0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.754+0000 7f14db7fe640 1 -- 10.20.193.28:0/2419964661 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f14e8014070 con 0x7f14f41140a0 2026-02-25T11:50:15.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.754+0000 7f14f1d74640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f14cc0b5760 0x7f14cc0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.754+0000 7f14fae90e00 1 Processor -- start 2026-02-25T11:50:15.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14fae90e00 1 -- start start 2026-02-25T11:50:15.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14f1d74640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f14cc0b5760 0x7f14cc0d5c10 secure :-1 s=READY pgs=288 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14f4128ec0 tx=0x7f14dc03a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:15.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14fae90e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893f06510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14fae90e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893f4b1b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14fae90e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 0x562893f6bad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14fae90e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562893e1c9b0 con 0x562893f06a50 2026-02-25T11:50:15.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14fae90e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562893e186e0 con 0x562893f4b6f0 2026-02-25T11:50:15.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14fae90e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562893e12840 con 0x562893f06140 2026-02-25T11:50:15.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14f1d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893f4b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14f1d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893f4b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42522/0 (socket says 10.20.193.28:42522) 2026-02-25T11:50:15.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14f1d74640 1 -- 10.20.193.28:0/2464172600 learned_addr learned my addr 10.20.193.28:0/2464172600 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:15.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14f1d74640 1 -- 10.20.193.28:0/2464172600 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 msgr2=0x562893f06510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14f1d74640 1 --2- 10.20.193.28:0/2464172600 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893f06510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14f1d74640 1 -- 10.20.193.28:0/2464172600 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 msgr2=0x562893f6bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14f1d74640 1 --2- 10.20.193.28:0/2464172600 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 0x562893f6bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.755+0000 7f14f1d74640 1 -- 10.20.193.28:0/2464172600 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562893dbdd80 con 0x562893f06a50 2026-02-25T11:50:15.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.756+0000 7f14f1d74640 1 --2- 10.20.193.28:0/2464172600 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893f4b1b0 secure :-1 s=READY pgs=504 gs=1 cs=0 l=1 c_cookie=12818444324557637992 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14dc004710 tx=0x7f14dc040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e44332ba614968 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:15.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.756+0000 7f14d8ff9640 1 -- 10.20.193.28:0/2464172600 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f14dc02fc90 con 0x562893f06a50 2026-02-25T11:50:15.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.756+0000 7f14d8ff9640 1 -- 10.20.193.28:0/2464172600 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f14dc03e040 con 0x562893f06a50 2026-02-25T11:50:15.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.756+0000 7f14d8ff9640 1 -- 10.20.193.28:0/2464172600 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f14dc044a60 con 0x562893f06a50 2026-02-25T11:50:15.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.756+0000 7f14fae90e00 1 -- 10.20.193.28:0/2464172600 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 msgr2=0x562893f4b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.756+0000 7f14fae90e00 1 --2- 10.20.193.28:0/2464172600 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893f4b1b0 secure :-1 s=READY pgs=504 gs=1 cs=0 l=1 c_cookie=12818444324557637992 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14dc004710 tx=0x7f14dc040000 comp rx=0 tx=0).stop 2026-02-25T11:50:15.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.756+0000 7f14fae90e00 1 -- 10.20.193.28:0/2464172600 shutdown_connections 2026-02-25T11:50:15.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.756+0000 7f14fae90e00 1 --2- 10.20.193.28:0/2464172600 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 0x562893f6bad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.756+0000 7f14fae90e00 1 --2- 10.20.193.28:0/2464172600 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893f4b1b0 unknown :-1 s=CLOSED pgs=504 gs=1 cs=0 l=1 c_cookie=12818444324557637992 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.756+0000 7f14fae90e00 1 --2- 10.20.193.28:0/2464172600 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893f06510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.756+0000 7f14fae90e00 1 -- 10.20.193.28:0/2464172600 >> 10.20.193.28:0/2464172600 conn(0x562893e25d40 msgr2=0x562893f04a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:15.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.756+0000 7f14fae90e00 1 -- 10.20.193.28:0/2464172600 shutdown_connections 2026-02-25T11:50:15.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.756+0000 7f14fae90e00 1 -- 10.20.193.28:0/2464172600 wait complete. 2026-02-25T11:50:15.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.758+0000 7f14fae90e00 1 -- start start 2026-02-25T11:50:15.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.758+0000 7f14fae90e00 1 Processor -- start 2026-02-25T11:50:15.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14fae90e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893f63500 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14fae90e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f06a50 0x562893f63a40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14fae90e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f4b6f0 0x562893f68a30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14fae90e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562893f68f70 con 0x562893f4b6f0 2026-02-25T11:50:15.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14fae90e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562893f690e0 con 0x562893f06a50 2026-02-25T11:50:15.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14fae90e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562893f692a0 con 0x562893f06140 2026-02-25T11:50:15.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14f2d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f4b6f0 0x562893f68a30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14f2d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f4b6f0 0x562893f68a30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42524/0 (socket says 10.20.193.28:42524) 2026-02-25T11:50:15.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14f2575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893f63500 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14f2d76640 1 -- 10.20.193.28:0/3937498891 learned_addr learned my addr 10.20.193.28:0/3937498891 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:15.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14f1d74640 1 --2- 10.20.193.28:0/3937498891 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f06a50 0x562893f63a40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14f2d76640 1 -- 10.20.193.28:0/3937498891 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 msgr2=0x562893f63500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14f2d76640 1 --2- 10.20.193.28:0/3937498891 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893f63500 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14f2d76640 1 -- 10.20.193.28:0/3937498891 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f06a50 msgr2=0x562893f63a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14f2d76640 1 --2- 10.20.193.28:0/3937498891 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f06a50 0x562893f63a40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14f2d76640 1 -- 10.20.193.28:0/3937498891 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562893d96350 con 0x562893f4b6f0 2026-02-25T11:50:15.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14f1d74640 1 --2- 10.20.193.28:0/3937498891 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f06a50 0x562893f63a40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:15.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14f2d76640 1 --2- 10.20.193.28:0/3937498891 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f4b6f0 0x562893f68a30 secure :-1 s=READY pgs=505 gs=2 cs=0 l=1 c_cookie=6876283121391880203 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14e4003c20 tx=0x7f14e400aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6d77111993c80b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:15.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.759+0000 7f14a3fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:15.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.760+0000 7f14a3fff640 1 -- 10.20.193.28:0/3937498891 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f14e4011860 con 0x562893f4b6f0 2026-02-25T11:50:15.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.760+0000 7f14a3fff640 1 -- 10.20.193.28:0/3937498891 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f14e4011e80 con 0x562893f4b6f0 2026-02-25T11:50:15.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.760+0000 7f14a3fff640 1 -- 10.20.193.28:0/3937498891 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f14e401b630 con 0x562893f4b6f0 2026-02-25T11:50:15.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.760+0000 7f14fae90e00 1 -- 10.20.193.28:0/3937498891 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f417af70 con 0x562893f4b6f0 2026-02-25T11:50:15.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.760+0000 7f14a3fff640 1 -- 10.20.193.28:0/3937498891 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f14e400fe10 con 0x562893f4b6f0 2026-02-25T11:50:15.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.760+0000 7f14fae90e00 10 client.15984 _ll_register_callbacks cb 0x562893d939a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:15.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.760+0000 7f14fae90e00 1 -- 10.20.193.28:0/3937498891 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562893aa86f0 con 0x562893f4b6f0 2026-02-25T11:50:15.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.760+0000 7f14a3fff640 1 -- 10.20.193.28:0/3937498891 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f14e40119c0 con 0x562893f4b6f0 2026-02-25T11:50:15.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.760+0000 7f14fae90e00 10 client.15984 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:15.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.760+0000 7f14fae90e00 1 -- 10.20.193.28:0/3937498891 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x562893da8150 con 0x562893f4b6f0 2026-02-25T11:50:15.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.761+0000 7f14a3fff640 1 -- 10.20.193.28:0/3937498891 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f14e401a030 con 0x562893f4b6f0 2026-02-25T11:50:15.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.761+0000 7f14fae90e00 10 client.15984 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:15.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.761+0000 7f14fae90e00 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:15.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.761+0000 7f14fae90e00 20 client.15984 populate_metadata read hostname 'trial028' 2026-02-25T11:50:15.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.761+0000 7f14fae90e00 1 --2- 10.20.193.28:0/3937498891 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562893f6ba40 0x562893f49420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.761+0000 7f14fae90e00 4 client.15984 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:50:15.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.761+0000 7f14fae90e00 1 -- 10.20.193.28:0/3937498891 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562893de4910 con 0x562893f6ba40 2026-02-25T11:50:15.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.761+0000 7f14f2575640 1 --2- 10.20.193.28:0/3937498891 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562893f6ba40 0x562893f49420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.762+0000 7f14f2575640 1 --2- 10.20.193.28:0/3937498891 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562893f6ba40 0x562893f49420 crc :-1 s=READY pgs=64 gs=4 cs=0 l=0 c_cookie=7665283715031780574 s_cookie=10719350698467649169 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a608eed290e4cde server_cookie=94c2c8aaacffe691 in_seq=0 out_seq=0 2026-02-25T11:50:15.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.762+0000 7f14a3fff640 10 client.15984 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:15.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14a3fff640 1 -- 10.20.193.28:0/3937498891 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x562893de4910 con 0x562893f6ba40 2026-02-25T11:50:15.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14a3fff640 10 client.15984 handle_command_reply: tid=0 2026-02-25T11:50:15.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14fae90e00 1 client.15984 shutdown 2026-02-25T11:50:15.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14fae90e00 1 -- 10.20.193.28:0/3937498891 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f4b6f0 msgr2=0x562893f68a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14fae90e00 1 --2- 10.20.193.28:0/3937498891 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f4b6f0 0x562893f68a30 secure :-1 s=READY pgs=505 gs=2 cs=0 l=1 c_cookie=6876283121391880203 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14e4003c20 tx=0x7f14e400aab0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14fae90e00 1 -- 10.20.193.28:0/3937498891 shutdown_connections 2026-02-25T11:50:15.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14fae90e00 1 --2- 10.20.193.28:0/3937498891 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562893f6ba40 0x562893f49420 crc :-1 s=READY pgs=64 gs=4 cs=0 l=0 c_cookie=7665283715031780574 s_cookie=10719350698467649169 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14fae90e00 1 --2- 10.20.193.28:0/3937498891 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f4b6f0 0x562893f68a30 unknown :-1 s=CLOSED pgs=505 gs=2 cs=0 l=1 c_cookie=6876283121391880203 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14fae90e00 1 --2- 10.20.193.28:0/3937498891 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f06a50 0x562893f63a40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14fae90e00 1 --2- 10.20.193.28:0/3937498891 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893f63500 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14a3fff640 0 client.15984 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:15.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14fae90e00 1 -- 10.20.193.28:0/3937498891 >> 10.20.193.28:0/3937498891 conn(0x562893e25d40 msgr2=0x562893f04230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:15.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14fae90e00 1 -- 10.20.193.28:0/3937498891 shutdown_connections 2026-02-25T11:50:15.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14fae90e00 1 -- 10.20.193.28:0/3937498891 wait complete. 2026-02-25T11:50:15.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.763+0000 7f14fae90e00 20 client.15984 trim_cache size 0 max 16384 2026-02-25T11:50:15.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.772+0000 7f14fae90e00 1 Processor -- start 2026-02-25T11:50:15.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.772+0000 7f14fae90e00 1 -- start start 2026-02-25T11:50:15.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.772+0000 7f14fae90e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893f4e1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.772+0000 7f14fae90e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893f606d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.772+0000 7f14fae90e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 0x562893f60c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.772+0000 7f14fae90e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562893e10470 con 0x562893f06a50 2026-02-25T11:50:15.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.772+0000 7f14fae90e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562893ad0700 con 0x562893f4b6f0 2026-02-25T11:50:15.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.772+0000 7f14fae90e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562893f6a5d0 con 0x562893f06140 2026-02-25T11:50:15.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14f1d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893f606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14f1d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893f606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42540/0 (socket says 10.20.193.28:42540) 2026-02-25T11:50:15.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14f1d74640 1 -- 10.20.193.28:0/731177528 learned_addr learned my addr 10.20.193.28:0/731177528 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:15.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14f2d76640 1 --2- 10.20.193.28:0/731177528 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 0x562893f60c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14f2575640 1 --2- 10.20.193.28:0/731177528 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893f4e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14f1d74640 1 -- 10.20.193.28:0/731177528 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 msgr2=0x562893f4e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14f1d74640 1 --2- 10.20.193.28:0/731177528 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893f4e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14f1d74640 1 -- 10.20.193.28:0/731177528 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 msgr2=0x562893f60c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14f1d74640 1 --2- 10.20.193.28:0/731177528 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 0x562893f60c10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14f1d74640 1 -- 10.20.193.28:0/731177528 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562893da2bd0 con 0x562893f06a50 2026-02-25T11:50:15.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14f1d74640 1 --2- 10.20.193.28:0/731177528 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893f606d0 secure :-1 s=READY pgs=506 gs=1 cs=0 l=1 c_cookie=11083022684603945039 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14dc009880 tx=0x7f14dc02f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cece69496a104f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:15.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14be4e6640 1 -- 10.20.193.28:0/731177528 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f14dc02fe70 con 0x562893f06a50 2026-02-25T11:50:15.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14be4e6640 1 -- 10.20.193.28:0/731177528 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f14dc038b60 con 0x562893f06a50 2026-02-25T11:50:15.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.773+0000 7f14be4e6640 1 -- 10.20.193.28:0/731177528 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f14dc044d10 con 0x562893f06a50 2026-02-25T11:50:15.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.774+0000 7f14fae90e00 1 -- 10.20.193.28:0/731177528 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 msgr2=0x562893f606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.774+0000 7f14fae90e00 1 --2- 10.20.193.28:0/731177528 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893f606d0 secure :-1 s=READY pgs=506 gs=1 cs=0 l=1 c_cookie=11083022684603945039 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14dc009880 tx=0x7f14dc02f190 comp rx=0 tx=0).stop 2026-02-25T11:50:15.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.774+0000 7f14fae90e00 1 -- 10.20.193.28:0/731177528 shutdown_connections 2026-02-25T11:50:15.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.774+0000 7f14fae90e00 1 --2- 10.20.193.28:0/731177528 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 0x562893f60c10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.774+0000 7f14fae90e00 1 --2- 10.20.193.28:0/731177528 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893f606d0 unknown :-1 s=CLOSED pgs=506 gs=1 cs=0 l=1 c_cookie=11083022684603945039 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.774+0000 7f14fae90e00 1 --2- 10.20.193.28:0/731177528 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893f4e1e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.774+0000 7f14fae90e00 1 -- 10.20.193.28:0/731177528 >> 10.20.193.28:0/731177528 conn(0x562893e25d40 msgr2=0x562893f4dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:15.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.774+0000 7f14fae90e00 1 -- 10.20.193.28:0/731177528 shutdown_connections 2026-02-25T11:50:15.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.774+0000 7f14fae90e00 1 -- 10.20.193.28:0/731177528 wait complete. 2026-02-25T11:50:15.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.775+0000 7f14fae90e00 1 -- start start 2026-02-25T11:50:15.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.775+0000 7f14fae90e00 1 Processor -- start 2026-02-25T11:50:15.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.775+0000 7f14fae90e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893ee5680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.775+0000 7f14fae90e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893ee5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.775+0000 7f14fae90e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 0x562893ee6100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.775+0000 7f14fae90e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562893f5ae40 con 0x562893f06a50 2026-02-25T11:50:15.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.775+0000 7f14fae90e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562893fb1530 con 0x562893f4b6f0 2026-02-25T11:50:15.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.775+0000 7f14fae90e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562893fb16f0 con 0x562893f06140 2026-02-25T11:50:15.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.775+0000 7f14f1d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893ee5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.775+0000 7f14f2575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893ee5680 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.775+0000 7f14f2d76640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 0x562893ee6100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.775+0000 7f14f1d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893ee5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:42556/0 (socket says 10.20.193.28:42556) 2026-02-25T11:50:15.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.775+0000 7f14f1d74640 1 -- 10.20.193.28:0/198805447 learned_addr learned my addr 10.20.193.28:0/198805447 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:15.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.776+0000 7f14f1d74640 1 -- 10.20.193.28:0/198805447 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 msgr2=0x562893ee5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.776+0000 7f14f1d74640 1 --2- 10.20.193.28:0/198805447 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893ee5680 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.776+0000 7f14f1d74640 1 -- 10.20.193.28:0/198805447 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 msgr2=0x562893ee6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:15.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.776+0000 7f14f1d74640 1 --2- 10.20.193.28:0/198805447 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 0x562893ee6100 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.776+0000 7f14f1d74640 1 -- 10.20.193.28:0/198805447 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562893d96680 con 0x562893f06a50 2026-02-25T11:50:15.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.776+0000 7f14f1d74640 1 --2- 10.20.193.28:0/198805447 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893ee5bc0 secure :-1 s=READY pgs=507 gs=1 cs=0 l=1 c_cookie=10258584029896452076 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14dc005c00 tx=0x7f14dc044ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5dcfd2a87927ec server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:15.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.776+0000 7f14a3fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:15.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.776+0000 7f14a3fff640 1 -- 10.20.193.28:0/198805447 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f14dc0444f0 con 0x562893f06a50 2026-02-25T11:50:15.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.776+0000 7f14fae90e00 1 -- 10.20.193.28:0/198805447 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562893dbdd80 con 0x562893f06a50 2026-02-25T11:50:15.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.776+0000 7f14fae90e00 10 client.15990 _ll_register_callbacks cb 0x562893d939a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:15.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.776+0000 7f14fae90e00 1 -- 10.20.193.28:0/198805447 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14dc03e040 con 0x562893f06a50 2026-02-25T11:50:15.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.776+0000 7f14a3fff640 1 -- 10.20.193.28:0/198805447 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f14dc03e040 con 0x562893f06a50 2026-02-25T11:50:15.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.776+0000 7f14a3fff640 1 -- 10.20.193.28:0/198805447 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f14dc05a440 con 0x562893f06a50 2026-02-25T11:50:15.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.777+0000 7f14a3fff640 1 -- 10.20.193.28:0/198805447 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f14dc05a5d0 con 0x562893f06a50 2026-02-25T11:50:15.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.777+0000 7f14a3fff640 1 -- 10.20.193.28:0/198805447 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f14dc05a840 con 0x562893f06a50 2026-02-25T11:50:15.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.777+0000 7f14fae90e00 10 client.15990 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:15.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.777+0000 7f14fae90e00 1 -- 10.20.193.28:0/198805447 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x562893da83b0 con 0x562893f06a50 2026-02-25T11:50:15.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.777+0000 7f14a3fff640 1 -- 10.20.193.28:0/198805447 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f14dc044650 con 0x562893f06a50 2026-02-25T11:50:15.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.777+0000 7f14fae90e00 10 client.15990 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:15.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.777+0000 7f14fae90e00 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:15.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.777+0000 7f14fae90e00 20 client.15990 populate_metadata read hostname 'trial028' 2026-02-25T11:50:15.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.777+0000 7f14fae90e00 1 --2- 10.20.193.28:0/198805447 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562893ff9080 0x562894019500 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:15.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.777+0000 7f14fae90e00 4 client.15990 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:50:15.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.777+0000 7f14fae90e00 1 -- 10.20.193.28:0/198805447 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562893aa5f40 con 0x562893ff9080 2026-02-25T11:50:15.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.777+0000 7f14f2575640 1 --2- 10.20.193.28:0/198805447 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562893ff9080 0x562894019500 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:15.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.778+0000 7f14f2575640 1 --2- 10.20.193.28:0/198805447 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562893ff9080 0x562894019500 crc :-1 s=READY pgs=65 gs=4 cs=0 l=0 c_cookie=2725417637289467961 s_cookie=14778113321988452136 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d2a0e6b7392039 server_cookie=cd1667c4bf96ef28 in_seq=0 out_seq=0 2026-02-25T11:50:15.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.778+0000 7f14a3fff640 10 client.15990 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:15.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14a3fff640 1 -- 10.20.193.28:0/198805447 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x562893aa5f40 con 0x562893ff9080 2026-02-25T11:50:15.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14a3fff640 10 client.15990 handle_command_reply: tid=0 2026-02-25T11:50:15.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14fae90e00 1 client.15990 shutdown 2026-02-25T11:50:15.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14fae90e00 1 -- 10.20.193.28:0/198805447 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 msgr2=0x562893ee5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14fae90e00 1 --2- 10.20.193.28:0/198805447 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893ee5bc0 secure :-1 s=READY pgs=507 gs=1 cs=0 l=1 c_cookie=10258584029896452076 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14dc005c00 tx=0x7f14dc044ca0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14fae90e00 1 -- 10.20.193.28:0/198805447 shutdown_connections 2026-02-25T11:50:15.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14fae90e00 1 --2- 10.20.193.28:0/198805447 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562893ff9080 0x562894019500 crc :-1 s=READY pgs=65 gs=4 cs=0 l=0 c_cookie=2725417637289467961 s_cookie=14778113321988452136 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14fae90e00 1 --2- 10.20.193.28:0/198805447 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562893f4b6f0 0x562893ee6100 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14fae90e00 1 --2- 10.20.193.28:0/198805447 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562893f06a50 0x562893ee5bc0 unknown :-1 s=CLOSED pgs=507 gs=1 cs=0 l=1 c_cookie=10258584029896452076 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14fae90e00 1 --2- 10.20.193.28:0/198805447 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562893f06140 0x562893ee5680 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14a3fff640 0 client.15990 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:15.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14fae90e00 1 -- 10.20.193.28:0/198805447 >> 10.20.193.28:0/198805447 conn(0x562893e25d40 msgr2=0x562893f499a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:15.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14fae90e00 1 -- 10.20.193.28:0/198805447 shutdown_connections 2026-02-25T11:50:15.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14fae90e00 1 -- 10.20.193.28:0/198805447 wait complete. 2026-02-25T11:50:15.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14fae90e00 20 client.15990 trim_cache size 0 max 16384 2026-02-25T11:50:15.786 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:50:15.787 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:50:15.787 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:50:15.787 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:50:15.787 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:15.787 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "92296b1b-2ea3-4dd6-a647-6be0790774fe", 2026-02-25T11:50:15.787 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:50:15.787 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:50:15.787 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:15.788 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:50:15.788 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:50:15.788 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:50:15.788 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:15.788 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:15.788 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:15.788 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:15.788 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:15.788 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:15.788 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:50:15.789 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:50:15.789 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:15.789 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:15.789 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:15.789 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:15.789 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:15.789 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:15.789 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:50:15.789 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:50:15.789 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:15.790 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:15.790 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:15.790 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:15.790 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:15.790 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:15.790 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:50:15.790 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:50:15.790 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:15.790 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:15.790 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:15.791 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:15.791 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:15.791 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:15.791 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:50:15.791 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:50:15.791 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:15.791 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:15.791 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:15.791 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:15.791 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:15.791 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:15.792 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:50:15.792 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:50:15.792 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:15.792 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:15.792 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:15.792 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:15.792 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:15.792 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:50:15.792 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:50:15.792 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:50:15.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14f3577640 1 -- 10.20.193.28:0/2419964661 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f14cc0b5760 msgr2=0x7f14cc0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.779+0000 7f14f3577640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f14cc0b5760 0x7f14cc0d5c10 secure :-1 s=READY pgs=288 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14f4128ec0 tx=0x7f14dc03a040 comp rx=0 tx=0).stop 2026-02-25T11:50:15.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.780+0000 7f14f3577640 1 -- 10.20.193.28:0/2419964661 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f14f41140a0 msgr2=0x7f14f4122570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:15.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.780+0000 7f14f3577640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f14f41140a0 0x7f14f4122570 secure :-1 s=READY pgs=138 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f14e800e7b0 tx=0x7f14e800ec80 comp rx=0 tx=0).stop 2026-02-25T11:50:15.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.780+0000 7f14f3577640 1 -- 10.20.193.28:0/2419964661 shutdown_connections 2026-02-25T11:50:15.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.780+0000 7f14f3577640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f14cc0b5760 0x7f14cc0d5c10 unknown :-1 s=CLOSED pgs=288 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.780+0000 7f14f3577640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f14f4119d30 0x7f14f4127e30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.780+0000 7f14f3577640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f14f4114a90 0x7f14f4122ab0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.780+0000 7f14f3577640 1 --2- 10.20.193.28:0/2419964661 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f14f41140a0 0x7f14f4122570 unknown :-1 s=CLOSED pgs=138 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:15.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.780+0000 7f14f3577640 1 -- 10.20.193.28:0/2419964661 >> 10.20.193.28:0/2419964661 conn(0x7f14f4064c40 msgr2=0x7f14f4117b80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:15.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.780+0000 7f14f3577640 1 -- 10.20.193.28:0/2419964661 shutdown_connections 2026-02-25T11:50:15.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:15.780+0000 7f14f3577640 1 -- 10.20.193.28:0/2419964661 wait complete. 2026-02-25T11:50:15.911 INFO:tasks.cephfs.filesystem:scrub status for tag:92296b1b-2ea3-4dd6-a647-6be0790774fe - {'path': '/', 'tag': '92296b1b-2ea3-4dd6-a647-6be0790774fe', 'options': 'recursive,force'} 2026-02-25T11:50:16.290 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:16 trial028 ceph-mon[39224]: pgmap v287: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:16.291 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:16 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:50:16.291 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:16 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:50:16.291 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:16 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:50:16.291 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:16 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:50:16.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:16 trial167 ceph-mon[43392]: pgmap v287: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:16.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:16 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:50:16.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:16 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:50:16.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:16 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:50:16.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:16 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:50:16.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:16 trial119 ceph-mon[42710]: pgmap v287: 97 pgs: 97 active+clean; 1.9 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:16.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:16 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:50:16.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:16 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:50:16.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:16 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:50:16.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:16 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:50:17.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:17 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:50:17.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:17 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:50:17.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:17 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:50:18.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:18 trial028 ceph-mon[39224]: pgmap v288: 97 pgs: 97 active+clean; 2.0 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:18 trial167 ceph-mon[43392]: pgmap v288: 97 pgs: 97 active+clean; 2.0 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:18.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:18 trial119 ceph-mon[42710]: pgmap v288: 97 pgs: 97 active+clean; 2.0 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:20 trial028 ceph-mon[39224]: pgmap v289: 97 pgs: 97 active+clean; 2.0 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:50:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:20 trial167 ceph-mon[43392]: pgmap v289: 97 pgs: 97 active+clean; 2.0 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:50:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:20 trial119 ceph-mon[42710]: pgmap v289: 97 pgs: 97 active+clean; 2.0 MiB data, 334 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:50:22.551 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:22 trial167 ceph-mon[43392]: pgmap v290: 97 pgs: 97 active+clean; 2.0 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:50:22.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:22 trial028 ceph-mon[39224]: pgmap v290: 97 pgs: 97 active+clean; 2.0 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:50:22.561 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:22 trial119 ceph-mon[42710]: pgmap v290: 97 pgs: 97 active+clean; 2.0 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:50:24.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:24 trial028 ceph-mon[39224]: pgmap v291: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:50:24.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:24 trial167 ceph-mon[43392]: pgmap v291: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:50:24.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:24 trial119 ceph-mon[42710]: pgmap v291: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:50:26.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:26 trial028 ceph-mon[39224]: pgmap v292: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:50:26.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:26 trial167 ceph-mon[43392]: pgmap v292: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:50:26.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:26 trial119 ceph-mon[42710]: pgmap v292: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:50:28.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:28 trial028 ceph-mon[39224]: pgmap v293: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:28.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:28 trial167 ceph-mon[43392]: pgmap v293: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:28.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:28 trial119 ceph-mon[42710]: pgmap v293: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:30.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:30 trial028 ceph-mon[39224]: pgmap v294: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:64/600: open fd = 3 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:65/600: open fd = 3 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:66/600: open fd = 3 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:67/600: open fd = 3 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.561 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:68/600: open fd = 3 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:69/600: open fd = 3 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:70/600: open fd = 3 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:71/600: open fd = 3 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:72/600: open fd = 3 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:73/600: open fd = 3 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:74/600: open fd = 3 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:75/600: open fd = 3 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.562 INFO:tasks.workunit.client.0.trial028.stdout:76/600: open fd = 3 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:77/600: open fd = 3 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:78/600: open fd = 3 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:79/600: open fd = 3 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:80/600: open fd = 3 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:81/600: open fd = 3 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:82/600: open fd = 3 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:83/600: open fd = 3 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:84/600: open fd = 3 2026-02-25T11:50:30.563 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:85/600: open fd = 3 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:86/600: open fd = 3 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:87/600: open fd = 3 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:88/600: open fd = 3 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:89/600: open fd = 3 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:90/600: open fd = 3 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:91/600: open fd = 3 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:92/600: open fd = 3 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.564 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:93/600: open fd = 3 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:94/600: open fd = 3 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:95/600: open fd = 3 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:96/600: open fd = 3 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:97/600: open fd = 3 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:98/600: open fd = 3 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:99/600: open fd = 3 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:100/600: open fd = 3 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.565 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:101/600: open fd = 3 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:102/600: open fd = 3 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:103/600: open fd = 3 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:104/600: open fd = 3 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:105/600: open fd = 3 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:106/600: open fd = 3 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:107/600: open fd = 3 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:108/600: open fd = 3 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:109/600: open fd = 3 2026-02-25T11:50:30.566 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:110/600: open fd = 3 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:111/600: open fd = 3 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:112/600: open fd = 3 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:113/600: open fd = 3 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:114/600: open fd = 3 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:115/600: open fd = 3 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:116/600: open fd = 3 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:117/600: open fd = 3 2026-02-25T11:50:30.567 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:118/600: open fd = 3 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:119/600: open fd = 3 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:120/600: open fd = 3 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:121/600: open fd = 3 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:122/600: open fd = 3 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:123/600: open fd = 3 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:124/600: open fd = 3 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:125/600: open fd = 3 2026-02-25T11:50:30.568 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.569 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.569 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:50:30.569 INFO:tasks.workunit.client.0.trial028.stdout:126/600: open fd = 3 2026-02-25T11:50:30.569 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:50:30.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:30 trial167 ceph-mon[43392]: pgmap v294: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:50:30.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:30 trial119 ceph-mon[42710]: pgmap v294: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:50:32.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:32 trial028 ceph-mon[39224]: pgmap v295: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:32.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:32 trial167 ceph-mon[43392]: pgmap v295: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:32.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:32 trial119 ceph-mon[42710]: pgmap v295: 97 pgs: 97 active+clean; 2.1 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:34.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:34 trial167 ceph-mon[43392]: pgmap v296: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:34.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:34 trial119 ceph-mon[42710]: pgmap v296: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:34.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:34 trial028 ceph-mon[39224]: pgmap v296: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:36.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:36 trial167 ceph-mon[43392]: pgmap v297: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:36.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:36 trial119 ceph-mon[42710]: pgmap v297: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:36.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:36 trial028 ceph-mon[39224]: pgmap v297: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:38.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:38 trial167 ceph-mon[43392]: pgmap v298: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:50:38.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:38 trial119 ceph-mon[42710]: pgmap v298: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:50:38.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:38 trial028 ceph-mon[39224]: pgmap v298: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:50:40.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:40 trial167 ceph-mon[43392]: pgmap v299: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 11 op/s 2026-02-25T11:50:40.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:40 trial119 ceph-mon[42710]: pgmap v299: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 11 op/s 2026-02-25T11:50:40.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:40 trial028 ceph-mon[39224]: pgmap v299: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 11 op/s 2026-02-25T11:50:42.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:42 trial167 ceph-mon[43392]: pgmap v300: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:42.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:42 trial119 ceph-mon[42710]: pgmap v300: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:42.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:42 trial028 ceph-mon[39224]: pgmap v300: 97 pgs: 97 active+clean; 2.2 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:44.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:44 trial167 ceph-mon[43392]: pgmap v301: 97 pgs: 97 active+clean; 2.3 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:44.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:44 trial119 ceph-mon[42710]: pgmap v301: 97 pgs: 97 active+clean; 2.3 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:44.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:44 trial028 ceph-mon[39224]: pgmap v301: 97 pgs: 97 active+clean; 2.3 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:45.912 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:50:46.099 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:46.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.286+0000 7f6fdbd3d640 1 -- 10.20.193.28:0/1096628862 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fd4115930 msgr2=0x7f6fd4176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.286+0000 7f6fdbd3d640 1 --2- 10.20.193.28:0/1096628862 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fd4115930 0x7f6fd4176300 secure :-1 s=READY pgs=508 gs=1 cs=0 l=1 c_cookie=16122466030467477178 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fd000aa30 tx=0x7f6fd0030270 comp rx=0 tx=0).stop 2026-02-25T11:50:46.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.286+0000 7f6fdbd3d640 1 -- 10.20.193.28:0/1096628862 shutdown_connections 2026-02-25T11:50:46.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.286+0000 7f6fdbd3d640 1 --2- 10.20.193.28:0/1096628862 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fd4115930 0x7f6fd4176300 unknown :-1 s=CLOSED pgs=508 gs=1 cs=0 l=1 c_cookie=16122466030467477178 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.286+0000 7f6fdbd3d640 1 --2- 10.20.193.28:0/1096628862 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6fd4110130 0x7f6fd4115180 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.286+0000 7f6fdbd3d640 1 --2- 10.20.193.28:0/1096628862 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6fd410f740 0x7f6fd410fb10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.286+0000 7f6fdbd3d640 1 -- 10.20.193.28:0/1096628862 >> 10.20.193.28:0/1096628862 conn(0x7f6fd40ecff0 msgr2=0x7f6fd410d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:46.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.286+0000 7f6fdbd3d640 1 -- 10.20.193.28:0/1096628862 shutdown_connections 2026-02-25T11:50:46.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.286+0000 7f6fdbd3d640 1 -- 10.20.193.28:0/1096628862 wait complete. 2026-02-25T11:50:46.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.287+0000 7f6fdbd3d640 1 Processor -- start 2026-02-25T11:50:46.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.287+0000 7f6fdbd3d640 1 -- start start 2026-02-25T11:50:46.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.287+0000 7f6fdbd3d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6fd410f740 0x7f6fd411e110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.287+0000 7f6fdbd3d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fd4110130 0x7f6fd411e650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.287+0000 7f6fdbd3d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6fd4115930 0x7f6fd4123a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.287+0000 7f6fdbd3d640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd4124040 con 0x7f6fd4110130 2026-02-25T11:50:46.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.287+0000 7f6fdbd3d640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd41241b0 con 0x7f6fd4115930 2026-02-25T11:50:46.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.287+0000 7f6fdbd3d640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd4124370 con 0x7f6fd410f740 2026-02-25T11:50:46.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.287+0000 7f6fd92b1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fd4110130 0x7f6fd411e650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.287+0000 7f6fd92b1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fd4110130 0x7f6fd411e650 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57750/0 (socket says 10.20.193.28:57750) 2026-02-25T11:50:46.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.287+0000 7f6fd92b1640 1 -- 10.20.193.28:0/1592195320 learned_addr learned my addr 10.20.193.28:0/1592195320 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:46.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.287+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6fd410f740 0x7f6fd411e110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.287+0000 7f6fda2b3640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6fd4115930 0x7f6fd4123a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.288+0000 7f6fd92b1640 1 -- 10.20.193.28:0/1592195320 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6fd410f740 msgr2=0x7f6fd411e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.288+0000 7f6fd92b1640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6fd410f740 0x7f6fd411e110 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.288+0000 7f6fd92b1640 1 -- 10.20.193.28:0/1592195320 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6fd4115930 msgr2=0x7f6fd4123a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.288+0000 7f6fd92b1640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6fd4115930 0x7f6fd4123a20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.288+0000 7f6fd92b1640 1 -- 10.20.193.28:0/1592195320 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd000a650 con 0x7f6fd4110130 2026-02-25T11:50:46.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.288+0000 7f6fd92b1640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fd4110130 0x7f6fd411e650 secure :-1 s=READY pgs=509 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fc800d950 tx=0x7f6fc800de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:46.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.288+0000 7f6fc2ffd640 1 -- 10.20.193.28:0/1592195320 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fc8004540 con 0x7f6fd4110130 2026-02-25T11:50:46.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.288+0000 7f6fc2ffd640 1 -- 10.20.193.28:0/1592195320 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6fc8009c60 con 0x7f6fd4110130 2026-02-25T11:50:46.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.288+0000 7f6fc2ffd640 1 -- 10.20.193.28:0/1592195320 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fc80106e0 con 0x7f6fd4110130 2026-02-25T11:50:46.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.288+0000 7f6fdbd3d640 1 -- 10.20.193.28:0/1592195320 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd4124650 con 0x7f6fd4110130 2026-02-25T11:50:46.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.288+0000 7f6fdbd3d640 1 -- 10.20.193.28:0/1592195320 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd4164d50 con 0x7f6fd4110130 2026-02-25T11:50:46.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.291+0000 7f6fc2ffd640 1 -- 10.20.193.28:0/1592195320 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f6fc80097a0 con 0x7f6fd4110130 2026-02-25T11:50:46.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.293+0000 7f6fc2ffd640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6fb00b5760 0x7f6fb00d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.293+0000 7f6fc2ffd640 1 -- 10.20.193.28:0/1592195320 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6fc80e6280 con 0x7f6fd4110130 2026-02-25T11:50:46.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.293+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6fb00b5760 0x7f6fb00d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.294+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6fb00b5760 0x7f6fb00d5c10 secure :-1 s=READY pgs=289 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fc4004740 tx=0x7f6fc400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:46.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.294+0000 7f6fddd38e00 1 Processor -- start 2026-02-25T11:50:46.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.294+0000 7f6fddd38e00 1 -- start start 2026-02-25T11:50:46.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.294+0000 7f6fddd38e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1140 0x55a75bcb1510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.294+0000 7f6fddd38e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcf61b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.294+0000 7f6fddd38e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcf66f0 0x55a75bd16ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.294+0000 7f6fddd38e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a75bbc79b0 con 0x55a75bcf66f0 2026-02-25T11:50:46.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.294+0000 7f6fddd38e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a75bbc36e0 con 0x55a75bcb1a50 2026-02-25T11:50:46.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.294+0000 7f6fddd38e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a75bbbd840 con 0x55a75bcb1140 2026-02-25T11:50:46.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fd9ab2640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1140 0x55a75bcb1510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fd9ab2640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1140 0x55a75bcb1510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:41304/0 (socket says 10.20.193.28:41304) 2026-02-25T11:50:46.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fd9ab2640 1 -- 10.20.193.28:0/521949798 learned_addr learned my addr 10.20.193.28:0/521949798 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:46.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fda2b3640 1 --2- 10.20.193.28:0/521949798 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcf66f0 0x55a75bd16ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fd92b1640 1 --2- 10.20.193.28:0/521949798 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcf61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fd9ab2640 1 -- 10.20.193.28:0/521949798 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 msgr2=0x55a75bcf61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:46.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/521949798 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcf61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fd9ab2640 1 -- 10.20.193.28:0/521949798 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcf66f0 msgr2=0x55a75bd16ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:46.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/521949798 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcf66f0 0x55a75bd16ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fd9ab2640 1 -- 10.20.193.28:0/521949798 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75bb68d80 con 0x55a75bcb1140 2026-02-25T11:50:46.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/521949798 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1140 0x55a75bcb1510 secure :-1 s=READY pgs=139 gs=1 cs=0 l=1 c_cookie=850746042068463248 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fc400b730 tx=0x7f6fc4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bce75048f8f6e90 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:46.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fa24e6640 1 -- 10.20.193.28:0/521949798 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fc401e640 con 0x55a75bcb1140 2026-02-25T11:50:46.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fa24e6640 1 -- 10.20.193.28:0/521949798 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6fc4015070 con 0x55a75bcb1140 2026-02-25T11:50:46.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.295+0000 7f6fa24e6640 1 -- 10.20.193.28:0/521949798 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fc4026c50 con 0x55a75bcb1140 2026-02-25T11:50:46.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.296+0000 7f6fddd38e00 1 -- 10.20.193.28:0/521949798 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1140 msgr2=0x55a75bcb1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.296+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/521949798 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1140 0x55a75bcb1510 secure :-1 s=READY pgs=139 gs=1 cs=0 l=1 c_cookie=850746042068463248 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fc400b730 tx=0x7f6fc4013000 comp rx=0 tx=0).stop 2026-02-25T11:50:46.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.296+0000 7f6fddd38e00 1 -- 10.20.193.28:0/521949798 shutdown_connections 2026-02-25T11:50:46.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.296+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/521949798 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcf66f0 0x55a75bd16ad0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.296+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/521949798 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcf61b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.296+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/521949798 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1140 0x55a75bcb1510 unknown :-1 s=CLOSED pgs=139 gs=1 cs=0 l=1 c_cookie=850746042068463248 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.296+0000 7f6fddd38e00 1 -- 10.20.193.28:0/521949798 >> 10.20.193.28:0/521949798 conn(0x55a75bbd0d40 msgr2=0x55a75bcafa20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:46.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.296+0000 7f6fddd38e00 1 -- 10.20.193.28:0/521949798 shutdown_connections 2026-02-25T11:50:46.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.296+0000 7f6fddd38e00 1 -- 10.20.193.28:0/521949798 wait complete. 2026-02-25T11:50:46.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.297+0000 7f6fddd38e00 1 -- start start 2026-02-25T11:50:46.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.297+0000 7f6fddd38e00 1 Processor -- start 2026-02-25T11:50:46.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fddd38e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcb1140 0x55a75bcb6380 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fddd38e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcb68c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fddd38e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcf66f0 0x55a75bcb5180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fddd38e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a75bcb56c0 con 0x55a75bcb1140 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fddd38e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a75bcb5830 con 0x55a75bcb1a50 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fddd38e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a75bcb59f0 con 0x55a75bcf66f0 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fd9ab2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcb1140 0x55a75bcb6380 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fd9ab2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcb1140 0x55a75bcb6380 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57776/0 (socket says 10.20.193.28:57776) 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fd9ab2640 1 -- 10.20.193.28:0/2143586516 learned_addr learned my addr 10.20.193.28:0/2143586516 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fd92b1640 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcb68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fda2b3640 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcf66f0 0x55a75bcb5180 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fd9ab2640 1 -- 10.20.193.28:0/2143586516 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcf66f0 msgr2=0x55a75bcb5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcf66f0 0x55a75bcb5180 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fd9ab2640 1 -- 10.20.193.28:0/2143586516 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 msgr2=0x55a75bcb68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcb68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fd9ab2640 1 -- 10.20.193.28:0/2143586516 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75bb41350 con 0x55a75bcb1140 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fda2b3640 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcf66f0 0x55a75bcb5180 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fd92b1640 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcb68c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.298+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcb1140 0x55a75bcb6380 secure :-1 s=READY pgs=510 gs=1 cs=0 l=1 c_cookie=6171472291548856439 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fc400b6d0 tx=0x7f6fc4026470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a57959eeef8077 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.299+0000 7f6f8f7fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:46.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.299+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/2143586516 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fc402a070 con 0x55a75bcb1140 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.299+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/2143586516 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6fc402b040 con 0x55a75bcb1140 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.299+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/2143586516 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fc40049f0 con 0x55a75bcb1140 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.299+0000 7f6fddd38e00 1 -- 10.20.193.28:0/2143586516 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc400bd10 con 0x55a75bcb1140 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.299+0000 7f6fddd38e00 10 client.16002 _ll_register_callbacks cb 0x55a75bb3e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.299+0000 7f6fddd38e00 1 -- 10.20.193.28:0/2143586516 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a75b8536f0 con 0x55a75bcb1140 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.299+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/2143586516 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6fc4017070 con 0x55a75bcb1140 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.299+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/2143586516 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6fc40045e0 con 0x55a75bcb1140 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.299+0000 7f6fddd38e00 10 client.16002 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.299+0000 7f6fddd38e00 1 -- 10.20.193.28:0/2143586516 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55a75bb53150 con 0x55a75bcb1140 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.300+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/2143586516 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6fc4024020 con 0x55a75bcb1140 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.300+0000 7f6fddd38e00 10 client.16002 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.300+0000 7f6fddd38e00 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.300+0000 7f6fddd38e00 20 client.16002 populate_metadata read hostname 'trial028' 2026-02-25T11:50:46.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.300+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a75bcdc5b0 0x55a75bcdca50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.300+0000 7f6fddd38e00 4 client.16002 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:50:46.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.300+0000 7f6fddd38e00 1 -- 10.20.193.28:0/2143586516 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a75bb8f910 con 0x55a75bcdc5b0 2026-02-25T11:50:46.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.300+0000 7f6fd92b1640 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a75bcdc5b0 0x55a75bcdca50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.301+0000 7f6fd92b1640 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a75bcdc5b0 0x55a75bcdca50 crc :-1 s=READY pgs=66 gs=4 cs=0 l=0 c_cookie=10357001405105433189 s_cookie=9348052925990606864 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fbb75e77987ea65 server_cookie=81baf49bab389810 in_seq=0 out_seq=0 2026-02-25T11:50:46.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.301+0000 7f6f8f7fe640 10 client.16002 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:46.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.301+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/2143586516 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55a75bb8f910 con 0x55a75bcdc5b0 2026-02-25T11:50:46.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.301+0000 7f6f8f7fe640 10 client.16002 handle_command_reply: tid=0 2026-02-25T11:50:46.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.301+0000 7f6fddd38e00 1 client.16002 shutdown 2026-02-25T11:50:46.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.301+0000 7f6fddd38e00 1 -- 10.20.193.28:0/2143586516 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcb1140 msgr2=0x55a75bcb6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.301+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcb1140 0x55a75bcb6380 secure :-1 s=READY pgs=510 gs=1 cs=0 l=1 c_cookie=6171472291548856439 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fc400b6d0 tx=0x7f6fc4026470 comp rx=0 tx=0).stop 2026-02-25T11:50:46.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.301+0000 7f6fddd38e00 1 -- 10.20.193.28:0/2143586516 shutdown_connections 2026-02-25T11:50:46.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.301+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a75bcdc5b0 0x55a75bcdca50 crc :-1 s=READY pgs=66 gs=4 cs=0 l=0 c_cookie=10357001405105433189 s_cookie=9348052925990606864 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.301+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcf66f0 0x55a75bcb5180 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.301+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcb68c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.301+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/2143586516 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcb1140 0x55a75bcb6380 unknown :-1 s=CLOSED pgs=510 gs=1 cs=0 l=1 c_cookie=6171472291548856439 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.301+0000 7f6fddd38e00 1 -- 10.20.193.28:0/2143586516 >> 10.20.193.28:0/2143586516 conn(0x55a75bbd0d40 msgr2=0x55a75bcaf230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:46.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.302+0000 7f6f8f7fe640 0 client.16002 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:46.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.302+0000 7f6fddd38e00 1 -- 10.20.193.28:0/2143586516 shutdown_connections 2026-02-25T11:50:46.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.302+0000 7f6fddd38e00 1 -- 10.20.193.28:0/2143586516 wait complete. 2026-02-25T11:50:46.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.302+0000 7f6fddd38e00 20 client.16002 trim_cache size 0 max 16384 2026-02-25T11:50:46.313 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.311+0000 7f6fddd38e00 1 Processor -- start 2026-02-25T11:50:46.313 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fddd38e00 1 -- start start 2026-02-25T11:50:46.313 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fddd38e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1140 0x55a75bcf60c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.313 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fddd38e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcf71d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fddd38e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcf66f0 0x55a75bd13eb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fddd38e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a75bbbb470 con 0x55a75bcf66f0 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fddd38e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a75b87b700 con 0x55a75bcb1a50 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fddd38e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a75bd16880 con 0x55a75bcb1140 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fda2b3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcf66f0 0x55a75bd13eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fd92b1640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcf71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fd92b1640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcf71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:56490/0 (socket says 10.20.193.28:56490) 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fd92b1640 1 -- 10.20.193.28:0/3310296260 learned_addr learned my addr 10.20.193.28:0/3310296260 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/3310296260 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1140 0x55a75bcf60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fd92b1640 1 -- 10.20.193.28:0/3310296260 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1140 msgr2=0x55a75bcf60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fd92b1640 1 --2- 10.20.193.28:0/3310296260 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1140 0x55a75bcf60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fd92b1640 1 -- 10.20.193.28:0/3310296260 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcf66f0 msgr2=0x55a75bd13eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fd92b1640 1 --2- 10.20.193.28:0/3310296260 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcf66f0 0x55a75bd13eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.312+0000 7f6fd92b1640 1 -- 10.20.193.28:0/3310296260 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75bb4dbd0 con 0x55a75bcb1a50 2026-02-25T11:50:46.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.313+0000 7f6fd92b1640 1 --2- 10.20.193.28:0/3310296260 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcf71d0 secure :-1 s=READY pgs=122 gs=1 cs=0 l=1 c_cookie=277237289790732206 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fc80e3d40 tx=0x7f6fc80026a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d8f1d8a077f7ae server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:46.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.313+0000 7f6fa1ce5640 1 -- 10.20.193.28:0/3310296260 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fc80b2150 con 0x55a75bcb1a50 2026-02-25T11:50:46.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.313+0000 7f6fa1ce5640 1 -- 10.20.193.28:0/3310296260 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6fc8009a80 con 0x55a75bcb1a50 2026-02-25T11:50:46.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.313+0000 7f6fa1ce5640 1 -- 10.20.193.28:0/3310296260 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fc80e3ee0 con 0x55a75bcb1a50 2026-02-25T11:50:46.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.313+0000 7f6fddd38e00 1 -- 10.20.193.28:0/3310296260 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 msgr2=0x55a75bcf71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.313+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/3310296260 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcf71d0 secure :-1 s=READY pgs=122 gs=1 cs=0 l=1 c_cookie=277237289790732206 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fc80e3d40 tx=0x7f6fc80026a0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.313+0000 7f6fddd38e00 1 -- 10.20.193.28:0/3310296260 shutdown_connections 2026-02-25T11:50:46.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.313+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/3310296260 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcf66f0 0x55a75bd13eb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.313+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/3310296260 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcb1a50 0x55a75bcf71d0 unknown :-1 s=CLOSED pgs=122 gs=1 cs=0 l=1 c_cookie=277237289790732206 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.313+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/3310296260 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1140 0x55a75bcf60c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.313+0000 7f6fddd38e00 1 -- 10.20.193.28:0/3310296260 >> 10.20.193.28:0/3310296260 conn(0x55a75bbd0d40 msgr2=0x55a75bcf6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:46.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.313+0000 7f6fddd38e00 1 -- 10.20.193.28:0/3310296260 shutdown_connections 2026-02-25T11:50:46.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.313+0000 7f6fddd38e00 1 -- 10.20.193.28:0/3310296260 wait complete. 2026-02-25T11:50:46.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.314+0000 7f6fddd38e00 1 -- start start 2026-02-25T11:50:46.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.314+0000 7f6fddd38e00 1 Processor -- start 2026-02-25T11:50:46.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.314+0000 7f6fddd38e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcb1140 0x55a75bcb0ae0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.314+0000 7f6fddd38e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1a50 0x55a75bcaf320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.314+0000 7f6fddd38e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcf66f0 0x55a75bcaf9f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.314+0000 7f6fddd38e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a75bcdcd40 con 0x55a75bcb1140 2026-02-25T11:50:46.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.314+0000 7f6fddd38e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a75bcb0120 con 0x55a75bcf66f0 2026-02-25T11:50:46.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.314+0000 7f6fddd38e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a75bce1150 con 0x55a75bcb1a50 2026-02-25T11:50:46.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.315+0000 7f6fda2b3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcf66f0 0x55a75bcaf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.315+0000 7f6fd92b1640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1a50 0x55a75bcaf320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.315+0000 7f6fda2b3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcf66f0 0x55a75bcaf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:56500/0 (socket says 10.20.193.28:56500) 2026-02-25T11:50:46.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.315+0000 7f6fda2b3640 1 -- 10.20.193.28:0/114827908 learned_addr learned my addr 10.20.193.28:0/114827908 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:46.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.315+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/114827908 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcb1140 0x55a75bcb0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.315+0000 7f6fda2b3640 1 -- 10.20.193.28:0/114827908 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1a50 msgr2=0x55a75bcaf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:46.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.315+0000 7f6fda2b3640 1 --2- 10.20.193.28:0/114827908 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1a50 0x55a75bcaf320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.315+0000 7f6fda2b3640 1 -- 10.20.193.28:0/114827908 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcb1140 msgr2=0x55a75bcb0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:46.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.315+0000 7f6fda2b3640 1 --2- 10.20.193.28:0/114827908 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcb1140 0x55a75bcb0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.315+0000 7f6fda2b3640 1 -- 10.20.193.28:0/114827908 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75bb41680 con 0x55a75bcf66f0 2026-02-25T11:50:46.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.315+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/114827908 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcb1140 0x55a75bcb0ae0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:46.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.315+0000 7f6fda2b3640 1 --2- 10.20.193.28:0/114827908 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcf66f0 0x55a75bcaf9f0 secure :-1 s=READY pgs=123 gs=1 cs=0 l=1 c_cookie=12226208844338476804 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fd0007010 tx=0x7f6fd00039b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9ac382aac88b704 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:46.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.315+0000 7f6f8f7fe640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:50:46.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.316+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/114827908 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fd003e040 con 0x55a75bcf66f0 2026-02-25T11:50:46.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.316+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/114827908 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6fd003f040 con 0x55a75bcf66f0 2026-02-25T11:50:46.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.316+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/114827908 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fd0047820 con 0x55a75bcf66f0 2026-02-25T11:50:46.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.316+0000 7f6fddd38e00 1 -- 10.20.193.28:0/114827908 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd4164d50 con 0x55a75bcf66f0 2026-02-25T11:50:46.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.316+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/114827908 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6fd00479b0 con 0x55a75bcf66f0 2026-02-25T11:50:46.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.316+0000 7f6fddd38e00 10 client.25817 _ll_register_callbacks cb 0x55a75bb3e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:46.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.316+0000 7f6fddd38e00 1 -- 10.20.193.28:0/114827908 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fc4015070 con 0x55a75bcf66f0 2026-02-25T11:50:46.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.316+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/114827908 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6fd00396a0 con 0x55a75bcf66f0 2026-02-25T11:50:46.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.316+0000 7f6fddd38e00 10 client.25817 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:46.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.316+0000 7f6fddd38e00 1 -- 10.20.193.28:0/114827908 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55a75bb533b0 con 0x55a75bcf66f0 2026-02-25T11:50:46.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.316+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/114827908 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6fd0058050 con 0x55a75bcf66f0 2026-02-25T11:50:46.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.317+0000 7f6fddd38e00 10 client.25817 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:46.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.317+0000 7f6fddd38e00 10 client.25817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:46.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.317+0000 7f6fddd38e00 20 client.25817 populate_metadata read hostname 'trial028' 2026-02-25T11:50:46.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.317+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/114827908 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a75bce2de0 0x55a75bdcd470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.317+0000 7f6fddd38e00 4 client.25817 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:50:46.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.317+0000 7f6fddd38e00 1 -- 10.20.193.28:0/114827908 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a75b850f40 con 0x55a75bce2de0 2026-02-25T11:50:46.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.317+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/114827908 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a75bce2de0 0x55a75bdcd470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.317+0000 7f6fd9ab2640 1 --2- 10.20.193.28:0/114827908 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a75bce2de0 0x55a75bdcd470 crc :-1 s=READY pgs=67 gs=4 cs=0 l=0 c_cookie=9820292742435172230 s_cookie=1826938902931724147 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8848b0381731ef86 server_cookie=195a9740af965773 in_seq=0 out_seq=0 2026-02-25T11:50:46.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.317+0000 7f6f8f7fe640 10 client.25817 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:46.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6f8f7fe640 1 -- 10.20.193.28:0/114827908 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x55a75b850f40 con 0x55a75bce2de0 2026-02-25T11:50:46.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6f8f7fe640 10 client.25817 handle_command_reply: tid=0 2026-02-25T11:50:46.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6fddd38e00 1 client.25817 shutdown 2026-02-25T11:50:46.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6fddd38e00 1 -- 10.20.193.28:0/114827908 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcf66f0 msgr2=0x55a75bcaf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/114827908 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcf66f0 0x55a75bcaf9f0 secure :-1 s=READY pgs=123 gs=1 cs=0 l=1 c_cookie=12226208844338476804 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fd0007010 tx=0x7f6fd00039b0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6fddd38e00 1 -- 10.20.193.28:0/114827908 shutdown_connections 2026-02-25T11:50:46.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/114827908 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a75bce2de0 0x55a75bdcd470 crc :-1 s=READY pgs=67 gs=4 cs=0 l=0 c_cookie=9820292742435172230 s_cookie=1826938902931724147 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/114827908 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a75bcf66f0 0x55a75bcaf9f0 unknown :-1 s=CLOSED pgs=123 gs=1 cs=0 l=1 c_cookie=12226208844338476804 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/114827908 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a75bcb1a50 0x55a75bcaf320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6fddd38e00 1 --2- 10.20.193.28:0/114827908 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a75bcb1140 0x55a75bcb0ae0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6fddd38e00 1 -- 10.20.193.28:0/114827908 >> 10.20.193.28:0/114827908 conn(0x55a75bbd0d40 msgr2=0x55a75bcf72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:46.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6f8f7fe640 0 client.25817 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:46.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6fddd38e00 1 -- 10.20.193.28:0/114827908 shutdown_connections 2026-02-25T11:50:46.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6fddd38e00 1 -- 10.20.193.28:0/114827908 wait complete. 2026-02-25T11:50:46.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.318+0000 7f6fddd38e00 20 client.25817 trim_cache size 0 max 16384 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:46.321 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.319+0000 7f6fdbd3d640 1 -- 10.20.193.28:0/1592195320 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6fb00b5760 msgr2=0x7f6fb00d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.319+0000 7f6fdbd3d640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6fb00b5760 0x7f6fb00d5c10 secure :-1 s=READY pgs=289 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fc4004740 tx=0x7f6fc400b410 comp rx=0 tx=0).stop 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.319+0000 7f6fdbd3d640 1 -- 10.20.193.28:0/1592195320 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fd4110130 msgr2=0x7f6fd411e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.319+0000 7f6fdbd3d640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fd4110130 0x7f6fd411e650 secure :-1 s=READY pgs=509 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fc800d950 tx=0x7f6fc800de20 comp rx=0 tx=0).stop 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.319+0000 7f6fdbd3d640 1 -- 10.20.193.28:0/1592195320 shutdown_connections 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.319+0000 7f6fdbd3d640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6fb00b5760 0x7f6fb00d5c10 unknown :-1 s=CLOSED pgs=289 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.319+0000 7f6fdbd3d640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6fd4115930 0x7f6fd4123a20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.319+0000 7f6fdbd3d640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fd4110130 0x7f6fd411e650 unknown :-1 s=CLOSED pgs=509 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.319+0000 7f6fdbd3d640 1 --2- 10.20.193.28:0/1592195320 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6fd410f740 0x7f6fd411e110 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.319+0000 7f6fdbd3d640 1 -- 10.20.193.28:0/1592195320 >> 10.20.193.28:0/1592195320 conn(0x7f6fd40ecff0 msgr2=0x7f6fd410d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.319+0000 7f6fdbd3d640 1 -- 10.20.193.28:0/1592195320 shutdown_connections 2026-02-25T11:50:46.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.319+0000 7f6fdbd3d640 1 -- 10.20.193.28:0/1592195320 wait complete. 2026-02-25T11:50:46.454 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:50:46.455 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:50:46.507 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:46 trial028 ceph-mon[39224]: pgmap v302: 97 pgs: 97 active+clean; 2.3 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:46.642 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:46.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:46 trial167 ceph-mon[43392]: pgmap v302: 97 pgs: 97 active+clean; 2.3 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:46.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:46 trial119 ceph-mon[42710]: pgmap v302: 97 pgs: 97 active+clean; 2.3 MiB data, 335 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:46.819 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.817+0000 7fb983577640 1 -- 10.20.193.28:0/3948887191 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb984110080 msgr2=0x7fb9841150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.819 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.817+0000 7fb983577640 1 --2- 10.20.193.28:0/3948887191 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb984110080 0x7fb9841150d0 secure :-1 s=READY pgs=124 gs=2 cs=0 l=1 c_cookie=17798701507805138518 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb96c009890 tx=0x7fb96c02f120 comp rx=0 tx=0).stop 2026-02-25T11:50:46.819 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.817+0000 7fb983577640 1 -- 10.20.193.28:0/3948887191 shutdown_connections 2026-02-25T11:50:46.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.817+0000 7fb983577640 1 --2- 10.20.193.28:0/3948887191 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb984115880 0x7fb984176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.817+0000 7fb983577640 1 --2- 10.20.193.28:0/3948887191 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb984110080 0x7fb9841150d0 unknown :-1 s=CLOSED pgs=124 gs=2 cs=0 l=1 c_cookie=17798701507805138518 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.817+0000 7fb983577640 1 --2- 10.20.193.28:0/3948887191 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb98410f690 0x7fb98410fa60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.817+0000 7fb983577640 1 -- 10.20.193.28:0/3948887191 >> 10.20.193.28:0/3948887191 conn(0x7fb9840ecf00 msgr2=0x7fb98410d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:46.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.817+0000 7fb983577640 1 -- 10.20.193.28:0/3948887191 shutdown_connections 2026-02-25T11:50:46.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.817+0000 7fb983577640 1 -- 10.20.193.28:0/3948887191 wait complete. 2026-02-25T11:50:46.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb983577640 1 Processor -- start 2026-02-25T11:50:46.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb983577640 1 -- start start 2026-02-25T11:50:46.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb983577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb98410f690 0x7fb98411e1a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb983577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb984110080 0x7fb98411e6e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb983577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb984115880 0x7fb984123a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb983577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb984124080 con 0x7fb98410f690 2026-02-25T11:50:46.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb983577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841241f0 con 0x7fb984115880 2026-02-25T11:50:46.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb983577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb9841243b0 con 0x7fb984110080 2026-02-25T11:50:46.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb982575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb98410f690 0x7fb98411e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb982575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb98410f690 0x7fb98411e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57838/0 (socket says 10.20.193.28:57838) 2026-02-25T11:50:46.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb982575640 1 -- 10.20.193.28:0/429777730 learned_addr learned my addr 10.20.193.28:0/429777730 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:46.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb982d76640 1 --2- 10.20.193.28:0/429777730 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb984115880 0x7fb984123a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb982575640 1 -- 10.20.193.28:0/429777730 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb984110080 msgr2=0x7fb98411e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb982575640 1 --2- 10.20.193.28:0/429777730 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb984110080 0x7fb98411e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb982575640 1 -- 10.20.193.28:0/429777730 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb984115880 msgr2=0x7fb984123a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb982575640 1 --2- 10.20.193.28:0/429777730 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb984115880 0x7fb984123a60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.818+0000 7fb982575640 1 -- 10.20.193.28:0/429777730 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c0094b0 con 0x7fb98410f690 2026-02-25T11:50:46.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.819+0000 7fb982575640 1 --2- 10.20.193.28:0/429777730 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb98410f690 0x7fb98411e1a0 secure :-1 s=READY pgs=511 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb978005940 tx=0x7fb97800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:46.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.819+0000 7fb96b7fe640 1 -- 10.20.193.28:0/429777730 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb97800cd10 con 0x7fb98410f690 2026-02-25T11:50:46.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.819+0000 7fb96b7fe640 1 -- 10.20.193.28:0/429777730 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb97800ce70 con 0x7fb98410f690 2026-02-25T11:50:46.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.819+0000 7fb96b7fe640 1 -- 10.20.193.28:0/429777730 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb9780106b0 con 0x7fb98410f690 2026-02-25T11:50:46.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.819+0000 7fb983577640 1 -- 10.20.193.28:0/429777730 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb984124690 con 0x7fb98410f690 2026-02-25T11:50:46.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.819+0000 7fb983577640 1 -- 10.20.193.28:0/429777730 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb984164d90 con 0x7fb98410f690 2026-02-25T11:50:46.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.820+0000 7fb983577640 1 -- 10.20.193.28:0/429777730 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9841248b0 con 0x7fb98410f690 2026-02-25T11:50:46.825 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.823+0000 7fb96b7fe640 1 -- 10.20.193.28:0/429777730 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fb97801f060 con 0x7fb98410f690 2026-02-25T11:50:46.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.825+0000 7fb96b7fe640 1 --2- 10.20.193.28:0/429777730 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb95c0b57e0 0x7fb95c0d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:46.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.825+0000 7fb96b7fe640 1 -- 10.20.193.28:0/429777730 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb978014070 con 0x7fb98410f690 2026-02-25T11:50:46.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.825+0000 7fb981d74640 1 --2- 10.20.193.28:0/429777730 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb95c0b57e0 0x7fb95c0d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:46.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.825+0000 7fb96b7fe640 1 -- 10.20.193.28:0/429777730 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fb9780e7090 con 0x7fb98410f690 2026-02-25T11:50:46.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.826+0000 7fb981d74640 1 --2- 10.20.193.28:0/429777730 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb95c0b57e0 0x7fb95c0d5c90 secure :-1 s=READY pgs=290 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb96c02f630 tx=0x7fb96c005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:46.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.967+0000 7fb983577640 1 -- 10.20.193.28:0/429777730 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb98411f2b0 con 0x7fb98410f690 2026-02-25T11:50:46.970 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.967+0000 7fb96b7fe640 1 -- 10.20.193.28:0/429777730 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7fb9780a98b0 con 0x7fb98410f690 2026-02-25T11:50:46.970 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:50:46.970 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:50:46.971 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:50:46.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.969+0000 7fb983577640 1 -- 10.20.193.28:0/429777730 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb95c0b57e0 msgr2=0x7fb95c0d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.969+0000 7fb983577640 1 --2- 10.20.193.28:0/429777730 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb95c0b57e0 0x7fb95c0d5c90 secure :-1 s=READY pgs=290 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb96c02f630 tx=0x7fb96c005900 comp rx=0 tx=0).stop 2026-02-25T11:50:46.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.969+0000 7fb983577640 1 -- 10.20.193.28:0/429777730 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb98410f690 msgr2=0x7fb98411e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:46.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.969+0000 7fb983577640 1 --2- 10.20.193.28:0/429777730 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb98410f690 0x7fb98411e1a0 secure :-1 s=READY pgs=511 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb978005940 tx=0x7fb97800ee60 comp rx=0 tx=0).stop 2026-02-25T11:50:46.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.969+0000 7fb983577640 1 -- 10.20.193.28:0/429777730 shutdown_connections 2026-02-25T11:50:46.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.969+0000 7fb983577640 1 --2- 10.20.193.28:0/429777730 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb95c0b57e0 0x7fb95c0d5c90 unknown :-1 s=CLOSED pgs=290 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.969+0000 7fb983577640 1 --2- 10.20.193.28:0/429777730 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb984115880 0x7fb984123a60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.969+0000 7fb983577640 1 --2- 10.20.193.28:0/429777730 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb984110080 0x7fb98411e6e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.969+0000 7fb983577640 1 --2- 10.20.193.28:0/429777730 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb98410f690 0x7fb98411e1a0 unknown :-1 s=CLOSED pgs=511 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:46.972 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.969+0000 7fb983577640 1 -- 10.20.193.28:0/429777730 >> 10.20.193.28:0/429777730 conn(0x7fb9840ecf00 msgr2=0x7fb98410d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:46.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.969+0000 7fb983577640 1 -- 10.20.193.28:0/429777730 shutdown_connections 2026-02-25T11:50:46.973 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:46.969+0000 7fb983577640 1 -- 10.20.193.28:0/429777730 wait complete. 2026-02-25T11:50:47.108 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:50:47.293 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:47.471 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.469+0000 7f376cd2e640 1 -- 10.20.193.28:0/1780232164 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3768110130 msgr2=0x7f3768115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:47.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.469+0000 7f376cd2e640 1 --2- 10.20.193.28:0/1780232164 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3768110130 0x7f3768115180 secure :-1 s=READY pgs=140 gs=2 cs=0 l=1 c_cookie=4338190002993653516 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3750009950 tx=0x7f375002f190 comp rx=0 tx=0).stop 2026-02-25T11:50:47.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.469+0000 7f376cd2e640 1 -- 10.20.193.28:0/1780232164 shutdown_connections 2026-02-25T11:50:47.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.469+0000 7f376cd2e640 1 --2- 10.20.193.28:0/1780232164 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3768115930 0x7f3768176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.469+0000 7f376cd2e640 1 --2- 10.20.193.28:0/1780232164 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3768110130 0x7f3768115180 unknown :-1 s=CLOSED pgs=140 gs=2 cs=0 l=1 c_cookie=4338190002993653516 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.469+0000 7f376cd2e640 1 --2- 10.20.193.28:0/1780232164 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f376810f740 0x7f376810fb10 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.469+0000 7f376cd2e640 1 -- 10.20.193.28:0/1780232164 >> 10.20.193.28:0/1780232164 conn(0x7f37680ecff0 msgr2=0x7f376810d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:47.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.469+0000 7f376cd2e640 1 -- 10.20.193.28:0/1780232164 shutdown_connections 2026-02-25T11:50:47.472 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.469+0000 7f376cd2e640 1 -- 10.20.193.28:0/1780232164 wait complete. 2026-02-25T11:50:47.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f376cd2e640 1 Processor -- start 2026-02-25T11:50:47.474 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f376cd2e640 1 -- start start 2026-02-25T11:50:47.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f376cd2e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f376810f740 0x7f37680f5a50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f376cd2e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3768110130 0x7f3768176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f376cd2e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3768115930 0x7f37680f60e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f376cd2e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f37680fb360 con 0x7f3768115930 2026-02-25T11:50:47.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f376cd2e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37680fb4d0 con 0x7f3768110130 2026-02-25T11:50:47.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f376cd2e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37680fb640 con 0x7f376810f740 2026-02-25T11:50:47.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f3766d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3768115930 0x7f37680f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f3766575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f376810f740 0x7f37680f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f3766575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f376810f740 0x7f37680f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:41360/0 (socket says 10.20.193.28:41360) 2026-02-25T11:50:47.475 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f3766575640 1 -- 10.20.193.28:0/751556115 learned_addr learned my addr 10.20.193.28:0/751556115 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:47.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f3765d74640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3768110130 0x7f3768176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.470+0000 7f3766575640 1 -- 10.20.193.28:0/751556115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3768110130 msgr2=0x7f3768176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:47.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.471+0000 7f3766575640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3768110130 0x7f3768176200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.471+0000 7f3766575640 1 -- 10.20.193.28:0/751556115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3768115930 msgr2=0x7f37680f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:47.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.471+0000 7f3766575640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3768115930 0x7f37680f60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.471+0000 7f3766575640 1 -- 10.20.193.28:0/751556115 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3750009570 con 0x7f376810f740 2026-02-25T11:50:47.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.471+0000 7f3766d76640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3768115930 0x7f37680f60e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:47.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.471+0000 7f3766575640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f376810f740 0x7f37680f5a50 secure :-1 s=READY pgs=141 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f375c009480 tx=0x7f375c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:47.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.471+0000 7f374f7fe640 1 -- 10.20.193.28:0/751556115 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f375c00cd40 con 0x7f376810f740 2026-02-25T11:50:47.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.471+0000 7f374f7fe640 1 -- 10.20.193.28:0/751556115 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f375c004d10 con 0x7f376810f740 2026-02-25T11:50:47.476 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.471+0000 7f374f7fe640 1 -- 10.20.193.28:0/751556115 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f375c0106b0 con 0x7f376810f740 2026-02-25T11:50:47.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.471+0000 7f376cd2e640 1 -- 10.20.193.28:0/751556115 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37680fb920 con 0x7f376810f740 2026-02-25T11:50:47.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.471+0000 7f376cd2e640 1 -- 10.20.193.28:0/751556115 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37680fbda0 con 0x7f376810f740 2026-02-25T11:50:47.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.474+0000 7f374f7fe640 1 -- 10.20.193.28:0/751556115 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f375c01f060 con 0x7f376810f740 2026-02-25T11:50:47.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.476+0000 7f374f7fe640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f37400b5760 0x7f37400d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.476+0000 7f374f7fe640 1 -- 10.20.193.28:0/751556115 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f375c014070 con 0x7f376810f740 2026-02-25T11:50:47.479 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.476+0000 7f3765d74640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f37400b5760 0x7f37400d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.482 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f376ed29e00 1 Processor -- start 2026-02-25T11:50:47.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f376ed29e00 1 -- start start 2026-02-25T11:50:47.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f376ed29e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278b7540 0x55eb278b7910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f376ed29e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7e50 0x55eb278fc5b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f376ed29e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278fcaf0 0x55eb2791ced0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f376ed29e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55eb277c48a0 con 0x55eb278fcaf0 2026-02-25T11:50:47.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f376ed29e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55eb277c05d0 con 0x55eb278b7540 2026-02-25T11:50:47.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f376ed29e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb277ba570 con 0x55eb278b7e50 2026-02-25T11:50:47.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f3765d74640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f37400b5760 0x7f37400d5c10 secure :-1 s=READY pgs=291 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f37500023d0 tx=0x7f3750031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:47.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f3766575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278b7540 0x55eb278b7910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.483 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f3765d74640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7e50 0x55eb278fc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f3766575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278b7540 0x55eb278b7910 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:56538/0 (socket says 10.20.193.28:56538) 2026-02-25T11:50:47.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f3766575640 1 -- 10.20.193.28:0/2738588272 learned_addr learned my addr 10.20.193.28:0/2738588272 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:47.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.477+0000 7f3766d76640 1 --2- 10.20.193.28:0/2738588272 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278fcaf0 0x55eb2791ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.478+0000 7f3766d76640 1 -- 10.20.193.28:0/2738588272 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7e50 msgr2=0x55eb278fc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:47.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.478+0000 7f3766d76640 1 --2- 10.20.193.28:0/2738588272 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7e50 0x55eb278fc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.478+0000 7f3766d76640 1 -- 10.20.193.28:0/2738588272 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278b7540 msgr2=0x55eb278b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:47.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.478+0000 7f3766d76640 1 --2- 10.20.193.28:0/2738588272 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278b7540 0x55eb278b7910 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.478+0000 7f3766d76640 1 -- 10.20.193.28:0/2738588272 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb27765d10 con 0x55eb278fcaf0 2026-02-25T11:50:47.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.478+0000 7f3766575640 1 --2- 10.20.193.28:0/2738588272 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278b7540 0x55eb278b7910 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:47.484 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.478+0000 7f3766d76640 1 --2- 10.20.193.28:0/2738588272 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278fcaf0 0x55eb2791ced0 secure :-1 s=READY pgs=512 gs=3 cs=0 l=1 c_cookie=5465832524383423804 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f37580051a0 tx=0x7f3758005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bda89b8c1a68d3c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:47.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.479+0000 7f374cff9640 1 -- 10.20.193.28:0/2738588272 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3758011710 con 0x55eb278fcaf0 2026-02-25T11:50:47.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.479+0000 7f374cff9640 1 -- 10.20.193.28:0/2738588272 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3758011d30 con 0x55eb278fcaf0 2026-02-25T11:50:47.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.479+0000 7f374cff9640 1 -- 10.20.193.28:0/2738588272 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f375801be20 con 0x55eb278fcaf0 2026-02-25T11:50:47.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.479+0000 7f376ed29e00 1 -- 10.20.193.28:0/2738588272 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278fcaf0 msgr2=0x55eb2791ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:47.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.479+0000 7f376ed29e00 1 --2- 10.20.193.28:0/2738588272 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278fcaf0 0x55eb2791ced0 secure :-1 s=READY pgs=512 gs=3 cs=0 l=1 c_cookie=5465832524383423804 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f37580051a0 tx=0x7f3758005670 comp rx=0 tx=0).stop 2026-02-25T11:50:47.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.479+0000 7f376ed29e00 1 -- 10.20.193.28:0/2738588272 shutdown_connections 2026-02-25T11:50:47.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.479+0000 7f376ed29e00 1 --2- 10.20.193.28:0/2738588272 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278fcaf0 0x55eb2791ced0 unknown :-1 s=CLOSED pgs=512 gs=3 cs=0 l=1 c_cookie=5465832524383423804 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.479+0000 7f376ed29e00 1 --2- 10.20.193.28:0/2738588272 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7e50 0x55eb278fc5b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.479+0000 7f376ed29e00 1 --2- 10.20.193.28:0/2738588272 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278b7540 0x55eb278b7910 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.479+0000 7f376ed29e00 1 -- 10.20.193.28:0/2738588272 >> 10.20.193.28:0/2738588272 conn(0x55eb277cd7a0 msgr2=0x55eb278ad410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:47.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.479+0000 7f376ed29e00 1 -- 10.20.193.28:0/2738588272 shutdown_connections 2026-02-25T11:50:47.486 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.479+0000 7f376ed29e00 1 -- 10.20.193.28:0/2738588272 wait complete. 2026-02-25T11:50:47.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.481+0000 7f376ed29e00 1 -- start start 2026-02-25T11:50:47.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.481+0000 7f376ed29e00 1 Processor -- start 2026-02-25T11:50:47.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.481+0000 7f376ed29e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278b7540 0x55eb278cf860 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.481+0000 7f376ed29e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7e50 0x55eb278cfda0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.481+0000 7f376ed29e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278fcaf0 0x55eb278d4d90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.481+0000 7f376ed29e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55eb278d52d0 con 0x55eb278b7e50 2026-02-25T11:50:47.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.481+0000 7f376ed29e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55eb278d5440 con 0x55eb278b7540 2026-02-25T11:50:47.490 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.481+0000 7f376ed29e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb278d5600 con 0x55eb278fcaf0 2026-02-25T11:50:47.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.482+0000 7f3766d76640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278fcaf0 0x55eb278d4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.482+0000 7f3766d76640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278fcaf0 0x55eb278d4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:41386/0 (socket says 10.20.193.28:41386) 2026-02-25T11:50:47.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.482+0000 7f3766d76640 1 -- 10.20.193.28:0/622812061 learned_addr learned my addr 10.20.193.28:0/622812061 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:47.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.482+0000 7f3766575640 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278b7540 0x55eb278cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.482+0000 7f3765d74640 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7e50 0x55eb278cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.482+0000 7f3766d76640 1 -- 10.20.193.28:0/622812061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278b7540 msgr2=0x55eb278cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:47.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.482+0000 7f3766d76640 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278b7540 0x55eb278cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.482+0000 7f3766d76640 1 -- 10.20.193.28:0/622812061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7e50 msgr2=0x55eb278cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:47.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.482+0000 7f3766d76640 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7e50 0x55eb278cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.482+0000 7f3766d76640 1 -- 10.20.193.28:0/622812061 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375800ad60 con 0x55eb278fcaf0 2026-02-25T11:50:47.491 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.482+0000 7f3766575640 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278b7540 0x55eb278cf860 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:47.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.482+0000 7f3766d76640 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278fcaf0 0x55eb278d4d90 secure :-1 s=READY pgs=142 gs=2 cs=0 l=1 c_cookie=7981141249241532455 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f375801b970 tx=0x7f375801bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec2b5ae27424827 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:47.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.482+0000 7f3717fff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:50:47.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f3717fff640 1 -- 10.20.193.28:0/622812061 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3758023070 con 0x55eb278fcaf0 2026-02-25T11:50:47.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f3717fff640 1 -- 10.20.193.28:0/622812061 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3758028420 con 0x55eb278fcaf0 2026-02-25T11:50:47.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f3717fff640 1 -- 10.20.193.28:0/622812061 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3758011bd0 con 0x55eb278fcaf0 2026-02-25T11:50:47.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f376ed29e00 1 -- 10.20.193.28:0/622812061 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37680fbda0 con 0x55eb278fcaf0 2026-02-25T11:50:47.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f3717fff640 1 -- 10.20.193.28:0/622812061 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f375801c680 con 0x55eb278fcaf0 2026-02-25T11:50:47.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f376ed29e00 10 client.25804 _ll_register_callbacks cb 0x55eb2773ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:47.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f376ed29e00 1 -- 10.20.193.28:0/622812061 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb274506f0 con 0x55eb278fcaf0 2026-02-25T11:50:47.492 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f3717fff640 1 -- 10.20.193.28:0/622812061 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3758028e90 con 0x55eb278fcaf0 2026-02-25T11:50:47.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f376ed29e00 10 client.25804 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:47.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f376ed29e00 1 -- 10.20.193.28:0/622812061 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55eb2773e3c0 con 0x55eb278fcaf0 2026-02-25T11:50:47.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f3717fff640 1 -- 10.20.193.28:0/622812061 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f3758031070 con 0x55eb278fcaf0 2026-02-25T11:50:47.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f376ed29e00 10 client.25804 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:47.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f376ed29e00 10 client.25804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:50:47.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.483+0000 7f376ed29e00 20 client.25804 populate_metadata read hostname 'trial028' 2026-02-25T11:50:47.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.484+0000 7f376ed29e00 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55eb278f9e90 0x55eb278fa330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.484+0000 7f376ed29e00 4 client.25804 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:50:47.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.484+0000 7f376ed29e00 1 -- 10.20.193.28:0/622812061 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb2778cfc0 con 0x55eb278f9e90 2026-02-25T11:50:47.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.484+0000 7f3766575640 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55eb278f9e90 0x55eb278fa330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.493 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.484+0000 7f3766575640 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55eb278f9e90 0x55eb278fa330 crc :-1 s=READY pgs=33 gs=4 cs=0 l=0 c_cookie=18367859748437391067 s_cookie=485059588837095374 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fee7bf1fc5ce76db server_cookie=6bb4720478837ce in_seq=0 out_seq=0 2026-02-25T11:50:47.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.484+0000 7f3717fff640 10 client.25804 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:50:47.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f3717fff640 1 -- 10.20.193.28:0/622812061 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55eb2778cfc0 con 0x55eb278f9e90 2026-02-25T11:50:47.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f3717fff640 10 client.25804 handle_command_reply: tid=0 2026-02-25T11:50:47.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f376ed29e00 1 client.25804 shutdown 2026-02-25T11:50:47.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f376ed29e00 1 -- 10.20.193.28:0/622812061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278fcaf0 msgr2=0x55eb278d4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:47.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f376ed29e00 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278fcaf0 0x55eb278d4d90 secure :-1 s=READY pgs=142 gs=2 cs=0 l=1 c_cookie=7981141249241532455 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f375801b970 tx=0x7f375801bf10 comp rx=0 tx=0).stop 2026-02-25T11:50:47.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f376ed29e00 1 -- 10.20.193.28:0/622812061 shutdown_connections 2026-02-25T11:50:47.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f376ed29e00 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55eb278f9e90 0x55eb278fa330 crc :-1 s=READY pgs=33 gs=4 cs=0 l=0 c_cookie=18367859748437391067 s_cookie=485059588837095374 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f376ed29e00 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278fcaf0 0x55eb278d4d90 unknown :-1 s=CLOSED pgs=142 gs=2 cs=0 l=1 c_cookie=7981141249241532455 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f376ed29e00 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7e50 0x55eb278cfda0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.494 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f376ed29e00 1 --2- 10.20.193.28:0/622812061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278b7540 0x55eb278cf860 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f3717fff640 0 client.25804 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:50:47.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f376ed29e00 1 -- 10.20.193.28:0/622812061 >> 10.20.193.28:0/622812061 conn(0x55eb277cd7a0 msgr2=0x55eb278adb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:47.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f376ed29e00 1 -- 10.20.193.28:0/622812061 shutdown_connections 2026-02-25T11:50:47.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f376ed29e00 1 -- 10.20.193.28:0/622812061 wait complete. 2026-02-25T11:50:47.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.485+0000 7f376ed29e00 20 client.25804 trim_cache size 0 max 16384 2026-02-25T11:50:47.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.493+0000 7f376ed29e00 1 Processor -- start 2026-02-25T11:50:47.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.493+0000 7f376ed29e00 1 -- start start 2026-02-25T11:50:47.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f376ed29e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7540 0x55eb278f5270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f376ed29e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7e50 0x55eb278f57b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f376ed29e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278fcaf0 0x55eb278cbce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f376ed29e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55eb277b81a0 con 0x55eb278b7e50 2026-02-25T11:50:47.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f376ed29e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55eb27478700 con 0x55eb278fcaf0 2026-02-25T11:50:47.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f376ed29e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb27889e80 con 0x55eb278b7540 2026-02-25T11:50:47.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f3766575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7540 0x55eb278f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f3766575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7540 0x55eb278f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:41390/0 (socket says 10.20.193.28:41390) 2026-02-25T11:50:47.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f3766575640 1 -- 10.20.193.28:0/945715681 learned_addr learned my addr 10.20.193.28:0/945715681 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:47.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f3766d76640 1 --2- 10.20.193.28:0/945715681 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278fcaf0 0x55eb278cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f3765d74640 1 --2- 10.20.193.28:0/945715681 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7e50 0x55eb278f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f3766575640 1 -- 10.20.193.28:0/945715681 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278fcaf0 msgr2=0x55eb278cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:47.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f3766575640 1 --2- 10.20.193.28:0/945715681 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278fcaf0 0x55eb278cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f3766575640 1 -- 10.20.193.28:0/945715681 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7e50 msgr2=0x55eb278f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:47.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f3766575640 1 --2- 10.20.193.28:0/945715681 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7e50 0x55eb278f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.494+0000 7f3766575640 1 -- 10.20.193.28:0/945715681 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb277500e0 con 0x55eb278b7540 2026-02-25T11:50:47.499 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.495+0000 7f3766575640 1 --2- 10.20.193.28:0/945715681 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7540 0x55eb278f5270 secure :-1 s=READY pgs=143 gs=1 cs=0 l=1 c_cookie=1314852788961911056 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f375c0acf90 tx=0x7f375c009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=123f4ba52a01a110 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:47.499 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.495+0000 7f37324e6640 1 -- 10.20.193.28:0/945715681 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f375c06f070 con 0x55eb278b7540 2026-02-25T11:50:47.499 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.495+0000 7f37324e6640 1 -- 10.20.193.28:0/945715681 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f375c0ad9b0 con 0x55eb278b7540 2026-02-25T11:50:47.499 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.495+0000 7f37324e6640 1 -- 10.20.193.28:0/945715681 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f375c0acdd0 con 0x55eb278b7540 2026-02-25T11:50:47.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.495+0000 7f376ed29e00 1 -- 10.20.193.28:0/945715681 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7540 msgr2=0x55eb278f5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:47.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.495+0000 7f376ed29e00 1 --2- 10.20.193.28:0/945715681 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7540 0x55eb278f5270 secure :-1 s=READY pgs=143 gs=1 cs=0 l=1 c_cookie=1314852788961911056 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f375c0acf90 tx=0x7f375c009f70 comp rx=0 tx=0).stop 2026-02-25T11:50:47.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.495+0000 7f376ed29e00 1 -- 10.20.193.28:0/945715681 shutdown_connections 2026-02-25T11:50:47.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.495+0000 7f376ed29e00 1 --2- 10.20.193.28:0/945715681 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278fcaf0 0x55eb278cbce0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.495+0000 7f376ed29e00 1 --2- 10.20.193.28:0/945715681 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7e50 0x55eb278f57b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.495+0000 7f376ed29e00 1 --2- 10.20.193.28:0/945715681 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7540 0x55eb278f5270 unknown :-1 s=CLOSED pgs=143 gs=1 cs=0 l=1 c_cookie=1314852788961911056 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.495+0000 7f376ed29e00 1 -- 10.20.193.28:0/945715681 >> 10.20.193.28:0/945715681 conn(0x55eb277cd7a0 msgr2=0x55eb278ba230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:47.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.495+0000 7f376ed29e00 1 -- 10.20.193.28:0/945715681 shutdown_connections 2026-02-25T11:50:47.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.495+0000 7f376ed29e00 1 -- 10.20.193.28:0/945715681 wait complete. 2026-02-25T11:50:47.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.496+0000 7f376ed29e00 1 -- start start 2026-02-25T11:50:47.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.496+0000 7f376ed29e00 1 Processor -- start 2026-02-25T11:50:47.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.496+0000 7f376ed29e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7540 0x55eb278db8e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.496+0000 7f376ed29e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7e50 0x55eb278d7180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.496+0000 7f376ed29e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278fcaf0 0x55eb278d76c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.496+0000 7f376ed29e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55eb278d7d60 con 0x55eb278b7540 2026-02-25T11:50:47.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.496+0000 7f376ed29e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55eb278d7ed0 con 0x55eb278fcaf0 2026-02-25T11:50:47.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.496+0000 7f376ed29e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb278d8090 con 0x55eb278b7e50 2026-02-25T11:50:47.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.496+0000 7f3766575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7540 0x55eb278db8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.496+0000 7f3765d74640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7e50 0x55eb278d7180 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.496+0000 7f3766575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7540 0x55eb278db8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57876/0 (socket says 10.20.193.28:57876) 2026-02-25T11:50:47.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.496+0000 7f3766575640 1 -- 10.20.193.28:0/4185087347 learned_addr learned my addr 10.20.193.28:0/4185087347 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:47.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.496+0000 7f3766d76640 1 --2- 10.20.193.28:0/4185087347 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278fcaf0 0x55eb278d76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f3766575640 1 -- 10.20.193.28:0/4185087347 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7e50 msgr2=0x55eb278d7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:47.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f3766575640 1 --2- 10.20.193.28:0/4185087347 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7e50 0x55eb278d7180 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f3766575640 1 -- 10.20.193.28:0/4185087347 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278fcaf0 msgr2=0x55eb278d76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:47.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f3766575640 1 --2- 10.20.193.28:0/4185087347 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278fcaf0 0x55eb278d76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f3766575640 1 -- 10.20.193.28:0/4185087347 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2774ab60 con 0x55eb278b7540 2026-02-25T11:50:47.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f3766575640 1 --2- 10.20.193.28:0/4185087347 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7540 0x55eb278db8e0 secure :-1 s=READY pgs=513 gs=1 cs=0 l=1 c_cookie=11881776941831916630 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f375c0a9580 tx=0x7f375c0a9680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e48d2d1e518c56 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:47.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f3717fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:47.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f3717fff640 1 -- 10.20.193.28:0/4185087347 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f375c06f040 con 0x55eb278b7540 2026-02-25T11:50:47.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f3717fff640 1 -- 10.20.193.28:0/4185087347 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f375c0b52b0 con 0x55eb278b7540 2026-02-25T11:50:47.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f3717fff640 1 -- 10.20.193.28:0/4185087347 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f375c0b7c60 con 0x55eb278b7540 2026-02-25T11:50:47.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f376ed29e00 1 -- 10.20.193.28:0/4185087347 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37680fb920 con 0x55eb278b7540 2026-02-25T11:50:47.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f3717fff640 1 -- 10.20.193.28:0/4185087347 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f375c0b7df0 con 0x55eb278b7540 2026-02-25T11:50:47.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f3717fff640 10 client.16032 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:50:47.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f376ed29e00 10 client.16032 _ll_register_callbacks cb 0x55eb2773ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:47.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f376ed29e00 1 -- 10.20.193.28:0/4185087347 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3758011d30 con 0x55eb278b7540 2026-02-25T11:50:47.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.497+0000 7f3717fff640 1 -- 10.20.193.28:0/4185087347 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f375c0b80f0 con 0x55eb278b7540 2026-02-25T11:50:47.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.498+0000 7f376ed29e00 10 client.16032 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:47.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.498+0000 7f376ed29e00 1 -- 10.20.193.28:0/4185087347 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55eb2773e6f0 con 0x55eb278b7540 2026-02-25T11:50:47.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.498+0000 7f3717fff640 1 -- 10.20.193.28:0/4185087347 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f375c0d2080 con 0x55eb278b7540 2026-02-25T11:50:47.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.498+0000 7f376ed29e00 10 client.16032 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:47.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.498+0000 7f376ed29e00 10 client.16032 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:50:47.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.498+0000 7f376ed29e00 20 client.16032 populate_metadata read hostname 'trial028' 2026-02-25T11:50:47.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.498+0000 7f376ed29e00 1 --2- 10.20.193.28:0/4185087347 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55eb279a9bf0 0x55eb279ca0a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:47.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.498+0000 7f376ed29e00 4 client.16032 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:50:47.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.498+0000 7f376ed29e00 1 -- 10.20.193.28:0/4185087347 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb2744df40 con 0x55eb279a9bf0 2026-02-25T11:50:47.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.498+0000 7f3765d74640 1 --2- 10.20.193.28:0/4185087347 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55eb279a9bf0 0x55eb279ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:47.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f3765d74640 1 --2- 10.20.193.28:0/4185087347 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55eb279a9bf0 0x55eb279ca0a0 crc :-1 s=READY pgs=34 gs=4 cs=0 l=0 c_cookie=10387723254527871853 s_cookie=1574780729027799840 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90289b437332cb6d server_cookie=15dabebac0d52b20 in_seq=0 out_seq=0 2026-02-25T11:50:47.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f3717fff640 10 client.16032 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:50:47.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f3717fff640 1 -- 10.20.193.28:0/4185087347 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb2744df40 con 0x55eb279a9bf0 2026-02-25T11:50:47.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f3717fff640 10 client.16032 handle_command_reply: tid=0 2026-02-25T11:50:47.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f376ed29e00 1 client.16032 shutdown 2026-02-25T11:50:47.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f376ed29e00 1 -- 10.20.193.28:0/4185087347 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7540 msgr2=0x55eb278db8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:47.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f376ed29e00 1 --2- 10.20.193.28:0/4185087347 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7540 0x55eb278db8e0 secure :-1 s=READY pgs=513 gs=1 cs=0 l=1 c_cookie=11881776941831916630 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f375c0a9580 tx=0x7f375c0a9680 comp rx=0 tx=0).stop 2026-02-25T11:50:47.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f376ed29e00 1 -- 10.20.193.28:0/4185087347 shutdown_connections 2026-02-25T11:50:47.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f376ed29e00 1 --2- 10.20.193.28:0/4185087347 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55eb279a9bf0 0x55eb279ca0a0 crc :-1 s=READY pgs=34 gs=4 cs=0 l=0 c_cookie=10387723254527871853 s_cookie=1574780729027799840 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f376ed29e00 1 --2- 10.20.193.28:0/4185087347 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55eb278fcaf0 0x55eb278d76c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f376ed29e00 1 --2- 10.20.193.28:0/4185087347 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55eb278b7e50 0x55eb278d7180 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f376ed29e00 1 --2- 10.20.193.28:0/4185087347 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55eb278b7540 0x55eb278db8e0 unknown :-1 s=CLOSED pgs=513 gs=1 cs=0 l=1 c_cookie=11881776941831916630 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f3717fff640 0 client.16032 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:50:47.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f376ed29e00 1 -- 10.20.193.28:0/4185087347 >> 10.20.193.28:0/4185087347 conn(0x55eb277cd7a0 msgr2=0x55eb278f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:47.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f376ed29e00 1 -- 10.20.193.28:0/4185087347 shutdown_connections 2026-02-25T11:50:47.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f376ed29e00 1 -- 10.20.193.28:0/4185087347 wait complete. 2026-02-25T11:50:47.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.499+0000 7f376ed29e00 20 client.16032 trim_cache size 0 max 16384 2026-02-25T11:50:47.508 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:50:47.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.500+0000 7f376cd2e640 1 -- 10.20.193.28:0/751556115 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f37400b5760 msgr2=0x7f37400d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:47.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.500+0000 7f376cd2e640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f37400b5760 0x7f37400d5c10 secure :-1 s=READY pgs=291 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f37500023d0 tx=0x7f3750031040 comp rx=0 tx=0).stop 2026-02-25T11:50:47.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.500+0000 7f376cd2e640 1 -- 10.20.193.28:0/751556115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f376810f740 msgr2=0x7f37680f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:47.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.500+0000 7f376cd2e640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f376810f740 0x7f37680f5a50 secure :-1 s=READY pgs=141 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f375c009480 tx=0x7f375c00ee60 comp rx=0 tx=0).stop 2026-02-25T11:50:47.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.500+0000 7f376cd2e640 1 -- 10.20.193.28:0/751556115 shutdown_connections 2026-02-25T11:50:47.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.500+0000 7f376cd2e640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f37400b5760 0x7f37400d5c10 unknown :-1 s=CLOSED pgs=291 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.500+0000 7f376cd2e640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3768115930 0x7f37680f60e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.500+0000 7f376cd2e640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3768110130 0x7f3768176200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.500+0000 7f376cd2e640 1 --2- 10.20.193.28:0/751556115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f376810f740 0x7f37680f5a50 unknown :-1 s=CLOSED pgs=141 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:47.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.500+0000 7f376cd2e640 1 -- 10.20.193.28:0/751556115 >> 10.20.193.28:0/751556115 conn(0x7f37680ecff0 msgr2=0x7f376810dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:47.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.500+0000 7f376cd2e640 1 -- 10.20.193.28:0/751556115 shutdown_connections 2026-02-25T11:50:47.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:47.500+0000 7f376cd2e640 1 -- 10.20.193.28:0/751556115 wait complete. 2026-02-25T11:50:47.588 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:47 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/429777730' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:50:47.638 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:50:47.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:47 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/429777730' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:50:47.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:47 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/429777730' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:50:47.823 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:48.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.005+0000 7f8a81254640 1 -- 10.20.193.28:0/1539070988 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a7c060b00 msgr2=0x7f8a7c112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.005+0000 7f8a81254640 1 --2- 10.20.193.28:0/1539070988 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a7c060b00 0x7f8a7c112e10 secure :-1 s=READY pgs=514 gs=2 cs=0 l=1 c_cookie=7626101736034216552 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a6c009950 tx=0x7f8a6c02f190 comp rx=0 tx=0).stop 2026-02-25T11:50:48.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.005+0000 7f8a81254640 1 -- 10.20.193.28:0/1539070988 shutdown_connections 2026-02-25T11:50:48.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.005+0000 7f8a81254640 1 --2- 10.20.193.28:0/1539070988 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8a7c1135c0 0x7f8a7c173f90 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.005+0000 7f8a81254640 1 --2- 10.20.193.28:0/1539070988 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a7c060b00 0x7f8a7c112e10 unknown :-1 s=CLOSED pgs=514 gs=2 cs=0 l=1 c_cookie=7626101736034216552 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.005+0000 7f8a81254640 1 --2- 10.20.193.28:0/1539070988 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8a7c060110 0x7f8a7c0604e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.005+0000 7f8a81254640 1 -- 10.20.193.28:0/1539070988 >> 10.20.193.28:0/1539070988 conn(0x7f8a7c0e9b40 msgr2=0x7f8a7c109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:48.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.005+0000 7f8a81254640 1 -- 10.20.193.28:0/1539070988 shutdown_connections 2026-02-25T11:50:48.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.005+0000 7f8a81254640 1 -- 10.20.193.28:0/1539070988 wait complete. 2026-02-25T11:50:48.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.006+0000 7f8a81254640 1 Processor -- start 2026-02-25T11:50:48.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.006+0000 7f8a81254640 1 -- start start 2026-02-25T11:50:48.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.006+0000 7f8a81254640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8a7c060110 0x7f8a7c0f4950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.006+0000 7f8a81254640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a7c060b00 0x7f8a7c173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.006+0000 7f8a81254640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8a7c1135c0 0x7f8a7c0f5050 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.006+0000 7f8a81254640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c0fa260 con 0x7f8a7c060b00 2026-02-25T11:50:48.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.006+0000 7f8a81254640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c0fa3d0 con 0x7f8a7c1135c0 2026-02-25T11:50:48.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.006+0000 7f8a81254640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c0fa590 con 0x7f8a7c060110 2026-02-25T11:50:48.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a7bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8a7c060110 0x7f8a7c0f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a7bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8a7c060110 0x7f8a7c0f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:41416/0 (socket says 10.20.193.28:41416) 2026-02-25T11:50:48.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a7bfff640 1 -- 10.20.193.28:0/981635348 learned_addr learned my addr 10.20.193.28:0/981635348 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:48.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a7c060b00 0x7f8a7c173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a80a53640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8a7c1135c0 0x7f8a7c0f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a7bfff640 1 -- 10.20.193.28:0/981635348 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8a7c1135c0 msgr2=0x7f8a7c0f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a7bfff640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8a7c1135c0 0x7f8a7c0f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a7bfff640 1 -- 10.20.193.28:0/981635348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a7c060b00 msgr2=0x7f8a7c173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a7bfff640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a7c060b00 0x7f8a7c173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a7bfff640 1 -- 10.20.193.28:0/981635348 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7000a590 con 0x7f8a7c060110 2026-02-25T11:50:48.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a7c060b00 0x7f8a7c173e90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:48.009 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a80a53640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8a7c1135c0 0x7f8a7c0f5050 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:48.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a7bfff640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8a7c060110 0x7f8a7c0f4950 secure :-1 s=READY pgs=144 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a68009480 tx=0x7f8a6800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:48.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a797fa640 1 -- 10.20.193.28:0/981635348 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a6800cd40 con 0x7f8a7c060110 2026-02-25T11:50:48.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a797fa640 1 -- 10.20.193.28:0/981635348 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8a68004d10 con 0x7f8a7c060110 2026-02-25T11:50:48.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.007+0000 7f8a81254640 1 -- 10.20.193.28:0/981635348 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a6c009570 con 0x7f8a7c060110 2026-02-25T11:50:48.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.008+0000 7f8a81254640 1 -- 10.20.193.28:0/981635348 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a7c0fabb0 con 0x7f8a7c060110 2026-02-25T11:50:48.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.008+0000 7f8a797fa640 1 -- 10.20.193.28:0/981635348 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a680106b0 con 0x7f8a7c060110 2026-02-25T11:50:48.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.011+0000 7f8a797fa640 1 -- 10.20.193.28:0/981635348 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f8a6801f060 con 0x7f8a7c060110 2026-02-25T11:50:48.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.012+0000 7f8a797fa640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8a500b56d0 0x7f8a500d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.012+0000 7f8a797fa640 1 -- 10.20.193.28:0/981635348 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8a68014070 con 0x7f8a7c060110 2026-02-25T11:50:48.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.012+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8a500b56d0 0x7f8a500d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.013+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8a500b56d0 0x7f8a500d5b80 secure :-1 s=READY pgs=292 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a6c0023d0 tx=0x7f8a6c031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:48.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.013+0000 7f8a844d8e00 1 Processor -- start 2026-02-25T11:50:48.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.013+0000 7f8a844d8e00 1 -- start start 2026-02-25T11:50:48.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.013+0000 7f8a844d8e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aa9b30 0x55e624aa9f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.013+0000 7f8a844d8e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624aeeba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.013+0000 7f8a844d8e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aef0e0 0x55e624b0f4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.013+0000 7f8a844d8e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e6249bf8a0 con 0x55e624aa9b30 2026-02-25T11:50:48.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.013+0000 7f8a844d8e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e6249bb5d0 con 0x55e624aaa440 2026-02-25T11:50:48.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.013+0000 7f8a844d8e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e6249b5570 con 0x55e624aef0e0 2026-02-25T11:50:48.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a7bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aa9b30 0x55e624aa9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a7bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aa9b30 0x55e624aa9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57898/0 (socket says 10.20.193.28:57898) 2026-02-25T11:50:48.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a7bfff640 1 -- 10.20.193.28:0/2823856100 learned_addr learned my addr 10.20.193.28:0/2823856100 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:48.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a7bfff640 1 -- 10.20.193.28:0/2823856100 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aef0e0 msgr2=0x55e624b0f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a80a53640 1 --2- 10.20.193.28:0/2823856100 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aef0e0 0x55e624b0f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/2823856100 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624aeeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a7bfff640 1 --2- 10.20.193.28:0/2823856100 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aef0e0 0x55e624b0f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a7bfff640 1 -- 10.20.193.28:0/2823856100 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 msgr2=0x55e624aeeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a7bfff640 1 --2- 10.20.193.28:0/2823856100 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624aeeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a7bfff640 1 -- 10.20.193.28:0/2823856100 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e624960d10 con 0x55e624aa9b30 2026-02-25T11:50:48.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a80a53640 1 --2- 10.20.193.28:0/2823856100 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aef0e0 0x55e624b0f4c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:48.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/2823856100 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624aeeba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:48.016 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a7bfff640 1 --2- 10.20.193.28:0/2823856100 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aa9b30 0x55e624aa9f00 secure :-1 s=READY pgs=515 gs=1 cs=0 l=1 c_cookie=8509908519661125630 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a680b7480 tx=0x7f8a680a8fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761944b61178bffe server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:48.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a4cce3640 1 -- 10.20.193.28:0/2823856100 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a680ac940 con 0x55e624aa9b30 2026-02-25T11:50:48.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a4cce3640 1 -- 10.20.193.28:0/2823856100 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8a68016ea0 con 0x55e624aa9b30 2026-02-25T11:50:48.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.014+0000 7f8a4cce3640 1 -- 10.20.193.28:0/2823856100 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a680ae050 con 0x55e624aa9b30 2026-02-25T11:50:48.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.015+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2823856100 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aa9b30 msgr2=0x55e624aa9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.015+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/2823856100 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aa9b30 0x55e624aa9f00 secure :-1 s=READY pgs=515 gs=1 cs=0 l=1 c_cookie=8509908519661125630 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a680b7480 tx=0x7f8a680a8fe0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.015+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2823856100 shutdown_connections 2026-02-25T11:50:48.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.015+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/2823856100 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aef0e0 0x55e624b0f4c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.015+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/2823856100 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624aeeba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.015+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/2823856100 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aa9b30 0x55e624aa9f00 unknown :-1 s=CLOSED pgs=515 gs=1 cs=0 l=1 c_cookie=8509908519661125630 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.015+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2823856100 >> 10.20.193.28:0/2823856100 conn(0x55e6249c87a0 msgr2=0x55e624aa8410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:48.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.015+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2823856100 shutdown_connections 2026-02-25T11:50:48.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.015+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2823856100 wait complete. 2026-02-25T11:50:48.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.016+0000 7f8a844d8e00 1 -- start start 2026-02-25T11:50:48.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.016+0000 7f8a844d8e00 1 Processor -- start 2026-02-25T11:50:48.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a844d8e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 0x55e624ac2210 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a844d8e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624ac2750 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a844d8e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aef0e0 0x55e624ac7740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a844d8e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e624ac7c80 con 0x55e624aef0e0 2026-02-25T11:50:48.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a844d8e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e624ac7df0 con 0x55e624aaa440 2026-02-25T11:50:48.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a844d8e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e624ac7fb0 con 0x55e624aa9b30 2026-02-25T11:50:48.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a7bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 0x55e624ac2210 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a7b7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624ac2750 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a7bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 0x55e624ac2210 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:41434/0 (socket says 10.20.193.28:41434) 2026-02-25T11:50:48.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a7bfff640 1 -- 10.20.193.28:0/4163606014 learned_addr learned my addr 10.20.193.28:0/4163606014 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:48.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a80a53640 1 --2- 10.20.193.28:0/4163606014 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aef0e0 0x55e624ac7740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a7bfff640 1 -- 10.20.193.28:0/4163606014 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 msgr2=0x55e624ac2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a7bfff640 1 --2- 10.20.193.28:0/4163606014 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624ac2750 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a7bfff640 1 -- 10.20.193.28:0/4163606014 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aef0e0 msgr2=0x55e624ac7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a7bfff640 1 --2- 10.20.193.28:0/4163606014 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aef0e0 0x55e624ac7740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a7bfff640 1 -- 10.20.193.28:0/4163606014 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6249393c0 con 0x55e624aa9b30 2026-02-25T11:50:48.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.017+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/4163606014 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624ac2750 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:48.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.018+0000 7f8a7bfff640 1 --2- 10.20.193.28:0/4163606014 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 0x55e624ac2210 secure :-1 s=READY pgs=145 gs=2 cs=0 l=1 c_cookie=17037582689943398539 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a680aa3b0 tx=0x7f8a680b8280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec71a730016ba88b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:48.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.018+0000 7f8a3dffb640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:50:48.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.018+0000 7f8a3dffb640 1 -- 10.20.193.28:0/4163606014 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a680ac940 con 0x55e624aa9b30 2026-02-25T11:50:48.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.018+0000 7f8a3dffb640 1 -- 10.20.193.28:0/4163606014 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8a680b2e60 con 0x55e624aa9b30 2026-02-25T11:50:48.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.018+0000 7f8a3dffb640 1 -- 10.20.193.28:0/4163606014 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a68072480 con 0x55e624aa9b30 2026-02-25T11:50:48.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.018+0000 7f8a844d8e00 1 -- 10.20.193.28:0/4163606014 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a7c0fabb0 con 0x55e624aa9b30 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.018+0000 7f8a844d8e00 10 client.25819 _ll_register_callbacks cb 0x55e624936a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.018+0000 7f8a844d8e00 1 -- 10.20.193.28:0/4163606014 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e62464b6f0 con 0x55e624aa9b30 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.018+0000 7f8a3dffb640 1 -- 10.20.193.28:0/4163606014 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8a680ab120 con 0x55e624aa9b30 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.018+0000 7f8a3dffb640 1 -- 10.20.193.28:0/4163606014 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8a680ab320 con 0x55e624aa9b30 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.018+0000 7f8a844d8e00 10 client.25819 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.018+0000 7f8a844d8e00 1 -- 10.20.193.28:0/4163606014 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55e62494b0e0 con 0x55e624aa9b30 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.019+0000 7f8a3dffb640 1 -- 10.20.193.28:0/4163606014 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f8a680aeb30 con 0x55e624aa9b30 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.019+0000 7f8a844d8e00 10 client.25819 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.019+0000 7f8a844d8e00 10 client.25819 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.019+0000 7f8a844d8e00 20 client.25819 populate_metadata read hostname 'trial028' 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.019+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/4163606014 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55e624aec580 0x55e624aec9f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.019+0000 7f8a844d8e00 4 client.25819 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.019+0000 7f8a844d8e00 1 -- 10.20.193.28:0/4163606014 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e624987fc0 con 0x55e624aec580 2026-02-25T11:50:48.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.019+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/4163606014 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55e624aec580 0x55e624aec9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.019+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/4163606014 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55e624aec580 0x55e624aec9f0 crc :-1 s=READY pgs=34 gs=4 cs=0 l=0 c_cookie=4925524141996109006 s_cookie=3052369124480099640 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=445afa1c5049ecce server_cookie=2a5c318bec01a538 in_seq=0 out_seq=0 2026-02-25T11:50:48.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.019+0000 7f8a3dffb640 10 client.25819 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:50:48.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.020+0000 7f8a3dffb640 1 -- 10.20.193.28:0/4163606014 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55e624987fc0 con 0x55e624aec580 2026-02-25T11:50:48.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.020+0000 7f8a3dffb640 10 client.25819 handle_command_reply: tid=0 2026-02-25T11:50:48.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.020+0000 7f8a844d8e00 1 client.25819 shutdown 2026-02-25T11:50:48.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.020+0000 7f8a844d8e00 1 -- 10.20.193.28:0/4163606014 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 msgr2=0x55e624ac2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.020+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/4163606014 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 0x55e624ac2210 secure :-1 s=READY pgs=145 gs=2 cs=0 l=1 c_cookie=17037582689943398539 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a680aa3b0 tx=0x7f8a680b8280 comp rx=0 tx=0).stop 2026-02-25T11:50:48.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.020+0000 7f8a844d8e00 1 -- 10.20.193.28:0/4163606014 shutdown_connections 2026-02-25T11:50:48.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.020+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/4163606014 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55e624aec580 0x55e624aec9f0 crc :-1 s=READY pgs=34 gs=4 cs=0 l=0 c_cookie=4925524141996109006 s_cookie=3052369124480099640 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.020+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/4163606014 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aef0e0 0x55e624ac7740 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.020+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/4163606014 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624ac2750 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.020+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/4163606014 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 0x55e624ac2210 unknown :-1 s=CLOSED pgs=145 gs=2 cs=0 l=1 c_cookie=17037582689943398539 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.020+0000 7f8a844d8e00 1 -- 10.20.193.28:0/4163606014 >> 10.20.193.28:0/4163606014 conn(0x55e6249c87a0 msgr2=0x55e624aa8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:48.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.021+0000 7f8a3dffb640 0 client.25819 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:50:48.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.021+0000 7f8a844d8e00 1 -- 10.20.193.28:0/4163606014 shutdown_connections 2026-02-25T11:50:48.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.021+0000 7f8a844d8e00 1 -- 10.20.193.28:0/4163606014 wait complete. 2026-02-25T11:50:48.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.021+0000 7f8a844d8e00 20 client.25819 trim_cache size 0 max 16384 2026-02-25T11:50:48.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a844d8e00 1 Processor -- start 2026-02-25T11:50:48.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a844d8e00 1 -- start start 2026-02-25T11:50:48.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a844d8e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 0x55e624aa8d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a844d8e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624aa9260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a844d8e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aef0e0 0x55e624ac5470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a844d8e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e6249b31a0 con 0x55e624aef0e0 2026-02-25T11:50:48.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a844d8e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e624673700 con 0x55e624aaa440 2026-02-25T11:50:48.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a844d8e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e624abb230 con 0x55e624aa9b30 2026-02-25T11:50:48.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a7bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 0x55e624aa8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a7bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 0x55e624aa8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:41436/0 (socket says 10.20.193.28:41436) 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a7bfff640 1 -- 10.20.193.28:0/3606111960 learned_addr learned my addr 10.20.193.28:0/3606111960 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a7b7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624aa9260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a80a53640 1 --2- 10.20.193.28:0/3606111960 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aef0e0 0x55e624ac5470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.029+0000 7f8a7b7fe640 1 -- 10.20.193.28:0/3606111960 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 msgr2=0x55e624aa8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/3606111960 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 0x55e624aa8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a7b7fe640 1 -- 10.20.193.28:0/3606111960 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aef0e0 msgr2=0x55e624ac5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/3606111960 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aef0e0 0x55e624ac5470 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a7b7fe640 1 -- 10.20.193.28:0/3606111960 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e624945b60 con 0x55e624aaa440 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a80a53640 1 --2- 10.20.193.28:0/3606111960 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aef0e0 0x55e624ac5470 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a7bfff640 1 --2- 10.20.193.28:0/3606111960 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 0x55e624aa8d20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/3606111960 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624aa9260 secure :-1 s=READY pgs=125 gs=2 cs=0 l=1 c_cookie=2909851380050752439 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a7c0fb2f0 tx=0x7f8a6c03b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2861de69c11af3b7 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a3ffff640 1 -- 10.20.193.28:0/3606111960 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a6c055170 con 0x55e624aaa440 2026-02-25T11:50:48.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a3ffff640 1 -- 10.20.193.28:0/3606111960 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8a6c04a040 con 0x55e624aaa440 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a3ffff640 1 -- 10.20.193.28:0/3606111960 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a6c055880 con 0x55e624aaa440 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a844d8e00 1 -- 10.20.193.28:0/3606111960 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 msgr2=0x55e624aa9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/3606111960 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624aa9260 secure :-1 s=READY pgs=125 gs=2 cs=0 l=1 c_cookie=2909851380050752439 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a7c0fb2f0 tx=0x7f8a6c03b000 comp rx=0 tx=0).stop 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a844d8e00 1 -- 10.20.193.28:0/3606111960 shutdown_connections 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/3606111960 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aef0e0 0x55e624ac5470 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/3606111960 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aaa440 0x55e624aa9260 unknown :-1 s=CLOSED pgs=125 gs=2 cs=0 l=1 c_cookie=2909851380050752439 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/3606111960 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aa9b30 0x55e624aa8d20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a844d8e00 1 -- 10.20.193.28:0/3606111960 >> 10.20.193.28:0/3606111960 conn(0x55e6249c87a0 msgr2=0x55e624abf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.030+0000 7f8a844d8e00 1 -- 10.20.193.28:0/3606111960 shutdown_connections 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.031+0000 7f8a844d8e00 1 -- 10.20.193.28:0/3606111960 wait complete. 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.031+0000 7f8a844d8e00 1 -- start start 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.031+0000 7f8a844d8e00 1 Processor -- start 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.031+0000 7f8a844d8e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aa9b30 0x55e624acb2a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.031+0000 7f8a844d8e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aaa440 0x55e624acb7e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.031+0000 7f8a844d8e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aef0e0 0x55e624b335f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.031+0000 7f8a844d8e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e624ae8fd0 con 0x55e624aa9b30 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.031+0000 7f8a844d8e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e624b33b90 con 0x55e624aef0e0 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.031+0000 7f8a844d8e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e624b33d50 con 0x55e624aaa440 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a7bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aa9b30 0x55e624acb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a7b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aaa440 0x55e624acb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a80a53640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aef0e0 0x55e624b335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a7b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aaa440 0x55e624acb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:41450/0 (socket says 10.20.193.28:41450) 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a7b7fe640 1 -- 10.20.193.28:0/2448876125 learned_addr learned my addr 10.20.193.28:0/2448876125 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a7bfff640 1 -- 10.20.193.28:0/2448876125 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aaa440 msgr2=0x55e624acb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a7bfff640 1 --2- 10.20.193.28:0/2448876125 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aaa440 0x55e624acb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a7bfff640 1 -- 10.20.193.28:0/2448876125 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aef0e0 msgr2=0x55e624b335f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a7bfff640 1 --2- 10.20.193.28:0/2448876125 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aef0e0 0x55e624b335f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a7bfff640 1 -- 10.20.193.28:0/2448876125 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a700053f0 con 0x55e624aa9b30 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a7bfff640 1 --2- 10.20.193.28:0/2448876125 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aa9b30 0x55e624acb2a0 secure :-1 s=READY pgs=516 gs=1 cs=0 l=1 c_cookie=1213624348544787601 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a680e4910 tx=0x7f8a680b81b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d7a8ea39c6e491 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:48.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a3dffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a3dffb640 1 -- 10.20.193.28:0/2448876125 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a680d24b0 con 0x55e624aa9b30 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a3dffb640 1 -- 10.20.193.28:0/2448876125 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55e62464b6f0 con 0x55e624aa9b30 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a3dffb640 1 -- 10.20.193.28:0/2448876125 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a68072480 con 0x55e624aa9b30 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.032+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2448876125 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a7c0fabb0 con 0x55e624aa9b30 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a3dffb640 1 -- 10.20.193.28:0/2448876125 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8a680addd0 con 0x55e624aa9b30 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a844d8e00 10 client.16050 _ll_register_callbacks cb 0x55e624936a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2448876125 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a68016ea0 con 0x55e624aa9b30 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a3dffb640 1 -- 10.20.193.28:0/2448876125 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8a680adb40 con 0x55e624aa9b30 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a844d8e00 10 client.16050 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2448876125 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x7f8a7c174800 con 0x55e624aa9b30 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a3dffb640 1 -- 10.20.193.28:0/2448876125 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f8a680ab120 con 0x55e624aa9b30 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a844d8e00 10 client.16050 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a844d8e00 10 client.16050 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a844d8e00 20 client.16050 populate_metadata read hostname 'trial028' 2026-02-25T11:50:48.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/2448876125 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55e624b9bb60 0x55e624bbbf40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a844d8e00 4 client.16050 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:50:48.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2448876125 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e624648f40 con 0x55e624b9bb60 2026-02-25T11:50:48.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.033+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/2448876125 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55e624b9bb60 0x55e624bbbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a7b7fe640 1 --2- 10.20.193.28:0/2448876125 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55e624b9bb60 0x55e624bbbf40 crc :-1 s=READY pgs=35 gs=4 cs=0 l=0 c_cookie=14497938708995444360 s_cookie=377645621259809073 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9330671450f0e88 server_cookie=53daab1122c3131 in_seq=0 out_seq=0 2026-02-25T11:50:48.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a3dffb640 10 client.16050 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:50:48.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a3dffb640 1 -- 10.20.193.28:0/2448876125 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e624648f40 con 0x55e624b9bb60 2026-02-25T11:50:48.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a3dffb640 10 client.16050 handle_command_reply: tid=0 2026-02-25T11:50:48.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a844d8e00 1 client.16050 shutdown 2026-02-25T11:50:48.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2448876125 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aa9b30 msgr2=0x55e624acb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/2448876125 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aa9b30 0x55e624acb2a0 secure :-1 s=READY pgs=516 gs=1 cs=0 l=1 c_cookie=1213624348544787601 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a680e4910 tx=0x7f8a680b81b0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2448876125 shutdown_connections 2026-02-25T11:50:48.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/2448876125 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55e624b9bb60 0x55e624bbbf40 crc :-1 s=READY pgs=35 gs=4 cs=0 l=0 c_cookie=14497938708995444360 s_cookie=377645621259809073 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/2448876125 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e624aef0e0 0x55e624b335f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/2448876125 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e624aaa440 0x55e624acb7e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a844d8e00 1 --2- 10.20.193.28:0/2448876125 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e624aa9b30 0x55e624acb2a0 unknown :-1 s=CLOSED pgs=516 gs=1 cs=0 l=1 c_cookie=1213624348544787601 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2448876125 >> 10.20.193.28:0/2448876125 conn(0x55e6249c87a0 msgr2=0x55e624ac03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:48.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a3dffb640 0 client.16050 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:50:48.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2448876125 shutdown_connections 2026-02-25T11:50:48.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.034+0000 7f8a844d8e00 1 -- 10.20.193.28:0/2448876125 wait complete. 2026-02-25T11:50:48.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.035+0000 7f8a844d8e00 20 client.16050 trim_cache size 0 max 16384 2026-02-25T11:50:48.037 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:50:48.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.035+0000 7f8a81254640 1 -- 10.20.193.28:0/981635348 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8a500b56d0 msgr2=0x7f8a500d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.035+0000 7f8a81254640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8a500b56d0 0x7f8a500d5b80 secure :-1 s=READY pgs=292 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a6c0023d0 tx=0x7f8a6c031040 comp rx=0 tx=0).stop 2026-02-25T11:50:48.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.035+0000 7f8a81254640 1 -- 10.20.193.28:0/981635348 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8a7c060110 msgr2=0x7f8a7c0f4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.035+0000 7f8a81254640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8a7c060110 0x7f8a7c0f4950 secure :-1 s=READY pgs=144 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a68009480 tx=0x7f8a6800ee60 comp rx=0 tx=0).stop 2026-02-25T11:50:48.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.035+0000 7f8a81254640 1 -- 10.20.193.28:0/981635348 shutdown_connections 2026-02-25T11:50:48.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.035+0000 7f8a81254640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8a500b56d0 0x7f8a500d5b80 unknown :-1 s=CLOSED pgs=292 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.035+0000 7f8a81254640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8a7c1135c0 0x7f8a7c0f5050 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.035+0000 7f8a81254640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a7c060b00 0x7f8a7c173e90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.035+0000 7f8a81254640 1 --2- 10.20.193.28:0/981635348 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8a7c060110 0x7f8a7c0f4950 unknown :-1 s=CLOSED pgs=144 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.035+0000 7f8a81254640 1 -- 10.20.193.28:0/981635348 >> 10.20.193.28:0/981635348 conn(0x7f8a7c0e9b40 msgr2=0x7f8a7c10fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:48.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.035+0000 7f8a81254640 1 -- 10.20.193.28:0/981635348 shutdown_connections 2026-02-25T11:50:48.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.035+0000 7f8a81254640 1 -- 10.20.193.28:0/981635348 wait complete. 2026-02-25T11:50:48.133 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:50:48.317 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:48.386 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:48 trial028 ceph-mon[39224]: pgmap v303: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:50:48.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.493+0000 7f359a1ee640 1 -- 10.20.193.28:0/1639074326 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3594115930 msgr2=0x7f3594176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.493+0000 7f359a1ee640 1 --2- 10.20.193.28:0/1639074326 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3594115930 0x7f3594176300 secure :-1 s=READY pgs=517 gs=1 cs=0 l=1 c_cookie=8457704815180931028 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f358800aa30 tx=0x7f3588030270 comp rx=0 tx=0).stop 2026-02-25T11:50:48.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.493+0000 7f359a1ee640 1 -- 10.20.193.28:0/1639074326 shutdown_connections 2026-02-25T11:50:48.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.493+0000 7f359a1ee640 1 --2- 10.20.193.28:0/1639074326 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3594115930 0x7f3594176300 unknown :-1 s=CLOSED pgs=517 gs=1 cs=0 l=1 c_cookie=8457704815180931028 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.493+0000 7f359a1ee640 1 --2- 10.20.193.28:0/1639074326 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3594110130 0x7f3594115180 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.493+0000 7f359a1ee640 1 --2- 10.20.193.28:0/1639074326 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f359410f740 0x7f359410fb10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.493+0000 7f359a1ee640 1 -- 10.20.193.28:0/1639074326 >> 10.20.193.28:0/1639074326 conn(0x7f35940ecff0 msgr2=0x7f359410d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:48.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.493+0000 7f359a1ee640 1 -- 10.20.193.28:0/1639074326 shutdown_connections 2026-02-25T11:50:48.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.493+0000 7f359a1ee640 1 -- 10.20.193.28:0/1639074326 wait complete. 2026-02-25T11:50:48.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.493+0000 7f359a1ee640 1 Processor -- start 2026-02-25T11:50:48.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f359a1ee640 1 -- start start 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f359a1ee640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f359410f740 0x7f35940f5a30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f359a1ee640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3594110130 0x7f3594176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f359a1ee640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3594115930 0x7f35940f60c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f359a1ee640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f35940fb340 con 0x7f359410f740 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f359a1ee640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f35940fb4b0 con 0x7f3594110130 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f359a1ee640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35940fb620 con 0x7f3594115930 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f35937fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f359410f740 0x7f35940f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f35937fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f359410f740 0x7f35940f5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57962/0 (socket says 10.20.193.28:57962) 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f35937fe640 1 -- 10.20.193.28:0/1332218148 learned_addr learned my addr 10.20.193.28:0/1332218148 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f3592ffd640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3594110130 0x7f3594176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f3593fff640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3594115930 0x7f35940f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f35937fe640 1 -- 10.20.193.28:0/1332218148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3594115930 msgr2=0x7f35940f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f35937fe640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3594115930 0x7f35940f60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f35937fe640 1 -- 10.20.193.28:0/1332218148 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3594110130 msgr2=0x7f3594176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f35937fe640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3594110130 0x7f3594176200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f35937fe640 1 -- 10.20.193.28:0/1332218148 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358800a650 con 0x7f359410f740 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f3593fff640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3594115930 0x7f35940f60c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:48.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.494+0000 7f35937fe640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f359410f740 0x7f35940f5a30 secure :-1 s=READY pgs=518 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f358000d950 tx=0x7f358000de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:48.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.495+0000 7f3590ff9640 1 -- 10.20.193.28:0/1332218148 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3580004540 con 0x7f359410f740 2026-02-25T11:50:48.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.495+0000 7f3590ff9640 1 -- 10.20.193.28:0/1332218148 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3580009ce0 con 0x7f359410f740 2026-02-25T11:50:48.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.495+0000 7f3590ff9640 1 -- 10.20.193.28:0/1332218148 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f35800106e0 con 0x7f359410f740 2026-02-25T11:50:48.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.495+0000 7f359a1ee640 1 -- 10.20.193.28:0/1332218148 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35940fb900 con 0x7f359410f740 2026-02-25T11:50:48.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.495+0000 7f359a1ee640 1 -- 10.20.193.28:0/1332218148 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35940fbd80 con 0x7f359410f740 2026-02-25T11:50:48.500 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.498+0000 7f3590ff9640 1 -- 10.20.193.28:0/1332218148 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f35800097c0 con 0x7f359410f740 2026-02-25T11:50:48.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.500+0000 7f3590ff9640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f35680b5760 0x7f35680d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.500+0000 7f3590ff9640 1 -- 10.20.193.28:0/1332218148 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f35800e6db0 con 0x7f359410f740 2026-02-25T11:50:48.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.500+0000 7f3592ffd640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f35680b5760 0x7f35680d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.500+0000 7f359c1e9e00 1 Processor -- start 2026-02-25T11:50:48.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.500+0000 7f359c1e9e00 1 -- start start 2026-02-25T11:50:48.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.500+0000 7f3592ffd640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f35680b5760 0x7f35680d5c10 secure :-1 s=READY pgs=293 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3584005970 tx=0x7f3584005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:48.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.500+0000 7f359c1e9e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 0x55aa06f69910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f359c1e9e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06fae5b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f359c1e9e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06faeaf0 0x55aa06fceed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f359c1e9e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55aa06e768a0 con 0x55aa06f69540 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f359c1e9e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55aa06e725d0 con 0x55aa06faeaf0 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f359c1e9e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55aa06e6c570 con 0x55aa06f69e50 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f35937fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 0x55aa06f69910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f35937fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 0x55aa06f69910 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57976/0 (socket says 10.20.193.28:57976) 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f35937fe640 1 -- 10.20.193.28:0/3217391247 learned_addr learned my addr 10.20.193.28:0/3217391247 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f3592ffd640 1 --2- 10.20.193.28:0/3217391247 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06fae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f3593fff640 1 --2- 10.20.193.28:0/3217391247 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06faeaf0 0x55aa06fceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f35937fe640 1 -- 10.20.193.28:0/3217391247 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 msgr2=0x55aa06fae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f35937fe640 1 --2- 10.20.193.28:0/3217391247 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06fae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f35937fe640 1 -- 10.20.193.28:0/3217391247 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06faeaf0 msgr2=0x55aa06fceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f35937fe640 1 --2- 10.20.193.28:0/3217391247 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06faeaf0 0x55aa06fceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f35937fe640 1 -- 10.20.193.28:0/3217391247 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa06e17d10 con 0x55aa06f69540 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f35937fe640 1 --2- 10.20.193.28:0/3217391247 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 0x55aa06f69910 secure :-1 s=READY pgs=519 gs=1 cs=0 l=1 c_cookie=3509270391020665481 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f35800a9700 tx=0x7f35800a8fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b36ed3e5953a89 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f3557fff640 1 -- 10.20.193.28:0/3217391247 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3580009e50 con 0x55aa06f69540 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f3557fff640 1 -- 10.20.193.28:0/3217391247 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f35800adb30 con 0x55aa06f69540 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.501+0000 7f3557fff640 1 -- 10.20.193.28:0/3217391247 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f35800a93a0 con 0x55aa06f69540 2026-02-25T11:50:48.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.502+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3217391247 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 msgr2=0x55aa06f69910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.502+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3217391247 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 0x55aa06f69910 secure :-1 s=READY pgs=519 gs=1 cs=0 l=1 c_cookie=3509270391020665481 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f35800a9700 tx=0x7f35800a8fd0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.502+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3217391247 shutdown_connections 2026-02-25T11:50:48.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.502+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3217391247 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06faeaf0 0x55aa06fceed0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.502+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3217391247 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06fae5b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.502+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3217391247 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 0x55aa06f69910 unknown :-1 s=CLOSED pgs=519 gs=1 cs=0 l=1 c_cookie=3509270391020665481 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.502+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3217391247 >> 10.20.193.28:0/3217391247 conn(0x55aa06e7f7a0 msgr2=0x55aa06f67e20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:48.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.502+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3217391247 shutdown_connections 2026-02-25T11:50:48.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.502+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3217391247 wait complete. 2026-02-25T11:50:48.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.503+0000 7f359c1e9e00 1 -- start start 2026-02-25T11:50:48.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.503+0000 7f359c1e9e00 1 Processor -- start 2026-02-25T11:50:48.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f359c1e9e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69540 0x55aa06f818b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f359c1e9e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06f69e50 0x55aa06f81df0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f359c1e9e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06faeaf0 0x55aa06f86de0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f359c1e9e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55aa06f87320 con 0x55aa06faeaf0 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f359c1e9e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55aa06f87490 con 0x55aa06f69e50 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f359c1e9e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55aa06f87650 con 0x55aa06f69540 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f3593fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06faeaf0 0x55aa06f86de0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f35937fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69540 0x55aa06f818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f35937fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69540 0x55aa06f818b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:41490/0 (socket says 10.20.193.28:41490) 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f35937fe640 1 -- 10.20.193.28:0/3280662405 learned_addr learned my addr 10.20.193.28:0/3280662405 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f3592ffd640 1 --2- 10.20.193.28:0/3280662405 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06f69e50 0x55aa06f81df0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f3593fff640 1 -- 10.20.193.28:0/3280662405 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69540 msgr2=0x55aa06f818b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f3593fff640 1 --2- 10.20.193.28:0/3280662405 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69540 0x55aa06f818b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f3593fff640 1 -- 10.20.193.28:0/3280662405 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06f69e50 msgr2=0x55aa06f81df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f3593fff640 1 --2- 10.20.193.28:0/3280662405 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06f69e50 0x55aa06f81df0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f3593fff640 1 -- 10.20.193.28:0/3280662405 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa06df03c0 con 0x55aa06faeaf0 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.504+0000 7f35937fe640 1 --2- 10.20.193.28:0/3280662405 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69540 0x55aa06f818b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.505+0000 7f3593fff640 1 --2- 10.20.193.28:0/3280662405 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06faeaf0 0x55aa06f86de0 secure :-1 s=READY pgs=520 gs=3 cs=0 l=1 c_cookie=5894246892468149301 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x55aa06ef20f0 tx=0x7f3588030d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cc9251d4039c35 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:48.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.505+0000 7f35557fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:48.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.505+0000 7f35557fa640 1 -- 10.20.193.28:0/3280662405 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f35880039a0 con 0x55aa06faeaf0 2026-02-25T11:50:48.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.505+0000 7f35557fa640 1 -- 10.20.193.28:0/3280662405 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3588003b00 con 0x55aa06faeaf0 2026-02-25T11:50:48.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.505+0000 7f35557fa640 1 -- 10.20.193.28:0/3280662405 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3588044680 con 0x55aa06faeaf0 2026-02-25T11:50:48.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.505+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3280662405 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35940fbd80 con 0x55aa06faeaf0 2026-02-25T11:50:48.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.505+0000 7f35557fa640 1 -- 10.20.193.28:0/3280662405 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f358804d3b0 con 0x55aa06faeaf0 2026-02-25T11:50:48.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.505+0000 7f359c1e9e00 10 client.16062 _ll_register_callbacks cb 0x55aa06deda10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:48.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.505+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3280662405 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aa06b026f0 con 0x55aa06faeaf0 2026-02-25T11:50:48.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.505+0000 7f35557fa640 1 -- 10.20.193.28:0/3280662405 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3588005660 con 0x55aa06faeaf0 2026-02-25T11:50:48.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.505+0000 7f359c1e9e00 10 client.16062 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:48.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.505+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3280662405 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55aa06e020e0 con 0x55aa06faeaf0 2026-02-25T11:50:48.507 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.506+0000 7f35557fa640 1 -- 10.20.193.28:0/3280662405 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f35880050c0 con 0x55aa06faeaf0 2026-02-25T11:50:48.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.506+0000 7f359c1e9e00 10 client.16062 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:48.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.506+0000 7f359c1e9e00 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:48.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.506+0000 7f359c1e9e00 20 client.16062 populate_metadata read hostname 'trial028' 2026-02-25T11:50:48.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.506+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3280662405 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aa06fac080 0x55aa06fac520 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.506+0000 7f359c1e9e00 4 client.16062 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:50:48.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.506+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3280662405 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa06e3efc0 con 0x55aa06fac080 2026-02-25T11:50:48.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.506+0000 7f35937fe640 1 --2- 10.20.193.28:0/3280662405 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aa06fac080 0x55aa06fac520 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.507+0000 7f35937fe640 1 --2- 10.20.193.28:0/3280662405 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aa06fac080 0x55aa06fac520 crc :-1 s=READY pgs=68 gs=4 cs=0 l=0 c_cookie=3784756093541795124 s_cookie=16025227703874775238 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3486279d550dc534 server_cookie=de650bb0fe3468c6 in_seq=0 out_seq=0 2026-02-25T11:50:48.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.507+0000 7f35557fa640 10 client.16062 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:48.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.507+0000 7f35557fa640 1 -- 10.20.193.28:0/3280662405 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55aa06e3efc0 con 0x55aa06fac080 2026-02-25T11:50:48.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.507+0000 7f35557fa640 10 client.16062 handle_command_reply: tid=0 2026-02-25T11:50:48.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.507+0000 7f359c1e9e00 1 client.16062 shutdown 2026-02-25T11:50:48.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.507+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3280662405 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06faeaf0 msgr2=0x55aa06f86de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.507+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3280662405 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06faeaf0 0x55aa06f86de0 secure :-1 s=READY pgs=520 gs=3 cs=0 l=1 c_cookie=5894246892468149301 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x55aa06ef20f0 tx=0x7f3588030d70 comp rx=0 tx=0).stop 2026-02-25T11:50:48.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.508+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3280662405 shutdown_connections 2026-02-25T11:50:48.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.508+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3280662405 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aa06fac080 0x55aa06fac520 crc :-1 s=READY pgs=68 gs=4 cs=0 l=0 c_cookie=3784756093541795124 s_cookie=16025227703874775238 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.508+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3280662405 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06faeaf0 0x55aa06f86de0 unknown :-1 s=CLOSED pgs=520 gs=3 cs=0 l=1 c_cookie=5894246892468149301 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.508+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3280662405 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06f69e50 0x55aa06f81df0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.509 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.508+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3280662405 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69540 0x55aa06f818b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.508+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3280662405 >> 10.20.193.28:0/3280662405 conn(0x55aa06e7f7a0 msgr2=0x55aa06f68610 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:48.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.508+0000 7f35557fa640 0 client.16062 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:48.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.508+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3280662405 shutdown_connections 2026-02-25T11:50:48.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.508+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3280662405 wait complete. 2026-02-25T11:50:48.510 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.508+0000 7f359c1e9e00 20 client.16062 trim_cache size 0 max 16384 2026-02-25T11:50:48.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.515+0000 7f359c1e9e00 1 Processor -- start 2026-02-25T11:50:48.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.515+0000 7f359c1e9e00 1 -- start start 2026-02-25T11:50:48.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.515+0000 7f359c1e9e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 0x55aa06f739e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.515+0000 7f359c1e9e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06f7eca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.515+0000 7f359c1e9e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06faeaf0 0x55aa06f7f1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.515+0000 7f359c1e9e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55aa06e6a1a0 con 0x55aa06f69540 2026-02-25T11:50:48.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.515+0000 7f359c1e9e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55aa06b2a700 con 0x55aa06faeaf0 2026-02-25T11:50:48.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.515+0000 7f359c1e9e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55aa06ecfd60 con 0x55aa06f69e50 2026-02-25T11:50:48.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.515+0000 7f35937fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 0x55aa06f739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.515+0000 7f35937fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 0x55aa06f739e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57986/0 (socket says 10.20.193.28:57986) 2026-02-25T11:50:48.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.515+0000 7f35937fe640 1 -- 10.20.193.28:0/20456186 learned_addr learned my addr 10.20.193.28:0/20456186 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:48.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.515+0000 7f3592ffd640 1 --2- 10.20.193.28:0/20456186 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06f7eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.516+0000 7f3593fff640 1 --2- 10.20.193.28:0/20456186 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06faeaf0 0x55aa06f7f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.516+0000 7f35937fe640 1 -- 10.20.193.28:0/20456186 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 msgr2=0x55aa06f7eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.516+0000 7f35937fe640 1 --2- 10.20.193.28:0/20456186 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06f7eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.516+0000 7f35937fe640 1 -- 10.20.193.28:0/20456186 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06faeaf0 msgr2=0x55aa06f7f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.516+0000 7f35937fe640 1 --2- 10.20.193.28:0/20456186 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06faeaf0 0x55aa06f7f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.516+0000 7f35937fe640 1 -- 10.20.193.28:0/20456186 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa06f741b0 con 0x55aa06f69540 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.516+0000 7f3593fff640 1 --2- 10.20.193.28:0/20456186 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06faeaf0 0x55aa06f7f1e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.516+0000 7f35937fe640 1 --2- 10.20.193.28:0/20456186 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 0x55aa06f739e0 secure :-1 s=READY pgs=521 gs=1 cs=0 l=1 c_cookie=10791492817217420978 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3580005940 tx=0x7f35800e4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c3158a9e6c7eb2 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.516+0000 7f35577fe640 1 -- 10.20.193.28:0/20456186 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3580009e50 con 0x55aa06f69540 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.516+0000 7f35577fe640 1 -- 10.20.193.28:0/20456186 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f35800b8270 con 0x55aa06f69540 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.516+0000 7f35577fe640 1 -- 10.20.193.28:0/20456186 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f35800addc0 con 0x55aa06f69540 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.517+0000 7f359c1e9e00 1 -- 10.20.193.28:0/20456186 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 msgr2=0x55aa06f739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.517+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/20456186 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 0x55aa06f739e0 secure :-1 s=READY pgs=521 gs=1 cs=0 l=1 c_cookie=10791492817217420978 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3580005940 tx=0x7f35800e4040 comp rx=0 tx=0).stop 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.517+0000 7f359c1e9e00 1 -- 10.20.193.28:0/20456186 shutdown_connections 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.517+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/20456186 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06faeaf0 0x55aa06f7f1e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.517+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/20456186 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06f7eca0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.517+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/20456186 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06f69540 0x55aa06f739e0 unknown :-1 s=CLOSED pgs=521 gs=1 cs=0 l=1 c_cookie=10791492817217420978 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.517+0000 7f359c1e9e00 1 -- 10.20.193.28:0/20456186 >> 10.20.193.28:0/20456186 conn(0x55aa06e7f7a0 msgr2=0x55aa06f6c230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:48.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.517+0000 7f359c1e9e00 1 -- 10.20.193.28:0/20456186 shutdown_connections 2026-02-25T11:50:48.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.517+0000 7f359c1e9e00 1 -- 10.20.193.28:0/20456186 wait complete. 2026-02-25T11:50:48.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.517+0000 7f359c1e9e00 1 -- start start 2026-02-25T11:50:48.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.517+0000 7f359c1e9e00 1 Processor -- start 2026-02-25T11:50:48.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f359c1e9e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06f69540 0x55aa06f8dbd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f359c1e9e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06f891d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f359c1e9e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06faeaf0 0x55aa06f89880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f359c1e9e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55aa06f3ee40 con 0x55aa06faeaf0 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f359c1e9e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55aa06f79880 con 0x55aa06f69540 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f359c1e9e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55aa06f8a030 con 0x55aa06f69e50 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f3592ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06f891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f3593fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06faeaf0 0x55aa06f89880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f35937fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06f69540 0x55aa06f8dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f3592ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06f891d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:41520/0 (socket says 10.20.193.28:41520) 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f35937fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06f69540 0x55aa06f8dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:56676/0 (socket says 10.20.193.28:56676) 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f3592ffd640 1 -- 10.20.193.28:0/3225951839 learned_addr learned my addr 10.20.193.28:0/3225951839 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f3592ffd640 1 -- 10.20.193.28:0/3225951839 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06f69540 msgr2=0x55aa06f8dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f3592ffd640 1 --2- 10.20.193.28:0/3225951839 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06f69540 0x55aa06f8dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f3592ffd640 1 -- 10.20.193.28:0/3225951839 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06faeaf0 msgr2=0x55aa06f89880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f3592ffd640 1 --2- 10.20.193.28:0/3225951839 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06faeaf0 0x55aa06f89880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f3592ffd640 1 -- 10.20.193.28:0/3225951839 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa06fa8a00 con 0x55aa06f69e50 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.518+0000 7f3593fff640 1 --2- 10.20.193.28:0/3225951839 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06faeaf0 0x55aa06f89880 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f3592ffd640 1 --2- 10.20.193.28:0/3225951839 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06f891d0 secure :-1 s=READY pgs=146 gs=3 cs=0 l=1 c_cookie=12416953427353874843 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3584004740 tx=0x7f3584004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac51e15ab5ff319b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:48.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f35557fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f35557fa640 1 -- 10.20.193.28:0/3225951839 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f35840119b0 con 0x55aa06f69e50 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3225951839 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa06e17d10 con 0x55aa06f69e50 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f359c1e9e00 10 client.25840 _ll_register_callbacks cb 0x55aa06deda10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3225951839 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35800adb30 con 0x55aa06f69e50 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f35557fa640 1 -- 10.20.193.28:0/3225951839 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3584025040 con 0x55aa06f69e50 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f35557fa640 1 -- 10.20.193.28:0/3225951839 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f358401d640 con 0x55aa06f69e50 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f35557fa640 1 -- 10.20.193.28:0/3225951839 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f358401ee30 con 0x55aa06f69e50 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f35557fa640 1 -- 10.20.193.28:0/3225951839 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f358401d7d0 con 0x55aa06f69e50 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f359c1e9e00 10 client.25840 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3225951839 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55aa06dfcb60 con 0x55aa06f69e50 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f35557fa640 1 -- 10.20.193.28:0/3225951839 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f3584032060 con 0x55aa06f69e50 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.519+0000 7f359c1e9e00 10 client.25840 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.520+0000 7f359c1e9e00 10 client.25840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.520+0000 7f359c1e9e00 20 client.25840 populate_metadata read hostname 'trial028' 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.520+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3225951839 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aa0705c060 0x55aa0707c510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.520+0000 7f359c1e9e00 4 client.25840 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.520+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3225951839 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa06afff40 con 0x55aa0705c060 2026-02-25T11:50:48.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.520+0000 7f35937fe640 1 --2- 10.20.193.28:0/3225951839 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aa0705c060 0x55aa0707c510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:48.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.520+0000 7f35937fe640 1 --2- 10.20.193.28:0/3225951839 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aa0705c060 0x55aa0707c510 crc :-1 s=READY pgs=69 gs=4 cs=0 l=0 c_cookie=14619375708501663769 s_cookie=3362533976441167622 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cae274c007376019 server_cookie=2eaa1ed60af51f06 in_seq=0 out_seq=0 2026-02-25T11:50:48.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.520+0000 7f35557fa640 10 client.25840 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:48.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.520+0000 7f35557fa640 1 -- 10.20.193.28:0/3225951839 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa06afff40 con 0x55aa0705c060 2026-02-25T11:50:48.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.520+0000 7f35557fa640 10 client.25840 handle_command_reply: tid=0 2026-02-25T11:50:48.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359c1e9e00 1 client.25840 shutdown 2026-02-25T11:50:48.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3225951839 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 msgr2=0x55aa06f891d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3225951839 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06f891d0 secure :-1 s=READY pgs=146 gs=3 cs=0 l=1 c_cookie=12416953427353874843 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3584004740 tx=0x7f3584004770 comp rx=0 tx=0).stop 2026-02-25T11:50:48.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3225951839 shutdown_connections 2026-02-25T11:50:48.522 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3225951839 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aa0705c060 0x55aa0707c510 crc :-1 s=READY pgs=69 gs=4 cs=0 l=0 c_cookie=14619375708501663769 s_cookie=3362533976441167622 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3225951839 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aa06faeaf0 0x55aa06f89880 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f35557fa640 0 client.25840 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3225951839 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aa06f69e50 0x55aa06f891d0 unknown :-1 s=CLOSED pgs=146 gs=3 cs=0 l=1 c_cookie=12416953427353874843 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359c1e9e00 1 --2- 10.20.193.28:0/3225951839 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aa06f69540 0x55aa06f8dbd0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3225951839 >> 10.20.193.28:0/3225951839 conn(0x55aa06e7f7a0 msgr2=0x55aa06f7dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3225951839 shutdown_connections 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359c1e9e00 1 -- 10.20.193.28:0/3225951839 wait complete. 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359c1e9e00 20 client.25840 trim_cache size 0 max 16384 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359a1ee640 1 -- 10.20.193.28:0/1332218148 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f35680b5760 msgr2=0x7f35680d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359a1ee640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f35680b5760 0x7f35680d5c10 secure :-1 s=READY pgs=293 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3584005970 tx=0x7f3584005900 comp rx=0 tx=0).stop 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359a1ee640 1 -- 10.20.193.28:0/1332218148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f359410f740 msgr2=0x7f35940f5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359a1ee640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f359410f740 0x7f35940f5a30 secure :-1 s=READY pgs=518 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f358000d950 tx=0x7f358000de20 comp rx=0 tx=0).stop 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359a1ee640 1 -- 10.20.193.28:0/1332218148 shutdown_connections 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359a1ee640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f35680b5760 0x7f35680d5c10 unknown :-1 s=CLOSED pgs=293 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359a1ee640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3594115930 0x7f35940f60c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359a1ee640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3594110130 0x7f3594176200 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359a1ee640 1 --2- 10.20.193.28:0/1332218148 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f359410f740 0x7f35940f5a30 unknown :-1 s=CLOSED pgs=518 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.521+0000 7f359a1ee640 1 -- 10.20.193.28:0/1332218148 >> 10.20.193.28:0/1332218148 conn(0x7f35940ecff0 msgr2=0x7f359410dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:48.523 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.522+0000 7f359a1ee640 1 -- 10.20.193.28:0/1332218148 shutdown_connections 2026-02-25T11:50:48.524 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:48.522+0000 7f359a1ee640 1 -- 10.20.193.28:0/1332218148 wait complete. 2026-02-25T11:50:48.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:48 trial167 ceph-mon[43392]: pgmap v303: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:50:48.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:48 trial119 ceph-mon[42710]: pgmap v303: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:50:49.623 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:50:49.623 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:50:49.808 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:49.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.992+0000 7fb24e05d640 1 -- 10.20.193.28:0/3290914557 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb2481140a0 msgr2=0x7fb248114470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:49.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.992+0000 7fb24e05d640 1 --2- 10.20.193.28:0/3290914557 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb2481140a0 0x7fb248114470 secure :-1 s=READY pgs=522 gs=1 cs=0 l=1 c_cookie=10820685773019766339 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb238009950 tx=0x7fb23802f190 comp rx=0 tx=0).stop 2026-02-25T11:50:49.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.992+0000 7fb24e05d640 1 -- 10.20.193.28:0/3290914557 shutdown_connections 2026-02-25T11:50:49.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.992+0000 7fb24e05d640 1 --2- 10.20.193.28:0/3290914557 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb248119d30 0x7fb24817a700 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:49.995 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.992+0000 7fb24e05d640 1 --2- 10.20.193.28:0/3290914557 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb248114a90 0x7fb248119580 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:49.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.992+0000 7fb24e05d640 1 --2- 10.20.193.28:0/3290914557 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb2481140a0 0x7fb248114470 unknown :-1 s=CLOSED pgs=522 gs=1 cs=0 l=1 c_cookie=10820685773019766339 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:49.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.992+0000 7fb24e05d640 1 -- 10.20.193.28:0/3290914557 >> 10.20.193.28:0/3290914557 conn(0x7fb248064c40 msgr2=0x7fb248065040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:49.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.992+0000 7fb24e05d640 1 -- 10.20.193.28:0/3290914557 shutdown_connections 2026-02-25T11:50:49.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.992+0000 7fb24e05d640 1 -- 10.20.193.28:0/3290914557 wait complete. 2026-02-25T11:50:49.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.993+0000 7fb24e05d640 1 Processor -- start 2026-02-25T11:50:49.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.993+0000 7fb24e05d640 1 -- start start 2026-02-25T11:50:49.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24e05d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb2481140a0 0x7fb248122550 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:49.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24e05d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb248114a90 0x7fb248122a90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:49.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24e05d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb248119d30 0x7fb248127e60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:49.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24e05d640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb248128480 con 0x7fb248114a90 2026-02-25T11:50:49.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24e05d640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481285f0 con 0x7fb2481140a0 2026-02-25T11:50:49.996 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24e05d640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481287b0 con 0x7fb248119d30 2026-02-25T11:50:49.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24c85a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb248114a90 0x7fb248122a90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:49.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24c85a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb248114a90 0x7fb248122a90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58022/0 (socket says 10.20.193.28:58022) 2026-02-25T11:50:49.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24c85a640 1 -- 10.20.193.28:0/143816253 learned_addr learned my addr 10.20.193.28:0/143816253 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:49.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24d85c640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb248119d30 0x7fb248127e60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:49.997 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24d05b640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb2481140a0 0x7fb248122550 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:49.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24c85a640 1 -- 10.20.193.28:0/143816253 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb248119d30 msgr2=0x7fb248127e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:49.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24c85a640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb248119d30 0x7fb248127e60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:49.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24c85a640 1 -- 10.20.193.28:0/143816253 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb2481140a0 msgr2=0x7fb248122550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:49.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24c85a640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb2481140a0 0x7fb248122550 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:49.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24c85a640 1 -- 10.20.193.28:0/143816253 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb238009570 con 0x7fb248114a90 2026-02-25T11:50:49.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.994+0000 7fb24c85a640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb248114a90 0x7fb248122a90 secure :-1 s=READY pgs=523 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb23c009480 tx=0x7fb23c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:49.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.995+0000 7fb2367fc640 1 -- 10.20.193.28:0/143816253 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb23c00cd40 con 0x7fb248114a90 2026-02-25T11:50:49.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.995+0000 7fb2367fc640 1 -- 10.20.193.28:0/143816253 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb23c004d10 con 0x7fb248114a90 2026-02-25T11:50:49.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.995+0000 7fb2367fc640 1 -- 10.20.193.28:0/143816253 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb23c0106b0 con 0x7fb248114a90 2026-02-25T11:50:49.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.995+0000 7fb24e05d640 1 -- 10.20.193.28:0/143816253 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb248128a90 con 0x7fb248114a90 2026-02-25T11:50:49.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.995+0000 7fb24e05d640 1 -- 10.20.193.28:0/143816253 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb248169190 con 0x7fb248114a90 2026-02-25T11:50:49.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.998+0000 7fb2367fc640 1 -- 10.20.193.28:0/143816253 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fb23c01f060 con 0x7fb248114a90 2026-02-25T11:50:50.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:49.999+0000 7fb2367fc640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb2240b5760 0x7fb2240d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.000+0000 7fb2367fc640 1 -- 10.20.193.28:0/143816253 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb23c014070 con 0x7fb248114a90 2026-02-25T11:50:50.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.000+0000 7fb24d05b640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb2240b5760 0x7fb2240d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.000+0000 7fb2512e1e00 1 Processor -- start 2026-02-25T11:50:50.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.000+0000 7fb24d05b640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb2240b5760 0x7fb2240d5c10 secure :-1 s=READY pgs=294 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb238005b30 tx=0x7fb23803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:50.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb2512e1e00 1 -- start start 2026-02-25T11:50:50.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb2512e1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9a750 0x564837d9ab20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb2512e1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9b060 0x564837ddf610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb2512e1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837dfff30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb2512e1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564837cb0d80 con 0x564837ddfb50 2026-02-25T11:50:50.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb2512e1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564837cacab0 con 0x564837d9a750 2026-02-25T11:50:50.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb2512e1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564837ca6a50 con 0x564837d9b060 2026-02-25T11:50:50.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb24c85a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9b060 0x564837ddf610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb24d85c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837dfff30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.004 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb24c85a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9b060 0x564837ddf610 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:41534/0 (socket says 10.20.193.28:41534) 2026-02-25T11:50:50.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb24c85a640 1 -- 10.20.193.28:0/2321367828 learned_addr learned my addr 10.20.193.28:0/2321367828 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:50.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb24d05b640 1 --2- 10.20.193.28:0/2321367828 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9a750 0x564837d9ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb24c85a640 1 -- 10.20.193.28:0/2321367828 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9a750 msgr2=0x564837d9ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb24c85a640 1 --2- 10.20.193.28:0/2321367828 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9a750 0x564837d9ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.001+0000 7fb24c85a640 1 -- 10.20.193.28:0/2321367828 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 msgr2=0x564837dfff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.002+0000 7fb24c85a640 1 --2- 10.20.193.28:0/2321367828 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837dfff30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.002+0000 7fb24c85a640 1 -- 10.20.193.28:0/2321367828 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564837c52120 con 0x564837d9b060 2026-02-25T11:50:50.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.002+0000 7fb24d85c640 1 --2- 10.20.193.28:0/2321367828 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837dfff30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:50.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.002+0000 7fb24c85a640 1 --2- 10.20.193.28:0/2321367828 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9b060 0x564837ddf610 secure :-1 s=READY pgs=147 gs=2 cs=0 l=1 c_cookie=9460781035911709948 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb23c0aa240 tx=0x7fb23c004f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=834b723a0f4504fc server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:50.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.002+0000 7fb215ce5640 1 -- 10.20.193.28:0/2321367828 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb23c0ac9b0 con 0x564837d9b060 2026-02-25T11:50:50.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.002+0000 7fb215ce5640 1 -- 10.20.193.28:0/2321367828 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb23c0ac7d0 con 0x564837d9b060 2026-02-25T11:50:50.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.002+0000 7fb215ce5640 1 -- 10.20.193.28:0/2321367828 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb23c0d2350 con 0x564837d9b060 2026-02-25T11:50:50.006 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.002+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2321367828 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9b060 msgr2=0x564837ddf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.006 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.002+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2321367828 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9b060 0x564837ddf610 secure :-1 s=READY pgs=147 gs=2 cs=0 l=1 c_cookie=9460781035911709948 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb23c0aa240 tx=0x7fb23c004f00 comp rx=0 tx=0).stop 2026-02-25T11:50:50.006 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.003+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2321367828 shutdown_connections 2026-02-25T11:50:50.006 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.003+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2321367828 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837dfff30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.006 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.003+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2321367828 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9b060 0x564837ddf610 unknown :-1 s=CLOSED pgs=147 gs=2 cs=0 l=1 c_cookie=9460781035911709948 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.006 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.003+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2321367828 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9a750 0x564837d9ab20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.006 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.003+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2321367828 >> 10.20.193.28:0/2321367828 conn(0x564837d788f0 msgr2=0x564837d98cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:50.006 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.003+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2321367828 shutdown_connections 2026-02-25T11:50:50.006 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.003+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2321367828 wait complete. 2026-02-25T11:50:50.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.004+0000 7fb2512e1e00 1 -- start start 2026-02-25T11:50:50.010 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.004+0000 7fb2512e1e00 1 Processor -- start 2026-02-25T11:50:50.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.004+0000 7fb2512e1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837df74c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.004+0000 7fb2512e1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9b060 0x564837df7a00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.004+0000 7fb2512e1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837dfc9f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb24c85a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9b060 0x564837df7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb24c85a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9b060 0x564837df7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:56708/0 (socket says 10.20.193.28:56708) 2026-02-25T11:50:50.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb24d05b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837df74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb24c85a640 1 -- 10.20.193.28:0/2357752007 learned_addr learned my addr 10.20.193.28:0/2357752007 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:50.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2357752007 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564837dfcf30 con 0x564837ddfb50 2026-02-25T11:50:50.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2357752007 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564837dfd0a0 con 0x564837d9b060 2026-02-25T11:50:50.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2357752007 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564837dfd260 con 0x564837d9a750 2026-02-25T11:50:50.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb24d85c640 1 --2- 10.20.193.28:0/2357752007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837dfc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb24d05b640 1 -- 10.20.193.28:0/2357752007 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9b060 msgr2=0x564837df7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb24d05b640 1 --2- 10.20.193.28:0/2357752007 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9b060 0x564837df7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb24d05b640 1 -- 10.20.193.28:0/2357752007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 msgr2=0x564837dfc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb24d05b640 1 --2- 10.20.193.28:0/2357752007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837dfc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb24d05b640 1 -- 10.20.193.28:0/2357752007 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb248169190 con 0x564837d9a750 2026-02-25T11:50:50.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.005+0000 7fb24d85c640 1 --2- 10.20.193.28:0/2357752007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837dfc9f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:50.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.006+0000 7fb24d05b640 1 --2- 10.20.193.28:0/2357752007 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837df74c0 secure :-1 s=READY pgs=148 gs=2 cs=0 l=1 c_cookie=18233204776267127320 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb238005f70 tx=0x7fb238009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd095b23ee5b9a18 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:50.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.006+0000 7fb20affd640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:50:50.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.006+0000 7fb20affd640 1 -- 10.20.193.28:0/2357752007 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb238038b90 con 0x564837d9a750 2026-02-25T11:50:50.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.006+0000 7fb20affd640 1 -- 10.20.193.28:0/2357752007 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb23803e040 con 0x564837d9a750 2026-02-25T11:50:50.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.006+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2357752007 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb244008840 con 0x564837d9a750 2026-02-25T11:50:50.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.006+0000 7fb2512e1e00 10 client.25852 _ll_register_callbacks cb 0x564837c27e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:50.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.006+0000 7fb20affd640 1 -- 10.20.193.28:0/2357752007 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb238050670 con 0x564837d9a750 2026-02-25T11:50:50.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.006+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2357752007 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5648379fe530 con 0x564837d9a750 2026-02-25T11:50:50.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.006+0000 7fb20affd640 1 -- 10.20.193.28:0/2357752007 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb238044e30 con 0x564837d9a750 2026-02-25T11:50:50.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.006+0000 7fb20affd640 1 -- 10.20.193.28:0/2357752007 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb238050800 con 0x564837d9a750 2026-02-25T11:50:50.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.006+0000 7fb2512e1e00 10 client.25852 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:50.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.006+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2357752007 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x564837c2a810 con 0x564837d9a750 2026-02-25T11:50:50.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.007+0000 7fb20affd640 1 -- 10.20.193.28:0/2357752007 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb23804f070 con 0x564837d9a750 2026-02-25T11:50:50.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.007+0000 7fb2512e1e00 10 client.25852 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:50.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.007+0000 7fb2512e1e00 10 client.25852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:50.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.007+0000 7fb2512e1e00 20 client.25852 populate_metadata read hostname 'trial028' 2026-02-25T11:50:50.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.007+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2357752007 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564837ddceb0 0x564837ddd350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.007+0000 7fb2512e1e00 4 client.25852 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:50:50.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.007+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2357752007 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564837c53b50 con 0x564837ddceb0 2026-02-25T11:50:50.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.007+0000 7fb24c85a640 1 --2- 10.20.193.28:0/2357752007 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564837ddceb0 0x564837ddd350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.008+0000 7fb24c85a640 1 --2- 10.20.193.28:0/2357752007 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564837ddceb0 0x564837ddd350 crc :-1 s=READY pgs=70 gs=4 cs=0 l=0 c_cookie=6226833388635282305 s_cookie=11748281252683762644 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=566a27f9cd25db81 server_cookie=a30a478de398cbd4 in_seq=0 out_seq=0 2026-02-25T11:50:50.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.008+0000 7fb20affd640 10 client.25852 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:50.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.008+0000 7fb20affd640 1 -- 10.20.193.28:0/2357752007 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x564837c53b50 con 0x564837ddceb0 2026-02-25T11:50:50.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.008+0000 7fb20affd640 10 client.25852 handle_command_reply: tid=0 2026-02-25T11:50:50.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.009+0000 7fb2512e1e00 1 client.25852 shutdown 2026-02-25T11:50:50.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.009+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2357752007 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 msgr2=0x564837df74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.009+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2357752007 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837df74c0 secure :-1 s=READY pgs=148 gs=2 cs=0 l=1 c_cookie=18233204776267127320 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb238005f70 tx=0x7fb238009310 comp rx=0 tx=0).stop 2026-02-25T11:50:50.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.009+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2357752007 shutdown_connections 2026-02-25T11:50:50.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.009+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2357752007 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564837ddceb0 0x564837ddd350 crc :-1 s=READY pgs=70 gs=4 cs=0 l=0 c_cookie=6226833388635282305 s_cookie=11748281252683762644 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.009+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2357752007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837dfc9f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.009+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2357752007 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9b060 0x564837df7a00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.009+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2357752007 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837df74c0 unknown :-1 s=CLOSED pgs=148 gs=2 cs=0 l=1 c_cookie=18233204776267127320 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.009+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2357752007 >> 10.20.193.28:0/2357752007 conn(0x564837d788f0 msgr2=0x564837d98cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:50.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.009+0000 7fb20affd640 0 client.25852 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:50.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.009+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2357752007 shutdown_connections 2026-02-25T11:50:50.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.009+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2357752007 wait complete. 2026-02-25T11:50:50.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.009+0000 7fb2512e1e00 20 client.25852 trim_cache size 0 max 16384 2026-02-25T11:50:50.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.018+0000 7fb2512e1e00 1 Processor -- start 2026-02-25T11:50:50.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.018+0000 7fb2512e1e00 1 -- start start 2026-02-25T11:50:50.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.018+0000 7fb2512e1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837dfbf60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.018+0000 7fb2512e1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837d9b060 0x564837dda740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.018+0000 7fb2512e1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837ddfb50 0x564837ddac80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.018+0000 7fb2512e1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564837de9820 con 0x564837d9b060 2026-02-25T11:50:50.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.018+0000 7fb2512e1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564837ca4680 con 0x564837ddfb50 2026-02-25T11:50:50.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.018+0000 7fb2512e1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564837964790 con 0x564837d9a750 2026-02-25T11:50:50.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb24d05b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837dfbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb24d85c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837ddfb50 0x564837ddac80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb24d05b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837dfbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:41542/0 (socket says 10.20.193.28:41542) 2026-02-25T11:50:50.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb24d05b640 1 -- 10.20.193.28:0/2897316261 learned_addr learned my addr 10.20.193.28:0/2897316261 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:50.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb24c85a640 1 --2- 10.20.193.28:0/2897316261 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837d9b060 0x564837dda740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb24c85a640 1 -- 10.20.193.28:0/2897316261 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 msgr2=0x564837dfbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb24c85a640 1 --2- 10.20.193.28:0/2897316261 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837dfbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb24c85a640 1 -- 10.20.193.28:0/2897316261 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837ddfb50 msgr2=0x564837ddac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb24c85a640 1 --2- 10.20.193.28:0/2897316261 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837ddfb50 0x564837ddac80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb24c85a640 1 -- 10.20.193.28:0/2897316261 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564837de9a50 con 0x564837d9b060 2026-02-25T11:50:50.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb24d85c640 1 --2- 10.20.193.28:0/2897316261 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837ddfb50 0x564837ddac80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:50.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb24c85a640 1 --2- 10.20.193.28:0/2897316261 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837d9b060 0x564837dda740 secure :-1 s=READY pgs=524 gs=3 cs=0 l=1 c_cookie=4309855007982339321 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb23c0a95a0 tx=0x7fb23c005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcfae4b9039c4f9 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:50.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb2154e4640 1 -- 10.20.193.28:0/2897316261 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb23c0ac9b0 con 0x564837d9b060 2026-02-25T11:50:50.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb2154e4640 1 -- 10.20.193.28:0/2897316261 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb23c0ad9d0 con 0x564837d9b060 2026-02-25T11:50:50.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.019+0000 7fb2154e4640 1 -- 10.20.193.28:0/2897316261 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb23c0e4be0 con 0x564837d9b060 2026-02-25T11:50:50.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.020+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2897316261 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837d9b060 msgr2=0x564837dda740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.020+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2897316261 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837d9b060 0x564837dda740 secure :-1 s=READY pgs=524 gs=3 cs=0 l=1 c_cookie=4309855007982339321 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb23c0a95a0 tx=0x7fb23c005540 comp rx=0 tx=0).stop 2026-02-25T11:50:50.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.020+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2897316261 shutdown_connections 2026-02-25T11:50:50.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.020+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2897316261 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837ddfb50 0x564837ddac80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.020+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2897316261 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837d9b060 0x564837dda740 unknown :-1 s=CLOSED pgs=524 gs=3 cs=0 l=1 c_cookie=4309855007982339321 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.020+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/2897316261 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837dfbf60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.020+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2897316261 >> 10.20.193.28:0/2897316261 conn(0x564837d788f0 msgr2=0x564837dfb850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:50.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.020+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2897316261 shutdown_connections 2026-02-25T11:50:50.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.020+0000 7fb2512e1e00 1 -- 10.20.193.28:0/2897316261 wait complete. 2026-02-25T11:50:50.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.020+0000 7fb2512e1e00 1 -- start start 2026-02-25T11:50:50.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb2512e1e00 1 Processor -- start 2026-02-25T11:50:50.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb2512e1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837d78e70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb2512e1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9b060 0x564837d793b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb2512e1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837d7dd10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb2512e1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564837964790 con 0x564837ddfb50 2026-02-25T11:50:50.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb2512e1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564837de4df0 con 0x564837d9b060 2026-02-25T11:50:50.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb2512e1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564837d75370 con 0x564837d9a750 2026-02-25T11:50:50.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb24c85a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9b060 0x564837d793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb24c85a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9b060 0x564837d793b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:56726/0 (socket says 10.20.193.28:56726) 2026-02-25T11:50:50.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb24c85a640 1 -- 10.20.193.28:0/3922486516 learned_addr learned my addr 10.20.193.28:0/3922486516 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:50.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb24d05b640 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837d78e70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb24d85c640 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837d7dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb24c85a640 1 -- 10.20.193.28:0/3922486516 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 msgr2=0x564837d78e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb24c85a640 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837d78e70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb24c85a640 1 -- 10.20.193.28:0/3922486516 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 msgr2=0x564837d7dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.021+0000 7fb24c85a640 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837d7dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb24c85a640 1 -- 10.20.193.28:0/3922486516 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c072040 con 0x564837d9b060 2026-02-25T11:50:50.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb24d85c640 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837d7dd10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:50.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb24c85a640 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9b060 0x564837d793b0 secure :-1 s=READY pgs=126 gs=1 cs=0 l=1 c_cookie=10777380915277258177 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb23c0b2eb0 tx=0x7fb23c0e4820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9590f2d7682cddc1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:50.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb20affd640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:50:50.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb20affd640 1 -- 10.20.193.28:0/3922486516 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb23c0ac9b0 con 0x564837d9b060 2026-02-25T11:50:50.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb20affd640 1 -- 10.20.193.28:0/3922486516 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb23c0b8110 con 0x564837d9b060 2026-02-25T11:50:50.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb20affd640 1 -- 10.20.193.28:0/3922486516 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb23c0b7c80 con 0x564837d9b060 2026-02-25T11:50:50.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb2512e1e00 1 -- 10.20.193.28:0/3922486516 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb244008bc0 con 0x564837d9b060 2026-02-25T11:50:50.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb20affd640 1 -- 10.20.193.28:0/3922486516 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb23c0b7e10 con 0x564837d9b060 2026-02-25T11:50:50.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb2512e1e00 10 client.25886 _ll_register_callbacks cb 0x564837c27e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:50.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb2512e1e00 1 -- 10.20.193.28:0/3922486516 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb23c0ac7d0 con 0x564837d9b060 2026-02-25T11:50:50.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb20affd640 1 -- 10.20.193.28:0/3922486516 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb23c0b2cf0 con 0x564837d9b060 2026-02-25T11:50:50.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb2512e1e00 10 client.25886 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:50.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.022+0000 7fb2512e1e00 1 -- 10.20.193.28:0/3922486516 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x564837c3c4f0 con 0x564837d9b060 2026-02-25T11:50:50.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.023+0000 7fb20affd640 1 -- 10.20.193.28:0/3922486516 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb23c0e4be0 con 0x564837d9b060 2026-02-25T11:50:50.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.023+0000 7fb2512e1e00 10 client.25886 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:50.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.023+0000 7fb2512e1e00 10 client.25886 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:50.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.023+0000 7fb2512e1e00 20 client.25886 populate_metadata read hostname 'trial028' 2026-02-25T11:50:50.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.023+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564837e8d340 0x564837ead7f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.023+0000 7fb2512e1e00 4 client.25886 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:50:50.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.023+0000 7fb2512e1e00 1 -- 10.20.193.28:0/3922486516 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564837b07690 con 0x564837e8d340 2026-02-25T11:50:50.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.023+0000 7fb24d05b640 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564837e8d340 0x564837ead7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.024+0000 7fb24d05b640 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564837e8d340 0x564837ead7f0 crc :-1 s=READY pgs=71 gs=4 cs=0 l=0 c_cookie=14650535347366467819 s_cookie=5764773400801201456 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb512846cd1518eb server_cookie=500096dd6d604d30 in_seq=0 out_seq=0 2026-02-25T11:50:50.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.024+0000 7fb20affd640 10 client.25886 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:50.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.024+0000 7fb20affd640 1 -- 10.20.193.28:0/3922486516 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564837b07690 con 0x564837e8d340 2026-02-25T11:50:50.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.024+0000 7fb20affd640 10 client.25886 handle_command_reply: tid=0 2026-02-25T11:50:50.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.024+0000 7fb2512e1e00 1 client.25886 shutdown 2026-02-25T11:50:50.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.024+0000 7fb2512e1e00 1 -- 10.20.193.28:0/3922486516 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9b060 msgr2=0x564837d793b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.024+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9b060 0x564837d793b0 secure :-1 s=READY pgs=126 gs=1 cs=0 l=1 c_cookie=10777380915277258177 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb23c0b2eb0 tx=0x7fb23c0e4820 comp rx=0 tx=0).stop 2026-02-25T11:50:50.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.024+0000 7fb2512e1e00 1 -- 10.20.193.28:0/3922486516 shutdown_connections 2026-02-25T11:50:50.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.024+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564837e8d340 0x564837ead7f0 crc :-1 s=READY pgs=71 gs=4 cs=0 l=0 c_cookie=14650535347366467819 s_cookie=5764773400801201456 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.024+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564837ddfb50 0x564837d7dd10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.024+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564837d9b060 0x564837d793b0 unknown :-1 s=CLOSED pgs=126 gs=1 cs=0 l=1 c_cookie=10777380915277258177 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.024+0000 7fb2512e1e00 1 --2- 10.20.193.28:0/3922486516 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564837d9a750 0x564837d78e70 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.025+0000 7fb20affd640 0 client.25886 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:50.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.025+0000 7fb2512e1e00 1 -- 10.20.193.28:0/3922486516 >> 10.20.193.28:0/3922486516 conn(0x564837d788f0 msgr2=0x564837df3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:50.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.025+0000 7fb2512e1e00 1 -- 10.20.193.28:0/3922486516 shutdown_connections 2026-02-25T11:50:50.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.025+0000 7fb2512e1e00 1 -- 10.20.193.28:0/3922486516 wait complete. 2026-02-25T11:50:50.032 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:50:50.032 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:50:50.032 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "36aaf888-2505-43bd-b817-3eafabc59fb7", 2026-02-25T11:50:50.032 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:50:50.032 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:50:50.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.025+0000 7fb2512e1e00 20 client.25886 trim_cache size 0 max 16384 2026-02-25T11:50:50.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.025+0000 7fb24e05d640 1 -- 10.20.193.28:0/143816253 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb2240b5760 msgr2=0x7fb2240d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.025+0000 7fb24e05d640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb2240b5760 0x7fb2240d5c10 secure :-1 s=READY pgs=294 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb238005b30 tx=0x7fb23803a040 comp rx=0 tx=0).stop 2026-02-25T11:50:50.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.025+0000 7fb24e05d640 1 -- 10.20.193.28:0/143816253 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb248114a90 msgr2=0x7fb248122a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.025+0000 7fb24e05d640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb248114a90 0x7fb248122a90 secure :-1 s=READY pgs=523 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb23c009480 tx=0x7fb23c00ee60 comp rx=0 tx=0).stop 2026-02-25T11:50:50.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.025+0000 7fb24e05d640 1 -- 10.20.193.28:0/143816253 shutdown_connections 2026-02-25T11:50:50.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.025+0000 7fb24e05d640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb2240b5760 0x7fb2240d5c10 unknown :-1 s=CLOSED pgs=294 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.025+0000 7fb24e05d640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb248119d30 0x7fb248127e60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.025+0000 7fb24e05d640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb248114a90 0x7fb248122a90 unknown :-1 s=CLOSED pgs=523 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.026+0000 7fb24e05d640 1 --2- 10.20.193.28:0/143816253 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb2481140a0 0x7fb248122550 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.026+0000 7fb24e05d640 1 -- 10.20.193.28:0/143816253 >> 10.20.193.28:0/143816253 conn(0x7fb248064c40 msgr2=0x7fb248117b80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:50.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.026+0000 7fb24e05d640 1 -- 10.20.193.28:0/143816253 shutdown_connections 2026-02-25T11:50:50.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.026+0000 7fb24e05d640 1 -- 10.20.193.28:0/143816253 wait complete. 2026-02-25T11:50:50.126 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:50:50.313 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:50:50.500 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.499+0000 7fd44f577640 1 -- 10.20.193.28:0/306622215 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd450114a90 msgr2=0x7fd450119580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.500 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.499+0000 7fd44f577640 1 --2- 10.20.193.28:0/306622215 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd450114a90 0x7fd450119580 secure :-1 s=READY pgs=525 gs=1 cs=0 l=1 c_cookie=6291614571621879930 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd438009950 tx=0x7fd43802f190 comp rx=0 tx=0).stop 2026-02-25T11:50:50.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.499+0000 7fd44f577640 1 -- 10.20.193.28:0/306622215 shutdown_connections 2026-02-25T11:50:50.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.499+0000 7fd44f577640 1 --2- 10.20.193.28:0/306622215 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd450119d30 0x7fd45017a700 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.499+0000 7fd44f577640 1 --2- 10.20.193.28:0/306622215 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd450114a90 0x7fd450119580 unknown :-1 s=CLOSED pgs=525 gs=1 cs=0 l=1 c_cookie=6291614571621879930 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.499+0000 7fd44f577640 1 --2- 10.20.193.28:0/306622215 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd4501140a0 0x7fd450114470 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.499+0000 7fd44f577640 1 -- 10.20.193.28:0/306622215 >> 10.20.193.28:0/306622215 conn(0x7fd450064c40 msgr2=0x7fd450065040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:50.501 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.499+0000 7fd44f577640 1 -- 10.20.193.28:0/306622215 shutdown_connections 2026-02-25T11:50:50.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.499+0000 7fd44f577640 1 -- 10.20.193.28:0/306622215 wait complete. 2026-02-25T11:50:50.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.500+0000 7fd44f577640 1 Processor -- start 2026-02-25T11:50:50.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.500+0000 7fd44f577640 1 -- start start 2026-02-25T11:50:50.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44f577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd4501140a0 0x7fd4500fbe30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44f577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd450114a90 0x7fd45017a600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44f577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd450119d30 0x7fd4500fc530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44f577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd450101740 con 0x7fd4501140a0 2026-02-25T11:50:50.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44f577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd4501018b0 con 0x7fd450119d30 2026-02-25T11:50:50.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44f577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd450101a70 con 0x7fd450114a90 2026-02-25T11:50:50.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd4501140a0 0x7fd4500fbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd4501140a0 0x7fd4500fbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58074/0 (socket says 10.20.193.28:58074) 2026-02-25T11:50:50.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44e575640 1 -- 10.20.193.28:0/1721475352 learned_addr learned my addr 10.20.193.28:0/1721475352 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:50.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44dd74640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd450114a90 0x7fd45017a600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44ed76640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd450119d30 0x7fd4500fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44e575640 1 -- 10.20.193.28:0/1721475352 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd450114a90 msgr2=0x7fd45017a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.504 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44e575640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd450114a90 0x7fd45017a600 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44e575640 1 -- 10.20.193.28:0/1721475352 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd450119d30 msgr2=0x7fd4500fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44e575640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd450119d30 0x7fd4500fc530 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44e575640 1 -- 10.20.193.28:0/1721475352 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd438009570 con 0x7fd4501140a0 2026-02-25T11:50:50.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.501+0000 7fd44e575640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd4501140a0 0x7fd4500fbe30 secure :-1 s=READY pgs=526 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd444005db0 tx=0x7fd44400ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:50.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.502+0000 7fd4377fe640 1 -- 10.20.193.28:0/1721475352 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd444004500 con 0x7fd4501140a0 2026-02-25T11:50:50.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.502+0000 7fd4377fe640 1 -- 10.20.193.28:0/1721475352 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd444004d10 con 0x7fd4501140a0 2026-02-25T11:50:50.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.502+0000 7fd44f577640 1 -- 10.20.193.28:0/1721475352 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd450101d50 con 0x7fd4501140a0 2026-02-25T11:50:50.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.502+0000 7fd4377fe640 1 -- 10.20.193.28:0/1721475352 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd4440052a0 con 0x7fd4501140a0 2026-02-25T11:50:50.505 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.502+0000 7fd44f577640 1 -- 10.20.193.28:0/1721475352 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd450102220 con 0x7fd4501140a0 2026-02-25T11:50:50.506 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.505+0000 7fd4377fe640 1 -- 10.20.193.28:0/1721475352 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fd444020020 con 0x7fd4501140a0 2026-02-25T11:50:50.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.506+0000 7fd4377fe640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd4280b5750 0x7fd4280d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.507+0000 7fd4377fe640 1 -- 10.20.193.28:0/1721475352 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd4440e65e0 con 0x7fd4501140a0 2026-02-25T11:50:50.508 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.507+0000 7fd44dd74640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd4280b5750 0x7fd4280d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.507+0000 7fd456dade00 1 Processor -- start 2026-02-25T11:50:50.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.507+0000 7fd44dd74640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd4280b5750 0x7fd4280d5c00 secure :-1 s=READY pgs=295 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd4380023d0 tx=0x7fd438031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:50.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.507+0000 7fd456dade00 1 -- start start 2026-02-25T11:50:50.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.507+0000 7fd456dade00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 0x565406b84510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd456dade00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84a50 0x565406bc91b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd456dade00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 0x565406be9ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd456dade00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x565406a9a9b0 con 0x565406b84a50 2026-02-25T11:50:50.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd456dade00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x565406a966e0 con 0x565406b84140 2026-02-25T11:50:50.511 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd456dade00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x565406a90840 con 0x565406bc96f0 2026-02-25T11:50:50.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd44dd74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84a50 0x565406bc91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd44e575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 0x565406b84510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd44e575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 0x565406b84510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:56746/0 (socket says 10.20.193.28:56746) 2026-02-25T11:50:50.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd44e575640 1 -- 10.20.193.28:0/322831224 learned_addr learned my addr 10.20.193.28:0/322831224 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:50.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd44ed76640 1 --2- 10.20.193.28:0/322831224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 0x565406be9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd44dd74640 1 -- 10.20.193.28:0/322831224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 msgr2=0x565406be9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd44dd74640 1 --2- 10.20.193.28:0/322831224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 0x565406be9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd44dd74640 1 -- 10.20.193.28:0/322831224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 msgr2=0x565406b84510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd44dd74640 1 --2- 10.20.193.28:0/322831224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 0x565406b84510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd44dd74640 1 -- 10.20.193.28:0/322831224 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565406a3bd80 con 0x565406b84a50 2026-02-25T11:50:50.512 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd44ed76640 1 --2- 10.20.193.28:0/322831224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 0x565406be9ad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:50.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd44e575640 1 --2- 10.20.193.28:0/322831224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 0x565406b84510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:50:50.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.508+0000 7fd44dd74640 1 --2- 10.20.193.28:0/322831224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84a50 0x565406bc91b0 secure :-1 s=READY pgs=527 gs=2 cs=0 l=1 c_cookie=1091547295118536548 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd438005b30 tx=0x7fd438040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25f47b4d142f64 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:50.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.509+0000 7fd434ff9640 1 -- 10.20.193.28:0/322831224 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd43802f9f0 con 0x565406b84a50 2026-02-25T11:50:50.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.509+0000 7fd434ff9640 1 -- 10.20.193.28:0/322831224 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd438038b60 con 0x565406b84a50 2026-02-25T11:50:50.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.509+0000 7fd434ff9640 1 -- 10.20.193.28:0/322831224 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd43804f660 con 0x565406b84a50 2026-02-25T11:50:50.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.509+0000 7fd456dade00 1 -- 10.20.193.28:0/322831224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84a50 msgr2=0x565406bc91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.509+0000 7fd456dade00 1 --2- 10.20.193.28:0/322831224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84a50 0x565406bc91b0 secure :-1 s=READY pgs=527 gs=2 cs=0 l=1 c_cookie=1091547295118536548 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd438005b30 tx=0x7fd438040000 comp rx=0 tx=0).stop 2026-02-25T11:50:50.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.509+0000 7fd456dade00 1 -- 10.20.193.28:0/322831224 shutdown_connections 2026-02-25T11:50:50.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.509+0000 7fd456dade00 1 --2- 10.20.193.28:0/322831224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 0x565406be9ad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.509+0000 7fd456dade00 1 --2- 10.20.193.28:0/322831224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84a50 0x565406bc91b0 unknown :-1 s=CLOSED pgs=527 gs=2 cs=0 l=1 c_cookie=1091547295118536548 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.509+0000 7fd456dade00 1 --2- 10.20.193.28:0/322831224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 0x565406b84510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.513 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.509+0000 7fd456dade00 1 -- 10.20.193.28:0/322831224 >> 10.20.193.28:0/322831224 conn(0x565406aa3d40 msgr2=0x565406b82a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:50.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.509+0000 7fd456dade00 1 -- 10.20.193.28:0/322831224 shutdown_connections 2026-02-25T11:50:50.514 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.509+0000 7fd456dade00 1 -- 10.20.193.28:0/322831224 wait complete. 2026-02-25T11:50:50.516 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.510+0000 7fd456dade00 1 -- start start 2026-02-25T11:50:50.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd456dade00 1 Processor -- start 2026-02-25T11:50:50.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd456dade00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 0x565406be1420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd456dade00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406b84a50 0x565406be1960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd456dade00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406bc96f0 0x565406be6950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd456dade00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x565406be6e90 con 0x565406b84140 2026-02-25T11:50:50.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd456dade00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x565406be7000 con 0x565406bc96f0 2026-02-25T11:50:50.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd456dade00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x565406be71c0 con 0x565406b84a50 2026-02-25T11:50:50.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd44e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 0x565406be1420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd44e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 0x565406be1420 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58090/0 (socket says 10.20.193.28:58090) 2026-02-25T11:50:50.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd44e575640 1 -- 10.20.193.28:0/482516048 learned_addr learned my addr 10.20.193.28:0/482516048 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:50.517 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd44dd74640 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406b84a50 0x565406be1960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd44e575640 1 -- 10.20.193.28:0/482516048 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406b84a50 msgr2=0x565406be1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd44ed76640 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406bc96f0 0x565406be6950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd44e575640 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406b84a50 0x565406be1960 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd44e575640 1 -- 10.20.193.28:0/482516048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406bc96f0 msgr2=0x565406be6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd44e575640 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406bc96f0 0x565406be6950 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.511+0000 7fd44e575640 1 -- 10.20.193.28:0/482516048 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45017af70 con 0x565406b84140 2026-02-25T11:50:50.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd44ed76640 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406bc96f0 0x565406be6950 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:50:50.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd44e575640 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 0x565406be1420 secure :-1 s=READY pgs=528 gs=1 cs=0 l=1 c_cookie=17514952564992932722 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd44400dfd0 tx=0x7fd4440ac160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3119c8f46387b72 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:50.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd3fbfff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:50.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd3fbfff640 1 -- 10.20.193.28:0/482516048 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd444009960 con 0x565406b84140 2026-02-25T11:50:50.518 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd3fbfff640 1 -- 10.20.193.28:0/482516048 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd4440b2680 con 0x565406b84140 2026-02-25T11:50:50.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd3fbfff640 1 -- 10.20.193.28:0/482516048 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd4440ac780 con 0x565406b84140 2026-02-25T11:50:50.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd456dade00 1 -- 10.20.193.28:0/482516048 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd450102220 con 0x565406b84140 2026-02-25T11:50:50.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd3fbfff640 1 -- 10.20.193.28:0/482516048 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd4440ad5f0 con 0x565406b84140 2026-02-25T11:50:50.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd3fbfff640 10 client.16098 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:50:50.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd456dade00 10 client.16098 _ll_register_callbacks cb 0x565406a119a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:50.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd456dade00 1 -- 10.20.193.28:0/482516048 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654067266f0 con 0x565406b84140 2026-02-25T11:50:50.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd3fbfff640 1 -- 10.20.193.28:0/482516048 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd4440b7aa0 con 0x565406b84140 2026-02-25T11:50:50.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd456dade00 10 client.16098 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:50.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.512+0000 7fd456dade00 1 -- 10.20.193.28:0/482516048 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x565406a14350 con 0x565406b84140 2026-02-25T11:50:50.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.513+0000 7fd3fbfff640 1 -- 10.20.193.28:0/482516048 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd4440b7610 con 0x565406b84140 2026-02-25T11:50:50.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.513+0000 7fd456dade00 10 client.16098 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:50.519 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.513+0000 7fd456dade00 10 client.16098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:50.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.513+0000 7fd456dade00 20 client.16098 populate_metadata read hostname 'trial028' 2026-02-25T11:50:50.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.513+0000 7fd456dade00 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x565406be9810 0x565406bc7420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.513+0000 7fd456dade00 4 client.16098 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:50:50.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.513+0000 7fd456dade00 1 -- 10.20.193.28:0/482516048 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565406a62910 con 0x565406be9810 2026-02-25T11:50:50.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.513+0000 7fd44dd74640 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x565406be9810 0x565406bc7420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd44dd74640 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x565406be9810 0x565406bc7420 crc :-1 s=READY pgs=72 gs=4 cs=0 l=0 c_cookie=3661435688893167107 s_cookie=6635433345569071368 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32d0085c246a3603 server_cookie=5c15cb78c56d2108 in_seq=0 out_seq=0 2026-02-25T11:50:50.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd3fbfff640 10 client.16098 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:50.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd3fbfff640 1 -- 10.20.193.28:0/482516048 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x565406a62910 con 0x565406be9810 2026-02-25T11:50:50.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd3fbfff640 10 client.16098 handle_command_reply: tid=0 2026-02-25T11:50:50.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd456dade00 1 client.16098 shutdown 2026-02-25T11:50:50.520 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd456dade00 1 -- 10.20.193.28:0/482516048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 msgr2=0x565406be1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd456dade00 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 0x565406be1420 secure :-1 s=READY pgs=528 gs=1 cs=0 l=1 c_cookie=17514952564992932722 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd44400dfd0 tx=0x7fd4440ac160 comp rx=0 tx=0).stop 2026-02-25T11:50:50.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd456dade00 1 -- 10.20.193.28:0/482516048 shutdown_connections 2026-02-25T11:50:50.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd456dade00 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x565406be9810 0x565406bc7420 crc :-1 s=READY pgs=72 gs=4 cs=0 l=0 c_cookie=3661435688893167107 s_cookie=6635433345569071368 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd456dade00 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406bc96f0 0x565406be6950 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd456dade00 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406b84a50 0x565406be1960 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd456dade00 1 --2- 10.20.193.28:0/482516048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 0x565406be1420 unknown :-1 s=CLOSED pgs=528 gs=1 cs=0 l=1 c_cookie=17514952564992932722 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd456dade00 1 -- 10.20.193.28:0/482516048 >> 10.20.193.28:0/482516048 conn(0x565406aa3d40 msgr2=0x565406b82230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:50.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd3fbfff640 0 client.16098 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:50.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd456dade00 1 -- 10.20.193.28:0/482516048 shutdown_connections 2026-02-25T11:50:50.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.514+0000 7fd456dade00 1 -- 10.20.193.28:0/482516048 wait complete. 2026-02-25T11:50:50.521 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.515+0000 7fd456dade00 20 client.16098 trim_cache size 0 max 16384 2026-02-25T11:50:50.525 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.524+0000 7fd456dade00 1 Processor -- start 2026-02-25T11:50:50.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.524+0000 7fd456dade00 1 -- start start 2026-02-25T11:50:50.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd456dade00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 0x565406bcc1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd456dade00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84a50 0x565406bde5f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd456dade00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 0x565406bdeb30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd456dade00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x565406a8e470 con 0x565406b84a50 2026-02-25T11:50:50.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd456dade00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56540674e700 con 0x565406b84140 2026-02-25T11:50:50.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd456dade00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x565406b84f30 con 0x565406bc96f0 2026-02-25T11:50:50.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd44dd74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84a50 0x565406bde5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd44e575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 0x565406bcc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd44e575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 0x565406bcc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:56754/0 (socket says 10.20.193.28:56754) 2026-02-25T11:50:50.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd44e575640 1 -- 10.20.193.28:0/2486160722 learned_addr learned my addr 10.20.193.28:0/2486160722 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:50.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd44ed76640 1 --2- 10.20.193.28:0/2486160722 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 0x565406bdeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd44e575640 1 -- 10.20.193.28:0/2486160722 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 msgr2=0x565406bdeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd44e575640 1 --2- 10.20.193.28:0/2486160722 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 0x565406bdeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd44e575640 1 -- 10.20.193.28:0/2486160722 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84a50 msgr2=0x565406bde5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd44e575640 1 --2- 10.20.193.28:0/2486160722 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84a50 0x565406bde5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd44e575640 1 -- 10.20.193.28:0/2486160722 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565406be1d10 con 0x565406b84140 2026-02-25T11:50:50.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.525+0000 7fd44e575640 1 --2- 10.20.193.28:0/2486160722 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 0x565406bcc1e0 secure :-1 s=READY pgs=127 gs=2 cs=0 l=1 c_cookie=5895234931656619583 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd4440ac940 tx=0x7fd4440c7000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51d014efa0ddf63f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:50.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.526+0000 7fd41a4e6640 1 -- 10.20.193.28:0/2486160722 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd4440a8fb0 con 0x565406b84140 2026-02-25T11:50:50.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.526+0000 7fd41a4e6640 1 -- 10.20.193.28:0/2486160722 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5654067266f0 con 0x565406b84140 2026-02-25T11:50:50.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.526+0000 7fd41a4e6640 1 -- 10.20.193.28:0/2486160722 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd4440a82b0 con 0x565406b84140 2026-02-25T11:50:50.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.526+0000 7fd456dade00 1 -- 10.20.193.28:0/2486160722 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 msgr2=0x565406bcc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.526+0000 7fd456dade00 1 --2- 10.20.193.28:0/2486160722 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 0x565406bcc1e0 secure :-1 s=READY pgs=127 gs=2 cs=0 l=1 c_cookie=5895234931656619583 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd4440ac940 tx=0x7fd4440c7000 comp rx=0 tx=0).stop 2026-02-25T11:50:50.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.526+0000 7fd456dade00 1 -- 10.20.193.28:0/2486160722 shutdown_connections 2026-02-25T11:50:50.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.526+0000 7fd456dade00 1 --2- 10.20.193.28:0/2486160722 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 0x565406bdeb30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.526+0000 7fd456dade00 1 --2- 10.20.193.28:0/2486160722 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84a50 0x565406bde5f0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.526+0000 7fd456dade00 1 --2- 10.20.193.28:0/2486160722 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84140 0x565406bcc1e0 unknown :-1 s=CLOSED pgs=127 gs=2 cs=0 l=1 c_cookie=5895234931656619583 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.526+0000 7fd456dade00 1 -- 10.20.193.28:0/2486160722 >> 10.20.193.28:0/2486160722 conn(0x565406aa3d40 msgr2=0x565406bcbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:50.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.526+0000 7fd456dade00 1 -- 10.20.193.28:0/2486160722 shutdown_connections 2026-02-25T11:50:50.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.526+0000 7fd456dade00 1 -- 10.20.193.28:0/2486160722 wait complete. 2026-02-25T11:50:50.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd456dade00 1 -- start start 2026-02-25T11:50:50.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd456dade00 1 Processor -- start 2026-02-25T11:50:50.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd456dade00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 0x565406b67970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd456dade00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84a50 0x565406b67eb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd456dade00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 0x565406bc4000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd456dade00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x565406bc4540 con 0x565406b84140 2026-02-25T11:50:50.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd456dade00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x565406bc46b0 con 0x565406b84a50 2026-02-25T11:50:50.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd456dade00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x565406bc4870 con 0x565406bc96f0 2026-02-25T11:50:50.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd44e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 0x565406b67970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd44e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 0x565406b67970 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58110/0 (socket says 10.20.193.28:58110) 2026-02-25T11:50:50.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd44e575640 1 -- 10.20.193.28:0/2236317754 learned_addr learned my addr 10.20.193.28:0/2236317754 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:50:50.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd44ed76640 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 0x565406bc4000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd44dd74640 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84a50 0x565406b67eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd44e575640 1 -- 10.20.193.28:0/2236317754 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 msgr2=0x565406bc4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.527+0000 7fd44e575640 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 0x565406bc4000 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd44e575640 1 -- 10.20.193.28:0/2236317754 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84a50 msgr2=0x565406b67eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:50:50.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd44e575640 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84a50 0x565406b67eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd44e575640 1 -- 10.20.193.28:0/2236317754 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565406a26150 con 0x565406b84140 2026-02-25T11:50:50.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd44dd74640 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84a50 0x565406b67eb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:50:50.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd44e575640 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 0x565406b67970 secure :-1 s=READY pgs=529 gs=1 cs=0 l=1 c_cookie=10806644399289480964 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd444009610 tx=0x7fd4440ac160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f8e9d3399d3704 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:50:50.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd3fbfff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:50:50.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd3fbfff640 1 -- 10.20.193.28:0/2236317754 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd4440a8fb0 con 0x565406b84140 2026-02-25T11:50:50.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd3fbfff640 1 -- 10.20.193.28:0/2236317754 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd4440960f0 con 0x565406b84140 2026-02-25T11:50:50.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd3fbfff640 1 -- 10.20.193.28:0/2236317754 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd444071a60 con 0x565406b84140 2026-02-25T11:50:50.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd456dade00 1 -- 10.20.193.28:0/2236317754 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565406a14350 con 0x565406b84140 2026-02-25T11:50:50.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd3fbfff640 1 -- 10.20.193.28:0/2236317754 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd444071bf0 con 0x565406b84140 2026-02-25T11:50:50.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd456dade00 10 client.16104 _ll_register_callbacks cb 0x565406a119a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:50:50.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd456dade00 1 -- 10.20.193.28:0/2236317754 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd438038b60 con 0x565406b84140 2026-02-25T11:50:50.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd3fbfff640 1 -- 10.20.193.28:0/2236317754 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd444009960 con 0x565406b84140 2026-02-25T11:50:50.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd456dade00 10 client.16104 fetch_fsmap learned FSMap version 23 2026-02-25T11:50:50.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.528+0000 7fd456dade00 1 -- 10.20.193.28:0/2236317754 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x565406a20bd0 con 0x565406b84140 2026-02-25T11:50:50.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.529+0000 7fd3fbfff640 1 -- 10.20.193.28:0/2236317754 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd44408f020 con 0x565406b84140 2026-02-25T11:50:50.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.529+0000 7fd456dade00 10 client.16104 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:50:50.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.529+0000 7fd456dade00 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:50:50.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.529+0000 7fd456dade00 20 client.16104 populate_metadata read hostname 'trial028' 2026-02-25T11:50:50.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.529+0000 7fd456dade00 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x565406b64010 0x565406c96c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:50:50.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.529+0000 7fd456dade00 4 client.16104 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:50:50.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.529+0000 7fd456dade00 1 -- 10.20.193.28:0/2236317754 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565406723f40 con 0x565406b64010 2026-02-25T11:50:50.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.529+0000 7fd44dd74640 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x565406b64010 0x565406c96c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:50:50.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.529+0000 7fd44dd74640 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x565406b64010 0x565406c96c90 crc :-1 s=READY pgs=73 gs=4 cs=0 l=0 c_cookie=14047841112587984662 s_cookie=2819422601299913309 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2f3f51030ef5f16 server_cookie=272099eafe306a5d in_seq=0 out_seq=0 2026-02-25T11:50:50.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.529+0000 7fd3fbfff640 10 client.16104 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:50.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd3fbfff640 1 -- 10.20.193.28:0/2236317754 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x565406723f40 con 0x565406b64010 2026-02-25T11:50:50.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd3fbfff640 10 client.16104 handle_command_reply: tid=0 2026-02-25T11:50:50.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd456dade00 1 client.16104 shutdown 2026-02-25T11:50:50.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd456dade00 1 -- 10.20.193.28:0/2236317754 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 msgr2=0x565406b67970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd456dade00 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 0x565406b67970 secure :-1 s=READY pgs=529 gs=1 cs=0 l=1 c_cookie=10806644399289480964 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd444009610 tx=0x7fd4440ac160 comp rx=0 tx=0).stop 2026-02-25T11:50:50.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd456dade00 1 -- 10.20.193.28:0/2236317754 shutdown_connections 2026-02-25T11:50:50.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd456dade00 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x565406b64010 0x565406c96c90 crc :-1 s=READY pgs=73 gs=4 cs=0 l=0 c_cookie=14047841112587984662 s_cookie=2819422601299913309 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd456dade00 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565406bc96f0 0x565406bc4000 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd456dade00 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565406b84a50 0x565406b67eb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd456dade00 1 --2- 10.20.193.28:0/2236317754 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565406b84140 0x565406b67970 unknown :-1 s=CLOSED pgs=529 gs=1 cs=0 l=1 c_cookie=10806644399289480964 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd3fbfff640 0 client.16104 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:50:50.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd456dade00 1 -- 10.20.193.28:0/2236317754 >> 10.20.193.28:0/2236317754 conn(0x565406aa3d40 msgr2=0x565406bdd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:50.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd456dade00 1 -- 10.20.193.28:0/2236317754 shutdown_connections 2026-02-25T11:50:50.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd456dade00 1 -- 10.20.193.28:0/2236317754 wait complete. 2026-02-25T11:50:50.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.530+0000 7fd456dade00 20 client.16104 trim_cache size 0 max 16384 2026-02-25T11:50:50.539 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:50:50.540 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:50:50.540 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:50:50.540 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:50:50.540 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:50.540 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "36aaf888-2505-43bd-b817-3eafabc59fb7", 2026-02-25T11:50:50.540 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:50:50.540 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:50:50.540 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:50.540 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:50:50.540 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:50:50.541 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:50:50.541 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:50.541 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:50.541 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:50.541 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:50.541 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:50.541 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:50.541 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:50:50.541 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:50:50.541 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:50.541 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:50.542 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:50.542 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:50.542 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:50.542 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:50.542 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:50:50.542 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:50:50.542 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:50.542 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:50.542 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:50.542 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:50.542 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:50.542 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:50.543 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:50:50.543 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:50:50.543 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:50.543 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:50.543 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:50.543 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:50.543 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:50.543 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:50.543 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:50:50.543 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:50:50.543 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:50.544 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:50.544 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:50.544 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:50.544 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:50.544 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:50.544 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:50:50.544 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:50:50.544 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:50.544 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:50.544 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:50.544 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:50.545 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:50.545 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:50:50.545 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:50:50.545 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:50:50.545 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:50:50.545 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:50:50.545 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:50:50.545 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:50:50.545 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:50:50.545 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:50:50.545 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:50:50.546 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:50:50.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.531+0000 7fd44f577640 1 -- 10.20.193.28:0/1721475352 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd4280b5750 msgr2=0x7fd4280d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.531+0000 7fd44f577640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd4280b5750 0x7fd4280d5c00 secure :-1 s=READY pgs=295 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd4380023d0 tx=0x7fd438031040 comp rx=0 tx=0).stop 2026-02-25T11:50:50.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.531+0000 7fd44f577640 1 -- 10.20.193.28:0/1721475352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd4501140a0 msgr2=0x7fd4500fbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:50:50.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.531+0000 7fd44f577640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd4501140a0 0x7fd4500fbe30 secure :-1 s=READY pgs=526 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd444005db0 tx=0x7fd44400ddf0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.531+0000 7fd44f577640 1 -- 10.20.193.28:0/1721475352 shutdown_connections 2026-02-25T11:50:50.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.531+0000 7fd44f577640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd4280b5750 0x7fd4280d5c00 unknown :-1 s=CLOSED pgs=295 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.531+0000 7fd44f577640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd450119d30 0x7fd4500fc530 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.531+0000 7fd44f577640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd450114a90 0x7fd45017a600 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.531+0000 7fd44f577640 1 --2- 10.20.193.28:0/1721475352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd4501140a0 0x7fd4500fbe30 unknown :-1 s=CLOSED pgs=526 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:50:50.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.531+0000 7fd44f577640 1 -- 10.20.193.28:0/1721475352 >> 10.20.193.28:0/1721475352 conn(0x7fd450064c40 msgr2=0x7fd450065fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:50:50.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.531+0000 7fd44f577640 1 -- 10.20.193.28:0/1721475352 shutdown_connections 2026-02-25T11:50:50.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:50:50.531+0000 7fd44f577640 1 -- 10.20.193.28:0/1721475352 wait complete. 2026-02-25T11:50:50.587 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:50 trial028 ceph-mon[39224]: pgmap v304: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:50:50.587 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:50 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:50:50.587 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:50 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:50:50.588 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:50 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:50:50.588 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:50 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:50:50.633 INFO:tasks.cephfs.filesystem:scrub status for tag:36aaf888-2505-43bd-b817-3eafabc59fb7 - {'path': '/', 'tag': '36aaf888-2505-43bd-b817-3eafabc59fb7', 'options': 'recursive,force'} 2026-02-25T11:50:50.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:50 trial167 ceph-mon[43392]: pgmap v304: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:50:50.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:50 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:50:50.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:50 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:50:50.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:50 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:50:50.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:50 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:50:50.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:50 trial119 ceph-mon[42710]: pgmap v304: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:50:50.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:50 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:50:50.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:50 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:50:50.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:50 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:50:50.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:50 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:50:52.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:52 trial167 ceph-mon[43392]: pgmap v305: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:52.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:52 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:50:52.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:52 trial119 ceph-mon[42710]: pgmap v305: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:52.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:52 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:50:52.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:52 trial028 ceph-mon[39224]: pgmap v305: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:50:52.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:52 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:50:54.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:54 trial167 ceph-mon[43392]: pgmap v306: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:50:54.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:54 trial119 ceph-mon[42710]: pgmap v306: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:50:54.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:54 trial028 ceph-mon[39224]: pgmap v306: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:50:55.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:55 trial167 ceph-mon[43392]: pgmap v307: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:50:55.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:55 trial119 ceph-mon[42710]: pgmap v307: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:50:55.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:55 trial028 ceph-mon[39224]: pgmap v307: 97 pgs: 97 active+clean; 2.4 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:50:58.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:50:58 trial167 ceph-mon[43392]: pgmap v308: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:58.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:50:58 trial119 ceph-mon[42710]: pgmap v308: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:50:58.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:50:58 trial028 ceph-mon[39224]: pgmap v308: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:00.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:00 trial167 ceph-mon[43392]: pgmap v309: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:51:00.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:00 trial119 ceph-mon[42710]: pgmap v309: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:51:00.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:00 trial028 ceph-mon[39224]: pgmap v309: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:51:02.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:02 trial167 ceph-mon[43392]: pgmap v310: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:51:02.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:02 trial119 ceph-mon[42710]: pgmap v310: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:51:02.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:02 trial028 ceph-mon[39224]: pgmap v310: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:51:04.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:04 trial167 ceph-mon[43392]: pgmap v311: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:04.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:04 trial119 ceph-mon[42710]: pgmap v311: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:04.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:04 trial028 ceph-mon[39224]: pgmap v311: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:06.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:06 trial167 ceph-mon[43392]: pgmap v312: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:51:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:06 trial119 ceph-mon[42710]: pgmap v312: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:51:06.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:06 trial028 ceph-mon[39224]: pgmap v312: 97 pgs: 97 active+clean; 2.5 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:51:08.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:08 trial167 ceph-mon[43392]: pgmap v313: 97 pgs: 97 active+clean; 2.6 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:08.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:08 trial119 ceph-mon[42710]: pgmap v313: 97 pgs: 97 active+clean; 2.6 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:08.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:08 trial028 ceph-mon[39224]: pgmap v313: 97 pgs: 97 active+clean; 2.6 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:10.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:10 trial167 ceph-mon[43392]: pgmap v314: 97 pgs: 97 active+clean; 2.6 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:51:10.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:10 trial119 ceph-mon[42710]: pgmap v314: 97 pgs: 97 active+clean; 2.6 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:51:10.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:10 trial028 ceph-mon[39224]: pgmap v314: 97 pgs: 97 active+clean; 2.6 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:51:12.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:12 trial167 ceph-mon[43392]: pgmap v315: 97 pgs: 97 active+clean; 2.6 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:12.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:12 trial119 ceph-mon[42710]: pgmap v315: 97 pgs: 97 active+clean; 2.6 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:12.526 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:12 trial028 ceph-mon[39224]: pgmap v315: 97 pgs: 97 active+clean; 2.6 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:14.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:14 trial167 ceph-mon[43392]: pgmap v316: 97 pgs: 97 active+clean; 2.6 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:14.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:14 trial119 ceph-mon[42710]: pgmap v316: 97 pgs: 97 active+clean; 2.6 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:14.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:14 trial028 ceph-mon[39224]: pgmap v316: 97 pgs: 97 active+clean; 2.6 MiB data, 336 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:15.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:51:15.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:51:15.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:51:15.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:51:15.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:51:15.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:51:15.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:51:15.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:51:15.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:51:15.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:51:15.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:51:15.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:51:15.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:51:15.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:51:15.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:51:16.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:16 trial167 ceph-mon[43392]: pgmap v317: 97 pgs: 97 active+clean; 2.6 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:16.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:16 trial119 ceph-mon[42710]: pgmap v317: 97 pgs: 97 active+clean; 2.6 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:16.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:16 trial028 ceph-mon[39224]: pgmap v317: 97 pgs: 97 active+clean; 2.6 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:18.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:18 trial167 ceph-mon[43392]: pgmap v318: 97 pgs: 97 active+clean; 2.7 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:18.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:18 trial119 ceph-mon[42710]: pgmap v318: 97 pgs: 97 active+clean; 2.7 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:18.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:18 trial028 ceph-mon[39224]: pgmap v318: 97 pgs: 97 active+clean; 2.7 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:20.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:20 trial167 ceph-mon[43392]: pgmap v319: 97 pgs: 97 active+clean; 2.7 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:51:20.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:20 trial119 ceph-mon[42710]: pgmap v319: 97 pgs: 97 active+clean; 2.7 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:51:20.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:20 trial028 ceph-mon[39224]: pgmap v319: 97 pgs: 97 active+clean; 2.7 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:51:20.634 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:51:20.821 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:20.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.995+0000 7fe368e93640 1 -- 10.20.193.28:0/1109243312 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe36410d3d0 msgr2=0x7fe36410d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:20.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.995+0000 7fe368e93640 1 --2- 10.20.193.28:0/1109243312 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe36410d3d0 0x7fe36410d7a0 secure :-1 s=READY pgs=530 gs=1 cs=0 l=1 c_cookie=6090790615772187808 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe350009950 tx=0x7fe35002f1d0 comp rx=0 tx=0).stop 2026-02-25T11:51:20.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.995+0000 7fe368e93640 1 -- 10.20.193.28:0/1109243312 shutdown_connections 2026-02-25T11:51:20.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.995+0000 7fe368e93640 1 --2- 10.20.193.28:0/1109243312 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe3641135c0 0x7fe364173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:20.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.995+0000 7fe368e93640 1 --2- 10.20.193.28:0/1109243312 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe36410ddc0 0x7fe364112e10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:20.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.995+0000 7fe368e93640 1 --2- 10.20.193.28:0/1109243312 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe36410d3d0 0x7fe36410d7a0 unknown :-1 s=CLOSED pgs=530 gs=1 cs=0 l=1 c_cookie=6090790615772187808 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:20.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.995+0000 7fe368e93640 1 -- 10.20.193.28:0/1109243312 >> 10.20.193.28:0/1109243312 conn(0x7fe3640eaca0 msgr2=0x7fe36410b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:20.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.995+0000 7fe368e93640 1 -- 10.20.193.28:0/1109243312 shutdown_connections 2026-02-25T11:51:20.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.995+0000 7fe368e93640 1 -- 10.20.193.28:0/1109243312 wait complete. 2026-02-25T11:51:20.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.996+0000 7fe368e93640 1 Processor -- start 2026-02-25T11:51:20.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.996+0000 7fe368e93640 1 -- start start 2026-02-25T11:51:21.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.996+0000 7fe368e93640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe36410d3d0 0x7fe3640f59a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.996+0000 7fe368e93640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe36410ddc0 0x7fe364173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.996+0000 7fe368e93640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3641135c0 0x7fe3640f60a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.996+0000 7fe368e93640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640fb260 con 0x7fe3641135c0 2026-02-25T11:51:21.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.996+0000 7fe368e93640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640fb3d0 con 0x7fe36410ddc0 2026-02-25T11:51:21.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.996+0000 7fe368e93640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe3640fb590 con 0x7fe36410d3d0 2026-02-25T11:51:21.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe363fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3641135c0 0x7fe3640f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe363fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3641135c0 0x7fe3640f60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:50988/0 (socket says 10.20.193.28:50988) 2026-02-25T11:51:21.001 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe363fff640 1 -- 10.20.193.28:0/1337482537 learned_addr learned my addr 10.20.193.28:0/1337482537 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:21.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe3637fe640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe36410d3d0 0x7fe3640f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe362ffd640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe36410ddc0 0x7fe364173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe363fff640 1 -- 10.20.193.28:0/1337482537 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe36410d3d0 msgr2=0x7fe3640f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:21.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe363fff640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe36410d3d0 0x7fe3640f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe363fff640 1 -- 10.20.193.28:0/1337482537 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe36410ddc0 msgr2=0x7fe364173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:21.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe363fff640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe36410ddc0 0x7fe364173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe363fff640 1 -- 10.20.193.28:0/1337482537 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe350009570 con 0x7fe3641135c0 2026-02-25T11:51:21.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe362ffd640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe36410ddc0 0x7fe364173e90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:21.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe363fff640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3641135c0 0x7fe3640f60a0 secure :-1 s=READY pgs=531 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe35800f8d0 tx=0x7fe35800fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:21.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe360ff9640 1 -- 10.20.193.28:0/1337482537 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe35800dce0 con 0x7fe3641135c0 2026-02-25T11:51:21.002 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe360ff9640 1 -- 10.20.193.28:0/1337482537 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe35800de40 con 0x7fe3641135c0 2026-02-25T11:51:21.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe360ff9640 1 -- 10.20.193.28:0/1337482537 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe3580116b0 con 0x7fe3641135c0 2026-02-25T11:51:21.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe368e93640 1 -- 10.20.193.28:0/1337482537 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3640fb870 con 0x7fe3641135c0 2026-02-25T11:51:21.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:20.997+0000 7fe368e93640 1 -- 10.20.193.28:0/1337482537 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3640fbcf0 con 0x7fe3641135c0 2026-02-25T11:51:21.003 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.001+0000 7fe360ff9640 1 -- 10.20.193.28:0/1337482537 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fe358020060 con 0x7fe3641135c0 2026-02-25T11:51:21.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.002+0000 7fe360ff9640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe3380b5750 0x7fe3380d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.003+0000 7fe360ff9640 1 -- 10.20.193.28:0/1337482537 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fe358015070 con 0x7fe3641135c0 2026-02-25T11:51:21.005 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.003+0000 7fe3637fe640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe3380b5750 0x7fe3380d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.006 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.003+0000 7fe36c117e00 1 Processor -- start 2026-02-25T11:51:21.006 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.003+0000 7fe36c117e00 1 -- start start 2026-02-25T11:51:21.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.003+0000 7fe36c117e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02140 0x55d5f2a02510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.003+0000 7fe36c117e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a02a50 0x55d5f2a471b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.003+0000 7fe36c117e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a476f0 0x55d5f2a67ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.003+0000 7fe36c117e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d5f29189b0 con 0x55d5f2a02140 2026-02-25T11:51:21.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.003+0000 7fe36c117e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d5f29146e0 con 0x55d5f2a02a50 2026-02-25T11:51:21.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.003+0000 7fe36c117e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d5f290e840 con 0x55d5f2a476f0 2026-02-25T11:51:21.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe363fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a476f0 0x55d5f2a67ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe3637fe640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe3380b5750 0x7fe3380d5c00 secure :-1 s=READY pgs=296 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe3500098a0 tx=0x7fe35003a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:21.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe3637fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02140 0x55d5f2a02510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe363fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a476f0 0x55d5f2a67ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44256/0 (socket says 10.20.193.28:44256) 2026-02-25T11:51:21.007 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe363fff640 1 -- 10.20.193.28:0/235917008 learned_addr learned my addr 10.20.193.28:0/235917008 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:21.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe362ffd640 1 --2- 10.20.193.28:0/235917008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a02a50 0x55d5f2a471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe363fff640 1 -- 10.20.193.28:0/235917008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a02a50 msgr2=0x55d5f2a471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:21.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe363fff640 1 --2- 10.20.193.28:0/235917008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a02a50 0x55d5f2a471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe363fff640 1 -- 10.20.193.28:0/235917008 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02140 msgr2=0x55d5f2a02510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:21.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe363fff640 1 --2- 10.20.193.28:0/235917008 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02140 0x55d5f2a02510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe363fff640 1 -- 10.20.193.28:0/235917008 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f28b9d80 con 0x55d5f2a476f0 2026-02-25T11:51:21.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe3637fe640 1 --2- 10.20.193.28:0/235917008 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02140 0x55d5f2a02510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:51:21.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe363fff640 1 --2- 10.20.193.28:0/235917008 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a476f0 0x55d5f2a67ad0 secure :-1 s=READY pgs=149 gs=2 cs=0 l=1 c_cookie=6657728141218361780 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe3580aa5e0 tx=0x7fe35800a4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c650078647131b4 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:21.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe327fff640 1 -- 10.20.193.28:0/235917008 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe3580068a0 con 0x55d5f2a476f0 2026-02-25T11:51:21.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.004+0000 7fe327fff640 1 -- 10.20.193.28:0/235917008 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe358097040 con 0x55d5f2a476f0 2026-02-25T11:51:21.008 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.005+0000 7fe327fff640 1 -- 10.20.193.28:0/235917008 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe3580ade30 con 0x55d5f2a476f0 2026-02-25T11:51:21.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.005+0000 7fe36c117e00 1 -- 10.20.193.28:0/235917008 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a476f0 msgr2=0x55d5f2a67ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:21.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.005+0000 7fe36c117e00 1 --2- 10.20.193.28:0/235917008 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a476f0 0x55d5f2a67ad0 secure :-1 s=READY pgs=149 gs=2 cs=0 l=1 c_cookie=6657728141218361780 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe3580aa5e0 tx=0x7fe35800a4a0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.005+0000 7fe36c117e00 1 -- 10.20.193.28:0/235917008 shutdown_connections 2026-02-25T11:51:21.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.005+0000 7fe36c117e00 1 --2- 10.20.193.28:0/235917008 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a476f0 0x55d5f2a67ad0 unknown :-1 s=CLOSED pgs=149 gs=2 cs=0 l=1 c_cookie=6657728141218361780 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.005+0000 7fe36c117e00 1 --2- 10.20.193.28:0/235917008 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a02a50 0x55d5f2a471b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.005+0000 7fe36c117e00 1 --2- 10.20.193.28:0/235917008 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02140 0x55d5f2a02510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.005+0000 7fe36c117e00 1 -- 10.20.193.28:0/235917008 >> 10.20.193.28:0/235917008 conn(0x55d5f2921d40 msgr2=0x55d5f2a00a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:21.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.005+0000 7fe36c117e00 1 -- 10.20.193.28:0/235917008 shutdown_connections 2026-02-25T11:51:21.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.005+0000 7fe36c117e00 1 -- 10.20.193.28:0/235917008 wait complete. 2026-02-25T11:51:21.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.006+0000 7fe36c117e00 1 -- start start 2026-02-25T11:51:21.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.006+0000 7fe36c117e00 1 Processor -- start 2026-02-25T11:51:21.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe36c117e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 0x55d5f2a06320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe36c117e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a06860 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe36c117e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a476f0 0x55d5f2a05120 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe36c117e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d5f2a05660 con 0x55d5f2a02a50 2026-02-25T11:51:21.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe36c117e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d5f2a057d0 con 0x55d5f2a476f0 2026-02-25T11:51:21.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe36c117e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d5f2a05990 con 0x55d5f2a02140 2026-02-25T11:51:21.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe3637fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 0x55d5f2a06320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe3637fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 0x55d5f2a06320 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44262/0 (socket says 10.20.193.28:44262) 2026-02-25T11:51:21.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe3637fe640 1 -- 10.20.193.28:0/889275563 learned_addr learned my addr 10.20.193.28:0/889275563 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:21.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe362ffd640 1 --2- 10.20.193.28:0/889275563 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a06860 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe363fff640 1 --2- 10.20.193.28:0/889275563 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a476f0 0x55d5f2a05120 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe3637fe640 1 -- 10.20.193.28:0/889275563 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a476f0 msgr2=0x55d5f2a05120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:21.013 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe3637fe640 1 --2- 10.20.193.28:0/889275563 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a476f0 0x55d5f2a05120 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe3637fe640 1 -- 10.20.193.28:0/889275563 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 msgr2=0x55d5f2a06860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:21.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe3637fe640 1 --2- 10.20.193.28:0/889275563 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a06860 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.007+0000 7fe3637fe640 1 -- 10.20.193.28:0/889275563 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3640fbcf0 con 0x55d5f2a02140 2026-02-25T11:51:21.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.008+0000 7fe3637fe640 1 --2- 10.20.193.28:0/889275563 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 0x55d5f2a06320 secure :-1 s=READY pgs=150 gs=1 cs=0 l=1 c_cookie=15288043794330913456 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe350004490 tx=0x7fe350009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d42a0ad4d7b58ab0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:21.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.008+0000 7fe3257fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:51:21.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.008+0000 7fe3257fa640 1 -- 10.20.193.28:0/889275563 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe350038ad0 con 0x55d5f2a02140 2026-02-25T11:51:21.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.008+0000 7fe3257fa640 1 -- 10.20.193.28:0/889275563 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe35003e040 con 0x55d5f2a02140 2026-02-25T11:51:21.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.008+0000 7fe36c117e00 1 -- 10.20.193.28:0/889275563 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe364174800 con 0x55d5f2a02140 2026-02-25T11:51:21.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.008+0000 7fe3257fa640 1 -- 10.20.193.28:0/889275563 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe3500503f0 con 0x55d5f2a02140 2026-02-25T11:51:21.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.008+0000 7fe36c117e00 10 client.25888 _ll_register_callbacks cb 0x55d5f288f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:21.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.008+0000 7fe3257fa640 1 -- 10.20.193.28:0/889275563 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fe350038dc0 con 0x55d5f2a02140 2026-02-25T11:51:21.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.008+0000 7fe36c117e00 1 -- 10.20.193.28:0/889275563 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d5f25a46f0 con 0x55d5f2a02140 2026-02-25T11:51:21.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.008+0000 7fe3257fa640 1 -- 10.20.193.28:0/889275563 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe3500503f0 con 0x55d5f2a02140 2026-02-25T11:51:21.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.008+0000 7fe36c117e00 10 client.25888 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:21.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.008+0000 7fe36c117e00 1 -- 10.20.193.28:0/889275563 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55d5f2892350 con 0x55d5f2a02140 2026-02-25T11:51:21.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.009+0000 7fe3257fa640 1 -- 10.20.193.28:0/889275563 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe35004f070 con 0x55d5f2a02140 2026-02-25T11:51:21.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.009+0000 7fe36c117e00 10 client.25888 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:21.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.009+0000 7fe36c117e00 10 client.25888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:21.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.009+0000 7fe36c117e00 20 client.25888 populate_metadata read hostname 'trial028' 2026-02-25T11:51:21.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.009+0000 7fe36c117e00 1 --2- 10.20.193.28:0/889275563 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d5f2a2d650 0x55d5f2a2daf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.009+0000 7fe36c117e00 4 client.25888 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:51:21.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.009+0000 7fe36c117e00 1 -- 10.20.193.28:0/889275563 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5f28e0910 con 0x55d5f2a2d650 2026-02-25T11:51:21.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.009+0000 7fe362ffd640 1 --2- 10.20.193.28:0/889275563 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d5f2a2d650 0x55d5f2a2daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.017 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.010+0000 7fe362ffd640 1 --2- 10.20.193.28:0/889275563 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d5f2a2d650 0x55d5f2a2daf0 crc :-1 s=READY pgs=74 gs=4 cs=0 l=0 c_cookie=11411922457220671362 s_cookie=17821203217270592739 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e5f4b030af0e782 server_cookie=f751a1e7c37654e3 in_seq=0 out_seq=0 2026-02-25T11:51:21.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.010+0000 7fe3257fa640 10 client.25888 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:21.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.010+0000 7fe3257fa640 1 -- 10.20.193.28:0/889275563 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55d5f28e0910 con 0x55d5f2a2d650 2026-02-25T11:51:21.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.010+0000 7fe3257fa640 10 client.25888 handle_command_reply: tid=0 2026-02-25T11:51:21.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.010+0000 7fe36c117e00 1 client.25888 shutdown 2026-02-25T11:51:21.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.011+0000 7fe36c117e00 1 -- 10.20.193.28:0/889275563 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 msgr2=0x55d5f2a06320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:21.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.011+0000 7fe36c117e00 1 --2- 10.20.193.28:0/889275563 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 0x55d5f2a06320 secure :-1 s=READY pgs=150 gs=1 cs=0 l=1 c_cookie=15288043794330913456 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe350004490 tx=0x7fe350009310 comp rx=0 tx=0).stop 2026-02-25T11:51:21.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.011+0000 7fe36c117e00 1 -- 10.20.193.28:0/889275563 shutdown_connections 2026-02-25T11:51:21.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.011+0000 7fe36c117e00 1 --2- 10.20.193.28:0/889275563 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d5f2a2d650 0x55d5f2a2daf0 crc :-1 s=READY pgs=74 gs=4 cs=0 l=0 c_cookie=11411922457220671362 s_cookie=17821203217270592739 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.011+0000 7fe36c117e00 1 --2- 10.20.193.28:0/889275563 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a476f0 0x55d5f2a05120 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.011+0000 7fe36c117e00 1 --2- 10.20.193.28:0/889275563 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a06860 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.011+0000 7fe36c117e00 1 --2- 10.20.193.28:0/889275563 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 0x55d5f2a06320 unknown :-1 s=CLOSED pgs=150 gs=1 cs=0 l=1 c_cookie=15288043794330913456 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.011+0000 7fe36c117e00 1 -- 10.20.193.28:0/889275563 >> 10.20.193.28:0/889275563 conn(0x55d5f2921d40 msgr2=0x55d5f2a00230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:21.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.011+0000 7fe3257fa640 0 client.25888 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:21.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.011+0000 7fe36c117e00 1 -- 10.20.193.28:0/889275563 shutdown_connections 2026-02-25T11:51:21.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.011+0000 7fe36c117e00 1 -- 10.20.193.28:0/889275563 wait complete. 2026-02-25T11:51:21.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.011+0000 7fe36c117e00 20 client.25888 trim_cache size 0 max 16384 2026-02-25T11:51:21.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.020+0000 7fe36c117e00 1 Processor -- start 2026-02-25T11:51:21.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.020+0000 7fe36c117e00 1 -- start start 2026-02-25T11:51:21.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.020+0000 7fe36c117e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 0x55d5f2a06840 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.020+0000 7fe36c117e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a46db0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.020+0000 7fe36c117e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a476f0 0x55d5f2a64f40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.020+0000 7fe36c117e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d5f290c470 con 0x55d5f2a02a50 2026-02-25T11:51:21.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.020+0000 7fe36c117e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d5f25cc700 con 0x55d5f2a476f0 2026-02-25T11:51:21.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.020+0000 7fe36c117e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d5f2a05ad0 con 0x55d5f2a02140 2026-02-25T11:51:21.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.020+0000 7fe3637fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 0x55d5f2a06840 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.020+0000 7fe362ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a46db0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.020+0000 7fe3637fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 0x55d5f2a06840 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44278/0 (socket says 10.20.193.28:44278) 2026-02-25T11:51:21.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.020+0000 7fe3637fe640 1 -- 10.20.193.28:0/3794931401 learned_addr learned my addr 10.20.193.28:0/3794931401 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:21.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.020+0000 7fe363fff640 1 --2- 10.20.193.28:0/3794931401 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a476f0 0x55d5f2a64f40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe3637fe640 1 -- 10.20.193.28:0/3794931401 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a476f0 msgr2=0x55d5f2a64f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:21.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe3637fe640 1 --2- 10.20.193.28:0/3794931401 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a476f0 0x55d5f2a64f40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe3637fe640 1 -- 10.20.193.28:0/3794931401 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 msgr2=0x55d5f2a46db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:21.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe3637fe640 1 --2- 10.20.193.28:0/3794931401 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a46db0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe3637fe640 1 -- 10.20.193.28:0/3794931401 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f28a4150 con 0x55d5f2a02140 2026-02-25T11:51:21.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe363fff640 1 --2- 10.20.193.28:0/3794931401 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a476f0 0x55d5f2a64f40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:21.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe362ffd640 1 --2- 10.20.193.28:0/3794931401 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a46db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:51:21.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe3637fe640 1 --2- 10.20.193.28:0/3794931401 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 0x55d5f2a06840 secure :-1 s=READY pgs=151 gs=1 cs=0 l=1 c_cookie=18240230875907170197 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe350002bb0 tx=0x7fe35005a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd2251573b8f1f95 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:21.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe3277fe640 1 -- 10.20.193.28:0/3794931401 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe350058e20 con 0x55d5f2a02140 2026-02-25T11:51:21.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe3277fe640 1 -- 10.20.193.28:0/3794931401 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe35003e070 con 0x55d5f2a02140 2026-02-25T11:51:21.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe3277fe640 1 -- 10.20.193.28:0/3794931401 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe350038b10 con 0x55d5f2a02140 2026-02-25T11:51:21.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe36c117e00 1 -- 10.20.193.28:0/3794931401 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 msgr2=0x55d5f2a06840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:21.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe36c117e00 1 --2- 10.20.193.28:0/3794931401 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 0x55d5f2a06840 secure :-1 s=READY pgs=151 gs=1 cs=0 l=1 c_cookie=18240230875907170197 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe350002bb0 tx=0x7fe35005a000 comp rx=0 tx=0).stop 2026-02-25T11:51:21.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe36c117e00 1 -- 10.20.193.28:0/3794931401 shutdown_connections 2026-02-25T11:51:21.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe36c117e00 1 --2- 10.20.193.28:0/3794931401 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a476f0 0x55d5f2a64f40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe36c117e00 1 --2- 10.20.193.28:0/3794931401 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a46db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe36c117e00 1 --2- 10.20.193.28:0/3794931401 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a02140 0x55d5f2a06840 unknown :-1 s=CLOSED pgs=151 gs=1 cs=0 l=1 c_cookie=18240230875907170197 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe36c117e00 1 -- 10.20.193.28:0/3794931401 >> 10.20.193.28:0/3794931401 conn(0x55d5f2921d40 msgr2=0x55d5f2a06130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:21.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.021+0000 7fe36c117e00 1 -- 10.20.193.28:0/3794931401 shutdown_connections 2026-02-25T11:51:21.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.022+0000 7fe36c117e00 1 -- 10.20.193.28:0/3794931401 wait complete. 2026-02-25T11:51:21.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.022+0000 7fe36c117e00 1 -- start start 2026-02-25T11:51:21.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.022+0000 7fe36c117e00 1 Processor -- start 2026-02-25T11:51:21.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe36c117e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a02140 0x55d5f2a01880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe36c117e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a00320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe36c117e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a476f0 0x55d5f2a00860 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe36c117e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d5f2a00dd0 con 0x55d5f2a02a50 2026-02-25T11:51:21.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe36c117e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d5f2a00f40 con 0x55d5f2a02140 2026-02-25T11:51:21.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe36c117e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d5f2a01100 con 0x55d5f2a476f0 2026-02-25T11:51:21.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe362ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a00320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe362ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a00320 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51026/0 (socket says 10.20.193.28:51026) 2026-02-25T11:51:21.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe3637fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a02140 0x55d5f2a01880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe362ffd640 1 -- 10.20.193.28:0/820723866 learned_addr learned my addr 10.20.193.28:0/820723866 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:21.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe363fff640 1 --2- 10.20.193.28:0/820723866 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a476f0 0x55d5f2a00860 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe362ffd640 1 -- 10.20.193.28:0/820723866 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a476f0 msgr2=0x55d5f2a00860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:21.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe362ffd640 1 --2- 10.20.193.28:0/820723866 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a476f0 0x55d5f2a00860 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe362ffd640 1 -- 10.20.193.28:0/820723866 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a02140 msgr2=0x55d5f2a01880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:21.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe362ffd640 1 --2- 10.20.193.28:0/820723866 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a02140 0x55d5f2a01880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe362ffd640 1 -- 10.20.193.28:0/820723866 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5f2892350 con 0x55d5f2a02a50 2026-02-25T11:51:21.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe362ffd640 1 --2- 10.20.193.28:0/820723866 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a00320 secure :-1 s=READY pgs=532 gs=1 cs=0 l=1 c_cookie=12605810220030956054 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe354009df0 tx=0x7fe354002a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef0d59b4eef1216 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:21.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe3257fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:21.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.023+0000 7fe3257fa640 1 -- 10.20.193.28:0/820723866 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe354016040 con 0x55d5f2a02a50 2026-02-25T11:51:21.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe36c117e00 1 -- 10.20.193.28:0/820723866 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe354009b80 con 0x55d5f2a02a50 2026-02-25T11:51:21.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe36c117e00 10 client.16122 _ll_register_callbacks cb 0x55d5f288f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:21.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe36c117e00 1 -- 10.20.193.28:0/820723866 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d5f26664a0 con 0x55d5f2a02a50 2026-02-25T11:51:21.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe3257fa640 1 -- 10.20.193.28:0/820723866 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55d5f26664a0 con 0x55d5f2a02a50 2026-02-25T11:51:21.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe3257fa640 1 -- 10.20.193.28:0/820723866 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe354011d30 con 0x55d5f2a02a50 2026-02-25T11:51:21.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe3257fa640 1 -- 10.20.193.28:0/820723866 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fe35401e400 con 0x55d5f2a02a50 2026-02-25T11:51:21.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe3257fa640 1 -- 10.20.193.28:0/820723866 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe35401e630 con 0x55d5f2a02a50 2026-02-25T11:51:21.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe36c117e00 10 client.16122 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:21.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe36c117e00 1 -- 10.20.193.28:0/820723866 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55d5f289ebd0 con 0x55d5f2a02a50 2026-02-25T11:51:21.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe3257fa640 1 -- 10.20.193.28:0/820723866 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe354011440 con 0x55d5f2a02a50 2026-02-25T11:51:21.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe36c117e00 10 client.16122 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:21.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe36c117e00 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:21.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe36c117e00 20 client.16122 populate_metadata read hostname 'trial028' 2026-02-25T11:51:21.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe36c117e00 1 --2- 10.20.193.28:0/820723866 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d5f2a405d0 0x55d5f2b1e5d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe36c117e00 4 client.16122 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:51:21.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe36c117e00 1 -- 10.20.193.28:0/820723866 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe358097040 con 0x55d5f2a405d0 2026-02-25T11:51:21.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.024+0000 7fe3637fe640 1 --2- 10.20.193.28:0/820723866 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d5f2a405d0 0x55d5f2b1e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.025+0000 7fe3637fe640 1 --2- 10.20.193.28:0/820723866 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d5f2a405d0 0x55d5f2b1e5d0 crc :-1 s=READY pgs=75 gs=4 cs=0 l=0 c_cookie=9778390896131203417 s_cookie=14457806516882389198 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b3d2b63ed06159 server_cookie=c8a4726d261a38ce in_seq=0 out_seq=0 2026-02-25T11:51:21.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.025+0000 7fe3257fa640 10 client.16122 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:21.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.025+0000 7fe3257fa640 1 -- 10.20.193.28:0/820723866 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x7fe358097040 con 0x55d5f2a405d0 2026-02-25T11:51:21.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.025+0000 7fe3257fa640 10 client.16122 handle_command_reply: tid=0 2026-02-25T11:51:21.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.025+0000 7fe36c117e00 1 client.16122 shutdown 2026-02-25T11:51:21.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.025+0000 7fe36c117e00 1 -- 10.20.193.28:0/820723866 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 msgr2=0x55d5f2a00320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:21.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.025+0000 7fe36c117e00 1 --2- 10.20.193.28:0/820723866 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a00320 secure :-1 s=READY pgs=532 gs=1 cs=0 l=1 c_cookie=12605810220030956054 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe354009df0 tx=0x7fe354002a70 comp rx=0 tx=0).stop 2026-02-25T11:51:21.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe36c117e00 1 -- 10.20.193.28:0/820723866 shutdown_connections 2026-02-25T11:51:21.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe36c117e00 1 --2- 10.20.193.28:0/820723866 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d5f2a405d0 0x55d5f2b1e5d0 crc :-1 s=READY pgs=75 gs=4 cs=0 l=0 c_cookie=9778390896131203417 s_cookie=14457806516882389198 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe36c117e00 1 --2- 10.20.193.28:0/820723866 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d5f2a476f0 0x55d5f2a00860 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe36c117e00 1 --2- 10.20.193.28:0/820723866 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d5f2a02a50 0x55d5f2a00320 unknown :-1 s=CLOSED pgs=532 gs=1 cs=0 l=1 c_cookie=12605810220030956054 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe36c117e00 1 --2- 10.20.193.28:0/820723866 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d5f2a02140 0x55d5f2a01880 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe36c117e00 1 -- 10.20.193.28:0/820723866 >> 10.20.193.28:0/820723866 conn(0x55d5f2921d40 msgr2=0x55d5f2a47ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:21.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe3257fa640 0 client.16122 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:21.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe36c117e00 1 -- 10.20.193.28:0/820723866 shutdown_connections 2026-02-25T11:51:21.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe36c117e00 1 -- 10.20.193.28:0/820723866 wait complete. 2026-02-25T11:51:21.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe36c117e00 20 client.16122 trim_cache size 0 max 16384 2026-02-25T11:51:21.034 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:51:21.035 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:51:21.035 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:51:21.035 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:51:21.035 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:51:21.035 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:51:21.035 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:21.035 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:21.035 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:21.035 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:21.035 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:21.035 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:21.036 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:51:21.036 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:51:21.036 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:21.036 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:21.036 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:21.036 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:21.036 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:21.036 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:21.036 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:51:21.036 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:51:21.036 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:21.036 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:21.037 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:21.037 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:21.037 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:21.037 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:21.037 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:51:21.037 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:51:21.037 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:21.037 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:21.037 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:21.037 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:21.037 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:21.037 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:21.038 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:51:21.038 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:51:21.038 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:21.038 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:21.038 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:21.038 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:21.038 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:21.038 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:21.038 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:51:21.038 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:51:21.038 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:21.038 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:21.038 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:21.039 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:21.039 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:21.039 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:21.039 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:51:21.039 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:51:21.039 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:21.039 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:21.039 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:21.039 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:21.040 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:21.040 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:51:21.040 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:51:21.040 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:51:21.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe368e93640 1 -- 10.20.193.28:0/1337482537 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe3380b5750 msgr2=0x7fe3380d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:21.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe368e93640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe3380b5750 0x7fe3380d5c00 secure :-1 s=READY pgs=296 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe3500098a0 tx=0x7fe35003a040 comp rx=0 tx=0).stop 2026-02-25T11:51:21.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe368e93640 1 -- 10.20.193.28:0/1337482537 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3641135c0 msgr2=0x7fe3640f60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:21.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe368e93640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3641135c0 0x7fe3640f60a0 secure :-1 s=READY pgs=531 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe35800f8d0 tx=0x7fe35800fda0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe368e93640 1 -- 10.20.193.28:0/1337482537 shutdown_connections 2026-02-25T11:51:21.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe368e93640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe3380b5750 0x7fe3380d5c00 unknown :-1 s=CLOSED pgs=296 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe368e93640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe3641135c0 0x7fe3640f60a0 unknown :-1 s=CLOSED pgs=531 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe368e93640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe36410ddc0 0x7fe364173e90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe368e93640 1 --2- 10.20.193.28:0/1337482537 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe36410d3d0 0x7fe3640f59a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.026+0000 7fe368e93640 1 -- 10.20.193.28:0/1337482537 >> 10.20.193.28:0/1337482537 conn(0x7fe3640eaca0 msgr2=0x7fe36410f530 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:21.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.027+0000 7fe368e93640 1 -- 10.20.193.28:0/1337482537 shutdown_connections 2026-02-25T11:51:21.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.027+0000 7fe368e93640 1 -- 10.20.193.28:0/1337482537 wait complete. 2026-02-25T11:51:21.128 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:51:21.129 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:51:21.316 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:21.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.493+0000 7fc2bb258640 1 -- 10.20.193.28:0/3354876729 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc2b410bf30 msgr2=0x7fc2b4110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:21.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.493+0000 7fc2bb258640 1 --2- 10.20.193.28:0/3354876729 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc2b410bf30 0x7fc2b4110da0 secure :-1 s=READY pgs=533 gs=1 cs=0 l=1 c_cookie=6193725502999374959 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc29c009950 tx=0x7fc29c02f190 comp rx=0 tx=0).stop 2026-02-25T11:51:21.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.493+0000 7fc2bb258640 1 -- 10.20.193.28:0/3354876729 shutdown_connections 2026-02-25T11:51:21.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.493+0000 7fc2bb258640 1 --2- 10.20.193.28:0/3354876729 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc2b4111530 0x7fc2b4111f10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.493+0000 7fc2bb258640 1 --2- 10.20.193.28:0/3354876729 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc2b410bf30 0x7fc2b4110da0 unknown :-1 s=CLOSED pgs=533 gs=1 cs=0 l=1 c_cookie=6193725502999374959 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.493+0000 7fc2bb258640 1 --2- 10.20.193.28:0/3354876729 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc2b410b540 0x7fc2b410b910 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.493+0000 7fc2bb258640 1 -- 10.20.193.28:0/3354876729 >> 10.20.193.28:0/3354876729 conn(0x7fc2b40e93e0 msgr2=0x7fc2b41097c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:21.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.493+0000 7fc2bb258640 1 -- 10.20.193.28:0/3354876729 shutdown_connections 2026-02-25T11:51:21.495 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.493+0000 7fc2bb258640 1 -- 10.20.193.28:0/3354876729 wait complete. 2026-02-25T11:51:21.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.494+0000 7fc2bb258640 1 Processor -- start 2026-02-25T11:51:21.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.494+0000 7fc2bb258640 1 -- start start 2026-02-25T11:51:21.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.494+0000 7fc2bb258640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc2b410b540 0x7fc2b4123970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.494+0000 7fc2bb258640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc2b410bf30 0x7fc2b4123eb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.494+0000 7fc2bb258640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc2b4111530 0x7fc2b4129230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.494+0000 7fc2bb258640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4129850 con 0x7fc2b410b540 2026-02-25T11:51:21.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.494+0000 7fc2bb258640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41299c0 con 0x7fc2b410bf30 2026-02-25T11:51:21.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.494+0000 7fc2bb258640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4129b80 con 0x7fc2b4111530 2026-02-25T11:51:21.496 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2b8fcd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc2b410b540 0x7fc2b4123970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2b8fcd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc2b410b540 0x7fc2b4123970 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51052/0 (socket says 10.20.193.28:51052) 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2b8fcd640 1 -- 10.20.193.28:0/969431934 learned_addr learned my addr 10.20.193.28:0/969431934 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2b97ce640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc2b4111530 0x7fc2b4129230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2b8fcd640 1 -- 10.20.193.28:0/969431934 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc2b4111530 msgr2=0x7fc2b4129230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2abfff640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc2b410bf30 0x7fc2b4123eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2b8fcd640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc2b4111530 0x7fc2b4129230 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2b8fcd640 1 -- 10.20.193.28:0/969431934 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc2b410bf30 msgr2=0x7fc2b4123eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2b8fcd640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc2b410bf30 0x7fc2b4123eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2b8fcd640 1 -- 10.20.193.28:0/969431934 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c009570 con 0x7fc2b410b540 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2abfff640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc2b410bf30 0x7fc2b4123eb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2b8fcd640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc2b410b540 0x7fc2b4123970 secure :-1 s=READY pgs=534 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc2a4009480 tx=0x7fc2a400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2a9ffb640 1 -- 10.20.193.28:0/969431934 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc2a400cd40 con 0x7fc2b410b540 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2a9ffb640 1 -- 10.20.193.28:0/969431934 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc2a4004d10 con 0x7fc2b410b540 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2a9ffb640 1 -- 10.20.193.28:0/969431934 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc2a40106b0 con 0x7fc2b410b540 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2bb258640 1 -- 10.20.193.28:0/969431934 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b4129e60 con 0x7fc2b410b540 2026-02-25T11:51:21.497 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.495+0000 7fc2bb258640 1 -- 10.20.193.28:0/969431934 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b416a560 con 0x7fc2b410b540 2026-02-25T11:51:21.498 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.496+0000 7fc2bb258640 1 -- 10.20.193.28:0/969431934 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b412a080 con 0x7fc2b410b540 2026-02-25T11:51:21.500 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.498+0000 7fc2a9ffb640 1 -- 10.20.193.28:0/969431934 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fc2a401f060 con 0x7fc2b410b540 2026-02-25T11:51:21.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.500+0000 7fc2a9ffb640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc2900b56e0 0x7fc2900d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:21.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.500+0000 7fc2a9ffb640 1 -- 10.20.193.28:0/969431934 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc2a4014070 con 0x7fc2b410b540 2026-02-25T11:51:21.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.500+0000 7fc2abfff640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc2900b56e0 0x7fc2900d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:21.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.500+0000 7fc2a9ffb640 1 -- 10.20.193.28:0/969431934 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fc2a40e71e0 con 0x7fc2b410b540 2026-02-25T11:51:21.503 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.501+0000 7fc2abfff640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc2900b56e0 0x7fc2900d5b90 secure :-1 s=READY pgs=297 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc29c0098a0 tx=0x7fc29c031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:21.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.645+0000 7fc2bb258640 1 -- 10.20.193.28:0/969431934 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2b4124a80 con 0x7fc2b410b540 2026-02-25T11:51:21.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.645+0000 7fc2a9ffb640 1 -- 10.20.193.28:0/969431934 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7fc2a406fec0 con 0x7fc2b410b540 2026-02-25T11:51:21.648 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:51:21.648 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:51:21.649 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:51:21.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.647+0000 7fc2bb258640 1 -- 10.20.193.28:0/969431934 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc2900b56e0 msgr2=0x7fc2900d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:21.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.647+0000 7fc2bb258640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc2900b56e0 0x7fc2900d5b90 secure :-1 s=READY pgs=297 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc29c0098a0 tx=0x7fc29c031040 comp rx=0 tx=0).stop 2026-02-25T11:51:21.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.647+0000 7fc2bb258640 1 -- 10.20.193.28:0/969431934 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc2b410b540 msgr2=0x7fc2b4123970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:21.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.647+0000 7fc2bb258640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc2b410b540 0x7fc2b4123970 secure :-1 s=READY pgs=534 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc2a4009480 tx=0x7fc2a400ee60 comp rx=0 tx=0).stop 2026-02-25T11:51:21.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.649+0000 7fc2bb258640 1 -- 10.20.193.28:0/969431934 shutdown_connections 2026-02-25T11:51:21.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.649+0000 7fc2bb258640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc2900b56e0 0x7fc2900d5b90 unknown :-1 s=CLOSED pgs=297 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.649+0000 7fc2bb258640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc2b4111530 0x7fc2b4129230 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.649+0000 7fc2bb258640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc2b410bf30 0x7fc2b4123eb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.649+0000 7fc2bb258640 1 --2- 10.20.193.28:0/969431934 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc2b410b540 0x7fc2b4123970 unknown :-1 s=CLOSED pgs=534 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:21.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.649+0000 7fc2bb258640 1 -- 10.20.193.28:0/969431934 >> 10.20.193.28:0/969431934 conn(0x7fc2b40e93e0 msgr2=0x7fc2b410f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:21.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.649+0000 7fc2bb258640 1 -- 10.20.193.28:0/969431934 shutdown_connections 2026-02-25T11:51:21.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:21.649+0000 7fc2bb258640 1 -- 10.20.193.28:0/969431934 wait complete. 2026-02-25T11:51:21.752 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:51:21.935 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:22.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.107+0000 7fadc2027640 1 -- 10.20.193.28:0/510226393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fadbc10ddc0 msgr2=0x7fadbc112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.107+0000 7fadc2027640 1 --2- 10.20.193.28:0/510226393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fadbc10ddc0 0x7fadbc112e10 secure :-1 s=READY pgs=535 gs=1 cs=0 l=1 c_cookie=17432767318921495758 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fadb0009950 tx=0x7fadb002f190 comp rx=0 tx=0).stop 2026-02-25T11:51:22.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.107+0000 7fadc2027640 1 -- 10.20.193.28:0/510226393 shutdown_connections 2026-02-25T11:51:22.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.107+0000 7fadc2027640 1 --2- 10.20.193.28:0/510226393 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fadbc1135c0 0x7fadbc173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.107+0000 7fadc2027640 1 --2- 10.20.193.28:0/510226393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fadbc10ddc0 0x7fadbc112e10 unknown :-1 s=CLOSED pgs=535 gs=1 cs=0 l=1 c_cookie=17432767318921495758 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.107+0000 7fadc2027640 1 --2- 10.20.193.28:0/510226393 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fadbc10d3d0 0x7fadbc10d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.107+0000 7fadc2027640 1 -- 10.20.193.28:0/510226393 >> 10.20.193.28:0/510226393 conn(0x7fadbc0eaca0 msgr2=0x7fadbc10b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:22.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.107+0000 7fadc2027640 1 -- 10.20.193.28:0/510226393 shutdown_connections 2026-02-25T11:51:22.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.107+0000 7fadc2027640 1 -- 10.20.193.28:0/510226393 wait complete. 2026-02-25T11:51:22.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.108+0000 7fadc2027640 1 Processor -- start 2026-02-25T11:51:22.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.108+0000 7fadc2027640 1 -- start start 2026-02-25T11:51:22.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.108+0000 7fadc2027640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fadbc10d3d0 0x7fadbc062970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.108+0000 7fadc2027640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fadbc10ddc0 0x7fadbc173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.108+0000 7fadc2027640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fadbc1135c0 0x7fadbc060ef0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.108+0000 7fadc2027640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc0614f0 con 0x7fadbc1135c0 2026-02-25T11:51:22.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.108+0000 7fadc2027640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc061660 con 0x7fadbc10ddc0 2026-02-25T11:51:22.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.108+0000 7fadc2027640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc061820 con 0x7fadbc10d3d0 2026-02-25T11:51:22.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.108+0000 7fadc0824640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fadbc10ddc0 0x7fadbc173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.109+0000 7fadc0824640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fadbc10ddc0 0x7fadbc173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:34454/0 (socket says 10.20.193.28:34454) 2026-02-25T11:51:22.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.109+0000 7fadc0824640 1 -- 10.20.193.28:0/2833753551 learned_addr learned my addr 10.20.193.28:0/2833753551 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:22.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.109+0000 7fadc1826640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fadbc1135c0 0x7fadbc060ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.109+0000 7fadc1025640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fadbc10d3d0 0x7fadbc062970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.109+0000 7fadc0824640 1 -- 10.20.193.28:0/2833753551 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fadbc10d3d0 msgr2=0x7fadbc062970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.109+0000 7fadc0824640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fadbc10d3d0 0x7fadbc062970 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.109+0000 7fadc0824640 1 -- 10.20.193.28:0/2833753551 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fadbc1135c0 msgr2=0x7fadbc060ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.109+0000 7fadc0824640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fadbc1135c0 0x7fadbc060ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.109+0000 7fadc0824640 1 -- 10.20.193.28:0/2833753551 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb0009570 con 0x7fadbc10ddc0 2026-02-25T11:51:22.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.109+0000 7fadc0824640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fadbc10ddc0 0x7fadbc173e90 secure :-1 s=READY pgs=128 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fadb00098a0 tx=0x7fadb00028d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:22.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.109+0000 7fadaa7fc640 1 -- 10.20.193.28:0/2833753551 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fadb003d070 con 0x7fadbc10ddc0 2026-02-25T11:51:22.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.110+0000 7fadaa7fc640 1 -- 10.20.193.28:0/2833753551 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fadb0002a70 con 0x7fadbc10ddc0 2026-02-25T11:51:22.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.110+0000 7fadc2027640 1 -- 10.20.193.28:0/2833753551 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadbc061a40 con 0x7fadbc10ddc0 2026-02-25T11:51:22.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.110+0000 7fadc2027640 1 -- 10.20.193.28:0/2833753551 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc100340 con 0x7fadbc10ddc0 2026-02-25T11:51:22.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.110+0000 7fadaa7fc640 1 -- 10.20.193.28:0/2833753551 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fadb0041830 con 0x7fadbc10ddc0 2026-02-25T11:51:22.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.113+0000 7fadaa7fc640 1 -- 10.20.193.28:0/2833753551 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fadb00386a0 con 0x7fadbc10ddc0 2026-02-25T11:51:22.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadaa7fc640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fad980b56e0 0x7fad980d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadc52abe00 1 Processor -- start 2026-02-25T11:51:22.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadaa7fc640 1 -- 10.20.193.28:0/2833753551 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fadb010b090 con 0x7fadbc10ddc0 2026-02-25T11:51:22.117 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadc52abe00 1 -- start start 2026-02-25T11:51:22.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadc1025640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fad980b56e0 0x7fad980d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadc52abe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a726b30 0x555a3a726f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadc52abe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a727440 0x555a3a76bba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadc52abe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a76c0e0 0x555a3a78c4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadc52abe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555a3a63c8a0 con 0x555a3a727440 2026-02-25T11:51:22.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadc52abe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555a3a6385d0 con 0x555a3a726b30 2026-02-25T11:51:22.118 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadc52abe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555a3a632570 con 0x555a3a76c0e0 2026-02-25T11:51:22.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadc0824640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a727440 0x555a3a76bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadc0824640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a727440 0x555a3a76bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51082/0 (socket says 10.20.193.28:51082) 2026-02-25T11:51:22.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadc0824640 1 -- 10.20.193.28:0/2994608313 learned_addr learned my addr 10.20.193.28:0/2994608313 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:22.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.115+0000 7fadc1826640 1 --2- 10.20.193.28:0/2994608313 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a76c0e0 0x555a3a78c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc0824640 1 -- 10.20.193.28:0/2994608313 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a76c0e0 msgr2=0x555a3a78c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc0824640 1 --2- 10.20.193.28:0/2994608313 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a76c0e0 0x555a3a78c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc0824640 1 -- 10.20.193.28:0/2994608313 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a726b30 msgr2=0x555a3a726f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc0824640 1 --2- 10.20.193.28:0/2994608313 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a726b30 0x555a3a726f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc0824640 1 -- 10.20.193.28:0/2994608313 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a3a5ddd10 con 0x555a3a727440 2026-02-25T11:51:22.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc0824640 1 --2- 10.20.193.28:0/2994608313 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a727440 0x555a3a76bba0 secure :-1 s=READY pgs=536 gs=1 cs=0 l=1 c_cookie=8159318376135158428 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fadb0004030 tx=0x7fadb0038f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713bb8d54ada6a9c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:22.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc1025640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fad980b56e0 0x7fad980d5b90 secure :-1 s=READY pgs=298 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fadac004740 tx=0x7fadac002ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:22.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fad89ce5640 1 -- 10.20.193.28:0/2994608313 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fadb00d0d00 con 0x555a3a727440 2026-02-25T11:51:22.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fad89ce5640 1 -- 10.20.193.28:0/2994608313 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fadb00d7170 con 0x555a3a727440 2026-02-25T11:51:22.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fad89ce5640 1 -- 10.20.193.28:0/2994608313 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fadb00cf760 con 0x555a3a727440 2026-02-25T11:51:22.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc52abe00 1 -- 10.20.193.28:0/2994608313 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a727440 msgr2=0x555a3a76bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2994608313 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a727440 0x555a3a76bba0 secure :-1 s=READY pgs=536 gs=1 cs=0 l=1 c_cookie=8159318376135158428 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fadb0004030 tx=0x7fadb0038f30 comp rx=0 tx=0).stop 2026-02-25T11:51:22.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc52abe00 1 -- 10.20.193.28:0/2994608313 shutdown_connections 2026-02-25T11:51:22.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2994608313 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a76c0e0 0x555a3a78c4c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2994608313 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a727440 0x555a3a76bba0 unknown :-1 s=CLOSED pgs=536 gs=1 cs=0 l=1 c_cookie=8159318376135158428 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2994608313 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a726b30 0x555a3a726f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc52abe00 1 -- 10.20.193.28:0/2994608313 >> 10.20.193.28:0/2994608313 conn(0x555a3a6457a0 msgr2=0x555a3a725410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:22.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc52abe00 1 -- 10.20.193.28:0/2994608313 shutdown_connections 2026-02-25T11:51:22.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.116+0000 7fadc52abe00 1 -- 10.20.193.28:0/2994608313 wait complete. 2026-02-25T11:51:22.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.118+0000 7fadc52abe00 1 -- start start 2026-02-25T11:51:22.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.118+0000 7fadc52abe00 1 Processor -- start 2026-02-25T11:51:22.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.118+0000 7fadc52abe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a726b30 0x555a3a73eaa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.118+0000 7fadc52abe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a727440 0x555a3a73efe0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.118+0000 7fadc52abe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a7440b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.118+0000 7fadc52abe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555a3a7445f0 con 0x555a3a76c0e0 2026-02-25T11:51:22.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.118+0000 7fadc52abe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555a3a744760 con 0x555a3a726b30 2026-02-25T11:51:22.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.118+0000 7fadc52abe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555a3a744920 con 0x555a3a727440 2026-02-25T11:51:22.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.118+0000 7fadc1826640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a7440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.118+0000 7fadc1826640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a7440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51084/0 (socket says 10.20.193.28:51084) 2026-02-25T11:51:22.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.118+0000 7fadc1826640 1 -- 10.20.193.28:0/2446349211 learned_addr learned my addr 10.20.193.28:0/2446349211 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:22.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.118+0000 7fadc1025640 1 --2- 10.20.193.28:0/2446349211 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a726b30 0x555a3a73eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fadc0824640 1 --2- 10.20.193.28:0/2446349211 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a727440 0x555a3a73efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fadc1826640 1 -- 10.20.193.28:0/2446349211 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a727440 msgr2=0x555a3a73efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fadc1826640 1 --2- 10.20.193.28:0/2446349211 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a727440 0x555a3a73efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fadc1826640 1 -- 10.20.193.28:0/2446349211 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a726b30 msgr2=0x555a3a73eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fadc1826640 1 --2- 10.20.193.28:0/2446349211 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a726b30 0x555a3a73eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fadc1826640 1 -- 10.20.193.28:0/2446349211 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a3a5b63c0 con 0x555a3a76c0e0 2026-02-25T11:51:22.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fadc1826640 1 --2- 10.20.193.28:0/2446349211 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a7440b0 secure :-1 s=READY pgs=537 gs=1 cs=0 l=1 c_cookie=6372831541951498765 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fadb8003ae0 tx=0x7fadb80055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5870d885cf6c8a0d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:22.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fad76ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:22.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fad76ffd640 1 -- 10.20.193.28:0/2446349211 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fadb8012660 con 0x555a3a76c0e0 2026-02-25T11:51:22.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fad76ffd640 1 -- 10.20.193.28:0/2446349211 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fadb8008e40 con 0x555a3a76c0e0 2026-02-25T11:51:22.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fad76ffd640 1 -- 10.20.193.28:0/2446349211 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fadb8011da0 con 0x555a3a76c0e0 2026-02-25T11:51:22.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fadc52abe00 1 -- 10.20.193.28:0/2446349211 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc100340 con 0x555a3a76c0e0 2026-02-25T11:51:22.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fadc52abe00 10 client.16140 _ll_register_callbacks cb 0x555a3a5b3a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:22.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fad76ffd640 1 -- 10.20.193.28:0/2446349211 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fadb80103c0 con 0x555a3a76c0e0 2026-02-25T11:51:22.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.119+0000 7fadc52abe00 1 -- 10.20.193.28:0/2446349211 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555a3a2c86f0 con 0x555a3a76c0e0 2026-02-25T11:51:22.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.120+0000 7fad76ffd640 1 -- 10.20.193.28:0/2446349211 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fadb8011440 con 0x555a3a76c0e0 2026-02-25T11:51:22.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.120+0000 7fadc52abe00 10 client.16140 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:22.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.120+0000 7fadc52abe00 1 -- 10.20.193.28:0/2446349211 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x555a3a5c80e0 con 0x555a3a76c0e0 2026-02-25T11:51:22.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.120+0000 7fad76ffd640 1 -- 10.20.193.28:0/2446349211 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fadb80115a0 con 0x555a3a76c0e0 2026-02-25T11:51:22.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.120+0000 7fadc52abe00 10 client.16140 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:22.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.120+0000 7fadc52abe00 10 client.16140 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:51:22.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.120+0000 7fadc52abe00 20 client.16140 populate_metadata read hostname 'trial028' 2026-02-25T11:51:22.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.120+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2446349211 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x555a3a769470 0x555a3a769910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.120+0000 7fadc52abe00 4 client.16140 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:51:22.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.120+0000 7fadc52abe00 1 -- 10.20.193.28:0/2446349211 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a3a604fc0 con 0x555a3a769470 2026-02-25T11:51:22.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.120+0000 7fadc1025640 1 --2- 10.20.193.28:0/2446349211 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x555a3a769470 0x555a3a769910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.121+0000 7fadc1025640 1 --2- 10.20.193.28:0/2446349211 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x555a3a769470 0x555a3a769910 crc :-1 s=READY pgs=35 gs=4 cs=0 l=0 c_cookie=11060614075725629727 s_cookie=8583177236476129346 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=997f31e6754fc91f server_cookie=771d92389060f042 in_seq=0 out_seq=0 2026-02-25T11:51:22.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.121+0000 7fad76ffd640 10 client.16140 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:51:22.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.122+0000 7fad76ffd640 1 -- 10.20.193.28:0/2446349211 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x555a3a604fc0 con 0x555a3a769470 2026-02-25T11:51:22.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.122+0000 7fad76ffd640 10 client.16140 handle_command_reply: tid=0 2026-02-25T11:51:22.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.122+0000 7fadc52abe00 1 client.16140 shutdown 2026-02-25T11:51:22.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.122+0000 7fadc52abe00 1 -- 10.20.193.28:0/2446349211 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 msgr2=0x555a3a7440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.122+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2446349211 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a7440b0 secure :-1 s=READY pgs=537 gs=1 cs=0 l=1 c_cookie=6372831541951498765 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fadb8003ae0 tx=0x7fadb80055c0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.123+0000 7fadc52abe00 1 -- 10.20.193.28:0/2446349211 shutdown_connections 2026-02-25T11:51:22.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.123+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2446349211 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x555a3a769470 0x555a3a769910 crc :-1 s=READY pgs=35 gs=4 cs=0 l=0 c_cookie=11060614075725629727 s_cookie=8583177236476129346 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.123+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2446349211 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a7440b0 unknown :-1 s=CLOSED pgs=537 gs=1 cs=0 l=1 c_cookie=6372831541951498765 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.123+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2446349211 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a727440 0x555a3a73efe0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.123+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2446349211 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a726b30 0x555a3a73eaa0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.123+0000 7fad76ffd640 0 client.16140 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:51:22.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.123+0000 7fadc52abe00 1 -- 10.20.193.28:0/2446349211 >> 10.20.193.28:0/2446349211 conn(0x555a3a6457a0 msgr2=0x555a3a725b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:22.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.123+0000 7fadc52abe00 1 -- 10.20.193.28:0/2446349211 shutdown_connections 2026-02-25T11:51:22.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.123+0000 7fadc52abe00 1 -- 10.20.193.28:0/2446349211 wait complete. 2026-02-25T11:51:22.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.123+0000 7fadc52abe00 20 client.16140 trim_cache size 0 max 16384 2026-02-25T11:51:22.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc52abe00 1 Processor -- start 2026-02-25T11:51:22.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc52abe00 1 -- start start 2026-02-25T11:51:22.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc52abe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a726b30 0x555a3a73bee0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc52abe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a727440 0x555a3a73c420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc52abe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a764590 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc52abe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555a3a6301a0 con 0x555a3a76c0e0 2026-02-25T11:51:22.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc52abe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555a3a2f0700 con 0x555a3a727440 2026-02-25T11:51:22.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc52abe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555a3a632570 con 0x555a3a726b30 2026-02-25T11:51:22.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc1826640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a764590 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc1826640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a764590 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51098/0 (socket says 10.20.193.28:51098) 2026-02-25T11:51:22.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc1826640 1 -- 10.20.193.28:0/2578582007 learned_addr learned my addr 10.20.193.28:0/2578582007 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:22.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc1025640 1 --2- 10.20.193.28:0/2578582007 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a726b30 0x555a3a73bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc0824640 1 --2- 10.20.193.28:0/2578582007 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a727440 0x555a3a73c420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc1826640 1 -- 10.20.193.28:0/2578582007 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a726b30 msgr2=0x555a3a73bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc1826640 1 --2- 10.20.193.28:0/2578582007 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a726b30 0x555a3a73bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc1826640 1 -- 10.20.193.28:0/2578582007 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a727440 msgr2=0x555a3a73c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc1826640 1 --2- 10.20.193.28:0/2578582007 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a727440 0x555a3a73c420 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc1826640 1 -- 10.20.193.28:0/2578582007 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a3a731410 con 0x555a3a76c0e0 2026-02-25T11:51:22.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.130+0000 7fadc0824640 1 --2- 10.20.193.28:0/2578582007 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a727440 0x555a3a73c420 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:51:22.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.131+0000 7fadc1826640 1 --2- 10.20.193.28:0/2578582007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a764590 secure :-1 s=READY pgs=538 gs=1 cs=0 l=1 c_cookie=8582284254247710656 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fadb8011920 tx=0x7fadb801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771a660eefdcefc0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:22.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.131+0000 7fad894e4640 1 -- 10.20.193.28:0/2578582007 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fadb802c070 con 0x555a3a76c0e0 2026-02-25T11:51:22.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.131+0000 7fad894e4640 1 -- 10.20.193.28:0/2578582007 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x555a3a2c86f0 con 0x555a3a76c0e0 2026-02-25T11:51:22.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.131+0000 7fad894e4640 1 -- 10.20.193.28:0/2578582007 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fadb8011440 con 0x555a3a76c0e0 2026-02-25T11:51:22.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.131+0000 7fadc52abe00 1 -- 10.20.193.28:0/2578582007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 msgr2=0x555a3a764590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.131+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2578582007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a764590 secure :-1 s=READY pgs=538 gs=1 cs=0 l=1 c_cookie=8582284254247710656 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fadb8011920 tx=0x7fadb801f000 comp rx=0 tx=0).stop 2026-02-25T11:51:22.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.131+0000 7fadc52abe00 1 -- 10.20.193.28:0/2578582007 shutdown_connections 2026-02-25T11:51:22.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.131+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2578582007 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a764590 unknown :-1 s=CLOSED pgs=538 gs=1 cs=0 l=1 c_cookie=8582284254247710656 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.131+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2578582007 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a727440 0x555a3a73c420 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.131+0000 7fadc52abe00 1 --2- 10.20.193.28:0/2578582007 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a726b30 0x555a3a73bee0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.131+0000 7fadc52abe00 1 -- 10.20.193.28:0/2578582007 >> 10.20.193.28:0/2578582007 conn(0x555a3a6457a0 msgr2=0x555a3a73d490 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:22.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.131+0000 7fadc52abe00 1 -- 10.20.193.28:0/2578582007 shutdown_connections 2026-02-25T11:51:22.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.131+0000 7fadc52abe00 1 -- 10.20.193.28:0/2578582007 wait complete. 2026-02-25T11:51:22.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.132+0000 7fadc52abe00 1 -- start start 2026-02-25T11:51:22.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.132+0000 7fadc52abe00 1 Processor -- start 2026-02-25T11:51:22.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.132+0000 7fadc52abe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a726b30 0x555a3a744ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.132+0000 7fadc52abe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a727440 0x555a3a745530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.132+0000 7fadc52abe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a749e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.132+0000 7fadc52abe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555a3a745c90 con 0x555a3a76c0e0 2026-02-25T11:51:22.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.132+0000 7fadc52abe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555a3a74a3d0 con 0x555a3a727440 2026-02-25T11:51:22.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.132+0000 7fadc52abe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555a3a74a540 con 0x555a3a726b30 2026-02-25T11:51:22.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.132+0000 7fadc1826640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a749e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.132+0000 7fadc1025640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a726b30 0x555a3a744ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.132+0000 7fadc1826640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a749e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51108/0 (socket says 10.20.193.28:51108) 2026-02-25T11:51:22.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fadc1826640 1 -- 10.20.193.28:0/1394784631 learned_addr learned my addr 10.20.193.28:0/1394784631 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:22.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fadc0824640 1 --2- 10.20.193.28:0/1394784631 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a727440 0x555a3a745530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fadc1826640 1 -- 10.20.193.28:0/1394784631 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a726b30 msgr2=0x555a3a744ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fadc1826640 1 --2- 10.20.193.28:0/1394784631 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a726b30 0x555a3a744ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fadc1826640 1 -- 10.20.193.28:0/1394784631 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a727440 msgr2=0x555a3a745530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fadc1826640 1 --2- 10.20.193.28:0/1394784631 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a727440 0x555a3a745530 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fadc1826640 1 -- 10.20.193.28:0/1394784631 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a3a5c2b60 con 0x555a3a76c0e0 2026-02-25T11:51:22.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fadc1025640 1 --2- 10.20.193.28:0/1394784631 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a726b30 0x555a3a744ff0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:22.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fadc1826640 1 --2- 10.20.193.28:0/1394784631 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a749e90 secure :-1 s=READY pgs=539 gs=1 cs=0 l=1 c_cookie=3460958925884246466 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fadb8006910 tx=0x7fadb8010680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3007cbce5f8c49c2 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:22.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fad76ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:22.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fad76ffd640 1 -- 10.20.193.28:0/1394784631 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fadb802c070 con 0x555a3a76c0e0 2026-02-25T11:51:22.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fadc52abe00 1 -- 10.20.193.28:0/1394784631 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a3a5c80e0 con 0x555a3a76c0e0 2026-02-25T11:51:22.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fadc52abe00 10 client.16146 _ll_register_callbacks cb 0x555a3a5b3a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:22.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fadc52abe00 1 -- 10.20.193.28:0/1394784631 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadb00d7170 con 0x555a3a76c0e0 2026-02-25T11:51:22.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fad76ffd640 1 -- 10.20.193.28:0/1394784631 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fadb00d7170 con 0x555a3a76c0e0 2026-02-25T11:51:22.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.133+0000 7fad76ffd640 1 -- 10.20.193.28:0/1394784631 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fadb8010c80 con 0x555a3a76c0e0 2026-02-25T11:51:22.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.134+0000 7fad76ffd640 1 -- 10.20.193.28:0/1394784631 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fadb8010e10 con 0x555a3a76c0e0 2026-02-25T11:51:22.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.134+0000 7fad76ffd640 1 -- 10.20.193.28:0/1394784631 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fadb801b9e0 con 0x555a3a76c0e0 2026-02-25T11:51:22.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.134+0000 7fadc52abe00 10 client.16146 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:22.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.134+0000 7fadc52abe00 1 -- 10.20.193.28:0/1394784631 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x555a3a5b66f0 con 0x555a3a76c0e0 2026-02-25T11:51:22.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.134+0000 7fad76ffd640 1 -- 10.20.193.28:0/1394784631 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fadb801bb40 con 0x555a3a76c0e0 2026-02-25T11:51:22.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.134+0000 7fadc52abe00 10 client.16146 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:22.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.134+0000 7fadc52abe00 10 client.16146 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:51:22.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.134+0000 7fadc52abe00 20 client.16146 populate_metadata read hostname 'trial028' 2026-02-25T11:51:22.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.134+0000 7fadc52abe00 1 --2- 10.20.193.28:0/1394784631 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x555a3a819560 0x555a3a839a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.134+0000 7fadc52abe00 4 client.16146 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:51:22.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.134+0000 7fadc52abe00 1 -- 10.20.193.28:0/1394784631 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555a3a2c5f40 con 0x555a3a819560 2026-02-25T11:51:22.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.135+0000 7fadc1025640 1 --2- 10.20.193.28:0/1394784631 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x555a3a819560 0x555a3a839a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fadc1025640 1 --2- 10.20.193.28:0/1394784631 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x555a3a819560 0x555a3a839a10 crc :-1 s=READY pgs=36 gs=4 cs=0 l=0 c_cookie=10217693415785862751 s_cookie=6376730719869237995 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8dcc8a06a7c3965f server_cookie=587eb2cdea207aeb in_seq=0 out_seq=0 2026-02-25T11:51:22.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fad76ffd640 10 client.16146 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:51:22.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fad76ffd640 1 -- 10.20.193.28:0/1394784631 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555a3a2c5f40 con 0x555a3a819560 2026-02-25T11:51:22.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fad76ffd640 10 client.16146 handle_command_reply: tid=0 2026-02-25T11:51:22.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fadc52abe00 1 client.16146 shutdown 2026-02-25T11:51:22.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fadc52abe00 1 -- 10.20.193.28:0/1394784631 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 msgr2=0x555a3a749e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fadc52abe00 1 --2- 10.20.193.28:0/1394784631 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a749e90 secure :-1 s=READY pgs=539 gs=1 cs=0 l=1 c_cookie=3460958925884246466 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fadb8006910 tx=0x7fadb8010680 comp rx=0 tx=0).stop 2026-02-25T11:51:22.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fadc52abe00 1 -- 10.20.193.28:0/1394784631 shutdown_connections 2026-02-25T11:51:22.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fadc52abe00 1 --2- 10.20.193.28:0/1394784631 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x555a3a819560 0x555a3a839a10 crc :-1 s=READY pgs=36 gs=4 cs=0 l=0 c_cookie=10217693415785862751 s_cookie=6376730719869237995 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fadc52abe00 1 --2- 10.20.193.28:0/1394784631 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555a3a76c0e0 0x555a3a749e90 unknown :-1 s=CLOSED pgs=539 gs=1 cs=0 l=1 c_cookie=3460958925884246466 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fadc52abe00 1 --2- 10.20.193.28:0/1394784631 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555a3a727440 0x555a3a745530 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fadc52abe00 1 --2- 10.20.193.28:0/1394784631 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555a3a726b30 0x555a3a744ff0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fadc52abe00 1 -- 10.20.193.28:0/1394784631 >> 10.20.193.28:0/1394784631 conn(0x555a3a6457a0 msgr2=0x555a3a73aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:22.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fad76ffd640 0 client.16146 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:51:22.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fadc52abe00 1 -- 10.20.193.28:0/1394784631 shutdown_connections 2026-02-25T11:51:22.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fadc52abe00 1 -- 10.20.193.28:0/1394784631 wait complete. 2026-02-25T11:51:22.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.136+0000 7fadc52abe00 20 client.16146 trim_cache size 0 max 16384 2026-02-25T11:51:22.145 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:51:22.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.137+0000 7fadc2027640 1 -- 10.20.193.28:0/2833753551 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fad980b56e0 msgr2=0x7fad980d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.137+0000 7fadc2027640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fad980b56e0 0x7fad980d5b90 secure :-1 s=READY pgs=298 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fadac004740 tx=0x7fadac002ea0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.137+0000 7fadc2027640 1 -- 10.20.193.28:0/2833753551 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fadbc10ddc0 msgr2=0x7fadbc173e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.137+0000 7fadc2027640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fadbc10ddc0 0x7fadbc173e90 secure :-1 s=READY pgs=128 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fadb00098a0 tx=0x7fadb00028d0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.137+0000 7fadc2027640 1 -- 10.20.193.28:0/2833753551 shutdown_connections 2026-02-25T11:51:22.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.137+0000 7fadc2027640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fad980b56e0 0x7fad980d5b90 unknown :-1 s=CLOSED pgs=298 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.137+0000 7fadc2027640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fadbc1135c0 0x7fadbc060ef0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.137+0000 7fadc2027640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fadbc10ddc0 0x7fadbc173e90 unknown :-1 s=CLOSED pgs=128 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.137+0000 7fadc2027640 1 --2- 10.20.193.28:0/2833753551 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fadbc10d3d0 0x7fadbc062970 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.137+0000 7fadc2027640 1 -- 10.20.193.28:0/2833753551 >> 10.20.193.28:0/2833753551 conn(0x7fadbc0eaca0 msgr2=0x7fadbc10fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:22.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.137+0000 7fadc2027640 1 -- 10.20.193.28:0/2833753551 shutdown_connections 2026-02-25T11:51:22.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.137+0000 7fadc2027640 1 -- 10.20.193.28:0/2833753551 wait complete. 2026-02-25T11:51:22.233 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:22 trial028 ceph-mon[39224]: pgmap v320: 97 pgs: 97 active+clean; 2.7 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:22.234 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:22 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/969431934' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:51:22.235 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:51:22.419 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:22.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:22 trial167 ceph-mon[43392]: pgmap v320: 97 pgs: 97 active+clean; 2.7 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:22.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:22 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/969431934' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:51:22.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:22 trial119 ceph-mon[42710]: pgmap v320: 97 pgs: 97 active+clean; 2.7 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:22.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:22 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/969431934' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:51:22.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.594+0000 7fa590dca640 1 -- 10.20.193.28:0/955022652 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa58c115930 msgr2=0x7fa58c176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.594+0000 7fa590dca640 1 --2- 10.20.193.28:0/955022652 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa58c115930 0x7fa58c176300 secure :-1 s=READY pgs=540 gs=1 cs=0 l=1 c_cookie=15567076766018032866 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa57c00aa30 tx=0x7fa57c030270 comp rx=0 tx=0).stop 2026-02-25T11:51:22.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.594+0000 7fa590dca640 1 -- 10.20.193.28:0/955022652 shutdown_connections 2026-02-25T11:51:22.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.594+0000 7fa590dca640 1 --2- 10.20.193.28:0/955022652 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa58c115930 0x7fa58c176300 unknown :-1 s=CLOSED pgs=540 gs=1 cs=0 l=1 c_cookie=15567076766018032866 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.594+0000 7fa590dca640 1 --2- 10.20.193.28:0/955022652 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa58c110130 0x7fa58c115180 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.594+0000 7fa590dca640 1 --2- 10.20.193.28:0/955022652 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa58c10f740 0x7fa58c10fb10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.594+0000 7fa590dca640 1 -- 10.20.193.28:0/955022652 >> 10.20.193.28:0/955022652 conn(0x7fa58c0ecff0 msgr2=0x7fa58c10d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:22.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.594+0000 7fa590dca640 1 -- 10.20.193.28:0/955022652 shutdown_connections 2026-02-25T11:51:22.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.594+0000 7fa590dca640 1 -- 10.20.193.28:0/955022652 wait complete. 2026-02-25T11:51:22.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.595+0000 7fa590dca640 1 Processor -- start 2026-02-25T11:51:22.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.595+0000 7fa590dca640 1 -- start start 2026-02-25T11:51:22.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.595+0000 7fa590dca640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa58c10f740 0x7fa58c11e110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.595+0000 7fa590dca640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa58c110130 0x7fa58c11e650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.595+0000 7fa590dca640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa58c115930 0x7fa58c123a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.595+0000 7fa590dca640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c124040 con 0x7fa58c115930 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa590dca640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c1241b0 con 0x7fa58c10f740 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa590dca640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa58c124370 con 0x7fa58c110130 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa58ad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa58c115930 0x7fa58c123a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa58ad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa58c115930 0x7fa58c123a20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51130/0 (socket says 10.20.193.28:51130) 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa58ad76640 1 -- 10.20.193.28:0/521221784 learned_addr learned my addr 10.20.193.28:0/521221784 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa589d74640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa58c110130 0x7fa58c11e650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa58a575640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa58c10f740 0x7fa58c11e110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa58ad76640 1 -- 10.20.193.28:0/521221784 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa58c110130 msgr2=0x7fa58c11e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa58ad76640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa58c110130 0x7fa58c11e650 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa58ad76640 1 -- 10.20.193.28:0/521221784 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa58c10f740 msgr2=0x7fa58c11e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa58ad76640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa58c10f740 0x7fa58c11e110 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa58ad76640 1 -- 10.20.193.28:0/521221784 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c00a650 con 0x7fa58c115930 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa58ad76640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa58c115930 0x7fa58c123a20 secure :-1 s=READY pgs=541 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa57c006a50 tx=0x7fa57c030ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa5737fe640 1 -- 10.20.193.28:0/521221784 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa57c03e070 con 0x7fa58c115930 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa5737fe640 1 -- 10.20.193.28:0/521221784 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa57c039de0 con 0x7fa58c115930 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa590dca640 1 -- 10.20.193.28:0/521221784 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa58c1245f0 con 0x7fa58c115930 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa5737fe640 1 -- 10.20.193.28:0/521221784 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa57c042820 con 0x7fa58c115930 2026-02-25T11:51:22.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.596+0000 7fa590dca640 1 -- 10.20.193.28:0/521221784 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa58c164d50 con 0x7fa58c115930 2026-02-25T11:51:22.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.599+0000 7fa5737fe640 1 -- 10.20.193.28:0/521221784 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fa57c04c440 con 0x7fa58c115930 2026-02-25T11:51:22.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.601+0000 7fa5737fe640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5640b5760 0x7fa5640d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.601+0000 7fa5737fe640 1 -- 10.20.193.28:0/521221784 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa57c10c6b0 con 0x7fa58c115930 2026-02-25T11:51:22.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.601+0000 7fa58a575640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5640b5760 0x7fa5640d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.602+0000 7fa58a575640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5640b5760 0x7fa5640d5c10 secure :-1 s=READY pgs=299 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa580004740 tx=0x7fa580005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:22.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.602+0000 7fa592dc5e00 1 Processor -- start 2026-02-25T11:51:22.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.602+0000 7fa592dc5e00 1 -- start start 2026-02-25T11:51:22.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.602+0000 7fa592dc5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b339cab30 0x557b339caf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.602+0000 7fa592dc5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cb440 0x557b33a0fba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.602+0000 7fa592dc5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b33a100e0 0x557b33a304c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.602+0000 7fa592dc5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x557b338e08a0 con 0x557b339cab30 2026-02-25T11:51:22.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.602+0000 7fa592dc5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x557b338dc5d0 con 0x557b339cb440 2026-02-25T11:51:22.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.602+0000 7fa592dc5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x557b338d6570 con 0x557b33a100e0 2026-02-25T11:51:22.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.603+0000 7fa58a575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b339cab30 0x557b339caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.603+0000 7fa58a575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b339cab30 0x557b339caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51140/0 (socket says 10.20.193.28:51140) 2026-02-25T11:51:22.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.603+0000 7fa58a575640 1 -- 10.20.193.28:0/1305293109 learned_addr learned my addr 10.20.193.28:0/1305293109 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:22.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.603+0000 7fa589d74640 1 --2- 10.20.193.28:0/1305293109 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cb440 0x557b33a0fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.603+0000 7fa58ad76640 1 --2- 10.20.193.28:0/1305293109 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b33a100e0 0x557b33a304c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.603+0000 7fa58ad76640 1 -- 10.20.193.28:0/1305293109 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cb440 msgr2=0x557b33a0fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.603+0000 7fa58ad76640 1 --2- 10.20.193.28:0/1305293109 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cb440 0x557b33a0fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.603+0000 7fa58ad76640 1 -- 10.20.193.28:0/1305293109 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b339cab30 msgr2=0x557b339caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.603+0000 7fa58ad76640 1 --2- 10.20.193.28:0/1305293109 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b339cab30 0x557b339caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.603+0000 7fa58ad76640 1 -- 10.20.193.28:0/1305293109 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b33881d10 con 0x557b33a100e0 2026-02-25T11:51:22.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.603+0000 7fa58a575640 1 --2- 10.20.193.28:0/1305293109 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b339cab30 0x557b339caf00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:51:22.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.603+0000 7fa58ad76640 1 --2- 10.20.193.28:0/1305293109 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b33a100e0 0x557b33a304c0 secure :-1 s=READY pgs=152 gs=3 cs=0 l=1 c_cookie=53278841992071271 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa57c10a170 tx=0x7fa57c030270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd48d312fb4867 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:22.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.603+0000 7fa570ff9640 1 -- 10.20.193.28:0/1305293109 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa57c0d4300 con 0x557b33a100e0 2026-02-25T11:51:22.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.604+0000 7fa570ff9640 1 -- 10.20.193.28:0/1305293109 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa57c0d9620 con 0x557b33a100e0 2026-02-25T11:51:22.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.604+0000 7fa570ff9640 1 -- 10.20.193.28:0/1305293109 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa57c0053e0 con 0x557b33a100e0 2026-02-25T11:51:22.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.604+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1305293109 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b33a100e0 msgr2=0x557b33a304c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.604+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1305293109 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b33a100e0 0x557b33a304c0 secure :-1 s=READY pgs=152 gs=3 cs=0 l=1 c_cookie=53278841992071271 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa57c10a170 tx=0x7fa57c030270 comp rx=0 tx=0).stop 2026-02-25T11:51:22.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.604+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1305293109 shutdown_connections 2026-02-25T11:51:22.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.604+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1305293109 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b33a100e0 0x557b33a304c0 unknown :-1 s=CLOSED pgs=152 gs=3 cs=0 l=1 c_cookie=53278841992071271 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.604+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1305293109 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cb440 0x557b33a0fba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.604+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1305293109 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b339cab30 0x557b339caf00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.604+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1305293109 >> 10.20.193.28:0/1305293109 conn(0x557b338e97a0 msgr2=0x557b339c9410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:22.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.604+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1305293109 shutdown_connections 2026-02-25T11:51:22.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.604+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1305293109 wait complete. 2026-02-25T11:51:22.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.605+0000 7fa592dc5e00 1 -- start start 2026-02-25T11:51:22.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.606+0000 7fa592dc5e00 1 Processor -- start 2026-02-25T11:51:22.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.606+0000 7fa592dc5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cab30 0x557b339e2ae0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.606+0000 7fa592dc5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b339e3020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.606+0000 7fa592dc5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b33a100e0 0x557b339e8010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.606+0000 7fa592dc5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x557b339e8550 con 0x557b33a100e0 2026-02-25T11:51:22.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.606+0000 7fa592dc5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x557b339e86c0 con 0x557b339cab30 2026-02-25T11:51:22.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.606+0000 7fa592dc5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x557b339e8880 con 0x557b339cb440 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.606+0000 7fa589d74640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b339e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.606+0000 7fa589d74640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b339e3020 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44384/0 (socket says 10.20.193.28:44384) 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.606+0000 7fa589d74640 1 -- 10.20.193.28:0/1398591029 learned_addr learned my addr 10.20.193.28:0/1398591029 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.606+0000 7fa58ad76640 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b33a100e0 0x557b339e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.606+0000 7fa58a575640 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cab30 0x557b339e2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa58ad76640 1 -- 10.20.193.28:0/1398591029 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 msgr2=0x557b339e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa58ad76640 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b339e3020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa58ad76640 1 -- 10.20.193.28:0/1398591029 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cab30 msgr2=0x557b339e2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa58ad76640 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cab30 0x557b339e2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa58ad76640 1 -- 10.20.193.28:0/1398591029 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa58c164d50 con 0x557b33a100e0 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa58a575640 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cab30 0x557b339e2ae0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa589d74640 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b339e3020 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa58ad76640 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b33a100e0 0x557b339e8010 secure :-1 s=READY pgs=542 gs=1 cs=0 l=1 c_cookie=6558211146992356605 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa57c006c10 tx=0x7fa57c0bd190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b03724a927c8cfd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:22.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa53bfff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa53bfff640 1 -- 10.20.193.28:0/1398591029 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa57c0d36c0 con 0x557b33a100e0 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa53bfff640 1 -- 10.20.193.28:0/1398591029 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa57c0dea40 con 0x557b33a100e0 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1398591029 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5800098c0 con 0x557b33a100e0 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa53bfff640 1 -- 10.20.193.28:0/1398591029 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa57c0de830 con 0x557b33a100e0 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa592dc5e00 10 client.16158 _ll_register_callbacks cb 0x557b33857a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.607+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1398591029 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557b3356c6f0 con 0x557b33a100e0 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.608+0000 7fa53bfff640 1 -- 10.20.193.28:0/1398591029 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa57c0d4800 con 0x557b33a100e0 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.608+0000 7fa53bfff640 1 -- 10.20.193.28:0/1398591029 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa57c098a90 con 0x557b33a100e0 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.608+0000 7fa592dc5e00 10 client.16158 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.608+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1398591029 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x557b3385a3c0 con 0x557b33a100e0 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.608+0000 7fa53bfff640 1 -- 10.20.193.28:0/1398591029 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa57c0d18e0 con 0x557b33a100e0 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.608+0000 7fa592dc5e00 10 client.16158 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.608+0000 7fa592dc5e00 10 client.16158 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.608+0000 7fa592dc5e00 20 client.16158 populate_metadata read hostname 'trial028' 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.608+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x557b33a0d000 0x557b33a0d4a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.608+0000 7fa592dc5e00 4 client.16158 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:51:22.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.608+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1398591029 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b338a8fc0 con 0x557b33a0d000 2026-02-25T11:51:22.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.608+0000 7fa58a575640 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x557b33a0d000 0x557b33a0d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.609+0000 7fa58a575640 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x557b33a0d000 0x557b33a0d4a0 crc :-1 s=READY pgs=36 gs=4 cs=0 l=0 c_cookie=3484569742023726103 s_cookie=17928046924692553090 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=305badb7e4661417 server_cookie=f8cd37b0eed57d82 in_seq=0 out_seq=0 2026-02-25T11:51:22.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.609+0000 7fa53bfff640 10 client.16158 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:51:22.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.609+0000 7fa53bfff640 1 -- 10.20.193.28:0/1398591029 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x557b338a8fc0 con 0x557b33a0d000 2026-02-25T11:51:22.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.609+0000 7fa53bfff640 10 client.16158 handle_command_reply: tid=0 2026-02-25T11:51:22.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.609+0000 7fa592dc5e00 1 client.16158 shutdown 2026-02-25T11:51:22.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.609+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1398591029 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b33a100e0 msgr2=0x557b339e8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.609+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b33a100e0 0x557b339e8010 secure :-1 s=READY pgs=542 gs=1 cs=0 l=1 c_cookie=6558211146992356605 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa57c006c10 tx=0x7fa57c0bd190 comp rx=0 tx=0).stop 2026-02-25T11:51:22.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.610+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1398591029 shutdown_connections 2026-02-25T11:51:22.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.610+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x557b33a0d000 0x557b33a0d4a0 crc :-1 s=READY pgs=36 gs=4 cs=0 l=0 c_cookie=3484569742023726103 s_cookie=17928046924692553090 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.610+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b33a100e0 0x557b339e8010 unknown :-1 s=CLOSED pgs=542 gs=1 cs=0 l=1 c_cookie=6558211146992356605 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.610+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b339e3020 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.610+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1398591029 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cab30 0x557b339e2ae0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.610+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1398591029 >> 10.20.193.28:0/1398591029 conn(0x557b338e97a0 msgr2=0x557b339c9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:22.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.610+0000 7fa53bfff640 0 client.16158 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:51:22.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.610+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1398591029 shutdown_connections 2026-02-25T11:51:22.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.610+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1398591029 wait complete. 2026-02-25T11:51:22.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.610+0000 7fa592dc5e00 20 client.16158 trim_cache size 0 max 16384 2026-02-25T11:51:22.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.617+0000 7fa592dc5e00 1 Processor -- start 2026-02-25T11:51:22.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.617+0000 7fa592dc5e00 1 -- start start 2026-02-25T11:51:22.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.617+0000 7fa592dc5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b339cab30 0x557b339cd820 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.617+0000 7fa592dc5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b33a0ed10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.617+0000 7fa592dc5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b33a100e0 0x557b33a0f250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.617+0000 7fa592dc5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x557b338d41a0 con 0x557b339cab30 2026-02-25T11:51:22.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.617+0000 7fa592dc5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x557b33594700 con 0x557b33a100e0 2026-02-25T11:51:22.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.617+0000 7fa592dc5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x557b33a0db60 con 0x557b339cb440 2026-02-25T11:51:22.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.617+0000 7fa58a575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b339cab30 0x557b339cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.617+0000 7fa589d74640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b33a0ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.617+0000 7fa589d74640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b33a0ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44388/0 (socket says 10.20.193.28:44388) 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.617+0000 7fa589d74640 1 -- 10.20.193.28:0/3861743580 learned_addr learned my addr 10.20.193.28:0/3861743580 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.618+0000 7fa589d74640 1 -- 10.20.193.28:0/3861743580 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b33a100e0 msgr2=0x557b33a0f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.618+0000 7fa58ad76640 1 --2- 10.20.193.28:0/3861743580 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b33a100e0 0x557b33a0f250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.618+0000 7fa589d74640 1 --2- 10.20.193.28:0/3861743580 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b33a100e0 0x557b33a0f250 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.618+0000 7fa589d74640 1 -- 10.20.193.28:0/3861743580 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b339cab30 msgr2=0x557b339cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.618+0000 7fa589d74640 1 --2- 10.20.193.28:0/3861743580 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b339cab30 0x557b339cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.618+0000 7fa589d74640 1 -- 10.20.193.28:0/3861743580 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b339d0890 con 0x557b339cb440 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.618+0000 7fa58a575640 1 --2- 10.20.193.28:0/3861743580 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b339cab30 0x557b339cd820 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.618+0000 7fa589d74640 1 --2- 10.20.193.28:0/3861743580 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b33a0ed10 secure :-1 s=READY pgs=153 gs=2 cs=0 l=1 c_cookie=13340013068230799208 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa574003cf0 tx=0x7fa574005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9213f3506f72f68 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.618+0000 7fa5564e6640 1 -- 10.20.193.28:0/3861743580 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa574016070 con 0x557b339cb440 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.618+0000 7fa5564e6640 1 -- 10.20.193.28:0/3861743580 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa574017040 con 0x557b339cb440 2026-02-25T11:51:22.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.618+0000 7fa5564e6640 1 -- 10.20.193.28:0/3861743580 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa57401b9b0 con 0x557b339cb440 2026-02-25T11:51:22.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.619+0000 7fa592dc5e00 1 -- 10.20.193.28:0/3861743580 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 msgr2=0x557b33a0ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.619+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/3861743580 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b33a0ed10 secure :-1 s=READY pgs=153 gs=2 cs=0 l=1 c_cookie=13340013068230799208 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa574003cf0 tx=0x7fa574005f90 comp rx=0 tx=0).stop 2026-02-25T11:51:22.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.619+0000 7fa592dc5e00 1 -- 10.20.193.28:0/3861743580 shutdown_connections 2026-02-25T11:51:22.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.619+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/3861743580 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b33a100e0 0x557b33a0f250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.619+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/3861743580 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b33a0ed10 unknown :-1 s=CLOSED pgs=153 gs=2 cs=0 l=1 c_cookie=13340013068230799208 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.619+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/3861743580 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b339cab30 0x557b339cd820 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.619+0000 7fa592dc5e00 1 -- 10.20.193.28:0/3861743580 >> 10.20.193.28:0/3861743580 conn(0x557b338e97a0 msgr2=0x557b339c8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:22.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.619+0000 7fa592dc5e00 1 -- 10.20.193.28:0/3861743580 shutdown_connections 2026-02-25T11:51:22.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.619+0000 7fa592dc5e00 1 -- 10.20.193.28:0/3861743580 wait complete. 2026-02-25T11:51:22.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.620+0000 7fa592dc5e00 1 -- start start 2026-02-25T11:51:22.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.620+0000 7fa592dc5e00 1 Processor -- start 2026-02-25T11:51:22.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.620+0000 7fa592dc5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cab30 0x557b339ee880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.620+0000 7fa592dc5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b339eede0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.620+0000 7fa592dc5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b33a100e0 0x557b339ea3c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.620+0000 7fa592dc5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x557b339d0750 con 0x557b33a100e0 2026-02-25T11:51:22.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.620+0000 7fa592dc5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x557b339eaaf0 con 0x557b339cab30 2026-02-25T11:51:22.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.620+0000 7fa592dc5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x557b339eacb0 con 0x557b339cb440 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.620+0000 7fa58ad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b33a100e0 0x557b339ea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.620+0000 7fa58ad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b33a100e0 0x557b339ea3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51160/0 (socket says 10.20.193.28:51160) 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.620+0000 7fa58ad76640 1 -- 10.20.193.28:0/1575576774 learned_addr learned my addr 10.20.193.28:0/1575576774 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.620+0000 7fa589d74640 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b339eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa58a575640 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cab30 0x557b339ee880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa58ad76640 1 -- 10.20.193.28:0/1575576774 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 msgr2=0x557b339eede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa58ad76640 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b339eede0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa58ad76640 1 -- 10.20.193.28:0/1575576774 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cab30 msgr2=0x557b339ee880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa58ad76640 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cab30 0x557b339ee880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa58ad76640 1 -- 10.20.193.28:0/1575576774 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa574003990 con 0x557b33a100e0 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa58a575640 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cab30 0x557b339ee880 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa589d74640 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b339eede0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa58ad76640 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b33a100e0 0x557b339ea3c0 secure :-1 s=READY pgs=543 gs=1 cs=0 l=1 c_cookie=7619828315975496516 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa57c0cfed0 tx=0x7fa57c005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bf117b572eb344 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:22.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa53bfff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:22.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa53bfff640 1 -- 10.20.193.28:0/1575576774 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa57c10c4d0 con 0x557b33a100e0 2026-02-25T11:51:22.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa53bfff640 1 -- 10.20.193.28:0/1575576774 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x557b3356c6f0 con 0x557b33a100e0 2026-02-25T11:51:22.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa53bfff640 1 -- 10.20.193.28:0/1575576774 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa57c0d34a0 con 0x557b33a100e0 2026-02-25T11:51:22.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1575576774 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa580009c80 con 0x557b33a100e0 2026-02-25T11:51:22.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa53bfff640 1 -- 10.20.193.28:0/1575576774 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa57c0d3600 con 0x557b33a100e0 2026-02-25T11:51:22.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa592dc5e00 10 client.16164 _ll_register_callbacks cb 0x557b33857a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:22.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.621+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1575576774 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa57c0d9620 con 0x557b33a100e0 2026-02-25T11:51:22.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.622+0000 7fa53bfff640 1 -- 10.20.193.28:0/1575576774 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa57c10c350 con 0x557b33a100e0 2026-02-25T11:51:22.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.622+0000 7fa592dc5e00 10 client.16164 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:22.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.622+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1575576774 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x557b3386c0e0 con 0x557b33a100e0 2026-02-25T11:51:22.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.622+0000 7fa53bfff640 1 -- 10.20.193.28:0/1575576774 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa57c098a90 con 0x557b33a100e0 2026-02-25T11:51:22.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.622+0000 7fa592dc5e00 10 client.16164 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:22.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.622+0000 7fa592dc5e00 10 client.16164 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:51:22.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.622+0000 7fa592dc5e00 20 client.16164 populate_metadata read hostname 'trial028' 2026-02-25T11:51:22.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.622+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x557b33abcb60 0x557b33adcf60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:22.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.622+0000 7fa592dc5e00 4 client.16164 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:51:22.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.622+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1575576774 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b33569f40 con 0x557b33abcb60 2026-02-25T11:51:22.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa58a575640 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x557b33abcb60 0x557b33adcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:22.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa58a575640 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x557b33abcb60 0x557b33adcf60 crc :-1 s=READY pgs=37 gs=4 cs=0 l=0 c_cookie=8249441649324036519 s_cookie=5104626603212478114 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=727be778ff4591a7 server_cookie=46d746d9a37d16a2 in_seq=0 out_seq=0 2026-02-25T11:51:22.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa53bfff640 10 client.16164 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:51:22.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa53bfff640 1 -- 10.20.193.28:0/1575576774 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b33569f40 con 0x557b33abcb60 2026-02-25T11:51:22.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa53bfff640 10 client.16164 handle_command_reply: tid=0 2026-02-25T11:51:22.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa592dc5e00 1 client.16164 shutdown 2026-02-25T11:51:22.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1575576774 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b33a100e0 msgr2=0x557b339ea3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b33a100e0 0x557b339ea3c0 secure :-1 s=READY pgs=543 gs=1 cs=0 l=1 c_cookie=7619828315975496516 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa57c0cfed0 tx=0x7fa57c005510 comp rx=0 tx=0).stop 2026-02-25T11:51:22.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1575576774 shutdown_connections 2026-02-25T11:51:22.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x557b33abcb60 0x557b33adcf60 crc :-1 s=READY pgs=37 gs=4 cs=0 l=0 c_cookie=8249441649324036519 s_cookie=5104626603212478114 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557b33a100e0 0x557b339ea3c0 unknown :-1 s=CLOSED pgs=543 gs=1 cs=0 l=1 c_cookie=7619828315975496516 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557b339cb440 0x557b339eede0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa592dc5e00 1 --2- 10.20.193.28:0/1575576774 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557b339cab30 0x557b339ee880 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa53bfff640 0 client.16164 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:51:22.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1575576774 >> 10.20.193.28:0/1575576774 conn(0x557b338e97a0 msgr2=0x557b339c9500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:22.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.623+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1575576774 shutdown_connections 2026-02-25T11:51:22.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa592dc5e00 1 -- 10.20.193.28:0/1575576774 wait complete. 2026-02-25T11:51:22.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa592dc5e00 20 client.16164 trim_cache size 0 max 16384 2026-02-25T11:51:22.625 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:51:22.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa590dca640 1 -- 10.20.193.28:0/521221784 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5640b5760 msgr2=0x7fa5640d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa590dca640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5640b5760 0x7fa5640d5c10 secure :-1 s=READY pgs=299 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa580004740 tx=0x7fa580005f90 comp rx=0 tx=0).stop 2026-02-25T11:51:22.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa590dca640 1 -- 10.20.193.28:0/521221784 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa58c115930 msgr2=0x7fa58c123a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:22.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa590dca640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa58c115930 0x7fa58c123a20 secure :-1 s=READY pgs=541 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa57c006a50 tx=0x7fa57c030ce0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa590dca640 1 -- 10.20.193.28:0/521221784 shutdown_connections 2026-02-25T11:51:22.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa590dca640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5640b5760 0x7fa5640d5c10 unknown :-1 s=CLOSED pgs=299 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa590dca640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa58c115930 0x7fa58c123a20 unknown :-1 s=CLOSED pgs=541 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa590dca640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa58c110130 0x7fa58c11e650 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa590dca640 1 --2- 10.20.193.28:0/521221784 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa58c10f740 0x7fa58c11e110 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:22.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa590dca640 1 -- 10.20.193.28:0/521221784 >> 10.20.193.28:0/521221784 conn(0x7fa58c0ecff0 msgr2=0x7fa58c10d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:22.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa590dca640 1 -- 10.20.193.28:0/521221784 shutdown_connections 2026-02-25T11:51:22.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:22.624+0000 7fa590dca640 1 -- 10.20.193.28:0/521221784 wait complete. 2026-02-25T11:51:22.722 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:51:22.906 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:23.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.089+0000 7f1d9585c640 1 -- 10.20.193.28:0/433132110 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1d90110130 msgr2=0x7f1d90115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:23.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.089+0000 7f1d9585c640 1 --2- 10.20.193.28:0/433132110 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1d90110130 0x7f1d90115180 secure :-1 s=READY pgs=129 gs=2 cs=0 l=1 c_cookie=2102454962519257398 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1d78009890 tx=0x7f1d7802f120 comp rx=0 tx=0).stop 2026-02-25T11:51:23.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.089+0000 7f1d9585c640 1 -- 10.20.193.28:0/433132110 shutdown_connections 2026-02-25T11:51:23.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.089+0000 7f1d9585c640 1 --2- 10.20.193.28:0/433132110 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1d90115930 0x7f1d90176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.089+0000 7f1d9585c640 1 --2- 10.20.193.28:0/433132110 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1d90110130 0x7f1d90115180 unknown :-1 s=CLOSED pgs=129 gs=2 cs=0 l=1 c_cookie=2102454962519257398 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.089+0000 7f1d9585c640 1 --2- 10.20.193.28:0/433132110 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1d9010f740 0x7f1d9010fb10 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.089+0000 7f1d9585c640 1 -- 10.20.193.28:0/433132110 >> 10.20.193.28:0/433132110 conn(0x7f1d900ecff0 msgr2=0x7f1d9010d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:23.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.089+0000 7f1d9585c640 1 -- 10.20.193.28:0/433132110 shutdown_connections 2026-02-25T11:51:23.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.089+0000 7f1d9585c640 1 -- 10.20.193.28:0/433132110 wait complete. 2026-02-25T11:51:23.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.089+0000 7f1d9585c640 1 Processor -- start 2026-02-25T11:51:23.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d9585c640 1 -- start start 2026-02-25T11:51:23.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d9585c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1d9010f740 0x7f1d900f5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d9585c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1d90110130 0x7f1d90176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d9585c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1d90115930 0x7f1d900f6250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d9585c640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1d900fb4d0 con 0x7f1d9010f740 2026-02-25T11:51:23.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d9585c640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1d900fb640 con 0x7f1d90110130 2026-02-25T11:51:23.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d9585c640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d900fb7b0 con 0x7f1d90115930 2026-02-25T11:51:23.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d8effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1d9010f740 0x7f1d900f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d8effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1d9010f740 0x7f1d900f5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51164/0 (socket says 10.20.193.28:51164) 2026-02-25T11:51:23.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d8effd640 1 -- 10.20.193.28:0/2090599718 learned_addr learned my addr 10.20.193.28:0/2090599718 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:23.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d8e7fc640 1 --2- 10.20.193.28:0/2090599718 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1d90110130 0x7f1d90176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d8effd640 1 -- 10.20.193.28:0/2090599718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1d90115930 msgr2=0x7f1d900f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:23.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d8effd640 1 --2- 10.20.193.28:0/2090599718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1d90115930 0x7f1d900f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d8effd640 1 -- 10.20.193.28:0/2090599718 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1d90110130 msgr2=0x7f1d90176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:23.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d8effd640 1 --2- 10.20.193.28:0/2090599718 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1d90110130 0x7f1d90176200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.090+0000 7f1d8effd640 1 -- 10.20.193.28:0/2090599718 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d780094b0 con 0x7f1d9010f740 2026-02-25T11:51:23.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.091+0000 7f1d8effd640 1 --2- 10.20.193.28:0/2090599718 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1d9010f740 0x7f1d900f5bc0 secure :-1 s=READY pgs=544 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1d84005940 tx=0x7f1d8400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:23.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.091+0000 7f1d9485a640 1 -- 10.20.193.28:0/2090599718 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1d8400cd10 con 0x7f1d9010f740 2026-02-25T11:51:23.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.091+0000 7f1d9485a640 1 -- 10.20.193.28:0/2090599718 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1d8400ce70 con 0x7f1d9010f740 2026-02-25T11:51:23.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.091+0000 7f1d9485a640 1 -- 10.20.193.28:0/2090599718 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1d84005280 con 0x7f1d9010f740 2026-02-25T11:51:23.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.091+0000 7f1d9585c640 1 -- 10.20.193.28:0/2090599718 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d900fba90 con 0x7f1d9010f740 2026-02-25T11:51:23.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.091+0000 7f1d9585c640 1 -- 10.20.193.28:0/2090599718 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d900fbf10 con 0x7f1d9010f740 2026-02-25T11:51:23.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.094+0000 7f1d9485a640 1 -- 10.20.193.28:0/2090599718 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f1d84005410 con 0x7f1d9010f740 2026-02-25T11:51:23.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.096+0000 7f1d9485a640 1 --2- 10.20.193.28:0/2090599718 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1d640b5760 0x7f1d640d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.096+0000 7f1d9485a640 1 -- 10.20.193.28:0/2090599718 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1d84014070 con 0x7f1d9010f740 2026-02-25T11:51:23.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.096+0000 7f1d8e7fc640 1 --2- 10.20.193.28:0/2090599718 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1d640b5760 0x7f1d640d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.096+0000 7f1d97857e00 1 Processor -- start 2026-02-25T11:51:23.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.096+0000 7f1d97857e00 1 -- start start 2026-02-25T11:51:23.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d97857e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 0x555e99840910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d97857e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e998855b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d97857e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 0x555e998a5ed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d97857e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555e9974d8a0 con 0x555e99840e50 2026-02-25T11:51:23.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d97857e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555e997495d0 con 0x555e99885af0 2026-02-25T11:51:23.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d97857e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555e99743570 con 0x555e99840540 2026-02-25T11:51:23.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d8effd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 0x555e99840910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d8effd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 0x555e99840910 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44428/0 (socket says 10.20.193.28:44428) 2026-02-25T11:51:23.100 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d8effd640 1 -- 10.20.193.28:0/3590429561 learned_addr learned my addr 10.20.193.28:0/3590429561 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:23.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d8e7fc640 1 --2- 10.20.193.28:0/3590429561 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e998855b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d8e7fc640 1 --2- 10.20.193.28:0/2090599718 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1d640b5760 0x7f1d640d5c10 secure :-1 s=READY pgs=300 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1d900fc560 tx=0x7f1d7803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:23.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d8f7fe640 1 --2- 10.20.193.28:0/3590429561 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 0x555e998a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d8effd640 1 -- 10.20.193.28:0/3590429561 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 msgr2=0x555e998a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:23.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d8effd640 1 --2- 10.20.193.28:0/3590429561 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 0x555e998a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d8effd640 1 -- 10.20.193.28:0/3590429561 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 msgr2=0x555e998855b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:23.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d8effd640 1 --2- 10.20.193.28:0/3590429561 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e998855b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d8effd640 1 -- 10.20.193.28:0/3590429561 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e996eed10 con 0x555e99840540 2026-02-25T11:51:23.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.097+0000 7f1d8effd640 1 --2- 10.20.193.28:0/3590429561 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 0x555e99840910 secure :-1 s=READY pgs=154 gs=1 cs=0 l=1 c_cookie=5462336531085767124 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1d840a9590 tx=0x7f1d840a8d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bce1e226301b1d4 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:23.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.098+0000 7f1d53fff640 1 -- 10.20.193.28:0/3590429561 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1d840ac6c0 con 0x555e99840540 2026-02-25T11:51:23.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.098+0000 7f1d53fff640 1 -- 10.20.193.28:0/3590429561 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1d840ad8c0 con 0x555e99840540 2026-02-25T11:51:23.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.098+0000 7f1d53fff640 1 -- 10.20.193.28:0/3590429561 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1d840aca60 con 0x555e99840540 2026-02-25T11:51:23.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.098+0000 7f1d97857e00 1 -- 10.20.193.28:0/3590429561 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 msgr2=0x555e99840910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:23.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.098+0000 7f1d97857e00 1 --2- 10.20.193.28:0/3590429561 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 0x555e99840910 secure :-1 s=READY pgs=154 gs=1 cs=0 l=1 c_cookie=5462336531085767124 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1d840a9590 tx=0x7f1d840a8d60 comp rx=0 tx=0).stop 2026-02-25T11:51:23.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.098+0000 7f1d97857e00 1 -- 10.20.193.28:0/3590429561 shutdown_connections 2026-02-25T11:51:23.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.098+0000 7f1d97857e00 1 --2- 10.20.193.28:0/3590429561 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 0x555e998a5ed0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.098+0000 7f1d97857e00 1 --2- 10.20.193.28:0/3590429561 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e998855b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.098+0000 7f1d97857e00 1 --2- 10.20.193.28:0/3590429561 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 0x555e99840910 unknown :-1 s=CLOSED pgs=154 gs=1 cs=0 l=1 c_cookie=5462336531085767124 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.098+0000 7f1d97857e00 1 -- 10.20.193.28:0/3590429561 >> 10.20.193.28:0/3590429561 conn(0x555e997567a0 msgr2=0x555e99836410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:23.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.098+0000 7f1d97857e00 1 -- 10.20.193.28:0/3590429561 shutdown_connections 2026-02-25T11:51:23.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.098+0000 7f1d97857e00 1 -- 10.20.193.28:0/3590429561 wait complete. 2026-02-25T11:51:23.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.099+0000 7f1d97857e00 1 -- start start 2026-02-25T11:51:23.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d97857e00 1 Processor -- start 2026-02-25T11:51:23.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d97857e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99840540 0x555e99858800 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d97857e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e99858d40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d97857e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99885af0 0x555e9985dd30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d97857e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555e9985e270 con 0x555e99840e50 2026-02-25T11:51:23.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d97857e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555e9985e3e0 con 0x555e99840540 2026-02-25T11:51:23.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d97857e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555e9985e5a0 con 0x555e99885af0 2026-02-25T11:51:23.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d8e7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e99858d40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d8e7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e99858d40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51186/0 (socket says 10.20.193.28:51186) 2026-02-25T11:51:23.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d8e7fc640 1 -- 10.20.193.28:0/1984053441 learned_addr learned my addr 10.20.193.28:0/1984053441 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:23.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d8f7fe640 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99885af0 0x555e9985dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d8effd640 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99840540 0x555e99858800 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d8e7fc640 1 -- 10.20.193.28:0/1984053441 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99885af0 msgr2=0x555e9985dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:23.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d8e7fc640 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99885af0 0x555e9985dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d8e7fc640 1 -- 10.20.193.28:0/1984053441 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99840540 msgr2=0x555e99858800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:23.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d8e7fc640 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99840540 0x555e99858800 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.100+0000 7f1d8e7fc640 1 -- 10.20.193.28:0/1984053441 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e996c73c0 con 0x555e99840e50 2026-02-25T11:51:23.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.101+0000 7f1d8effd640 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99840540 0x555e99858800 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:23.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.101+0000 7f1d8e7fc640 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e99858d40 secure :-1 s=READY pgs=545 gs=1 cs=0 l=1 c_cookie=2641117348086759997 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1d78004500 tx=0x7f1d78002390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a7223c1ed5423d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:23.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.101+0000 7f1d517fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:23.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.101+0000 7f1d517fa640 1 -- 10.20.193.28:0/1984053441 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1d78002ad0 con 0x555e99840e50 2026-02-25T11:51:23.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.101+0000 7f1d517fa640 1 -- 10.20.193.28:0/1984053441 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1d7804a040 con 0x555e99840e50 2026-02-25T11:51:23.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.101+0000 7f1d517fa640 1 -- 10.20.193.28:0/1984053441 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1d78042400 con 0x555e99840e50 2026-02-25T11:51:23.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.101+0000 7f1d97857e00 1 -- 10.20.193.28:0/1984053441 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d900fbf10 con 0x555e99840e50 2026-02-25T11:51:23.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.101+0000 7f1d517fa640 1 -- 10.20.193.28:0/1984053441 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1d78042590 con 0x555e99840e50 2026-02-25T11:51:23.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.101+0000 7f1d97857e00 10 client.16176 _ll_register_callbacks cb 0x555e996c4a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:23.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.101+0000 7f1d97857e00 1 -- 10.20.193.28:0/1984053441 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e993d96f0 con 0x555e99840e50 2026-02-25T11:51:23.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.101+0000 7f1d517fa640 1 -- 10.20.193.28:0/1984053441 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1d78002c30 con 0x555e99840e50 2026-02-25T11:51:23.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.101+0000 7f1d97857e00 10 client.16176 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:23.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.101+0000 7f1d97857e00 1 -- 10.20.193.28:0/1984053441 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x555e996d90e0 con 0x555e99840e50 2026-02-25T11:51:23.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.102+0000 7f1d517fa640 1 -- 10.20.193.28:0/1984053441 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f1d78050440 con 0x555e99840e50 2026-02-25T11:51:23.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.102+0000 7f1d97857e00 10 client.16176 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:23.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.102+0000 7f1d97857e00 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:23.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.102+0000 7f1d97857e00 20 client.16176 populate_metadata read hostname 'trial028' 2026-02-25T11:51:23.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.102+0000 7f1d97857e00 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555e99882f90 0x555e99883430 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.102+0000 7f1d97857e00 4 client.16176 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:51:23.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.102+0000 7f1d97857e00 1 -- 10.20.193.28:0/1984053441 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e99715fc0 con 0x555e99882f90 2026-02-25T11:51:23.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.102+0000 7f1d8effd640 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555e99882f90 0x555e99883430 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.103+0000 7f1d8effd640 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555e99882f90 0x555e99883430 crc :-1 s=READY pgs=76 gs=4 cs=0 l=0 c_cookie=8311094686513089093 s_cookie=3990992480683520580 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7356f095543fca45 server_cookie=3762da8418657244 in_seq=0 out_seq=0 2026-02-25T11:51:23.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.103+0000 7f1d517fa640 10 client.16176 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:23.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.103+0000 7f1d517fa640 1 -- 10.20.193.28:0/1984053441 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x555e99715fc0 con 0x555e99882f90 2026-02-25T11:51:23.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.103+0000 7f1d517fa640 10 client.16176 handle_command_reply: tid=0 2026-02-25T11:51:23.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.104+0000 7f1d97857e00 1 client.16176 shutdown 2026-02-25T11:51:23.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.104+0000 7f1d97857e00 1 -- 10.20.193.28:0/1984053441 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 msgr2=0x555e99858d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:23.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.104+0000 7f1d97857e00 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e99858d40 secure :-1 s=READY pgs=545 gs=1 cs=0 l=1 c_cookie=2641117348086759997 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1d78004500 tx=0x7f1d78002390 comp rx=0 tx=0).stop 2026-02-25T11:51:23.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.104+0000 7f1d97857e00 1 -- 10.20.193.28:0/1984053441 shutdown_connections 2026-02-25T11:51:23.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.104+0000 7f1d97857e00 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555e99882f90 0x555e99883430 crc :-1 s=READY pgs=76 gs=4 cs=0 l=0 c_cookie=8311094686513089093 s_cookie=3990992480683520580 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.104+0000 7f1d97857e00 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99885af0 0x555e9985dd30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.104+0000 7f1d97857e00 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e99858d40 unknown :-1 s=CLOSED pgs=545 gs=1 cs=0 l=1 c_cookie=2641117348086759997 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.104+0000 7f1d97857e00 1 --2- 10.20.193.28:0/1984053441 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99840540 0x555e99858800 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.104+0000 7f1d517fa640 0 client.16176 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:23.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.104+0000 7f1d97857e00 1 -- 10.20.193.28:0/1984053441 >> 10.20.193.28:0/1984053441 conn(0x555e997567a0 msgr2=0x555e99836b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:23.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.104+0000 7f1d97857e00 1 -- 10.20.193.28:0/1984053441 shutdown_connections 2026-02-25T11:51:23.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.104+0000 7f1d97857e00 1 -- 10.20.193.28:0/1984053441 wait complete. 2026-02-25T11:51:23.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.104+0000 7f1d97857e00 20 client.16176 trim_cache size 0 max 16384 2026-02-25T11:51:23.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.111+0000 7f1d97857e00 1 Processor -- start 2026-02-25T11:51:23.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.111+0000 7f1d97857e00 1 -- start start 2026-02-25T11:51:23.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.111+0000 7f1d97857e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840540 0x555e99884b00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.111+0000 7f1d97857e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840e50 0x555e99885040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.111+0000 7f1d97857e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 0x555e99885580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.111+0000 7f1d97857e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555e997411a0 con 0x555e99840540 2026-02-25T11:51:23.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.111+0000 7f1d97857e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555e99401700 con 0x555e99885af0 2026-02-25T11:51:23.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.111+0000 7f1d97857e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555e9984d770 con 0x555e99840e50 2026-02-25T11:51:23.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.111+0000 7f1d8effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840540 0x555e99884b00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d8effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840540 0x555e99884b00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51190/0 (socket says 10.20.193.28:51190) 2026-02-25T11:51:23.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d8effd640 1 -- 10.20.193.28:0/2195267233 learned_addr learned my addr 10.20.193.28:0/2195267233 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:23.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d8e7fc640 1 --2- 10.20.193.28:0/2195267233 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840e50 0x555e99885040 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d8f7fe640 1 --2- 10.20.193.28:0/2195267233 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 0x555e99885580 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d8effd640 1 -- 10.20.193.28:0/2195267233 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840e50 msgr2=0x555e99885040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:23.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d8effd640 1 --2- 10.20.193.28:0/2195267233 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840e50 0x555e99885040 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d8effd640 1 -- 10.20.193.28:0/2195267233 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 msgr2=0x555e99885580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:23.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d8effd640 1 --2- 10.20.193.28:0/2195267233 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 0x555e99885580 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d8effd640 1 -- 10.20.193.28:0/2195267233 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e996d3b60 con 0x555e99840540 2026-02-25T11:51:23.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d8e7fc640 1 --2- 10.20.193.28:0/2195267233 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840e50 0x555e99885040 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:23.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d8f7fe640 1 --2- 10.20.193.28:0/2195267233 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 0x555e99885580 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:23.115 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d8effd640 1 --2- 10.20.193.28:0/2195267233 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840540 0x555e99884b00 secure :-1 s=READY pgs=546 gs=1 cs=0 l=1 c_cookie=9374063017683847928 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1d840a9590 tx=0x7f1d840d2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82175ca60f2c0af8 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:23.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d537fe640 1 -- 10.20.193.28:0/2195267233 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1d840b2be0 con 0x555e99840540 2026-02-25T11:51:23.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d537fe640 1 -- 10.20.193.28:0/2195267233 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1d840b51c0 con 0x555e99840540 2026-02-25T11:51:23.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.112+0000 7f1d537fe640 1 -- 10.20.193.28:0/2195267233 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1d840adb50 con 0x555e99840540 2026-02-25T11:51:23.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.113+0000 7f1d97857e00 1 -- 10.20.193.28:0/2195267233 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840540 msgr2=0x555e99884b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:23.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.113+0000 7f1d97857e00 1 --2- 10.20.193.28:0/2195267233 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840540 0x555e99884b00 secure :-1 s=READY pgs=546 gs=1 cs=0 l=1 c_cookie=9374063017683847928 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1d840a9590 tx=0x7f1d840d2040 comp rx=0 tx=0).stop 2026-02-25T11:51:23.119 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.113+0000 7f1d97857e00 1 -- 10.20.193.28:0/2195267233 shutdown_connections 2026-02-25T11:51:23.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.113+0000 7f1d97857e00 1 --2- 10.20.193.28:0/2195267233 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 0x555e99885580 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.113+0000 7f1d97857e00 1 --2- 10.20.193.28:0/2195267233 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840e50 0x555e99885040 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.113+0000 7f1d97857e00 1 --2- 10.20.193.28:0/2195267233 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840540 0x555e99884b00 unknown :-1 s=CLOSED pgs=546 gs=1 cs=0 l=1 c_cookie=9374063017683847928 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.113+0000 7f1d97857e00 1 -- 10.20.193.28:0/2195267233 >> 10.20.193.28:0/2195267233 conn(0x555e997567a0 msgr2=0x555e99854c00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:23.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.113+0000 7f1d97857e00 1 -- 10.20.193.28:0/2195267233 shutdown_connections 2026-02-25T11:51:23.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.113+0000 7f1d97857e00 1 -- 10.20.193.28:0/2195267233 wait complete. 2026-02-25T11:51:23.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.113+0000 7f1d97857e00 1 -- start start 2026-02-25T11:51:23.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.114+0000 7f1d97857e00 1 Processor -- start 2026-02-25T11:51:23.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.114+0000 7f1d97857e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 0x555e99864760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.114+0000 7f1d97857e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e99860030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.114+0000 7f1d97857e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 0x555e99860570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.114+0000 7f1d97857e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555e997411a0 con 0x555e99840e50 2026-02-25T11:51:23.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.114+0000 7f1d97857e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555e99401700 con 0x555e99885af0 2026-02-25T11:51:23.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.114+0000 7f1d97857e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555e99860b60 con 0x555e99840540 2026-02-25T11:51:23.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.114+0000 7f1d8effd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 0x555e99864760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.114+0000 7f1d8effd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 0x555e99864760 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44452/0 (socket says 10.20.193.28:44452) 2026-02-25T11:51:23.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.114+0000 7f1d8effd640 1 -- 10.20.193.28:0/824533733 learned_addr learned my addr 10.20.193.28:0/824533733 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:23.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.114+0000 7f1d8f7fe640 1 --2- 10.20.193.28:0/824533733 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 0x555e99860570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d8e7fc640 1 --2- 10.20.193.28:0/824533733 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e99860030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d8effd640 1 -- 10.20.193.28:0/824533733 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 msgr2=0x555e99860570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:23.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d8effd640 1 --2- 10.20.193.28:0/824533733 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 0x555e99860570 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d8effd640 1 -- 10.20.193.28:0/824533733 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 msgr2=0x555e99860030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:23.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d8effd640 1 --2- 10.20.193.28:0/824533733 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e99860030 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d8effd640 1 -- 10.20.193.28:0/824533733 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e996c76f0 con 0x555e99840540 2026-02-25T11:51:23.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d8effd640 1 --2- 10.20.193.28:0/824533733 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 0x555e99864760 secure :-1 s=READY pgs=155 gs=1 cs=0 l=1 c_cookie=1845859174351480918 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1d840af150 tx=0x7f1d840a8d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=199dcf23a89da856 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:23.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d517fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:51:23.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d517fa640 1 -- 10.20.193.28:0/824533733 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1d840adb50 con 0x555e99840540 2026-02-25T11:51:23.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d97857e00 1 -- 10.20.193.28:0/824533733 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e996eed10 con 0x555e99840540 2026-02-25T11:51:23.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d97857e00 10 client.25951 _ll_register_callbacks cb 0x555e996c4a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:23.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d97857e00 1 -- 10.20.193.28:0/824533733 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d840ad8c0 con 0x555e99840540 2026-02-25T11:51:23.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d517fa640 1 -- 10.20.193.28:0/824533733 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1d840b8000 con 0x555e99840540 2026-02-25T11:51:23.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.115+0000 7f1d517fa640 1 -- 10.20.193.28:0/824533733 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1d840b7df0 con 0x555e99840540 2026-02-25T11:51:23.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.116+0000 7f1d517fa640 1 -- 10.20.193.28:0/824533733 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1d8408a070 con 0x555e99840540 2026-02-25T11:51:23.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.116+0000 7f1d517fa640 1 -- 10.20.193.28:0/824533733 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1d840a92c0 con 0x555e99840540 2026-02-25T11:51:23.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.116+0000 7f1d97857e00 10 client.25951 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:23.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.116+0000 7f1d97857e00 1 -- 10.20.193.28:0/824533733 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x555e996d9340 con 0x555e99840540 2026-02-25T11:51:23.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.116+0000 7f1d517fa640 1 -- 10.20.193.28:0/824533733 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f1d840c4d20 con 0x555e99840540 2026-02-25T11:51:23.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.116+0000 7f1d97857e00 10 client.25951 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:23.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.116+0000 7f1d97857e00 10 client.25951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:23.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.116+0000 7f1d97857e00 20 client.25951 populate_metadata read hostname 'trial028' 2026-02-25T11:51:23.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.116+0000 7f1d97857e00 1 --2- 10.20.193.28:0/824533733 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555e99932630 0x555e99952ae0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:23.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.116+0000 7f1d97857e00 4 client.25951 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:51:23.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.116+0000 7f1d97857e00 1 -- 10.20.193.28:0/824533733 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e993d6f40 con 0x555e99932630 2026-02-25T11:51:23.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.116+0000 7f1d8e7fc640 1 --2- 10.20.193.28:0/824533733 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555e99932630 0x555e99952ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:23.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.117+0000 7f1d8e7fc640 1 --2- 10.20.193.28:0/824533733 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555e99932630 0x555e99952ae0 crc :-1 s=READY pgs=77 gs=4 cs=0 l=0 c_cookie=17811516184179076750 s_cookie=2056232701788658887 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f72f37999d1d6e8e server_cookie=1c8934bf75a79cc7 in_seq=0 out_seq=0 2026-02-25T11:51:23.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.117+0000 7f1d517fa640 10 client.25951 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:23.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.117+0000 7f1d517fa640 1 -- 10.20.193.28:0/824533733 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e993d6f40 con 0x555e99932630 2026-02-25T11:51:23.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.117+0000 7f1d517fa640 10 client.25951 handle_command_reply: tid=0 2026-02-25T11:51:23.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.117+0000 7f1d97857e00 1 client.25951 shutdown 2026-02-25T11:51:23.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.117+0000 7f1d97857e00 1 -- 10.20.193.28:0/824533733 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 msgr2=0x555e99864760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:23.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.117+0000 7f1d97857e00 1 --2- 10.20.193.28:0/824533733 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 0x555e99864760 secure :-1 s=READY pgs=155 gs=1 cs=0 l=1 c_cookie=1845859174351480918 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1d840af150 tx=0x7f1d840a8d60 comp rx=0 tx=0).stop 2026-02-25T11:51:23.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d97857e00 1 -- 10.20.193.28:0/824533733 shutdown_connections 2026-02-25T11:51:23.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d97857e00 1 --2- 10.20.193.28:0/824533733 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555e99932630 0x555e99952ae0 crc :-1 s=READY pgs=77 gs=4 cs=0 l=0 c_cookie=17811516184179076750 s_cookie=2056232701788658887 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d97857e00 1 --2- 10.20.193.28:0/824533733 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555e99885af0 0x555e99860570 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d97857e00 1 --2- 10.20.193.28:0/824533733 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555e99840e50 0x555e99860030 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d97857e00 1 --2- 10.20.193.28:0/824533733 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555e99840540 0x555e99864760 unknown :-1 s=CLOSED pgs=155 gs=1 cs=0 l=1 c_cookie=1845859174351480918 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d517fa640 0 client.25951 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:23.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d97857e00 1 -- 10.20.193.28:0/824533733 >> 10.20.193.28:0/824533733 conn(0x555e997567a0 msgr2=0x555e998553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:23.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d97857e00 1 -- 10.20.193.28:0/824533733 shutdown_connections 2026-02-25T11:51:23.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d97857e00 1 -- 10.20.193.28:0/824533733 wait complete. 2026-02-25T11:51:23.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d97857e00 20 client.25951 trim_cache size 0 max 16384 2026-02-25T11:51:23.125 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:51:23.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d9585c640 1 -- 10.20.193.28:0/2090599718 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1d640b5760 msgr2=0x7f1d640d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:23.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d9585c640 1 --2- 10.20.193.28:0/2090599718 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1d640b5760 0x7f1d640d5c10 secure :-1 s=READY pgs=300 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1d900fc560 tx=0x7f1d7803a040 comp rx=0 tx=0).stop 2026-02-25T11:51:23.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d9585c640 1 -- 10.20.193.28:0/2090599718 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1d9010f740 msgr2=0x7f1d900f5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:23.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d9585c640 1 --2- 10.20.193.28:0/2090599718 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1d9010f740 0x7f1d900f5bc0 secure :-1 s=READY pgs=544 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1d84005940 tx=0x7f1d8400ee60 comp rx=0 tx=0).stop 2026-02-25T11:51:23.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d9585c640 1 -- 10.20.193.28:0/2090599718 shutdown_connections 2026-02-25T11:51:23.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d9585c640 1 --2- 10.20.193.28:0/2090599718 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1d640b5760 0x7f1d640d5c10 unknown :-1 s=CLOSED pgs=300 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d9585c640 1 --2- 10.20.193.28:0/2090599718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1d90115930 0x7f1d900f6250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d9585c640 1 --2- 10.20.193.28:0/2090599718 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1d90110130 0x7f1d90176200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d9585c640 1 --2- 10.20.193.28:0/2090599718 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1d9010f740 0x7f1d900f5bc0 unknown :-1 s=CLOSED pgs=544 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:23.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.118+0000 7f1d9585c640 1 -- 10.20.193.28:0/2090599718 >> 10.20.193.28:0/2090599718 conn(0x7f1d900ecff0 msgr2=0x7f1d9010dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:23.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.119+0000 7f1d9585c640 1 -- 10.20.193.28:0/2090599718 shutdown_connections 2026-02-25T11:51:23.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:23.119+0000 7f1d9585c640 1 -- 10.20.193.28:0/2090599718 wait complete. 2026-02-25T11:51:24.221 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:51:24.222 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:51:24.406 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:24.470 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:24 trial028 ceph-mon[39224]: pgmap v321: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:24.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:24 trial167 ceph-mon[43392]: pgmap v321: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:24.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:24 trial119 ceph-mon[42710]: pgmap v321: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:24.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.574+0000 7f8309878640 1 -- 10.20.193.28:0/1487878059 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f830410f650 msgr2=0x7f830410fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:24.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.574+0000 7f8309878640 1 --2- 10.20.193.28:0/1487878059 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f830410f650 0x7f830410fa20 secure :-1 s=READY pgs=156 gs=1 cs=0 l=1 c_cookie=17944589208730539023 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f82f4009950 tx=0x7f82f402f190 comp rx=0 tx=0).stop 2026-02-25T11:51:24.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.574+0000 7f8309878640 1 -- 10.20.193.28:0/1487878059 shutdown_connections 2026-02-25T11:51:24.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.574+0000 7f8309878640 1 --2- 10.20.193.28:0/1487878059 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8304115840 0x7f8304176210 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.574+0000 7f8309878640 1 --2- 10.20.193.28:0/1487878059 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8304110040 0x7f8304115090 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.574+0000 7f8309878640 1 --2- 10.20.193.28:0/1487878059 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f830410f650 0x7f830410fa20 unknown :-1 s=CLOSED pgs=156 gs=1 cs=0 l=1 c_cookie=17944589208730539023 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.574+0000 7f8309878640 1 -- 10.20.193.28:0/1487878059 >> 10.20.193.28:0/1487878059 conn(0x7f83040ecf00 msgr2=0x7f830410d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:24.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.574+0000 7f8309878640 1 -- 10.20.193.28:0/1487878059 shutdown_connections 2026-02-25T11:51:24.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.574+0000 7f8309878640 1 -- 10.20.193.28:0/1487878059 wait complete. 2026-02-25T11:51:24.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.575+0000 7f8309878640 1 Processor -- start 2026-02-25T11:51:24.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.575+0000 7f8309878640 1 -- start start 2026-02-25T11:51:24.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.575+0000 7f8309878640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f830410f650 0x7f83040f5950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.575+0000 7f8309878640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8304110040 0x7f8304176110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.575+0000 7f8309878640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8304115840 0x7f83040f6050 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.575+0000 7f8309878640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f83040fb260 con 0x7f8304115840 2026-02-25T11:51:24.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.575+0000 7f8309878640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f83040fb3d0 con 0x7f830410f650 2026-02-25T11:51:24.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.575+0000 7f8309878640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f83040fb590 con 0x7f8304110040 2026-02-25T11:51:24.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f8309077640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8304115840 0x7f83040f6050 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f82fbfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8304110040 0x7f8304176110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f8309077640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8304115840 0x7f83040f6050 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51200/0 (socket says 10.20.193.28:51200) 2026-02-25T11:51:24.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f8309077640 1 -- 10.20.193.28:0/3487478501 learned_addr learned my addr 10.20.193.28:0/3487478501 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:24.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f8308876640 1 --2- 10.20.193.28:0/3487478501 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f830410f650 0x7f83040f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f8309077640 1 -- 10.20.193.28:0/3487478501 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8304110040 msgr2=0x7f8304176110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:24.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f8309077640 1 --2- 10.20.193.28:0/3487478501 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8304110040 0x7f8304176110 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f8309077640 1 -- 10.20.193.28:0/3487478501 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f830410f650 msgr2=0x7f83040f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:24.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f8309077640 1 --2- 10.20.193.28:0/3487478501 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f830410f650 0x7f83040f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f8309077640 1 -- 10.20.193.28:0/3487478501 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f4009570 con 0x7f8304115840 2026-02-25T11:51:24.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f8309077640 1 --2- 10.20.193.28:0/3487478501 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8304115840 0x7f83040f6050 secure :-1 s=READY pgs=547 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f830000f8d0 tx=0x7f830000fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:24.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f82f9ffb640 1 -- 10.20.193.28:0/3487478501 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f830000dce0 con 0x7f8304115840 2026-02-25T11:51:24.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f8309878640 1 -- 10.20.193.28:0/3487478501 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83040fb870 con 0x7f8304115840 2026-02-25T11:51:24.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f8309878640 1 -- 10.20.193.28:0/3487478501 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83040fbd90 con 0x7f8304115840 2026-02-25T11:51:24.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f82f9ffb640 1 -- 10.20.193.28:0/3487478501 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f830000de40 con 0x7f8304115840 2026-02-25T11:51:24.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.576+0000 7f82f9ffb640 1 -- 10.20.193.28:0/3487478501 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f83000116b0 con 0x7f8304115840 2026-02-25T11:51:24.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.579+0000 7f82f9ffb640 1 -- 10.20.193.28:0/3487478501 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f8300020060 con 0x7f8304115840 2026-02-25T11:51:24.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.581+0000 7f82f9ffb640 1 --2- 10.20.193.28:0/3487478501 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f82e00b5750 0x7f82e00d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.581+0000 7f82f9ffb640 1 -- 10.20.193.28:0/3487478501 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8300015070 con 0x7f8304115840 2026-02-25T11:51:24.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.581+0000 7f8308876640 1 --2- 10.20.193.28:0/3487478501 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f82e00b5750 0x7f82e00d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f8308876640 1 --2- 10.20.193.28:0/3487478501 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f82e00b5750 0x7f82e00d5c00 secure :-1 s=READY pgs=301 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f82f40023d0 tx=0x7f82f4031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:24.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f830cafce00 1 Processor -- start 2026-02-25T11:51:24.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f830cafce00 1 -- start start 2026-02-25T11:51:24.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f830cafce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5a8750 0x55d6db5a8b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f830cafce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 0x55d6db5ed610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f830cafce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5edb50 0x55d6db60df30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f830cafce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d6db4bed80 con 0x55d6db5a9060 2026-02-25T11:51:24.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f830cafce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d6db4baab0 con 0x55d6db5a8750 2026-02-25T11:51:24.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f830cafce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d6db4b4a50 con 0x55d6db5edb50 2026-02-25T11:51:24.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f82fbfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 0x55d6db5ed610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f82fbfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 0x55d6db5ed610 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51212/0 (socket says 10.20.193.28:51212) 2026-02-25T11:51:24.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f82fbfff640 1 -- 10.20.193.28:0/3292596165 learned_addr learned my addr 10.20.193.28:0/3292596165 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:24.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f8309077640 1 --2- 10.20.193.28:0/3292596165 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5edb50 0x55d6db60df30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f8308876640 1 --2- 10.20.193.28:0/3292596165 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5a8750 0x55d6db5a8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.582+0000 7f82fbfff640 1 -- 10.20.193.28:0/3292596165 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5edb50 msgr2=0x55d6db60df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:24.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f82fbfff640 1 --2- 10.20.193.28:0/3292596165 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5edb50 0x55d6db60df30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f82fbfff640 1 -- 10.20.193.28:0/3292596165 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5a8750 msgr2=0x55d6db5a8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:24.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f82fbfff640 1 --2- 10.20.193.28:0/3292596165 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5a8750 0x55d6db5a8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f82fbfff640 1 -- 10.20.193.28:0/3292596165 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6db460120 con 0x55d6db5a9060 2026-02-25T11:51:24.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f82fbfff640 1 --2- 10.20.193.28:0/3292596165 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 0x55d6db5ed610 secure :-1 s=READY pgs=548 gs=1 cs=0 l=1 c_cookie=9876686162316909691 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f82ec00bc90 tx=0x7f82ec00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891109bc56ea5c7b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:24.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f82d14e4640 1 -- 10.20.193.28:0/3292596165 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f82ec00bd90 con 0x55d6db5a9060 2026-02-25T11:51:24.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f82d14e4640 1 -- 10.20.193.28:0/3292596165 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f82ec0026a0 con 0x55d6db5a9060 2026-02-25T11:51:24.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f82d14e4640 1 -- 10.20.193.28:0/3292596165 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f82ec0203e0 con 0x55d6db5a9060 2026-02-25T11:51:24.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f830cafce00 1 -- 10.20.193.28:0/3292596165 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 msgr2=0x55d6db5ed610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:24.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f830cafce00 1 --2- 10.20.193.28:0/3292596165 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 0x55d6db5ed610 secure :-1 s=READY pgs=548 gs=1 cs=0 l=1 c_cookie=9876686162316909691 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f82ec00bc90 tx=0x7f82ec00e720 comp rx=0 tx=0).stop 2026-02-25T11:51:24.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f830cafce00 1 -- 10.20.193.28:0/3292596165 shutdown_connections 2026-02-25T11:51:24.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f830cafce00 1 --2- 10.20.193.28:0/3292596165 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5edb50 0x55d6db60df30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f830cafce00 1 --2- 10.20.193.28:0/3292596165 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 0x55d6db5ed610 unknown :-1 s=CLOSED pgs=548 gs=1 cs=0 l=1 c_cookie=9876686162316909691 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f830cafce00 1 --2- 10.20.193.28:0/3292596165 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5a8750 0x55d6db5a8b20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f830cafce00 1 -- 10.20.193.28:0/3292596165 >> 10.20.193.28:0/3292596165 conn(0x55d6db5868f0 msgr2=0x55d6db5a6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:24.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f830cafce00 1 -- 10.20.193.28:0/3292596165 shutdown_connections 2026-02-25T11:51:24.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.583+0000 7f830cafce00 1 -- 10.20.193.28:0/3292596165 wait complete. 2026-02-25T11:51:24.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.585+0000 7f830cafce00 1 -- start start 2026-02-25T11:51:24.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.585+0000 7f830cafce00 1 Processor -- start 2026-02-25T11:51:24.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.585+0000 7f830cafce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5a8750 0x55d6db6054b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.585+0000 7f830cafce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a9060 0x55d6db6059f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.585+0000 7f830cafce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5edb50 0x55d6db60a9e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.585+0000 7f830cafce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d6db60af20 con 0x55d6db5edb50 2026-02-25T11:51:24.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.585+0000 7f830cafce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d6db60b090 con 0x55d6db5a8750 2026-02-25T11:51:24.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.585+0000 7f830cafce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d6db60b250 con 0x55d6db5a9060 2026-02-25T11:51:24.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.585+0000 7f8308876640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5a8750 0x55d6db6054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.585+0000 7f8308876640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5a8750 0x55d6db6054b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:34642/0 (socket says 10.20.193.28:34642) 2026-02-25T11:51:24.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.585+0000 7f8308876640 1 -- 10.20.193.28:0/1292237857 learned_addr learned my addr 10.20.193.28:0/1292237857 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:24.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.585+0000 7f82fbfff640 1 --2- 10.20.193.28:0/1292237857 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a9060 0x55d6db6059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.586+0000 7f8309077640 1 --2- 10.20.193.28:0/1292237857 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5edb50 0x55d6db60a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.586+0000 7f82fbfff640 1 -- 10.20.193.28:0/1292237857 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5a8750 msgr2=0x55d6db6054b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:24.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.586+0000 7f82fbfff640 1 --2- 10.20.193.28:0/1292237857 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5a8750 0x55d6db6054b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.586+0000 7f82fbfff640 1 -- 10.20.193.28:0/1292237857 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5edb50 msgr2=0x55d6db60a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:24.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.586+0000 7f82fbfff640 1 --2- 10.20.193.28:0/1292237857 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5edb50 0x55d6db60a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.586+0000 7f82fbfff640 1 -- 10.20.193.28:0/1292237857 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6db438810 con 0x55d6db5a9060 2026-02-25T11:51:24.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.586+0000 7f82fbfff640 1 --2- 10.20.193.28:0/1292237857 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a9060 0x55d6db6059f0 secure :-1 s=READY pgs=157 gs=2 cs=0 l=1 c_cookie=7069738608046792211 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f82ec00bc90 tx=0x7f82ec002a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=621cc1ce937e5e13 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:24.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.586+0000 7f82c27fc640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:51:24.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.586+0000 7f82c27fc640 1 -- 10.20.193.28:0/1292237857 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f82ec0203e0 con 0x55d6db5a9060 2026-02-25T11:51:24.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.586+0000 7f82c27fc640 1 -- 10.20.193.28:0/1292237857 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f82ec00bd90 con 0x55d6db5a9060 2026-02-25T11:51:24.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.586+0000 7f82c27fc640 1 -- 10.20.193.28:0/1292237857 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f82ec004810 con 0x55d6db5a9060 2026-02-25T11:51:24.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.586+0000 7f830cafce00 1 -- 10.20.193.28:0/1292237857 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ec00b8b0 con 0x55d6db5a9060 2026-02-25T11:51:24.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f82c27fc640 1 -- 10.20.193.28:0/1292237857 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f82ec0049a0 con 0x55d6db5a9060 2026-02-25T11:51:24.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f830cafce00 10 client.25963 _ll_register_callbacks cb 0x55d6db435e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:24.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f830cafce00 1 -- 10.20.193.28:0/1292237857 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d6db20c530 con 0x55d6db5a9060 2026-02-25T11:51:24.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f82c27fc640 1 -- 10.20.193.28:0/1292237857 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f82ec0042e0 con 0x55d6db5a9060 2026-02-25T11:51:24.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f830cafce00 10 client.25963 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:24.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f830cafce00 1 -- 10.20.193.28:0/1292237857 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55d6db44a4f0 con 0x55d6db5a9060 2026-02-25T11:51:24.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f82c27fc640 1 -- 10.20.193.28:0/1292237857 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f82ec030020 con 0x55d6db5a9060 2026-02-25T11:51:24.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f830cafce00 10 client.25963 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:24.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f830cafce00 10 client.25963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:24.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f830cafce00 20 client.25963 populate_metadata read hostname 'trial028' 2026-02-25T11:51:24.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f830cafce00 1 --2- 10.20.193.28:0/1292237857 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d6db5ead40 0x55d6db5eb1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.594 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f830cafce00 4 client.25963 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:51:24.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f830cafce00 1 -- 10.20.193.28:0/1292237857 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6db461b50 con 0x55d6db5ead40 2026-02-25T11:51:24.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.587+0000 7f8308876640 1 --2- 10.20.193.28:0/1292237857 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d6db5ead40 0x55d6db5eb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.588+0000 7f8308876640 1 --2- 10.20.193.28:0/1292237857 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d6db5ead40 0x55d6db5eb1e0 crc :-1 s=READY pgs=78 gs=4 cs=0 l=0 c_cookie=14632192676258602158 s_cookie=7678233840701807681 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb0ffdb6e1775cae server_cookie=6a8e90ff416a1c41 in_seq=0 out_seq=0 2026-02-25T11:51:24.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.588+0000 7f82c27fc640 10 client.25963 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:24.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f82c27fc640 1 -- 10.20.193.28:0/1292237857 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55d6db461b50 con 0x55d6db5ead40 2026-02-25T11:51:24.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f82c27fc640 10 client.25963 handle_command_reply: tid=0 2026-02-25T11:51:24.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f830cafce00 1 client.25963 shutdown 2026-02-25T11:51:24.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f830cafce00 1 -- 10.20.193.28:0/1292237857 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a9060 msgr2=0x55d6db6059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:24.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f830cafce00 1 --2- 10.20.193.28:0/1292237857 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a9060 0x55d6db6059f0 secure :-1 s=READY pgs=157 gs=2 cs=0 l=1 c_cookie=7069738608046792211 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f82ec00bc90 tx=0x7f82ec002a40 comp rx=0 tx=0).stop 2026-02-25T11:51:24.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f830cafce00 1 -- 10.20.193.28:0/1292237857 shutdown_connections 2026-02-25T11:51:24.595 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f830cafce00 1 --2- 10.20.193.28:0/1292237857 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d6db5ead40 0x55d6db5eb1e0 crc :-1 s=READY pgs=78 gs=4 cs=0 l=0 c_cookie=14632192676258602158 s_cookie=7678233840701807681 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f830cafce00 1 --2- 10.20.193.28:0/1292237857 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5edb50 0x55d6db60a9e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f830cafce00 1 --2- 10.20.193.28:0/1292237857 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a9060 0x55d6db6059f0 unknown :-1 s=CLOSED pgs=157 gs=2 cs=0 l=1 c_cookie=7069738608046792211 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f830cafce00 1 --2- 10.20.193.28:0/1292237857 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5a8750 0x55d6db6054b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f82c27fc640 0 client.25963 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:24.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f830cafce00 1 -- 10.20.193.28:0/1292237857 >> 10.20.193.28:0/1292237857 conn(0x55d6db5868f0 msgr2=0x55d6db5a6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:24.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f830cafce00 1 -- 10.20.193.28:0/1292237857 shutdown_connections 2026-02-25T11:51:24.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f830cafce00 1 -- 10.20.193.28:0/1292237857 wait complete. 2026-02-25T11:51:24.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.589+0000 7f830cafce00 20 client.25963 trim_cache size 0 max 16384 2026-02-25T11:51:24.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f830cafce00 1 Processor -- start 2026-02-25T11:51:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f830cafce00 1 -- start start 2026-02-25T11:51:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f830cafce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a8750 0x55d6db602e70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f830cafce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 0x55d6db5e7730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f830cafce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5edb50 0x55d6db5e7c70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f830cafce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d6db5f2ef0 con 0x55d6db5a9060 2026-02-25T11:51:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f830cafce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d6db4b2680 con 0x55d6db5edb50 2026-02-25T11:51:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f830cafce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d6db172790 con 0x55d6db5a8750 2026-02-25T11:51:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f82fbfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 0x55d6db5e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f8308876640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a8750 0x55d6db602e70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f82fbfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 0x55d6db5e7730 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51222/0 (socket says 10.20.193.28:51222) 2026-02-25T11:51:24.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f82fbfff640 1 -- 10.20.193.28:0/4201999198 learned_addr learned my addr 10.20.193.28:0/4201999198 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f8309077640 1 --2- 10.20.193.28:0/4201999198 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5edb50 0x55d6db5e7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f82fbfff640 1 -- 10.20.193.28:0/4201999198 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a8750 msgr2=0x55d6db602e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f82fbfff640 1 --2- 10.20.193.28:0/4201999198 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a8750 0x55d6db602e70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f82fbfff640 1 -- 10.20.193.28:0/4201999198 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5edb50 msgr2=0x55d6db5e7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f82fbfff640 1 --2- 10.20.193.28:0/4201999198 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5edb50 0x55d6db5e7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f82fbfff640 1 -- 10.20.193.28:0/4201999198 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6db444f70 con 0x55d6db5a9060 2026-02-25T11:51:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.599+0000 7f8309077640 1 --2- 10.20.193.28:0/4201999198 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5edb50 0x55d6db5e7c70 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.600+0000 7f82fbfff640 1 --2- 10.20.193.28:0/4201999198 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 0x55d6db5e7730 secure :-1 s=READY pgs=549 gs=1 cs=0 l=1 c_cookie=16133284414042693951 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f82ec00ec30 tx=0x7f82ec032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe4f0b25f5c613f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.600+0000 7f82d0ce3640 1 -- 10.20.193.28:0/4201999198 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f82ec00bd90 con 0x55d6db5a9060 2026-02-25T11:51:24.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.600+0000 7f82d0ce3640 1 -- 10.20.193.28:0/4201999198 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55d6db20c530 con 0x55d6db5a9060 2026-02-25T11:51:24.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.600+0000 7f82d0ce3640 1 -- 10.20.193.28:0/4201999198 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f82ec028a50 con 0x55d6db5a9060 2026-02-25T11:51:24.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.600+0000 7f830cafce00 1 -- 10.20.193.28:0/4201999198 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 msgr2=0x55d6db5e7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:24.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.600+0000 7f830cafce00 1 --2- 10.20.193.28:0/4201999198 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 0x55d6db5e7730 secure :-1 s=READY pgs=549 gs=1 cs=0 l=1 c_cookie=16133284414042693951 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f82ec00ec30 tx=0x7f82ec032000 comp rx=0 tx=0).stop 2026-02-25T11:51:24.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.600+0000 7f830cafce00 1 -- 10.20.193.28:0/4201999198 shutdown_connections 2026-02-25T11:51:24.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.600+0000 7f830cafce00 1 --2- 10.20.193.28:0/4201999198 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5edb50 0x55d6db5e7c70 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.600+0000 7f830cafce00 1 --2- 10.20.193.28:0/4201999198 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a9060 0x55d6db5e7730 unknown :-1 s=CLOSED pgs=549 gs=1 cs=0 l=1 c_cookie=16133284414042693951 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.600+0000 7f830cafce00 1 --2- 10.20.193.28:0/4201999198 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a8750 0x55d6db602e70 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.600+0000 7f830cafce00 1 -- 10.20.193.28:0/4201999198 >> 10.20.193.28:0/4201999198 conn(0x55d6db5868f0 msgr2=0x55d6db601750 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:24.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.600+0000 7f830cafce00 1 -- 10.20.193.28:0/4201999198 shutdown_connections 2026-02-25T11:51:24.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.600+0000 7f830cafce00 1 -- 10.20.193.28:0/4201999198 wait complete. 2026-02-25T11:51:24.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.601+0000 7f830cafce00 1 -- start start 2026-02-25T11:51:24.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.601+0000 7f830cafce00 1 Processor -- start 2026-02-25T11:51:24.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.601+0000 7f830cafce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a8750 0x55d6db5881e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.601+0000 7f830cafce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a9060 0x55d6db588720 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.601+0000 7f830cafce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5edb50 0x55d6db588c60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.601+0000 7f830cafce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d6db586330 con 0x55d6db5a8750 2026-02-25T11:51:24.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.601+0000 7f830cafce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d6db60aaf0 con 0x55d6db5edb50 2026-02-25T11:51:24.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.601+0000 7f830cafce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d6db653a70 con 0x55d6db5a9060 2026-02-25T11:51:24.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f8308876640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a8750 0x55d6db5881e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f8308876640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a8750 0x55d6db5881e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51238/0 (socket says 10.20.193.28:51238) 2026-02-25T11:51:24.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f8308876640 1 -- 10.20.193.28:0/3963481586 learned_addr learned my addr 10.20.193.28:0/3963481586 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f82fbfff640 1 --2- 10.20.193.28:0/3963481586 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a9060 0x55d6db588720 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f8309077640 1 --2- 10.20.193.28:0/3963481586 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5edb50 0x55d6db588c60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f8308876640 1 -- 10.20.193.28:0/3963481586 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a9060 msgr2=0x55d6db588720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f8308876640 1 --2- 10.20.193.28:0/3963481586 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a9060 0x55d6db588720 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f8308876640 1 -- 10.20.193.28:0/3963481586 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5edb50 msgr2=0x55d6db588c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f8308876640 1 --2- 10.20.193.28:0/3963481586 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5edb50 0x55d6db588c60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f8308876640 1 -- 10.20.193.28:0/3963481586 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6db438b40 con 0x55d6db5a8750 2026-02-25T11:51:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f8308876640 1 --2- 10.20.193.28:0/3963481586 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a8750 0x55d6db5881e0 secure :-1 s=READY pgs=550 gs=1 cs=0 l=1 c_cookie=2895321520834890901 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f82f4002ee0 tx=0x7f82f402f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282e3f9547e84c95 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f82c27fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f82c27fc640 1 -- 10.20.193.28:0/3963481586 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f82f402fc10 con 0x55d6db5a8750 2026-02-25T11:51:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f830cafce00 1 -- 10.20.193.28:0/3963481586 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6db44a4f0 con 0x55d6db5a8750 2026-02-25T11:51:24.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f82c27fc640 1 -- 10.20.193.28:0/3963481586 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f82f402fd70 con 0x55d6db5a8750 2026-02-25T11:51:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f82c27fc640 1 -- 10.20.193.28:0/3963481586 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f82f4057ac0 con 0x55d6db5a8750 2026-02-25T11:51:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f830cafce00 10 client.16200 _ll_register_callbacks cb 0x55d6db435e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.602+0000 7f830cafce00 1 -- 10.20.193.28:0/3963481586 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82ec0026a0 con 0x55d6db5a8750 2026-02-25T11:51:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.603+0000 7f82c27fc640 1 -- 10.20.193.28:0/3963481586 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f82f4057c50 con 0x55d6db5a8750 2026-02-25T11:51:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.603+0000 7f82c27fc640 1 -- 10.20.193.28:0/3963481586 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f82f4057e10 con 0x55d6db5a8750 2026-02-25T11:51:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.603+0000 7f830cafce00 10 client.16200 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.603+0000 7f830cafce00 1 -- 10.20.193.28:0/3963481586 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55d6db44a750 con 0x55d6db5a8750 2026-02-25T11:51:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.603+0000 7f82c27fc640 1 -- 10.20.193.28:0/3963481586 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f82f40386a0 con 0x55d6db5a8750 2026-02-25T11:51:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.603+0000 7f830cafce00 10 client.16200 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.603+0000 7f830cafce00 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.603+0000 7f830cafce00 20 client.16200 populate_metadata read hostname 'trial028' 2026-02-25T11:51:24.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.603+0000 7f830cafce00 1 --2- 10.20.193.28:0/3963481586 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d6db69bc10 0x55d6db6bc070 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.603+0000 7f830cafce00 4 client.16200 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:51:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.603+0000 7f830cafce00 1 -- 10.20.193.28:0/3963481586 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d6db315690 con 0x55d6db69bc10 2026-02-25T11:51:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.603+0000 7f82fbfff640 1 --2- 10.20.193.28:0/3963481586 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d6db69bc10 0x55d6db6bc070 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.604+0000 7f82fbfff640 1 --2- 10.20.193.28:0/3963481586 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d6db69bc10 0x55d6db6bc070 crc :-1 s=READY pgs=79 gs=4 cs=0 l=0 c_cookie=14033564263285510578 s_cookie=6900637818803309934 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2c13c582dbfd1b2 server_cookie=5fc3fd88f965cd6e in_seq=0 out_seq=0 2026-02-25T11:51:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.604+0000 7f82c27fc640 10 client.16200 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f82c27fc640 1 -- 10.20.193.28:0/3963481586 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d6db315690 con 0x55d6db69bc10 2026-02-25T11:51:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f82c27fc640 10 client.16200 handle_command_reply: tid=0 2026-02-25T11:51:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f830cafce00 1 client.16200 shutdown 2026-02-25T11:51:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f830cafce00 1 -- 10.20.193.28:0/3963481586 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a8750 msgr2=0x55d6db5881e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f830cafce00 1 --2- 10.20.193.28:0/3963481586 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a8750 0x55d6db5881e0 secure :-1 s=READY pgs=550 gs=1 cs=0 l=1 c_cookie=2895321520834890901 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f82f4002ee0 tx=0x7f82f402f190 comp rx=0 tx=0).stop 2026-02-25T11:51:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f830cafce00 1 -- 10.20.193.28:0/3963481586 shutdown_connections 2026-02-25T11:51:24.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f830cafce00 1 --2- 10.20.193.28:0/3963481586 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d6db69bc10 0x55d6db6bc070 crc :-1 s=READY pgs=79 gs=4 cs=0 l=0 c_cookie=14033564263285510578 s_cookie=6900637818803309934 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f830cafce00 1 --2- 10.20.193.28:0/3963481586 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d6db5edb50 0x55d6db588c60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f830cafce00 1 --2- 10.20.193.28:0/3963481586 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d6db5a9060 0x55d6db588720 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f830cafce00 1 --2- 10.20.193.28:0/3963481586 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d6db5a8750 0x55d6db5881e0 unknown :-1 s=CLOSED pgs=550 gs=1 cs=0 l=1 c_cookie=2895321520834890901 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f830cafce00 1 -- 10.20.193.28:0/3963481586 >> 10.20.193.28:0/3963481586 conn(0x55d6db5868f0 msgr2=0x55d6db601f40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f82c27fc640 0 client.16200 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:24.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f830cafce00 1 -- 10.20.193.28:0/3963481586 shutdown_connections 2026-02-25T11:51:24.612 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:51:24.612 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:51:24.613 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "11ba92dd-980a-495c-a63e-d2ded1de1c9f", 2026-02-25T11:51:24.613 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:51:24.613 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:51:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f830cafce00 1 -- 10.20.193.28:0/3963481586 wait complete. 2026-02-25T11:51:24.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f830cafce00 20 client.16200 trim_cache size 0 max 16384 2026-02-25T11:51:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f8309878640 1 -- 10.20.193.28:0/3487478501 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f82e00b5750 msgr2=0x7f82e00d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f8309878640 1 --2- 10.20.193.28:0/3487478501 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f82e00b5750 0x7f82e00d5c00 secure :-1 s=READY pgs=301 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f82f40023d0 tx=0x7f82f4031040 comp rx=0 tx=0).stop 2026-02-25T11:51:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f8309878640 1 -- 10.20.193.28:0/3487478501 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8304115840 msgr2=0x7f83040f6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.605+0000 7f8309878640 1 --2- 10.20.193.28:0/3487478501 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8304115840 0x7f83040f6050 secure :-1 s=READY pgs=547 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f830000f8d0 tx=0x7f830000fda0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.606+0000 7f8309878640 1 -- 10.20.193.28:0/3487478501 shutdown_connections 2026-02-25T11:51:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.606+0000 7f8309878640 1 --2- 10.20.193.28:0/3487478501 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f82e00b5750 0x7f82e00d5c00 unknown :-1 s=CLOSED pgs=301 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.606+0000 7f8309878640 1 --2- 10.20.193.28:0/3487478501 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8304115840 0x7f83040f6050 unknown :-1 s=CLOSED pgs=547 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.606+0000 7f8309878640 1 --2- 10.20.193.28:0/3487478501 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8304110040 0x7f8304176110 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.606+0000 7f8309878640 1 --2- 10.20.193.28:0/3487478501 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f830410f650 0x7f83040f5950 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:24.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.606+0000 7f8309878640 1 -- 10.20.193.28:0/3487478501 >> 10.20.193.28:0/3487478501 conn(0x7f83040ecf00 msgr2=0x7f8304111a90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:24.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.606+0000 7f8309878640 1 -- 10.20.193.28:0/3487478501 shutdown_connections 2026-02-25T11:51:24.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:24.606+0000 7f8309878640 1 -- 10.20.193.28:0/3487478501 wait complete. 2026-02-25T11:51:24.704 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:51:24.888 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:25.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.067+0000 7f1e2fb46640 1 -- 10.20.193.28:0/2465725499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1e28115930 msgr2=0x7f1e28176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.067+0000 7f1e2fb46640 1 --2- 10.20.193.28:0/2465725499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1e28115930 0x7f1e28176300 secure :-1 s=READY pgs=130 gs=2 cs=0 l=1 c_cookie=8519242980423235373 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1e2400aa30 tx=0x7f1e24030270 comp rx=0 tx=0).stop 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.067+0000 7f1e2fb46640 1 -- 10.20.193.28:0/2465725499 shutdown_connections 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.067+0000 7f1e2fb46640 1 --2- 10.20.193.28:0/2465725499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1e28115930 0x7f1e28176300 unknown :-1 s=CLOSED pgs=130 gs=2 cs=0 l=1 c_cookie=8519242980423235373 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.067+0000 7f1e2fb46640 1 --2- 10.20.193.28:0/2465725499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e28110130 0x7f1e28115180 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.067+0000 7f1e2fb46640 1 --2- 10.20.193.28:0/2465725499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1e2810f740 0x7f1e2810fb10 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.067+0000 7f1e2fb46640 1 -- 10.20.193.28:0/2465725499 >> 10.20.193.28:0/2465725499 conn(0x7f1e280ecff0 msgr2=0x7f1e2810d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.067+0000 7f1e2fb46640 1 -- 10.20.193.28:0/2465725499 shutdown_connections 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.067+0000 7f1e2fb46640 1 -- 10.20.193.28:0/2465725499 wait complete. 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.068+0000 7f1e2fb46640 1 Processor -- start 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.068+0000 7f1e2fb46640 1 -- start start 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.068+0000 7f1e2fb46640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e2810f740 0x7f1e280f5a40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.068+0000 7f1e2fb46640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1e28110130 0x7f1e28176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.068+0000 7f1e2fb46640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1e28115930 0x7f1e280f60d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.068+0000 7f1e2fb46640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1e280fb350 con 0x7f1e2810f740 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.068+0000 7f1e2fb46640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1e280fb4c0 con 0x7f1e28110130 2026-02-25T11:51:25.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.068+0000 7f1e2fb46640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1e280fb630 con 0x7f1e28115930 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e2d8bb640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e2810f740 0x7f1e280f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e2e0bc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1e28115930 0x7f1e280f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e2d0ba640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1e28110130 0x7f1e28176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e2e0bc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1e28115930 0x7f1e280f60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44538/0 (socket says 10.20.193.28:44538) 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e2e0bc640 1 -- 10.20.193.28:0/130869764 learned_addr learned my addr 10.20.193.28:0/130869764 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e2d0ba640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1e28110130 0x7f1e28176200 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:34682/0 (socket says 10.20.193.28:34682) 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e2d8bb640 1 -- 10.20.193.28:0/130869764 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1e28115930 msgr2=0x7f1e280f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e2d8bb640 1 --2- 10.20.193.28:0/130869764 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1e28115930 0x7f1e280f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e2d8bb640 1 -- 10.20.193.28:0/130869764 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1e28110130 msgr2=0x7f1e28176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e2d8bb640 1 --2- 10.20.193.28:0/130869764 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1e28110130 0x7f1e28176200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e2d8bb640 1 -- 10.20.193.28:0/130869764 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2400a650 con 0x7f1e2810f740 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e2d8bb640 1 --2- 10.20.193.28:0/130869764 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e2810f740 0x7f1e280f5a40 secure :-1 s=READY pgs=551 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1e1800e7b0 tx=0x7f1e1800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e16ffd640 1 -- 10.20.193.28:0/130869764 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1e1800cc60 con 0x7f1e2810f740 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e16ffd640 1 -- 10.20.193.28:0/130869764 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1e1800cdc0 con 0x7f1e2810f740 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.069+0000 7f1e16ffd640 1 -- 10.20.193.28:0/130869764 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1e180052a0 con 0x7f1e2810f740 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.070+0000 7f1e2fb46640 1 -- 10.20.193.28:0/130869764 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e280fb910 con 0x7f1e2810f740 2026-02-25T11:51:25.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.070+0000 7f1e2fb46640 1 -- 10.20.193.28:0/130869764 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e280fbd90 con 0x7f1e2810f740 2026-02-25T11:51:25.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.073+0000 7f1e16ffd640 1 -- 10.20.193.28:0/130869764 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f1e18005430 con 0x7f1e2810f740 2026-02-25T11:51:25.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.074+0000 7f1e16ffd640 1 --2- 10.20.193.28:0/130869764 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1e040b5760 0x7f1e040d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.074+0000 7f1e16ffd640 1 -- 10.20.193.28:0/130869764 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1e18014070 con 0x7f1e2810f740 2026-02-25T11:51:25.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.074+0000 7f1e2d0ba640 1 --2- 10.20.193.28:0/130869764 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1e040b5760 0x7f1e040d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e31b41e00 1 Processor -- start 2026-02-25T11:51:25.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e31b41e00 1 -- start start 2026-02-25T11:51:25.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e2d0ba640 1 --2- 10.20.193.28:0/130869764 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1e040b5760 0x7f1e040d5c10 secure :-1 s=READY pgs=302 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1e280fc3e0 tx=0x7f1e1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e31b41e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff007510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e31b41e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007a50 0x55aeff04c1b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e31b41e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff04c6f0 0x55aeff06cad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e31b41e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55aefef1d9b0 con 0x55aeff04c6f0 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e31b41e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55aefef196e0 con 0x55aeff007140 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e31b41e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55aefef13840 con 0x55aeff007a50 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e2e0bc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff04c6f0 0x55aeff06cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e2d8bb640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff007510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e2d0ba640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007a50 0x55aeff04c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e2d0ba640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007a50 0x55aeff04c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44540/0 (socket says 10.20.193.28:44540) 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e2d8bb640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff007510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:34684/0 (socket says 10.20.193.28:34684) 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.075+0000 7f1e2d0ba640 1 -- 10.20.193.28:0/3555817432 learned_addr learned my addr 10.20.193.28:0/3555817432 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.076+0000 7f1e2d0ba640 1 -- 10.20.193.28:0/3555817432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 msgr2=0x55aeff007510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.076+0000 7f1e2d0ba640 1 --2- 10.20.193.28:0/3555817432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff007510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.076+0000 7f1e2d0ba640 1 -- 10.20.193.28:0/3555817432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff04c6f0 msgr2=0x55aeff06cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.076+0000 7f1e2d0ba640 1 --2- 10.20.193.28:0/3555817432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff04c6f0 0x55aeff06cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.076+0000 7f1e2d0ba640 1 -- 10.20.193.28:0/3555817432 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aefeebed80 con 0x55aeff007a50 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.076+0000 7f1e2e0bc640 1 --2- 10.20.193.28:0/3555817432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff04c6f0 0x55aeff06cad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.076+0000 7f1e2d0ba640 1 --2- 10.20.193.28:0/3555817432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007a50 0x55aeff04c1b0 secure :-1 s=READY pgs=158 gs=1 cs=0 l=1 c_cookie=16035651840726811653 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1e1c009c90 tx=0x7f1e1c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de8a14637c5c4405 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.076+0000 7f1df64e6640 1 -- 10.20.193.28:0/3555817432 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1e1c009d90 con 0x55aeff007a50 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.076+0000 7f1df64e6640 1 -- 10.20.193.28:0/3555817432 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1e1c0198c0 con 0x55aeff007a50 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.076+0000 7f1df64e6640 1 -- 10.20.193.28:0/3555817432 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1e1c01b940 con 0x55aeff007a50 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.076+0000 7f1e31b41e00 1 -- 10.20.193.28:0/3555817432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007a50 msgr2=0x55aeff04c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.076+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/3555817432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007a50 0x55aeff04c1b0 secure :-1 s=READY pgs=158 gs=1 cs=0 l=1 c_cookie=16035651840726811653 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1e1c009c90 tx=0x7f1e1c014000 comp rx=0 tx=0).stop 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.077+0000 7f1e31b41e00 1 -- 10.20.193.28:0/3555817432 shutdown_connections 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.077+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/3555817432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff04c6f0 0x55aeff06cad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.077+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/3555817432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007a50 0x55aeff04c1b0 unknown :-1 s=CLOSED pgs=158 gs=1 cs=0 l=1 c_cookie=16035651840726811653 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.077+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/3555817432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff007510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.077+0000 7f1e31b41e00 1 -- 10.20.193.28:0/3555817432 >> 10.20.193.28:0/3555817432 conn(0x55aefef26d40 msgr2=0x55aeff005a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.077+0000 7f1e31b41e00 1 -- 10.20.193.28:0/3555817432 shutdown_connections 2026-02-25T11:51:25.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.077+0000 7f1e31b41e00 1 -- 10.20.193.28:0/3555817432 wait complete. 2026-02-25T11:51:25.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.078+0000 7f1e31b41e00 1 -- start start 2026-02-25T11:51:25.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.078+0000 7f1e31b41e00 1 Processor -- start 2026-02-25T11:51:25.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.078+0000 7f1e31b41e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff064300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.078+0000 7f1e31b41e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007a50 0x55aeff064840 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e31b41e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff04c6f0 0x55aeff069830 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e31b41e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55aeff069d70 con 0x55aeff04c6f0 2026-02-25T11:51:25.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e31b41e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55aeff069ee0 con 0x55aeff007140 2026-02-25T11:51:25.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e31b41e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55aeff06a0a0 con 0x55aeff007a50 2026-02-25T11:51:25.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e2d8bb640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff064300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e2d8bb640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff064300 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:34698/0 (socket says 10.20.193.28:34698) 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e2d8bb640 1 -- 10.20.193.28:0/131892247 learned_addr learned my addr 10.20.193.28:0/131892247 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e2e0bc640 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff04c6f0 0x55aeff069830 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e2d0ba640 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007a50 0x55aeff064840 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e2d8bb640 1 -- 10.20.193.28:0/131892247 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007a50 msgr2=0x55aeff064840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e2d8bb640 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007a50 0x55aeff064840 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e2d8bb640 1 -- 10.20.193.28:0/131892247 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff04c6f0 msgr2=0x55aeff069830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e2d8bb640 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff04c6f0 0x55aeff069830 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e2d8bb640 1 -- 10.20.193.28:0/131892247 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aefee97350 con 0x55aeff007140 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e2e0bc640 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff04c6f0 0x55aeff069830 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1e2d8bb640 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff064300 secure :-1 s=READY pgs=131 gs=2 cs=0 l=1 c_cookie=4625569675429471278 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1e1800c6a0 tx=0x7f1e180acdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4031531ccbcae42e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1ddf7fe640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/131892247 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1e180b2970 con 0x55aeff007140 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/131892247 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1e180b4f50 con 0x55aeff007140 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.079+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/131892247 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1e180ad8e0 con 0x55aeff007140 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1e31b41e00 1 -- 10.20.193.28:0/131892247 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e1c009930 con 0x55aeff007140 2026-02-25T11:51:25.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/131892247 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1e180ada70 con 0x55aeff007140 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1ddf7fe640 10 client.25997 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1e31b41e00 10 client.25997 _ll_register_callbacks cb 0x55aefee949a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1e31b41e00 1 -- 10.20.193.28:0/131892247 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55aefeba96f0 con 0x55aeff007140 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/131892247 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1e180ac7f0 con 0x55aeff007140 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1e31b41e00 10 client.25997 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1e31b41e00 1 -- 10.20.193.28:0/131892247 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55aefeea9150 con 0x55aeff007140 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/131892247 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f1e180aac30 con 0x55aeff007140 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1e31b41e00 10 client.25997 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1e31b41e00 10 client.25997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1e31b41e00 20 client.25997 populate_metadata read hostname 'trial028' 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aeff06c7d0 0x55aeff04a470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.080+0000 7f1e31b41e00 4 client.25997 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.081+0000 7f1e31b41e00 1 -- 10.20.193.28:0/131892247 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aefeee5910 con 0x55aeff06c7d0 2026-02-25T11:51:25.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.081+0000 7f1e2d0ba640 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aeff06c7d0 0x55aeff04a470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.081+0000 7f1e2d0ba640 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aeff06c7d0 0x55aeff04a470 crc :-1 s=READY pgs=80 gs=4 cs=0 l=0 c_cookie=14865622363418225312 s_cookie=7090194786823121783 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce4d4cc730a56aa0 server_cookie=62656e97d8eb2b77 in_seq=0 out_seq=0 2026-02-25T11:51:25.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.081+0000 7f1ddf7fe640 10 client.25997 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:25.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/131892247 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55aefeee5910 con 0x55aeff06c7d0 2026-02-25T11:51:25.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1ddf7fe640 10 client.25997 handle_command_reply: tid=0 2026-02-25T11:51:25.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1e31b41e00 1 client.25997 shutdown 2026-02-25T11:51:25.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1e31b41e00 1 -- 10.20.193.28:0/131892247 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 msgr2=0x55aeff064300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:25.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff064300 secure :-1 s=READY pgs=131 gs=2 cs=0 l=1 c_cookie=4625569675429471278 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1e1800c6a0 tx=0x7f1e180acdd0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1e31b41e00 1 -- 10.20.193.28:0/131892247 shutdown_connections 2026-02-25T11:51:25.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aeff06c7d0 0x55aeff04a470 crc :-1 s=READY pgs=80 gs=4 cs=0 l=0 c_cookie=14865622363418225312 s_cookie=7090194786823121783 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff04c6f0 0x55aeff069830 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007a50 0x55aeff064840 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/131892247 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff064300 unknown :-1 s=CLOSED pgs=131 gs=2 cs=0 l=1 c_cookie=4625569675429471278 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1ddf7fe640 0 client.25997 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:25.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1e31b41e00 1 -- 10.20.193.28:0/131892247 >> 10.20.193.28:0/131892247 conn(0x55aefef26d40 msgr2=0x55aeff005230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:25.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1e31b41e00 1 -- 10.20.193.28:0/131892247 shutdown_connections 2026-02-25T11:51:25.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1e31b41e00 1 -- 10.20.193.28:0/131892247 wait complete. 2026-02-25T11:51:25.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.082+0000 7f1e31b41e00 20 client.25997 trim_cache size 0 max 16384 2026-02-25T11:51:25.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.091+0000 7f1e31b41e00 1 Processor -- start 2026-02-25T11:51:25.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.091+0000 7f1e31b41e00 1 -- start start 2026-02-25T11:51:25.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e31b41e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff04f1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e31b41e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff007a50 0x55aeff0614d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e31b41e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff04c6f0 0x55aeff061a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e31b41e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55aefef11470 con 0x55aeff007a50 2026-02-25T11:51:25.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e31b41e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55aefebd1700 con 0x55aeff007140 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e31b41e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55aeff064e20 con 0x55aeff04c6f0 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e2d8bb640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff04f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e2d0ba640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff007a50 0x55aeff0614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e2d8bb640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff04f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:34700/0 (socket says 10.20.193.28:34700) 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e2d8bb640 1 -- 10.20.193.28:0/2648724978 learned_addr learned my addr 10.20.193.28:0/2648724978 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e2e0bc640 1 --2- 10.20.193.28:0/2648724978 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff04c6f0 0x55aeff061a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e2d8bb640 1 -- 10.20.193.28:0/2648724978 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff04c6f0 msgr2=0x55aeff061a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e2d8bb640 1 --2- 10.20.193.28:0/2648724978 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff04c6f0 0x55aeff061a10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e2d8bb640 1 -- 10.20.193.28:0/2648724978 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff007a50 msgr2=0x55aeff0614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e2d8bb640 1 --2- 10.20.193.28:0/2648724978 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff007a50 0x55aeff0614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e2d8bb640 1 -- 10.20.193.28:0/2648724978 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aefeea3bd0 con 0x55aeff007140 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.092+0000 7f1e2d8bb640 1 --2- 10.20.193.28:0/2648724978 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff04f1e0 secure :-1 s=READY pgs=132 gs=1 cs=0 l=1 c_cookie=4473801802580803643 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1e180a9220 tx=0x7f1e1806e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e1623094654c03b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.093+0000 7f1df5ce5640 1 -- 10.20.193.28:0/2648724978 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1e180ad8e0 con 0x55aeff007140 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.093+0000 7f1df5ce5640 1 -- 10.20.193.28:0/2648724978 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55aefeba96f0 con 0x55aeff007140 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.093+0000 7f1df5ce5640 1 -- 10.20.193.28:0/2648724978 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1e180b2970 con 0x55aeff007140 2026-02-25T11:51:25.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.093+0000 7f1e31b41e00 1 -- 10.20.193.28:0/2648724978 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 msgr2=0x55aeff04f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:25.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.093+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/2648724978 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff04f1e0 secure :-1 s=READY pgs=132 gs=1 cs=0 l=1 c_cookie=4473801802580803643 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1e180a9220 tx=0x7f1e1806e000 comp rx=0 tx=0).stop 2026-02-25T11:51:25.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.093+0000 7f1e31b41e00 1 -- 10.20.193.28:0/2648724978 shutdown_connections 2026-02-25T11:51:25.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.093+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/2648724978 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff04c6f0 0x55aeff061a10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.093+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/2648724978 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff007a50 0x55aeff0614d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.093+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/2648724978 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff007140 0x55aeff04f1e0 unknown :-1 s=CLOSED pgs=132 gs=1 cs=0 l=1 c_cookie=4473801802580803643 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.093+0000 7f1e31b41e00 1 -- 10.20.193.28:0/2648724978 >> 10.20.193.28:0/2648724978 conn(0x55aefef26d40 msgr2=0x55aeff04ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:25.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.093+0000 7f1e31b41e00 1 -- 10.20.193.28:0/2648724978 shutdown_connections 2026-02-25T11:51:25.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.093+0000 7f1e31b41e00 1 -- 10.20.193.28:0/2648724978 wait complete. 2026-02-25T11:51:25.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.094+0000 7f1e31b41e00 1 -- start start 2026-02-25T11:51:25.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.094+0000 7f1e31b41e00 1 Processor -- start 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.094+0000 7f1e31b41e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007140 0x55aefefeaa40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.094+0000 7f1e31b41e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff007a50 0x55aefefeaf80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.094+0000 7f1e31b41e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff04c6f0 0x55aeff047010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.094+0000 7f1e31b41e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55aefef777a0 con 0x55aeff007a50 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.094+0000 7f1e31b41e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55aeff0475b0 con 0x55aeff04c6f0 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.094+0000 7f1e31b41e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55aeff047770 con 0x55aeff007140 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.094+0000 7f1e2d0ba640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff007a50 0x55aefefeaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.094+0000 7f1e2e0bc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff04c6f0 0x55aeff047010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.094+0000 7f1e2d8bb640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007140 0x55aefefeaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.094+0000 7f1e2d0ba640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff007a50 0x55aefefeaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:51284/0 (socket says 10.20.193.28:51284) 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.094+0000 7f1e2d0ba640 1 -- 10.20.193.28:0/1925244123 learned_addr learned my addr 10.20.193.28:0/1925244123 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1e2e0bc640 1 -- 10.20.193.28:0/1925244123 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007140 msgr2=0x55aefefeaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1e2e0bc640 1 --2- 10.20.193.28:0/1925244123 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007140 0x55aefefeaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1e2e0bc640 1 -- 10.20.193.28:0/1925244123 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff007a50 msgr2=0x55aefefeaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1e2e0bc640 1 --2- 10.20.193.28:0/1925244123 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff007a50 0x55aefefeaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1e2e0bc640 1 -- 10.20.193.28:0/1925244123 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aefee97680 con 0x55aeff04c6f0 2026-02-25T11:51:25.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1e2d8bb640 1 --2- 10.20.193.28:0/1925244123 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007140 0x55aefefeaa40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1e2d0ba640 1 --2- 10.20.193.28:0/1925244123 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff007a50 0x55aefefeaf80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1e2e0bc640 1 --2- 10.20.193.28:0/1925244123 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff04c6f0 0x55aeff047010 secure :-1 s=READY pgs=133 gs=2 cs=0 l=1 c_cookie=11509320592207066944 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x55aefee970a0 tx=0x7f1e240396a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fb9521980973740 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1ddf7fe640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/1925244123 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1e2403e040 con 0x55aeff04c6f0 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1e31b41e00 1 -- 10.20.193.28:0/1925244123 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aefeea9150 con 0x55aeff04c6f0 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/1925244123 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1e2403f040 con 0x55aeff04c6f0 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/1925244123 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1e2404e400 con 0x55aeff04c6f0 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1e31b41e00 10 client.26003 _ll_register_callbacks cb 0x55aeff007030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1e31b41e00 1 -- 10.20.193.28:0/1925244123 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e1c0198c0 con 0x55aeff04c6f0 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.095+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/1925244123 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1e2404e590 con 0x55aeff04c6f0 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.096+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/1925244123 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1e24039920 con 0x55aeff04c6f0 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.096+0000 7f1e31b41e00 10 client.26003 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.096+0000 7f1e31b41e00 1 -- 10.20.193.28:0/1925244123 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55aefeea93b0 con 0x55aeff04c6f0 2026-02-25T11:51:25.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.096+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/1925244123 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f1e2404c020 con 0x55aeff04c6f0 2026-02-25T11:51:25.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.096+0000 7f1e31b41e00 10 client.26003 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:25.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.096+0000 7f1e31b41e00 10 client.26003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:25.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.096+0000 7f1e31b41e00 20 client.26003 populate_metadata read hostname 'trial028' 2026-02-25T11:51:25.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.096+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/1925244123 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aeff0f9fd0 0x55aeff11a430 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:25.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.096+0000 7f1e31b41e00 4 client.26003 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:51:25.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.096+0000 7f1e31b41e00 1 -- 10.20.193.28:0/1925244123 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aefeba6f40 con 0x55aeff0f9fd0 2026-02-25T11:51:25.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.096+0000 7f1e2d8bb640 1 --2- 10.20.193.28:0/1925244123 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aeff0f9fd0 0x55aeff11a430 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:25.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1e2d8bb640 1 --2- 10.20.193.28:0/1925244123 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aeff0f9fd0 0x55aeff11a430 crc :-1 s=READY pgs=81 gs=4 cs=0 l=0 c_cookie=12302878587686121440 s_cookie=13940992517454993283 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aabc9ae45dca6fe0 server_cookie=c1785ad503489783 in_seq=0 out_seq=0 2026-02-25T11:51:25.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1ddf7fe640 10 client.26003 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:25.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1ddf7fe640 1 -- 10.20.193.28:0/1925244123 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x55aefeba6f40 con 0x55aeff0f9fd0 2026-02-25T11:51:25.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1ddf7fe640 10 client.26003 handle_command_reply: tid=0 2026-02-25T11:51:25.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1e31b41e00 1 client.26003 shutdown 2026-02-25T11:51:25.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1e31b41e00 1 -- 10.20.193.28:0/1925244123 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff04c6f0 msgr2=0x55aeff047010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:25.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/1925244123 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff04c6f0 0x55aeff047010 secure :-1 s=READY pgs=133 gs=2 cs=0 l=1 c_cookie=11509320592207066944 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x55aefee970a0 tx=0x7f1e240396a0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1e31b41e00 1 -- 10.20.193.28:0/1925244123 shutdown_connections 2026-02-25T11:51:25.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/1925244123 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55aeff0f9fd0 0x55aeff11a430 crc :-1 s=READY pgs=81 gs=4 cs=0 l=0 c_cookie=12302878587686121440 s_cookie=13940992517454993283 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/1925244123 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55aeff04c6f0 0x55aeff047010 unknown :-1 s=CLOSED pgs=133 gs=2 cs=0 l=1 c_cookie=11509320592207066944 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/1925244123 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55aeff007a50 0x55aefefeaf80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1e31b41e00 1 --2- 10.20.193.28:0/1925244123 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55aeff007140 0x55aefefeaa40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1ddf7fe640 0 client.26003 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:25.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1e31b41e00 1 -- 10.20.193.28:0/1925244123 >> 10.20.193.28:0/1925244123 conn(0x55aefef26d40 msgr2=0x55aeff068690 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:25.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1e31b41e00 1 -- 10.20.193.28:0/1925244123 shutdown_connections 2026-02-25T11:51:25.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.097+0000 7f1e31b41e00 1 -- 10.20.193.28:0/1925244123 wait complete. 2026-02-25T11:51:25.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.098+0000 7f1e31b41e00 20 client.26003 trim_cache size 0 max 16384 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "11ba92dd-980a-495c-a63e-d2ded1de1c9f", 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:25.099 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:51:25.100 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.098+0000 7f1e2fb46640 1 -- 10.20.193.28:0/130869764 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1e040b5760 msgr2=0x7f1e040d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.098+0000 7f1e2fb46640 1 --2- 10.20.193.28:0/130869764 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1e040b5760 0x7f1e040d5c10 secure :-1 s=READY pgs=302 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1e280fc3e0 tx=0x7f1e1c005f90 comp rx=0 tx=0).stop 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.098+0000 7f1e2fb46640 1 -- 10.20.193.28:0/130869764 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e2810f740 msgr2=0x7f1e280f5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.098+0000 7f1e2fb46640 1 --2- 10.20.193.28:0/130869764 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e2810f740 0x7f1e280f5a40 secure :-1 s=READY pgs=551 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1e1800e7b0 tx=0x7f1e1800ec80 comp rx=0 tx=0).stop 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.098+0000 7f1e2fb46640 1 -- 10.20.193.28:0/130869764 shutdown_connections 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.098+0000 7f1e2fb46640 1 --2- 10.20.193.28:0/130869764 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1e040b5760 0x7f1e040d5c10 unknown :-1 s=CLOSED pgs=302 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.098+0000 7f1e2fb46640 1 --2- 10.20.193.28:0/130869764 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1e28115930 0x7f1e280f60d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.098+0000 7f1e2fb46640 1 --2- 10.20.193.28:0/130869764 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1e28110130 0x7f1e28176200 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.098+0000 7f1e2fb46640 1 --2- 10.20.193.28:0/130869764 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1e2810f740 0x7f1e280f5a40 unknown :-1 s=CLOSED pgs=551 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.098+0000 7f1e2fb46640 1 -- 10.20.193.28:0/130869764 >> 10.20.193.28:0/130869764 conn(0x7f1e280ecff0 msgr2=0x7f1e2810dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.098+0000 7f1e2fb46640 1 -- 10.20.193.28:0/130869764 shutdown_connections 2026-02-25T11:51:25.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:25.098+0000 7f1e2fb46640 1 -- 10.20.193.28:0/130869764 wait complete. 2026-02-25T11:51:25.196 INFO:tasks.cephfs.filesystem:scrub status for tag:11ba92dd-980a-495c-a63e-d2ded1de1c9f - {'path': '/', 'tag': '11ba92dd-980a-495c-a63e-d2ded1de1c9f', 'options': 'recursive,force'} 2026-02-25T11:51:25.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:25 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:51:25.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:25 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:51:25.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:25 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:51:25.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:25 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:51:25.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:25 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:51:25.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:25 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:51:25.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:25 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:51:25.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:25 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:51:25.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:25 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:51:25.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:25 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:51:25.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:25 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:51:25.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:25 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:51:26.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:26 trial167 ceph-mon[43392]: pgmap v322: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:26.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:26 trial119 ceph-mon[42710]: pgmap v322: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:26.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:26 trial028 ceph-mon[39224]: pgmap v322: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:27.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:27 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:51:27.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:27 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:51:27.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:27 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:51:28.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:28 trial167 ceph-mon[43392]: pgmap v323: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:28.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:28 trial119 ceph-mon[42710]: pgmap v323: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:28.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:28 trial028 ceph-mon[39224]: pgmap v323: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:30.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:30 trial167 ceph-mon[43392]: pgmap v324: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:51:30.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:30 trial119 ceph-mon[42710]: pgmap v324: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:51:30.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:30 trial028 ceph-mon[39224]: pgmap v324: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:51:32.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:32 trial167 ceph-mon[43392]: pgmap v325: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:51:32.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:32 trial119 ceph-mon[42710]: pgmap v325: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:51:32.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:32 trial028 ceph-mon[39224]: pgmap v325: 97 pgs: 97 active+clean; 2.8 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:51:32.849 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.849 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.849 INFO:tasks.workunit.client.0.trial028.stdout:127/600: open fd = 3 2026-02-25T11:51:32.849 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.849 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.849 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.849 INFO:tasks.workunit.client.0.trial028.stdout:128/600: open fd = 3 2026-02-25T11:51:32.849 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.849 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.849 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.849 INFO:tasks.workunit.client.0.trial028.stdout:129/600: open fd = 3 2026-02-25T11:51:32.849 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:130/600: open fd = 3 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:131/600: open fd = 3 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:132/600: open fd = 3 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:133/600: open fd = 3 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.850 INFO:tasks.workunit.client.0.trial028.stdout:134/600: open fd = 3 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:135/600: open fd = 3 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:136/600: open fd = 3 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:137/600: open fd = 3 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:138/600: open fd = 3 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:139/600: open fd = 3 2026-02-25T11:51:32.851 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:140/600: open fd = 3 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:141/600: open fd = 3 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:142/600: open fd = 3 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:143/600: open fd = 3 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.852 INFO:tasks.workunit.client.0.trial028.stdout:144/600: open fd = 3 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:145/600: open fd = 3 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:146/600: open fd = 3 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:147/600: open fd = 3 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:148/600: open fd = 3 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.853 INFO:tasks.workunit.client.0.trial028.stdout:149/600: open fd = 3 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:150/600: open fd = 3 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:151/600: open fd = 3 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:152/600: open fd = 3 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:153/600: open fd = 3 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:154/600: open fd = 3 2026-02-25T11:51:32.854 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:155/600: open fd = 3 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:156/600: open fd = 3 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:157/600: open fd = 3 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:158/600: open fd = 3 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:159/600: open fd = 3 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.855 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:160/600: open fd = 3 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:161/600: open fd = 3 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:162/600: open fd = 3 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:163/600: open fd = 3 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:164/600: open fd = 3 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:165/600: open fd = 3 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.856 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:166/600: open fd = 3 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:167/600: open fd = 3 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:168/600: open fd = 3 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:169/600: open fd = 3 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:170/600: open fd = 3 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.857 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:171/600: open fd = 3 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:172/600: open fd = 3 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:173/600: open fd = 3 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:174/600: open fd = 3 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:175/600: open fd = 3 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.858 INFO:tasks.workunit.client.0.trial028.stdout:176/600: open fd = 3 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:177/600: open fd = 3 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:178/600: open fd = 3 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:179/600: open fd = 3 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:180/600: open fd = 3 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:181/600: open fd = 3 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:182/600: open fd = 3 2026-02-25T11:51:32.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:183/600: open fd = 3 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:184/600: open fd = 3 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:185/600: open fd = 3 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:186/600: open fd = 3 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:187/600: open fd = 3 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:188/600: open fd = 3 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:32.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:51:34.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:34 trial167 ceph-mon[43392]: pgmap v326: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:51:34.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:34 trial119 ceph-mon[42710]: pgmap v326: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:51:34.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:34 trial028 ceph-mon[39224]: pgmap v326: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:51:36.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:36 trial167 ceph-mon[43392]: pgmap v327: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:51:36.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:36 trial119 ceph-mon[42710]: pgmap v327: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:51:36.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:36 trial028 ceph-mon[39224]: pgmap v327: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:51:38.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:38 trial167 ceph-mon[43392]: pgmap v328: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:38.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:38 trial119 ceph-mon[42710]: pgmap v328: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:38.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:38 trial028 ceph-mon[39224]: pgmap v328: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:51:40.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:40 trial167 ceph-mon[43392]: pgmap v329: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 11 op/s 2026-02-25T11:51:40.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:40 trial119 ceph-mon[42710]: pgmap v329: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 11 op/s 2026-02-25T11:51:40.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:40 trial028 ceph-mon[39224]: pgmap v329: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 11 op/s 2026-02-25T11:51:42.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:42 trial167 ceph-mon[43392]: pgmap v330: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:51:42.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:42 trial119 ceph-mon[42710]: pgmap v330: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:51:42.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:42 trial028 ceph-mon[39224]: pgmap v330: 97 pgs: 97 active+clean; 2.9 MiB data, 337 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:51:44.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:44 trial167 ceph-mon[43392]: pgmap v331: 97 pgs: 97 active+clean; 3.0 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:44.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:44 trial119 ceph-mon[42710]: pgmap v331: 97 pgs: 97 active+clean; 3.0 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:44.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:44 trial028 ceph-mon[39224]: pgmap v331: 97 pgs: 97 active+clean; 3.0 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:46.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:46 trial167 ceph-mon[43392]: pgmap v332: 97 pgs: 97 active+clean; 3.0 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 10 op/s 2026-02-25T11:51:46.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:46 trial119 ceph-mon[42710]: pgmap v332: 97 pgs: 97 active+clean; 3.0 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 10 op/s 2026-02-25T11:51:46.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:46 trial028 ceph-mon[39224]: pgmap v332: 97 pgs: 97 active+clean; 3.0 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 10 op/s 2026-02-25T11:51:48.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:48 trial167 ceph-mon[43392]: pgmap v333: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2026-02-25T11:51:48.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:48 trial119 ceph-mon[42710]: pgmap v333: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2026-02-25T11:51:48.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:48 trial028 ceph-mon[39224]: pgmap v333: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2026-02-25T11:51:50.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:50 trial167 ceph-mon[43392]: pgmap v334: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:51:50.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:50 trial119 ceph-mon[42710]: pgmap v334: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:51:50.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:50 trial028 ceph-mon[39224]: pgmap v334: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:51:52.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:52 trial167 ceph-mon[43392]: pgmap v335: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:51:52.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:52 trial119 ceph-mon[42710]: pgmap v335: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:51:52.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:52 trial028 ceph-mon[39224]: pgmap v335: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:51:54.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:54 trial167 ceph-mon[43392]: pgmap v336: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:54.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:54 trial119 ceph-mon[42710]: pgmap v336: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:54.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:54 trial028 ceph-mon[39224]: pgmap v336: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:55.197 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:51:55.386 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:55.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.569+0000 7f6a7204d640 1 -- 10.20.193.28:0/2319931473 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6a6c114a90 msgr2=0x7f6a6c119580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:55.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.569+0000 7f6a7204d640 1 --2- 10.20.193.28:0/2319931473 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6a6c114a90 0x7f6a6c119580 secure :-1 s=READY pgs=552 gs=3 cs=0 l=1 c_cookie=458337532023636702 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6a60009950 tx=0x7f6a6002f190 comp rx=0 tx=0).stop 2026-02-25T11:51:55.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.569+0000 7f6a7204d640 1 -- 10.20.193.28:0/2319931473 shutdown_connections 2026-02-25T11:51:55.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.569+0000 7f6a7204d640 1 --2- 10.20.193.28:0/2319931473 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6a6c119d30 0x7f6a6c17a700 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.569+0000 7f6a7204d640 1 --2- 10.20.193.28:0/2319931473 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6a6c114a90 0x7f6a6c119580 unknown :-1 s=CLOSED pgs=552 gs=3 cs=0 l=1 c_cookie=458337532023636702 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.569+0000 7f6a7204d640 1 --2- 10.20.193.28:0/2319931473 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6a6c1140a0 0x7f6a6c114470 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.569+0000 7f6a7204d640 1 -- 10.20.193.28:0/2319931473 >> 10.20.193.28:0/2319931473 conn(0x7f6a6c064c40 msgr2=0x7f6a6c065040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:55.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.570+0000 7f6a7204d640 1 -- 10.20.193.28:0/2319931473 shutdown_connections 2026-02-25T11:51:55.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.570+0000 7f6a7204d640 1 -- 10.20.193.28:0/2319931473 wait complete. 2026-02-25T11:51:55.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.570+0000 7f6a7204d640 1 Processor -- start 2026-02-25T11:51:55.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.570+0000 7f6a7204d640 1 -- start start 2026-02-25T11:51:55.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7204d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6a6c1140a0 0x7f6a6c122550 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7204d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6a6c114a90 0x7f6a6c122a90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7204d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6a6c119d30 0x7f6a6c127e60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7204d640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c128480 con 0x7f6a6c114a90 2026-02-25T11:51:55.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7204d640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c1285f0 con 0x7f6a6c119d30 2026-02-25T11:51:55.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7204d640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c1287b0 con 0x7f6a6c1140a0 2026-02-25T11:51:55.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7184c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6a6c119d30 0x7f6a6c127e60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7104b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6a6c1140a0 0x7f6a6c122550 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7184c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6a6c119d30 0x7f6a6c127e60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:36148/0 (socket says 10.20.193.28:36148) 2026-02-25T11:51:55.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7184c640 1 -- 10.20.193.28:0/1177011141 learned_addr learned my addr 10.20.193.28:0/1177011141 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:55.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7184c640 1 -- 10.20.193.28:0/1177011141 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6a6c1140a0 msgr2=0x7f6a6c122550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:55.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7184c640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6a6c1140a0 0x7f6a6c122550 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7184c640 1 -- 10.20.193.28:0/1177011141 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6a6c114a90 msgr2=0x7f6a6c122a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:55.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7084a640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6a6c114a90 0x7f6a6c122a90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7184c640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6a6c114a90 0x7f6a6c122a90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7184c640 1 -- 10.20.193.28:0/1177011141 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a60009570 con 0x7f6a6c119d30 2026-02-25T11:51:55.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.571+0000 7f6a7084a640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6a6c114a90 0x7f6a6c122a90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:51:55.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.572+0000 7f6a7184c640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6a6c119d30 0x7f6a6c127e60 secure :-1 s=READY pgs=134 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6a6800f8d0 tx=0x7f6a6800fda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:55.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.572+0000 7f6a5a7fc640 1 -- 10.20.193.28:0/1177011141 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6a6800dce0 con 0x7f6a6c119d30 2026-02-25T11:51:55.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.572+0000 7f6a5a7fc640 1 -- 10.20.193.28:0/1177011141 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6a6800de40 con 0x7f6a6c119d30 2026-02-25T11:51:55.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.572+0000 7f6a5a7fc640 1 -- 10.20.193.28:0/1177011141 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6a680116b0 con 0x7f6a6c119d30 2026-02-25T11:51:55.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.572+0000 7f6a7204d640 1 -- 10.20.193.28:0/1177011141 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6c128a90 con 0x7f6a6c119d30 2026-02-25T11:51:55.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.572+0000 7f6a7204d640 1 -- 10.20.193.28:0/1177011141 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a6c169190 con 0x7f6a6c119d30 2026-02-25T11:51:55.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.575+0000 7f6a5a7fc640 1 -- 10.20.193.28:0/1177011141 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f6a68020060 con 0x7f6a6c119d30 2026-02-25T11:51:55.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.577+0000 7f6a5a7fc640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6a480b56e0 0x7f6a480d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.577+0000 7f6a5a7fc640 1 -- 10.20.193.28:0/1177011141 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6a68015070 con 0x7f6a6c119d30 2026-02-25T11:51:55.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.577+0000 7f6a7104b640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6a480b56e0 0x7f6a480d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.577+0000 7f6a752d1e00 1 Processor -- start 2026-02-25T11:51:55.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.577+0000 7f6a7104b640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6a480b56e0 0x7f6a480d5b90 secure :-1 s=READY pgs=303 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6a5c004740 tx=0x7f6a5c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:55.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.577+0000 7f6a752d1e00 1 -- start start 2026-02-25T11:51:55.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a752d1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1fa140 0x559e0f1fa510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a752d1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f1faa50 0x559e0f23f1b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a752d1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f23f6f0 0x559e0f25fad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a752d1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559e0f1109b0 con 0x559e0f1faa50 2026-02-25T11:51:55.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a752d1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559e0f10c6e0 con 0x559e0f23f6f0 2026-02-25T11:51:55.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a752d1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559e0f106840 con 0x559e0f1fa140 2026-02-25T11:51:55.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a7104b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1fa140 0x559e0f1fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a7184c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f23f6f0 0x559e0f25fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a7104b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1fa140 0x559e0f1fa510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48778/0 (socket says 10.20.193.28:48778) 2026-02-25T11:51:55.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a7104b640 1 -- 10.20.193.28:0/4181940293 learned_addr learned my addr 10.20.193.28:0/4181940293 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:55.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a7104b640 1 -- 10.20.193.28:0/4181940293 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f23f6f0 msgr2=0x559e0f25fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:55.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a7104b640 1 --2- 10.20.193.28:0/4181940293 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f23f6f0 0x559e0f25fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a7104b640 1 -- 10.20.193.28:0/4181940293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f1faa50 msgr2=0x559e0f23f1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-25T11:51:55.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a7084a640 1 --2- 10.20.193.28:0/4181940293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f1faa50 0x559e0f23f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a7104b640 1 --2- 10.20.193.28:0/4181940293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f1faa50 0x559e0f23f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.578+0000 7f6a7104b640 1 -- 10.20.193.28:0/4181940293 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0f0b1d80 con 0x559e0f1fa140 2026-02-25T11:51:55.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.579+0000 7f6a7084a640 1 --2- 10.20.193.28:0/4181940293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f1faa50 0x559e0f23f1b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:51:55.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.579+0000 7f6a7104b640 1 --2- 10.20.193.28:0/4181940293 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1fa140 0x559e0f1fa510 secure :-1 s=READY pgs=159 gs=1 cs=0 l=1 c_cookie=16078939393910669541 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6a5c009cc0 tx=0x7f6a5c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df23de305b1480e5 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:55.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.579+0000 7f6a39ce5640 1 -- 10.20.193.28:0/4181940293 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6a5c011e60 con 0x559e0f1fa140 2026-02-25T11:51:55.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.579+0000 7f6a39ce5640 1 -- 10.20.193.28:0/4181940293 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6a5c0198c0 con 0x559e0f1fa140 2026-02-25T11:51:55.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.579+0000 7f6a39ce5640 1 -- 10.20.193.28:0/4181940293 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6a5c01bae0 con 0x559e0f1fa140 2026-02-25T11:51:55.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.580+0000 7f6a752d1e00 1 -- 10.20.193.28:0/4181940293 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1fa140 msgr2=0x559e0f1fa510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:55.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.580+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/4181940293 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1fa140 0x559e0f1fa510 secure :-1 s=READY pgs=159 gs=1 cs=0 l=1 c_cookie=16078939393910669541 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6a5c009cc0 tx=0x7f6a5c014000 comp rx=0 tx=0).stop 2026-02-25T11:51:55.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.580+0000 7f6a752d1e00 1 -- 10.20.193.28:0/4181940293 shutdown_connections 2026-02-25T11:51:55.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.580+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/4181940293 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f23f6f0 0x559e0f25fad0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.580+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/4181940293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f1faa50 0x559e0f23f1b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.580+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/4181940293 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1fa140 0x559e0f1fa510 unknown :-1 s=CLOSED pgs=159 gs=1 cs=0 l=1 c_cookie=16078939393910669541 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.580+0000 7f6a752d1e00 1 -- 10.20.193.28:0/4181940293 >> 10.20.193.28:0/4181940293 conn(0x559e0f119d40 msgr2=0x559e0f1f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:55.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.580+0000 7f6a752d1e00 1 -- 10.20.193.28:0/4181940293 shutdown_connections 2026-02-25T11:51:55.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.580+0000 7f6a752d1e00 1 -- 10.20.193.28:0/4181940293 wait complete. 2026-02-25T11:51:55.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.581+0000 7f6a752d1e00 1 -- start start 2026-02-25T11:51:55.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.581+0000 7f6a752d1e00 1 Processor -- start 2026-02-25T11:51:55.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a752d1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f1fa140 0x559e0f257460 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a752d1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1faa50 0x559e0f2579a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a752d1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f23f6f0 0x559e0f259730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a752d1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559e0f259ca0 con 0x559e0f1fa140 2026-02-25T11:51:55.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a752d1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559e0f259e10 con 0x559e0f23f6f0 2026-02-25T11:51:55.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a752d1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559e0f259fd0 con 0x559e0f1faa50 2026-02-25T11:51:55.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a7104b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f1fa140 0x559e0f257460 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a7104b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f1fa140 0x559e0f257460 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35580/0 (socket says 10.20.193.28:35580) 2026-02-25T11:51:55.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a7104b640 1 -- 10.20.193.28:0/2003041646 learned_addr learned my addr 10.20.193.28:0/2003041646 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:55.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a7184c640 1 --2- 10.20.193.28:0/2003041646 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f23f6f0 0x559e0f259730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a7104b640 1 -- 10.20.193.28:0/2003041646 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1faa50 msgr2=0x559e0f2579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:55.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a7104b640 1 --2- 10.20.193.28:0/2003041646 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1faa50 0x559e0f2579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a7104b640 1 -- 10.20.193.28:0/2003041646 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f23f6f0 msgr2=0x559e0f259730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:55.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a7104b640 1 --2- 10.20.193.28:0/2003041646 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f23f6f0 0x559e0f259730 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a7104b640 1 -- 10.20.193.28:0/2003041646 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0f08a350 con 0x559e0f1fa140 2026-02-25T11:51:55.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a7104b640 1 --2- 10.20.193.28:0/2003041646 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f1fa140 0x559e0f257460 secure :-1 s=READY pgs=553 gs=1 cs=0 l=1 c_cookie=16094726477201817617 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6a5c0041e0 tx=0x7f6a5c012ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5bf4751eb04011 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:55.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.582+0000 7f6a26ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:55.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a26ffd640 1 -- 10.20.193.28:0/2003041646 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6a5c019a30 con 0x559e0f1fa140 2026-02-25T11:51:55.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a752d1e00 1 -- 10.20.193.28:0/2003041646 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5c009960 con 0x559e0f1fa140 2026-02-25T11:51:55.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a752d1e00 10 client.16224 _ll_register_callbacks cb 0x559e0f0879a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:55.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a752d1e00 1 -- 10.20.193.28:0/2003041646 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e0ed9c6f0 con 0x559e0f1fa140 2026-02-25T11:51:55.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a26ffd640 1 -- 10.20.193.28:0/2003041646 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6a5c019b90 con 0x559e0f1fa140 2026-02-25T11:51:55.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a26ffd640 1 -- 10.20.193.28:0/2003041646 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6a5c0243b0 con 0x559e0f1fa140 2026-02-25T11:51:55.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a26ffd640 1 -- 10.20.193.28:0/2003041646 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6a5c029070 con 0x559e0f1fa140 2026-02-25T11:51:55.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a26ffd640 1 -- 10.20.193.28:0/2003041646 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6a5c024540 con 0x559e0f1fa140 2026-02-25T11:51:55.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a752d1e00 10 client.16224 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:55.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a752d1e00 1 -- 10.20.193.28:0/2003041646 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x559e0f09c150 con 0x559e0f1fa140 2026-02-25T11:51:55.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a26ffd640 1 -- 10.20.193.28:0/2003041646 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6a5c035070 con 0x559e0f1fa140 2026-02-25T11:51:55.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a752d1e00 10 client.16224 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:55.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a752d1e00 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:55.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a752d1e00 20 client.16224 populate_metadata read hostname 'trial028' 2026-02-25T11:51:55.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/2003041646 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559e0f25f930 0x559e0f23d420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a752d1e00 4 client.16224 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:51:55.590 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.583+0000 7f6a752d1e00 1 -- 10.20.193.28:0/2003041646 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e0f0d8910 con 0x559e0f25f930 2026-02-25T11:51:55.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.584+0000 7f6a7084a640 1 --2- 10.20.193.28:0/2003041646 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559e0f25f930 0x559e0f23d420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.584+0000 7f6a7084a640 1 --2- 10.20.193.28:0/2003041646 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559e0f25f930 0x559e0f23d420 crc :-1 s=READY pgs=82 gs=4 cs=0 l=0 c_cookie=3107957070629377708 s_cookie=6923543627724039683 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b21ae7d340716ac server_cookie=60155e3f35692603 in_seq=0 out_seq=0 2026-02-25T11:51:55.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.584+0000 7f6a26ffd640 10 client.16224 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:55.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a26ffd640 1 -- 10.20.193.28:0/2003041646 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x559e0f0d8910 con 0x559e0f25f930 2026-02-25T11:51:55.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a26ffd640 10 client.16224 handle_command_reply: tid=0 2026-02-25T11:51:55.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a752d1e00 1 client.16224 shutdown 2026-02-25T11:51:55.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a752d1e00 1 -- 10.20.193.28:0/2003041646 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f1fa140 msgr2=0x559e0f257460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:55.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/2003041646 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f1fa140 0x559e0f257460 secure :-1 s=READY pgs=553 gs=1 cs=0 l=1 c_cookie=16094726477201817617 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6a5c0041e0 tx=0x7f6a5c012ef0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a752d1e00 1 -- 10.20.193.28:0/2003041646 shutdown_connections 2026-02-25T11:51:55.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/2003041646 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559e0f25f930 0x559e0f23d420 crc :-1 s=READY pgs=82 gs=4 cs=0 l=0 c_cookie=3107957070629377708 s_cookie=6923543627724039683 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/2003041646 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f23f6f0 0x559e0f259730 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/2003041646 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1faa50 0x559e0f2579a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/2003041646 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f1fa140 0x559e0f257460 unknown :-1 s=CLOSED pgs=553 gs=1 cs=0 l=1 c_cookie=16094726477201817617 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a26ffd640 0 client.16224 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:55.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a752d1e00 1 -- 10.20.193.28:0/2003041646 >> 10.20.193.28:0/2003041646 conn(0x559e0f119d40 msgr2=0x559e0f1f8230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:55.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a752d1e00 1 -- 10.20.193.28:0/2003041646 shutdown_connections 2026-02-25T11:51:55.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a752d1e00 1 -- 10.20.193.28:0/2003041646 wait complete. 2026-02-25T11:51:55.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.585+0000 7f6a752d1e00 20 client.16224 trim_cache size 0 max 16384 2026-02-25T11:51:55.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a752d1e00 1 Processor -- start 2026-02-25T11:51:55.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a752d1e00 1 -- start start 2026-02-25T11:51:55.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a752d1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f1fa140 0x559e0f1f9650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a752d1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1faa50 0x559e0f1f9b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a752d1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f23f6f0 0x559e0f2545d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a752d1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559e0f104470 con 0x559e0f23f6f0 2026-02-25T11:51:55.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a752d1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559e0edc4700 con 0x559e0f1fa140 2026-02-25T11:51:55.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a752d1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559e0f25d9c0 con 0x559e0f1faa50 2026-02-25T11:51:55.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a7104b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f1fa140 0x559e0f1f9650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a7104b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f1fa140 0x559e0f1f9650 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:36168/0 (socket says 10.20.193.28:36168) 2026-02-25T11:51:55.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a7104b640 1 -- 10.20.193.28:0/3832590540 learned_addr learned my addr 10.20.193.28:0/3832590540 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:55.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a7184c640 1 --2- 10.20.193.28:0/3832590540 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f23f6f0 0x559e0f2545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a7084a640 1 --2- 10.20.193.28:0/3832590540 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1faa50 0x559e0f1f9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a7184c640 1 -- 10.20.193.28:0/3832590540 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1faa50 msgr2=0x559e0f1f9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:55.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.595+0000 7f6a7184c640 1 --2- 10.20.193.28:0/3832590540 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1faa50 0x559e0f1f9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a7184c640 1 -- 10.20.193.28:0/3832590540 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f1fa140 msgr2=0x559e0f1f9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:55.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a7184c640 1 --2- 10.20.193.28:0/3832590540 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f1fa140 0x559e0f1f9650 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a7184c640 1 -- 10.20.193.28:0/3832590540 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0f096bd0 con 0x559e0f23f6f0 2026-02-25T11:51:55.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a7184c640 1 --2- 10.20.193.28:0/3832590540 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f23f6f0 0x559e0f2545d0 secure :-1 s=READY pgs=554 gs=3 cs=0 l=1 c_cookie=16318250011061202927 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6a680adff0 tx=0x7f6a6800a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27611ed61512fef server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:55.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a394e4640 1 -- 10.20.193.28:0/3832590540 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6a680aea10 con 0x559e0f23f6f0 2026-02-25T11:51:55.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a394e4640 1 -- 10.20.193.28:0/3832590540 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6a68097040 con 0x559e0f23f6f0 2026-02-25T11:51:55.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a394e4640 1 -- 10.20.193.28:0/3832590540 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6a680adbb0 con 0x559e0f23f6f0 2026-02-25T11:51:55.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a752d1e00 1 -- 10.20.193.28:0/3832590540 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f23f6f0 msgr2=0x559e0f2545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:55.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/3832590540 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f23f6f0 0x559e0f2545d0 secure :-1 s=READY pgs=554 gs=3 cs=0 l=1 c_cookie=16318250011061202927 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6a680adff0 tx=0x7f6a6800a4a0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a752d1e00 1 -- 10.20.193.28:0/3832590540 shutdown_connections 2026-02-25T11:51:55.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/3832590540 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f23f6f0 0x559e0f2545d0 unknown :-1 s=CLOSED pgs=554 gs=3 cs=0 l=1 c_cookie=16318250011061202927 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/3832590540 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1faa50 0x559e0f1f9b90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/3832590540 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f1fa140 0x559e0f1f9650 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a752d1e00 1 -- 10.20.193.28:0/3832590540 >> 10.20.193.28:0/3832590540 conn(0x559e0f119d40 msgr2=0x559e0f25d040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:55.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a752d1e00 1 -- 10.20.193.28:0/3832590540 shutdown_connections 2026-02-25T11:51:55.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.596+0000 7f6a752d1e00 1 -- 10.20.193.28:0/3832590540 wait complete. 2026-02-25T11:51:55.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.597+0000 7f6a752d1e00 1 -- start start 2026-02-25T11:51:55.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.597+0000 7f6a752d1e00 1 Processor -- start 2026-02-25T11:51:55.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.597+0000 7f6a752d1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1fa140 0x559e0f1d8220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.597+0000 7f6a752d1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f1faa50 0x559e0f1d8760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.597+0000 7f6a752d1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f23f6f0 0x559e0f1d8ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.597+0000 7f6a752d1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559e0f23ac90 con 0x559e0f23f6f0 2026-02-25T11:51:55.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.597+0000 7f6a752d1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559e0f23ae00 con 0x559e0f1faa50 2026-02-25T11:51:55.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.597+0000 7f6a752d1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559e0f23afc0 con 0x559e0f1fa140 2026-02-25T11:51:55.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.597+0000 7f6a7084a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f1faa50 0x559e0f1d8760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a7084a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f1faa50 0x559e0f1d8760 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:36184/0 (socket says 10.20.193.28:36184) 2026-02-25T11:51:55.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a7084a640 1 -- 10.20.193.28:0/1184381082 learned_addr learned my addr 10.20.193.28:0/1184381082 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:55.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a7184c640 1 --2- 10.20.193.28:0/1184381082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f23f6f0 0x559e0f1d8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a7104b640 1 --2- 10.20.193.28:0/1184381082 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1fa140 0x559e0f1d8220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a7084a640 1 -- 10.20.193.28:0/1184381082 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1fa140 msgr2=0x559e0f1d8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:55.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a7084a640 1 --2- 10.20.193.28:0/1184381082 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1fa140 0x559e0f1d8220 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a7084a640 1 -- 10.20.193.28:0/1184381082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f23f6f0 msgr2=0x559e0f1d8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:55.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a7084a640 1 --2- 10.20.193.28:0/1184381082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f23f6f0 0x559e0f1d8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a7084a640 1 -- 10.20.193.28:0/1184381082 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0f08a680 con 0x559e0f1faa50 2026-02-25T11:51:55.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a7084a640 1 --2- 10.20.193.28:0/1184381082 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f1faa50 0x559e0f1d8760 secure :-1 s=READY pgs=135 gs=2 cs=0 l=1 c_cookie=14018401458376100020 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x559e0f169b00 tx=0x7f6a60038470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c28b5dda8d5348b4 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:55.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a26ffd640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:51:55.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a26ffd640 1 -- 10.20.193.28:0/1184381082 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6a60004250 con 0x559e0f1faa50 2026-02-25T11:51:55.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a752d1e00 1 -- 10.20.193.28:0/1184381082 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a6c169190 con 0x559e0f1faa50 2026-02-25T11:51:55.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a752d1e00 10 client.26021 _ll_register_callbacks cb 0x559e0f0879a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:55.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a752d1e00 1 -- 10.20.193.28:0/1184381082 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a5c0198c0 con 0x559e0f1faa50 2026-02-25T11:51:55.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a26ffd640 1 -- 10.20.193.28:0/1184381082 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6a600043b0 con 0x559e0f1faa50 2026-02-25T11:51:55.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.598+0000 7f6a26ffd640 1 -- 10.20.193.28:0/1184381082 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6a60038c80 con 0x559e0f1faa50 2026-02-25T11:51:55.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.599+0000 7f6a26ffd640 1 -- 10.20.193.28:0/1184381082 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6a60038e10 con 0x559e0f1faa50 2026-02-25T11:51:55.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.599+0000 7f6a26ffd640 1 -- 10.20.193.28:0/1184381082 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6a60046510 con 0x559e0f1faa50 2026-02-25T11:51:55.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.599+0000 7f6a752d1e00 10 client.26021 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:55.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.599+0000 7f6a752d1e00 1 -- 10.20.193.28:0/1184381082 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x559e0f09c3b0 con 0x559e0f1faa50 2026-02-25T11:51:55.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.599+0000 7f6a26ffd640 1 -- 10.20.193.28:0/1184381082 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6a60043070 con 0x559e0f1faa50 2026-02-25T11:51:55.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.599+0000 7f6a752d1e00 10 client.26021 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:55.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.599+0000 7f6a752d1e00 10 client.26021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:55.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.599+0000 7f6a752d1e00 20 client.26021 populate_metadata read hostname 'trial028' 2026-02-25T11:51:55.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.599+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/1184381082 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559e0f2ed810 0x559e0f30dcc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:55.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.599+0000 7f6a752d1e00 4 client.26021 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:51:55.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.599+0000 7f6a752d1e00 1 -- 10.20.193.28:0/1184381082 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e0ed99f40 con 0x559e0f2ed810 2026-02-25T11:51:55.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.599+0000 7f6a7104b640 1 --2- 10.20.193.28:0/1184381082 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559e0f2ed810 0x559e0f30dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:55.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a7104b640 1 --2- 10.20.193.28:0/1184381082 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559e0f2ed810 0x559e0f30dcc0 crc :-1 s=READY pgs=83 gs=4 cs=0 l=0 c_cookie=13779367175141357314 s_cookie=17650974444824103176 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf3a257087fa7702 server_cookie=f4f4dbbd1b3d0908 in_seq=0 out_seq=0 2026-02-25T11:51:55.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a26ffd640 10 client.26021 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:55.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a26ffd640 1 -- 10.20.193.28:0/1184381082 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x559e0ed99f40 con 0x559e0f2ed810 2026-02-25T11:51:55.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a26ffd640 10 client.26021 handle_command_reply: tid=0 2026-02-25T11:51:55.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a752d1e00 1 client.26021 shutdown 2026-02-25T11:51:55.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a752d1e00 1 -- 10.20.193.28:0/1184381082 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f1faa50 msgr2=0x559e0f1d8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:55.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/1184381082 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f1faa50 0x559e0f1d8760 secure :-1 s=READY pgs=135 gs=2 cs=0 l=1 c_cookie=14018401458376100020 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x559e0f169b00 tx=0x7f6a60038470 comp rx=0 tx=0).stop 2026-02-25T11:51:55.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a752d1e00 1 -- 10.20.193.28:0/1184381082 shutdown_connections 2026-02-25T11:51:55.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/1184381082 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559e0f2ed810 0x559e0f30dcc0 crc :-1 s=READY pgs=83 gs=4 cs=0 l=0 c_cookie=13779367175141357314 s_cookie=17650974444824103176 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/1184381082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559e0f23f6f0 0x559e0f1d8ca0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/1184381082 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559e0f1faa50 0x559e0f1d8760 unknown :-1 s=CLOSED pgs=135 gs=2 cs=0 l=1 c_cookie=14018401458376100020 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a752d1e00 1 --2- 10.20.193.28:0/1184381082 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559e0f1fa140 0x559e0f1d8220 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a26ffd640 0 client.26021 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:55.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a752d1e00 1 -- 10.20.193.28:0/1184381082 >> 10.20.193.28:0/1184381082 conn(0x559e0f119d40 msgr2=0x559e0f256ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:55.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a752d1e00 1 -- 10.20.193.28:0/1184381082 shutdown_connections 2026-02-25T11:51:55.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a752d1e00 1 -- 10.20.193.28:0/1184381082 wait complete. 2026-02-25T11:51:55.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.600+0000 7f6a752d1e00 20 client.26021 trim_cache size 0 max 16384 2026-02-25T11:51:55.608 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:51:55.608 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:51:55.609 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:51:55.609 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:51:55.609 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:51:55.609 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:51:55.609 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:55.609 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:55.609 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:55.609 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:55.609 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:55.609 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:55.609 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:51:55.609 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:51:55.609 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:55.610 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:55.610 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:55.610 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:55.610 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:55.610 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:55.610 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:51:55.610 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:51:55.610 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:55.610 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:55.610 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:55.610 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:55.610 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:55.610 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:55.611 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:55.612 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:55.613 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:55.613 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:55.613 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:51:55.613 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:51:55.613 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:55.613 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:55.613 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:55.613 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:55.613 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:55.613 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:51:55.613 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:51:55.613 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:51:55.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.602+0000 7f6a7204d640 1 -- 10.20.193.28:0/1177011141 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6a480b56e0 msgr2=0x7f6a480d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:55.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.602+0000 7f6a7204d640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6a480b56e0 0x7f6a480d5b90 secure :-1 s=READY pgs=303 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6a5c004740 tx=0x7f6a5c005f90 comp rx=0 tx=0).stop 2026-02-25T11:51:55.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.602+0000 7f6a7204d640 1 -- 10.20.193.28:0/1177011141 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6a6c119d30 msgr2=0x7f6a6c127e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:55.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.602+0000 7f6a7204d640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6a6c119d30 0x7f6a6c127e60 secure :-1 s=READY pgs=134 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6a6800f8d0 tx=0x7f6a6800fda0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.603+0000 7f6a7204d640 1 -- 10.20.193.28:0/1177011141 shutdown_connections 2026-02-25T11:51:55.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.603+0000 7f6a7204d640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6a480b56e0 0x7f6a480d5b90 unknown :-1 s=CLOSED pgs=303 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.603+0000 7f6a7204d640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6a6c119d30 0x7f6a6c127e60 unknown :-1 s=CLOSED pgs=134 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.603+0000 7f6a7204d640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6a6c114a90 0x7f6a6c122a90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.603+0000 7f6a7204d640 1 --2- 10.20.193.28:0/1177011141 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6a6c1140a0 0x7f6a6c122550 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:55.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.603+0000 7f6a7204d640 1 -- 10.20.193.28:0/1177011141 >> 10.20.193.28:0/1177011141 conn(0x7f6a6c064c40 msgr2=0x7f6a6c117b80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:55.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.603+0000 7f6a7204d640 1 -- 10.20.193.28:0/1177011141 shutdown_connections 2026-02-25T11:51:55.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:55.603+0000 7f6a7204d640 1 -- 10.20.193.28:0/1177011141 wait complete. 2026-02-25T11:51:55.702 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:51:55.703 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:51:55.887 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:56.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.078+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3257329032 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4e90110080 msgr2=0x7f4e901150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.078+0000 7f4e8ffff640 1 --2- 10.20.193.28:0/3257329032 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4e90110080 0x7f4e901150d0 secure :-1 s=READY pgs=555 gs=1 cs=0 l=1 c_cookie=9014876232273925451 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4e78009950 tx=0x7f4e7802f190 comp rx=0 tx=0).stop 2026-02-25T11:51:56.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.078+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3257329032 shutdown_connections 2026-02-25T11:51:56.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.078+0000 7f4e8ffff640 1 --2- 10.20.193.28:0/3257329032 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4e90115880 0x7f4e90176250 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.078+0000 7f4e8ffff640 1 --2- 10.20.193.28:0/3257329032 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4e90110080 0x7f4e901150d0 unknown :-1 s=CLOSED pgs=555 gs=1 cs=0 l=1 c_cookie=9014876232273925451 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.078+0000 7f4e8ffff640 1 --2- 10.20.193.28:0/3257329032 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4e9010f690 0x7f4e9010fa60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.078+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3257329032 >> 10.20.193.28:0/3257329032 conn(0x7f4e900ecf00 msgr2=0x7f4e9010d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:56.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.079+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3257329032 shutdown_connections 2026-02-25T11:51:56.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.079+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3257329032 wait complete. 2026-02-25T11:51:56.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.079+0000 7f4e8ffff640 1 Processor -- start 2026-02-25T11:51:56.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.079+0000 7f4e8ffff640 1 -- start start 2026-02-25T11:51:56.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4e9010f690 0x7f4e9011e0a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4e90110080 0x7f4e9011e5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8ffff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4e90115880 0x7f4e901239b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8ffff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90123fd0 con 0x7f4e9010f690 2026-02-25T11:51:56.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8ffff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90124140 con 0x7f4e90115880 2026-02-25T11:51:56.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8ffff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e90124300 con 0x7f4e90110080 2026-02-25T11:51:56.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4e9010f690 0x7f4e9011e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8f7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4e90115880 0x7f4e901239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8f7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4e90115880 0x7f4e901239b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43706/0 (socket says 10.20.193.28:43706) 2026-02-25T11:51:56.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8f7fe640 1 -- 10.20.193.28:0/3596504940 learned_addr learned my addr 10.20.193.28:0/3596504940 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:56.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8e7fc640 1 --2- 10.20.193.28:0/3596504940 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4e90110080 0x7f4e9011e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8effd640 1 -- 10.20.193.28:0/3596504940 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4e90110080 msgr2=0x7f4e9011e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8effd640 1 --2- 10.20.193.28:0/3596504940 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4e90110080 0x7f4e9011e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8effd640 1 -- 10.20.193.28:0/3596504940 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4e90115880 msgr2=0x7f4e901239b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8effd640 1 --2- 10.20.193.28:0/3596504940 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4e90115880 0x7f4e901239b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.080+0000 7f4e8effd640 1 -- 10.20.193.28:0/3596504940 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e78009570 con 0x7f4e9010f690 2026-02-25T11:51:56.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.081+0000 7f4e8effd640 1 --2- 10.20.193.28:0/3596504940 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4e9010f690 0x7f4e9011e0a0 secure :-1 s=READY pgs=556 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4e8400d720 tx=0x7f4e8400dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:56.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.081+0000 7f4e948f4640 1 -- 10.20.193.28:0/3596504940 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4e84004500 con 0x7f4e9010f690 2026-02-25T11:51:56.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.081+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3596504940 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e901245e0 con 0x7f4e9010f690 2026-02-25T11:51:56.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.081+0000 7f4e948f4640 1 -- 10.20.193.28:0/3596504940 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4e84004d10 con 0x7f4e9010f690 2026-02-25T11:51:56.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.081+0000 7f4e948f4640 1 -- 10.20.193.28:0/3596504940 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4e840052a0 con 0x7f4e9010f690 2026-02-25T11:51:56.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.081+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3596504940 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e90164ce0 con 0x7f4e9010f690 2026-02-25T11:51:56.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.081+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3596504940 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e90110970 con 0x7f4e9010f690 2026-02-25T11:51:56.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.084+0000 7f4e948f4640 1 -- 10.20.193.28:0/3596504940 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f4e84005430 con 0x7f4e9010f690 2026-02-25T11:51:56.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.085+0000 7f4e948f4640 1 --2- 10.20.193.28:0/3596504940 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4e640b57e0 0x7f4e640d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.085+0000 7f4e948f4640 1 -- 10.20.193.28:0/3596504940 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4e840e6830 con 0x7f4e9010f690 2026-02-25T11:51:56.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.085+0000 7f4e948f4640 1 -- 10.20.193.28:0/3596504940 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f4e840e6c30 con 0x7f4e9010f690 2026-02-25T11:51:56.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.086+0000 7f4e8e7fc640 1 --2- 10.20.193.28:0/3596504940 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4e640b57e0 0x7f4e640d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.087+0000 7f4e8e7fc640 1 --2- 10.20.193.28:0/3596504940 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4e640b57e0 0x7f4e640d5c90 secure :-1 s=READY pgs=304 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4e90124a40 tx=0x7f4e7803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:56.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.227+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3596504940 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e9010fa60 con 0x7f4e9010f690 2026-02-25T11:51:56.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.227+0000 7f4e948f4640 1 -- 10.20.193.28:0/3596504940 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f4e840a92b0 con 0x7f4e9010f690 2026-02-25T11:51:56.229 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:51:56.229 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:51:56.230 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:51:56.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.228+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3596504940 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4e640b57e0 msgr2=0x7f4e640d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.228+0000 7f4e8ffff640 1 --2- 10.20.193.28:0/3596504940 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4e640b57e0 0x7f4e640d5c90 secure :-1 s=READY pgs=304 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4e90124a40 tx=0x7f4e7803a040 comp rx=0 tx=0).stop 2026-02-25T11:51:56.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.228+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3596504940 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4e9010f690 msgr2=0x7f4e9011e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.228+0000 7f4e8ffff640 1 --2- 10.20.193.28:0/3596504940 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4e9010f690 0x7f4e9011e0a0 secure :-1 s=READY pgs=556 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4e8400d720 tx=0x7f4e8400dbf0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.229+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3596504940 shutdown_connections 2026-02-25T11:51:56.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.229+0000 7f4e8ffff640 1 --2- 10.20.193.28:0/3596504940 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4e640b57e0 0x7f4e640d5c90 unknown :-1 s=CLOSED pgs=304 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.229+0000 7f4e8ffff640 1 --2- 10.20.193.28:0/3596504940 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4e90115880 0x7f4e901239b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.229+0000 7f4e8ffff640 1 --2- 10.20.193.28:0/3596504940 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4e90110080 0x7f4e9011e5e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.229+0000 7f4e8ffff640 1 --2- 10.20.193.28:0/3596504940 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4e9010f690 0x7f4e9011e0a0 unknown :-1 s=CLOSED pgs=556 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.229+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3596504940 >> 10.20.193.28:0/3596504940 conn(0x7f4e900ecf00 msgr2=0x7f4e9010d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:56.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.229+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3596504940 shutdown_connections 2026-02-25T11:51:56.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.229+0000 7f4e8ffff640 1 -- 10.20.193.28:0/3596504940 wait complete. 2026-02-25T11:51:56.271 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:56 trial028 ceph-mon[39224]: pgmap v337: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:56.271 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:56 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3596504940' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:51:56.363 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:51:56.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:56 trial167 ceph-mon[43392]: pgmap v337: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:56.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:56 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3596504940' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:51:56.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:56 trial119 ceph-mon[42710]: pgmap v337: 97 pgs: 97 active+clean; 3.1 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:51:56.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:56 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3596504940' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:51:56.546 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:56.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.717+0000 7f6ae8ba4640 1 -- 10.20.193.28:0/976012199 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ae410b360 msgr2=0x7f6ae410b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.717+0000 7f6ae8ba4640 1 --2- 10.20.193.28:0/976012199 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ae410b360 0x7f6ae410b730 secure :-1 s=READY pgs=557 gs=1 cs=0 l=1 c_cookie=12522649508386201123 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ad0009950 tx=0x7f6ad002f1b0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.717+0000 7f6ae8ba4640 1 -- 10.20.193.28:0/976012199 shutdown_connections 2026-02-25T11:51:56.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.717+0000 7f6ae8ba4640 1 --2- 10.20.193.28:0/976012199 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6ae4111370 0x7f6ae4111d50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.717+0000 7f6ae8ba4640 1 --2- 10.20.193.28:0/976012199 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6ae410bd50 0x7f6ae4110be0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.717+0000 7f6ae8ba4640 1 --2- 10.20.193.28:0/976012199 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ae410b360 0x7f6ae410b730 unknown :-1 s=CLOSED pgs=557 gs=1 cs=0 l=1 c_cookie=12522649508386201123 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.717+0000 7f6ae8ba4640 1 -- 10.20.193.28:0/976012199 >> 10.20.193.28:0/976012199 conn(0x7f6ae40e8e70 msgr2=0x7f6ae4109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:56.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.717+0000 7f6ae8ba4640 1 -- 10.20.193.28:0/976012199 shutdown_connections 2026-02-25T11:51:56.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.717+0000 7f6ae8ba4640 1 -- 10.20.193.28:0/976012199 wait complete. 2026-02-25T11:51:56.718 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.718+0000 7f6ae8ba4640 1 Processor -- start 2026-02-25T11:51:56.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.718+0000 7f6ae8ba4640 1 -- start start 2026-02-25T11:51:56.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.718+0000 7f6ae8ba4640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6ae410b360 0x7f6ae4123680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.718+0000 7f6ae8ba4640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ae410bd50 0x7f6ae4123bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.718+0000 7f6ae8ba4640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6ae4111370 0x7f6ae4128f90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.718+0000 7f6ae8ba4640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41295b0 con 0x7f6ae410bd50 2026-02-25T11:51:56.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.718+0000 7f6ae8ba4640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae4129720 con 0x7f6ae4111370 2026-02-25T11:51:56.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.718+0000 7f6ae8ba4640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae41298e0 con 0x7f6ae410b360 2026-02-25T11:51:56.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.718+0000 7f6ae37fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6ae410b360 0x7f6ae4123680 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.718+0000 7f6ae2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ae410bd50 0x7f6ae4123bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.718+0000 7f6ae37fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6ae410b360 0x7f6ae4123680 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:59996/0 (socket says 10.20.193.28:59996) 2026-02-25T11:51:56.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.718+0000 7f6ae37fe640 1 -- 10.20.193.28:0/1675078785 learned_addr learned my addr 10.20.193.28:0/1675078785 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:56.719 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.719+0000 7f6ae3fff640 1 --2- 10.20.193.28:0/1675078785 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6ae4111370 0x7f6ae4128f90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.719+0000 7f6ae2ffd640 1 -- 10.20.193.28:0/1675078785 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6ae410b360 msgr2=0x7f6ae4123680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.719+0000 7f6ae2ffd640 1 --2- 10.20.193.28:0/1675078785 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6ae410b360 0x7f6ae4123680 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.719+0000 7f6ae2ffd640 1 -- 10.20.193.28:0/1675078785 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6ae4111370 msgr2=0x7f6ae4128f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.719+0000 7f6ae2ffd640 1 --2- 10.20.193.28:0/1675078785 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6ae4111370 0x7f6ae4128f90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.719+0000 7f6ae2ffd640 1 -- 10.20.193.28:0/1675078785 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad0009570 con 0x7f6ae410bd50 2026-02-25T11:51:56.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.719+0000 7f6ae2ffd640 1 --2- 10.20.193.28:0/1675078785 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ae410bd50 0x7f6ae4123bc0 secure :-1 s=READY pgs=558 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ad4004740 tx=0x7f6ad400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:56.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.719+0000 7f6ae0ff9640 1 -- 10.20.193.28:0/1675078785 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6ad4004500 con 0x7f6ae410bd50 2026-02-25T11:51:56.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.719+0000 7f6ae0ff9640 1 -- 10.20.193.28:0/1675078785 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6ad4004d10 con 0x7f6ae410bd50 2026-02-25T11:51:56.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.719+0000 7f6ae0ff9640 1 -- 10.20.193.28:0/1675078785 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6ad40052a0 con 0x7f6ae410bd50 2026-02-25T11:51:56.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.719+0000 7f6ae8ba4640 1 -- 10.20.193.28:0/1675078785 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae4129bc0 con 0x7f6ae410bd50 2026-02-25T11:51:56.720 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.719+0000 7f6ae8ba4640 1 -- 10.20.193.28:0/1675078785 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae416a2c0 con 0x7f6ae410bd50 2026-02-25T11:51:56.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.722+0000 7f6ae0ff9640 1 -- 10.20.193.28:0/1675078785 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f6ad4005430 con 0x7f6ae410bd50 2026-02-25T11:51:56.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.724+0000 7f6ae0ff9640 1 --2- 10.20.193.28:0/1675078785 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ab80b5760 0x7f6ab80d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.724+0000 7f6ae0ff9640 1 -- 10.20.193.28:0/1675078785 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6ad40e6480 con 0x7f6ae410bd50 2026-02-25T11:51:56.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.724+0000 7f6ae37fe640 1 --2- 10.20.193.28:0/1675078785 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ab80b5760 0x7f6ab80d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.725 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.724+0000 7f6ae37fe640 1 --2- 10.20.193.28:0/1675078785 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ab80b5760 0x7f6ab80d5c10 secure :-1 s=READY pgs=305 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ad0005b30 tx=0x7f6ad003a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:56.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.725+0000 7f6aebe28e00 1 Processor -- start 2026-02-25T11:51:56.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.725+0000 7f6aebe28e00 1 -- start start 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6aebe28e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eab30 0x564e7c6eaf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6aebe28e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c6eb440 0x564e7c72fba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6aebe28e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c7300e0 0x564e7c7504c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6aebe28e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564e7c6008a0 con 0x564e7c6eb440 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6aebe28e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564e7c5fc5d0 con 0x564e7c7300e0 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6aebe28e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564e7c5f6570 con 0x564e7c6eab30 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6ae2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c6eb440 0x564e7c72fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6ae2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c6eb440 0x564e7c72fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59686/0 (socket says 10.20.193.28:59686) 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6ae2ffd640 1 -- 10.20.193.28:0/2405684006 learned_addr learned my addr 10.20.193.28:0/2405684006 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6ae37fe640 1 --2- 10.20.193.28:0/2405684006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eab30 0x564e7c6eaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6ae3fff640 1 --2- 10.20.193.28:0/2405684006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c7300e0 0x564e7c7504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6ae2ffd640 1 -- 10.20.193.28:0/2405684006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eab30 msgr2=0x564e7c6eaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6ae2ffd640 1 --2- 10.20.193.28:0/2405684006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eab30 0x564e7c6eaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6ae2ffd640 1 -- 10.20.193.28:0/2405684006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c7300e0 msgr2=0x564e7c7504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6ae2ffd640 1 --2- 10.20.193.28:0/2405684006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c7300e0 0x564e7c7504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6ae2ffd640 1 -- 10.20.193.28:0/2405684006 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e7c5a1d10 con 0x564e7c6eb440 2026-02-25T11:51:56.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6ae2ffd640 1 --2- 10.20.193.28:0/2405684006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c6eb440 0x564e7c72fba0 secure :-1 s=READY pgs=559 gs=1 cs=0 l=1 c_cookie=17286862253307415489 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ad40a8dd0 tx=0x7f6ad4002390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe745a1317f4bc1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:56.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6aa7fff640 1 -- 10.20.193.28:0/2405684006 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6ad4004e80 con 0x564e7c6eb440 2026-02-25T11:51:56.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6aa7fff640 1 -- 10.20.193.28:0/2405684006 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6ad40e3040 con 0x564e7c6eb440 2026-02-25T11:51:56.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.726+0000 7f6aa7fff640 1 -- 10.20.193.28:0/2405684006 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6ad40a8a70 con 0x564e7c6eb440 2026-02-25T11:51:56.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.727+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2405684006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c6eb440 msgr2=0x564e7c72fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.727+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2405684006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c6eb440 0x564e7c72fba0 secure :-1 s=READY pgs=559 gs=1 cs=0 l=1 c_cookie=17286862253307415489 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ad40a8dd0 tx=0x7f6ad4002390 comp rx=0 tx=0).stop 2026-02-25T11:51:56.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.727+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2405684006 shutdown_connections 2026-02-25T11:51:56.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.727+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2405684006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c7300e0 0x564e7c7504c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.727+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2405684006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c6eb440 0x564e7c72fba0 unknown :-1 s=CLOSED pgs=559 gs=1 cs=0 l=1 c_cookie=17286862253307415489 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.727+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2405684006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eab30 0x564e7c6eaf00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.727+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2405684006 >> 10.20.193.28:0/2405684006 conn(0x564e7c6097a0 msgr2=0x564e7c6e9410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:56.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.727+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2405684006 shutdown_connections 2026-02-25T11:51:56.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.727+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2405684006 wait complete. 2026-02-25T11:51:56.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.728+0000 7f6aebe28e00 1 -- start start 2026-02-25T11:51:56.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.728+0000 7f6aebe28e00 1 Processor -- start 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6aebe28e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eab30 0x564e7c702fb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6aebe28e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eb440 0x564e7c7034f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6aebe28e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c7084e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6aebe28e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564e7c708a20 con 0x564e7c7300e0 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6aebe28e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564e7c708b90 con 0x564e7c6eab30 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6aebe28e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564e7c708d50 con 0x564e7c6eb440 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6ae2ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eb440 0x564e7c7034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6ae2ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eb440 0x564e7c7034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:60012/0 (socket says 10.20.193.28:60012) 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6ae2ffd640 1 -- 10.20.193.28:0/2340603198 learned_addr learned my addr 10.20.193.28:0/2340603198 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6ae3fff640 1 --2- 10.20.193.28:0/2340603198 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c7084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6ae2ffd640 1 -- 10.20.193.28:0/2340603198 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eab30 msgr2=0x564e7c702fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6ae2ffd640 1 --2- 10.20.193.28:0/2340603198 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eab30 0x564e7c702fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6ae2ffd640 1 -- 10.20.193.28:0/2340603198 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 msgr2=0x564e7c7084e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.729+0000 7f6ae2ffd640 1 --2- 10.20.193.28:0/2340603198 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c7084e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6ae2ffd640 1 -- 10.20.193.28:0/2340603198 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e7c57a3c0 con 0x564e7c6eb440 2026-02-25T11:51:56.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6ae3fff640 1 --2- 10.20.193.28:0/2340603198 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c7084e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:56.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6ae2ffd640 1 --2- 10.20.193.28:0/2340603198 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eb440 0x564e7c7034f0 secure :-1 s=READY pgs=160 gs=1 cs=0 l=1 c_cookie=628004743839118612 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ad40a86e0 tx=0x7f6ad400b5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b71efcb9ca7514 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:56.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6aa57fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:51:56.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2340603198 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6ad40b2520 con 0x564e7c6eb440 2026-02-25T11:51:56.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2340603198 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6ad4009e30 con 0x564e7c6eb440 2026-02-25T11:51:56.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2340603198 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6ad40b74b0 con 0x564e7c6eb440 2026-02-25T11:51:56.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2340603198 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae416a2c0 con 0x564e7c6eb440 2026-02-25T11:51:56.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6aebe28e00 10 client.26020 _ll_register_callbacks cb 0x564e7c577a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:56.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2340603198 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564e7c28c6f0 con 0x564e7c6eb440 2026-02-25T11:51:56.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2340603198 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6ad40b7640 con 0x564e7c6eb440 2026-02-25T11:51:56.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2340603198 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6ad40ad490 con 0x564e7c6eb440 2026-02-25T11:51:56.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6aebe28e00 10 client.26020 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:56.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.730+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2340603198 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x564e7c58c0e0 con 0x564e7c6eb440 2026-02-25T11:51:56.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.731+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2340603198 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6ad40d0110 con 0x564e7c6eb440 2026-02-25T11:51:56.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.731+0000 7f6aebe28e00 10 client.26020 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:56.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.731+0000 7f6aebe28e00 10 client.26020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:51:56.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.731+0000 7f6aebe28e00 20 client.26020 populate_metadata read hostname 'trial028' 2026-02-25T11:51:56.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.731+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2340603198 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x564e7c72d450 0x564e7c72d8f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.731+0000 7f6aebe28e00 4 client.26020 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:51:56.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.731+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2340603198 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e7c5c8fc0 con 0x564e7c72d450 2026-02-25T11:51:56.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.731+0000 7f6ae37fe640 1 --2- 10.20.193.28:0/2340603198 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x564e7c72d450 0x564e7c72d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.732+0000 7f6ae37fe640 1 --2- 10.20.193.28:0/2340603198 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x564e7c72d450 0x564e7c72d8f0 crc :-1 s=READY pgs=37 gs=4 cs=0 l=0 c_cookie=6370467579895186470 s_cookie=4113850307292037426 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5868728300cb8426 server_cookie=3917550ef1460132 in_seq=0 out_seq=0 2026-02-25T11:51:56.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.732+0000 7f6aa57fa640 10 client.26020 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:51:56.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.732+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2340603198 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x564e7c5c8fc0 con 0x564e7c72d450 2026-02-25T11:51:56.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.732+0000 7f6aa57fa640 10 client.26020 handle_command_reply: tid=0 2026-02-25T11:51:56.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.732+0000 7f6aebe28e00 1 client.26020 shutdown 2026-02-25T11:51:56.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.733+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2340603198 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eb440 msgr2=0x564e7c7034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.733+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2340603198 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eb440 0x564e7c7034f0 secure :-1 s=READY pgs=160 gs=1 cs=0 l=1 c_cookie=628004743839118612 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ad40a86e0 tx=0x7f6ad400b5d0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.733+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2340603198 shutdown_connections 2026-02-25T11:51:56.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.733+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2340603198 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x564e7c72d450 0x564e7c72d8f0 crc :-1 s=READY pgs=37 gs=4 cs=0 l=0 c_cookie=6370467579895186470 s_cookie=4113850307292037426 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.733+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2340603198 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c7084e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.733+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2340603198 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eb440 0x564e7c7034f0 unknown :-1 s=CLOSED pgs=160 gs=1 cs=0 l=1 c_cookie=628004743839118612 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.733+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2340603198 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eab30 0x564e7c702fb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.733+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2340603198 >> 10.20.193.28:0/2340603198 conn(0x564e7c6097a0 msgr2=0x564e7c6e9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:56.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.733+0000 7f6aa57fa640 0 client.26020 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:51:56.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.733+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2340603198 shutdown_connections 2026-02-25T11:51:56.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.733+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2340603198 wait complete. 2026-02-25T11:51:56.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.733+0000 7f6aebe28e00 20 client.26020 trim_cache size 0 max 16384 2026-02-25T11:51:56.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.740+0000 7f6aebe28e00 1 Processor -- start 2026-02-25T11:51:56.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.740+0000 7f6aebe28e00 1 -- start start 2026-02-25T11:51:56.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.740+0000 7f6aebe28e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eab30 0x564e7c7033d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.740+0000 7f6aebe28e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eb440 0x564e7c6e9d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.740+0000 7f6aebe28e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c6ea260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.740+0000 7f6aebe28e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564e7c5f41a0 con 0x564e7c7300e0 2026-02-25T11:51:56.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.740+0000 7f6aebe28e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564e7c2b4700 con 0x564e7c6eb440 2026-02-25T11:51:56.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.740+0000 7f6aebe28e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564e7c729cd0 con 0x564e7c6eab30 2026-02-25T11:51:56.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6ae3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c6ea260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6ae3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c6ea260 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59706/0 (socket says 10.20.193.28:59706) 2026-02-25T11:51:56.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6ae3fff640 1 -- 10.20.193.28:0/2719311080 learned_addr learned my addr 10.20.193.28:0/2719311080 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:56.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6ae37fe640 1 --2- 10.20.193.28:0/2719311080 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eab30 0x564e7c7033d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6ae2ffd640 1 --2- 10.20.193.28:0/2719311080 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eb440 0x564e7c6e9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6ae3fff640 1 -- 10.20.193.28:0/2719311080 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eab30 msgr2=0x564e7c7033d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:56.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6ae3fff640 1 --2- 10.20.193.28:0/2719311080 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eab30 0x564e7c7033d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6ae3fff640 1 -- 10.20.193.28:0/2719311080 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eb440 msgr2=0x564e7c6e9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6ae3fff640 1 --2- 10.20.193.28:0/2719311080 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eb440 0x564e7c6e9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6ae3fff640 1 -- 10.20.193.28:0/2719311080 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e7c586b60 con 0x564e7c7300e0 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6ae3fff640 1 --2- 10.20.193.28:0/2719311080 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c6ea260 secure :-1 s=READY pgs=560 gs=2 cs=0 l=1 c_cookie=9844508932509884820 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ad8003ab0 tx=0x7f6ad8003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889eb8b736696194 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6aa77fe640 1 -- 10.20.193.28:0/2719311080 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6ad8014b70 con 0x564e7c7300e0 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6aa77fe640 1 -- 10.20.193.28:0/2719311080 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6ad8014cd0 con 0x564e7c7300e0 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.741+0000 7f6aa77fe640 1 -- 10.20.193.28:0/2719311080 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6ad80126d0 con 0x564e7c7300e0 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.742+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2719311080 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 msgr2=0x564e7c6ea260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.742+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2719311080 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c6ea260 secure :-1 s=READY pgs=560 gs=2 cs=0 l=1 c_cookie=9844508932509884820 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ad8003ab0 tx=0x7f6ad8003f80 comp rx=0 tx=0).stop 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.742+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2719311080 shutdown_connections 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.742+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2719311080 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c6ea260 unknown :-1 s=CLOSED pgs=560 gs=2 cs=0 l=1 c_cookie=9844508932509884820 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.742+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2719311080 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eb440 0x564e7c6e9d20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.742+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2719311080 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eab30 0x564e7c7033d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.742+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2719311080 >> 10.20.193.28:0/2719311080 conn(0x564e7c6097a0 msgr2=0x564e7c72d450 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:56.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.742+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2719311080 shutdown_connections 2026-02-25T11:51:56.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.742+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2719311080 wait complete. 2026-02-25T11:51:56.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.743+0000 7f6aebe28e00 1 -- start start 2026-02-25T11:51:56.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.743+0000 7f6aebe28e00 1 Processor -- start 2026-02-25T11:51:56.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.743+0000 7f6aebe28e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eab30 0x564e7c706d10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.743+0000 7f6aebe28e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eb440 0x564e7c707250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.743+0000 7f6aebe28e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c70d5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.743+0000 7f6aebe28e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564e7c7079d0 con 0x564e7c7300e0 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.743+0000 7f6aebe28e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564e7c70db20 con 0x564e7c6eab30 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.743+0000 7f6aebe28e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564e7c70dcb0 con 0x564e7c6eb440 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.743+0000 7f6ae3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c70d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.743+0000 7f6ae3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c70d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59714/0 (socket says 10.20.193.28:59714) 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.744+0000 7f6ae3fff640 1 -- 10.20.193.28:0/2616261811 learned_addr learned my addr 10.20.193.28:0/2616261811 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.743+0000 7f6ae2ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eb440 0x564e7c707250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.744+0000 7f6ae37fe640 1 --2- 10.20.193.28:0/2616261811 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eab30 0x564e7c706d10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.744+0000 7f6ae3fff640 1 -- 10.20.193.28:0/2616261811 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eb440 msgr2=0x564e7c707250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.744+0000 7f6ae3fff640 1 --2- 10.20.193.28:0/2616261811 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eb440 0x564e7c707250 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.744+0000 7f6ae3fff640 1 -- 10.20.193.28:0/2616261811 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eab30 msgr2=0x564e7c706d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.744+0000 7f6ae3fff640 1 --2- 10.20.193.28:0/2616261811 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eab30 0x564e7c706d10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.744+0000 7f6ae3fff640 1 -- 10.20.193.28:0/2616261811 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e7c57a6f0 con 0x564e7c7300e0 2026-02-25T11:51:56.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.744+0000 7f6ae3fff640 1 --2- 10.20.193.28:0/2616261811 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c70d5e0 secure :-1 s=READY pgs=561 gs=1 cs=0 l=1 c_cookie=11982809016955101412 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ad80055b0 tx=0x7f6ad801dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64b7d5032f4e0e4 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:56.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.744+0000 7f6aa57fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:56.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.744+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2616261811 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6ad8022070 con 0x564e7c7300e0 2026-02-25T11:51:56.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.744+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2616261811 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6ad8012680 con 0x564e7c7300e0 2026-02-25T11:51:56.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.744+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2616261811 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6ad80057c0 con 0x564e7c7300e0 2026-02-25T11:51:56.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.744+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2616261811 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad80055e0 con 0x564e7c7300e0 2026-02-25T11:51:56.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.745+0000 7f6aebe28e00 10 client.16254 _ll_register_callbacks cb 0x564e7c577a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:56.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.745+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2616261811 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad40e3040 con 0x564e7c7300e0 2026-02-25T11:51:56.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.745+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2616261811 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6ad8014e40 con 0x564e7c7300e0 2026-02-25T11:51:56.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.745+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2616261811 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6ad801d440 con 0x564e7c7300e0 2026-02-25T11:51:56.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.745+0000 7f6aebe28e00 10 client.16254 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:56.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.745+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2616261811 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x564e7c58c340 con 0x564e7c7300e0 2026-02-25T11:51:56.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.745+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2616261811 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6ad802e060 con 0x564e7c7300e0 2026-02-25T11:51:56.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.745+0000 7f6aebe28e00 10 client.16254 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:56.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.745+0000 7f6aebe28e00 10 client.16254 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:51:56.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.745+0000 7f6aebe28e00 20 client.16254 populate_metadata read hostname 'trial028' 2026-02-25T11:51:56.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.745+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2616261811 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x564e7c7dcc40 0x564e7c7fd0f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:56.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.745+0000 7f6aebe28e00 4 client.16254 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:51:56.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.745+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2616261811 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e7c289f40 con 0x564e7c7dcc40 2026-02-25T11:51:56.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.746+0000 7f6ae37fe640 1 --2- 10.20.193.28:0/2616261811 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x564e7c7dcc40 0x564e7c7fd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:56.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.746+0000 7f6ae37fe640 1 --2- 10.20.193.28:0/2616261811 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x564e7c7dcc40 0x564e7c7fd0f0 crc :-1 s=READY pgs=38 gs=4 cs=0 l=0 c_cookie=18349260977162969279 s_cookie=14664079692854391803 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fea5aba3e2bc60bf server_cookie=cb8146c97a9af3fb in_seq=0 out_seq=0 2026-02-25T11:51:56.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.746+0000 7f6aa57fa640 10 client.16254 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.746+0000 7f6aa57fa640 1 -- 10.20.193.28:0/2616261811 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e7c289f40 con 0x564e7c7dcc40 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.746+0000 7f6aa57fa640 10 client.16254 handle_command_reply: tid=0 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.746+0000 7f6aebe28e00 1 client.16254 shutdown 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.746+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2616261811 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 msgr2=0x564e7c70d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.746+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2616261811 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c70d5e0 secure :-1 s=READY pgs=561 gs=1 cs=0 l=1 c_cookie=11982809016955101412 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ad80055b0 tx=0x7f6ad801dd50 comp rx=0 tx=0).stop 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2616261811 shutdown_connections 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2616261811 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x564e7c7dcc40 0x564e7c7fd0f0 crc :-1 s=READY pgs=38 gs=4 cs=0 l=0 c_cookie=18349260977162969279 s_cookie=14664079692854391803 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2616261811 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564e7c7300e0 0x564e7c70d5e0 unknown :-1 s=CLOSED pgs=561 gs=1 cs=0 l=1 c_cookie=11982809016955101412 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2616261811 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564e7c6eb440 0x564e7c707250 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6aebe28e00 1 --2- 10.20.193.28:0/2616261811 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564e7c6eab30 0x564e7c706d10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2616261811 >> 10.20.193.28:0/2616261811 conn(0x564e7c6097a0 msgr2=0x564e7c6ed9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6aa57fa640 0 client.16254 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2616261811 shutdown_connections 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6aebe28e00 1 -- 10.20.193.28:0/2616261811 wait complete. 2026-02-25T11:51:56.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6aebe28e00 20 client.16254 trim_cache size 0 max 16384 2026-02-25T11:51:56.749 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:51:56.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6ae8ba4640 1 -- 10.20.193.28:0/1675078785 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ab80b5760 msgr2=0x7f6ab80d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6ae8ba4640 1 --2- 10.20.193.28:0/1675078785 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ab80b5760 0x7f6ab80d5c10 secure :-1 s=READY pgs=305 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ad0005b30 tx=0x7f6ad003a040 comp rx=0 tx=0).stop 2026-02-25T11:51:56.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6ae8ba4640 1 -- 10.20.193.28:0/1675078785 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ae410bd50 msgr2=0x7f6ae4123bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:56.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6ae8ba4640 1 --2- 10.20.193.28:0/1675078785 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ae410bd50 0x7f6ae4123bc0 secure :-1 s=READY pgs=558 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6ad4004740 tx=0x7f6ad400dbc0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.747+0000 7f6ae8ba4640 1 -- 10.20.193.28:0/1675078785 shutdown_connections 2026-02-25T11:51:56.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.748+0000 7f6ae8ba4640 1 --2- 10.20.193.28:0/1675078785 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ab80b5760 0x7f6ab80d5c10 unknown :-1 s=CLOSED pgs=305 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.748+0000 7f6ae8ba4640 1 --2- 10.20.193.28:0/1675078785 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6ae4111370 0x7f6ae4128f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.748+0000 7f6ae8ba4640 1 --2- 10.20.193.28:0/1675078785 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6ae410bd50 0x7f6ae4123bc0 unknown :-1 s=CLOSED pgs=558 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.748+0000 7f6ae8ba4640 1 --2- 10.20.193.28:0/1675078785 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6ae410b360 0x7f6ae4123680 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:56.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.748+0000 7f6ae8ba4640 1 -- 10.20.193.28:0/1675078785 >> 10.20.193.28:0/1675078785 conn(0x7f6ae40e8e70 msgr2=0x7f6ae4109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:56.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.748+0000 7f6ae8ba4640 1 -- 10.20.193.28:0/1675078785 shutdown_connections 2026-02-25T11:51:56.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:56.748+0000 7f6ae8ba4640 1 -- 10.20.193.28:0/1675078785 wait complete. 2026-02-25T11:51:56.849 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:51:57.031 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:57.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.209+0000 7f5adde2d640 1 -- 10.20.193.28:0/954779168 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad8110080 msgr2=0x7f5ad81150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.209+0000 7f5adde2d640 1 --2- 10.20.193.28:0/954779168 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad8110080 0x7f5ad81150d0 secure :-1 s=READY pgs=562 gs=2 cs=0 l=1 c_cookie=13299773756647647953 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac0009950 tx=0x7f5ac002f1d0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.209+0000 7f5adde2d640 1 -- 10.20.193.28:0/954779168 shutdown_connections 2026-02-25T11:51:57.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.209+0000 7f5adde2d640 1 --2- 10.20.193.28:0/954779168 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad8115880 0x7f5ad8176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.209+0000 7f5adde2d640 1 --2- 10.20.193.28:0/954779168 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad8110080 0x7f5ad81150d0 unknown :-1 s=CLOSED pgs=562 gs=2 cs=0 l=1 c_cookie=13299773756647647953 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.209+0000 7f5adde2d640 1 --2- 10.20.193.28:0/954779168 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ad810f690 0x7f5ad810fa60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.209+0000 7f5adde2d640 1 -- 10.20.193.28:0/954779168 >> 10.20.193.28:0/954779168 conn(0x7f5ad80ecf00 msgr2=0x7f5ad810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:57.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.209+0000 7f5adde2d640 1 -- 10.20.193.28:0/954779168 shutdown_connections 2026-02-25T11:51:57.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.209+0000 7f5adde2d640 1 -- 10.20.193.28:0/954779168 wait complete. 2026-02-25T11:51:57.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.210+0000 7f5adde2d640 1 Processor -- start 2026-02-25T11:51:57.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.210+0000 7f5adde2d640 1 -- start start 2026-02-25T11:51:57.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.210+0000 7f5adde2d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ad810f690 0x7f5ad80f5a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.210+0000 7f5adde2d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad8110080 0x7f5ad8176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.210+0000 7f5adde2d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad8115880 0x7f5ad80f6120 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.210+0000 7f5adde2d640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad80fb330 con 0x7f5ad8110080 2026-02-25T11:51:57.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.210+0000 7f5adde2d640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad80fb4a0 con 0x7f5ad8115880 2026-02-25T11:51:57.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.210+0000 7f5adde2d640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad80fb660 con 0x7f5ad810f690 2026-02-25T11:51:57.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.210+0000 7f5acffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad8110080 0x7f5ad8176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.210+0000 7f5acffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad8110080 0x7f5ad8176150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59722/0 (socket says 10.20.193.28:59722) 2026-02-25T11:51:57.214 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.210+0000 7f5acffff640 1 -- 10.20.193.28:0/1747213721 learned_addr learned my addr 10.20.193.28:0/1747213721 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:57.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.210+0000 7f5adce2b640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ad810f690 0x7f5ad80f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5acffff640 1 -- 10.20.193.28:0/1747213721 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ad810f690 msgr2=0x7f5ad80f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5acffff640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ad810f690 0x7f5ad80f5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5acffff640 1 -- 10.20.193.28:0/1747213721 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad8115880 msgr2=0x7f5ad80f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5add62c640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad8115880 0x7f5ad80f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5acffff640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad8115880 0x7f5ad80f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5acffff640 1 -- 10.20.193.28:0/1747213721 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac0009570 con 0x7f5ad8110080 2026-02-25T11:51:57.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5adce2b640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ad810f690 0x7f5ad80f5a20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:57.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5add62c640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad8115880 0x7f5ad80f6120 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:51:57.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5acffff640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad8110080 0x7f5ad8176150 secure :-1 s=READY pgs=563 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac00098a0 tx=0x7f5ac00028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:57.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5acdffb640 1 -- 10.20.193.28:0/1747213721 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5ac003d070 con 0x7f5ad8110080 2026-02-25T11:51:57.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5adde2d640 1 -- 10.20.193.28:0/1747213721 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad80fb8e0 con 0x7f5ad8110080 2026-02-25T11:51:57.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5acdffb640 1 -- 10.20.193.28:0/1747213721 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5ac0002a40 con 0x7f5ad8110080 2026-02-25T11:51:57.216 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5acdffb640 1 -- 10.20.193.28:0/1747213721 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5ac0041720 con 0x7f5ad8110080 2026-02-25T11:51:57.216 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.211+0000 7f5adde2d640 1 -- 10.20.193.28:0/1747213721 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad80fbda0 con 0x7f5ad8110080 2026-02-25T11:51:57.216 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.214+0000 7f5acdffb640 1 -- 10.20.193.28:0/1747213721 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f5ac00386a0 con 0x7f5ad8110080 2026-02-25T11:51:57.217 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.216+0000 7f5acdffb640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ab40b5750 0x7f5ab40d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.217 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.216+0000 7f5acdffb640 1 -- 10.20.193.28:0/1747213721 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f5ac010b190 con 0x7f5ad8110080 2026-02-25T11:51:57.217 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.216+0000 7f5adce2b640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ab40b5750 0x7f5ab40d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.218 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.217+0000 7f5adce2b640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ab40b5750 0x7f5ab40d5c00 secure :-1 s=READY pgs=306 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac8004740 tx=0x7f5ac8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:57.218 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.217+0000 7f5ae10b1e00 1 Processor -- start 2026-02-25T11:51:57.219 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.217+0000 7f5ae10b1e00 1 -- start start 2026-02-25T11:51:57.219 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.217+0000 7f5ae10b1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcd6bb30 0x5601dcd6bf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.220 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.217+0000 7f5ae10b1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6c440 0x5601dcdb0ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.220 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.217+0000 7f5ae10b1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcdb10e0 0x5601dcdd14c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.220 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.217+0000 7f5ae10b1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5601dcc818a0 con 0x5601dcdb10e0 2026-02-25T11:51:57.220 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.217+0000 7f5ae10b1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5601dcc7d5d0 con 0x5601dcd6c440 2026-02-25T11:51:57.220 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5ae10b1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5601dcc77570 con 0x5601dcd6bb30 2026-02-25T11:51:57.220 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5add62c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcdb10e0 0x5601dcdd14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.220 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5add62c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcdb10e0 0x5601dcdd14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59732/0 (socket says 10.20.193.28:59732) 2026-02-25T11:51:57.220 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5add62c640 1 -- 10.20.193.28:0/1791559997 learned_addr learned my addr 10.20.193.28:0/1791559997 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:57.220 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5adce2b640 1 --2- 10.20.193.28:0/1791559997 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcd6bb30 0x5601dcd6bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.220 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5add62c640 1 -- 10.20.193.28:0/1791559997 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcd6bb30 msgr2=0x5601dcd6bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.220 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5add62c640 1 --2- 10.20.193.28:0/1791559997 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcd6bb30 0x5601dcd6bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5add62c640 1 -- 10.20.193.28:0/1791559997 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6c440 msgr2=0x5601dcdb0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5acffff640 1 --2- 10.20.193.28:0/1791559997 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6c440 0x5601dcdb0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5add62c640 1 --2- 10.20.193.28:0/1791559997 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6c440 0x5601dcdb0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5add62c640 1 -- 10.20.193.28:0/1791559997 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601dcc22d10 con 0x5601dcdb10e0 2026-02-25T11:51:57.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5acffff640 1 --2- 10.20.193.28:0/1791559997 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6c440 0x5601dcdb0ba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:51:57.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5add62c640 1 --2- 10.20.193.28:0/1791559997 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcdb10e0 0x5601dcdd14c0 secure :-1 s=READY pgs=564 gs=2 cs=0 l=1 c_cookie=14270523371734875422 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ad400a2e0 tx=0x7f5ad4011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60b1565eb2abd1e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:57.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5aa54e4640 1 -- 10.20.193.28:0/1791559997 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5ad40050d0 con 0x5601dcdb10e0 2026-02-25T11:51:57.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5aa54e4640 1 -- 10.20.193.28:0/1791559997 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5ad4003780 con 0x5601dcdb10e0 2026-02-25T11:51:57.221 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.218+0000 7f5aa54e4640 1 -- 10.20.193.28:0/1791559997 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5ad4013820 con 0x5601dcdb10e0 2026-02-25T11:51:57.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.219+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1791559997 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcdb10e0 msgr2=0x5601dcdd14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.219+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1791559997 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcdb10e0 0x5601dcdd14c0 secure :-1 s=READY pgs=564 gs=2 cs=0 l=1 c_cookie=14270523371734875422 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ad400a2e0 tx=0x7f5ad4011740 comp rx=0 tx=0).stop 2026-02-25T11:51:57.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.219+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1791559997 shutdown_connections 2026-02-25T11:51:57.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.219+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1791559997 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcdb10e0 0x5601dcdd14c0 unknown :-1 s=CLOSED pgs=564 gs=2 cs=0 l=1 c_cookie=14270523371734875422 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.219+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1791559997 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6c440 0x5601dcdb0ba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.219+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1791559997 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcd6bb30 0x5601dcd6bf00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.219+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1791559997 >> 10.20.193.28:0/1791559997 conn(0x5601dcc8a7a0 msgr2=0x5601dcd6a410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:57.224 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.219+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1791559997 shutdown_connections 2026-02-25T11:51:57.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.219+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1791559997 wait complete. 2026-02-25T11:51:57.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.220+0000 7f5ae10b1e00 1 -- start start 2026-02-25T11:51:57.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5ae10b1e00 1 Processor -- start 2026-02-25T11:51:57.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5ae10b1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6bb30 0x5601dcd83ea0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5ae10b1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 0x5601dcd843e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5ae10b1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcdb10e0 0x5601dcd89440 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5ae10b1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5601dcd89980 con 0x5601dcd6c440 2026-02-25T11:51:57.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5ae10b1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5601dcd89af0 con 0x5601dcd6bb30 2026-02-25T11:51:57.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5ae10b1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5601dcd89cb0 con 0x5601dcdb10e0 2026-02-25T11:51:57.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5acffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 0x5601dcd843e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5acffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 0x5601dcd843e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59736/0 (socket says 10.20.193.28:59736) 2026-02-25T11:51:57.225 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5acffff640 1 -- 10.20.193.28:0/3120850417 learned_addr learned my addr 10.20.193.28:0/3120850417 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:57.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5adce2b640 1 --2- 10.20.193.28:0/3120850417 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6bb30 0x5601dcd83ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5add62c640 1 --2- 10.20.193.28:0/3120850417 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcdb10e0 0x5601dcd89440 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5acffff640 1 -- 10.20.193.28:0/3120850417 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcdb10e0 msgr2=0x5601dcd89440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5acffff640 1 --2- 10.20.193.28:0/3120850417 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcdb10e0 0x5601dcd89440 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5acffff640 1 -- 10.20.193.28:0/3120850417 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6bb30 msgr2=0x5601dcd83ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5acffff640 1 --2- 10.20.193.28:0/3120850417 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6bb30 0x5601dcd83ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.221+0000 7f5acffff640 1 -- 10.20.193.28:0/3120850417 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601dcbfb3c0 con 0x5601dcd6c440 2026-02-25T11:51:57.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.222+0000 7f5acffff640 1 --2- 10.20.193.28:0/3120850417 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 0x5601dcd843e0 secure :-1 s=READY pgs=565 gs=1 cs=0 l=1 c_cookie=17378951644766166742 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac0004030 tx=0x7f5ac00cd3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12e70717afa5ed6 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:57.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.222+0000 7f5a967fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:57.226 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.222+0000 7f5a967fc640 1 -- 10.20.193.28:0/3120850417 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5ac00d7070 con 0x5601dcd6c440 2026-02-25T11:51:57.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.222+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/3120850417 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad400a900 con 0x5601dcd6c440 2026-02-25T11:51:57.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.222+0000 7f5ae10b1e00 10 client.16266 _ll_register_callbacks cb 0x5601dcbf8a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:57.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.222+0000 7f5a967fc640 1 -- 10.20.193.28:0/3120850417 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5ac00d0d10 con 0x5601dcd6c440 2026-02-25T11:51:57.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.222+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/3120850417 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5601dc90d6f0 con 0x5601dcd6c440 2026-02-25T11:51:57.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.222+0000 7f5a967fc640 1 -- 10.20.193.28:0/3120850417 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5ac0096610 con 0x5601dcd6c440 2026-02-25T11:51:57.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.222+0000 7f5a967fc640 1 -- 10.20.193.28:0/3120850417 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f5ac00d1f10 con 0x5601dcd6c440 2026-02-25T11:51:57.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.222+0000 7f5a967fc640 1 -- 10.20.193.28:0/3120850417 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5ac0048e80 con 0x5601dcd6c440 2026-02-25T11:51:57.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.222+0000 7f5ae10b1e00 10 client.16266 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:57.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.222+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/3120850417 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5601dcc0d0e0 con 0x5601dcd6c440 2026-02-25T11:51:57.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.223+0000 7f5a967fc640 1 -- 10.20.193.28:0/3120850417 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f5ac00cf4f0 con 0x5601dcd6c440 2026-02-25T11:51:57.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.223+0000 7f5ae10b1e00 10 client.16266 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:57.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.223+0000 7f5ae10b1e00 10 client.16266 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:51:57.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.223+0000 7f5ae10b1e00 20 client.16266 populate_metadata read hostname 'trial028' 2026-02-25T11:51:57.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.223+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/3120850417 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5601dcdae4b0 0x5601dcdae920 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.223+0000 7f5ae10b1e00 4 client.16266 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:51:57.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.223+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/3120850417 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601dcc49fc0 con 0x5601dcdae4b0 2026-02-25T11:51:57.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.223+0000 7f5adce2b640 1 --2- 10.20.193.28:0/3120850417 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5601dcdae4b0 0x5601dcdae920 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.223+0000 7f5adce2b640 1 --2- 10.20.193.28:0/3120850417 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5601dcdae4b0 0x5601dcdae920 crc :-1 s=READY pgs=38 gs=4 cs=0 l=0 c_cookie=12866088125360667530 s_cookie=16193331885402962957 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b28d86fb88fe7b8a server_cookie=e0ba458dc7733c0d in_seq=0 out_seq=0 2026-02-25T11:51:57.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.223+0000 7f5a967fc640 10 client.16266 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:51:57.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5a967fc640 1 -- 10.20.193.28:0/3120850417 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5601dcc49fc0 con 0x5601dcdae4b0 2026-02-25T11:51:57.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5a967fc640 10 client.16266 handle_command_reply: tid=0 2026-02-25T11:51:57.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5ae10b1e00 1 client.16266 shutdown 2026-02-25T11:51:57.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/3120850417 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 msgr2=0x5601dcd843e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/3120850417 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 0x5601dcd843e0 secure :-1 s=READY pgs=565 gs=1 cs=0 l=1 c_cookie=17378951644766166742 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac0004030 tx=0x7f5ac00cd3b0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/3120850417 shutdown_connections 2026-02-25T11:51:57.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/3120850417 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5601dcdae4b0 0x5601dcdae920 crc :-1 s=READY pgs=38 gs=4 cs=0 l=0 c_cookie=12866088125360667530 s_cookie=16193331885402962957 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/3120850417 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcdb10e0 0x5601dcd89440 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/3120850417 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 0x5601dcd843e0 unknown :-1 s=CLOSED pgs=565 gs=1 cs=0 l=1 c_cookie=17378951644766166742 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/3120850417 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6bb30 0x5601dcd83ea0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/3120850417 >> 10.20.193.28:0/3120850417 conn(0x5601dcc8a7a0 msgr2=0x5601dcd6ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:57.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5a967fc640 0 client.16266 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:51:57.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/3120850417 shutdown_connections 2026-02-25T11:51:57.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.224+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/3120850417 wait complete. 2026-02-25T11:51:57.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.225+0000 7f5ae10b1e00 20 client.16266 trim_cache size 0 max 16384 2026-02-25T11:51:57.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.232+0000 7f5ae10b1e00 1 Processor -- start 2026-02-25T11:51:57.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.232+0000 7f5ae10b1e00 1 -- start start 2026-02-25T11:51:57.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.232+0000 7f5ae10b1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcd6bb30 0x5601dcd6ad20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.232+0000 7f5ae10b1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 0x5601dcd6b260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.232+0000 7f5ae10b1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcdb10e0 0x5601dcdaf200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.232+0000 7f5ae10b1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5601dcc751a0 con 0x5601dcd6c440 2026-02-25T11:51:57.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.232+0000 7f5ae10b1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5601dc935700 con 0x5601dcdb10e0 2026-02-25T11:51:57.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.232+0000 7f5ae10b1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5601dcd7d280 con 0x5601dcd6bb30 2026-02-25T11:51:57.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.232+0000 7f5acffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 0x5601dcd6b260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.232+0000 7f5acffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 0x5601dcd6b260 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59740/0 (socket says 10.20.193.28:59740) 2026-02-25T11:51:57.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.232+0000 7f5acffff640 1 -- 10.20.193.28:0/1701934345 learned_addr learned my addr 10.20.193.28:0/1701934345 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:57.235 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5add62c640 1 --2- 10.20.193.28:0/1701934345 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcdb10e0 0x5601dcdaf200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5acffff640 1 -- 10.20.193.28:0/1701934345 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcd6bb30 msgr2=0x5601dcd6ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5adce2b640 1 --2- 10.20.193.28:0/1701934345 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcd6bb30 0x5601dcd6ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5acffff640 1 --2- 10.20.193.28:0/1701934345 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcd6bb30 0x5601dcd6ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5acffff640 1 -- 10.20.193.28:0/1701934345 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcdb10e0 msgr2=0x5601dcdaf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5acffff640 1 --2- 10.20.193.28:0/1701934345 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcdb10e0 0x5601dcdaf200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5acffff640 1 -- 10.20.193.28:0/1701934345 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601dcc07b60 con 0x5601dcd6c440 2026-02-25T11:51:57.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5add62c640 1 --2- 10.20.193.28:0/1701934345 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcdb10e0 0x5601dcdaf200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:51:57.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5adce2b640 1 --2- 10.20.193.28:0/1701934345 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcd6bb30 0x5601dcd6ad20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:51:57.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5acffff640 1 --2- 10.20.193.28:0/1701934345 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 0x5601dcd6b260 secure :-1 s=READY pgs=566 gs=1 cs=0 l=1 c_cookie=6027582086022431073 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac00d0560 tx=0x7f5ac0096000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a645f8c3e00161 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:57.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5aa4ce3640 1 -- 10.20.193.28:0/1701934345 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5ac00d7040 con 0x5601dcd6c440 2026-02-25T11:51:57.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5aa4ce3640 1 -- 10.20.193.28:0/1701934345 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5601dc90d6f0 con 0x5601dcd6c440 2026-02-25T11:51:57.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5aa4ce3640 1 -- 10.20.193.28:0/1701934345 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5ac0048e80 con 0x5601dcd6c440 2026-02-25T11:51:57.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1701934345 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 msgr2=0x5601dcd6b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1701934345 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 0x5601dcd6b260 secure :-1 s=READY pgs=566 gs=1 cs=0 l=1 c_cookie=6027582086022431073 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac00d0560 tx=0x7f5ac0096000 comp rx=0 tx=0).stop 2026-02-25T11:51:57.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1701934345 shutdown_connections 2026-02-25T11:51:57.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1701934345 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcdb10e0 0x5601dcdaf200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1701934345 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6c440 0x5601dcd6b260 unknown :-1 s=CLOSED pgs=566 gs=1 cs=0 l=1 c_cookie=6027582086022431073 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1701934345 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcd6bb30 0x5601dcd6ad20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1701934345 >> 10.20.193.28:0/1701934345 conn(0x5601dcc8a7a0 msgr2=0x5601dcd6e9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:57.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1701934345 shutdown_connections 2026-02-25T11:51:57.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.233+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1701934345 wait complete. 2026-02-25T11:51:57.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.234+0000 7f5ae10b1e00 1 -- start start 2026-02-25T11:51:57.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.234+0000 7f5ae10b1e00 1 Processor -- start 2026-02-25T11:51:57.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5ae10b1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6bb30 0x5601dcd8fd70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5ae10b1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6c440 0x5601dcd8b760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5ae10b1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcdb10e0 0x5601dcd8bca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5ae10b1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5601dcd46e80 con 0x5601dcd6bb30 2026-02-25T11:51:57.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5ae10b1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5601dcd89b70 con 0x5601dcd6c440 2026-02-25T11:51:57.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5ae10b1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5601dcd8c1e0 con 0x5601dcdb10e0 2026-02-25T11:51:57.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5adce2b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6bb30 0x5601dcd8fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5adce2b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6bb30 0x5601dcd8fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59754/0 (socket says 10.20.193.28:59754) 2026-02-25T11:51:57.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5adce2b640 1 -- 10.20.193.28:0/1242756371 learned_addr learned my addr 10.20.193.28:0/1242756371 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:57.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5add62c640 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcdb10e0 0x5601dcd8bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5adce2b640 1 -- 10.20.193.28:0/1242756371 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcdb10e0 msgr2=0x5601dcd8bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5adce2b640 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcdb10e0 0x5601dcd8bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5acffff640 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6c440 0x5601dcd8b760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5adce2b640 1 -- 10.20.193.28:0/1242756371 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6c440 msgr2=0x5601dcd8b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5adce2b640 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6c440 0x5601dcd8b760 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5adce2b640 1 -- 10.20.193.28:0/1242756371 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601dcbfb6f0 con 0x5601dcd6bb30 2026-02-25T11:51:57.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5acffff640 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6c440 0x5601dcd8b760 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:51:57.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5adce2b640 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6bb30 0x5601dcd8fd70 secure :-1 s=READY pgs=567 gs=1 cs=0 l=1 c_cookie=14311471976947697559 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac8009ce0 tx=0x7f5ac80047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69c8ff006ecc797 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:57.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.235+0000 7f5a967fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:57.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5a967fc640 1 -- 10.20.193.28:0/1242756371 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5ac801db30 con 0x5601dcd6bb30 2026-02-25T11:51:57.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5a967fc640 1 -- 10.20.193.28:0/1242756371 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5ac802e070 con 0x5601dcd6bb30 2026-02-25T11:51:57.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5a967fc640 1 -- 10.20.193.28:0/1242756371 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5ac801b6c0 con 0x5601dcd6bb30 2026-02-25T11:51:57.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1242756371 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601dcc0d0e0 con 0x5601dcd6bb30 2026-02-25T11:51:57.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5ae10b1e00 10 client.16272 _ll_register_callbacks cb 0x5601dcbf8a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:57.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1242756371 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5601dc9cf4a0 con 0x5601dcd6bb30 2026-02-25T11:51:57.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5a967fc640 1 -- 10.20.193.28:0/1242756371 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f5ac801dc90 con 0x5601dcd6bb30 2026-02-25T11:51:57.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5a967fc640 1 -- 10.20.193.28:0/1242756371 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5ac801de90 con 0x5601dcd6bb30 2026-02-25T11:51:57.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5ae10b1e00 10 client.16272 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:57.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1242756371 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5601dcc0d340 con 0x5601dcd6bb30 2026-02-25T11:51:57.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5a967fc640 1 -- 10.20.193.28:0/1242756371 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f5ac8028070 con 0x5601dcd6bb30 2026-02-25T11:51:57.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5ae10b1e00 10 client.16272 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:57.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5ae10b1e00 10 client.16272 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:51:57.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5ae10b1e00 20 client.16272 populate_metadata read hostname 'trial028' 2026-02-25T11:51:57.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5601dce66bb0 0x5601dce87060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5ae10b1e00 4 client.16272 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:51:57.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.236+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1242756371 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5ad4003780 con 0x5601dce66bb0 2026-02-25T11:51:57.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.237+0000 7f5acffff640 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5601dce66bb0 0x5601dce87060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.237+0000 7f5acffff640 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5601dce66bb0 0x5601dce87060 crc :-1 s=READY pgs=39 gs=4 cs=0 l=0 c_cookie=9554301845944992834 s_cookie=5202707986943050594 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8497b2e874690442 server_cookie=4833bb5957d40362 in_seq=0 out_seq=0 2026-02-25T11:51:57.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.237+0000 7f5a967fc640 10 client.16272 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:51:57.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.237+0000 7f5a967fc640 1 -- 10.20.193.28:0/1242756371 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5ad4003780 con 0x5601dce66bb0 2026-02-25T11:51:57.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.237+0000 7f5a967fc640 10 client.16272 handle_command_reply: tid=0 2026-02-25T11:51:57.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.237+0000 7f5ae10b1e00 1 client.16272 shutdown 2026-02-25T11:51:57.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.238+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1242756371 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6bb30 msgr2=0x5601dcd8fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.238+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6bb30 0x5601dcd8fd70 secure :-1 s=READY pgs=567 gs=1 cs=0 l=1 c_cookie=14311471976947697559 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac8009ce0 tx=0x7f5ac80047e0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.238+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1242756371 shutdown_connections 2026-02-25T11:51:57.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.238+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5601dce66bb0 0x5601dce87060 crc :-1 s=READY pgs=39 gs=4 cs=0 l=0 c_cookie=9554301845944992834 s_cookie=5202707986943050594 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.238+0000 7f5a967fc640 0 client.16272 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:51:57.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.238+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601dcdb10e0 0x5601dcd8bca0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.238+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601dcd6c440 0x5601dcd8b760 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.238+0000 7f5ae10b1e00 1 --2- 10.20.193.28:0/1242756371 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601dcd6bb30 0x5601dcd8fd70 unknown :-1 s=CLOSED pgs=567 gs=1 cs=0 l=1 c_cookie=14311471976947697559 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.238+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1242756371 >> 10.20.193.28:0/1242756371 conn(0x5601dcc8a7a0 msgr2=0x5601dcd69d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:57.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.238+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1242756371 shutdown_connections 2026-02-25T11:51:57.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.238+0000 7f5ae10b1e00 1 -- 10.20.193.28:0/1242756371 wait complete. 2026-02-25T11:51:57.245 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:51:57.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.238+0000 7f5ae10b1e00 20 client.16272 trim_cache size 0 max 16384 2026-02-25T11:51:57.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.239+0000 7f5adde2d640 1 -- 10.20.193.28:0/1747213721 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ab40b5750 msgr2=0x7f5ab40d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.239+0000 7f5adde2d640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ab40b5750 0x7f5ab40d5c00 secure :-1 s=READY pgs=306 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac8004740 tx=0x7f5ac8005f90 comp rx=0 tx=0).stop 2026-02-25T11:51:57.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.239+0000 7f5adde2d640 1 -- 10.20.193.28:0/1747213721 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad8110080 msgr2=0x7f5ad8176150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.239+0000 7f5adde2d640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad8110080 0x7f5ad8176150 secure :-1 s=READY pgs=563 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ac00098a0 tx=0x7f5ac00028d0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.239+0000 7f5adde2d640 1 -- 10.20.193.28:0/1747213721 shutdown_connections 2026-02-25T11:51:57.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.239+0000 7f5adde2d640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5ab40b5750 0x7f5ab40d5c00 unknown :-1 s=CLOSED pgs=306 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.239+0000 7f5adde2d640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ad8115880 0x7f5ad80f6120 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.239+0000 7f5adde2d640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ad8110080 0x7f5ad8176150 unknown :-1 s=CLOSED pgs=563 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.239+0000 7f5adde2d640 1 --2- 10.20.193.28:0/1747213721 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ad810f690 0x7f5ad80f5a20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.239+0000 7f5adde2d640 1 -- 10.20.193.28:0/1747213721 >> 10.20.193.28:0/1747213721 conn(0x7f5ad80ecf00 msgr2=0x7f5ad8111e90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:57.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.239+0000 7f5adde2d640 1 -- 10.20.193.28:0/1747213721 shutdown_connections 2026-02-25T11:51:57.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.239+0000 7f5adde2d640 1 -- 10.20.193.28:0/1747213721 wait complete. 2026-02-25T11:51:57.373 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:51:57.556 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:57.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.721+0000 7fec97577640 1 -- 10.20.193.28:0/235962301 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec9810f690 msgr2=0x7fec9810fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.721+0000 7fec97577640 1 --2- 10.20.193.28:0/235962301 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec9810f690 0x7fec9810fa60 secure :-1 s=READY pgs=568 gs=1 cs=0 l=1 c_cookie=18104250883072318702 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec8c009950 tx=0x7fec8c02f1b0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.721+0000 7fec97577640 1 -- 10.20.193.28:0/235962301 shutdown_connections 2026-02-25T11:51:57.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.721+0000 7fec97577640 1 --2- 10.20.193.28:0/235962301 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec98115880 0x7fec98176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.721+0000 7fec97577640 1 --2- 10.20.193.28:0/235962301 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec98110080 0x7fec981150d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.721+0000 7fec97577640 1 --2- 10.20.193.28:0/235962301 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec9810f690 0x7fec9810fa60 unknown :-1 s=CLOSED pgs=568 gs=1 cs=0 l=1 c_cookie=18104250883072318702 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.721+0000 7fec97577640 1 -- 10.20.193.28:0/235962301 >> 10.20.193.28:0/235962301 conn(0x7fec980ecf00 msgr2=0x7fec9810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:57.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.721+0000 7fec97577640 1 -- 10.20.193.28:0/235962301 shutdown_connections 2026-02-25T11:51:57.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.721+0000 7fec97577640 1 -- 10.20.193.28:0/235962301 wait complete. 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.722+0000 7fec97577640 1 Processor -- start 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.722+0000 7fec97577640 1 -- start start 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.722+0000 7fec97577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec9810f690 0x7fec980f5960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.722+0000 7fec97577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec98110080 0x7fec98176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.722+0000 7fec97577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec98115880 0x7fec980f6060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.722+0000 7fec97577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fec980fb270 con 0x7fec98115880 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.722+0000 7fec97577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fec980fb3e0 con 0x7fec9810f690 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.722+0000 7fec97577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fec980fb5a0 con 0x7fec98110080 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.722+0000 7fec96d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec98115880 0x7fec980f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.722+0000 7fec96d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec98115880 0x7fec980f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59778/0 (socket says 10.20.193.28:59778) 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec96d76640 1 -- 10.20.193.28:0/3639898464 learned_addr learned my addr 10.20.193.28:0/3639898464 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec95d74640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec98110080 0x7fec98176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec96575640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec9810f690 0x7fec980f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec96d76640 1 -- 10.20.193.28:0/3639898464 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec98110080 msgr2=0x7fec98176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec96d76640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec98110080 0x7fec98176150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec96d76640 1 -- 10.20.193.28:0/3639898464 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec9810f690 msgr2=0x7fec980f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec96d76640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec9810f690 0x7fec980f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec96d76640 1 -- 10.20.193.28:0/3639898464 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8c009570 con 0x7fec98115880 2026-02-25T11:51:57.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec96d76640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec98115880 0x7fec980f6060 secure :-1 s=READY pgs=569 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec88006e90 tx=0x7fec8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:57.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec7b7fe640 1 -- 10.20.193.28:0/3639898464 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fec88005500 con 0x7fec98115880 2026-02-25T11:51:57.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec7b7fe640 1 -- 10.20.193.28:0/3639898464 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fec88005df0 con 0x7fec98115880 2026-02-25T11:51:57.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec7b7fe640 1 -- 10.20.193.28:0/3639898464 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fec88003a00 con 0x7fec98115880 2026-02-25T11:51:57.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec97577640 1 -- 10.20.193.28:0/3639898464 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec980fb880 con 0x7fec98115880 2026-02-25T11:51:57.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.723+0000 7fec97577640 1 -- 10.20.193.28:0/3639898464 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec980fbd00 con 0x7fec98115880 2026-02-25T11:51:57.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.726+0000 7fec7b7fe640 1 -- 10.20.193.28:0/3639898464 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fec88003b60 con 0x7fec98115880 2026-02-25T11:51:57.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.728+0000 7fec7b7fe640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec700b5750 0x7fec700d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.728+0000 7fec7b7fe640 1 -- 10.20.193.28:0/3639898464 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fec880e77e0 con 0x7fec98115880 2026-02-25T11:51:57.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.729+0000 7fec96575640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec700b5750 0x7fec700d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.729+0000 7fec9ee8be00 1 Processor -- start 2026-02-25T11:51:57.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.729+0000 7fec96575640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec700b5750 0x7fec700d5c00 secure :-1 s=READY pgs=307 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec8c005b30 tx=0x7fec8c03a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:57.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.729+0000 7fec9ee8be00 1 -- start start 2026-02-25T11:51:57.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.729+0000 7fec9ee8be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd73e2f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.729+0000 7fec9ee8be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd73e3440 0x555fd7427ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.729+0000 7fec9ee8be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd74280e0 0x555fd74484c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.729+0000 7fec9ee8be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555fd72f88a0 con 0x555fd73e3440 2026-02-25T11:51:57.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.729+0000 7fec9ee8be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555fd72f45d0 con 0x555fd73e2b30 2026-02-25T11:51:57.731 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.729+0000 7fec9ee8be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555fd72ee570 con 0x555fd74280e0 2026-02-25T11:51:57.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.730+0000 7fec95d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd73e3440 0x555fd7427ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.730+0000 7fec96575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd73e2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.730+0000 7fec96575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd73e2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43846/0 (socket says 10.20.193.28:43846) 2026-02-25T11:51:57.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.730+0000 7fec96575640 1 -- 10.20.193.28:0/712299799 learned_addr learned my addr 10.20.193.28:0/712299799 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:57.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.730+0000 7fec96d76640 1 --2- 10.20.193.28:0/712299799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd74280e0 0x555fd74484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.730+0000 7fec95d74640 1 -- 10.20.193.28:0/712299799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd74280e0 msgr2=0x555fd74484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.730+0000 7fec95d74640 1 --2- 10.20.193.28:0/712299799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd74280e0 0x555fd74484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.730+0000 7fec95d74640 1 -- 10.20.193.28:0/712299799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 msgr2=0x555fd73e2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.730+0000 7fec95d74640 1 --2- 10.20.193.28:0/712299799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd73e2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.730+0000 7fec95d74640 1 -- 10.20.193.28:0/712299799 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd7299d10 con 0x555fd73e3440 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.730+0000 7fec95d74640 1 --2- 10.20.193.28:0/712299799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd73e3440 0x555fd7427ba0 secure :-1 s=READY pgs=570 gs=1 cs=0 l=1 c_cookie=4712968698263536811 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec80004740 tx=0x7fec800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4167d40f898884ab server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.731+0000 7fec78ff9640 1 -- 10.20.193.28:0/712299799 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fec800040d0 con 0x555fd73e3440 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.731+0000 7fec78ff9640 1 -- 10.20.193.28:0/712299799 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fec800026a0 con 0x555fd73e3440 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.731+0000 7fec78ff9640 1 -- 10.20.193.28:0/712299799 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fec80011750 con 0x555fd73e3440 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.731+0000 7fec9ee8be00 1 -- 10.20.193.28:0/712299799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd73e3440 msgr2=0x555fd7427ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.731+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/712299799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd73e3440 0x555fd7427ba0 secure :-1 s=READY pgs=570 gs=1 cs=0 l=1 c_cookie=4712968698263536811 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec80004740 tx=0x7fec800107a0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.731+0000 7fec9ee8be00 1 -- 10.20.193.28:0/712299799 shutdown_connections 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.731+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/712299799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd74280e0 0x555fd74484c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.731+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/712299799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd73e3440 0x555fd7427ba0 unknown :-1 s=CLOSED pgs=570 gs=1 cs=0 l=1 c_cookie=4712968698263536811 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.731+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/712299799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd73e2f00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.731+0000 7fec9ee8be00 1 -- 10.20.193.28:0/712299799 >> 10.20.193.28:0/712299799 conn(0x555fd73017a0 msgr2=0x555fd73e1410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:57.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.731+0000 7fec9ee8be00 1 -- 10.20.193.28:0/712299799 shutdown_connections 2026-02-25T11:51:57.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.731+0000 7fec9ee8be00 1 -- 10.20.193.28:0/712299799 wait complete. 2026-02-25T11:51:57.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.733+0000 7fec9ee8be00 1 -- start start 2026-02-25T11:51:57.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.733+0000 7fec9ee8be00 1 Processor -- start 2026-02-25T11:51:57.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.733+0000 7fec9ee8be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd73e2b30 0x555fd74039d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.733+0000 7fec9ee8be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e3440 0x555fd7403f10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.733+0000 7fec9ee8be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd74280e0 0x555fd7408f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.733+0000 7fec9ee8be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555fd7409440 con 0x555fd74280e0 2026-02-25T11:51:57.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.733+0000 7fec9ee8be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555fd74095b0 con 0x555fd73e3440 2026-02-25T11:51:57.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.733+0000 7fec9ee8be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555fd7409770 con 0x555fd73e2b30 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.733+0000 7fec96d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd74280e0 0x555fd7408f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.733+0000 7fec96d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd74280e0 0x555fd7408f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59790/0 (socket says 10.20.193.28:59790) 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.733+0000 7fec96d76640 1 -- 10.20.193.28:0/1838324259 learned_addr learned my addr 10.20.193.28:0/1838324259 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec95d74640 1 --2- 10.20.193.28:0/1838324259 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e3440 0x555fd7403f10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec96d76640 1 -- 10.20.193.28:0/1838324259 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd73e2b30 msgr2=0x555fd74039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec96d76640 1 --2- 10.20.193.28:0/1838324259 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd73e2b30 0x555fd74039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec96d76640 1 -- 10.20.193.28:0/1838324259 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e3440 msgr2=0x555fd7403f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec96d76640 1 --2- 10.20.193.28:0/1838324259 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e3440 0x555fd7403f10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec96d76640 1 -- 10.20.193.28:0/1838324259 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd72723c0 con 0x555fd74280e0 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec95d74640 1 --2- 10.20.193.28:0/1838324259 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e3440 0x555fd7403f10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec96d76640 1 --2- 10.20.193.28:0/1838324259 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd74280e0 0x555fd7408f00 secure :-1 s=READY pgs=571 gs=1 cs=0 l=1 c_cookie=13262439233634909319 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec8800cfd0 tx=0x7fec880b3880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80da63730033c87 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec43fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec43fff640 1 -- 10.20.193.28:0/1838324259 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fec880e50b0 con 0x555fd74280e0 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec9ee8be00 1 -- 10.20.193.28:0/1838324259 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec8000b800 con 0x555fd74280e0 2026-02-25T11:51:57.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec9ee8be00 10 client.16284 _ll_register_callbacks cb 0x555fd726fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:57.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec9ee8be00 1 -- 10.20.193.28:0/1838324259 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555fd6f846f0 con 0x555fd74280e0 2026-02-25T11:51:57.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec43fff640 1 -- 10.20.193.28:0/1838324259 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fec880b6e90 con 0x555fd74280e0 2026-02-25T11:51:57.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec43fff640 1 -- 10.20.193.28:0/1838324259 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fec880aa990 con 0x555fd74280e0 2026-02-25T11:51:57.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec43fff640 1 -- 10.20.193.28:0/1838324259 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fec88098330 con 0x555fd74280e0 2026-02-25T11:51:57.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec43fff640 1 -- 10.20.193.28:0/1838324259 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fec8807a140 con 0x555fd74280e0 2026-02-25T11:51:57.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec9ee8be00 10 client.16284 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:57.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.734+0000 7fec9ee8be00 1 -- 10.20.193.28:0/1838324259 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x555fd72840e0 con 0x555fd74280e0 2026-02-25T11:51:57.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.735+0000 7fec43fff640 1 -- 10.20.193.28:0/1838324259 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fec8806dae0 con 0x555fd74280e0 2026-02-25T11:51:57.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.735+0000 7fec9ee8be00 10 client.16284 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.735+0000 7fec9ee8be00 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.735+0000 7fec9ee8be00 20 client.16284 populate_metadata read hostname 'trial028' 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.735+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/1838324259 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555fd740e2c0 0x555fd740e760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.735+0000 7fec9ee8be00 4 client.16284 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.735+0000 7fec9ee8be00 1 -- 10.20.193.28:0/1838324259 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fd72c0fc0 con 0x555fd740e2c0 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.735+0000 7fec96575640 1 --2- 10.20.193.28:0/1838324259 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555fd740e2c0 0x555fd740e760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.736+0000 7fec96575640 1 --2- 10.20.193.28:0/1838324259 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555fd740e2c0 0x555fd740e760 crc :-1 s=READY pgs=84 gs=4 cs=0 l=0 c_cookie=2086130222424660818 s_cookie=7518086165532492766 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cf36c626ed41b52 server_cookie=68559b88f757c3de in_seq=0 out_seq=0 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.736+0000 7fec43fff640 10 client.16284 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.736+0000 7fec43fff640 1 -- 10.20.193.28:0/1838324259 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x555fd72c0fc0 con 0x555fd740e2c0 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.736+0000 7fec43fff640 10 client.16284 handle_command_reply: tid=0 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.736+0000 7fec9ee8be00 1 client.16284 shutdown 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.737+0000 7fec9ee8be00 1 -- 10.20.193.28:0/1838324259 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd74280e0 msgr2=0x555fd7408f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.737+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/1838324259 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd74280e0 0x555fd7408f00 secure :-1 s=READY pgs=571 gs=1 cs=0 l=1 c_cookie=13262439233634909319 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec8800cfd0 tx=0x7fec880b3880 comp rx=0 tx=0).stop 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.737+0000 7fec9ee8be00 1 -- 10.20.193.28:0/1838324259 shutdown_connections 2026-02-25T11:51:57.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.737+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/1838324259 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555fd740e2c0 0x555fd740e760 crc :-1 s=READY pgs=84 gs=4 cs=0 l=0 c_cookie=2086130222424660818 s_cookie=7518086165532492766 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.737+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/1838324259 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd74280e0 0x555fd7408f00 unknown :-1 s=CLOSED pgs=571 gs=1 cs=0 l=1 c_cookie=13262439233634909319 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.737+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/1838324259 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e3440 0x555fd7403f10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.737+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/1838324259 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd73e2b30 0x555fd74039d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.737+0000 7fec9ee8be00 1 -- 10.20.193.28:0/1838324259 >> 10.20.193.28:0/1838324259 conn(0x555fd73017a0 msgr2=0x555fd73e1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:57.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.737+0000 7fec43fff640 0 client.16284 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:57.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.737+0000 7fec9ee8be00 1 -- 10.20.193.28:0/1838324259 shutdown_connections 2026-02-25T11:51:57.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.737+0000 7fec9ee8be00 1 -- 10.20.193.28:0/1838324259 wait complete. 2026-02-25T11:51:57.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.737+0000 7fec9ee8be00 20 client.16284 trim_cache size 0 max 16384 2026-02-25T11:51:57.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.744+0000 7fec9ee8be00 1 Processor -- start 2026-02-25T11:51:57.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.744+0000 7fec9ee8be00 1 -- start start 2026-02-25T11:51:57.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.744+0000 7fec9ee8be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd73f83b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.744+0000 7fec9ee8be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd73e3440 0x555fd73f88f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.744+0000 7fec9ee8be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd74280e0 0x555fd73e1d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.744+0000 7fec9ee8be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555fd72ec1a0 con 0x555fd73e3440 2026-02-25T11:51:57.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.744+0000 7fec9ee8be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555fd6fac700 con 0x555fd73e2b30 2026-02-25T11:51:57.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.744+0000 7fec9ee8be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555fd740ab80 con 0x555fd74280e0 2026-02-25T11:51:57.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.744+0000 7fec96d76640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd74280e0 0x555fd73e1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.744+0000 7fec96575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd73f83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.745+0000 7fec96d76640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd74280e0 0x555fd73e1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:60156/0 (socket says 10.20.193.28:60156) 2026-02-25T11:51:57.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.745+0000 7fec96d76640 1 -- 10.20.193.28:0/3766309962 learned_addr learned my addr 10.20.193.28:0/3766309962 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:57.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.745+0000 7fec95d74640 1 --2- 10.20.193.28:0/3766309962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd73e3440 0x555fd73f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.745+0000 7fec96d76640 1 -- 10.20.193.28:0/3766309962 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 msgr2=0x555fd73f83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.745+0000 7fec96d76640 1 --2- 10.20.193.28:0/3766309962 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd73f83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.745+0000 7fec96d76640 1 -- 10.20.193.28:0/3766309962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd73e3440 msgr2=0x555fd73f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.745+0000 7fec96d76640 1 --2- 10.20.193.28:0/3766309962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd73e3440 0x555fd73f88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.745+0000 7fec96d76640 1 -- 10.20.193.28:0/3766309962 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd73ed8e0 con 0x555fd74280e0 2026-02-25T11:51:57.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.745+0000 7fec96d76640 1 --2- 10.20.193.28:0/3766309962 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd74280e0 0x555fd73e1d20 secure :-1 s=READY pgs=161 gs=3 cs=0 l=1 c_cookie=400548801639967544 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec880abef0 tx=0x7fec880b3000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58f09034cf56b38 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:57.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.745+0000 7fec624e6640 1 -- 10.20.193.28:0/3766309962 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fec880b6e90 con 0x555fd74280e0 2026-02-25T11:51:57.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.745+0000 7fec624e6640 1 -- 10.20.193.28:0/3766309962 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x555fd6f846f0 con 0x555fd74280e0 2026-02-25T11:51:57.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.745+0000 7fec624e6640 1 -- 10.20.193.28:0/3766309962 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fec8806dae0 con 0x555fd74280e0 2026-02-25T11:51:57.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.746+0000 7fec9ee8be00 1 -- 10.20.193.28:0/3766309962 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd74280e0 msgr2=0x555fd73e1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.746+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/3766309962 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd74280e0 0x555fd73e1d20 secure :-1 s=READY pgs=161 gs=3 cs=0 l=1 c_cookie=400548801639967544 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec880abef0 tx=0x7fec880b3000 comp rx=0 tx=0).stop 2026-02-25T11:51:57.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.746+0000 7fec9ee8be00 1 -- 10.20.193.28:0/3766309962 shutdown_connections 2026-02-25T11:51:57.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.746+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/3766309962 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd74280e0 0x555fd73e1d20 unknown :-1 s=CLOSED pgs=161 gs=3 cs=0 l=1 c_cookie=400548801639967544 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.746+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/3766309962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd73e3440 0x555fd73f88f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.746+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/3766309962 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd73f83b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.746+0000 7fec9ee8be00 1 -- 10.20.193.28:0/3766309962 >> 10.20.193.28:0/3766309962 conn(0x555fd73017a0 msgr2=0x555fd73e59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.746+0000 7fec9ee8be00 1 -- 10.20.193.28:0/3766309962 shutdown_connections 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.746+0000 7fec9ee8be00 1 -- 10.20.193.28:0/3766309962 wait complete. 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.746+0000 7fec9ee8be00 1 -- start start 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.747+0000 7fec9ee8be00 1 Processor -- start 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.747+0000 7fec9ee8be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd740f2d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.747+0000 7fec9ee8be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd73e3440 0x555fd740f810 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.747+0000 7fec9ee8be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd74280e0 0x555fd740fd50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.747+0000 7fec9ee8be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x555fd73fa840 con 0x555fd74280e0 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.747+0000 7fec9ee8be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555fd73bde80 con 0x555fd73e2b30 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.747+0000 7fec9ee8be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555fd73fad00 con 0x555fd73e3440 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.747+0000 7fec96d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd74280e0 0x555fd740fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.747+0000 7fec96575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd740f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.747+0000 7fec96575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd740f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43874/0 (socket says 10.20.193.28:43874) 2026-02-25T11:51:57.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.747+0000 7fec96575640 1 -- 10.20.193.28:0/2793773285 learned_addr learned my addr 10.20.193.28:0/2793773285 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:57.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.747+0000 7fec95d74640 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd73e3440 0x555fd740f810 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec96d76640 1 -- 10.20.193.28:0/2793773285 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd73e3440 msgr2=0x555fd740f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec96d76640 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd73e3440 0x555fd740f810 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec96d76640 1 -- 10.20.193.28:0/2793773285 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 msgr2=0x555fd740f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:57.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec96d76640 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd740f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec96d76640 1 -- 10.20.193.28:0/2793773285 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd727eb60 con 0x555fd74280e0 2026-02-25T11:51:57.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec95d74640 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd73e3440 0x555fd740f810 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:57.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec96575640 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd740f2d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:51:57.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec96d76640 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd74280e0 0x555fd740fd50 secure :-1 s=READY pgs=572 gs=1 cs=0 l=1 c_cookie=874715772106170034 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x555fd735cce0 tx=0x7fec88005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c239d5c3bd1deb2 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:57.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec43fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:57.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec43fff640 1 -- 10.20.193.28:0/2793773285 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fec880b8cf0 con 0x555fd74280e0 2026-02-25T11:51:57.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec43fff640 1 -- 10.20.193.28:0/2793773285 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fec8800c8d0 con 0x555fd74280e0 2026-02-25T11:51:57.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec43fff640 1 -- 10.20.193.28:0/2793773285 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fec88074d20 con 0x555fd74280e0 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec9ee8be00 1 -- 10.20.193.28:0/2793773285 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fd72840e0 con 0x555fd74280e0 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec43fff640 1 -- 10.20.193.28:0/2793773285 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fec880980e0 con 0x555fd74280e0 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec43fff640 10 client.16290 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec9ee8be00 10 client.16290 _ll_register_callbacks cb 0x555fd726fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec9ee8be00 1 -- 10.20.193.28:0/2793773285 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec800026a0 con 0x555fd74280e0 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec43fff640 1 -- 10.20.193.28:0/2793773285 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fec88098360 con 0x555fd74280e0 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec9ee8be00 10 client.16290 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.748+0000 7fec9ee8be00 1 -- 10.20.193.28:0/2793773285 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x555fd72726f0 con 0x555fd74280e0 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.749+0000 7fec43fff640 1 -- 10.20.193.28:0/2793773285 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fec880ae9c0 con 0x555fd74280e0 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.749+0000 7fec9ee8be00 10 client.16290 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.749+0000 7fec9ee8be00 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.749+0000 7fec9ee8be00 20 client.16290 populate_metadata read hostname 'trial028' 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.749+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555fd73fc090 0x555fd73fc500 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.749+0000 7fec9ee8be00 4 client.16290 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.749+0000 7fec9ee8be00 1 -- 10.20.193.28:0/2793773285 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555fd6f81f40 con 0x555fd73fc090 2026-02-25T11:51:57.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.749+0000 7fec96575640 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555fd73fc090 0x555fd73fc500 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec96575640 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555fd73fc090 0x555fd73fc500 crc :-1 s=READY pgs=85 gs=4 cs=0 l=0 c_cookie=10559639075412816357 s_cookie=14168604986667162615 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=928b5fcaae50b1e5 server_cookie=c4a0ff2ab91fdbf7 in_seq=0 out_seq=0 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec43fff640 10 client.16290 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec43fff640 1 -- 10.20.193.28:0/2793773285 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555fd6f81f40 con 0x555fd73fc090 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec43fff640 10 client.16290 handle_command_reply: tid=0 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec9ee8be00 1 client.16290 shutdown 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec9ee8be00 1 -- 10.20.193.28:0/2793773285 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd74280e0 msgr2=0x555fd740fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd74280e0 0x555fd740fd50 secure :-1 s=READY pgs=572 gs=1 cs=0 l=1 c_cookie=874715772106170034 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x555fd735cce0 tx=0x7fec88005730 comp rx=0 tx=0).stop 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec9ee8be00 1 -- 10.20.193.28:0/2793773285 shutdown_connections 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x555fd73fc090 0x555fd73fc500 crc :-1 s=READY pgs=85 gs=4 cs=0 l=0 c_cookie=10559639075412816357 s_cookie=14168604986667162615 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x555fd74280e0 0x555fd740fd50 unknown :-1 s=CLOSED pgs=572 gs=1 cs=0 l=1 c_cookie=874715772106170034 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555fd73e3440 0x555fd740f810 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec9ee8be00 1 --2- 10.20.193.28:0/2793773285 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555fd73e2b30 0x555fd740f2d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec43fff640 0 client.16290 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec9ee8be00 1 -- 10.20.193.28:0/2793773285 >> 10.20.193.28:0/2793773285 conn(0x555fd73017a0 msgr2=0x555fd7403710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:57.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec9ee8be00 1 -- 10.20.193.28:0/2793773285 shutdown_connections 2026-02-25T11:51:57.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec9ee8be00 1 -- 10.20.193.28:0/2793773285 wait complete. 2026-02-25T11:51:57.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.750+0000 7fec9ee8be00 20 client.16290 trim_cache size 0 max 16384 2026-02-25T11:51:57.755 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:51:57.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.751+0000 7fec97577640 1 -- 10.20.193.28:0/3639898464 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec700b5750 msgr2=0x7fec700d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.751+0000 7fec97577640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec700b5750 0x7fec700d5c00 secure :-1 s=READY pgs=307 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec8c005b30 tx=0x7fec8c03a040 comp rx=0 tx=0).stop 2026-02-25T11:51:57.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.751+0000 7fec97577640 1 -- 10.20.193.28:0/3639898464 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec98115880 msgr2=0x7fec980f6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:57.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.751+0000 7fec97577640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec98115880 0x7fec980f6060 secure :-1 s=READY pgs=569 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec88006e90 tx=0x7fec8800edf0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.751+0000 7fec97577640 1 -- 10.20.193.28:0/3639898464 shutdown_connections 2026-02-25T11:51:57.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.751+0000 7fec97577640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec700b5750 0x7fec700d5c00 unknown :-1 s=CLOSED pgs=307 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.751+0000 7fec97577640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec98115880 0x7fec980f6060 unknown :-1 s=CLOSED pgs=569 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.751+0000 7fec97577640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec98110080 0x7fec98176150 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.751+0000 7fec97577640 1 --2- 10.20.193.28:0/3639898464 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec9810f690 0x7fec980f5960 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:57.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.751+0000 7fec97577640 1 -- 10.20.193.28:0/3639898464 >> 10.20.193.28:0/3639898464 conn(0x7fec980ecf00 msgr2=0x7fec98111b30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:57.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.751+0000 7fec97577640 1 -- 10.20.193.28:0/3639898464 shutdown_connections 2026-02-25T11:51:57.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:57.751+0000 7fec97577640 1 -- 10.20.193.28:0/3639898464 wait complete. 2026-02-25T11:51:58.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:51:58 trial167 ceph-mon[43392]: pgmap v338: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:58.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:51:58 trial119 ceph-mon[42710]: pgmap v338: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:58.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:51:58 trial028 ceph-mon[39224]: pgmap v338: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:51:58.852 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:51:58.853 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:51:59.038 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:59.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.226+0000 7f735bfff640 1 -- 10.20.193.28:0/4241300954 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f735c110080 msgr2=0x7f735c1150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.226+0000 7f735bfff640 1 --2- 10.20.193.28:0/4241300954 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f735c110080 0x7f735c1150d0 secure :-1 s=READY pgs=573 gs=1 cs=0 l=1 c_cookie=14830679223895083330 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7344009950 tx=0x7f734402f190 comp rx=0 tx=0).stop 2026-02-25T11:51:59.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.227+0000 7f735bfff640 1 -- 10.20.193.28:0/4241300954 shutdown_connections 2026-02-25T11:51:59.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.227+0000 7f735bfff640 1 --2- 10.20.193.28:0/4241300954 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f735c115880 0x7f735c176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.227+0000 7f735bfff640 1 --2- 10.20.193.28:0/4241300954 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f735c110080 0x7f735c1150d0 unknown :-1 s=CLOSED pgs=573 gs=1 cs=0 l=1 c_cookie=14830679223895083330 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.227+0000 7f735bfff640 1 --2- 10.20.193.28:0/4241300954 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f735c10f690 0x7f735c10fa60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.227+0000 7f735bfff640 1 -- 10.20.193.28:0/4241300954 >> 10.20.193.28:0/4241300954 conn(0x7f735c0ecf00 msgr2=0x7f735c10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:59.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.227+0000 7f735bfff640 1 -- 10.20.193.28:0/4241300954 shutdown_connections 2026-02-25T11:51:59.229 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.227+0000 7f735bfff640 1 -- 10.20.193.28:0/4241300954 wait complete. 2026-02-25T11:51:59.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.227+0000 7f735bfff640 1 Processor -- start 2026-02-25T11:51:59.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.227+0000 7f735bfff640 1 -- start start 2026-02-25T11:51:59.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f735c10f690 0x7f735c0f59d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f735c110080 0x7f735c176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f735c115880 0x7f735c0f60d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735bfff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f735c0fb2e0 con 0x7f735c110080 2026-02-25T11:51:59.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735bfff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f735c0fb450 con 0x7f735c115880 2026-02-25T11:51:59.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735bfff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f735c0fb610 con 0x7f735c10f690 2026-02-25T11:51:59.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735affd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f735c10f690 0x7f735c0f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735affd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f735c10f690 0x7f735c0f59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:60180/0 (socket says 10.20.193.28:60180) 2026-02-25T11:51:59.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735affd640 1 -- 10.20.193.28:0/1779038216 learned_addr learned my addr 10.20.193.28:0/1779038216 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:59.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735a7fc640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f735c110080 0x7f735c176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735b7fe640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f735c115880 0x7f735c0f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735affd640 1 -- 10.20.193.28:0/1779038216 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f735c115880 msgr2=0x7f735c0f60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735affd640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f735c115880 0x7f735c0f60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735affd640 1 -- 10.20.193.28:0/1779038216 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f735c110080 msgr2=0x7f735c176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735affd640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f735c110080 0x7f735c176150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.228+0000 7f735affd640 1 -- 10.20.193.28:0/1779038216 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7344009570 con 0x7f735c10f690 2026-02-25T11:51:59.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.229+0000 7f735affd640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f735c10f690 0x7f735c0f59d0 secure :-1 s=READY pgs=162 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7350009480 tx=0x7f735000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:59.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.229+0000 7f736088f640 1 -- 10.20.193.28:0/1779038216 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f735000cd40 con 0x7f735c10f690 2026-02-25T11:51:59.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.229+0000 7f735bfff640 1 -- 10.20.193.28:0/1779038216 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f735c0fb8f0 con 0x7f735c10f690 2026-02-25T11:51:59.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.229+0000 7f735bfff640 1 -- 10.20.193.28:0/1779038216 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f735c0fbe10 con 0x7f735c10f690 2026-02-25T11:51:59.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.229+0000 7f736088f640 1 -- 10.20.193.28:0/1779038216 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7350004d10 con 0x7f735c10f690 2026-02-25T11:51:59.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.229+0000 7f736088f640 1 -- 10.20.193.28:0/1779038216 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f73500106b0 con 0x7f735c10f690 2026-02-25T11:51:59.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.235+0000 7f736088f640 1 -- 10.20.193.28:0/1779038216 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f735001f060 con 0x7f735c10f690 2026-02-25T11:51:59.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.235+0000 7f736388ce00 1 Processor -- start 2026-02-25T11:51:59.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.235+0000 7f736388ce00 1 -- start start 2026-02-25T11:51:59.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.235+0000 7f736388ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 0x559f3454e530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.235+0000 7f736388ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 0x559f34593020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.235+0000 7f736388ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f34593560 0x559f345b3940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.235+0000 7f736388ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559f3445bd80 con 0x559f3454e160 2026-02-25T11:51:59.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.235+0000 7f736388ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559f34457ab0 con 0x559f3454ea70 2026-02-25T11:51:59.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.235+0000 7f736388ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559f34451a50 con 0x559f34593560 2026-02-25T11:51:59.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.235+0000 7f735b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 0x559f3454e530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.235+0000 7f735affd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 0x559f34593020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.235+0000 7f735b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 0x559f3454e530 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59834/0 (socket says 10.20.193.28:59834) 2026-02-25T11:51:59.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.235+0000 7f735b7fe640 1 -- 10.20.193.28:0/1682927611 learned_addr learned my addr 10.20.193.28:0/1682927611 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:59.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.236+0000 7f735a7fc640 1 --2- 10.20.193.28:0/1682927611 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f34593560 0x559f345b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.236+0000 7f735b7fe640 1 -- 10.20.193.28:0/1682927611 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f34593560 msgr2=0x559f345b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.236+0000 7f735b7fe640 1 --2- 10.20.193.28:0/1682927611 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f34593560 0x559f345b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.236+0000 7f735b7fe640 1 -- 10.20.193.28:0/1682927611 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 msgr2=0x559f34593020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.236+0000 7f735b7fe640 1 --2- 10.20.193.28:0/1682927611 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 0x559f34593020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.236+0000 7f735b7fe640 1 -- 10.20.193.28:0/1682927611 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f343fd120 con 0x559f3454e160 2026-02-25T11:51:59.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.236+0000 7f735b7fe640 1 --2- 10.20.193.28:0/1682927611 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 0x559f3454e530 secure :-1 s=READY pgs=574 gs=3 cs=0 l=1 c_cookie=12528860318069197138 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f734c003a00 tx=0x7f734c005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addf74141b008952 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:59.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.236+0000 7f731ffff640 1 -- 10.20.193.28:0/1682927611 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f734c011bd0 con 0x559f3454e160 2026-02-25T11:51:59.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.236+0000 7f731ffff640 1 -- 10.20.193.28:0/1682927611 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f734c011d30 con 0x559f3454e160 2026-02-25T11:51:59.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.236+0000 7f731ffff640 1 -- 10.20.193.28:0/1682927611 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f734c012890 con 0x559f3454e160 2026-02-25T11:51:59.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.236+0000 7f736388ce00 1 -- 10.20.193.28:0/1682927611 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 msgr2=0x559f3454e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.236+0000 7f736388ce00 1 --2- 10.20.193.28:0/1682927611 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 0x559f3454e530 secure :-1 s=READY pgs=574 gs=3 cs=0 l=1 c_cookie=12528860318069197138 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f734c003a00 tx=0x7f734c005790 comp rx=0 tx=0).stop 2026-02-25T11:51:59.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.236+0000 7f736088f640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73300b57d0 0x7f73300d5c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.237+0000 7f735b7fe640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73300b57d0 0x7f73300d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.237+0000 7f736388ce00 1 -- 10.20.193.28:0/1682927611 shutdown_connections 2026-02-25T11:51:59.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.237+0000 7f736388ce00 1 --2- 10.20.193.28:0/1682927611 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f34593560 0x559f345b3940 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.237+0000 7f736388ce00 1 --2- 10.20.193.28:0/1682927611 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 0x559f34593020 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.240 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.237+0000 7f736388ce00 1 --2- 10.20.193.28:0/1682927611 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 0x559f3454e530 unknown :-1 s=CLOSED pgs=574 gs=3 cs=0 l=1 c_cookie=12528860318069197138 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.237+0000 7f736388ce00 1 -- 10.20.193.28:0/1682927611 >> 10.20.193.28:0/1682927611 conn(0x559f345238f0 msgr2=0x559f34543cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:59.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.237+0000 7f736088f640 1 -- 10.20.193.28:0/1779038216 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f7350014070 con 0x7f735c10f690 2026-02-25T11:51:59.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.237+0000 7f736388ce00 1 -- 10.20.193.28:0/1682927611 shutdown_connections 2026-02-25T11:51:59.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.237+0000 7f736388ce00 1 -- 10.20.193.28:0/1682927611 wait complete. 2026-02-25T11:51:59.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.237+0000 7f735b7fe640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73300b57d0 0x7f73300d5c80 secure :-1 s=READY pgs=308 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f734c0050f0 tx=0x7f734c005050 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:59.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.238+0000 7f736388ce00 1 -- start start 2026-02-25T11:51:59.241 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.239+0000 7f736388ce00 1 Processor -- start 2026-02-25T11:51:59.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.240+0000 7f736388ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 0x559f345aadc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.240+0000 7f736388ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 0x559f345ab300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.240+0000 7f736388ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f34593560 0x559f345b02f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.240+0000 7f736388ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559f345b0830 con 0x559f34593560 2026-02-25T11:51:59.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.240+0000 7f736388ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559f345b09a0 con 0x559f3454ea70 2026-02-25T11:51:59.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.240+0000 7f736388ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559f345b0b60 con 0x559f3454e160 2026-02-25T11:51:59.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.240+0000 7f735affd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 0x559f345aadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.240+0000 7f735affd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 0x559f345aadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:60196/0 (socket says 10.20.193.28:60196) 2026-02-25T11:51:59.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.240+0000 7f735affd640 1 -- 10.20.193.28:0/3020420115 learned_addr learned my addr 10.20.193.28:0/3020420115 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:59.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.240+0000 7f735a7fc640 1 --2- 10.20.193.28:0/3020420115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 0x559f345ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.240+0000 7f735b7fe640 1 --2- 10.20.193.28:0/3020420115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f34593560 0x559f345b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.241+0000 7f735affd640 1 -- 10.20.193.28:0/3020420115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 msgr2=0x559f345ab300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.241+0000 7f735affd640 1 --2- 10.20.193.28:0/3020420115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 0x559f345ab300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.241+0000 7f735affd640 1 -- 10.20.193.28:0/3020420115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f34593560 msgr2=0x559f345b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.241+0000 7f735affd640 1 --2- 10.20.193.28:0/3020420115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f34593560 0x559f345b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.241+0000 7f735affd640 1 -- 10.20.193.28:0/3020420115 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f343d5810 con 0x559f3454e160 2026-02-25T11:51:59.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.241+0000 7f735affd640 1 --2- 10.20.193.28:0/3020420115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 0x559f345aadc0 secure :-1 s=READY pgs=163 gs=1 cs=0 l=1 c_cookie=6367341036132979357 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f735000c6a0 tx=0x7f7350005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=585d56efce79529d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:59.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.241+0000 7f731d7fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:51:59.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.241+0000 7f731d7fa640 1 -- 10.20.193.28:0/3020420115 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f73500adef0 con 0x559f3454e160 2026-02-25T11:51:59.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.241+0000 7f731d7fa640 1 -- 10.20.193.28:0/3020420115 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f73500ad0f0 con 0x559f3454e160 2026-02-25T11:51:59.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.241+0000 7f731d7fa640 1 -- 10.20.193.28:0/3020420115 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f73500b7f70 con 0x559f3454e160 2026-02-25T11:51:59.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.241+0000 7f736388ce00 1 -- 10.20.193.28:0/3020420115 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c005550 con 0x559f3454e160 2026-02-25T11:51:59.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.242+0000 7f731d7fa640 1 -- 10.20.193.28:0/3020420115 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f73500b8180 con 0x559f3454e160 2026-02-25T11:51:59.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.242+0000 7f736388ce00 10 client.26068 _ll_register_callbacks cb 0x559f343d2e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:59.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.242+0000 7f736388ce00 1 -- 10.20.193.28:0/3020420115 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559f341a9530 con 0x559f3454e160 2026-02-25T11:51:59.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.242+0000 7f731d7fa640 1 -- 10.20.193.28:0/3020420115 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f73500aca20 con 0x559f3454e160 2026-02-25T11:51:59.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.242+0000 7f736388ce00 10 client.26068 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:59.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.242+0000 7f736388ce00 1 -- 10.20.193.28:0/3020420115 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x559f343e74f0 con 0x559f3454e160 2026-02-25T11:51:59.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.242+0000 7f731d7fa640 1 -- 10.20.193.28:0/3020420115 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f7350072140 con 0x559f3454e160 2026-02-25T11:51:59.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.242+0000 7f736388ce00 10 client.26068 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:59.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.242+0000 7f736388ce00 10 client.26068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:59.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.242+0000 7f736388ce00 20 client.26068 populate_metadata read hostname 'trial028' 2026-02-25T11:51:59.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.242+0000 7f736388ce00 1 --2- 10.20.193.28:0/3020420115 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559f345905c0 0x559f34590a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.242+0000 7f736388ce00 4 client.26068 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:51:59.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.242+0000 7f736388ce00 1 -- 10.20.193.28:0/3020420115 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f343feb50 con 0x559f345905c0 2026-02-25T11:51:59.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.243+0000 7f735a7fc640 1 --2- 10.20.193.28:0/3020420115 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559f345905c0 0x559f34590a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.243+0000 7f735a7fc640 1 --2- 10.20.193.28:0/3020420115 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559f345905c0 0x559f34590a60 crc :-1 s=READY pgs=86 gs=4 cs=0 l=0 c_cookie=3377093657995819971 s_cookie=5694223067369789508 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eddd8ca1210bbc3 server_cookie=4f05f1b5e2695844 in_seq=0 out_seq=0 2026-02-25T11:51:59.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.243+0000 7f731d7fa640 10 client.26068 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:59.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f731d7fa640 1 -- 10.20.193.28:0/3020420115 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x559f343feb50 con 0x559f345905c0 2026-02-25T11:51:59.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f731d7fa640 10 client.26068 handle_command_reply: tid=0 2026-02-25T11:51:59.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f736388ce00 1 client.26068 shutdown 2026-02-25T11:51:59.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f736388ce00 1 -- 10.20.193.28:0/3020420115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 msgr2=0x559f345aadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f736388ce00 1 --2- 10.20.193.28:0/3020420115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 0x559f345aadc0 secure :-1 s=READY pgs=163 gs=1 cs=0 l=1 c_cookie=6367341036132979357 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f735000c6a0 tx=0x7f7350005880 comp rx=0 tx=0).stop 2026-02-25T11:51:59.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f736388ce00 1 -- 10.20.193.28:0/3020420115 shutdown_connections 2026-02-25T11:51:59.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f736388ce00 1 --2- 10.20.193.28:0/3020420115 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559f345905c0 0x559f34590a60 crc :-1 s=READY pgs=86 gs=4 cs=0 l=0 c_cookie=3377093657995819971 s_cookie=5694223067369789508 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f736388ce00 1 --2- 10.20.193.28:0/3020420115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f34593560 0x559f345b02f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f731d7fa640 0 client.26068 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:59.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f736388ce00 1 --2- 10.20.193.28:0/3020420115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 0x559f345ab300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f736388ce00 1 --2- 10.20.193.28:0/3020420115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 0x559f345aadc0 unknown :-1 s=CLOSED pgs=163 gs=1 cs=0 l=1 c_cookie=6367341036132979357 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f736388ce00 1 -- 10.20.193.28:0/3020420115 >> 10.20.193.28:0/3020420115 conn(0x559f345238f0 msgr2=0x559f34543cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:59.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f736388ce00 1 -- 10.20.193.28:0/3020420115 shutdown_connections 2026-02-25T11:51:59.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f736388ce00 1 -- 10.20.193.28:0/3020420115 wait complete. 2026-02-25T11:51:59.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.244+0000 7f736388ce00 20 client.26068 trim_cache size 0 max 16384 2026-02-25T11:51:59.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f736388ce00 1 Processor -- start 2026-02-25T11:51:59.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f736388ce00 1 -- start start 2026-02-25T11:51:59.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f736388ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 0x559f3458e150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f736388ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 0x559f3458e690 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f736388ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f34593560 0x559f345a7060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f736388ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559f3444f680 con 0x559f3454e160 2026-02-25T11:51:59.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f736388ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559f3410f790 con 0x559f3454ea70 2026-02-25T11:51:59.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f736388ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559f345ab8e0 con 0x559f34593560 2026-02-25T11:51:59.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f735affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 0x559f3458e150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f735affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 0x559f3458e150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59840/0 (socket says 10.20.193.28:59840) 2026-02-25T11:51:59.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f735affd640 1 -- 10.20.193.28:0/2296195082 learned_addr learned my addr 10.20.193.28:0/2296195082 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:59.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f735b7fe640 1 --2- 10.20.193.28:0/2296195082 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f34593560 0x559f345a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f735a7fc640 1 --2- 10.20.193.28:0/2296195082 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 0x559f3458e690 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f735affd640 1 -- 10.20.193.28:0/2296195082 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f34593560 msgr2=0x559f345a7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f735affd640 1 --2- 10.20.193.28:0/2296195082 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f34593560 0x559f345a7060 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f735affd640 1 -- 10.20.193.28:0/2296195082 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 msgr2=0x559f3458e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f735affd640 1 --2- 10.20.193.28:0/2296195082 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 0x559f3458e690 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.254+0000 7f735affd640 1 -- 10.20.193.28:0/2296195082 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f3459d350 con 0x559f3454e160 2026-02-25T11:51:59.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.255+0000 7f735affd640 1 --2- 10.20.193.28:0/2296195082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 0x559f3458e150 secure :-1 s=READY pgs=575 gs=1 cs=0 l=1 c_cookie=5552639464162560001 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f73500a9610 tx=0x7f73500eb000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0ef02c6b398801 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:59.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.255+0000 7f731f7fe640 1 -- 10.20.193.28:0/2296195082 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7350072ea0 con 0x559f3454e160 2026-02-25T11:51:59.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.255+0000 7f731f7fe640 1 -- 10.20.193.28:0/2296195082 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x559f341a9530 con 0x559f3454e160 2026-02-25T11:51:59.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.255+0000 7f731f7fe640 1 -- 10.20.193.28:0/2296195082 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f73500b8180 con 0x559f3454e160 2026-02-25T11:51:59.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.255+0000 7f736388ce00 1 -- 10.20.193.28:0/2296195082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 msgr2=0x559f3458e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.255+0000 7f736388ce00 1 --2- 10.20.193.28:0/2296195082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 0x559f3458e150 secure :-1 s=READY pgs=575 gs=1 cs=0 l=1 c_cookie=5552639464162560001 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f73500a9610 tx=0x7f73500eb000 comp rx=0 tx=0).stop 2026-02-25T11:51:59.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.255+0000 7f736388ce00 1 -- 10.20.193.28:0/2296195082 shutdown_connections 2026-02-25T11:51:59.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.255+0000 7f736388ce00 1 --2- 10.20.193.28:0/2296195082 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f34593560 0x559f345a7060 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.255+0000 7f736388ce00 1 --2- 10.20.193.28:0/2296195082 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f3454ea70 0x559f3458e690 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.255+0000 7f736388ce00 1 --2- 10.20.193.28:0/2296195082 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454e160 0x559f3458e150 unknown :-1 s=CLOSED pgs=575 gs=1 cs=0 l=1 c_cookie=5552639464162560001 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.255+0000 7f736388ce00 1 -- 10.20.193.28:0/2296195082 >> 10.20.193.28:0/2296195082 conn(0x559f345238f0 msgr2=0x559f345af940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:59.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.255+0000 7f736388ce00 1 -- 10.20.193.28:0/2296195082 shutdown_connections 2026-02-25T11:51:59.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.255+0000 7f736388ce00 1 -- 10.20.193.28:0/2296195082 wait complete. 2026-02-25T11:51:59.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.256+0000 7f736388ce00 1 -- start start 2026-02-25T11:51:59.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.256+0000 7f736388ce00 1 Processor -- start 2026-02-25T11:51:59.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.256+0000 7f736388ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 0x559f34523cc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.256+0000 7f736388ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454ea70 0x559f34524200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.256+0000 7f736388ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f34593560 0x559f34524740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.256+0000 7f736388ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559f3454e770 con 0x559f3454ea70 2026-02-25T11:51:59.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.256+0000 7f736388ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559f345fa3f0 con 0x559f34593560 2026-02-25T11:51:59.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.256+0000 7f736388ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559f345fa5b0 con 0x559f3454e160 2026-02-25T11:51:59.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.256+0000 7f735a7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454ea70 0x559f34524200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f735affd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 0x559f34523cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f735b7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f34593560 0x559f34524740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f735affd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 0x559f34523cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:60224/0 (socket says 10.20.193.28:60224) 2026-02-25T11:51:59.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f735affd640 1 -- 10.20.193.28:0/2675872218 learned_addr learned my addr 10.20.193.28:0/2675872218 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:59.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f735a7fc640 1 --2- 10.20.193.28:0/2675872218 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454ea70 0x559f34524200 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59850/0 (socket says 10.20.193.28:59850) 2026-02-25T11:51:59.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f735affd640 1 -- 10.20.193.28:0/2675872218 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f34593560 msgr2=0x559f34524740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f735affd640 1 --2- 10.20.193.28:0/2675872218 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f34593560 0x559f34524740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f735affd640 1 -- 10.20.193.28:0/2675872218 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454ea70 msgr2=0x559f34524200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f735affd640 1 --2- 10.20.193.28:0/2675872218 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454ea70 0x559f34524200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f735affd640 1 -- 10.20.193.28:0/2675872218 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f343e1f70 con 0x559f3454e160 2026-02-25T11:51:59.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f735b7fe640 1 --2- 10.20.193.28:0/2675872218 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f34593560 0x559f34524740 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:51:59.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f735affd640 1 --2- 10.20.193.28:0/2675872218 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 0x559f34523cc0 secure :-1 s=READY pgs=164 gs=1 cs=0 l=1 c_cookie=3758713785046758154 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f73500e6e40 tx=0x7f73500e4b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3429a245e213730a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:59.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f731d7fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:51:59.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f731d7fa640 1 -- 10.20.193.28:0/2675872218 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f73500c4ea0 con 0x559f3454e160 2026-02-25T11:51:59.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f731d7fa640 1 -- 10.20.193.28:0/2675872218 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f73500ac840 con 0x559f3454e160 2026-02-25T11:51:59.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f731d7fa640 1 -- 10.20.193.28:0/2675872218 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7350072ea0 con 0x559f3454e160 2026-02-25T11:51:59.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.257+0000 7f736388ce00 1 -- 10.20.193.28:0/2675872218 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c005550 con 0x559f3454e160 2026-02-25T11:51:59.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f731d7fa640 1 -- 10.20.193.28:0/2675872218 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f73500050b0 con 0x559f3454e160 2026-02-25T11:51:59.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f736388ce00 10 client.26074 _ll_register_callbacks cb 0x559f343d2e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:59.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f736388ce00 1 -- 10.20.193.28:0/2675872218 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f734c011d30 con 0x559f3454e160 2026-02-25T11:51:59.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f731d7fa640 1 -- 10.20.193.28:0/2675872218 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f73500a9690 con 0x559f3454e160 2026-02-25T11:51:59.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f736388ce00 10 client.26074 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:59.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f736388ce00 1 -- 10.20.193.28:0/2675872218 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x559f343d5b40 con 0x559f3454e160 2026-02-25T11:51:59.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f731d7fa640 1 -- 10.20.193.28:0/2675872218 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f73500aea30 con 0x559f3454e160 2026-02-25T11:51:59.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f736388ce00 10 client.26074 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:59.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f736388ce00 10 client.26074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:59.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f736388ce00 20 client.26074 populate_metadata read hostname 'trial028' 2026-02-25T11:51:59.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f736388ce00 1 --2- 10.20.193.28:0/2675872218 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559f34640e50 0x559f34661270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f736388ce00 4 client.26074 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:51:59.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f736388ce00 1 -- 10.20.193.28:0/2675872218 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f342b2690 con 0x559f34640e50 2026-02-25T11:51:59.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.258+0000 7f735a7fc640 1 --2- 10.20.193.28:0/2675872218 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559f34640e50 0x559f34661270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.259+0000 7f735a7fc640 1 --2- 10.20.193.28:0/2675872218 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559f34640e50 0x559f34661270 crc :-1 s=READY pgs=87 gs=4 cs=0 l=0 c_cookie=15777031429934337738 s_cookie=4661049099503272297 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daf3467ec768ceca server_cookie=40af5f75f08c0169 in_seq=0 out_seq=0 2026-02-25T11:51:59.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.259+0000 7f731d7fa640 10 client.26074 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:59.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f731d7fa640 1 -- 10.20.193.28:0/2675872218 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f342b2690 con 0x559f34640e50 2026-02-25T11:51:59.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f731d7fa640 10 client.26074 handle_command_reply: tid=0 2026-02-25T11:51:59.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f736388ce00 1 client.26074 shutdown 2026-02-25T11:51:59.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f736388ce00 1 -- 10.20.193.28:0/2675872218 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 msgr2=0x559f34523cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f736388ce00 1 --2- 10.20.193.28:0/2675872218 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 0x559f34523cc0 secure :-1 s=READY pgs=164 gs=1 cs=0 l=1 c_cookie=3758713785046758154 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f73500e6e40 tx=0x7f73500e4b30 comp rx=0 tx=0).stop 2026-02-25T11:51:59.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f736388ce00 1 -- 10.20.193.28:0/2675872218 shutdown_connections 2026-02-25T11:51:59.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f736388ce00 1 --2- 10.20.193.28:0/2675872218 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559f34640e50 0x559f34661270 crc :-1 s=READY pgs=87 gs=4 cs=0 l=0 c_cookie=15777031429934337738 s_cookie=4661049099503272297 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f736388ce00 1 --2- 10.20.193.28:0/2675872218 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559f34593560 0x559f34524740 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f736388ce00 1 --2- 10.20.193.28:0/2675872218 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559f3454ea70 0x559f34524200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f736388ce00 1 --2- 10.20.193.28:0/2675872218 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559f3454e160 0x559f34523cc0 unknown :-1 s=CLOSED pgs=164 gs=1 cs=0 l=1 c_cookie=3758713785046758154 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f736388ce00 1 -- 10.20.193.28:0/2675872218 >> 10.20.193.28:0/2675872218 conn(0x559f345238f0 msgr2=0x559f345ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:59.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f731d7fa640 0 client.26074 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:59.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f736388ce00 1 -- 10.20.193.28:0/2675872218 shutdown_connections 2026-02-25T11:51:59.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f736388ce00 1 -- 10.20.193.28:0/2675872218 wait complete. 2026-02-25T11:51:59.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f736388ce00 20 client.26074 trim_cache size 0 max 16384 2026-02-25T11:51:59.267 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:51:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:51:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "23922697-db31-470c-b97e-bd685d22642e", 2026-02-25T11:51:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:51:59.267 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:51:59.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f735bfff640 1 -- 10.20.193.28:0/1779038216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73300b57d0 msgr2=0x7f73300d5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f735bfff640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73300b57d0 0x7f73300d5c80 secure :-1 s=READY pgs=308 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f734c0050f0 tx=0x7f734c005050 comp rx=0 tx=0).stop 2026-02-25T11:51:59.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f735bfff640 1 -- 10.20.193.28:0/1779038216 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f735c10f690 msgr2=0x7f735c0f59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.260+0000 7f735bfff640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f735c10f690 0x7f735c0f59d0 secure :-1 s=READY pgs=162 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7350009480 tx=0x7f735000ee60 comp rx=0 tx=0).stop 2026-02-25T11:51:59.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.261+0000 7f735bfff640 1 -- 10.20.193.28:0/1779038216 shutdown_connections 2026-02-25T11:51:59.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.261+0000 7f735bfff640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f73300b57d0 0x7f73300d5c80 unknown :-1 s=CLOSED pgs=308 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.261+0000 7f735bfff640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f735c115880 0x7f735c0f60d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.261+0000 7f735bfff640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f735c110080 0x7f735c176150 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.261+0000 7f735bfff640 1 --2- 10.20.193.28:0/1779038216 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f735c10f690 0x7f735c0f59d0 unknown :-1 s=CLOSED pgs=162 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.261+0000 7f735bfff640 1 -- 10.20.193.28:0/1779038216 >> 10.20.193.28:0/1779038216 conn(0x7f735c0ecf00 msgr2=0x7f735c111e90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:59.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.261+0000 7f735bfff640 1 -- 10.20.193.28:0/1779038216 shutdown_connections 2026-02-25T11:51:59.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.261+0000 7f735bfff640 1 -- 10.20.193.28:0/1779038216 wait complete. 2026-02-25T11:51:59.397 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:51:59.582 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:51:59.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.765+0000 7f534e31c640 1 -- 10.20.193.28:0/2872317563 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f53481135c0 msgr2=0x7f5348173f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.765+0000 7f534e31c640 1 --2- 10.20.193.28:0/2872317563 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f53481135c0 0x7f5348173f90 secure :-1 s=READY pgs=576 gs=2 cs=0 l=1 c_cookie=9572577929506312324 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f534400aa30 tx=0x7f5344030270 comp rx=0 tx=0).stop 2026-02-25T11:51:59.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.765+0000 7f534e31c640 1 -- 10.20.193.28:0/2872317563 shutdown_connections 2026-02-25T11:51:59.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.765+0000 7f534e31c640 1 --2- 10.20.193.28:0/2872317563 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f53481135c0 0x7f5348173f90 unknown :-1 s=CLOSED pgs=576 gs=2 cs=0 l=1 c_cookie=9572577929506312324 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.765+0000 7f534e31c640 1 --2- 10.20.193.28:0/2872317563 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5348060b00 0x7f5348112e10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.765+0000 7f534e31c640 1 --2- 10.20.193.28:0/2872317563 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5348060110 0x7f53480604e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.765+0000 7f534e31c640 1 -- 10.20.193.28:0/2872317563 >> 10.20.193.28:0/2872317563 conn(0x7f53480e9b40 msgr2=0x7f5348109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:59.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.766+0000 7f534e31c640 1 -- 10.20.193.28:0/2872317563 shutdown_connections 2026-02-25T11:51:59.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.766+0000 7f534e31c640 1 -- 10.20.193.28:0/2872317563 wait complete. 2026-02-25T11:51:59.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.766+0000 7f534e31c640 1 Processor -- start 2026-02-25T11:51:59.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.766+0000 7f534e31c640 1 -- start start 2026-02-25T11:51:59.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534e31c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5348060110 0x7f53480f4800 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534e31c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5348060b00 0x7f5348173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534e31c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f53481135c0 0x7f53480f4f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534e31c640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f53480fa110 con 0x7f5348060b00 2026-02-25T11:51:59.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534e31c640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f53480fa280 con 0x7f53481135c0 2026-02-25T11:51:59.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534e31c640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53480fa440 con 0x7f5348060110 2026-02-25T11:51:59.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534cb19640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5348060b00 0x7f5348173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534d31a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5348060110 0x7f53480f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534cb19640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5348060b00 0x7f5348173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59882/0 (socket says 10.20.193.28:59882) 2026-02-25T11:51:59.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534cb19640 1 -- 10.20.193.28:0/2273311457 learned_addr learned my addr 10.20.193.28:0/2273311457 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:59.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534db1b640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f53481135c0 0x7f53480f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534cb19640 1 -- 10.20.193.28:0/2273311457 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5348060110 msgr2=0x7f53480f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534cb19640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5348060110 0x7f53480f4800 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534cb19640 1 -- 10.20.193.28:0/2273311457 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f53481135c0 msgr2=0x7f53480f4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534cb19640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f53481135c0 0x7f53480f4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534cb19640 1 -- 10.20.193.28:0/2273311457 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534400a650 con 0x7f5348060b00 2026-02-25T11:51:59.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534db1b640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f53481135c0 0x7f53480f4f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:51:59.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.767+0000 7f534cb19640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5348060b00 0x7f5348173e90 secure :-1 s=READY pgs=577 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f533c009480 tx=0x7f533c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:59.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.768+0000 7f53367fc640 1 -- 10.20.193.28:0/2273311457 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f533c00cd40 con 0x7f5348060b00 2026-02-25T11:51:59.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.768+0000 7f534e31c640 1 -- 10.20.193.28:0/2273311457 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53480fa720 con 0x7f5348060b00 2026-02-25T11:51:59.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.768+0000 7f534e31c640 1 -- 10.20.193.28:0/2273311457 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53480fabf0 con 0x7f5348060b00 2026-02-25T11:51:59.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.768+0000 7f53367fc640 1 -- 10.20.193.28:0/2273311457 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f533c004d10 con 0x7f5348060b00 2026-02-25T11:51:59.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.768+0000 7f53367fc640 1 -- 10.20.193.28:0/2273311457 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f533c0106b0 con 0x7f5348060b00 2026-02-25T11:51:59.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.771+0000 7f53367fc640 1 -- 10.20.193.28:0/2273311457 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f533c01f060 con 0x7f5348060b00 2026-02-25T11:51:59.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.772+0000 7f53367fc640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f53240b5750 0x7f53240d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.772+0000 7f53367fc640 1 -- 10.20.193.28:0/2273311457 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f533c014070 con 0x7f5348060b00 2026-02-25T11:51:59.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.772+0000 7f534d31a640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f53240b5750 0x7f53240d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.773+0000 7f534d31a640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f53240b5750 0x7f53240d5c00 secure :-1 s=READY pgs=309 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5338004740 tx=0x7f5338005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:59.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.773+0000 7f53515a0e00 1 Processor -- start 2026-02-25T11:51:59.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.773+0000 7f53515a0e00 1 -- start start 2026-02-25T11:51:59.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.773+0000 7f53515a0e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bafbb50 0x556e9bafbf20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.773+0000 7f53515a0e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafc460 0x556e9bb40bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.773+0000 7f53515a0e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bb41100 0x556e9bb614e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.773+0000 7f53515a0e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x556e9ba099b0 con 0x556e9bafc460 2026-02-25T11:51:59.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.773+0000 7f53515a0e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x556e9ba056e0 con 0x556e9bb41100 2026-02-25T11:51:59.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.773+0000 7f53515a0e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x556e9b9ff840 con 0x556e9bafbb50 2026-02-25T11:51:59.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.774+0000 7f534cb19640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafc460 0x556e9bb40bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.774+0000 7f534d31a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bafbb50 0x556e9bafbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.774+0000 7f534d31a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bafbb50 0x556e9bafbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:60260/0 (socket says 10.20.193.28:60260) 2026-02-25T11:51:59.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.774+0000 7f534d31a640 1 -- 10.20.193.28:0/1116928304 learned_addr learned my addr 10.20.193.28:0/1116928304 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:59.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.774+0000 7f534db1b640 1 --2- 10.20.193.28:0/1116928304 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bb41100 0x556e9bb614e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.774+0000 7f534cb19640 1 -- 10.20.193.28:0/1116928304 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bafbb50 msgr2=0x556e9bafbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.774+0000 7f534cb19640 1 --2- 10.20.193.28:0/1116928304 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bafbb50 0x556e9bafbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.774+0000 7f534cb19640 1 -- 10.20.193.28:0/1116928304 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bb41100 msgr2=0x556e9bb614e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.774+0000 7f534cb19640 1 --2- 10.20.193.28:0/1116928304 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bb41100 0x556e9bb614e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.774+0000 7f534cb19640 1 -- 10.20.193.28:0/1116928304 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e9b9aad80 con 0x556e9bafc460 2026-02-25T11:51:59.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.774+0000 7f534db1b640 1 --2- 10.20.193.28:0/1116928304 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bb41100 0x556e9bb614e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:51:59.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.774+0000 7f534cb19640 1 --2- 10.20.193.28:0/1116928304 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafc460 0x556e9bb40bc0 secure :-1 s=READY pgs=578 gs=3 cs=0 l=1 c_cookie=11709830272102947975 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f533c009230 tx=0x7f533c0a8fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a281ac9754cfdc87 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:59.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.774+0000 7f5315ce5640 1 -- 10.20.193.28:0/1116928304 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f533c005140 con 0x556e9bafc460 2026-02-25T11:51:59.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.775+0000 7f5315ce5640 1 -- 10.20.193.28:0/1116928304 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f533c016ea0 con 0x556e9bafc460 2026-02-25T11:51:59.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.775+0000 7f5315ce5640 1 -- 10.20.193.28:0/1116928304 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f533c0acc10 con 0x556e9bafc460 2026-02-25T11:51:59.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.775+0000 7f53515a0e00 1 -- 10.20.193.28:0/1116928304 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafc460 msgr2=0x556e9bb40bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.775+0000 7f53515a0e00 1 --2- 10.20.193.28:0/1116928304 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafc460 0x556e9bb40bc0 secure :-1 s=READY pgs=578 gs=3 cs=0 l=1 c_cookie=11709830272102947975 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f533c009230 tx=0x7f533c0a8fe0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.775+0000 7f53515a0e00 1 -- 10.20.193.28:0/1116928304 shutdown_connections 2026-02-25T11:51:59.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.775+0000 7f53515a0e00 1 --2- 10.20.193.28:0/1116928304 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bb41100 0x556e9bb614e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.775+0000 7f53515a0e00 1 --2- 10.20.193.28:0/1116928304 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafc460 0x556e9bb40bc0 unknown :-1 s=CLOSED pgs=578 gs=3 cs=0 l=1 c_cookie=11709830272102947975 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.775+0000 7f53515a0e00 1 --2- 10.20.193.28:0/1116928304 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bafbb50 0x556e9bafbf20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.775+0000 7f53515a0e00 1 -- 10.20.193.28:0/1116928304 >> 10.20.193.28:0/1116928304 conn(0x556e9ba12d40 msgr2=0x556e9baf1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:59.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.775+0000 7f53515a0e00 1 -- 10.20.193.28:0/1116928304 shutdown_connections 2026-02-25T11:51:59.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.775+0000 7f53515a0e00 1 -- 10.20.193.28:0/1116928304 wait complete. 2026-02-25T11:51:59.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.776+0000 7f53515a0e00 1 -- start start 2026-02-25T11:51:59.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.776+0000 7f53515a0e00 1 Processor -- start 2026-02-25T11:51:59.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f53515a0e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 0x556e9bb58f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f53515a0e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafc460 0x556e9bb59440 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f53515a0e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 0x556e9bb5e430 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f53515a0e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x556e9bb5e970 con 0x556e9bafbb50 2026-02-25T11:51:59.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f53515a0e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x556e9bb5eae0 con 0x556e9bafc460 2026-02-25T11:51:59.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f53515a0e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x556e9bb5eca0 con 0x556e9bb41100 2026-02-25T11:51:59.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f534d31a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 0x556e9bb58f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f534d31a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 0x556e9bb58f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59910/0 (socket says 10.20.193.28:59910) 2026-02-25T11:51:59.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f534d31a640 1 -- 10.20.193.28:0/552046539 learned_addr learned my addr 10.20.193.28:0/552046539 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:59.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f534cb19640 1 --2- 10.20.193.28:0/552046539 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafc460 0x556e9bb59440 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f534db1b640 1 --2- 10.20.193.28:0/552046539 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 0x556e9bb5e430 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f534d31a640 1 -- 10.20.193.28:0/552046539 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 msgr2=0x556e9bb5e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f534d31a640 1 --2- 10.20.193.28:0/552046539 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 0x556e9bb5e430 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f534d31a640 1 -- 10.20.193.28:0/552046539 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafc460 msgr2=0x556e9bb59440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f534d31a640 1 --2- 10.20.193.28:0/552046539 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafc460 0x556e9bb59440 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.777+0000 7f534d31a640 1 -- 10.20.193.28:0/552046539 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53480fabf0 con 0x556e9bafbb50 2026-02-25T11:51:59.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f534d31a640 1 --2- 10.20.193.28:0/552046539 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 0x556e9bb58f00 secure :-1 s=READY pgs=579 gs=1 cs=0 l=1 c_cookie=4973976317804726746 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f53380041f0 tx=0x7f5338013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45071d1b97f4c9da server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:59.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f5302ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:59.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f5302ffd640 1 -- 10.20.193.28:0/552046539 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5338021040 con 0x556e9bafbb50 2026-02-25T11:51:59.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f53515a0e00 1 -- 10.20.193.28:0/552046539 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5348174800 con 0x556e9bafbb50 2026-02-25T11:51:59.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f53515a0e00 10 client.16320 _ll_register_callbacks cb 0x556e9b9809a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:59.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f53515a0e00 1 -- 10.20.193.28:0/552046539 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556e9b6956f0 con 0x556e9bafbb50 2026-02-25T11:51:59.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f5302ffd640 1 -- 10.20.193.28:0/552046539 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f533801bb90 con 0x556e9bafbb50 2026-02-25T11:51:59.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f5302ffd640 1 -- 10.20.193.28:0/552046539 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5338025710 con 0x556e9bafbb50 2026-02-25T11:51:59.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f5302ffd640 1 -- 10.20.193.28:0/552046539 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f53380258a0 con 0x556e9bafbb50 2026-02-25T11:51:59.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f5302ffd640 1 -- 10.20.193.28:0/552046539 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5338025ac0 con 0x556e9bafbb50 2026-02-25T11:51:59.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f53515a0e00 10 client.16320 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:59.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f53515a0e00 1 -- 10.20.193.28:0/552046539 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x556e9b983350 con 0x556e9bafbb50 2026-02-25T11:51:59.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f5302ffd640 1 -- 10.20.193.28:0/552046539 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f533801a460 con 0x556e9bafbb50 2026-02-25T11:51:59.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f53515a0e00 10 client.16320 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:59.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f53515a0e00 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:59.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f53515a0e00 20 client.16320 populate_metadata read hostname 'trial028' 2026-02-25T11:51:59.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f53515a0e00 1 --2- 10.20.193.28:0/552046539 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e9bb61540 0x556e9bb3ee30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f53515a0e00 4 client.16320 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:51:59.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.778+0000 7f53515a0e00 1 -- 10.20.193.28:0/552046539 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e9b9d1910 con 0x556e9bb61540 2026-02-25T11:51:59.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.779+0000 7f534cb19640 1 --2- 10.20.193.28:0/552046539 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e9bb61540 0x556e9bb3ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.779+0000 7f534cb19640 1 --2- 10.20.193.28:0/552046539 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e9bb61540 0x556e9bb3ee30 crc :-1 s=READY pgs=88 gs=4 cs=0 l=0 c_cookie=396854087240375250 s_cookie=16724416222911476266 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=581e8b0919fdbd2 server_cookie=e8190ff1ea6b322a in_seq=0 out_seq=0 2026-02-25T11:51:59.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.779+0000 7f5302ffd640 10 client.16320 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:59.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f5302ffd640 1 -- 10.20.193.28:0/552046539 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x556e9b9d1910 con 0x556e9bb61540 2026-02-25T11:51:59.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f5302ffd640 10 client.16320 handle_command_reply: tid=0 2026-02-25T11:51:59.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f53515a0e00 1 client.16320 shutdown 2026-02-25T11:51:59.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f53515a0e00 1 -- 10.20.193.28:0/552046539 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 msgr2=0x556e9bb58f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f53515a0e00 1 --2- 10.20.193.28:0/552046539 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 0x556e9bb58f00 secure :-1 s=READY pgs=579 gs=1 cs=0 l=1 c_cookie=4973976317804726746 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f53380041f0 tx=0x7f5338013040 comp rx=0 tx=0).stop 2026-02-25T11:51:59.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f53515a0e00 1 -- 10.20.193.28:0/552046539 shutdown_connections 2026-02-25T11:51:59.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f53515a0e00 1 --2- 10.20.193.28:0/552046539 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e9bb61540 0x556e9bb3ee30 crc :-1 s=READY pgs=88 gs=4 cs=0 l=0 c_cookie=396854087240375250 s_cookie=16724416222911476266 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f53515a0e00 1 --2- 10.20.193.28:0/552046539 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 0x556e9bb5e430 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f53515a0e00 1 --2- 10.20.193.28:0/552046539 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafc460 0x556e9bb59440 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f53515a0e00 1 --2- 10.20.193.28:0/552046539 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 0x556e9bb58f00 unknown :-1 s=CLOSED pgs=579 gs=1 cs=0 l=1 c_cookie=4973976317804726746 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f5302ffd640 0 client.16320 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:59.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f53515a0e00 1 -- 10.20.193.28:0/552046539 >> 10.20.193.28:0/552046539 conn(0x556e9ba12d40 msgr2=0x556e9baf1230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:59.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f53515a0e00 1 -- 10.20.193.28:0/552046539 shutdown_connections 2026-02-25T11:51:59.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f53515a0e00 1 -- 10.20.193.28:0/552046539 wait complete. 2026-02-25T11:51:59.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.780+0000 7f53515a0e00 20 client.16320 trim_cache size 0 max 16384 2026-02-25T11:51:59.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f53515a0e00 1 Processor -- start 2026-02-25T11:51:59.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f53515a0e00 1 -- start start 2026-02-25T11:51:59.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f53515a0e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 0x556e9bb43bf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f53515a0e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafc460 0x556e9bb40500 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f53515a0e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 0x556e9bb40a40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f53515a0e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x556e9b9fd470 con 0x556e9bafbb50 2026-02-25T11:51:59.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f53515a0e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x556e9b6bd700 con 0x556e9bafc460 2026-02-25T11:51:59.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f53515a0e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x556e9bb60d80 con 0x556e9bb41100 2026-02-25T11:51:59.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f534d31a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 0x556e9bb43bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f534d31a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 0x556e9bb43bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59916/0 (socket says 10.20.193.28:59916) 2026-02-25T11:51:59.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f534d31a640 1 -- 10.20.193.28:0/884107767 learned_addr learned my addr 10.20.193.28:0/884107767 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:59.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f534cb19640 1 --2- 10.20.193.28:0/884107767 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafc460 0x556e9bb40500 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f534d31a640 1 -- 10.20.193.28:0/884107767 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 msgr2=0x556e9bb40a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f534d31a640 1 --2- 10.20.193.28:0/884107767 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 0x556e9bb40a40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f534d31a640 1 -- 10.20.193.28:0/884107767 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafc460 msgr2=0x556e9bb40500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f534d31a640 1 --2- 10.20.193.28:0/884107767 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafc460 0x556e9bb40500 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.790+0000 7f534d31a640 1 -- 10.20.193.28:0/884107767 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e9b995150 con 0x556e9bafbb50 2026-02-25T11:51:59.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.791+0000 7f534d31a640 1 --2- 10.20.193.28:0/884107767 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 0x556e9bb43bf0 secure :-1 s=READY pgs=580 gs=2 cs=0 l=1 c_cookie=17593345705371699777 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5338004220 tx=0x7f533802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4281eb49b530a41 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:59.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.791+0000 7f53154e4640 1 -- 10.20.193.28:0/884107767 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f533800eaa0 con 0x556e9bafbb50 2026-02-25T11:51:59.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.791+0000 7f53154e4640 1 -- 10.20.193.28:0/884107767 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x556e9b6956f0 con 0x556e9bafbb50 2026-02-25T11:51:59.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.791+0000 7f53154e4640 1 -- 10.20.193.28:0/884107767 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f533800e3e0 con 0x556e9bafbb50 2026-02-25T11:51:59.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.791+0000 7f53515a0e00 1 -- 10.20.193.28:0/884107767 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 msgr2=0x556e9bb43bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.791+0000 7f53515a0e00 1 --2- 10.20.193.28:0/884107767 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 0x556e9bb43bf0 secure :-1 s=READY pgs=580 gs=2 cs=0 l=1 c_cookie=17593345705371699777 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5338004220 tx=0x7f533802c000 comp rx=0 tx=0).stop 2026-02-25T11:51:59.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.791+0000 7f53515a0e00 1 -- 10.20.193.28:0/884107767 shutdown_connections 2026-02-25T11:51:59.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.791+0000 7f53515a0e00 1 --2- 10.20.193.28:0/884107767 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 0x556e9bb40a40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.791+0000 7f53515a0e00 1 --2- 10.20.193.28:0/884107767 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafc460 0x556e9bb40500 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.791+0000 7f53515a0e00 1 --2- 10.20.193.28:0/884107767 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafbb50 0x556e9bb43bf0 unknown :-1 s=CLOSED pgs=580 gs=2 cs=0 l=1 c_cookie=17593345705371699777 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.791+0000 7f53515a0e00 1 -- 10.20.193.28:0/884107767 >> 10.20.193.28:0/884107767 conn(0x556e9ba12d40 msgr2=0x556e9bb434e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:59.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.791+0000 7f53515a0e00 1 -- 10.20.193.28:0/884107767 shutdown_connections 2026-02-25T11:51:59.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.791+0000 7f53515a0e00 1 -- 10.20.193.28:0/884107767 wait complete. 2026-02-25T11:51:59.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.792+0000 7f53515a0e00 1 -- start start 2026-02-25T11:51:59.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.792+0000 7f53515a0e00 1 Processor -- start 2026-02-25T11:51:59.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.792+0000 7f53515a0e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafbb50 0x556e9bad64b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.792+0000 7f53515a0e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafc460 0x556e9bad69f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.792+0000 7f53515a0e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 0x556e9bad5260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.792+0000 7f53515a0e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x556e9bace490 con 0x556e9bafc460 2026-02-25T11:51:59.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.792+0000 7f53515a0e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x556e9bb50840 con 0x556e9bafbb50 2026-02-25T11:51:59.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.792+0000 7f53515a0e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x556e9bad6fb0 con 0x556e9bb41100 2026-02-25T11:51:59.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.792+0000 7f534cb19640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafc460 0x556e9bad69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.792+0000 7f534cb19640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafc460 0x556e9bad69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59924/0 (socket says 10.20.193.28:59924) 2026-02-25T11:51:59.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.792+0000 7f534cb19640 1 -- 10.20.193.28:0/4048204424 learned_addr learned my addr 10.20.193.28:0/4048204424 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:51:59.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f534d31a640 1 --2- 10.20.193.28:0/4048204424 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafbb50 0x556e9bad64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f534db1b640 1 --2- 10.20.193.28:0/4048204424 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 0x556e9bad5260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f534cb19640 1 -- 10.20.193.28:0/4048204424 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 msgr2=0x556e9bad5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f534cb19640 1 --2- 10.20.193.28:0/4048204424 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 0x556e9bad5260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f534cb19640 1 -- 10.20.193.28:0/4048204424 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafbb50 msgr2=0x556e9bad64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:51:59.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f534cb19640 1 --2- 10.20.193.28:0/4048204424 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafbb50 0x556e9bad64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f534cb19640 1 -- 10.20.193.28:0/4048204424 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e9b98fbd0 con 0x556e9bafc460 2026-02-25T11:51:59.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f534cb19640 1 --2- 10.20.193.28:0/4048204424 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafc460 0x556e9bad69f0 secure :-1 s=READY pgs=581 gs=1 cs=0 l=1 c_cookie=12116156103366891898 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f533c0ad050 tx=0x7f533c0acb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8253bc8285d597a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:51:59.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f5302ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:51:59.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f5302ffd640 1 -- 10.20.193.28:0/4048204424 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f533c0ada70 con 0x556e9bafc460 2026-02-25T11:51:59.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f5302ffd640 1 -- 10.20.193.28:0/4048204424 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f533c0b2d90 con 0x556e9bafc460 2026-02-25T11:51:59.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f5302ffd640 1 -- 10.20.193.28:0/4048204424 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f533c0adf80 con 0x556e9bafc460 2026-02-25T11:51:59.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f53515a0e00 1 -- 10.20.193.28:0/4048204424 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e9b983350 con 0x556e9bafc460 2026-02-25T11:51:59.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f5302ffd640 1 -- 10.20.193.28:0/4048204424 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f533c0b7d20 con 0x556e9bafc460 2026-02-25T11:51:59.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.793+0000 7f5302ffd640 10 client.16326 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:51:59.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.794+0000 7f53515a0e00 10 client.16326 _ll_register_callbacks cb 0x556e9b9809a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:51:59.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.794+0000 7f53515a0e00 1 -- 10.20.193.28:0/4048204424 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f533c016ea0 con 0x556e9bafc460 2026-02-25T11:51:59.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.794+0000 7f5302ffd640 1 -- 10.20.193.28:0/4048204424 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f533c0b5370 con 0x556e9bafc460 2026-02-25T11:51:59.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.794+0000 7f53515a0e00 10 client.16326 fetch_fsmap learned FSMap version 23 2026-02-25T11:51:59.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.794+0000 7f53515a0e00 1 -- 10.20.193.28:0/4048204424 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x556e9b983680 con 0x556e9bafc460 2026-02-25T11:51:59.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.794+0000 7f5302ffd640 1 -- 10.20.193.28:0/4048204424 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f533c072170 con 0x556e9bafc460 2026-02-25T11:51:59.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.794+0000 7f53515a0e00 10 client.16326 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:51:59.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.794+0000 7f53515a0e00 10 client.16326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:51:59.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.794+0000 7f53515a0e00 20 client.16326 populate_metadata read hostname 'trial028' 2026-02-25T11:51:59.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.794+0000 7f53515a0e00 1 --2- 10.20.193.28:0/4048204424 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e9bbef1e0 0x556e9bc0f660 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:51:59.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.794+0000 7f53515a0e00 4 client.16326 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:51:59.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.794+0000 7f53515a0e00 1 -- 10.20.193.28:0/4048204424 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e9b692f40 con 0x556e9bbef1e0 2026-02-25T11:51:59.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.794+0000 7f534d31a640 1 --2- 10.20.193.28:0/4048204424 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e9bbef1e0 0x556e9bc0f660 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:51:59.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.795+0000 7f534d31a640 1 --2- 10.20.193.28:0/4048204424 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e9bbef1e0 0x556e9bc0f660 crc :-1 s=READY pgs=89 gs=4 cs=0 l=0 c_cookie=11104035832655296944 s_cookie=13638355427682290336 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1975c212a0b9b0 server_cookie=bd452c009b860aa0 in_seq=0 out_seq=0 2026-02-25T11:51:59.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.795+0000 7f5302ffd640 10 client.16326 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:59.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f5302ffd640 1 -- 10.20.193.28:0/4048204424 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x556e9b692f40 con 0x556e9bbef1e0 2026-02-25T11:51:59.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f5302ffd640 10 client.16326 handle_command_reply: tid=0 2026-02-25T11:51:59.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f53515a0e00 1 client.16326 shutdown 2026-02-25T11:51:59.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f53515a0e00 1 -- 10.20.193.28:0/4048204424 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafc460 msgr2=0x556e9bad69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f53515a0e00 1 --2- 10.20.193.28:0/4048204424 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafc460 0x556e9bad69f0 secure :-1 s=READY pgs=581 gs=1 cs=0 l=1 c_cookie=12116156103366891898 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f533c0ad050 tx=0x7f533c0acb50 comp rx=0 tx=0).stop 2026-02-25T11:51:59.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f53515a0e00 1 -- 10.20.193.28:0/4048204424 shutdown_connections 2026-02-25T11:51:59.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f53515a0e00 1 --2- 10.20.193.28:0/4048204424 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e9bbef1e0 0x556e9bc0f660 crc :-1 s=READY pgs=89 gs=4 cs=0 l=0 c_cookie=11104035832655296944 s_cookie=13638355427682290336 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f53515a0e00 1 --2- 10.20.193.28:0/4048204424 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e9bb41100 0x556e9bad5260 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f53515a0e00 1 --2- 10.20.193.28:0/4048204424 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e9bafc460 0x556e9bad69f0 unknown :-1 s=CLOSED pgs=581 gs=1 cs=0 l=1 c_cookie=12116156103366891898 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f53515a0e00 1 --2- 10.20.193.28:0/4048204424 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e9bafbb50 0x556e9bad64b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f53515a0e00 1 -- 10.20.193.28:0/4048204424 >> 10.20.193.28:0/4048204424 conn(0x556e9ba12d40 msgr2=0x556e9bb3f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:59.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f5302ffd640 0 client.16326 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:51:59.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f53515a0e00 1 -- 10.20.193.28:0/4048204424 shutdown_connections 2026-02-25T11:51:59.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f53515a0e00 1 -- 10.20.193.28:0/4048204424 wait complete. 2026-02-25T11:51:59.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f53515a0e00 20 client.16326 trim_cache size 0 max 16384 2026-02-25T11:51:59.803 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:51:59.803 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:51:59.803 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:51:59.803 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:51:59.803 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:59.803 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "23922697-db31-470c-b97e-bd685d22642e", 2026-02-25T11:51:59.803 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:51:59.804 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:51:59.804 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:59.804 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:51:59.804 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:51:59.804 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:51:59.804 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:59.804 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:59.804 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:59.804 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:59.804 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:59.804 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:59.804 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:51:59.805 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:51:59.805 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:59.805 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:59.805 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:59.805 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:59.805 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:59.805 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:59.805 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:51:59.805 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:51:59.805 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:59.806 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:59.806 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:59.806 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:59.806 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:59.806 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:59.806 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:51:59.806 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:51:59.806 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:59.806 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:59.806 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:59.806 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:59.806 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:59.807 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:59.807 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:51:59.807 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:51:59.807 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:59.807 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:59.807 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:59.807 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:59.807 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:59.807 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:59.807 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:51:59.807 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:51:59.807 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:59.808 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:59.808 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:59.808 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:59.808 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:59.808 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:59.808 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:51:59.808 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:51:59.808 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:59.808 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:59.808 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:59.808 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:59.809 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:59.809 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:59.809 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:51:59.809 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:51:59.809 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:59.809 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:59.809 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:59.809 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:59.809 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:59.809 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:51:59.809 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:51:59.809 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:51:59.810 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:51:59.810 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:51:59.810 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:51:59.810 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:51:59.810 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:51:59.810 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:51:59.810 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:51:59.810 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:51:59.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f534e31c640 1 -- 10.20.193.28:0/2273311457 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f53240b5750 msgr2=0x7f53240d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f534e31c640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f53240b5750 0x7f53240d5c00 secure :-1 s=READY pgs=309 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5338004740 tx=0x7f5338005f90 comp rx=0 tx=0).stop 2026-02-25T11:51:59.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f534e31c640 1 -- 10.20.193.28:0/2273311457 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5348060b00 msgr2=0x7f5348173e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:51:59.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.796+0000 7f534e31c640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5348060b00 0x7f5348173e90 secure :-1 s=READY pgs=577 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f533c009480 tx=0x7f533c00ee60 comp rx=0 tx=0).stop 2026-02-25T11:51:59.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.797+0000 7f534e31c640 1 -- 10.20.193.28:0/2273311457 shutdown_connections 2026-02-25T11:51:59.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.797+0000 7f534e31c640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f53240b5750 0x7f53240d5c00 unknown :-1 s=CLOSED pgs=309 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.797+0000 7f534e31c640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f53481135c0 0x7f53480f4f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.797+0000 7f534e31c640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5348060b00 0x7f5348173e90 unknown :-1 s=CLOSED pgs=577 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.797+0000 7f534e31c640 1 --2- 10.20.193.28:0/2273311457 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5348060110 0x7f53480f4800 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:51:59.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.797+0000 7f534e31c640 1 -- 10.20.193.28:0/2273311457 >> 10.20.193.28:0/2273311457 conn(0x7f53480e9b40 msgr2=0x7f534810a700 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:51:59.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.797+0000 7f534e31c640 1 -- 10.20.193.28:0/2273311457 shutdown_connections 2026-02-25T11:51:59.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:51:59.797+0000 7f534e31c640 1 -- 10.20.193.28:0/2273311457 wait complete. 2026-02-25T11:51:59.884 INFO:tasks.cephfs.filesystem:scrub status for tag:23922697-db31-470c-b97e-bd685d22642e - {'path': '/', 'tag': '23922697-db31-470c-b97e-bd685d22642e', 'options': 'recursive,force'} 2026-02-25T11:52:00.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:00 trial167 ceph-mon[43392]: pgmap v339: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:52:00.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:00 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:52:00.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:00 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:52:00.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:00 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:52:00.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:00 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:52:00.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:00 trial119 ceph-mon[42710]: pgmap v339: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:52:00.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:00 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:52:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:00 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:52:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:00 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:52:00.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:00 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:52:00.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:00 trial028 ceph-mon[39224]: pgmap v339: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:52:00.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:00 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:52:00.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:00 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:52:00.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:00 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:52:00.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:00 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:52:02.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:02 trial167 ceph-mon[43392]: pgmap v340: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:02.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:02 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:52:02.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:02 trial119 ceph-mon[42710]: pgmap v340: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:02.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:02 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:52:02.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:02 trial028 ceph-mon[39224]: pgmap v340: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:02.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:02 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:52:04.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:04 trial167 ceph-mon[43392]: pgmap v341: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:52:04.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:04 trial119 ceph-mon[42710]: pgmap v341: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:52:04.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:04 trial028 ceph-mon[39224]: pgmap v341: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:52:06.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:06 trial167 ceph-mon[43392]: pgmap v342: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:52:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:06 trial119 ceph-mon[42710]: pgmap v342: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:52:06.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:06 trial028 ceph-mon[39224]: pgmap v342: 97 pgs: 97 active+clean; 3.2 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:52:08.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:08 trial119 ceph-mon[42710]: pgmap v343: 97 pgs: 97 active+clean; 3.3 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:08.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:08 trial028 ceph-mon[39224]: pgmap v343: 97 pgs: 97 active+clean; 3.3 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:08.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:08 trial167 ceph-mon[43392]: pgmap v343: 97 pgs: 97 active+clean; 3.3 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:10.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:10 trial119 ceph-mon[42710]: pgmap v344: 97 pgs: 97 active+clean; 3.3 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:52:10.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:10 trial028 ceph-mon[39224]: pgmap v344: 97 pgs: 97 active+clean; 3.3 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:52:10.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:10 trial167 ceph-mon[43392]: pgmap v344: 97 pgs: 97 active+clean; 3.3 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:52:12.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:12 trial119 ceph-mon[42710]: pgmap v345: 97 pgs: 97 active+clean; 3.3 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:52:12.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:12 trial028 ceph-mon[39224]: pgmap v345: 97 pgs: 97 active+clean; 3.3 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:52:12.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:12 trial167 ceph-mon[43392]: pgmap v345: 97 pgs: 97 active+clean; 3.3 MiB data, 338 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:52:14.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:14 trial028 ceph-mon[39224]: pgmap v346: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:52:14.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:14 trial167 ceph-mon[43392]: pgmap v346: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:52:14.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:14 trial119 ceph-mon[42710]: pgmap v346: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:52:15.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:15 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:52:15.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:15 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:52:15.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:15 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:52:16.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:16 trial028 ceph-mon[39224]: pgmap v347: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:16.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:52:16.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:52:16.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:52:16.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:52:16.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:16 trial167 ceph-mon[43392]: pgmap v347: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:16.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:52:16.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:52:16.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:52:16.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:52:16.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:16 trial119 ceph-mon[42710]: pgmap v347: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:16.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:52:16.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:52:16.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:52:16.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:52:18.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:18 trial028 ceph-mon[39224]: pgmap v348: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:52:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:18 trial167 ceph-mon[43392]: pgmap v348: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:52:18.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:18 trial119 ceph-mon[42710]: pgmap v348: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:52:20.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:20 trial028 ceph-mon[39224]: pgmap v349: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:52:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:20 trial167 ceph-mon[43392]: pgmap v349: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:52:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:20 trial119 ceph-mon[42710]: pgmap v349: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:52:22.552 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:22 trial167 ceph-mon[43392]: pgmap v350: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:22.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:22 trial028 ceph-mon[39224]: pgmap v350: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:22.562 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:22 trial119 ceph-mon[42710]: pgmap v350: 97 pgs: 97 active+clean; 3.4 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:24.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:24 trial028 ceph-mon[39224]: pgmap v351: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:24.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:24 trial167 ceph-mon[43392]: pgmap v351: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:24.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:24 trial119 ceph-mon[42710]: pgmap v351: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:26.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:26 trial028 ceph-mon[39224]: pgmap v352: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:26.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:26 trial167 ceph-mon[43392]: pgmap v352: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:26.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:26 trial119 ceph-mon[42710]: pgmap v352: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:28.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:28 trial028 ceph-mon[39224]: pgmap v353: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:52:28.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:28 trial167 ceph-mon[43392]: pgmap v353: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:52:28.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:28 trial119 ceph-mon[42710]: pgmap v353: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:52:29.885 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:52:30.073 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:52:30.255 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.253+0000 7fc493ac9640 1 -- 10.20.193.28:0/946538982 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c1115d0 msgr2=0x7fc48c111a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.253+0000 7fc493ac9640 1 --2- 10.20.193.28:0/946538982 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c1115d0 0x7fc48c111a70 secure :-1 s=READY pgs=582 gs=1 cs=0 l=1 c_cookie=14357233753451126574 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc48800aa60 tx=0x7fc4880302b0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.253+0000 7fc493ac9640 1 -- 10.20.193.28:0/946538982 shutdown_connections 2026-02-25T11:52:30.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.253+0000 7fc493ac9640 1 --2- 10.20.193.28:0/946538982 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c1115d0 0x7fc48c111a70 unknown :-1 s=CLOSED pgs=582 gs=1 cs=0 l=1 c_cookie=14357233753451126574 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.253+0000 7fc493ac9640 1 --2- 10.20.193.28:0/946538982 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc48c10c9e0 0x7fc48c110fb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.253+0000 7fc493ac9640 1 --2- 10.20.193.28:0/946538982 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc48c10bff0 0x7fc48c10c3c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.253+0000 7fc493ac9640 1 -- 10.20.193.28:0/946538982 >> 10.20.193.28:0/946538982 conn(0x7fc48c0e9c80 msgr2=0x7fc48c10a090 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:30.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.253+0000 7fc493ac9640 1 -- 10.20.193.28:0/946538982 shutdown_connections 2026-02-25T11:52:30.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.253+0000 7fc493ac9640 1 -- 10.20.193.28:0/946538982 wait complete. 2026-02-25T11:52:30.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.254+0000 7fc493ac9640 1 Processor -- start 2026-02-25T11:52:30.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.254+0000 7fc493ac9640 1 -- start start 2026-02-25T11:52:30.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.254+0000 7fc493ac9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c10bff0 0x7fc48c122a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.254+0000 7fc493ac9640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc48c10c9e0 0x7fc48c122f60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.254+0000 7fc493ac9640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc48c1115d0 0x7fc48c128330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.254+0000 7fc493ac9640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c128950 con 0x7fc48c10bff0 2026-02-25T11:52:30.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.254+0000 7fc493ac9640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c128ac0 con 0x7fc48c10c9e0 2026-02-25T11:52:30.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.254+0000 7fc493ac9640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c128c80 con 0x7fc48c1115d0 2026-02-25T11:52:30.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc49183e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c10bff0 0x7fc48c122a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc49183e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c10bff0 0x7fc48c122a20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58264/0 (socket says 10.20.193.28:58264) 2026-02-25T11:52:30.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc49183e640 1 -- 10.20.193.28:0/1377395086 learned_addr learned my addr 10.20.193.28:0/1377395086 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:30.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc49103d640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc48c10c9e0 0x7fc48c122f60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc49203f640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc48c1115d0 0x7fc48c128330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc49183e640 1 -- 10.20.193.28:0/1377395086 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc48c1115d0 msgr2=0x7fc48c128330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc49183e640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc48c1115d0 0x7fc48c128330 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc49183e640 1 -- 10.20.193.28:0/1377395086 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc48c10c9e0 msgr2=0x7fc48c122f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc49183e640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc48c10c9e0 0x7fc48c122f60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc49183e640 1 -- 10.20.193.28:0/1377395086 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48800a650 con 0x7fc48c10bff0 2026-02-25T11:52:30.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc49183e640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c10bff0 0x7fc48c122a20 secure :-1 s=READY pgs=583 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc47c009480 tx=0x7fc47c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:30.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc47affd640 1 -- 10.20.193.28:0/1377395086 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc47c00ccc0 con 0x7fc48c10bff0 2026-02-25T11:52:30.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc47affd640 1 -- 10.20.193.28:0/1377395086 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc47c00ce20 con 0x7fc48c10bff0 2026-02-25T11:52:30.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc47affd640 1 -- 10.20.193.28:0/1377395086 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc47c0106b0 con 0x7fc48c10bff0 2026-02-25T11:52:30.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc493ac9640 1 -- 10.20.193.28:0/1377395086 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48c128f60 con 0x7fc48c10bff0 2026-02-25T11:52:30.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.255+0000 7fc493ac9640 1 -- 10.20.193.28:0/1377395086 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48c169660 con 0x7fc48c10bff0 2026-02-25T11:52:30.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.258+0000 7fc47affd640 1 -- 10.20.193.28:0/1377395086 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fc47c01f060 con 0x7fc48c10bff0 2026-02-25T11:52:30.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.260+0000 7fc47affd640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4680b56e0 0x7fc4680d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.260+0000 7fc47affd640 1 -- 10.20.193.28:0/1377395086 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc47c014070 con 0x7fc48c10bff0 2026-02-25T11:52:30.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.260+0000 7fc49103d640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4680b56e0 0x7fc4680d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.261+0000 7fc49103d640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4680b56e0 0x7fc4680d5b90 secure :-1 s=READY pgs=310 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc48c1293c0 tx=0x7fc48000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:30.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.261+0000 7fc495ac4e00 1 Processor -- start 2026-02-25T11:52:30.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.261+0000 7fc495ac4e00 1 -- start start 2026-02-25T11:52:30.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.261+0000 7fc495ac4e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 0x558b2ed49510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.261+0000 7fc495ac4e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2ed8e1b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.261+0000 7fc495ac4e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 0x558b2edaead0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.261+0000 7fc495ac4e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558b2ec5f9b0 con 0x558b2ed49a50 2026-02-25T11:52:30.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.261+0000 7fc495ac4e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558b2ec5b6e0 con 0x558b2ed49140 2026-02-25T11:52:30.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.261+0000 7fc495ac4e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558b2ec55840 con 0x558b2ed8e6f0 2026-02-25T11:52:30.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.262+0000 7fc49183e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 0x558b2ed49510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.262+0000 7fc49183e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 0x558b2ed49510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43086/0 (socket says 10.20.193.28:43086) 2026-02-25T11:52:30.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.262+0000 7fc49183e640 1 -- 10.20.193.28:0/2366708659 learned_addr learned my addr 10.20.193.28:0/2366708659 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:30.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.262+0000 7fc49203f640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 0x558b2edaead0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.262+0000 7fc49103d640 1 --2- 10.20.193.28:0/2366708659 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2ed8e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.262+0000 7fc49203f640 1 -- 10.20.193.28:0/2366708659 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 msgr2=0x558b2ed49510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:30.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.262+0000 7fc49203f640 1 --2- 10.20.193.28:0/2366708659 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 0x558b2ed49510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.262+0000 7fc49203f640 1 -- 10.20.193.28:0/2366708659 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 msgr2=0x558b2ed8e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:30.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.262+0000 7fc49203f640 1 --2- 10.20.193.28:0/2366708659 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2ed8e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.262+0000 7fc49203f640 1 -- 10.20.193.28:0/2366708659 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b2ec00d80 con 0x558b2ed8e6f0 2026-02-25T11:52:30.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.262+0000 7fc49103d640 1 --2- 10.20.193.28:0/2366708659 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2ed8e1b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:52:30.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.262+0000 7fc49183e640 1 --2- 10.20.193.28:0/2366708659 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 0x558b2ed49510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:52:30.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.262+0000 7fc49203f640 1 --2- 10.20.193.28:0/2366708659 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 0x558b2edaead0 secure :-1 s=READY pgs=165 gs=3 cs=0 l=1 c_cookie=6545078318426289617 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc488033040 tx=0x7fc488006f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ad4ca0da4e8f5d1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:30.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.263+0000 7fc45a4e6640 1 -- 10.20.193.28:0/2366708659 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc48803e050 con 0x558b2ed8e6f0 2026-02-25T11:52:30.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.263+0000 7fc45a4e6640 1 -- 10.20.193.28:0/2366708659 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc48803f040 con 0x558b2ed8e6f0 2026-02-25T11:52:30.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.263+0000 7fc45a4e6640 1 -- 10.20.193.28:0/2366708659 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc4880050c0 con 0x558b2ed8e6f0 2026-02-25T11:52:30.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.263+0000 7fc495ac4e00 1 -- 10.20.193.28:0/2366708659 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 msgr2=0x558b2edaead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.263+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/2366708659 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 0x558b2edaead0 secure :-1 s=READY pgs=165 gs=3 cs=0 l=1 c_cookie=6545078318426289617 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc488033040 tx=0x7fc488006f80 comp rx=0 tx=0).stop 2026-02-25T11:52:30.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.263+0000 7fc495ac4e00 1 -- 10.20.193.28:0/2366708659 shutdown_connections 2026-02-25T11:52:30.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.263+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/2366708659 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 0x558b2edaead0 unknown :-1 s=CLOSED pgs=165 gs=3 cs=0 l=1 c_cookie=6545078318426289617 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.263+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/2366708659 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2ed8e1b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.263+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/2366708659 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 0x558b2ed49510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.263+0000 7fc495ac4e00 1 -- 10.20.193.28:0/2366708659 >> 10.20.193.28:0/2366708659 conn(0x558b2ec68d40 msgr2=0x558b2ed47a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:30.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.263+0000 7fc495ac4e00 1 -- 10.20.193.28:0/2366708659 shutdown_connections 2026-02-25T11:52:30.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.263+0000 7fc495ac4e00 1 -- 10.20.193.28:0/2366708659 wait complete. 2026-02-25T11:52:30.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.265+0000 7fc495ac4e00 1 -- start start 2026-02-25T11:52:30.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.265+0000 7fc495ac4e00 1 Processor -- start 2026-02-25T11:52:30.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.265+0000 7fc495ac4e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 0x558b2eda6350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.265+0000 7fc495ac4e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2eda6890 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.265+0000 7fc495ac4e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 0x558b2edab880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.265+0000 7fc495ac4e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558b2edabdc0 con 0x558b2ed49a50 2026-02-25T11:52:30.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.265+0000 7fc495ac4e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558b2edabf30 con 0x558b2ed49140 2026-02-25T11:52:30.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.265+0000 7fc495ac4e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558b2edac0f0 con 0x558b2ed8e6f0 2026-02-25T11:52:30.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.265+0000 7fc49183e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 0x558b2eda6350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.265+0000 7fc49183e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 0x558b2eda6350 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43100/0 (socket says 10.20.193.28:43100) 2026-02-25T11:52:30.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.265+0000 7fc49183e640 1 -- 10.20.193.28:0/987986111 learned_addr learned my addr 10.20.193.28:0/987986111 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:30.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.265+0000 7fc49103d640 1 --2- 10.20.193.28:0/987986111 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2eda6890 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc49183e640 1 -- 10.20.193.28:0/987986111 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 msgr2=0x558b2edab880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:30.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc49183e640 1 --2- 10.20.193.28:0/987986111 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 0x558b2edab880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc49183e640 1 -- 10.20.193.28:0/987986111 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 msgr2=0x558b2eda6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:30.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc49183e640 1 --2- 10.20.193.28:0/987986111 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2eda6890 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc49183e640 1 -- 10.20.193.28:0/987986111 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48c17b140 con 0x558b2ed49140 2026-02-25T11:52:30.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc49183e640 1 --2- 10.20.193.28:0/987986111 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 0x558b2eda6350 secure :-1 s=READY pgs=136 gs=1 cs=0 l=1 c_cookie=11519024517537146231 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc47c0ad290 tx=0x7fc47c0af1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fdbcbc4aed12577 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:30.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc4437fe640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:52:30.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc4437fe640 1 -- 10.20.193.28:0/987986111 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc47c0ac820 con 0x558b2ed49140 2026-02-25T11:52:30.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc4437fe640 1 -- 10.20.193.28:0/987986111 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc47c0ada20 con 0x558b2ed49140 2026-02-25T11:52:30.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc4437fe640 1 -- 10.20.193.28:0/987986111 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc47c0b7f50 con 0x558b2ed49140 2026-02-25T11:52:30.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc495ac4e00 1 -- 10.20.193.28:0/987986111 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48c169660 con 0x558b2ed49140 2026-02-25T11:52:30.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc4437fe640 1 -- 10.20.193.28:0/987986111 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc47c0adcb0 con 0x558b2ed49140 2026-02-25T11:52:30.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc495ac4e00 10 client.26114 _ll_register_callbacks cb 0x558b2ebd69a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:52:30.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.266+0000 7fc495ac4e00 1 -- 10.20.193.28:0/987986111 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558b2e8eb6f0 con 0x558b2ed49140 2026-02-25T11:52:30.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.267+0000 7fc4437fe640 1 -- 10.20.193.28:0/987986111 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc47c0ace60 con 0x558b2ed49140 2026-02-25T11:52:30.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.267+0000 7fc495ac4e00 10 client.26114 fetch_fsmap learned FSMap version 23 2026-02-25T11:52:30.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.267+0000 7fc495ac4e00 1 -- 10.20.193.28:0/987986111 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x558b2ebd9350 con 0x558b2ed49140 2026-02-25T11:52:30.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.267+0000 7fc4437fe640 1 -- 10.20.193.28:0/987986111 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fc47c0ab000 con 0x558b2ed49140 2026-02-25T11:52:30.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.267+0000 7fc495ac4e00 10 client.26114 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:52:30.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.267+0000 7fc495ac4e00 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:52:30.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.267+0000 7fc495ac4e00 20 client.26114 populate_metadata read hostname 'trial028' 2026-02-25T11:52:30.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.267+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/987986111 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558b2edae920 0x558b2ed8c470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.267+0000 7fc495ac4e00 4 client.26114 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:52:30.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.267+0000 7fc495ac4e00 1 -- 10.20.193.28:0/987986111 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b2ec27910 con 0x558b2edae920 2026-02-25T11:52:30.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.267+0000 7fc49103d640 1 --2- 10.20.193.28:0/987986111 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558b2edae920 0x558b2ed8c470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.268+0000 7fc49103d640 1 --2- 10.20.193.28:0/987986111 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558b2edae920 0x558b2ed8c470 crc :-1 s=READY pgs=90 gs=4 cs=0 l=0 c_cookie=16150189737327294052 s_cookie=5750012305179045431 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e020ffffac8d7664 server_cookie=4fcc25ba08526e37 in_seq=0 out_seq=0 2026-02-25T11:52:30.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.268+0000 7fc4437fe640 10 client.26114 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:30.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.268+0000 7fc4437fe640 1 -- 10.20.193.28:0/987986111 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x558b2ec27910 con 0x558b2edae920 2026-02-25T11:52:30.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.268+0000 7fc4437fe640 10 client.26114 handle_command_reply: tid=0 2026-02-25T11:52:30.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.268+0000 7fc495ac4e00 1 client.26114 shutdown 2026-02-25T11:52:30.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.269+0000 7fc495ac4e00 1 -- 10.20.193.28:0/987986111 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 msgr2=0x558b2eda6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.269+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/987986111 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 0x558b2eda6350 secure :-1 s=READY pgs=136 gs=1 cs=0 l=1 c_cookie=11519024517537146231 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc47c0ad290 tx=0x7fc47c0af1e0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.269+0000 7fc495ac4e00 1 -- 10.20.193.28:0/987986111 shutdown_connections 2026-02-25T11:52:30.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.269+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/987986111 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558b2edae920 0x558b2ed8c470 crc :-1 s=READY pgs=90 gs=4 cs=0 l=0 c_cookie=16150189737327294052 s_cookie=5750012305179045431 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.269+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/987986111 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 0x558b2edab880 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.269+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/987986111 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2eda6890 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.269+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/987986111 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49140 0x558b2eda6350 unknown :-1 s=CLOSED pgs=136 gs=1 cs=0 l=1 c_cookie=11519024517537146231 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.269+0000 7fc495ac4e00 1 -- 10.20.193.28:0/987986111 >> 10.20.193.28:0/987986111 conn(0x558b2ec68d40 msgr2=0x558b2ed47230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:30.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.269+0000 7fc4437fe640 0 client.26114 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:30.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.269+0000 7fc495ac4e00 1 -- 10.20.193.28:0/987986111 shutdown_connections 2026-02-25T11:52:30.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.269+0000 7fc495ac4e00 1 -- 10.20.193.28:0/987986111 wait complete. 2026-02-25T11:52:30.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.269+0000 7fc495ac4e00 20 client.26114 trim_cache size 0 max 16384 2026-02-25T11:52:30.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.278+0000 7fc495ac4e00 1 Processor -- start 2026-02-25T11:52:30.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.278+0000 7fc495ac4e00 1 -- start start 2026-02-25T11:52:30.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc495ac4e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49140 0x558b2edaadf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc495ac4e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49a50 0x558b2eda3520 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc495ac4e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 0x558b2eda3a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc495ac4e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558b2ec53470 con 0x558b2ed49140 2026-02-25T11:52:30.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc495ac4e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558b2e913700 con 0x558b2ed49a50 2026-02-25T11:52:30.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc495ac4e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558b2ecbd6e0 con 0x558b2ed8e6f0 2026-02-25T11:52:30.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc49183e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49140 0x558b2edaadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc49183e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49140 0x558b2edaadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58286/0 (socket says 10.20.193.28:58286) 2026-02-25T11:52:30.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc49183e640 1 -- 10.20.193.28:0/592928489 learned_addr learned my addr 10.20.193.28:0/592928489 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:30.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc49203f640 1 --2- 10.20.193.28:0/592928489 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 0x558b2eda3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc49103d640 1 --2- 10.20.193.28:0/592928489 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49a50 0x558b2eda3520 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc49183e640 1 -- 10.20.193.28:0/592928489 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 msgr2=0x558b2eda3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:30.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc49183e640 1 --2- 10.20.193.28:0/592928489 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 0x558b2eda3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc49183e640 1 -- 10.20.193.28:0/592928489 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49a50 msgr2=0x558b2eda3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:30.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc49183e640 1 --2- 10.20.193.28:0/592928489 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49a50 0x558b2eda3520 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc49183e640 1 -- 10.20.193.28:0/592928489 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b2ed985c0 con 0x558b2ed49140 2026-02-25T11:52:30.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.279+0000 7fc49183e640 1 --2- 10.20.193.28:0/592928489 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49140 0x558b2edaadf0 secure :-1 s=READY pgs=584 gs=1 cs=0 l=1 c_cookie=12030123666068698734 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc47c0a8d10 tx=0x7fc47c0e6070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f395bc680b766e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:30.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.280+0000 7fc459ce5640 1 -- 10.20.193.28:0/592928489 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc47c0b8160 con 0x558b2ed49140 2026-02-25T11:52:30.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.280+0000 7fc459ce5640 1 -- 10.20.193.28:0/592928489 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x558b2e8eb6f0 con 0x558b2ed49140 2026-02-25T11:52:30.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.280+0000 7fc459ce5640 1 -- 10.20.193.28:0/592928489 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc47c0acdd0 con 0x558b2ed49140 2026-02-25T11:52:30.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.280+0000 7fc495ac4e00 1 -- 10.20.193.28:0/592928489 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49140 msgr2=0x558b2edaadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.280+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/592928489 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49140 0x558b2edaadf0 secure :-1 s=READY pgs=584 gs=1 cs=0 l=1 c_cookie=12030123666068698734 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc47c0a8d10 tx=0x7fc47c0e6070 comp rx=0 tx=0).stop 2026-02-25T11:52:30.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.280+0000 7fc495ac4e00 1 -- 10.20.193.28:0/592928489 shutdown_connections 2026-02-25T11:52:30.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.280+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/592928489 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed8e6f0 0x558b2eda3a60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.280+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/592928489 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed49a50 0x558b2eda3520 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.280+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/592928489 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49140 0x558b2edaadf0 unknown :-1 s=CLOSED pgs=584 gs=1 cs=0 l=1 c_cookie=12030123666068698734 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.280+0000 7fc495ac4e00 1 -- 10.20.193.28:0/592928489 >> 10.20.193.28:0/592928489 conn(0x558b2ec68d40 msgr2=0x558b2edaa6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:30.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.280+0000 7fc495ac4e00 1 -- 10.20.193.28:0/592928489 shutdown_connections 2026-02-25T11:52:30.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.280+0000 7fc495ac4e00 1 -- 10.20.193.28:0/592928489 wait complete. 2026-02-25T11:52:30.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.281+0000 7fc495ac4e00 1 -- start start 2026-02-25T11:52:30.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.281+0000 7fc495ac4e00 1 Processor -- start 2026-02-25T11:52:30.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.281+0000 7fc495ac4e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed49140 0x558b2ed27220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.281+0000 7fc495ac4e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2ed27760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.281+0000 7fc495ac4e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed8e6f0 0x558b2ed27ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.281+0000 7fc495ac4e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558b2edab360 con 0x558b2ed49a50 2026-02-25T11:52:30.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.281+0000 7fc495ac4e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558b2edab4d0 con 0x558b2ed8e6f0 2026-02-25T11:52:30.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.281+0000 7fc495ac4e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558b2ed2c270 con 0x558b2ed49140 2026-02-25T11:52:30.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.281+0000 7fc49103d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2ed27760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.281+0000 7fc49183e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed49140 0x558b2ed27220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.281+0000 7fc49103d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2ed27760 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58300/0 (socket says 10.20.193.28:58300) 2026-02-25T11:52:30.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.281+0000 7fc49103d640 1 -- 10.20.193.28:0/3083415615 learned_addr learned my addr 10.20.193.28:0/3083415615 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:30.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.281+0000 7fc49203f640 1 --2- 10.20.193.28:0/3083415615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed8e6f0 0x558b2ed27ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc49103d640 1 -- 10.20.193.28:0/3083415615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed49140 msgr2=0x558b2ed27220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:30.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc49103d640 1 --2- 10.20.193.28:0/3083415615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed49140 0x558b2ed27220 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc49103d640 1 -- 10.20.193.28:0/3083415615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed8e6f0 msgr2=0x558b2ed27ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:30.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc49103d640 1 --2- 10.20.193.28:0/3083415615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed8e6f0 0x558b2ed27ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc49103d640 1 -- 10.20.193.28:0/3083415615 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b2ebeb150 con 0x558b2ed49a50 2026-02-25T11:52:30.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc49103d640 1 --2- 10.20.193.28:0/3083415615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2ed27760 secure :-1 s=READY pgs=585 gs=2 cs=0 l=1 c_cookie=6079413648001608046 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x558b2ece37f0 tx=0x7fc4800047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545e6a80b3cc296e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:30.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc4437fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:52:30.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc4437fe640 1 -- 10.20.193.28:0/3083415615 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc480024040 con 0x558b2ed49a50 2026-02-25T11:52:30.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc4437fe640 1 -- 10.20.193.28:0/3083415615 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc480015070 con 0x558b2ed49a50 2026-02-25T11:52:30.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc495ac4e00 1 -- 10.20.193.28:0/3083415615 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b2ebd9350 con 0x558b2ed49a50 2026-02-25T11:52:30.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc4437fe640 1 -- 10.20.193.28:0/3083415615 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc48001fb20 con 0x558b2ed49a50 2026-02-25T11:52:30.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc495ac4e00 10 client.16344 _ll_register_callbacks cb 0x558b2ebd69a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:52:30.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc495ac4e00 1 -- 10.20.193.28:0/3083415615 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc48803f040 con 0x558b2ed49a50 2026-02-25T11:52:30.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc4437fe640 1 -- 10.20.193.28:0/3083415615 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc480017070 con 0x558b2ed49a50 2026-02-25T11:52:30.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc4437fe640 1 -- 10.20.193.28:0/3083415615 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc48001fcb0 con 0x558b2ed49a50 2026-02-25T11:52:30.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc495ac4e00 10 client.16344 fetch_fsmap learned FSMap version 23 2026-02-25T11:52:30.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.282+0000 7fc495ac4e00 1 -- 10.20.193.28:0/3083415615 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x558b2ebe5bd0 con 0x558b2ed49a50 2026-02-25T11:52:30.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.283+0000 7fc4437fe640 1 -- 10.20.193.28:0/3083415615 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fc480032020 con 0x558b2ed49a50 2026-02-25T11:52:30.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.283+0000 7fc495ac4e00 10 client.16344 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:52:30.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.283+0000 7fc495ac4e00 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:52:30.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.283+0000 7fc495ac4e00 20 client.16344 populate_metadata read hostname 'trial028' 2026-02-25T11:52:30.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.283+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/3083415615 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558b2ee3c470 0x558b2ee5c8f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.283+0000 7fc495ac4e00 4 client.16344 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:52:30.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.283+0000 7fc495ac4e00 1 -- 10.20.193.28:0/3083415615 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b2e8e8f40 con 0x558b2ee3c470 2026-02-25T11:52:30.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.283+0000 7fc49183e640 1 --2- 10.20.193.28:0/3083415615 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558b2ee3c470 0x558b2ee5c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.283+0000 7fc49183e640 1 --2- 10.20.193.28:0/3083415615 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558b2ee3c470 0x558b2ee5c8f0 crc :-1 s=READY pgs=91 gs=4 cs=0 l=0 c_cookie=1217138398026269911 s_cookie=5198279935498715630 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e424eca0d804d7 server_cookie=4824000f467101ee in_seq=0 out_seq=0 2026-02-25T11:52:30.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.283+0000 7fc4437fe640 10 client.16344 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:30.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc4437fe640 1 -- 10.20.193.28:0/3083415615 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x558b2e8e8f40 con 0x558b2ee3c470 2026-02-25T11:52:30.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc4437fe640 10 client.16344 handle_command_reply: tid=0 2026-02-25T11:52:30.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc495ac4e00 1 client.16344 shutdown 2026-02-25T11:52:30.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc495ac4e00 1 -- 10.20.193.28:0/3083415615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 msgr2=0x558b2ed27760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/3083415615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2ed27760 secure :-1 s=READY pgs=585 gs=2 cs=0 l=1 c_cookie=6079413648001608046 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x558b2ece37f0 tx=0x7fc4800047e0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc495ac4e00 1 -- 10.20.193.28:0/3083415615 shutdown_connections 2026-02-25T11:52:30.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/3083415615 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558b2ee3c470 0x558b2ee5c8f0 crc :-1 s=READY pgs=91 gs=4 cs=0 l=0 c_cookie=1217138398026269911 s_cookie=5198279935498715630 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/3083415615 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558b2ed8e6f0 0x558b2ed27ca0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/3083415615 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558b2ed49a50 0x558b2ed27760 unknown :-1 s=CLOSED pgs=585 gs=2 cs=0 l=1 c_cookie=6079413648001608046 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc495ac4e00 1 --2- 10.20.193.28:0/3083415615 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558b2ed49140 0x558b2ed27220 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc495ac4e00 1 -- 10.20.193.28:0/3083415615 >> 10.20.193.28:0/3083415615 conn(0x558b2ec68d40 msgr2=0x558b2ed8d370 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:30.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc4437fe640 0 client.16344 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:30.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc495ac4e00 1 -- 10.20.193.28:0/3083415615 shutdown_connections 2026-02-25T11:52:30.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc495ac4e00 1 -- 10.20.193.28:0/3083415615 wait complete. 2026-02-25T11:52:30.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc495ac4e00 20 client.16344 trim_cache size 0 max 16384 2026-02-25T11:52:30.294 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:52:30.294 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:52:30.294 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:52:30.294 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:52:30.295 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:52:30.295 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:52:30.295 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:30.295 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:30.295 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:30.295 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:30.295 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:30.295 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:30.295 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:52:30.295 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:52:30.295 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:30.296 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:30.296 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:30.296 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:30.296 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:30.296 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:30.296 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:52:30.296 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:52:30.296 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:30.296 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:30.296 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:30.296 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:30.296 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:30.297 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:30.297 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:52:30.297 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:52:30.297 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:30.297 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:30.297 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:30.297 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:30.297 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:30.297 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:30.297 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:52:30.297 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:52:30.297 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:30.298 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:30.298 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:30.298 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:30.298 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:30.298 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:30.298 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:52:30.298 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:52:30.298 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:30.298 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:30.298 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:30.298 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:30.298 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:30.298 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:30.299 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:52:30.299 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:52:30.299 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:30.299 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:30.299 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:30.299 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:30.299 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:30.299 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:30.299 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:52:30.299 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:52:30.299 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:30.299 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:30.299 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:30.300 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:30.300 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:30.300 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:30.300 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:52:30.300 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:52:30.300 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:30.300 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:30.300 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:30.300 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:30.300 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:30.300 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:52:30.300 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:52:30.301 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:52:30.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc493ac9640 1 -- 10.20.193.28:0/1377395086 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4680b56e0 msgr2=0x7fc4680d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc493ac9640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4680b56e0 0x7fc4680d5b90 secure :-1 s=READY pgs=310 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc48c1293c0 tx=0x7fc48000b410 comp rx=0 tx=0).stop 2026-02-25T11:52:30.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc493ac9640 1 -- 10.20.193.28:0/1377395086 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c10bff0 msgr2=0x7fc48c122a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.284+0000 7fc493ac9640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c10bff0 0x7fc48c122a20 secure :-1 s=READY pgs=583 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc47c009480 tx=0x7fc47c00ee60 comp rx=0 tx=0).stop 2026-02-25T11:52:30.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.285+0000 7fc493ac9640 1 -- 10.20.193.28:0/1377395086 shutdown_connections 2026-02-25T11:52:30.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.285+0000 7fc493ac9640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc4680b56e0 0x7fc4680d5b90 unknown :-1 s=CLOSED pgs=310 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.285+0000 7fc493ac9640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc48c1115d0 0x7fc48c128330 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.285+0000 7fc493ac9640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc48c10c9e0 0x7fc48c122f60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.285+0000 7fc493ac9640 1 --2- 10.20.193.28:0/1377395086 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc48c10bff0 0x7fc48c122a20 unknown :-1 s=CLOSED pgs=583 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.285+0000 7fc493ac9640 1 -- 10.20.193.28:0/1377395086 >> 10.20.193.28:0/1377395086 conn(0x7fc48c0e9c80 msgr2=0x7fc48c10a060 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:30.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.285+0000 7fc493ac9640 1 -- 10.20.193.28:0/1377395086 shutdown_connections 2026-02-25T11:52:30.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.285+0000 7fc493ac9640 1 -- 10.20.193.28:0/1377395086 wait complete. 2026-02-25T11:52:30.386 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:52:30.386 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:52:30.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:30 trial028 ceph-mon[39224]: pgmap v354: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:52:30.570 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:52:30.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:30 trial167 ceph-mon[43392]: pgmap v354: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:52:30.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:30 trial119 ceph-mon[42710]: pgmap v354: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:52:30.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.758+0000 7f4759f3e640 1 -- 10.20.193.28:0/3245481927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f475410f840 msgr2=0x7f475410fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.758+0000 7f4759f3e640 1 --2- 10.20.193.28:0/3245481927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f475410f840 0x7f475410fc10 secure :-1 s=READY pgs=586 gs=1 cs=0 l=1 c_cookie=5674235316424788273 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4740009950 tx=0x7f474002f190 comp rx=0 tx=0).stop 2026-02-25T11:52:30.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.758+0000 7f4759f3e640 1 -- 10.20.193.28:0/3245481927 shutdown_connections 2026-02-25T11:52:30.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.758+0000 7f4759f3e640 1 --2- 10.20.193.28:0/3245481927 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4754115a30 0x7f4754176400 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.758+0000 7f4759f3e640 1 --2- 10.20.193.28:0/3245481927 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4754110230 0x7f4754115280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.758+0000 7f4759f3e640 1 --2- 10.20.193.28:0/3245481927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f475410f840 0x7f475410fc10 unknown :-1 s=CLOSED pgs=586 gs=1 cs=0 l=1 c_cookie=5674235316424788273 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.758+0000 7f4759f3e640 1 -- 10.20.193.28:0/3245481927 >> 10.20.193.28:0/3245481927 conn(0x7f47540ed130 msgr2=0x7f475410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:30.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.759+0000 7f4759f3e640 1 -- 10.20.193.28:0/3245481927 shutdown_connections 2026-02-25T11:52:30.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.759+0000 7f4759f3e640 1 -- 10.20.193.28:0/3245481927 wait complete. 2026-02-25T11:52:30.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.760+0000 7f4759f3e640 1 Processor -- start 2026-02-25T11:52:30.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.760+0000 7f4759f3e640 1 -- start start 2026-02-25T11:52:30.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.760+0000 7f4759f3e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f475410f840 0x7f47540f5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.760+0000 7f4759f3e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4754110230 0x7f4754176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.760+0000 7f4759f3e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4754115a30 0x7f47540f6240 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.760+0000 7f4759f3e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f47540fb450 con 0x7f4754110230 2026-02-25T11:52:30.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.760+0000 7f4759f3e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f47540fb5c0 con 0x7f475410f840 2026-02-25T11:52:30.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.760+0000 7f4759f3e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f47540fb780 con 0x7f4754115a30 2026-02-25T11:52:30.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.760+0000 7f4752ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4754110230 0x7f4754176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.760+0000 7f4752ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4754110230 0x7f4754176300 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58318/0 (socket says 10.20.193.28:58318) 2026-02-25T11:52:30.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.760+0000 7f4752ffd640 1 -- 10.20.193.28:0/1230377324 learned_addr learned my addr 10.20.193.28:0/1230377324 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:30.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.761+0000 7f47537fe640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f475410f840 0x7f47540f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.764 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.761+0000 7f4753fff640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4754115a30 0x7f47540f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.761+0000 7f4752ffd640 1 -- 10.20.193.28:0/1230377324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4754115a30 msgr2=0x7f47540f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.761+0000 7f4752ffd640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4754115a30 0x7f47540f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.761+0000 7f4752ffd640 1 -- 10.20.193.28:0/1230377324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f475410f840 msgr2=0x7f47540f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.761+0000 7f4752ffd640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f475410f840 0x7f47540f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.761+0000 7f4752ffd640 1 -- 10.20.193.28:0/1230377324 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4740009570 con 0x7f4754110230 2026-02-25T11:52:30.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.761+0000 7f4752ffd640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4754110230 0x7f4754176300 secure :-1 s=READY pgs=587 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4744005db0 tx=0x7f474400ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:30.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.761+0000 7f4750ff9640 1 -- 10.20.193.28:0/1230377324 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4744004500 con 0x7f4754110230 2026-02-25T11:52:30.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.761+0000 7f4750ff9640 1 -- 10.20.193.28:0/1230377324 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4744004d10 con 0x7f4754110230 2026-02-25T11:52:30.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.761+0000 7f4750ff9640 1 -- 10.20.193.28:0/1230377324 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4744005280 con 0x7f4754110230 2026-02-25T11:52:30.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.761+0000 7f4759f3e640 1 -- 10.20.193.28:0/1230377324 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47540fba60 con 0x7f4754110230 2026-02-25T11:52:30.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.761+0000 7f4759f3e640 1 -- 10.20.193.28:0/1230377324 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47540fbee0 con 0x7f4754110230 2026-02-25T11:52:30.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.762+0000 7f4759f3e640 1 -- 10.20.193.28:0/1230377324 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4754110b20 con 0x7f4754110230 2026-02-25T11:52:30.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.764+0000 7f4750ff9640 1 -- 10.20.193.28:0/1230377324 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f4744005410 con 0x7f4754110230 2026-02-25T11:52:30.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.766+0000 7f4750ff9640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f47280b5750 0x7f47280d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:30.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.766+0000 7f4750ff9640 1 -- 10.20.193.28:0/1230377324 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f47440e7070 con 0x7f4754110230 2026-02-25T11:52:30.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.766+0000 7f4750ff9640 1 -- 10.20.193.28:0/1230377324 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f47440e7560 con 0x7f4754110230 2026-02-25T11:52:30.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.766+0000 7f47537fe640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f47280b5750 0x7f47280d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:30.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.766+0000 7f47537fe640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f47280b5750 0x7f47280d5c00 secure :-1 s=READY pgs=311 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f47400023d0 tx=0x7f4740031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:30.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.914+0000 7f4759f3e640 1 -- 10.20.193.28:0/1230377324 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f475410fc10 con 0x7f4754110230 2026-02-25T11:52:30.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.915+0000 7f4750ff9640 1 -- 10.20.193.28:0/1230377324 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f47440a9a80 con 0x7f4754110230 2026-02-25T11:52:30.917 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:52:30.917 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:52:30.918 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:52:30.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.916+0000 7f4759f3e640 1 -- 10.20.193.28:0/1230377324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f47280b5750 msgr2=0x7f47280d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.916+0000 7f4759f3e640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f47280b5750 0x7f47280d5c00 secure :-1 s=READY pgs=311 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f47400023d0 tx=0x7f4740031040 comp rx=0 tx=0).stop 2026-02-25T11:52:30.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.916+0000 7f4759f3e640 1 -- 10.20.193.28:0/1230377324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4754110230 msgr2=0x7f4754176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:30.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.916+0000 7f4759f3e640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4754110230 0x7f4754176300 secure :-1 s=READY pgs=587 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4744005db0 tx=0x7f474400ddf0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.916+0000 7f4759f3e640 1 -- 10.20.193.28:0/1230377324 shutdown_connections 2026-02-25T11:52:30.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.916+0000 7f4759f3e640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f47280b5750 0x7f47280d5c00 unknown :-1 s=CLOSED pgs=311 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.916+0000 7f4759f3e640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4754115a30 0x7f47540f6240 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.916+0000 7f4759f3e640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4754110230 0x7f4754176300 unknown :-1 s=CLOSED pgs=587 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.916+0000 7f4759f3e640 1 --2- 10.20.193.28:0/1230377324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f475410f840 0x7f47540f5b40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:30.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.916+0000 7f4759f3e640 1 -- 10.20.193.28:0/1230377324 >> 10.20.193.28:0/1230377324 conn(0x7f47540ed130 msgr2=0x7f4754111c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:30.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.916+0000 7f4759f3e640 1 -- 10.20.193.28:0/1230377324 shutdown_connections 2026-02-25T11:52:30.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:30.916+0000 7f4759f3e640 1 -- 10.20.193.28:0/1230377324 wait complete. 2026-02-25T11:52:31.020 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:52:31.205 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:52:31.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.384+0000 7f5127fff640 1 -- 10.20.193.28:0/1353567957 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f51281135c0 msgr2=0x7f5128173f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.384+0000 7f5127fff640 1 --2- 10.20.193.28:0/1353567957 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f51281135c0 0x7f5128173f90 secure :-1 s=READY pgs=588 gs=2 cs=0 l=1 c_cookie=4485430702219543453 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f511800aa30 tx=0x7f5118030270 comp rx=0 tx=0).stop 2026-02-25T11:52:31.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.384+0000 7f5127fff640 1 -- 10.20.193.28:0/1353567957 shutdown_connections 2026-02-25T11:52:31.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.384+0000 7f5127fff640 1 --2- 10.20.193.28:0/1353567957 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f51281135c0 0x7f5128173f90 unknown :-1 s=CLOSED pgs=588 gs=2 cs=0 l=1 c_cookie=4485430702219543453 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.384+0000 7f5127fff640 1 --2- 10.20.193.28:0/1353567957 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f512810ddc0 0x7f5128112e10 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.384+0000 7f5127fff640 1 --2- 10.20.193.28:0/1353567957 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f512810d3d0 0x7f512810d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.384+0000 7f5127fff640 1 -- 10.20.193.28:0/1353567957 >> 10.20.193.28:0/1353567957 conn(0x7f51280eaca0 msgr2=0x7f512810b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:31.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.384+0000 7f5127fff640 1 -- 10.20.193.28:0/1353567957 shutdown_connections 2026-02-25T11:52:31.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.384+0000 7f5127fff640 1 -- 10.20.193.28:0/1353567957 wait complete. 2026-02-25T11:52:31.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.385+0000 7f5127fff640 1 Processor -- start 2026-02-25T11:52:31.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.385+0000 7f5127fff640 1 -- start start 2026-02-25T11:52:31.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.385+0000 7f5127fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f512810d3d0 0x7f51280f5920 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.385+0000 7f5127fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f512810ddc0 0x7f5128173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.385+0000 7f5127fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f51281135c0 0x7f51280f6020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.385+0000 7f5127fff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f51280fb230 con 0x7f512810ddc0 2026-02-25T11:52:31.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.385+0000 7f5127fff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f51280fb3a0 con 0x7f51281135c0 2026-02-25T11:52:31.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.385+0000 7f5127fff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f51280fb560 con 0x7f512810d3d0 2026-02-25T11:52:31.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.385+0000 7f51267fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f512810ddc0 0x7f5128173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.385+0000 7f5126ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f512810d3d0 0x7f51280f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.385+0000 7f51267fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f512810ddc0 0x7f5128173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58334/0 (socket says 10.20.193.28:58334) 2026-02-25T11:52:31.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.385+0000 7f51277fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f51281135c0 0x7f51280f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.385+0000 7f51267fc640 1 -- 10.20.193.28:0/1563853324 learned_addr learned my addr 10.20.193.28:0/1563853324 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:31.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.386+0000 7f51277fe640 1 -- 10.20.193.28:0/1563853324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f512810d3d0 msgr2=0x7f51280f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.386+0000 7f51277fe640 1 --2- 10.20.193.28:0/1563853324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f512810d3d0 0x7f51280f5920 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.386+0000 7f51277fe640 1 -- 10.20.193.28:0/1563853324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f512810ddc0 msgr2=0x7f5128173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.386+0000 7f51277fe640 1 --2- 10.20.193.28:0/1563853324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f512810ddc0 0x7f5128173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.386+0000 7f51277fe640 1 -- 10.20.193.28:0/1563853324 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511800a650 con 0x7f51281135c0 2026-02-25T11:52:31.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.386+0000 7f51277fe640 1 --2- 10.20.193.28:0/1563853324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f51281135c0 0x7f51280f6020 secure :-1 s=READY pgs=137 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5118033040 tx=0x7f5118030d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:31.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.386+0000 7f5107fff640 1 -- 10.20.193.28:0/1563853324 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f511803e070 con 0x7f51281135c0 2026-02-25T11:52:31.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.386+0000 7f5107fff640 1 -- 10.20.193.28:0/1563853324 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5118003ad0 con 0x7f51281135c0 2026-02-25T11:52:31.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.386+0000 7f5127fff640 1 -- 10.20.193.28:0/1563853324 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51280fb7e0 con 0x7f51281135c0 2026-02-25T11:52:31.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.386+0000 7f5107fff640 1 -- 10.20.193.28:0/1563853324 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f51180428c0 con 0x7f51281135c0 2026-02-25T11:52:31.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.386+0000 7f5127fff640 1 -- 10.20.193.28:0/1563853324 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51280fbc50 con 0x7f51281135c0 2026-02-25T11:52:31.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.389+0000 7f5107fff640 1 -- 10.20.193.28:0/1563853324 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f5118005380 con 0x7f51281135c0 2026-02-25T11:52:31.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.391+0000 7f5107fff640 1 --2- 10.20.193.28:0/1563853324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f50fc0b5750 0x7f50fc0d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.391+0000 7f5107fff640 1 -- 10.20.193.28:0/1563853324 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f511810c500 con 0x7f51281135c0 2026-02-25T11:52:31.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.392+0000 7f512f2bbe00 1 Processor -- start 2026-02-25T11:52:31.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.392+0000 7f5126ffd640 1 --2- 10.20.193.28:0/1563853324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f50fc0b5750 0x7f50fc0d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.392+0000 7f512f2bbe00 1 -- start start 2026-02-25T11:52:31.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.392+0000 7f512f2bbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 0x55f547908910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.392+0000 7f512f2bbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f547908e50 0x55f54794d5b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.392+0000 7f512f2bbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f54794daf0 0x55f54796ded0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.392+0000 7f512f2bbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f5478158a0 con 0x55f547908e50 2026-02-25T11:52:31.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.392+0000 7f512f2bbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f5478115d0 con 0x55f54794daf0 2026-02-25T11:52:31.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.392+0000 7f512f2bbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f54780b570 con 0x55f547908540 2026-02-25T11:52:31.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.393+0000 7f5126ffd640 1 --2- 10.20.193.28:0/1563853324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f50fc0b5750 0x7f50fc0d5c00 secure :-1 s=READY pgs=312 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f511c004740 tx=0x7f511c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:31.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.393+0000 7f5126ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 0x55f547908910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.393+0000 7f5126ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 0x55f547908910 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48124/0 (socket says 10.20.193.28:48124) 2026-02-25T11:52:31.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.393+0000 7f51277fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f54794daf0 0x55f54796ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.393+0000 7f5126ffd640 1 -- 10.20.193.28:0/4028169048 learned_addr learned my addr 10.20.193.28:0/4028169048 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:31.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.393+0000 7f51267fc640 1 --2- 10.20.193.28:0/4028169048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f547908e50 0x55f54794d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.393+0000 7f5126ffd640 1 -- 10.20.193.28:0/4028169048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f54794daf0 msgr2=0x55f54796ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.393+0000 7f5126ffd640 1 --2- 10.20.193.28:0/4028169048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f54794daf0 0x55f54796ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.393+0000 7f5126ffd640 1 -- 10.20.193.28:0/4028169048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f547908e50 msgr2=0x55f54794d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.393+0000 7f5126ffd640 1 --2- 10.20.193.28:0/4028169048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f547908e50 0x55f54794d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.393+0000 7f5126ffd640 1 -- 10.20.193.28:0/4028169048 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5477b6d10 con 0x55f547908540 2026-02-25T11:52:31.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.393+0000 7f51277fe640 1 --2- 10.20.193.28:0/4028169048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f54794daf0 0x55f54796ded0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:52:31.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.393+0000 7f5126ffd640 1 --2- 10.20.193.28:0/4028169048 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 0x55f547908910 secure :-1 s=READY pgs=166 gs=2 cs=0 l=1 c_cookie=2154696980555033973 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f511c00b770 tx=0x7f511c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1de7057cdc96ed75 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:31.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.394+0000 7f50f37fe640 1 -- 10.20.193.28:0/4028169048 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f511c0276a0 con 0x55f547908540 2026-02-25T11:52:31.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.394+0000 7f50f37fe640 1 -- 10.20.193.28:0/4028169048 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f511c015070 con 0x55f547908540 2026-02-25T11:52:31.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.394+0000 7f50f37fe640 1 -- 10.20.193.28:0/4028169048 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f511c01ed70 con 0x55f547908540 2026-02-25T11:52:31.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.394+0000 7f512f2bbe00 1 -- 10.20.193.28:0/4028169048 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 msgr2=0x55f547908910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.394+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/4028169048 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 0x55f547908910 secure :-1 s=READY pgs=166 gs=2 cs=0 l=1 c_cookie=2154696980555033973 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f511c00b770 tx=0x7f511c013000 comp rx=0 tx=0).stop 2026-02-25T11:52:31.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.394+0000 7f512f2bbe00 1 -- 10.20.193.28:0/4028169048 shutdown_connections 2026-02-25T11:52:31.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.394+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/4028169048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f54794daf0 0x55f54796ded0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.394+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/4028169048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f547908e50 0x55f54794d5b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.394+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/4028169048 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 0x55f547908910 unknown :-1 s=CLOSED pgs=166 gs=2 cs=0 l=1 c_cookie=2154696980555033973 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.394+0000 7f512f2bbe00 1 -- 10.20.193.28:0/4028169048 >> 10.20.193.28:0/4028169048 conn(0x55f54781e7a0 msgr2=0x55f5478fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:31.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.394+0000 7f512f2bbe00 1 -- 10.20.193.28:0/4028169048 shutdown_connections 2026-02-25T11:52:31.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.394+0000 7f512f2bbe00 1 -- 10.20.193.28:0/4028169048 wait complete. 2026-02-25T11:52:31.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.396+0000 7f512f2bbe00 1 -- start start 2026-02-25T11:52:31.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.396+0000 7f512f2bbe00 1 Processor -- start 2026-02-25T11:52:31.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f512f2bbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f547908540 0x55f5479209b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f512f2bbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908e50 0x55f547920ef0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f512f2bbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f54794daf0 0x55f547925ee0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f5126ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f547908540 0x55f5479209b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f512f2bbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f547926420 con 0x55f547908540 2026-02-25T11:52:31.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f5126ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f547908540 0x55f5479209b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58358/0 (socket says 10.20.193.28:58358) 2026-02-25T11:52:31.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f5126ffd640 1 -- 10.20.193.28:0/2369284992 learned_addr learned my addr 10.20.193.28:0/2369284992 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:31.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2369284992 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f547926590 con 0x55f54794daf0 2026-02-25T11:52:31.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2369284992 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f547926750 con 0x55f547908e50 2026-02-25T11:52:31.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f51267fc640 1 --2- 10.20.193.28:0/2369284992 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908e50 0x55f547920ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f51277fe640 1 --2- 10.20.193.28:0/2369284992 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f54794daf0 0x55f547925ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f5126ffd640 1 -- 10.20.193.28:0/2369284992 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908e50 msgr2=0x55f547920ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f5126ffd640 1 --2- 10.20.193.28:0/2369284992 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908e50 0x55f547920ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f5126ffd640 1 -- 10.20.193.28:0/2369284992 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f54794daf0 msgr2=0x55f547925ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f5126ffd640 1 --2- 10.20.193.28:0/2369284992 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f54794daf0 0x55f547925ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f5126ffd640 1 -- 10.20.193.28:0/2369284992 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511c00bd50 con 0x55f547908540 2026-02-25T11:52:31.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f5126ffd640 1 --2- 10.20.193.28:0/2369284992 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f547908540 0x55f5479209b0 secure :-1 s=READY pgs=589 gs=1 cs=0 l=1 c_cookie=4082033874741136985 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f511c01ec10 tx=0x7f511c01e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a64c2e790e8e59 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:31.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f50f0ff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:52:31.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.397+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2369284992 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f511c02b040 con 0x55f547908540 2026-02-25T11:52:31.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2369284992 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f511c01e720 con 0x55f547908540 2026-02-25T11:52:31.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2369284992 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f511c004c90 con 0x55f547908540 2026-02-25T11:52:31.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2369284992 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51280fbc50 con 0x55f547908540 2026-02-25T11:52:31.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2369284992 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f511c022070 con 0x55f547908540 2026-02-25T11:52:31.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f512f2bbe00 10 client.16362 _ll_register_callbacks cb 0x55f54778ca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:52:31.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2369284992 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f5474a16f0 con 0x55f547908540 2026-02-25T11:52:31.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2369284992 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f511c01e420 con 0x55f547908540 2026-02-25T11:52:31.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f512f2bbe00 10 client.16362 fetch_fsmap learned FSMap version 23 2026-02-25T11:52:31.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2369284992 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55f54778f3c0 con 0x55f547908540 2026-02-25T11:52:31.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2369284992 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f511c037070 con 0x55f547908540 2026-02-25T11:52:31.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f512f2bbe00 10 client.16362 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:52:31.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f512f2bbe00 10 client.16362 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:52:31.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f512f2bbe00 20 client.16362 populate_metadata read hostname 'trial028' 2026-02-25T11:52:31.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/2369284992 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55f54794b120 0x55f54794b5c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f512f2bbe00 4 client.16362 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:52:31.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.398+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2369284992 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5477ddfc0 con 0x55f54794b120 2026-02-25T11:52:31.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.399+0000 7f51267fc640 1 --2- 10.20.193.28:0/2369284992 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55f54794b120 0x55f54794b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.399+0000 7f51267fc640 1 --2- 10.20.193.28:0/2369284992 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55f54794b120 0x55f54794b5c0 crc :-1 s=READY pgs=39 gs=4 cs=0 l=0 c_cookie=16383087973959794678 s_cookie=4866770269073799452 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e35c6bb603d8fff6 server_cookie=438a3dc64393791c in_seq=0 out_seq=0 2026-02-25T11:52:31.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.399+0000 7f50f0ff9640 10 client.16362 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:52:31.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2369284992 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55f5477ddfc0 con 0x55f54794b120 2026-02-25T11:52:31.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f50f0ff9640 10 client.16362 handle_command_reply: tid=0 2026-02-25T11:52:31.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f512f2bbe00 1 client.16362 shutdown 2026-02-25T11:52:31.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2369284992 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f547908540 msgr2=0x55f5479209b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/2369284992 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f547908540 0x55f5479209b0 secure :-1 s=READY pgs=589 gs=1 cs=0 l=1 c_cookie=4082033874741136985 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f511c01ec10 tx=0x7f511c01e680 comp rx=0 tx=0).stop 2026-02-25T11:52:31.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2369284992 shutdown_connections 2026-02-25T11:52:31.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/2369284992 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55f54794b120 0x55f54794b5c0 crc :-1 s=READY pgs=39 gs=4 cs=0 l=0 c_cookie=16383087973959794678 s_cookie=4866770269073799452 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/2369284992 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f54794daf0 0x55f547925ee0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/2369284992 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908e50 0x55f547920ef0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/2369284992 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f547908540 0x55f5479209b0 unknown :-1 s=CLOSED pgs=589 gs=1 cs=0 l=1 c_cookie=4082033874741136985 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f50f0ff9640 0 client.16362 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:52:31.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2369284992 >> 10.20.193.28:0/2369284992 conn(0x55f54781e7a0 msgr2=0x55f5478feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:31.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2369284992 shutdown_connections 2026-02-25T11:52:31.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2369284992 wait complete. 2026-02-25T11:52:31.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.400+0000 7f512f2bbe00 20 client.16362 trim_cache size 0 max 16384 2026-02-25T11:52:31.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.408+0000 7f512f2bbe00 1 Processor -- start 2026-02-25T11:52:31.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.408+0000 7f512f2bbe00 1 -- start start 2026-02-25T11:52:31.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.408+0000 7f512f2bbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 0x55f547920dd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.408+0000 7f512f2bbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f547908e50 0x55f5478fed20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.408+0000 7f512f2bbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f54794daf0 0x55f5478ff260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.408+0000 7f512f2bbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f5478091a0 con 0x55f54794daf0 2026-02-25T11:52:31.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.408+0000 7f512f2bbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f5474c9700 con 0x55f547908e50 2026-02-25T11:52:31.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.408+0000 7f512f2bbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f547947980 con 0x55f547908540 2026-02-25T11:52:31.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.408+0000 7f51277fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f54794daf0 0x55f5478ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.408+0000 7f5126ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 0x55f547920dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.409+0000 7f5126ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 0x55f547920dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48132/0 (socket says 10.20.193.28:48132) 2026-02-25T11:52:31.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.409+0000 7f5126ffd640 1 -- 10.20.193.28:0/3189444337 learned_addr learned my addr 10.20.193.28:0/3189444337 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:31.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.409+0000 7f51267fc640 1 --2- 10.20.193.28:0/3189444337 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f547908e50 0x55f5478fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.409+0000 7f5126ffd640 1 -- 10.20.193.28:0/3189444337 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f547908e50 msgr2=0x55f5478fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.409+0000 7f5126ffd640 1 --2- 10.20.193.28:0/3189444337 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f547908e50 0x55f5478fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.409+0000 7f5126ffd640 1 -- 10.20.193.28:0/3189444337 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f54794daf0 msgr2=0x55f5478ff260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.409+0000 7f5126ffd640 1 --2- 10.20.193.28:0/3189444337 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f54794daf0 0x55f5478ff260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.409+0000 7f5126ffd640 1 -- 10.20.193.28:0/3189444337 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5477a10e0 con 0x55f547908540 2026-02-25T11:52:31.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.409+0000 7f51277fe640 1 --2- 10.20.193.28:0/3189444337 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f54794daf0 0x55f5478ff260 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:52:31.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.409+0000 7f5126ffd640 1 --2- 10.20.193.28:0/3189444337 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 0x55f547920dd0 secure :-1 s=READY pgs=167 gs=1 cs=0 l=1 c_cookie=14960918082576030340 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f511c01e5e0 tx=0x7f511c037000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf9fdbbad1a99a84 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:31.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.409+0000 7f50f2ffd640 1 -- 10.20.193.28:0/3189444337 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f511c02b070 con 0x55f547908540 2026-02-25T11:52:31.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.409+0000 7f50f2ffd640 1 -- 10.20.193.28:0/3189444337 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55f5474a16f0 con 0x55f547908540 2026-02-25T11:52:31.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.409+0000 7f50f2ffd640 1 -- 10.20.193.28:0/3189444337 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f511c01e930 con 0x55f547908540 2026-02-25T11:52:31.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.410+0000 7f512f2bbe00 1 -- 10.20.193.28:0/3189444337 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 msgr2=0x55f547920dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.410+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/3189444337 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 0x55f547920dd0 secure :-1 s=READY pgs=167 gs=1 cs=0 l=1 c_cookie=14960918082576030340 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f511c01e5e0 tx=0x7f511c037000 comp rx=0 tx=0).stop 2026-02-25T11:52:31.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.410+0000 7f512f2bbe00 1 -- 10.20.193.28:0/3189444337 shutdown_connections 2026-02-25T11:52:31.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.410+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/3189444337 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f54794daf0 0x55f5478ff260 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.410+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/3189444337 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f547908e50 0x55f5478fed20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.410+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/3189444337 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908540 0x55f547920dd0 unknown :-1 s=CLOSED pgs=167 gs=1 cs=0 l=1 c_cookie=14960918082576030340 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.410+0000 7f512f2bbe00 1 -- 10.20.193.28:0/3189444337 >> 10.20.193.28:0/3189444337 conn(0x55f54781e7a0 msgr2=0x55f54790b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:31.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.410+0000 7f512f2bbe00 1 -- 10.20.193.28:0/3189444337 shutdown_connections 2026-02-25T11:52:31.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.410+0000 7f512f2bbe00 1 -- 10.20.193.28:0/3189444337 wait complete. 2026-02-25T11:52:31.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.410+0000 7f512f2bbe00 1 -- start start 2026-02-25T11:52:31.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.410+0000 7f512f2bbe00 1 Processor -- start 2026-02-25T11:52:31.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f512f2bbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f547908540 0x55f547925310 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f512f2bbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908e50 0x55f547925850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f512f2bbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f54794daf0 0x55f547926eb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f512f2bbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f547927470 con 0x55f54794daf0 2026-02-25T11:52:31.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f512f2bbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f5479275e0 con 0x55f547908540 2026-02-25T11:52:31.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f512f2bbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5479277a0 con 0x55f547908e50 2026-02-25T11:52:31.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f51277fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f54794daf0 0x55f547926eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f51277fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f54794daf0 0x55f547926eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58370/0 (socket says 10.20.193.28:58370) 2026-02-25T11:52:31.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f51277fe640 1 -- 10.20.193.28:0/2943767352 learned_addr learned my addr 10.20.193.28:0/2943767352 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:31.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f51267fc640 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908e50 0x55f547925850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f5126ffd640 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f547908540 0x55f547925310 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f51277fe640 1 -- 10.20.193.28:0/2943767352 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908e50 msgr2=0x55f547925850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f51277fe640 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908e50 0x55f547925850 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f51277fe640 1 -- 10.20.193.28:0/2943767352 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f547908540 msgr2=0x55f547925310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f51277fe640 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f547908540 0x55f547925310 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f51277fe640 1 -- 10.20.193.28:0/2943767352 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f54778f3c0 con 0x55f54794daf0 2026-02-25T11:52:31.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f5126ffd640 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f547908540 0x55f547925310 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:52:31.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f51267fc640 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908e50 0x55f547925850 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:52:31.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f51277fe640 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f54794daf0 0x55f547926eb0 secure :-1 s=READY pgs=590 gs=1 cs=0 l=1 c_cookie=6112877035157505918 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5118109ff0 tx=0x7f5118030270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d54d46ac34c77e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:31.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.411+0000 7f50f0ff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:52:31.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2943767352 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f51180d8370 con 0x55f54794daf0 2026-02-25T11:52:31.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2943767352 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f511810c2e0 con 0x55f54794daf0 2026-02-25T11:52:31.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2943767352 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f51180dd790 con 0x55f54794daf0 2026-02-25T11:52:31.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2943767352 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51280fb7e0 con 0x55f54794daf0 2026-02-25T11:52:31.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2943767352 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f51180d0910 con 0x55f54794daf0 2026-02-25T11:52:31.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f512f2bbe00 10 client.16368 _ll_register_callbacks cb 0x55f54778ca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:52:31.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2943767352 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f511c015070 con 0x55f54794daf0 2026-02-25T11:52:31.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2943767352 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f51180dab80 con 0x55f54794daf0 2026-02-25T11:52:31.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f512f2bbe00 10 client.16368 fetch_fsmap learned FSMap version 23 2026-02-25T11:52:31.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2943767352 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55f54779bb60 con 0x55f54794daf0 2026-02-25T11:52:31.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2943767352 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f51180d4ba0 con 0x55f54794daf0 2026-02-25T11:52:31.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f512f2bbe00 10 client.16368 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:52:31.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f512f2bbe00 10 client.16368 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:52:31.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f512f2bbe00 20 client.16368 populate_metadata read hostname 'trial028' 2026-02-25T11:52:31.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55f5479fa950 0x55f547a1add0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f512f2bbe00 4 client.16368 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:52:31.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.412+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2943767352 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f54749ef40 con 0x55f5479fa950 2026-02-25T11:52:31.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.413+0000 7f5126ffd640 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55f5479fa950 0x55f547a1add0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.413+0000 7f5126ffd640 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55f5479fa950 0x55f547a1add0 crc :-1 s=READY pgs=40 gs=4 cs=0 l=0 c_cookie=3720370323371123836 s_cookie=9129174347563794070 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33a169190570e47c server_cookie=7eb157b35f25fe96 in_seq=0 out_seq=0 2026-02-25T11:52:31.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.413+0000 7f50f0ff9640 10 client.16368 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:52:31.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.413+0000 7f50f0ff9640 1 -- 10.20.193.28:0/2943767352 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f54749ef40 con 0x55f5479fa950 2026-02-25T11:52:31.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.413+0000 7f50f0ff9640 10 client.16368 handle_command_reply: tid=0 2026-02-25T11:52:31.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.413+0000 7f512f2bbe00 1 client.16368 shutdown 2026-02-25T11:52:31.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.413+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2943767352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f54794daf0 msgr2=0x55f547926eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.413+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f54794daf0 0x55f547926eb0 secure :-1 s=READY pgs=590 gs=1 cs=0 l=1 c_cookie=6112877035157505918 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5118109ff0 tx=0x7f5118030270 comp rx=0 tx=0).stop 2026-02-25T11:52:31.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.413+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2943767352 shutdown_connections 2026-02-25T11:52:31.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.413+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55f5479fa950 0x55f547a1add0 crc :-1 s=READY pgs=40 gs=4 cs=0 l=0 c_cookie=3720370323371123836 s_cookie=9129174347563794070 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.413+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f54794daf0 0x55f547926eb0 unknown :-1 s=CLOSED pgs=590 gs=1 cs=0 l=1 c_cookie=6112877035157505918 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.413+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f547908e50 0x55f547925850 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.413+0000 7f512f2bbe00 1 --2- 10.20.193.28:0/2943767352 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f547908540 0x55f547925310 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f50f0ff9640 0 client.16368 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:52:31.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2943767352 >> 10.20.193.28:0/2943767352 conn(0x55f54781e7a0 msgr2=0x55f5478fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:31.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2943767352 shutdown_connections 2026-02-25T11:52:31.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f512f2bbe00 1 -- 10.20.193.28:0/2943767352 wait complete. 2026-02-25T11:52:31.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f512f2bbe00 20 client.16368 trim_cache size 0 max 16384 2026-02-25T11:52:31.425 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:52:31.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f5127fff640 1 -- 10.20.193.28:0/1563853324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f50fc0b5750 msgr2=0x7f50fc0d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f5127fff640 1 --2- 10.20.193.28:0/1563853324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f50fc0b5750 0x7f50fc0d5c00 secure :-1 s=READY pgs=312 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f511c004740 tx=0x7f511c00b410 comp rx=0 tx=0).stop 2026-02-25T11:52:31.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f5127fff640 1 -- 10.20.193.28:0/1563853324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f51281135c0 msgr2=0x7f51280f6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f5127fff640 1 --2- 10.20.193.28:0/1563853324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f51281135c0 0x7f51280f6020 secure :-1 s=READY pgs=137 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5118033040 tx=0x7f5118030d70 comp rx=0 tx=0).stop 2026-02-25T11:52:31.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f5127fff640 1 -- 10.20.193.28:0/1563853324 shutdown_connections 2026-02-25T11:52:31.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f5127fff640 1 --2- 10.20.193.28:0/1563853324 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f50fc0b5750 0x7f50fc0d5c00 unknown :-1 s=CLOSED pgs=312 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f5127fff640 1 --2- 10.20.193.28:0/1563853324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f51281135c0 0x7f51280f6020 unknown :-1 s=CLOSED pgs=137 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f5127fff640 1 --2- 10.20.193.28:0/1563853324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f512810ddc0 0x7f5128173e90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f5127fff640 1 --2- 10.20.193.28:0/1563853324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f512810d3d0 0x7f51280f5920 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.414+0000 7f5127fff640 1 -- 10.20.193.28:0/1563853324 >> 10.20.193.28:0/1563853324 conn(0x7f51280eaca0 msgr2=0x7f51281115b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:31.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.415+0000 7f5127fff640 1 -- 10.20.193.28:0/1563853324 shutdown_connections 2026-02-25T11:52:31.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.415+0000 7f5127fff640 1 -- 10.20.193.28:0/1563853324 wait complete. 2026-02-25T11:52:31.503 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:31 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1230377324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:52:31.554 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:52:31.739 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:52:31.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:31 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1230377324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:52:31.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:31 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1230377324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:52:31.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.925+0000 7fc69a567640 1 -- 10.20.193.28:0/3086647631 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc69410b360 msgr2=0x7fc69410b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.925+0000 7fc69a567640 1 --2- 10.20.193.28:0/3086647631 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc69410b360 0x7fc69410b730 secure :-1 s=READY pgs=591 gs=1 cs=0 l=1 c_cookie=10253602505307832263 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc684009950 tx=0x7fc68402f190 comp rx=0 tx=0).stop 2026-02-25T11:52:31.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.925+0000 7fc69a567640 1 -- 10.20.193.28:0/3086647631 shutdown_connections 2026-02-25T11:52:31.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.925+0000 7fc69a567640 1 --2- 10.20.193.28:0/3086647631 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc694111370 0x7fc694111d50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.925+0000 7fc69a567640 1 --2- 10.20.193.28:0/3086647631 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc69410bd50 0x7fc694110be0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.925+0000 7fc69a567640 1 --2- 10.20.193.28:0/3086647631 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc69410b360 0x7fc69410b730 unknown :-1 s=CLOSED pgs=591 gs=1 cs=0 l=1 c_cookie=10253602505307832263 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.925+0000 7fc69a567640 1 -- 10.20.193.28:0/3086647631 >> 10.20.193.28:0/3086647631 conn(0x7fc6940e8e70 msgr2=0x7fc694109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:31.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.926+0000 7fc69a567640 1 -- 10.20.193.28:0/3086647631 shutdown_connections 2026-02-25T11:52:31.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.926+0000 7fc69a567640 1 -- 10.20.193.28:0/3086647631 wait complete. 2026-02-25T11:52:31.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.927+0000 7fc69a567640 1 Processor -- start 2026-02-25T11:52:31.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.927+0000 7fc69a567640 1 -- start start 2026-02-25T11:52:31.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.927+0000 7fc69a567640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc69410b360 0x7fc6941236b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.927+0000 7fc69a567640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc69410bd50 0x7fc694123bf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.927+0000 7fc69a567640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc694111370 0x7fc694128fc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.927+0000 7fc69a567640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc6941295e0 con 0x7fc69410b360 2026-02-25T11:52:31.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.927+0000 7fc69a567640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc694129750 con 0x7fc69410bd50 2026-02-25T11:52:31.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.927+0000 7fc69a567640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc694129910 con 0x7fc694111370 2026-02-25T11:52:31.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.927+0000 7fc699565640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc69410b360 0x7fc6941236b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.927+0000 7fc699565640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc69410b360 0x7fc6941236b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58382/0 (socket says 10.20.193.28:58382) 2026-02-25T11:52:31.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.927+0000 7fc699565640 1 -- 10.20.193.28:0/313138606 learned_addr learned my addr 10.20.193.28:0/313138606 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:31.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.928+0000 7fc699d66640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc694111370 0x7fc694128fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.928+0000 7fc698d64640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc69410bd50 0x7fc694123bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.928+0000 7fc699565640 1 -- 10.20.193.28:0/313138606 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc694111370 msgr2=0x7fc694128fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.928+0000 7fc699565640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc694111370 0x7fc694128fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.928+0000 7fc699565640 1 -- 10.20.193.28:0/313138606 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc69410bd50 msgr2=0x7fc694123bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.928+0000 7fc699565640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc69410bd50 0x7fc694123bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.928+0000 7fc699565640 1 -- 10.20.193.28:0/313138606 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc684009570 con 0x7fc69410b360 2026-02-25T11:52:31.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.928+0000 7fc699565640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc69410b360 0x7fc6941236b0 secure :-1 s=READY pgs=592 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc6840098a0 tx=0x7fc6840028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:31.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.928+0000 7fc6827fc640 1 -- 10.20.193.28:0/313138606 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc68403d070 con 0x7fc69410b360 2026-02-25T11:52:31.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.928+0000 7fc6827fc640 1 -- 10.20.193.28:0/313138606 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc684002a40 con 0x7fc69410b360 2026-02-25T11:52:31.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.928+0000 7fc6827fc640 1 -- 10.20.193.28:0/313138606 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc684041720 con 0x7fc69410b360 2026-02-25T11:52:31.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.928+0000 7fc69a567640 1 -- 10.20.193.28:0/313138606 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc694129b90 con 0x7fc69410b360 2026-02-25T11:52:31.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.928+0000 7fc69a567640 1 -- 10.20.193.28:0/313138606 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69416a150 con 0x7fc69410b360 2026-02-25T11:52:31.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.931+0000 7fc6827fc640 1 -- 10.20.193.28:0/313138606 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fc6840386a0 con 0x7fc69410b360 2026-02-25T11:52:31.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.933+0000 7fc6827fc640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc6700b5760 0x7fc6700d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.933+0000 7fc6827fc640 1 -- 10.20.193.28:0/313138606 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc68410b0f0 con 0x7fc69410b360 2026-02-25T11:52:31.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.933+0000 7fc698d64640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc6700b5760 0x7fc6700d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.934+0000 7fc69d7ebe00 1 Processor -- start 2026-02-25T11:52:31.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.934+0000 7fc698d64640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc6700b5760 0x7fc6700d5c10 secure :-1 s=READY pgs=313 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc69412a050 tx=0x7fc688005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:31.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.934+0000 7fc69d7ebe00 1 -- start start 2026-02-25T11:52:31.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.934+0000 7fc69d7ebe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 0x559d76557f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.934+0000 7fc69d7ebe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76558440 0x559d7659cba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.934+0000 7fc69d7ebe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d7659d0e0 0x559d765bd4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.934+0000 7fc69d7ebe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559d7646d8a0 con 0x559d76557b30 2026-02-25T11:52:31.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.934+0000 7fc69d7ebe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559d764695d0 con 0x559d76558440 2026-02-25T11:52:31.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.934+0000 7fc69d7ebe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559d76463570 con 0x559d7659d0e0 2026-02-25T11:52:31.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc699565640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 0x559d76557f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc699565640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 0x559d76557f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58388/0 (socket says 10.20.193.28:58388) 2026-02-25T11:52:31.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc699565640 1 -- 10.20.193.28:0/1069359280 learned_addr learned my addr 10.20.193.28:0/1069359280 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:31.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc698d64640 1 --2- 10.20.193.28:0/1069359280 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76558440 0x559d7659cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc699d66640 1 --2- 10.20.193.28:0/1069359280 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d7659d0e0 0x559d765bd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc699565640 1 -- 10.20.193.28:0/1069359280 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d7659d0e0 msgr2=0x559d765bd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc699565640 1 --2- 10.20.193.28:0/1069359280 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d7659d0e0 0x559d765bd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc699565640 1 -- 10.20.193.28:0/1069359280 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76558440 msgr2=0x559d7659cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc699565640 1 --2- 10.20.193.28:0/1069359280 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76558440 0x559d7659cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc699565640 1 -- 10.20.193.28:0/1069359280 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7640ed10 con 0x559d76557b30 2026-02-25T11:52:31.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc699565640 1 --2- 10.20.193.28:0/1069359280 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 0x559d76557f00 secure :-1 s=READY pgs=593 gs=3 cs=0 l=1 c_cookie=10252282091031716200 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc684009480 tx=0x7fc68402fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e476c3e2d0b6168 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:31.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc661ce5640 1 -- 10.20.193.28:0/1069359280 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc6840f6040 con 0x559d76557b30 2026-02-25T11:52:31.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc661ce5640 1 -- 10.20.193.28:0/1069359280 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc6840d0c70 con 0x559d76557b30 2026-02-25T11:52:31.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc661ce5640 1 -- 10.20.193.28:0/1069359280 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc6840d2100 con 0x559d76557b30 2026-02-25T11:52:31.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1069359280 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 msgr2=0x559d76557f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.935+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1069359280 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 0x559d76557f00 secure :-1 s=READY pgs=593 gs=3 cs=0 l=1 c_cookie=10252282091031716200 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc684009480 tx=0x7fc68402fe20 comp rx=0 tx=0).stop 2026-02-25T11:52:31.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.936+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1069359280 shutdown_connections 2026-02-25T11:52:31.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.936+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1069359280 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d7659d0e0 0x559d765bd4c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.936+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1069359280 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76558440 0x559d7659cba0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.936+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1069359280 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 0x559d76557f00 unknown :-1 s=CLOSED pgs=593 gs=3 cs=0 l=1 c_cookie=10252282091031716200 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.936+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1069359280 >> 10.20.193.28:0/1069359280 conn(0x559d764767a0 msgr2=0x559d76556410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:31.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.936+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1069359280 shutdown_connections 2026-02-25T11:52:31.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.936+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1069359280 wait complete. 2026-02-25T11:52:31.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.937+0000 7fc69d7ebe00 1 -- start start 2026-02-25T11:52:31.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.937+0000 7fc69d7ebe00 1 Processor -- start 2026-02-25T11:52:31.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc69d7ebe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 0x559d7656fd60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc69d7ebe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76558440 0x559d765702a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc69d7ebe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d7659d0e0 0x559d76575290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc69d7ebe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559d765757d0 con 0x559d76557b30 2026-02-25T11:52:31.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc69d7ebe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559d76575940 con 0x559d76558440 2026-02-25T11:52:31.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc69d7ebe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559d76575b00 con 0x559d7659d0e0 2026-02-25T11:52:31.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc699565640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 0x559d7656fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc699565640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 0x559d7656fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58404/0 (socket says 10.20.193.28:58404) 2026-02-25T11:52:31.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc699565640 1 -- 10.20.193.28:0/1274932915 learned_addr learned my addr 10.20.193.28:0/1274932915 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:31.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc698d64640 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76558440 0x559d765702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc699565640 1 -- 10.20.193.28:0/1274932915 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d7659d0e0 msgr2=0x559d76575290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc699d66640 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d7659d0e0 0x559d76575290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc699565640 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d7659d0e0 0x559d76575290 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc699565640 1 -- 10.20.193.28:0/1274932915 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76558440 msgr2=0x559d765702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc699565640 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76558440 0x559d765702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc699565640 1 -- 10.20.193.28:0/1274932915 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d763e73c0 con 0x559d76557b30 2026-02-25T11:52:31.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc698d64640 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76558440 0x559d765702a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:52:31.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc699d66640 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d7659d0e0 0x559d76575290 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:52:31.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc699565640 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 0x559d7656fd60 secure :-1 s=READY pgs=594 gs=1 cs=0 l=1 c_cookie=7339119280314722454 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc684005b30 tx=0x7fc6840d12c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d9ca19e3612896 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:31.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.938+0000 7fc64effd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:52:31.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc64effd640 1 -- 10.20.193.28:0/1274932915 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc6840f6040 con 0x559d76557b30 2026-02-25T11:52:31.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc64effd640 1 -- 10.20.193.28:0/1274932915 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc6840d1e70 con 0x559d76557b30 2026-02-25T11:52:31.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc64effd640 1 -- 10.20.193.28:0/1274932915 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc684096600 con 0x559d76557b30 2026-02-25T11:52:31.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1274932915 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69416a150 con 0x559d76557b30 2026-02-25T11:52:31.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc69d7ebe00 10 client.16380 _ll_register_callbacks cb 0x559d763e4a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:52:31.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1274932915 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559d760f96f0 con 0x559d76557b30 2026-02-25T11:52:31.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc64effd640 1 -- 10.20.193.28:0/1274932915 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc6840d7190 con 0x559d76557b30 2026-02-25T11:52:31.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc64effd640 1 -- 10.20.193.28:0/1274932915 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc6840d9770 con 0x559d76557b30 2026-02-25T11:52:31.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc69d7ebe00 10 client.16380 fetch_fsmap learned FSMap version 23 2026-02-25T11:52:31.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1274932915 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x559d763f90e0 con 0x559d76557b30 2026-02-25T11:52:31.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc64effd640 1 -- 10.20.193.28:0/1274932915 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fc684091020 con 0x559d76557b30 2026-02-25T11:52:31.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc69d7ebe00 10 client.16380 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:52:31.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc69d7ebe00 10 client.16380 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:52:31.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc69d7ebe00 20 client.16380 populate_metadata read hostname 'trial028' 2026-02-25T11:52:31.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559d7659a0a0 0x559d7659a540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc69d7ebe00 4 client.16380 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:52:31.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.939+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1274932915 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d76435fc0 con 0x559d7659a0a0 2026-02-25T11:52:31.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.940+0000 7fc698d64640 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559d7659a0a0 0x559d7659a540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.940+0000 7fc698d64640 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559d7659a0a0 0x559d7659a540 crc :-1 s=READY pgs=40 gs=4 cs=0 l=0 c_cookie=8873239905783057535 s_cookie=4203648577762341691 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b2414ae8f050c7f server_cookie=3a565c1c0e19b33b in_seq=0 out_seq=0 2026-02-25T11:52:31.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.940+0000 7fc64effd640 10 client.16380 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:52:31.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc64effd640 1 -- 10.20.193.28:0/1274932915 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x559d76435fc0 con 0x559d7659a0a0 2026-02-25T11:52:31.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc64effd640 10 client.16380 handle_command_reply: tid=0 2026-02-25T11:52:31.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc69d7ebe00 1 client.16380 shutdown 2026-02-25T11:52:31.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1274932915 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 msgr2=0x559d7656fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 0x559d7656fd60 secure :-1 s=READY pgs=594 gs=1 cs=0 l=1 c_cookie=7339119280314722454 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc684005b30 tx=0x7fc6840d12c0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1274932915 shutdown_connections 2026-02-25T11:52:31.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559d7659a0a0 0x559d7659a540 crc :-1 s=READY pgs=40 gs=4 cs=0 l=0 c_cookie=8873239905783057535 s_cookie=4203648577762341691 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d7659d0e0 0x559d76575290 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76558440 0x559d765702a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1274932915 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76557b30 0x559d7656fd60 unknown :-1 s=CLOSED pgs=594 gs=1 cs=0 l=1 c_cookie=7339119280314722454 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1274932915 >> 10.20.193.28:0/1274932915 conn(0x559d764767a0 msgr2=0x559d76556b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:31.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc64effd640 0 client.16380 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:52:31.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1274932915 shutdown_connections 2026-02-25T11:52:31.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.941+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1274932915 wait complete. 2026-02-25T11:52:31.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.942+0000 7fc69d7ebe00 20 client.16380 trim_cache size 0 max 16384 2026-02-25T11:52:31.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.949+0000 7fc69d7ebe00 1 Processor -- start 2026-02-25T11:52:31.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.949+0000 7fc69d7ebe00 1 -- start start 2026-02-25T11:52:31.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.949+0000 7fc69d7ebe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d76557b30 0x559d76570290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.949+0000 7fc69d7ebe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76558440 0x559d76561eb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.949+0000 7fc69d7ebe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d7659d0e0 0x559d76556d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.949+0000 7fc69d7ebe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559d764611a0 con 0x559d76558440 2026-02-25T11:52:31.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.949+0000 7fc69d7ebe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559d76121700 con 0x559d7659d0e0 2026-02-25T11:52:31.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.949+0000 7fc69d7ebe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559d764695d0 con 0x559d76557b30 2026-02-25T11:52:31.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.949+0000 7fc699d66640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d7659d0e0 0x559d76556d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.949+0000 7fc699565640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d76557b30 0x559d76570290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.949+0000 7fc699d66640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d7659d0e0 0x559d76556d20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43248/0 (socket says 10.20.193.28:43248) 2026-02-25T11:52:31.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.949+0000 7fc699d66640 1 -- 10.20.193.28:0/1463613362 learned_addr learned my addr 10.20.193.28:0/1463613362 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:31.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.949+0000 7fc698d64640 1 --2- 10.20.193.28:0/1463613362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76558440 0x559d76561eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.950+0000 7fc699d66640 1 -- 10.20.193.28:0/1463613362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d76557b30 msgr2=0x559d76570290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.950+0000 7fc699d66640 1 --2- 10.20.193.28:0/1463613362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d76557b30 0x559d76570290 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.950+0000 7fc699d66640 1 -- 10.20.193.28:0/1463613362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76558440 msgr2=0x559d76561eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.950+0000 7fc699d66640 1 --2- 10.20.193.28:0/1463613362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76558440 0x559d76561eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.950+0000 7fc699d66640 1 -- 10.20.193.28:0/1463613362 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d76562680 con 0x559d7659d0e0 2026-02-25T11:52:31.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.950+0000 7fc699d66640 1 --2- 10.20.193.28:0/1463613362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d7659d0e0 0x559d76556d20 secure :-1 s=READY pgs=138 gs=1 cs=0 l=1 c_cookie=12925090670245449478 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc690005780 tx=0x7fc6900057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b35f257c09b20f06 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:31.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.950+0000 7fc6614e4640 1 -- 10.20.193.28:0/1463613362 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc690019070 con 0x559d7659d0e0 2026-02-25T11:52:31.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.950+0000 7fc6614e4640 1 -- 10.20.193.28:0/1463613362 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc69000ad30 con 0x559d7659d0e0 2026-02-25T11:52:31.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.950+0000 7fc6614e4640 1 -- 10.20.193.28:0/1463613362 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc69001e7c0 con 0x559d7659d0e0 2026-02-25T11:52:31.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.951+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1463613362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d7659d0e0 msgr2=0x559d76556d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.951+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1463613362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d7659d0e0 0x559d76556d20 secure :-1 s=READY pgs=138 gs=1 cs=0 l=1 c_cookie=12925090670245449478 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc690005780 tx=0x7fc6900057b0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.951+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1463613362 shutdown_connections 2026-02-25T11:52:31.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.951+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1463613362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d7659d0e0 0x559d76556d20 unknown :-1 s=CLOSED pgs=138 gs=1 cs=0 l=1 c_cookie=12925090670245449478 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.951+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1463613362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d76558440 0x559d76561eb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.951+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/1463613362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d76557b30 0x559d76570290 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.951+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1463613362 >> 10.20.193.28:0/1463613362 conn(0x559d764767a0 msgr2=0x559d7656fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:31.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.951+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1463613362 shutdown_connections 2026-02-25T11:52:31.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.951+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/1463613362 wait complete. 2026-02-25T11:52:31.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.951+0000 7fc69d7ebe00 1 -- start start 2026-02-25T11:52:31.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc69d7ebe00 1 Processor -- start 2026-02-25T11:52:31.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc69d7ebe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76557b30 0x559d7657bbe0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc69d7ebe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d76558440 0x559d76577190 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc69d7ebe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d7659d0e0 0x559d765776d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc69d7ebe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559d765755d0 con 0x559d7659d0e0 2026-02-25T11:52:31.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc69d7ebe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559d76596a70 con 0x559d76557b30 2026-02-25T11:52:31.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc69d7ebe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559d7646d8a0 con 0x559d76558440 2026-02-25T11:52:31.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc699d66640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d7659d0e0 0x559d765776d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc699d66640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d7659d0e0 0x559d765776d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58426/0 (socket says 10.20.193.28:58426) 2026-02-25T11:52:31.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc699d66640 1 -- 10.20.193.28:0/873838124 learned_addr learned my addr 10.20.193.28:0/873838124 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:31.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc699565640 1 --2- 10.20.193.28:0/873838124 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76557b30 0x559d7657bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc698d64640 1 --2- 10.20.193.28:0/873838124 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d76558440 0x559d76577190 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc699d66640 1 -- 10.20.193.28:0/873838124 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d76558440 msgr2=0x559d76577190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc699d66640 1 --2- 10.20.193.28:0/873838124 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d76558440 0x559d76577190 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc699d66640 1 -- 10.20.193.28:0/873838124 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76557b30 msgr2=0x559d7657bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:31.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc699d66640 1 --2- 10.20.193.28:0/873838124 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76557b30 0x559d7657bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.952+0000 7fc699d66640 1 -- 10.20.193.28:0/873838124 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d763f3b60 con 0x559d7659d0e0 2026-02-25T11:52:31.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.953+0000 7fc699d66640 1 --2- 10.20.193.28:0/873838124 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d7659d0e0 0x559d765776d0 secure :-1 s=READY pgs=595 gs=1 cs=0 l=1 c_cookie=17332867571394734681 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc690005020 tx=0x7fc6900128f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08ab73903deea59 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:31.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.953+0000 7fc64effd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:52:31.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.953+0000 7fc64effd640 1 -- 10.20.193.28:0/873838124 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc690019040 con 0x559d7659d0e0 2026-02-25T11:52:31.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.953+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/873838124 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc690005400 con 0x559d7659d0e0 2026-02-25T11:52:31.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.953+0000 7fc69d7ebe00 10 client.16386 _ll_register_callbacks cb 0x559d763e4a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:52:31.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.953+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/873838124 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6840d0c70 con 0x559d7659d0e0 2026-02-25T11:52:31.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.953+0000 7fc64effd640 1 -- 10.20.193.28:0/873838124 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc690006ec0 con 0x559d7659d0e0 2026-02-25T11:52:31.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.953+0000 7fc64effd640 1 -- 10.20.193.28:0/873838124 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc690006420 con 0x559d7659d0e0 2026-02-25T11:52:31.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.953+0000 7fc64effd640 1 -- 10.20.193.28:0/873838124 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc690006580 con 0x559d7659d0e0 2026-02-25T11:52:31.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.953+0000 7fc64effd640 1 -- 10.20.193.28:0/873838124 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc6900067a0 con 0x559d7659d0e0 2026-02-25T11:52:31.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.953+0000 7fc69d7ebe00 10 client.16386 fetch_fsmap learned FSMap version 23 2026-02-25T11:52:31.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.953+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/873838124 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x559d763e76f0 con 0x559d7659d0e0 2026-02-25T11:52:31.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.953+0000 7fc64effd640 1 -- 10.20.193.28:0/873838124 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fc690025020 con 0x559d7659d0e0 2026-02-25T11:52:31.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.954+0000 7fc69d7ebe00 10 client.16386 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:52:31.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.954+0000 7fc69d7ebe00 10 client.16386 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:52:31.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.954+0000 7fc69d7ebe00 20 client.16386 populate_metadata read hostname 'trial028' 2026-02-25T11:52:31.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.954+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/873838124 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559d7664a020 0x559d7666a4d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:31.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.954+0000 7fc69d7ebe00 4 client.16386 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:52:31.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.954+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/873838124 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d760f6f40 con 0x559d7664a020 2026-02-25T11:52:31.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.954+0000 7fc699565640 1 --2- 10.20.193.28:0/873838124 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559d7664a020 0x559d7666a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:31.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.954+0000 7fc699565640 1 --2- 10.20.193.28:0/873838124 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559d7664a020 0x559d7666a4d0 crc :-1 s=READY pgs=41 gs=4 cs=0 l=0 c_cookie=16917766305949137322 s_cookie=6766390051373976753 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eac7fad2332bb9aa server_cookie=5de70be6dfe20cb1 in_seq=0 out_seq=0 2026-02-25T11:52:31.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.954+0000 7fc64effd640 10 client.16386 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:52:31.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.954+0000 7fc64effd640 1 -- 10.20.193.28:0/873838124 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d760f6f40 con 0x559d7664a020 2026-02-25T11:52:31.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.954+0000 7fc64effd640 10 client.16386 handle_command_reply: tid=0 2026-02-25T11:52:31.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.954+0000 7fc69d7ebe00 1 client.16386 shutdown 2026-02-25T11:52:31.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.955+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/873838124 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d7659d0e0 msgr2=0x559d765776d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.955+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/873838124 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d7659d0e0 0x559d765776d0 secure :-1 s=READY pgs=595 gs=1 cs=0 l=1 c_cookie=17332867571394734681 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc690005020 tx=0x7fc6900128f0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.955+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/873838124 shutdown_connections 2026-02-25T11:52:31.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.955+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/873838124 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x559d7664a020 0x559d7666a4d0 crc :-1 s=READY pgs=41 gs=4 cs=0 l=0 c_cookie=16917766305949137322 s_cookie=6766390051373976753 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.955+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/873838124 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559d7659d0e0 0x559d765776d0 unknown :-1 s=CLOSED pgs=595 gs=1 cs=0 l=1 c_cookie=17332867571394734681 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.955+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/873838124 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559d76558440 0x559d76577190 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.955+0000 7fc69d7ebe00 1 --2- 10.20.193.28:0/873838124 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559d76557b30 0x559d7657bbe0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.955+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/873838124 >> 10.20.193.28:0/873838124 conn(0x559d764767a0 msgr2=0x559d765748e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:31.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.955+0000 7fc64effd640 0 client.16386 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:52:31.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.955+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/873838124 shutdown_connections 2026-02-25T11:52:31.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.955+0000 7fc69d7ebe00 1 -- 10.20.193.28:0/873838124 wait complete. 2026-02-25T11:52:31.964 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.955+0000 7fc69d7ebe00 20 client.16386 trim_cache size 0 max 16384 2026-02-25T11:52:31.965 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:52:31.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.956+0000 7fc69a567640 1 -- 10.20.193.28:0/313138606 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc6700b5760 msgr2=0x7fc6700d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.956+0000 7fc69a567640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc6700b5760 0x7fc6700d5c10 secure :-1 s=READY pgs=313 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc69412a050 tx=0x7fc688005f90 comp rx=0 tx=0).stop 2026-02-25T11:52:31.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.956+0000 7fc69a567640 1 -- 10.20.193.28:0/313138606 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc69410b360 msgr2=0x7fc6941236b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:31.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.956+0000 7fc69a567640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc69410b360 0x7fc6941236b0 secure :-1 s=READY pgs=592 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc6840098a0 tx=0x7fc6840028d0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.956+0000 7fc69a567640 1 -- 10.20.193.28:0/313138606 shutdown_connections 2026-02-25T11:52:31.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.956+0000 7fc69a567640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc6700b5760 0x7fc6700d5c10 unknown :-1 s=CLOSED pgs=313 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.956+0000 7fc69a567640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc694111370 0x7fc694128fc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.956+0000 7fc69a567640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc69410bd50 0x7fc694123bf0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.956+0000 7fc69a567640 1 --2- 10.20.193.28:0/313138606 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc69410b360 0x7fc6941236b0 unknown :-1 s=CLOSED pgs=592 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:31.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.956+0000 7fc69a567640 1 -- 10.20.193.28:0/313138606 >> 10.20.193.28:0/313138606 conn(0x7fc6940e8e70 msgr2=0x7fc694109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:31.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.956+0000 7fc69a567640 1 -- 10.20.193.28:0/313138606 shutdown_connections 2026-02-25T11:52:31.966 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:31.956+0000 7fc69a567640 1 -- 10.20.193.28:0/313138606 wait complete. 2026-02-25T11:52:32.056 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:52:32.241 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:52:32.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:32 trial028 ceph-mon[39224]: pgmap v355: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:32.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.420+0000 7ff3d255e640 1 -- 10.20.193.28:0/2441175813 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3cc10f690 msgr2=0x7ff3cc10fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:32.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.420+0000 7ff3d255e640 1 --2- 10.20.193.28:0/2441175813 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3cc10f690 0x7ff3cc10fa60 secure :-1 s=READY pgs=596 gs=1 cs=0 l=1 c_cookie=9812266989924209247 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3bc009950 tx=0x7ff3bc02f190 comp rx=0 tx=0).stop 2026-02-25T11:52:32.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.421+0000 7ff3d255e640 1 -- 10.20.193.28:0/2441175813 shutdown_connections 2026-02-25T11:52:32.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.421+0000 7ff3d255e640 1 --2- 10.20.193.28:0/2441175813 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff3cc115880 0x7ff3cc176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.421+0000 7ff3d255e640 1 --2- 10.20.193.28:0/2441175813 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff3cc110080 0x7ff3cc1150d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.421+0000 7ff3d255e640 1 --2- 10.20.193.28:0/2441175813 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3cc10f690 0x7ff3cc10fa60 unknown :-1 s=CLOSED pgs=596 gs=1 cs=0 l=1 c_cookie=9812266989924209247 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.421+0000 7ff3d255e640 1 -- 10.20.193.28:0/2441175813 >> 10.20.193.28:0/2441175813 conn(0x7ff3cc0ecf00 msgr2=0x7ff3cc10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:32.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.421+0000 7ff3d255e640 1 -- 10.20.193.28:0/2441175813 shutdown_connections 2026-02-25T11:52:32.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.421+0000 7ff3d255e640 1 -- 10.20.193.28:0/2441175813 wait complete. 2026-02-25T11:52:32.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.422+0000 7ff3d255e640 1 Processor -- start 2026-02-25T11:52:32.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.422+0000 7ff3d255e640 1 -- start start 2026-02-25T11:52:32.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.422+0000 7ff3d255e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3cc10f690 0x7ff3cc0f5960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.422+0000 7ff3d255e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff3cc110080 0x7ff3cc176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.422+0000 7ff3d255e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff3cc115880 0x7ff3cc0f6060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.422+0000 7ff3d255e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc0fb270 con 0x7ff3cc10f690 2026-02-25T11:52:32.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.422+0000 7ff3d255e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc0fb3e0 con 0x7ff3cc115880 2026-02-25T11:52:32.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.422+0000 7ff3d255e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc0fb5a0 con 0x7ff3cc110080 2026-02-25T11:52:32.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.422+0000 7ff3d155c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3cc10f690 0x7ff3cc0f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.422+0000 7ff3d155c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3cc10f690 0x7ff3cc0f5960 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58448/0 (socket says 10.20.193.28:58448) 2026-02-25T11:52:32.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.422+0000 7ff3d155c640 1 -- 10.20.193.28:0/869577640 learned_addr learned my addr 10.20.193.28:0/869577640 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:32.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3d1d5d640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff3cc115880 0x7ff3cc0f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3d0d5b640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff3cc110080 0x7ff3cc176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3d155c640 1 -- 10.20.193.28:0/869577640 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff3cc110080 msgr2=0x7ff3cc176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:32.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3d155c640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff3cc110080 0x7ff3cc176150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3d155c640 1 -- 10.20.193.28:0/869577640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff3cc115880 msgr2=0x7ff3cc0f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:32.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3d155c640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff3cc115880 0x7ff3cc0f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3d155c640 1 -- 10.20.193.28:0/869577640 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3bc009570 con 0x7ff3cc10f690 2026-02-25T11:52:32.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3d0d5b640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff3cc110080 0x7ff3cc176150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:52:32.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3d1d5d640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff3cc115880 0x7ff3cc0f6060 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:52:32.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3d155c640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3cc10f690 0x7ff3cc0f5960 secure :-1 s=READY pgs=597 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3bc0098a0 tx=0x7ff3bc0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:32.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3ba7fc640 1 -- 10.20.193.28:0/869577640 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3bc03d070 con 0x7ff3cc10f690 2026-02-25T11:52:32.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3ba7fc640 1 -- 10.20.193.28:0/869577640 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff3bc002a70 con 0x7ff3cc10f690 2026-02-25T11:52:32.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3d255e640 1 -- 10.20.193.28:0/869577640 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3cc0fb820 con 0x7ff3cc10f690 2026-02-25T11:52:32.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3d255e640 1 -- 10.20.193.28:0/869577640 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3cc0fbc90 con 0x7ff3cc10f690 2026-02-25T11:52:32.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.423+0000 7ff3ba7fc640 1 -- 10.20.193.28:0/869577640 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3bc041890 con 0x7ff3cc10f690 2026-02-25T11:52:32.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.428+0000 7ff3ba7fc640 1 -- 10.20.193.28:0/869577640 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7ff3bc041a20 con 0x7ff3cc10f690 2026-02-25T11:52:32.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.428+0000 7ff3d57e2e00 1 Processor -- start 2026-02-25T11:52:32.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d57e2e00 1 -- start start 2026-02-25T11:52:32.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d57e2e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e5b30 0x55cad02e5f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d57e2e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e6440 0x55cad032aba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d57e2e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 0x55cad034b4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d57e2e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55cad01fb8a0 con 0x55cad02e5b30 2026-02-25T11:52:32.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d57e2e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55cad01f75d0 con 0x55cad032b0e0 2026-02-25T11:52:32.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d57e2e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55cad01f1570 con 0x55cad02e6440 2026-02-25T11:52:32.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d1d5d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e5b30 0x55cad02e5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d1d5d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e5b30 0x55cad02e5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58460/0 (socket says 10.20.193.28:58460) 2026-02-25T11:52:32.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d1d5d640 1 -- 10.20.193.28:0/2049006788 learned_addr learned my addr 10.20.193.28:0/2049006788 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:32.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d155c640 1 --2- 10.20.193.28:0/2049006788 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e6440 0x55cad032aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d1d5d640 1 -- 10.20.193.28:0/2049006788 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e6440 msgr2=0x55cad032aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:32.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d1d5d640 1 --2- 10.20.193.28:0/2049006788 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e6440 0x55cad032aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d1d5d640 1 -- 10.20.193.28:0/2049006788 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 msgr2=0x55cad034b4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-25T11:52:32.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d1d5d640 1 --2- 10.20.193.28:0/2049006788 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 0x55cad034b4c0 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3ba7fc640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff3a80b56d0 0x7ff3a80d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.429+0000 7ff3d1d5d640 1 -- 10.20.193.28:0/2049006788 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad019cd10 con 0x55cad02e5b30 2026-02-25T11:52:32.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.430+0000 7ff3ba7fc640 1 -- 10.20.193.28:0/869577640 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ff3bc10b2d0 con 0x7ff3cc10f690 2026-02-25T11:52:32.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.430+0000 7ff3d1d5d640 1 --2- 10.20.193.28:0/2049006788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e5b30 0x55cad02e5f00 secure :-1 s=READY pgs=598 gs=1 cs=0 l=1 c_cookie=218275568889561788 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3c8003a90 tx=0x7ff3c8003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307787932cf02bc server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:32.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.430+0000 7ff3d1d5d640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff3a80b56d0 0x7ff3a80d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.430+0000 7ff399ce5640 1 -- 10.20.193.28:0/2049006788 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3c8012660 con 0x55cad02e5b30 2026-02-25T11:52:32.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.430+0000 7ff399ce5640 1 -- 10.20.193.28:0/2049006788 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff3c8012d00 con 0x55cad02e5b30 2026-02-25T11:52:32.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.430+0000 7ff399ce5640 1 -- 10.20.193.28:0/2049006788 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3c80113d0 con 0x55cad02e5b30 2026-02-25T11:52:32.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.431+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/2049006788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e5b30 msgr2=0x55cad02e5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:32.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.431+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/2049006788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e5b30 0x55cad02e5f00 secure :-1 s=READY pgs=598 gs=1 cs=0 l=1 c_cookie=218275568889561788 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3c8003a90 tx=0x7ff3c8003f60 comp rx=0 tx=0).stop 2026-02-25T11:52:32.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.431+0000 7ff3d1d5d640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff3a80b56d0 0x7ff3a80d5b80 secure :-1 s=READY pgs=314 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3c80115c0 tx=0x7ff3c8011a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:32.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.431+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/2049006788 shutdown_connections 2026-02-25T11:52:32.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.431+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/2049006788 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 0x55cad034b4c0 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.431+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/2049006788 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e6440 0x55cad032aba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.431+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/2049006788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e5b30 0x55cad02e5f00 unknown :-1 s=CLOSED pgs=598 gs=1 cs=0 l=1 c_cookie=218275568889561788 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.431+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/2049006788 >> 10.20.193.28:0/2049006788 conn(0x55cad02047a0 msgr2=0x55cad02e4410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:32.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.431+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/2049006788 shutdown_connections 2026-02-25T11:52:32.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.431+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/2049006788 wait complete. 2026-02-25T11:52:32.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.432+0000 7ff3d57e2e00 1 -- start start 2026-02-25T11:52:32.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.432+0000 7ff3d57e2e00 1 Processor -- start 2026-02-25T11:52:32.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d57e2e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e5b30 0x55cad0306810 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d57e2e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 0x55cad0306d50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d57e2e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 0x55cad030be90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d57e2e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55cad030c3d0 con 0x55cad02e6440 2026-02-25T11:52:32.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d57e2e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55cad030c540 con 0x55cad032b0e0 2026-02-25T11:52:32.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d57e2e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55cad030c700 con 0x55cad02e5b30 2026-02-25T11:52:32.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d0d5b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 0x55cad0306d50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d0d5b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 0x55cad0306d50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58476/0 (socket says 10.20.193.28:58476) 2026-02-25T11:52:32.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d0d5b640 1 -- 10.20.193.28:0/4208497098 learned_addr learned my addr 10.20.193.28:0/4208497098 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:32.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d1d5d640 1 --2- 10.20.193.28:0/4208497098 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 0x55cad030be90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d155c640 1 --2- 10.20.193.28:0/4208497098 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e5b30 0x55cad0306810 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d0d5b640 1 -- 10.20.193.28:0/4208497098 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e5b30 msgr2=0x55cad0306810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:32.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d0d5b640 1 --2- 10.20.193.28:0/4208497098 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e5b30 0x55cad0306810 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d0d5b640 1 -- 10.20.193.28:0/4208497098 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 msgr2=0x55cad030be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:32.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d0d5b640 1 --2- 10.20.193.28:0/4208497098 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 0x55cad030be90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d0d5b640 1 -- 10.20.193.28:0/4208497098 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad01753c0 con 0x55cad02e6440 2026-02-25T11:52:32.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff3d0d5b640 1 --2- 10.20.193.28:0/4208497098 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 0x55cad0306d50 secure :-1 s=READY pgs=599 gs=1 cs=0 l=1 c_cookie=14811497195792594036 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3c0005970 tx=0x7ff3c000eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8d023992e97474 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:32.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.433+0000 7ff386ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:52:32.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.434+0000 7ff386ffd640 1 -- 10.20.193.28:0/4208497098 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3c0004500 con 0x55cad02e6440 2026-02-25T11:52:32.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.434+0000 7ff386ffd640 1 -- 10.20.193.28:0/4208497098 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff3c0002b60 con 0x55cad02e6440 2026-02-25T11:52:32.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.434+0000 7ff386ffd640 1 -- 10.20.193.28:0/4208497098 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3c000f420 con 0x55cad02e6440 2026-02-25T11:52:32.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.434+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/4208497098 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c80055f0 con 0x55cad02e6440 2026-02-25T11:52:32.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.434+0000 7ff386ffd640 1 -- 10.20.193.28:0/4208497098 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ff3c000f5b0 con 0x55cad02e6440 2026-02-25T11:52:32.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.434+0000 7ff3d57e2e00 10 client.16398 _ll_register_callbacks cb 0x55cad0172a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:52:32.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.434+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/4208497098 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cacfe876f0 con 0x55cad02e6440 2026-02-25T11:52:32.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.434+0000 7ff386ffd640 1 -- 10.20.193.28:0/4208497098 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff3c0002cd0 con 0x55cad02e6440 2026-02-25T11:52:32.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.434+0000 7ff3d57e2e00 10 client.16398 fetch_fsmap learned FSMap version 23 2026-02-25T11:52:32.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.434+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/4208497098 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55cad01870e0 con 0x55cad02e6440 2026-02-25T11:52:32.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.434+0000 7ff386ffd640 1 -- 10.20.193.28:0/4208497098 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7ff3c00026a0 con 0x55cad02e6440 2026-02-25T11:52:32.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.435+0000 7ff3d57e2e00 10 client.16398 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:52:32.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.435+0000 7ff3d57e2e00 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:52:32.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.435+0000 7ff3d57e2e00 20 client.16398 populate_metadata read hostname 'trial028' 2026-02-25T11:52:32.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.435+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/4208497098 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cad0311190 0x55cad0311630 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.435+0000 7ff3d57e2e00 4 client.16398 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:52:32.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.435+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/4208497098 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cad01c3fc0 con 0x55cad0311190 2026-02-25T11:52:32.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.435+0000 7ff3d155c640 1 --2- 10.20.193.28:0/4208497098 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cad0311190 0x55cad0311630 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.435+0000 7ff3d155c640 1 --2- 10.20.193.28:0/4208497098 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cad0311190 0x55cad0311630 crc :-1 s=READY pgs=92 gs=4 cs=0 l=0 c_cookie=10494812449169132799 s_cookie=5477596357918143769 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91a510519112ccff server_cookie=4c0454ddb0af3519 in_seq=0 out_seq=0 2026-02-25T11:52:32.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.435+0000 7ff386ffd640 10 client.16398 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:32.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.436+0000 7ff386ffd640 1 -- 10.20.193.28:0/4208497098 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55cad01c3fc0 con 0x55cad0311190 2026-02-25T11:52:32.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.436+0000 7ff386ffd640 10 client.16398 handle_command_reply: tid=0 2026-02-25T11:52:32.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.436+0000 7ff3d57e2e00 1 client.16398 shutdown 2026-02-25T11:52:32.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.436+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/4208497098 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 msgr2=0x55cad0306d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:32.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.436+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/4208497098 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 0x55cad0306d50 secure :-1 s=READY pgs=599 gs=1 cs=0 l=1 c_cookie=14811497195792594036 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3c0005970 tx=0x7ff3c000eaf0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.437+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/4208497098 shutdown_connections 2026-02-25T11:52:32.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.437+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/4208497098 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cad0311190 0x55cad0311630 crc :-1 s=READY pgs=92 gs=4 cs=0 l=0 c_cookie=10494812449169132799 s_cookie=5477596357918143769 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.437+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/4208497098 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 0x55cad030be90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.437+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/4208497098 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 0x55cad0306d50 unknown :-1 s=CLOSED pgs=599 gs=1 cs=0 l=1 c_cookie=14811497195792594036 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.437+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/4208497098 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e5b30 0x55cad0306810 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.437+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/4208497098 >> 10.20.193.28:0/4208497098 conn(0x55cad02047a0 msgr2=0x55cad02e4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:32.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.437+0000 7ff386ffd640 0 client.16398 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:32.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.437+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/4208497098 shutdown_connections 2026-02-25T11:52:32.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.437+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/4208497098 wait complete. 2026-02-25T11:52:32.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.437+0000 7ff3d57e2e00 20 client.16398 trim_cache size 0 max 16384 2026-02-25T11:52:32.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.444+0000 7ff3d57e2e00 1 Processor -- start 2026-02-25T11:52:32.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.444+0000 7ff3d57e2e00 1 -- start start 2026-02-25T11:52:32.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.444+0000 7ff3d57e2e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e5b30 0x55cad02e4d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.444+0000 7ff3d57e2e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 0x55cad02e5260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.444+0000 7ff3d57e2e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 0x55cad0303ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.444+0000 7ff3d57e2e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55cad01ef1a0 con 0x55cad02e6440 2026-02-25T11:52:32.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.444+0000 7ff3d57e2e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55cacfeaf700 con 0x55cad032b0e0 2026-02-25T11:52:32.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.444+0000 7ff3d57e2e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55cad02c0e80 con 0x55cad02e5b30 2026-02-25T11:52:32.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.444+0000 7ff3d0d5b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 0x55cad02e5260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.444+0000 7ff3d0d5b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 0x55cad02e5260 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58490/0 (socket says 10.20.193.28:58490) 2026-02-25T11:52:32.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.444+0000 7ff3d0d5b640 1 -- 10.20.193.28:0/3058739056 learned_addr learned my addr 10.20.193.28:0/3058739056 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:32.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.444+0000 7ff3d1d5d640 1 --2- 10.20.193.28:0/3058739056 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 0x55cad0303ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.445+0000 7ff3d155c640 1 --2- 10.20.193.28:0/3058739056 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e5b30 0x55cad02e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.445+0000 7ff3d0d5b640 1 -- 10.20.193.28:0/3058739056 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e5b30 msgr2=0x55cad02e4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:32.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.445+0000 7ff3d0d5b640 1 --2- 10.20.193.28:0/3058739056 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e5b30 0x55cad02e4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.445+0000 7ff3d0d5b640 1 -- 10.20.193.28:0/3058739056 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 msgr2=0x55cad0303ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:32.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.445+0000 7ff3d0d5b640 1 --2- 10.20.193.28:0/3058739056 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 0x55cad0303ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.445+0000 7ff3d0d5b640 1 -- 10.20.193.28:0/3058739056 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad02f07c0 con 0x55cad02e6440 2026-02-25T11:52:32.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.445+0000 7ff3d1d5d640 1 --2- 10.20.193.28:0/3058739056 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 0x55cad0303ca0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:52:32.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.445+0000 7ff3d0d5b640 1 --2- 10.20.193.28:0/3058739056 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 0x55cad02e5260 secure :-1 s=READY pgs=600 gs=1 cs=0 l=1 c_cookie=13442070875470422134 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3c000fe20 tx=0x7ff3c001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8bd43dfa09e076 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:32.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.445+0000 7ff3994e4640 1 -- 10.20.193.28:0/3058739056 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3c000f420 con 0x55cad02e6440 2026-02-25T11:52:32.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.445+0000 7ff3994e4640 1 -- 10.20.193.28:0/3058739056 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55cacfe876f0 con 0x55cad02e6440 2026-02-25T11:52:32.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.445+0000 7ff3994e4640 1 -- 10.20.193.28:0/3058739056 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3c0002780 con 0x55cad02e6440 2026-02-25T11:52:32.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.445+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/3058739056 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 msgr2=0x55cad02e5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:32.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.445+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/3058739056 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 0x55cad02e5260 secure :-1 s=READY pgs=600 gs=1 cs=0 l=1 c_cookie=13442070875470422134 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3c000fe20 tx=0x7ff3c001c000 comp rx=0 tx=0).stop 2026-02-25T11:52:32.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.446+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/3058739056 shutdown_connections 2026-02-25T11:52:32.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.446+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/3058739056 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad032b0e0 0x55cad0303ca0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.446+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/3058739056 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad02e6440 0x55cad02e5260 unknown :-1 s=CLOSED pgs=600 gs=1 cs=0 l=1 c_cookie=13442070875470422134 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.446+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/3058739056 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e5b30 0x55cad02e4d20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.446+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/3058739056 >> 10.20.193.28:0/3058739056 conn(0x55cad02047a0 msgr2=0x55cad02e89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:32.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.446+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/3058739056 shutdown_connections 2026-02-25T11:52:32.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.446+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/3058739056 wait complete. 2026-02-25T11:52:32.454 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.446+0000 7ff3d57e2e00 1 -- start start 2026-02-25T11:52:32.454 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.446+0000 7ff3d57e2e00 1 Processor -- start 2026-02-25T11:52:32.455 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d57e2e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad02e5b30 0x55cad0317990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.455 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d57e2e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e6440 0x55cad0311fd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.455 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d57e2e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad032b0e0 0x55cad0312510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.455 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d57e2e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55cad0312b70 con 0x55cad032b0e0 2026-02-25T11:52:32.455 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d57e2e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55cad0312ce0 con 0x55cad02e5b30 2026-02-25T11:52:32.455 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d57e2e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55cad0312ea0 con 0x55cad02e6440 2026-02-25T11:52:32.455 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d1d5d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad032b0e0 0x55cad0312510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.455 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d1d5d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad032b0e0 0x55cad0312510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58506/0 (socket says 10.20.193.28:58506) 2026-02-25T11:52:32.455 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d1d5d640 1 -- 10.20.193.28:0/1256990801 learned_addr learned my addr 10.20.193.28:0/1256990801 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:32.455 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d155c640 1 --2- 10.20.193.28:0/1256990801 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad02e5b30 0x55cad0317990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.455 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d0d5b640 1 --2- 10.20.193.28:0/1256990801 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e6440 0x55cad0311fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d1d5d640 1 -- 10.20.193.28:0/1256990801 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e6440 msgr2=0x55cad0311fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:32.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d1d5d640 1 --2- 10.20.193.28:0/1256990801 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e6440 0x55cad0311fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d1d5d640 1 -- 10.20.193.28:0/1256990801 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad02e5b30 msgr2=0x55cad0317990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:32.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d1d5d640 1 --2- 10.20.193.28:0/1256990801 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad02e5b30 0x55cad0317990 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.447+0000 7ff3d1d5d640 1 -- 10.20.193.28:0/1256990801 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad01870e0 con 0x55cad032b0e0 2026-02-25T11:52:32.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.448+0000 7ff386ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:52:32.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.448+0000 7ff3d1d5d640 1 --2- 10.20.193.28:0/1256990801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad032b0e0 0x55cad0312510 secure :-1 s=READY pgs=601 gs=1 cs=0 l=1 c_cookie=15641583289209210288 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3c80109e0 tx=0x7ff3c8024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9121120fe0045b0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:32.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.448+0000 7ff386ffd640 1 -- 10.20.193.28:0/1256990801 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3c8038070 con 0x55cad032b0e0 2026-02-25T11:52:32.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.448+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/1256990801 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c8012e70 con 0x55cad032b0e0 2026-02-25T11:52:32.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.448+0000 7ff3d57e2e00 10 client.16404 _ll_register_callbacks cb 0x55cad0172a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:52:32.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.448+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/1256990801 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3c8012d00 con 0x55cad032b0e0 2026-02-25T11:52:32.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.448+0000 7ff386ffd640 1 -- 10.20.193.28:0/1256990801 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff3c802e050 con 0x55cad032b0e0 2026-02-25T11:52:32.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.448+0000 7ff386ffd640 1 -- 10.20.193.28:0/1256990801 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3c801dc40 con 0x55cad032b0e0 2026-02-25T11:52:32.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.448+0000 7ff386ffd640 1 -- 10.20.193.28:0/1256990801 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ff3c801dda0 con 0x55cad032b0e0 2026-02-25T11:52:32.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.448+0000 7ff386ffd640 1 -- 10.20.193.28:0/1256990801 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff3c8034470 con 0x55cad032b0e0 2026-02-25T11:52:32.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.448+0000 7ff3d57e2e00 10 client.16404 fetch_fsmap learned FSMap version 23 2026-02-25T11:52:32.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.448+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/1256990801 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55cad0181b60 con 0x55cad032b0e0 2026-02-25T11:52:32.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.448+0000 7ff386ffd640 1 -- 10.20.193.28:0/1256990801 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7ff3c801a040 con 0x55cad032b0e0 2026-02-25T11:52:32.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.449+0000 7ff3d57e2e00 10 client.16404 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:52:32.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.449+0000 7ff3d57e2e00 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:52:32.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.449+0000 7ff3d57e2e00 20 client.16404 populate_metadata read hostname 'trial028' 2026-02-25T11:52:32.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.449+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/1256990801 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cad03e1280 0x55cad0401730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:32.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.449+0000 7ff3d57e2e00 4 client.16404 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:52:32.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.449+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/1256990801 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cacfe84f40 con 0x55cad03e1280 2026-02-25T11:52:32.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.449+0000 7ff3d155c640 1 --2- 10.20.193.28:0/1256990801 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cad03e1280 0x55cad0401730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:32.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.450+0000 7ff3d155c640 1 --2- 10.20.193.28:0/1256990801 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cad03e1280 0x55cad0401730 crc :-1 s=READY pgs=93 gs=4 cs=0 l=0 c_cookie=1635861057910993238 s_cookie=1001751794923799847 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16b3bef70aadf156 server_cookie=de6eff3318b0527 in_seq=0 out_seq=0 2026-02-25T11:52:32.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.450+0000 7ff386ffd640 10 client.16404 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:32.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.450+0000 7ff386ffd640 1 -- 10.20.193.28:0/1256990801 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cacfe84f40 con 0x55cad03e1280 2026-02-25T11:52:32.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.450+0000 7ff386ffd640 10 client.16404 handle_command_reply: tid=0 2026-02-25T11:52:32.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.450+0000 7ff3d57e2e00 1 client.16404 shutdown 2026-02-25T11:52:32.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.450+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/1256990801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad032b0e0 msgr2=0x55cad0312510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:32.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.450+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/1256990801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad032b0e0 0x55cad0312510 secure :-1 s=READY pgs=601 gs=1 cs=0 l=1 c_cookie=15641583289209210288 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3c80109e0 tx=0x7ff3c8024000 comp rx=0 tx=0).stop 2026-02-25T11:52:32.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.451+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/1256990801 shutdown_connections 2026-02-25T11:52:32.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.451+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/1256990801 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cad03e1280 0x55cad0401730 crc :-1 s=READY pgs=93 gs=4 cs=0 l=0 c_cookie=1635861057910993238 s_cookie=1001751794923799847 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.451+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/1256990801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cad032b0e0 0x55cad0312510 unknown :-1 s=CLOSED pgs=601 gs=1 cs=0 l=1 c_cookie=15641583289209210288 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.451+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/1256990801 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cad02e6440 0x55cad0311fd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.451+0000 7ff3d57e2e00 1 --2- 10.20.193.28:0/1256990801 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cad02e5b30 0x55cad0317990 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.451+0000 7ff386ffd640 0 client.16404 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:32.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.451+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/1256990801 >> 10.20.193.28:0/1256990801 conn(0x55cad02047a0 msgr2=0x55cad030acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:32.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.451+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/1256990801 shutdown_connections 2026-02-25T11:52:32.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.451+0000 7ff3d57e2e00 1 -- 10.20.193.28:0/1256990801 wait complete. 2026-02-25T11:52:32.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.451+0000 7ff3d57e2e00 20 client.16404 trim_cache size 0 max 16384 2026-02-25T11:52:32.460 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:52:32.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.452+0000 7ff3d255e640 1 -- 10.20.193.28:0/869577640 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff3a80b56d0 msgr2=0x7ff3a80d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:32.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.452+0000 7ff3d255e640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff3a80b56d0 0x7ff3a80d5b80 secure :-1 s=READY pgs=314 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3c80115c0 tx=0x7ff3c8011a90 comp rx=0 tx=0).stop 2026-02-25T11:52:32.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.452+0000 7ff3d255e640 1 -- 10.20.193.28:0/869577640 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3cc10f690 msgr2=0x7ff3cc0f5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:32.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.452+0000 7ff3d255e640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3cc10f690 0x7ff3cc0f5960 secure :-1 s=READY pgs=597 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3bc0098a0 tx=0x7ff3bc0028d0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.452+0000 7ff3d255e640 1 -- 10.20.193.28:0/869577640 shutdown_connections 2026-02-25T11:52:32.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.452+0000 7ff3d255e640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff3a80b56d0 0x7ff3a80d5b80 unknown :-1 s=CLOSED pgs=314 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.452+0000 7ff3d255e640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff3cc115880 0x7ff3cc0f6060 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.452+0000 7ff3d255e640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff3cc110080 0x7ff3cc176150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.452+0000 7ff3d255e640 1 --2- 10.20.193.28:0/869577640 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3cc10f690 0x7ff3cc0f5960 unknown :-1 s=CLOSED pgs=597 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:32.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.452+0000 7ff3d255e640 1 -- 10.20.193.28:0/869577640 >> 10.20.193.28:0/869577640 conn(0x7ff3cc0ecf00 msgr2=0x7ff3cc111b30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:32.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.452+0000 7ff3d255e640 1 -- 10.20.193.28:0/869577640 shutdown_connections 2026-02-25T11:52:32.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:32.452+0000 7ff3d255e640 1 -- 10.20.193.28:0/869577640 wait complete. 2026-02-25T11:52:32.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:32 trial167 ceph-mon[43392]: pgmap v355: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:32.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:32 trial119 ceph-mon[42710]: pgmap v355: 97 pgs: 97 active+clean; 3.5 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:33.550 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:52:33.551 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:52:33.738 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:52:33.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.926+0000 7ff31ffff640 1 -- 10.20.193.28:0/857200247 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff32010ddc0 msgr2=0x7ff320112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:33.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.926+0000 7ff31ffff640 1 --2- 10.20.193.28:0/857200247 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff32010ddc0 0x7ff320112e10 secure :-1 s=READY pgs=139 gs=1 cs=0 l=1 c_cookie=4092628918269241758 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff308009950 tx=0x7ff30802f190 comp rx=0 tx=0).stop 2026-02-25T11:52:33.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.926+0000 7ff31ffff640 1 -- 10.20.193.28:0/857200247 shutdown_connections 2026-02-25T11:52:33.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.926+0000 7ff31ffff640 1 --2- 10.20.193.28:0/857200247 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff3201135c0 0x7ff320173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.926+0000 7ff31ffff640 1 --2- 10.20.193.28:0/857200247 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff32010ddc0 0x7ff320112e10 unknown :-1 s=CLOSED pgs=139 gs=1 cs=0 l=1 c_cookie=4092628918269241758 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.926+0000 7ff31ffff640 1 --2- 10.20.193.28:0/857200247 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff32010d3d0 0x7ff32010d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.926+0000 7ff31ffff640 1 -- 10.20.193.28:0/857200247 >> 10.20.193.28:0/857200247 conn(0x7ff3200eaca0 msgr2=0x7ff32010b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:33.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.926+0000 7ff31ffff640 1 -- 10.20.193.28:0/857200247 shutdown_connections 2026-02-25T11:52:33.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.926+0000 7ff31ffff640 1 -- 10.20.193.28:0/857200247 wait complete. 2026-02-25T11:52:33.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.927+0000 7ff31ffff640 1 Processor -- start 2026-02-25T11:52:33.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.927+0000 7ff31ffff640 1 -- start start 2026-02-25T11:52:33.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.927+0000 7ff31ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff32010d3d0 0x7ff320062970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.927+0000 7ff31ffff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff32010ddc0 0x7ff320173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.927+0000 7ff31ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3201135c0 0x7ff320060ef0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.927+0000 7ff31ffff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff3200614f0 con 0x7ff3201135c0 2026-02-25T11:52:33.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.927+0000 7ff31ffff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff320061660 con 0x7ff32010ddc0 2026-02-25T11:52:33.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.927+0000 7ff31ffff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff320061820 con 0x7ff32010d3d0 2026-02-25T11:52:33.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.927+0000 7ff31e7fc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff32010ddc0 0x7ff320173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.927+0000 7ff31e7fc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff32010ddc0 0x7ff320173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43334/0 (socket says 10.20.193.28:43334) 2026-02-25T11:52:33.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.927+0000 7ff31f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3201135c0 0x7ff320060ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.927+0000 7ff31e7fc640 1 -- 10.20.193.28:0/518880493 learned_addr learned my addr 10.20.193.28:0/518880493 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:33.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.928+0000 7ff31effd640 1 --2- 10.20.193.28:0/518880493 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff32010d3d0 0x7ff320062970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.928+0000 7ff31f7fe640 1 -- 10.20.193.28:0/518880493 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff32010d3d0 msgr2=0x7ff320062970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:33.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.928+0000 7ff31f7fe640 1 --2- 10.20.193.28:0/518880493 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff32010d3d0 0x7ff320062970 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.928+0000 7ff31f7fe640 1 -- 10.20.193.28:0/518880493 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff32010ddc0 msgr2=0x7ff320173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:33.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.928+0000 7ff31f7fe640 1 --2- 10.20.193.28:0/518880493 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff32010ddc0 0x7ff320173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.928+0000 7ff31f7fe640 1 -- 10.20.193.28:0/518880493 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff308009570 con 0x7ff3201135c0 2026-02-25T11:52:33.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.928+0000 7ff31f7fe640 1 --2- 10.20.193.28:0/518880493 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3201135c0 0x7ff320060ef0 secure :-1 s=READY pgs=602 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff31000f940 tx=0x7ff31000fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:33.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.928+0000 7ff2fffff640 1 -- 10.20.193.28:0/518880493 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff31000dce0 con 0x7ff3201135c0 2026-02-25T11:52:33.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.928+0000 7ff31ffff640 1 -- 10.20.193.28:0/518880493 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff320061aa0 con 0x7ff3201135c0 2026-02-25T11:52:33.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.928+0000 7ff31ffff640 1 -- 10.20.193.28:0/518880493 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff320100500 con 0x7ff3201135c0 2026-02-25T11:52:33.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.928+0000 7ff2fffff640 1 -- 10.20.193.28:0/518880493 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff31000de40 con 0x7ff3201135c0 2026-02-25T11:52:33.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.928+0000 7ff2fffff640 1 -- 10.20.193.28:0/518880493 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3100116b0 con 0x7ff3201135c0 2026-02-25T11:52:33.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.931+0000 7ff2fffff640 1 -- 10.20.193.28:0/518880493 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7ff310020060 con 0x7ff3201135c0 2026-02-25T11:52:33.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.933+0000 7ff2fffff640 1 --2- 10.20.193.28:0/518880493 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff2f40b57e0 0x7ff2f40d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.933+0000 7ff2fffff640 1 -- 10.20.193.28:0/518880493 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ff310015070 con 0x7ff3201135c0 2026-02-25T11:52:33.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.933+0000 7ff31effd640 1 --2- 10.20.193.28:0/518880493 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff2f40b57e0 0x7ff2f40d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.934+0000 7ff32750ee00 1 Processor -- start 2026-02-25T11:52:33.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.934+0000 7ff32750ee00 1 -- start start 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.934+0000 7ff31effd640 1 --2- 10.20.193.28:0/518880493 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff2f40b57e0 0x7ff2f40d5c90 secure :-1 s=READY pgs=315 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff314004740 tx=0x7ff314005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.934+0000 7ff32750ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 0x55bad4bceb20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.934+0000 7ff32750ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bcf060 0x55bad4c13610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.934+0000 7ff32750ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4c33f30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.934+0000 7ff32750ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55bad4ae4d80 con 0x55bad4c13b50 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.934+0000 7ff32750ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55bad4ae0ab0 con 0x55bad4bcf060 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.934+0000 7ff32750ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55bad4adaa50 con 0x55bad4bce750 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.934+0000 7ff31f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4c33f30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.934+0000 7ff31f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4c33f30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58540/0 (socket says 10.20.193.28:58540) 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.934+0000 7ff31f7fe640 1 -- 10.20.193.28:0/119428348 learned_addr learned my addr 10.20.193.28:0/119428348 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.934+0000 7ff31e7fc640 1 --2- 10.20.193.28:0/119428348 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bcf060 0x55bad4c13610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff31f7fe640 1 -- 10.20.193.28:0/119428348 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 msgr2=0x55bad4bceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff31f7fe640 1 --2- 10.20.193.28:0/119428348 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 0x55bad4bceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff31f7fe640 1 -- 10.20.193.28:0/119428348 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bcf060 msgr2=0x55bad4c13610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff31f7fe640 1 --2- 10.20.193.28:0/119428348 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bcf060 0x55bad4c13610 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff31f7fe640 1 -- 10.20.193.28:0/119428348 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad4a86120 con 0x55bad4c13b50 2026-02-25T11:52:33.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff31f7fe640 1 --2- 10.20.193.28:0/119428348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4c33f30 secure :-1 s=READY pgs=603 gs=2 cs=0 l=1 c_cookie=18153481396797671727 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3100aa4d0 tx=0x7ff3100adfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbee1f262cc9892f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:33.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff2eb7fe640 1 -- 10.20.193.28:0/119428348 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff310006140 con 0x55bad4c13b50 2026-02-25T11:52:33.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff2eb7fe640 1 -- 10.20.193.28:0/119428348 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff3100ae900 con 0x55bad4c13b50 2026-02-25T11:52:33.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff2eb7fe640 1 -- 10.20.193.28:0/119428348 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3100d30c0 con 0x55bad4c13b50 2026-02-25T11:52:33.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff32750ee00 1 -- 10.20.193.28:0/119428348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 msgr2=0x55bad4c33f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:33.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff32750ee00 1 --2- 10.20.193.28:0/119428348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4c33f30 secure :-1 s=READY pgs=603 gs=2 cs=0 l=1 c_cookie=18153481396797671727 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3100aa4d0 tx=0x7ff3100adfb0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff32750ee00 1 -- 10.20.193.28:0/119428348 shutdown_connections 2026-02-25T11:52:33.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff32750ee00 1 --2- 10.20.193.28:0/119428348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4c33f30 unknown :-1 s=CLOSED pgs=603 gs=2 cs=0 l=1 c_cookie=18153481396797671727 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff32750ee00 1 --2- 10.20.193.28:0/119428348 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bcf060 0x55bad4c13610 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff32750ee00 1 --2- 10.20.193.28:0/119428348 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 0x55bad4bceb20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.935+0000 7ff32750ee00 1 -- 10.20.193.28:0/119428348 >> 10.20.193.28:0/119428348 conn(0x55bad4bac8f0 msgr2=0x55bad4bcccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:33.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.936+0000 7ff32750ee00 1 -- 10.20.193.28:0/119428348 shutdown_connections 2026-02-25T11:52:33.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.936+0000 7ff32750ee00 1 -- 10.20.193.28:0/119428348 wait complete. 2026-02-25T11:52:33.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.937+0000 7ff32750ee00 1 -- start start 2026-02-25T11:52:33.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.937+0000 7ff32750ee00 1 Processor -- start 2026-02-25T11:52:33.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.937+0000 7ff32750ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bce750 0x55bad4c2b4f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.937+0000 7ff32750ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bcf060 0x55bad4c2ba30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.937+0000 7ff32750ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4c30a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.937+0000 7ff32750ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55bad4c30f60 con 0x55bad4c13b50 2026-02-25T11:52:33.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.937+0000 7ff32750ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55bad4c310d0 con 0x55bad4bce750 2026-02-25T11:52:33.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.937+0000 7ff32750ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55bad4c31290 con 0x55bad4bcf060 2026-02-25T11:52:33.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff31f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4c30a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff31effd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bce750 0x55bad4c2b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff31effd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bce750 0x55bad4c2b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43338/0 (socket says 10.20.193.28:43338) 2026-02-25T11:52:33.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff31effd640 1 -- 10.20.193.28:0/2076884471 learned_addr learned my addr 10.20.193.28:0/2076884471 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff31e7fc640 1 --2- 10.20.193.28:0/2076884471 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bcf060 0x55bad4c2ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff31f7fe640 1 -- 10.20.193.28:0/2076884471 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bcf060 msgr2=0x55bad4c2ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff31f7fe640 1 --2- 10.20.193.28:0/2076884471 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bcf060 0x55bad4c2ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff31f7fe640 1 -- 10.20.193.28:0/2076884471 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bce750 msgr2=0x55bad4c2b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff31f7fe640 1 --2- 10.20.193.28:0/2076884471 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bce750 0x55bad4c2b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff31f7fe640 1 -- 10.20.193.28:0/2076884471 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad4a5e810 con 0x55bad4c13b50 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff31f7fe640 1 --2- 10.20.193.28:0/2076884471 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4c30a20 secure :-1 s=READY pgs=604 gs=2 cs=0 l=1 c_cookie=6540747230922408509 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3100a9f10 tx=0x7ff3100063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac566f3c0c0023d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff2e8ff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/2076884471 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3100aedb0 con 0x55bad4c13b50 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/2076884471 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff310006140 con 0x55bad4c13b50 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/2076884471 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3100b8e10 con 0x55bad4c13b50 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff32750ee00 1 -- 10.20.193.28:0/2076884471 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff320100500 con 0x55bad4c13b50 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff32750ee00 10 client.16416 _ll_register_callbacks cb 0x55bad4a5be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.938+0000 7ff32750ee00 1 -- 10.20.193.28:0/2076884471 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bad4832530 con 0x55bad4c13b50 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.939+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/2076884471 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ff3100adcb0 con 0x55bad4c13b50 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.939+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/2076884471 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff310073090 con 0x55bad4c13b50 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.939+0000 7ff32750ee00 10 client.16416 fetch_fsmap learned FSMap version 23 2026-02-25T11:52:33.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.939+0000 7ff32750ee00 1 -- 10.20.193.28:0/2076884471 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55bad4a704f0 con 0x55bad4c13b50 2026-02-25T11:52:33.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.939+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/2076884471 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7ff3100d1bc0 con 0x55bad4c13b50 2026-02-25T11:52:33.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.939+0000 7ff32750ee00 10 client.16416 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:52:33.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.939+0000 7ff32750ee00 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:52:33.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.939+0000 7ff32750ee00 20 client.16416 populate_metadata read hostname 'trial028' 2026-02-25T11:52:33.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.939+0000 7ff32750ee00 1 --2- 10.20.193.28:0/2076884471 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bad4c10b20 0x55bad4c10fc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.939+0000 7ff32750ee00 4 client.16416 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:52:33.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.939+0000 7ff32750ee00 1 -- 10.20.193.28:0/2076884471 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bad4a87b50 con 0x55bad4c10b20 2026-02-25T11:52:33.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.939+0000 7ff31effd640 1 --2- 10.20.193.28:0/2076884471 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bad4c10b20 0x55bad4c10fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.940+0000 7ff31effd640 1 --2- 10.20.193.28:0/2076884471 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bad4c10b20 0x55bad4c10fc0 crc :-1 s=READY pgs=94 gs=4 cs=0 l=0 c_cookie=16187721330014119943 s_cookie=4133306354304845873 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0a656c88e816807 server_cookie=395c743ae4348831 in_seq=0 out_seq=0 2026-02-25T11:52:33.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.940+0000 7ff2e8ff9640 10 client.16416 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:33.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.940+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/2076884471 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55bad4a87b50 con 0x55bad4c10b20 2026-02-25T11:52:33.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.940+0000 7ff2e8ff9640 10 client.16416 handle_command_reply: tid=0 2026-02-25T11:52:33.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.940+0000 7ff32750ee00 1 client.16416 shutdown 2026-02-25T11:52:33.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.940+0000 7ff32750ee00 1 -- 10.20.193.28:0/2076884471 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 msgr2=0x55bad4c30a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:33.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.940+0000 7ff32750ee00 1 --2- 10.20.193.28:0/2076884471 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4c30a20 secure :-1 s=READY pgs=604 gs=2 cs=0 l=1 c_cookie=6540747230922408509 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3100a9f10 tx=0x7ff3100063a0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.941+0000 7ff32750ee00 1 -- 10.20.193.28:0/2076884471 shutdown_connections 2026-02-25T11:52:33.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.941+0000 7ff32750ee00 1 --2- 10.20.193.28:0/2076884471 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bad4c10b20 0x55bad4c10fc0 crc :-1 s=READY pgs=94 gs=4 cs=0 l=0 c_cookie=16187721330014119943 s_cookie=4133306354304845873 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.941+0000 7ff32750ee00 1 --2- 10.20.193.28:0/2076884471 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4c30a20 unknown :-1 s=CLOSED pgs=604 gs=2 cs=0 l=1 c_cookie=6540747230922408509 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.941+0000 7ff32750ee00 1 --2- 10.20.193.28:0/2076884471 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bcf060 0x55bad4c2ba30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.941+0000 7ff32750ee00 1 --2- 10.20.193.28:0/2076884471 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bce750 0x55bad4c2b4f0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.941+0000 7ff2e8ff9640 0 client.16416 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:33.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.941+0000 7ff32750ee00 1 -- 10.20.193.28:0/2076884471 >> 10.20.193.28:0/2076884471 conn(0x55bad4bac8f0 msgr2=0x55bad4bcccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:33.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.941+0000 7ff32750ee00 1 -- 10.20.193.28:0/2076884471 shutdown_connections 2026-02-25T11:52:33.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.941+0000 7ff32750ee00 1 -- 10.20.193.28:0/2076884471 wait complete. 2026-02-25T11:52:33.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.941+0000 7ff32750ee00 20 client.16416 trim_cache size 0 max 16384 2026-02-25T11:52:33.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.951+0000 7ff32750ee00 1 Processor -- start 2026-02-25T11:52:33.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.951+0000 7ff32750ee00 1 -- start start 2026-02-25T11:52:33.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.951+0000 7ff32750ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 0x55bad4c28eb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.951+0000 7ff32750ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4bcf060 0x55bad4c128b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.951+0000 7ff32750ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4c13b50 0x55bad4c12df0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.951+0000 7ff32750ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55bad4c18fb0 con 0x55bad4bcf060 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.951+0000 7ff32750ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55bad4ad8680 con 0x55bad4c13b50 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.951+0000 7ff32750ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55bad4798790 con 0x55bad4bce750 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff31effd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 0x55bad4c28eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff31effd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 0x55bad4c28eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48266/0 (socket says 10.20.193.28:48266) 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff31effd640 1 -- 10.20.193.28:0/2777430019 learned_addr learned my addr 10.20.193.28:0/2777430019 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff31f7fe640 1 --2- 10.20.193.28:0/2777430019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4c13b50 0x55bad4c12df0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff31e7fc640 1 --2- 10.20.193.28:0/2777430019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4bcf060 0x55bad4c128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff31effd640 1 -- 10.20.193.28:0/2777430019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4c13b50 msgr2=0x55bad4c12df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff31effd640 1 --2- 10.20.193.28:0/2777430019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4c13b50 0x55bad4c12df0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff31effd640 1 -- 10.20.193.28:0/2777430019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4bcf060 msgr2=0x55bad4c128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff31effd640 1 --2- 10.20.193.28:0/2777430019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4bcf060 0x55bad4c128b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff31effd640 1 -- 10.20.193.28:0/2777430019 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad4a6af70 con 0x55bad4bce750 2026-02-25T11:52:33.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff31e7fc640 1 --2- 10.20.193.28:0/2777430019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4bcf060 0x55bad4c128b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:52:33.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff31effd640 1 --2- 10.20.193.28:0/2777430019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 0x55bad4c28eb0 secure :-1 s=READY pgs=168 gs=1 cs=0 l=1 c_cookie=12744936062657087132 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3140046c0 tx=0x7ff314013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0df1bd2cbbdc69c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:33.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff2eaffd640 1 -- 10.20.193.28:0/2777430019 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff314022040 con 0x55bad4bce750 2026-02-25T11:52:33.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff2eaffd640 1 -- 10.20.193.28:0/2777430019 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff31401dc30 con 0x55bad4bce750 2026-02-25T11:52:33.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.952+0000 7ff2eaffd640 1 -- 10.20.193.28:0/2777430019 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3140116c0 con 0x55bad4bce750 2026-02-25T11:52:33.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.953+0000 7ff32750ee00 1 -- 10.20.193.28:0/2777430019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 msgr2=0x55bad4c28eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:33.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.953+0000 7ff32750ee00 1 --2- 10.20.193.28:0/2777430019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 0x55bad4c28eb0 secure :-1 s=READY pgs=168 gs=1 cs=0 l=1 c_cookie=12744936062657087132 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3140046c0 tx=0x7ff314013000 comp rx=0 tx=0).stop 2026-02-25T11:52:33.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.953+0000 7ff32750ee00 1 -- 10.20.193.28:0/2777430019 shutdown_connections 2026-02-25T11:52:33.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.953+0000 7ff32750ee00 1 --2- 10.20.193.28:0/2777430019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4c13b50 0x55bad4c12df0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.953+0000 7ff32750ee00 1 --2- 10.20.193.28:0/2777430019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4bcf060 0x55bad4c128b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.953+0000 7ff32750ee00 1 --2- 10.20.193.28:0/2777430019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 0x55bad4c28eb0 unknown :-1 s=CLOSED pgs=168 gs=1 cs=0 l=1 c_cookie=12744936062657087132 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.953+0000 7ff32750ee00 1 -- 10.20.193.28:0/2777430019 >> 10.20.193.28:0/2777430019 conn(0x55bad4bac8f0 msgr2=0x55bad4c27790 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:33.955 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.953+0000 7ff32750ee00 1 -- 10.20.193.28:0/2777430019 shutdown_connections 2026-02-25T11:52:33.955 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.953+0000 7ff32750ee00 1 -- 10.20.193.28:0/2777430019 wait complete. 2026-02-25T11:52:33.955 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.953+0000 7ff32750ee00 1 -- start start 2026-02-25T11:52:33.955 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff32750ee00 1 Processor -- start 2026-02-25T11:52:33.955 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff32750ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 0x55bad4bae030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff32750ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bcf060 0x55bad4bae570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff32750ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4baeab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff32750ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55bad4c31150 con 0x55bad4c13b50 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff32750ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55bad4c18e70 con 0x55bad4bcf060 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff32750ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55bad4c79990 con 0x55bad4bce750 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff31f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4baeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff31f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4baeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58582/0 (socket says 10.20.193.28:58582) 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff31f7fe640 1 -- 10.20.193.28:0/4014757935 learned_addr learned my addr 10.20.193.28:0/4014757935 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff31effd640 1 --2- 10.20.193.28:0/4014757935 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 0x55bad4bae030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff31e7fc640 1 --2- 10.20.193.28:0/4014757935 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bcf060 0x55bad4bae570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff31f7fe640 1 -- 10.20.193.28:0/4014757935 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 msgr2=0x55bad4bae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff31f7fe640 1 --2- 10.20.193.28:0/4014757935 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 0x55bad4bae030 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff31f7fe640 1 -- 10.20.193.28:0/4014757935 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bcf060 msgr2=0x55bad4bae570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff31f7fe640 1 --2- 10.20.193.28:0/4014757935 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bcf060 0x55bad4bae570 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.954+0000 7ff31f7fe640 1 -- 10.20.193.28:0/4014757935 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad4a5eb40 con 0x55bad4c13b50 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.955+0000 7ff31f7fe640 1 --2- 10.20.193.28:0/4014757935 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4baeab0 secure :-1 s=READY pgs=605 gs=1 cs=0 l=1 c_cookie=3191584962407714589 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3100d3220 tx=0x7ff310006330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4ac99d12ca4f1d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:33.956 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.955+0000 7ff2e8ff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:52:33.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.955+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/4014757935 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff3100adcb0 con 0x55bad4c13b50 2026-02-25T11:52:33.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.955+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/4014757935 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55bad4832530 con 0x55bad4c13b50 2026-02-25T11:52:33.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.955+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/4014757935 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff31000c2c0 con 0x55bad4c13b50 2026-02-25T11:52:33.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.955+0000 7ff32750ee00 1 -- 10.20.193.28:0/4014757935 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff314009980 con 0x55bad4c13b50 2026-02-25T11:52:33.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.955+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/4014757935 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ff310073090 con 0x55bad4c13b50 2026-02-25T11:52:33.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.955+0000 7ff2e8ff9640 10 client.16419 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:52:33.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.955+0000 7ff32750ee00 10 client.16419 _ll_register_callbacks cb 0x55bad4a5be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:52:33.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.955+0000 7ff32750ee00 1 -- 10.20.193.28:0/4014757935 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bad476e220 con 0x55bad4c13b50 2026-02-25T11:52:33.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.955+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/4014757935 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff3100b9040 con 0x55bad4c13b50 2026-02-25T11:52:33.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.955+0000 7ff32750ee00 10 client.16419 fetch_fsmap learned FSMap version 23 2026-02-25T11:52:33.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.955+0000 7ff32750ee00 1 -- 10.20.193.28:0/4014757935 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55bad4a70750 con 0x55bad4c13b50 2026-02-25T11:52:33.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.956+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/4014757935 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7ff3100aeb90 con 0x55bad4c13b50 2026-02-25T11:52:33.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.956+0000 7ff32750ee00 10 client.16419 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:52:33.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.956+0000 7ff32750ee00 10 client.16419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:52:33.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.956+0000 7ff32750ee00 20 client.16419 populate_metadata read hostname 'trial028' 2026-02-25T11:52:33.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.956+0000 7ff32750ee00 1 --2- 10.20.193.28:0/4014757935 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bad4cc1190 0x55bad4ce1610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:33.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.956+0000 7ff32750ee00 4 client.16419 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:52:33.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.956+0000 7ff32750ee00 1 -- 10.20.193.28:0/4014757935 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7ff3100ae900 con 0x55bad4cc1190 2026-02-25T11:52:33.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.956+0000 7ff31effd640 1 --2- 10.20.193.28:0/4014757935 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bad4cc1190 0x55bad4ce1610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:33.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.956+0000 7ff31effd640 1 --2- 10.20.193.28:0/4014757935 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bad4cc1190 0x55bad4ce1610 crc :-1 s=READY pgs=95 gs=4 cs=0 l=0 c_cookie=11540808218816091296 s_cookie=13813241164383902398 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0292feddf7114a0 server_cookie=bfb27da73fe7dabe in_seq=0 out_seq=0 2026-02-25T11:52:33.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.956+0000 7ff2e8ff9640 10 client.16419 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.957+0000 7ff2e8ff9640 1 -- 10.20.193.28:0/4014757935 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7ff3100ae900 con 0x55bad4cc1190 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.957+0000 7ff2e8ff9640 10 client.16419 handle_command_reply: tid=0 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.957+0000 7ff32750ee00 1 client.16419 shutdown 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.957+0000 7ff32750ee00 1 -- 10.20.193.28:0/4014757935 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 msgr2=0x55bad4baeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.957+0000 7ff32750ee00 1 --2- 10.20.193.28:0/4014757935 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4baeab0 secure :-1 s=READY pgs=605 gs=1 cs=0 l=1 c_cookie=3191584962407714589 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff3100d3220 tx=0x7ff310006330 comp rx=0 tx=0).stop 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.957+0000 7ff32750ee00 1 -- 10.20.193.28:0/4014757935 shutdown_connections 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.957+0000 7ff32750ee00 1 --2- 10.20.193.28:0/4014757935 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55bad4cc1190 0x55bad4ce1610 crc :-1 s=READY pgs=95 gs=4 cs=0 l=0 c_cookie=11540808218816091296 s_cookie=13813241164383902398 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.957+0000 7ff32750ee00 1 --2- 10.20.193.28:0/4014757935 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55bad4c13b50 0x55bad4baeab0 unknown :-1 s=CLOSED pgs=605 gs=1 cs=0 l=1 c_cookie=3191584962407714589 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.957+0000 7ff32750ee00 1 --2- 10.20.193.28:0/4014757935 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55bad4bcf060 0x55bad4bae570 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.957+0000 7ff32750ee00 1 --2- 10.20.193.28:0/4014757935 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55bad4bce750 0x55bad4bae030 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.957+0000 7ff2e8ff9640 0 client.16419 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.957+0000 7ff32750ee00 1 -- 10.20.193.28:0/4014757935 >> 10.20.193.28:0/4014757935 conn(0x55bad4bac8f0 msgr2=0x55bad4c27f80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff32750ee00 1 -- 10.20.193.28:0/4014757935 shutdown_connections 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff32750ee00 1 -- 10.20.193.28:0/4014757935 wait complete. 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff32750ee00 20 client.16419 trim_cache size 0 max 16384 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:52:33.959 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "3c6fdafc-4d02-40d2-962e-24e4d16fa603", 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff31ffff640 1 -- 10.20.193.28:0/518880493 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff2f40b57e0 msgr2=0x7ff2f40d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff31ffff640 1 --2- 10.20.193.28:0/518880493 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff2f40b57e0 0x7ff2f40d5c90 secure :-1 s=READY pgs=315 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff314004740 tx=0x7ff314005f90 comp rx=0 tx=0).stop 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff31ffff640 1 -- 10.20.193.28:0/518880493 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3201135c0 msgr2=0x7ff320060ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff31ffff640 1 --2- 10.20.193.28:0/518880493 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3201135c0 0x7ff320060ef0 secure :-1 s=READY pgs=602 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff31000f940 tx=0x7ff31000fe10 comp rx=0 tx=0).stop 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff31ffff640 1 -- 10.20.193.28:0/518880493 shutdown_connections 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff31ffff640 1 --2- 10.20.193.28:0/518880493 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff2f40b57e0 0x7ff2f40d5c90 unknown :-1 s=CLOSED pgs=315 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff31ffff640 1 --2- 10.20.193.28:0/518880493 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff3201135c0 0x7ff320060ef0 unknown :-1 s=CLOSED pgs=602 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff31ffff640 1 --2- 10.20.193.28:0/518880493 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff32010ddc0 0x7ff320173e90 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff31ffff640 1 --2- 10.20.193.28:0/518880493 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff32010d3d0 0x7ff320062970 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff31ffff640 1 -- 10.20.193.28:0/518880493 >> 10.20.193.28:0/518880493 conn(0x7ff3200eaca0 msgr2=0x7ff32010fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff31ffff640 1 -- 10.20.193.28:0/518880493 shutdown_connections 2026-02-25T11:52:33.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:33.958+0000 7ff31ffff640 1 -- 10.20.193.28:0/518880493 wait complete. 2026-02-25T11:52:34.061 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:52:34.246 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:52:34.362 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:34 trial028 ceph-mon[39224]: pgmap v356: 97 pgs: 97 active+clean; 3.6 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:34.363 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:34 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:52:34.363 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:34 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:52:34.363 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:34 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:52:34.363 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:34 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:52:34.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.430+0000 7fb63a53d640 1 -- 10.20.193.28:0/4055126371 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb634110130 msgr2=0x7fb634115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:34.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.430+0000 7fb63a53d640 1 --2- 10.20.193.28:0/4055126371 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb634110130 0x7fb634115180 secure :-1 s=READY pgs=169 gs=1 cs=0 l=1 c_cookie=11523374430029998337 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb624009890 tx=0x7fb62402f120 comp rx=0 tx=0).stop 2026-02-25T11:52:34.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.430+0000 7fb63a53d640 1 -- 10.20.193.28:0/4055126371 shutdown_connections 2026-02-25T11:52:34.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.430+0000 7fb63a53d640 1 --2- 10.20.193.28:0/4055126371 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb634115930 0x7fb634176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.430+0000 7fb63a53d640 1 --2- 10.20.193.28:0/4055126371 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb634110130 0x7fb634115180 unknown :-1 s=CLOSED pgs=169 gs=1 cs=0 l=1 c_cookie=11523374430029998337 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.430+0000 7fb63a53d640 1 --2- 10.20.193.28:0/4055126371 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb63410f740 0x7fb63410fb10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.430+0000 7fb63a53d640 1 -- 10.20.193.28:0/4055126371 >> 10.20.193.28:0/4055126371 conn(0x7fb6340ecff0 msgr2=0x7fb63410d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:34.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.430+0000 7fb63a53d640 1 -- 10.20.193.28:0/4055126371 shutdown_connections 2026-02-25T11:52:34.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.430+0000 7fb63a53d640 1 -- 10.20.193.28:0/4055126371 wait complete. 2026-02-25T11:52:34.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.431+0000 7fb63a53d640 1 Processor -- start 2026-02-25T11:52:34.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.431+0000 7fb63a53d640 1 -- start start 2026-02-25T11:52:34.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb63a53d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb63410f740 0x7fb6340f5bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb63a53d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb634110130 0x7fb634176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb63a53d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb634115930 0x7fb6340f6250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb63a53d640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb6340fb4d0 con 0x7fb634115930 2026-02-25T11:52:34.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb63a53d640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb6340fb640 con 0x7fb63410f740 2026-02-25T11:52:34.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb63a53d640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb6340fb7b0 con 0x7fb634110130 2026-02-25T11:52:34.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb638ab3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb634115930 0x7fb6340f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb633fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb63410f740 0x7fb6340f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb6337fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb634110130 0x7fb634176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb633fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb63410f740 0x7fb6340f5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43370/0 (socket says 10.20.193.28:43370) 2026-02-25T11:52:34.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb633fff640 1 -- 10.20.193.28:0/1468455840 learned_addr learned my addr 10.20.193.28:0/1468455840 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:34.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb6337fe640 1 -- 10.20.193.28:0/1468455840 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb63410f740 msgr2=0x7fb6340f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:34.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb6337fe640 1 --2- 10.20.193.28:0/1468455840 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb63410f740 0x7fb6340f5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb6337fe640 1 -- 10.20.193.28:0/1468455840 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb634115930 msgr2=0x7fb6340f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:34.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb6337fe640 1 --2- 10.20.193.28:0/1468455840 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb634115930 0x7fb6340f6250 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb6337fe640 1 -- 10.20.193.28:0/1468455840 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6240094b0 con 0x7fb634110130 2026-02-25T11:52:34.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.432+0000 7fb638ab3640 1 --2- 10.20.193.28:0/1468455840 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb634115930 0x7fb6340f6250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:52:34.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.433+0000 7fb6337fe640 1 --2- 10.20.193.28:0/1468455840 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb634110130 0x7fb634176200 secure :-1 s=READY pgs=170 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb624009890 tx=0x7fb624004660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:34.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.433+0000 7fb6317fa640 1 -- 10.20.193.28:0/1468455840 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb62403d070 con 0x7fb634110130 2026-02-25T11:52:34.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.433+0000 7fb6317fa640 1 -- 10.20.193.28:0/1468455840 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb624005d40 con 0x7fb634110130 2026-02-25T11:52:34.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.433+0000 7fb6317fa640 1 -- 10.20.193.28:0/1468455840 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb6240417b0 con 0x7fb634110130 2026-02-25T11:52:34.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.433+0000 7fb63a53d640 1 -- 10.20.193.28:0/1468455840 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6340fba30 con 0x7fb634110130 2026-02-25T11:52:34.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.433+0000 7fb63a53d640 1 -- 10.20.193.28:0/1468455840 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6340fbe50 con 0x7fb634110130 2026-02-25T11:52:34.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.435+0000 7fb6317fa640 1 -- 10.20.193.28:0/1468455840 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fb624048020 con 0x7fb634110130 2026-02-25T11:52:34.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.437+0000 7fb6317fa640 1 --2- 10.20.193.28:0/1468455840 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb6080b5760 0x7fb6080d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.437+0000 7fb6317fa640 1 -- 10.20.193.28:0/1468455840 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb62410ab50 con 0x7fb634110130 2026-02-25T11:52:34.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.438+0000 7fb633fff640 1 --2- 10.20.193.28:0/1468455840 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb6080b5760 0x7fb6080d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.438+0000 7fb633fff640 1 --2- 10.20.193.28:0/1468455840 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb6080b5760 0x7fb6080d5c10 secure :-1 s=READY pgs=316 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb6200097c0 tx=0x7fb620009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:34.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.438+0000 7fb63c538e00 1 Processor -- start 2026-02-25T11:52:34.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb63c538e00 1 -- start start 2026-02-25T11:52:34.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb63c538e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9140 0x56231ddc9510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb63c538e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de0e1b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb63c538e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231de0e6f0 0x56231de2ead0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb63c538e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56231dcdf9b0 con 0x56231ddc9a50 2026-02-25T11:52:34.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb63c538e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56231dcdb6e0 con 0x56231de0e6f0 2026-02-25T11:52:34.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb63c538e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56231dcd5840 con 0x56231ddc9140 2026-02-25T11:52:34.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb633fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9140 0x56231ddc9510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb638ab3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231de0e6f0 0x56231de2ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb6337fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de0e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb638ab3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231de0e6f0 0x56231de2ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43378/0 (socket says 10.20.193.28:43378) 2026-02-25T11:52:34.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb638ab3640 1 -- 10.20.193.28:0/3096887357 learned_addr learned my addr 10.20.193.28:0/3096887357 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:34.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb6337fe640 1 -- 10.20.193.28:0/3096887357 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9140 msgr2=0x56231ddc9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:34.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb6337fe640 1 --2- 10.20.193.28:0/3096887357 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9140 0x56231ddc9510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb6337fe640 1 -- 10.20.193.28:0/3096887357 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231de0e6f0 msgr2=0x56231de2ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:34.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb6337fe640 1 --2- 10.20.193.28:0/3096887357 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231de0e6f0 0x56231de2ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.439+0000 7fb6337fe640 1 -- 10.20.193.28:0/3096887357 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231dc80d80 con 0x56231ddc9a50 2026-02-25T11:52:34.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.440+0000 7fb6337fe640 1 --2- 10.20.193.28:0/3096887357 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de0e1b0 secure :-1 s=READY pgs=606 gs=2 cs=0 l=1 c_cookie=13429850122559301459 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb624006010 tx=0x7fb6240d0a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba60698835e22753 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:34.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.440+0000 7fb604ce3640 1 -- 10.20.193.28:0/3096887357 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb6240d91d0 con 0x56231ddc9a50 2026-02-25T11:52:34.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.440+0000 7fb604ce3640 1 -- 10.20.193.28:0/3096887357 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb6240ba040 con 0x56231ddc9a50 2026-02-25T11:52:34.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.440+0000 7fb604ce3640 1 -- 10.20.193.28:0/3096887357 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb6240026a0 con 0x56231ddc9a50 2026-02-25T11:52:34.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.440+0000 7fb63c538e00 1 -- 10.20.193.28:0/3096887357 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 msgr2=0x56231de0e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:34.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.440+0000 7fb63c538e00 1 --2- 10.20.193.28:0/3096887357 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de0e1b0 secure :-1 s=READY pgs=606 gs=2 cs=0 l=1 c_cookie=13429850122559301459 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb624006010 tx=0x7fb6240d0a30 comp rx=0 tx=0).stop 2026-02-25T11:52:34.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.440+0000 7fb63c538e00 1 -- 10.20.193.28:0/3096887357 shutdown_connections 2026-02-25T11:52:34.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.440+0000 7fb63c538e00 1 --2- 10.20.193.28:0/3096887357 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231de0e6f0 0x56231de2ead0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.440+0000 7fb63c538e00 1 --2- 10.20.193.28:0/3096887357 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de0e1b0 unknown :-1 s=CLOSED pgs=606 gs=2 cs=0 l=1 c_cookie=13429850122559301459 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.440+0000 7fb63c538e00 1 --2- 10.20.193.28:0/3096887357 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9140 0x56231ddc9510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.440+0000 7fb63c538e00 1 -- 10.20.193.28:0/3096887357 >> 10.20.193.28:0/3096887357 conn(0x56231dce8d40 msgr2=0x56231ddc7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:34.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.440+0000 7fb63c538e00 1 -- 10.20.193.28:0/3096887357 shutdown_connections 2026-02-25T11:52:34.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.440+0000 7fb63c538e00 1 -- 10.20.193.28:0/3096887357 wait complete. 2026-02-25T11:52:34.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb63c538e00 1 -- start start 2026-02-25T11:52:34.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb63c538e00 1 Processor -- start 2026-02-25T11:52:34.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb63c538e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231ddc9140 0x56231de263d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb63c538e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de26910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb63c538e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231de0e6f0 0x56231de2b900 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb63c538e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56231de2be40 con 0x56231ddc9a50 2026-02-25T11:52:34.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb63c538e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56231de2bfb0 con 0x56231ddc9140 2026-02-25T11:52:34.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb63c538e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56231de2c170 con 0x56231de0e6f0 2026-02-25T11:52:34.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb6337fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de26910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb633fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231ddc9140 0x56231de263d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb633fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231ddc9140 0x56231de263d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43392/0 (socket says 10.20.193.28:43392) 2026-02-25T11:52:34.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb633fff640 1 -- 10.20.193.28:0/1929455291 learned_addr learned my addr 10.20.193.28:0/1929455291 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:34.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb6337fe640 1 --2- 10.20.193.28:0/1929455291 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de26910 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58602/0 (socket says 10.20.193.28:58602) 2026-02-25T11:52:34.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.442+0000 7fb638ab3640 1 --2- 10.20.193.28:0/1929455291 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231de0e6f0 0x56231de2b900 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb633fff640 1 -- 10.20.193.28:0/1929455291 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231de0e6f0 msgr2=0x56231de2b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:34.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb633fff640 1 --2- 10.20.193.28:0/1929455291 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231de0e6f0 0x56231de2b900 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb633fff640 1 -- 10.20.193.28:0/1929455291 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 msgr2=0x56231de26910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:34.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb633fff640 1 --2- 10.20.193.28:0/1929455291 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de26910 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb633fff640 1 -- 10.20.193.28:0/1929455291 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231dc59350 con 0x56231ddc9140 2026-02-25T11:52:34.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb633fff640 1 --2- 10.20.193.28:0/1929455291 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231ddc9140 0x56231de263d0 secure :-1 s=READY pgs=140 gs=1 cs=0 l=1 c_cookie=5040035626488519958 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb6200118a0 tx=0x7fb6200047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45f1cdb2f4295d16 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:34.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb5f5ffb640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:52:34.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/1929455291 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb620021070 con 0x56231ddc9140 2026-02-25T11:52:34.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/1929455291 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb620019af0 con 0x56231ddc9140 2026-02-25T11:52:34.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb63c538e00 1 -- 10.20.193.28:0/1929455291 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6340fbe50 con 0x56231ddc9140 2026-02-25T11:52:34.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/1929455291 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb620026d00 con 0x56231ddc9140 2026-02-25T11:52:34.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/1929455291 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb62001c410 con 0x56231ddc9140 2026-02-25T11:52:34.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb63c538e00 10 client.26204 _ll_register_callbacks cb 0x56231dc569a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:52:34.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.443+0000 7fb63c538e00 1 -- 10.20.193.28:0/1929455291 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56231d96b6f0 con 0x56231ddc9140 2026-02-25T11:52:34.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.444+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/1929455291 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb620026e90 con 0x56231ddc9140 2026-02-25T11:52:34.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.444+0000 7fb63c538e00 10 client.26204 fetch_fsmap learned FSMap version 23 2026-02-25T11:52:34.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.444+0000 7fb63c538e00 1 -- 10.20.193.28:0/1929455291 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x56231dc6b150 con 0x56231ddc9140 2026-02-25T11:52:34.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.444+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/1929455291 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb620002d20 con 0x56231ddc9140 2026-02-25T11:52:34.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.444+0000 7fb63c538e00 10 client.26204 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:52:34.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.444+0000 7fb63c538e00 10 client.26204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:52:34.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.444+0000 7fb63c538e00 20 client.26204 populate_metadata read hostname 'trial028' 2026-02-25T11:52:34.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.444+0000 7fb63c538e00 1 --2- 10.20.193.28:0/1929455291 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56231de2e920 0x56231de0c470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.444+0000 7fb63c538e00 4 client.26204 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:52:34.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.444+0000 7fb63c538e00 1 -- 10.20.193.28:0/1929455291 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56231dca7910 con 0x56231de2e920 2026-02-25T11:52:34.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.444+0000 7fb6337fe640 1 --2- 10.20.193.28:0/1929455291 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56231de2e920 0x56231de0c470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.445+0000 7fb6337fe640 1 --2- 10.20.193.28:0/1929455291 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56231de2e920 0x56231de0c470 crc :-1 s=READY pgs=96 gs=4 cs=0 l=0 c_cookie=12363348369305490608 s_cookie=2390622694606344369 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab936fd68adbd4b0 server_cookie=212d32acb7ff90b1 in_seq=0 out_seq=0 2026-02-25T11:52:34.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.445+0000 7fb5f5ffb640 10 client.26204 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:34.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.445+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/1929455291 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x56231dca7910 con 0x56231de2e920 2026-02-25T11:52:34.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.445+0000 7fb5f5ffb640 10 client.26204 handle_command_reply: tid=0 2026-02-25T11:52:34.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.445+0000 7fb63c538e00 1 client.26204 shutdown 2026-02-25T11:52:34.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.445+0000 7fb63c538e00 1 -- 10.20.193.28:0/1929455291 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231ddc9140 msgr2=0x56231de263d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:34.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.445+0000 7fb63c538e00 1 --2- 10.20.193.28:0/1929455291 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231ddc9140 0x56231de263d0 secure :-1 s=READY pgs=140 gs=1 cs=0 l=1 c_cookie=5040035626488519958 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb6200118a0 tx=0x7fb6200047e0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.446+0000 7fb63c538e00 1 -- 10.20.193.28:0/1929455291 shutdown_connections 2026-02-25T11:52:34.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.446+0000 7fb63c538e00 1 --2- 10.20.193.28:0/1929455291 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56231de2e920 0x56231de0c470 crc :-1 s=READY pgs=96 gs=4 cs=0 l=0 c_cookie=12363348369305490608 s_cookie=2390622694606344369 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.446+0000 7fb63c538e00 1 --2- 10.20.193.28:0/1929455291 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231de0e6f0 0x56231de2b900 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.446+0000 7fb63c538e00 1 --2- 10.20.193.28:0/1929455291 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de26910 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.446+0000 7fb63c538e00 1 --2- 10.20.193.28:0/1929455291 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231ddc9140 0x56231de263d0 unknown :-1 s=CLOSED pgs=140 gs=1 cs=0 l=1 c_cookie=5040035626488519958 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.446+0000 7fb5f5ffb640 0 client.26204 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:34.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.446+0000 7fb63c538e00 1 -- 10.20.193.28:0/1929455291 >> 10.20.193.28:0/1929455291 conn(0x56231dce8d40 msgr2=0x56231ddc7230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:34.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.446+0000 7fb63c538e00 1 -- 10.20.193.28:0/1929455291 shutdown_connections 2026-02-25T11:52:34.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.446+0000 7fb63c538e00 1 -- 10.20.193.28:0/1929455291 wait complete. 2026-02-25T11:52:34.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.446+0000 7fb63c538e00 20 client.26204 trim_cache size 0 max 16384 2026-02-25T11:52:34.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.455+0000 7fb63c538e00 1 Processor -- start 2026-02-25T11:52:34.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.455+0000 7fb63c538e00 1 -- start start 2026-02-25T11:52:34.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.455+0000 7fb63c538e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9140 0x56231de111e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.455+0000 7fb63c538e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de235a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.456 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.455+0000 7fb63c538e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231de0e6f0 0x56231de23ae0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.455+0000 7fb63c538e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56231dcd3470 con 0x56231ddc9a50 2026-02-25T11:52:34.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.455+0000 7fb63c538e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56231d993700 con 0x56231de0e6f0 2026-02-25T11:52:34.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.455+0000 7fb63c538e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56231dd636b0 con 0x56231ddc9140 2026-02-25T11:52:34.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.455+0000 7fb6337fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de235a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.455+0000 7fb6337fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de235a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58616/0 (socket says 10.20.193.28:58616) 2026-02-25T11:52:34.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.455+0000 7fb6337fe640 1 -- 10.20.193.28:0/656605904 learned_addr learned my addr 10.20.193.28:0/656605904 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:34.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.455+0000 7fb638ab3640 1 --2- 10.20.193.28:0/656605904 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231de0e6f0 0x56231de23ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.455+0000 7fb633fff640 1 --2- 10.20.193.28:0/656605904 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9140 0x56231de111e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.456+0000 7fb638ab3640 1 -- 10.20.193.28:0/656605904 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9140 msgr2=0x56231de111e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:34.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.456+0000 7fb638ab3640 1 --2- 10.20.193.28:0/656605904 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9140 0x56231de111e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.456+0000 7fb638ab3640 1 -- 10.20.193.28:0/656605904 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 msgr2=0x56231de235a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:34.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.456+0000 7fb638ab3640 1 --2- 10.20.193.28:0/656605904 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de235a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.456+0000 7fb638ab3640 1 -- 10.20.193.28:0/656605904 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56231dc65bd0 con 0x56231de0e6f0 2026-02-25T11:52:34.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.456+0000 7fb6337fe640 1 --2- 10.20.193.28:0/656605904 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de235a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:52:34.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.456+0000 7fb638ab3640 1 --2- 10.20.193.28:0/656605904 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231de0e6f0 0x56231de23ae0 secure :-1 s=READY pgs=141 gs=3 cs=0 l=1 c_cookie=13281789608592910189 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x56231dc590a0 tx=0x7fb628003b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b852654771a7a76d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:34.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.456+0000 7fb5f7fff640 1 -- 10.20.193.28:0/656605904 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb628012b10 con 0x56231de0e6f0 2026-02-25T11:52:34.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.456+0000 7fb5f7fff640 1 -- 10.20.193.28:0/656605904 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb628019040 con 0x56231de0e6f0 2026-02-25T11:52:34.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.456+0000 7fb5f7fff640 1 -- 10.20.193.28:0/656605904 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb628010640 con 0x56231de0e6f0 2026-02-25T11:52:34.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.456+0000 7fb63c538e00 1 -- 10.20.193.28:0/656605904 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231de0e6f0 msgr2=0x56231de23ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:34.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.456+0000 7fb63c538e00 1 --2- 10.20.193.28:0/656605904 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231de0e6f0 0x56231de23ae0 secure :-1 s=READY pgs=141 gs=3 cs=0 l=1 c_cookie=13281789608592910189 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x56231dc590a0 tx=0x7fb628003b50 comp rx=0 tx=0).stop 2026-02-25T11:52:34.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 -- 10.20.193.28:0/656605904 shutdown_connections 2026-02-25T11:52:34.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 --2- 10.20.193.28:0/656605904 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231de0e6f0 0x56231de23ae0 unknown :-1 s=CLOSED pgs=141 gs=3 cs=0 l=1 c_cookie=13281789608592910189 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 --2- 10.20.193.28:0/656605904 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231ddc9a50 0x56231de235a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 --2- 10.20.193.28:0/656605904 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9140 0x56231de111e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 -- 10.20.193.28:0/656605904 >> 10.20.193.28:0/656605904 conn(0x56231dce8d40 msgr2=0x56231de10ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:34.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 -- 10.20.193.28:0/656605904 shutdown_connections 2026-02-25T11:52:34.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 -- 10.20.193.28:0/656605904 wait complete. 2026-02-25T11:52:34.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 -- start start 2026-02-25T11:52:34.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 Processor -- start 2026-02-25T11:52:34.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231ddc9140 0x56231dda7220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9a50 0x56231dda7760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231de0e6f0 0x56231dda7ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56231ddac270 con 0x56231de0e6f0 2026-02-25T11:52:34.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56231ddac3e0 con 0x56231ddc9140 2026-02-25T11:52:34.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.457+0000 7fb63c538e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56231ddac5a0 con 0x56231ddc9a50 2026-02-25T11:52:34.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb633fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231ddc9140 0x56231dda7220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb6337fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9a50 0x56231dda7760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb633fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231ddc9140 0x56231dda7220 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:43414/0 (socket says 10.20.193.28:43414) 2026-02-25T11:52:34.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb633fff640 1 -- 10.20.193.28:0/2439263137 learned_addr learned my addr 10.20.193.28:0/2439263137 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:52:34.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb638ab3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231de0e6f0 0x56231dda7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb6337fe640 1 -- 10.20.193.28:0/2439263137 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231ddc9140 msgr2=0x56231dda7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:34.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb6337fe640 1 --2- 10.20.193.28:0/2439263137 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231ddc9140 0x56231dda7220 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb6337fe640 1 -- 10.20.193.28:0/2439263137 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231de0e6f0 msgr2=0x56231dda7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:52:34.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb6337fe640 1 --2- 10.20.193.28:0/2439263137 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231de0e6f0 0x56231dda7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb6337fe640 1 -- 10.20.193.28:0/2439263137 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb628005100 con 0x56231ddc9a50 2026-02-25T11:52:34.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb6337fe640 1 --2- 10.20.193.28:0/2439263137 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9a50 0x56231dda7760 secure :-1 s=READY pgs=171 gs=2 cs=0 l=1 c_cookie=4689351211987650156 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb6240d0f60 tx=0x7fb624005b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4113ec150192466c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:52:34.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb5f5ffb640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:52:34.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/2439263137 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb6240d91d0 con 0x56231ddc9a50 2026-02-25T11:52:34.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/2439263137 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb62403f310 con 0x56231ddc9a50 2026-02-25T11:52:34.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/2439263137 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb624002a80 con 0x56231ddc9a50 2026-02-25T11:52:34.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.458+0000 7fb63c538e00 1 -- 10.20.193.28:0/2439263137 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56231dc80d80 con 0x56231ddc9a50 2026-02-25T11:52:34.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/2439263137 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb624002be0 con 0x56231ddc9a50 2026-02-25T11:52:34.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb63c538e00 10 client.26182 _ll_register_callbacks cb 0x56231dc569a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:52:34.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb63c538e00 1 -- 10.20.193.28:0/2439263137 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6240ba040 con 0x56231ddc9a50 2026-02-25T11:52:34.466 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/2439263137 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb6240027c0 con 0x56231ddc9a50 2026-02-25T11:52:34.466 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb63c538e00 10 client.26182 fetch_fsmap learned FSMap version 23 2026-02-25T11:52:34.466 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb63c538e00 1 -- 10.20.193.28:0/2439263137 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x56231dc59680 con 0x56231ddc9a50 2026-02-25T11:52:34.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/2439263137 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb6240dbb80 con 0x56231ddc9a50 2026-02-25T11:52:34.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb63c538e00 10 client.26182 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:52:34.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb63c538e00 10 client.26182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:52:34.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb63c538e00 20 client.26182 populate_metadata read hostname 'trial028' 2026-02-25T11:52:34.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb63c538e00 1 --2- 10.20.193.28:0/2439263137 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56231de5e700 0x56231de5eba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:52:34.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb63c538e00 4 client.26182 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:52:34.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb63c538e00 1 -- 10.20.193.28:0/2439263137 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56231d968f40 con 0x56231de5e700 2026-02-25T11:52:34.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.459+0000 7fb633fff640 1 --2- 10.20.193.28:0/2439263137 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56231de5e700 0x56231de5eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:52:34.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb633fff640 1 --2- 10.20.193.28:0/2439263137 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56231de5e700 0x56231de5eba0 crc :-1 s=READY pgs=97 gs=4 cs=0 l=0 c_cookie=7463400427273562670 s_cookie=16173702210831029007 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679353255423aa2e server_cookie=e07488780388930f in_seq=0 out_seq=0 2026-02-25T11:52:34.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb5f5ffb640 10 client.26182 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:34.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb5f5ffb640 1 -- 10.20.193.28:0/2439263137 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x56231d968f40 con 0x56231de5e700 2026-02-25T11:52:34.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb5f5ffb640 10 client.26182 handle_command_reply: tid=0 2026-02-25T11:52:34.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb63c538e00 1 client.26182 shutdown 2026-02-25T11:52:34.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb63c538e00 1 -- 10.20.193.28:0/2439263137 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9a50 msgr2=0x56231dda7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:34.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb63c538e00 1 --2- 10.20.193.28:0/2439263137 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9a50 0x56231dda7760 secure :-1 s=READY pgs=171 gs=2 cs=0 l=1 c_cookie=4689351211987650156 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb6240d0f60 tx=0x7fb624005b60 comp rx=0 tx=0).stop 2026-02-25T11:52:34.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb63c538e00 1 -- 10.20.193.28:0/2439263137 shutdown_connections 2026-02-25T11:52:34.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb63c538e00 1 --2- 10.20.193.28:0/2439263137 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56231de5e700 0x56231de5eba0 crc :-1 s=READY pgs=97 gs=4 cs=0 l=0 c_cookie=7463400427273562670 s_cookie=16173702210831029007 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb63c538e00 1 --2- 10.20.193.28:0/2439263137 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56231de0e6f0 0x56231dda7ca0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb63c538e00 1 --2- 10.20.193.28:0/2439263137 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56231ddc9a50 0x56231dda7760 unknown :-1 s=CLOSED pgs=171 gs=2 cs=0 l=1 c_cookie=4689351211987650156 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb63c538e00 1 --2- 10.20.193.28:0/2439263137 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56231ddc9140 0x56231dda7220 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb5f5ffb640 0 client.26182 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:52:34.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.460+0000 7fb63c538e00 1 -- 10.20.193.28:0/2439263137 >> 10.20.193.28:0/2439263137 conn(0x56231dce8d40 msgr2=0x56231de22590 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:34.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.461+0000 7fb63c538e00 1 -- 10.20.193.28:0/2439263137 shutdown_connections 2026-02-25T11:52:34.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.461+0000 7fb63c538e00 1 -- 10.20.193.28:0/2439263137 wait complete. 2026-02-25T11:52:34.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.461+0000 7fb63c538e00 20 client.26182 trim_cache size 0 max 16384 2026-02-25T11:52:34.469 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:52:34.469 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:52:34.470 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:52:34.470 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:52:34.470 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:34.470 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "3c6fdafc-4d02-40d2-962e-24e4d16fa603", 2026-02-25T11:52:34.470 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:52:34.470 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:52:34.470 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:34.470 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:52:34.470 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:52:34.470 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:52:34.471 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:34.471 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:34.471 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:34.471 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:34.471 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:34.471 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:34.471 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:52:34.471 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:52:34.471 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:34.471 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:34.471 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:34.471 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:34.472 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:34.472 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:34.472 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:52:34.472 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:52:34.472 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:34.472 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:34.472 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:34.472 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:34.472 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:34.472 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:34.473 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:52:34.473 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:52:34.473 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:34.473 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:34.473 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:34.473 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:34.473 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:34.473 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:34.473 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:52:34.473 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:52:34.473 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:34.473 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:34.474 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:34.474 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:34.474 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:34.474 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:34.474 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:52:34.474 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:52:34.474 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:34.474 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:34.474 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:34.474 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:34.474 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:34.474 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:34.475 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:52:34.475 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:52:34.475 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:34.475 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:34.475 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:34.475 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:34.475 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:34.475 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:34.475 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:52:34.475 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:52:34.475 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:34.475 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:34.475 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:34.476 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:34.476 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:34.476 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:34.476 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:52:34.476 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:52:34.476 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:34.476 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:34.476 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:34.476 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:34.476 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:34.476 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:52:34.476 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:52:34.477 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:52:34.477 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:52:34.477 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:52:34.477 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:52:34.477 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:52:34.477 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:52:34.477 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:52:34.477 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:52:34.477 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:52:34.477 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.461+0000 7fb63a53d640 1 -- 10.20.193.28:0/1468455840 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb6080b5760 msgr2=0x7fb6080d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:34.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.461+0000 7fb63a53d640 1 --2- 10.20.193.28:0/1468455840 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb6080b5760 0x7fb6080d5c10 secure :-1 s=READY pgs=316 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb6200097c0 tx=0x7fb620009310 comp rx=0 tx=0).stop 2026-02-25T11:52:34.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.461+0000 7fb63a53d640 1 -- 10.20.193.28:0/1468455840 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb634110130 msgr2=0x7fb634176200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:52:34.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.461+0000 7fb63a53d640 1 --2- 10.20.193.28:0/1468455840 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb634110130 0x7fb634176200 secure :-1 s=READY pgs=170 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb624009890 tx=0x7fb624004660 comp rx=0 tx=0).stop 2026-02-25T11:52:34.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.461+0000 7fb63a53d640 1 -- 10.20.193.28:0/1468455840 shutdown_connections 2026-02-25T11:52:34.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.461+0000 7fb63a53d640 1 --2- 10.20.193.28:0/1468455840 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb6080b5760 0x7fb6080d5c10 unknown :-1 s=CLOSED pgs=316 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.461+0000 7fb63a53d640 1 --2- 10.20.193.28:0/1468455840 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb634115930 0x7fb6340f6250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.461+0000 7fb63a53d640 1 --2- 10.20.193.28:0/1468455840 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb634110130 0x7fb634176200 unknown :-1 s=CLOSED pgs=170 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.461+0000 7fb63a53d640 1 --2- 10.20.193.28:0/1468455840 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb63410f740 0x7fb6340f5bc0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:52:34.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.461+0000 7fb63a53d640 1 -- 10.20.193.28:0/1468455840 >> 10.20.193.28:0/1468455840 conn(0x7fb6340ecff0 msgr2=0x7fb63410dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:52:34.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.462+0000 7fb63a53d640 1 -- 10.20.193.28:0/1468455840 shutdown_connections 2026-02-25T11:52:34.478 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:52:34.462+0000 7fb63a53d640 1 -- 10.20.193.28:0/1468455840 wait complete. 2026-02-25T11:52:34.562 INFO:tasks.cephfs.filesystem:scrub status for tag:3c6fdafc-4d02-40d2-962e-24e4d16fa603 - {'path': '/', 'tag': '3c6fdafc-4d02-40d2-962e-24e4d16fa603', 'options': 'recursive,force'} 2026-02-25T11:52:34.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:34 trial167 ceph-mon[43392]: pgmap v356: 97 pgs: 97 active+clean; 3.6 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:34.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:34 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:52:34.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:34 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:52:34.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:34 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:52:34.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:34 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:52:34.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:34 trial119 ceph-mon[42710]: pgmap v356: 97 pgs: 97 active+clean; 3.6 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:34.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:34 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:52:34.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:34 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:52:34.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:34 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:52:34.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:34 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:52:35.148 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.148 INFO:tasks.workunit.client.0.trial028.stdout:189/600: open fd = 3 2026-02-25T11:52:35.148 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.148 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.148 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.149 INFO:tasks.workunit.client.0.trial028.stdout:190/600: open fd = 3 2026-02-25T11:52:35.149 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.149 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.149 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.149 INFO:tasks.workunit.client.0.trial028.stdout:191/600: open fd = 3 2026-02-25T11:52:35.149 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.149 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.150 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.150 INFO:tasks.workunit.client.0.trial028.stdout:192/600: open fd = 3 2026-02-25T11:52:35.150 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.150 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.150 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.150 INFO:tasks.workunit.client.0.trial028.stdout:193/600: open fd = 3 2026-02-25T11:52:35.150 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.150 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.150 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.150 INFO:tasks.workunit.client.0.trial028.stdout:194/600: open fd = 3 2026-02-25T11:52:35.151 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.151 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.151 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.151 INFO:tasks.workunit.client.0.trial028.stdout:195/600: open fd = 3 2026-02-25T11:52:35.151 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.151 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.152 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.152 INFO:tasks.workunit.client.0.trial028.stdout:196/600: open fd = 3 2026-02-25T11:52:35.152 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.152 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.152 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.152 INFO:tasks.workunit.client.0.trial028.stdout:197/600: open fd = 3 2026-02-25T11:52:35.153 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.153 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.153 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.153 INFO:tasks.workunit.client.0.trial028.stdout:198/600: open fd = 3 2026-02-25T11:52:35.153 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.153 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.153 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.153 INFO:tasks.workunit.client.0.trial028.stdout:199/600: open fd = 3 2026-02-25T11:52:35.153 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.154 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.154 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.154 INFO:tasks.workunit.client.0.trial028.stdout:200/600: open fd = 3 2026-02-25T11:52:35.154 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.154 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.154 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.154 INFO:tasks.workunit.client.0.trial028.stdout:201/600: open fd = 3 2026-02-25T11:52:35.154 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.154 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.154 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.155 INFO:tasks.workunit.client.0.trial028.stdout:202/600: open fd = 3 2026-02-25T11:52:35.155 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.155 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.155 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.155 INFO:tasks.workunit.client.0.trial028.stdout:203/600: open fd = 3 2026-02-25T11:52:35.155 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.156 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.156 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.156 INFO:tasks.workunit.client.0.trial028.stdout:204/600: open fd = 3 2026-02-25T11:52:35.156 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.156 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.156 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.156 INFO:tasks.workunit.client.0.trial028.stdout:205/600: open fd = 3 2026-02-25T11:52:35.156 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.156 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.157 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.157 INFO:tasks.workunit.client.0.trial028.stdout:206/600: open fd = 3 2026-02-25T11:52:35.157 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.157 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.157 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.157 INFO:tasks.workunit.client.0.trial028.stdout:207/600: open fd = 3 2026-02-25T11:52:35.157 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.157 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.157 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.157 INFO:tasks.workunit.client.0.trial028.stdout:208/600: open fd = 3 2026-02-25T11:52:35.157 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.157 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:209/600: open fd = 3 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:210/600: open fd = 3 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:211/600: open fd = 3 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.158 INFO:tasks.workunit.client.0.trial028.stdout:212/600: open fd = 3 2026-02-25T11:52:35.159 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.159 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.159 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.159 INFO:tasks.workunit.client.0.trial028.stdout:213/600: open fd = 3 2026-02-25T11:52:35.159 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.159 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.159 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.159 INFO:tasks.workunit.client.0.trial028.stdout:214/600: open fd = 3 2026-02-25T11:52:35.159 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.159 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.159 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.159 INFO:tasks.workunit.client.0.trial028.stdout:215/600: open fd = 3 2026-02-25T11:52:35.159 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.160 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.160 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.160 INFO:tasks.workunit.client.0.trial028.stdout:216/600: open fd = 3 2026-02-25T11:52:35.160 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.160 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.160 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.160 INFO:tasks.workunit.client.0.trial028.stdout:217/600: open fd = 3 2026-02-25T11:52:35.160 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.160 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.160 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.160 INFO:tasks.workunit.client.0.trial028.stdout:218/600: open fd = 3 2026-02-25T11:52:35.160 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.160 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.161 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.161 INFO:tasks.workunit.client.0.trial028.stdout:219/600: open fd = 3 2026-02-25T11:52:35.161 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.161 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.161 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.161 INFO:tasks.workunit.client.0.trial028.stdout:220/600: open fd = 3 2026-02-25T11:52:35.161 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.161 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.161 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.161 INFO:tasks.workunit.client.0.trial028.stdout:221/600: open fd = 3 2026-02-25T11:52:35.161 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.161 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.161 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.162 INFO:tasks.workunit.client.0.trial028.stdout:222/600: open fd = 3 2026-02-25T11:52:35.162 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.162 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.162 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.162 INFO:tasks.workunit.client.0.trial028.stdout:223/600: open fd = 3 2026-02-25T11:52:35.162 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.162 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.162 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.162 INFO:tasks.workunit.client.0.trial028.stdout:224/600: open fd = 3 2026-02-25T11:52:35.162 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.162 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.162 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.162 INFO:tasks.workunit.client.0.trial028.stdout:225/600: open fd = 3 2026-02-25T11:52:35.163 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.163 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.163 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.163 INFO:tasks.workunit.client.0.trial028.stdout:226/600: open fd = 3 2026-02-25T11:52:35.163 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.163 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.163 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.163 INFO:tasks.workunit.client.0.trial028.stdout:227/600: open fd = 3 2026-02-25T11:52:35.163 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.163 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.163 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.163 INFO:tasks.workunit.client.0.trial028.stdout:228/600: open fd = 3 2026-02-25T11:52:35.163 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:229/600: open fd = 3 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:230/600: open fd = 3 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:231/600: open fd = 3 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.164 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.165 INFO:tasks.workunit.client.0.trial028.stdout:232/600: open fd = 3 2026-02-25T11:52:35.165 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.165 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.165 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.165 INFO:tasks.workunit.client.0.trial028.stdout:233/600: open fd = 3 2026-02-25T11:52:35.165 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.165 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.165 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.165 INFO:tasks.workunit.client.0.trial028.stdout:234/600: open fd = 3 2026-02-25T11:52:35.165 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.165 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.165 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.166 INFO:tasks.workunit.client.0.trial028.stdout:235/600: open fd = 3 2026-02-25T11:52:35.166 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.166 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.166 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.166 INFO:tasks.workunit.client.0.trial028.stdout:236/600: open fd = 3 2026-02-25T11:52:35.166 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.166 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.166 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.166 INFO:tasks.workunit.client.0.trial028.stdout:237/600: open fd = 3 2026-02-25T11:52:35.166 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.166 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.166 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.166 INFO:tasks.workunit.client.0.trial028.stdout:238/600: open fd = 3 2026-02-25T11:52:35.167 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.167 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.167 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.167 INFO:tasks.workunit.client.0.trial028.stdout:239/600: open fd = 3 2026-02-25T11:52:35.167 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.167 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.167 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.167 INFO:tasks.workunit.client.0.trial028.stdout:240/600: open fd = 3 2026-02-25T11:52:35.167 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.167 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.167 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.167 INFO:tasks.workunit.client.0.trial028.stdout:241/600: open fd = 3 2026-02-25T11:52:35.167 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.168 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.168 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.168 INFO:tasks.workunit.client.0.trial028.stdout:242/600: open fd = 3 2026-02-25T11:52:35.168 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.168 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.168 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.168 INFO:tasks.workunit.client.0.trial028.stdout:243/600: open fd = 3 2026-02-25T11:52:35.168 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.168 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.168 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.168 INFO:tasks.workunit.client.0.trial028.stdout:244/600: open fd = 3 2026-02-25T11:52:35.168 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.169 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.169 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.169 INFO:tasks.workunit.client.0.trial028.stdout:245/600: open fd = 3 2026-02-25T11:52:35.169 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.169 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.169 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.169 INFO:tasks.workunit.client.0.trial028.stdout:246/600: open fd = 3 2026-02-25T11:52:35.169 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.169 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.169 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.169 INFO:tasks.workunit.client.0.trial028.stdout:247/600: open fd = 3 2026-02-25T11:52:35.170 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.170 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.170 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.170 INFO:tasks.workunit.client.0.trial028.stdout:248/600: open fd = 3 2026-02-25T11:52:35.170 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.170 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.170 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.170 INFO:tasks.workunit.client.0.trial028.stdout:249/600: open fd = 3 2026-02-25T11:52:35.170 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.171 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.171 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:52:35.171 INFO:tasks.workunit.client.0.trial028.stdout:250/600: open fd = 3 2026-02-25T11:52:35.171 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:35.171 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:52:36.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:36 trial028 ceph-mon[39224]: pgmap v357: 97 pgs: 97 active+clean; 3.6 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:36.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:36 trial167 ceph-mon[43392]: pgmap v357: 97 pgs: 97 active+clean; 3.6 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:36.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:36 trial119 ceph-mon[42710]: pgmap v357: 97 pgs: 97 active+clean; 3.6 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:37.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:37 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:52:37.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:37 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:52:37.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:37 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:52:38.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:38 trial028 ceph-mon[39224]: pgmap v358: 97 pgs: 97 active+clean; 3.7 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:38.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:38 trial167 ceph-mon[43392]: pgmap v358: 97 pgs: 97 active+clean; 3.7 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:38.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:38 trial119 ceph-mon[42710]: pgmap v358: 97 pgs: 97 active+clean; 3.7 MiB data, 339 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:40.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:40 trial028 ceph-mon[39224]: pgmap v359: 97 pgs: 97 active+clean; 3.7 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:52:40.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:40 trial167 ceph-mon[43392]: pgmap v359: 97 pgs: 97 active+clean; 3.7 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:52:40.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:40 trial119 ceph-mon[42710]: pgmap v359: 97 pgs: 97 active+clean; 3.7 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:52:42.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:42 trial028 ceph-mon[39224]: pgmap v360: 97 pgs: 97 active+clean; 3.7 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:52:42.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:42 trial167 ceph-mon[43392]: pgmap v360: 97 pgs: 97 active+clean; 3.7 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:52:42.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:42 trial119 ceph-mon[42710]: pgmap v360: 97 pgs: 97 active+clean; 3.7 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:52:44.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:44 trial028 ceph-mon[39224]: pgmap v361: 97 pgs: 97 active+clean; 3.7 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:52:44.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:44 trial167 ceph-mon[43392]: pgmap v361: 97 pgs: 97 active+clean; 3.7 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:52:44.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:44 trial119 ceph-mon[42710]: pgmap v361: 97 pgs: 97 active+clean; 3.7 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:52:46.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:46 trial028 ceph-mon[39224]: pgmap v362: 97 pgs: 97 active+clean; 3.7 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:52:46.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:46 trial167 ceph-mon[43392]: pgmap v362: 97 pgs: 97 active+clean; 3.7 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:52:46.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:46 trial119 ceph-mon[42710]: pgmap v362: 97 pgs: 97 active+clean; 3.7 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:52:48.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:48 trial028 ceph-mon[39224]: pgmap v363: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:48.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:48 trial167 ceph-mon[43392]: pgmap v363: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:48.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:48 trial119 ceph-mon[42710]: pgmap v363: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:50.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:50 trial028 ceph-mon[39224]: pgmap v364: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:52:50.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:50 trial167 ceph-mon[43392]: pgmap v364: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:52:50.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:50 trial119 ceph-mon[42710]: pgmap v364: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:52:52.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:52 trial028 ceph-mon[39224]: pgmap v365: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:52.562 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:52 trial119 ceph-mon[42710]: pgmap v365: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:52.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:52 trial167 ceph-mon[43392]: pgmap v365: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:54.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:54 trial028 ceph-mon[39224]: pgmap v366: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:54.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:54 trial167 ceph-mon[43392]: pgmap v366: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:54.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:54 trial119 ceph-mon[42710]: pgmap v366: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:52:56.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:56 trial167 ceph-mon[43392]: pgmap v367: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:56.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:56 trial119 ceph-mon[42710]: pgmap v367: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:56.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:56 trial028 ceph-mon[39224]: pgmap v367: 97 pgs: 97 active+clean; 3.8 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:52:58.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:52:58 trial167 ceph-mon[43392]: pgmap v368: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:52:58.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:52:58 trial119 ceph-mon[42710]: pgmap v368: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:52:58.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:52:58 trial028 ceph-mon[39224]: pgmap v368: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:53:00.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:00 trial167 ceph-mon[43392]: pgmap v369: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:53:00.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:00 trial119 ceph-mon[42710]: pgmap v369: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:53:00.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:00 trial028 ceph-mon[39224]: pgmap v369: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:53:02.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:02 trial167 ceph-mon[43392]: pgmap v370: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:02.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:02 trial119 ceph-mon[42710]: pgmap v370: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:02.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:02 trial028 ceph-mon[39224]: pgmap v370: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:04.563 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:53:04.615 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:04 trial028 ceph-mon[39224]: pgmap v371: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:04.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:04 trial167 ceph-mon[43392]: pgmap v371: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:04.752 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:04.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:04 trial119 ceph-mon[42710]: pgmap v371: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:04.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.921+0000 7f9d9532b640 1 -- 10.20.193.28:0/4054806369 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d9010f740 msgr2=0x7f9d9010fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:04.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.921+0000 7f9d9532b640 1 --2- 10.20.193.28:0/4054806369 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d9010f740 0x7f9d9010fb10 secure :-1 s=READY pgs=607 gs=1 cs=0 l=1 c_cookie=8042280945774765844 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d84009950 tx=0x7f9d8402f190 comp rx=0 tx=0).stop 2026-02-25T11:53:04.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.921+0000 7f9d9532b640 1 -- 10.20.193.28:0/4054806369 shutdown_connections 2026-02-25T11:53:04.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.921+0000 7f9d9532b640 1 --2- 10.20.193.28:0/4054806369 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9d90115930 0x7f9d90176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.921+0000 7f9d9532b640 1 --2- 10.20.193.28:0/4054806369 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9d90110130 0x7f9d90115180 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.921+0000 7f9d9532b640 1 --2- 10.20.193.28:0/4054806369 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d9010f740 0x7f9d9010fb10 unknown :-1 s=CLOSED pgs=607 gs=1 cs=0 l=1 c_cookie=8042280945774765844 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.921+0000 7f9d9532b640 1 -- 10.20.193.28:0/4054806369 >> 10.20.193.28:0/4054806369 conn(0x7f9d900ecff0 msgr2=0x7f9d9010d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:04.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.921+0000 7f9d9532b640 1 -- 10.20.193.28:0/4054806369 shutdown_connections 2026-02-25T11:53:04.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.921+0000 7f9d9532b640 1 -- 10.20.193.28:0/4054806369 wait complete. 2026-02-25T11:53:04.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.922+0000 7f9d9532b640 1 Processor -- start 2026-02-25T11:53:04.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.922+0000 7f9d9532b640 1 -- start start 2026-02-25T11:53:04.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.922+0000 7f9d9532b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d9010f740 0x7f9d9011e110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.922+0000 7f9d9532b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9d90110130 0x7f9d9011e650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.922+0000 7f9d9532b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9d90115930 0x7f9d90123a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.922+0000 7f9d9532b640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9d90124040 con 0x7f9d9010f740 2026-02-25T11:53:04.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.922+0000 7f9d9532b640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9d901241b0 con 0x7f9d90110130 2026-02-25T11:53:04.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.922+0000 7f9d9532b640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d90124370 con 0x7f9d90115930 2026-02-25T11:53:04.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.922+0000 7f9d8effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d9010f740 0x7f9d9011e110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.922+0000 7f9d8effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d9010f740 0x7f9d9011e110 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52548/0 (socket says 10.20.193.28:52548) 2026-02-25T11:53:04.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.922+0000 7f9d8effd640 1 -- 10.20.193.28:0/410729738 learned_addr learned my addr 10.20.193.28:0/410729738 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:04.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.922+0000 7f9d8e7fc640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9d90110130 0x7f9d9011e650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.922+0000 7f9d8f7fe640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9d90115930 0x7f9d90123a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.923+0000 7f9d8effd640 1 -- 10.20.193.28:0/410729738 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9d90115930 msgr2=0x7f9d90123a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:04.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.923+0000 7f9d8effd640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9d90115930 0x7f9d90123a20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.923+0000 7f9d8effd640 1 -- 10.20.193.28:0/410729738 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9d90110130 msgr2=0x7f9d9011e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:04.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.923+0000 7f9d8effd640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9d90110130 0x7f9d9011e650 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.923+0000 7f9d8effd640 1 -- 10.20.193.28:0/410729738 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d84009570 con 0x7f9d9010f740 2026-02-25T11:53:04.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.923+0000 7f9d8effd640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d9010f740 0x7f9d9011e110 secure :-1 s=READY pgs=608 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d840098a0 tx=0x7f9d8402fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:04.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.923+0000 7f9d6ffff640 1 -- 10.20.193.28:0/410729738 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9d84005e40 con 0x7f9d9010f740 2026-02-25T11:53:04.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.923+0000 7f9d6ffff640 1 -- 10.20.193.28:0/410729738 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9d84002a10 con 0x7f9d9010f740 2026-02-25T11:53:04.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.923+0000 7f9d6ffff640 1 -- 10.20.193.28:0/410729738 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9d84041930 con 0x7f9d9010f740 2026-02-25T11:53:04.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.923+0000 7f9d9532b640 1 -- 10.20.193.28:0/410729738 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d901245f0 con 0x7f9d9010f740 2026-02-25T11:53:04.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.923+0000 7f9d9532b640 1 -- 10.20.193.28:0/410729738 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d90164bb0 con 0x7f9d9010f740 2026-02-25T11:53:04.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.926+0000 7f9d6ffff640 1 -- 10.20.193.28:0/410729738 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f9d840388a0 con 0x7f9d9010f740 2026-02-25T11:53:04.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.928+0000 7f9d6ffff640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9d640b5760 0x7f9d640d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.928+0000 7f9d6ffff640 1 -- 10.20.193.28:0/410729738 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9d8410b490 con 0x7f9d9010f740 2026-02-25T11:53:04.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.928+0000 7f9d8e7fc640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9d640b5760 0x7f9d640d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.929+0000 7f9d8e7fc640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9d640b5760 0x7f9d640d5c10 secure :-1 s=READY pgs=317 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d90124ab0 tx=0x7f9d7800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:04.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.929+0000 7f9d97326e00 1 Processor -- start 2026-02-25T11:53:04.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d97326e00 1 -- start start 2026-02-25T11:53:04.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d97326e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb6d6b50 0x55a6eb6d6f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d97326e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 0x55a6eb71bbc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d97326e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb71c100 0x55a6eb73c4e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d97326e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb5e49b0 con 0x55a6eb71c100 2026-02-25T11:53:04.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d97326e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb5e06e0 con 0x55a6eb6d6b50 2026-02-25T11:53:04.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d97326e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb5da840 con 0x55a6eb6d7460 2026-02-25T11:53:04.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d8f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb71c100 0x55a6eb73c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d8f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb71c100 0x55a6eb73c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52560/0 (socket says 10.20.193.28:52560) 2026-02-25T11:53:04.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d8f7fe640 1 -- 10.20.193.28:0/2496247959 learned_addr learned my addr 10.20.193.28:0/2496247959 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:04.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d8effd640 1 --2- 10.20.193.28:0/2496247959 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb6d6b50 0x55a6eb6d6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d8e7fc640 1 --2- 10.20.193.28:0/2496247959 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 0x55a6eb71bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d8f7fe640 1 -- 10.20.193.28:0/2496247959 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 msgr2=0x55a6eb71bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:04.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d8f7fe640 1 --2- 10.20.193.28:0/2496247959 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 0x55a6eb71bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d8f7fe640 1 -- 10.20.193.28:0/2496247959 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb6d6b50 msgr2=0x55a6eb6d6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:04.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d8f7fe640 1 --2- 10.20.193.28:0/2496247959 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb6d6b50 0x55a6eb6d6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d8f7fe640 1 -- 10.20.193.28:0/2496247959 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6eb585d80 con 0x55a6eb71c100 2026-02-25T11:53:04.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.930+0000 7f9d8effd640 1 --2- 10.20.193.28:0/2496247959 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb6d6b50 0x55a6eb6d6f20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:53:04.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.931+0000 7f9d8f7fe640 1 --2- 10.20.193.28:0/2496247959 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb71c100 0x55a6eb73c4e0 secure :-1 s=READY pgs=609 gs=1 cs=0 l=1 c_cookie=8640594142188847919 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d80008d10 tx=0x7f9d800117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e98e97ad7cc32f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:04.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.931+0000 7f9d5b7fe640 1 -- 10.20.193.28:0/2496247959 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9d800052f0 con 0x55a6eb71c100 2026-02-25T11:53:04.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.931+0000 7f9d5b7fe640 1 -- 10.20.193.28:0/2496247959 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9d80008e10 con 0x55a6eb71c100 2026-02-25T11:53:04.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.931+0000 7f9d97326e00 1 -- 10.20.193.28:0/2496247959 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb71c100 msgr2=0x55a6eb73c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:04.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.931+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2496247959 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb71c100 0x55a6eb73c4e0 secure :-1 s=READY pgs=609 gs=1 cs=0 l=1 c_cookie=8640594142188847919 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d80008d10 tx=0x7f9d800117a0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.931+0000 7f9d97326e00 1 -- 10.20.193.28:0/2496247959 shutdown_connections 2026-02-25T11:53:04.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.931+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2496247959 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb71c100 0x55a6eb73c4e0 unknown :-1 s=CLOSED pgs=609 gs=1 cs=0 l=1 c_cookie=8640594142188847919 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.931+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2496247959 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 0x55a6eb71bbc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.931+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2496247959 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb6d6b50 0x55a6eb6d6f20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.931+0000 7f9d97326e00 1 -- 10.20.193.28:0/2496247959 >> 10.20.193.28:0/2496247959 conn(0x55a6eb5edd40 msgr2=0x55a6eb6cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:04.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.931+0000 7f9d97326e00 1 -- 10.20.193.28:0/2496247959 shutdown_connections 2026-02-25T11:53:04.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.931+0000 7f9d97326e00 1 -- 10.20.193.28:0/2496247959 wait complete. 2026-02-25T11:53:04.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.932+0000 7f9d97326e00 1 -- start start 2026-02-25T11:53:04.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.933+0000 7f9d97326e00 1 Processor -- start 2026-02-25T11:53:04.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.933+0000 7f9d97326e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d6b50 0x55a6eb733b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.933+0000 7f9d97326e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 0x55a6eb734080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.933+0000 7f9d97326e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 0x55a6eb739070 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.933+0000 7f9d97326e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb7395b0 con 0x55a6eb6d6b50 2026-02-25T11:53:04.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.933+0000 7f9d97326e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb739720 con 0x55a6eb71c100 2026-02-25T11:53:04.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.933+0000 7f9d97326e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb7398e0 con 0x55a6eb6d7460 2026-02-25T11:53:04.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.933+0000 7f9d8effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d6b50 0x55a6eb733b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.933+0000 7f9d8f7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 0x55a6eb739070 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.933+0000 7f9d8effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d6b50 0x55a6eb733b40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52562/0 (socket says 10.20.193.28:52562) 2026-02-25T11:53:04.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.933+0000 7f9d8effd640 1 -- 10.20.193.28:0/2302259519 learned_addr learned my addr 10.20.193.28:0/2302259519 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:04.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d8effd640 1 -- 10.20.193.28:0/2302259519 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 msgr2=0x55a6eb734080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:04.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d8effd640 1 --2- 10.20.193.28:0/2302259519 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 0x55a6eb734080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d8effd640 1 -- 10.20.193.28:0/2302259519 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 msgr2=0x55a6eb739070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:04.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d8effd640 1 --2- 10.20.193.28:0/2302259519 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 0x55a6eb739070 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d8effd640 1 -- 10.20.193.28:0/2302259519 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6eb55e350 con 0x55a6eb6d6b50 2026-02-25T11:53:04.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d8effd640 1 --2- 10.20.193.28:0/2302259519 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d6b50 0x55a6eb733b40 secure :-1 s=READY pgs=610 gs=1 cs=0 l=1 c_cookie=17256185003174401535 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d84108f80 tx=0x7f9d8402f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7a48d54ca215ff server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:04.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d58ff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:04.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d58ff9640 1 -- 10.20.193.28:0/2302259519 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9d840d7070 con 0x55a6eb6d6b50 2026-02-25T11:53:04.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d58ff9640 1 -- 10.20.193.28:0/2302259519 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9d840d11f0 con 0x55a6eb6d6b50 2026-02-25T11:53:04.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d58ff9640 1 -- 10.20.193.28:0/2302259519 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9d840dc740 con 0x55a6eb6d6b50 2026-02-25T11:53:04.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d97326e00 1 -- 10.20.193.28:0/2302259519 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d80008930 con 0x55a6eb6d6b50 2026-02-25T11:53:04.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d58ff9640 1 -- 10.20.193.28:0/2302259519 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9d840d9b10 con 0x55a6eb6d6b50 2026-02-25T11:53:04.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d58ff9640 10 client.16449 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:53:04.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d97326e00 10 client.16449 _ll_register_callbacks cb 0x55a6eb55b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:04.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.934+0000 7f9d97326e00 1 -- 10.20.193.28:0/2302259519 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a6eb2706f0 con 0x55a6eb6d6b50 2026-02-25T11:53:04.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.935+0000 7f9d58ff9640 1 -- 10.20.193.28:0/2302259519 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9d840d1010 con 0x55a6eb6d6b50 2026-02-25T11:53:04.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.935+0000 7f9d97326e00 10 client.16449 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:04.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.935+0000 7f9d97326e00 1 -- 10.20.193.28:0/2302259519 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55a6eb570150 con 0x55a6eb6d6b50 2026-02-25T11:53:04.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.935+0000 7f9d58ff9640 1 -- 10.20.193.28:0/2302259519 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9d840d24a0 con 0x55a6eb6d6b50 2026-02-25T11:53:04.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.935+0000 7f9d97326e00 10 client.16449 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:04.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.935+0000 7f9d97326e00 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:04.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.935+0000 7f9d97326e00 20 client.16449 populate_metadata read hostname 'trial028' 2026-02-25T11:53:04.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.935+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2302259519 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a6eb7191c0 0x55a6eb719660 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.935+0000 7f9d97326e00 4 client.16449 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:53:04.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.935+0000 7f9d97326e00 1 -- 10.20.193.28:0/2302259519 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6eb5ac910 con 0x55a6eb7191c0 2026-02-25T11:53:04.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.935+0000 7f9d8e7fc640 1 --2- 10.20.193.28:0/2302259519 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a6eb7191c0 0x55a6eb719660 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.936+0000 7f9d8e7fc640 1 --2- 10.20.193.28:0/2302259519 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a6eb7191c0 0x55a6eb719660 crc :-1 s=READY pgs=98 gs=4 cs=0 l=0 c_cookie=5246241967951407491 s_cookie=1858861245909965642 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48ce6546297d6983 server_cookie=19cc007459341b4a in_seq=0 out_seq=0 2026-02-25T11:53:04.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.936+0000 7f9d58ff9640 10 client.16449 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:04.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d58ff9640 1 -- 10.20.193.28:0/2302259519 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55a6eb5ac910 con 0x55a6eb7191c0 2026-02-25T11:53:04.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d58ff9640 10 client.16449 handle_command_reply: tid=0 2026-02-25T11:53:04.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d97326e00 1 client.16449 shutdown 2026-02-25T11:53:04.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d97326e00 1 -- 10.20.193.28:0/2302259519 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d6b50 msgr2=0x55a6eb733b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:04.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2302259519 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d6b50 0x55a6eb733b40 secure :-1 s=READY pgs=610 gs=1 cs=0 l=1 c_cookie=17256185003174401535 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d84108f80 tx=0x7f9d8402f190 comp rx=0 tx=0).stop 2026-02-25T11:53:04.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d97326e00 1 -- 10.20.193.28:0/2302259519 shutdown_connections 2026-02-25T11:53:04.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2302259519 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a6eb7191c0 0x55a6eb719660 crc :-1 s=READY pgs=98 gs=4 cs=0 l=0 c_cookie=5246241967951407491 s_cookie=1858861245909965642 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2302259519 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 0x55a6eb739070 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2302259519 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 0x55a6eb734080 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2302259519 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d6b50 0x55a6eb733b40 unknown :-1 s=CLOSED pgs=610 gs=1 cs=0 l=1 c_cookie=17256185003174401535 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d58ff9640 0 client.16449 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:04.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d97326e00 1 -- 10.20.193.28:0/2302259519 >> 10.20.193.28:0/2302259519 conn(0x55a6eb5edd40 msgr2=0x55a6eb6cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:04.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d97326e00 1 -- 10.20.193.28:0/2302259519 shutdown_connections 2026-02-25T11:53:04.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d97326e00 1 -- 10.20.193.28:0/2302259519 wait complete. 2026-02-25T11:53:04.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.937+0000 7f9d97326e00 20 client.16449 trim_cache size 0 max 16384 2026-02-25T11:53:04.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.947+0000 7f9d97326e00 1 Processor -- start 2026-02-25T11:53:04.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.947+0000 7f9d97326e00 1 -- start start 2026-02-25T11:53:04.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.947+0000 7f9d97326e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d6b50 0x55a6eb71ebf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.947+0000 7f9d97326e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d7460 0x55a6eb730d40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.947+0000 7f9d97326e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 0x55a6eb731280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.947+0000 7f9d97326e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb5d8470 con 0x55a6eb6d7460 2026-02-25T11:53:04.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.947+0000 7f9d97326e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb298700 con 0x55a6eb71c100 2026-02-25T11:53:04.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.947+0000 7f9d97326e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb63d9d0 con 0x55a6eb6d6b50 2026-02-25T11:53:04.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.947+0000 7f9d8e7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d7460 0x55a6eb730d40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.947+0000 7f9d8e7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d7460 0x55a6eb730d40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52576/0 (socket says 10.20.193.28:52576) 2026-02-25T11:53:04.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.947+0000 7f9d8e7fc640 1 -- 10.20.193.28:0/2832356295 learned_addr learned my addr 10.20.193.28:0/2832356295 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:04.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.947+0000 7f9d8effd640 1 --2- 10.20.193.28:0/2832356295 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d6b50 0x55a6eb71ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.948+0000 7f9d8f7fe640 1 --2- 10.20.193.28:0/2832356295 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 0x55a6eb731280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.948+0000 7f9d8e7fc640 1 -- 10.20.193.28:0/2832356295 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d6b50 msgr2=0x55a6eb71ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:04.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.948+0000 7f9d8e7fc640 1 --2- 10.20.193.28:0/2832356295 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d6b50 0x55a6eb71ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.948+0000 7f9d8e7fc640 1 -- 10.20.193.28:0/2832356295 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 msgr2=0x55a6eb731280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:04.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.948+0000 7f9d8e7fc640 1 --2- 10.20.193.28:0/2832356295 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 0x55a6eb731280 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.948+0000 7f9d8e7fc640 1 -- 10.20.193.28:0/2832356295 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6eb725f40 con 0x55a6eb6d7460 2026-02-25T11:53:04.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.948+0000 7f9d8e7fc640 1 --2- 10.20.193.28:0/2832356295 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d7460 0x55a6eb730d40 secure :-1 s=READY pgs=611 gs=1 cs=0 l=1 c_cookie=17518857957776982934 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x55a6eb719ed0 tx=0x7f9d78013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31f7c7e63d28396 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:04.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.948+0000 7f9d5affd640 1 -- 10.20.193.28:0/2832356295 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9d78024040 con 0x55a6eb6d7460 2026-02-25T11:53:04.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.948+0000 7f9d5affd640 1 -- 10.20.193.28:0/2832356295 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9d78015070 con 0x55a6eb6d7460 2026-02-25T11:53:04.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.948+0000 7f9d5affd640 1 -- 10.20.193.28:0/2832356295 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9d7801f8e0 con 0x55a6eb6d7460 2026-02-25T11:53:04.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.948+0000 7f9d97326e00 1 -- 10.20.193.28:0/2832356295 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d7460 msgr2=0x55a6eb730d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:04.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.948+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2832356295 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d7460 0x55a6eb730d40 secure :-1 s=READY pgs=611 gs=1 cs=0 l=1 c_cookie=17518857957776982934 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x55a6eb719ed0 tx=0x7f9d78013000 comp rx=0 tx=0).stop 2026-02-25T11:53:04.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.949+0000 7f9d97326e00 1 -- 10.20.193.28:0/2832356295 shutdown_connections 2026-02-25T11:53:04.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.949+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2832356295 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 0x55a6eb731280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.949+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2832356295 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d7460 0x55a6eb730d40 unknown :-1 s=CLOSED pgs=611 gs=1 cs=0 l=1 c_cookie=17518857957776982934 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.949+0000 7f9d97326e00 1 --2- 10.20.193.28:0/2832356295 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d6b50 0x55a6eb71ebf0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.949+0000 7f9d97326e00 1 -- 10.20.193.28:0/2832356295 >> 10.20.193.28:0/2832356295 conn(0x55a6eb5edd40 msgr2=0x55a6eb71e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:04.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.949+0000 7f9d97326e00 1 -- 10.20.193.28:0/2832356295 shutdown_connections 2026-02-25T11:53:04.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.949+0000 7f9d97326e00 1 -- 10.20.193.28:0/2832356295 wait complete. 2026-02-25T11:53:04.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.949+0000 7f9d97326e00 1 -- start start 2026-02-25T11:53:04.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.949+0000 7f9d97326e00 1 Processor -- start 2026-02-25T11:53:04.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d97326e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d6b50 0x55a6eb6b1880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d97326e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 0x55a6eb6b1dc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d97326e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 0x55a6eb716a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d97326e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb7830e0 con 0x55a6eb6d6b50 2026-02-25T11:53:04.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d97326e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb716f60 con 0x55a6eb71c100 2026-02-25T11:53:04.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d97326e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a6eb717120 con 0x55a6eb6d7460 2026-02-25T11:53:04.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d8e7fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 0x55a6eb6b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d8e7fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 0x55a6eb6b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:37360/0 (socket says 10.20.193.28:37360) 2026-02-25T11:53:04.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d8e7fc640 1 -- 10.20.193.28:0/1262268582 learned_addr learned my addr 10.20.193.28:0/1262268582 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:04.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d8f7fe640 1 --2- 10.20.193.28:0/1262268582 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 0x55a6eb716a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d8f7fe640 1 -- 10.20.193.28:0/1262268582 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 msgr2=0x55a6eb6b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:04.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d8f7fe640 1 --2- 10.20.193.28:0/1262268582 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 0x55a6eb6b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d8f7fe640 1 -- 10.20.193.28:0/1262268582 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d6b50 msgr2=0x55a6eb6b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:04.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d8f7fe640 1 --2- 10.20.193.28:0/1262268582 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d6b50 0x55a6eb6b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.954 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.950+0000 7f9d8f7fe640 1 -- 10.20.193.28:0/1262268582 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6eb56abd0 con 0x55a6eb71c100 2026-02-25T11:53:04.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.951+0000 7f9d8f7fe640 1 --2- 10.20.193.28:0/1262268582 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 0x55a6eb716a20 secure :-1 s=READY pgs=142 gs=2 cs=0 l=1 c_cookie=13871464402829261565 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d800034b0 tx=0x7f9d800067f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0815761551d3afd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:04.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.951+0000 7f9d58ff9640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:53:04.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.951+0000 7f9d58ff9640 1 -- 10.20.193.28:0/1262268582 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9d80024040 con 0x55a6eb71c100 2026-02-25T11:53:04.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.951+0000 7f9d58ff9640 1 -- 10.20.193.28:0/1262268582 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9d80005590 con 0x55a6eb71c100 2026-02-25T11:53:04.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.951+0000 7f9d58ff9640 1 -- 10.20.193.28:0/1262268582 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9d800296d0 con 0x55a6eb71c100 2026-02-25T11:53:04.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.951+0000 7f9d97326e00 1 -- 10.20.193.28:0/1262268582 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d78029470 con 0x55a6eb71c100 2026-02-25T11:53:04.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.951+0000 7f9d58ff9640 1 -- 10.20.193.28:0/1262268582 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9d8001b420 con 0x55a6eb71c100 2026-02-25T11:53:04.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.951+0000 7f9d58ff9640 10 client.26228 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:53:04.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.951+0000 7f9d97326e00 10 client.26228 _ll_register_callbacks cb 0x55a6eb55b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:04.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.951+0000 7f9d97326e00 1 -- 10.20.193.28:0/1262268582 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d80008e10 con 0x55a6eb71c100 2026-02-25T11:53:04.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.952+0000 7f9d58ff9640 1 -- 10.20.193.28:0/1262268582 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9d80003e80 con 0x55a6eb71c100 2026-02-25T11:53:04.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.952+0000 7f9d97326e00 10 client.26228 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:04.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.952+0000 7f9d97326e00 1 -- 10.20.193.28:0/1262268582 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55a6eb55e680 con 0x55a6eb71c100 2026-02-25T11:53:04.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.952+0000 7f9d58ff9640 1 -- 10.20.193.28:0/1262268582 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9d80027070 con 0x55a6eb71c100 2026-02-25T11:53:04.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.952+0000 7f9d97326e00 10 client.26228 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:04.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.952+0000 7f9d97326e00 10 client.26228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:04.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.952+0000 7f9d97326e00 20 client.26228 populate_metadata read hostname 'trial028' 2026-02-25T11:53:04.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.952+0000 7f9d97326e00 1 --2- 10.20.193.28:0/1262268582 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a6eb7c9860 0x55a6eb7e9ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:04.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.952+0000 7f9d97326e00 4 client.26228 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:53:04.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.952+0000 7f9d97326e00 1 -- 10.20.193.28:0/1262268582 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a6eb26df40 con 0x55a6eb7c9860 2026-02-25T11:53:04.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.952+0000 7f9d8effd640 1 --2- 10.20.193.28:0/1262268582 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a6eb7c9860 0x55a6eb7e9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:04.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.953+0000 7f9d8effd640 1 --2- 10.20.193.28:0/1262268582 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a6eb7c9860 0x55a6eb7e9ce0 crc :-1 s=READY pgs=99 gs=4 cs=0 l=0 c_cookie=10891106951478917155 s_cookie=16577543199385644912 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9724fc119db03823 server_cookie=e60f43b55012ef70 in_seq=0 out_seq=0 2026-02-25T11:53:04.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.953+0000 7f9d58ff9640 10 client.26228 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:04.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.953+0000 7f9d58ff9640 1 -- 10.20.193.28:0/1262268582 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x55a6eb26df40 con 0x55a6eb7c9860 2026-02-25T11:53:04.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.953+0000 7f9d58ff9640 10 client.26228 handle_command_reply: tid=0 2026-02-25T11:53:04.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.953+0000 7f9d97326e00 1 client.26228 shutdown 2026-02-25T11:53:04.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d97326e00 1 -- 10.20.193.28:0/1262268582 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 msgr2=0x55a6eb716a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:04.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d97326e00 1 --2- 10.20.193.28:0/1262268582 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 0x55a6eb716a20 secure :-1 s=READY pgs=142 gs=2 cs=0 l=1 c_cookie=13871464402829261565 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d800034b0 tx=0x7f9d800067f0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d97326e00 1 -- 10.20.193.28:0/1262268582 shutdown_connections 2026-02-25T11:53:04.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d97326e00 1 --2- 10.20.193.28:0/1262268582 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a6eb7c9860 0x55a6eb7e9ce0 crc :-1 s=READY pgs=99 gs=4 cs=0 l=0 c_cookie=10891106951478917155 s_cookie=16577543199385644912 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d97326e00 1 --2- 10.20.193.28:0/1262268582 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6eb71c100 0x55a6eb716a20 unknown :-1 s=CLOSED pgs=142 gs=2 cs=0 l=1 c_cookie=13871464402829261565 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d97326e00 1 --2- 10.20.193.28:0/1262268582 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6eb6d7460 0x55a6eb6b1dc0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d97326e00 1 --2- 10.20.193.28:0/1262268582 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6eb6d6b50 0x55a6eb6b1880 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d58ff9640 0 client.26228 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:04.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d97326e00 1 -- 10.20.193.28:0/1262268582 >> 10.20.193.28:0/1262268582 conn(0x55a6eb5edd40 msgr2=0x55a6eb737ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:04.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d97326e00 1 -- 10.20.193.28:0/1262268582 shutdown_connections 2026-02-25T11:53:04.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d97326e00 1 -- 10.20.193.28:0/1262268582 wait complete. 2026-02-25T11:53:04.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d97326e00 20 client.26228 trim_cache size 0 max 16384 2026-02-25T11:53:04.961 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:53:04.961 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:53:04.961 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:53:04.961 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:53:04.961 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:53:04.961 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:53:04.961 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:04.961 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:04.961 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:04.961 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:04.962 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:04.962 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:04.962 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:53:04.962 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:53:04.962 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:04.962 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:04.962 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:04.962 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:04.962 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:04.962 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:04.962 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:53:04.962 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:53:04.963 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:04.963 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:04.963 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:04.963 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:04.963 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:04.963 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:04.963 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:53:04.963 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:53:04.963 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:04.963 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:04.963 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:04.964 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:04.964 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:04.964 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:04.964 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:53:04.964 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:53:04.964 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:04.964 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:04.964 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:04.964 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:04.964 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:04.964 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:04.964 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:53:04.965 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:53:04.965 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:04.965 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:04.965 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:04.965 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:04.965 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:04.965 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:04.965 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:53:04.965 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:53:04.965 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:04.965 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:04.965 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:04.966 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:04.966 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:04.966 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:04.966 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:53:04.966 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:53:04.966 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:04.966 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:04.966 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:04.966 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:04.966 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:04.966 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:04.966 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:53:04.967 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:53:04.967 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:04.967 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:04.967 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:04.967 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:04.967 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:04.967 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:04.967 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:53:04.967 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:53:04.967 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:04.967 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:04.967 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:04.968 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:04.968 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:04.968 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:53:04.968 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:53:04.968 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:53:04.968 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d9532b640 1 -- 10.20.193.28:0/410729738 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9d640b5760 msgr2=0x7f9d640d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:04.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d9532b640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9d640b5760 0x7f9d640d5c10 secure :-1 s=READY pgs=317 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d90124ab0 tx=0x7f9d7800b410 comp rx=0 tx=0).stop 2026-02-25T11:53:04.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d9532b640 1 -- 10.20.193.28:0/410729738 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d9010f740 msgr2=0x7f9d9011e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:04.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.954+0000 7f9d9532b640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d9010f740 0x7f9d9011e110 secure :-1 s=READY pgs=608 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9d840098a0 tx=0x7f9d8402fc20 comp rx=0 tx=0).stop 2026-02-25T11:53:04.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.955+0000 7f9d9532b640 1 -- 10.20.193.28:0/410729738 shutdown_connections 2026-02-25T11:53:04.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.955+0000 7f9d9532b640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9d640b5760 0x7f9d640d5c10 unknown :-1 s=CLOSED pgs=317 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.955+0000 7f9d9532b640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9d90115930 0x7f9d90123a20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.955+0000 7f9d9532b640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9d90110130 0x7f9d9011e650 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.955+0000 7f9d9532b640 1 --2- 10.20.193.28:0/410729738 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9d9010f740 0x7f9d9011e110 unknown :-1 s=CLOSED pgs=608 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:04.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.955+0000 7f9d9532b640 1 -- 10.20.193.28:0/410729738 >> 10.20.193.28:0/410729738 conn(0x7f9d900ecff0 msgr2=0x7f9d9010d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:04.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.955+0000 7f9d9532b640 1 -- 10.20.193.28:0/410729738 shutdown_connections 2026-02-25T11:53:04.969 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:04.956+0000 7f9d9532b640 1 -- 10.20.193.28:0/410729738 wait complete. 2026-02-25T11:53:05.058 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:53:05.058 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:53:05.243 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:05.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.416+0000 7f05179dc640 1 -- 10.20.193.28:0/330231527 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f051010f840 msgr2=0x7f051010fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:05.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.416+0000 7f05179dc640 1 --2- 10.20.193.28:0/330231527 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f051010f840 0x7f051010fc10 secure :-1 s=READY pgs=612 gs=2 cs=0 l=1 c_cookie=6415471769778757539 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0500009950 tx=0x7f050002f190 comp rx=0 tx=0).stop 2026-02-25T11:53:05.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.417+0000 7f05179dc640 1 -- 10.20.193.28:0/330231527 shutdown_connections 2026-02-25T11:53:05.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.417+0000 7f05179dc640 1 --2- 10.20.193.28:0/330231527 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0510115a30 0x7f0510176400 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:05.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.417+0000 7f05179dc640 1 --2- 10.20.193.28:0/330231527 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0510110230 0x7f0510115280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:05.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.417+0000 7f05179dc640 1 --2- 10.20.193.28:0/330231527 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f051010f840 0x7f051010fc10 unknown :-1 s=CLOSED pgs=612 gs=2 cs=0 l=1 c_cookie=6415471769778757539 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:05.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.417+0000 7f05179dc640 1 -- 10.20.193.28:0/330231527 >> 10.20.193.28:0/330231527 conn(0x7f05100ed130 msgr2=0x7f051010d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:05.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.417+0000 7f05179dc640 1 -- 10.20.193.28:0/330231527 shutdown_connections 2026-02-25T11:53:05.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.417+0000 7f05179dc640 1 -- 10.20.193.28:0/330231527 wait complete. 2026-02-25T11:53:05.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.417+0000 7f05179dc640 1 Processor -- start 2026-02-25T11:53:05.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.417+0000 7f05179dc640 1 -- start start 2026-02-25T11:53:05.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f05179dc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f051010f840 0x7f05100f5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:05.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f05179dc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0510110230 0x7f0510176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:05.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f05179dc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0510115a30 0x7f05100f6240 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:05.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f05179dc640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f05100fb450 con 0x7f051010f840 2026-02-25T11:53:05.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f05179dc640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f05100fb5c0 con 0x7f0510115a30 2026-02-25T11:53:05.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f05179dc640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f05100fb780 con 0x7f0510110230 2026-02-25T11:53:05.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f0515751640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f051010f840 0x7f05100f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:05.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f0515751640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f051010f840 0x7f05100f5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52600/0 (socket says 10.20.193.28:52600) 2026-02-25T11:53:05.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f0515751640 1 -- 10.20.193.28:0/147221955 learned_addr learned my addr 10.20.193.28:0/147221955 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:05.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f0514f50640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0510110230 0x7f0510176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:05.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f0515f52640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0510115a30 0x7f05100f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:05.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f0515751640 1 -- 10.20.193.28:0/147221955 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0510110230 msgr2=0x7f0510176300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:05.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f0515751640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0510110230 0x7f0510176300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:05.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f0515751640 1 -- 10.20.193.28:0/147221955 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0510115a30 msgr2=0x7f05100f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:05.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f0515751640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0510115a30 0x7f05100f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:05.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.418+0000 7f0515751640 1 -- 10.20.193.28:0/147221955 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0500009570 con 0x7f051010f840 2026-02-25T11:53:05.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.419+0000 7f0515751640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f051010f840 0x7f05100f5b40 secure :-1 s=READY pgs=613 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05000098a0 tx=0x7f05000028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:05.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.419+0000 7f04fe7fc640 1 -- 10.20.193.28:0/147221955 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f050003d070 con 0x7f051010f840 2026-02-25T11:53:05.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.419+0000 7f04fe7fc640 1 -- 10.20.193.28:0/147221955 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0500002a40 con 0x7f051010f840 2026-02-25T11:53:05.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.419+0000 7f04fe7fc640 1 -- 10.20.193.28:0/147221955 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0500041820 con 0x7f051010f840 2026-02-25T11:53:05.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.419+0000 7f05179dc640 1 -- 10.20.193.28:0/147221955 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05100fba00 con 0x7f051010f840 2026-02-25T11:53:05.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.419+0000 7f05179dc640 1 -- 10.20.193.28:0/147221955 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05100fbe20 con 0x7f051010f840 2026-02-25T11:53:05.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.419+0000 7f05179dc640 1 -- 10.20.193.28:0/147221955 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0510110b20 con 0x7f051010f840 2026-02-25T11:53:05.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.422+0000 7f04fe7fc640 1 -- 10.20.193.28:0/147221955 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f05000419c0 con 0x7f051010f840 2026-02-25T11:53:05.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.423+0000 7f04fe7fc640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f04ec0b56d0 0x7f04ec0d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:05.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.424+0000 7f04fe7fc640 1 -- 10.20.193.28:0/147221955 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f050010b190 con 0x7f051010f840 2026-02-25T11:53:05.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.424+0000 7f0514f50640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f04ec0b56d0 0x7f04ec0d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:05.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.424+0000 7f0514f50640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f04ec0b56d0 0x7f04ec0d5b80 secure :-1 s=READY pgs=318 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05100fc4e0 tx=0x7f0504005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:05.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.425+0000 7f04fe7fc640 1 -- 10.20.193.28:0/147221955 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f05000cdb60 con 0x7f051010f840 2026-02-25T11:53:05.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.572+0000 7f05179dc640 1 -- 10.20.193.28:0/147221955 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f051010fc10 con 0x7f051010f840 2026-02-25T11:53:05.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.573+0000 7f04fe7fc640 1 -- 10.20.193.28:0/147221955 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f0500048e80 con 0x7f051010f840 2026-02-25T11:53:05.575 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:53:05.575 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:53:05.576 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:53:05.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.574+0000 7f05179dc640 1 -- 10.20.193.28:0/147221955 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f04ec0b56d0 msgr2=0x7f04ec0d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:05.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.574+0000 7f05179dc640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f04ec0b56d0 0x7f04ec0d5b80 secure :-1 s=READY pgs=318 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05100fc4e0 tx=0x7f0504005f90 comp rx=0 tx=0).stop 2026-02-25T11:53:05.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.574+0000 7f05179dc640 1 -- 10.20.193.28:0/147221955 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f051010f840 msgr2=0x7f05100f5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:05.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.574+0000 7f05179dc640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f051010f840 0x7f05100f5b40 secure :-1 s=READY pgs=613 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05000098a0 tx=0x7f05000028d0 comp rx=0 tx=0).stop 2026-02-25T11:53:05.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.574+0000 7f05179dc640 1 -- 10.20.193.28:0/147221955 shutdown_connections 2026-02-25T11:53:05.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.574+0000 7f05179dc640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f04ec0b56d0 0x7f04ec0d5b80 unknown :-1 s=CLOSED pgs=318 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:05.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.574+0000 7f05179dc640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0510115a30 0x7f05100f6240 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:05.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.574+0000 7f05179dc640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0510110230 0x7f0510176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:05.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.574+0000 7f05179dc640 1 --2- 10.20.193.28:0/147221955 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f051010f840 0x7f05100f5b40 unknown :-1 s=CLOSED pgs=613 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:05.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.574+0000 7f05179dc640 1 -- 10.20.193.28:0/147221955 >> 10.20.193.28:0/147221955 conn(0x7f05100ed130 msgr2=0x7f0510111c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:05.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.574+0000 7f05179dc640 1 -- 10.20.193.28:0/147221955 shutdown_connections 2026-02-25T11:53:05.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:05.574+0000 7f05179dc640 1 -- 10.20.193.28:0/147221955 wait complete. 2026-02-25T11:53:05.712 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:53:05.897 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:06.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.075+0000 7fa63d7a8640 1 -- 10.20.193.28:0/1250487293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa638110080 msgr2=0x7fa6381150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.075+0000 7fa63d7a8640 1 --2- 10.20.193.28:0/1250487293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa638110080 0x7fa6381150d0 secure :-1 s=READY pgs=614 gs=1 cs=0 l=1 c_cookie=13245212453154983295 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa628009950 tx=0x7fa62802f190 comp rx=0 tx=0).stop 2026-02-25T11:53:06.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.075+0000 7fa63d7a8640 1 -- 10.20.193.28:0/1250487293 shutdown_connections 2026-02-25T11:53:06.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.075+0000 7fa63d7a8640 1 --2- 10.20.193.28:0/1250487293 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa638115880 0x7fa638176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.075+0000 7fa63d7a8640 1 --2- 10.20.193.28:0/1250487293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa638110080 0x7fa6381150d0 unknown :-1 s=CLOSED pgs=614 gs=1 cs=0 l=1 c_cookie=13245212453154983295 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.075+0000 7fa63d7a8640 1 --2- 10.20.193.28:0/1250487293 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa63810f690 0x7fa63810fa60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.075+0000 7fa63d7a8640 1 -- 10.20.193.28:0/1250487293 >> 10.20.193.28:0/1250487293 conn(0x7fa6380ecf00 msgr2=0x7fa63810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:06.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.075+0000 7fa63d7a8640 1 -- 10.20.193.28:0/1250487293 shutdown_connections 2026-02-25T11:53:06.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.075+0000 7fa63d7a8640 1 -- 10.20.193.28:0/1250487293 wait complete. 2026-02-25T11:53:06.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.076+0000 7fa63d7a8640 1 Processor -- start 2026-02-25T11:53:06.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.076+0000 7fa63d7a8640 1 -- start start 2026-02-25T11:53:06.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.076+0000 7fa63d7a8640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa63810f690 0x7fa63811e0b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.076+0000 7fa63d7a8640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa638110080 0x7fa63811e5f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.076+0000 7fa63d7a8640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa638115880 0x7fa6381239c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.076+0000 7fa63d7a8640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa638123fe0 con 0x7fa63810f690 2026-02-25T11:53:06.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.076+0000 7fa63d7a8640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa638124150 con 0x7fa638115880 2026-02-25T11:53:06.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.076+0000 7fa63d7a8640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa638124310 con 0x7fa638110080 2026-02-25T11:53:06.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.076+0000 7fa637fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa63810f690 0x7fa63811e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa637fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa63810f690 0x7fa63811e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37070/0 (socket says 10.20.193.28:37070) 2026-02-25T11:53:06.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa637fff640 1 -- 10.20.193.28:0/225895373 learned_addr learned my addr 10.20.193.28:0/225895373 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:06.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa6377fe640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa638110080 0x7fa63811e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa63cfa7640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa638115880 0x7fa6381239c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa637fff640 1 -- 10.20.193.28:0/225895373 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa638110080 msgr2=0x7fa63811e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa637fff640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa638110080 0x7fa63811e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa637fff640 1 -- 10.20.193.28:0/225895373 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa638115880 msgr2=0x7fa6381239c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa637fff640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa638115880 0x7fa6381239c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa637fff640 1 -- 10.20.193.28:0/225895373 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa628009570 con 0x7fa63810f690 2026-02-25T11:53:06.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa637fff640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa63810f690 0x7fa63811e0b0 secure :-1 s=READY pgs=615 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa62400d950 tx=0x7fa62400de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:06.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa6357fa640 1 -- 10.20.193.28:0/225895373 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa624004540 con 0x7fa63810f690 2026-02-25T11:53:06.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa6357fa640 1 -- 10.20.193.28:0/225895373 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa624009ce0 con 0x7fa63810f690 2026-02-25T11:53:06.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa6357fa640 1 -- 10.20.193.28:0/225895373 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa6240106e0 con 0x7fa63810f690 2026-02-25T11:53:06.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa63d7a8640 1 -- 10.20.193.28:0/225895373 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6381245f0 con 0x7fa63810f690 2026-02-25T11:53:06.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.077+0000 7fa63d7a8640 1 -- 10.20.193.28:0/225895373 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa638164cf0 con 0x7fa63810f690 2026-02-25T11:53:06.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.080+0000 7fa6357fa640 1 -- 10.20.193.28:0/225895373 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fa6240097c0 con 0x7fa63810f690 2026-02-25T11:53:06.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.082+0000 7fa6357fa640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa60c0b5760 0x7fa60c0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.082+0000 7fa6357fa640 1 -- 10.20.193.28:0/225895373 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa6240e6db0 con 0x7fa63810f690 2026-02-25T11:53:06.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.082+0000 7fa6377fe640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa60c0b5760 0x7fa60c0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa640a2ce00 1 Processor -- start 2026-02-25T11:53:06.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa640a2ce00 1 -- start start 2026-02-25T11:53:06.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa6377fe640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa60c0b5760 0x7fa60c0d5c10 secure :-1 s=READY pgs=319 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa6280023d0 tx=0x7fa628031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:06.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa640a2ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa1f910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa640a2ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa1fe50 0x5631ffa645b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa640a2ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa64af0 0x5631ffa84ed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa640a2ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5631ff92c8a0 con 0x5631ffa1fe50 2026-02-25T11:53:06.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa640a2ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5631ff9285d0 con 0x5631ffa1f540 2026-02-25T11:53:06.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa640a2ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5631ff922570 con 0x5631ffa64af0 2026-02-25T11:53:06.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa6377fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa1fe50 0x5631ffa645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa637fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa1f910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa637fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa1f910 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:57308/0 (socket says 10.20.193.28:57308) 2026-02-25T11:53:06.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa637fff640 1 -- 10.20.193.28:0/3515337758 learned_addr learned my addr 10.20.193.28:0/3515337758 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:06.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.083+0000 7fa63cfa7640 1 --2- 10.20.193.28:0/3515337758 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa64af0 0x5631ffa84ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa637fff640 1 -- 10.20.193.28:0/3515337758 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa64af0 msgr2=0x5631ffa84ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa637fff640 1 --2- 10.20.193.28:0/3515337758 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa64af0 0x5631ffa84ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa637fff640 1 -- 10.20.193.28:0/3515337758 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa1fe50 msgr2=0x5631ffa645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa637fff640 1 --2- 10.20.193.28:0/3515337758 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa1fe50 0x5631ffa645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa637fff640 1 -- 10.20.193.28:0/3515337758 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631ff8cdd10 con 0x5631ffa1f540 2026-02-25T11:53:06.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa637fff640 1 --2- 10.20.193.28:0/3515337758 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa1f910 secure :-1 s=READY pgs=143 gs=2 cs=0 l=1 c_cookie=2176192369160644717 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa6240aa3a0 tx=0x7fa6240a8fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e33636e06bc686d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:06.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa608ce3640 1 -- 10.20.193.28:0/3515337758 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa624009e50 con 0x5631ffa1f540 2026-02-25T11:53:06.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa608ce3640 1 -- 10.20.193.28:0/3515337758 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa6240adb30 con 0x5631ffa1f540 2026-02-25T11:53:06.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa608ce3640 1 -- 10.20.193.28:0/3515337758 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa6240a93a0 con 0x5631ffa1f540 2026-02-25T11:53:06.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3515337758 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 msgr2=0x5631ffa1f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/3515337758 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa1f910 secure :-1 s=READY pgs=143 gs=2 cs=0 l=1 c_cookie=2176192369160644717 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa6240aa3a0 tx=0x7fa6240a8fd0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3515337758 shutdown_connections 2026-02-25T11:53:06.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/3515337758 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa64af0 0x5631ffa84ed0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/3515337758 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa1fe50 0x5631ffa645b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/3515337758 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa1f910 unknown :-1 s=CLOSED pgs=143 gs=2 cs=0 l=1 c_cookie=2176192369160644717 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3515337758 >> 10.20.193.28:0/3515337758 conn(0x5631ff9357a0 msgr2=0x5631ffa15410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:06.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3515337758 shutdown_connections 2026-02-25T11:53:06.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.084+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3515337758 wait complete. 2026-02-25T11:53:06.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.086+0000 7fa640a2ce00 1 -- start start 2026-02-25T11:53:06.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.086+0000 7fa640a2ce00 1 Processor -- start 2026-02-25T11:53:06.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.086+0000 7fa640a2ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa1f540 0x5631ffa37980 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.086+0000 7fa640a2ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 0x5631ffa37ec0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.086+0000 7fa640a2ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa64af0 0x5631ffa3ceb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.086+0000 7fa640a2ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5631ffa3d3f0 con 0x5631ffa1f540 2026-02-25T11:53:06.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.086+0000 7fa640a2ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5631ffa3d560 con 0x5631ffa64af0 2026-02-25T11:53:06.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.086+0000 7fa640a2ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5631ffa3d720 con 0x5631ffa1fe50 2026-02-25T11:53:06.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.086+0000 7fa637fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa1f540 0x5631ffa37980 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.086+0000 7fa637fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa1f540 0x5631ffa37980 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37100/0 (socket says 10.20.193.28:37100) 2026-02-25T11:53:06.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.086+0000 7fa637fff640 1 -- 10.20.193.28:0/3606658499 learned_addr learned my addr 10.20.193.28:0/3606658499 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:06.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.086+0000 7fa63cfa7640 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa64af0 0x5631ffa3ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa6377fe640 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 0x5631ffa37ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa637fff640 1 -- 10.20.193.28:0/3606658499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 msgr2=0x5631ffa37ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa637fff640 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 0x5631ffa37ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa637fff640 1 -- 10.20.193.28:0/3606658499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa64af0 msgr2=0x5631ffa3ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa637fff640 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa64af0 0x5631ffa3ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa637fff640 1 -- 10.20.193.28:0/3606658499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631ff8a63c0 con 0x5631ffa1f540 2026-02-25T11:53:06.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa63cfa7640 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa64af0 0x5631ffa3ceb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:53:06.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa637fff640 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa1f540 0x5631ffa37980 secure :-1 s=READY pgs=616 gs=1 cs=0 l=1 c_cookie=866277839095743690 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa6240ad0c0 tx=0x7fa6240e6f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05a31adc1844ca server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:06.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa5f9ffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:06.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/3606658499 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa6240b5430 con 0x5631ffa1f540 2026-02-25T11:53:06.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/3606658499 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa6240b8270 con 0x5631ffa1f540 2026-02-25T11:53:06.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/3606658499 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa6240b8060 con 0x5631ffa1f540 2026-02-25T11:53:06.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3606658499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa638164cf0 con 0x5631ffa1f540 2026-02-25T11:53:06.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa640a2ce00 10 client.16470 _ll_register_callbacks cb 0x5631ff8a3a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:06.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3606658499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5631ff5b86f0 con 0x5631ffa1f540 2026-02-25T11:53:06.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.087+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/3606658499 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa6240a93a0 con 0x5631ffa1f540 2026-02-25T11:53:06.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.088+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/3606658499 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa6240722c0 con 0x5631ffa1f540 2026-02-25T11:53:06.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.088+0000 7fa640a2ce00 10 client.16470 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:06.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.088+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3606658499 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5631ff8b80e0 con 0x5631ffa1f540 2026-02-25T11:53:06.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.088+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/3606658499 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa6240ab110 con 0x5631ffa1f540 2026-02-25T11:53:06.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.088+0000 7fa640a2ce00 10 client.16470 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:06.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.088+0000 7fa640a2ce00 10 client.16470 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:53:06.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.088+0000 7fa640a2ce00 20 client.16470 populate_metadata read hostname 'trial028' 2026-02-25T11:53:06.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.088+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5631ffa61c00 0x5631ffa620a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.088+0000 7fa640a2ce00 4 client.16470 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:53:06.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.088+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3606658499 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631ff8f4fc0 con 0x5631ffa61c00 2026-02-25T11:53:06.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.088+0000 7fa6377fe640 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5631ffa61c00 0x5631ffa620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.089+0000 7fa6377fe640 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5631ffa61c00 0x5631ffa620a0 crc :-1 s=READY pgs=41 gs=4 cs=0 l=0 c_cookie=14992039253265219801 s_cookie=4690235715884280402 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d00e6c45059600d9 server_cookie=411710889e21ea52 in_seq=0 out_seq=0 2026-02-25T11:53:06.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.089+0000 7fa5f9ffb640 10 client.16470 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:53:06.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.089+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/3606658499 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5631ff8f4fc0 con 0x5631ffa61c00 2026-02-25T11:53:06.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.089+0000 7fa5f9ffb640 10 client.16470 handle_command_reply: tid=0 2026-02-25T11:53:06.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.089+0000 7fa640a2ce00 1 client.16470 shutdown 2026-02-25T11:53:06.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.090+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3606658499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa1f540 msgr2=0x5631ffa37980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.090+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa1f540 0x5631ffa37980 secure :-1 s=READY pgs=616 gs=1 cs=0 l=1 c_cookie=866277839095743690 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa6240ad0c0 tx=0x7fa6240e6f70 comp rx=0 tx=0).stop 2026-02-25T11:53:06.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.090+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3606658499 shutdown_connections 2026-02-25T11:53:06.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.090+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5631ffa61c00 0x5631ffa620a0 crc :-1 s=READY pgs=41 gs=4 cs=0 l=0 c_cookie=14992039253265219801 s_cookie=4690235715884280402 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.090+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa64af0 0x5631ffa3ceb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.090+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 0x5631ffa37ec0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.090+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/3606658499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa1f540 0x5631ffa37980 unknown :-1 s=CLOSED pgs=616 gs=1 cs=0 l=1 c_cookie=866277839095743690 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.090+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3606658499 >> 10.20.193.28:0/3606658499 conn(0x5631ff9357a0 msgr2=0x5631ffa15b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:06.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.090+0000 7fa5f9ffb640 0 client.16470 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:53:06.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.090+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3606658499 shutdown_connections 2026-02-25T11:53:06.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.090+0000 7fa640a2ce00 1 -- 10.20.193.28:0/3606658499 wait complete. 2026-02-25T11:53:06.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.090+0000 7fa640a2ce00 20 client.16470 trim_cache size 0 max 16384 2026-02-25T11:53:06.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.097+0000 7fa640a2ce00 1 Processor -- start 2026-02-25T11:53:06.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.097+0000 7fa640a2ce00 1 -- start start 2026-02-25T11:53:06.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.097+0000 7fa640a2ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa3c420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.097+0000 7fa640a2ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 0x5631ffa34d70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.097+0000 7fa640a2ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 0x5631ffa352b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.097+0000 7fa640a2ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5631ff9201a0 con 0x5631ffa64af0 2026-02-25T11:53:06.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.097+0000 7fa640a2ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5631ff5e0700 con 0x5631ffa1f540 2026-02-25T11:53:06.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.097+0000 7fa640a2ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5631ffa5e7a0 con 0x5631ffa1fe50 2026-02-25T11:53:06.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.098+0000 7fa63cfa7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 0x5631ffa352b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.098+0000 7fa63cfa7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 0x5631ffa352b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37104/0 (socket says 10.20.193.28:37104) 2026-02-25T11:53:06.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.098+0000 7fa63cfa7640 1 -- 10.20.193.28:0/2350774699 learned_addr learned my addr 10.20.193.28:0/2350774699 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:06.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.098+0000 7fa6377fe640 1 --2- 10.20.193.28:0/2350774699 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 0x5631ffa34d70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.098+0000 7fa637fff640 1 --2- 10.20.193.28:0/2350774699 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa3c420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.098+0000 7fa63cfa7640 1 -- 10.20.193.28:0/2350774699 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 msgr2=0x5631ffa34d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.098+0000 7fa63cfa7640 1 --2- 10.20.193.28:0/2350774699 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 0x5631ffa34d70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.098+0000 7fa63cfa7640 1 -- 10.20.193.28:0/2350774699 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 msgr2=0x5631ffa3c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.098+0000 7fa63cfa7640 1 --2- 10.20.193.28:0/2350774699 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa3c420 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.098+0000 7fa63cfa7640 1 -- 10.20.193.28:0/2350774699 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631ffa2a2a0 con 0x5631ffa64af0 2026-02-25T11:53:06.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.098+0000 7fa637fff640 1 --2- 10.20.193.28:0/2350774699 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa3c420 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:53:06.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.098+0000 7fa63cfa7640 1 --2- 10.20.193.28:0/2350774699 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 0x5631ffa352b0 secure :-1 s=READY pgs=617 gs=1 cs=0 l=1 c_cookie=18298431707332785955 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa62c003bc0 tx=0x7fa62c013820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf116b04650b723 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:06.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.098+0000 7fa5fbfff640 1 -- 10.20.193.28:0/2350774699 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa62c018070 con 0x5631ffa64af0 2026-02-25T11:53:06.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.099+0000 7fa5fbfff640 1 -- 10.20.193.28:0/2350774699 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa62c011890 con 0x5631ffa64af0 2026-02-25T11:53:06.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.099+0000 7fa5fbfff640 1 -- 10.20.193.28:0/2350774699 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa62c01c7d0 con 0x5631ffa64af0 2026-02-25T11:53:06.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.099+0000 7fa640a2ce00 1 -- 10.20.193.28:0/2350774699 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 msgr2=0x5631ffa352b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.099+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/2350774699 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 0x5631ffa352b0 secure :-1 s=READY pgs=617 gs=1 cs=0 l=1 c_cookie=18298431707332785955 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa62c003bc0 tx=0x7fa62c013820 comp rx=0 tx=0).stop 2026-02-25T11:53:06.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.099+0000 7fa640a2ce00 1 -- 10.20.193.28:0/2350774699 shutdown_connections 2026-02-25T11:53:06.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.099+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/2350774699 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 0x5631ffa352b0 unknown :-1 s=CLOSED pgs=617 gs=1 cs=0 l=1 c_cookie=18298431707332785955 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.099+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/2350774699 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 0x5631ffa34d70 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.099+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/2350774699 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa3c420 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.099+0000 7fa640a2ce00 1 -- 10.20.193.28:0/2350774699 >> 10.20.193.28:0/2350774699 conn(0x5631ff9357a0 msgr2=0x5631ffa3bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:06.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.099+0000 7fa640a2ce00 1 -- 10.20.193.28:0/2350774699 shutdown_connections 2026-02-25T11:53:06.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.099+0000 7fa640a2ce00 1 -- 10.20.193.28:0/2350774699 wait complete. 2026-02-25T11:53:06.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.099+0000 7fa640a2ce00 1 -- start start 2026-02-25T11:53:06.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa640a2ce00 1 Processor -- start 2026-02-25T11:53:06.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa640a2ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa156c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa640a2ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 0x5631ffa15c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa640a2ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 0x5631ffa3fdc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa640a2ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5631ffa40300 con 0x5631ffa64af0 2026-02-25T11:53:06.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa640a2ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5631ffa40470 con 0x5631ffa1f540 2026-02-25T11:53:06.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa640a2ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5631ffa40630 con 0x5631ffa1fe50 2026-02-25T11:53:06.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa63cfa7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 0x5631ffa3fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa63cfa7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 0x5631ffa3fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37108/0 (socket says 10.20.193.28:37108) 2026-02-25T11:53:06.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa63cfa7640 1 -- 10.20.193.28:0/1720101496 learned_addr learned my addr 10.20.193.28:0/1720101496 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:06.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa637fff640 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa156c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa6377fe640 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 0x5631ffa15c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa63cfa7640 1 -- 10.20.193.28:0/1720101496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 msgr2=0x5631ffa15c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa63cfa7640 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 0x5631ffa15c00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa63cfa7640 1 -- 10.20.193.28:0/1720101496 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 msgr2=0x5631ffa156c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa63cfa7640 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa156c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.100+0000 7fa63cfa7640 1 -- 10.20.193.28:0/1720101496 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c003860 con 0x5631ffa64af0 2026-02-25T11:53:06.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.101+0000 7fa6377fe640 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 0x5631ffa15c00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:53:06.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.101+0000 7fa63cfa7640 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 0x5631ffa3fdc0 secure :-1 s=READY pgs=618 gs=1 cs=0 l=1 c_cookie=1841708914975744222 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa62c003470 tx=0x7fa62c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198f108018cca0de server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:06.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.101+0000 7fa5f9ffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:06.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.101+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/1720101496 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa62c02a070 con 0x5631ffa64af0 2026-02-25T11:53:06.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.101+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/1720101496 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa62c011a00 con 0x5631ffa64af0 2026-02-25T11:53:06.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.101+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/1720101496 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa62c018040 con 0x5631ffa64af0 2026-02-25T11:53:06.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.101+0000 7fa640a2ce00 1 -- 10.20.193.28:0/1720101496 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631ff8b80e0 con 0x5631ffa64af0 2026-02-25T11:53:06.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.101+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/1720101496 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa62c005820 con 0x5631ffa64af0 2026-02-25T11:53:06.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.101+0000 7fa640a2ce00 10 client.16476 _ll_register_callbacks cb 0x5631ff8a3a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:06.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.101+0000 7fa640a2ce00 1 -- 10.20.193.28:0/1720101496 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6240adb30 con 0x5631ffa64af0 2026-02-25T11:53:06.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.101+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/1720101496 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa62c01c960 con 0x5631ffa64af0 2026-02-25T11:53:06.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.101+0000 7fa640a2ce00 10 client.16476 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:06.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.102+0000 7fa640a2ce00 1 -- 10.20.193.28:0/1720101496 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5631ff8b2b60 con 0x5631ffa64af0 2026-02-25T11:53:06.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.102+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/1720101496 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa62c011b70 con 0x5631ffa64af0 2026-02-25T11:53:06.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.102+0000 7fa640a2ce00 10 client.16476 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:06.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.102+0000 7fa640a2ce00 10 client.16476 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:53:06.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.102+0000 7fa640a2ce00 20 client.16476 populate_metadata read hostname 'trial028' 2026-02-25T11:53:06.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.102+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5631ffb22bc0 0x5631ffb43040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.102+0000 7fa640a2ce00 4 client.16476 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:53:06.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.102+0000 7fa640a2ce00 1 -- 10.20.193.28:0/1720101496 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631ff5b5f40 con 0x5631ffb22bc0 2026-02-25T11:53:06.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.102+0000 7fa637fff640 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5631ffb22bc0 0x5631ffb43040 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa637fff640 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5631ffb22bc0 0x5631ffb43040 crc :-1 s=READY pgs=42 gs=4 cs=0 l=0 c_cookie=14453098985434083102 s_cookie=2295224668526958421 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c893b8f3925b6f1e server_cookie=1fda46ace7435755 in_seq=0 out_seq=0 2026-02-25T11:53:06.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa5f9ffb640 10 client.16476 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:53:06.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa5f9ffb640 1 -- 10.20.193.28:0/1720101496 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631ff5b5f40 con 0x5631ffb22bc0 2026-02-25T11:53:06.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa5f9ffb640 10 client.16476 handle_command_reply: tid=0 2026-02-25T11:53:06.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa640a2ce00 1 client.16476 shutdown 2026-02-25T11:53:06.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa640a2ce00 1 -- 10.20.193.28:0/1720101496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 msgr2=0x5631ffa3fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 0x5631ffa3fdc0 secure :-1 s=READY pgs=618 gs=1 cs=0 l=1 c_cookie=1841708914975744222 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa62c003470 tx=0x7fa62c01c430 comp rx=0 tx=0).stop 2026-02-25T11:53:06.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa640a2ce00 1 -- 10.20.193.28:0/1720101496 shutdown_connections 2026-02-25T11:53:06.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5631ffb22bc0 0x5631ffb43040 crc :-1 s=READY pgs=42 gs=4 cs=0 l=0 c_cookie=14453098985434083102 s_cookie=2295224668526958421 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5631ffa64af0 0x5631ffa3fdc0 unknown :-1 s=CLOSED pgs=618 gs=1 cs=0 l=1 c_cookie=1841708914975744222 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5631ffa1fe50 0x5631ffa15c00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa640a2ce00 1 --2- 10.20.193.28:0/1720101496 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5631ffa1f540 0x5631ffa156c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa5f9ffb640 0 client.16476 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:53:06.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa640a2ce00 1 -- 10.20.193.28:0/1720101496 >> 10.20.193.28:0/1720101496 conn(0x5631ff9357a0 msgr2=0x5631ffa3c500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:06.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa640a2ce00 1 -- 10.20.193.28:0/1720101496 shutdown_connections 2026-02-25T11:53:06.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa640a2ce00 1 -- 10.20.193.28:0/1720101496 wait complete. 2026-02-25T11:53:06.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.103+0000 7fa640a2ce00 20 client.16476 trim_cache size 0 max 16384 2026-02-25T11:53:06.113 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:53:06.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.104+0000 7fa63d7a8640 1 -- 10.20.193.28:0/225895373 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa60c0b5760 msgr2=0x7fa60c0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.113 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.104+0000 7fa63d7a8640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa60c0b5760 0x7fa60c0d5c10 secure :-1 s=READY pgs=319 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa6280023d0 tx=0x7fa628031040 comp rx=0 tx=0).stop 2026-02-25T11:53:06.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.104+0000 7fa63d7a8640 1 -- 10.20.193.28:0/225895373 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa63810f690 msgr2=0x7fa63811e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.104+0000 7fa63d7a8640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa63810f690 0x7fa63811e0b0 secure :-1 s=READY pgs=615 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa62400d950 tx=0x7fa62400de20 comp rx=0 tx=0).stop 2026-02-25T11:53:06.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.104+0000 7fa63d7a8640 1 -- 10.20.193.28:0/225895373 shutdown_connections 2026-02-25T11:53:06.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.104+0000 7fa63d7a8640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa60c0b5760 0x7fa60c0d5c10 unknown :-1 s=CLOSED pgs=319 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.104+0000 7fa63d7a8640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa638115880 0x7fa6381239c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.104+0000 7fa63d7a8640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa638110080 0x7fa63811e5f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.104+0000 7fa63d7a8640 1 --2- 10.20.193.28:0/225895373 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa63810f690 0x7fa63811e0b0 unknown :-1 s=CLOSED pgs=615 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.104+0000 7fa63d7a8640 1 -- 10.20.193.28:0/225895373 >> 10.20.193.28:0/225895373 conn(0x7fa6380ecf00 msgr2=0x7fa63810d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:06.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.104+0000 7fa63d7a8640 1 -- 10.20.193.28:0/225895373 shutdown_connections 2026-02-25T11:53:06.114 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.104+0000 7fa63d7a8640 1 -- 10.20.193.28:0/225895373 wait complete. 2026-02-25T11:53:06.202 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:53:06.390 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:06.456 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:06 trial028 ceph-mon[39224]: pgmap v372: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:06.456 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:06 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/147221955' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:53:06.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.560+0000 7fe01ccab640 1 -- 10.20.193.28:0/2901768312 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe01810bd50 msgr2=0x7fe018110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.560+0000 7fe01ccab640 1 --2- 10.20.193.28:0/2901768312 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe01810bd50 0x7fe018110be0 secure :-1 s=READY pgs=619 gs=3 cs=0 l=1 c_cookie=18214714062920516392 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe008009950 tx=0x7fe00802f1d0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.560+0000 7fe01ccab640 1 -- 10.20.193.28:0/2901768312 shutdown_connections 2026-02-25T11:53:06.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.560+0000 7fe01ccab640 1 --2- 10.20.193.28:0/2901768312 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe018111370 0x7fe018111d50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.560+0000 7fe01ccab640 1 --2- 10.20.193.28:0/2901768312 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe01810bd50 0x7fe018110be0 unknown :-1 s=CLOSED pgs=619 gs=3 cs=0 l=1 c_cookie=18214714062920516392 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.560+0000 7fe01ccab640 1 --2- 10.20.193.28:0/2901768312 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe01810b360 0x7fe01810b730 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.560+0000 7fe01ccab640 1 -- 10.20.193.28:0/2901768312 >> 10.20.193.28:0/2901768312 conn(0x7fe0180e8e70 msgr2=0x7fe018109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:06.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.560+0000 7fe01ccab640 1 -- 10.20.193.28:0/2901768312 shutdown_connections 2026-02-25T11:53:06.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.560+0000 7fe01ccab640 1 -- 10.20.193.28:0/2901768312 wait complete. 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.561+0000 7fe01ccab640 1 Processor -- start 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.561+0000 7fe01ccab640 1 -- start start 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.561+0000 7fe01ccab640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe01810b360 0x7fe018123650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.561+0000 7fe01ccab640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe01810bd50 0x7fe018123b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.561+0000 7fe01ccab640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe018111370 0x7fe018128f60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.561+0000 7fe01ccab640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe018129580 con 0x7fe018111370 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.561+0000 7fe01ccab640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181296f0 con 0x7fe01810bd50 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.561+0000 7fe01ccab640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181298b0 con 0x7fe01810b360 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe017fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe018111370 0x7fe018128f60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe017fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe018111370 0x7fe018128f60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37124/0 (socket says 10.20.193.28:37124) 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe017fff640 1 -- 10.20.193.28:0/26712890 learned_addr learned my addr 10.20.193.28:0/26712890 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe0177fe640 1 --2- 10.20.193.28:0/26712890 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe01810b360 0x7fe018123650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe017fff640 1 -- 10.20.193.28:0/26712890 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe01810b360 msgr2=0x7fe018123650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe017fff640 1 --2- 10.20.193.28:0/26712890 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe01810b360 0x7fe018123650 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe017fff640 1 -- 10.20.193.28:0/26712890 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe01810bd50 msgr2=0x7fe018123b90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe017fff640 1 --2- 10.20.193.28:0/26712890 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe01810bd50 0x7fe018123b90 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe017fff640 1 -- 10.20.193.28:0/26712890 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe008009570 con 0x7fe018111370 2026-02-25T11:53:06.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe017fff640 1 --2- 10.20.193.28:0/26712890 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe018111370 0x7fe018128f60 secure :-1 s=READY pgs=620 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe00c00f8d0 tx=0x7fe00c00fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:06.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe014ff9640 1 -- 10.20.193.28:0/26712890 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe00c00dce0 con 0x7fe018111370 2026-02-25T11:53:06.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe014ff9640 1 -- 10.20.193.28:0/26712890 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe00c00de40 con 0x7fe018111370 2026-02-25T11:53:06.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe014ff9640 1 -- 10.20.193.28:0/26712890 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe00c0116b0 con 0x7fe018111370 2026-02-25T11:53:06.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe01ccab640 1 -- 10.20.193.28:0/26712890 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe018129b90 con 0x7fe018111370 2026-02-25T11:53:06.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.562+0000 7fe01ccab640 1 -- 10.20.193.28:0/26712890 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01816a290 con 0x7fe018111370 2026-02-25T11:53:06.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.566+0000 7fe014ff9640 1 -- 10.20.193.28:0/26712890 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fe00c020060 con 0x7fe018111370 2026-02-25T11:53:06.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.567+0000 7fe014ff9640 1 --2- 10.20.193.28:0/26712890 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdfec0b5760 0x7fdfec0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.567+0000 7fe014ff9640 1 -- 10.20.193.28:0/26712890 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fe00c015070 con 0x7fe018111370 2026-02-25T11:53:06.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.567+0000 7fe01ff2fe00 1 Processor -- start 2026-02-25T11:53:06.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe0177fe640 1 --2- 10.20.193.28:0/26712890 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdfec0b5760 0x7fdfec0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe01ff2fe00 1 -- start start 2026-02-25T11:53:06.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe01ff2fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 0x558188785f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe01ff2fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 0x5581887caba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe01ff2fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5581887cb0e0 0x5581887eb4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe01ff2fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55818869b8a0 con 0x558188786440 2026-02-25T11:53:06.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe01ff2fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5581886975d0 con 0x558188785b30 2026-02-25T11:53:06.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe01ff2fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558188691570 con 0x5581887cb0e0 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe016ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 0x5581887caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe016ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 0x5581887caba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37126/0 (socket says 10.20.193.28:37126) 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe016ffd640 1 -- 10.20.193.28:0/2356659672 learned_addr learned my addr 10.20.193.28:0/2356659672 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe0177fe640 1 --2- 10.20.193.28:0/2356659672 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 0x558188785f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe017fff640 1 --2- 10.20.193.28:0/2356659672 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5581887cb0e0 0x5581887eb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe0177fe640 1 --2- 10.20.193.28:0/26712890 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdfec0b5760 0x7fdfec0d5c10 secure :-1 s=READY pgs=320 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe004004740 tx=0x7fe004005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe016ffd640 1 -- 10.20.193.28:0/2356659672 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5581887cb0e0 msgr2=0x5581887eb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe016ffd640 1 --2- 10.20.193.28:0/2356659672 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5581887cb0e0 0x5581887eb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe016ffd640 1 -- 10.20.193.28:0/2356659672 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 msgr2=0x558188785f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe016ffd640 1 --2- 10.20.193.28:0/2356659672 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 0x558188785f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.568+0000 7fe016ffd640 1 -- 10.20.193.28:0/2356659672 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55818863cd10 con 0x558188786440 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.569+0000 7fe016ffd640 1 --2- 10.20.193.28:0/2356659672 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 0x5581887caba0 secure :-1 s=READY pgs=621 gs=1 cs=0 l=1 c_cookie=10876046863663690846 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe008005cc0 tx=0x7fe008005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ef7affad2cf45e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.569+0000 7fdfdbfff640 1 -- 10.20.193.28:0/2356659672 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe00803d070 con 0x558188786440 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.569+0000 7fdfdbfff640 1 -- 10.20.193.28:0/2356659672 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe008005dd0 con 0x558188786440 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.569+0000 7fdfdbfff640 1 -- 10.20.193.28:0/2356659672 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe00802fea0 con 0x558188786440 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.569+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2356659672 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 msgr2=0x5581887caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.569+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/2356659672 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 0x5581887caba0 secure :-1 s=READY pgs=621 gs=1 cs=0 l=1 c_cookie=10876046863663690846 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe008005cc0 tx=0x7fe008005ae0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.569+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2356659672 shutdown_connections 2026-02-25T11:53:06.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.569+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/2356659672 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5581887cb0e0 0x5581887eb4c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.569+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/2356659672 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 0x5581887caba0 unknown :-1 s=CLOSED pgs=621 gs=1 cs=0 l=1 c_cookie=10876046863663690846 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.569+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/2356659672 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 0x558188785f00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.569+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2356659672 >> 10.20.193.28:0/2356659672 conn(0x5581886a47a0 msgr2=0x558188784410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:06.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.569+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2356659672 shutdown_connections 2026-02-25T11:53:06.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.569+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2356659672 wait complete. 2026-02-25T11:53:06.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.570+0000 7fe01ff2fe00 1 -- start start 2026-02-25T11:53:06.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe01ff2fe00 1 Processor -- start 2026-02-25T11:53:06.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe01ff2fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 0x55818879df30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe01ff2fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558188786440 0x55818879e470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe01ff2fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 0x5581887a3460 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe01ff2fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5581887a39a0 con 0x5581887cb0e0 2026-02-25T11:53:06.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe01ff2fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5581887a3b10 con 0x558188785b30 2026-02-25T11:53:06.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe01ff2fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5581887a3cd0 con 0x558188786440 2026-02-25T11:53:06.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe017fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 0x5581887a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe017fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 0x5581887a3460 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37130/0 (socket says 10.20.193.28:37130) 2026-02-25T11:53:06.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe017fff640 1 -- 10.20.193.28:0/2693783485 learned_addr learned my addr 10.20.193.28:0/2693783485 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:06.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe016ffd640 1 --2- 10.20.193.28:0/2693783485 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558188786440 0x55818879e470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe0177fe640 1 --2- 10.20.193.28:0/2693783485 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 0x55818879df30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe017fff640 1 -- 10.20.193.28:0/2693783485 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558188786440 msgr2=0x55818879e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe017fff640 1 --2- 10.20.193.28:0/2693783485 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558188786440 0x55818879e470 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe017fff640 1 -- 10.20.193.28:0/2693783485 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 msgr2=0x55818879df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe017fff640 1 --2- 10.20.193.28:0/2693783485 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 0x55818879df30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.571+0000 7fe017fff640 1 -- 10.20.193.28:0/2693783485 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581886153c0 con 0x5581887cb0e0 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fe017fff640 1 --2- 10.20.193.28:0/2693783485 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 0x5581887a3460 secure :-1 s=READY pgs=622 gs=1 cs=0 l=1 c_cookie=3403317065204075237 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe00c00a4e0 tx=0x7fe00c0b01f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3b02d6d9838ae5 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fdfd97fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fdfd97fa640 1 -- 10.20.193.28:0/2693783485 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe00c073390 con 0x5581887cb0e0 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2693783485 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe018112610 con 0x5581887cb0e0 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fdfd97fa640 1 -- 10.20.193.28:0/2693783485 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe00c0068a0 con 0x5581887cb0e0 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fe01ff2fe00 10 client.16488 _ll_register_callbacks cb 0x558188612a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fdfd97fa640 1 -- 10.20.193.28:0/2693783485 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe00c073390 con 0x5581887cb0e0 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2693783485 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5581883276f0 con 0x5581887cb0e0 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fdfd97fa640 1 -- 10.20.193.28:0/2693783485 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fe00c0c5e40 con 0x5581887cb0e0 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fdfd97fa640 1 -- 10.20.193.28:0/2693783485 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe00c0aec70 con 0x5581887cb0e0 2026-02-25T11:53:06.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fe01ff2fe00 10 client.16488 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2693783485 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5581886270e0 con 0x5581887cb0e0 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fdfd97fa640 1 -- 10.20.193.28:0/2693783485 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe00c0b8c90 con 0x5581887cb0e0 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fe01ff2fe00 10 client.16488 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fe01ff2fe00 10 client.16488 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fe01ff2fe00 20 client.16488 populate_metadata read hostname 'trial028' 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/2693783485 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5581887c8810 0x5581887c8c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fe01ff2fe00 4 client.16488 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.572+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2693783485 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558188663fc0 con 0x5581887c8810 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.573+0000 7fe0177fe640 1 --2- 10.20.193.28:0/2693783485 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5581887c8810 0x5581887c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.573+0000 7fe0177fe640 1 --2- 10.20.193.28:0/2693783485 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5581887c8810 0x5581887c8c80 crc :-1 s=READY pgs=42 gs=4 cs=0 l=0 c_cookie=4951945011871034428 s_cookie=9177399863289582666 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=44b8d7c073d72c3c server_cookie=7f5cac8d3d18144a in_seq=0 out_seq=0 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.573+0000 7fdfd97fa640 10 client.16488 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.573+0000 7fdfd97fa640 1 -- 10.20.193.28:0/2693783485 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x558188663fc0 con 0x5581887c8810 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.573+0000 7fdfd97fa640 10 client.16488 handle_command_reply: tid=0 2026-02-25T11:53:06.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.574+0000 7fe01ff2fe00 1 client.16488 shutdown 2026-02-25T11:53:06.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.574+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2693783485 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 msgr2=0x5581887a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.574+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/2693783485 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 0x5581887a3460 secure :-1 s=READY pgs=622 gs=1 cs=0 l=1 c_cookie=3403317065204075237 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe00c00a4e0 tx=0x7fe00c0b01f0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.574+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2693783485 shutdown_connections 2026-02-25T11:53:06.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.574+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/2693783485 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5581887c8810 0x5581887c8c80 crc :-1 s=READY pgs=42 gs=4 cs=0 l=0 c_cookie=4951945011871034428 s_cookie=9177399863289582666 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.574+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/2693783485 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 0x5581887a3460 unknown :-1 s=CLOSED pgs=622 gs=1 cs=0 l=1 c_cookie=3403317065204075237 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.574+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/2693783485 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558188786440 0x55818879e470 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.574+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/2693783485 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 0x55818879df30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.574+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2693783485 >> 10.20.193.28:0/2693783485 conn(0x5581886a47a0 msgr2=0x558188784b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:06.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.574+0000 7fdfd97fa640 0 client.16488 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:53:06.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.574+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2693783485 shutdown_connections 2026-02-25T11:53:06.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.574+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/2693783485 wait complete. 2026-02-25T11:53:06.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.574+0000 7fe01ff2fe00 20 client.16488 trim_cache size 0 max 16384 2026-02-25T11:53:06.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe01ff2fe00 1 Processor -- start 2026-02-25T11:53:06.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe01ff2fe00 1 -- start start 2026-02-25T11:53:06.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe01ff2fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 0x5581887c3940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe01ff2fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 0x5581887c3e80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe01ff2fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5581887cb0e0 0x5581887c9560 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe01ff2fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55818868f1a0 con 0x558188786440 2026-02-25T11:53:06.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe01ff2fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55818834f700 con 0x558188785b30 2026-02-25T11:53:06.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe01ff2fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55818874ab10 con 0x5581887cb0e0 2026-02-25T11:53:06.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe016ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 0x5581887c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe016ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 0x5581887c3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37140/0 (socket says 10.20.193.28:37140) 2026-02-25T11:53:06.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe016ffd640 1 -- 10.20.193.28:0/3855647788 learned_addr learned my addr 10.20.193.28:0/3855647788 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:06.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe0177fe640 1 --2- 10.20.193.28:0/3855647788 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 0x5581887c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe016ffd640 1 -- 10.20.193.28:0/3855647788 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5581887cb0e0 msgr2=0x5581887c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.581+0000 7fe016ffd640 1 --2- 10.20.193.28:0/3855647788 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5581887cb0e0 0x5581887c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fe016ffd640 1 -- 10.20.193.28:0/3855647788 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 msgr2=0x5581887c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fe016ffd640 1 --2- 10.20.193.28:0/3855647788 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 0x5581887c3940 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fe016ffd640 1 -- 10.20.193.28:0/3855647788 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558188790850 con 0x558188786440 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fe016ffd640 1 --2- 10.20.193.28:0/3855647788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 0x5581887c3e80 secure :-1 s=READY pgs=623 gs=1 cs=0 l=1 c_cookie=6901232412286285633 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe0080385a0 tx=0x7fe00802f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc61a508ebc7741 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fdfdb7fe640 1 -- 10.20.193.28:0/3855647788 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe00803d070 con 0x558188786440 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fdfdb7fe640 1 -- 10.20.193.28:0/3855647788 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe008004290 con 0x558188786440 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fdfdb7fe640 1 -- 10.20.193.28:0/3855647788 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe008038bd0 con 0x558188786440 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/3855647788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 msgr2=0x5581887c3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/3855647788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 0x5581887c3e80 secure :-1 s=READY pgs=623 gs=1 cs=0 l=1 c_cookie=6901232412286285633 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe0080385a0 tx=0x7fe00802f9f0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/3855647788 shutdown_connections 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/3855647788 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5581887cb0e0 0x5581887c9560 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/3855647788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558188786440 0x5581887c3e80 unknown :-1 s=CLOSED pgs=623 gs=1 cs=0 l=1 c_cookie=6901232412286285633 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/3855647788 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188785b30 0x5581887c3940 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/3855647788 >> 10.20.193.28:0/3855647788 conn(0x5581886a47a0 msgr2=0x558188783d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:06.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.582+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/3855647788 shutdown_connections 2026-02-25T11:53:06.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.583+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/3855647788 wait complete. 2026-02-25T11:53:06.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.583+0000 7fe01ff2fe00 1 -- start start 2026-02-25T11:53:06.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.583+0000 7fe01ff2fe00 1 Processor -- start 2026-02-25T11:53:06.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe01ff2fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558188785b30 0x5581887a5a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe01ff2fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188786440 0x5581887a5fa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe01ff2fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 0x5581887a64e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe01ff2fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558188830600 con 0x5581887cb0e0 2026-02-25T11:53:06.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe01ff2fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558188830770 con 0x558188786440 2026-02-25T11:53:06.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe01ff2fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558188830930 con 0x558188785b30 2026-02-25T11:53:06.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe017fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 0x5581887a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe017fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 0x5581887a64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37142/0 (socket says 10.20.193.28:37142) 2026-02-25T11:53:06.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe017fff640 1 -- 10.20.193.28:0/1312319799 learned_addr learned my addr 10.20.193.28:0/1312319799 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:06.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe016ffd640 1 --2- 10.20.193.28:0/1312319799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188786440 0x5581887a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe017fff640 1 -- 10.20.193.28:0/1312319799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558188785b30 msgr2=0x5581887a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe017fff640 1 --2- 10.20.193.28:0/1312319799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558188785b30 0x5581887a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe017fff640 1 -- 10.20.193.28:0/1312319799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188786440 msgr2=0x5581887a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:06.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe017fff640 1 --2- 10.20.193.28:0/1312319799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188786440 0x5581887a5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe017fff640 1 -- 10.20.193.28:0/1312319799 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558188621b60 con 0x5581887cb0e0 2026-02-25T11:53:06.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe017fff640 1 --2- 10.20.193.28:0/1312319799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 0x5581887a64e0 secure :-1 s=READY pgs=624 gs=1 cs=0 l=1 c_cookie=5138779578533890140 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe00c0ae250 tx=0x7fe00c0b01f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47509cccdd8b485c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:06.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fdfd97fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:06.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fdfd97fa640 1 -- 10.20.193.28:0/1312319799 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe00c0b6330 con 0x5581887cb0e0 2026-02-25T11:53:06.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fdfd97fa640 1 -- 10.20.193.28:0/1312319799 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5581883276f0 con 0x5581887cb0e0 2026-02-25T11:53:06.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fdfd97fa640 1 -- 10.20.193.28:0/1312319799 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe00c0aec70 con 0x5581887cb0e0 2026-02-25T11:53:06.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.584+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/1312319799 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55818863cd10 con 0x5581887cb0e0 2026-02-25T11:53:06.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fdfd97fa640 1 -- 10.20.193.28:0/1312319799 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fe00c0c5e40 con 0x5581887cb0e0 2026-02-25T11:53:06.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fdfd97fa640 10 client.16494 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:53:06.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fe01ff2fe00 10 client.16494 _ll_register_callbacks cb 0x558188612a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:06.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/1312319799 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe008005dd0 con 0x5581887cb0e0 2026-02-25T11:53:06.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fdfd97fa640 1 -- 10.20.193.28:0/1312319799 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe00c0ae9e0 con 0x5581887cb0e0 2026-02-25T11:53:06.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fe01ff2fe00 10 client.16494 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:06.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/1312319799 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5581886156f0 con 0x5581887cb0e0 2026-02-25T11:53:06.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fdfd97fa640 1 -- 10.20.193.28:0/1312319799 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe00c08b070 con 0x5581887cb0e0 2026-02-25T11:53:06.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fe01ff2fe00 10 client.16494 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:06.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fe01ff2fe00 10 client.16494 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:53:06.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fe01ff2fe00 20 client.16494 populate_metadata read hostname 'trial028' 2026-02-25T11:53:06.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/1312319799 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x558188877f50 0x5581888983d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:06.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fe01ff2fe00 4 client.16494 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:53:06.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.585+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/1312319799 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558188324f40 con 0x558188877f50 2026-02-25T11:53:06.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fe0177fe640 1 --2- 10.20.193.28:0/1312319799 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x558188877f50 0x5581888983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:06.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fe0177fe640 1 --2- 10.20.193.28:0/1312319799 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x558188877f50 0x5581888983d0 crc :-1 s=READY pgs=43 gs=4 cs=0 l=0 c_cookie=10492350007037046830 s_cookie=8634883569113890077 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=919c50bd9482b02e server_cookie=77d544db45e3fd1d in_seq=0 out_seq=0 2026-02-25T11:53:06.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fdfd97fa640 10 client.16494 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:53:06.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fdfd97fa640 1 -- 10.20.193.28:0/1312319799 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558188324f40 con 0x558188877f50 2026-02-25T11:53:06.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fdfd97fa640 10 client.16494 handle_command_reply: tid=0 2026-02-25T11:53:06.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fe01ff2fe00 1 client.16494 shutdown 2026-02-25T11:53:06.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/1312319799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 msgr2=0x5581887a64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/1312319799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 0x5581887a64e0 secure :-1 s=READY pgs=624 gs=1 cs=0 l=1 c_cookie=5138779578533890140 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe00c0ae250 tx=0x7fe00c0b01f0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/1312319799 shutdown_connections 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/1312319799 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x558188877f50 0x5581888983d0 crc :-1 s=READY pgs=43 gs=4 cs=0 l=0 c_cookie=10492350007037046830 s_cookie=8634883569113890077 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/1312319799 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5581887cb0e0 0x5581887a64e0 unknown :-1 s=CLOSED pgs=624 gs=1 cs=0 l=1 c_cookie=5138779578533890140 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/1312319799 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558188786440 0x5581887a5fa0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fe01ff2fe00 1 --2- 10.20.193.28:0/1312319799 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558188785b30 0x5581887a5a60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/1312319799 >> 10.20.193.28:0/1312319799 conn(0x5581886a47a0 msgr2=0x558188784500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.586+0000 7fdfd97fa640 0 client.16494 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/1312319799 shutdown_connections 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ff2fe00 1 -- 10.20.193.28:0/1312319799 wait complete. 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ff2fe00 20 client.16494 trim_cache size 0 max 16384 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ccab640 1 -- 10.20.193.28:0/26712890 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdfec0b5760 msgr2=0x7fdfec0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ccab640 1 --2- 10.20.193.28:0/26712890 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdfec0b5760 0x7fdfec0d5c10 secure :-1 s=READY pgs=320 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe004004740 tx=0x7fe004005f90 comp rx=0 tx=0).stop 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ccab640 1 -- 10.20.193.28:0/26712890 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe018111370 msgr2=0x7fe018128f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:06.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ccab640 1 --2- 10.20.193.28:0/26712890 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe018111370 0x7fe018128f60 secure :-1 s=READY pgs=620 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe00c00f8d0 tx=0x7fe00c00fda0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ccab640 1 -- 10.20.193.28:0/26712890 shutdown_connections 2026-02-25T11:53:06.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ccab640 1 --2- 10.20.193.28:0/26712890 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdfec0b5760 0x7fdfec0d5c10 unknown :-1 s=CLOSED pgs=320 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ccab640 1 --2- 10.20.193.28:0/26712890 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe018111370 0x7fe018128f60 unknown :-1 s=CLOSED pgs=620 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ccab640 1 --2- 10.20.193.28:0/26712890 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe01810bd50 0x7fe018123b90 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ccab640 1 --2- 10.20.193.28:0/26712890 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe01810b360 0x7fe018123650 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:06.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ccab640 1 -- 10.20.193.28:0/26712890 >> 10.20.193.28:0/26712890 conn(0x7fe0180e8e70 msgr2=0x7fe018109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:06.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ccab640 1 -- 10.20.193.28:0/26712890 shutdown_connections 2026-02-25T11:53:06.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:06.587+0000 7fe01ccab640 1 -- 10.20.193.28:0/26712890 wait complete. 2026-02-25T11:53:06.685 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:53:06.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:06 trial167 ceph-mon[43392]: pgmap v372: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:06.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:06 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/147221955' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:53:06.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:06 trial119 ceph-mon[42710]: pgmap v372: 97 pgs: 97 active+clean; 3.9 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:06.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:06 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/147221955' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:53:06.871 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:07.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.055+0000 7fb5d3fff640 1 -- 10.20.193.28:0/33667459 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb5d410f650 msgr2=0x7fb5d410fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:07.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.055+0000 7fb5d3fff640 1 --2- 10.20.193.28:0/33667459 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb5d410f650 0x7fb5d410fa20 secure :-1 s=READY pgs=625 gs=1 cs=0 l=1 c_cookie=14577944046684410653 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5c8009950 tx=0x7fb5c802f190 comp rx=0 tx=0).stop 2026-02-25T11:53:07.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.055+0000 7fb5d3fff640 1 -- 10.20.193.28:0/33667459 shutdown_connections 2026-02-25T11:53:07.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.055+0000 7fb5d3fff640 1 --2- 10.20.193.28:0/33667459 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb5d4115840 0x7fb5d4176210 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.055+0000 7fb5d3fff640 1 --2- 10.20.193.28:0/33667459 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb5d4110040 0x7fb5d4115090 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.055+0000 7fb5d3fff640 1 --2- 10.20.193.28:0/33667459 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb5d410f650 0x7fb5d410fa20 unknown :-1 s=CLOSED pgs=625 gs=1 cs=0 l=1 c_cookie=14577944046684410653 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.055+0000 7fb5d3fff640 1 -- 10.20.193.28:0/33667459 >> 10.20.193.28:0/33667459 conn(0x7fb5d40ecf00 msgr2=0x7fb5d410d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:07.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.055+0000 7fb5d3fff640 1 -- 10.20.193.28:0/33667459 shutdown_connections 2026-02-25T11:53:07.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.055+0000 7fb5d3fff640 1 -- 10.20.193.28:0/33667459 wait complete. 2026-02-25T11:53:07.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.056+0000 7fb5d3fff640 1 Processor -- start 2026-02-25T11:53:07.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.056+0000 7fb5d3fff640 1 -- start start 2026-02-25T11:53:07.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.056+0000 7fb5d3fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb5d410f650 0x7fb5d411df50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.056+0000 7fb5d3fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb5d4110040 0x7fb5d411e490 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.056+0000 7fb5d3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb5d4115840 0x7fb5d4123860 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.056+0000 7fb5d3fff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4123df0 con 0x7fb5d4115840 2026-02-25T11:53:07.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.056+0000 7fb5d3fff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4123f60 con 0x7fb5d4110040 2026-02-25T11:53:07.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.056+0000 7fb5d3fff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d4124120 con 0x7fb5d410f650 2026-02-25T11:53:07.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.057+0000 7fb5d2ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb5d410f650 0x7fb5d411df50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.057+0000 7fb5d2ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb5d410f650 0x7fb5d411df50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52108/0 (socket says 10.20.193.28:52108) 2026-02-25T11:53:07.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.057+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/3853541518 learned_addr learned my addr 10.20.193.28:0/3853541518 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:07.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.057+0000 7fb5d27fc640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb5d4110040 0x7fb5d411e490 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.057+0000 7fb5d37fe640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb5d4115840 0x7fb5d4123860 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.057+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/3853541518 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb5d4110040 msgr2=0x7fb5d411e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:07.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.057+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb5d4110040 0x7fb5d411e490 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.057+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/3853541518 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb5d4115840 msgr2=0x7fb5d4123860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:07.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.057+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb5d4115840 0x7fb5d4123860 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.057+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/3853541518 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c8009570 con 0x7fb5d410f650 2026-02-25T11:53:07.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.057+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb5d410f650 0x7fb5d411df50 secure :-1 s=READY pgs=172 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5c8004820 tx=0x7fb5c80028d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:07.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.058+0000 7fb5d8957640 1 -- 10.20.193.28:0/3853541518 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb5c803d070 con 0x7fb5d410f650 2026-02-25T11:53:07.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.058+0000 7fb5d8957640 1 -- 10.20.193.28:0/3853541518 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb5c80043b0 con 0x7fb5d410f650 2026-02-25T11:53:07.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.058+0000 7fb5d8957640 1 -- 10.20.193.28:0/3853541518 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb5c8041960 con 0x7fb5d410f650 2026-02-25T11:53:07.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.058+0000 7fb5d3fff640 1 -- 10.20.193.28:0/3853541518 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d41243a0 con 0x7fb5d410f650 2026-02-25T11:53:07.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.058+0000 7fb5d3fff640 1 -- 10.20.193.28:0/3853541518 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d41649f0 con 0x7fb5d410f650 2026-02-25T11:53:07.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.063+0000 7fb5db954e00 1 Processor -- start 2026-02-25T11:53:07.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.063+0000 7fb5db954e00 1 -- start start 2026-02-25T11:53:07.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5d8957640 1 -- 10.20.193.28:0/3853541518 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fb5c8041af0 con 0x7fb5d410f650 2026-02-25T11:53:07.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5db954e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3540 0x5601096b3910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5db954e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096b3e50 0x5601096f85b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5db954e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096f8af0 0x560109718ed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5db954e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5601095c08a0 con 0x5601096f8af0 2026-02-25T11:53:07.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5db954e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5601095bc5d0 con 0x5601096b3e50 2026-02-25T11:53:07.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5db954e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5601095b6570 con 0x5601096b3540 2026-02-25T11:53:07.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5d37fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3540 0x5601096b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5d37fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3540 0x5601096b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52114/0 (socket says 10.20.193.28:52114) 2026-02-25T11:53:07.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5d37fe640 1 -- 10.20.193.28:0/943331506 learned_addr learned my addr 10.20.193.28:0/943331506 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:07.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5d2ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096b3e50 0x5601096f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/943331506 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3540 msgr2=0x5601096b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:07.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/943331506 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3540 0x5601096b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/943331506 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096f8af0 msgr2=0x560109718ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:07.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/943331506 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096f8af0 0x560109718ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.064+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/943331506 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560109561d10 con 0x5601096b3e50 2026-02-25T11:53:07.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.065+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/943331506 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096b3e50 0x5601096f85b0 secure :-1 s=READY pgs=144 gs=2 cs=0 l=1 c_cookie=13294751515450422454 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5c8106de0 tx=0x7fb5c8113580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b88072108f8234b6 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:07.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.065+0000 7fb597fff640 1 -- 10.20.193.28:0/943331506 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb5c810d2f0 con 0x5601096b3e50 2026-02-25T11:53:07.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.065+0000 7fb597fff640 1 -- 10.20.193.28:0/943331506 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb5c810ba00 con 0x5601096b3e50 2026-02-25T11:53:07.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.065+0000 7fb597fff640 1 -- 10.20.193.28:0/943331506 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb5c8119ad0 con 0x5601096b3e50 2026-02-25T11:53:07.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.065+0000 7fb5db954e00 1 -- 10.20.193.28:0/943331506 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096b3e50 msgr2=0x5601096f85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:07.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.065+0000 7fb5db954e00 1 --2- 10.20.193.28:0/943331506 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096b3e50 0x5601096f85b0 secure :-1 s=READY pgs=144 gs=2 cs=0 l=1 c_cookie=13294751515450422454 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5c8106de0 tx=0x7fb5c8113580 comp rx=0 tx=0).stop 2026-02-25T11:53:07.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.065+0000 7fb5db954e00 1 -- 10.20.193.28:0/943331506 shutdown_connections 2026-02-25T11:53:07.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.065+0000 7fb5db954e00 1 --2- 10.20.193.28:0/943331506 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096f8af0 0x560109718ed0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.065+0000 7fb5db954e00 1 --2- 10.20.193.28:0/943331506 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096b3e50 0x5601096f85b0 unknown :-1 s=CLOSED pgs=144 gs=2 cs=0 l=1 c_cookie=13294751515450422454 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.065+0000 7fb5db954e00 1 --2- 10.20.193.28:0/943331506 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3540 0x5601096b3910 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.066+0000 7fb5db954e00 1 -- 10.20.193.28:0/943331506 >> 10.20.193.28:0/943331506 conn(0x5601095c97a0 msgr2=0x5601096a9410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:07.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.066+0000 7fb5db954e00 1 -- 10.20.193.28:0/943331506 shutdown_connections 2026-02-25T11:53:07.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.066+0000 7fb5db954e00 1 -- 10.20.193.28:0/943331506 wait complete. 2026-02-25T11:53:07.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.066+0000 7fb5d8957640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb5a80b5790 0x7fb5a80d5c40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.066+0000 7fb5d8957640 1 -- 10.20.193.28:0/3853541518 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb5c810b150 con 0x7fb5d410f650 2026-02-25T11:53:07.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.066+0000 7fb5d37fe640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb5a80b5790 0x7fb5a80d5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.067+0000 7fb5d37fe640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb5a80b5790 0x7fb5a80d5c40 secure :-1 s=READY pgs=321 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5c4005150 tx=0x7fb5c4006f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:07.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.067+0000 7fb5db954e00 1 -- start start 2026-02-25T11:53:07.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.067+0000 7fb5db954e00 1 Processor -- start 2026-02-25T11:53:07.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.067+0000 7fb5db954e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096cb730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.067+0000 7fb5db954e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3e50 0x5601096cbc70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.067+0000 7fb5db954e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096f8af0 0x5601096d0c60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.067+0000 7fb5db954e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5601096d11a0 con 0x5601096b3540 2026-02-25T11:53:07.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.067+0000 7fb5db954e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5601096d1310 con 0x5601096f8af0 2026-02-25T11:53:07.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.067+0000 7fb5db954e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5601096d14d0 con 0x5601096b3e50 2026-02-25T11:53:07.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5d2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5d2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37176/0 (socket says 10.20.193.28:37176) 2026-02-25T11:53:07.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/2135079763 learned_addr learned my addr 10.20.193.28:0/2135079763 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:07.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5d27fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3e50 0x5601096cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5d37fe640 1 --2- 10.20.193.28:0/2135079763 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096f8af0 0x5601096d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/2135079763 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3e50 msgr2=0x5601096cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:07.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/2135079763 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3e50 0x5601096cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/2135079763 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096f8af0 msgr2=0x5601096d0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:07.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/2135079763 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096f8af0 0x5601096d0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/2135079763 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56010953a3c0 con 0x5601096b3540 2026-02-25T11:53:07.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/2135079763 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096cb730 secure :-1 s=READY pgs=626 gs=1 cs=0 l=1 c_cookie=15591338019242251208 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5c80d17f0 tx=0x7fb5c8041410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85f8f5271fa13c8 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:07.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5957fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:07.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5957fa640 1 -- 10.20.193.28:0/2135079763 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb5c8002a40 con 0x5601096b3540 2026-02-25T11:53:07.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5957fa640 1 -- 10.20.193.28:0/2135079763 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb5c80cdbd0 con 0x5601096b3540 2026-02-25T11:53:07.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5957fa640 1 -- 10.20.193.28:0/2135079763 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb5c810b310 con 0x5601096b3540 2026-02-25T11:53:07.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5db954e00 1 -- 10.20.193.28:0/2135079763 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d41649f0 con 0x5601096b3540 2026-02-25T11:53:07.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.068+0000 7fb5957fa640 1 -- 10.20.193.28:0/2135079763 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb5c80f6050 con 0x5601096b3540 2026-02-25T11:53:07.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.069+0000 7fb5957fa640 10 client.16503 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:53:07.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.069+0000 7fb5db954e00 10 client.16503 _ll_register_callbacks cb 0x560109537a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:07.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.069+0000 7fb5db954e00 1 -- 10.20.193.28:0/2135079763 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56010924c6f0 con 0x5601096b3540 2026-02-25T11:53:07.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.069+0000 7fb5957fa640 1 -- 10.20.193.28:0/2135079763 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb5c80d13d0 con 0x5601096b3540 2026-02-25T11:53:07.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.069+0000 7fb5db954e00 10 client.16503 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:07.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.069+0000 7fb5db954e00 1 -- 10.20.193.28:0/2135079763 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x56010954c0e0 con 0x5601096b3540 2026-02-25T11:53:07.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.069+0000 7fb5957fa640 1 -- 10.20.193.28:0/2135079763 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb5c80dc180 con 0x5601096b3540 2026-02-25T11:53:07.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.069+0000 7fb5db954e00 10 client.16503 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:07.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.069+0000 7fb5db954e00 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:07.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.069+0000 7fb5db954e00 20 client.16503 populate_metadata read hostname 'trial028' 2026-02-25T11:53:07.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.069+0000 7fb5db954e00 1 --2- 10.20.193.28:0/2135079763 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5601096f5950 0x5601096f5df0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.069+0000 7fb5db954e00 4 client.16503 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:53:07.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.069+0000 7fb5db954e00 1 -- 10.20.193.28:0/2135079763 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560109588fc0 con 0x5601096f5950 2026-02-25T11:53:07.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.070+0000 7fb5d27fc640 1 --2- 10.20.193.28:0/2135079763 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5601096f5950 0x5601096f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.070+0000 7fb5d27fc640 1 --2- 10.20.193.28:0/2135079763 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5601096f5950 0x5601096f5df0 crc :-1 s=READY pgs=100 gs=4 cs=0 l=0 c_cookie=15455678027597435308 s_cookie=14742755869497896964 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67d994745c151ac server_cookie=cc98ca5a53ba3004 in_seq=0 out_seq=0 2026-02-25T11:53:07.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.070+0000 7fb5957fa640 10 client.16503 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:07.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5957fa640 1 -- 10.20.193.28:0/2135079763 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x560109588fc0 con 0x5601096f5950 2026-02-25T11:53:07.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5957fa640 10 client.16503 handle_command_reply: tid=0 2026-02-25T11:53:07.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5db954e00 1 client.16503 shutdown 2026-02-25T11:53:07.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5db954e00 1 -- 10.20.193.28:0/2135079763 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 msgr2=0x5601096cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:07.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5db954e00 1 --2- 10.20.193.28:0/2135079763 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096cb730 secure :-1 s=READY pgs=626 gs=1 cs=0 l=1 c_cookie=15591338019242251208 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5c80d17f0 tx=0x7fb5c8041410 comp rx=0 tx=0).stop 2026-02-25T11:53:07.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5db954e00 1 -- 10.20.193.28:0/2135079763 shutdown_connections 2026-02-25T11:53:07.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5db954e00 1 --2- 10.20.193.28:0/2135079763 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5601096f5950 0x5601096f5df0 crc :-1 s=READY pgs=100 gs=4 cs=0 l=0 c_cookie=15455678027597435308 s_cookie=14742755869497896964 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5db954e00 1 --2- 10.20.193.28:0/2135079763 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096f8af0 0x5601096d0c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5db954e00 1 --2- 10.20.193.28:0/2135079763 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3e50 0x5601096cbc70 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5db954e00 1 --2- 10.20.193.28:0/2135079763 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096cb730 unknown :-1 s=CLOSED pgs=626 gs=1 cs=0 l=1 c_cookie=15591338019242251208 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5957fa640 0 client.16503 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:07.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5db954e00 1 -- 10.20.193.28:0/2135079763 >> 10.20.193.28:0/2135079763 conn(0x5601095c97a0 msgr2=0x5601096a9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:07.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5db954e00 1 -- 10.20.193.28:0/2135079763 shutdown_connections 2026-02-25T11:53:07.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5db954e00 1 -- 10.20.193.28:0/2135079763 wait complete. 2026-02-25T11:53:07.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.071+0000 7fb5db954e00 20 client.16503 trim_cache size 0 max 16384 2026-02-25T11:53:07.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.078+0000 7fb5db954e00 1 Processor -- start 2026-02-25T11:53:07.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.078+0000 7fb5db954e00 1 -- start start 2026-02-25T11:53:07.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5db954e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096c8d30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5db954e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3e50 0x5601096d01d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5db954e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096f8af0 0x5601096f76e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5db954e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5601095b41a0 con 0x5601096b3540 2026-02-25T11:53:07.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5db954e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560109274700 con 0x5601096f8af0 2026-02-25T11:53:07.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5db954e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5601095c08a0 con 0x5601096b3e50 2026-02-25T11:53:07.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5d2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5d2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37180/0 (socket says 10.20.193.28:37180) 2026-02-25T11:53:07.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/1418341265 learned_addr learned my addr 10.20.193.28:0/1418341265 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:07.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5d27fc640 1 --2- 10.20.193.28:0/1418341265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3e50 0x5601096d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5d37fe640 1 --2- 10.20.193.28:0/1418341265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096f8af0 0x5601096f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/1418341265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3e50 msgr2=0x5601096d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:07.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/1418341265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3e50 0x5601096d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/1418341265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096f8af0 msgr2=0x5601096f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:07.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/1418341265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096f8af0 0x5601096f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/1418341265 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560109546b60 con 0x5601096b3540 2026-02-25T11:53:07.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5d37fe640 1 --2- 10.20.193.28:0/1418341265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096f8af0 0x5601096f76e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:53:07.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.079+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/1418341265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096c8d30 secure :-1 s=READY pgs=627 gs=1 cs=0 l=1 c_cookie=5198839541904294247 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5c810bcf0 tx=0x7fb5c80ea000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4825fd04cb75c167 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:07.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.080+0000 7fb5977fe640 1 -- 10.20.193.28:0/1418341265 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb5c810b310 con 0x5601096b3540 2026-02-25T11:53:07.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.080+0000 7fb5977fe640 1 -- 10.20.193.28:0/1418341265 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb5c80d7020 con 0x5601096b3540 2026-02-25T11:53:07.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.080+0000 7fb5977fe640 1 -- 10.20.193.28:0/1418341265 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb5c810e680 con 0x5601096b3540 2026-02-25T11:53:07.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.080+0000 7fb5db954e00 1 -- 10.20.193.28:0/1418341265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 msgr2=0x5601096c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:07.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.080+0000 7fb5db954e00 1 --2- 10.20.193.28:0/1418341265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096c8d30 secure :-1 s=READY pgs=627 gs=1 cs=0 l=1 c_cookie=5198839541904294247 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5c810bcf0 tx=0x7fb5c80ea000 comp rx=0 tx=0).stop 2026-02-25T11:53:07.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.080+0000 7fb5db954e00 1 -- 10.20.193.28:0/1418341265 shutdown_connections 2026-02-25T11:53:07.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.080+0000 7fb5db954e00 1 --2- 10.20.193.28:0/1418341265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096f8af0 0x5601096f76e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.080+0000 7fb5db954e00 1 --2- 10.20.193.28:0/1418341265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096b3e50 0x5601096d01d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.080+0000 7fb5db954e00 1 --2- 10.20.193.28:0/1418341265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096c8d30 unknown :-1 s=CLOSED pgs=627 gs=1 cs=0 l=1 c_cookie=5198839541904294247 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.080+0000 7fb5db954e00 1 -- 10.20.193.28:0/1418341265 >> 10.20.193.28:0/1418341265 conn(0x5601095c97a0 msgr2=0x5601096cfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:07.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.080+0000 7fb5db954e00 1 -- 10.20.193.28:0/1418341265 shutdown_connections 2026-02-25T11:53:07.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.080+0000 7fb5db954e00 1 -- 10.20.193.28:0/1418341265 wait complete. 2026-02-25T11:53:07.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.081+0000 7fb5db954e00 1 -- start start 2026-02-25T11:53:07.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.081+0000 7fb5db954e00 1 Processor -- start 2026-02-25T11:53:07.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.081+0000 7fb5db954e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096d7290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.081+0000 7fb5db954e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096b3e50 0x5601096d77f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.081+0000 7fb5db954e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096f8af0 0x5601096d2f70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.081+0000 7fb5db954e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5601096f5f80 con 0x5601096b3540 2026-02-25T11:53:07.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.081+0000 7fb5db954e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5601096d36a0 con 0x5601096b3e50 2026-02-25T11:53:07.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.081+0000 7fb5db954e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5601096d3860 con 0x5601096f8af0 2026-02-25T11:53:07.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.081+0000 7fb5d2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096d7290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.081+0000 7fb5d2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096d7290 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37192/0 (socket says 10.20.193.28:37192) 2026-02-25T11:53:07.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.081+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/2172448920 learned_addr learned my addr 10.20.193.28:0/2172448920 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:07.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.081+0000 7fb5d27fc640 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096b3e50 0x5601096d77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5d37fe640 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096f8af0 0x5601096d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/2172448920 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096f8af0 msgr2=0x5601096d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:07.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096f8af0 0x5601096d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/2172448920 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096b3e50 msgr2=0x5601096d77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:07.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096b3e50 0x5601096d77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5d2ffd640 1 -- 10.20.193.28:0/2172448920 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56010953a6f0 con 0x5601096b3540 2026-02-25T11:53:07.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5d37fe640 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096f8af0 0x5601096d2f70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:53:07.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5d2ffd640 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096d7290 secure :-1 s=READY pgs=628 gs=3 cs=0 l=1 c_cookie=7089060028256185282 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5c80cd590 tx=0x7fb5c8041410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6261668947a16bc2 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:07.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5957fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:07.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5957fa640 1 -- 10.20.193.28:0/2172448920 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb5c8002a40 con 0x5601096b3540 2026-02-25T11:53:07.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5db954e00 1 -- 10.20.193.28:0/2172448920 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d41649f0 con 0x5601096b3540 2026-02-25T11:53:07.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5db954e00 10 client.16509 _ll_register_callbacks cb 0x560109537a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:07.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5db954e00 1 -- 10.20.193.28:0/2172448920 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5c810ba00 con 0x5601096b3540 2026-02-25T11:53:07.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5957fa640 1 -- 10.20.193.28:0/2172448920 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x56010924c6f0 con 0x5601096b3540 2026-02-25T11:53:07.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5957fa640 1 -- 10.20.193.28:0/2172448920 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb5c802fe20 con 0x5601096b3540 2026-02-25T11:53:07.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5957fa640 1 -- 10.20.193.28:0/2172448920 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb5c80f6070 con 0x5601096b3540 2026-02-25T11:53:07.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5957fa640 1 -- 10.20.193.28:0/2172448920 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb5c80d97d0 con 0x5601096b3540 2026-02-25T11:53:07.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5db954e00 10 client.16509 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:07.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.082+0000 7fb5db954e00 1 -- 10.20.193.28:0/2172448920 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x56010954c340 con 0x5601096b3540 2026-02-25T11:53:07.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.083+0000 7fb5957fa640 1 -- 10.20.193.28:0/2172448920 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb5c80d3390 con 0x5601096b3540 2026-02-25T11:53:07.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.083+0000 7fb5db954e00 10 client.16509 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:07.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.083+0000 7fb5db954e00 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:07.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.083+0000 7fb5db954e00 20 client.16509 populate_metadata read hostname 'trial028' 2026-02-25T11:53:07.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.083+0000 7fb5db954e00 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5601097a5570 0x5601097c5950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:07.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.083+0000 7fb5db954e00 4 client.16509 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:53:07.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.083+0000 7fb5db954e00 1 -- 10.20.193.28:0/2172448920 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560109249f40 con 0x5601097a5570 2026-02-25T11:53:07.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.083+0000 7fb5d27fc640 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5601097a5570 0x5601097c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:07.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.083+0000 7fb5d27fc640 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5601097a5570 0x5601097c5950 crc :-1 s=READY pgs=101 gs=4 cs=0 l=0 c_cookie=13426795247329712315 s_cookie=7316427787644234811 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba558f23a4fd98bb server_cookie=65892c4f0b54ec3b in_seq=0 out_seq=0 2026-02-25T11:53:07.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.083+0000 7fb5957fa640 10 client.16509 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:07.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5957fa640 1 -- 10.20.193.28:0/2172448920 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560109249f40 con 0x5601097a5570 2026-02-25T11:53:07.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5957fa640 10 client.16509 handle_command_reply: tid=0 2026-02-25T11:53:07.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5db954e00 1 client.16509 shutdown 2026-02-25T11:53:07.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5db954e00 1 -- 10.20.193.28:0/2172448920 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 msgr2=0x5601096d7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:07.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5db954e00 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096d7290 secure :-1 s=READY pgs=628 gs=3 cs=0 l=1 c_cookie=7089060028256185282 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5c80cd590 tx=0x7fb5c8041410 comp rx=0 tx=0).stop 2026-02-25T11:53:07.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5db954e00 1 -- 10.20.193.28:0/2172448920 shutdown_connections 2026-02-25T11:53:07.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5db954e00 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5601097a5570 0x5601097c5950 crc :-1 s=READY pgs=101 gs=4 cs=0 l=0 c_cookie=13426795247329712315 s_cookie=7316427787644234811 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5db954e00 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5601096f8af0 0x5601096d2f70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5db954e00 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5601096b3e50 0x5601096d77f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5db954e00 1 --2- 10.20.193.28:0/2172448920 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5601096b3540 0x5601096d7290 unknown :-1 s=CLOSED pgs=628 gs=3 cs=0 l=1 c_cookie=7089060028256185282 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5db954e00 1 -- 10.20.193.28:0/2172448920 >> 10.20.193.28:0/2172448920 conn(0x5601095c97a0 msgr2=0x5601096a9500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:07.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5957fa640 0 client.16509 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:07.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5db954e00 1 -- 10.20.193.28:0/2172448920 shutdown_connections 2026-02-25T11:53:07.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5db954e00 1 -- 10.20.193.28:0/2172448920 wait complete. 2026-02-25T11:53:07.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.084+0000 7fb5db954e00 20 client.16509 trim_cache size 0 max 16384 2026-02-25T11:53:07.093 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:53:07.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.085+0000 7fb5d3fff640 1 -- 10.20.193.28:0/3853541518 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb5a80b5790 msgr2=0x7fb5a80d5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:07.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.085+0000 7fb5d3fff640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb5a80b5790 0x7fb5a80d5c40 secure :-1 s=READY pgs=321 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5c4005150 tx=0x7fb5c4006f90 comp rx=0 tx=0).stop 2026-02-25T11:53:07.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.085+0000 7fb5d3fff640 1 -- 10.20.193.28:0/3853541518 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb5d410f650 msgr2=0x7fb5d411df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:07.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.085+0000 7fb5d3fff640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb5d410f650 0x7fb5d411df50 secure :-1 s=READY pgs=172 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb5c8004820 tx=0x7fb5c80028d0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.085+0000 7fb5d3fff640 1 -- 10.20.193.28:0/3853541518 shutdown_connections 2026-02-25T11:53:07.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.085+0000 7fb5d3fff640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb5a80b5790 0x7fb5a80d5c40 unknown :-1 s=CLOSED pgs=321 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.085+0000 7fb5d3fff640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb5d4115840 0x7fb5d4123860 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.085+0000 7fb5d3fff640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb5d4110040 0x7fb5d411e490 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.085+0000 7fb5d3fff640 1 --2- 10.20.193.28:0/3853541518 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb5d410f650 0x7fb5d411df50 unknown :-1 s=CLOSED pgs=172 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:07.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.085+0000 7fb5d3fff640 1 -- 10.20.193.28:0/3853541518 >> 10.20.193.28:0/3853541518 conn(0x7fb5d40ecf00 msgr2=0x7fb5d410d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:07.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.085+0000 7fb5d3fff640 1 -- 10.20.193.28:0/3853541518 shutdown_connections 2026-02-25T11:53:07.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:07.085+0000 7fb5d3fff640 1 -- 10.20.193.28:0/3853541518 wait complete. 2026-02-25T11:53:08.191 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:53:08.192 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:53:08.379 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:08.496 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:08 trial028 ceph-mon[39224]: pgmap v373: 97 pgs: 97 active+clean; 4.0 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:53:08.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.564+0000 7f219e210640 1 -- 10.20.193.28:0/2845196261 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f219810ddc0 msgr2=0x7f2198112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:08.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.564+0000 7f219e210640 1 --2- 10.20.193.28:0/2845196261 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f219810ddc0 0x7f2198112e10 secure :-1 s=READY pgs=629 gs=1 cs=0 l=1 c_cookie=54471718363767311 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f218c009950 tx=0x7f218c02f190 comp rx=0 tx=0).stop 2026-02-25T11:53:08.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.564+0000 7f219e210640 1 -- 10.20.193.28:0/2845196261 shutdown_connections 2026-02-25T11:53:08.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.564+0000 7f219e210640 1 --2- 10.20.193.28:0/2845196261 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21981135c0 0x7f2198173f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.564+0000 7f219e210640 1 --2- 10.20.193.28:0/2845196261 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f219810ddc0 0x7f2198112e10 unknown :-1 s=CLOSED pgs=629 gs=1 cs=0 l=1 c_cookie=54471718363767311 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.564+0000 7f219e210640 1 --2- 10.20.193.28:0/2845196261 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f219810d3d0 0x7f219810d7a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.564+0000 7f219e210640 1 -- 10.20.193.28:0/2845196261 >> 10.20.193.28:0/2845196261 conn(0x7f21980eaca0 msgr2=0x7f219810b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:08.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.564+0000 7f219e210640 1 -- 10.20.193.28:0/2845196261 shutdown_connections 2026-02-25T11:53:08.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.564+0000 7f219e210640 1 -- 10.20.193.28:0/2845196261 wait complete. 2026-02-25T11:53:08.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219e210640 1 Processor -- start 2026-02-25T11:53:08.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219e210640 1 -- start start 2026-02-25T11:53:08.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219e210640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f219810d3d0 0x7f21980f59b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219e210640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f219810ddc0 0x7f2198173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219e210640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21981135c0 0x7f21980f60b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219e210640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f21980fb2c0 con 0x7f219810ddc0 2026-02-25T11:53:08.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219e210640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f21980fb430 con 0x7f219810d3d0 2026-02-25T11:53:08.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219e210640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f21980fb5f0 con 0x7f21981135c0 2026-02-25T11:53:08.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219ca0d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f219810ddc0 0x7f2198173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219d20e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f219810d3d0 0x7f21980f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219ca0d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f219810ddc0 0x7f2198173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37208/0 (socket says 10.20.193.28:37208) 2026-02-25T11:53:08.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219ca0d640 1 -- 10.20.193.28:0/2505960994 learned_addr learned my addr 10.20.193.28:0/2505960994 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:08.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219da0f640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21981135c0 0x7f21980f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219ca0d640 1 -- 10.20.193.28:0/2505960994 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21981135c0 msgr2=0x7f21980f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:08.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219ca0d640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21981135c0 0x7f21980f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219ca0d640 1 -- 10.20.193.28:0/2505960994 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f219810d3d0 msgr2=0x7f21980f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:08.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219ca0d640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f219810d3d0 0x7f21980f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219ca0d640 1 -- 10.20.193.28:0/2505960994 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f218c009570 con 0x7f219810ddc0 2026-02-25T11:53:08.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.565+0000 7f219da0f640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21981135c0 0x7f21980f60b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:53:08.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.566+0000 7f219ca0d640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f219810ddc0 0x7f2198173e90 secure :-1 s=READY pgs=630 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f218c0098a0 tx=0x7f218c0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:08.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.566+0000 7f21867fc640 1 -- 10.20.193.28:0/2505960994 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f218c03d070 con 0x7f219810ddc0 2026-02-25T11:53:08.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.566+0000 7f21867fc640 1 -- 10.20.193.28:0/2505960994 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f218c002a40 con 0x7f219810ddc0 2026-02-25T11:53:08.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.566+0000 7f219e210640 1 -- 10.20.193.28:0/2505960994 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21980fb870 con 0x7f219810ddc0 2026-02-25T11:53:08.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.566+0000 7f21867fc640 1 -- 10.20.193.28:0/2505960994 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f218c041720 con 0x7f219810ddc0 2026-02-25T11:53:08.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.566+0000 7f219e210640 1 -- 10.20.193.28:0/2505960994 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21980fbd30 con 0x7f219810ddc0 2026-02-25T11:53:08.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.569+0000 7f21867fc640 1 -- 10.20.193.28:0/2505960994 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f218c0386a0 con 0x7f219810ddc0 2026-02-25T11:53:08.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.570+0000 7f21867fc640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f21740b5750 0x7f21740d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.571+0000 7f21867fc640 1 -- 10.20.193.28:0/2505960994 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f218c10b060 con 0x7f219810ddc0 2026-02-25T11:53:08.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.571+0000 7f219d20e640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f21740b5750 0x7f21740d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.571+0000 7f21a1494e00 1 Processor -- start 2026-02-25T11:53:08.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.571+0000 7f219d20e640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f21740b5750 0x7f21740d5c00 secure :-1 s=READY pgs=322 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2188004740 tx=0x7f2188005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:08.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f21a1494e00 1 -- start start 2026-02-25T11:53:08.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f21a1494e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb25160 0x564fdbb25530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f21a1494e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25a70 0x564fdbb6a020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f21a1494e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb6a560 0x564fdbb8a940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f21a1494e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564fdba32d80 con 0x564fdbb25a70 2026-02-25T11:53:08.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f21a1494e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564fdba2eab0 con 0x564fdbb6a560 2026-02-25T11:53:08.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f21a1494e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564fdba28a50 con 0x564fdbb25160 2026-02-25T11:53:08.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f219ca0d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25a70 0x564fdbb6a020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f219da0f640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb6a560 0x564fdbb8a940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f219ca0d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25a70 0x564fdbb6a020 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37220/0 (socket says 10.20.193.28:37220) 2026-02-25T11:53:08.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f219ca0d640 1 -- 10.20.193.28:0/4009635630 learned_addr learned my addr 10.20.193.28:0/4009635630 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:08.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f219d20e640 1 --2- 10.20.193.28:0/4009635630 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb25160 0x564fdbb25530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f219da0f640 1 -- 10.20.193.28:0/4009635630 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb25160 msgr2=0x564fdbb25530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:08.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f219da0f640 1 --2- 10.20.193.28:0/4009635630 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb25160 0x564fdbb25530 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f219da0f640 1 -- 10.20.193.28:0/4009635630 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25a70 msgr2=0x564fdbb6a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:08.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f219da0f640 1 --2- 10.20.193.28:0/4009635630 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25a70 0x564fdbb6a020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.572+0000 7f219da0f640 1 -- 10.20.193.28:0/4009635630 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fdb9d4120 con 0x564fdbb6a560 2026-02-25T11:53:08.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.573+0000 7f219da0f640 1 --2- 10.20.193.28:0/4009635630 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb6a560 0x564fdbb8a940 secure :-1 s=READY pgs=145 gs=2 cs=0 l=1 c_cookie=4319656341785466235 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f21940055e0 tx=0x7f2194003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bf2808e6be2d57b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:08.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.573+0000 7f2165ce5640 1 -- 10.20.193.28:0/4009635630 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f219400add0 con 0x564fdbb6a560 2026-02-25T11:53:08.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.573+0000 7f2165ce5640 1 -- 10.20.193.28:0/4009635630 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2194003e30 con 0x564fdbb6a560 2026-02-25T11:53:08.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.573+0000 7f2165ce5640 1 -- 10.20.193.28:0/4009635630 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f21940128f0 con 0x564fdbb6a560 2026-02-25T11:53:08.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.573+0000 7f21a1494e00 1 -- 10.20.193.28:0/4009635630 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb6a560 msgr2=0x564fdbb8a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:08.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.573+0000 7f21a1494e00 1 --2- 10.20.193.28:0/4009635630 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb6a560 0x564fdbb8a940 secure :-1 s=READY pgs=145 gs=2 cs=0 l=1 c_cookie=4319656341785466235 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f21940055e0 tx=0x7f2194003c30 comp rx=0 tx=0).stop 2026-02-25T11:53:08.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.573+0000 7f21a1494e00 1 -- 10.20.193.28:0/4009635630 shutdown_connections 2026-02-25T11:53:08.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.573+0000 7f21a1494e00 1 --2- 10.20.193.28:0/4009635630 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb6a560 0x564fdbb8a940 unknown :-1 s=CLOSED pgs=145 gs=2 cs=0 l=1 c_cookie=4319656341785466235 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.573+0000 7f21a1494e00 1 --2- 10.20.193.28:0/4009635630 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25a70 0x564fdbb6a020 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.573+0000 7f21a1494e00 1 --2- 10.20.193.28:0/4009635630 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb25160 0x564fdbb25530 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.573+0000 7f21a1494e00 1 -- 10.20.193.28:0/4009635630 >> 10.20.193.28:0/4009635630 conn(0x564fdbafa8f0 msgr2=0x564fdbb1acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:08.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.573+0000 7f21a1494e00 1 -- 10.20.193.28:0/4009635630 shutdown_connections 2026-02-25T11:53:08.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.573+0000 7f21a1494e00 1 -- 10.20.193.28:0/4009635630 wait complete. 2026-02-25T11:53:08.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.574+0000 7f21a1494e00 1 -- start start 2026-02-25T11:53:08.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.575+0000 7f21a1494e00 1 Processor -- start 2026-02-25T11:53:08.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.575+0000 7f21a1494e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25160 0x564fdbb81ea0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.575+0000 7f21a1494e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25a70 0x564fdbb823e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.575+0000 7f21a1494e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb6a560 0x564fdbb873d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.575+0000 7f21a1494e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564fdbb87910 con 0x564fdbb25a70 2026-02-25T11:53:08.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.575+0000 7f21a1494e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564fdbb87a80 con 0x564fdbb25160 2026-02-25T11:53:08.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.575+0000 7f21a1494e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564fdbb87c40 con 0x564fdbb6a560 2026-02-25T11:53:08.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.575+0000 7f219ca0d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25a70 0x564fdbb823e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.575+0000 7f219ca0d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25a70 0x564fdbb823e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37230/0 (socket says 10.20.193.28:37230) 2026-02-25T11:53:08.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.575+0000 7f219ca0d640 1 -- 10.20.193.28:0/56266199 learned_addr learned my addr 10.20.193.28:0/56266199 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:08.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.575+0000 7f219da0f640 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb6a560 0x564fdbb873d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.575+0000 7f219d20e640 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25160 0x564fdbb81ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f219da0f640 1 -- 10.20.193.28:0/56266199 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25160 msgr2=0x564fdbb81ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:08.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f219da0f640 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25160 0x564fdbb81ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f219da0f640 1 -- 10.20.193.28:0/56266199 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25a70 msgr2=0x564fdbb823e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:08.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f219da0f640 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25a70 0x564fdbb823e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f219da0f640 1 -- 10.20.193.28:0/56266199 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fdb9ac810 con 0x564fdbb6a560 2026-02-25T11:53:08.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f219ca0d640 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25a70 0x564fdbb823e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:53:08.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f219da0f640 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb6a560 0x564fdbb873d0 secure :-1 s=READY pgs=173 gs=1 cs=0 l=1 c_cookie=11055181548124340413 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2194012480 tx=0x7f2194004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=996be50b8f6780bd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:08.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f214effd640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:53:08.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f214effd640 1 -- 10.20.193.28:0/56266199 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2194012a80 con 0x564fdbb6a560 2026-02-25T11:53:08.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f21a1494e00 1 -- 10.20.193.28:0/56266199 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2194005200 con 0x564fdbb6a560 2026-02-25T11:53:08.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f214effd640 1 -- 10.20.193.28:0/56266199 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f219401b930 con 0x564fdbb6a560 2026-02-25T11:53:08.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f214effd640 1 -- 10.20.193.28:0/56266199 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2194026730 con 0x564fdbb6a560 2026-02-25T11:53:08.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f21a1494e00 10 client.26257 _ll_register_callbacks cb 0x564fdb9a9e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:08.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f21a1494e00 1 -- 10.20.193.28:0/56266199 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564fdb780530 con 0x564fdbb6a560 2026-02-25T11:53:08.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.576+0000 7f214effd640 1 -- 10.20.193.28:0/56266199 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f21940268c0 con 0x564fdbb6a560 2026-02-25T11:53:08.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.577+0000 7f214effd640 1 -- 10.20.193.28:0/56266199 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2194026aa0 con 0x564fdbb6a560 2026-02-25T11:53:08.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.577+0000 7f21a1494e00 10 client.26257 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:08.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.577+0000 7f21a1494e00 1 -- 10.20.193.28:0/56266199 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x564fdb9be4f0 con 0x564fdbb6a560 2026-02-25T11:53:08.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.577+0000 7f214effd640 1 -- 10.20.193.28:0/56266199 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f2194028d10 con 0x564fdbb6a560 2026-02-25T11:53:08.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.577+0000 7f21a1494e00 10 client.26257 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:08.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.577+0000 7f21a1494e00 10 client.26257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:08.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.577+0000 7f21a1494e00 20 client.26257 populate_metadata read hostname 'trial028' 2026-02-25T11:53:08.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.577+0000 7f21a1494e00 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564fdbb67780 0x564fdbb67c20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.577+0000 7f21a1494e00 4 client.26257 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:53:08.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.577+0000 7f21a1494e00 1 -- 10.20.193.28:0/56266199 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fdb9d5b50 con 0x564fdbb67780 2026-02-25T11:53:08.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.577+0000 7f219d20e640 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564fdbb67780 0x564fdbb67c20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.578+0000 7f219d20e640 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564fdbb67780 0x564fdbb67c20 crc :-1 s=READY pgs=102 gs=4 cs=0 l=0 c_cookie=18278418484884911687 s_cookie=9658083237048294396 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fda9fcc4cf7fb647 server_cookie=860867888b6b1bfc in_seq=0 out_seq=0 2026-02-25T11:53:08.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.578+0000 7f214effd640 10 client.26257 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:08.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.578+0000 7f214effd640 1 -- 10.20.193.28:0/56266199 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x564fdb9d5b50 con 0x564fdbb67780 2026-02-25T11:53:08.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.578+0000 7f214effd640 10 client.26257 handle_command_reply: tid=0 2026-02-25T11:53:08.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.578+0000 7f21a1494e00 1 client.26257 shutdown 2026-02-25T11:53:08.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.579+0000 7f21a1494e00 1 -- 10.20.193.28:0/56266199 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb6a560 msgr2=0x564fdbb873d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:08.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.579+0000 7f21a1494e00 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb6a560 0x564fdbb873d0 secure :-1 s=READY pgs=173 gs=1 cs=0 l=1 c_cookie=11055181548124340413 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2194012480 tx=0x7f2194004fb0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.579+0000 7f21a1494e00 1 -- 10.20.193.28:0/56266199 shutdown_connections 2026-02-25T11:53:08.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.579+0000 7f21a1494e00 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564fdbb67780 0x564fdbb67c20 crc :-1 s=READY pgs=102 gs=4 cs=0 l=0 c_cookie=18278418484884911687 s_cookie=9658083237048294396 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.579+0000 7f214effd640 0 client.26257 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:08.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.579+0000 7f21a1494e00 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb6a560 0x564fdbb873d0 unknown :-1 s=CLOSED pgs=173 gs=1 cs=0 l=1 c_cookie=11055181548124340413 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.579+0000 7f21a1494e00 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25a70 0x564fdbb823e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.579+0000 7f21a1494e00 1 --2- 10.20.193.28:0/56266199 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25160 0x564fdbb81ea0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.579+0000 7f21a1494e00 1 -- 10.20.193.28:0/56266199 >> 10.20.193.28:0/56266199 conn(0x564fdbafa8f0 msgr2=0x564fdbb1acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:08.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.579+0000 7f21a1494e00 1 -- 10.20.193.28:0/56266199 shutdown_connections 2026-02-25T11:53:08.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.579+0000 7f21a1494e00 1 -- 10.20.193.28:0/56266199 wait complete. 2026-02-25T11:53:08.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.579+0000 7f21a1494e00 20 client.26257 trim_cache size 0 max 16384 2026-02-25T11:53:08.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.589+0000 7f21a1494e00 1 Processor -- start 2026-02-25T11:53:08.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.589+0000 7f21a1494e00 1 -- start start 2026-02-25T11:53:08.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.589+0000 7f21a1494e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25160 0x564fdbb7f040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.589+0000 7f21a1494e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb25a70 0x564fdbb7f580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.589+0000 7f21a1494e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb67780 0x564fdbb65150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.589+0000 7f21a1494e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564fdba26680 con 0x564fdbb67780 2026-02-25T11:53:08.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.589+0000 7f21a1494e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564fdb6e6790 con 0x564fdbb25160 2026-02-25T11:53:08.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.589+0000 7f21a1494e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564fdbb67d80 con 0x564fdbb25a70 2026-02-25T11:53:08.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.589+0000 7f219d20e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25160 0x564fdbb7f040 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.589+0000 7f219d20e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25160 0x564fdbb7f040 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:57470/0 (socket says 10.20.193.28:57470) 2026-02-25T11:53:08.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.589+0000 7f219d20e640 1 -- 10.20.193.28:0/3583032518 learned_addr learned my addr 10.20.193.28:0/3583032518 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:08.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.589+0000 7f219da0f640 1 --2- 10.20.193.28:0/3583032518 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb67780 0x564fdbb65150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f219ca0d640 1 --2- 10.20.193.28:0/3583032518 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb25a70 0x564fdbb7f580 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f219d20e640 1 -- 10.20.193.28:0/3583032518 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb25a70 msgr2=0x564fdbb7f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:08.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f219d20e640 1 --2- 10.20.193.28:0/3583032518 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb25a70 0x564fdbb7f580 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f219d20e640 1 -- 10.20.193.28:0/3583032518 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb67780 msgr2=0x564fdbb65150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:08.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f219d20e640 1 --2- 10.20.193.28:0/3583032518 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb67780 0x564fdbb65150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f219d20e640 1 -- 10.20.193.28:0/3583032518 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fdbb74430 con 0x564fdbb25160 2026-02-25T11:53:08.593 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f219ca0d640 1 --2- 10.20.193.28:0/3583032518 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb25a70 0x564fdbb7f580 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:53:08.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f219d20e640 1 --2- 10.20.193.28:0/3583032518 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25160 0x564fdbb7f040 secure :-1 s=READY pgs=146 gs=1 cs=0 l=1 c_cookie=4061037438538310326 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2188009aa0 tx=0x7f2188014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=385bb408b7a362b6 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:08.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f21654e4640 1 -- 10.20.193.28:0/3583032518 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2188021030 con 0x564fdbb25160 2026-02-25T11:53:08.596 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f21654e4640 1 -- 10.20.193.28:0/3583032518 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2188022040 con 0x564fdbb25160 2026-02-25T11:53:08.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f21654e4640 1 -- 10.20.193.28:0/3583032518 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2188011800 con 0x564fdbb25160 2026-02-25T11:53:08.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f21a1494e00 1 -- 10.20.193.28:0/3583032518 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25160 msgr2=0x564fdbb7f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:08.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f21a1494e00 1 --2- 10.20.193.28:0/3583032518 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25160 0x564fdbb7f040 secure :-1 s=READY pgs=146 gs=1 cs=0 l=1 c_cookie=4061037438538310326 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2188009aa0 tx=0x7f2188014000 comp rx=0 tx=0).stop 2026-02-25T11:53:08.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f21a1494e00 1 -- 10.20.193.28:0/3583032518 shutdown_connections 2026-02-25T11:53:08.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f21a1494e00 1 --2- 10.20.193.28:0/3583032518 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb67780 0x564fdbb65150 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f21a1494e00 1 --2- 10.20.193.28:0/3583032518 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb25a70 0x564fdbb7f580 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f21a1494e00 1 --2- 10.20.193.28:0/3583032518 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25160 0x564fdbb7f040 unknown :-1 s=CLOSED pgs=146 gs=1 cs=0 l=1 c_cookie=4061037438538310326 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f21a1494e00 1 -- 10.20.193.28:0/3583032518 >> 10.20.193.28:0/3583032518 conn(0x564fdbafa8f0 msgr2=0x564fdbb7e930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:08.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.590+0000 7f21a1494e00 1 -- 10.20.193.28:0/3583032518 shutdown_connections 2026-02-25T11:53:08.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.591+0000 7f21a1494e00 1 -- 10.20.193.28:0/3583032518 wait complete. 2026-02-25T11:53:08.597 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.591+0000 7f21a1494e00 1 -- start start 2026-02-25T11:53:08.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.591+0000 7f21a1494e00 1 Processor -- start 2026-02-25T11:53:08.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f21a1494e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25160 0x564fdbafc790 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f21a1494e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25a70 0x564fdbafccd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f21a1494e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb67780 0x564fdbbd03a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f21a1494e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564fdbbd08e0 con 0x564fdbb25160 2026-02-25T11:53:08.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f21a1494e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564fdbbd0a50 con 0x564fdbb25a70 2026-02-25T11:53:08.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f21a1494e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564fdbbd0c10 con 0x564fdbb67780 2026-02-25T11:53:08.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f219d20e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25160 0x564fdbafc790 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f219ca0d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25a70 0x564fdbafccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f219d20e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25160 0x564fdbafc790 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37244/0 (socket says 10.20.193.28:37244) 2026-02-25T11:53:08.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f219d20e640 1 -- 10.20.193.28:0/2001143863 learned_addr learned my addr 10.20.193.28:0/2001143863 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:08.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f219da0f640 1 --2- 10.20.193.28:0/2001143863 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb67780 0x564fdbbd03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.598 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f219d20e640 1 -- 10.20.193.28:0/2001143863 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb67780 msgr2=0x564fdbbd03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:08.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f219d20e640 1 --2- 10.20.193.28:0/2001143863 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb67780 0x564fdbbd03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f219d20e640 1 -- 10.20.193.28:0/2001143863 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25a70 msgr2=0x564fdbafccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:08.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f219d20e640 1 --2- 10.20.193.28:0/2001143863 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25a70 0x564fdbafccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f219d20e640 1 -- 10.20.193.28:0/2001143863 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fdb9b8f70 con 0x564fdbb25160 2026-02-25T11:53:08.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f219d20e640 1 --2- 10.20.193.28:0/2001143863 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25160 0x564fdbafc790 secure :-1 s=READY pgs=631 gs=1 cs=0 l=1 c_cookie=4393666292004378302 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f21880046c0 tx=0x7f21880053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf97036c48756be server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:08.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.592+0000 7f214effd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:08.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f214effd640 1 -- 10.20.193.28:0/2001143863 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2188021070 con 0x564fdbb25160 2026-02-25T11:53:08.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f21a1494e00 1 -- 10.20.193.28:0/2001143863 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2188009e10 con 0x564fdbb25160 2026-02-25T11:53:08.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f214effd640 1 -- 10.20.193.28:0/2001143863 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f21880117a0 con 0x564fdbb25160 2026-02-25T11:53:08.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f214effd640 1 -- 10.20.193.28:0/2001143863 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2188021070 con 0x564fdbb25160 2026-02-25T11:53:08.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f21a1494e00 10 client.16527 _ll_register_callbacks cb 0x564fdb9a9e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:08.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f21a1494e00 1 -- 10.20.193.28:0/2001143863 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2194003e30 con 0x564fdbb25160 2026-02-25T11:53:08.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f214effd640 1 -- 10.20.193.28:0/2001143863 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f21880285e0 con 0x564fdbb25160 2026-02-25T11:53:08.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f214effd640 1 -- 10.20.193.28:0/2001143863 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f21880287e0 con 0x564fdbb25160 2026-02-25T11:53:08.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f21a1494e00 10 client.16527 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:08.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f21a1494e00 1 -- 10.20.193.28:0/2001143863 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x564fdb9acb40 con 0x564fdbb25160 2026-02-25T11:53:08.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f214effd640 1 -- 10.20.193.28:0/2001143863 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f2188004770 con 0x564fdbb25160 2026-02-25T11:53:08.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f21a1494e00 10 client.16527 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:08.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f21a1494e00 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:08.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f21a1494e00 20 client.16527 populate_metadata read hostname 'trial028' 2026-02-25T11:53:08.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f21a1494e00 1 --2- 10.20.193.28:0/2001143863 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564fdbb6a560 0x564fdbc38a70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:08.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f21a1494e00 4 client.16527 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:53:08.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f21a1494e00 1 -- 10.20.193.28:0/2001143863 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564fdb889690 con 0x564fdbb6a560 2026-02-25T11:53:08.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.593+0000 7f219ca0d640 1 --2- 10.20.193.28:0/2001143863 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564fdbb6a560 0x564fdbc38a70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:08.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.594+0000 7f219ca0d640 1 --2- 10.20.193.28:0/2001143863 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564fdbb6a560 0x564fdbc38a70 crc :-1 s=READY pgs=103 gs=4 cs=0 l=0 c_cookie=10178779545777371226 s_cookie=6592759097039619362 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d424a114112785a server_cookie=5b7e2f7804c58522 in_seq=0 out_seq=0 2026-02-25T11:53:08.601 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.594+0000 7f214effd640 10 client.16527 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:08.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f214effd640 1 -- 10.20.193.28:0/2001143863 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564fdb889690 con 0x564fdbb6a560 2026-02-25T11:53:08.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f214effd640 10 client.16527 handle_command_reply: tid=0 2026-02-25T11:53:08.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f21a1494e00 1 client.16527 shutdown 2026-02-25T11:53:08.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f21a1494e00 1 -- 10.20.193.28:0/2001143863 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25160 msgr2=0x564fdbafc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:08.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f21a1494e00 1 --2- 10.20.193.28:0/2001143863 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25160 0x564fdbafc790 secure :-1 s=READY pgs=631 gs=1 cs=0 l=1 c_cookie=4393666292004378302 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f21880046c0 tx=0x7f21880053e0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f21a1494e00 1 -- 10.20.193.28:0/2001143863 shutdown_connections 2026-02-25T11:53:08.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f21a1494e00 1 --2- 10.20.193.28:0/2001143863 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564fdbb6a560 0x564fdbc38a70 crc :-1 s=READY pgs=103 gs=4 cs=0 l=0 c_cookie=10178779545777371226 s_cookie=6592759097039619362 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f21a1494e00 1 --2- 10.20.193.28:0/2001143863 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564fdbb67780 0x564fdbbd03a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f21a1494e00 1 --2- 10.20.193.28:0/2001143863 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564fdbb25a70 0x564fdbafccd0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f21a1494e00 1 --2- 10.20.193.28:0/2001143863 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564fdbb25160 0x564fdbafc790 unknown :-1 s=CLOSED pgs=631 gs=1 cs=0 l=1 c_cookie=4393666292004378302 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f21a1494e00 1 -- 10.20.193.28:0/2001143863 >> 10.20.193.28:0/2001143863 conn(0x564fdbafa8f0 msgr2=0x564fdbb7f040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:08.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f214effd640 0 client.16527 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:08.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f21a1494e00 1 -- 10.20.193.28:0/2001143863 shutdown_connections 2026-02-25T11:53:08.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f21a1494e00 1 -- 10.20.193.28:0/2001143863 wait complete. 2026-02-25T11:53:08.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.595+0000 7f21a1494e00 20 client.16527 trim_cache size 0 max 16384 2026-02-25T11:53:08.604 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:53:08.604 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:53:08.604 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847", 2026-02-25T11:53:08.604 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:53:08.604 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:53:08.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.596+0000 7f219e210640 1 -- 10.20.193.28:0/2505960994 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f21740b5750 msgr2=0x7f21740d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:08.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.596+0000 7f219e210640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f21740b5750 0x7f21740d5c00 secure :-1 s=READY pgs=322 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2188004740 tx=0x7f2188005f90 comp rx=0 tx=0).stop 2026-02-25T11:53:08.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.596+0000 7f219e210640 1 -- 10.20.193.28:0/2505960994 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f219810ddc0 msgr2=0x7f2198173e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:08.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.596+0000 7f219e210640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f219810ddc0 0x7f2198173e90 secure :-1 s=READY pgs=630 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f218c0098a0 tx=0x7f218c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.596+0000 7f219e210640 1 -- 10.20.193.28:0/2505960994 shutdown_connections 2026-02-25T11:53:08.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.596+0000 7f219e210640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f21740b5750 0x7f21740d5c00 unknown :-1 s=CLOSED pgs=322 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.596+0000 7f219e210640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f21981135c0 0x7f21980f60b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.596+0000 7f219e210640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f219810ddc0 0x7f2198173e90 unknown :-1 s=CLOSED pgs=630 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.596+0000 7f219e210640 1 --2- 10.20.193.28:0/2505960994 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f219810d3d0 0x7f21980f59b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:08.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.596+0000 7f219e210640 1 -- 10.20.193.28:0/2505960994 >> 10.20.193.28:0/2505960994 conn(0x7f21980eaca0 msgr2=0x7f219810fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:08.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.596+0000 7f219e210640 1 -- 10.20.193.28:0/2505960994 shutdown_connections 2026-02-25T11:53:08.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:08.596+0000 7f219e210640 1 -- 10.20.193.28:0/2505960994 wait complete. 2026-02-25T11:53:08.696 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:53:08.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:08 trial167 ceph-mon[43392]: pgmap v373: 97 pgs: 97 active+clean; 4.0 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:53:08.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:08 trial119 ceph-mon[42710]: pgmap v373: 97 pgs: 97 active+clean; 4.0 MiB data, 340 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:53:08.881 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:09.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.051+0000 7fac8ee0c640 1 -- 10.20.193.28:0/2869807957 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fac88115930 msgr2=0x7fac88176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:09.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.051+0000 7fac8ee0c640 1 --2- 10.20.193.28:0/2869807957 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fac88115930 0x7fac88176300 secure :-1 s=READY pgs=632 gs=1 cs=0 l=1 c_cookie=17527934688419709529 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fac8400aa30 tx=0x7fac84030270 comp rx=0 tx=0).stop 2026-02-25T11:53:09.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.051+0000 7fac8ee0c640 1 -- 10.20.193.28:0/2869807957 shutdown_connections 2026-02-25T11:53:09.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.051+0000 7fac8ee0c640 1 --2- 10.20.193.28:0/2869807957 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fac88115930 0x7fac88176300 unknown :-1 s=CLOSED pgs=632 gs=1 cs=0 l=1 c_cookie=17527934688419709529 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.051+0000 7fac8ee0c640 1 --2- 10.20.193.28:0/2869807957 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fac88110130 0x7fac88115180 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.051+0000 7fac8ee0c640 1 --2- 10.20.193.28:0/2869807957 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fac8810f740 0x7fac8810fb10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.051+0000 7fac8ee0c640 1 -- 10.20.193.28:0/2869807957 >> 10.20.193.28:0/2869807957 conn(0x7fac880ecff0 msgr2=0x7fac8810d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:09.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.051+0000 7fac8ee0c640 1 -- 10.20.193.28:0/2869807957 shutdown_connections 2026-02-25T11:53:09.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.051+0000 7fac8ee0c640 1 -- 10.20.193.28:0/2869807957 wait complete. 2026-02-25T11:53:09.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.051+0000 7fac8ee0c640 1 Processor -- start 2026-02-25T11:53:09.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8ee0c640 1 -- start start 2026-02-25T11:53:09.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8ee0c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fac8810f740 0x7fac880f5a30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8ee0c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fac88110130 0x7fac88176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8ee0c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fac88115930 0x7fac880f60c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8ee0c640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fac880fb340 con 0x7fac88115930 2026-02-25T11:53:09.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8ee0c640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fac880fb4b0 con 0x7fac88110130 2026-02-25T11:53:09.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8ee0c640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac880fb620 con 0x7fac8810f740 2026-02-25T11:53:09.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8cb81640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fac8810f740 0x7fac880f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8cb81640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fac8810f740 0x7fac880f5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52224/0 (socket says 10.20.193.28:52224) 2026-02-25T11:53:09.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8cb81640 1 -- 10.20.193.28:0/1546272196 learned_addr learned my addr 10.20.193.28:0/1546272196 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:09.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac7ffff640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fac88110130 0x7fac88176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8d382640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fac88115930 0x7fac880f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8cb81640 1 -- 10.20.193.28:0/1546272196 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fac88110130 msgr2=0x7fac88176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:09.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8cb81640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fac88110130 0x7fac88176200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8cb81640 1 -- 10.20.193.28:0/1546272196 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fac88115930 msgr2=0x7fac880f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:09.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8cb81640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fac88115930 0x7fac880f60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.052+0000 7fac8cb81640 1 -- 10.20.193.28:0/1546272196 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8400a650 con 0x7fac8810f740 2026-02-25T11:53:09.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.053+0000 7fac8cb81640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fac8810f740 0x7fac880f5a30 secure :-1 s=READY pgs=174 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fac7800d950 tx=0x7fac7800de20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:09.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.053+0000 7fac7dffb640 1 -- 10.20.193.28:0/1546272196 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fac78004540 con 0x7fac8810f740 2026-02-25T11:53:09.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.053+0000 7fac7dffb640 1 -- 10.20.193.28:0/1546272196 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fac78009ce0 con 0x7fac8810f740 2026-02-25T11:53:09.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.053+0000 7fac8ee0c640 1 -- 10.20.193.28:0/1546272196 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac880fb900 con 0x7fac8810f740 2026-02-25T11:53:09.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.053+0000 7fac7dffb640 1 -- 10.20.193.28:0/1546272196 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fac780106e0 con 0x7fac8810f740 2026-02-25T11:53:09.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.053+0000 7fac8ee0c640 1 -- 10.20.193.28:0/1546272196 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac880fbdd0 con 0x7fac8810f740 2026-02-25T11:53:09.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.056+0000 7fac7dffb640 1 -- 10.20.193.28:0/1546272196 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fac780097c0 con 0x7fac8810f740 2026-02-25T11:53:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.058+0000 7fac7dffb640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fac640b56e0 0x7fac640d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.058+0000 7fac7dffb640 1 -- 10.20.193.28:0/1546272196 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fac780e6a10 con 0x7fac8810f740 2026-02-25T11:53:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.058+0000 7fac7ffff640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fac640b56e0 0x7fac640d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.058+0000 7fac7ffff640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fac640b56e0 0x7fac640d5b90 secure :-1 s=READY pgs=323 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fac880fc3d0 tx=0x7fac70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:09.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.059+0000 7fac90e07e00 1 Processor -- start 2026-02-25T11:53:09.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.059+0000 7fac90e07e00 1 -- start start 2026-02-25T11:53:09.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.059+0000 7fac90e07e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 0x5594ee502510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.059+0000 7fac90e07e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 0x5594ee5471b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.059+0000 7fac90e07e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee567ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac90e07e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5594ee4189b0 con 0x5594ee5476f0 2026-02-25T11:53:09.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac90e07e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5594ee4146e0 con 0x5594ee502140 2026-02-25T11:53:09.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac90e07e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5594ee40e840 con 0x5594ee502a50 2026-02-25T11:53:09.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac8d382640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee567ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac8d382640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee567ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37264/0 (socket says 10.20.193.28:37264) 2026-02-25T11:53:09.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac8d382640 1 -- 10.20.193.28:0/2856777206 learned_addr learned my addr 10.20.193.28:0/2856777206 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:09.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac8cb81640 1 --2- 10.20.193.28:0/2856777206 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 0x5594ee502510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac7ffff640 1 --2- 10.20.193.28:0/2856777206 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 0x5594ee5471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac8d382640 1 -- 10.20.193.28:0/2856777206 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 msgr2=0x5594ee5471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:09.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac8d382640 1 --2- 10.20.193.28:0/2856777206 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 0x5594ee5471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac8d382640 1 -- 10.20.193.28:0/2856777206 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 msgr2=0x5594ee502510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:09.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac8d382640 1 --2- 10.20.193.28:0/2856777206 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 0x5594ee502510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac8d382640 1 -- 10.20.193.28:0/2856777206 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ee3b9d80 con 0x5594ee5476f0 2026-02-25T11:53:09.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac8cb81640 1 --2- 10.20.193.28:0/2856777206 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 0x5594ee502510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:53:09.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.060+0000 7fac8d382640 1 --2- 10.20.193.28:0/2856777206 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee567ad0 secure :-1 s=READY pgs=633 gs=1 cs=0 l=1 c_cookie=537617873559619777 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fac84006cf0 tx=0x7fac84006d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776009b8a6c08c1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:09.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.061+0000 7fac554e4640 1 -- 10.20.193.28:0/2856777206 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fac8403e040 con 0x5594ee5476f0 2026-02-25T11:53:09.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.061+0000 7fac554e4640 1 -- 10.20.193.28:0/2856777206 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fac84030ad0 con 0x5594ee5476f0 2026-02-25T11:53:09.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.061+0000 7fac90e07e00 1 -- 10.20.193.28:0/2856777206 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 msgr2=0x5594ee567ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:09.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.061+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2856777206 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee567ad0 secure :-1 s=READY pgs=633 gs=1 cs=0 l=1 c_cookie=537617873559619777 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fac84006cf0 tx=0x7fac84006d20 comp rx=0 tx=0).stop 2026-02-25T11:53:09.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.061+0000 7fac90e07e00 1 -- 10.20.193.28:0/2856777206 shutdown_connections 2026-02-25T11:53:09.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.061+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2856777206 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee567ad0 unknown :-1 s=CLOSED pgs=633 gs=1 cs=0 l=1 c_cookie=537617873559619777 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.061+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2856777206 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 0x5594ee5471b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.061+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2856777206 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 0x5594ee502510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.061+0000 7fac90e07e00 1 -- 10.20.193.28:0/2856777206 >> 10.20.193.28:0/2856777206 conn(0x5594ee421d40 msgr2=0x5594ee500a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:09.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.061+0000 7fac90e07e00 1 -- 10.20.193.28:0/2856777206 shutdown_connections 2026-02-25T11:53:09.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.061+0000 7fac90e07e00 1 -- 10.20.193.28:0/2856777206 wait complete. 2026-02-25T11:53:09.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.063+0000 7fac90e07e00 1 -- start start 2026-02-25T11:53:09.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.063+0000 7fac90e07e00 1 Processor -- start 2026-02-25T11:53:09.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.063+0000 7fac90e07e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 0x5594ee508470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.063+0000 7fac90e07e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 0x5594ee5089b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.063+0000 7fac90e07e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee503150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.063+0000 7fac90e07e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5594ee503820 con 0x5594ee5476f0 2026-02-25T11:53:09.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.063+0000 7fac90e07e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5594ee503990 con 0x5594ee502140 2026-02-25T11:53:09.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.063+0000 7fac90e07e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5594ee503b50 con 0x5594ee502a50 2026-02-25T11:53:09.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.063+0000 7fac8d382640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee503150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.063+0000 7fac8d382640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee503150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37266/0 (socket says 10.20.193.28:37266) 2026-02-25T11:53:09.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.063+0000 7fac8d382640 1 -- 10.20.193.28:0/2887020638 learned_addr learned my addr 10.20.193.28:0/2887020638 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:09.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.063+0000 7fac8cb81640 1 --2- 10.20.193.28:0/2887020638 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 0x5594ee508470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac8d382640 1 -- 10.20.193.28:0/2887020638 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 msgr2=0x5594ee5089b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:09.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac8d382640 1 --2- 10.20.193.28:0/2887020638 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 0x5594ee5089b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac8d382640 1 -- 10.20.193.28:0/2887020638 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 msgr2=0x5594ee508470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:09.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac8d382640 1 --2- 10.20.193.28:0/2887020638 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 0x5594ee508470 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac8d382640 1 -- 10.20.193.28:0/2887020638 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac88176a30 con 0x5594ee5476f0 2026-02-25T11:53:09.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac8d382640 1 --2- 10.20.193.28:0/2887020638 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee503150 secure :-1 s=READY pgs=634 gs=1 cs=0 l=1 c_cookie=14246777729074438280 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fac84031fd0 tx=0x7fac84030910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b6b8dca76da088 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:09.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac467fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:09.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac467fc640 1 -- 10.20.193.28:0/2887020638 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fac8403e040 con 0x5594ee5476f0 2026-02-25T11:53:09.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac467fc640 1 -- 10.20.193.28:0/2887020638 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fac84039470 con 0x5594ee5476f0 2026-02-25T11:53:09.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac467fc640 1 -- 10.20.193.28:0/2887020638 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fac8404baf0 con 0x5594ee5476f0 2026-02-25T11:53:09.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac90e07e00 1 -- 10.20.193.28:0/2887020638 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac880fbdd0 con 0x5594ee5476f0 2026-02-25T11:53:09.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac467fc640 1 -- 10.20.193.28:0/2887020638 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fac8404bc80 con 0x5594ee5476f0 2026-02-25T11:53:09.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac90e07e00 10 client.16539 _ll_register_callbacks cb 0x5594ee38f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:09.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.064+0000 7fac90e07e00 1 -- 10.20.193.28:0/2887020638 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5594ee0a46f0 con 0x5594ee5476f0 2026-02-25T11:53:09.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.065+0000 7fac467fc640 1 -- 10.20.193.28:0/2887020638 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fac8404b6e0 con 0x5594ee5476f0 2026-02-25T11:53:09.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.065+0000 7fac90e07e00 10 client.16539 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:09.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.065+0000 7fac90e07e00 1 -- 10.20.193.28:0/2887020638 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5594ee392350 con 0x5594ee5476f0 2026-02-25T11:53:09.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.065+0000 7fac467fc640 1 -- 10.20.193.28:0/2887020638 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fac840050c0 con 0x5594ee5476f0 2026-02-25T11:53:09.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.065+0000 7fac90e07e00 10 client.16539 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:09.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.065+0000 7fac90e07e00 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:09.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.065+0000 7fac90e07e00 20 client.16539 populate_metadata read hostname 'trial028' 2026-02-25T11:53:09.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.065+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2887020638 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5594ee54ca10 0x5594ee54ce80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.065+0000 7fac90e07e00 4 client.16539 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:53:09.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.065+0000 7fac90e07e00 1 -- 10.20.193.28:0/2887020638 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594ee3e0910 con 0x5594ee54ca10 2026-02-25T11:53:09.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.065+0000 7fac8cb81640 1 --2- 10.20.193.28:0/2887020638 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5594ee54ca10 0x5594ee54ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.066+0000 7fac8cb81640 1 --2- 10.20.193.28:0/2887020638 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5594ee54ca10 0x5594ee54ce80 crc :-1 s=READY pgs=104 gs=4 cs=0 l=0 c_cookie=6343649381633947441 s_cookie=5418995799348191597 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58092b80a3eaf731 server_cookie=4b3423f7eb7ba16d in_seq=0 out_seq=0 2026-02-25T11:53:09.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.066+0000 7fac467fc640 10 client.16539 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:09.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.066+0000 7fac467fc640 1 -- 10.20.193.28:0/2887020638 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5594ee3e0910 con 0x5594ee54ca10 2026-02-25T11:53:09.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.066+0000 7fac467fc640 10 client.16539 handle_command_reply: tid=0 2026-02-25T11:53:09.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.066+0000 7fac90e07e00 1 client.16539 shutdown 2026-02-25T11:53:09.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.067+0000 7fac90e07e00 1 -- 10.20.193.28:0/2887020638 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 msgr2=0x5594ee503150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:09.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.067+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2887020638 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee503150 secure :-1 s=READY pgs=634 gs=1 cs=0 l=1 c_cookie=14246777729074438280 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fac84031fd0 tx=0x7fac84030910 comp rx=0 tx=0).stop 2026-02-25T11:53:09.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.067+0000 7fac90e07e00 1 -- 10.20.193.28:0/2887020638 shutdown_connections 2026-02-25T11:53:09.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.067+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2887020638 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5594ee54ca10 0x5594ee54ce80 crc :-1 s=READY pgs=104 gs=4 cs=0 l=0 c_cookie=6343649381633947441 s_cookie=5418995799348191597 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.067+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2887020638 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee503150 unknown :-1 s=CLOSED pgs=634 gs=1 cs=0 l=1 c_cookie=14246777729074438280 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.067+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2887020638 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 0x5594ee5089b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.067+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2887020638 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 0x5594ee508470 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.067+0000 7fac90e07e00 1 -- 10.20.193.28:0/2887020638 >> 10.20.193.28:0/2887020638 conn(0x5594ee421d40 msgr2=0x5594ee500230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:09.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.067+0000 7fac467fc640 0 client.16539 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:09.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.067+0000 7fac90e07e00 1 -- 10.20.193.28:0/2887020638 shutdown_connections 2026-02-25T11:53:09.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.067+0000 7fac90e07e00 1 -- 10.20.193.28:0/2887020638 wait complete. 2026-02-25T11:53:09.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.067+0000 7fac90e07e00 20 client.16539 trim_cache size 0 max 16384 2026-02-25T11:53:09.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.076+0000 7fac90e07e00 1 Processor -- start 2026-02-25T11:53:09.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.076+0000 7fac90e07e00 1 -- start start 2026-02-25T11:53:09.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.076+0000 7fac90e07e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502140 0x5594ee546d00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.076+0000 7fac90e07e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502a50 0x5594ee564d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.076+0000 7fac90e07e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee565260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.076+0000 7fac90e07e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5594ee40c470 con 0x5594ee5476f0 2026-02-25T11:53:09.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.076+0000 7fac90e07e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5594ee0cc700 con 0x5594ee502a50 2026-02-25T11:53:09.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.076+0000 7fac90e07e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5594ee4719d0 con 0x5594ee502140 2026-02-25T11:53:09.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac8d382640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee565260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac8d382640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee565260 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37280/0 (socket says 10.20.193.28:37280) 2026-02-25T11:53:09.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac8d382640 1 -- 10.20.193.28:0/4144230258 learned_addr learned my addr 10.20.193.28:0/4144230258 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:09.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac7ffff640 1 --2- 10.20.193.28:0/4144230258 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502a50 0x5594ee564d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac8d382640 1 -- 10.20.193.28:0/4144230258 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502140 msgr2=0x5594ee546d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:09.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac8d382640 1 --2- 10.20.193.28:0/4144230258 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502140 0x5594ee546d00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac8d382640 1 -- 10.20.193.28:0/4144230258 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502a50 msgr2=0x5594ee564d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:09.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac8d382640 1 --2- 10.20.193.28:0/4144230258 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502a50 0x5594ee564d20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac8d382640 1 -- 10.20.193.28:0/4144230258 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ee3a4150 con 0x5594ee5476f0 2026-02-25T11:53:09.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac7ffff640 1 --2- 10.20.193.28:0/4144230258 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502a50 0x5594ee564d20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:53:09.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac8d382640 1 --2- 10.20.193.28:0/4144230258 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee565260 secure :-1 s=READY pgs=635 gs=1 cs=0 l=1 c_cookie=9276124161410840605 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fac84056e40 tx=0x7fac84057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bb69c71d4fb81d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:09.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac54ce3640 1 -- 10.20.193.28:0/4144230258 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fac8403e050 con 0x5594ee5476f0 2026-02-25T11:53:09.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac54ce3640 1 -- 10.20.193.28:0/4144230258 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5594ee0a46f0 con 0x5594ee5476f0 2026-02-25T11:53:09.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.077+0000 7fac54ce3640 1 -- 10.20.193.28:0/4144230258 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fac84003990 con 0x5594ee5476f0 2026-02-25T11:53:09.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.078+0000 7fac90e07e00 1 -- 10.20.193.28:0/4144230258 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 msgr2=0x5594ee565260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:09.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.078+0000 7fac90e07e00 1 --2- 10.20.193.28:0/4144230258 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee565260 secure :-1 s=READY pgs=635 gs=1 cs=0 l=1 c_cookie=9276124161410840605 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fac84056e40 tx=0x7fac84057000 comp rx=0 tx=0).stop 2026-02-25T11:53:09.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.078+0000 7fac90e07e00 1 -- 10.20.193.28:0/4144230258 shutdown_connections 2026-02-25T11:53:09.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.078+0000 7fac90e07e00 1 --2- 10.20.193.28:0/4144230258 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee565260 unknown :-1 s=CLOSED pgs=635 gs=1 cs=0 l=1 c_cookie=9276124161410840605 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.078+0000 7fac90e07e00 1 --2- 10.20.193.28:0/4144230258 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502a50 0x5594ee564d20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.078+0000 7fac90e07e00 1 --2- 10.20.193.28:0/4144230258 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502140 0x5594ee546d00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.078+0000 7fac90e07e00 1 -- 10.20.193.28:0/4144230258 >> 10.20.193.28:0/4144230258 conn(0x5594ee421d40 msgr2=0x5594ee5482b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:09.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.078+0000 7fac90e07e00 1 -- 10.20.193.28:0/4144230258 shutdown_connections 2026-02-25T11:53:09.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.078+0000 7fac90e07e00 1 -- 10.20.193.28:0/4144230258 wait complete. 2026-02-25T11:53:09.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.078+0000 7fac90e07e00 1 -- start start 2026-02-25T11:53:09.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.078+0000 7fac90e07e00 1 Processor -- start 2026-02-25T11:53:09.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac90e07e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 0x5594ee566160 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac90e07e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 0x5594ee5666a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac90e07e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee50d2a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac90e07e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5594ee49c6b0 con 0x5594ee5476f0 2026-02-25T11:53:09.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac90e07e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5594ee566e90 con 0x5594ee502140 2026-02-25T11:53:09.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac90e07e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5594ee50d7e0 con 0x5594ee502a50 2026-02-25T11:53:09.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac8d382640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee50d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac8d382640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee50d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37284/0 (socket says 10.20.193.28:37284) 2026-02-25T11:53:09.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac8d382640 1 -- 10.20.193.28:0/2698980050 learned_addr learned my addr 10.20.193.28:0/2698980050 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:09.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac7ffff640 1 --2- 10.20.193.28:0/2698980050 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 0x5594ee5666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac8cb81640 1 --2- 10.20.193.28:0/2698980050 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 0x5594ee566160 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac8d382640 1 -- 10.20.193.28:0/2698980050 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 msgr2=0x5594ee5666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:09.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac8d382640 1 --2- 10.20.193.28:0/2698980050 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 0x5594ee5666a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac8d382640 1 -- 10.20.193.28:0/2698980050 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 msgr2=0x5594ee566160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:09.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac8d382640 1 --2- 10.20.193.28:0/2698980050 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 0x5594ee566160 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac8d382640 1 -- 10.20.193.28:0/2698980050 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594ee39ebd0 con 0x5594ee5476f0 2026-02-25T11:53:09.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac8d382640 1 --2- 10.20.193.28:0/2698980050 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee50d2a0 secure :-1 s=READY pgs=636 gs=1 cs=0 l=1 c_cookie=12065138061804371709 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fac84005010 tx=0x7fac84006fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76ffb24b3e4fefd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:09.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.079+0000 7fac467fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:09.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac467fc640 1 -- 10.20.193.28:0/2698980050 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fac8403e070 con 0x5594ee5476f0 2026-02-25T11:53:09.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac467fc640 1 -- 10.20.193.28:0/2698980050 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fac84039690 con 0x5594ee5476f0 2026-02-25T11:53:09.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac467fc640 1 -- 10.20.193.28:0/2698980050 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fac84056410 con 0x5594ee5476f0 2026-02-25T11:53:09.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac90e07e00 1 -- 10.20.193.28:0/2698980050 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594ee392350 con 0x5594ee5476f0 2026-02-25T11:53:09.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac467fc640 1 -- 10.20.193.28:0/2698980050 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fac840565b0 con 0x5594ee5476f0 2026-02-25T11:53:09.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac467fc640 10 client.16545 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:53:09.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac90e07e00 10 client.16545 _ll_register_callbacks cb 0x5594ee38f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:09.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac90e07e00 1 -- 10.20.193.28:0/2698980050 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5594ee1664a0 con 0x5594ee5476f0 2026-02-25T11:53:09.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac467fc640 1 -- 10.20.193.28:0/2698980050 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fac84006cb0 con 0x5594ee5476f0 2026-02-25T11:53:09.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac90e07e00 10 client.16545 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:09.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac90e07e00 1 -- 10.20.193.28:0/2698980050 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5594ee392680 con 0x5594ee5476f0 2026-02-25T11:53:09.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac467fc640 1 -- 10.20.193.28:0/2698980050 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fac8404b420 con 0x5594ee5476f0 2026-02-25T11:53:09.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac90e07e00 10 client.16545 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:09.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac90e07e00 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:09.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac90e07e00 20 client.16545 populate_metadata read hostname 'trial028' 2026-02-25T11:53:09.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2698980050 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5594ee50eae0 0x5594ee50ef30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:09.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac90e07e00 4 client.16545 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:53:09.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.080+0000 7fac90e07e00 1 -- 10.20.193.28:0/2698980050 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fac84030ad0 con 0x5594ee50eae0 2026-02-25T11:53:09.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.081+0000 7fac8cb81640 1 --2- 10.20.193.28:0/2698980050 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5594ee50eae0 0x5594ee50ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:09.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.081+0000 7fac8cb81640 1 --2- 10.20.193.28:0/2698980050 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5594ee50eae0 0x5594ee50ef30 crc :-1 s=READY pgs=105 gs=4 cs=0 l=0 c_cookie=5056445796298620525 s_cookie=8640010730740730991 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=462c1aa95a657e6d server_cookie=77e77bfb9d5d4c6f in_seq=0 out_seq=0 2026-02-25T11:53:09.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.081+0000 7fac467fc640 10 client.16545 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:09.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.081+0000 7fac467fc640 1 -- 10.20.193.28:0/2698980050 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x7fac84030ad0 con 0x5594ee50eae0 2026-02-25T11:53:09.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.081+0000 7fac467fc640 10 client.16545 handle_command_reply: tid=0 2026-02-25T11:53:09.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.081+0000 7fac90e07e00 1 client.16545 shutdown 2026-02-25T11:53:09.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac90e07e00 1 -- 10.20.193.28:0/2698980050 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 msgr2=0x5594ee50d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:09.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2698980050 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee50d2a0 secure :-1 s=READY pgs=636 gs=1 cs=0 l=1 c_cookie=12065138061804371709 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fac84005010 tx=0x7fac84006fb0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac90e07e00 1 -- 10.20.193.28:0/2698980050 shutdown_connections 2026-02-25T11:53:09.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2698980050 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5594ee50eae0 0x5594ee50ef30 crc :-1 s=READY pgs=105 gs=4 cs=0 l=0 c_cookie=5056445796298620525 s_cookie=8640010730740730991 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2698980050 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594ee5476f0 0x5594ee50d2a0 unknown :-1 s=CLOSED pgs=636 gs=1 cs=0 l=1 c_cookie=12065138061804371709 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2698980050 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594ee502a50 0x5594ee5666a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac90e07e00 1 --2- 10.20.193.28:0/2698980050 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594ee502140 0x5594ee566160 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac467fc640 0 client.16545 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:09.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac90e07e00 1 -- 10.20.193.28:0/2698980050 >> 10.20.193.28:0/2698980050 conn(0x5594ee421d40 msgr2=0x5594ee507d90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:09.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac90e07e00 1 -- 10.20.193.28:0/2698980050 shutdown_connections 2026-02-25T11:53:09.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac90e07e00 1 -- 10.20.193.28:0/2698980050 wait complete. 2026-02-25T11:53:09.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac90e07e00 20 client.16545 trim_cache size 0 max 16384 2026-02-25T11:53:09.090 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:53:09.090 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:53:09.091 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:53:09.091 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:53:09.091 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:09.091 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847", 2026-02-25T11:53:09.091 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:53:09.091 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:53:09.091 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:09.091 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:53:09.091 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:53:09.091 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:53:09.091 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:09.091 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:09.092 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:09.092 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:09.092 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:09.092 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:09.092 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:53:09.092 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:53:09.092 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:09.092 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:09.092 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:09.092 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:09.092 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:09.092 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:09.093 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:53:09.093 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:53:09.093 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:09.093 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:09.093 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:09.093 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:09.093 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:09.093 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:09.093 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:53:09.093 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:53:09.093 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:09.093 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:09.094 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:09.094 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:09.094 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:09.094 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:09.094 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:53:09.094 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:53:09.094 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:09.094 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:09.094 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:09.094 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:09.094 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:09.094 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:09.094 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:53:09.095 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:53:09.095 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:09.095 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:09.095 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:09.095 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:09.095 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:09.095 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:09.095 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:53:09.095 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:53:09.095 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:09.095 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:09.095 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:09.096 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:09.096 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:09.096 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:09.096 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:53:09.096 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:53:09.096 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:09.096 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:09.096 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:09.096 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:09.096 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:09.096 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:09.097 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:09.098 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:09.098 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:09.098 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:53:09.098 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:53:09.098 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:09.098 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:09.098 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:09.098 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:09.098 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:09.098 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:53:09.098 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:53:09.098 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:53:09.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac8ee0c640 1 -- 10.20.193.28:0/1546272196 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fac640b56e0 msgr2=0x7fac640d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:09.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac8ee0c640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fac640b56e0 0x7fac640d5b90 secure :-1 s=READY pgs=323 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fac880fc3d0 tx=0x7fac70005f90 comp rx=0 tx=0).stop 2026-02-25T11:53:09.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac8ee0c640 1 -- 10.20.193.28:0/1546272196 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fac8810f740 msgr2=0x7fac880f5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:09.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac8ee0c640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fac8810f740 0x7fac880f5a30 secure :-1 s=READY pgs=174 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fac7800d950 tx=0x7fac7800de20 comp rx=0 tx=0).stop 2026-02-25T11:53:09.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac8ee0c640 1 -- 10.20.193.28:0/1546272196 shutdown_connections 2026-02-25T11:53:09.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac8ee0c640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fac640b56e0 0x7fac640d5b90 unknown :-1 s=CLOSED pgs=323 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac8ee0c640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fac88115930 0x7fac880f60c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac8ee0c640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fac88110130 0x7fac88176200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac8ee0c640 1 --2- 10.20.193.28:0/1546272196 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fac8810f740 0x7fac880f5a30 unknown :-1 s=CLOSED pgs=174 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:09.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac8ee0c640 1 -- 10.20.193.28:0/1546272196 >> 10.20.193.28:0/1546272196 conn(0x7fac880ecff0 msgr2=0x7fac8810dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:09.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac8ee0c640 1 -- 10.20.193.28:0/1546272196 shutdown_connections 2026-02-25T11:53:09.099 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:09.082+0000 7fac8ee0c640 1 -- 10.20.193.28:0/1546272196 wait complete. 2026-02-25T11:53:09.181 INFO:tasks.cephfs.filesystem:scrub status for tag:a45f08eb-b06d-4dcb-8fdc-f2ced26d8847 - {'path': '/', 'tag': 'a45f08eb-b06d-4dcb-8fdc-f2ced26d8847', 'options': 'recursive,force'} 2026-02-25T11:53:09.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:09 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:53:09.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:09 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:53:09.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:09 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:53:09.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:09 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:53:09.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:09 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:53:09.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:09 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:53:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:09 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:53:09.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:09 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:53:09.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:09 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:53:09.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:09 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:53:09.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:09 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:53:09.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:09 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:53:10.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:10 trial167 ceph-mon[43392]: pgmap v374: 97 pgs: 97 active+clean; 4.0 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:53:10.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:10 trial119 ceph-mon[42710]: pgmap v374: 97 pgs: 97 active+clean; 4.0 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:53:10.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:10 trial028 ceph-mon[39224]: pgmap v374: 97 pgs: 97 active+clean; 4.0 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:53:12.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:12 trial167 ceph-mon[43392]: pgmap v375: 97 pgs: 97 active+clean; 4.0 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:12.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:12 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:53:12.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:12 trial119 ceph-mon[42710]: pgmap v375: 97 pgs: 97 active+clean; 4.0 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:12.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:12 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:53:12.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:12 trial028 ceph-mon[39224]: pgmap v375: 97 pgs: 97 active+clean; 4.0 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:12.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:12 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:53:14.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:14 trial167 ceph-mon[43392]: pgmap v376: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:53:14.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:14 trial119 ceph-mon[42710]: pgmap v376: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:53:14.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:14 trial028 ceph-mon[39224]: pgmap v376: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:53:16.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:16 trial167 ceph-mon[43392]: pgmap v377: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:53:16.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:53:16.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:53:16.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:16 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:53:16.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:16 trial119 ceph-mon[42710]: pgmap v377: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:53:16.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:53:16.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:53:16.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:16 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:53:16.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:16 trial028 ceph-mon[39224]: pgmap v377: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:53:16.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:53:16.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:53:16.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:16 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:53:17.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:53:17.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:53:17.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:53:17.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:53:17.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:53:17.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:53:17.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:53:17.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:53:17.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:53:17.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:53:17.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:53:17.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:53:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:18 trial167 ceph-mon[43392]: pgmap v378: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:53:18.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:18 trial119 ceph-mon[42710]: pgmap v378: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:53:18.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:18 trial028 ceph-mon[39224]: pgmap v378: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:53:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:20 trial167 ceph-mon[43392]: pgmap v379: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:53:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:20 trial119 ceph-mon[42710]: pgmap v379: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:53:20.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:20 trial028 ceph-mon[39224]: pgmap v379: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:53:22.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:22 trial167 ceph-mon[43392]: pgmap v380: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:22.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:22 trial119 ceph-mon[42710]: pgmap v380: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:22.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:22 trial028 ceph-mon[39224]: pgmap v380: 97 pgs: 97 active+clean; 4.1 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:24.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:24 trial167 ceph-mon[43392]: pgmap v381: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:24.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:24 trial119 ceph-mon[42710]: pgmap v381: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:24.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:24 trial028 ceph-mon[39224]: pgmap v381: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:26.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:26 trial167 ceph-mon[43392]: pgmap v382: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:53:26.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:26 trial119 ceph-mon[42710]: pgmap v382: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:53:26.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:26 trial028 ceph-mon[39224]: pgmap v382: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:53:28.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:28 trial167 ceph-mon[43392]: pgmap v383: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:53:28.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:28 trial119 ceph-mon[42710]: pgmap v383: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:53:28.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:28 trial028 ceph-mon[39224]: pgmap v383: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:53:29.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:29 trial167 ceph-mon[43392]: pgmap v384: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:53:29.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:29 trial119 ceph-mon[42710]: pgmap v384: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:53:29.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:29 trial028 ceph-mon[39224]: pgmap v384: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:53:32.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:32 trial167 ceph-mon[43392]: pgmap v385: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:32.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:32 trial119 ceph-mon[42710]: pgmap v385: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:32.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:32 trial028 ceph-mon[39224]: pgmap v385: 97 pgs: 97 active+clean; 4.2 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:34.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:34 trial167 ceph-mon[43392]: pgmap v386: 97 pgs: 97 active+clean; 4.3 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:34.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:34 trial119 ceph-mon[42710]: pgmap v386: 97 pgs: 97 active+clean; 4.3 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:34.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:34 trial028 ceph-mon[39224]: pgmap v386: 97 pgs: 97 active+clean; 4.3 MiB data, 341 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:36.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:36 trial167 ceph-mon[43392]: pgmap v387: 97 pgs: 97 active+clean; 4.3 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:36.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:36 trial119 ceph-mon[42710]: pgmap v387: 97 pgs: 97 active+clean; 4.3 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:36.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:36 trial028 ceph-mon[39224]: pgmap v387: 97 pgs: 97 active+clean; 4.3 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:37.443 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.443 INFO:tasks.workunit.client.0.trial028.stdout:251/600: open fd = 3 2026-02-25T11:53:37.444 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.444 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.444 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.444 INFO:tasks.workunit.client.0.trial028.stdout:252/600: open fd = 3 2026-02-25T11:53:37.444 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.444 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.444 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.444 INFO:tasks.workunit.client.0.trial028.stdout:253/600: open fd = 3 2026-02-25T11:53:37.444 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.444 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.445 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.445 INFO:tasks.workunit.client.0.trial028.stdout:254/600: open fd = 3 2026-02-25T11:53:37.445 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.445 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.445 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.445 INFO:tasks.workunit.client.0.trial028.stdout:255/600: open fd = 3 2026-02-25T11:53:37.445 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.445 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.445 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.445 INFO:tasks.workunit.client.0.trial028.stdout:256/600: open fd = 3 2026-02-25T11:53:37.445 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.445 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.446 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.446 INFO:tasks.workunit.client.0.trial028.stdout:257/600: open fd = 3 2026-02-25T11:53:37.446 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.446 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.446 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.446 INFO:tasks.workunit.client.0.trial028.stdout:258/600: open fd = 3 2026-02-25T11:53:37.446 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.446 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.446 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.446 INFO:tasks.workunit.client.0.trial028.stdout:259/600: open fd = 3 2026-02-25T11:53:37.446 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.446 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.447 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.447 INFO:tasks.workunit.client.0.trial028.stdout:260/600: open fd = 3 2026-02-25T11:53:37.447 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.447 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.447 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.447 INFO:tasks.workunit.client.0.trial028.stdout:261/600: open fd = 3 2026-02-25T11:53:37.447 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.447 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.447 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.447 INFO:tasks.workunit.client.0.trial028.stdout:262/600: open fd = 3 2026-02-25T11:53:37.447 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.447 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.447 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.448 INFO:tasks.workunit.client.0.trial028.stdout:263/600: open fd = 3 2026-02-25T11:53:37.448 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.448 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.448 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.448 INFO:tasks.workunit.client.0.trial028.stdout:264/600: open fd = 3 2026-02-25T11:53:37.448 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.448 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.448 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.448 INFO:tasks.workunit.client.0.trial028.stdout:265/600: open fd = 3 2026-02-25T11:53:37.448 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.448 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.448 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.448 INFO:tasks.workunit.client.0.trial028.stdout:266/600: open fd = 3 2026-02-25T11:53:37.449 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.449 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.449 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.449 INFO:tasks.workunit.client.0.trial028.stdout:267/600: open fd = 3 2026-02-25T11:53:37.449 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.449 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.449 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.449 INFO:tasks.workunit.client.0.trial028.stdout:268/600: open fd = 3 2026-02-25T11:53:37.449 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.449 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.449 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.449 INFO:tasks.workunit.client.0.trial028.stdout:269/600: open fd = 3 2026-02-25T11:53:37.449 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.450 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.450 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.450 INFO:tasks.workunit.client.0.trial028.stdout:270/600: open fd = 3 2026-02-25T11:53:37.450 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.450 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.450 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.450 INFO:tasks.workunit.client.0.trial028.stdout:271/600: open fd = 3 2026-02-25T11:53:37.450 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.450 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.450 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.450 INFO:tasks.workunit.client.0.trial028.stdout:272/600: open fd = 3 2026-02-25T11:53:37.450 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.451 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.451 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.451 INFO:tasks.workunit.client.0.trial028.stdout:273/600: open fd = 3 2026-02-25T11:53:37.451 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.451 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.451 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.451 INFO:tasks.workunit.client.0.trial028.stdout:274/600: open fd = 3 2026-02-25T11:53:37.451 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.451 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.451 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.451 INFO:tasks.workunit.client.0.trial028.stdout:275/600: open fd = 3 2026-02-25T11:53:37.451 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.451 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.452 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.452 INFO:tasks.workunit.client.0.trial028.stdout:276/600: open fd = 3 2026-02-25T11:53:37.452 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.452 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.452 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.452 INFO:tasks.workunit.client.0.trial028.stdout:277/600: open fd = 3 2026-02-25T11:53:37.452 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.452 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.452 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.452 INFO:tasks.workunit.client.0.trial028.stdout:278/600: open fd = 3 2026-02-25T11:53:37.452 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.452 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.453 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.453 INFO:tasks.workunit.client.0.trial028.stdout:279/600: open fd = 3 2026-02-25T11:53:37.453 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.453 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.453 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.453 INFO:tasks.workunit.client.0.trial028.stdout:280/600: open fd = 3 2026-02-25T11:53:37.453 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.453 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.453 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.453 INFO:tasks.workunit.client.0.trial028.stdout:281/600: open fd = 3 2026-02-25T11:53:37.453 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.453 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.453 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.454 INFO:tasks.workunit.client.0.trial028.stdout:282/600: open fd = 3 2026-02-25T11:53:37.454 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.454 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.454 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.454 INFO:tasks.workunit.client.0.trial028.stdout:283/600: open fd = 3 2026-02-25T11:53:37.454 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.454 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.454 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.454 INFO:tasks.workunit.client.0.trial028.stdout:284/600: open fd = 3 2026-02-25T11:53:37.454 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.454 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.454 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:285/600: open fd = 3 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:286/600: open fd = 3 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:287/600: open fd = 3 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:288/600: open fd = 3 2026-02-25T11:53:37.455 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.456 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.456 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.456 INFO:tasks.workunit.client.0.trial028.stdout:289/600: open fd = 3 2026-02-25T11:53:37.456 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.456 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.456 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.456 INFO:tasks.workunit.client.0.trial028.stdout:290/600: open fd = 3 2026-02-25T11:53:37.456 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.456 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.456 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.456 INFO:tasks.workunit.client.0.trial028.stdout:291/600: open fd = 3 2026-02-25T11:53:37.456 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.456 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.457 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.457 INFO:tasks.workunit.client.0.trial028.stdout:292/600: open fd = 3 2026-02-25T11:53:37.457 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.457 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.457 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.457 INFO:tasks.workunit.client.0.trial028.stdout:293/600: open fd = 3 2026-02-25T11:53:37.457 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.457 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.457 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.457 INFO:tasks.workunit.client.0.trial028.stdout:294/600: open fd = 3 2026-02-25T11:53:37.457 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.457 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.457 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:295/600: open fd = 3 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:296/600: open fd = 3 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:297/600: open fd = 3 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:298/600: open fd = 3 2026-02-25T11:53:37.458 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.459 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.459 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.459 INFO:tasks.workunit.client.0.trial028.stdout:299/600: open fd = 3 2026-02-25T11:53:37.459 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.459 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.459 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.459 INFO:tasks.workunit.client.0.trial028.stdout:300/600: open fd = 3 2026-02-25T11:53:37.459 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.459 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.459 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.459 INFO:tasks.workunit.client.0.trial028.stdout:301/600: open fd = 3 2026-02-25T11:53:37.459 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.459 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.460 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.460 INFO:tasks.workunit.client.0.trial028.stdout:302/600: open fd = 3 2026-02-25T11:53:37.460 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.460 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.460 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.460 INFO:tasks.workunit.client.0.trial028.stdout:303/600: open fd = 3 2026-02-25T11:53:37.460 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.460 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.460 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.460 INFO:tasks.workunit.client.0.trial028.stdout:304/600: open fd = 3 2026-02-25T11:53:37.460 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.460 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.460 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.461 INFO:tasks.workunit.client.0.trial028.stdout:305/600: open fd = 3 2026-02-25T11:53:37.461 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.461 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.461 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.461 INFO:tasks.workunit.client.0.trial028.stdout:306/600: open fd = 3 2026-02-25T11:53:37.461 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.461 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.461 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.461 INFO:tasks.workunit.client.0.trial028.stdout:307/600: open fd = 3 2026-02-25T11:53:37.461 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.461 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.461 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.461 INFO:tasks.workunit.client.0.trial028.stdout:308/600: open fd = 3 2026-02-25T11:53:37.462 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.462 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.462 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.462 INFO:tasks.workunit.client.0.trial028.stdout:309/600: open fd = 3 2026-02-25T11:53:37.462 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.462 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.462 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.462 INFO:tasks.workunit.client.0.trial028.stdout:310/600: open fd = 3 2026-02-25T11:53:37.462 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.462 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.462 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.462 INFO:tasks.workunit.client.0.trial028.stdout:311/600: open fd = 3 2026-02-25T11:53:37.462 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.463 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.463 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:53:37.463 INFO:tasks.workunit.client.0.trial028.stdout:312/600: open fd = 3 2026-02-25T11:53:37.463 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:37.463 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:53:38.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:38 trial167 ceph-mon[43392]: pgmap v388: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:53:38.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:38 trial119 ceph-mon[42710]: pgmap v388: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:53:38.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:38 trial028 ceph-mon[39224]: pgmap v388: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:53:39.182 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:53:39.372 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:39.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.543+0000 7f7d07756640 1 -- 10.20.193.28:0/596883567 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7d0010f740 msgr2=0x7f7d0010fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:39.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.543+0000 7f7d07756640 1 --2- 10.20.193.28:0/596883567 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7d0010f740 0x7f7d0010fb10 secure :-1 s=READY pgs=637 gs=1 cs=0 l=1 c_cookie=433966206516182586 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7cf0009950 tx=0x7f7cf002f190 comp rx=0 tx=0).stop 2026-02-25T11:53:39.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.543+0000 7f7d07756640 1 -- 10.20.193.28:0/596883567 shutdown_connections 2026-02-25T11:53:39.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.543+0000 7f7d07756640 1 --2- 10.20.193.28:0/596883567 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f7d00115930 0x7f7d00176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.543+0000 7f7d07756640 1 --2- 10.20.193.28:0/596883567 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7d00110130 0x7f7d00115180 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.543+0000 7f7d07756640 1 --2- 10.20.193.28:0/596883567 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7d0010f740 0x7f7d0010fb10 unknown :-1 s=CLOSED pgs=637 gs=1 cs=0 l=1 c_cookie=433966206516182586 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.543+0000 7f7d07756640 1 -- 10.20.193.28:0/596883567 >> 10.20.193.28:0/596883567 conn(0x7f7d000ecff0 msgr2=0x7f7d0010d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:39.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.543+0000 7f7d07756640 1 -- 10.20.193.28:0/596883567 shutdown_connections 2026-02-25T11:53:39.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.543+0000 7f7d07756640 1 -- 10.20.193.28:0/596883567 wait complete. 2026-02-25T11:53:39.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.544+0000 7f7d07756640 1 Processor -- start 2026-02-25T11:53:39.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.544+0000 7f7d07756640 1 -- start start 2026-02-25T11:53:39.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.544+0000 7f7d07756640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f7d0010f740 0x7f7d0011e110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.544+0000 7f7d07756640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7d00110130 0x7f7d0011e650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.544+0000 7f7d07756640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7d00115930 0x7f7d00123a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.544+0000 7f7d07756640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7d00124040 con 0x7f7d00110130 2026-02-25T11:53:39.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.544+0000 7f7d07756640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7d001241b0 con 0x7f7d00115930 2026-02-25T11:53:39.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.544+0000 7f7d07756640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d00124370 con 0x7f7d0010f740 2026-02-25T11:53:39.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.544+0000 7f7d04cca640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7d00110130 0x7f7d0011e650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.544+0000 7f7d04cca640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7d00110130 0x7f7d0011e650 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37074/0 (socket says 10.20.193.28:37074) 2026-02-25T11:53:39.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.544+0000 7f7d04cca640 1 -- 10.20.193.28:0/3411428301 learned_addr learned my addr 10.20.193.28:0/3411428301 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:39.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.544+0000 7f7d05ccc640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7d00115930 0x7f7d00123a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.544+0000 7f7d054cb640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f7d0010f740 0x7f7d0011e110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.545+0000 7f7d04cca640 1 -- 10.20.193.28:0/3411428301 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f7d0010f740 msgr2=0x7f7d0011e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:39.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.545+0000 7f7d04cca640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f7d0010f740 0x7f7d0011e110 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.545+0000 7f7d04cca640 1 -- 10.20.193.28:0/3411428301 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7d00115930 msgr2=0x7f7d00123a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:39.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.545+0000 7f7d04cca640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7d00115930 0x7f7d00123a20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.545+0000 7f7d04cca640 1 -- 10.20.193.28:0/3411428301 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf0009570 con 0x7f7d00110130 2026-02-25T11:53:39.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.545+0000 7f7d05ccc640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7d00115930 0x7f7d00123a20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:53:39.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.545+0000 7f7d054cb640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f7d0010f740 0x7f7d0011e110 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:53:39.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.545+0000 7f7d04cca640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7d00110130 0x7f7d0011e650 secure :-1 s=READY pgs=638 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7cf400d950 tx=0x7f7cf400de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:39.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.545+0000 7f7cee7fc640 1 -- 10.20.193.28:0/3411428301 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7cf4004540 con 0x7f7d00110130 2026-02-25T11:53:39.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.545+0000 7f7d07756640 1 -- 10.20.193.28:0/3411428301 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d00124650 con 0x7f7d00110130 2026-02-25T11:53:39.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.545+0000 7f7cee7fc640 1 -- 10.20.193.28:0/3411428301 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7cf4009ce0 con 0x7f7d00110130 2026-02-25T11:53:39.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.545+0000 7f7cee7fc640 1 -- 10.20.193.28:0/3411428301 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7cf40106e0 con 0x7f7d00110130 2026-02-25T11:53:39.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.545+0000 7f7d07756640 1 -- 10.20.193.28:0/3411428301 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d00164d50 con 0x7f7d00110130 2026-02-25T11:53:39.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.548+0000 7f7cee7fc640 1 -- 10.20.193.28:0/3411428301 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f7cf40097c0 con 0x7f7d00110130 2026-02-25T11:53:39.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.550+0000 7f7cee7fc640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f7cdc0b5760 0x7f7cdc0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.550+0000 7f7cee7fc640 1 -- 10.20.193.28:0/3411428301 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f7cf40e6db0 con 0x7f7d00110130 2026-02-25T11:53:39.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.550+0000 7f7d054cb640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f7cdc0b5760 0x7f7cdc0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.550+0000 7f7d054cb640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f7cdc0b5760 0x7f7cdc0d5c10 secure :-1 s=READY pgs=324 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7cf00023d0 tx=0x7f7cf0031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:39.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.551+0000 7f7d09751e00 1 Processor -- start 2026-02-25T11:53:39.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.551+0000 7f7d09751e00 1 -- start start 2026-02-25T11:53:39.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.551+0000 7f7d09751e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92d140 0x558fae92d510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.551+0000 7f7d09751e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92da50 0x558fae9721b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.551+0000 7f7d09751e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae9726f0 0x558fae992ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.551+0000 7f7d09751e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558fae8439b0 con 0x558fae92da50 2026-02-25T11:53:39.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.551+0000 7f7d09751e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558fae83f6e0 con 0x558fae9726f0 2026-02-25T11:53:39.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.551+0000 7f7d09751e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558fae839840 con 0x558fae92d140 2026-02-25T11:53:39.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7d04cca640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92da50 0x558fae9721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7d054cb640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92d140 0x558fae92d510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7d054cb640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92d140 0x558fae92d510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34820/0 (socket says 10.20.193.28:34820) 2026-02-25T11:53:39.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7d054cb640 1 -- 10.20.193.28:0/3729900393 learned_addr learned my addr 10.20.193.28:0/3729900393 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:39.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7d04cca640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92da50 0x558fae9721b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37084/0 (socket says 10.20.193.28:37084) 2026-02-25T11:53:39.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7d05ccc640 1 --2- 10.20.193.28:0/3729900393 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae9726f0 0x558fae992ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7d04cca640 1 -- 10.20.193.28:0/3729900393 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92d140 msgr2=0x558fae92d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:39.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7d04cca640 1 --2- 10.20.193.28:0/3729900393 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92d140 0x558fae92d510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7d04cca640 1 -- 10.20.193.28:0/3729900393 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae9726f0 msgr2=0x558fae992ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:39.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7d04cca640 1 --2- 10.20.193.28:0/3729900393 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae9726f0 0x558fae992ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7d04cca640 1 -- 10.20.193.28:0/3729900393 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fae7e4d80 con 0x558fae92da50 2026-02-25T11:53:39.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7d04cca640 1 --2- 10.20.193.28:0/3729900393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92da50 0x558fae9721b0 secure :-1 s=READY pgs=639 gs=2 cs=0 l=1 c_cookie=13597585354919628354 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7cf40a9700 tx=0x7f7cf40a9870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb453d65309ce42 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:39.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7ccdce5640 1 -- 10.20.193.28:0/3729900393 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7cf40ac930 con 0x558fae92da50 2026-02-25T11:53:39.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7ccdce5640 1 -- 10.20.193.28:0/3729900393 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7cf4009e50 con 0x558fae92da50 2026-02-25T11:53:39.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.552+0000 7f7ccdce5640 1 -- 10.20.193.28:0/3729900393 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7cf40a93a0 con 0x558fae92da50 2026-02-25T11:53:39.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.553+0000 7f7d09751e00 1 -- 10.20.193.28:0/3729900393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92da50 msgr2=0x558fae9721b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:39.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.553+0000 7f7d09751e00 1 --2- 10.20.193.28:0/3729900393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92da50 0x558fae9721b0 secure :-1 s=READY pgs=639 gs=2 cs=0 l=1 c_cookie=13597585354919628354 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7cf40a9700 tx=0x7f7cf40a9870 comp rx=0 tx=0).stop 2026-02-25T11:53:39.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.553+0000 7f7d09751e00 1 -- 10.20.193.28:0/3729900393 shutdown_connections 2026-02-25T11:53:39.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.553+0000 7f7d09751e00 1 --2- 10.20.193.28:0/3729900393 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae9726f0 0x558fae992ad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.553+0000 7f7d09751e00 1 --2- 10.20.193.28:0/3729900393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92da50 0x558fae9721b0 unknown :-1 s=CLOSED pgs=639 gs=2 cs=0 l=1 c_cookie=13597585354919628354 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.553+0000 7f7d09751e00 1 --2- 10.20.193.28:0/3729900393 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92d140 0x558fae92d510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.553+0000 7f7d09751e00 1 -- 10.20.193.28:0/3729900393 >> 10.20.193.28:0/3729900393 conn(0x558fae84cd40 msgr2=0x558fae92ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:39.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.553+0000 7f7d09751e00 1 -- 10.20.193.28:0/3729900393 shutdown_connections 2026-02-25T11:53:39.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.553+0000 7f7d09751e00 1 -- 10.20.193.28:0/3729900393 wait complete. 2026-02-25T11:53:39.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.554+0000 7f7d09751e00 1 -- start start 2026-02-25T11:53:39.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.554+0000 7f7d09751e00 1 Processor -- start 2026-02-25T11:53:39.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d09751e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92d140 0x558fae98a3a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d09751e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92da50 0x558fae98a8e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d09751e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae9726f0 0x558fae98f8d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d09751e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558fae98fe10 con 0x558fae92da50 2026-02-25T11:53:39.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d09751e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558fae98ff80 con 0x558fae9726f0 2026-02-25T11:53:39.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d09751e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558fae990140 con 0x558fae92d140 2026-02-25T11:53:39.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d05ccc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae9726f0 0x558fae98f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d054cb640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92d140 0x558fae98a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d05ccc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae9726f0 0x558fae98f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:47572/0 (socket says 10.20.193.28:47572) 2026-02-25T11:53:39.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d05ccc640 1 -- 10.20.193.28:0/2450025861 learned_addr learned my addr 10.20.193.28:0/2450025861 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:39.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d04cca640 1 --2- 10.20.193.28:0/2450025861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92da50 0x558fae98a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d04cca640 1 -- 10.20.193.28:0/2450025861 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92d140 msgr2=0x558fae98a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d04cca640 1 --2- 10.20.193.28:0/2450025861 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92d140 0x558fae98a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d04cca640 1 -- 10.20.193.28:0/2450025861 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae9726f0 msgr2=0x558fae98f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d04cca640 1 --2- 10.20.193.28:0/2450025861 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae9726f0 0x558fae98f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.555+0000 7f7d04cca640 1 -- 10.20.193.28:0/2450025861 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fae7bd350 con 0x558fae92da50 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.556+0000 7f7d04cca640 1 --2- 10.20.193.28:0/2450025861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92da50 0x558fae98a8e0 secure :-1 s=READY pgs=640 gs=3 cs=0 l=1 c_cookie=4282134872415775786 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7cf40ad0c0 tx=0x7f7cf40ac930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6d32fa8ea7ac2a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.556+0000 7f7cbaffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.556+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2450025861 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7cf40b5430 con 0x558fae92da50 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.556+0000 7f7d09751e00 1 -- 10.20.193.28:0/2450025861 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d00164d50 con 0x558fae92da50 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.556+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2450025861 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7cf40adb30 con 0x558fae92da50 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.556+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2450025861 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7cf40b7de0 con 0x558fae92da50 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.556+0000 7f7d09751e00 10 client.16557 _ll_register_callbacks cb 0x558fae7ba9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.556+0000 7f7d09751e00 1 -- 10.20.193.28:0/2450025861 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558fae4cf6f0 con 0x558fae92da50 2026-02-25T11:53:39.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.556+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2450025861 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f7cf40b8270 con 0x558fae92da50 2026-02-25T11:53:39.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.556+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2450025861 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7cf40b7f40 con 0x558fae92da50 2026-02-25T11:53:39.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.556+0000 7f7d09751e00 10 client.16557 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:39.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.556+0000 7f7d09751e00 1 -- 10.20.193.28:0/2450025861 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x558fae7cf150 con 0x558fae92da50 2026-02-25T11:53:39.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.557+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2450025861 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f7cf40a93a0 con 0x558fae92da50 2026-02-25T11:53:39.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.557+0000 7f7d09751e00 10 client.16557 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:39.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.557+0000 7f7d09751e00 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:39.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.557+0000 7f7d09751e00 20 client.16557 populate_metadata read hostname 'trial028' 2026-02-25T11:53:39.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.557+0000 7f7d09751e00 1 --2- 10.20.193.28:0/2450025861 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558fae9925e0 0x558fae992a80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.557+0000 7f7d09751e00 4 client.16557 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:53:39.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.557+0000 7f7d09751e00 1 -- 10.20.193.28:0/2450025861 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fae80b910 con 0x558fae9925e0 2026-02-25T11:53:39.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.557+0000 7f7d054cb640 1 --2- 10.20.193.28:0/2450025861 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558fae9925e0 0x558fae992a80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.558+0000 7f7d054cb640 1 --2- 10.20.193.28:0/2450025861 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558fae9925e0 0x558fae992a80 crc :-1 s=READY pgs=106 gs=4 cs=0 l=0 c_cookie=7576184584808988601 s_cookie=16695527868573401231 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692403bd52547fb9 server_cookie=e7b26e23c7f3c08f in_seq=0 out_seq=0 2026-02-25T11:53:39.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.558+0000 7f7cbaffd640 10 client.16557 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:39.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.558+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2450025861 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x558fae80b910 con 0x558fae9925e0 2026-02-25T11:53:39.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.558+0000 7f7cbaffd640 10 client.16557 handle_command_reply: tid=0 2026-02-25T11:53:39.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.558+0000 7f7d09751e00 1 client.16557 shutdown 2026-02-25T11:53:39.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.559+0000 7f7d09751e00 1 -- 10.20.193.28:0/2450025861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92da50 msgr2=0x558fae98a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:39.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.559+0000 7f7d09751e00 1 --2- 10.20.193.28:0/2450025861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92da50 0x558fae98a8e0 secure :-1 s=READY pgs=640 gs=3 cs=0 l=1 c_cookie=4282134872415775786 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7cf40ad0c0 tx=0x7f7cf40ac930 comp rx=0 tx=0).stop 2026-02-25T11:53:39.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.559+0000 7f7d09751e00 1 -- 10.20.193.28:0/2450025861 shutdown_connections 2026-02-25T11:53:39.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.559+0000 7f7d09751e00 1 --2- 10.20.193.28:0/2450025861 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558fae9925e0 0x558fae992a80 crc :-1 s=READY pgs=106 gs=4 cs=0 l=0 c_cookie=7576184584808988601 s_cookie=16695527868573401231 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.559+0000 7f7d09751e00 1 --2- 10.20.193.28:0/2450025861 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae9726f0 0x558fae98f8d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.559+0000 7f7d09751e00 1 --2- 10.20.193.28:0/2450025861 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92da50 0x558fae98a8e0 unknown :-1 s=CLOSED pgs=640 gs=3 cs=0 l=1 c_cookie=4282134872415775786 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.559+0000 7f7d09751e00 1 --2- 10.20.193.28:0/2450025861 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92d140 0x558fae98a3a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.559+0000 7f7d09751e00 1 -- 10.20.193.28:0/2450025861 >> 10.20.193.28:0/2450025861 conn(0x558fae84cd40 msgr2=0x558fae92b230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:39.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.559+0000 7f7cbaffd640 0 client.16557 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:39.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.559+0000 7f7d09751e00 1 -- 10.20.193.28:0/2450025861 shutdown_connections 2026-02-25T11:53:39.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.559+0000 7f7d09751e00 1 -- 10.20.193.28:0/2450025861 wait complete. 2026-02-25T11:53:39.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.559+0000 7f7d09751e00 20 client.16557 trim_cache size 0 max 16384 2026-02-25T11:53:39.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.568+0000 7f7d09751e00 1 Processor -- start 2026-02-25T11:53:39.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.568+0000 7f7d09751e00 1 -- start start 2026-02-25T11:53:39.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d09751e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae92d140 0x558fae9751e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d09751e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92da50 0x558fae987570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d09751e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae9726f0 0x558fae987ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d09751e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558fae837470 con 0x558fae9726f0 2026-02-25T11:53:39.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d09751e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558fae4f7700 con 0x558fae92d140 2026-02-25T11:53:39.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d09751e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558fae8a16e0 con 0x558fae92da50 2026-02-25T11:53:39.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d05ccc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae9726f0 0x558fae987ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d05ccc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae9726f0 0x558fae987ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37110/0 (socket says 10.20.193.28:37110) 2026-02-25T11:53:39.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d05ccc640 1 -- 10.20.193.28:0/1092197694 learned_addr learned my addr 10.20.193.28:0/1092197694 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:39.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d04cca640 1 --2- 10.20.193.28:0/1092197694 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92da50 0x558fae987570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d05ccc640 1 -- 10.20.193.28:0/1092197694 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92da50 msgr2=0x558fae987570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:39.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d05ccc640 1 --2- 10.20.193.28:0/1092197694 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92da50 0x558fae987570 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d05ccc640 1 -- 10.20.193.28:0/1092197694 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae92d140 msgr2=0x558fae9751e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:39.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d05ccc640 1 --2- 10.20.193.28:0/1092197694 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae92d140 0x558fae9751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d05ccc640 1 -- 10.20.193.28:0/1092197694 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fae7c9bd0 con 0x558fae9726f0 2026-02-25T11:53:39.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7d05ccc640 1 --2- 10.20.193.28:0/1092197694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae9726f0 0x558fae987ab0 secure :-1 s=READY pgs=641 gs=1 cs=0 l=1 c_cookie=847767267812427065 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7cfc003cb0 tx=0x7f7cfc013820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3dfd6a229c539 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:39.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7ccd4e4640 1 -- 10.20.193.28:0/1092197694 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7cfc018070 con 0x558fae9726f0 2026-02-25T11:53:39.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.569+0000 7f7ccd4e4640 1 -- 10.20.193.28:0/1092197694 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7cfc003e70 con 0x558fae9726f0 2026-02-25T11:53:39.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.570+0000 7f7ccd4e4640 1 -- 10.20.193.28:0/1092197694 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7cfc010920 con 0x558fae9726f0 2026-02-25T11:53:39.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.570+0000 7f7d09751e00 1 -- 10.20.193.28:0/1092197694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae9726f0 msgr2=0x558fae987ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:39.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.570+0000 7f7d09751e00 1 --2- 10.20.193.28:0/1092197694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae9726f0 0x558fae987ab0 secure :-1 s=READY pgs=641 gs=1 cs=0 l=1 c_cookie=847767267812427065 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7cfc003cb0 tx=0x7f7cfc013820 comp rx=0 tx=0).stop 2026-02-25T11:53:39.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.570+0000 7f7d09751e00 1 -- 10.20.193.28:0/1092197694 shutdown_connections 2026-02-25T11:53:39.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.570+0000 7f7d09751e00 1 --2- 10.20.193.28:0/1092197694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae9726f0 0x558fae987ab0 unknown :-1 s=CLOSED pgs=641 gs=1 cs=0 l=1 c_cookie=847767267812427065 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.570+0000 7f7d09751e00 1 --2- 10.20.193.28:0/1092197694 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae92da50 0x558fae987570 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.570+0000 7f7d09751e00 1 --2- 10.20.193.28:0/1092197694 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae92d140 0x558fae9751e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.570+0000 7f7d09751e00 1 -- 10.20.193.28:0/1092197694 >> 10.20.193.28:0/1092197694 conn(0x558fae84cd40 msgr2=0x558fae974ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:39.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.570+0000 7f7d09751e00 1 -- 10.20.193.28:0/1092197694 shutdown_connections 2026-02-25T11:53:39.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.570+0000 7f7d09751e00 1 -- 10.20.193.28:0/1092197694 wait complete. 2026-02-25T11:53:39.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.571+0000 7f7d09751e00 1 -- start start 2026-02-25T11:53:39.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.571+0000 7f7d09751e00 1 Processor -- start 2026-02-25T11:53:39.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.571+0000 7f7d09751e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92d140 0x558fae90c8c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.571+0000 7f7d09751e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae92da50 0x558fae90ce00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.571+0000 7f7d09751e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae9726f0 0x558fae90d340 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.571+0000 7f7d09751e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558fae908490 con 0x558fae92d140 2026-02-25T11:53:39.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.571+0000 7f7d09751e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558fae990000 con 0x558fae92da50 2026-02-25T11:53:39.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.571+0000 7f7d09751e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558fae8c76b0 con 0x558fae9726f0 2026-02-25T11:53:39.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.571+0000 7f7d054cb640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92d140 0x558fae90c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.571+0000 7f7d054cb640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92d140 0x558fae90c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37122/0 (socket says 10.20.193.28:37122) 2026-02-25T11:53:39.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.571+0000 7f7d054cb640 1 -- 10.20.193.28:0/2022737131 learned_addr learned my addr 10.20.193.28:0/2022737131 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:39.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.571+0000 7f7d04cca640 1 --2- 10.20.193.28:0/2022737131 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae92da50 0x558fae90ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.571+0000 7f7d05ccc640 1 --2- 10.20.193.28:0/2022737131 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae9726f0 0x558fae90d340 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7d054cb640 1 -- 10.20.193.28:0/2022737131 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae9726f0 msgr2=0x558fae90d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:39.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7d054cb640 1 --2- 10.20.193.28:0/2022737131 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae9726f0 0x558fae90d340 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7d054cb640 1 -- 10.20.193.28:0/2022737131 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae92da50 msgr2=0x558fae90ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:39.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7d054cb640 1 --2- 10.20.193.28:0/2022737131 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae92da50 0x558fae90ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7d054cb640 1 -- 10.20.193.28:0/2022737131 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fae7bd680 con 0x558fae92d140 2026-02-25T11:53:39.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7d054cb640 1 --2- 10.20.193.28:0/2022737131 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92d140 0x558fae90c8c0 secure :-1 s=READY pgs=642 gs=2 cs=0 l=1 c_cookie=17210086840052497341 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7cf00098c0 tx=0x7f7cf0004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed682cc4ef43bbd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:39.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7cbaffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:39.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2022737131 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7cf002f9f0 con 0x558fae92d140 2026-02-25T11:53:39.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2022737131 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7cf0058bb0 con 0x558fae92d140 2026-02-25T11:53:39.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2022737131 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f7cf0038a40 con 0x558fae92d140 2026-02-25T11:53:39.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7d09751e00 1 -- 10.20.193.28:0/2022737131 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cfc003950 con 0x558fae92d140 2026-02-25T11:53:39.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2022737131 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f7cf0038bd0 con 0x558fae92d140 2026-02-25T11:53:39.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7d09751e00 10 client.16563 _ll_register_callbacks cb 0x558fae7ba9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:39.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.572+0000 7f7d09751e00 1 -- 10.20.193.28:0/2022737131 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cf4009e50 con 0x558fae92d140 2026-02-25T11:53:39.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.573+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2022737131 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7cf0038d90 con 0x558fae92d140 2026-02-25T11:53:39.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.573+0000 7f7d09751e00 10 client.16563 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:39.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.573+0000 7f7d09751e00 1 -- 10.20.193.28:0/2022737131 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x558fae7cf3b0 con 0x558fae92d140 2026-02-25T11:53:39.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.573+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2022737131 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f7cf0058d20 con 0x558fae92d140 2026-02-25T11:53:39.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.573+0000 7f7d09751e00 10 client.16563 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:39.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.573+0000 7f7d09751e00 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:39.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.573+0000 7f7d09751e00 20 client.16563 populate_metadata read hostname 'trial028' 2026-02-25T11:53:39.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.573+0000 7f7d09751e00 1 --2- 10.20.193.28:0/2022737131 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558fae9c2b10 0x558fae9c2fb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:39.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.573+0000 7f7d09751e00 4 client.16563 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:53:39.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.573+0000 7f7d09751e00 1 -- 10.20.193.28:0/2022737131 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558fae4ccf40 con 0x558fae9c2b10 2026-02-25T11:53:39.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.573+0000 7f7d04cca640 1 --2- 10.20.193.28:0/2022737131 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558fae9c2b10 0x558fae9c2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:39.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.574+0000 7f7d04cca640 1 --2- 10.20.193.28:0/2022737131 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558fae9c2b10 0x558fae9c2fb0 crc :-1 s=READY pgs=107 gs=4 cs=0 l=0 c_cookie=16979152567746898712 s_cookie=6243178036059061389 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba2114d0bb72b18 server_cookie=56a4395894d23c8d in_seq=0 out_seq=0 2026-02-25T11:53:39.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.574+0000 7f7cbaffd640 10 client.16563 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:39.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.574+0000 7f7cbaffd640 1 -- 10.20.193.28:0/2022737131 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x558fae4ccf40 con 0x558fae9c2b10 2026-02-25T11:53:39.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.574+0000 7f7cbaffd640 10 client.16563 handle_command_reply: tid=0 2026-02-25T11:53:39.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.574+0000 7f7d09751e00 1 client.16563 shutdown 2026-02-25T11:53:39.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.574+0000 7f7d09751e00 1 -- 10.20.193.28:0/2022737131 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92d140 msgr2=0x558fae90c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:39.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.574+0000 7f7d09751e00 1 --2- 10.20.193.28:0/2022737131 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92d140 0x558fae90c8c0 secure :-1 s=READY pgs=642 gs=2 cs=0 l=1 c_cookie=17210086840052497341 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7cf00098c0 tx=0x7f7cf0004460 comp rx=0 tx=0).stop 2026-02-25T11:53:39.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.575+0000 7f7d09751e00 1 -- 10.20.193.28:0/2022737131 shutdown_connections 2026-02-25T11:53:39.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.575+0000 7f7d09751e00 1 --2- 10.20.193.28:0/2022737131 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558fae9c2b10 0x558fae9c2fb0 crc :-1 s=READY pgs=107 gs=4 cs=0 l=0 c_cookie=16979152567746898712 s_cookie=6243178036059061389 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.575+0000 7f7d09751e00 1 --2- 10.20.193.28:0/2022737131 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fae9726f0 0x558fae90d340 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.575+0000 7f7d09751e00 1 --2- 10.20.193.28:0/2022737131 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fae92da50 0x558fae90ce00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.575+0000 7f7d09751e00 1 --2- 10.20.193.28:0/2022737131 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fae92d140 0x558fae90c8c0 unknown :-1 s=CLOSED pgs=642 gs=2 cs=0 l=1 c_cookie=17210086840052497341 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.575+0000 7f7d09751e00 1 -- 10.20.193.28:0/2022737131 >> 10.20.193.28:0/2022737131 conn(0x558fae84cd40 msgr2=0x558fae986560 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:39.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.575+0000 7f7cbaffd640 0 client.16563 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:39.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.575+0000 7f7d09751e00 1 -- 10.20.193.28:0/2022737131 shutdown_connections 2026-02-25T11:53:39.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.575+0000 7f7d09751e00 1 -- 10.20.193.28:0/2022737131 wait complete. 2026-02-25T11:53:39.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.575+0000 7f7d09751e00 20 client.16563 trim_cache size 0 max 16384 2026-02-25T11:53:39.583 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:53:39.584 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:53:39.584 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:53:39.584 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:53:39.584 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:53:39.584 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:53:39.584 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:39.584 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:39.584 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:39.584 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:39.584 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:39.584 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:39.585 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:53:39.585 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:53:39.585 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:39.585 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:39.585 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:39.585 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:39.585 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:39.585 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:39.585 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:53:39.585 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:53:39.585 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:39.585 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:39.585 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:53:39.586 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:39.587 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:39.588 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:39.589 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:39.590 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:39.590 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:39.590 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:53:39.590 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:53:39.590 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:39.590 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:39.590 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:39.590 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:39.590 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:39.590 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:53:39.590 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:53:39.590 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:53:39.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.577+0000 7f7d07756640 1 -- 10.20.193.28:0/3411428301 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f7cdc0b5760 msgr2=0x7f7cdc0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:39.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.577+0000 7f7d07756640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f7cdc0b5760 0x7f7cdc0d5c10 secure :-1 s=READY pgs=324 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7cf00023d0 tx=0x7f7cf0031040 comp rx=0 tx=0).stop 2026-02-25T11:53:39.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.577+0000 7f7d07756640 1 -- 10.20.193.28:0/3411428301 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7d00110130 msgr2=0x7f7d0011e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:39.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.577+0000 7f7d07756640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7d00110130 0x7f7d0011e650 secure :-1 s=READY pgs=638 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7cf400d950 tx=0x7f7cf400de20 comp rx=0 tx=0).stop 2026-02-25T11:53:39.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.577+0000 7f7d07756640 1 -- 10.20.193.28:0/3411428301 shutdown_connections 2026-02-25T11:53:39.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.577+0000 7f7d07756640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f7cdc0b5760 0x7f7cdc0d5c10 unknown :-1 s=CLOSED pgs=324 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.591 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.577+0000 7f7d07756640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7d00115930 0x7f7d00123a20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.577+0000 7f7d07756640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7d00110130 0x7f7d0011e650 unknown :-1 s=CLOSED pgs=638 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.577+0000 7f7d07756640 1 --2- 10.20.193.28:0/3411428301 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f7d0010f740 0x7f7d0011e110 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:39.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.577+0000 7f7d07756640 1 -- 10.20.193.28:0/3411428301 >> 10.20.193.28:0/3411428301 conn(0x7f7d000ecff0 msgr2=0x7f7d0010d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:39.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.577+0000 7f7d07756640 1 -- 10.20.193.28:0/3411428301 shutdown_connections 2026-02-25T11:53:39.592 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:39.577+0000 7f7d07756640 1 -- 10.20.193.28:0/3411428301 wait complete. 2026-02-25T11:53:39.683 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:53:39.683 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:53:39.870 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:40.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.050+0000 7f6f7b577640 1 -- 10.20.193.28:0/2540061449 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f7c115880 msgr2=0x7f6f7c176250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.050+0000 7f6f7b577640 1 --2- 10.20.193.28:0/2540061449 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f7c115880 0x7f6f7c176250 secure :-1 s=READY pgs=643 gs=1 cs=0 l=1 c_cookie=16202684198336338489 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6f6c00aa30 tx=0x7f6f6c030270 comp rx=0 tx=0).stop 2026-02-25T11:53:40.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.051+0000 7f6f7b577640 1 -- 10.20.193.28:0/2540061449 shutdown_connections 2026-02-25T11:53:40.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.051+0000 7f6f7b577640 1 --2- 10.20.193.28:0/2540061449 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f7c115880 0x7f6f7c176250 unknown :-1 s=CLOSED pgs=643 gs=1 cs=0 l=1 c_cookie=16202684198336338489 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.051+0000 7f6f7b577640 1 --2- 10.20.193.28:0/2540061449 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6f7c110080 0x7f6f7c1150d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.051+0000 7f6f7b577640 1 --2- 10.20.193.28:0/2540061449 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6f7c10f690 0x7f6f7c10fa60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.051+0000 7f6f7b577640 1 -- 10.20.193.28:0/2540061449 >> 10.20.193.28:0/2540061449 conn(0x7f6f7c0ecf00 msgr2=0x7f6f7c10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:40.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.051+0000 7f6f7b577640 1 -- 10.20.193.28:0/2540061449 shutdown_connections 2026-02-25T11:53:40.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.051+0000 7f6f7b577640 1 -- 10.20.193.28:0/2540061449 wait complete. 2026-02-25T11:53:40.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.051+0000 7f6f7b577640 1 Processor -- start 2026-02-25T11:53:40.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.051+0000 7f6f7b577640 1 -- start start 2026-02-25T11:53:40.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7b577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6f7c10f690 0x7f6f7c11e010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7b577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6f7c110080 0x7f6f7c11e550 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7b577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f7c115880 0x7f6f7c123920 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7b577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c123f40 con 0x7f6f7c115880 2026-02-25T11:53:40.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7b577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c1240b0 con 0x7f6f7c10f690 2026-02-25T11:53:40.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7b577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7c124270 con 0x7f6f7c110080 2026-02-25T11:53:40.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7ad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f7c115880 0x7f6f7c123920 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7a575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6f7c10f690 0x7f6f7c11e010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7a575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6f7c10f690 0x7f6f7c11e010 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:47610/0 (socket says 10.20.193.28:47610) 2026-02-25T11:53:40.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7a575640 1 -- 10.20.193.28:0/596121511 learned_addr learned my addr 10.20.193.28:0/596121511 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:40.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f79d74640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6f7c110080 0x7f6f7c11e550 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7ad76640 1 -- 10.20.193.28:0/596121511 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6f7c110080 msgr2=0x7f6f7c11e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7ad76640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6f7c110080 0x7f6f7c11e550 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7ad76640 1 -- 10.20.193.28:0/596121511 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6f7c10f690 msgr2=0x7f6f7c11e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7ad76640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6f7c10f690 0x7f6f7c11e010 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f7ad76640 1 -- 10.20.193.28:0/596121511 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f6c00a650 con 0x7f6f7c115880 2026-02-25T11:53:40.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.052+0000 7f6f79d74640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6f7c110080 0x7f6f7c11e550 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:53:40.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.053+0000 7f6f7ad76640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f7c115880 0x7f6f7c123920 secure :-1 s=READY pgs=644 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6f6c033040 tx=0x7f6f6c030d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:40.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.053+0000 7f6f637fe640 1 -- 10.20.193.28:0/596121511 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6f6c03e070 con 0x7f6f7c115880 2026-02-25T11:53:40.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.053+0000 7f6f637fe640 1 -- 10.20.193.28:0/596121511 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6f6c039de0 con 0x7f6f7c115880 2026-02-25T11:53:40.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.053+0000 7f6f7b577640 1 -- 10.20.193.28:0/596121511 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7c1244f0 con 0x7f6f7c115880 2026-02-25T11:53:40.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.053+0000 7f6f637fe640 1 -- 10.20.193.28:0/596121511 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6f6c042790 con 0x7f6f7c115880 2026-02-25T11:53:40.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.053+0000 7f6f7b577640 1 -- 10.20.193.28:0/596121511 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f7c164ab0 con 0x7f6f7c115880 2026-02-25T11:53:40.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.053+0000 7f6f7b577640 1 -- 10.20.193.28:0/596121511 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f7c110970 con 0x7f6f7c115880 2026-02-25T11:53:40.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.056+0000 7f6f637fe640 1 -- 10.20.193.28:0/596121511 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f6f6c0052e0 con 0x7f6f7c115880 2026-02-25T11:53:40.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.057+0000 7f6f637fe640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6f540b56e0 0x7f6f540d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.058+0000 7f6f637fe640 1 -- 10.20.193.28:0/596121511 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6f6c10c4a0 con 0x7f6f7c115880 2026-02-25T11:53:40.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.058+0000 7f6f7a575640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6f540b56e0 0x7f6f540d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.058+0000 7f6f637fe640 1 -- 10.20.193.28:0/596121511 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f6f6c0d22b0 con 0x7f6f7c115880 2026-02-25T11:53:40.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.058+0000 7f6f7a575640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6f540b56e0 0x7f6f540d5b90 secure :-1 s=READY pgs=325 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6f700095d0 tx=0x7f6f70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:40.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.199+0000 7f6f7b577640 1 -- 10.20.193.28:0/596121511 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f7c10fa60 con 0x7f6f7c115880 2026-02-25T11:53:40.201 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.200+0000 7f6f637fe640 1 -- 10.20.193.28:0/596121511 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f6f6c0d24a0 con 0x7f6f7c115880 2026-02-25T11:53:40.202 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:53:40.202 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:53:40.202 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:53:40.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.201+0000 7f6f7b577640 1 -- 10.20.193.28:0/596121511 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6f540b56e0 msgr2=0x7f6f540d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.201+0000 7f6f7b577640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6f540b56e0 0x7f6f540d5b90 secure :-1 s=READY pgs=325 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6f700095d0 tx=0x7f6f70005f90 comp rx=0 tx=0).stop 2026-02-25T11:53:40.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.201+0000 7f6f7b577640 1 -- 10.20.193.28:0/596121511 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f7c115880 msgr2=0x7f6f7c123920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.201+0000 7f6f7b577640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f7c115880 0x7f6f7c123920 secure :-1 s=READY pgs=644 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6f6c033040 tx=0x7f6f6c030d00 comp rx=0 tx=0).stop 2026-02-25T11:53:40.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.202+0000 7f6f7b577640 1 -- 10.20.193.28:0/596121511 shutdown_connections 2026-02-25T11:53:40.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.202+0000 7f6f7b577640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6f540b56e0 0x7f6f540d5b90 unknown :-1 s=CLOSED pgs=325 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.202+0000 7f6f7b577640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6f7c115880 0x7f6f7c123920 unknown :-1 s=CLOSED pgs=644 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.202+0000 7f6f7b577640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6f7c110080 0x7f6f7c11e550 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.202+0000 7f6f7b577640 1 --2- 10.20.193.28:0/596121511 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6f7c10f690 0x7f6f7c11e010 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.202+0000 7f6f7b577640 1 -- 10.20.193.28:0/596121511 >> 10.20.193.28:0/596121511 conn(0x7f6f7c0ecf00 msgr2=0x7f6f7c10d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:40.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.202+0000 7f6f7b577640 1 -- 10.20.193.28:0/596121511 shutdown_connections 2026-02-25T11:53:40.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.202+0000 7f6f7b577640 1 -- 10.20.193.28:0/596121511 wait complete. 2026-02-25T11:53:40.308 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:53:40.493 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:40.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:40 trial167 ceph-mon[43392]: pgmap v389: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 11 op/s 2026-02-25T11:53:40.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:40 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/596121511' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:53:40.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:40 trial119 ceph-mon[42710]: pgmap v389: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 11 op/s 2026-02-25T11:53:40.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:40 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/596121511' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:53:40.536 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:40 trial028 ceph-mon[39224]: pgmap v389: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 17 KiB/s wr, 11 op/s 2026-02-25T11:53:40.536 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:40 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/596121511' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:53:40.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.666+0000 7f846cb03640 1 -- 10.20.193.28:0/2443742055 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8468110080 msgr2=0x7f84681150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.666+0000 7f846cb03640 1 --2- 10.20.193.28:0/2443742055 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8468110080 0x7f84681150d0 secure :-1 s=READY pgs=645 gs=1 cs=0 l=1 c_cookie=4908851455288115554 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8458009950 tx=0x7f845802f190 comp rx=0 tx=0).stop 2026-02-25T11:53:40.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.666+0000 7f846cb03640 1 -- 10.20.193.28:0/2443742055 shutdown_connections 2026-02-25T11:53:40.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.666+0000 7f846cb03640 1 --2- 10.20.193.28:0/2443742055 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8468115880 0x7f8468176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.666+0000 7f846cb03640 1 --2- 10.20.193.28:0/2443742055 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8468110080 0x7f84681150d0 unknown :-1 s=CLOSED pgs=645 gs=1 cs=0 l=1 c_cookie=4908851455288115554 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.666+0000 7f846cb03640 1 --2- 10.20.193.28:0/2443742055 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f846810f690 0x7f846810fa60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.666+0000 7f846cb03640 1 -- 10.20.193.28:0/2443742055 >> 10.20.193.28:0/2443742055 conn(0x7f84680ecf00 msgr2=0x7f846810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:40.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.667+0000 7f846cb03640 1 -- 10.20.193.28:0/2443742055 shutdown_connections 2026-02-25T11:53:40.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.667+0000 7f846cb03640 1 -- 10.20.193.28:0/2443742055 wait complete. 2026-02-25T11:53:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.667+0000 7f846cb03640 1 Processor -- start 2026-02-25T11:53:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.667+0000 7f846cb03640 1 -- start start 2026-02-25T11:53:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f846cb03640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f846810f690 0x7f846811e0a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f846cb03640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8468110080 0x7f846811e5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f846cb03640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8468115880 0x7f84681239b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f846cb03640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8468123fd0 con 0x7f8468115880 2026-02-25T11:53:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f846cb03640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8468124140 con 0x7f8468110080 2026-02-25T11:53:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f846cb03640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8468124300 con 0x7f846810f690 2026-02-25T11:53:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f84677fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f846810f690 0x7f846811e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f84677fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f846810f690 0x7f846811e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34888/0 (socket says 10.20.193.28:34888) 2026-02-25T11:53:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f84677fe640 1 -- 10.20.193.28:0/1209058298 learned_addr learned my addr 10.20.193.28:0/1209058298 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f8466ffd640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8468110080 0x7f846811e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f8467fff640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8468115880 0x7f84681239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f84677fe640 1 -- 10.20.193.28:0/1209058298 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8468110080 msgr2=0x7f846811e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f84677fe640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8468110080 0x7f846811e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f84677fe640 1 -- 10.20.193.28:0/1209058298 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8468115880 msgr2=0x7f84681239b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f84677fe640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8468115880 0x7f84681239b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.668+0000 7f84677fe640 1 -- 10.20.193.28:0/1209058298 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8458009570 con 0x7f846810f690 2026-02-25T11:53:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.669+0000 7f84677fe640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f846810f690 0x7f846811e0a0 secure :-1 s=READY pgs=175 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8454009480 tx=0x7f845400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.669+0000 7f8464ff9640 1 -- 10.20.193.28:0/1209058298 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f845400cd40 con 0x7f846810f690 2026-02-25T11:53:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.669+0000 7f846cb03640 1 -- 10.20.193.28:0/1209058298 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84681245e0 con 0x7f846810f690 2026-02-25T11:53:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.669+0000 7f8464ff9640 1 -- 10.20.193.28:0/1209058298 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8454004d10 con 0x7f846810f690 2026-02-25T11:53:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.669+0000 7f8464ff9640 1 -- 10.20.193.28:0/1209058298 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f84540106b0 con 0x7f846810f690 2026-02-25T11:53:40.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.669+0000 7f846cb03640 1 -- 10.20.193.28:0/1209058298 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8468164ce0 con 0x7f846810f690 2026-02-25T11:53:40.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.672+0000 7f8464ff9640 1 -- 10.20.193.28:0/1209058298 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f845401f060 con 0x7f846810f690 2026-02-25T11:53:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.674+0000 7f8464ff9640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f843c0b5760 0x7f843c0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.674+0000 7f8464ff9640 1 -- 10.20.193.28:0/1209058298 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8454014070 con 0x7f846810f690 2026-02-25T11:53:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.674+0000 7f8466ffd640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f843c0b5760 0x7f843c0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.674+0000 7f8466ffd640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f843c0b5760 0x7f843c0d5c10 secure :-1 s=READY pgs=326 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8468124a40 tx=0x7f845803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.674+0000 7f846fd87e00 1 Processor -- start 2026-02-25T11:53:40.676 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f846fd87e00 1 -- start start 2026-02-25T11:53:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f846fd87e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e061f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f846fd87e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e062440 0x55849e0a6ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f846fd87e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e0a70e0 0x55849e0c74c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f846fd87e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55849df778a0 con 0x55849e062440 2026-02-25T11:53:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f846fd87e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55849df735d0 con 0x55849e0a70e0 2026-02-25T11:53:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f846fd87e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55849df6d570 con 0x55849e061b30 2026-02-25T11:53:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f84677fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e061f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f84677fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e061f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34890/0 (socket says 10.20.193.28:34890) 2026-02-25T11:53:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f84677fe640 1 -- 10.20.193.28:0/3357629502 learned_addr learned my addr 10.20.193.28:0/3357629502 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f8466ffd640 1 --2- 10.20.193.28:0/3357629502 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e062440 0x55849e0a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.677 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f8467fff640 1 --2- 10.20.193.28:0/3357629502 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e0a70e0 0x55849e0c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f8466ffd640 1 -- 10.20.193.28:0/3357629502 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 msgr2=0x55849e061f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f8466ffd640 1 --2- 10.20.193.28:0/3357629502 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e061f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f8466ffd640 1 -- 10.20.193.28:0/3357629502 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e0a70e0 msgr2=0x55849e0c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f8466ffd640 1 --2- 10.20.193.28:0/3357629502 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e0a70e0 0x55849e0c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.675+0000 7f8466ffd640 1 -- 10.20.193.28:0/3357629502 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55849df18d10 con 0x55849e062440 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.676+0000 7f8466ffd640 1 --2- 10.20.193.28:0/3357629502 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e062440 0x55849e0a6ba0 secure :-1 s=READY pgs=646 gs=1 cs=0 l=1 c_cookie=605590684604670392 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8458005b30 tx=0x7f8458040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8677d84e325a1b8 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.676+0000 7f842bfff640 1 -- 10.20.193.28:0/3357629502 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8458002a00 con 0x55849e062440 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.676+0000 7f842bfff640 1 -- 10.20.193.28:0/3357629502 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f845803e040 con 0x55849e062440 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.676+0000 7f842bfff640 1 -- 10.20.193.28:0/3357629502 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f845804f6c0 con 0x55849e062440 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.676+0000 7f846fd87e00 1 -- 10.20.193.28:0/3357629502 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e062440 msgr2=0x55849e0a6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.676+0000 7f846fd87e00 1 --2- 10.20.193.28:0/3357629502 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e062440 0x55849e0a6ba0 secure :-1 s=READY pgs=646 gs=1 cs=0 l=1 c_cookie=605590684604670392 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8458005b30 tx=0x7f8458040000 comp rx=0 tx=0).stop 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.676+0000 7f846fd87e00 1 -- 10.20.193.28:0/3357629502 shutdown_connections 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.676+0000 7f846fd87e00 1 --2- 10.20.193.28:0/3357629502 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e0a70e0 0x55849e0c74c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.676+0000 7f846fd87e00 1 --2- 10.20.193.28:0/3357629502 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e062440 0x55849e0a6ba0 unknown :-1 s=CLOSED pgs=646 gs=1 cs=0 l=1 c_cookie=605590684604670392 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.676+0000 7f846fd87e00 1 --2- 10.20.193.28:0/3357629502 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e061f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.676+0000 7f846fd87e00 1 -- 10.20.193.28:0/3357629502 >> 10.20.193.28:0/3357629502 conn(0x55849df807a0 msgr2=0x55849e060410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.676+0000 7f846fd87e00 1 -- 10.20.193.28:0/3357629502 shutdown_connections 2026-02-25T11:53:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.676+0000 7f846fd87e00 1 -- 10.20.193.28:0/3357629502 wait complete. 2026-02-25T11:53:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.677+0000 7f846fd87e00 1 -- start start 2026-02-25T11:53:40.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.678+0000 7f846fd87e00 1 Processor -- start 2026-02-25T11:53:40.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.678+0000 7f846fd87e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e079ed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.678+0000 7f846fd87e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e062440 0x55849e07a410 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.678+0000 7f846fd87e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e0a70e0 0x55849e07f400 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.678+0000 7f846fd87e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55849e07f940 con 0x55849e0a70e0 2026-02-25T11:53:40.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.678+0000 7f846fd87e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55849e07fab0 con 0x55849e062440 2026-02-25T11:53:40.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.678+0000 7f846fd87e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55849e07fc70 con 0x55849e061b30 2026-02-25T11:53:40.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.678+0000 7f84677fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e079ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.678+0000 7f8467fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e0a70e0 0x55849e07f400 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.678+0000 7f84677fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e079ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34896/0 (socket says 10.20.193.28:34896) 2026-02-25T11:53:40.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.678+0000 7f84677fe640 1 -- 10.20.193.28:0/1246772757 learned_addr learned my addr 10.20.193.28:0/1246772757 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:40.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.678+0000 7f8466ffd640 1 --2- 10.20.193.28:0/1246772757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e062440 0x55849e07a410 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f8467fff640 1 -- 10.20.193.28:0/1246772757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 msgr2=0x55849e079ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:40.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f8467fff640 1 --2- 10.20.193.28:0/1246772757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e079ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f8467fff640 1 -- 10.20.193.28:0/1246772757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e062440 msgr2=0x55849e07a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:40.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f8467fff640 1 --2- 10.20.193.28:0/1246772757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e062440 0x55849e07a410 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f8467fff640 1 -- 10.20.193.28:0/1246772757 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55849def13c0 con 0x55849e0a70e0 2026-02-25T11:53:40.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f8467fff640 1 --2- 10.20.193.28:0/1246772757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e0a70e0 0x55849e07f400 secure :-1 s=READY pgs=647 gs=1 cs=0 l=1 c_cookie=3318611677958381100 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f845c003bb0 tx=0x7f845c00aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0e13bd042c1e2c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:40.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f8466ffd640 1 --2- 10.20.193.28:0/1246772757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e062440 0x55849e07a410 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:53:40.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f84297fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:40.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f84297fa640 1 -- 10.20.193.28:0/1246772757 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f845c018070 con 0x55849e0a70e0 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f84297fa640 1 -- 10.20.193.28:0/1246772757 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f845c011d20 con 0x55849e0a70e0 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f84297fa640 1 -- 10.20.193.28:0/1246772757 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f845c0103c0 con 0x55849e0a70e0 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f846fd87e00 1 -- 10.20.193.28:0/1246772757 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8468176ac0 con 0x55849e0a70e0 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f846fd87e00 10 client.16581 _ll_register_callbacks cb 0x55849deeea10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f846fd87e00 1 -- 10.20.193.28:0/1246772757 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55849dc036f0 con 0x55849e0a70e0 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f84297fa640 1 -- 10.20.193.28:0/1246772757 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f845c011860 con 0x55849e0a70e0 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f84297fa640 1 -- 10.20.193.28:0/1246772757 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f845c011a20 con 0x55849e0a70e0 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f846fd87e00 10 client.16581 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.679+0000 7f846fd87e00 1 -- 10.20.193.28:0/1246772757 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55849df030e0 con 0x55849e0a70e0 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.680+0000 7f84297fa640 1 -- 10.20.193.28:0/1246772757 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f845c01c3c0 con 0x55849e0a70e0 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.680+0000 7f846fd87e00 10 client.16581 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.680+0000 7f846fd87e00 10 client.16581 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.680+0000 7f846fd87e00 20 client.16581 populate_metadata read hostname 'trial028' 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.680+0000 7f846fd87e00 1 --2- 10.20.193.28:0/1246772757 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55849e0a47d0 0x55849e0a4c70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.680+0000 7f846fd87e00 4 client.16581 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.680+0000 7f846fd87e00 1 -- 10.20.193.28:0/1246772757 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55849df3ffc0 con 0x55849e0a47d0 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.680+0000 7f84677fe640 1 --2- 10.20.193.28:0/1246772757 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55849e0a47d0 0x55849e0a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.681+0000 7f84677fe640 1 --2- 10.20.193.28:0/1246772757 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55849e0a47d0 0x55849e0a4c70 crc :-1 s=READY pgs=43 gs=4 cs=0 l=0 c_cookie=7873227369328759578 s_cookie=10780039860143311811 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d43529408b9531a server_cookie=959a65234064cbc3 in_seq=0 out_seq=0 2026-02-25T11:53:40.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.681+0000 7f84297fa640 10 client.16581 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:53:40.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.681+0000 7f84297fa640 1 -- 10.20.193.28:0/1246772757 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55849df3ffc0 con 0x55849e0a47d0 2026-02-25T11:53:40.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.681+0000 7f84297fa640 10 client.16581 handle_command_reply: tid=0 2026-02-25T11:53:40.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.681+0000 7f846fd87e00 1 client.16581 shutdown 2026-02-25T11:53:40.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.681+0000 7f846fd87e00 1 -- 10.20.193.28:0/1246772757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e0a70e0 msgr2=0x55849e07f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.681+0000 7f846fd87e00 1 --2- 10.20.193.28:0/1246772757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e0a70e0 0x55849e07f400 secure :-1 s=READY pgs=647 gs=1 cs=0 l=1 c_cookie=3318611677958381100 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f845c003bb0 tx=0x7f845c00aab0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.682+0000 7f846fd87e00 1 -- 10.20.193.28:0/1246772757 shutdown_connections 2026-02-25T11:53:40.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.682+0000 7f846fd87e00 1 --2- 10.20.193.28:0/1246772757 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55849e0a47d0 0x55849e0a4c70 crc :-1 s=READY pgs=43 gs=4 cs=0 l=0 c_cookie=7873227369328759578 s_cookie=10780039860143311811 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.682+0000 7f846fd87e00 1 --2- 10.20.193.28:0/1246772757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e0a70e0 0x55849e07f400 unknown :-1 s=CLOSED pgs=647 gs=1 cs=0 l=1 c_cookie=3318611677958381100 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.682+0000 7f846fd87e00 1 --2- 10.20.193.28:0/1246772757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e062440 0x55849e07a410 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.682+0000 7f846fd87e00 1 --2- 10.20.193.28:0/1246772757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e079ed0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.682+0000 7f846fd87e00 1 -- 10.20.193.28:0/1246772757 >> 10.20.193.28:0/1246772757 conn(0x55849df807a0 msgr2=0x55849e060b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:40.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.682+0000 7f84297fa640 0 client.16581 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:53:40.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.682+0000 7f846fd87e00 1 -- 10.20.193.28:0/1246772757 shutdown_connections 2026-02-25T11:53:40.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.682+0000 7f846fd87e00 1 -- 10.20.193.28:0/1246772757 wait complete. 2026-02-25T11:53:40.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.682+0000 7f846fd87e00 20 client.16581 trim_cache size 0 max 16384 2026-02-25T11:53:40.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f846fd87e00 1 Processor -- start 2026-02-25T11:53:40.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f846fd87e00 1 -- start start 2026-02-25T11:53:40.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f846fd87e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e0772f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f846fd87e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e062440 0x55849e077830 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f846fd87e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e0a70e0 0x55849e0a6560 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f846fd87e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55849df6b1a0 con 0x55849e0a70e0 2026-02-25T11:53:40.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f846fd87e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55849dc2b700 con 0x55849e062440 2026-02-25T11:53:40.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f846fd87e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55849e03ce80 con 0x55849e061b30 2026-02-25T11:53:40.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f84677fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e0772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f8467fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e0a70e0 0x55849e0a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f84677fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e0772f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34898/0 (socket says 10.20.193.28:34898) 2026-02-25T11:53:40.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f84677fe640 1 -- 10.20.193.28:0/2925107439 learned_addr learned my addr 10.20.193.28:0/2925107439 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:40.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f8466ffd640 1 --2- 10.20.193.28:0/2925107439 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e062440 0x55849e077830 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f8467fff640 1 -- 10.20.193.28:0/2925107439 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 msgr2=0x55849e0772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:40.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f8467fff640 1 --2- 10.20.193.28:0/2925107439 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e0772f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f8467fff640 1 -- 10.20.193.28:0/2925107439 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e062440 msgr2=0x55849e077830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:40.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f8467fff640 1 --2- 10.20.193.28:0/2925107439 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e062440 0x55849e077830 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.690+0000 7f8467fff640 1 -- 10.20.193.28:0/2925107439 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55849defdb60 con 0x55849e0a70e0 2026-02-25T11:53:40.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.691+0000 7f8467fff640 1 --2- 10.20.193.28:0/2925107439 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e0a70e0 0x55849e0a6560 secure :-1 s=READY pgs=648 gs=2 cs=0 l=1 c_cookie=24985448359549398 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f845c027870 tx=0x7f845c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c422060659d6 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:40.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.691+0000 7f842b7fe640 1 -- 10.20.193.28:0/2925107439 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f845c02d070 con 0x55849e0a70e0 2026-02-25T11:53:40.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.691+0000 7f842b7fe640 1 -- 10.20.193.28:0/2925107439 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55849dc036f0 con 0x55849e0a70e0 2026-02-25T11:53:40.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.691+0000 7f842b7fe640 1 -- 10.20.193.28:0/2925107439 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f845c018040 con 0x55849e0a70e0 2026-02-25T11:53:40.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.691+0000 7f846fd87e00 1 -- 10.20.193.28:0/2925107439 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e0a70e0 msgr2=0x55849e0a6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.691+0000 7f846fd87e00 1 --2- 10.20.193.28:0/2925107439 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e0a70e0 0x55849e0a6560 secure :-1 s=READY pgs=648 gs=2 cs=0 l=1 c_cookie=24985448359549398 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f845c027870 tx=0x7f845c020000 comp rx=0 tx=0).stop 2026-02-25T11:53:40.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.691+0000 7f846fd87e00 1 -- 10.20.193.28:0/2925107439 shutdown_connections 2026-02-25T11:53:40.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.691+0000 7f846fd87e00 1 --2- 10.20.193.28:0/2925107439 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e0a70e0 0x55849e0a6560 unknown :-1 s=CLOSED pgs=648 gs=2 cs=0 l=1 c_cookie=24985448359549398 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.691+0000 7f846fd87e00 1 --2- 10.20.193.28:0/2925107439 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e062440 0x55849e077830 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.691+0000 7f846fd87e00 1 --2- 10.20.193.28:0/2925107439 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e061b30 0x55849e0772f0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.691+0000 7f846fd87e00 1 -- 10.20.193.28:0/2925107439 >> 10.20.193.28:0/2925107439 conn(0x55849df807a0 msgr2=0x55849e078af0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:40.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.691+0000 7f846fd87e00 1 -- 10.20.193.28:0/2925107439 shutdown_connections 2026-02-25T11:53:40.694 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.692+0000 7f846fd87e00 1 -- 10.20.193.28:0/2925107439 wait complete. 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.692+0000 7f846fd87e00 1 -- start start 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.692+0000 7f846fd87e00 1 Processor -- start 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.692+0000 7f846fd87e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e061b30 0x55849e080670 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.692+0000 7f846fd87e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e062440 0x55849e080bb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.692+0000 7f846fd87e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e0a70e0 0x55849e085510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.692+0000 7f846fd87e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55849e0811c0 con 0x55849e062440 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.692+0000 7f846fd87e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55849e081330 con 0x55849e061b30 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.692+0000 7f846fd87e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55849e085a80 con 0x55849e0a70e0 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f8466ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e062440 0x55849e080bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f84677fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e061b30 0x55849e080670 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f84677fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e061b30 0x55849e080670 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:47666/0 (socket says 10.20.193.28:47666) 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f84677fe640 1 -- 10.20.193.28:0/2839422189 learned_addr learned my addr 10.20.193.28:0/2839422189 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f8467fff640 1 --2- 10.20.193.28:0/2839422189 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e0a70e0 0x55849e085510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f8466ffd640 1 -- 10.20.193.28:0/2839422189 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e0a70e0 msgr2=0x55849e085510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f8466ffd640 1 --2- 10.20.193.28:0/2839422189 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e0a70e0 0x55849e085510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f8466ffd640 1 -- 10.20.193.28:0/2839422189 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e061b30 msgr2=0x55849e080670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:40.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f8466ffd640 1 --2- 10.20.193.28:0/2839422189 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e061b30 0x55849e080670 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f8466ffd640 1 -- 10.20.193.28:0/2839422189 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55849def16f0 con 0x55849e062440 2026-02-25T11:53:40.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f8466ffd640 1 --2- 10.20.193.28:0/2839422189 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e062440 0x55849e080bb0 secure :-1 s=READY pgs=649 gs=1 cs=0 l=1 c_cookie=4518753056578208262 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f845802f6a0 tx=0x7f845802fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb5d5f6ad0e0206 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:40.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f84297fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:40.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f84297fa640 1 -- 10.20.193.28:0/2839422189 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f84580388c0 con 0x55849e062440 2026-02-25T11:53:40.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f84297fa640 1 -- 10.20.193.28:0/2839422189 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f845804f440 con 0x55849e062440 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f84297fa640 1 -- 10.20.193.28:0/2839422189 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8458059690 con 0x55849e062440 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.693+0000 7f846fd87e00 1 -- 10.20.193.28:0/2839422189 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55849df030e0 con 0x55849e062440 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f84297fa640 1 -- 10.20.193.28:0/2839422189 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8458059820 con 0x55849e062440 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f84297fa640 10 client.16587 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f846fd87e00 10 client.16587 _ll_register_callbacks cb 0x55849deeea10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f846fd87e00 1 -- 10.20.193.28:0/2839422189 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f845803e040 con 0x55849e062440 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f84297fa640 1 -- 10.20.193.28:0/2839422189 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f845804f920 con 0x55849e062440 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f846fd87e00 10 client.16587 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f846fd87e00 1 -- 10.20.193.28:0/2839422189 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55849df03340 con 0x55849e062440 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f84297fa640 1 -- 10.20.193.28:0/2839422189 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f845804f5b0 con 0x55849e062440 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f846fd87e00 10 client.16587 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f846fd87e00 10 client.16587 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f846fd87e00 20 client.16587 populate_metadata read hostname 'trial028' 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f846fd87e00 1 --2- 10.20.193.28:0/2839422189 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55849e1547a0 0x55849e174c50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f846fd87e00 4 client.16587 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.694+0000 7f846fd87e00 1 -- 10.20.193.28:0/2839422189 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55849dc00f40 con 0x55849e1547a0 2026-02-25T11:53:40.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.695+0000 7f84677fe640 1 --2- 10.20.193.28:0/2839422189 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55849e1547a0 0x55849e174c50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.695+0000 7f84677fe640 1 --2- 10.20.193.28:0/2839422189 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55849e1547a0 0x55849e174c50 crc :-1 s=READY pgs=44 gs=4 cs=0 l=0 c_cookie=3664963104417468889 s_cookie=12376290527748837085 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32dc908691b751d9 server_cookie=abc16aa99f804edd in_seq=0 out_seq=0 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.695+0000 7f84297fa640 10 client.16587 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.695+0000 7f84297fa640 1 -- 10.20.193.28:0/2839422189 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55849dc00f40 con 0x55849e1547a0 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.695+0000 7f84297fa640 10 client.16587 handle_command_reply: tid=0 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.695+0000 7f846fd87e00 1 client.16587 shutdown 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.695+0000 7f846fd87e00 1 -- 10.20.193.28:0/2839422189 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e062440 msgr2=0x55849e080bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.695+0000 7f846fd87e00 1 --2- 10.20.193.28:0/2839422189 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e062440 0x55849e080bb0 secure :-1 s=READY pgs=649 gs=1 cs=0 l=1 c_cookie=4518753056578208262 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f845802f6a0 tx=0x7f845802fc00 comp rx=0 tx=0).stop 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.695+0000 7f846fd87e00 1 -- 10.20.193.28:0/2839422189 shutdown_connections 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.695+0000 7f846fd87e00 1 --2- 10.20.193.28:0/2839422189 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55849e1547a0 0x55849e174c50 crc :-1 s=READY pgs=44 gs=4 cs=0 l=0 c_cookie=3664963104417468889 s_cookie=12376290527748837085 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846fd87e00 1 --2- 10.20.193.28:0/2839422189 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55849e0a70e0 0x55849e085510 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846fd87e00 1 --2- 10.20.193.28:0/2839422189 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55849e062440 0x55849e080bb0 unknown :-1 s=CLOSED pgs=649 gs=1 cs=0 l=1 c_cookie=4518753056578208262 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846fd87e00 1 --2- 10.20.193.28:0/2839422189 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55849e061b30 0x55849e080670 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846fd87e00 1 -- 10.20.193.28:0/2839422189 >> 10.20.193.28:0/2839422189 conn(0x55849df807a0 msgr2=0x55849e05fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f84297fa640 0 client.16587 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846fd87e00 1 -- 10.20.193.28:0/2839422189 shutdown_connections 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846fd87e00 1 -- 10.20.193.28:0/2839422189 wait complete. 2026-02-25T11:53:40.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846fd87e00 20 client.16587 trim_cache size 0 max 16384 2026-02-25T11:53:40.699 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:53:40.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846cb03640 1 -- 10.20.193.28:0/1209058298 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f843c0b5760 msgr2=0x7f843c0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846cb03640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f843c0b5760 0x7f843c0d5c10 secure :-1 s=READY pgs=326 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8468124a40 tx=0x7f845803a040 comp rx=0 tx=0).stop 2026-02-25T11:53:40.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846cb03640 1 -- 10.20.193.28:0/1209058298 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f846810f690 msgr2=0x7f846811e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:40.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846cb03640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f846810f690 0x7f846811e0a0 secure :-1 s=READY pgs=175 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8454009480 tx=0x7f845400ee60 comp rx=0 tx=0).stop 2026-02-25T11:53:40.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846cb03640 1 -- 10.20.193.28:0/1209058298 shutdown_connections 2026-02-25T11:53:40.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846cb03640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f843c0b5760 0x7f843c0d5c10 unknown :-1 s=CLOSED pgs=326 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846cb03640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8468115880 0x7f84681239b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846cb03640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8468110080 0x7f846811e5e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846cb03640 1 --2- 10.20.193.28:0/1209058298 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f846810f690 0x7f846811e0a0 unknown :-1 s=CLOSED pgs=175 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:40.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846cb03640 1 -- 10.20.193.28:0/1209058298 >> 10.20.193.28:0/1209058298 conn(0x7f84680ecf00 msgr2=0x7f846810d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:40.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846cb03640 1 -- 10.20.193.28:0/1209058298 shutdown_connections 2026-02-25T11:53:40.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:40.696+0000 7f846cb03640 1 -- 10.20.193.28:0/1209058298 wait complete. 2026-02-25T11:53:40.798 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:53:40.983 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:41.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.149+0000 7f80d971b640 1 -- 10.20.193.28:0/4049671768 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f80d410e190 msgr2=0x7f80d410e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.149+0000 7f80d971b640 1 --2- 10.20.193.28:0/4049671768 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f80d410e190 0x7f80d410e560 secure :-1 s=READY pgs=650 gs=1 cs=0 l=1 c_cookie=14493505232964965118 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f80c0009950 tx=0x7f80c002f190 comp rx=0 tx=0).stop 2026-02-25T11:53:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.149+0000 7f80d971b640 1 -- 10.20.193.28:0/4049671768 shutdown_connections 2026-02-25T11:53:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.149+0000 7f80d971b640 1 --2- 10.20.193.28:0/4049671768 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f80d4153740 0x7f80d4173b60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.149+0000 7f80d971b640 1 --2- 10.20.193.28:0/4049671768 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f80d410eb80 0x7f80d4153200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.149+0000 7f80d971b640 1 --2- 10.20.193.28:0/4049671768 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f80d410e190 0x7f80d410e560 unknown :-1 s=CLOSED pgs=650 gs=1 cs=0 l=1 c_cookie=14493505232964965118 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.149+0000 7f80d971b640 1 -- 10.20.193.28:0/4049671768 >> 10.20.193.28:0/4049671768 conn(0x7f80d40e9b40 msgr2=0x7f80d4109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.149+0000 7f80d971b640 1 -- 10.20.193.28:0/4049671768 shutdown_connections 2026-02-25T11:53:41.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.149+0000 7f80d971b640 1 -- 10.20.193.28:0/4049671768 wait complete. 2026-02-25T11:53:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.150+0000 7f80d971b640 1 Processor -- start 2026-02-25T11:53:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.150+0000 7f80d971b640 1 -- start start 2026-02-25T11:53:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.150+0000 7f80d971b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f80d410e190 0x7f80d4162170 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.150+0000 7f80d971b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f80d410eb80 0x7f80d41626b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.150+0000 7f80d971b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f80d4153740 0x7f80d4167a80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.150+0000 7f80d971b640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41680a0 con 0x7f80d410e190 2026-02-25T11:53:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.150+0000 7f80d971b640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f80d4168210 con 0x7f80d410eb80 2026-02-25T11:53:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.150+0000 7f80d971b640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80d41683d0 con 0x7f80d4153740 2026-02-25T11:53:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.150+0000 7f80d3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f80d410e190 0x7f80d4162170 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.150+0000 7f80d3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f80d410e190 0x7f80d4162170 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37222/0 (socket says 10.20.193.28:37222) 2026-02-25T11:53:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.150+0000 7f80d3fff640 1 -- 10.20.193.28:0/3815979422 learned_addr learned my addr 10.20.193.28:0/3815979422 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.150+0000 7f80d37fe640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f80d410eb80 0x7f80d41626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.150+0000 7f80d8f1a640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f80d4153740 0x7f80d4167a80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.151+0000 7f80d3fff640 1 -- 10.20.193.28:0/3815979422 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f80d4153740 msgr2=0x7f80d4167a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.151+0000 7f80d3fff640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f80d4153740 0x7f80d4167a80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.151+0000 7f80d3fff640 1 -- 10.20.193.28:0/3815979422 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f80d410eb80 msgr2=0x7f80d41626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.151+0000 7f80d3fff640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f80d410eb80 0x7f80d41626b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.151+0000 7f80d3fff640 1 -- 10.20.193.28:0/3815979422 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c0009570 con 0x7f80d410e190 2026-02-25T11:53:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.151+0000 7f80d3fff640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f80d410e190 0x7f80d4162170 secure :-1 s=READY pgs=651 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f80c00098a0 tx=0x7f80c00028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.151+0000 7f80d17fa640 1 -- 10.20.193.28:0/3815979422 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f80c003d070 con 0x7f80d410e190 2026-02-25T11:53:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.151+0000 7f80d17fa640 1 -- 10.20.193.28:0/3815979422 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f80c0002a40 con 0x7f80d410e190 2026-02-25T11:53:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.151+0000 7f80d971b640 1 -- 10.20.193.28:0/3815979422 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d4168650 con 0x7f80d410e190 2026-02-25T11:53:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.151+0000 7f80d17fa640 1 -- 10.20.193.28:0/3815979422 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f80c0041820 con 0x7f80d410e190 2026-02-25T11:53:41.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.151+0000 7f80d971b640 1 -- 10.20.193.28:0/3815979422 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d414f360 con 0x7f80d410e190 2026-02-25T11:53:41.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.154+0000 7f80d17fa640 1 -- 10.20.193.28:0/3815979422 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f80c00419c0 con 0x7f80d410e190 2026-02-25T11:53:41.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.156+0000 7f80d17fa640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80a80b5760 0x7f80a80d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.156+0000 7f80dc99fe00 1 Processor -- start 2026-02-25T11:53:41.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.156+0000 7f80dc99fe00 1 -- start start 2026-02-25T11:53:41.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.156+0000 7f80d17fa640 1 -- 10.20.193.28:0/3815979422 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f80c010b170 con 0x7f80d410e190 2026-02-25T11:53:41.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.156+0000 7f80dc99fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c55b30 0x55a195c55f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80dc99fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c9aba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80dc99fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c9b0e0 0x55a195cbb4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80dc99fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a195b6b8a0 con 0x55a195c55b30 2026-02-25T11:53:41.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80dc99fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a195b675d0 con 0x55a195c56440 2026-02-25T11:53:41.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80dc99fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a195b61570 con 0x55a195c9b0e0 2026-02-25T11:53:41.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80d37fe640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80a80b5760 0x7f80a80d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80d3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c55b30 0x55a195c55f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80d3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c55b30 0x55a195c55f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37232/0 (socket says 10.20.193.28:37232) 2026-02-25T11:53:41.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80d3fff640 1 -- 10.20.193.28:0/2966872307 learned_addr learned my addr 10.20.193.28:0/2966872307 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:41.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80d8f1a640 1 --2- 10.20.193.28:0/2966872307 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c9b0e0 0x55a195cbb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80d37fe640 1 --2- 10.20.193.28:0/2966872307 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c9aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80d8f1a640 1 -- 10.20.193.28:0/2966872307 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 msgr2=0x55a195c9aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80d8f1a640 1 --2- 10.20.193.28:0/2966872307 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c9aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80d8f1a640 1 -- 10.20.193.28:0/2966872307 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c55b30 msgr2=0x55a195c55f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80d8f1a640 1 --2- 10.20.193.28:0/2966872307 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c55b30 0x55a195c55f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80d8f1a640 1 -- 10.20.193.28:0/2966872307 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a195b0cd10 con 0x55a195c9b0e0 2026-02-25T11:53:41.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.157+0000 7f80d37fe640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80a80b5760 0x7f80a80d5c10 secure :-1 s=READY pgs=327 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f80d4168b10 tx=0x7f80c40042c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.158+0000 7f80d8f1a640 1 --2- 10.20.193.28:0/2966872307 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c9b0e0 0x55a195cbb4c0 secure :-1 s=READY pgs=176 gs=2 cs=0 l=1 c_cookie=11556557282759219460 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f80c8005770 tx=0x7f80c8003dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a061239e91511904 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.158+0000 7f80a4ce3640 1 -- 10.20.193.28:0/2966872307 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f80c8010c20 con 0x55a195c9b0e0 2026-02-25T11:53:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.158+0000 7f80a4ce3640 1 -- 10.20.193.28:0/2966872307 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f80c8010d80 con 0x55a195c9b0e0 2026-02-25T11:53:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.158+0000 7f80a4ce3640 1 -- 10.20.193.28:0/2966872307 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f80c8011770 con 0x55a195c9b0e0 2026-02-25T11:53:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.158+0000 7f80dc99fe00 1 -- 10.20.193.28:0/2966872307 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c9b0e0 msgr2=0x55a195cbb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.158+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/2966872307 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c9b0e0 0x55a195cbb4c0 secure :-1 s=READY pgs=176 gs=2 cs=0 l=1 c_cookie=11556557282759219460 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f80c8005770 tx=0x7f80c8003dd0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.158+0000 7f80dc99fe00 1 -- 10.20.193.28:0/2966872307 shutdown_connections 2026-02-25T11:53:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.158+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/2966872307 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c9b0e0 0x55a195cbb4c0 unknown :-1 s=CLOSED pgs=176 gs=2 cs=0 l=1 c_cookie=11556557282759219460 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.158+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/2966872307 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c9aba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.158+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/2966872307 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c55b30 0x55a195c55f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.158+0000 7f80dc99fe00 1 -- 10.20.193.28:0/2966872307 >> 10.20.193.28:0/2966872307 conn(0x55a195b747a0 msgr2=0x55a195c54410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.158+0000 7f80dc99fe00 1 -- 10.20.193.28:0/2966872307 shutdown_connections 2026-02-25T11:53:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.158+0000 7f80dc99fe00 1 -- 10.20.193.28:0/2966872307 wait complete. 2026-02-25T11:53:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.159+0000 7f80dc99fe00 1 -- start start 2026-02-25T11:53:41.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80dc99fe00 1 Processor -- start 2026-02-25T11:53:41.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80dc99fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 0x55a195c6df50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80dc99fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c6e490 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80dc99fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c9b0e0 0x55a195c73480 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80dc99fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a195c739c0 con 0x55a195c9b0e0 2026-02-25T11:53:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80dc99fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a195c73b30 con 0x55a195c56440 2026-02-25T11:53:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80dc99fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a195c73cf0 con 0x55a195c55b30 2026-02-25T11:53:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80d3fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 0x55a195c6df50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80d3fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 0x55a195c6df50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34934/0 (socket says 10.20.193.28:34934) 2026-02-25T11:53:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80d3fff640 1 -- 10.20.193.28:0/376755753 learned_addr learned my addr 10.20.193.28:0/376755753 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80d37fe640 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c6e490 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80d8f1a640 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c9b0e0 0x55a195c73480 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80d3fff640 1 -- 10.20.193.28:0/376755753 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 msgr2=0x55a195c6e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80d3fff640 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c6e490 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80d3fff640 1 -- 10.20.193.28:0/376755753 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c9b0e0 msgr2=0x55a195c73480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.160+0000 7f80d3fff640 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c9b0e0 0x55a195c73480 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.161+0000 7f80d3fff640 1 -- 10.20.193.28:0/376755753 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a195ae53c0 con 0x55a195c55b30 2026-02-25T11:53:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.161+0000 7f80d8f1a640 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c9b0e0 0x55a195c73480 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:53:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.161+0000 7f80d3fff640 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 0x55a195c6df50 secure :-1 s=READY pgs=177 gs=1 cs=0 l=1 c_cookie=13098027616858360441 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f80c0002bb0 tx=0x7f80c002f190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5c58ab8a1194a79 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.161+0000 7f8095ffb640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:53:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.161+0000 7f8095ffb640 1 -- 10.20.193.28:0/376755753 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f80c00d1ef0 con 0x55a195c55b30 2026-02-25T11:53:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.161+0000 7f8095ffb640 1 -- 10.20.193.28:0/376755753 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f80c0048e30 con 0x55a195c55b30 2026-02-25T11:53:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.161+0000 7f8095ffb640 1 -- 10.20.193.28:0/376755753 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f80c00cf750 con 0x55a195c55b30 2026-02-25T11:53:41.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.161+0000 7f80dc99fe00 1 -- 10.20.193.28:0/376755753 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c8005460 con 0x55a195c55b30 2026-02-25T11:53:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.161+0000 7f8095ffb640 1 -- 10.20.193.28:0/376755753 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f80c00968f0 con 0x55a195c55b30 2026-02-25T11:53:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.161+0000 7f8095ffb640 10 client.26326 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:53:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.162+0000 7f80dc99fe00 10 client.26326 _ll_register_callbacks cb 0x55a195ae2a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.162+0000 7f80dc99fe00 1 -- 10.20.193.28:0/376755753 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a1957f76f0 con 0x55a195c55b30 2026-02-25T11:53:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.162+0000 7f8095ffb640 1 -- 10.20.193.28:0/376755753 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f80c00d7210 con 0x55a195c55b30 2026-02-25T11:53:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.162+0000 7f80dc99fe00 10 client.26326 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.162+0000 7f80dc99fe00 1 -- 10.20.193.28:0/376755753 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55a195af70e0 con 0x55a195c55b30 2026-02-25T11:53:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.162+0000 7f8095ffb640 1 -- 10.20.193.28:0/376755753 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f80c00961b0 con 0x55a195c55b30 2026-02-25T11:53:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.162+0000 7f80dc99fe00 10 client.26326 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.162+0000 7f80dc99fe00 10 client.26326 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:53:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.162+0000 7f80dc99fe00 20 client.26326 populate_metadata read hostname 'trial028' 2026-02-25T11:53:41.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.162+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a195c98430 0x55a195c988d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.162+0000 7f80dc99fe00 4 client.26326 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:53:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.162+0000 7f80dc99fe00 1 -- 10.20.193.28:0/376755753 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a195b33fc0 con 0x55a195c98430 2026-02-25T11:53:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.162+0000 7f80d37fe640 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a195c98430 0x55a195c988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.163+0000 7f80d37fe640 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a195c98430 0x55a195c988d0 crc :-1 s=READY pgs=44 gs=4 cs=0 l=0 c_cookie=13144840564768593838 s_cookie=12919262218723126013 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b66bdad96c2d57ae server_cookie=b34a7089eb1deafd in_seq=0 out_seq=0 2026-02-25T11:53:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.163+0000 7f8095ffb640 10 client.26326 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:53:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.163+0000 7f8095ffb640 1 -- 10.20.193.28:0/376755753 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55a195b33fc0 con 0x55a195c98430 2026-02-25T11:53:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.163+0000 7f8095ffb640 10 client.26326 handle_command_reply: tid=0 2026-02-25T11:53:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.164+0000 7f80dc99fe00 1 client.26326 shutdown 2026-02-25T11:53:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.164+0000 7f80dc99fe00 1 -- 10.20.193.28:0/376755753 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 msgr2=0x55a195c6df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.164+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 0x55a195c6df50 secure :-1 s=READY pgs=177 gs=1 cs=0 l=1 c_cookie=13098027616858360441 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f80c0002bb0 tx=0x7f80c002f190 comp rx=0 tx=0).stop 2026-02-25T11:53:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.164+0000 7f80dc99fe00 1 -- 10.20.193.28:0/376755753 shutdown_connections 2026-02-25T11:53:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.164+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a195c98430 0x55a195c988d0 crc :-1 s=READY pgs=44 gs=4 cs=0 l=0 c_cookie=13144840564768593838 s_cookie=12919262218723126013 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.164+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c9b0e0 0x55a195c73480 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.164+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c6e490 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.164+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/376755753 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 0x55a195c6df50 unknown :-1 s=CLOSED pgs=177 gs=1 cs=0 l=1 c_cookie=13098027616858360441 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.164+0000 7f80dc99fe00 1 -- 10.20.193.28:0/376755753 >> 10.20.193.28:0/376755753 conn(0x55a195b747a0 msgr2=0x55a195c54b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.164+0000 7f8095ffb640 0 client.26326 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:53:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.164+0000 7f80dc99fe00 1 -- 10.20.193.28:0/376755753 shutdown_connections 2026-02-25T11:53:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.164+0000 7f80dc99fe00 1 -- 10.20.193.28:0/376755753 wait complete. 2026-02-25T11:53:41.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.164+0000 7f80dc99fe00 20 client.26326 trim_cache size 0 max 16384 2026-02-25T11:53:41.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.171+0000 7f80dc99fe00 1 Processor -- start 2026-02-25T11:53:41.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.171+0000 7f80dc99fe00 1 -- start start 2026-02-25T11:53:41.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80dc99fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c55b30 0x55a195c9a1c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80dc99fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c9a700 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80dc99fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c9b0e0 0x55a195c53d10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80dc99fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a195b5f1a0 con 0x55a195c55b30 2026-02-25T11:53:41.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80dc99fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a19581f700 con 0x55a195c56440 2026-02-25T11:53:41.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80dc99fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a195c5b8a0 con 0x55a195c9b0e0 2026-02-25T11:53:41.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80d37fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c9a700 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80d37fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c9a700 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:47718/0 (socket says 10.20.193.28:47718) 2026-02-25T11:53:41.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80d37fe640 1 -- 10.20.193.28:0/1338283287 learned_addr learned my addr 10.20.193.28:0/1338283287 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:41.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80d8f1a640 1 --2- 10.20.193.28:0/1338283287 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c9b0e0 0x55a195c53d10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80d3fff640 1 --2- 10.20.193.28:0/1338283287 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c55b30 0x55a195c9a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80d37fe640 1 -- 10.20.193.28:0/1338283287 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c9b0e0 msgr2=0x55a195c53d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80d37fe640 1 --2- 10.20.193.28:0/1338283287 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c9b0e0 0x55a195c53d10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.172+0000 7f80d37fe640 1 -- 10.20.193.28:0/1338283287 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c55b30 msgr2=0x55a195c9a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.173+0000 7f80d37fe640 1 --2- 10.20.193.28:0/1338283287 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c55b30 0x55a195c9a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.173+0000 7f80d37fe640 1 -- 10.20.193.28:0/1338283287 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a195c60870 con 0x55a195c56440 2026-02-25T11:53:41.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.173+0000 7f80d8f1a640 1 --2- 10.20.193.28:0/1338283287 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c9b0e0 0x55a195c53d10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:53:41.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.173+0000 7f80d3fff640 1 --2- 10.20.193.28:0/1338283287 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c55b30 0x55a195c9a1c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:53:41.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.173+0000 7f80d37fe640 1 --2- 10.20.193.28:0/1338283287 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c9a700 secure :-1 s=READY pgs=147 gs=1 cs=0 l=1 c_cookie=6901195528965988552 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f80c4004740 tx=0x7f80c4015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fc5f8c4fd8820c8 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:41.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.173+0000 7f8097fff640 1 -- 10.20.193.28:0/1338283287 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f80c4022040 con 0x55a195c56440 2026-02-25T11:53:41.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.173+0000 7f8097fff640 1 -- 10.20.193.28:0/1338283287 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f80c401d430 con 0x55a195c56440 2026-02-25T11:53:41.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.173+0000 7f8097fff640 1 -- 10.20.193.28:0/1338283287 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f80c4028320 con 0x55a195c56440 2026-02-25T11:53:41.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.173+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1338283287 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 msgr2=0x55a195c9a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.173+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/1338283287 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c9a700 secure :-1 s=READY pgs=147 gs=1 cs=0 l=1 c_cookie=6901195528965988552 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f80c4004740 tx=0x7f80c4015000 comp rx=0 tx=0).stop 2026-02-25T11:53:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.174+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1338283287 shutdown_connections 2026-02-25T11:53:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.174+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/1338283287 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c9b0e0 0x55a195c53d10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.174+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/1338283287 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195c9a700 unknown :-1 s=CLOSED pgs=147 gs=1 cs=0 l=1 c_cookie=6901195528965988552 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.174+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/1338283287 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c55b30 0x55a195c9a1c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.174+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1338283287 >> 10.20.193.28:0/1338283287 conn(0x55a195b747a0 msgr2=0x55a195c72ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.174+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1338283287 shutdown_connections 2026-02-25T11:53:41.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.174+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1338283287 wait complete. 2026-02-25T11:53:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.174+0000 7f80dc99fe00 1 -- start start 2026-02-25T11:53:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.174+0000 7f80dc99fe00 1 Processor -- start 2026-02-25T11:53:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.174+0000 7f80dc99fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 0x55a195cdf820 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80dc99fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195cdfd80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80dc99fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c9b0e0 0x55a195d00880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80dc99fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a195d00e10 con 0x55a195c9b0e0 2026-02-25T11:53:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80dc99fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a195d00f80 con 0x55a195c56440 2026-02-25T11:53:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80dc99fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a195d01140 con 0x55a195c55b30 2026-02-25T11:53:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80d3fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 0x55a195cdf820 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80d3fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 0x55a195cdf820 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34960/0 (socket says 10.20.193.28:34960) 2026-02-25T11:53:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80d3fff640 1 -- 10.20.193.28:0/1467565879 learned_addr learned my addr 10.20.193.28:0/1467565879 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:41.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80d37fe640 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195cdfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80d3fff640 1 -- 10.20.193.28:0/1467565879 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 msgr2=0x55a195cdfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80d3fff640 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195cdfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80d3fff640 1 -- 10.20.193.28:0/1467565879 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c9b0e0 msgr2=0x55a195d00880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80d8f1a640 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c9b0e0 0x55a195d00880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80d3fff640 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c9b0e0 0x55a195d00880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80d3fff640 1 -- 10.20.193.28:0/1467565879 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c4009c20 con 0x55a195c55b30 2026-02-25T11:53:41.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.175+0000 7f80d8f1a640 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c9b0e0 0x55a195d00880 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:53:41.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f80d3fff640 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 0x55a195cdf820 secure :-1 s=READY pgs=178 gs=1 cs=0 l=1 c_cookie=6219298281995334397 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f80c00d1760 tx=0x7f80c0005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=564f62d6128772fd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:41.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f8095ffb640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:53:41.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f8095ffb640 1 -- 10.20.193.28:0/1467565879 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f80c00ef020 con 0x55a195c55b30 2026-02-25T11:53:41.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f8095ffb640 1 -- 10.20.193.28:0/1467565879 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55a1957f76f0 con 0x55a195c55b30 2026-02-25T11:53:41.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1467565879 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c8005460 con 0x55a195c55b30 2026-02-25T11:53:41.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f80dc99fe00 10 client.26329 _ll_register_callbacks cb 0x55a195ae2a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:41.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1467565879 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80c8010d80 con 0x55a195c55b30 2026-02-25T11:53:41.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f8095ffb640 1 -- 10.20.193.28:0/1467565879 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f80c00961b0 con 0x55a195c55b30 2026-02-25T11:53:41.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f8095ffb640 1 -- 10.20.193.28:0/1467565879 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f80c0096310 con 0x55a195c55b30 2026-02-25T11:53:41.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f8095ffb640 1 -- 10.20.193.28:0/1467565879 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f80c0096680 con 0x55a195c55b30 2026-02-25T11:53:41.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f80dc99fe00 10 client.26329 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:41.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1467565879 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55a195af1b60 con 0x55a195c55b30 2026-02-25T11:53:41.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f8095ffb640 1 -- 10.20.193.28:0/1467565879 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f80c00cf4d0 con 0x55a195c55b30 2026-02-25T11:53:41.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f80dc99fe00 10 client.26329 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:41.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f80dc99fe00 10 client.26329 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:53:41.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f80dc99fe00 20 client.26329 populate_metadata read hostname 'trial028' 2026-02-25T11:53:41.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a195d47cc0 0x55a195d68170 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f80dc99fe00 4 client.26329 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:53:41.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1467565879 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1957f4f40 con 0x55a195d47cc0 2026-02-25T11:53:41.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.176+0000 7f80d37fe640 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a195d47cc0 0x55a195d68170 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f80d37fe640 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a195d47cc0 0x55a195d68170 crc :-1 s=READY pgs=45 gs=4 cs=0 l=0 c_cookie=10699415041588636808 s_cookie=2338214913158084371 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=947bf54ada5cac88 server_cookie=207302133b4ef713 in_seq=0 out_seq=0 2026-02-25T11:53:41.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f8095ffb640 10 client.26329 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:53:41.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f8095ffb640 1 -- 10.20.193.28:0/1467565879 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1957f4f40 con 0x55a195d47cc0 2026-02-25T11:53:41.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f8095ffb640 10 client.26329 handle_command_reply: tid=0 2026-02-25T11:53:41.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f80dc99fe00 1 client.26329 shutdown 2026-02-25T11:53:41.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1467565879 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 msgr2=0x55a195cdf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 0x55a195cdf820 secure :-1 s=READY pgs=178 gs=1 cs=0 l=1 c_cookie=6219298281995334397 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f80c00d1760 tx=0x7f80c0005f00 comp rx=0 tx=0).stop 2026-02-25T11:53:41.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1467565879 shutdown_connections 2026-02-25T11:53:41.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a195d47cc0 0x55a195d68170 crc :-1 s=READY pgs=45 gs=4 cs=0 l=0 c_cookie=10699415041588636808 s_cookie=2338214913158084371 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a195c9b0e0 0x55a195d00880 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a195c56440 0x55a195cdfd80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f80dc99fe00 1 --2- 10.20.193.28:0/1467565879 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a195c55b30 0x55a195cdf820 unknown :-1 s=CLOSED pgs=178 gs=1 cs=0 l=1 c_cookie=6219298281995334397 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1467565879 >> 10.20.193.28:0/1467565879 conn(0x55a195b747a0 msgr2=0x55a195c54c10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:41.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f8095ffb640 0 client.26329 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:53:41.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1467565879 shutdown_connections 2026-02-25T11:53:41.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f80dc99fe00 1 -- 10.20.193.28:0/1467565879 wait complete. 2026-02-25T11:53:41.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.177+0000 7f80dc99fe00 20 client.26329 trim_cache size 0 max 16384 2026-02-25T11:53:41.187 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:53:41.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.178+0000 7f80d971b640 1 -- 10.20.193.28:0/3815979422 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80a80b5760 msgr2=0x7f80a80d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.188 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.178+0000 7f80d971b640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80a80b5760 0x7f80a80d5c10 secure :-1 s=READY pgs=327 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f80d4168b10 tx=0x7f80c40042c0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.188 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.178+0000 7f80d971b640 1 -- 10.20.193.28:0/3815979422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f80d410e190 msgr2=0x7f80d4162170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.188 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.178+0000 7f80d971b640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f80d410e190 0x7f80d4162170 secure :-1 s=READY pgs=651 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f80c00098a0 tx=0x7f80c00028d0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.188 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.178+0000 7f80d971b640 1 -- 10.20.193.28:0/3815979422 shutdown_connections 2026-02-25T11:53:41.188 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.178+0000 7f80d971b640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f80a80b5760 0x7f80a80d5c10 unknown :-1 s=CLOSED pgs=327 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.188 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.178+0000 7f80d971b640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f80d4153740 0x7f80d4167a80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.188 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.178+0000 7f80d971b640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f80d410eb80 0x7f80d41626b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.188 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.178+0000 7f80d971b640 1 --2- 10.20.193.28:0/3815979422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f80d410e190 0x7f80d4162170 unknown :-1 s=CLOSED pgs=651 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.188 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.178+0000 7f80d971b640 1 -- 10.20.193.28:0/3815979422 >> 10.20.193.28:0/3815979422 conn(0x7f80d40e9b40 msgr2=0x7f80d4109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:41.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.178+0000 7f80d971b640 1 -- 10.20.193.28:0/3815979422 shutdown_connections 2026-02-25T11:53:41.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.178+0000 7f80d971b640 1 -- 10.20.193.28:0/3815979422 wait complete. 2026-02-25T11:53:41.278 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:53:41.460 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:41.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.629+0000 7f1951cba640 1 -- 10.20.193.28:0/704219063 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f194c1115d0 msgr2=0x7f194c111a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.629+0000 7f1951cba640 1 --2- 10.20.193.28:0/704219063 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f194c1115d0 0x7f194c111a70 secure :-1 s=READY pgs=652 gs=1 cs=0 l=1 c_cookie=11210181621757502760 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f194000aa60 tx=0x7f1940030270 comp rx=0 tx=0).stop 2026-02-25T11:53:41.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.630+0000 7f1951cba640 1 -- 10.20.193.28:0/704219063 shutdown_connections 2026-02-25T11:53:41.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.630+0000 7f1951cba640 1 --2- 10.20.193.28:0/704219063 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f194c1115d0 0x7f194c111a70 unknown :-1 s=CLOSED pgs=652 gs=1 cs=0 l=1 c_cookie=11210181621757502760 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.630+0000 7f1951cba640 1 --2- 10.20.193.28:0/704219063 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f194c10c9e0 0x7f194c110fb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.630+0000 7f1951cba640 1 --2- 10.20.193.28:0/704219063 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f194c10bff0 0x7f194c10c3c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.630+0000 7f1951cba640 1 -- 10.20.193.28:0/704219063 >> 10.20.193.28:0/704219063 conn(0x7f194c0e9c40 msgr2=0x7f194c10a050 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:41.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.630+0000 7f1951cba640 1 -- 10.20.193.28:0/704219063 shutdown_connections 2026-02-25T11:53:41.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.630+0000 7f1951cba640 1 -- 10.20.193.28:0/704219063 wait complete. 2026-02-25T11:53:41.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.630+0000 7f1951cba640 1 Processor -- start 2026-02-25T11:53:41.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.630+0000 7f1951cba640 1 -- start start 2026-02-25T11:53:41.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f1951cba640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f194c10bff0 0x7f194c122a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f1951cba640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f194c10c9e0 0x7f194c122f50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f1951cba640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f194c1115d0 0x7f194c128320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f1951cba640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f194c128940 con 0x7f194c1115d0 2026-02-25T11:53:41.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f1951cba640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f194c128ab0 con 0x7f194c10c9e0 2026-02-25T11:53:41.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f1951cba640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f194c128c70 con 0x7f194c10bff0 2026-02-25T11:53:41.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f194bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f194c1115d0 0x7f194c128320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f194b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f194c10bff0 0x7f194c122a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f194b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f194c10bff0 0x7f194c122a10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34986/0 (socket says 10.20.193.28:34986) 2026-02-25T11:53:41.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f194b7fe640 1 -- 10.20.193.28:0/3184264614 learned_addr learned my addr 10.20.193.28:0/3184264614 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:41.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f194affd640 1 --2- 10.20.193.28:0/3184264614 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f194c10c9e0 0x7f194c122f50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f194b7fe640 1 -- 10.20.193.28:0/3184264614 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f194c10c9e0 msgr2=0x7f194c122f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f194b7fe640 1 --2- 10.20.193.28:0/3184264614 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f194c10c9e0 0x7f194c122f50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f194b7fe640 1 -- 10.20.193.28:0/3184264614 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f194c1115d0 msgr2=0x7f194c128320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f194b7fe640 1 --2- 10.20.193.28:0/3184264614 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f194c1115d0 0x7f194c128320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.631+0000 7f194b7fe640 1 -- 10.20.193.28:0/3184264614 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194000a650 con 0x7f194c10bff0 2026-02-25T11:53:41.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.632+0000 7f194b7fe640 1 --2- 10.20.193.28:0/3184264614 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f194c10bff0 0x7f194c122a10 secure :-1 s=READY pgs=179 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1938009480 tx=0x7f193800ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:41.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.632+0000 7f1948ff9640 1 -- 10.20.193.28:0/3184264614 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f193800cc80 con 0x7f194c10bff0 2026-02-25T11:53:41.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.632+0000 7f1948ff9640 1 -- 10.20.193.28:0/3184264614 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f193800cde0 con 0x7f194c10bff0 2026-02-25T11:53:41.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.632+0000 7f1948ff9640 1 -- 10.20.193.28:0/3184264614 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f19380106b0 con 0x7f194c10bff0 2026-02-25T11:53:41.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.632+0000 7f1951cba640 1 -- 10.20.193.28:0/3184264614 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f194c128f50 con 0x7f194c10bff0 2026-02-25T11:53:41.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.632+0000 7f1951cba640 1 -- 10.20.193.28:0/3184264614 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f194c169650 con 0x7f194c10bff0 2026-02-25T11:53:41.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.635+0000 7f1948ff9640 1 -- 10.20.193.28:0/3184264614 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f193801f060 con 0x7f194c10bff0 2026-02-25T11:53:41.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.637+0000 7f1948ff9640 1 --2- 10.20.193.28:0/3184264614 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f19200b5760 0x7f19200d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.637+0000 7f1948ff9640 1 -- 10.20.193.28:0/3184264614 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1938014070 con 0x7f194c10bff0 2026-02-25T11:53:41.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.637+0000 7f194affd640 1 --2- 10.20.193.28:0/3184264614 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f19200b5760 0x7f19200d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.637+0000 7f194affd640 1 --2- 10.20.193.28:0/3184264614 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f19200b5760 0x7f19200d5c10 secure :-1 s=READY pgs=328 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f194c1293b0 tx=0x7f193c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:41.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.638+0000 7f1953cb5e00 1 Processor -- start 2026-02-25T11:53:41.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.638+0000 7f1953cb5e00 1 -- start start 2026-02-25T11:53:41.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.638+0000 7f1953cb5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 0x560fa19c2f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.638+0000 7f1953cb5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 0x560fa1a07ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.638+0000 7f1953cb5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa1a080e0 0x560fa1a284c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.638+0000 7f1953cb5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x560fa18d88a0 con 0x560fa1a080e0 2026-02-25T11:53:41.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.638+0000 7f1953cb5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560fa18d45d0 con 0x560fa19c3440 2026-02-25T11:53:41.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.638+0000 7f1953cb5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560fa18ce570 con 0x560fa19c2b30 2026-02-25T11:53:41.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.639+0000 7f194b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 0x560fa19c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.639+0000 7f194b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 0x560fa19c2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:35000/0 (socket says 10.20.193.28:35000) 2026-02-25T11:53:41.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.639+0000 7f194b7fe640 1 -- 10.20.193.28:0/1029208201 learned_addr learned my addr 10.20.193.28:0/1029208201 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:41.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.639+0000 7f194bfff640 1 --2- 10.20.193.28:0/1029208201 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa1a080e0 0x560fa1a284c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.639+0000 7f194affd640 1 --2- 10.20.193.28:0/1029208201 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 0x560fa1a07ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.639+0000 7f194b7fe640 1 -- 10.20.193.28:0/1029208201 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 msgr2=0x560fa1a07ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.639+0000 7f194b7fe640 1 --2- 10.20.193.28:0/1029208201 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 0x560fa1a07ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.639+0000 7f194b7fe640 1 -- 10.20.193.28:0/1029208201 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa1a080e0 msgr2=0x560fa1a284c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.639+0000 7f194b7fe640 1 --2- 10.20.193.28:0/1029208201 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa1a080e0 0x560fa1a284c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.639+0000 7f194b7fe640 1 -- 10.20.193.28:0/1029208201 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa1879d10 con 0x560fa19c2b30 2026-02-25T11:53:41.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.639+0000 7f194bfff640 1 --2- 10.20.193.28:0/1029208201 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa1a080e0 0x560fa1a284c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:53:41.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.639+0000 7f194b7fe640 1 --2- 10.20.193.28:0/1029208201 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 0x560fa19c2f00 secure :-1 s=READY pgs=180 gs=1 cs=0 l=1 c_cookie=1933340914607978299 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f19380a9460 tx=0x7f19380ad010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ad49b518ceff73b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:41.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.640+0000 7f190ffff640 1 -- 10.20.193.28:0/1029208201 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f19380ad890 con 0x560fa19c2b30 2026-02-25T11:53:41.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.640+0000 7f190ffff640 1 -- 10.20.193.28:0/1029208201 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f19380b2bb0 con 0x560fa19c2b30 2026-02-25T11:53:41.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.640+0000 7f190ffff640 1 -- 10.20.193.28:0/1029208201 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f19380aca30 con 0x560fa19c2b30 2026-02-25T11:53:41.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.640+0000 7f1953cb5e00 1 -- 10.20.193.28:0/1029208201 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 msgr2=0x560fa19c2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.640+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/1029208201 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 0x560fa19c2f00 secure :-1 s=READY pgs=180 gs=1 cs=0 l=1 c_cookie=1933340914607978299 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f19380a9460 tx=0x7f19380ad010 comp rx=0 tx=0).stop 2026-02-25T11:53:41.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.640+0000 7f1953cb5e00 1 -- 10.20.193.28:0/1029208201 shutdown_connections 2026-02-25T11:53:41.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.640+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/1029208201 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa1a080e0 0x560fa1a284c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.640+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/1029208201 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 0x560fa1a07ba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.640+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/1029208201 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 0x560fa19c2f00 unknown :-1 s=CLOSED pgs=180 gs=1 cs=0 l=1 c_cookie=1933340914607978299 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.640+0000 7f1953cb5e00 1 -- 10.20.193.28:0/1029208201 >> 10.20.193.28:0/1029208201 conn(0x560fa18e17a0 msgr2=0x560fa19c1410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:41.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.640+0000 7f1953cb5e00 1 -- 10.20.193.28:0/1029208201 shutdown_connections 2026-02-25T11:53:41.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.640+0000 7f1953cb5e00 1 -- 10.20.193.28:0/1029208201 wait complete. 2026-02-25T11:53:41.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.641+0000 7f1953cb5e00 1 -- start start 2026-02-25T11:53:41.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.642+0000 7f1953cb5e00 1 Processor -- start 2026-02-25T11:53:41.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.642+0000 7f1953cb5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 0x560fa19dadb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.642+0000 7f1953cb5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 0x560fa19db2f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.642+0000 7f1953cb5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa1a080e0 0x560fa19e02e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.642+0000 7f1953cb5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x560fa19e0820 con 0x560fa1a080e0 2026-02-25T11:53:41.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.642+0000 7f1953cb5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560fa19e0990 con 0x560fa19c3440 2026-02-25T11:53:41.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.642+0000 7f1953cb5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560fa19e0b50 con 0x560fa19c2b30 2026-02-25T11:53:41.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.642+0000 7f194b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 0x560fa19dadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.642+0000 7f194bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa1a080e0 0x560fa19e02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.642+0000 7f194bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa1a080e0 0x560fa19e02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37308/0 (socket says 10.20.193.28:37308) 2026-02-25T11:53:41.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.642+0000 7f194bfff640 1 -- 10.20.193.28:0/3793148263 learned_addr learned my addr 10.20.193.28:0/3793148263 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:41.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.642+0000 7f194affd640 1 --2- 10.20.193.28:0/3793148263 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 0x560fa19db2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.642+0000 7f194b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 0x560fa19dadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:35016/0 (socket says 10.20.193.28:35016) 2026-02-25T11:53:41.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f194bfff640 1 -- 10.20.193.28:0/3793148263 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 msgr2=0x560fa19dadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f194bfff640 1 --2- 10.20.193.28:0/3793148263 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 0x560fa19dadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f194bfff640 1 -- 10.20.193.28:0/3793148263 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 msgr2=0x560fa19db2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f194bfff640 1 --2- 10.20.193.28:0/3793148263 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 0x560fa19db2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f194bfff640 1 -- 10.20.193.28:0/3793148263 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194c169650 con 0x560fa1a080e0 2026-02-25T11:53:41.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f194b7fe640 1 --2- 10.20.193.28:0/3793148263 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 0x560fa19dadb0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:53:41.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f194bfff640 1 --2- 10.20.193.28:0/3793148263 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa1a080e0 0x560fa19e02e0 secure :-1 s=READY pgs=653 gs=2 cs=0 l=1 c_cookie=1310809588926291381 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f194000a230 tx=0x7f19400396a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1230ee604c209db5 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:41.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f190d7fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:41.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f190d7fa640 1 -- 10.20.193.28:0/3793148263 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f194003e040 con 0x560fa1a080e0 2026-02-25T11:53:41.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f190d7fa640 1 -- 10.20.193.28:0/3793148263 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f194003f040 con 0x560fa1a080e0 2026-02-25T11:53:41.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f190d7fa640 1 -- 10.20.193.28:0/3793148263 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1940045450 con 0x560fa1a080e0 2026-02-25T11:53:41.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f1953cb5e00 1 -- 10.20.193.28:0/3793148263 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f194c17b140 con 0x560fa1a080e0 2026-02-25T11:53:41.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f190d7fa640 1 -- 10.20.193.28:0/3793148263 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f19400455e0 con 0x560fa1a080e0 2026-02-25T11:53:41.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.643+0000 7f190d7fa640 10 client.16614 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:53:41.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.644+0000 7f1953cb5e00 10 client.16614 _ll_register_callbacks cb 0x560fa184fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:41.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.644+0000 7f1953cb5e00 1 -- 10.20.193.28:0/3793148263 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560fa15646f0 con 0x560fa1a080e0 2026-02-25T11:53:41.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.644+0000 7f190d7fa640 1 -- 10.20.193.28:0/3793148263 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f19400050c0 con 0x560fa1a080e0 2026-02-25T11:53:41.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.644+0000 7f1953cb5e00 10 client.16614 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:41.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.644+0000 7f1953cb5e00 1 -- 10.20.193.28:0/3793148263 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x560fa18523c0 con 0x560fa1a080e0 2026-02-25T11:53:41.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.644+0000 7f190d7fa640 1 -- 10.20.193.28:0/3793148263 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f1940055020 con 0x560fa1a080e0 2026-02-25T11:53:41.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.644+0000 7f1953cb5e00 10 client.16614 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:41.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.644+0000 7f1953cb5e00 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:41.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.644+0000 7f1953cb5e00 20 client.16614 populate_metadata read hostname 'trial028' 2026-02-25T11:53:41.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.644+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/3793148263 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560fa1a055e0 0x560fa1a05a80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.644+0000 7f1953cb5e00 4 client.16614 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:53:41.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.644+0000 7f1953cb5e00 1 -- 10.20.193.28:0/3793148263 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fa18a0fc0 con 0x560fa1a055e0 2026-02-25T11:53:41.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.645+0000 7f194b7fe640 1 --2- 10.20.193.28:0/3793148263 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560fa1a055e0 0x560fa1a05a80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.645+0000 7f194b7fe640 1 --2- 10.20.193.28:0/3793148263 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560fa1a055e0 0x560fa1a05a80 crc :-1 s=READY pgs=108 gs=4 cs=0 l=0 c_cookie=16440704719622168072 s_cookie=17935139650534150292 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4291dd5fe2ade08 server_cookie=f8e66a7cdb1c9094 in_seq=0 out_seq=0 2026-02-25T11:53:41.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.645+0000 7f190d7fa640 10 client.16614 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:41.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f190d7fa640 1 -- 10.20.193.28:0/3793148263 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x560fa18a0fc0 con 0x560fa1a055e0 2026-02-25T11:53:41.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f190d7fa640 10 client.16614 handle_command_reply: tid=0 2026-02-25T11:53:41.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f1953cb5e00 1 client.16614 shutdown 2026-02-25T11:53:41.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f1953cb5e00 1 -- 10.20.193.28:0/3793148263 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa1a080e0 msgr2=0x560fa19e02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/3793148263 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa1a080e0 0x560fa19e02e0 secure :-1 s=READY pgs=653 gs=2 cs=0 l=1 c_cookie=1310809588926291381 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f194000a230 tx=0x7f19400396a0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f1953cb5e00 1 -- 10.20.193.28:0/3793148263 shutdown_connections 2026-02-25T11:53:41.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/3793148263 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560fa1a055e0 0x560fa1a05a80 crc :-1 s=READY pgs=108 gs=4 cs=0 l=0 c_cookie=16440704719622168072 s_cookie=17935139650534150292 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/3793148263 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa1a080e0 0x560fa19e02e0 unknown :-1 s=CLOSED pgs=653 gs=2 cs=0 l=1 c_cookie=1310809588926291381 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/3793148263 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 0x560fa19db2f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/3793148263 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c2b30 0x560fa19dadb0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f190d7fa640 0 client.16614 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:41.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f1953cb5e00 1 -- 10.20.193.28:0/3793148263 >> 10.20.193.28:0/3793148263 conn(0x560fa18e17a0 msgr2=0x560fa19c1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:41.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f1953cb5e00 1 -- 10.20.193.28:0/3793148263 shutdown_connections 2026-02-25T11:53:41.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f1953cb5e00 1 -- 10.20.193.28:0/3793148263 wait complete. 2026-02-25T11:53:41.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.646+0000 7f1953cb5e00 20 client.16614 trim_cache size 0 max 16384 2026-02-25T11:53:41.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.654+0000 7f1953cb5e00 1 Processor -- start 2026-02-25T11:53:41.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.654+0000 7f1953cb5e00 1 -- start start 2026-02-25T11:53:41.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.654+0000 7f1953cb5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 0x560fa19d83b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.654+0000 7f1953cb5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c3440 0x560fa19c1d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.654+0000 7f1953cb5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa1a080e0 0x560fa19c2260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.654+0000 7f1953cb5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x560fa18cc1a0 con 0x560fa19c2b30 2026-02-25T11:53:41.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.654+0000 7f1953cb5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560fa158c700 con 0x560fa1a080e0 2026-02-25T11:53:41.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.654+0000 7f1953cb5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560fa199de80 con 0x560fa19c3440 2026-02-25T11:53:41.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.654+0000 7f194b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 0x560fa19d83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.654+0000 7f194b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 0x560fa19d83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37316/0 (socket says 10.20.193.28:37316) 2026-02-25T11:53:41.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.654+0000 7f194b7fe640 1 -- 10.20.193.28:0/136506521 learned_addr learned my addr 10.20.193.28:0/136506521 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:41.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.654+0000 7f194affd640 1 --2- 10.20.193.28:0/136506521 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c3440 0x560fa19c1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f194bfff640 1 --2- 10.20.193.28:0/136506521 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa1a080e0 0x560fa19c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f194b7fe640 1 -- 10.20.193.28:0/136506521 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c3440 msgr2=0x560fa19c1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f194b7fe640 1 --2- 10.20.193.28:0/136506521 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c3440 0x560fa19c1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f194b7fe640 1 -- 10.20.193.28:0/136506521 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa1a080e0 msgr2=0x560fa19c2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f194b7fe640 1 --2- 10.20.193.28:0/136506521 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa1a080e0 0x560fa19c2260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f194b7fe640 1 -- 10.20.193.28:0/136506521 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa18640e0 con 0x560fa19c2b30 2026-02-25T11:53:41.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f194b7fe640 1 --2- 10.20.193.28:0/136506521 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 0x560fa19d83b0 secure :-1 s=READY pgs=654 gs=1 cs=0 l=1 c_cookie=6923645292542985156 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f193800c6a0 tx=0x7f1938009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6015bab5e4d483c4 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:41.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f190f7fe640 1 -- 10.20.193.28:0/136506521 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1938071040 con 0x560fa19c2b30 2026-02-25T11:53:41.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f190f7fe640 1 -- 10.20.193.28:0/136506521 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f19380ad890 con 0x560fa19c2b30 2026-02-25T11:53:41.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f190f7fe640 1 -- 10.20.193.28:0/136506521 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f19380acb50 con 0x560fa19c2b30 2026-02-25T11:53:41.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f1953cb5e00 1 -- 10.20.193.28:0/136506521 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 msgr2=0x560fa19d83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/136506521 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 0x560fa19d83b0 secure :-1 s=READY pgs=654 gs=1 cs=0 l=1 c_cookie=6923645292542985156 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f193800c6a0 tx=0x7f1938009f90 comp rx=0 tx=0).stop 2026-02-25T11:53:41.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f1953cb5e00 1 -- 10.20.193.28:0/136506521 shutdown_connections 2026-02-25T11:53:41.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/136506521 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa1a080e0 0x560fa19c2260 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/136506521 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa19c3440 0x560fa19c1d20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/136506521 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 0x560fa19d83b0 unknown :-1 s=CLOSED pgs=654 gs=1 cs=0 l=1 c_cookie=6923645292542985156 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.655+0000 7f1953cb5e00 1 -- 10.20.193.28:0/136506521 >> 10.20.193.28:0/136506521 conn(0x560fa18e17a0 msgr2=0x560fa19c5820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:41.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.656+0000 7f1953cb5e00 1 -- 10.20.193.28:0/136506521 shutdown_connections 2026-02-25T11:53:41.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.656+0000 7f1953cb5e00 1 -- 10.20.193.28:0/136506521 wait complete. 2026-02-25T11:53:41.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.656+0000 7f1953cb5e00 1 -- start start 2026-02-25T11:53:41.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.656+0000 7f1953cb5e00 1 Processor -- start 2026-02-25T11:53:41.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f1953cb5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 0x560fa19e6b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f1953cb5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 0x560fa19e7060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f1953cb5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa1a080e0 0x560fa19e2810 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f1953cb5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x560fa18cc1a0 con 0x560fa19c2b30 2026-02-25T11:53:41.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f1953cb5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560fa19e2f40 con 0x560fa19c3440 2026-02-25T11:53:41.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f1953cb5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560fa19e3100 con 0x560fa1a080e0 2026-02-25T11:53:41.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f194b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 0x560fa19e6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f194b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 0x560fa19e6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37320/0 (socket says 10.20.193.28:37320) 2026-02-25T11:53:41.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f194b7fe640 1 -- 10.20.193.28:0/8378332 learned_addr learned my addr 10.20.193.28:0/8378332 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:41.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f194bfff640 1 --2- 10.20.193.28:0/8378332 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa1a080e0 0x560fa19e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f194affd640 1 --2- 10.20.193.28:0/8378332 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 0x560fa19e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f194b7fe640 1 -- 10.20.193.28:0/8378332 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa1a080e0 msgr2=0x560fa19e2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f194b7fe640 1 --2- 10.20.193.28:0/8378332 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa1a080e0 0x560fa19e2810 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f194b7fe640 1 -- 10.20.193.28:0/8378332 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 msgr2=0x560fa19e7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:41.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f194b7fe640 1 --2- 10.20.193.28:0/8378332 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 0x560fa19e7060 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.657+0000 7f194b7fe640 1 -- 10.20.193.28:0/8378332 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa185eb60 con 0x560fa19c2b30 2026-02-25T11:53:41.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f194b7fe640 1 --2- 10.20.193.28:0/8378332 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 0x560fa19e6b20 secure :-1 s=READY pgs=655 gs=1 cs=0 l=1 c_cookie=12057391651857350866 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f19380a8ea0 tx=0x7f19380ad010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75475d316b7c0d2 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:41.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f190d7fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:41.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f190d7fa640 1 -- 10.20.193.28:0/8378332 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1938071070 con 0x560fa19c2b30 2026-02-25T11:53:41.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f190d7fa640 1 -- 10.20.193.28:0/8378332 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f193800b2c0 con 0x560fa19c2b30 2026-02-25T11:53:41.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f190d7fa640 1 -- 10.20.193.28:0/8378332 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f19380add80 con 0x560fa19c2b30 2026-02-25T11:53:41.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f1953cb5e00 1 -- 10.20.193.28:0/8378332 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa1879d10 con 0x560fa19c2b30 2026-02-25T11:53:41.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f1953cb5e00 10 client.16620 _ll_register_callbacks cb 0x560fa184fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:41.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f1953cb5e00 1 -- 10.20.193.28:0/8378332 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19380b2bb0 con 0x560fa19c2b30 2026-02-25T11:53:41.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f190d7fa640 1 -- 10.20.193.28:0/8378332 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f19380aae70 con 0x560fa19c2b30 2026-02-25T11:53:41.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f190d7fa640 1 -- 10.20.193.28:0/8378332 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f19380b7dd0 con 0x560fa19c2b30 2026-02-25T11:53:41.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f1953cb5e00 10 client.16620 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:41.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f1953cb5e00 1 -- 10.20.193.28:0/8378332 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x560fa18526f0 con 0x560fa19c2b30 2026-02-25T11:53:41.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f190d7fa640 1 -- 10.20.193.28:0/8378332 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f193808d020 con 0x560fa19c2b30 2026-02-25T11:53:41.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f1953cb5e00 10 client.16620 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:41.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f1953cb5e00 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:41.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f1953cb5e00 20 client.16620 populate_metadata read hostname 'trial028' 2026-02-25T11:53:41.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/8378332 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560fa1ab4b60 0x560fa1ad4f40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:41.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f1953cb5e00 4 client.16620 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:53:41.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.658+0000 7f1953cb5e00 1 -- 10.20.193.28:0/8378332 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560fa1561f40 con 0x560fa1ab4b60 2026-02-25T11:53:41.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.659+0000 7f194affd640 1 --2- 10.20.193.28:0/8378332 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560fa1ab4b60 0x560fa1ad4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:41.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.659+0000 7f194affd640 1 --2- 10.20.193.28:0/8378332 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560fa1ab4b60 0x560fa1ad4f40 crc :-1 s=READY pgs=109 gs=4 cs=0 l=0 c_cookie=13652378311433675381 s_cookie=6501913662398356013 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd76fdbda8406275 server_cookie=5a3b7006ff835a2d in_seq=0 out_seq=0 2026-02-25T11:53:41.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.659+0000 7f190d7fa640 10 client.16620 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:41.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.659+0000 7f190d7fa640 1 -- 10.20.193.28:0/8378332 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560fa1561f40 con 0x560fa1ab4b60 2026-02-25T11:53:41.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.659+0000 7f190d7fa640 10 client.16620 handle_command_reply: tid=0 2026-02-25T11:53:41.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.659+0000 7f1953cb5e00 1 client.16620 shutdown 2026-02-25T11:53:41.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1953cb5e00 1 -- 10.20.193.28:0/8378332 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 msgr2=0x560fa19e6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/8378332 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 0x560fa19e6b20 secure :-1 s=READY pgs=655 gs=1 cs=0 l=1 c_cookie=12057391651857350866 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f19380a8ea0 tx=0x7f19380ad010 comp rx=0 tx=0).stop 2026-02-25T11:53:41.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1953cb5e00 1 -- 10.20.193.28:0/8378332 shutdown_connections 2026-02-25T11:53:41.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/8378332 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560fa1ab4b60 0x560fa1ad4f40 crc :-1 s=READY pgs=109 gs=4 cs=0 l=0 c_cookie=13652378311433675381 s_cookie=6501913662398356013 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/8378332 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560fa1a080e0 0x560fa19e2810 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/8378332 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560fa19c3440 0x560fa19e7060 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1953cb5e00 1 --2- 10.20.193.28:0/8378332 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560fa19c2b30 0x560fa19e6b20 unknown :-1 s=CLOSED pgs=655 gs=1 cs=0 l=1 c_cookie=12057391651857350866 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f190d7fa640 0 client.16620 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:41.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1953cb5e00 1 -- 10.20.193.28:0/8378332 >> 10.20.193.28:0/8378332 conn(0x560fa18e17a0 msgr2=0x560fa19c0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:41.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1953cb5e00 1 -- 10.20.193.28:0/8378332 shutdown_connections 2026-02-25T11:53:41.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1953cb5e00 1 -- 10.20.193.28:0/8378332 wait complete. 2026-02-25T11:53:41.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1953cb5e00 20 client.16620 trim_cache size 0 max 16384 2026-02-25T11:53:41.667 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:53:41.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1951cba640 1 -- 10.20.193.28:0/3184264614 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f19200b5760 msgr2=0x7f19200d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1951cba640 1 --2- 10.20.193.28:0/3184264614 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f19200b5760 0x7f19200d5c10 secure :-1 s=READY pgs=328 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f194c1293b0 tx=0x7f193c00b410 comp rx=0 tx=0).stop 2026-02-25T11:53:41.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1951cba640 1 -- 10.20.193.28:0/3184264614 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f194c10bff0 msgr2=0x7f194c122a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:41.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1951cba640 1 --2- 10.20.193.28:0/3184264614 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f194c10bff0 0x7f194c122a10 secure :-1 s=READY pgs=179 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1938009480 tx=0x7f193800ee60 comp rx=0 tx=0).stop 2026-02-25T11:53:41.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1951cba640 1 -- 10.20.193.28:0/3184264614 shutdown_connections 2026-02-25T11:53:41.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1951cba640 1 --2- 10.20.193.28:0/3184264614 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f19200b5760 0x7f19200d5c10 unknown :-1 s=CLOSED pgs=328 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1951cba640 1 --2- 10.20.193.28:0/3184264614 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f194c1115d0 0x7f194c128320 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1951cba640 1 --2- 10.20.193.28:0/3184264614 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f194c10c9e0 0x7f194c122f50 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1951cba640 1 --2- 10.20.193.28:0/3184264614 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f194c10bff0 0x7f194c122a10 unknown :-1 s=CLOSED pgs=179 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:41.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.660+0000 7f1951cba640 1 -- 10.20.193.28:0/3184264614 >> 10.20.193.28:0/3184264614 conn(0x7f194c0e9c40 msgr2=0x7f194c10a020 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:41.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.661+0000 7f1951cba640 1 -- 10.20.193.28:0/3184264614 shutdown_connections 2026-02-25T11:53:41.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:41.661+0000 7f1951cba640 1 -- 10.20.193.28:0/3184264614 wait complete. 2026-02-25T11:53:42.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:42 trial167 ceph-mon[43392]: pgmap v390: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:42.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:42 trial119 ceph-mon[42710]: pgmap v390: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:42.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:42 trial028 ceph-mon[39224]: pgmap v390: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:42.761 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:53:42.762 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:53:42.947 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:43.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.117+0000 7fa61dc96640 1 -- 10.20.193.28:0/3570881788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa61810e190 msgr2=0x7fa61810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.117+0000 7fa61dc96640 1 --2- 10.20.193.28:0/3570881788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa61810e190 0x7fa61810e560 secure :-1 s=READY pgs=656 gs=1 cs=0 l=1 c_cookie=5400410971794211138 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa608009950 tx=0x7fa60802f190 comp rx=0 tx=0).stop 2026-02-25T11:53:43.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.118+0000 7fa61dc96640 1 -- 10.20.193.28:0/3570881788 shutdown_connections 2026-02-25T11:53:43.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.118+0000 7fa61dc96640 1 --2- 10.20.193.28:0/3570881788 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa618153740 0x7fa618173b60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.118+0000 7fa61dc96640 1 --2- 10.20.193.28:0/3570881788 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa61810eb80 0x7fa618153200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.118+0000 7fa61dc96640 1 --2- 10.20.193.28:0/3570881788 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa61810e190 0x7fa61810e560 unknown :-1 s=CLOSED pgs=656 gs=1 cs=0 l=1 c_cookie=5400410971794211138 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.118+0000 7fa61dc96640 1 -- 10.20.193.28:0/3570881788 >> 10.20.193.28:0/3570881788 conn(0x7fa6180e9b40 msgr2=0x7fa618109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:43.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.118+0000 7fa61dc96640 1 -- 10.20.193.28:0/3570881788 shutdown_connections 2026-02-25T11:53:43.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.118+0000 7fa61dc96640 1 -- 10.20.193.28:0/3570881788 wait complete. 2026-02-25T11:53:43.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.118+0000 7fa61dc96640 1 Processor -- start 2026-02-25T11:53:43.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61dc96640 1 -- start start 2026-02-25T11:53:43.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61dc96640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa61810e190 0x7fa618162170 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61dc96640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa61810eb80 0x7fa6181626b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61dc96640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa618153740 0x7fa618167a80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61dc96640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa6181680a0 con 0x7fa61810e190 2026-02-25T11:53:43.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61dc96640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa618168210 con 0x7fa618153740 2026-02-25T11:53:43.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61dc96640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa6181683d0 con 0x7fa61810eb80 2026-02-25T11:53:43.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61cc94640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa61810e190 0x7fa618162170 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61cc94640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa61810e190 0x7fa618162170 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37338/0 (socket says 10.20.193.28:37338) 2026-02-25T11:53:43.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61cc94640 1 -- 10.20.193.28:0/4122513735 learned_addr learned my addr 10.20.193.28:0/4122513735 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:43.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61d495640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa618153740 0x7fa618167a80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa60ffff640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa61810eb80 0x7fa6181626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61cc94640 1 -- 10.20.193.28:0/4122513735 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa61810eb80 msgr2=0x7fa6181626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61cc94640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa61810eb80 0x7fa6181626b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61cc94640 1 -- 10.20.193.28:0/4122513735 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa618153740 msgr2=0x7fa618167a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61cc94640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa618153740 0x7fa618167a80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61cc94640 1 -- 10.20.193.28:0/4122513735 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa608009570 con 0x7fa61810e190 2026-02-25T11:53:43.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.119+0000 7fa61d495640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa618153740 0x7fa618167a80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:53:43.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.120+0000 7fa61cc94640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa61810e190 0x7fa618162170 secure :-1 s=READY pgs=657 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa6080098a0 tx=0x7fa6080028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:43.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.120+0000 7fa60dffb640 1 -- 10.20.193.28:0/4122513735 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa60803d070 con 0x7fa61810e190 2026-02-25T11:53:43.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.120+0000 7fa60dffb640 1 -- 10.20.193.28:0/4122513735 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa608002a70 con 0x7fa61810e190 2026-02-25T11:53:43.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.120+0000 7fa60dffb640 1 -- 10.20.193.28:0/4122513735 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa608041890 con 0x7fa61810e190 2026-02-25T11:53:43.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.120+0000 7fa61dc96640 1 -- 10.20.193.28:0/4122513735 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa618168650 con 0x7fa61810e190 2026-02-25T11:53:43.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.120+0000 7fa61dc96640 1 -- 10.20.193.28:0/4122513735 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61814f1c0 con 0x7fa61810e190 2026-02-25T11:53:43.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.123+0000 7fa60dffb640 1 -- 10.20.193.28:0/4122513735 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fa608041a20 con 0x7fa61810e190 2026-02-25T11:53:43.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.125+0000 7fa60dffb640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5f40b5760 0x7fa5f40d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.125+0000 7fa60dffb640 1 -- 10.20.193.28:0/4122513735 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa60810afc0 con 0x7fa61810e190 2026-02-25T11:53:43.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.125+0000 7fa60ffff640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5f40b5760 0x7fa5f40d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.125+0000 7fa620f1ae00 1 Processor -- start 2026-02-25T11:53:43.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.125+0000 7fa620f1ae00 1 -- start start 2026-02-25T11:53:43.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa620f1ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9528750 0x5602c9528b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa620f1ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9529060 0x5602c956d610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa620f1ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c958df30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa620f1ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5602c943ed80 con 0x5602c9529060 2026-02-25T11:53:43.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa620f1ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5602c943aab0 con 0x5602c956db50 2026-02-25T11:53:43.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa620f1ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5602c9434a50 con 0x5602c9528750 2026-02-25T11:53:43.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa60ffff640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5f40b5760 0x7fa5f40d5c10 secure :-1 s=READY pgs=329 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa618168b10 tx=0x7fa600005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:43.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa61cc94640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9528750 0x5602c9528b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa61d495640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c958df30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa61cc94640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9528750 0x5602c9528b20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:35070/0 (socket says 10.20.193.28:35070) 2026-02-25T11:53:43.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa61cc94640 1 -- 10.20.193.28:0/3618751478 learned_addr learned my addr 10.20.193.28:0/3618751478 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:43.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa60ffff640 1 --2- 10.20.193.28:0/3618751478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9529060 0x5602c956d610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa61d495640 1 -- 10.20.193.28:0/3618751478 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9528750 msgr2=0x5602c9528b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa61d495640 1 --2- 10.20.193.28:0/3618751478 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9528750 0x5602c9528b20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa61d495640 1 -- 10.20.193.28:0/3618751478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9529060 msgr2=0x5602c956d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa61d495640 1 --2- 10.20.193.28:0/3618751478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9529060 0x5602c956d610 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa61d495640 1 -- 10.20.193.28:0/3618751478 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c93e0120 con 0x5602c956db50 2026-02-25T11:53:43.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa60ffff640 1 --2- 10.20.193.28:0/3618751478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9529060 0x5602c956d610 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:53:43.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.126+0000 7fa61d495640 1 --2- 10.20.193.28:0/3618751478 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c958df30 secure :-1 s=READY pgs=148 gs=1 cs=0 l=1 c_cookie=12942037571286320112 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa614003940 tx=0x7fa614003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b39b5a99ea47c3f0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:43.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.127+0000 7fa5e54e4640 1 -- 10.20.193.28:0/3618751478 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa614010bc0 con 0x5602c956db50 2026-02-25T11:53:43.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.127+0000 7fa5e54e4640 1 -- 10.20.193.28:0/3618751478 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa614010d20 con 0x5602c956db50 2026-02-25T11:53:43.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.127+0000 7fa5e54e4640 1 -- 10.20.193.28:0/3618751478 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa614011850 con 0x5602c956db50 2026-02-25T11:53:43.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.127+0000 7fa620f1ae00 1 -- 10.20.193.28:0/3618751478 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 msgr2=0x5602c958df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.127+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/3618751478 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c958df30 secure :-1 s=READY pgs=148 gs=1 cs=0 l=1 c_cookie=12942037571286320112 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa614003940 tx=0x7fa614003e10 comp rx=0 tx=0).stop 2026-02-25T11:53:43.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.127+0000 7fa620f1ae00 1 -- 10.20.193.28:0/3618751478 shutdown_connections 2026-02-25T11:53:43.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.127+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/3618751478 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c958df30 unknown :-1 s=CLOSED pgs=148 gs=1 cs=0 l=1 c_cookie=12942037571286320112 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.127+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/3618751478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9529060 0x5602c956d610 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.127+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/3618751478 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9528750 0x5602c9528b20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.127+0000 7fa620f1ae00 1 -- 10.20.193.28:0/3618751478 >> 10.20.193.28:0/3618751478 conn(0x5602c95068f0 msgr2=0x5602c9526cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:43.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.127+0000 7fa620f1ae00 1 -- 10.20.193.28:0/3618751478 shutdown_connections 2026-02-25T11:53:43.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.127+0000 7fa620f1ae00 1 -- 10.20.193.28:0/3618751478 wait complete. 2026-02-25T11:53:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.128+0000 7fa620f1ae00 1 -- start start 2026-02-25T11:53:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.129+0000 7fa620f1ae00 1 Processor -- start 2026-02-25T11:53:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.129+0000 7fa620f1ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9528750 0x5602c9529740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.129+0000 7fa620f1ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 0x5602c9529c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.129+0000 7fa620f1ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c954e5f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.129+0000 7fa620f1ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5602c954eb30 con 0x5602c9528750 2026-02-25T11:53:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.129+0000 7fa620f1ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5602c954eca0 con 0x5602c956db50 2026-02-25T11:53:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.129+0000 7fa620f1ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5602c954ee60 con 0x5602c9529060 2026-02-25T11:53:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.129+0000 7fa60ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 0x5602c9529c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.129+0000 7fa61d495640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c954e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.129+0000 7fa61d495640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c954e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:47806/0 (socket says 10.20.193.28:47806) 2026-02-25T11:53:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.129+0000 7fa61d495640 1 -- 10.20.193.28:0/357092506 learned_addr learned my addr 10.20.193.28:0/357092506 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.129+0000 7fa61cc94640 1 --2- 10.20.193.28:0/357092506 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9528750 0x5602c9529740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa61d495640 1 -- 10.20.193.28:0/357092506 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 msgr2=0x5602c9529c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa61d495640 1 --2- 10.20.193.28:0/357092506 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 0x5602c9529c80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa61d495640 1 -- 10.20.193.28:0/357092506 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9528750 msgr2=0x5602c9529740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa61d495640 1 --2- 10.20.193.28:0/357092506 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9528750 0x5602c9529740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa61d495640 1 -- 10.20.193.28:0/357092506 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c93b8810 con 0x5602c956db50 2026-02-25T11:53:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa61cc94640 1 --2- 10.20.193.28:0/357092506 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9528750 0x5602c9529740 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:53:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa61d495640 1 --2- 10.20.193.28:0/357092506 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c954e5f0 secure :-1 s=READY pgs=149 gs=1 cs=0 l=1 c_cookie=6502437929536480062 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa614014010 tx=0x7fa614010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a3d4cd8740c633e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa5d67fc640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:53:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa5d67fc640 1 -- 10.20.193.28:0/357092506 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa614011800 con 0x5602c956db50 2026-02-25T11:53:43.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa620f1ae00 1 -- 10.20.193.28:0/357092506 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa614005580 con 0x5602c956db50 2026-02-25T11:53:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa620f1ae00 10 client.26396 _ll_register_callbacks cb 0x5602c93b5e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa620f1ae00 1 -- 10.20.193.28:0/357092506 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5602c918c530 con 0x5602c956db50 2026-02-25T11:53:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa5d67fc640 1 -- 10.20.193.28:0/357092506 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa614011ea0 con 0x5602c956db50 2026-02-25T11:53:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa5d67fc640 1 -- 10.20.193.28:0/357092506 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa61401b6d0 con 0x5602c956db50 2026-02-25T11:53:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa5d67fc640 1 -- 10.20.193.28:0/357092506 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa61401b860 con 0x5602c956db50 2026-02-25T11:53:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa5d67fc640 1 -- 10.20.193.28:0/357092506 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa61401ba80 con 0x5602c956db50 2026-02-25T11:53:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa620f1ae00 10 client.26396 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.130+0000 7fa620f1ae00 1 -- 10.20.193.28:0/357092506 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5602c93ca4f0 con 0x5602c956db50 2026-02-25T11:53:43.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.131+0000 7fa5d67fc640 1 -- 10.20.193.28:0/357092506 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa61401bbe0 con 0x5602c956db50 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.131+0000 7fa620f1ae00 10 client.26396 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.131+0000 7fa620f1ae00 10 client.26396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.131+0000 7fa620f1ae00 20 client.26396 populate_metadata read hostname 'trial028' 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.131+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/357092506 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5602c9553650 0x5602c9553ac0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.131+0000 7fa620f1ae00 4 client.26396 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.131+0000 7fa620f1ae00 1 -- 10.20.193.28:0/357092506 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602c93e1b50 con 0x5602c9553650 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.131+0000 7fa61cc94640 1 --2- 10.20.193.28:0/357092506 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5602c9553650 0x5602c9553ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.132+0000 7fa61cc94640 1 --2- 10.20.193.28:0/357092506 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5602c9553650 0x5602c9553ac0 crc :-1 s=READY pgs=110 gs=4 cs=0 l=0 c_cookie=12096489423067135046 s_cookie=16592688691146420190 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7df5d0a532fbc46 server_cookie=e6451273e86c6fde in_seq=0 out_seq=0 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.132+0000 7fa5d67fc640 10 client.26396 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.132+0000 7fa5d67fc640 1 -- 10.20.193.28:0/357092506 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5602c93e1b50 con 0x5602c9553650 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.132+0000 7fa5d67fc640 10 client.26396 handle_command_reply: tid=0 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.132+0000 7fa620f1ae00 1 client.26396 shutdown 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.133+0000 7fa620f1ae00 1 -- 10.20.193.28:0/357092506 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 msgr2=0x5602c954e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.133+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/357092506 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c954e5f0 secure :-1 s=READY pgs=149 gs=1 cs=0 l=1 c_cookie=6502437929536480062 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa614014010 tx=0x7fa614010bd0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.133+0000 7fa620f1ae00 1 -- 10.20.193.28:0/357092506 shutdown_connections 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.133+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/357092506 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5602c9553650 0x5602c9553ac0 crc :-1 s=READY pgs=110 gs=4 cs=0 l=0 c_cookie=12096489423067135046 s_cookie=16592688691146420190 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.133+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/357092506 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c954e5f0 unknown :-1 s=CLOSED pgs=149 gs=1 cs=0 l=1 c_cookie=6502437929536480062 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.133+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/357092506 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 0x5602c9529c80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.133+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/357092506 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9528750 0x5602c9529740 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.133+0000 7fa620f1ae00 1 -- 10.20.193.28:0/357092506 >> 10.20.193.28:0/357092506 conn(0x5602c95068f0 msgr2=0x5602c9526cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.133+0000 7fa5d67fc640 0 client.26396 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.133+0000 7fa620f1ae00 1 -- 10.20.193.28:0/357092506 shutdown_connections 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.133+0000 7fa620f1ae00 1 -- 10.20.193.28:0/357092506 wait complete. 2026-02-25T11:53:43.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.133+0000 7fa620f1ae00 20 client.26396 trim_cache size 0 max 16384 2026-02-25T11:53:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.142+0000 7fa620f1ae00 1 Processor -- start 2026-02-25T11:53:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.142+0000 7fa620f1ae00 1 -- start start 2026-02-25T11:53:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.142+0000 7fa620f1ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9528750 0x5602c956d050 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.142+0000 7fa620f1ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 0x5602c956d590 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.142+0000 7fa620f1ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c958b0c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.142+0000 7fa620f1ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5602c9432680 con 0x5602c9528750 2026-02-25T11:53:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.142+0000 7fa620f1ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5602c90f2790 con 0x5602c956db50 2026-02-25T11:53:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.142+0000 7fa620f1ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5602c9586f70 con 0x5602c9529060 2026-02-25T11:53:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.142+0000 7fa61cc94640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9528750 0x5602c956d050 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.142+0000 7fa60ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 0x5602c956d590 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.143+0000 7fa60ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 0x5602c956d590 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:35098/0 (socket says 10.20.193.28:35098) 2026-02-25T11:53:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.143+0000 7fa60ffff640 1 -- 10.20.193.28:0/3016631279 learned_addr learned my addr 10.20.193.28:0/3016631279 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.143+0000 7fa61d495640 1 --2- 10.20.193.28:0/3016631279 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c958b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.143+0000 7fa61cc94640 1 -- 10.20.193.28:0/3016631279 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 msgr2=0x5602c956d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.143+0000 7fa61cc94640 1 --2- 10.20.193.28:0/3016631279 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 0x5602c956d590 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.143+0000 7fa61cc94640 1 -- 10.20.193.28:0/3016631279 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 msgr2=0x5602c958b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.143+0000 7fa61cc94640 1 --2- 10.20.193.28:0/3016631279 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c958b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.143+0000 7fa61cc94640 1 -- 10.20.193.28:0/3016631279 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c93c4f70 con 0x5602c9528750 2026-02-25T11:53:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.143+0000 7fa61cc94640 1 --2- 10.20.193.28:0/3016631279 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9528750 0x5602c956d050 secure :-1 s=READY pgs=658 gs=1 cs=0 l=1 c_cookie=3950915475037106612 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa608002940 tx=0x7fa60803a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d478b1132e31b4 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.143+0000 7fa5e4ce3640 1 -- 10.20.193.28:0/3016631279 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa6080d7060 con 0x5602c9528750 2026-02-25T11:53:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.143+0000 7fa5e4ce3640 1 -- 10.20.193.28:0/3016631279 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa6080d9640 con 0x5602c9528750 2026-02-25T11:53:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.143+0000 7fa5e4ce3640 1 -- 10.20.193.28:0/3016631279 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa6080d2250 con 0x5602c9528750 2026-02-25T11:53:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.144+0000 7fa620f1ae00 1 -- 10.20.193.28:0/3016631279 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9528750 msgr2=0x5602c956d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.144+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/3016631279 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9528750 0x5602c956d050 secure :-1 s=READY pgs=658 gs=1 cs=0 l=1 c_cookie=3950915475037106612 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa608002940 tx=0x7fa60803a040 comp rx=0 tx=0).stop 2026-02-25T11:53:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.144+0000 7fa620f1ae00 1 -- 10.20.193.28:0/3016631279 shutdown_connections 2026-02-25T11:53:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.144+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/3016631279 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c956db50 0x5602c958b0c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.144+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/3016631279 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 0x5602c956d590 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.144+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/3016631279 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c9528750 0x5602c956d050 unknown :-1 s=CLOSED pgs=658 gs=1 cs=0 l=1 c_cookie=3950915475037106612 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.144+0000 7fa620f1ae00 1 -- 10.20.193.28:0/3016631279 >> 10.20.193.28:0/3016631279 conn(0x5602c95068f0 msgr2=0x5602c956c940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:43.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.144+0000 7fa620f1ae00 1 -- 10.20.193.28:0/3016631279 shutdown_connections 2026-02-25T11:53:43.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.144+0000 7fa620f1ae00 1 -- 10.20.193.28:0/3016631279 wait complete. 2026-02-25T11:53:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.144+0000 7fa620f1ae00 1 -- start start 2026-02-25T11:53:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.144+0000 7fa620f1ae00 1 Processor -- start 2026-02-25T11:53:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.145+0000 7fa620f1ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c9528750 0x5602c9556db0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.145+0000 7fa620f1ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 0x5602c95572f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.145+0000 7fa620f1ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c956db50 0x5602c9622a30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.145+0000 7fa620f1ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5602c9585a70 con 0x5602c956db50 2026-02-25T11:53:43.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.145+0000 7fa620f1ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5602c9553c50 con 0x5602c9528750 2026-02-25T11:53:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.145+0000 7fa620f1ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5602c9623050 con 0x5602c9529060 2026-02-25T11:53:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.145+0000 7fa61cc94640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c9528750 0x5602c9556db0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.145+0000 7fa61cc94640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c9528750 0x5602c9556db0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:47826/0 (socket says 10.20.193.28:47826) 2026-02-25T11:53:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.145+0000 7fa61cc94640 1 -- 10.20.193.28:0/2674893188 learned_addr learned my addr 10.20.193.28:0/2674893188 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.145+0000 7fa61d495640 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c956db50 0x5602c9622a30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.145+0000 7fa60ffff640 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 0x5602c95572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa61d495640 1 -- 10.20.193.28:0/2674893188 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 msgr2=0x5602c95572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa61d495640 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 0x5602c95572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa61d495640 1 -- 10.20.193.28:0/2674893188 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c9528750 msgr2=0x5602c9556db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa61d495640 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c9528750 0x5602c9556db0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa61d495640 1 -- 10.20.193.28:0/2674893188 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602c93b8b40 con 0x5602c956db50 2026-02-25T11:53:43.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa61cc94640 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c9528750 0x5602c9556db0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa61d495640 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c956db50 0x5602c9622a30 secure :-1 s=READY pgs=659 gs=3 cs=0 l=1 c_cookie=8806606127843067299 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa61402fc40 tx=0x7fa6140068c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a37599d26dd45a3 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa5d67fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa5d67fc640 1 -- 10.20.193.28:0/2674893188 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa61403d070 con 0x5602c956db50 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa5d67fc640 1 -- 10.20.193.28:0/2674893188 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fa61401f070 con 0x5602c956db50 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa620f1ae00 1 -- 10.20.193.28:0/2674893188 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61814f1c0 con 0x5602c956db50 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa5d67fc640 1 -- 10.20.193.28:0/2674893188 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fa61402f3b0 con 0x5602c956db50 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa620f1ae00 10 client.16635 _ll_register_callbacks cb 0x5602c93b5e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.146+0000 7fa620f1ae00 1 -- 10.20.193.28:0/2674893188 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa614010d20 con 0x5602c956db50 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.147+0000 7fa5d67fc640 1 -- 10.20.193.28:0/2674893188 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fa614012510 con 0x5602c956db50 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.147+0000 7fa5d67fc640 1 -- 10.20.193.28:0/2674893188 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa614006c00 con 0x5602c956db50 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.147+0000 7fa620f1ae00 10 client.16635 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.147+0000 7fa620f1ae00 1 -- 10.20.193.28:0/2674893188 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5602c93ca750 con 0x5602c956db50 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.147+0000 7fa5d67fc640 1 -- 10.20.193.28:0/2674893188 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fa6140128d0 con 0x5602c956db50 2026-02-25T11:53:43.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.147+0000 7fa620f1ae00 10 client.16635 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.147+0000 7fa620f1ae00 10 client.16635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.147+0000 7fa620f1ae00 20 client.16635 populate_metadata read hostname 'trial028' 2026-02-25T11:53:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.147+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5602c9624610 0x5602c9644ac0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.147+0000 7fa620f1ae00 4 client.16635 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:53:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.147+0000 7fa620f1ae00 1 -- 10.20.193.28:0/2674893188 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5602c9295690 con 0x5602c9624610 2026-02-25T11:53:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.148+0000 7fa61cc94640 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5602c9624610 0x5602c9644ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.148+0000 7fa61cc94640 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5602c9624610 0x5602c9644ac0 crc :-1 s=READY pgs=111 gs=4 cs=0 l=0 c_cookie=7772631537447614022 s_cookie=5462592066459687306 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bddef33bf903646 server_cookie=4bcf068ada32598a in_seq=0 out_seq=0 2026-02-25T11:53:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.148+0000 7fa5d67fc640 10 client.16635 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.149+0000 7fa5d67fc640 1 -- 10.20.193.28:0/2674893188 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5602c9295690 con 0x5602c9624610 2026-02-25T11:53:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.149+0000 7fa5d67fc640 10 client.16635 handle_command_reply: tid=0 2026-02-25T11:53:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.149+0000 7fa620f1ae00 1 client.16635 shutdown 2026-02-25T11:53:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.149+0000 7fa620f1ae00 1 -- 10.20.193.28:0/2674893188 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c956db50 msgr2=0x5602c9622a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.149+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c956db50 0x5602c9622a30 secure :-1 s=READY pgs=659 gs=3 cs=0 l=1 c_cookie=8806606127843067299 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa61402fc40 tx=0x7fa6140068c0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.149+0000 7fa620f1ae00 1 -- 10.20.193.28:0/2674893188 shutdown_connections 2026-02-25T11:53:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.149+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5602c9624610 0x5602c9644ac0 crc :-1 s=READY pgs=111 gs=4 cs=0 l=0 c_cookie=7772631537447614022 s_cookie=5462592066459687306 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.149+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5602c956db50 0x5602c9622a30 unknown :-1 s=CLOSED pgs=659 gs=3 cs=0 l=1 c_cookie=8806606127843067299 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.149+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5602c9529060 0x5602c95572f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.149+0000 7fa620f1ae00 1 --2- 10.20.193.28:0/2674893188 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5602c9528750 0x5602c9556db0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.149+0000 7fa620f1ae00 1 -- 10.20.193.28:0/2674893188 >> 10.20.193.28:0/2674893188 conn(0x5602c95068f0 msgr2=0x5602c956d130 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.149+0000 7fa5d67fc640 0 client.16635 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.149+0000 7fa620f1ae00 1 -- 10.20.193.28:0/2674893188 shutdown_connections 2026-02-25T11:53:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.150+0000 7fa620f1ae00 1 -- 10.20.193.28:0/2674893188 wait complete. 2026-02-25T11:53:43.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.150+0000 7fa620f1ae00 20 client.16635 trim_cache size 0 max 16384 2026-02-25T11:53:43.158 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:53:43.158 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:53:43.158 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "0d9e5421-597c-4231-88a4-8abf309c266b", 2026-02-25T11:53:43.158 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:53:43.158 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:53:43.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.150+0000 7fa61dc96640 1 -- 10.20.193.28:0/4122513735 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5f40b5760 msgr2=0x7fa5f40d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.150+0000 7fa61dc96640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5f40b5760 0x7fa5f40d5c10 secure :-1 s=READY pgs=329 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa618168b10 tx=0x7fa600005d00 comp rx=0 tx=0).stop 2026-02-25T11:53:43.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.150+0000 7fa61dc96640 1 -- 10.20.193.28:0/4122513735 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa61810e190 msgr2=0x7fa618162170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.150+0000 7fa61dc96640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa61810e190 0x7fa618162170 secure :-1 s=READY pgs=657 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fa6080098a0 tx=0x7fa6080028d0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.150+0000 7fa61dc96640 1 -- 10.20.193.28:0/4122513735 shutdown_connections 2026-02-25T11:53:43.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.150+0000 7fa61dc96640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fa5f40b5760 0x7fa5f40d5c10 unknown :-1 s=CLOSED pgs=329 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.150+0000 7fa61dc96640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fa618153740 0x7fa618167a80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.150+0000 7fa61dc96640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fa61810eb80 0x7fa6181626b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.150+0000 7fa61dc96640 1 --2- 10.20.193.28:0/4122513735 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fa61810e190 0x7fa618162170 unknown :-1 s=CLOSED pgs=657 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.150+0000 7fa61dc96640 1 -- 10.20.193.28:0/4122513735 >> 10.20.193.28:0/4122513735 conn(0x7fa6180e9b40 msgr2=0x7fa618109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:43.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.151+0000 7fa61dc96640 1 -- 10.20.193.28:0/4122513735 shutdown_connections 2026-02-25T11:53:43.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.151+0000 7fa61dc96640 1 -- 10.20.193.28:0/4122513735 wait complete. 2026-02-25T11:53:43.251 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:53:43.439 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:53:43.484 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:43 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:53:43.484 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:43 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:53:43.484 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:43 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:53:43.484 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:43 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:53:43.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:43 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:53:43.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:43 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:53:43.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:43 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:53:43.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:43 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:53:43.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:43 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:53:43.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:43 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:53:43.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:43 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:53:43.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:43 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:53:43.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.624+0000 7f62637c8640 1 -- 10.20.193.28:0/3821495732 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f625c115930 msgr2=0x7f625c176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.624+0000 7f62637c8640 1 --2- 10.20.193.28:0/3821495732 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f625c115930 0x7f625c176300 secure :-1 s=READY pgs=660 gs=1 cs=0 l=1 c_cookie=14735616946626554826 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f625800aa30 tx=0x7f6258030270 comp rx=0 tx=0).stop 2026-02-25T11:53:43.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.624+0000 7f62637c8640 1 -- 10.20.193.28:0/3821495732 shutdown_connections 2026-02-25T11:53:43.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.624+0000 7f62637c8640 1 --2- 10.20.193.28:0/3821495732 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f625c115930 0x7f625c176300 unknown :-1 s=CLOSED pgs=660 gs=1 cs=0 l=1 c_cookie=14735616946626554826 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.624+0000 7f62637c8640 1 --2- 10.20.193.28:0/3821495732 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f625c110130 0x7f625c115180 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.624+0000 7f62637c8640 1 --2- 10.20.193.28:0/3821495732 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f625c10f740 0x7f625c10fb10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.624+0000 7f62637c8640 1 -- 10.20.193.28:0/3821495732 >> 10.20.193.28:0/3821495732 conn(0x7f625c0ecff0 msgr2=0x7f625c10d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:43.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.625+0000 7f62637c8640 1 -- 10.20.193.28:0/3821495732 shutdown_connections 2026-02-25T11:53:43.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.625+0000 7f62637c8640 1 -- 10.20.193.28:0/3821495732 wait complete. 2026-02-25T11:53:43.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f62637c8640 1 Processor -- start 2026-02-25T11:53:43.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f62637c8640 1 -- start start 2026-02-25T11:53:43.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f62637c8640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f625c10f740 0x7f625c0f5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f62637c8640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f625c110130 0x7f625c176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f62637c8640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f625c115930 0x7f625c0f6140 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f62637c8640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f625c0fb320 con 0x7f625c10f740 2026-02-25T11:53:43.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f62637c8640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f625c0fb490 con 0x7f625c110130 2026-02-25T11:53:43.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f62637c8640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f625c0fb650 con 0x7f625c115930 2026-02-25T11:53:43.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f626153d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f625c10f740 0x7f625c0f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f626153d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f625c10f740 0x7f625c0f5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37412/0 (socket says 10.20.193.28:37412) 2026-02-25T11:53:43.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f626153d640 1 -- 10.20.193.28:0/1241220605 learned_addr learned my addr 10.20.193.28:0/1241220605 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:43.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f6261d3e640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f625c115930 0x7f625c0f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f6260d3c640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f625c110130 0x7f625c176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f626153d640 1 -- 10.20.193.28:0/1241220605 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f625c115930 msgr2=0x7f625c0f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f626153d640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f625c115930 0x7f625c0f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f626153d640 1 -- 10.20.193.28:0/1241220605 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f625c110130 msgr2=0x7f625c176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f626153d640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f625c110130 0x7f625c176200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.626+0000 7f626153d640 1 -- 10.20.193.28:0/1241220605 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625800a650 con 0x7f625c10f740 2026-02-25T11:53:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.627+0000 7f626153d640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f625c10f740 0x7f625c0f5ab0 secure :-1 s=READY pgs=661 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f624c00c9b0 tx=0x7f624c00ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.627+0000 7f624a7fc640 1 -- 10.20.193.28:0/1241220605 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f624c0044a0 con 0x7f625c10f740 2026-02-25T11:53:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.627+0000 7f624a7fc640 1 -- 10.20.193.28:0/1241220605 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f624c007d80 con 0x7f625c10f740 2026-02-25T11:53:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.627+0000 7f624a7fc640 1 -- 10.20.193.28:0/1241220605 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f624c00f6d0 con 0x7f625c10f740 2026-02-25T11:53:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.627+0000 7f62637c8640 1 -- 10.20.193.28:0/1241220605 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625c0fb930 con 0x7f625c10f740 2026-02-25T11:53:43.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.627+0000 7f62637c8640 1 -- 10.20.193.28:0/1241220605 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f625c0fbdb0 con 0x7f625c10f740 2026-02-25T11:53:43.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.630+0000 7f624a7fc640 1 -- 10.20.193.28:0/1241220605 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f624c0078c0 con 0x7f625c10f740 2026-02-25T11:53:43.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.632+0000 7f624a7fc640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f62380b5760 0x7f62380d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.632+0000 7f624a7fc640 1 -- 10.20.193.28:0/1241220605 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f624c0e6810 con 0x7f625c10f740 2026-02-25T11:53:43.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.632+0000 7f6260d3c640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f62380b5760 0x7f62380d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.632+0000 7f62657c3e00 1 Processor -- start 2026-02-25T11:53:43.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.632+0000 7f62657c3e00 1 -- start start 2026-02-25T11:53:43.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f62657c3e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279bfb50 0x564c279bff20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f62657c3e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 0x564c27a04bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f62657c3e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 0x564c27a254e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f62657c3e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564c278cd9b0 con 0x564c279c0460 2026-02-25T11:53:43.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f62657c3e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564c278c96e0 con 0x564c27a05100 2026-02-25T11:53:43.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f62657c3e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564c278c3840 con 0x564c279bfb50 2026-02-25T11:53:43.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f6260d3c640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f62380b5760 0x7f62380d5c10 secure :-1 s=READY pgs=330 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6250005fd0 tx=0x7f625000a580 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:43.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f6260d3c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 0x564c27a04bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f626153d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279bfb50 0x564c279bff20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f6260d3c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 0x564c27a04bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37422/0 (socket says 10.20.193.28:37422) 2026-02-25T11:53:43.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f6260d3c640 1 -- 10.20.193.28:0/332433633 learned_addr learned my addr 10.20.193.28:0/332433633 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:43.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f6261d3e640 1 --2- 10.20.193.28:0/332433633 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 0x564c27a254e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f626153d640 1 -- 10.20.193.28:0/332433633 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 msgr2=0x564c27a254e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f626153d640 1 --2- 10.20.193.28:0/332433633 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 0x564c27a254e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f626153d640 1 -- 10.20.193.28:0/332433633 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 msgr2=0x564c27a04bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f626153d640 1 --2- 10.20.193.28:0/332433633 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 0x564c27a04bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f626153d640 1 -- 10.20.193.28:0/332433633 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c2786ed80 con 0x564c279bfb50 2026-02-25T11:53:43.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.633+0000 7f6260d3c640 1 --2- 10.20.193.28:0/332433633 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 0x564c27a04bc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:53:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.634+0000 7f626153d640 1 --2- 10.20.193.28:0/332433633 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279bfb50 0x564c279bff20 secure :-1 s=READY pgs=181 gs=1 cs=0 l=1 c_cookie=5639819321690360554 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f624c0e4300 tx=0x7f624c0046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e44a9cac2bd0aea server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.634+0000 7f6229ce5640 1 -- 10.20.193.28:0/332433633 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f624c0e40d0 con 0x564c279bfb50 2026-02-25T11:53:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.634+0000 7f6229ce5640 1 -- 10.20.193.28:0/332433633 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f624c0026a0 con 0x564c279bfb50 2026-02-25T11:53:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.634+0000 7f6229ce5640 1 -- 10.20.193.28:0/332433633 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f624c0b7cd0 con 0x564c279bfb50 2026-02-25T11:53:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.634+0000 7f62657c3e00 1 -- 10.20.193.28:0/332433633 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279bfb50 msgr2=0x564c279bff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.634+0000 7f62657c3e00 1 --2- 10.20.193.28:0/332433633 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279bfb50 0x564c279bff20 secure :-1 s=READY pgs=181 gs=1 cs=0 l=1 c_cookie=5639819321690360554 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f624c0e4300 tx=0x7f624c0046d0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.634+0000 7f62657c3e00 1 -- 10.20.193.28:0/332433633 shutdown_connections 2026-02-25T11:53:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.634+0000 7f62657c3e00 1 --2- 10.20.193.28:0/332433633 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 0x564c27a254e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.634+0000 7f62657c3e00 1 --2- 10.20.193.28:0/332433633 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 0x564c27a04bc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.634+0000 7f62657c3e00 1 --2- 10.20.193.28:0/332433633 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279bfb50 0x564c279bff20 unknown :-1 s=CLOSED pgs=181 gs=1 cs=0 l=1 c_cookie=5639819321690360554 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.634+0000 7f62657c3e00 1 -- 10.20.193.28:0/332433633 >> 10.20.193.28:0/332433633 conn(0x564c278d6d40 msgr2=0x564c279b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:43.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.635+0000 7f62657c3e00 1 -- 10.20.193.28:0/332433633 shutdown_connections 2026-02-25T11:53:43.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.635+0000 7f62657c3e00 1 -- 10.20.193.28:0/332433633 wait complete. 2026-02-25T11:53:43.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.636+0000 7f62657c3e00 1 -- start start 2026-02-25T11:53:43.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.636+0000 7f62657c3e00 1 Processor -- start 2026-02-25T11:53:43.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f62657c3e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c279bfb50 0x564c27a1cd40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f62657c3e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279c0460 0x564c27a1d280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f62657c3e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c27a05100 0x564c27a22270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f62657c3e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564c27a227b0 con 0x564c27a05100 2026-02-25T11:53:43.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f62657c3e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564c27a22920 con 0x564c279bfb50 2026-02-25T11:53:43.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f62657c3e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564c27a22ae0 con 0x564c279c0460 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f6261d3e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c27a05100 0x564c27a22270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f6261d3e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c27a05100 0x564c27a22270 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37428/0 (socket says 10.20.193.28:37428) 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f6261d3e640 1 -- 10.20.193.28:0/1751009223 learned_addr learned my addr 10.20.193.28:0/1751009223 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f626153d640 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c279bfb50 0x564c27a1cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f6260d3c640 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279c0460 0x564c27a1d280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f6261d3e640 1 -- 10.20.193.28:0/1751009223 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279c0460 msgr2=0x564c27a1d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f6261d3e640 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279c0460 0x564c27a1d280 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f6261d3e640 1 -- 10.20.193.28:0/1751009223 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c279bfb50 msgr2=0x564c27a1cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f6261d3e640 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c279bfb50 0x564c27a1cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f6261d3e640 1 -- 10.20.193.28:0/1751009223 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625c0fbdb0 con 0x564c27a05100 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.637+0000 7f626153d640 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c279bfb50 0x564c27a1cd40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.638+0000 7f6261d3e640 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c27a05100 0x564c27a22270 secure :-1 s=READY pgs=662 gs=1 cs=0 l=1 c_cookie=9581446494340111011 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f625800a230 tx=0x7f62580396a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f822d2698ddaa3 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.638+0000 7f6212ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:43.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.638+0000 7f6212ffd640 1 -- 10.20.193.28:0/1751009223 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f625803e040 con 0x564c27a05100 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.638+0000 7f62657c3e00 1 -- 10.20.193.28:0/1751009223 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f625000ebe0 con 0x564c27a05100 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.638+0000 7f6212ffd640 1 -- 10.20.193.28:0/1751009223 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6258039de0 con 0x564c27a05100 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.638+0000 7f6212ffd640 1 -- 10.20.193.28:0/1751009223 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6258044810 con 0x564c27a05100 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.638+0000 7f62657c3e00 10 client.16647 _ll_register_callbacks cb 0x564c278449a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.638+0000 7f62657c3e00 1 -- 10.20.193.28:0/1751009223 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564c275596f0 con 0x564c27a05100 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.638+0000 7f6212ffd640 1 -- 10.20.193.28:0/1751009223 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f62580051e0 con 0x564c27a05100 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.638+0000 7f6212ffd640 1 -- 10.20.193.28:0/1751009223 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6258039920 con 0x564c27a05100 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.638+0000 7f62657c3e00 10 client.16647 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.638+0000 7f62657c3e00 1 -- 10.20.193.28:0/1751009223 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x564c27847350 con 0x564c27a05100 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.639+0000 7f6212ffd640 1 -- 10.20.193.28:0/1751009223 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6258039ac0 con 0x564c27a05100 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.639+0000 7f62657c3e00 10 client.16647 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.639+0000 7f62657c3e00 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.639+0000 7f62657c3e00 20 client.16647 populate_metadata read hostname 'trial028' 2026-02-25T11:53:43.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.639+0000 7f62657c3e00 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564c27a25360 0x564c27a02e80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.639+0000 7f62657c3e00 4 client.16647 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:53:43.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.639+0000 7f62657c3e00 1 -- 10.20.193.28:0/1751009223 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c27895910 con 0x564c27a25360 2026-02-25T11:53:43.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.639+0000 7f626153d640 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564c27a25360 0x564c27a02e80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.640+0000 7f626153d640 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564c27a25360 0x564c27a02e80 crc :-1 s=READY pgs=112 gs=4 cs=0 l=0 c_cookie=476435489012381654 s_cookie=7130982469089498365 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69ca38d90f8f7d6 server_cookie=62f656c6299e14fd in_seq=0 out_seq=0 2026-02-25T11:53:43.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.640+0000 7f6212ffd640 10 client.16647 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f6212ffd640 1 -- 10.20.193.28:0/1751009223 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x564c27895910 con 0x564c27a25360 2026-02-25T11:53:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f6212ffd640 10 client.16647 handle_command_reply: tid=0 2026-02-25T11:53:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f62657c3e00 1 client.16647 shutdown 2026-02-25T11:53:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f62657c3e00 1 -- 10.20.193.28:0/1751009223 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c27a05100 msgr2=0x564c27a22270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f62657c3e00 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c27a05100 0x564c27a22270 secure :-1 s=READY pgs=662 gs=1 cs=0 l=1 c_cookie=9581446494340111011 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f625800a230 tx=0x7f62580396a0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f62657c3e00 1 -- 10.20.193.28:0/1751009223 shutdown_connections 2026-02-25T11:53:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f62657c3e00 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564c27a25360 0x564c27a02e80 crc :-1 s=READY pgs=112 gs=4 cs=0 l=0 c_cookie=476435489012381654 s_cookie=7130982469089498365 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f62657c3e00 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c27a05100 0x564c27a22270 unknown :-1 s=CLOSED pgs=662 gs=1 cs=0 l=1 c_cookie=9581446494340111011 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f62657c3e00 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279c0460 0x564c27a1d280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f62657c3e00 1 --2- 10.20.193.28:0/1751009223 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c279bfb50 0x564c27a1cd40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f62657c3e00 1 -- 10.20.193.28:0/1751009223 >> 10.20.193.28:0/1751009223 conn(0x564c278d6d40 msgr2=0x564c279b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f6212ffd640 0 client.16647 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f62657c3e00 1 -- 10.20.193.28:0/1751009223 shutdown_connections 2026-02-25T11:53:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f62657c3e00 1 -- 10.20.193.28:0/1751009223 wait complete. 2026-02-25T11:53:43.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.641+0000 7f62657c3e00 20 client.16647 trim_cache size 0 max 16384 2026-02-25T11:53:43.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.650+0000 7f62657c3e00 1 Processor -- start 2026-02-25T11:53:43.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f62657c3e00 1 -- start start 2026-02-25T11:53:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f62657c3e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279bfb50 0x564c27a1a570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f62657c3e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 0x564c27a03e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f62657c3e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 0x564c27a043d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f62657c3e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564c278c1470 con 0x564c279c0460 2026-02-25T11:53:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f62657c3e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564c27581700 con 0x564c27a05100 2026-02-25T11:53:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f62657c3e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564c2792b6e0 con 0x564c279bfb50 2026-02-25T11:53:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f6260d3c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 0x564c27a03e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f6260d3c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 0x564c27a03e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37440/0 (socket says 10.20.193.28:37440) 2026-02-25T11:53:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f6260d3c640 1 -- 10.20.193.28:0/763225321 learned_addr learned my addr 10.20.193.28:0/763225321 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f626153d640 1 --2- 10.20.193.28:0/763225321 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279bfb50 0x564c27a1a570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f6260d3c640 1 -- 10.20.193.28:0/763225321 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279bfb50 msgr2=0x564c27a1a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f6260d3c640 1 --2- 10.20.193.28:0/763225321 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279bfb50 0x564c27a1a570 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f6260d3c640 1 -- 10.20.193.28:0/763225321 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 msgr2=0x564c27a043d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f6260d3c640 1 --2- 10.20.193.28:0/763225321 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 0x564c27a043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f6260d3c640 1 -- 10.20.193.28:0/763225321 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c27a0ef00 con 0x564c279c0460 2026-02-25T11:53:43.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.651+0000 7f6260d3c640 1 --2- 10.20.193.28:0/763225321 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 0x564c27a03e90 secure :-1 s=READY pgs=663 gs=2 cs=0 l=1 c_cookie=5997383911027991430 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6250004430 tx=0x7f6250011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533afce43e776386 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:43.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.652+0000 7f62294e4640 1 -- 10.20.193.28:0/763225321 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f625001e870 con 0x564c279c0460 2026-02-25T11:53:43.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.652+0000 7f62294e4640 1 -- 10.20.193.28:0/763225321 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f625001ee90 con 0x564c279c0460 2026-02-25T11:53:43.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.652+0000 7f62294e4640 1 -- 10.20.193.28:0/763225321 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f625001d440 con 0x564c279c0460 2026-02-25T11:53:43.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.652+0000 7f62657c3e00 1 -- 10.20.193.28:0/763225321 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 msgr2=0x564c27a03e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.652+0000 7f62657c3e00 1 --2- 10.20.193.28:0/763225321 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 0x564c27a03e90 secure :-1 s=READY pgs=663 gs=2 cs=0 l=1 c_cookie=5997383911027991430 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6250004430 tx=0x7f6250011000 comp rx=0 tx=0).stop 2026-02-25T11:53:43.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.652+0000 7f62657c3e00 1 -- 10.20.193.28:0/763225321 shutdown_connections 2026-02-25T11:53:43.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.652+0000 7f62657c3e00 1 --2- 10.20.193.28:0/763225321 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 0x564c27a043d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.652+0000 7f62657c3e00 1 --2- 10.20.193.28:0/763225321 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279c0460 0x564c27a03e90 unknown :-1 s=CLOSED pgs=663 gs=2 cs=0 l=1 c_cookie=5997383911027991430 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.652+0000 7f62657c3e00 1 --2- 10.20.193.28:0/763225321 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279bfb50 0x564c27a1a570 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.652+0000 7f62657c3e00 1 -- 10.20.193.28:0/763225321 >> 10.20.193.28:0/763225321 conn(0x564c278d6d40 msgr2=0x564c27a18e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:43.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.652+0000 7f62657c3e00 1 -- 10.20.193.28:0/763225321 shutdown_connections 2026-02-25T11:53:43.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.652+0000 7f62657c3e00 1 -- 10.20.193.28:0/763225321 wait complete. 2026-02-25T11:53:43.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.652+0000 7f62657c3e00 1 -- start start 2026-02-25T11:53:43.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f62657c3e00 1 Processor -- start 2026-02-25T11:53:43.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f62657c3e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279bfb50 0x564c27996910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f62657c3e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279c0460 0x564c27996e50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f62657c3e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 0x564c27997390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f62657c3e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x564c279516b0 con 0x564c279bfb50 2026-02-25T11:53:43.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f62657c3e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x564c27a6ad10 con 0x564c27a05100 2026-02-25T11:53:43.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f62657c3e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x564c27a6aed0 con 0x564c279c0460 2026-02-25T11:53:43.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f626153d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279bfb50 0x564c27996910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f626153d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279bfb50 0x564c27996910 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:37454/0 (socket says 10.20.193.28:37454) 2026-02-25T11:53:43.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f626153d640 1 -- 10.20.193.28:0/2250521486 learned_addr learned my addr 10.20.193.28:0/2250521486 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:53:43.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f6260d3c640 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279c0460 0x564c27996e50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f626153d640 1 -- 10.20.193.28:0/2250521486 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279c0460 msgr2=0x564c27996e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.661 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f626153d640 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279c0460 0x564c27996e50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f6261d3e640 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 0x564c27997390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f626153d640 1 -- 10.20.193.28:0/2250521486 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 msgr2=0x564c27997390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:53:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.653+0000 7f626153d640 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 0x564c27997390 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f626153d640 1 -- 10.20.193.28:0/2250521486 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c27859150 con 0x564c279bfb50 2026-02-25T11:53:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f6261d3e640 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 0x564c27997390 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:53:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f626153d640 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279bfb50 0x564c27996910 secure :-1 s=READY pgs=664 gs=2 cs=0 l=1 c_cookie=12424676380153447567 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f624c0ace00 tx=0x7f624c0076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6d5156c83b848f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:53:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f6212ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:53:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f6212ffd640 1 -- 10.20.193.28:0/2250521486 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f624c0acc40 con 0x564c279bfb50 2026-02-25T11:53:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f62657c3e00 1 -- 10.20.193.28:0/2250521486 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62500040d0 con 0x564c279bfb50 2026-02-25T11:53:43.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f62657c3e00 10 client.16653 _ll_register_callbacks cb 0x564c278449a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:53:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f62657c3e00 1 -- 10.20.193.28:0/2250521486 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f624c0026a0 con 0x564c279bfb50 2026-02-25T11:53:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f6212ffd640 1 -- 10.20.193.28:0/2250521486 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f624c007ba0 con 0x564c279bfb50 2026-02-25T11:53:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f6212ffd640 1 -- 10.20.193.28:0/2250521486 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f624c0adaa0 con 0x564c279bfb50 2026-02-25T11:53:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f6212ffd640 1 -- 10.20.193.28:0/2250521486 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f624c0b7840 con 0x564c279bfb50 2026-02-25T11:53:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f6212ffd640 1 -- 10.20.193.28:0/2250521486 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f624c0b7a20 con 0x564c279bfb50 2026-02-25T11:53:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f62657c3e00 10 client.16653 fetch_fsmap learned FSMap version 23 2026-02-25T11:53:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f62657c3e00 1 -- 10.20.193.28:0/2250521486 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x564c27853bd0 con 0x564c279bfb50 2026-02-25T11:53:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f6212ffd640 1 -- 10.20.193.28:0/2250521486 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f624c071070 con 0x564c279bfb50 2026-02-25T11:53:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f62657c3e00 10 client.16653 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:53:43.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f62657c3e00 10 client.16653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:53:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f62657c3e00 20 client.16653 populate_metadata read hostname 'trial028' 2026-02-25T11:53:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f62657c3e00 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564c27ab2ab0 0x564c27ad2f60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:53:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f62657c3e00 4 client.16653 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:53:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.654+0000 7f62657c3e00 1 -- 10.20.193.28:0/2250521486 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564c27556f40 con 0x564c27ab2ab0 2026-02-25T11:53:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.655+0000 7f6260d3c640 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564c27ab2ab0 0x564c27ad2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:53:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f6260d3c640 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564c27ab2ab0 0x564c27ad2f60 crc :-1 s=READY pgs=113 gs=4 cs=0 l=0 c_cookie=7391649190113761368 s_cookie=12193646458177672318 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=669469c69908d858 server_cookie=a93888d95861547e in_seq=0 out_seq=0 2026-02-25T11:53:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f6212ffd640 10 client.16653 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f6212ffd640 1 -- 10.20.193.28:0/2250521486 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x564c27556f40 con 0x564c27ab2ab0 2026-02-25T11:53:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f6212ffd640 10 client.16653 handle_command_reply: tid=0 2026-02-25T11:53:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f62657c3e00 1 client.16653 shutdown 2026-02-25T11:53:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f62657c3e00 1 -- 10.20.193.28:0/2250521486 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279bfb50 msgr2=0x564c27996910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f62657c3e00 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279bfb50 0x564c27996910 secure :-1 s=READY pgs=664 gs=2 cs=0 l=1 c_cookie=12424676380153447567 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f624c0ace00 tx=0x7f624c0076c0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f62657c3e00 1 -- 10.20.193.28:0/2250521486 shutdown_connections 2026-02-25T11:53:43.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f62657c3e00 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x564c27ab2ab0 0x564c27ad2f60 crc :-1 s=READY pgs=113 gs=4 cs=0 l=0 c_cookie=7391649190113761368 s_cookie=12193646458177672318 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f62657c3e00 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x564c27a05100 0x564c27997390 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f62657c3e00 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x564c279c0460 0x564c27996e50 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f62657c3e00 1 --2- 10.20.193.28:0/2250521486 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x564c279bfb50 0x564c27996910 unknown :-1 s=CLOSED pgs=664 gs=2 cs=0 l=1 c_cookie=12424676380153447567 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f62657c3e00 1 -- 10.20.193.28:0/2250521486 >> 10.20.193.28:0/2250521486 conn(0x564c278d6d40 msgr2=0x564c27a19640 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:43.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f6212ffd640 0 client.16653 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:53:43.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.656+0000 7f62657c3e00 1 -- 10.20.193.28:0/2250521486 shutdown_connections 2026-02-25T11:53:43.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62657c3e00 1 -- 10.20.193.28:0/2250521486 wait complete. 2026-02-25T11:53:43.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62657c3e00 20 client.16653 trim_cache size 0 max 16384 2026-02-25T11:53:43.666 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:53:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:53:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:53:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:53:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "0d9e5421-597c-4231-88a4-8abf309c266b", 2026-02-25T11:53:43.666 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:53:43.666 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:53:43.667 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:53:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:53:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:53:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:43.667 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:43.667 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:53:43.668 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:53:43.668 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:43.668 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:43.668 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:43.668 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:43.668 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:43.668 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:43.668 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:53:43.668 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:53:43.668 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:43.668 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:43.668 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:43.669 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:43.669 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:43.669 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:43.669 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:53:43.669 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:53:43.669 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:43.669 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:43.669 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:43.669 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:43.669 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:43.669 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:43.669 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:53:43.670 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:53:43.670 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:43.670 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:43.670 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:43.670 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:43.670 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:43.670 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:43.670 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:53:43.670 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:53:43.670 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:43.670 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:43.670 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:43.670 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:43.671 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:43.672 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:43.672 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:43.672 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:43.672 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:43.672 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:53:43.672 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:53:43.672 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:43.672 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:43.672 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:43.672 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:43.672 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:43.672 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:43.672 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:43.673 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:43.674 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:53:43.674 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:53:43.674 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:53:43.674 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:53:43.674 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:53:43.674 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:53:43.674 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:53:43.674 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:53:43.674 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:53:43.674 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:53:43.674 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:53:43.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62637c8640 1 -- 10.20.193.28:0/1241220605 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f62380b5760 msgr2=0x7f62380d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62637c8640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f62380b5760 0x7f62380d5c10 secure :-1 s=READY pgs=330 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6250005fd0 tx=0x7f625000a580 comp rx=0 tx=0).stop 2026-02-25T11:53:43.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62637c8640 1 -- 10.20.193.28:0/1241220605 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f625c10f740 msgr2=0x7f625c0f5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:53:43.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62637c8640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f625c10f740 0x7f625c0f5ab0 secure :-1 s=READY pgs=661 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f624c00c9b0 tx=0x7f624c00ce80 comp rx=0 tx=0).stop 2026-02-25T11:53:43.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62637c8640 1 -- 10.20.193.28:0/1241220605 shutdown_connections 2026-02-25T11:53:43.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62637c8640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f62380b5760 0x7f62380d5c10 unknown :-1 s=CLOSED pgs=330 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62637c8640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f625c115930 0x7f625c0f6140 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62637c8640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f625c110130 0x7f625c176200 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62637c8640 1 --2- 10.20.193.28:0/1241220605 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f625c10f740 0x7f625c0f5ab0 unknown :-1 s=CLOSED pgs=661 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:53:43.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62637c8640 1 -- 10.20.193.28:0/1241220605 >> 10.20.193.28:0/1241220605 conn(0x7f625c0ecff0 msgr2=0x7f625c10dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:53:43.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62637c8640 1 -- 10.20.193.28:0/1241220605 shutdown_connections 2026-02-25T11:53:43.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:53:43.657+0000 7f62637c8640 1 -- 10.20.193.28:0/1241220605 wait complete. 2026-02-25T11:53:43.758 INFO:tasks.cephfs.filesystem:scrub status for tag:0d9e5421-597c-4231-88a4-8abf309c266b - {'path': '/', 'tag': '0d9e5421-597c-4231-88a4-8abf309c266b', 'options': 'recursive,force'} 2026-02-25T11:53:44.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:44 trial167 ceph-mon[43392]: pgmap v391: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:44.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:44 trial119 ceph-mon[42710]: pgmap v391: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:44.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:44 trial028 ceph-mon[39224]: pgmap v391: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:46.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:46 trial167 ceph-mon[43392]: pgmap v392: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:46.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:46 trial119 ceph-mon[42710]: pgmap v392: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:46.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:46 trial028 ceph-mon[39224]: pgmap v392: 97 pgs: 97 active+clean; 4.4 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:53:47.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:47 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:53:47.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:47 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:53:47.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:47 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:53:48.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:48 trial167 ceph-mon[43392]: pgmap v393: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:48.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:48 trial119 ceph-mon[42710]: pgmap v393: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:48.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:48 trial028 ceph-mon[39224]: pgmap v393: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:53:50.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:50 trial167 ceph-mon[43392]: pgmap v394: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:53:50.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:50 trial119 ceph-mon[42710]: pgmap v394: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:53:50.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:50 trial028 ceph-mon[39224]: pgmap v394: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:53:52.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:52 trial167 ceph-mon[43392]: pgmap v395: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:53:52.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:52 trial119 ceph-mon[42710]: pgmap v395: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:53:52.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:52 trial028 ceph-mon[39224]: pgmap v395: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:53:54.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:54 trial167 ceph-mon[43392]: pgmap v396: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:53:54.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:54 trial119 ceph-mon[42710]: pgmap v396: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:53:54.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:54 trial028 ceph-mon[39224]: pgmap v396: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:53:56.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:56 trial167 ceph-mon[43392]: pgmap v397: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:53:56.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:56 trial119 ceph-mon[42710]: pgmap v397: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:53:56.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:56 trial028 ceph-mon[39224]: pgmap v397: 97 pgs: 97 active+clean; 4.5 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:53:58.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:53:58 trial167 ceph-mon[43392]: pgmap v398: 97 pgs: 97 active+clean; 4.6 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 13 op/s 2026-02-25T11:53:58.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:53:58 trial119 ceph-mon[42710]: pgmap v398: 97 pgs: 97 active+clean; 4.6 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 13 op/s 2026-02-25T11:53:58.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:53:58 trial028 ceph-mon[39224]: pgmap v398: 97 pgs: 97 active+clean; 4.6 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 17 KiB/s rd, 19 KiB/s wr, 13 op/s 2026-02-25T11:54:00.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:00 trial167 ceph-mon[43392]: pgmap v399: 97 pgs: 97 active+clean; 4.6 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 19 KiB/s wr, 13 op/s 2026-02-25T11:54:00.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:00 trial119 ceph-mon[42710]: pgmap v399: 97 pgs: 97 active+clean; 4.6 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 19 KiB/s wr, 13 op/s 2026-02-25T11:54:00.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:00 trial028 ceph-mon[39224]: pgmap v399: 97 pgs: 97 active+clean; 4.6 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 19 KiB/s wr, 13 op/s 2026-02-25T11:54:02.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:02 trial167 ceph-mon[43392]: pgmap v400: 97 pgs: 97 active+clean; 4.6 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:54:02.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:02 trial119 ceph-mon[42710]: pgmap v400: 97 pgs: 97 active+clean; 4.6 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:54:02.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:02 trial028 ceph-mon[39224]: pgmap v400: 97 pgs: 97 active+clean; 4.6 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:54:04.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:04 trial167 ceph-mon[43392]: pgmap v401: 97 pgs: 97 active+clean; 4.6 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 13 op/s 2026-02-25T11:54:04.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:04 trial119 ceph-mon[42710]: pgmap v401: 97 pgs: 97 active+clean; 4.6 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 13 op/s 2026-02-25T11:54:04.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:04 trial028 ceph-mon[39224]: pgmap v401: 97 pgs: 97 active+clean; 4.6 MiB data, 342 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 13 op/s 2026-02-25T11:54:06.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:06 trial167 ceph-mon[43392]: pgmap v402: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 19 KiB/s wr, 12 op/s 2026-02-25T11:54:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:06 trial119 ceph-mon[42710]: pgmap v402: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 19 KiB/s wr, 12 op/s 2026-02-25T11:54:06.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:06 trial028 ceph-mon[39224]: pgmap v402: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 19 KiB/s wr, 12 op/s 2026-02-25T11:54:08.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:08 trial167 ceph-mon[43392]: pgmap v403: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 13 op/s 2026-02-25T11:54:08.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:08 trial119 ceph-mon[42710]: pgmap v403: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 13 op/s 2026-02-25T11:54:08.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:08 trial028 ceph-mon[39224]: pgmap v403: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 19 KiB/s wr, 13 op/s 2026-02-25T11:54:10.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:10 trial119 ceph-mon[42710]: pgmap v404: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:54:10.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:10 trial028 ceph-mon[39224]: pgmap v404: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:54:10.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:10 trial167 ceph-mon[43392]: pgmap v404: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:54:12.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:12 trial119 ceph-mon[42710]: pgmap v405: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:54:12.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:12 trial028 ceph-mon[39224]: pgmap v405: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:54:12.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:12 trial167 ceph-mon[43392]: pgmap v405: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:54:13.759 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:54:13.947 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:14.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.127+0000 7f11cf7db640 1 -- 10.20.193.28:0/488514801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11c8110130 msgr2=0x7f11c8115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.127+0000 7f11cf7db640 1 --2- 10.20.193.28:0/488514801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11c8110130 0x7f11c8115180 secure :-1 s=READY pgs=665 gs=1 cs=0 l=1 c_cookie=16394457953763900378 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11bc009950 tx=0x7f11bc02f190 comp rx=0 tx=0).stop 2026-02-25T11:54:14.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.127+0000 7f11cf7db640 1 -- 10.20.193.28:0/488514801 shutdown_connections 2026-02-25T11:54:14.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.127+0000 7f11cf7db640 1 --2- 10.20.193.28:0/488514801 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f11c8115930 0x7f11c8176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.127+0000 7f11cf7db640 1 --2- 10.20.193.28:0/488514801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11c8110130 0x7f11c8115180 unknown :-1 s=CLOSED pgs=665 gs=1 cs=0 l=1 c_cookie=16394457953763900378 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.127+0000 7f11cf7db640 1 --2- 10.20.193.28:0/488514801 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f11c810f740 0x7f11c810fb10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.127+0000 7f11cf7db640 1 -- 10.20.193.28:0/488514801 >> 10.20.193.28:0/488514801 conn(0x7f11c80ecff0 msgr2=0x7f11c810d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:14.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.127+0000 7f11cf7db640 1 -- 10.20.193.28:0/488514801 shutdown_connections 2026-02-25T11:54:14.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.127+0000 7f11cf7db640 1 -- 10.20.193.28:0/488514801 wait complete. 2026-02-25T11:54:14.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.128+0000 7f11cf7db640 1 Processor -- start 2026-02-25T11:54:14.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.128+0000 7f11cf7db640 1 -- start start 2026-02-25T11:54:14.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.128+0000 7f11cf7db640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11c810f740 0x7f11c80f5b10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.128+0000 7f11cf7db640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f11c8110130 0x7f11c8176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.128+0000 7f11cf7db640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f11c8115930 0x7f11c80f61a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.128+0000 7f11cf7db640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f11c80fb420 con 0x7f11c810f740 2026-02-25T11:54:14.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.128+0000 7f11cf7db640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f11c80fb590 con 0x7f11c8110130 2026-02-25T11:54:14.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.128+0000 7f11cf7db640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f11c80fb750 con 0x7f11c8115930 2026-02-25T11:54:14.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.128+0000 7f11cd550640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11c810f740 0x7f11c80f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.128+0000 7f11cd550640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11c810f740 0x7f11c80f5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52776/0 (socket says 10.20.193.28:52776) 2026-02-25T11:54:14.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.128+0000 7f11cd550640 1 -- 10.20.193.28:0/108650477 learned_addr learned my addr 10.20.193.28:0/108650477 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:14.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.128+0000 7f11ccd4f640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f11c8110130 0x7f11c8176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.128+0000 7f11cdd51640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f11c8115930 0x7f11c80f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.129+0000 7f11cd550640 1 -- 10.20.193.28:0/108650477 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f11c8115930 msgr2=0x7f11c80f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.129+0000 7f11cd550640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f11c8115930 0x7f11c80f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.129+0000 7f11cd550640 1 -- 10.20.193.28:0/108650477 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f11c8110130 msgr2=0x7f11c8176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.129+0000 7f11cd550640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f11c8110130 0x7f11c8176200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.129+0000 7f11cd550640 1 -- 10.20.193.28:0/108650477 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc009570 con 0x7f11c810f740 2026-02-25T11:54:14.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.129+0000 7f11cd550640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11c810f740 0x7f11c80f5b10 secure :-1 s=READY pgs=666 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11b80072a0 tx=0x7f11b800ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:14.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.129+0000 7f11b67fc640 1 -- 10.20.193.28:0/108650477 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11b8004500 con 0x7f11c810f740 2026-02-25T11:54:14.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.129+0000 7f11b67fc640 1 -- 10.20.193.28:0/108650477 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f11b800bd00 con 0x7f11c810f740 2026-02-25T11:54:14.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.129+0000 7f11b67fc640 1 -- 10.20.193.28:0/108650477 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11b80106e0 con 0x7f11c810f740 2026-02-25T11:54:14.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.129+0000 7f11cf7db640 1 -- 10.20.193.28:0/108650477 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c80fba30 con 0x7f11c810f740 2026-02-25T11:54:14.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.129+0000 7f11cf7db640 1 -- 10.20.193.28:0/108650477 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c80fbeb0 con 0x7f11c810f740 2026-02-25T11:54:14.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.132+0000 7f11b67fc640 1 -- 10.20.193.28:0/108650477 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f11b800b840 con 0x7f11c810f740 2026-02-25T11:54:14.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.134+0000 7f11b67fc640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f11a40b5760 0x7f11a40d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.136 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.134+0000 7f11b67fc640 1 -- 10.20.193.28:0/108650477 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f11b80e69d0 con 0x7f11c810f740 2026-02-25T11:54:14.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.134+0000 7f11ccd4f640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f11a40b5760 0x7f11a40d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11d17d6e00 1 Processor -- start 2026-02-25T11:54:14.137 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11d17d6e00 1 -- start start 2026-02-25T11:54:14.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11d17d6e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 0x5577b17d8f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11d17d6e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d9460 0x5577b181dbc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11d17d6e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b181e100 0x5577b183e4e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11d17d6e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5577b16e69b0 con 0x5577b17d8b50 2026-02-25T11:54:14.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11d17d6e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5577b16e26e0 con 0x5577b17d9460 2026-02-25T11:54:14.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11d17d6e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5577b16dc840 con 0x5577b181e100 2026-02-25T11:54:14.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11cd550640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 0x5577b17d8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11cd550640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 0x5577b17d8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52790/0 (socket says 10.20.193.28:52790) 2026-02-25T11:54:14.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11cd550640 1 -- 10.20.193.28:0/1389854061 learned_addr learned my addr 10.20.193.28:0/1389854061 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:14.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11cd550640 1 -- 10.20.193.28:0/1389854061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b181e100 msgr2=0x5577b183e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:14.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11cdd51640 1 --2- 10.20.193.28:0/1389854061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b181e100 0x5577b183e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11cd550640 1 --2- 10.20.193.28:0/1389854061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b181e100 0x5577b183e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11cd550640 1 -- 10.20.193.28:0/1389854061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d9460 msgr2=0x5577b181dbc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2026-02-25T11:54:14.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11cd550640 1 --2- 10.20.193.28:0/1389854061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d9460 0x5577b181dbc0 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11cd550640 1 -- 10.20.193.28:0/1389854061 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b1687d80 con 0x5577b17d8b50 2026-02-25T11:54:14.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.135+0000 7f11cdd51640 1 --2- 10.20.193.28:0/1389854061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b181e100 0x5577b183e4e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:54:14.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f11cd550640 1 --2- 10.20.193.28:0/1389854061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 0x5577b17d8f20 secure :-1 s=READY pgs=667 gs=1 cs=0 l=1 c_cookie=17667715462849027441 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11b80a93d0 tx=0x7f11b8002390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f530559b1bcadd71 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:14.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f1195ce5640 1 -- 10.20.193.28:0/1389854061 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11b80d20d0 con 0x5577b17d8b50 2026-02-25T11:54:14.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f1195ce5640 1 -- 10.20.193.28:0/1389854061 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f11b80a8ca0 con 0x5577b17d8b50 2026-02-25T11:54:14.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f1195ce5640 1 -- 10.20.193.28:0/1389854061 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11b80b2a70 con 0x5577b17d8b50 2026-02-25T11:54:14.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1389854061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 msgr2=0x5577b17d8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/1389854061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 0x5577b17d8f20 secure :-1 s=READY pgs=667 gs=1 cs=0 l=1 c_cookie=17667715462849027441 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11b80a93d0 tx=0x7f11b8002390 comp rx=0 tx=0).stop 2026-02-25T11:54:14.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f11ccd4f640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f11a40b5760 0x7f11a40d5c10 secure :-1 s=READY pgs=331 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11c80fc4b0 tx=0x7f11bc03a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:14.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1389854061 shutdown_connections 2026-02-25T11:54:14.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/1389854061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b181e100 0x5577b183e4e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/1389854061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d9460 0x5577b181dbc0 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/1389854061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 0x5577b17d8f20 unknown :-1 s=CLOSED pgs=667 gs=1 cs=0 l=1 c_cookie=17667715462849027441 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1389854061 >> 10.20.193.28:0/1389854061 conn(0x5577b16efd40 msgr2=0x5577b17d7430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:14.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1389854061 shutdown_connections 2026-02-25T11:54:14.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.136+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1389854061 wait complete. 2026-02-25T11:54:14.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.137+0000 7f11d17d6e00 1 -- start start 2026-02-25T11:54:14.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.138+0000 7f11d17d6e00 1 Processor -- start 2026-02-25T11:54:14.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.138+0000 7f11d17d6e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d8b50 0x5577b17e22d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.138+0000 7f11d17d6e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d9460 0x5577b17e2810 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.138+0000 7f11d17d6e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 0x5577b17e7800 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.138+0000 7f11d17d6e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5577b17e7d40 con 0x5577b181e100 2026-02-25T11:54:14.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.138+0000 7f11d17d6e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5577b17e7eb0 con 0x5577b17d8b50 2026-02-25T11:54:14.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.138+0000 7f11d17d6e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5577b17e8070 con 0x5577b17d9460 2026-02-25T11:54:14.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.138+0000 7f11cdd51640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 0x5577b17e7800 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.138+0000 7f11cd550640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d8b50 0x5577b17e22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.138+0000 7f11ccd4f640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d9460 0x5577b17e2810 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.138+0000 7f11cdd51640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 0x5577b17e7800 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52798/0 (socket says 10.20.193.28:52798) 2026-02-25T11:54:14.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.138+0000 7f11cdd51640 1 -- 10.20.193.28:0/2741784957 learned_addr learned my addr 10.20.193.28:0/2741784957 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:14.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.138+0000 7f11cd550640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d8b50 0x5577b17e22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:49298/0 (socket says 10.20.193.28:49298) 2026-02-25T11:54:14.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f11cdd51640 1 -- 10.20.193.28:0/2741784957 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d9460 msgr2=0x5577b17e2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:14.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f11cdd51640 1 --2- 10.20.193.28:0/2741784957 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d9460 0x5577b17e2810 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f11cdd51640 1 -- 10.20.193.28:0/2741784957 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d8b50 msgr2=0x5577b17e22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:14.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f11cdd51640 1 --2- 10.20.193.28:0/2741784957 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d8b50 0x5577b17e22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f11cdd51640 1 -- 10.20.193.28:0/2741784957 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b1660350 con 0x5577b181e100 2026-02-25T11:54:14.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f11cd550640 1 --2- 10.20.193.28:0/2741784957 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d8b50 0x5577b17e22d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:14.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f11cdd51640 1 --2- 10.20.193.28:0/2741784957 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 0x5577b17e7800 secure :-1 s=READY pgs=668 gs=2 cs=0 l=1 c_cookie=7513085923329907799 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11c4003e90 tx=0x7f11c4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6843d7d75b111c57 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:14.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f117effd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:14.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f117effd640 1 -- 10.20.193.28:0/2741784957 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11c40139a0 con 0x5577b181e100 2026-02-25T11:54:14.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f11d17d6e00 1 -- 10.20.193.28:0/2741784957 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c80fbeb0 con 0x5577b181e100 2026-02-25T11:54:14.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f117effd640 1 -- 10.20.193.28:0/2741784957 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f11c401c470 con 0x5577b181e100 2026-02-25T11:54:14.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f117effd640 1 -- 10.20.193.28:0/2741784957 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11c40106c0 con 0x5577b181e100 2026-02-25T11:54:14.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f117effd640 1 -- 10.20.193.28:0/2741784957 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f11c4010850 con 0x5577b181e100 2026-02-25T11:54:14.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.139+0000 7f117effd640 10 client.16665 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:54:14.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.140+0000 7f11d17d6e00 10 client.16665 _ll_register_callbacks cb 0x5577b165d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:14.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.140+0000 7f11d17d6e00 1 -- 10.20.193.28:0/2741784957 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5577b13726f0 con 0x5577b181e100 2026-02-25T11:54:14.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.140+0000 7f117effd640 1 -- 10.20.193.28:0/2741784957 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f11c401c990 con 0x5577b181e100 2026-02-25T11:54:14.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.140+0000 7f11d17d6e00 10 client.16665 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:14.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.140+0000 7f11d17d6e00 1 -- 10.20.193.28:0/2741784957 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5577b1672150 con 0x5577b181e100 2026-02-25T11:54:14.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.140+0000 7f117effd640 1 -- 10.20.193.28:0/2741784957 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f11c401c5e0 con 0x5577b181e100 2026-02-25T11:54:14.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.140+0000 7f11d17d6e00 10 client.16665 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:14.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.140+0000 7f11d17d6e00 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:14.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.140+0000 7f11d17d6e00 20 client.16665 populate_metadata read hostname 'trial028' 2026-02-25T11:54:14.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.140+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/2741784957 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5577b180cb50 0x5577b180cff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.140+0000 7f11d17d6e00 4 client.16665 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:54:14.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.140+0000 7f11d17d6e00 1 -- 10.20.193.28:0/2741784957 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577b16ae910 con 0x5577b180cb50 2026-02-25T11:54:14.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.140+0000 7f11cd550640 1 --2- 10.20.193.28:0/2741784957 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5577b180cb50 0x5577b180cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.141+0000 7f11cd550640 1 --2- 10.20.193.28:0/2741784957 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5577b180cb50 0x5577b180cff0 crc :-1 s=READY pgs=114 gs=4 cs=0 l=0 c_cookie=8944226329983993352 s_cookie=16337011380332440416 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c20467565644608 server_cookie=e2b8b94b103aab60 in_seq=0 out_seq=0 2026-02-25T11:54:14.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.141+0000 7f117effd640 10 client.16665 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:14.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.141+0000 7f117effd640 1 -- 10.20.193.28:0/2741784957 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5577b16ae910 con 0x5577b180cb50 2026-02-25T11:54:14.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.141+0000 7f117effd640 10 client.16665 handle_command_reply: tid=0 2026-02-25T11:54:14.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.142+0000 7f11d17d6e00 1 client.16665 shutdown 2026-02-25T11:54:14.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.142+0000 7f11d17d6e00 1 -- 10.20.193.28:0/2741784957 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 msgr2=0x5577b17e7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.142+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/2741784957 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 0x5577b17e7800 secure :-1 s=READY pgs=668 gs=2 cs=0 l=1 c_cookie=7513085923329907799 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11c4003e90 tx=0x7f11c4003f90 comp rx=0 tx=0).stop 2026-02-25T11:54:14.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.142+0000 7f11d17d6e00 1 -- 10.20.193.28:0/2741784957 shutdown_connections 2026-02-25T11:54:14.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.142+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/2741784957 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5577b180cb50 0x5577b180cff0 crc :-1 s=READY pgs=114 gs=4 cs=0 l=0 c_cookie=8944226329983993352 s_cookie=16337011380332440416 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.142+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/2741784957 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 0x5577b17e7800 unknown :-1 s=CLOSED pgs=668 gs=2 cs=0 l=1 c_cookie=7513085923329907799 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.142+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/2741784957 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d9460 0x5577b17e2810 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.142+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/2741784957 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d8b50 0x5577b17e22d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.142+0000 7f117effd640 0 client.16665 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:14.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.142+0000 7f11d17d6e00 1 -- 10.20.193.28:0/2741784957 >> 10.20.193.28:0/2741784957 conn(0x5577b16efd40 msgr2=0x5577b17d7430 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:14.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.142+0000 7f11d17d6e00 1 -- 10.20.193.28:0/2741784957 shutdown_connections 2026-02-25T11:54:14.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.142+0000 7f11d17d6e00 1 -- 10.20.193.28:0/2741784957 wait complete. 2026-02-25T11:54:14.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.142+0000 7f11d17d6e00 20 client.16665 trim_cache size 0 max 16384 2026-02-25T11:54:14.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.151+0000 7f11d17d6e00 1 Processor -- start 2026-02-25T11:54:14.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.151+0000 7f11d17d6e00 1 -- start start 2026-02-25T11:54:14.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.151+0000 7f11d17d6e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d8b50 0x5577b17dfbc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.151+0000 7f11d17d6e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d9460 0x5577b181e4d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.151+0000 7f11d17d6e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 0x5577b18310e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.151+0000 7f11d17d6e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5577b16da470 con 0x5577b181e100 2026-02-25T11:54:14.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.151+0000 7f11d17d6e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5577b139a700 con 0x5577b17d9460 2026-02-25T11:54:14.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.151+0000 7f11d17d6e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5577b1809400 con 0x5577b17d8b50 2026-02-25T11:54:14.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.152+0000 7f11cdd51640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 0x5577b18310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.152+0000 7f11ccd4f640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d9460 0x5577b181e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.152+0000 7f11cdd51640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 0x5577b18310e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52812/0 (socket says 10.20.193.28:52812) 2026-02-25T11:54:14.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.152+0000 7f11cdd51640 1 -- 10.20.193.28:0/3819595478 learned_addr learned my addr 10.20.193.28:0/3819595478 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:14.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.152+0000 7f11cd550640 1 --2- 10.20.193.28:0/3819595478 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d8b50 0x5577b17dfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.152+0000 7f11cdd51640 1 -- 10.20.193.28:0/3819595478 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d8b50 msgr2=0x5577b17dfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:14.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.152+0000 7f11cdd51640 1 --2- 10.20.193.28:0/3819595478 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d8b50 0x5577b17dfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.152+0000 7f11cdd51640 1 -- 10.20.193.28:0/3819595478 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d9460 msgr2=0x5577b181e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:14.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.152+0000 7f11cdd51640 1 --2- 10.20.193.28:0/3819595478 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d9460 0x5577b181e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.152+0000 7f11cdd51640 1 -- 10.20.193.28:0/3819595478 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b181eb30 con 0x5577b181e100 2026-02-25T11:54:14.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.152+0000 7f11cd550640 1 --2- 10.20.193.28:0/3819595478 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d8b50 0x5577b17dfbc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:54:14.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.152+0000 7f11cdd51640 1 --2- 10.20.193.28:0/3819595478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 0x5577b18310e0 secure :-1 s=READY pgs=669 gs=3 cs=0 l=1 c_cookie=14068811828177885740 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11c40034b0 tx=0x7f11c4015070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33e75d1665b5a2c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:14.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.153+0000 7f11954e4640 1 -- 10.20.193.28:0/3819595478 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11c401c950 con 0x5577b181e100 2026-02-25T11:54:14.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.153+0000 7f11954e4640 1 -- 10.20.193.28:0/3819595478 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5577b13726f0 con 0x5577b181e100 2026-02-25T11:54:14.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.153+0000 7f11954e4640 1 -- 10.20.193.28:0/3819595478 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11c401c6f0 con 0x5577b181e100 2026-02-25T11:54:14.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.153+0000 7f11d17d6e00 1 -- 10.20.193.28:0/3819595478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 msgr2=0x5577b18310e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.153+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/3819595478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 0x5577b18310e0 secure :-1 s=READY pgs=669 gs=3 cs=0 l=1 c_cookie=14068811828177885740 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11c40034b0 tx=0x7f11c4015070 comp rx=0 tx=0).stop 2026-02-25T11:54:14.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.153+0000 7f11d17d6e00 1 -- 10.20.193.28:0/3819595478 shutdown_connections 2026-02-25T11:54:14.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.153+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/3819595478 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b181e100 0x5577b18310e0 unknown :-1 s=CLOSED pgs=669 gs=3 cs=0 l=1 c_cookie=14068811828177885740 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.153+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/3819595478 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b17d9460 0x5577b181e4d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.153+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/3819595478 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d8b50 0x5577b17dfbc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.153+0000 7f11d17d6e00 1 -- 10.20.193.28:0/3819595478 >> 10.20.193.28:0/3819595478 conn(0x5577b16efd40 msgr2=0x5577b17de4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:14.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.153+0000 7f11d17d6e00 1 -- 10.20.193.28:0/3819595478 shutdown_connections 2026-02-25T11:54:14.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.153+0000 7f11d17d6e00 1 -- 10.20.193.28:0/3819595478 wait complete. 2026-02-25T11:54:14.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11d17d6e00 1 -- start start 2026-02-25T11:54:14.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11d17d6e00 1 Processor -- start 2026-02-25T11:54:14.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11d17d6e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 0x5577b187b100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11d17d6e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d9460 0x5577b187b660 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11d17d6e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b181e100 0x5577b1812a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11d17d6e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5577b183d990 con 0x5577b17d8b50 2026-02-25T11:54:14.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11d17d6e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5577b1812fe0 con 0x5577b181e100 2026-02-25T11:54:14.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11d17d6e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5577b18131a0 con 0x5577b17d9460 2026-02-25T11:54:14.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11cd550640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 0x5577b187b100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11cd550640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 0x5577b187b100 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52820/0 (socket says 10.20.193.28:52820) 2026-02-25T11:54:14.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11cd550640 1 -- 10.20.193.28:0/1115928482 learned_addr learned my addr 10.20.193.28:0/1115928482 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:14.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11cd550640 1 -- 10.20.193.28:0/1115928482 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d9460 msgr2=0x5577b187b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:14.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11cd550640 1 --2- 10.20.193.28:0/1115928482 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d9460 0x5577b187b660 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11cd550640 1 -- 10.20.193.28:0/1115928482 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b181e100 msgr2=0x5577b1812a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:14.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11cd550640 1 --2- 10.20.193.28:0/1115928482 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b181e100 0x5577b1812a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.154+0000 7f11cd550640 1 -- 10.20.193.28:0/1115928482 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b166cbd0 con 0x5577b17d8b50 2026-02-25T11:54:14.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.155+0000 7f11cd550640 1 --2- 10.20.193.28:0/1115928482 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 0x5577b187b100 secure :-1 s=READY pgs=670 gs=1 cs=0 l=1 c_cookie=8654772605905551455 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11b80a8e10 tx=0x7f11b8096040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781bedd482c8405f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:14.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.155+0000 7f117effd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:14.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.155+0000 7f117effd640 1 -- 10.20.193.28:0/1115928482 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11b80e4310 con 0x5577b17d8b50 2026-02-25T11:54:14.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.155+0000 7f117effd640 1 -- 10.20.193.28:0/1115928482 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f11b80ac600 con 0x5577b17d8b50 2026-02-25T11:54:14.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.155+0000 7f117effd640 1 -- 10.20.193.28:0/1115928482 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f11b80b7c80 con 0x5577b17d8b50 2026-02-25T11:54:14.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.155+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1115928482 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c402b040 con 0x5577b17d8b50 2026-02-25T11:54:14.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.155+0000 7f11d17d6e00 10 client.16671 _ll_register_callbacks cb 0x5577b165d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:14.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.155+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1115928482 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b80a8ca0 con 0x5577b17d8b50 2026-02-25T11:54:14.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.155+0000 7f117effd640 1 -- 10.20.193.28:0/1115928482 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f11b8071070 con 0x5577b17d8b50 2026-02-25T11:54:14.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.155+0000 7f117effd640 1 -- 10.20.193.28:0/1115928482 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f11b80722d0 con 0x5577b17d8b50 2026-02-25T11:54:14.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.155+0000 7f11d17d6e00 10 client.16671 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:14.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.155+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1115928482 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5577b1660680 con 0x5577b17d8b50 2026-02-25T11:54:14.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.156+0000 7f117effd640 1 -- 10.20.193.28:0/1115928482 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f11b80aade0 con 0x5577b17d8b50 2026-02-25T11:54:14.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.156+0000 7f11d17d6e00 10 client.16671 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:14.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.156+0000 7f11d17d6e00 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:14.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.156+0000 7f11d17d6e00 20 client.16671 populate_metadata read hostname 'trial028' 2026-02-25T11:54:14.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.156+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/1115928482 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5577b18dd240 0x5577b18fd6f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.156+0000 7f11d17d6e00 4 client.16671 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:54:14.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.156+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1115928482 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5577b136ff40 con 0x5577b18dd240 2026-02-25T11:54:14.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.156+0000 7f11ccd4f640 1 --2- 10.20.193.28:0/1115928482 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5577b18dd240 0x5577b18fd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.157+0000 7f117effd640 10 client.16671 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:14.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.157+0000 7f11ccd4f640 1 --2- 10.20.193.28:0/1115928482 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5577b18dd240 0x5577b18fd6f0 crc :-1 s=READY pgs=115 gs=4 cs=0 l=0 c_cookie=13097634697290460037 s_cookie=6841685233369083384 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c4255ce9d42b85 server_cookie=5ef28c788fffedf8 in_seq=0 out_seq=0 2026-02-25T11:54:14.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.157+0000 7f117effd640 1 -- 10.20.193.28:0/1115928482 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x5577b136ff40 con 0x5577b18dd240 2026-02-25T11:54:14.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.157+0000 7f117effd640 10 client.16671 handle_command_reply: tid=0 2026-02-25T11:54:14.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.157+0000 7f11d17d6e00 1 client.16671 shutdown 2026-02-25T11:54:14.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1115928482 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 msgr2=0x5577b187b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/1115928482 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 0x5577b187b100 secure :-1 s=READY pgs=670 gs=1 cs=0 l=1 c_cookie=8654772605905551455 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11b80a8e10 tx=0x7f11b8096040 comp rx=0 tx=0).stop 2026-02-25T11:54:14.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1115928482 shutdown_connections 2026-02-25T11:54:14.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/1115928482 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5577b18dd240 0x5577b18fd6f0 crc :-1 s=READY pgs=115 gs=4 cs=0 l=0 c_cookie=13097634697290460037 s_cookie=6841685233369083384 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/1115928482 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5577b181e100 0x5577b1812a20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/1115928482 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5577b17d9460 0x5577b187b660 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11d17d6e00 1 --2- 10.20.193.28:0/1115928482 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5577b17d8b50 0x5577b187b100 unknown :-1 s=CLOSED pgs=670 gs=1 cs=0 l=1 c_cookie=8654772605905551455 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f117effd640 0 client.16671 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:14.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1115928482 >> 10.20.193.28:0/1115928482 conn(0x5577b16efd40 msgr2=0x5577b17dec90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:14.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1115928482 shutdown_connections 2026-02-25T11:54:14.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11d17d6e00 1 -- 10.20.193.28:0/1115928482 wait complete. 2026-02-25T11:54:14.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11d17d6e00 20 client.16671 trim_cache size 0 max 16384 2026-02-25T11:54:14.168 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:54:14.168 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:54:14.168 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:54:14.168 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:54:14.168 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:54:14.168 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:54:14.168 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:14.169 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:14.169 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:14.169 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:14.169 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:14.169 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:14.169 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:54:14.169 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:54:14.169 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:14.169 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:14.169 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:14.170 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:14.170 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:14.170 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:14.170 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:54:14.170 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:54:14.170 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:14.170 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:14.170 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:14.170 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:14.170 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:14.171 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:14.171 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:54:14.171 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:54:14.171 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:14.171 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:14.171 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:14.171 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:14.171 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:14.171 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:14.171 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:54:14.171 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:54:14.172 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:14.172 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:14.172 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:14.172 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:14.172 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:14.172 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:14.172 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:54:14.172 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:54:14.172 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:14.172 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:14.173 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:14.173 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:14.173 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:14.173 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:14.173 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:54:14.173 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:54:14.173 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:14.173 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:14.173 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:14.173 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:14.173 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:14.174 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:14.174 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:54:14.174 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:54:14.174 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:14.174 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:14.174 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:14.174 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:14.174 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:14.174 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:14.174 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:54:14.174 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:54:14.174 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:14.174 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:54:14.175 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:14.176 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:54:14.177 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:54:14.177 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:54:14.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11cf7db640 1 -- 10.20.193.28:0/108650477 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f11a40b5760 msgr2=0x7f11a40d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11cf7db640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f11a40b5760 0x7f11a40d5c10 secure :-1 s=READY pgs=331 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11c80fc4b0 tx=0x7f11bc03a040 comp rx=0 tx=0).stop 2026-02-25T11:54:14.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11cf7db640 1 -- 10.20.193.28:0/108650477 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11c810f740 msgr2=0x7f11c80f5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.158+0000 7f11cf7db640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11c810f740 0x7f11c80f5b10 secure :-1 s=READY pgs=666 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f11b80072a0 tx=0x7f11b800ddf0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.159+0000 7f11cf7db640 1 -- 10.20.193.28:0/108650477 shutdown_connections 2026-02-25T11:54:14.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.159+0000 7f11cf7db640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f11a40b5760 0x7f11a40d5c10 unknown :-1 s=CLOSED pgs=331 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.159+0000 7f11cf7db640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f11c8115930 0x7f11c80f61a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.159+0000 7f11cf7db640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f11c8110130 0x7f11c8176200 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.159+0000 7f11cf7db640 1 --2- 10.20.193.28:0/108650477 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f11c810f740 0x7f11c80f5b10 unknown :-1 s=CLOSED pgs=666 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.159+0000 7f11cf7db640 1 -- 10.20.193.28:0/108650477 >> 10.20.193.28:0/108650477 conn(0x7f11c80ecff0 msgr2=0x7f11c8111f40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:14.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.159+0000 7f11cf7db640 1 -- 10.20.193.28:0/108650477 shutdown_connections 2026-02-25T11:54:14.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.159+0000 7f11cf7db640 1 -- 10.20.193.28:0/108650477 wait complete. 2026-02-25T11:54:14.259 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:54:14.259 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:54:14.445 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:14.488 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:14 trial028 ceph-mon[39224]: pgmap v406: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:54:14.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:14 trial119 ceph-mon[42710]: pgmap v406: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:54:14.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.621+0000 7fe2a947b640 1 -- 10.20.193.28:0/3667023765 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe2a4115a30 msgr2=0x7fe2a4176400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.621+0000 7fe2a947b640 1 --2- 10.20.193.28:0/3667023765 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe2a4115a30 0x7fe2a4176400 secure :-1 s=READY pgs=671 gs=1 cs=0 l=1 c_cookie=17843016165358207211 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe29400aa30 tx=0x7fe294030270 comp rx=0 tx=0).stop 2026-02-25T11:54:14.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.621+0000 7fe2a947b640 1 -- 10.20.193.28:0/3667023765 shutdown_connections 2026-02-25T11:54:14.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.621+0000 7fe2a947b640 1 --2- 10.20.193.28:0/3667023765 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe2a4115a30 0x7fe2a4176400 unknown :-1 s=CLOSED pgs=671 gs=1 cs=0 l=1 c_cookie=17843016165358207211 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.621+0000 7fe2a947b640 1 --2- 10.20.193.28:0/3667023765 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe2a4110230 0x7fe2a4115280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.621+0000 7fe2a947b640 1 --2- 10.20.193.28:0/3667023765 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe2a410f840 0x7fe2a410fc10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.621+0000 7fe2a947b640 1 -- 10.20.193.28:0/3667023765 >> 10.20.193.28:0/3667023765 conn(0x7fe2a40ed130 msgr2=0x7fe2a410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:14.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.621+0000 7fe2a947b640 1 -- 10.20.193.28:0/3667023765 shutdown_connections 2026-02-25T11:54:14.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.621+0000 7fe2a947b640 1 -- 10.20.193.28:0/3667023765 wait complete. 2026-02-25T11:54:14.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.622+0000 7fe2a947b640 1 Processor -- start 2026-02-25T11:54:14.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.622+0000 7fe2a947b640 1 -- start start 2026-02-25T11:54:14.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.622+0000 7fe2a947b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe2a410f840 0x7fe2a411e180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.622+0000 7fe2a947b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe2a4110230 0x7fe2a411e6c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.622+0000 7fe2a947b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe2a4115a30 0x7fe2a4123a90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.622+0000 7fe2a947b640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a41240b0 con 0x7fe2a4110230 2026-02-25T11:54:14.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.622+0000 7fe2a947b640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a4124220 con 0x7fe2a410f840 2026-02-25T11:54:14.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.622+0000 7fe2a947b640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a41243e0 con 0x7fe2a4115a30 2026-02-25T11:54:14.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.622+0000 7fe2a27fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe2a4110230 0x7fe2a411e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.622+0000 7fe2a27fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe2a4110230 0x7fe2a411e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52838/0 (socket says 10.20.193.28:52838) 2026-02-25T11:54:14.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.622+0000 7fe2a2ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe2a410f840 0x7fe2a411e180 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.622+0000 7fe2a27fc640 1 -- 10.20.193.28:0/441350676 learned_addr learned my addr 10.20.193.28:0/441350676 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:14.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.622+0000 7fe2a37fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe2a4115a30 0x7fe2a4123a90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.623+0000 7fe2a37fe640 1 -- 10.20.193.28:0/441350676 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe2a410f840 msgr2=0x7fe2a411e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.623+0000 7fe2a37fe640 1 --2- 10.20.193.28:0/441350676 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe2a410f840 0x7fe2a411e180 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.623+0000 7fe2a37fe640 1 -- 10.20.193.28:0/441350676 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe2a4110230 msgr2=0x7fe2a411e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.623+0000 7fe2a37fe640 1 --2- 10.20.193.28:0/441350676 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe2a4110230 0x7fe2a411e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.623+0000 7fe2a37fe640 1 -- 10.20.193.28:0/441350676 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29400a650 con 0x7fe2a4115a30 2026-02-25T11:54:14.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.623+0000 7fe2a27fc640 1 --2- 10.20.193.28:0/441350676 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe2a4110230 0x7fe2a411e6c0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:14.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.623+0000 7fe2a37fe640 1 --2- 10.20.193.28:0/441350676 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe2a4115a30 0x7fe2a4123a90 secure :-1 s=READY pgs=182 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe294033040 tx=0x7fe294030d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:14.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.623+0000 7fe283fff640 1 -- 10.20.193.28:0/441350676 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe29403e070 con 0x7fe2a4115a30 2026-02-25T11:54:14.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.623+0000 7fe283fff640 1 -- 10.20.193.28:0/441350676 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe294039de0 con 0x7fe2a4115a30 2026-02-25T11:54:14.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.623+0000 7fe2a947b640 1 -- 10.20.193.28:0/441350676 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a4124660 con 0x7fe2a4115a30 2026-02-25T11:54:14.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.623+0000 7fe283fff640 1 -- 10.20.193.28:0/441350676 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe2940427b0 con 0x7fe2a4115a30 2026-02-25T11:54:14.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.623+0000 7fe2a947b640 1 -- 10.20.193.28:0/441350676 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a4164dc0 con 0x7fe2a4115a30 2026-02-25T11:54:14.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.624+0000 7fe2a947b640 1 -- 10.20.193.28:0/441350676 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a4124930 con 0x7fe2a4115a30 2026-02-25T11:54:14.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.627+0000 7fe283fff640 1 -- 10.20.193.28:0/441350676 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fe2940052e0 con 0x7fe2a4115a30 2026-02-25T11:54:14.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.628+0000 7fe283fff640 1 --2- 10.20.193.28:0/441350676 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe2780b5760 0x7fe2780d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:14.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.628+0000 7fe283fff640 1 -- 10.20.193.28:0/441350676 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fe29410c480 con 0x7fe2a4115a30 2026-02-25T11:54:14.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.629+0000 7fe2a2ffd640 1 --2- 10.20.193.28:0/441350676 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe2780b5760 0x7fe2780d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:14.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.629+0000 7fe283fff640 1 -- 10.20.193.28:0/441350676 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fe29410c900 con 0x7fe2a4115a30 2026-02-25T11:54:14.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.629+0000 7fe2a2ffd640 1 --2- 10.20.193.28:0/441350676 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe2780b5760 0x7fe2780d5c10 secure :-1 s=READY pgs=332 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe298004740 tx=0x7fe298005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:14.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:14 trial167 ceph-mon[43392]: pgmap v406: 97 pgs: 97 active+clean; 4.7 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:54:14.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.773+0000 7fe2a947b640 1 -- 10.20.193.28:0/441350676 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2a411f2e0 con 0x7fe2a4115a30 2026-02-25T11:54:14.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.773+0000 7fe283fff640 1 -- 10.20.193.28:0/441350676 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7fe2940cef00 con 0x7fe2a4115a30 2026-02-25T11:54:14.776 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:54:14.776 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:54:14.776 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:54:14.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.775+0000 7fe2a947b640 1 -- 10.20.193.28:0/441350676 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe2780b5760 msgr2=0x7fe2780d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.775+0000 7fe2a947b640 1 --2- 10.20.193.28:0/441350676 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe2780b5760 0x7fe2780d5c10 secure :-1 s=READY pgs=332 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe298004740 tx=0x7fe298005f90 comp rx=0 tx=0).stop 2026-02-25T11:54:14.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.775+0000 7fe2a947b640 1 -- 10.20.193.28:0/441350676 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe2a4115a30 msgr2=0x7fe2a4123a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:14.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.775+0000 7fe2a947b640 1 --2- 10.20.193.28:0/441350676 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe2a4115a30 0x7fe2a4123a90 secure :-1 s=READY pgs=182 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe294033040 tx=0x7fe294030d00 comp rx=0 tx=0).stop 2026-02-25T11:54:14.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.775+0000 7fe2a947b640 1 -- 10.20.193.28:0/441350676 shutdown_connections 2026-02-25T11:54:14.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.775+0000 7fe2a947b640 1 --2- 10.20.193.28:0/441350676 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe2780b5760 0x7fe2780d5c10 unknown :-1 s=CLOSED pgs=332 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.775+0000 7fe2a947b640 1 --2- 10.20.193.28:0/441350676 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe2a4115a30 0x7fe2a4123a90 unknown :-1 s=CLOSED pgs=182 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.775+0000 7fe2a947b640 1 --2- 10.20.193.28:0/441350676 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe2a4110230 0x7fe2a411e6c0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.775+0000 7fe2a947b640 1 --2- 10.20.193.28:0/441350676 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe2a410f840 0x7fe2a411e180 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:14.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.775+0000 7fe2a947b640 1 -- 10.20.193.28:0/441350676 >> 10.20.193.28:0/441350676 conn(0x7fe2a40ed130 msgr2=0x7fe2a410d510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:14.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.775+0000 7fe2a947b640 1 -- 10.20.193.28:0/441350676 shutdown_connections 2026-02-25T11:54:14.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:14.775+0000 7fe2a947b640 1 -- 10.20.193.28:0/441350676 wait complete. 2026-02-25T11:54:14.907 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:54:15.091 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:15.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.263+0000 7f465f377640 1 -- 10.20.193.28:0/1166170060 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4658115880 msgr2=0x7f4658176250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.263+0000 7f465f377640 1 --2- 10.20.193.28:0/1166170060 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4658115880 0x7f4658176250 secure :-1 s=READY pgs=672 gs=2 cs=0 l=1 c_cookie=8799381762371184844 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f465400aa30 tx=0x7f4654030270 comp rx=0 tx=0).stop 2026-02-25T11:54:15.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.263+0000 7f465f377640 1 -- 10.20.193.28:0/1166170060 shutdown_connections 2026-02-25T11:54:15.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.263+0000 7f465f377640 1 --2- 10.20.193.28:0/1166170060 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4658115880 0x7f4658176250 unknown :-1 s=CLOSED pgs=672 gs=2 cs=0 l=1 c_cookie=8799381762371184844 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.263+0000 7f465f377640 1 --2- 10.20.193.28:0/1166170060 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4658110080 0x7f46581150d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.263+0000 7f465f377640 1 --2- 10.20.193.28:0/1166170060 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f465810f690 0x7f465810fa60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.263+0000 7f465f377640 1 -- 10.20.193.28:0/1166170060 >> 10.20.193.28:0/1166170060 conn(0x7f46580ecf00 msgr2=0x7f465810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:15.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.263+0000 7f465f377640 1 -- 10.20.193.28:0/1166170060 shutdown_connections 2026-02-25T11:54:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.263+0000 7f465f377640 1 -- 10.20.193.28:0/1166170060 wait complete. 2026-02-25T11:54:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.264+0000 7f465f377640 1 Processor -- start 2026-02-25T11:54:15.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.264+0000 7f465f377640 1 -- start start 2026-02-25T11:54:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465f377640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f465810f690 0x7f465811e0b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465f377640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4658110080 0x7f465811e5f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465f377640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4658115880 0x7f46581239c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465f377640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4658123fe0 con 0x7f4658115880 2026-02-25T11:54:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465f377640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4658124150 con 0x7f4658110080 2026-02-25T11:54:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465f377640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4658124310 con 0x7f465810f690 2026-02-25T11:54:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465eb76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4658115880 0x7f46581239c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465db74640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4658110080 0x7f465811e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465e375640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f465810f690 0x7f465811e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465eb76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4658115880 0x7f46581239c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52862/0 (socket says 10.20.193.28:52862) 2026-02-25T11:54:15.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465eb76640 1 -- 10.20.193.28:0/782646139 learned_addr learned my addr 10.20.193.28:0/782646139 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465db74640 1 -- 10.20.193.28:0/782646139 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f465810f690 msgr2=0x7f465811e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465db74640 1 --2- 10.20.193.28:0/782646139 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f465810f690 0x7f465811e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465db74640 1 -- 10.20.193.28:0/782646139 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4658115880 msgr2=0x7f46581239c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465db74640 1 --2- 10.20.193.28:0/782646139 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4658115880 0x7f46581239c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465db74640 1 -- 10.20.193.28:0/782646139 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f465400a650 con 0x7f4658110080 2026-02-25T11:54:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.265+0000 7f465eb76640 1 --2- 10.20.193.28:0/782646139 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4658115880 0x7f46581239c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:15.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.266+0000 7f465db74640 1 --2- 10.20.193.28:0/782646139 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4658110080 0x7f465811e5f0 secure :-1 s=READY pgs=150 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f464c00d700 tx=0x7f464c00dbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:15.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.266+0000 7f463f7fe640 1 -- 10.20.193.28:0/782646139 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f464c004500 con 0x7f4658110080 2026-02-25T11:54:15.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.266+0000 7f463f7fe640 1 -- 10.20.193.28:0/782646139 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f464c00de30 con 0x7f4658110080 2026-02-25T11:54:15.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.266+0000 7f463f7fe640 1 -- 10.20.193.28:0/782646139 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f464c00bea0 con 0x7f4658110080 2026-02-25T11:54:15.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.266+0000 7f465f377640 1 -- 10.20.193.28:0/782646139 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46581245f0 con 0x7f4658110080 2026-02-25T11:54:15.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.266+0000 7f465f377640 1 -- 10.20.193.28:0/782646139 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4658164cf0 con 0x7f4658110080 2026-02-25T11:54:15.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.269+0000 7f463f7fe640 1 -- 10.20.193.28:0/782646139 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f464c0052d0 con 0x7f4658110080 2026-02-25T11:54:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.271+0000 7f463f7fe640 1 --2- 10.20.193.28:0/782646139 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f46340b5760 0x7f46340d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.271+0000 7f463f7fe640 1 -- 10.20.193.28:0/782646139 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f464c0e6670 con 0x7f4658110080 2026-02-25T11:54:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.271+0000 7f465e375640 1 --2- 10.20.193.28:0/782646139 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f46340b5760 0x7f46340d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.271+0000 7f46625fbe00 1 Processor -- start 2026-02-25T11:54:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.271+0000 7f465e375640 1 --2- 10.20.193.28:0/782646139 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f46340b5760 0x7f46340d5c10 secure :-1 s=READY pgs=333 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4648004740 tx=0x7f4648005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f46625fbe00 1 -- start start 2026-02-25T11:54:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f46625fbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b6f8b30 0x55724b6f8f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f46625fbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f9440 0x55724b73dba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f46625fbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b73e0e0 0x55724b75e4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f46625fbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55724b60e8a0 con 0x55724b73e0e0 2026-02-25T11:54:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f46625fbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55724b60a5d0 con 0x55724b6f8b30 2026-02-25T11:54:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f46625fbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55724b604570 con 0x55724b6f9440 2026-02-25T11:54:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f465eb76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b73e0e0 0x55724b75e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f465db74640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f9440 0x55724b73dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f465db74640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f9440 0x55724b73dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:53174/0 (socket says 10.20.193.28:53174) 2026-02-25T11:54:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f465db74640 1 -- 10.20.193.28:0/3433105407 learned_addr learned my addr 10.20.193.28:0/3433105407 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f465e375640 1 --2- 10.20.193.28:0/3433105407 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b6f8b30 0x55724b6f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f465db74640 1 -- 10.20.193.28:0/3433105407 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b6f8b30 msgr2=0x55724b6f8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f465db74640 1 --2- 10.20.193.28:0/3433105407 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b6f8b30 0x55724b6f8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f465db74640 1 -- 10.20.193.28:0/3433105407 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b73e0e0 msgr2=0x55724b75e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f465db74640 1 --2- 10.20.193.28:0/3433105407 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b73e0e0 0x55724b75e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f465db74640 1 -- 10.20.193.28:0/3433105407 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724b5afd10 con 0x55724b6f9440 2026-02-25T11:54:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.272+0000 7f465e375640 1 --2- 10.20.193.28:0/3433105407 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b6f8b30 0x55724b6f8f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:54:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.273+0000 7f465db74640 1 --2- 10.20.193.28:0/3433105407 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f9440 0x55724b73dba0 secure :-1 s=READY pgs=183 gs=3 cs=0 l=1 c_cookie=17835994758006670945 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f464c0e4160 tx=0x7f464c005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7862ebbb6b78661 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.273+0000 7f463cb0a640 1 -- 10.20.193.28:0/3433105407 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f464c00b7e0 con 0x55724b6f9440 2026-02-25T11:54:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.273+0000 7f463cb0a640 1 -- 10.20.193.28:0/3433105407 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f464c0b2710 con 0x55724b6f9440 2026-02-25T11:54:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.273+0000 7f463cb0a640 1 -- 10.20.193.28:0/3433105407 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f464c0ac8c0 con 0x55724b6f9440 2026-02-25T11:54:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.273+0000 7f46625fbe00 1 -- 10.20.193.28:0/3433105407 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f9440 msgr2=0x55724b73dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.273+0000 7f46625fbe00 1 --2- 10.20.193.28:0/3433105407 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f9440 0x55724b73dba0 secure :-1 s=READY pgs=183 gs=3 cs=0 l=1 c_cookie=17835994758006670945 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f464c0e4160 tx=0x7f464c005e70 comp rx=0 tx=0).stop 2026-02-25T11:54:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.273+0000 7f46625fbe00 1 -- 10.20.193.28:0/3433105407 shutdown_connections 2026-02-25T11:54:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.273+0000 7f46625fbe00 1 --2- 10.20.193.28:0/3433105407 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b73e0e0 0x55724b75e4c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.273+0000 7f46625fbe00 1 --2- 10.20.193.28:0/3433105407 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f9440 0x55724b73dba0 unknown :-1 s=CLOSED pgs=183 gs=3 cs=0 l=1 c_cookie=17835994758006670945 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.273+0000 7f46625fbe00 1 --2- 10.20.193.28:0/3433105407 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b6f8b30 0x55724b6f8f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.273+0000 7f46625fbe00 1 -- 10.20.193.28:0/3433105407 >> 10.20.193.28:0/3433105407 conn(0x55724b6177a0 msgr2=0x55724b6f7410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.273+0000 7f46625fbe00 1 -- 10.20.193.28:0/3433105407 shutdown_connections 2026-02-25T11:54:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.273+0000 7f46625fbe00 1 -- 10.20.193.28:0/3433105407 wait complete. 2026-02-25T11:54:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.274+0000 7f46625fbe00 1 -- start start 2026-02-25T11:54:15.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.274+0000 7f46625fbe00 1 Processor -- start 2026-02-25T11:54:15.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f46625fbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f8b30 0x55724b710f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f46625fbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f9440 0x55724b711460 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f46625fbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b73e0e0 0x55724b716450 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f46625fbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55724b716990 con 0x55724b6f9440 2026-02-25T11:54:15.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f46625fbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55724b716b00 con 0x55724b73e0e0 2026-02-25T11:54:15.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f46625fbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55724b716cc0 con 0x55724b6f8b30 2026-02-25T11:54:15.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f465e375640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f8b30 0x55724b710f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f465db74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f9440 0x55724b711460 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f465e375640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f8b30 0x55724b710f20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:53182/0 (socket says 10.20.193.28:53182) 2026-02-25T11:54:15.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f465e375640 1 -- 10.20.193.28:0/1877087834 learned_addr learned my addr 10.20.193.28:0/1877087834 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:15.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f465eb76640 1 --2- 10.20.193.28:0/1877087834 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b73e0e0 0x55724b716450 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f465e375640 1 -- 10.20.193.28:0/1877087834 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b73e0e0 msgr2=0x55724b716450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f465e375640 1 --2- 10.20.193.28:0/1877087834 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b73e0e0 0x55724b716450 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f465e375640 1 -- 10.20.193.28:0/1877087834 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f9440 msgr2=0x55724b711460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f465e375640 1 --2- 10.20.193.28:0/1877087834 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f9440 0x55724b711460 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f465e375640 1 -- 10.20.193.28:0/1877087834 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724b5883c0 con 0x55724b6f8b30 2026-02-25T11:54:15.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.275+0000 7f465db74640 1 --2- 10.20.193.28:0/1877087834 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f9440 0x55724b711460 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:15.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.276+0000 7f465e375640 1 --2- 10.20.193.28:0/1877087834 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f8b30 0x55724b710f20 secure :-1 s=READY pgs=184 gs=2 cs=0 l=1 c_cookie=1512993318440402150 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4648004200 tx=0x7f46480047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14ff3b682d2dfce6 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:15.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.276+0000 7f460bfff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:54:15.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.276+0000 7f460bfff640 1 -- 10.20.193.28:0/1877087834 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4648021070 con 0x55724b6f8b30 2026-02-25T11:54:15.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.276+0000 7f460bfff640 1 -- 10.20.193.28:0/1877087834 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f464801ce30 con 0x55724b6f8b30 2026-02-25T11:54:15.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.276+0000 7f46625fbe00 1 -- 10.20.193.28:0/1877087834 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4658164cf0 con 0x55724b6f8b30 2026-02-25T11:54:15.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.276+0000 7f460bfff640 1 -- 10.20.193.28:0/1877087834 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f46480193d0 con 0x55724b6f8b30 2026-02-25T11:54:15.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.276+0000 7f46625fbe00 10 client.26407 _ll_register_callbacks cb 0x55724b585a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.276+0000 7f46625fbe00 1 -- 10.20.193.28:0/1877087834 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55724b29a6f0 con 0x55724b6f8b30 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.276+0000 7f460bfff640 1 -- 10.20.193.28:0/1877087834 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4648019560 con 0x55724b6f8b30 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.276+0000 7f460bfff640 1 -- 10.20.193.28:0/1877087834 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4648019760 con 0x55724b6f8b30 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.276+0000 7f46625fbe00 10 client.26407 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.276+0000 7f46625fbe00 1 -- 10.20.193.28:0/1877087834 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55724b59a0e0 con 0x55724b6f8b30 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.277+0000 7f460bfff640 1 -- 10.20.193.28:0/1877087834 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f464802d060 con 0x55724b6f8b30 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.277+0000 7f46625fbe00 10 client.26407 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.277+0000 7f46625fbe00 10 client.26407 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.277+0000 7f46625fbe00 20 client.26407 populate_metadata read hostname 'trial028' 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.277+0000 7f46625fbe00 1 --2- 10.20.193.28:0/1877087834 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55724b73b780 0x55724b73bc20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.277+0000 7f46625fbe00 4 client.26407 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.277+0000 7f46625fbe00 1 -- 10.20.193.28:0/1877087834 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55724b5d6fc0 con 0x55724b73b780 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.277+0000 7f465db74640 1 --2- 10.20.193.28:0/1877087834 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55724b73b780 0x55724b73bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.277+0000 7f465db74640 1 --2- 10.20.193.28:0/1877087834 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55724b73b780 0x55724b73bc20 crc :-1 s=READY pgs=45 gs=4 cs=0 l=0 c_cookie=7768233276129563074 s_cookie=11416068174632933157 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6bce4f01bc1251c2 server_cookie=9e6e058518a35725 in_seq=0 out_seq=0 2026-02-25T11:54:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.277+0000 7f460bfff640 10 client.26407 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:54:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f460bfff640 1 -- 10.20.193.28:0/1877087834 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55724b5d6fc0 con 0x55724b73b780 2026-02-25T11:54:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f460bfff640 10 client.26407 handle_command_reply: tid=0 2026-02-25T11:54:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f46625fbe00 1 client.26407 shutdown 2026-02-25T11:54:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f46625fbe00 1 -- 10.20.193.28:0/1877087834 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f8b30 msgr2=0x55724b710f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f46625fbe00 1 --2- 10.20.193.28:0/1877087834 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f8b30 0x55724b710f20 secure :-1 s=READY pgs=184 gs=2 cs=0 l=1 c_cookie=1512993318440402150 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4648004200 tx=0x7f46480047e0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f46625fbe00 1 -- 10.20.193.28:0/1877087834 shutdown_connections 2026-02-25T11:54:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f46625fbe00 1 --2- 10.20.193.28:0/1877087834 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55724b73b780 0x55724b73bc20 crc :-1 s=READY pgs=45 gs=4 cs=0 l=0 c_cookie=7768233276129563074 s_cookie=11416068174632933157 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f46625fbe00 1 --2- 10.20.193.28:0/1877087834 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b73e0e0 0x55724b716450 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f46625fbe00 1 --2- 10.20.193.28:0/1877087834 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f9440 0x55724b711460 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f46625fbe00 1 --2- 10.20.193.28:0/1877087834 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f8b30 0x55724b710f20 unknown :-1 s=CLOSED pgs=184 gs=2 cs=0 l=1 c_cookie=1512993318440402150 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f46625fbe00 1 -- 10.20.193.28:0/1877087834 >> 10.20.193.28:0/1877087834 conn(0x55724b6177a0 msgr2=0x55724b6f7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:15.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f460bfff640 0 client.26407 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:54:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f46625fbe00 1 -- 10.20.193.28:0/1877087834 shutdown_connections 2026-02-25T11:54:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f46625fbe00 1 -- 10.20.193.28:0/1877087834 wait complete. 2026-02-25T11:54:15.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.278+0000 7f46625fbe00 20 client.26407 trim_cache size 0 max 16384 2026-02-25T11:54:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.286+0000 7f46625fbe00 1 Processor -- start 2026-02-25T11:54:15.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.286+0000 7f46625fbe00 1 -- start start 2026-02-25T11:54:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.286+0000 7f46625fbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f8b30 0x55724b7159c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.286+0000 7f46625fbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f9440 0x55724b73d2f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.286+0000 7f46625fbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b73e0e0 0x55724b73d830 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.286+0000 7f46625fbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55724b6021a0 con 0x55724b6f8b30 2026-02-25T11:54:15.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.286+0000 7f46625fbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55724b2c2700 con 0x55724b73e0e0 2026-02-25T11:54:15.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.286+0000 7f46625fbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55724b738070 con 0x55724b6f9440 2026-02-25T11:54:15.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.286+0000 7f465e375640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f8b30 0x55724b7159c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.286+0000 7f465e375640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f8b30 0x55724b7159c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52886/0 (socket says 10.20.193.28:52886) 2026-02-25T11:54:15.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.286+0000 7f465e375640 1 -- 10.20.193.28:0/1362874757 learned_addr learned my addr 10.20.193.28:0/1362874757 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:15.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.286+0000 7f465db74640 1 --2- 10.20.193.28:0/1362874757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f9440 0x55724b73d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.287+0000 7f465e375640 1 -- 10.20.193.28:0/1362874757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f9440 msgr2=0x55724b73d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.287+0000 7f465e375640 1 --2- 10.20.193.28:0/1362874757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f9440 0x55724b73d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.287+0000 7f465e375640 1 -- 10.20.193.28:0/1362874757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b73e0e0 msgr2=0x55724b73d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.287+0000 7f465e375640 1 --2- 10.20.193.28:0/1362874757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b73e0e0 0x55724b73d830 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.287+0000 7f465e375640 1 -- 10.20.193.28:0/1362874757 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724b594b60 con 0x55724b6f8b30 2026-02-25T11:54:15.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.287+0000 7f465e375640 1 --2- 10.20.193.28:0/1362874757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f8b30 0x55724b7159c0 secure :-1 s=READY pgs=673 gs=1 cs=0 l=1 c_cookie=16504787574116407833 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4648026d50 tx=0x7f464802b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50cc8da407d5a19 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:15.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.287+0000 7f46264e6640 1 -- 10.20.193.28:0/1362874757 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4648021070 con 0x55724b6f8b30 2026-02-25T11:54:15.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.287+0000 7f46264e6640 1 -- 10.20.193.28:0/1362874757 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55724b29a6f0 con 0x55724b6f8b30 2026-02-25T11:54:15.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.287+0000 7f46264e6640 1 -- 10.20.193.28:0/1362874757 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f464801c040 con 0x55724b6f8b30 2026-02-25T11:54:15.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.287+0000 7f46625fbe00 1 -- 10.20.193.28:0/1362874757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f8b30 msgr2=0x55724b7159c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.287+0000 7f46625fbe00 1 --2- 10.20.193.28:0/1362874757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f8b30 0x55724b7159c0 secure :-1 s=READY pgs=673 gs=1 cs=0 l=1 c_cookie=16504787574116407833 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4648026d50 tx=0x7f464802b000 comp rx=0 tx=0).stop 2026-02-25T11:54:15.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.288+0000 7f46625fbe00 1 -- 10.20.193.28:0/1362874757 shutdown_connections 2026-02-25T11:54:15.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.288+0000 7f46625fbe00 1 --2- 10.20.193.28:0/1362874757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b73e0e0 0x55724b73d830 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.288+0000 7f46625fbe00 1 --2- 10.20.193.28:0/1362874757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b6f9440 0x55724b73d2f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.288+0000 7f46625fbe00 1 --2- 10.20.193.28:0/1362874757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f8b30 0x55724b7159c0 unknown :-1 s=CLOSED pgs=673 gs=1 cs=0 l=1 c_cookie=16504787574116407833 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.288+0000 7f46625fbe00 1 -- 10.20.193.28:0/1362874757 >> 10.20.193.28:0/1362874757 conn(0x55724b6177a0 msgr2=0x55724b7152b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:15.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.288+0000 7f46625fbe00 1 -- 10.20.193.28:0/1362874757 shutdown_connections 2026-02-25T11:54:15.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.288+0000 7f46625fbe00 1 -- 10.20.193.28:0/1362874757 wait complete. 2026-02-25T11:54:15.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.288+0000 7f46625fbe00 1 -- start start 2026-02-25T11:54:15.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.288+0000 7f46625fbe00 1 Processor -- start 2026-02-25T11:54:15.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f46625fbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b6f8b30 0x55724b715c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f46625fbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f9440 0x55724b717450 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f46625fbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b73e0e0 0x55724b717990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f46625fbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55724b718060 con 0x55724b6f9440 2026-02-25T11:54:15.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f46625fbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55724b7181d0 con 0x55724b6f8b30 2026-02-25T11:54:15.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f46625fbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55724b71c4a0 con 0x55724b73e0e0 2026-02-25T11:54:15.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f465e375640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b6f8b30 0x55724b715c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f465e375640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b6f8b30 0x55724b715c10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:49378/0 (socket says 10.20.193.28:49378) 2026-02-25T11:54:15.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f465e375640 1 -- 10.20.193.28:0/3452394894 learned_addr learned my addr 10.20.193.28:0/3452394894 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:15.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f465eb76640 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b73e0e0 0x55724b717990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f465db74640 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f9440 0x55724b717450 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f465e375640 1 -- 10.20.193.28:0/3452394894 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b73e0e0 msgr2=0x55724b717990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f465e375640 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b73e0e0 0x55724b717990 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f465e375640 1 -- 10.20.193.28:0/3452394894 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f9440 msgr2=0x55724b717450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f465e375640 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f9440 0x55724b717450 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f465e375640 1 -- 10.20.193.28:0/3452394894 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55724b5886f0 con 0x55724b6f8b30 2026-02-25T11:54:15.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f465db74640 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f9440 0x55724b717450 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:54:15.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f465e375640 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b6f8b30 0x55724b715c10 secure :-1 s=READY pgs=151 gs=2 cs=0 l=1 c_cookie=6209086157125537213 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4648016040 tx=0x7f4648009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=562b1af677434dbd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:15.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.289+0000 7f460bfff640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:54:15.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f460bfff640 1 -- 10.20.193.28:0/3452394894 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4648021040 con 0x55724b6f8b30 2026-02-25T11:54:15.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f46625fbe00 1 -- 10.20.193.28:0/3452394894 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55724b59a0e0 con 0x55724b6f8b30 2026-02-25T11:54:15.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f460bfff640 1 -- 10.20.193.28:0/3452394894 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4648019490 con 0x55724b6f8b30 2026-02-25T11:54:15.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f46625fbe00 10 client.26441 _ll_register_callbacks cb 0x55724b585a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:15.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f46625fbe00 1 -- 10.20.193.28:0/3452394894 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f464c0b2710 con 0x55724b6f8b30 2026-02-25T11:54:15.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f460bfff640 1 -- 10.20.193.28:0/3452394894 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4648019ba0 con 0x55724b6f8b30 2026-02-25T11:54:15.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f460bfff640 1 -- 10.20.193.28:0/3452394894 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4648019d00 con 0x55724b6f8b30 2026-02-25T11:54:15.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f460bfff640 1 -- 10.20.193.28:0/3452394894 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f464801a870 con 0x55724b6f8b30 2026-02-25T11:54:15.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f46625fbe00 10 client.26441 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:15.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f46625fbe00 1 -- 10.20.193.28:0/3452394894 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55724b59a340 con 0x55724b6f8b30 2026-02-25T11:54:15.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f460bfff640 1 -- 10.20.193.28:0/3452394894 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f464804a060 con 0x55724b6f8b30 2026-02-25T11:54:15.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f46625fbe00 10 client.26441 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:15.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f46625fbe00 10 client.26441 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:54:15.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f46625fbe00 20 client.26441 populate_metadata read hostname 'trial028' 2026-02-25T11:54:15.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f46625fbe00 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55724b78d4a0 0x55724b78d940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f46625fbe00 4 client.26441 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:54:15.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f46625fbe00 1 -- 10.20.193.28:0/3452394894 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55724b297f40 con 0x55724b78d4a0 2026-02-25T11:54:15.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.290+0000 7f465db74640 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55724b78d4a0 0x55724b78d940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.291+0000 7f465db74640 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55724b78d4a0 0x55724b78d940 crc :-1 s=READY pgs=46 gs=4 cs=0 l=0 c_cookie=9955781281938821516 s_cookie=4872781009024592175 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a2a0a53f372518c server_cookie=439f9882b72e292f in_seq=0 out_seq=0 2026-02-25T11:54:15.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.291+0000 7f460bfff640 10 client.26441 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:54:15.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.291+0000 7f460bfff640 1 -- 10.20.193.28:0/3452394894 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55724b297f40 con 0x55724b78d4a0 2026-02-25T11:54:15.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.291+0000 7f460bfff640 10 client.26441 handle_command_reply: tid=0 2026-02-25T11:54:15.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.291+0000 7f46625fbe00 1 client.26441 shutdown 2026-02-25T11:54:15.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f46625fbe00 1 -- 10.20.193.28:0/3452394894 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b6f8b30 msgr2=0x55724b715c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f46625fbe00 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b6f8b30 0x55724b715c10 secure :-1 s=READY pgs=151 gs=2 cs=0 l=1 c_cookie=6209086157125537213 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4648016040 tx=0x7f4648009300 comp rx=0 tx=0).stop 2026-02-25T11:54:15.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f46625fbe00 1 -- 10.20.193.28:0/3452394894 shutdown_connections 2026-02-25T11:54:15.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f46625fbe00 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55724b78d4a0 0x55724b78d940 crc :-1 s=READY pgs=46 gs=4 cs=0 l=0 c_cookie=9955781281938821516 s_cookie=4872781009024592175 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f46625fbe00 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55724b73e0e0 0x55724b717990 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f46625fbe00 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55724b6f9440 0x55724b717450 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f46625fbe00 1 --2- 10.20.193.28:0/3452394894 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55724b6f8b30 0x55724b715c10 unknown :-1 s=CLOSED pgs=151 gs=2 cs=0 l=1 c_cookie=6209086157125537213 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f46625fbe00 1 -- 10.20.193.28:0/3452394894 >> 10.20.193.28:0/3452394894 conn(0x55724b6177a0 msgr2=0x55724b736110 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:15.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f460bfff640 0 client.26441 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:54:15.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f46625fbe00 1 -- 10.20.193.28:0/3452394894 shutdown_connections 2026-02-25T11:54:15.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f46625fbe00 1 -- 10.20.193.28:0/3452394894 wait complete. 2026-02-25T11:54:15.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f46625fbe00 20 client.26441 trim_cache size 0 max 16384 2026-02-25T11:54:15.299 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:54:15.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f465f377640 1 -- 10.20.193.28:0/782646139 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f46340b5760 msgr2=0x7f46340d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f465f377640 1 --2- 10.20.193.28:0/782646139 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f46340b5760 0x7f46340d5c10 secure :-1 s=READY pgs=333 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4648004740 tx=0x7f4648005f90 comp rx=0 tx=0).stop 2026-02-25T11:54:15.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f465f377640 1 -- 10.20.193.28:0/782646139 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4658110080 msgr2=0x7f465811e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f465f377640 1 --2- 10.20.193.28:0/782646139 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4658110080 0x7f465811e5f0 secure :-1 s=READY pgs=150 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f464c00d700 tx=0x7f464c00dbd0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f465f377640 1 -- 10.20.193.28:0/782646139 shutdown_connections 2026-02-25T11:54:15.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f465f377640 1 --2- 10.20.193.28:0/782646139 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f46340b5760 0x7f46340d5c10 unknown :-1 s=CLOSED pgs=333 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f465f377640 1 --2- 10.20.193.28:0/782646139 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4658115880 0x7f46581239c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f465f377640 1 --2- 10.20.193.28:0/782646139 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4658110080 0x7f465811e5f0 unknown :-1 s=CLOSED pgs=150 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f465f377640 1 --2- 10.20.193.28:0/782646139 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f465810f690 0x7f465811e0b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f465f377640 1 -- 10.20.193.28:0/782646139 >> 10.20.193.28:0/782646139 conn(0x7f46580ecf00 msgr2=0x7f465810d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:15.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f465f377640 1 -- 10.20.193.28:0/782646139 shutdown_connections 2026-02-25T11:54:15.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.292+0000 7f465f377640 1 -- 10.20.193.28:0/782646139 wait complete. 2026-02-25T11:54:15.396 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:15 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/441350676' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:54:15.430 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:54:15.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:15 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/441350676' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:54:15.614 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:15.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:15 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/441350676' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:54:15.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.784+0000 7f5693577640 1 -- 10.20.193.28:0/2145182658 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f569410de10 msgr2=0x7f5694112e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.784+0000 7f5693577640 1 --2- 10.20.193.28:0/2145182658 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f569410de10 0x7f5694112e60 secure :-1 s=READY pgs=674 gs=1 cs=0 l=1 c_cookie=2352696013376160544 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f567c009950 tx=0x7f567c02f190 comp rx=0 tx=0).stop 2026-02-25T11:54:15.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.784+0000 7f5693577640 1 -- 10.20.193.28:0/2145182658 shutdown_connections 2026-02-25T11:54:15.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.784+0000 7f5693577640 1 --2- 10.20.193.28:0/2145182658 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5694113610 0x7f5694173fe0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.784+0000 7f5693577640 1 --2- 10.20.193.28:0/2145182658 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f569410de10 0x7f5694112e60 unknown :-1 s=CLOSED pgs=674 gs=1 cs=0 l=1 c_cookie=2352696013376160544 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.784+0000 7f5693577640 1 --2- 10.20.193.28:0/2145182658 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f569410d420 0x7f569410d7f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.784+0000 7f5693577640 1 -- 10.20.193.28:0/2145182658 >> 10.20.193.28:0/2145182658 conn(0x7f56940eacb0 msgr2=0x7f569410b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:15.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.784+0000 7f5693577640 1 -- 10.20.193.28:0/2145182658 shutdown_connections 2026-02-25T11:54:15.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.784+0000 7f5693577640 1 -- 10.20.193.28:0/2145182658 wait complete. 2026-02-25T11:54:15.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.785+0000 7f5693577640 1 Processor -- start 2026-02-25T11:54:15.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.785+0000 7f5693577640 1 -- start start 2026-02-25T11:54:15.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5693577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f569410d420 0x7f56940f59c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5693577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f569410de10 0x7f5694173ee0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5693577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5694113610 0x7f56940f60c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5693577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f56940fb2d0 con 0x7f569410de10 2026-02-25T11:54:15.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5693577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56940fb440 con 0x7f569410d420 2026-02-25T11:54:15.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5693577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56940fb600 con 0x7f5694113610 2026-02-25T11:54:15.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5691d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f569410de10 0x7f5694173ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5691d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f569410de10 0x7f5694173ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52920/0 (socket says 10.20.193.28:52920) 2026-02-25T11:54:15.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5691d74640 1 -- 10.20.193.28:0/1952566340 learned_addr learned my addr 10.20.193.28:0/1952566340 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:15.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5692575640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f569410d420 0x7f56940f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5692d76640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5694113610 0x7f56940f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5691d74640 1 -- 10.20.193.28:0/1952566340 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5694113610 msgr2=0x7f56940f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5691d74640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5694113610 0x7f56940f60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5691d74640 1 -- 10.20.193.28:0/1952566340 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f569410d420 msgr2=0x7f56940f59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5691d74640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f569410d420 0x7f56940f59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.786+0000 7f5691d74640 1 -- 10.20.193.28:0/1952566340 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c009570 con 0x7f569410de10 2026-02-25T11:54:15.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.787+0000 7f5691d74640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f569410de10 0x7f5694173ee0 secure :-1 s=READY pgs=675 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f567c0098a0 tx=0x7f567c0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:15.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.787+0000 7f567b7fe640 1 -- 10.20.193.28:0/1952566340 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f567c03d070 con 0x7f569410de10 2026-02-25T11:54:15.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.787+0000 7f567b7fe640 1 -- 10.20.193.28:0/1952566340 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f567c002a70 con 0x7f569410de10 2026-02-25T11:54:15.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.787+0000 7f567b7fe640 1 -- 10.20.193.28:0/1952566340 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f567c041890 con 0x7f569410de10 2026-02-25T11:54:15.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.787+0000 7f5693577640 1 -- 10.20.193.28:0/1952566340 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56940fb880 con 0x7f569410de10 2026-02-25T11:54:15.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.787+0000 7f5693577640 1 -- 10.20.193.28:0/1952566340 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56940fbd40 con 0x7f569410de10 2026-02-25T11:54:15.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.789+0000 7f567b7fe640 1 -- 10.20.193.28:0/1952566340 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f567c041a20 con 0x7f569410de10 2026-02-25T11:54:15.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.791+0000 7f567b7fe640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f566c0b5750 0x7f566c0d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.791+0000 7f567b7fe640 1 -- 10.20.193.28:0/1952566340 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f567c10b280 con 0x7f569410de10 2026-02-25T11:54:15.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.791+0000 7f5692575640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f566c0b5750 0x7f566c0d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.792+0000 7f5692575640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f566c0b5750 0x7f566c0d5c00 secure :-1 s=READY pgs=334 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5688005970 tx=0x7f5688005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:15.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.792+0000 7f569aedbe00 1 Processor -- start 2026-02-25T11:54:15.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.792+0000 7f569aedbe00 1 -- start start 2026-02-25T11:54:15.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f569aedbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353cb30 0x5636e353cf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f569aedbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e3581ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f569aedbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e35a24c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f569aedbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5636e34528a0 con 0x5636e35820e0 2026-02-25T11:54:15.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f569aedbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5636e344e5d0 con 0x5636e353d440 2026-02-25T11:54:15.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f569aedbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5636e3448570 con 0x5636e353cb30 2026-02-25T11:54:15.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f5692d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e35a24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f5692d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e35a24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52928/0 (socket says 10.20.193.28:52928) 2026-02-25T11:54:15.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f5692d76640 1 -- 10.20.193.28:0/341794972 learned_addr learned my addr 10.20.193.28:0/341794972 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:15.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f5692575640 1 --2- 10.20.193.28:0/341794972 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353cb30 0x5636e353cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f5691d74640 1 --2- 10.20.193.28:0/341794972 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e3581ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f5692d76640 1 -- 10.20.193.28:0/341794972 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353cb30 msgr2=0x5636e353cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f5692d76640 1 --2- 10.20.193.28:0/341794972 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353cb30 0x5636e353cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f5692d76640 1 -- 10.20.193.28:0/341794972 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 msgr2=0x5636e3581ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f5692d76640 1 --2- 10.20.193.28:0/341794972 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e3581ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f5692d76640 1 -- 10.20.193.28:0/341794972 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e33f3d10 con 0x5636e35820e0 2026-02-25T11:54:15.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.793+0000 7f5692d76640 1 --2- 10.20.193.28:0/341794972 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e35a24c0 secure :-1 s=READY pgs=676 gs=2 cs=0 l=1 c_cookie=5177671082325163670 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5684006d50 tx=0x7f5684003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47dac86ab9cdbe96 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:15.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.794+0000 7f5678ff9640 1 -- 10.20.193.28:0/341794972 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5684010c50 con 0x5636e35820e0 2026-02-25T11:54:15.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.794+0000 7f5678ff9640 1 -- 10.20.193.28:0/341794972 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5684010db0 con 0x5636e35820e0 2026-02-25T11:54:15.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.794+0000 7f5678ff9640 1 -- 10.20.193.28:0/341794972 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5684011920 con 0x5636e35820e0 2026-02-25T11:54:15.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.794+0000 7f569aedbe00 1 -- 10.20.193.28:0/341794972 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 msgr2=0x5636e35a24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.794+0000 7f569aedbe00 1 --2- 10.20.193.28:0/341794972 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e35a24c0 secure :-1 s=READY pgs=676 gs=2 cs=0 l=1 c_cookie=5177671082325163670 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5684006d50 tx=0x7f5684003ea0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.794+0000 7f569aedbe00 1 -- 10.20.193.28:0/341794972 shutdown_connections 2026-02-25T11:54:15.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.794+0000 7f569aedbe00 1 --2- 10.20.193.28:0/341794972 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e35a24c0 unknown :-1 s=CLOSED pgs=676 gs=2 cs=0 l=1 c_cookie=5177671082325163670 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.794+0000 7f569aedbe00 1 --2- 10.20.193.28:0/341794972 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e3581ba0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.794+0000 7f569aedbe00 1 --2- 10.20.193.28:0/341794972 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353cb30 0x5636e353cf00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.794+0000 7f569aedbe00 1 -- 10.20.193.28:0/341794972 >> 10.20.193.28:0/341794972 conn(0x5636e345b7a0 msgr2=0x5636e353b410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:15.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.794+0000 7f569aedbe00 1 -- 10.20.193.28:0/341794972 shutdown_connections 2026-02-25T11:54:15.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.795+0000 7f569aedbe00 1 -- 10.20.193.28:0/341794972 wait complete. 2026-02-25T11:54:15.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.796+0000 7f569aedbe00 1 -- start start 2026-02-25T11:54:15.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.796+0000 7f569aedbe00 1 Processor -- start 2026-02-25T11:54:15.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.797+0000 7f569aedbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353cb30 0x5636e3554f70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.797+0000 7f569aedbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e35554b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.797+0000 7f569aedbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e355a4a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.797+0000 7f569aedbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5636e355a9e0 con 0x5636e35820e0 2026-02-25T11:54:15.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.797+0000 7f569aedbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5636e355ab50 con 0x5636e353d440 2026-02-25T11:54:15.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.797+0000 7f569aedbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5636e355ad10 con 0x5636e353cb30 2026-02-25T11:54:15.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.797+0000 7f5692d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e355a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.797+0000 7f5692d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e355a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52936/0 (socket says 10.20.193.28:52936) 2026-02-25T11:54:15.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.797+0000 7f5692d76640 1 -- 10.20.193.28:0/305489134 learned_addr learned my addr 10.20.193.28:0/305489134 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:15.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.797+0000 7f5692575640 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353cb30 0x5636e3554f70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.797+0000 7f5691d74640 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e35554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.798+0000 7f5692d76640 1 -- 10.20.193.28:0/305489134 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353cb30 msgr2=0x5636e3554f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.798+0000 7f5692d76640 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353cb30 0x5636e3554f70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.798+0000 7f5692d76640 1 -- 10.20.193.28:0/305489134 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 msgr2=0x5636e35554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.798+0000 7f5692d76640 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e35554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.798+0000 7f5692d76640 1 -- 10.20.193.28:0/305489134 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e33cc3c0 con 0x5636e35820e0 2026-02-25T11:54:15.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.798+0000 7f5691d74640 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e35554b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:15.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.798+0000 7f5692575640 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353cb30 0x5636e3554f70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:15.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.798+0000 7f5692d76640 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e355a4a0 secure :-1 s=READY pgs=677 gs=3 cs=0 l=1 c_cookie=11536942060865317394 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5684006d20 tx=0x7f5684010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a01b73add423ea12 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:15.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.798+0000 7f5643fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:15.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.798+0000 7f5643fff640 1 -- 10.20.193.28:0/305489134 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5684012ea0 con 0x5636e35820e0 2026-02-25T11:54:15.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.798+0000 7f569aedbe00 1 -- 10.20.193.28:0/305489134 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5684005550 con 0x5636e35820e0 2026-02-25T11:54:15.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.798+0000 7f5643fff640 1 -- 10.20.193.28:0/305489134 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5684012440 con 0x5636e35820e0 2026-02-25T11:54:15.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.798+0000 7f5643fff640 1 -- 10.20.193.28:0/305489134 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f568401b6d0 con 0x5636e35820e0 2026-02-25T11:54:15.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.799+0000 7f569aedbe00 10 client.16707 _ll_register_callbacks cb 0x5636e33c9a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:15.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.799+0000 7f569aedbe00 1 -- 10.20.193.28:0/305489134 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5636e30de6f0 con 0x5636e35820e0 2026-02-25T11:54:15.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.799+0000 7f5643fff640 1 -- 10.20.193.28:0/305489134 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f568401b870 con 0x5636e35820e0 2026-02-25T11:54:15.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.799+0000 7f5643fff640 1 -- 10.20.193.28:0/305489134 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5684011ab0 con 0x5636e35820e0 2026-02-25T11:54:15.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.799+0000 7f569aedbe00 10 client.16707 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:15.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.799+0000 7f569aedbe00 1 -- 10.20.193.28:0/305489134 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5636e33de0e0 con 0x5636e35820e0 2026-02-25T11:54:15.805 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.799+0000 7f5643fff640 1 -- 10.20.193.28:0/305489134 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f56840125b0 con 0x5636e35820e0 2026-02-25T11:54:15.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.799+0000 7f569aedbe00 10 client.16707 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:15.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.799+0000 7f569aedbe00 10 client.16707 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:54:15.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.799+0000 7f569aedbe00 20 client.16707 populate_metadata read hostname 'trial028' 2026-02-25T11:54:15.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.799+0000 7f569aedbe00 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5636e357f590 0x5636e357fa30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.799+0000 7f569aedbe00 4 client.16707 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:54:15.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.799+0000 7f569aedbe00 1 -- 10.20.193.28:0/305489134 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636e341afc0 con 0x5636e357f590 2026-02-25T11:54:15.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.800+0000 7f5692575640 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5636e357f590 0x5636e357fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.800+0000 7f5692575640 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5636e357f590 0x5636e357fa30 crc :-1 s=READY pgs=46 gs=4 cs=0 l=0 c_cookie=6571956195725134322 s_cookie=11219316304847470819 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5b3447571d9fc9f2 server_cookie=9bb304bc9d2e80e3 in_seq=0 out_seq=0 2026-02-25T11:54:15.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.800+0000 7f5643fff640 10 client.16707 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:54:15.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f5643fff640 1 -- 10.20.193.28:0/305489134 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5636e341afc0 con 0x5636e357f590 2026-02-25T11:54:15.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f5643fff640 10 client.16707 handle_command_reply: tid=0 2026-02-25T11:54:15.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f569aedbe00 1 client.16707 shutdown 2026-02-25T11:54:15.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f569aedbe00 1 -- 10.20.193.28:0/305489134 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 msgr2=0x5636e355a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f569aedbe00 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e355a4a0 secure :-1 s=READY pgs=677 gs=3 cs=0 l=1 c_cookie=11536942060865317394 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5684006d20 tx=0x7f5684010c60 comp rx=0 tx=0).stop 2026-02-25T11:54:15.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f569aedbe00 1 -- 10.20.193.28:0/305489134 shutdown_connections 2026-02-25T11:54:15.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f569aedbe00 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5636e357f590 0x5636e357fa30 crc :-1 s=READY pgs=46 gs=4 cs=0 l=0 c_cookie=6571956195725134322 s_cookie=11219316304847470819 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f569aedbe00 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e355a4a0 unknown :-1 s=CLOSED pgs=677 gs=3 cs=0 l=1 c_cookie=11536942060865317394 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f569aedbe00 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e35554b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f569aedbe00 1 --2- 10.20.193.28:0/305489134 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353cb30 0x5636e3554f70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f569aedbe00 1 -- 10.20.193.28:0/305489134 >> 10.20.193.28:0/305489134 conn(0x5636e345b7a0 msgr2=0x5636e353bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:15.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f5643fff640 0 client.16707 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:54:15.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f569aedbe00 1 -- 10.20.193.28:0/305489134 shutdown_connections 2026-02-25T11:54:15.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f569aedbe00 1 -- 10.20.193.28:0/305489134 wait complete. 2026-02-25T11:54:15.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.801+0000 7f569aedbe00 20 client.16707 trim_cache size 0 max 16384 2026-02-25T11:54:15.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f569aedbe00 1 Processor -- start 2026-02-25T11:54:15.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f569aedbe00 1 -- start start 2026-02-25T11:54:15.813 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f569aedbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e353cb30 0x5636e353bd20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.813 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f569aedbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e353c260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.813 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f569aedbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e35820e0 0x5636e3581320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.813 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f569aedbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5636e34461a0 con 0x5636e353cb30 2026-02-25T11:54:15.813 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f569aedbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5636e3106700 con 0x5636e353d440 2026-02-25T11:54:15.813 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f569aedbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5636e355ae50 con 0x5636e35820e0 2026-02-25T11:54:15.813 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f5692575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e353cb30 0x5636e353bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.813 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f5692575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e353cb30 0x5636e353bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52952/0 (socket says 10.20.193.28:52952) 2026-02-25T11:54:15.813 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f5692575640 1 -- 10.20.193.28:0/4272461132 learned_addr learned my addr 10.20.193.28:0/4272461132 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:15.813 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f5691d74640 1 --2- 10.20.193.28:0/4272461132 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e353c260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.813 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f5692575640 1 -- 10.20.193.28:0/4272461132 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e35820e0 msgr2=0x5636e3581320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f5692d76640 1 --2- 10.20.193.28:0/4272461132 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e35820e0 0x5636e3581320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f5692575640 1 --2- 10.20.193.28:0/4272461132 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e35820e0 0x5636e3581320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f5692575640 1 -- 10.20.193.28:0/4272461132 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 msgr2=0x5636e353c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f5692575640 1 --2- 10.20.193.28:0/4272461132 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e353c260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f5692575640 1 -- 10.20.193.28:0/4272461132 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e33d8b60 con 0x5636e353cb30 2026-02-25T11:54:15.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f5691d74640 1 --2- 10.20.193.28:0/4272461132 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e353c260 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:54:15.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.809+0000 7f5692d76640 1 --2- 10.20.193.28:0/4272461132 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e35820e0 0x5636e3581320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:54:15.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.810+0000 7f5692575640 1 --2- 10.20.193.28:0/4272461132 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e353cb30 0x5636e353bd20 secure :-1 s=READY pgs=678 gs=1 cs=0 l=1 c_cookie=2660594165723949860 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5688020520 tx=0x7f568800b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ec544c1b73eb24 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:15.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.810+0000 7f565e4e6640 1 -- 10.20.193.28:0/4272461132 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5688025040 con 0x5636e353cb30 2026-02-25T11:54:15.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.810+0000 7f565e4e6640 1 -- 10.20.193.28:0/4272461132 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5688011b20 con 0x5636e353cb30 2026-02-25T11:54:15.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.810+0000 7f565e4e6640 1 -- 10.20.193.28:0/4272461132 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5688029cb0 con 0x5636e353cb30 2026-02-25T11:54:15.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.810+0000 7f569aedbe00 1 -- 10.20.193.28:0/4272461132 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e353cb30 msgr2=0x5636e353bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.810+0000 7f569aedbe00 1 --2- 10.20.193.28:0/4272461132 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e353cb30 0x5636e353bd20 secure :-1 s=READY pgs=678 gs=1 cs=0 l=1 c_cookie=2660594165723949860 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5688020520 tx=0x7f568800b000 comp rx=0 tx=0).stop 2026-02-25T11:54:15.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.810+0000 7f569aedbe00 1 -- 10.20.193.28:0/4272461132 shutdown_connections 2026-02-25T11:54:15.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.810+0000 7f569aedbe00 1 --2- 10.20.193.28:0/4272461132 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e35820e0 0x5636e3581320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.810+0000 7f569aedbe00 1 --2- 10.20.193.28:0/4272461132 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353d440 0x5636e353c260 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.810+0000 7f569aedbe00 1 --2- 10.20.193.28:0/4272461132 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e353cb30 0x5636e353bd20 unknown :-1 s=CLOSED pgs=678 gs=1 cs=0 l=1 c_cookie=2660594165723949860 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.810+0000 7f569aedbe00 1 -- 10.20.193.28:0/4272461132 >> 10.20.193.28:0/4272461132 conn(0x5636e345b7a0 msgr2=0x5636e353f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:15.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.810+0000 7f569aedbe00 1 -- 10.20.193.28:0/4272461132 shutdown_connections 2026-02-25T11:54:15.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.810+0000 7f569aedbe00 1 -- 10.20.193.28:0/4272461132 wait complete. 2026-02-25T11:54:15.818 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.811+0000 7f569aedbe00 1 -- start start 2026-02-25T11:54:15.819 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.811+0000 7f569aedbe00 1 Processor -- start 2026-02-25T11:54:15.819 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.811+0000 7f569aedbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353cb30 0x5636e3560f40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.819 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.811+0000 7f569aedbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353d440 0x5636e355c730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.819 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.811+0000 7f569aedbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e355cc70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.819 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.811+0000 7f569aedbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5636e35555e0 con 0x5636e35820e0 2026-02-25T11:54:15.819 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f569aedbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5636e355d1e0 con 0x5636e353cb30 2026-02-25T11:54:15.819 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f569aedbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5636e355d3a0 con 0x5636e353d440 2026-02-25T11:54:15.819 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5692d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e355cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5692d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e355cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52964/0 (socket says 10.20.193.28:52964) 2026-02-25T11:54:15.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5692d76640 1 -- 10.20.193.28:0/4077980927 learned_addr learned my addr 10.20.193.28:0/4077980927 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:15.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5691d74640 1 --2- 10.20.193.28:0/4077980927 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353d440 0x5636e355c730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5692575640 1 --2- 10.20.193.28:0/4077980927 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353cb30 0x5636e3560f40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5692d76640 1 -- 10.20.193.28:0/4077980927 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353d440 msgr2=0x5636e355c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5692d76640 1 --2- 10.20.193.28:0/4077980927 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353d440 0x5636e355c730 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5692d76640 1 -- 10.20.193.28:0/4077980927 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353cb30 msgr2=0x5636e3560f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:15.820 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5692d76640 1 --2- 10.20.193.28:0/4077980927 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353cb30 0x5636e3560f40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.821 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5692d76640 1 -- 10.20.193.28:0/4077980927 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636e33cc6f0 con 0x5636e35820e0 2026-02-25T11:54:15.821 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5692d76640 1 --2- 10.20.193.28:0/4077980927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e355cc70 secure :-1 s=READY pgs=679 gs=1 cs=0 l=1 c_cookie=8732231091809626161 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f568402f8e0 tx=0x7f568402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792f1de9a3ef2831 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:15.821 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5643fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:15.821 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5643fff640 1 -- 10.20.193.28:0/4077980927 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f568402f5d0 con 0x5636e35820e0 2026-02-25T11:54:15.821 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5643fff640 1 -- 10.20.193.28:0/4077980927 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f568401f070 con 0x5636e35820e0 2026-02-25T11:54:15.821 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f5643fff640 1 -- 10.20.193.28:0/4077980927 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56840126c0 con 0x5636e35820e0 2026-02-25T11:54:15.821 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.812+0000 7f569aedbe00 1 -- 10.20.193.28:0/4077980927 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568800fe70 con 0x5636e35820e0 2026-02-25T11:54:15.821 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f5643fff640 1 -- 10.20.193.28:0/4077980927 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f5684012850 con 0x5636e35820e0 2026-02-25T11:54:15.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f5643fff640 10 client.16713 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:54:15.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f569aedbe00 10 client.16713 _ll_register_callbacks cb 0x5636e33c9a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:15.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f569aedbe00 1 -- 10.20.193.28:0/4077980927 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5684010db0 con 0x5636e35820e0 2026-02-25T11:54:15.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f5643fff640 1 -- 10.20.193.28:0/4077980927 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f56840052a0 con 0x5636e35820e0 2026-02-25T11:54:15.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f569aedbe00 10 client.16713 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:15.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f569aedbe00 1 -- 10.20.193.28:0/4077980927 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5636e33de340 con 0x5636e35820e0 2026-02-25T11:54:15.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f5643fff640 1 -- 10.20.193.28:0/4077980927 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f5684011a30 con 0x5636e35820e0 2026-02-25T11:54:15.822 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f569aedbe00 10 client.16713 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:15.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f569aedbe00 10 client.16713 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:54:15.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f569aedbe00 20 client.16713 populate_metadata read hostname 'trial028' 2026-02-25T11:54:15.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f569aedbe00 1 --2- 10.20.193.28:0/4077980927 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5636e362ed80 0x5636e364f230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:15.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f569aedbe00 4 client.16713 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:54:15.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f569aedbe00 1 -- 10.20.193.28:0/4077980927 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636e30dbf40 con 0x5636e362ed80 2026-02-25T11:54:15.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.813+0000 7f5692575640 1 --2- 10.20.193.28:0/4077980927 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5636e362ed80 0x5636e364f230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:15.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f5692575640 1 --2- 10.20.193.28:0/4077980927 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5636e362ed80 0x5636e364f230 crc :-1 s=READY pgs=47 gs=4 cs=0 l=0 c_cookie=10657887500318658601 s_cookie=16652650612531126229 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93e86c36b091d029 server_cookie=e71a1980a89bdfd5 in_seq=0 out_seq=0 2026-02-25T11:54:15.823 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f5643fff640 10 client.16713 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:54:15.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f5643fff640 1 -- 10.20.193.28:0/4077980927 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636e30dbf40 con 0x5636e362ed80 2026-02-25T11:54:15.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f5643fff640 10 client.16713 handle_command_reply: tid=0 2026-02-25T11:54:15.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f569aedbe00 1 client.16713 shutdown 2026-02-25T11:54:15.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f569aedbe00 1 -- 10.20.193.28:0/4077980927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 msgr2=0x5636e355cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f569aedbe00 1 --2- 10.20.193.28:0/4077980927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e355cc70 secure :-1 s=READY pgs=679 gs=1 cs=0 l=1 c_cookie=8732231091809626161 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f568402f8e0 tx=0x7f568402e000 comp rx=0 tx=0).stop 2026-02-25T11:54:15.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f569aedbe00 1 -- 10.20.193.28:0/4077980927 shutdown_connections 2026-02-25T11:54:15.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f569aedbe00 1 --2- 10.20.193.28:0/4077980927 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5636e362ed80 0x5636e364f230 crc :-1 s=READY pgs=47 gs=4 cs=0 l=0 c_cookie=10657887500318658601 s_cookie=16652650612531126229 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.824 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f569aedbe00 1 --2- 10.20.193.28:0/4077980927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5636e35820e0 0x5636e355cc70 unknown :-1 s=CLOSED pgs=679 gs=1 cs=0 l=1 c_cookie=8732231091809626161 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.825 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f569aedbe00 1 --2- 10.20.193.28:0/4077980927 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5636e353d440 0x5636e355c730 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.825 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f569aedbe00 1 --2- 10.20.193.28:0/4077980927 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5636e353cb30 0x5636e3560f40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.825 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f569aedbe00 1 -- 10.20.193.28:0/4077980927 >> 10.20.193.28:0/4077980927 conn(0x5636e345b7a0 msgr2=0x5636e353ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:15.825 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f5643fff640 0 client.16713 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:54:15.825 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f569aedbe00 1 -- 10.20.193.28:0/4077980927 shutdown_connections 2026-02-25T11:54:15.825 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.814+0000 7f569aedbe00 1 -- 10.20.193.28:0/4077980927 wait complete. 2026-02-25T11:54:15.825 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.815+0000 7f569aedbe00 20 client.16713 trim_cache size 0 max 16384 2026-02-25T11:54:15.826 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:54:15.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.815+0000 7f5693577640 1 -- 10.20.193.28:0/1952566340 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f566c0b5750 msgr2=0x7f566c0d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.815+0000 7f5693577640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f566c0b5750 0x7f566c0d5c00 secure :-1 s=READY pgs=334 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5688005970 tx=0x7f5688005900 comp rx=0 tx=0).stop 2026-02-25T11:54:15.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.815+0000 7f5693577640 1 -- 10.20.193.28:0/1952566340 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f569410de10 msgr2=0x7f5694173ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:15.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.815+0000 7f5693577640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f569410de10 0x7f5694173ee0 secure :-1 s=READY pgs=675 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f567c0098a0 tx=0x7f567c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.815+0000 7f5693577640 1 -- 10.20.193.28:0/1952566340 shutdown_connections 2026-02-25T11:54:15.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.815+0000 7f5693577640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f566c0b5750 0x7f566c0d5c00 unknown :-1 s=CLOSED pgs=334 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.815+0000 7f5693577640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5694113610 0x7f56940f60c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.815+0000 7f5693577640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f569410de10 0x7f5694173ee0 unknown :-1 s=CLOSED pgs=675 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.815+0000 7f5693577640 1 --2- 10.20.193.28:0/1952566340 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f569410d420 0x7f56940f59c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:15.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.815+0000 7f5693577640 1 -- 10.20.193.28:0/1952566340 >> 10.20.193.28:0/1952566340 conn(0x7f56940eacb0 msgr2=0x7f569410fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:15.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.815+0000 7f5693577640 1 -- 10.20.193.28:0/1952566340 shutdown_connections 2026-02-25T11:54:15.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:15.815+0000 7f5693577640 1 -- 10.20.193.28:0/1952566340 wait complete. 2026-02-25T11:54:15.951 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:54:16.133 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:16.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.308+0000 7f0565011640 1 -- 10.20.193.28:0/1947267508 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f056010bec0 msgr2=0x7f056010c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:16.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.308+0000 7f0565011640 1 --2- 10.20.193.28:0/1947267508 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f056010bec0 0x7f056010c290 secure :-1 s=READY pgs=680 gs=1 cs=0 l=1 c_cookie=1947550682438802022 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f054c009950 tx=0x7f054c02f190 comp rx=0 tx=0).stop 2026-02-25T11:54:16.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.308+0000 7f0565011640 1 -- 10.20.193.28:0/1947267508 shutdown_connections 2026-02-25T11:54:16.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.308+0000 7f0565011640 1 --2- 10.20.193.28:0/1947267508 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0560155bf0 0x7f0560175fd0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.308+0000 7f0565011640 1 --2- 10.20.193.28:0/1947267508 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f056010c7d0 0x7f05601556b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.308+0000 7f0565011640 1 --2- 10.20.193.28:0/1947267508 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f056010bec0 0x7f056010c290 unknown :-1 s=CLOSED pgs=680 gs=1 cs=0 l=1 c_cookie=1947550682438802022 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.308+0000 7f0565011640 1 -- 10.20.193.28:0/1947267508 >> 10.20.193.28:0/1947267508 conn(0x7f05600e9b40 msgr2=0x7f0560109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:16.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.308+0000 7f0565011640 1 -- 10.20.193.28:0/1947267508 shutdown_connections 2026-02-25T11:54:16.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.308+0000 7f0565011640 1 -- 10.20.193.28:0/1947267508 wait complete. 2026-02-25T11:54:16.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.309+0000 7f0565011640 1 Processor -- start 2026-02-25T11:54:16.313 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.309+0000 7f0565011640 1 -- start start 2026-02-25T11:54:16.313 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.309+0000 7f0565011640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f056010bec0 0x7f0560161350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.309+0000 7f0565011640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f056010c7d0 0x7f0560161890 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.309+0000 7f0565011640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0560155bf0 0x7f0560166c60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.309+0000 7f0565011640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f05601671f0 con 0x7f056010bec0 2026-02-25T11:54:16.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.309+0000 7f0565011640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0560167360 con 0x7f0560155bf0 2026-02-25T11:54:16.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.309+0000 7f0565011640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0560167520 con 0x7f056010c7d0 2026-02-25T11:54:16.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.309+0000 7f055ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f056010bec0 0x7f0560161350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.309+0000 7f055ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f056010bec0 0x7f0560161350 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38174/0 (socket says 10.20.193.28:38174) 2026-02-25T11:54:16.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.309+0000 7f055ffff640 1 -- 10.20.193.28:0/485774169 learned_addr learned my addr 10.20.193.28:0/485774169 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:16.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.309+0000 7f0564810640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0560155bf0 0x7f0560166c60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.310+0000 7f055f7fe640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f056010c7d0 0x7f0560161890 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.314 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.310+0000 7f055ffff640 1 -- 10.20.193.28:0/485774169 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f056010c7d0 msgr2=0x7f0560161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:16.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.310+0000 7f055ffff640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f056010c7d0 0x7f0560161890 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.310+0000 7f055ffff640 1 -- 10.20.193.28:0/485774169 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0560155bf0 msgr2=0x7f0560166c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:16.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.310+0000 7f055ffff640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0560155bf0 0x7f0560166c60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.310+0000 7f055ffff640 1 -- 10.20.193.28:0/485774169 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c009570 con 0x7f056010bec0 2026-02-25T11:54:16.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.310+0000 7f055ffff640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f056010bec0 0x7f0560161350 secure :-1 s=READY pgs=681 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f054c0098a0 tx=0x7f054c0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:16.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.310+0000 7f055d7fa640 1 -- 10.20.193.28:0/485774169 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f054c03d070 con 0x7f056010bec0 2026-02-25T11:54:16.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.310+0000 7f055d7fa640 1 -- 10.20.193.28:0/485774169 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f054c002a70 con 0x7f056010bec0 2026-02-25T11:54:16.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.310+0000 7f055d7fa640 1 -- 10.20.193.28:0/485774169 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f054c041890 con 0x7f056010bec0 2026-02-25T11:54:16.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.310+0000 7f0565011640 1 -- 10.20.193.28:0/485774169 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05601677a0 con 0x7f056010bec0 2026-02-25T11:54:16.315 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.310+0000 7f0565011640 1 -- 10.20.193.28:0/485774169 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0560152680 con 0x7f056010bec0 2026-02-25T11:54:16.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.313+0000 7f055d7fa640 1 -- 10.20.193.28:0/485774169 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f054c041a20 con 0x7f056010bec0 2026-02-25T11:54:16.316 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.314+0000 7f055d7fa640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f05340b57e0 0x7f05340d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.314+0000 7f055d7fa640 1 -- 10.20.193.28:0/485774169 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f054c10ae80 con 0x7f056010bec0 2026-02-25T11:54:16.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.315+0000 7f055f7fe640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f05340b57e0 0x7f05340d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.315+0000 7f055f7fe640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f05340b57e0 0x7f05340d5c90 secure :-1 s=READY pgs=335 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0560167cf0 tx=0x7f055000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:16.317 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f0568295e00 1 Processor -- start 2026-02-25T11:54:16.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f0568295e00 1 -- start start 2026-02-25T11:54:16.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f0568295e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266a82b30 0x559266a82f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f0568295e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266ac7ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f0568295e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266ac80e0 0x559266ae84c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f0568295e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5592669988a0 con 0x559266a82b30 2026-02-25T11:54:16.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f0568295e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5592669945d0 con 0x559266a83440 2026-02-25T11:54:16.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f0568295e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55926698e570 con 0x559266ac80e0 2026-02-25T11:54:16.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f055f7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266ac7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f055f7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266ac7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:51248/0 (socket says 10.20.193.28:51248) 2026-02-25T11:54:16.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f055f7fe640 1 -- 10.20.193.28:0/979237714 learned_addr learned my addr 10.20.193.28:0/979237714 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:16.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f0564810640 1 --2- 10.20.193.28:0/979237714 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266ac80e0 0x559266ae84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f055f7fe640 1 -- 10.20.193.28:0/979237714 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266ac80e0 msgr2=0x559266ae84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:16.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f055f7fe640 1 --2- 10.20.193.28:0/979237714 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266ac80e0 0x559266ae84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f055f7fe640 1 -- 10.20.193.28:0/979237714 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266a82b30 msgr2=0x559266a82f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:16.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f055ffff640 1 --2- 10.20.193.28:0/979237714 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266a82b30 0x559266a82f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f055f7fe640 1 --2- 10.20.193.28:0/979237714 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266a82b30 0x559266a82f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.316+0000 7f055f7fe640 1 -- 10.20.193.28:0/979237714 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559266939d10 con 0x559266a83440 2026-02-25T11:54:16.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.317+0000 7f055ffff640 1 --2- 10.20.193.28:0/979237714 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266a82b30 0x559266a82f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:54:16.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.317+0000 7f055f7fe640 1 --2- 10.20.193.28:0/979237714 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266ac7ba0 secure :-1 s=READY pgs=152 gs=1 cs=0 l=1 c_cookie=93014240446392011 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f055000ba20 tx=0x7f0550013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14a73f578dc26cb server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:16.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.317+0000 7f0530ce3640 1 -- 10.20.193.28:0/979237714 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f055001e640 con 0x559266a83440 2026-02-25T11:54:16.321 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.317+0000 7f0530ce3640 1 -- 10.20.193.28:0/979237714 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0550015070 con 0x559266a83440 2026-02-25T11:54:16.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.317+0000 7f0530ce3640 1 -- 10.20.193.28:0/979237714 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f055001cd00 con 0x559266a83440 2026-02-25T11:54:16.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.317+0000 7f0568295e00 1 -- 10.20.193.28:0/979237714 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 msgr2=0x559266ac7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:16.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.317+0000 7f0568295e00 1 --2- 10.20.193.28:0/979237714 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266ac7ba0 secure :-1 s=READY pgs=152 gs=1 cs=0 l=1 c_cookie=93014240446392011 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f055000ba20 tx=0x7f0550013000 comp rx=0 tx=0).stop 2026-02-25T11:54:16.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.317+0000 7f0568295e00 1 -- 10.20.193.28:0/979237714 shutdown_connections 2026-02-25T11:54:16.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.317+0000 7f0568295e00 1 --2- 10.20.193.28:0/979237714 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266ac80e0 0x559266ae84c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.317+0000 7f0568295e00 1 --2- 10.20.193.28:0/979237714 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266ac7ba0 unknown :-1 s=CLOSED pgs=152 gs=1 cs=0 l=1 c_cookie=93014240446392011 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.317+0000 7f0568295e00 1 --2- 10.20.193.28:0/979237714 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266a82b30 0x559266a82f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.317+0000 7f0568295e00 1 -- 10.20.193.28:0/979237714 >> 10.20.193.28:0/979237714 conn(0x5592669a17a0 msgr2=0x559266a81410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:16.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.318+0000 7f0568295e00 1 -- 10.20.193.28:0/979237714 shutdown_connections 2026-02-25T11:54:16.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.318+0000 7f0568295e00 1 -- 10.20.193.28:0/979237714 wait complete. 2026-02-25T11:54:16.327 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.319+0000 7f0568295e00 1 -- start start 2026-02-25T11:54:16.327 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.319+0000 7f0568295e00 1 Processor -- start 2026-02-25T11:54:16.327 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.319+0000 7f0568295e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266a82b30 0x559266a9ae70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.327 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.319+0000 7f0568295e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a83440 0x559266a9b3b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.327 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.319+0000 7f0568295e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266ac80e0 0x559266aa0410 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.327 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.319+0000 7f0568295e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559266aa0950 con 0x559266a82b30 2026-02-25T11:54:16.327 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.319+0000 7f0568295e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559266aa0ac0 con 0x559266ac80e0 2026-02-25T11:54:16.327 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.319+0000 7f0568295e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559266aa0c80 con 0x559266a83440 2026-02-25T11:54:16.327 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.320+0000 7f0564810640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266ac80e0 0x559266aa0410 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.327 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.320+0000 7f055f7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a83440 0x559266a9b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.320+0000 7f0564810640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266ac80e0 0x559266aa0410 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:51262/0 (socket says 10.20.193.28:51262) 2026-02-25T11:54:16.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.320+0000 7f0564810640 1 -- 10.20.193.28:0/2415533915 learned_addr learned my addr 10.20.193.28:0/2415533915 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:16.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.320+0000 7f055ffff640 1 --2- 10.20.193.28:0/2415533915 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266a82b30 0x559266a9ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.320+0000 7f0564810640 1 -- 10.20.193.28:0/2415533915 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a83440 msgr2=0x559266a9b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:16.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.320+0000 7f0564810640 1 --2- 10.20.193.28:0/2415533915 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a83440 0x559266a9b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.320+0000 7f0564810640 1 -- 10.20.193.28:0/2415533915 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266a82b30 msgr2=0x559266a9ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:16.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.320+0000 7f0564810640 1 --2- 10.20.193.28:0/2415533915 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266a82b30 0x559266a9ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.320+0000 7f0564810640 1 -- 10.20.193.28:0/2415533915 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592669123c0 con 0x559266ac80e0 2026-02-25T11:54:16.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.320+0000 7f055ffff640 1 --2- 10.20.193.28:0/2415533915 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266a82b30 0x559266a9ae70 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:54:16.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.321+0000 7f0564810640 1 --2- 10.20.193.28:0/2415533915 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266ac80e0 0x559266aa0410 secure :-1 s=READY pgs=153 gs=3 cs=0 l=1 c_cookie=10444593116712645053 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0554003fb0 tx=0x7f055400ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90f2a61a1125b1bd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:16.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.321+0000 7f0521ffb640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:54:16.328 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.321+0000 7f0521ffb640 1 -- 10.20.193.28:0/2415533915 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0554016070 con 0x559266ac80e0 2026-02-25T11:54:16.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.321+0000 7f0521ffb640 1 -- 10.20.193.28:0/2415533915 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0554010430 con 0x559266ac80e0 2026-02-25T11:54:16.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.321+0000 7f0568295e00 1 -- 10.20.193.28:0/2415533915 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055000bd10 con 0x559266ac80e0 2026-02-25T11:54:16.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.321+0000 7f0521ffb640 1 -- 10.20.193.28:0/2415533915 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f055401a640 con 0x559266ac80e0 2026-02-25T11:54:16.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.321+0000 7f0521ffb640 1 -- 10.20.193.28:0/2415533915 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f05540105a0 con 0x559266ac80e0 2026-02-25T11:54:16.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.321+0000 7f0568295e00 10 client.26465 _ll_register_callbacks cb 0x55926690fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:16.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.322+0000 7f0568295e00 1 -- 10.20.193.28:0/2415533915 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5592666246f0 con 0x559266ac80e0 2026-02-25T11:54:16.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.322+0000 7f0521ffb640 1 -- 10.20.193.28:0/2415533915 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0554010980 con 0x559266ac80e0 2026-02-25T11:54:16.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.322+0000 7f0568295e00 10 client.26465 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:16.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.322+0000 7f0568295e00 1 -- 10.20.193.28:0/2415533915 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5592669240e0 con 0x559266ac80e0 2026-02-25T11:54:16.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.322+0000 7f0521ffb640 1 -- 10.20.193.28:0/2415533915 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f0554022060 con 0x559266ac80e0 2026-02-25T11:54:16.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.322+0000 7f0568295e00 10 client.26465 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:16.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.322+0000 7f0568295e00 10 client.26465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:16.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.322+0000 7f0568295e00 20 client.26465 populate_metadata read hostname 'trial028' 2026-02-25T11:54:16.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.322+0000 7f0568295e00 1 --2- 10.20.193.28:0/2415533915 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559266ac5830 0x559266ac5cd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.322+0000 7f0568295e00 4 client.26465 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:54:16.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.322+0000 7f0568295e00 1 -- 10.20.193.28:0/2415533915 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559266960fc0 con 0x559266ac5830 2026-02-25T11:54:16.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.323+0000 7f055ffff640 1 --2- 10.20.193.28:0/2415533915 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559266ac5830 0x559266ac5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.323+0000 7f055ffff640 1 --2- 10.20.193.28:0/2415533915 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559266ac5830 0x559266ac5cd0 crc :-1 s=READY pgs=116 gs=4 cs=0 l=0 c_cookie=9360376894216016884 s_cookie=407259772768519489 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81e6bd311f8513f4 server_cookie=5a6e09b072c8541 in_seq=0 out_seq=0 2026-02-25T11:54:16.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.323+0000 7f0521ffb640 10 client.26465 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:16.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0521ffb640 1 -- 10.20.193.28:0/2415533915 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x559266960fc0 con 0x559266ac5830 2026-02-25T11:54:16.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0521ffb640 10 client.26465 handle_command_reply: tid=0 2026-02-25T11:54:16.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0568295e00 1 client.26465 shutdown 2026-02-25T11:54:16.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0568295e00 1 -- 10.20.193.28:0/2415533915 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266ac80e0 msgr2=0x559266aa0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:16.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0568295e00 1 --2- 10.20.193.28:0/2415533915 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266ac80e0 0x559266aa0410 secure :-1 s=READY pgs=153 gs=3 cs=0 l=1 c_cookie=10444593116712645053 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0554003fb0 tx=0x7f055400ba50 comp rx=0 tx=0).stop 2026-02-25T11:54:16.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0568295e00 1 -- 10.20.193.28:0/2415533915 shutdown_connections 2026-02-25T11:54:16.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0568295e00 1 --2- 10.20.193.28:0/2415533915 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559266ac5830 0x559266ac5cd0 crc :-1 s=READY pgs=116 gs=4 cs=0 l=0 c_cookie=9360376894216016884 s_cookie=407259772768519489 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0568295e00 1 --2- 10.20.193.28:0/2415533915 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266ac80e0 0x559266aa0410 unknown :-1 s=CLOSED pgs=153 gs=3 cs=0 l=1 c_cookie=10444593116712645053 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0568295e00 1 --2- 10.20.193.28:0/2415533915 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a83440 0x559266a9b3b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0568295e00 1 --2- 10.20.193.28:0/2415533915 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266a82b30 0x559266a9ae70 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0568295e00 1 -- 10.20.193.28:0/2415533915 >> 10.20.193.28:0/2415533915 conn(0x5592669a17a0 msgr2=0x559266a81b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:16.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0521ffb640 0 client.26465 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:16.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0568295e00 1 -- 10.20.193.28:0/2415533915 shutdown_connections 2026-02-25T11:54:16.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0568295e00 1 -- 10.20.193.28:0/2415533915 wait complete. 2026-02-25T11:54:16.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.324+0000 7f0568295e00 20 client.26465 trim_cache size 0 max 16384 2026-02-25T11:54:16.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0568295e00 1 Processor -- start 2026-02-25T11:54:16.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0568295e00 1 -- start start 2026-02-25T11:54:16.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0568295e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a82b30 0x559266ac1000 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0568295e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266a8d030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0568295e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266ac80e0 0x559266a98240 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0568295e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55926698c1a0 con 0x559266ac80e0 2026-02-25T11:54:16.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0568295e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55926664c700 con 0x559266a83440 2026-02-25T11:54:16.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0568295e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559266a941e0 con 0x559266a82b30 2026-02-25T11:54:16.337 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0564810640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266ac80e0 0x559266a98240 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.337 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f055f7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266a8d030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.337 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0564810640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266ac80e0 0x559266a98240 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38192/0 (socket says 10.20.193.28:38192) 2026-02-25T11:54:16.337 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0564810640 1 -- 10.20.193.28:0/1680460015 learned_addr learned my addr 10.20.193.28:0/1680460015 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:16.337 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f055ffff640 1 --2- 10.20.193.28:0/1680460015 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a82b30 0x559266ac1000 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.337 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0564810640 1 -- 10.20.193.28:0/1680460015 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a82b30 msgr2=0x559266ac1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:16.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0564810640 1 --2- 10.20.193.28:0/1680460015 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a82b30 0x559266ac1000 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0564810640 1 -- 10.20.193.28:0/1680460015 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 msgr2=0x559266a8d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:16.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0564810640 1 --2- 10.20.193.28:0/1680460015 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266a8d030 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f0564810640 1 -- 10.20.193.28:0/1680460015 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55926691eb60 con 0x559266ac80e0 2026-02-25T11:54:16.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.332+0000 7f055ffff640 1 --2- 10.20.193.28:0/1680460015 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a82b30 0x559266ac1000 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:54:16.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.333+0000 7f0564810640 1 --2- 10.20.193.28:0/1680460015 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266ac80e0 0x559266a98240 secure :-1 s=READY pgs=682 gs=2 cs=0 l=1 c_cookie=13574341626065099984 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f055401bad0 tx=0x7f0554020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc61bfc9f9689cd0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:16.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.333+0000 7f0523fff640 1 -- 10.20.193.28:0/1680460015 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0554016070 con 0x559266ac80e0 2026-02-25T11:54:16.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.333+0000 7f0523fff640 1 -- 10.20.193.28:0/1680460015 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5592666246f0 con 0x559266ac80e0 2026-02-25T11:54:16.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.333+0000 7f0523fff640 1 -- 10.20.193.28:0/1680460015 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0554010940 con 0x559266ac80e0 2026-02-25T11:54:16.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.333+0000 7f0568295e00 1 -- 10.20.193.28:0/1680460015 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266ac80e0 msgr2=0x559266a98240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:16.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.333+0000 7f0568295e00 1 --2- 10.20.193.28:0/1680460015 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266ac80e0 0x559266a98240 secure :-1 s=READY pgs=682 gs=2 cs=0 l=1 c_cookie=13574341626065099984 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f055401bad0 tx=0x7f0554020000 comp rx=0 tx=0).stop 2026-02-25T11:54:16.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.334+0000 7f0568295e00 1 -- 10.20.193.28:0/1680460015 shutdown_connections 2026-02-25T11:54:16.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.334+0000 7f0568295e00 1 --2- 10.20.193.28:0/1680460015 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266ac80e0 0x559266a98240 unknown :-1 s=CLOSED pgs=682 gs=2 cs=0 l=1 c_cookie=13574341626065099984 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.334+0000 7f0568295e00 1 --2- 10.20.193.28:0/1680460015 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266a8d030 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.334+0000 7f0568295e00 1 --2- 10.20.193.28:0/1680460015 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a82b30 0x559266ac1000 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.334+0000 7f0568295e00 1 -- 10.20.193.28:0/1680460015 >> 10.20.193.28:0/1680460015 conn(0x5592669a17a0 msgr2=0x559266abf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:16.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.334+0000 7f0568295e00 1 -- 10.20.193.28:0/1680460015 shutdown_connections 2026-02-25T11:54:16.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.334+0000 7f0568295e00 1 -- 10.20.193.28:0/1680460015 wait complete. 2026-02-25T11:54:16.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.334+0000 7f0568295e00 1 -- start start 2026-02-25T11:54:16.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.334+0000 7f0568295e00 1 Processor -- start 2026-02-25T11:54:16.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f0568295e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a82b30 0x559266aa2c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.339 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f0568295e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266aa31d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f0568295e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266ac80e0 0x559266b2d600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f0568295e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x559266a88860 con 0x559266ac80e0 2026-02-25T11:54:16.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f0568295e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x559266aa3b50 con 0x559266a83440 2026-02-25T11:54:16.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f0568295e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x559266b2db40 con 0x559266a82b30 2026-02-25T11:54:16.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f055ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a82b30 0x559266aa2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f055ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a82b30 0x559266aa2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44886/0 (socket says 10.20.193.28:44886) 2026-02-25T11:54:16.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f055ffff640 1 -- 10.20.193.28:0/2774374422 learned_addr learned my addr 10.20.193.28:0/2774374422 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:16.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f055f7fe640 1 --2- 10.20.193.28:0/2774374422 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266aa31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f0564810640 1 --2- 10.20.193.28:0/2774374422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266ac80e0 0x559266b2d600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.343 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f0564810640 1 -- 10.20.193.28:0/2774374422 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a82b30 msgr2=0x559266aa2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:16.343 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f0564810640 1 --2- 10.20.193.28:0/2774374422 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a82b30 0x559266aa2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f0564810640 1 -- 10.20.193.28:0/2774374422 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 msgr2=0x559266aa31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:16.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f0564810640 1 --2- 10.20.193.28:0/2774374422 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266aa31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.335+0000 7f0564810640 1 -- 10.20.193.28:0/2774374422 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592669126f0 con 0x559266ac80e0 2026-02-25T11:54:16.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.336+0000 7f0564810640 1 --2- 10.20.193.28:0/2774374422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266ac80e0 0x559266b2d600 secure :-1 s=READY pgs=683 gs=2 cs=0 l=1 c_cookie=11080752231159874529 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f055401b950 tx=0x7f055401b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c6bd722666c3e1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:16.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.336+0000 7f0521ffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:16.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.336+0000 7f0521ffb640 1 -- 10.20.193.28:0/2774374422 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0554016040 con 0x559266ac80e0 2026-02-25T11:54:16.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.336+0000 7f0521ffb640 1 -- 10.20.193.28:0/2774374422 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0554006d30 con 0x559266ac80e0 2026-02-25T11:54:16.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.336+0000 7f0521ffb640 1 -- 10.20.193.28:0/2774374422 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0554011ce0 con 0x559266ac80e0 2026-02-25T11:54:16.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.336+0000 7f0568295e00 1 -- 10.20.193.28:0/2774374422 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592669240e0 con 0x559266ac80e0 2026-02-25T11:54:16.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.336+0000 7f0568295e00 10 client.16728 _ll_register_callbacks cb 0x55926690fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:16.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.336+0000 7f0521ffb640 1 -- 10.20.193.28:0/2774374422 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f055401bb00 con 0x559266ac80e0 2026-02-25T11:54:16.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.336+0000 7f0568295e00 1 -- 10.20.193.28:0/2774374422 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0550015070 con 0x559266ac80e0 2026-02-25T11:54:16.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.336+0000 7f0521ffb640 1 -- 10.20.193.28:0/2774374422 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0554011e70 con 0x559266ac80e0 2026-02-25T11:54:16.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.336+0000 7f0568295e00 10 client.16728 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:16.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.336+0000 7f0568295e00 1 -- 10.20.193.28:0/2774374422 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x559266924340 con 0x559266ac80e0 2026-02-25T11:54:16.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.337+0000 7f0521ffb640 1 -- 10.20.193.28:0/2774374422 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f055403d020 con 0x559266ac80e0 2026-02-25T11:54:16.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.337+0000 7f0568295e00 10 client.16728 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:16.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.337+0000 7f0568295e00 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:16.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.337+0000 7f0568295e00 20 client.16728 populate_metadata read hostname 'trial028' 2026-02-25T11:54:16.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.337+0000 7f0568295e00 1 --2- 10.20.193.28:0/2774374422 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559266b759c0 0x559266b95e70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:16.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.337+0000 7f0568295e00 4 client.16728 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:54:16.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.337+0000 7f0568295e00 1 -- 10.20.193.28:0/2774374422 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559266621f40 con 0x559266b759c0 2026-02-25T11:54:16.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.337+0000 7f055ffff640 1 --2- 10.20.193.28:0/2774374422 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559266b759c0 0x559266b95e70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:16.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.337+0000 7f055ffff640 1 --2- 10.20.193.28:0/2774374422 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559266b759c0 0x559266b95e70 crc :-1 s=READY pgs=117 gs=4 cs=0 l=0 c_cookie=4832067997496483662 s_cookie=10145099739968304567 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=430ef43e094f9f4e server_cookie=8ccaa27663b7f5b7 in_seq=0 out_seq=0 2026-02-25T11:54:16.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.337+0000 7f0521ffb640 10 client.16728 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:16.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0521ffb640 1 -- 10.20.193.28:0/2774374422 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559266621f40 con 0x559266b759c0 2026-02-25T11:54:16.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0521ffb640 10 client.16728 handle_command_reply: tid=0 2026-02-25T11:54:16.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0568295e00 1 client.16728 shutdown 2026-02-25T11:54:16.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0568295e00 1 -- 10.20.193.28:0/2774374422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266ac80e0 msgr2=0x559266b2d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:16.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0568295e00 1 --2- 10.20.193.28:0/2774374422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266ac80e0 0x559266b2d600 secure :-1 s=READY pgs=683 gs=2 cs=0 l=1 c_cookie=11080752231159874529 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f055401b950 tx=0x7f055401b9b0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0568295e00 1 -- 10.20.193.28:0/2774374422 shutdown_connections 2026-02-25T11:54:16.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0568295e00 1 --2- 10.20.193.28:0/2774374422 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x559266b759c0 0x559266b95e70 crc :-1 s=READY pgs=117 gs=4 cs=0 l=0 c_cookie=4832067997496483662 s_cookie=10145099739968304567 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0568295e00 1 --2- 10.20.193.28:0/2774374422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x559266ac80e0 0x559266b2d600 unknown :-1 s=CLOSED pgs=683 gs=2 cs=0 l=1 c_cookie=11080752231159874529 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0568295e00 1 --2- 10.20.193.28:0/2774374422 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x559266a83440 0x559266aa31d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0568295e00 1 --2- 10.20.193.28:0/2774374422 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x559266a82b30 0x559266aa2c90 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0568295e00 1 -- 10.20.193.28:0/2774374422 >> 10.20.193.28:0/2774374422 conn(0x5592669a17a0 msgr2=0x559266ac00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:16.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0521ffb640 0 client.16728 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:16.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0568295e00 1 -- 10.20.193.28:0/2774374422 shutdown_connections 2026-02-25T11:54:16.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0568295e00 1 -- 10.20.193.28:0/2774374422 wait complete. 2026-02-25T11:54:16.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0568295e00 20 client.16728 trim_cache size 0 max 16384 2026-02-25T11:54:16.347 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:54:16.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0565011640 1 -- 10.20.193.28:0/485774169 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f05340b57e0 msgr2=0x7f05340d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:16.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.338+0000 7f0565011640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f05340b57e0 0x7f05340d5c90 secure :-1 s=READY pgs=335 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0560167cf0 tx=0x7f055000b410 comp rx=0 tx=0).stop 2026-02-25T11:54:16.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.339+0000 7f0565011640 1 -- 10.20.193.28:0/485774169 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f056010bec0 msgr2=0x7f0560161350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:16.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.339+0000 7f0565011640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f056010bec0 0x7f0560161350 secure :-1 s=READY pgs=681 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f054c0098a0 tx=0x7f054c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.339+0000 7f0565011640 1 -- 10.20.193.28:0/485774169 shutdown_connections 2026-02-25T11:54:16.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.339+0000 7f0565011640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f05340b57e0 0x7f05340d5c90 unknown :-1 s=CLOSED pgs=335 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.339+0000 7f0565011640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0560155bf0 0x7f0560166c60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.339+0000 7f0565011640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f056010c7d0 0x7f0560161890 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.339+0000 7f0565011640 1 --2- 10.20.193.28:0/485774169 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f056010bec0 0x7f0560161350 unknown :-1 s=CLOSED pgs=681 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:16.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.339+0000 7f0565011640 1 -- 10.20.193.28:0/485774169 >> 10.20.193.28:0/485774169 conn(0x7f05600e9b40 msgr2=0x7f0560109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:16.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.339+0000 7f0565011640 1 -- 10.20.193.28:0/485774169 shutdown_connections 2026-02-25T11:54:16.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:16.339+0000 7f0565011640 1 -- 10.20.193.28:0/485774169 wait complete. 2026-02-25T11:54:16.425 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:16 trial028 ceph-mon[39224]: pgmap v407: 97 pgs: 97 active+clean; 4.8 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:54:16.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:16 trial119 ceph-mon[42710]: pgmap v407: 97 pgs: 97 active+clean; 4.8 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:54:16.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:16 trial167 ceph-mon[43392]: pgmap v407: 97 pgs: 97 active+clean; 4.8 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:54:17.474 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:54:17.475 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:54:17.528 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:17 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:54:17.663 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:17.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:17 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:54:17.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:17 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:54:17.839 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.837+0000 7fd7a60dd640 1 -- 10.20.193.28:0/3075360142 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7a010b840 msgr2=0x7fd7a010bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:17.840 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.837+0000 7fd7a60dd640 1 --2- 10.20.193.28:0/3075360142 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7a010b840 0x7fd7a010bc10 secure :-1 s=READY pgs=684 gs=3 cs=0 l=1 c_cookie=5338425924592606599 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd790009950 tx=0x7fd79002f1d0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.840 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.837+0000 7fd7a60dd640 1 -- 10.20.193.28:0/3075360142 shutdown_connections 2026-02-25T11:54:17.840 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.837+0000 7fd7a60dd640 1 --2- 10.20.193.28:0/3075360142 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7a0114700 0x7fd7a0176230 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.840 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.837+0000 7fd7a60dd640 1 --2- 10.20.193.28:0/3075360142 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7a010c1e0 0x7fd7a0113f80 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.840 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.837+0000 7fd7a60dd640 1 --2- 10.20.193.28:0/3075360142 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7a010b840 0x7fd7a010bc10 unknown :-1 s=CLOSED pgs=684 gs=3 cs=0 l=1 c_cookie=5338425924592606599 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.840 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.837+0000 7fd7a60dd640 1 -- 10.20.193.28:0/3075360142 >> 10.20.193.28:0/3075360142 conn(0x7fd7a00e9290 msgr2=0x7fd7a01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:17.840 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.837+0000 7fd7a60dd640 1 -- 10.20.193.28:0/3075360142 shutdown_connections 2026-02-25T11:54:17.840 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.837+0000 7fd7a60dd640 1 -- 10.20.193.28:0/3075360142 wait complete. 2026-02-25T11:54:17.840 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.837+0000 7fd7a60dd640 1 Processor -- start 2026-02-25T11:54:17.840 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a60dd640 1 -- start start 2026-02-25T11:54:17.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a60dd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7a010b840 0x7fd7a00f6160 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a60dd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7a010c1e0 0x7fd7a01761a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a60dd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7a0114700 0x7fd7a00f6920 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a60dd640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a00fbb00 con 0x7fd7a0114700 2026-02-25T11:54:17.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a60dd640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a00fbc70 con 0x7fd7a010b840 2026-02-25T11:54:17.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a60dd640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a00fbe30 con 0x7fd7a010c1e0 2026-02-25T11:54:17.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a50db640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7a010b840 0x7fd7a00f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a50db640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7a010b840 0x7fd7a00f6160 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:51310/0 (socket says 10.20.193.28:51310) 2026-02-25T11:54:17.843 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a50db640 1 -- 10.20.193.28:0/3800546766 learned_addr learned my addr 10.20.193.28:0/3800546766 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:17.843 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a48da640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7a010c1e0 0x7fd7a01761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.843 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a58dc640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7a0114700 0x7fd7a00f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.843 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a48da640 1 -- 10.20.193.28:0/3800546766 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7a010b840 msgr2=0x7fd7a00f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:17.843 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a48da640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7a010b840 0x7fd7a00f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.843 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a48da640 1 -- 10.20.193.28:0/3800546766 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7a0114700 msgr2=0x7fd7a00f6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:17.843 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a48da640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7a0114700 0x7fd7a00f6920 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.843 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.838+0000 7fd7a48da640 1 -- 10.20.193.28:0/3800546766 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd790009570 con 0x7fd7a010c1e0 2026-02-25T11:54:17.843 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.839+0000 7fd7a48da640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7a010c1e0 0x7fd7a01761a0 secure :-1 s=READY pgs=185 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd794004740 tx=0x7fd79400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:17.843 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.839+0000 7fd78e7fc640 1 -- 10.20.193.28:0/3800546766 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd79400cc80 con 0x7fd7a010c1e0 2026-02-25T11:54:17.843 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.839+0000 7fd7a60dd640 1 -- 10.20.193.28:0/3800546766 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a00fc110 con 0x7fd7a010c1e0 2026-02-25T11:54:17.843 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.839+0000 7fd78e7fc640 1 -- 10.20.193.28:0/3800546766 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd79400cde0 con 0x7fd7a010c1e0 2026-02-25T11:54:17.844 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.839+0000 7fd78e7fc640 1 -- 10.20.193.28:0/3800546766 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd794010430 con 0x7fd7a010c1e0 2026-02-25T11:54:17.844 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.839+0000 7fd7a60dd640 1 -- 10.20.193.28:0/3800546766 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a00fc630 con 0x7fd7a010c1e0 2026-02-25T11:54:17.844 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.842+0000 7fd78e7fc640 1 -- 10.20.193.28:0/3800546766 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fd79401f060 con 0x7fd7a010c1e0 2026-02-25T11:54:17.846 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.844+0000 7fd78e7fc640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd77c0b56e0 0x7fd77c0d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.846 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.844+0000 7fd78e7fc640 1 -- 10.20.193.28:0/3800546766 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd794014070 con 0x7fd7a010c1e0 2026-02-25T11:54:17.846 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.844+0000 7fd7a50db640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd77c0b56e0 0x7fd77c0d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.846 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.844+0000 7fd7a9361e00 1 Processor -- start 2026-02-25T11:54:17.847 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a9361e00 1 -- start start 2026-02-25T11:54:17.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a50db640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd77c0b56e0 0x7fd77c0d5b90 secure :-1 s=READY pgs=336 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd790005b30 tx=0x7fd79003a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:17.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a9361e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa0750 0x5571ecfa0b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a9361e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa1060 0x5571ecfe5610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a9361e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfe5b50 0x5571ed005f30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a9361e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5571eceb6d80 con 0x5571ecfa1060 2026-02-25T11:54:17.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a9361e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5571eceb2ab0 con 0x5571ecfa0750 2026-02-25T11:54:17.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a9361e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5571eceaca50 con 0x5571ecfe5b50 2026-02-25T11:54:17.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a48da640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa1060 0x5571ecfe5610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a50db640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa0750 0x5571ecfa0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a48da640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa1060 0x5571ecfe5610 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38228/0 (socket says 10.20.193.28:38228) 2026-02-25T11:54:17.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a48da640 1 -- 10.20.193.28:0/3803985814 learned_addr learned my addr 10.20.193.28:0/3803985814 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:17.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a58dc640 1 --2- 10.20.193.28:0/3803985814 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfe5b50 0x5571ed005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a48da640 1 -- 10.20.193.28:0/3803985814 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfe5b50 msgr2=0x5571ed005f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:17.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a48da640 1 --2- 10.20.193.28:0/3803985814 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfe5b50 0x5571ed005f30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a48da640 1 -- 10.20.193.28:0/3803985814 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa0750 msgr2=0x5571ecfa0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:17.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a48da640 1 --2- 10.20.193.28:0/3803985814 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa0750 0x5571ecfa0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.845+0000 7fd7a48da640 1 -- 10.20.193.28:0/3803985814 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ece58120 con 0x5571ecfa1060 2026-02-25T11:54:17.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.846+0000 7fd7a48da640 1 --2- 10.20.193.28:0/3803985814 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa1060 0x5571ecfe5610 secure :-1 s=READY pgs=685 gs=1 cs=0 l=1 c_cookie=4767108428089739880 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd7940051e0 tx=0x7fd7940a95d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42282bdbae0f4a68 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:17.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.846+0000 7fd76dce5640 1 -- 10.20.193.28:0/3803985814 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd7940ac660 con 0x5571ecfa1060 2026-02-25T11:54:17.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.846+0000 7fd76dce5640 1 -- 10.20.193.28:0/3803985814 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd794016ea0 con 0x5571ecfa1060 2026-02-25T11:54:17.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.846+0000 7fd76dce5640 1 -- 10.20.193.28:0/3803985814 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd7940d21e0 con 0x5571ecfa1060 2026-02-25T11:54:17.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.846+0000 7fd7a9361e00 1 -- 10.20.193.28:0/3803985814 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa1060 msgr2=0x5571ecfe5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:17.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.846+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/3803985814 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa1060 0x5571ecfe5610 secure :-1 s=READY pgs=685 gs=1 cs=0 l=1 c_cookie=4767108428089739880 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd7940051e0 tx=0x7fd7940a95d0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.846+0000 7fd7a9361e00 1 -- 10.20.193.28:0/3803985814 shutdown_connections 2026-02-25T11:54:17.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.846+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/3803985814 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfe5b50 0x5571ed005f30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.846+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/3803985814 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa1060 0x5571ecfe5610 unknown :-1 s=CLOSED pgs=685 gs=1 cs=0 l=1 c_cookie=4767108428089739880 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.846+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/3803985814 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa0750 0x5571ecfa0b20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.846+0000 7fd7a9361e00 1 -- 10.20.193.28:0/3803985814 >> 10.20.193.28:0/3803985814 conn(0x5571ecf7e8f0 msgr2=0x5571ecf9ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:17.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.846+0000 7fd7a9361e00 1 -- 10.20.193.28:0/3803985814 shutdown_connections 2026-02-25T11:54:17.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.846+0000 7fd7a9361e00 1 -- 10.20.193.28:0/3803985814 wait complete. 2026-02-25T11:54:17.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.848+0000 7fd7a9361e00 1 -- start start 2026-02-25T11:54:17.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.848+0000 7fd7a9361e00 1 Processor -- start 2026-02-25T11:54:17.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.848+0000 7fd7a9361e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa0750 0x5571ecffd510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.848+0000 7fd7a9361e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa1060 0x5571ecffda50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.848+0000 7fd7a9361e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfe5b50 0x5571ed002a40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.848+0000 7fd7a9361e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5571ed002f80 con 0x5571ecfa0750 2026-02-25T11:54:17.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.848+0000 7fd7a9361e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5571ed0030f0 con 0x5571ecfa1060 2026-02-25T11:54:17.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.848+0000 7fd7a9361e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5571ed0032b0 con 0x5571ecfe5b50 2026-02-25T11:54:17.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.848+0000 7fd7a48da640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa1060 0x5571ecffda50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.848+0000 7fd7a50db640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa0750 0x5571ecffd510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.848+0000 7fd7a48da640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa1060 0x5571ecffda50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:51334/0 (socket says 10.20.193.28:51334) 2026-02-25T11:54:17.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.848+0000 7fd7a48da640 1 -- 10.20.193.28:0/1123392905 learned_addr learned my addr 10.20.193.28:0/1123392905 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:17.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd7a58dc640 1 --2- 10.20.193.28:0/1123392905 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfe5b50 0x5571ed002a40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd7a50db640 1 -- 10.20.193.28:0/1123392905 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfe5b50 msgr2=0x5571ed002a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:17.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd7a50db640 1 --2- 10.20.193.28:0/1123392905 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfe5b50 0x5571ed002a40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd7a50db640 1 -- 10.20.193.28:0/1123392905 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa1060 msgr2=0x5571ecffda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:17.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd7a50db640 1 --2- 10.20.193.28:0/1123392905 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa1060 0x5571ecffda50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd7a50db640 1 -- 10.20.193.28:0/1123392905 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ece30810 con 0x5571ecfa0750 2026-02-25T11:54:17.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd7a48da640 1 --2- 10.20.193.28:0/1123392905 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa1060 0x5571ecffda50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:17.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd7a50db640 1 --2- 10.20.193.28:0/1123392905 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa0750 0x5571ecffd510 secure :-1 s=READY pgs=686 gs=1 cs=0 l=1 c_cookie=16191867672349013561 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd790004820 tx=0x7fd790009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b511dc1c7e7639 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:17.857 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd756ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:17.857 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd756ffd640 1 -- 10.20.193.28:0/1123392905 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd790038470 con 0x5571ecfa0750 2026-02-25T11:54:17.857 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd7a9361e00 1 -- 10.20.193.28:0/1123392905 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a00fc630 con 0x5571ecfa0750 2026-02-25T11:54:17.857 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd7a9361e00 10 client.16740 _ll_register_callbacks cb 0x5571ece2de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:17.857 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd7a9361e00 1 -- 10.20.193.28:0/1123392905 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5571ecc04530 con 0x5571ecfa0750 2026-02-25T11:54:17.857 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd756ffd640 1 -- 10.20.193.28:0/1123392905 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd79003e040 con 0x5571ecfa0750 2026-02-25T11:54:17.857 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.849+0000 7fd756ffd640 1 -- 10.20.193.28:0/1123392905 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd7900503f0 con 0x5571ecfa0750 2026-02-25T11:54:17.857 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.850+0000 7fd756ffd640 1 -- 10.20.193.28:0/1123392905 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd790044410 con 0x5571ecfa0750 2026-02-25T11:54:17.857 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.850+0000 7fd756ffd640 1 -- 10.20.193.28:0/1123392905 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd790044680 con 0x5571ecfa0750 2026-02-25T11:54:17.857 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.850+0000 7fd7a9361e00 10 client.16740 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:17.857 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.850+0000 7fd7a9361e00 1 -- 10.20.193.28:0/1123392905 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5571ece424f0 con 0x5571ecfa0750 2026-02-25T11:54:17.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.850+0000 7fd756ffd640 1 -- 10.20.193.28:0/1123392905 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd790060020 con 0x5571ecfa0750 2026-02-25T11:54:17.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.850+0000 7fd7a9361e00 10 client.16740 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:17.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.850+0000 7fd7a9361e00 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:17.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.850+0000 7fd7a9361e00 20 client.16740 populate_metadata read hostname 'trial028' 2026-02-25T11:54:17.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.850+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/1123392905 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571ecfe2ec0 0x5571ecfe3360 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.850+0000 7fd7a9361e00 4 client.16740 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:54:17.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.850+0000 7fd7a9361e00 1 -- 10.20.193.28:0/1123392905 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571ece59b50 con 0x5571ecfe2ec0 2026-02-25T11:54:17.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.851+0000 7fd7a48da640 1 --2- 10.20.193.28:0/1123392905 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571ecfe2ec0 0x5571ecfe3360 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.851+0000 7fd7a48da640 1 --2- 10.20.193.28:0/1123392905 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571ecfe2ec0 0x5571ecfe3360 crc :-1 s=READY pgs=118 gs=4 cs=0 l=0 c_cookie=17611951496825102422 s_cookie=11918791442179631945 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f46a389301ab6056 server_cookie=a5680dab0f152b49 in_seq=0 out_seq=0 2026-02-25T11:54:17.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.851+0000 7fd756ffd640 10 client.16740 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:17.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd756ffd640 1 -- 10.20.193.28:0/1123392905 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5571ece59b50 con 0x5571ecfe2ec0 2026-02-25T11:54:17.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd756ffd640 10 client.16740 handle_command_reply: tid=0 2026-02-25T11:54:17.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd7a9361e00 1 client.16740 shutdown 2026-02-25T11:54:17.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd7a9361e00 1 -- 10.20.193.28:0/1123392905 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa0750 msgr2=0x5571ecffd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:17.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/1123392905 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa0750 0x5571ecffd510 secure :-1 s=READY pgs=686 gs=1 cs=0 l=1 c_cookie=16191867672349013561 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd790004820 tx=0x7fd790009310 comp rx=0 tx=0).stop 2026-02-25T11:54:17.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd7a9361e00 1 -- 10.20.193.28:0/1123392905 shutdown_connections 2026-02-25T11:54:17.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/1123392905 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571ecfe2ec0 0x5571ecfe3360 crc :-1 s=READY pgs=118 gs=4 cs=0 l=0 c_cookie=17611951496825102422 s_cookie=11918791442179631945 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/1123392905 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfe5b50 0x5571ed002a40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/1123392905 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa1060 0x5571ecffda50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/1123392905 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa0750 0x5571ecffd510 unknown :-1 s=CLOSED pgs=686 gs=1 cs=0 l=1 c_cookie=16191867672349013561 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd756ffd640 0 client.16740 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:17.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd7a9361e00 1 -- 10.20.193.28:0/1123392905 >> 10.20.193.28:0/1123392905 conn(0x5571ecf7e8f0 msgr2=0x5571ecf9ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:17.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd7a9361e00 1 -- 10.20.193.28:0/1123392905 shutdown_connections 2026-02-25T11:54:17.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd7a9361e00 1 -- 10.20.193.28:0/1123392905 wait complete. 2026-02-25T11:54:17.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.852+0000 7fd7a9361e00 20 client.16740 trim_cache size 0 max 16384 2026-02-25T11:54:17.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.861+0000 7fd7a9361e00 1 Processor -- start 2026-02-25T11:54:17.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.861+0000 7fd7a9361e00 1 -- start start 2026-02-25T11:54:17.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a9361e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfa0750 0x5571ed001fb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a9361e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa1060 0x5571ecff97b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a9361e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfe5b50 0x5571ecff9cf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a9361e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5571ecfeaff0 con 0x5571ecfe5b50 2026-02-25T11:54:17.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a9361e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5571eceaa680 con 0x5571ecfa1060 2026-02-25T11:54:17.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a9361e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5571ecb6a790 con 0x5571ecfa0750 2026-02-25T11:54:17.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a58dc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfe5b50 0x5571ecff9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a50db640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfa0750 0x5571ed001fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a50db640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfa0750 0x5571ed001fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44912/0 (socket says 10.20.193.28:44912) 2026-02-25T11:54:17.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a50db640 1 -- 10.20.193.28:0/3544207293 learned_addr learned my addr 10.20.193.28:0/3544207293 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:17.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a48da640 1 --2- 10.20.193.28:0/3544207293 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa1060 0x5571ecff97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a58dc640 1 -- 10.20.193.28:0/3544207293 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfa0750 msgr2=0x5571ed001fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:17.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a58dc640 1 --2- 10.20.193.28:0/3544207293 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfa0750 0x5571ed001fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a58dc640 1 -- 10.20.193.28:0/3544207293 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa1060 msgr2=0x5571ecff97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:17.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a58dc640 1 --2- 10.20.193.28:0/3544207293 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa1060 0x5571ecff97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a58dc640 1 -- 10.20.193.28:0/3544207293 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ece3cf70 con 0x5571ecfe5b50 2026-02-25T11:54:17.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.862+0000 7fd7a58dc640 1 --2- 10.20.193.28:0/3544207293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfe5b50 0x5571ecff9cf0 secure :-1 s=READY pgs=687 gs=2 cs=0 l=1 c_cookie=14906539930807000240 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd79c0048e0 tx=0x7fd79c00a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedeab16bded5cb0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:17.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.863+0000 7fd76d4e4640 1 -- 10.20.193.28:0/3544207293 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd79c018070 con 0x5571ecfe5b50 2026-02-25T11:54:17.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.863+0000 7fd76d4e4640 1 -- 10.20.193.28:0/3544207293 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd79c019040 con 0x5571ecfe5b50 2026-02-25T11:54:17.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.863+0000 7fd76d4e4640 1 -- 10.20.193.28:0/3544207293 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd79c0118e0 con 0x5571ecfe5b50 2026-02-25T11:54:17.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.863+0000 7fd7a9361e00 1 -- 10.20.193.28:0/3544207293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfe5b50 msgr2=0x5571ecff9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:17.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.863+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/3544207293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfe5b50 0x5571ecff9cf0 secure :-1 s=READY pgs=687 gs=2 cs=0 l=1 c_cookie=14906539930807000240 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd79c0048e0 tx=0x7fd79c00a310 comp rx=0 tx=0).stop 2026-02-25T11:54:17.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.863+0000 7fd7a9361e00 1 -- 10.20.193.28:0/3544207293 shutdown_connections 2026-02-25T11:54:17.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.863+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/3544207293 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfe5b50 0x5571ecff9cf0 unknown :-1 s=CLOSED pgs=687 gs=2 cs=0 l=1 c_cookie=14906539930807000240 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.863+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/3544207293 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfa1060 0x5571ecff97b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.863+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/3544207293 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfa0750 0x5571ed001fb0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.863+0000 7fd7a9361e00 1 -- 10.20.193.28:0/3544207293 >> 10.20.193.28:0/3544207293 conn(0x5571ecf7e8f0 msgr2=0x5571ed0018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:17.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.863+0000 7fd7a9361e00 1 -- 10.20.193.28:0/3544207293 shutdown_connections 2026-02-25T11:54:17.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.863+0000 7fd7a9361e00 1 -- 10.20.193.28:0/3544207293 wait complete. 2026-02-25T11:54:17.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.863+0000 7fd7a9361e00 1 -- start start 2026-02-25T11:54:17.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a9361e00 1 Processor -- start 2026-02-25T11:54:17.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a9361e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa0750 0x5571ecf808c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a9361e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfa1060 0x5571ecf80e00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a9361e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfe5b50 0x5571ed02b980 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a9361e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5571ecfeaeb0 con 0x5571ecfa0750 2026-02-25T11:54:17.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a9361e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5571ecf7e330 con 0x5571ecfe5b50 2026-02-25T11:54:17.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a9361e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5571ecf7b370 con 0x5571ecfa1060 2026-02-25T11:54:17.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a50db640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa0750 0x5571ecf808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a50db640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa0750 0x5571ecf808c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38252/0 (socket says 10.20.193.28:38252) 2026-02-25T11:54:17.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a50db640 1 -- 10.20.193.28:0/518817006 learned_addr learned my addr 10.20.193.28:0/518817006 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:17.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a48da640 1 --2- 10.20.193.28:0/518817006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfa1060 0x5571ecf80e00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a58dc640 1 --2- 10.20.193.28:0/518817006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfe5b50 0x5571ed02b980 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a50db640 1 -- 10.20.193.28:0/518817006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfa1060 msgr2=0x5571ecf80e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:17.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a50db640 1 --2- 10.20.193.28:0/518817006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfa1060 0x5571ecf80e00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a50db640 1 -- 10.20.193.28:0/518817006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfe5b50 msgr2=0x5571ed02b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:17.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a50db640 1 --2- 10.20.193.28:0/518817006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfe5b50 0x5571ed02b980 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.864+0000 7fd7a50db640 1 -- 10.20.193.28:0/518817006 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571ece30b40 con 0x5571ecfa0750 2026-02-25T11:54:17.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.865+0000 7fd7a50db640 1 --2- 10.20.193.28:0/518817006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa0750 0x5571ecf808c0 secure :-1 s=READY pgs=688 gs=1 cs=0 l=1 c_cookie=8569357493382612189 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd7900023b0 tx=0x7fd790038560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ec793ce26704dd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:17.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.865+0000 7fd756ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:17.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.865+0000 7fd756ffd640 1 -- 10.20.193.28:0/518817006 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd7900594c0 con 0x5571ecfa0750 2026-02-25T11:54:17.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.865+0000 7fd7a9361e00 1 -- 10.20.193.28:0/518817006 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79c0054c0 con 0x5571ecfa0750 2026-02-25T11:54:17.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.865+0000 7fd7a9361e00 10 client.16746 _ll_register_callbacks cb 0x5571ece2de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:17.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.865+0000 7fd7a9361e00 1 -- 10.20.193.28:0/518817006 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd794016ea0 con 0x5571ecfa0750 2026-02-25T11:54:17.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.865+0000 7fd756ffd640 1 -- 10.20.193.28:0/518817006 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd794016ea0 con 0x5571ecfa0750 2026-02-25T11:54:17.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.865+0000 7fd756ffd640 1 -- 10.20.193.28:0/518817006 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd7900596d0 con 0x5571ecfa0750 2026-02-25T11:54:17.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.865+0000 7fd756ffd640 1 -- 10.20.193.28:0/518817006 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd790059850 con 0x5571ecfa0750 2026-02-25T11:54:17.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.865+0000 7fd756ffd640 1 -- 10.20.193.28:0/518817006 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd790059a80 con 0x5571ecfa0750 2026-02-25T11:54:17.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.865+0000 7fd7a9361e00 10 client.16746 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:17.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.865+0000 7fd7a9361e00 1 -- 10.20.193.28:0/518817006 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5571ece42750 con 0x5571ecfa0750 2026-02-25T11:54:17.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.865+0000 7fd756ffd640 1 -- 10.20.193.28:0/518817006 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd790072070 con 0x5571ecfa0750 2026-02-25T11:54:17.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.866+0000 7fd7a9361e00 10 client.16746 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:17.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.866+0000 7fd7a9361e00 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:17.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.866+0000 7fd7a9361e00 20 client.16746 populate_metadata read hostname 'trial028' 2026-02-25T11:54:17.873 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.866+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/518817006 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571ed093de0 0x5571ed0b4290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:17.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.866+0000 7fd7a9361e00 4 client.16746 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:54:17.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.866+0000 7fd7a9361e00 1 -- 10.20.193.28:0/518817006 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5571ecd0d690 con 0x5571ed093de0 2026-02-25T11:54:17.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.866+0000 7fd7a48da640 1 --2- 10.20.193.28:0/518817006 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571ed093de0 0x5571ed0b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:17.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.867+0000 7fd7a48da640 1 --2- 10.20.193.28:0/518817006 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571ed093de0 0x5571ed0b4290 crc :-1 s=READY pgs=119 gs=4 cs=0 l=0 c_cookie=11404635148510511477 s_cookie=250391485411153056 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e45673e43e74175 server_cookie=37991bab6b6d4a0 in_seq=0 out_seq=0 2026-02-25T11:54:17.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.867+0000 7fd756ffd640 10 client.16746 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:17.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd756ffd640 1 -- 10.20.193.28:0/518817006 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5571ecd0d690 con 0x5571ed093de0 2026-02-25T11:54:17.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd756ffd640 10 client.16746 handle_command_reply: tid=0 2026-02-25T11:54:17.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd7a9361e00 1 client.16746 shutdown 2026-02-25T11:54:17.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd7a9361e00 1 -- 10.20.193.28:0/518817006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa0750 msgr2=0x5571ecf808c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:17.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/518817006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa0750 0x5571ecf808c0 secure :-1 s=READY pgs=688 gs=1 cs=0 l=1 c_cookie=8569357493382612189 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd7900023b0 tx=0x7fd790038560 comp rx=0 tx=0).stop 2026-02-25T11:54:17.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd7a9361e00 1 -- 10.20.193.28:0/518817006 shutdown_connections 2026-02-25T11:54:17.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/518817006 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5571ed093de0 0x5571ed0b4290 crc :-1 s=READY pgs=119 gs=4 cs=0 l=0 c_cookie=11404635148510511477 s_cookie=250391485411153056 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/518817006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5571ecfe5b50 0x5571ed02b980 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/518817006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5571ecfa1060 0x5571ecf80e00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd7a9361e00 1 --2- 10.20.193.28:0/518817006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5571ecfa0750 0x5571ecf808c0 unknown :-1 s=CLOSED pgs=688 gs=1 cs=0 l=1 c_cookie=8569357493382612189 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd7a9361e00 1 -- 10.20.193.28:0/518817006 >> 10.20.193.28:0/518817006 conn(0x5571ecf7e8f0 msgr2=0x5571ed002090 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:17.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd756ffd640 0 client.16746 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:17.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd7a9361e00 1 -- 10.20.193.28:0/518817006 shutdown_connections 2026-02-25T11:54:17.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd7a9361e00 1 -- 10.20.193.28:0/518817006 wait complete. 2026-02-25T11:54:17.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.868+0000 7fd7a9361e00 20 client.16746 trim_cache size 0 max 16384 2026-02-25T11:54:17.875 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:54:17.876 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:54:17.876 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "641fdd6e-6811-41fc-b473-48f98e83e9f2", 2026-02-25T11:54:17.876 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:54:17.876 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:54:17.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.869+0000 7fd7a60dd640 1 -- 10.20.193.28:0/3800546766 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd77c0b56e0 msgr2=0x7fd77c0d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:17.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.869+0000 7fd7a60dd640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd77c0b56e0 0x7fd77c0d5b90 secure :-1 s=READY pgs=336 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd790005b30 tx=0x7fd79003a040 comp rx=0 tx=0).stop 2026-02-25T11:54:17.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.869+0000 7fd7a60dd640 1 -- 10.20.193.28:0/3800546766 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7a010c1e0 msgr2=0x7fd7a01761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:17.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.869+0000 7fd7a60dd640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7a010c1e0 0x7fd7a01761a0 secure :-1 s=READY pgs=185 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd794004740 tx=0x7fd79400ee60 comp rx=0 tx=0).stop 2026-02-25T11:54:17.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.869+0000 7fd7a60dd640 1 -- 10.20.193.28:0/3800546766 shutdown_connections 2026-02-25T11:54:17.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.869+0000 7fd7a60dd640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd77c0b56e0 0x7fd77c0d5b90 unknown :-1 s=CLOSED pgs=336 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.869+0000 7fd7a60dd640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd7a0114700 0x7fd7a00f6920 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.869+0000 7fd7a60dd640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd7a010c1e0 0x7fd7a01761a0 unknown :-1 s=CLOSED pgs=185 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.869+0000 7fd7a60dd640 1 --2- 10.20.193.28:0/3800546766 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd7a010b840 0x7fd7a00f6160 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:17.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.869+0000 7fd7a60dd640 1 -- 10.20.193.28:0/3800546766 >> 10.20.193.28:0/3800546766 conn(0x7fd7a00e9290 msgr2=0x7fd7a010d630 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:17.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.869+0000 7fd7a60dd640 1 -- 10.20.193.28:0/3800546766 shutdown_connections 2026-02-25T11:54:17.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:17.869+0000 7fd7a60dd640 1 -- 10.20.193.28:0/3800546766 wait complete. 2026-02-25T11:54:18.004 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:54:18.187 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:18.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.366+0000 7fc937fff640 1 -- 10.20.193.28:0/3353444502 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc93810f690 msgr2=0x7fc93810fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:18.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.366+0000 7fc937fff640 1 --2- 10.20.193.28:0/3353444502 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc93810f690 0x7fc93810fa60 secure :-1 s=READY pgs=689 gs=1 cs=0 l=1 c_cookie=6146243149132901508 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc92c009950 tx=0x7fc92c02f190 comp rx=0 tx=0).stop 2026-02-25T11:54:18.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.366+0000 7fc937fff640 1 -- 10.20.193.28:0/3353444502 shutdown_connections 2026-02-25T11:54:18.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.366+0000 7fc937fff640 1 --2- 10.20.193.28:0/3353444502 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc938115880 0x7fc938176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.366+0000 7fc937fff640 1 --2- 10.20.193.28:0/3353444502 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc938110080 0x7fc9381150d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.366+0000 7fc937fff640 1 --2- 10.20.193.28:0/3353444502 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc93810f690 0x7fc93810fa60 unknown :-1 s=CLOSED pgs=689 gs=1 cs=0 l=1 c_cookie=6146243149132901508 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.366+0000 7fc937fff640 1 -- 10.20.193.28:0/3353444502 >> 10.20.193.28:0/3353444502 conn(0x7fc9380ecf00 msgr2=0x7fc93810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:18.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.366+0000 7fc937fff640 1 -- 10.20.193.28:0/3353444502 shutdown_connections 2026-02-25T11:54:18.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.366+0000 7fc937fff640 1 -- 10.20.193.28:0/3353444502 wait complete. 2026-02-25T11:54:18.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.367+0000 7fc937fff640 1 Processor -- start 2026-02-25T11:54:18.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.367+0000 7fc937fff640 1 -- start start 2026-02-25T11:54:18.372 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc937fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc93810f690 0x7fc93805e4d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.372 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc937fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc938110080 0x7fc938176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.372 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc937fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc938115880 0x7fc93805a740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.372 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc937fff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc93805ade0 con 0x7fc938110080 2026-02-25T11:54:18.372 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc937fff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc93805af50 con 0x7fc938115880 2026-02-25T11:54:18.372 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc937fff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc93805b110 con 0x7fc93810f690 2026-02-25T11:54:18.372 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc9367fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc938110080 0x7fc938176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.372 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc936ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc93810f690 0x7fc93805e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc9367fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc938110080 0x7fc938176150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38280/0 (socket says 10.20.193.28:38280) 2026-02-25T11:54:18.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc9367fc640 1 -- 10.20.193.28:0/182996114 learned_addr learned my addr 10.20.193.28:0/182996114 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:18.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc9377fe640 1 --2- 10.20.193.28:0/182996114 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc938115880 0x7fc93805a740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc9367fc640 1 -- 10.20.193.28:0/182996114 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc93810f690 msgr2=0x7fc93805e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:18.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc9367fc640 1 --2- 10.20.193.28:0/182996114 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc93810f690 0x7fc93805e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc9367fc640 1 -- 10.20.193.28:0/182996114 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc938115880 msgr2=0x7fc93805a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:18.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc9367fc640 1 --2- 10.20.193.28:0/182996114 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc938115880 0x7fc93805a740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc9367fc640 1 -- 10.20.193.28:0/182996114 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc92c009570 con 0x7fc938110080 2026-02-25T11:54:18.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.368+0000 7fc9367fc640 1 --2- 10.20.193.28:0/182996114 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc938110080 0x7fc938176150 secure :-1 s=READY pgs=690 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc920009480 tx=0x7fc92000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:18.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.369+0000 7fc917fff640 1 -- 10.20.193.28:0/182996114 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc92000cd40 con 0x7fc938110080 2026-02-25T11:54:18.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.369+0000 7fc917fff640 1 -- 10.20.193.28:0/182996114 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc920004d10 con 0x7fc938110080 2026-02-25T11:54:18.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.369+0000 7fc917fff640 1 -- 10.20.193.28:0/182996114 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc9200106b0 con 0x7fc938110080 2026-02-25T11:54:18.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.369+0000 7fc937fff640 1 -- 10.20.193.28:0/182996114 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc93805b3f0 con 0x7fc938110080 2026-02-25T11:54:18.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.369+0000 7fc937fff640 1 -- 10.20.193.28:0/182996114 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc938104990 con 0x7fc938110080 2026-02-25T11:54:18.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.373+0000 7fc917fff640 1 -- 10.20.193.28:0/182996114 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fc92001f060 con 0x7fc938110080 2026-02-25T11:54:18.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.375+0000 7fc917fff640 1 --2- 10.20.193.28:0/182996114 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc90c0b5760 0x7fc90c0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.375+0000 7fc917fff640 1 -- 10.20.193.28:0/182996114 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc920014070 con 0x7fc938110080 2026-02-25T11:54:18.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.375+0000 7fc936ffd640 1 --2- 10.20.193.28:0/182996114 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc90c0b5760 0x7fc90c0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.376+0000 7fc936ffd640 1 --2- 10.20.193.28:0/182996114 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc90c0b5760 0x7fc90c0d5c10 secure :-1 s=READY pgs=337 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc92c0098a0 tx=0x7fc92c03a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:18.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.377+0000 7fc93f38ae00 1 Processor -- start 2026-02-25T11:54:18.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.377+0000 7fc93f38ae00 1 -- start start 2026-02-25T11:54:18.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.377+0000 7fc93f38ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58140 0x55d2ceb58510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.377+0000 7fc93f38ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58a50 0x55d2ceb9d1b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.377+0000 7fc93f38ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebbdad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.377+0000 7fc93f38ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d2cea6e9b0 con 0x55d2ceb58140 2026-02-25T11:54:18.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.377+0000 7fc93f38ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d2cea6a6e0 con 0x55d2ceb58a50 2026-02-25T11:54:18.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.377+0000 7fc93f38ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d2cea64840 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.377+0000 7fc936ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58140 0x55d2ceb58510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.377+0000 7fc936ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58140 0x55d2ceb58510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38286/0 (socket says 10.20.193.28:38286) 2026-02-25T11:54:18.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.377+0000 7fc936ffd640 1 -- 10.20.193.28:0/1303569651 learned_addr learned my addr 10.20.193.28:0/1303569651 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:18.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.377+0000 7fc9377fe640 1 --2- 10.20.193.28:0/1303569651 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebbdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.377+0000 7fc9367fc640 1 --2- 10.20.193.28:0/1303569651 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58a50 0x55d2ceb9d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc936ffd640 1 -- 10.20.193.28:0/1303569651 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb9d6f0 msgr2=0x55d2cebbdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:18.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc936ffd640 1 --2- 10.20.193.28:0/1303569651 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebbdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc936ffd640 1 -- 10.20.193.28:0/1303569651 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58a50 msgr2=0x55d2ceb9d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:18.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc936ffd640 1 --2- 10.20.193.28:0/1303569651 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58a50 0x55d2ceb9d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc936ffd640 1 -- 10.20.193.28:0/1303569651 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2cea0fd80 con 0x55d2ceb58140 2026-02-25T11:54:18.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc9377fe640 1 --2- 10.20.193.28:0/1303569651 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebbdad0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:54:18.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc936ffd640 1 --2- 10.20.193.28:0/1303569651 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58140 0x55d2ceb58510 secure :-1 s=READY pgs=691 gs=1 cs=0 l=1 c_cookie=17022076276371472557 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc92c005ad0 tx=0x7fc92c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3a902fc0b9c0ad server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:18.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc9037fe640 1 -- 10.20.193.28:0/1303569651 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc92c02f9f0 con 0x55d2ceb58140 2026-02-25T11:54:18.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc9037fe640 1 -- 10.20.193.28:0/1303569651 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc92c03e040 con 0x55d2ceb58140 2026-02-25T11:54:18.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc9037fe640 1 -- 10.20.193.28:0/1303569651 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc92c04f6c0 con 0x55d2ceb58140 2026-02-25T11:54:18.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1303569651 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58140 msgr2=0x55d2ceb58510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:18.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/1303569651 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58140 0x55d2ceb58510 secure :-1 s=READY pgs=691 gs=1 cs=0 l=1 c_cookie=17022076276371472557 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc92c005ad0 tx=0x7fc92c040000 comp rx=0 tx=0).stop 2026-02-25T11:54:18.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1303569651 shutdown_connections 2026-02-25T11:54:18.386 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/1303569651 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebbdad0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/1303569651 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58a50 0x55d2ceb9d1b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/1303569651 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58140 0x55d2ceb58510 unknown :-1 s=CLOSED pgs=691 gs=1 cs=0 l=1 c_cookie=17022076276371472557 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.378+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1303569651 >> 10.20.193.28:0/1303569651 conn(0x55d2cea77d40 msgr2=0x55d2ceb56a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:18.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.379+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1303569651 shutdown_connections 2026-02-25T11:54:18.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.379+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1303569651 wait complete. 2026-02-25T11:54:18.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.380+0000 7fc93f38ae00 1 -- start start 2026-02-25T11:54:18.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.380+0000 7fc93f38ae00 1 Processor -- start 2026-02-25T11:54:18.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.380+0000 7fc93f38ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58140 0x55d2cebb5470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.380+0000 7fc93f38ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58a50 0x55d2cebb59b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.380+0000 7fc93f38ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebba9a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.380+0000 7fc93f38ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d2cebbaee0 con 0x55d2ceb58a50 2026-02-25T11:54:18.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.380+0000 7fc93f38ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d2cebbb050 con 0x55d2ceb58140 2026-02-25T11:54:18.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.380+0000 7fc93f38ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d2cebbb210 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.381+0000 7fc936ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58140 0x55d2cebb5470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.381+0000 7fc936ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58140 0x55d2cebb5470 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:51372/0 (socket says 10.20.193.28:51372) 2026-02-25T11:54:18.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.381+0000 7fc936ffd640 1 -- 10.20.193.28:0/3417653229 learned_addr learned my addr 10.20.193.28:0/3417653229 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:18.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.381+0000 7fc9377fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebba9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.381+0000 7fc9367fc640 1 --2- 10.20.193.28:0/3417653229 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58a50 0x55d2cebb59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.381+0000 7fc9377fe640 1 -- 10.20.193.28:0/3417653229 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58140 msgr2=0x55d2cebb5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:18.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.381+0000 7fc9377fe640 1 --2- 10.20.193.28:0/3417653229 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58140 0x55d2cebb5470 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.381+0000 7fc9377fe640 1 -- 10.20.193.28:0/3417653229 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58a50 msgr2=0x55d2cebb59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:18.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.381+0000 7fc9377fe640 1 --2- 10.20.193.28:0/3417653229 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58a50 0x55d2cebb59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.381+0000 7fc9377fe640 1 -- 10.20.193.28:0/3417653229 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ce9e8350 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.381+0000 7fc9367fc640 1 --2- 10.20.193.28:0/3417653229 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58a50 0x55d2cebb59b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:18.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.382+0000 7fc9377fe640 1 --2- 10.20.193.28:0/3417653229 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebba9a0 secure :-1 s=READY pgs=186 gs=2 cs=0 l=1 c_cookie=11705360292859768639 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc928005820 tx=0x7fc9280137c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a271cb2b2fde6f3f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:18.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.382+0000 7fc900ff9640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:54:18.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.382+0000 7fc900ff9640 1 -- 10.20.193.28:0/3417653229 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc9280116d0 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.382+0000 7fc900ff9640 1 -- 10.20.193.28:0/3417653229 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc928011cf0 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.382+0000 7fc900ff9640 1 -- 10.20.193.28:0/3417653229 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc92801bd90 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.382+0000 7fc93f38ae00 1 -- 10.20.193.28:0/3417653229 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc938176ac0 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.382+0000 7fc93f38ae00 10 client.26476 _ll_register_callbacks cb 0x55d2ce9e59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:18.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.382+0000 7fc93f38ae00 1 -- 10.20.193.28:0/3417653229 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d2ce6fa6f0 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.382+0000 7fc900ff9640 1 -- 10.20.193.28:0/3417653229 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc928011e60 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.382+0000 7fc900ff9640 1 -- 10.20.193.28:0/3417653229 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc928011830 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.382+0000 7fc93f38ae00 10 client.26476 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:18.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.382+0000 7fc93f38ae00 1 -- 10.20.193.28:0/3417653229 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55d2ce9fa150 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.383+0000 7fc900ff9640 1 -- 10.20.193.28:0/3417653229 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fc928023060 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.383+0000 7fc93f38ae00 10 client.26476 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:18.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.383+0000 7fc93f38ae00 10 client.26476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:18.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.383+0000 7fc93f38ae00 20 client.26476 populate_metadata read hostname 'trial028' 2026-02-25T11:54:18.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.383+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/3417653229 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d2cebbda60 0x55d2ceb9b420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.383+0000 7fc93f38ae00 4 client.26476 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:54:18.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.383+0000 7fc93f38ae00 1 -- 10.20.193.28:0/3417653229 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2cea36910 con 0x55d2cebbda60 2026-02-25T11:54:18.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.383+0000 7fc936ffd640 1 --2- 10.20.193.28:0/3417653229 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d2cebbda60 0x55d2ceb9b420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.384+0000 7fc936ffd640 1 --2- 10.20.193.28:0/3417653229 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d2cebbda60 0x55d2ceb9b420 crc :-1 s=READY pgs=120 gs=4 cs=0 l=0 c_cookie=16065463205792139677 s_cookie=8017345544913290418 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def3fdaacd59499d server_cookie=6f43554b9e1e04b2 in_seq=0 out_seq=0 2026-02-25T11:54:18.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.384+0000 7fc900ff9640 10 client.26476 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:18.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.385+0000 7fc900ff9640 1 -- 10.20.193.28:0/3417653229 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55d2cea36910 con 0x55d2cebbda60 2026-02-25T11:54:18.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.385+0000 7fc900ff9640 10 client.26476 handle_command_reply: tid=0 2026-02-25T11:54:18.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.385+0000 7fc93f38ae00 1 client.26476 shutdown 2026-02-25T11:54:18.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.385+0000 7fc93f38ae00 1 -- 10.20.193.28:0/3417653229 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb9d6f0 msgr2=0x55d2cebba9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:18.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.385+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/3417653229 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebba9a0 secure :-1 s=READY pgs=186 gs=2 cs=0 l=1 c_cookie=11705360292859768639 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc928005820 tx=0x7fc9280137c0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.385+0000 7fc93f38ae00 1 -- 10.20.193.28:0/3417653229 shutdown_connections 2026-02-25T11:54:18.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.385+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/3417653229 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d2cebbda60 0x55d2ceb9b420 crc :-1 s=READY pgs=120 gs=4 cs=0 l=0 c_cookie=16065463205792139677 s_cookie=8017345544913290418 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.385+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/3417653229 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebba9a0 unknown :-1 s=CLOSED pgs=186 gs=2 cs=0 l=1 c_cookie=11705360292859768639 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.385+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/3417653229 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58a50 0x55d2cebb59b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.385+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/3417653229 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58140 0x55d2cebb5470 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.385+0000 7fc900ff9640 0 client.26476 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:18.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.385+0000 7fc93f38ae00 1 -- 10.20.193.28:0/3417653229 >> 10.20.193.28:0/3417653229 conn(0x55d2cea77d40 msgr2=0x55d2ceb56230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:18.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.385+0000 7fc93f38ae00 1 -- 10.20.193.28:0/3417653229 shutdown_connections 2026-02-25T11:54:18.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.386+0000 7fc93f38ae00 1 -- 10.20.193.28:0/3417653229 wait complete. 2026-02-25T11:54:18.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.386+0000 7fc93f38ae00 20 client.26476 trim_cache size 0 max 16384 2026-02-25T11:54:18.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc93f38ae00 1 Processor -- start 2026-02-25T11:54:18.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc93f38ae00 1 -- start start 2026-02-25T11:54:18.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc93f38ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb58140 0x55d2ceb98300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc93f38ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58a50 0x55d2ceb98840 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc93f38ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebb1630 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc93f38ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d2cea62470 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc93f38ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d2ce722700 con 0x55d2ceb58a50 2026-02-25T11:54:18.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc93f38ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d2ceb33490 con 0x55d2ceb58140 2026-02-25T11:54:18.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc9377fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebb1630 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc9377fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebb1630 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38300/0 (socket says 10.20.193.28:38300) 2026-02-25T11:54:18.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc9377fe640 1 -- 10.20.193.28:0/2696383006 learned_addr learned my addr 10.20.193.28:0/2696383006 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:18.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc9367fc640 1 --2- 10.20.193.28:0/2696383006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58a50 0x55d2ceb98840 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc9377fe640 1 -- 10.20.193.28:0/2696383006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb58140 msgr2=0x55d2ceb98300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:18.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc9377fe640 1 --2- 10.20.193.28:0/2696383006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb58140 0x55d2ceb98300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc9377fe640 1 -- 10.20.193.28:0/2696383006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58a50 msgr2=0x55d2ceb98840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:18.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc9377fe640 1 --2- 10.20.193.28:0/2696383006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58a50 0x55d2ceb98840 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.395+0000 7fc9377fe640 1 -- 10.20.193.28:0/2696383006 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ceba76e0 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.396+0000 7fc9377fe640 1 --2- 10.20.193.28:0/2696383006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebb1630 secure :-1 s=READY pgs=692 gs=1 cs=0 l=1 c_cookie=1916380763489329461 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc928025800 tx=0x7fc928021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a985a26a62c6135 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:18.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.396+0000 7fc902ffd640 1 -- 10.20.193.28:0/2696383006 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc928003cb0 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.396+0000 7fc902ffd640 1 -- 10.20.193.28:0/2696383006 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55d2ce6fa6f0 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.396+0000 7fc902ffd640 1 -- 10.20.193.28:0/2696383006 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc928005d50 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.396+0000 7fc93f38ae00 1 -- 10.20.193.28:0/2696383006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb9d6f0 msgr2=0x55d2cebb1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:18.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.396+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/2696383006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebb1630 secure :-1 s=READY pgs=692 gs=1 cs=0 l=1 c_cookie=1916380763489329461 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc928025800 tx=0x7fc928021000 comp rx=0 tx=0).stop 2026-02-25T11:54:18.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.396+0000 7fc93f38ae00 1 -- 10.20.193.28:0/2696383006 shutdown_connections 2026-02-25T11:54:18.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.396+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/2696383006 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb9d6f0 0x55d2cebb1630 unknown :-1 s=CLOSED pgs=692 gs=1 cs=0 l=1 c_cookie=1916380763489329461 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.396+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/2696383006 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb58a50 0x55d2ceb98840 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.396+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/2696383006 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb58140 0x55d2ceb98300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.396+0000 7fc93f38ae00 1 -- 10.20.193.28:0/2696383006 >> 10.20.193.28:0/2696383006 conn(0x55d2cea77d40 msgr2=0x55d2ceb9bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:18.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.396+0000 7fc93f38ae00 1 -- 10.20.193.28:0/2696383006 shutdown_connections 2026-02-25T11:54:18.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.396+0000 7fc93f38ae00 1 -- 10.20.193.28:0/2696383006 wait complete. 2026-02-25T11:54:18.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.397+0000 7fc93f38ae00 1 -- start start 2026-02-25T11:54:18.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.397+0000 7fc93f38ae00 1 Processor -- start 2026-02-25T11:54:18.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.397+0000 7fc93f38ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb58140 0x55d2ceb378f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.397+0000 7fc93f38ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58a50 0x55d2ceb37e30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.397+0000 7fc93f38ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb9d6f0 0x55d2ceb38390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.397+0000 7fc93f38ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d2cec03690 con 0x55d2ceb58a50 2026-02-25T11:54:18.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.397+0000 7fc93f38ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d2cec03800 con 0x55d2ceb9d6f0 2026-02-25T11:54:18.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.397+0000 7fc93f38ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d2cec039c0 con 0x55d2ceb58140 2026-02-25T11:54:18.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc9367fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58a50 0x55d2ceb37e30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc9367fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58a50 0x55d2ceb37e30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38314/0 (socket says 10.20.193.28:38314) 2026-02-25T11:54:18.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc9367fc640 1 -- 10.20.193.28:0/1022653795 learned_addr learned my addr 10.20.193.28:0/1022653795 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:18.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc936ffd640 1 --2- 10.20.193.28:0/1022653795 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb58140 0x55d2ceb378f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc9377fe640 1 --2- 10.20.193.28:0/1022653795 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb9d6f0 0x55d2ceb38390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc9367fc640 1 -- 10.20.193.28:0/1022653795 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb58140 msgr2=0x55d2ceb378f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:18.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc9367fc640 1 --2- 10.20.193.28:0/1022653795 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb58140 0x55d2ceb378f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc9367fc640 1 -- 10.20.193.28:0/1022653795 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb9d6f0 msgr2=0x55d2ceb38390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:18.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc9367fc640 1 --2- 10.20.193.28:0/1022653795 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb9d6f0 0x55d2ceb38390 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc9367fc640 1 -- 10.20.193.28:0/1022653795 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ce9f4bd0 con 0x55d2ceb58a50 2026-02-25T11:54:18.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc9367fc640 1 --2- 10.20.193.28:0/1022653795 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58a50 0x55d2ceb37e30 secure :-1 s=READY pgs=693 gs=3 cs=0 l=1 c_cookie=2208026477787790607 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9200ad030 tx=0x7fc9200aca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea47c620732bd0f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:18.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc900ff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:18.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc900ff9640 1 -- 10.20.193.28:0/1022653795 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc920016ea0 con 0x55d2ceb58a50 2026-02-25T11:54:18.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc900ff9640 1 -- 10.20.193.28:0/1022653795 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc92006be90 con 0x55d2ceb58a50 2026-02-25T11:54:18.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1022653795 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2ce9fa150 con 0x55d2ceb58a50 2026-02-25T11:54:18.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc900ff9640 1 -- 10.20.193.28:0/1022653795 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc9200721e0 con 0x55d2ceb58a50 2026-02-25T11:54:18.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.398+0000 7fc93f38ae00 10 client.16764 _ll_register_callbacks cb 0x55d2ce9e59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:18.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.399+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1022653795 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc92c03e040 con 0x55d2ceb58a50 2026-02-25T11:54:18.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.399+0000 7fc900ff9640 1 -- 10.20.193.28:0/1022653795 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc920072370 con 0x55d2ceb58a50 2026-02-25T11:54:18.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.399+0000 7fc900ff9640 1 -- 10.20.193.28:0/1022653795 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc9200ab030 con 0x55d2ceb58a50 2026-02-25T11:54:18.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.399+0000 7fc93f38ae00 10 client.16764 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:18.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.399+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1022653795 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55d2ce9e8680 con 0x55d2ceb58a50 2026-02-25T11:54:18.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.399+0000 7fc900ff9640 1 -- 10.20.193.28:0/1022653795 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fc9200cd030 con 0x55d2ceb58a50 2026-02-25T11:54:18.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.399+0000 7fc93f38ae00 10 client.16764 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:18.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.399+0000 7fc93f38ae00 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:18.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.399+0000 7fc93f38ae00 20 client.16764 populate_metadata read hostname 'trial028' 2026-02-25T11:54:18.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.399+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/1022653795 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d2cec4b810 0x55d2cec6bcc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:18.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.399+0000 7fc93f38ae00 4 client.16764 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:54:18.406 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.399+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1022653795 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d2ce6f7f40 con 0x55d2cec4b810 2026-02-25T11:54:18.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.399+0000 7fc936ffd640 1 --2- 10.20.193.28:0/1022653795 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d2cec4b810 0x55d2cec6bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:18.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.400+0000 7fc936ffd640 1 --2- 10.20.193.28:0/1022653795 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d2cec4b810 0x55d2cec6bcc0 crc :-1 s=READY pgs=121 gs=4 cs=0 l=0 c_cookie=12433090941916460718 s_cookie=7593368264954822915 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac8b36569d0eb6ae server_cookie=69611034a15b4103 in_seq=0 out_seq=0 2026-02-25T11:54:18.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.400+0000 7fc900ff9640 10 client.16764 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:18.407 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc900ff9640 1 -- 10.20.193.28:0/1022653795 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x55d2ce6f7f40 con 0x55d2cec4b810 2026-02-25T11:54:18.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc900ff9640 10 client.16764 handle_command_reply: tid=0 2026-02-25T11:54:18.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc93f38ae00 1 client.16764 shutdown 2026-02-25T11:54:18.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1022653795 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58a50 msgr2=0x55d2ceb37e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:18.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/1022653795 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58a50 0x55d2ceb37e30 secure :-1 s=READY pgs=693 gs=3 cs=0 l=1 c_cookie=2208026477787790607 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9200ad030 tx=0x7fc9200aca30 comp rx=0 tx=0).stop 2026-02-25T11:54:18.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1022653795 shutdown_connections 2026-02-25T11:54:18.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/1022653795 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55d2cec4b810 0x55d2cec6bcc0 crc :-1 s=READY pgs=121 gs=4 cs=0 l=0 c_cookie=12433090941916460718 s_cookie=7593368264954822915 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/1022653795 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d2ceb9d6f0 0x55d2ceb38390 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/1022653795 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d2ceb58a50 0x55d2ceb37e30 unknown :-1 s=CLOSED pgs=693 gs=3 cs=0 l=1 c_cookie=2208026477787790607 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc93f38ae00 1 --2- 10.20.193.28:0/1022653795 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d2ceb58140 0x55d2ceb378f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc900ff9640 0 client.16764 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:18.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1022653795 >> 10.20.193.28:0/1022653795 conn(0x55d2cea77d40 msgr2=0x55d2ceb9c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:18.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1022653795 shutdown_connections 2026-02-25T11:54:18.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc93f38ae00 1 -- 10.20.193.28:0/1022653795 wait complete. 2026-02-25T11:54:18.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.401+0000 7fc93f38ae00 20 client.16764 trim_cache size 0 max 16384 2026-02-25T11:54:18.409 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:54:18.409 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:54:18.409 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:54:18.409 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:54:18.409 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.410 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "641fdd6e-6811-41fc-b473-48f98e83e9f2", 2026-02-25T11:54:18.410 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:54:18.410 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:54:18.410 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:18.410 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:54:18.410 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:54:18.410 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:54:18.410 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.410 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:18.410 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:18.410 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:18.410 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:18.411 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:18.411 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:54:18.411 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:54:18.411 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.411 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:18.411 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:18.411 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:18.411 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:18.411 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:18.411 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:54:18.411 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:54:18.411 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.411 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:18.412 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:18.412 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:18.412 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:18.412 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:18.412 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:54:18.412 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:54:18.412 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.412 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:18.412 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:18.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:18.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:18.413 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:18.413 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:54:18.413 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:54:18.413 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:18.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:18.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:18.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:18.413 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:18.413 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:54:18.413 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:54:18.414 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:18.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:18.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:18.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:18.414 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:18.414 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:54:18.414 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:54:18.414 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:18.415 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:18.415 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:18.415 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:18.415 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:18.415 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:54:18.415 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:54:18.415 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.415 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:18.415 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:18.415 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:18.415 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:18.415 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:18.415 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:54:18.416 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:54:18.416 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.416 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:18.416 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:18.416 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:18.416 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:18.416 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:18.416 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:54:18.416 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:54:18.416 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.416 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:18.416 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:18.416 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:18.417 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:18.417 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:18.417 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:54:18.417 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:54:18.417 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.417 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:18.417 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:18.417 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:18.417 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:18.417 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:18.417 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:54:18.417 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:54:18.417 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.418 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:18.418 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:18.418 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:18.418 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:18.418 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:18.418 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:54:18.418 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:54:18.418 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:18.418 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:18.418 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:18.418 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:18.418 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:18.418 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:54:18.419 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:54:18.419 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:54:18.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.402+0000 7fc937fff640 1 -- 10.20.193.28:0/182996114 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc90c0b5760 msgr2=0x7fc90c0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:18.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.402+0000 7fc937fff640 1 --2- 10.20.193.28:0/182996114 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc90c0b5760 0x7fc90c0d5c10 secure :-1 s=READY pgs=337 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc92c0098a0 tx=0x7fc92c03a040 comp rx=0 tx=0).stop 2026-02-25T11:54:18.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.402+0000 7fc937fff640 1 -- 10.20.193.28:0/182996114 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc938110080 msgr2=0x7fc938176150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:18.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.402+0000 7fc937fff640 1 --2- 10.20.193.28:0/182996114 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc938110080 0x7fc938176150 secure :-1 s=READY pgs=690 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc920009480 tx=0x7fc92000ee60 comp rx=0 tx=0).stop 2026-02-25T11:54:18.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.402+0000 7fc937fff640 1 -- 10.20.193.28:0/182996114 shutdown_connections 2026-02-25T11:54:18.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.402+0000 7fc937fff640 1 --2- 10.20.193.28:0/182996114 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc90c0b5760 0x7fc90c0d5c10 unknown :-1 s=CLOSED pgs=337 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.402+0000 7fc937fff640 1 --2- 10.20.193.28:0/182996114 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc938115880 0x7fc93805a740 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.402+0000 7fc937fff640 1 --2- 10.20.193.28:0/182996114 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc938110080 0x7fc938176150 unknown :-1 s=CLOSED pgs=690 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.402+0000 7fc937fff640 1 --2- 10.20.193.28:0/182996114 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc93810f690 0x7fc93805e4d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:18.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.402+0000 7fc937fff640 1 -- 10.20.193.28:0/182996114 >> 10.20.193.28:0/182996114 conn(0x7fc9380ecf00 msgr2=0x7fc938111b30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:18.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.402+0000 7fc937fff640 1 -- 10.20.193.28:0/182996114 shutdown_connections 2026-02-25T11:54:18.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:18.402+0000 7fc937fff640 1 -- 10.20.193.28:0/182996114 wait complete. 2026-02-25T11:54:18.502 INFO:tasks.cephfs.filesystem:scrub status for tag:641fdd6e-6811-41fc-b473-48f98e83e9f2 - {'path': '/', 'tag': '641fdd6e-6811-41fc-b473-48f98e83e9f2', 'options': 'recursive,force'} 2026-02-25T11:54:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:18 trial167 ceph-mon[43392]: pgmap v408: 97 pgs: 97 active+clean; 4.8 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:18 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:54:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:18 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:54:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:18 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:54:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:18 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:54:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:54:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:54:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:54:18.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:18 trial119 ceph-mon[42710]: pgmap v408: 97 pgs: 97 active+clean; 4.8 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:18.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:18 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:54:18.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:18 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:54:18.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:18 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:54:18.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:18 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:54:18.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:54:18.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:54:18.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:54:18.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:18 trial028 ceph-mon[39224]: pgmap v408: 97 pgs: 97 active+clean; 4.8 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:18.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:18 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:54:18.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:18 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:54:18.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:18 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:54:18.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:18 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:54:18.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:54:18.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:54:18.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:54:18.811 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:54:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:20 trial167 ceph-mon[43392]: pgmap v409: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:54:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:20 trial119 ceph-mon[42710]: pgmap v409: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:54:20.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:20 trial028 ceph-mon[39224]: pgmap v409: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:54:22.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:22 trial167 ceph-mon[43392]: pgmap v410: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:22.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:22 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:54:22.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:22 trial119 ceph-mon[42710]: pgmap v410: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:22.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:22 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:54:22.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:22 trial028 ceph-mon[39224]: pgmap v410: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:22.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:22 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:54:24.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:24 trial167 ceph-mon[43392]: pgmap v411: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:54:24.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:24 trial119 ceph-mon[42710]: pgmap v411: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:54:24.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:24 trial028 ceph-mon[39224]: pgmap v411: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:54:26.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:26 trial167 ceph-mon[43392]: pgmap v412: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:54:26.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:26 trial119 ceph-mon[42710]: pgmap v412: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:54:26.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:26 trial028 ceph-mon[39224]: pgmap v412: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:54:28.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:28 trial167 ceph-mon[43392]: pgmap v413: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:28.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:28 trial119 ceph-mon[42710]: pgmap v413: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:28.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:28 trial028 ceph-mon[39224]: pgmap v413: 97 pgs: 97 active+clean; 4.9 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:30.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:30 trial167 ceph-mon[43392]: pgmap v414: 97 pgs: 97 active+clean; 5.0 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:54:30.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:30 trial119 ceph-mon[42710]: pgmap v414: 97 pgs: 97 active+clean; 5.0 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:54:30.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:30 trial028 ceph-mon[39224]: pgmap v414: 97 pgs: 97 active+clean; 5.0 MiB data, 343 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:54:32.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:32 trial167 ceph-mon[43392]: pgmap v415: 97 pgs: 97 active+clean; 5.0 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:54:32.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:32 trial119 ceph-mon[42710]: pgmap v415: 97 pgs: 97 active+clean; 5.0 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:54:32.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:32 trial028 ceph-mon[39224]: pgmap v415: 97 pgs: 97 active+clean; 5.0 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:54:34.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:34 trial167 ceph-mon[43392]: pgmap v416: 97 pgs: 97 active+clean; 5.0 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:54:34.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:34 trial119 ceph-mon[42710]: pgmap v416: 97 pgs: 97 active+clean; 5.0 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:54:34.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:34 trial028 ceph-mon[39224]: pgmap v416: 97 pgs: 97 active+clean; 5.0 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:54:36.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:36 trial167 ceph-mon[43392]: pgmap v417: 97 pgs: 97 active+clean; 5.0 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:54:36.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:36 trial119 ceph-mon[42710]: pgmap v417: 97 pgs: 97 active+clean; 5.0 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:54:36.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:36 trial028 ceph-mon[39224]: pgmap v417: 97 pgs: 97 active+clean; 5.0 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:54:38.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:38 trial167 ceph-mon[43392]: pgmap v418: 97 pgs: 97 active+clean; 5.0 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:38.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:38 trial119 ceph-mon[42710]: pgmap v418: 97 pgs: 97 active+clean; 5.0 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:38.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:38 trial028 ceph-mon[39224]: pgmap v418: 97 pgs: 97 active+clean; 5.0 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:313/600: open fd = 3 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:314/600: open fd = 3 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:315/600: open fd = 3 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:316/600: open fd = 3 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.741 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:317/600: open fd = 3 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:318/600: open fd = 3 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:319/600: open fd = 3 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:320/600: open fd = 3 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:321/600: open fd = 3 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:322/600: open fd = 3 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:323/600: open fd = 3 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:324/600: open fd = 3 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.742 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:325/600: open fd = 3 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:326/600: open fd = 3 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:327/600: open fd = 3 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:328/600: open fd = 3 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:329/600: open fd = 3 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:330/600: open fd = 3 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:331/600: open fd = 3 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.743 INFO:tasks.workunit.client.0.trial028.stdout:332/600: open fd = 3 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:333/600: open fd = 3 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:334/600: open fd = 3 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:335/600: open fd = 3 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:336/600: open fd = 3 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:337/600: open fd = 3 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:338/600: open fd = 3 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:339/600: open fd = 3 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.744 INFO:tasks.workunit.client.0.trial028.stdout:340/600: open fd = 3 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:341/600: open fd = 3 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:342/600: open fd = 3 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:343/600: open fd = 3 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:344/600: open fd = 3 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:345/600: open fd = 3 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:346/600: open fd = 3 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:347/600: open fd = 3 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.745 INFO:tasks.workunit.client.0.trial028.stdout:348/600: open fd = 3 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:349/600: open fd = 3 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:350/600: open fd = 3 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:351/600: open fd = 3 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:352/600: open fd = 3 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:353/600: open fd = 3 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:354/600: open fd = 3 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:355/600: open fd = 3 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.746 INFO:tasks.workunit.client.0.trial028.stdout:356/600: open fd = 3 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:357/600: open fd = 3 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:358/600: open fd = 3 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:359/600: open fd = 3 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:360/600: open fd = 3 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:361/600: open fd = 3 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:362/600: open fd = 3 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:363/600: open fd = 3 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.747 INFO:tasks.workunit.client.0.trial028.stdout:364/600: open fd = 3 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:365/600: open fd = 3 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:366/600: open fd = 3 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:367/600: open fd = 3 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:368/600: open fd = 3 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:369/600: open fd = 3 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:370/600: open fd = 3 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:371/600: open fd = 3 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.748 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.749 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.749 INFO:tasks.workunit.client.0.trial028.stdout:372/600: open fd = 3 2026-02-25T11:54:39.749 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.749 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.749 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.749 INFO:tasks.workunit.client.0.trial028.stdout:373/600: open fd = 3 2026-02-25T11:54:39.749 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.749 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.749 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:39.749 INFO:tasks.workunit.client.0.trial028.stdout:374/600: open fd = 3 2026-02-25T11:54:39.749 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.749 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:54:39.749 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:54:40.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:40 trial167 ceph-mon[43392]: pgmap v419: 97 pgs: 97 active+clean; 5.1 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:54:40.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:40 trial119 ceph-mon[42710]: pgmap v419: 97 pgs: 97 active+clean; 5.1 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:54:40.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:40 trial028 ceph-mon[39224]: pgmap v419: 97 pgs: 97 active+clean; 5.1 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:54:42.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:42 trial167 ceph-mon[43392]: pgmap v420: 97 pgs: 97 active+clean; 5.1 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:54:42.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:42 trial119 ceph-mon[42710]: pgmap v420: 97 pgs: 97 active+clean; 5.1 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:54:42.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:42 trial028 ceph-mon[39224]: pgmap v420: 97 pgs: 97 active+clean; 5.1 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:54:44.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:44 trial167 ceph-mon[43392]: pgmap v421: 97 pgs: 97 active+clean; 5.1 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:54:44.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:44 trial119 ceph-mon[42710]: pgmap v421: 97 pgs: 97 active+clean; 5.1 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:54:44.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:44 trial028 ceph-mon[39224]: pgmap v421: 97 pgs: 97 active+clean; 5.1 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:54:46.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:46 trial167 ceph-mon[43392]: pgmap v422: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:54:46.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:46 trial119 ceph-mon[42710]: pgmap v422: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:54:46.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:46 trial028 ceph-mon[39224]: pgmap v422: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:54:48.503 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:54:48.692 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:48.735 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:48 trial028 ceph-mon[39224]: pgmap v423: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:48.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:48 trial167 ceph-mon[43392]: pgmap v423: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:48.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:48 trial119 ceph-mon[42710]: pgmap v423: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:48.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.859+0000 7f69ed13e640 1 -- 10.20.193.28:0/756386386 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e8111370 msgr2=0x7f69e8111d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:48.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.859+0000 7f69ed13e640 1 --2- 10.20.193.28:0/756386386 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e8111370 0x7f69e8111d50 secure :-1 s=READY pgs=694 gs=1 cs=0 l=1 c_cookie=12788905713524862304 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69dc00aa30 tx=0x7f69dc030270 comp rx=0 tx=0).stop 2026-02-25T11:54:48.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.859+0000 7f69ed13e640 1 -- 10.20.193.28:0/756386386 shutdown_connections 2026-02-25T11:54:48.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.859+0000 7f69ed13e640 1 --2- 10.20.193.28:0/756386386 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e8111370 0x7f69e8111d50 unknown :-1 s=CLOSED pgs=694 gs=1 cs=0 l=1 c_cookie=12788905713524862304 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.859+0000 7f69ed13e640 1 --2- 10.20.193.28:0/756386386 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e810bd50 0x7f69e8110be0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.859+0000 7f69ed13e640 1 --2- 10.20.193.28:0/756386386 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69e810b360 0x7f69e810b730 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.859+0000 7f69ed13e640 1 -- 10.20.193.28:0/756386386 >> 10.20.193.28:0/756386386 conn(0x7f69e80e8e70 msgr2=0x7f69e8109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:48.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.859+0000 7f69ed13e640 1 -- 10.20.193.28:0/756386386 shutdown_connections 2026-02-25T11:54:48.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.859+0000 7f69ed13e640 1 -- 10.20.193.28:0/756386386 wait complete. 2026-02-25T11:54:48.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.860+0000 7f69ed13e640 1 Processor -- start 2026-02-25T11:54:48.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.860+0000 7f69ed13e640 1 -- start start 2026-02-25T11:54:48.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.860+0000 7f69ed13e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e810b360 0x7f69e81235f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.860+0000 7f69ed13e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e810bd50 0x7f69e8123b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.860+0000 7f69ed13e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69e8111370 0x7f69e8128f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.860+0000 7f69ed13e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8129520 con 0x7f69e810bd50 2026-02-25T11:54:48.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.860+0000 7f69ed13e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8129690 con 0x7f69e8111370 2026-02-25T11:54:48.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.860+0000 7f69ed13e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8129850 con 0x7f69e810b360 2026-02-25T11:54:48.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.861+0000 7f69e7fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e810b360 0x7f69e81235f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.861+0000 7f69e7fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e810b360 0x7f69e81235f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:43806/0 (socket says 10.20.193.28:43806) 2026-02-25T11:54:48.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.861+0000 7f69e7fff640 1 -- 10.20.193.28:0/304645140 learned_addr learned my addr 10.20.193.28:0/304645140 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:48.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.861+0000 7f69ec93d640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69e8111370 0x7f69e8128f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.861+0000 7f69e77fe640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e810bd50 0x7f69e8123b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.861+0000 7f69e7fff640 1 -- 10.20.193.28:0/304645140 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69e8111370 msgr2=0x7f69e8128f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:48.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.861+0000 7f69e7fff640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69e8111370 0x7f69e8128f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.861+0000 7f69e7fff640 1 -- 10.20.193.28:0/304645140 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e810bd50 msgr2=0x7f69e8123b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:48.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.861+0000 7f69e7fff640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e810bd50 0x7f69e8123b30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.861+0000 7f69e7fff640 1 -- 10.20.193.28:0/304645140 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69dc00a650 con 0x7f69e810b360 2026-02-25T11:54:48.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.861+0000 7f69ec93d640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69e8111370 0x7f69e8128f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:48.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.861+0000 7f69e77fe640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e810bd50 0x7f69e8123b30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:48.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.861+0000 7f69e7fff640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e810b360 0x7f69e81235f0 secure :-1 s=READY pgs=187 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69d4009480 tx=0x7f69d400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:48.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.862+0000 7f69e57fa640 1 -- 10.20.193.28:0/304645140 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f69d400cd40 con 0x7f69e810b360 2026-02-25T11:54:48.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.862+0000 7f69e57fa640 1 -- 10.20.193.28:0/304645140 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f69d4004d10 con 0x7f69e810b360 2026-02-25T11:54:48.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.862+0000 7f69ed13e640 1 -- 10.20.193.28:0/304645140 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e8129b30 con 0x7f69e810b360 2026-02-25T11:54:48.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.862+0000 7f69e57fa640 1 -- 10.20.193.28:0/304645140 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f69d40106b0 con 0x7f69e810b360 2026-02-25T11:54:48.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.862+0000 7f69ed13e640 1 -- 10.20.193.28:0/304645140 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e816a230 con 0x7f69e810b360 2026-02-25T11:54:48.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.864+0000 7f69e57fa640 1 -- 10.20.193.28:0/304645140 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f69d401f060 con 0x7f69e810b360 2026-02-25T11:54:48.868 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.866+0000 7f69e57fa640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69bc0b5760 0x7f69bc0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.868 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.866+0000 7f69e57fa640 1 -- 10.20.193.28:0/304645140 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f69d4014070 con 0x7f69e810b360 2026-02-25T11:54:48.868 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.866+0000 7f69e77fe640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69bc0b5760 0x7f69bc0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.867+0000 7f69e77fe640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69bc0b5760 0x7f69bc0d5c10 secure :-1 s=READY pgs=338 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69e8129f90 tx=0x7f69d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:48.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.867+0000 7f69f03c2e00 1 Processor -- start 2026-02-25T11:54:48.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.867+0000 7f69f03c2e00 1 -- start start 2026-02-25T11:54:48.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.867+0000 7f69f03c2e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280140 0x55e9c6280510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.867+0000 7f69f03c2e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280a50 0x55e9c62c51b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.867+0000 7f69f03c2e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62e5ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.867+0000 7f69f03c2e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e9c61969b0 con 0x55e9c62c56f0 2026-02-25T11:54:48.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.867+0000 7f69f03c2e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e9c61926e0 con 0x55e9c6280a50 2026-02-25T11:54:48.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.867+0000 7f69f03c2e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e9c618c840 con 0x55e9c6280140 2026-02-25T11:54:48.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.868+0000 7f69e7fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280140 0x55e9c6280510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.868+0000 7f69e7fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280140 0x55e9c6280510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:43814/0 (socket says 10.20.193.28:43814) 2026-02-25T11:54:48.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.868+0000 7f69ec93d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62e5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.868+0000 7f69e7fff640 1 -- 10.20.193.28:0/863301220 learned_addr learned my addr 10.20.193.28:0/863301220 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:48.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.868+0000 7f69e77fe640 1 --2- 10.20.193.28:0/863301220 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280a50 0x55e9c62c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.868+0000 7f69e7fff640 1 -- 10.20.193.28:0/863301220 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280a50 msgr2=0x55e9c62c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:48.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.868+0000 7f69e7fff640 1 --2- 10.20.193.28:0/863301220 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280a50 0x55e9c62c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.868+0000 7f69e7fff640 1 -- 10.20.193.28:0/863301220 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 msgr2=0x55e9c62e5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:48.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.868+0000 7f69e7fff640 1 --2- 10.20.193.28:0/863301220 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62e5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.868+0000 7f69e7fff640 1 -- 10.20.193.28:0/863301220 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c6137d80 con 0x55e9c6280140 2026-02-25T11:54:48.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.868+0000 7f69e77fe640 1 --2- 10.20.193.28:0/863301220 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280a50 0x55e9c62c51b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:54:48.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.868+0000 7f69e7fff640 1 --2- 10.20.193.28:0/863301220 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280140 0x55e9c6280510 secure :-1 s=READY pgs=188 gs=2 cs=0 l=1 c_cookie=12066833088207026351 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69d40052c0 tx=0x7f69d40a8720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a77600c2ca8278af server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:48.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.868+0000 7f69b8ce3640 1 -- 10.20.193.28:0/863301220 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f69d40ad880 con 0x55e9c6280140 2026-02-25T11:54:48.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.869+0000 7f69b8ce3640 1 -- 10.20.193.28:0/863301220 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f69d40b2ba0 con 0x55e9c6280140 2026-02-25T11:54:48.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.869+0000 7f69f03c2e00 1 -- 10.20.193.28:0/863301220 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280140 msgr2=0x55e9c6280510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:48.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.869+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/863301220 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280140 0x55e9c6280510 secure :-1 s=READY pgs=188 gs=2 cs=0 l=1 c_cookie=12066833088207026351 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69d40052c0 tx=0x7f69d40a8720 comp rx=0 tx=0).stop 2026-02-25T11:54:48.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.869+0000 7f69f03c2e00 1 -- 10.20.193.28:0/863301220 shutdown_connections 2026-02-25T11:54:48.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.869+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/863301220 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62e5ad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.869+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/863301220 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280a50 0x55e9c62c51b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.869+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/863301220 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280140 0x55e9c6280510 unknown :-1 s=CLOSED pgs=188 gs=2 cs=0 l=1 c_cookie=12066833088207026351 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.869+0000 7f69f03c2e00 1 -- 10.20.193.28:0/863301220 >> 10.20.193.28:0/863301220 conn(0x55e9c619fd40 msgr2=0x55e9c627ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:48.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.869+0000 7f69f03c2e00 1 -- 10.20.193.28:0/863301220 shutdown_connections 2026-02-25T11:54:48.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.869+0000 7f69f03c2e00 1 -- 10.20.193.28:0/863301220 wait complete. 2026-02-25T11:54:48.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.870+0000 7f69f03c2e00 1 -- start start 2026-02-25T11:54:48.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.871+0000 7f69f03c2e00 1 Processor -- start 2026-02-25T11:54:48.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.871+0000 7f69f03c2e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c6280140 0x55e9c62dd270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.871+0000 7f69f03c2e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280a50 0x55e9c62dd7b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.871+0000 7f69f03c2e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c62c56f0 0x55e9c62e27a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.871+0000 7f69f03c2e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e9c62e2ce0 con 0x55e9c6280140 2026-02-25T11:54:48.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.871+0000 7f69f03c2e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e9c62e2e50 con 0x55e9c62c56f0 2026-02-25T11:54:48.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.871+0000 7f69f03c2e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e9c62e3010 con 0x55e9c6280a50 2026-02-25T11:54:48.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.871+0000 7f69e7fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c6280140 0x55e9c62dd270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69e7fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c6280140 0x55e9c62dd270 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57406/0 (socket says 10.20.193.28:57406) 2026-02-25T11:54:48.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69e7fff640 1 -- 10.20.193.28:0/4103821393 learned_addr learned my addr 10.20.193.28:0/4103821393 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:48.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69e77fe640 1 --2- 10.20.193.28:0/4103821393 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280a50 0x55e9c62dd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69ec93d640 1 --2- 10.20.193.28:0/4103821393 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c62c56f0 0x55e9c62e27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69e7fff640 1 -- 10.20.193.28:0/4103821393 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280a50 msgr2=0x55e9c62dd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:48.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69e7fff640 1 --2- 10.20.193.28:0/4103821393 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280a50 0x55e9c62dd7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69e7fff640 1 -- 10.20.193.28:0/4103821393 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c62c56f0 msgr2=0x55e9c62e27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:48.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69e7fff640 1 --2- 10.20.193.28:0/4103821393 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c62c56f0 0x55e9c62e27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69e7fff640 1 -- 10.20.193.28:0/4103821393 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c6110350 con 0x55e9c6280140 2026-02-25T11:54:48.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69e7fff640 1 --2- 10.20.193.28:0/4103821393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c6280140 0x55e9c62dd270 secure :-1 s=READY pgs=695 gs=1 cs=0 l=1 c_cookie=12694358709662543879 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69d400c6a0 tx=0x7f69d40a90f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02b6bfd24d27007 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:48.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69a9ffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:48.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69a9ffb640 1 -- 10.20.193.28:0/4103821393 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f69d40ad880 con 0x55e9c6280140 2026-02-25T11:54:48.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69a9ffb640 1 -- 10.20.193.28:0/4103821393 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f69d40b5180 con 0x55e9c6280140 2026-02-25T11:54:48.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69f03c2e00 1 -- 10.20.193.28:0/4103821393 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e816a230 con 0x55e9c6280140 2026-02-25T11:54:48.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69a9ffb640 1 -- 10.20.193.28:0/4103821393 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f69d40ab0c0 con 0x55e9c6280140 2026-02-25T11:54:48.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69f03c2e00 10 client.16776 _ll_register_callbacks cb 0x55e9c610d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:48.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.872+0000 7f69f03c2e00 1 -- 10.20.193.28:0/4103821393 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e9c5e226f0 con 0x55e9c6280140 2026-02-25T11:54:48.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.873+0000 7f69a9ffb640 1 -- 10.20.193.28:0/4103821393 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f69d40b7fc0 con 0x55e9c6280140 2026-02-25T11:54:48.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.873+0000 7f69a9ffb640 1 -- 10.20.193.28:0/4103821393 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f69d40b7de0 con 0x55e9c6280140 2026-02-25T11:54:48.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.873+0000 7f69f03c2e00 10 client.16776 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:48.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.873+0000 7f69f03c2e00 1 -- 10.20.193.28:0/4103821393 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55e9c6122150 con 0x55e9c6280140 2026-02-25T11:54:48.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.873+0000 7f69a9ffb640 1 -- 10.20.193.28:0/4103821393 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f69d40ae870 con 0x55e9c6280140 2026-02-25T11:54:48.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.873+0000 7f69f03c2e00 10 client.16776 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:48.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.873+0000 7f69f03c2e00 10 client.16776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:48.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.873+0000 7f69f03c2e00 20 client.16776 populate_metadata read hostname 'trial028' 2026-02-25T11:54:48.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.873+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/4103821393 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e9c62c2880 0x55e9c62c2d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.873+0000 7f69f03c2e00 4 client.16776 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:54:48.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.873+0000 7f69f03c2e00 1 -- 10.20.193.28:0/4103821393 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9c615e910 con 0x55e9c62c2880 2026-02-25T11:54:48.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.874+0000 7f69e77fe640 1 --2- 10.20.193.28:0/4103821393 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e9c62c2880 0x55e9c62c2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.874+0000 7f69a9ffb640 10 client.16776 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:48.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.874+0000 7f69e77fe640 1 --2- 10.20.193.28:0/4103821393 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e9c62c2880 0x55e9c62c2d20 crc :-1 s=READY pgs=122 gs=4 cs=0 l=0 c_cookie=10662206956960069455 s_cookie=5144811900864235513 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93f7c4bc8f4db74f server_cookie=47660b2a5b3707f9 in_seq=0 out_seq=0 2026-02-25T11:54:48.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69a9ffb640 1 -- 10.20.193.28:0/4103821393 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55e9c615e910 con 0x55e9c62c2880 2026-02-25T11:54:48.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69a9ffb640 10 client.16776 handle_command_reply: tid=0 2026-02-25T11:54:48.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69f03c2e00 1 client.16776 shutdown 2026-02-25T11:54:48.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69f03c2e00 1 -- 10.20.193.28:0/4103821393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c6280140 msgr2=0x55e9c62dd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:48.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/4103821393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c6280140 0x55e9c62dd270 secure :-1 s=READY pgs=695 gs=1 cs=0 l=1 c_cookie=12694358709662543879 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69d400c6a0 tx=0x7f69d40a90f0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69f03c2e00 1 -- 10.20.193.28:0/4103821393 shutdown_connections 2026-02-25T11:54:48.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/4103821393 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e9c62c2880 0x55e9c62c2d20 crc :-1 s=READY pgs=122 gs=4 cs=0 l=0 c_cookie=10662206956960069455 s_cookie=5144811900864235513 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/4103821393 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c62c56f0 0x55e9c62e27a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/4103821393 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280a50 0x55e9c62dd7b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/4103821393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c6280140 0x55e9c62dd270 unknown :-1 s=CLOSED pgs=695 gs=1 cs=0 l=1 c_cookie=12694358709662543879 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69f03c2e00 1 -- 10.20.193.28:0/4103821393 >> 10.20.193.28:0/4103821393 conn(0x55e9c619fd40 msgr2=0x55e9c627e230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:48.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69a9ffb640 0 client.16776 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:48.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69f03c2e00 1 -- 10.20.193.28:0/4103821393 shutdown_connections 2026-02-25T11:54:48.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69f03c2e00 1 -- 10.20.193.28:0/4103821393 wait complete. 2026-02-25T11:54:48.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.875+0000 7f69f03c2e00 20 client.16776 trim_cache size 0 max 16384 2026-02-25T11:54:48.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.884+0000 7f69f03c2e00 1 Processor -- start 2026-02-25T11:54:48.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.884+0000 7f69f03c2e00 1 -- start start 2026-02-25T11:54:48.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.884+0000 7f69f03c2e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280140 0x55e9c62c81e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69f03c2e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280a50 0x55e9c62da470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69f03c2e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62da9b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69f03c2e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e9c618a470 con 0x55e9c62c56f0 2026-02-25T11:54:48.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69f03c2e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e9c5e4a700 con 0x55e9c6280a50 2026-02-25T11:54:48.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69f03c2e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e9c62e2ed0 con 0x55e9c6280140 2026-02-25T11:54:48.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69e77fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280a50 0x55e9c62da470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69e7fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280140 0x55e9c62c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69ec93d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62da9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69e77fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280a50 0x55e9c62da470 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:35042/0 (socket says 10.20.193.28:35042) 2026-02-25T11:54:48.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69e77fe640 1 -- 10.20.193.28:0/3482830020 learned_addr learned my addr 10.20.193.28:0/3482830020 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:48.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69e77fe640 1 -- 10.20.193.28:0/3482830020 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280140 msgr2=0x55e9c62c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:48.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69e77fe640 1 --2- 10.20.193.28:0/3482830020 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280140 0x55e9c62c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69e77fe640 1 -- 10.20.193.28:0/3482830020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 msgr2=0x55e9c62da9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:48.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69e77fe640 1 --2- 10.20.193.28:0/3482830020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62da9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69e77fe640 1 -- 10.20.193.28:0/3482830020 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c62cf670 con 0x55e9c6280a50 2026-02-25T11:54:48.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.885+0000 7f69e77fe640 1 --2- 10.20.193.28:0/3482830020 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280a50 0x55e9c62da470 secure :-1 s=READY pgs=154 gs=1 cs=0 l=1 c_cookie=16857874444619209955 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69d80297d0 tx=0x7f69d8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f3337d93a9dce3 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:48.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.886+0000 7f69abfff640 1 -- 10.20.193.28:0/3482830020 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f69d8012bf0 con 0x55e9c6280a50 2026-02-25T11:54:48.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.886+0000 7f69abfff640 1 -- 10.20.193.28:0/3482830020 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f69d8022040 con 0x55e9c6280a50 2026-02-25T11:54:48.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.886+0000 7f69abfff640 1 -- 10.20.193.28:0/3482830020 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f69d801b650 con 0x55e9c6280a50 2026-02-25T11:54:48.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.886+0000 7f69f03c2e00 1 -- 10.20.193.28:0/3482830020 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280a50 msgr2=0x55e9c62da470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:48.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.886+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/3482830020 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280a50 0x55e9c62da470 secure :-1 s=READY pgs=154 gs=1 cs=0 l=1 c_cookie=16857874444619209955 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69d80297d0 tx=0x7f69d8014000 comp rx=0 tx=0).stop 2026-02-25T11:54:48.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.886+0000 7f69f03c2e00 1 -- 10.20.193.28:0/3482830020 shutdown_connections 2026-02-25T11:54:48.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.886+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/3482830020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62da9b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.886+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/3482830020 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280a50 0x55e9c62da470 unknown :-1 s=CLOSED pgs=154 gs=1 cs=0 l=1 c_cookie=16857874444619209955 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.886+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/3482830020 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280140 0x55e9c62c81e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.886+0000 7f69f03c2e00 1 -- 10.20.193.28:0/3482830020 >> 10.20.193.28:0/3482830020 conn(0x55e9c619fd40 msgr2=0x55e9c62c7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:48.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.886+0000 7f69f03c2e00 1 -- 10.20.193.28:0/3482830020 shutdown_connections 2026-02-25T11:54:48.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.886+0000 7f69f03c2e00 1 -- 10.20.193.28:0/3482830020 wait complete. 2026-02-25T11:54:48.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.887+0000 7f69f03c2e00 1 -- start start 2026-02-25T11:54:48.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.887+0000 7f69f03c2e00 1 Processor -- start 2026-02-25T11:54:48.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.887+0000 7f69f03c2e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280140 0x55e9c6263ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.887+0000 7f69f03c2e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280a50 0x55e9c6263ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.887+0000 7f69f03c2e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62c4440 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.887+0000 7f69f03c2e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e9c62c4a20 con 0x55e9c62c56f0 2026-02-25T11:54:48.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.887+0000 7f69f03c2e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e9c62c4b90 con 0x55e9c6280140 2026-02-25T11:54:48.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.887+0000 7f69f03c2e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e9c62c4d50 con 0x55e9c6280a50 2026-02-25T11:54:48.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.887+0000 7f69ec93d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.887+0000 7f69ec93d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62c4440 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57420/0 (socket says 10.20.193.28:57420) 2026-02-25T11:54:48.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.887+0000 7f69ec93d640 1 -- 10.20.193.28:0/2148559820 learned_addr learned my addr 10.20.193.28:0/2148559820 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:48.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.887+0000 7f69e7fff640 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280140 0x55e9c6263ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.887+0000 7f69e77fe640 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280a50 0x55e9c6263ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69ec93d640 1 -- 10.20.193.28:0/2148559820 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280a50 msgr2=0x55e9c6263ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:48.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69ec93d640 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280a50 0x55e9c6263ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69ec93d640 1 -- 10.20.193.28:0/2148559820 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280140 msgr2=0x55e9c6263ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:48.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69ec93d640 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280140 0x55e9c6263ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69ec93d640 1 -- 10.20.193.28:0/2148559820 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9c611cbd0 con 0x55e9c62c56f0 2026-02-25T11:54:48.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69e7fff640 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280140 0x55e9c6263ab0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:48.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69ec93d640 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62c4440 secure :-1 s=READY pgs=696 gs=2 cs=0 l=1 c_cookie=6615524506675716071 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69dc00a230 tx=0x7f69dc0039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcf107cff4c1fe7 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:48.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69a9ffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:48.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69a9ffb640 1 -- 10.20.193.28:0/2148559820 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f69dc03e040 con 0x55e9c62c56f0 2026-02-25T11:54:48.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69a9ffb640 1 -- 10.20.193.28:0/2148559820 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f69dc03f040 con 0x55e9c62c56f0 2026-02-25T11:54:48.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69a9ffb640 1 -- 10.20.193.28:0/2148559820 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f69dc0509c0 con 0x55e9c62c56f0 2026-02-25T11:54:48.895 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69f03c2e00 1 -- 10.20.193.28:0/2148559820 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69d8029500 con 0x55e9c62c56f0 2026-02-25T11:54:48.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69a9ffb640 1 -- 10.20.193.28:0/2148559820 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f69dc050b50 con 0x55e9c62c56f0 2026-02-25T11:54:48.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.888+0000 7f69a9ffb640 10 client.16782 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:54:48.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.889+0000 7f69f03c2e00 10 client.16782 _ll_register_callbacks cb 0x55e9c610d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:48.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.889+0000 7f69f03c2e00 1 -- 10.20.193.28:0/2148559820 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69d40b2ba0 con 0x55e9c62c56f0 2026-02-25T11:54:48.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.889+0000 7f69a9ffb640 1 -- 10.20.193.28:0/2148559820 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f69dc003b20 con 0x55e9c62c56f0 2026-02-25T11:54:48.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.889+0000 7f69f03c2e00 10 client.16782 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:48.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.889+0000 7f69f03c2e00 1 -- 10.20.193.28:0/2148559820 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55e9c6110680 con 0x55e9c62c56f0 2026-02-25T11:54:48.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.889+0000 7f69a9ffb640 1 -- 10.20.193.28:0/2148559820 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f69dc044040 con 0x55e9c62c56f0 2026-02-25T11:54:48.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.889+0000 7f69f03c2e00 10 client.16782 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:48.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.889+0000 7f69f03c2e00 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:48.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.889+0000 7f69f03c2e00 20 client.16782 populate_metadata read hostname 'trial028' 2026-02-25T11:54:48.896 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.889+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e9c63730b0 0x55e9c6393560 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:48.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.889+0000 7f69f03c2e00 4 client.16782 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:54:48.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.889+0000 7f69f03c2e00 1 -- 10.20.193.28:0/2148559820 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9c5e1ff40 con 0x55e9c63730b0 2026-02-25T11:54:48.897 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.889+0000 7f69e7fff640 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e9c63730b0 0x55e9c6393560 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:48.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.890+0000 7f69e7fff640 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e9c63730b0 0x55e9c6393560 crc :-1 s=READY pgs=123 gs=4 cs=0 l=0 c_cookie=11391909185380286675 s_cookie=6594315822265600351 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e18310c141a00d3 server_cookie=5b83b74d5ae1095f in_seq=0 out_seq=0 2026-02-25T11:54:48.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.890+0000 7f69a9ffb640 10 client.16782 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:48.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.890+0000 7f69a9ffb640 1 -- 10.20.193.28:0/2148559820 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x55e9c5e1ff40 con 0x55e9c63730b0 2026-02-25T11:54:48.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.890+0000 7f69a9ffb640 10 client.16782 handle_command_reply: tid=0 2026-02-25T11:54:48.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.890+0000 7f69f03c2e00 1 client.16782 shutdown 2026-02-25T11:54:48.898 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.890+0000 7f69f03c2e00 1 -- 10.20.193.28:0/2148559820 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 msgr2=0x55e9c62c4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:48.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.890+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62c4440 secure :-1 s=READY pgs=696 gs=2 cs=0 l=1 c_cookie=6615524506675716071 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69dc00a230 tx=0x7f69dc0039b0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69f03c2e00 1 -- 10.20.193.28:0/2148559820 shutdown_connections 2026-02-25T11:54:48.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e9c63730b0 0x55e9c6393560 crc :-1 s=READY pgs=123 gs=4 cs=0 l=0 c_cookie=11391909185380286675 s_cookie=6594315822265600351 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e9c62c56f0 0x55e9c62c4440 unknown :-1 s=CLOSED pgs=696 gs=2 cs=0 l=1 c_cookie=6615524506675716071 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e9c6280a50 0x55e9c6263ff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69f03c2e00 1 --2- 10.20.193.28:0/2148559820 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e9c6280140 0x55e9c6263ab0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69f03c2e00 1 -- 10.20.193.28:0/2148559820 >> 10.20.193.28:0/2148559820 conn(0x55e9c619fd40 msgr2=0x55e9c62d9460 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:48.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69a9ffb640 0 client.16782 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:48.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69f03c2e00 1 -- 10.20.193.28:0/2148559820 shutdown_connections 2026-02-25T11:54:48.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69f03c2e00 1 -- 10.20.193.28:0/2148559820 wait complete. 2026-02-25T11:54:48.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69f03c2e00 20 client.16782 trim_cache size 0 max 16384 2026-02-25T11:54:48.900 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:54:48.900 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:54:48.900 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:54:48.900 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:54:48.900 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:54:48.900 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:54:48.900 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:48.901 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:48.901 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:48.901 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:48.901 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:48.901 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:48.901 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:54:48.901 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:54:48.901 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:48.901 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:48.901 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:48.901 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:48.901 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:48.902 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:48.902 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:54:48.902 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:54:48.902 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:48.902 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:48.902 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:48.902 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:48.902 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:48.902 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:48.902 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:54:48.902 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:54:48.902 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:48.903 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:48.903 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:48.903 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:48.903 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:48.903 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:48.903 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:54:48.903 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:54:48.903 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:48.903 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:48.903 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:48.903 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:48.903 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:48.904 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:48.904 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:54:48.904 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:54:48.904 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:48.904 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:48.904 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:48.904 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:48.904 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:48.904 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:48.904 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:54:48.904 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:54:48.904 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:48.905 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:48.905 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:48.905 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:48.905 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:48.905 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:48.905 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:54:48.905 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:54:48.905 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:48.905 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:48.905 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:48.905 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:48.906 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:48.906 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:48.906 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:54:48.906 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:54:48.906 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:48.906 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:48.906 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:48.907 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:48.907 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:48.907 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:48.907 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:54:48.907 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:54:48.907 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:48.908 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:48.908 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:48.908 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:48.908 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:48.908 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:48.908 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:54:48.908 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:54:48.908 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:48.909 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:54:48.910 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:54:48.910 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:48.910 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:48.910 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:48.910 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:48.910 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:48.910 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:54:48.910 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:54:48.910 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:54:48.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69ed13e640 1 -- 10.20.193.28:0/304645140 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69bc0b5760 msgr2=0x7f69bc0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:48.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69ed13e640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69bc0b5760 0x7f69bc0d5c10 secure :-1 s=READY pgs=338 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69e8129f90 tx=0x7f69d8005f90 comp rx=0 tx=0).stop 2026-02-25T11:54:48.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69ed13e640 1 -- 10.20.193.28:0/304645140 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e810b360 msgr2=0x7f69e81235f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:48.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.891+0000 7f69ed13e640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e810b360 0x7f69e81235f0 secure :-1 s=READY pgs=187 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f69d4009480 tx=0x7f69d400ee60 comp rx=0 tx=0).stop 2026-02-25T11:54:48.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.892+0000 7f69ed13e640 1 -- 10.20.193.28:0/304645140 shutdown_connections 2026-02-25T11:54:48.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.892+0000 7f69ed13e640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f69bc0b5760 0x7f69bc0d5c10 unknown :-1 s=CLOSED pgs=338 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.892+0000 7f69ed13e640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69e8111370 0x7f69e8128f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.892+0000 7f69ed13e640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f69e810bd50 0x7f69e8123b30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.892+0000 7f69ed13e640 1 --2- 10.20.193.28:0/304645140 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f69e810b360 0x7f69e81235f0 unknown :-1 s=CLOSED pgs=187 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:48.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.892+0000 7f69ed13e640 1 -- 10.20.193.28:0/304645140 >> 10.20.193.28:0/304645140 conn(0x7f69e80e8e70 msgr2=0x7f69e8109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:48.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.892+0000 7f69ed13e640 1 -- 10.20.193.28:0/304645140 shutdown_connections 2026-02-25T11:54:48.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:48.892+0000 7f69ed13e640 1 -- 10.20.193.28:0/304645140 wait complete. 2026-02-25T11:54:49.034 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:54:49.034 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:54:49.219 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:49.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.395+0000 7f69057ed640 1 -- 10.20.193.28:0/1608671568 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f690010c8e0 msgr2=0x7f6900110eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:49.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.395+0000 7f69057ed640 1 --2- 10.20.193.28:0/1608671568 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f690010c8e0 0x7f6900110eb0 secure :-1 s=READY pgs=697 gs=1 cs=0 l=1 c_cookie=16436833929512917049 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f68e8009980 tx=0x7f68e802f190 comp rx=0 tx=0).stop 2026-02-25T11:54:49.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.395+0000 7f69057ed640 1 -- 10.20.193.28:0/1608671568 shutdown_connections 2026-02-25T11:54:49.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.395+0000 7f69057ed640 1 --2- 10.20.193.28:0/1608671568 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69001114d0 0x7f6900111970 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:49.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.395+0000 7f69057ed640 1 --2- 10.20.193.28:0/1608671568 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f690010c8e0 0x7f6900110eb0 unknown :-1 s=CLOSED pgs=697 gs=1 cs=0 l=1 c_cookie=16436833929512917049 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:49.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.395+0000 7f69057ed640 1 --2- 10.20.193.28:0/1608671568 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f690010bef0 0x7f690010c2c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:49.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.395+0000 7f69057ed640 1 -- 10.20.193.28:0/1608671568 >> 10.20.193.28:0/1608671568 conn(0x7f69000e9b40 msgr2=0x7f6900109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:49.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.395+0000 7f69057ed640 1 -- 10.20.193.28:0/1608671568 shutdown_connections 2026-02-25T11:54:49.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.396+0000 7f69057ed640 1 -- 10.20.193.28:0/1608671568 wait complete. 2026-02-25T11:54:49.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.396+0000 7f69057ed640 1 Processor -- start 2026-02-25T11:54:49.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.396+0000 7f69057ed640 1 -- start start 2026-02-25T11:54:49.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f69057ed640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f690010bef0 0x7f6900122970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:49.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f69057ed640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f690010c8e0 0x7f6900122eb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:49.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f69057ed640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69001114d0 0x7f6900128280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:49.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f69057ed640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f69001288a0 con 0x7f690010bef0 2026-02-25T11:54:49.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f69057ed640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6900128a10 con 0x7f69001114d0 2026-02-25T11:54:49.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f69057ed640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6900128bd0 con 0x7f690010c8e0 2026-02-25T11:54:49.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f68feffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f690010bef0 0x7f6900122970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:49.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f68feffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f690010bef0 0x7f6900122970 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57448/0 (socket says 10.20.193.28:57448) 2026-02-25T11:54:49.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f68feffd640 1 -- 10.20.193.28:0/1956351962 learned_addr learned my addr 10.20.193.28:0/1956351962 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:49.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f68ff7fe640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69001114d0 0x7f6900128280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:49.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f68fe7fc640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f690010c8e0 0x7f6900122eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:49.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f68feffd640 1 -- 10.20.193.28:0/1956351962 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f690010c8e0 msgr2=0x7f6900122eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:49.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f68feffd640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f690010c8e0 0x7f6900122eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:49.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f68feffd640 1 -- 10.20.193.28:0/1956351962 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69001114d0 msgr2=0x7f6900128280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:49.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f68feffd640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69001114d0 0x7f6900128280 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:49.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.397+0000 7f68feffd640 1 -- 10.20.193.28:0/1956351962 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e8009570 con 0x7f690010bef0 2026-02-25T11:54:49.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.398+0000 7f68feffd640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f690010bef0 0x7f6900122970 secure :-1 s=READY pgs=698 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f68f400b500 tx=0x7f68f400b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:49.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.398+0000 7f68dffff640 1 -- 10.20.193.28:0/1956351962 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f68f40044a0 con 0x7f690010bef0 2026-02-25T11:54:49.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.398+0000 7f69057ed640 1 -- 10.20.193.28:0/1956351962 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6900128eb0 con 0x7f690010bef0 2026-02-25T11:54:49.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.398+0000 7f69057ed640 1 -- 10.20.193.28:0/1956351962 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6900169630 con 0x7f690010bef0 2026-02-25T11:54:49.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.398+0000 7f68dffff640 1 -- 10.20.193.28:0/1956351962 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f68f4002b60 con 0x7f690010bef0 2026-02-25T11:54:49.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.398+0000 7f68dffff640 1 -- 10.20.193.28:0/1956351962 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f68f400fd20 con 0x7f690010bef0 2026-02-25T11:54:49.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.399+0000 7f69057ed640 1 -- 10.20.193.28:0/1956351962 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69000604a0 con 0x7f690010bef0 2026-02-25T11:54:49.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.401+0000 7f68dffff640 1 -- 10.20.193.28:0/1956351962 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f68f4002cd0 con 0x7f690010bef0 2026-02-25T11:54:49.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.402+0000 7f68dffff640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f68d40b5760 0x7f68d40d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:49.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.402+0000 7f68dffff640 1 -- 10.20.193.28:0/1956351962 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f68f40e65f0 con 0x7f690010bef0 2026-02-25T11:54:49.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.403+0000 7f68fe7fc640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f68d40b5760 0x7f68d40d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:49.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.403+0000 7f68dffff640 1 -- 10.20.193.28:0/1956351962 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f68f40a8fc0 con 0x7f690010bef0 2026-02-25T11:54:49.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.403+0000 7f68fe7fc640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f68d40b5760 0x7f68d40d5c10 secure :-1 s=READY pgs=339 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6900129310 tx=0x7f68e803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:49.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.543+0000 7f69057ed640 1 -- 10.20.193.28:0/1956351962 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6900123ad0 con 0x7f690010bef0 2026-02-25T11:54:49.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.543+0000 7f68dffff640 1 -- 10.20.193.28:0/1956351962 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f68f40ac170 con 0x7f690010bef0 2026-02-25T11:54:49.546 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:54:49.546 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:54:49.547 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:54:49.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.544+0000 7f69057ed640 1 -- 10.20.193.28:0/1956351962 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f68d40b5760 msgr2=0x7f68d40d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:49.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.544+0000 7f69057ed640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f68d40b5760 0x7f68d40d5c10 secure :-1 s=READY pgs=339 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6900129310 tx=0x7f68e803a040 comp rx=0 tx=0).stop 2026-02-25T11:54:49.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.544+0000 7f69057ed640 1 -- 10.20.193.28:0/1956351962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f690010bef0 msgr2=0x7f6900122970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:49.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.544+0000 7f69057ed640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f690010bef0 0x7f6900122970 secure :-1 s=READY pgs=698 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f68f400b500 tx=0x7f68f400b9d0 comp rx=0 tx=0).stop 2026-02-25T11:54:49.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.545+0000 7f69057ed640 1 -- 10.20.193.28:0/1956351962 shutdown_connections 2026-02-25T11:54:49.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.545+0000 7f69057ed640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f68d40b5760 0x7f68d40d5c10 unknown :-1 s=CLOSED pgs=339 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:49.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.545+0000 7f69057ed640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f69001114d0 0x7f6900128280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:49.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.545+0000 7f69057ed640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f690010c8e0 0x7f6900122eb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:49.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.545+0000 7f69057ed640 1 --2- 10.20.193.28:0/1956351962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f690010bef0 0x7f6900122970 unknown :-1 s=CLOSED pgs=698 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:49.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.545+0000 7f69057ed640 1 -- 10.20.193.28:0/1956351962 >> 10.20.193.28:0/1956351962 conn(0x7f69000e9b40 msgr2=0x7f6900109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:49.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.545+0000 7f69057ed640 1 -- 10.20.193.28:0/1956351962 shutdown_connections 2026-02-25T11:54:49.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:49.545+0000 7f69057ed640 1 -- 10.20.193.28:0/1956351962 wait complete. 2026-02-25T11:54:49.679 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:54:49.864 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:50.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.042+0000 7f56be444640 1 -- 10.20.193.28:0/3984283840 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56b8110080 msgr2=0x7f56b81150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.042+0000 7f56be444640 1 --2- 10.20.193.28:0/3984283840 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56b8110080 0x7f56b81150d0 secure :-1 s=READY pgs=699 gs=1 cs=0 l=1 c_cookie=12069007249878284794 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56ac009950 tx=0x7f56ac02f190 comp rx=0 tx=0).stop 2026-02-25T11:54:50.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.042+0000 7f56be444640 1 -- 10.20.193.28:0/3984283840 shutdown_connections 2026-02-25T11:54:50.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.042+0000 7f56be444640 1 --2- 10.20.193.28:0/3984283840 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56b8115880 0x7f56b8176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.042+0000 7f56be444640 1 --2- 10.20.193.28:0/3984283840 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56b8110080 0x7f56b81150d0 unknown :-1 s=CLOSED pgs=699 gs=1 cs=0 l=1 c_cookie=12069007249878284794 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.042+0000 7f56be444640 1 --2- 10.20.193.28:0/3984283840 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f56b810f690 0x7f56b810fa60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.042+0000 7f56be444640 1 -- 10.20.193.28:0/3984283840 >> 10.20.193.28:0/3984283840 conn(0x7f56b80ecf00 msgr2=0x7f56b810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:50.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.042+0000 7f56be444640 1 -- 10.20.193.28:0/3984283840 shutdown_connections 2026-02-25T11:54:50.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.042+0000 7f56be444640 1 -- 10.20.193.28:0/3984283840 wait complete. 2026-02-25T11:54:50.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.043+0000 7f56be444640 1 Processor -- start 2026-02-25T11:54:50.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.043+0000 7f56be444640 1 -- start start 2026-02-25T11:54:50.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56be444640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56b810f690 0x7f56b80fe880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56be444640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56b8110080 0x7f56b8176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56be444640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f56b8115880 0x7f56b80fef80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56be444640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f56b8103c90 con 0x7f56b8110080 2026-02-25T11:54:50.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56be444640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56b8103e00 con 0x7f56b8115880 2026-02-25T11:54:50.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56be444640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56b8103fc0 con 0x7f56b810f690 2026-02-25T11:54:50.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56bd442640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56b810f690 0x7f56b80fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56bd442640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56b810f690 0x7f56b80fe880 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:43862/0 (socket says 10.20.193.28:43862) 2026-02-25T11:54:50.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56bd442640 1 -- 10.20.193.28:0/704386829 learned_addr learned my addr 10.20.193.28:0/704386829 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:50.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56bcc41640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56b8110080 0x7f56b8176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56bdc43640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f56b8115880 0x7f56b80fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56bd442640 1 -- 10.20.193.28:0/704386829 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f56b8115880 msgr2=0x7f56b80fef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56bd442640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f56b8115880 0x7f56b80fef80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56bd442640 1 -- 10.20.193.28:0/704386829 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56b8110080 msgr2=0x7f56b8176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56bd442640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56b8110080 0x7f56b8176150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.044+0000 7f56bd442640 1 -- 10.20.193.28:0/704386829 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac009570 con 0x7f56b810f690 2026-02-25T11:54:50.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.045+0000 7f56bd442640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56b810f690 0x7f56b80fe880 secure :-1 s=READY pgs=189 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a800d950 tx=0x7f56a800de20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:50.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.045+0000 7f56a67fc640 1 -- 10.20.193.28:0/704386829 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56a8004540 con 0x7f56b810f690 2026-02-25T11:54:50.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.045+0000 7f56a67fc640 1 -- 10.20.193.28:0/704386829 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f56a8009ce0 con 0x7f56b810f690 2026-02-25T11:54:50.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.045+0000 7f56be444640 1 -- 10.20.193.28:0/704386829 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b81042a0 con 0x7f56b810f690 2026-02-25T11:54:50.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.045+0000 7f56be444640 1 -- 10.20.193.28:0/704386829 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b81047c0 con 0x7f56b810f690 2026-02-25T11:54:50.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.045+0000 7f56a67fc640 1 -- 10.20.193.28:0/704386829 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56a80106e0 con 0x7f56b810f690 2026-02-25T11:54:50.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.048+0000 7f56a67fc640 1 -- 10.20.193.28:0/704386829 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f56a80097c0 con 0x7f56b810f690 2026-02-25T11:54:50.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.050+0000 7f56a67fc640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56940b5750 0x7f56940d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.050+0000 7f56a67fc640 1 -- 10.20.193.28:0/704386829 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f56a80e6d80 con 0x7f56b810f690 2026-02-25T11:54:50.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.050+0000 7f56bcc41640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56940b5750 0x7f56940d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.050+0000 7f56bcc41640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56940b5750 0x7f56940d5c00 secure :-1 s=READY pgs=340 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56b8104d20 tx=0x7f56ac03a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:50.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.050+0000 7f56c16c8e00 1 Processor -- start 2026-02-25T11:54:50.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56c16c8e00 1 -- start start 2026-02-25T11:54:50.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56c16c8e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a35fe5b30 0x561a35fe5f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56c16c8e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe6440 0x561a3602aba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56c16c8e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a3602b0e0 0x561a3604b4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56c16c8e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x561a35efb8a0 con 0x561a35fe6440 2026-02-25T11:54:50.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56c16c8e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561a35ef75d0 con 0x561a35fe5b30 2026-02-25T11:54:50.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56c16c8e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x561a35ef1570 con 0x561a3602b0e0 2026-02-25T11:54:50.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56bd442640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a35fe5b30 0x561a35fe5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56bdc43640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a3602b0e0 0x561a3604b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56bd442640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a35fe5b30 0x561a35fe5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:35074/0 (socket says 10.20.193.28:35074) 2026-02-25T11:54:50.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56bd442640 1 -- 10.20.193.28:0/2677361778 learned_addr learned my addr 10.20.193.28:0/2677361778 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:50.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56bcc41640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe6440 0x561a3602aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56bd442640 1 -- 10.20.193.28:0/2677361778 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a3602b0e0 msgr2=0x561a3604b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56bd442640 1 --2- 10.20.193.28:0/2677361778 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a3602b0e0 0x561a3604b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56bd442640 1 -- 10.20.193.28:0/2677361778 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe6440 msgr2=0x561a3602aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56bd442640 1 --2- 10.20.193.28:0/2677361778 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe6440 0x561a3602aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.051+0000 7f56bd442640 1 -- 10.20.193.28:0/2677361778 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a35e9cd10 con 0x561a35fe5b30 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.052+0000 7f56bd442640 1 --2- 10.20.193.28:0/2677361778 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a35fe5b30 0x561a35fe5f00 secure :-1 s=READY pgs=155 gs=1 cs=0 l=1 c_cookie=2890136207043266394 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a8005aa0 tx=0x7f56a80a9070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=281bd391554f075a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.052+0000 7f5685ce5640 1 -- 10.20.193.28:0/2677361778 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56a8009e50 con 0x561a35fe5b30 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.052+0000 7f5685ce5640 1 -- 10.20.193.28:0/2677361778 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f56a80adb00 con 0x561a35fe5b30 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.052+0000 7f5685ce5640 1 -- 10.20.193.28:0/2677361778 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56a80add90 con 0x561a35fe5b30 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.052+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2677361778 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a35fe5b30 msgr2=0x561a35fe5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.052+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/2677361778 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a35fe5b30 0x561a35fe5f00 secure :-1 s=READY pgs=155 gs=1 cs=0 l=1 c_cookie=2890136207043266394 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a8005aa0 tx=0x7f56a80a9070 comp rx=0 tx=0).stop 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.052+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2677361778 shutdown_connections 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.052+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/2677361778 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a3602b0e0 0x561a3604b4c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.052+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/2677361778 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe6440 0x561a3602aba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.052+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/2677361778 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a35fe5b30 0x561a35fe5f00 unknown :-1 s=CLOSED pgs=155 gs=1 cs=0 l=1 c_cookie=2890136207043266394 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.052+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2677361778 >> 10.20.193.28:0/2677361778 conn(0x561a35f047a0 msgr2=0x561a35fe4410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.052+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2677361778 shutdown_connections 2026-02-25T11:54:50.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.052+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2677361778 wait complete. 2026-02-25T11:54:50.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.054+0000 7f56c16c8e00 1 -- start start 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.054+0000 7f56c16c8e00 1 Processor -- start 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.054+0000 7f56c16c8e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe5b30 0x561a36006980 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.054+0000 7f56c16c8e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a35fe6440 0x561a36006ec0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.054+0000 7f56c16c8e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a3602b0e0 0x561a3600beb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.054+0000 7f56c16c8e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x561a3600c3f0 con 0x561a3602b0e0 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.054+0000 7f56c16c8e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561a3600c560 con 0x561a35fe6440 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.054+0000 7f56c16c8e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x561a3600c720 con 0x561a35fe5b30 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.054+0000 7f56bdc43640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a3602b0e0 0x561a3600beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.054+0000 7f56bd442640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe5b30 0x561a36006980 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.054+0000 7f56bd442640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe5b30 0x561a36006980 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:43880/0 (socket says 10.20.193.28:43880) 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.054+0000 7f56bd442640 1 -- 10.20.193.28:0/2978874168 learned_addr learned my addr 10.20.193.28:0/2978874168 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.054+0000 7f56bcc41640 1 --2- 10.20.193.28:0/2978874168 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a35fe6440 0x561a36006ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f56bdc43640 1 -- 10.20.193.28:0/2978874168 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe5b30 msgr2=0x561a36006980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f56bdc43640 1 --2- 10.20.193.28:0/2978874168 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe5b30 0x561a36006980 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f56bdc43640 1 -- 10.20.193.28:0/2978874168 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a35fe6440 msgr2=0x561a36006ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f56bdc43640 1 --2- 10.20.193.28:0/2978874168 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a35fe6440 0x561a36006ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f56bdc43640 1 -- 10.20.193.28:0/2978874168 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a35e753c0 con 0x561a3602b0e0 2026-02-25T11:54:50.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f56bdc43640 1 --2- 10.20.193.28:0/2978874168 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a3602b0e0 0x561a3600beb0 secure :-1 s=READY pgs=700 gs=2 cs=0 l=1 c_cookie=8932008980861090682 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56b4003c00 tx=0x7f56b4013820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf4ded822d2fb7a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:50.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f5672ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:50.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f5672ffd640 1 -- 10.20.193.28:0/2978874168 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56b401c440 con 0x561a3602b0e0 2026-02-25T11:54:50.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f5672ffd640 1 -- 10.20.193.28:0/2978874168 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f56b401ca60 con 0x561a3602b0e0 2026-02-25T11:54:50.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2978874168 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b81047c0 con 0x561a3602b0e0 2026-02-25T11:54:50.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f5672ffd640 1 -- 10.20.193.28:0/2978874168 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56b4010c10 con 0x561a3602b0e0 2026-02-25T11:54:50.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f56c16c8e00 10 client.16800 _ll_register_callbacks cb 0x561a35e72a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:50.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2978874168 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561a35b876f0 con 0x561a3602b0e0 2026-02-25T11:54:50.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f5672ffd640 1 -- 10.20.193.28:0/2978874168 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f56b40103d0 con 0x561a3602b0e0 2026-02-25T11:54:50.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.055+0000 7f5672ffd640 1 -- 10.20.193.28:0/2978874168 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f56b40105c0 con 0x561a3602b0e0 2026-02-25T11:54:50.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.056+0000 7f56c16c8e00 10 client.16800 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:50.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.056+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2978874168 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x561a35e870e0 con 0x561a3602b0e0 2026-02-25T11:54:50.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.056+0000 7f5672ffd640 1 -- 10.20.193.28:0/2978874168 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f56b401cbd0 con 0x561a3602b0e0 2026-02-25T11:54:50.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.056+0000 7f56c16c8e00 10 client.16800 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:50.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.056+0000 7f56c16c8e00 10 client.16800 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:54:50.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.056+0000 7f56c16c8e00 20 client.16800 populate_metadata read hostname 'trial028' 2026-02-25T11:54:50.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.056+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/2978874168 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x561a36011210 0x561a360116b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.056+0000 7f56c16c8e00 4 client.16800 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:54:50.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.056+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2978874168 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a35ec3fc0 con 0x561a36011210 2026-02-25T11:54:50.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.056+0000 7f56bd442640 1 --2- 10.20.193.28:0/2978874168 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x561a36011210 0x561a360116b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.057+0000 7f56bd442640 1 --2- 10.20.193.28:0/2978874168 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x561a36011210 0x561a360116b0 crc :-1 s=READY pgs=47 gs=4 cs=0 l=0 c_cookie=13698782583565167907 s_cookie=13155447907031792942 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be1bda2e34b5ad23 server_cookie=b6918a2bbfc93d2e in_seq=0 out_seq=0 2026-02-25T11:54:50.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.057+0000 7f5672ffd640 10 client.16800 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:54:50.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.057+0000 7f5672ffd640 1 -- 10.20.193.28:0/2978874168 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x561a35ec3fc0 con 0x561a36011210 2026-02-25T11:54:50.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.057+0000 7f5672ffd640 10 client.16800 handle_command_reply: tid=0 2026-02-25T11:54:50.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.057+0000 7f56c16c8e00 1 client.16800 shutdown 2026-02-25T11:54:50.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.058+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2978874168 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a3602b0e0 msgr2=0x561a3600beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.058+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/2978874168 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a3602b0e0 0x561a3600beb0 secure :-1 s=READY pgs=700 gs=2 cs=0 l=1 c_cookie=8932008980861090682 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56b4003c00 tx=0x7f56b4013820 comp rx=0 tx=0).stop 2026-02-25T11:54:50.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.058+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2978874168 shutdown_connections 2026-02-25T11:54:50.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.058+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/2978874168 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x561a36011210 0x561a360116b0 crc :-1 s=READY pgs=47 gs=4 cs=0 l=0 c_cookie=13698782583565167907 s_cookie=13155447907031792942 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.058+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/2978874168 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a3602b0e0 0x561a3600beb0 unknown :-1 s=CLOSED pgs=700 gs=2 cs=0 l=1 c_cookie=8932008980861090682 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.058+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/2978874168 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a35fe6440 0x561a36006ec0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.058+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/2978874168 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe5b30 0x561a36006980 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.058+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2978874168 >> 10.20.193.28:0/2978874168 conn(0x561a35f047a0 msgr2=0x561a35fe4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:50.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.058+0000 7f5672ffd640 0 client.16800 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:54:50.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.058+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2978874168 shutdown_connections 2026-02-25T11:54:50.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.058+0000 7f56c16c8e00 1 -- 10.20.193.28:0/2978874168 wait complete. 2026-02-25T11:54:50.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.058+0000 7f56c16c8e00 20 client.16800 trim_cache size 0 max 16384 2026-02-25T11:54:50.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.065+0000 7f56c16c8e00 1 Processor -- start 2026-02-25T11:54:50.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.065+0000 7f56c16c8e00 1 -- start start 2026-02-25T11:54:50.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.065+0000 7f56c16c8e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 0x561a35ffb360 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.065+0000 7f56c16c8e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe6440 0x561a35ffb8a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.065+0000 7f56c16c8e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a3602b0e0 0x561a360066c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.065+0000 7f56c16c8e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x561a35eef1a0 con 0x561a35fe5b30 2026-02-25T11:54:50.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.065+0000 7f56c16c8e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561a35baf700 con 0x561a3602b0e0 2026-02-25T11:54:50.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.065+0000 7f56c16c8e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x561a3600bfc0 con 0x561a35fe6440 2026-02-25T11:54:50.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.065+0000 7f56bd442640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 0x561a35ffb360 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.065+0000 7f56bcc41640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe6440 0x561a35ffb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.065+0000 7f56bdc43640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a3602b0e0 0x561a360066c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.065+0000 7f56bd442640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 0x561a35ffb360 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57502/0 (socket says 10.20.193.28:57502) 2026-02-25T11:54:50.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.065+0000 7f56bd442640 1 -- 10.20.193.28:0/4060284789 learned_addr learned my addr 10.20.193.28:0/4060284789 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:50.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56bd442640 1 -- 10.20.193.28:0/4060284789 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe6440 msgr2=0x561a35ffb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56bd442640 1 --2- 10.20.193.28:0/4060284789 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe6440 0x561a35ffb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56bd442640 1 -- 10.20.193.28:0/4060284789 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a3602b0e0 msgr2=0x561a360066c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56bd442640 1 --2- 10.20.193.28:0/4060284789 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a3602b0e0 0x561a360066c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56bd442640 1 -- 10.20.193.28:0/4060284789 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a35ff0890 con 0x561a35fe5b30 2026-02-25T11:54:50.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56bd442640 1 --2- 10.20.193.28:0/4060284789 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 0x561a35ffb360 secure :-1 s=READY pgs=701 gs=3 cs=0 l=1 c_cookie=747806915675381875 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a800bfd0 tx=0x7f56a80d2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60be6d81696073 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:50.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56854e4640 1 -- 10.20.193.28:0/4060284789 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56a80b8240 con 0x561a35fe5b30 2026-02-25T11:54:50.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56854e4640 1 -- 10.20.193.28:0/4060284789 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f56a80027c0 con 0x561a35fe5b30 2026-02-25T11:54:50.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56854e4640 1 -- 10.20.193.28:0/4060284789 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56a80b8030 con 0x561a35fe5b30 2026-02-25T11:54:50.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56c16c8e00 1 -- 10.20.193.28:0/4060284789 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 msgr2=0x561a35ffb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/4060284789 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 0x561a35ffb360 secure :-1 s=READY pgs=701 gs=3 cs=0 l=1 c_cookie=747806915675381875 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a800bfd0 tx=0x7f56a80d2040 comp rx=0 tx=0).stop 2026-02-25T11:54:50.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56c16c8e00 1 -- 10.20.193.28:0/4060284789 shutdown_connections 2026-02-25T11:54:50.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/4060284789 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a3602b0e0 0x561a360066c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/4060284789 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe6440 0x561a35ffb8a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/4060284789 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 0x561a35ffb360 unknown :-1 s=CLOSED pgs=701 gs=3 cs=0 l=1 c_cookie=747806915675381875 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56c16c8e00 1 -- 10.20.193.28:0/4060284789 >> 10.20.193.28:0/4060284789 conn(0x561a35f047a0 msgr2=0x561a35fe89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:50.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56c16c8e00 1 -- 10.20.193.28:0/4060284789 shutdown_connections 2026-02-25T11:54:50.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.066+0000 7f56c16c8e00 1 -- 10.20.193.28:0/4060284789 wait complete. 2026-02-25T11:54:50.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.067+0000 7f56c16c8e00 1 -- start start 2026-02-25T11:54:50.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.067+0000 7f56c16c8e00 1 Processor -- start 2026-02-25T11:54:50.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.067+0000 7f56c16c8e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 0x561a36017740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.067+0000 7f56c16c8e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe6440 0x561a36013dc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.067+0000 7f56c16c8e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a3602b0e0 0x561a36014300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.067+0000 7f56c16c8e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x561a36014840 con 0x561a35fe5b30 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.067+0000 7f56c16c8e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561a360149b0 con 0x561a3602b0e0 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.067+0000 7f56c16c8e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x561a36014b70 con 0x561a35fe6440 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.067+0000 7f56bd442640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 0x561a36017740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f56bd442640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 0x561a36017740 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57510/0 (socket says 10.20.193.28:57510) 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f56bd442640 1 -- 10.20.193.28:0/1850825265 learned_addr learned my addr 10.20.193.28:0/1850825265 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f56bcc41640 1 --2- 10.20.193.28:0/1850825265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe6440 0x561a36013dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f56bdc43640 1 --2- 10.20.193.28:0/1850825265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a3602b0e0 0x561a36014300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f56bd442640 1 -- 10.20.193.28:0/1850825265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe6440 msgr2=0x561a36013dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f56bd442640 1 --2- 10.20.193.28:0/1850825265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe6440 0x561a36013dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f56bd442640 1 -- 10.20.193.28:0/1850825265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a3602b0e0 msgr2=0x561a36014300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f56bd442640 1 --2- 10.20.193.28:0/1850825265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a3602b0e0 0x561a36014300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f56bd442640 1 -- 10.20.193.28:0/1850825265 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a35e81b60 con 0x561a35fe5b30 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f56bd442640 1 --2- 10.20.193.28:0/1850825265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 0x561a36017740 secure :-1 s=READY pgs=702 gs=1 cs=0 l=1 c_cookie=13733773073325798086 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a80d2010 tx=0x7f56a80adc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9829d87568b2c6 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f5672ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f5672ffd640 1 -- 10.20.193.28:0/1850825265 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56a80b8240 con 0x561a35fe5b30 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f5672ffd640 1 -- 10.20.193.28:0/1850825265 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f56a80b5400 con 0x561a35fe5b30 2026-02-25T11:54:50.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f5672ffd640 1 -- 10.20.193.28:0/1850825265 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56a8072200 con 0x561a35fe5b30 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f56c16c8e00 1 -- 10.20.193.28:0/1850825265 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a35e9cd10 con 0x561a35fe5b30 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f5672ffd640 1 -- 10.20.193.28:0/1850825265 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f56a8072390 con 0x561a35fe5b30 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f5672ffd640 10 client.16806 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f56c16c8e00 10 client.16806 _ll_register_callbacks cb 0x561a35e72a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.068+0000 7f56c16c8e00 1 -- 10.20.193.28:0/1850825265 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56a80adb00 con 0x561a35fe5b30 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.069+0000 7f5672ffd640 1 -- 10.20.193.28:0/1850825265 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f56a80add90 con 0x561a35fe5b30 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.069+0000 7f56c16c8e00 10 client.16806 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.069+0000 7f56c16c8e00 1 -- 10.20.193.28:0/1850825265 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x561a35e756f0 con 0x561a35fe5b30 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.069+0000 7f5672ffd640 1 -- 10.20.193.28:0/1850825265 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f56a80ab0e0 con 0x561a35fe5b30 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.069+0000 7f56c16c8e00 10 client.16806 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.069+0000 7f56c16c8e00 10 client.16806 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.069+0000 7f56c16c8e00 20 client.16806 populate_metadata read hostname 'trial028' 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.069+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/1850825265 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x561a360e0a10 0x561a36100ec0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.069+0000 7f56c16c8e00 4 client.16806 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.069+0000 7f56c16c8e00 1 -- 10.20.193.28:0/1850825265 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a35b84f40 con 0x561a360e0a10 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.069+0000 7f56bcc41640 1 --2- 10.20.193.28:0/1850825265 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x561a360e0a10 0x561a36100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f56bcc41640 1 --2- 10.20.193.28:0/1850825265 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x561a360e0a10 0x561a36100ec0 crc :-1 s=READY pgs=48 gs=4 cs=0 l=0 c_cookie=3221669734345732661 s_cookie=15286544486824544328 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2cb5ab8de34df235 server_cookie=d424b7381c620848 in_seq=0 out_seq=0 2026-02-25T11:54:50.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f5672ffd640 10 client.16806 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f5672ffd640 1 -- 10.20.193.28:0/1850825265 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a35b84f40 con 0x561a360e0a10 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f5672ffd640 10 client.16806 handle_command_reply: tid=0 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f56c16c8e00 1 client.16806 shutdown 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f56c16c8e00 1 -- 10.20.193.28:0/1850825265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 msgr2=0x561a36017740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/1850825265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 0x561a36017740 secure :-1 s=READY pgs=702 gs=1 cs=0 l=1 c_cookie=13733773073325798086 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a80d2010 tx=0x7f56a80adc70 comp rx=0 tx=0).stop 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f56c16c8e00 1 -- 10.20.193.28:0/1850825265 shutdown_connections 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/1850825265 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x561a360e0a10 0x561a36100ec0 crc :-1 s=READY pgs=48 gs=4 cs=0 l=0 c_cookie=3221669734345732661 s_cookie=15286544486824544328 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/1850825265 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561a3602b0e0 0x561a36014300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/1850825265 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561a35fe6440 0x561a36013dc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f56c16c8e00 1 --2- 10.20.193.28:0/1850825265 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x561a35fe5b30 0x561a36017740 unknown :-1 s=CLOSED pgs=702 gs=1 cs=0 l=1 c_cookie=13733773073325798086 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f5672ffd640 0 client.16806 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f56c16c8e00 1 -- 10.20.193.28:0/1850825265 >> 10.20.193.28:0/1850825265 conn(0x561a35f047a0 msgr2=0x561a360066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f56c16c8e00 1 -- 10.20.193.28:0/1850825265 shutdown_connections 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f56c16c8e00 1 -- 10.20.193.28:0/1850825265 wait complete. 2026-02-25T11:54:50.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.070+0000 7f56c16c8e00 20 client.16806 trim_cache size 0 max 16384 2026-02-25T11:54:50.076 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:54:50.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.071+0000 7f56be444640 1 -- 10.20.193.28:0/704386829 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56940b5750 msgr2=0x7f56940d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.071+0000 7f56be444640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56940b5750 0x7f56940d5c00 secure :-1 s=READY pgs=340 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56b8104d20 tx=0x7f56ac03a040 comp rx=0 tx=0).stop 2026-02-25T11:54:50.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.071+0000 7f56be444640 1 -- 10.20.193.28:0/704386829 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56b810f690 msgr2=0x7f56b80fe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.071+0000 7f56be444640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56b810f690 0x7f56b80fe880 secure :-1 s=READY pgs=189 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a800d950 tx=0x7f56a800de20 comp rx=0 tx=0).stop 2026-02-25T11:54:50.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.071+0000 7f56be444640 1 -- 10.20.193.28:0/704386829 shutdown_connections 2026-02-25T11:54:50.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.071+0000 7f56be444640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56940b5750 0x7f56940d5c00 unknown :-1 s=CLOSED pgs=340 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.071+0000 7f56be444640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f56b8115880 0x7f56b80fef80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.071+0000 7f56be444640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56b8110080 0x7f56b8176150 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.071+0000 7f56be444640 1 --2- 10.20.193.28:0/704386829 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56b810f690 0x7f56b80fe880 unknown :-1 s=CLOSED pgs=189 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.071+0000 7f56be444640 1 -- 10.20.193.28:0/704386829 >> 10.20.193.28:0/704386829 conn(0x7f56b80ecf00 msgr2=0x7f56b8111e90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:50.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.071+0000 7f56be444640 1 -- 10.20.193.28:0/704386829 shutdown_connections 2026-02-25T11:54:50.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.071+0000 7f56be444640 1 -- 10.20.193.28:0/704386829 wait complete. 2026-02-25T11:54:50.171 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:54:50.356 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:50.435 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:50 trial028 ceph-mon[39224]: pgmap v424: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:54:50.435 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:50 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1956351962' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:54:50.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.550+0000 7f88291bb640 1 -- 10.20.193.28:0/2421433868 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8824057970 msgr2=0x7f8824057d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.550+0000 7f88291bb640 1 --2- 10.20.193.28:0/2421433868 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8824057970 0x7f8824057d40 secure :-1 s=READY pgs=703 gs=2 cs=0 l=1 c_cookie=5005692889437045428 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8818009950 tx=0x7f881802f190 comp rx=0 tx=0).stop 2026-02-25T11:54:50.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.550+0000 7f88291bb640 1 -- 10.20.193.28:0/2421433868 shutdown_connections 2026-02-25T11:54:50.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.550+0000 7f88291bb640 1 --2- 10.20.193.28:0/2421433868 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f882415e610 0x7f882417e9f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.550+0000 7f88291bb640 1 --2- 10.20.193.28:0/2421433868 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8824058280 0x7f882415e0d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.550+0000 7f88291bb640 1 --2- 10.20.193.28:0/2421433868 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8824057970 0x7f8824057d40 unknown :-1 s=CLOSED pgs=703 gs=2 cs=0 l=1 c_cookie=5005692889437045428 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.550+0000 7f88291bb640 1 -- 10.20.193.28:0/2421433868 >> 10.20.193.28:0/2421433868 conn(0x7f882405dbd0 msgr2=0x7f882405dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:50.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.550+0000 7f88291bb640 1 -- 10.20.193.28:0/2421433868 shutdown_connections 2026-02-25T11:54:50.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.550+0000 7f88291bb640 1 -- 10.20.193.28:0/2421433868 wait complete. 2026-02-25T11:54:50.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.551+0000 7f88291bb640 1 Processor -- start 2026-02-25T11:54:50.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.551+0000 7f88291bb640 1 -- start start 2026-02-25T11:54:50.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f88291bb640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8824057970 0x7f882410a0b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f88291bb640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8824058280 0x7f882410a5f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f88291bb640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f882415e610 0x7f882410f9c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f88291bb640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f882410ffe0 con 0x7f8824057970 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f88291bb640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8824110150 con 0x7f8824058280 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f88291bb640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8824110310 con 0x7f882415e610 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8822d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8824057970 0x7f882410a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8822d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8824057970 0x7f882410a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57532/0 (socket says 10.20.193.28:57532) 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8822d76640 1 -- 10.20.193.28:0/3205967423 learned_addr learned my addr 10.20.193.28:0/3205967423 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8822575640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8824058280 0x7f882410a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8823577640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f882415e610 0x7f882410f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8822d76640 1 -- 10.20.193.28:0/3205967423 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f882415e610 msgr2=0x7f882410f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8822d76640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f882415e610 0x7f882410f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8822d76640 1 -- 10.20.193.28:0/3205967423 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8824058280 msgr2=0x7f882410a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8822d76640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8824058280 0x7f882410a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8822d76640 1 -- 10.20.193.28:0/3205967423 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8818009570 con 0x7f8824057970 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8822d76640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8824057970 0x7f882410a0b0 secure :-1 s=READY pgs=704 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f88180098a0 tx=0x7f88180028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8803fff640 1 -- 10.20.193.28:0/3205967423 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f881803d070 con 0x7f8824057970 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8803fff640 1 -- 10.20.193.28:0/3205967423 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8818002a40 con 0x7f8824057970 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.552+0000 7f8803fff640 1 -- 10.20.193.28:0/3205967423 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8818041720 con 0x7f8824057970 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.553+0000 7f88291bb640 1 -- 10.20.193.28:0/3205967423 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8824110590 con 0x7f8824057970 2026-02-25T11:54:50.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.553+0000 7f88291bb640 1 -- 10.20.193.28:0/3205967423 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8824110b30 con 0x7f8824057970 2026-02-25T11:54:50.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.556+0000 7f8803fff640 1 -- 10.20.193.28:0/3205967423 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f88180386a0 con 0x7f8824057970 2026-02-25T11:54:50.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.558+0000 7f8803fff640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f87f80b5760 0x7f87f80d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.558+0000 7f8803fff640 1 -- 10.20.193.28:0/3205967423 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f881810aff0 con 0x7f8824057970 2026-02-25T11:54:50.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.558+0000 7f882b1b6e00 1 Processor -- start 2026-02-25T11:54:50.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.558+0000 7f8822575640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f87f80b5760 0x7f87f80d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.558+0000 7f882b1b6e00 1 -- start start 2026-02-25T11:54:50.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.558+0000 7f882b1b6e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88743b30 0x55ae88743f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.558+0000 7f882b1b6e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88744440 0x55ae88788ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.558+0000 7f882b1b6e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae887890e0 0x55ae887a94c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.558+0000 7f882b1b6e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ae886598a0 con 0x55ae88743b30 2026-02-25T11:54:50.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.558+0000 7f882b1b6e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ae886555d0 con 0x55ae887890e0 2026-02-25T11:54:50.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.558+0000 7f882b1b6e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae8864f570 con 0x55ae88744440 2026-02-25T11:54:50.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f8822575640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f87f80b5760 0x7f87f80d5c10 secure :-1 s=READY pgs=341 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8824110a50 tx=0x7f880c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:50.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f8822575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88744440 0x55ae88788ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f8822575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88744440 0x55ae88788ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:43940/0 (socket says 10.20.193.28:43940) 2026-02-25T11:54:50.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f8822575640 1 -- 10.20.193.28:0/2641492632 learned_addr learned my addr 10.20.193.28:0/2641492632 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:50.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f8822d76640 1 --2- 10.20.193.28:0/2641492632 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88743b30 0x55ae88743f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f8823577640 1 --2- 10.20.193.28:0/2641492632 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae887890e0 0x55ae887a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f8822575640 1 -- 10.20.193.28:0/2641492632 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae887890e0 msgr2=0x55ae887a94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f8822575640 1 --2- 10.20.193.28:0/2641492632 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae887890e0 0x55ae887a94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f8822575640 1 -- 10.20.193.28:0/2641492632 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88743b30 msgr2=0x55ae88743f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f8822575640 1 --2- 10.20.193.28:0/2641492632 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88743b30 0x55ae88743f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f8822575640 1 -- 10.20.193.28:0/2641492632 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae885fad10 con 0x55ae88744440 2026-02-25T11:54:50.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f8822d76640 1 --2- 10.20.193.28:0/2641492632 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88743b30 0x55ae88743f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:50.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f8822575640 1 --2- 10.20.193.28:0/2641492632 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88744440 0x55ae88788ba0 secure :-1 s=READY pgs=190 gs=2 cs=0 l=1 c_cookie=5838380447722590316 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f880c004120 tx=0x7f880c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510618158a17506c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:50.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.559+0000 7f87ef7fe640 1 -- 10.20.193.28:0/2641492632 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f880c002e40 con 0x55ae88744440 2026-02-25T11:54:50.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.560+0000 7f87ef7fe640 1 -- 10.20.193.28:0/2641492632 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f880c01d8e0 con 0x55ae88744440 2026-02-25T11:54:50.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.560+0000 7f87ef7fe640 1 -- 10.20.193.28:0/2641492632 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f880c01c970 con 0x55ae88744440 2026-02-25T11:54:50.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.560+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2641492632 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88744440 msgr2=0x55ae88788ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.560+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/2641492632 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88744440 0x55ae88788ba0 secure :-1 s=READY pgs=190 gs=2 cs=0 l=1 c_cookie=5838380447722590316 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f880c004120 tx=0x7f880c014000 comp rx=0 tx=0).stop 2026-02-25T11:54:50.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.560+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2641492632 shutdown_connections 2026-02-25T11:54:50.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.560+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/2641492632 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae887890e0 0x55ae887a94c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.560+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/2641492632 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88744440 0x55ae88788ba0 unknown :-1 s=CLOSED pgs=190 gs=2 cs=0 l=1 c_cookie=5838380447722590316 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.560+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/2641492632 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88743b30 0x55ae88743f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.560+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2641492632 >> 10.20.193.28:0/2641492632 conn(0x55ae886627a0 msgr2=0x55ae88742410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:50.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.560+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2641492632 shutdown_connections 2026-02-25T11:54:50.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.560+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2641492632 wait complete. 2026-02-25T11:54:50.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.561+0000 7f882b1b6e00 1 -- start start 2026-02-25T11:54:50.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.562+0000 7f882b1b6e00 1 Processor -- start 2026-02-25T11:54:50.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.562+0000 7f882b1b6e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae88743b30 0x55ae8875bde0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.562+0000 7f882b1b6e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88744440 0x55ae8875c320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.562+0000 7f882b1b6e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae887890e0 0x55ae88761310 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.562+0000 7f882b1b6e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ae88761850 con 0x55ae887890e0 2026-02-25T11:54:50.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.562+0000 7f882b1b6e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ae887619c0 con 0x55ae88743b30 2026-02-25T11:54:50.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.562+0000 7f882b1b6e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae88761b80 con 0x55ae88744440 2026-02-25T11:54:50.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.562+0000 7f8822d76640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae88743b30 0x55ae8875bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.562+0000 7f8822d76640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae88743b30 0x55ae8875bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:35132/0 (socket says 10.20.193.28:35132) 2026-02-25T11:54:50.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.562+0000 7f8822d76640 1 -- 10.20.193.28:0/2600393264 learned_addr learned my addr 10.20.193.28:0/2600393264 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:50.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.562+0000 7f8823577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae887890e0 0x55ae88761310 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.562+0000 7f8822575640 1 --2- 10.20.193.28:0/2600393264 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88744440 0x55ae8875c320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f8822d76640 1 -- 10.20.193.28:0/2600393264 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88744440 msgr2=0x55ae8875c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f8822d76640 1 --2- 10.20.193.28:0/2600393264 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88744440 0x55ae8875c320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f8822d76640 1 -- 10.20.193.28:0/2600393264 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae887890e0 msgr2=0x55ae88761310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f8822d76640 1 --2- 10.20.193.28:0/2600393264 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae887890e0 0x55ae88761310 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f8822d76640 1 -- 10.20.193.28:0/2600393264 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c002a10 con 0x55ae88743b30 2026-02-25T11:54:50.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f8823577640 1 --2- 10.20.193.28:0/2600393264 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae887890e0 0x55ae88761310 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:54:50.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f8822d76640 1 --2- 10.20.193.28:0/2600393264 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae88743b30 0x55ae8875bde0 secure :-1 s=READY pgs=156 gs=1 cs=0 l=1 c_cookie=14361340180195100281 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f88180d1600 tx=0x7f8818048f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c74dbacde312c679 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:50.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f87ecff9640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:54:50.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f87ecff9640 1 -- 10.20.193.28:0/2600393264 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f88180d0d70 con 0x55ae88743b30 2026-02-25T11:54:50.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f87ecff9640 1 -- 10.20.193.28:0/2600393264 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f88180d0b90 con 0x55ae88743b30 2026-02-25T11:54:50.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f87ecff9640 1 -- 10.20.193.28:0/2600393264 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f88180cf370 con 0x55ae88743b30 2026-02-25T11:54:50.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2600393264 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8824110b30 con 0x55ae88743b30 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f87ecff9640 1 -- 10.20.193.28:0/2600393264 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f88180cf4d0 con 0x55ae88743b30 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.563+0000 7f87ecff9640 10 client.26558 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.564+0000 7f882b1b6e00 10 client.26558 _ll_register_callbacks cb 0x55ae885d0a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.564+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2600393264 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ae882e56f0 con 0x55ae88743b30 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.564+0000 7f87ecff9640 1 -- 10.20.193.28:0/2600393264 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f88180d70b0 con 0x55ae88743b30 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.564+0000 7f882b1b6e00 10 client.26558 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.564+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2600393264 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55ae885d33c0 con 0x55ae88743b30 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.564+0000 7f87ecff9640 1 -- 10.20.193.28:0/2600393264 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f88180d2d80 con 0x55ae88743b30 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.564+0000 7f882b1b6e00 10 client.26558 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.564+0000 7f882b1b6e00 10 client.26558 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.564+0000 7f882b1b6e00 20 client.26558 populate_metadata read hostname 'trial028' 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.564+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/2600393264 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55ae88786340 0x55ae887867e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.564+0000 7f882b1b6e00 4 client.26558 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.564+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2600393264 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae88621fc0 con 0x55ae88786340 2026-02-25T11:54:50.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.564+0000 7f8822575640 1 --2- 10.20.193.28:0/2600393264 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55ae88786340 0x55ae887867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.565+0000 7f87ecff9640 10 client.26558 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:54:50.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.565+0000 7f8822575640 1 --2- 10.20.193.28:0/2600393264 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55ae88786340 0x55ae887867e0 crc :-1 s=READY pgs=48 gs=4 cs=0 l=0 c_cookie=5503271317133174206 s_cookie=6606460817803240545 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c5f8c1af851a5be server_cookie=5baedd1c7bb2e461 in_seq=0 out_seq=0 2026-02-25T11:54:50.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.565+0000 7f87ecff9640 1 -- 10.20.193.28:0/2600393264 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55ae88621fc0 con 0x55ae88786340 2026-02-25T11:54:50.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.565+0000 7f87ecff9640 10 client.26558 handle_command_reply: tid=0 2026-02-25T11:54:50.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.565+0000 7f882b1b6e00 1 client.26558 shutdown 2026-02-25T11:54:50.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.566+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2600393264 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae88743b30 msgr2=0x55ae8875bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.566+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/2600393264 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae88743b30 0x55ae8875bde0 secure :-1 s=READY pgs=156 gs=1 cs=0 l=1 c_cookie=14361340180195100281 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f88180d1600 tx=0x7f8818048f50 comp rx=0 tx=0).stop 2026-02-25T11:54:50.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.566+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2600393264 shutdown_connections 2026-02-25T11:54:50.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.566+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/2600393264 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55ae88786340 0x55ae887867e0 crc :-1 s=READY pgs=48 gs=4 cs=0 l=0 c_cookie=5503271317133174206 s_cookie=6606460817803240545 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.566+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/2600393264 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae887890e0 0x55ae88761310 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.566+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/2600393264 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88744440 0x55ae8875c320 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.566+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/2600393264 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae88743b30 0x55ae8875bde0 unknown :-1 s=CLOSED pgs=156 gs=1 cs=0 l=1 c_cookie=14361340180195100281 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.566+0000 7f87ecff9640 0 client.26558 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:54:50.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.566+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2600393264 >> 10.20.193.28:0/2600393264 conn(0x55ae886627a0 msgr2=0x55ae88742b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:50.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.566+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2600393264 shutdown_connections 2026-02-25T11:54:50.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.566+0000 7f882b1b6e00 1 -- 10.20.193.28:0/2600393264 wait complete. 2026-02-25T11:54:50.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.566+0000 7f882b1b6e00 20 client.26558 trim_cache size 0 max 16384 2026-02-25T11:54:50.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.573+0000 7f882b1b6e00 1 Processor -- start 2026-02-25T11:54:50.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.573+0000 7f882b1b6e00 1 -- start start 2026-02-25T11:54:50.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.573+0000 7f882b1b6e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae88743b30 0x55ae88760880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.573+0000 7f882b1b6e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 0x55ae88787eb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.573+0000 7f882b1b6e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae887890e0 0x55ae887883f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.573+0000 7f882b1b6e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ae8864d1a0 con 0x55ae88744440 2026-02-25T11:54:50.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.573+0000 7f882b1b6e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ae8830d700 con 0x55ae88743b30 2026-02-25T11:54:50.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.573+0000 7f882b1b6e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae88718a80 con 0x55ae887890e0 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f8822575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 0x55ae88787eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f8822575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 0x55ae88787eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57562/0 (socket says 10.20.193.28:57562) 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f8822575640 1 -- 10.20.193.28:0/3794293424 learned_addr learned my addr 10.20.193.28:0/3794293424 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f8822d76640 1 --2- 10.20.193.28:0/3794293424 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae88743b30 0x55ae88760880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f8823577640 1 --2- 10.20.193.28:0/3794293424 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae887890e0 0x55ae887883f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f8822575640 1 -- 10.20.193.28:0/3794293424 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae887890e0 msgr2=0x55ae887883f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f8822575640 1 --2- 10.20.193.28:0/3794293424 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae887890e0 0x55ae887883f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f8822575640 1 -- 10.20.193.28:0/3794293424 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae88743b30 msgr2=0x55ae88760880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f8822575640 1 --2- 10.20.193.28:0/3794293424 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae88743b30 0x55ae88760880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f8822575640 1 -- 10.20.193.28:0/3794293424 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae8874e750 con 0x55ae88744440 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f8822d76640 1 --2- 10.20.193.28:0/3794293424 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae88743b30 0x55ae88760880 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f8823577640 1 --2- 10.20.193.28:0/3794293424 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae887890e0 0x55ae887883f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f8822575640 1 --2- 10.20.193.28:0/3794293424 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 0x55ae88787eb0 secure :-1 s=READY pgs=705 gs=1 cs=0 l=1 c_cookie=962968031230270133 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f880c005830 tx=0x7f880c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d265287899ab5 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f87eeffd640 1 -- 10.20.193.28:0/3794293424 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f880c02b040 con 0x55ae88744440 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f87eeffd640 1 -- 10.20.193.28:0/3794293424 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f880c02d040 con 0x55ae88744440 2026-02-25T11:54:50.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.574+0000 7f87eeffd640 1 -- 10.20.193.28:0/3794293424 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f880c034910 con 0x55ae88744440 2026-02-25T11:54:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.575+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3794293424 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 msgr2=0x55ae88787eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.575+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/3794293424 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 0x55ae88787eb0 secure :-1 s=READY pgs=705 gs=1 cs=0 l=1 c_cookie=962968031230270133 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f880c005830 tx=0x7f880c013040 comp rx=0 tx=0).stop 2026-02-25T11:54:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.575+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3794293424 shutdown_connections 2026-02-25T11:54:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.575+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/3794293424 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae887890e0 0x55ae887883f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.575+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/3794293424 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 0x55ae88787eb0 unknown :-1 s=CLOSED pgs=705 gs=1 cs=0 l=1 c_cookie=962968031230270133 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.575+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/3794293424 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae88743b30 0x55ae88760880 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.575+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3794293424 >> 10.20.193.28:0/3794293424 conn(0x55ae886627a0 msgr2=0x55ae88760170 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.575+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3794293424 shutdown_connections 2026-02-25T11:54:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.575+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3794293424 wait complete. 2026-02-25T11:54:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.575+0000 7f882b1b6e00 1 -- start start 2026-02-25T11:54:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.576+0000 7f882b1b6e00 1 Processor -- start 2026-02-25T11:54:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.576+0000 7f882b1b6e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88743b30 0x55ae887636e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.576+0000 7f882b1b6e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 0x55ae88763c20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.576+0000 7f882b1b6e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae887890e0 0x55ae887cd5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.576+0000 7f882b1b6e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ae887cdb50 con 0x55ae88744440 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.576+0000 7f882b1b6e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ae887cdcc0 con 0x55ae887890e0 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.576+0000 7f882b1b6e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae887cde80 con 0x55ae88743b30 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.576+0000 7f8822575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 0x55ae88763c20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.576+0000 7f8823577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae887890e0 0x55ae887cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.576+0000 7f8822d76640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88743b30 0x55ae887636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.576+0000 7f8822575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 0x55ae88763c20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57574/0 (socket says 10.20.193.28:57574) 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.576+0000 7f8822575640 1 -- 10.20.193.28:0/3886807945 learned_addr learned my addr 10.20.193.28:0/3886807945 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.576+0000 7f8823577640 1 -- 10.20.193.28:0/3886807945 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88743b30 msgr2=0x55ae887636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f8823577640 1 --2- 10.20.193.28:0/3886807945 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88743b30 0x55ae887636e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f8823577640 1 -- 10.20.193.28:0/3886807945 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 msgr2=0x55ae88763c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f8823577640 1 --2- 10.20.193.28:0/3886807945 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 0x55ae88763c20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f8823577640 1 -- 10.20.193.28:0/3886807945 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae885fad10 con 0x55ae887890e0 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f8822575640 1 --2- 10.20.193.28:0/3886807945 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 0x55ae88763c20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f8822d76640 1 --2- 10.20.193.28:0/3886807945 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88743b30 0x55ae887636e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f8823577640 1 --2- 10.20.193.28:0/3886807945 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae887890e0 0x55ae887cd5e0 secure :-1 s=READY pgs=157 gs=3 cs=0 l=1 c_cookie=10564022033642639227 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8814005820 tx=0x7f881400aa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=929af211aaab4f7b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f87ecff9640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f87ecff9640 1 -- 10.20.193.28:0/3886807945 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8814019070 con 0x55ae887890e0 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f87ecff9640 1 -- 10.20.193.28:0/3886807945 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8814013d40 con 0x55ae887890e0 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3886807945 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae885d33c0 con 0x55ae887890e0 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f87ecff9640 1 -- 10.20.193.28:0/3886807945 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f881401e420 con 0x55ae887890e0 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f882b1b6e00 10 client.26564 _ll_register_callbacks cb 0x55ae885d0a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3886807945 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f880c01d8e0 con 0x55ae887890e0 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f87ecff9640 1 -- 10.20.193.28:0/3886807945 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f88140263a0 con 0x55ae887890e0 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f87ecff9640 1 -- 10.20.193.28:0/3886807945 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f88140265c0 con 0x55ae887890e0 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f882b1b6e00 10 client.26564 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:50.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.577+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3886807945 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55ae885e50e0 con 0x55ae887890e0 2026-02-25T11:54:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.578+0000 7f87ecff9640 1 -- 10.20.193.28:0/3886807945 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f8814013880 con 0x55ae887890e0 2026-02-25T11:54:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.578+0000 7f882b1b6e00 10 client.26564 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.578+0000 7f882b1b6e00 10 client.26564 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:54:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.578+0000 7f882b1b6e00 20 client.26564 populate_metadata read hostname 'trial028' 2026-02-25T11:54:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.578+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/3886807945 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55ae88836040 0x55ae888564f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.578+0000 7f882b1b6e00 4 client.26564 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:54:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.578+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3886807945 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae882e2f40 con 0x55ae88836040 2026-02-25T11:54:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.578+0000 7f8822d76640 1 --2- 10.20.193.28:0/3886807945 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55ae88836040 0x55ae888564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.578+0000 7f8822d76640 1 --2- 10.20.193.28:0/3886807945 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55ae88836040 0x55ae888564f0 crc :-1 s=READY pgs=49 gs=4 cs=0 l=0 c_cookie=18060883365561592906 s_cookie=17045025971611459814 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=faa525baf60f6c4a server_cookie=ec8c18d0113560e6 in_seq=0 out_seq=0 2026-02-25T11:54:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.578+0000 7f87ecff9640 10 client.26564 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:54:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.578+0000 7f87ecff9640 1 -- 10.20.193.28:0/3886807945 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae882e2f40 con 0x55ae88836040 2026-02-25T11:54:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.578+0000 7f87ecff9640 10 client.26564 handle_command_reply: tid=0 2026-02-25T11:54:50.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.578+0000 7f882b1b6e00 1 client.26564 shutdown 2026-02-25T11:54:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3886807945 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae887890e0 msgr2=0x55ae887cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/3886807945 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae887890e0 0x55ae887cd5e0 secure :-1 s=READY pgs=157 gs=3 cs=0 l=1 c_cookie=10564022033642639227 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8814005820 tx=0x7f881400aa00 comp rx=0 tx=0).stop 2026-02-25T11:54:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3886807945 shutdown_connections 2026-02-25T11:54:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/3886807945 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55ae88836040 0x55ae888564f0 crc :-1 s=READY pgs=49 gs=4 cs=0 l=0 c_cookie=18060883365561592906 s_cookie=17045025971611459814 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/3886807945 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ae887890e0 0x55ae887cd5e0 unknown :-1 s=CLOSED pgs=157 gs=3 cs=0 l=1 c_cookie=10564022033642639227 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/3886807945 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ae88744440 0x55ae88763c20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f882b1b6e00 1 --2- 10.20.193.28:0/3886807945 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ae88743b30 0x55ae887636e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f87ecff9640 0 client.26564 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:54:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3886807945 >> 10.20.193.28:0/3886807945 conn(0x55ae886627a0 msgr2=0x55ae88742500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3886807945 shutdown_connections 2026-02-25T11:54:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f882b1b6e00 1 -- 10.20.193.28:0/3886807945 wait complete. 2026-02-25T11:54:50.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f882b1b6e00 20 client.26564 trim_cache size 0 max 16384 2026-02-25T11:54:50.581 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:54:50.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f88291bb640 1 -- 10.20.193.28:0/3205967423 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f87f80b5760 msgr2=0x7f87f80d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f88291bb640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f87f80b5760 0x7f87f80d5c10 secure :-1 s=READY pgs=341 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8824110a50 tx=0x7f880c005f90 comp rx=0 tx=0).stop 2026-02-25T11:54:50.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f88291bb640 1 -- 10.20.193.28:0/3205967423 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8824057970 msgr2=0x7f882410a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:50.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.579+0000 7f88291bb640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8824057970 0x7f882410a0b0 secure :-1 s=READY pgs=704 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f88180098a0 tx=0x7f88180028d0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.580+0000 7f88291bb640 1 -- 10.20.193.28:0/3205967423 shutdown_connections 2026-02-25T11:54:50.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.580+0000 7f88291bb640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f87f80b5760 0x7f87f80d5c10 unknown :-1 s=CLOSED pgs=341 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.580+0000 7f88291bb640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f882415e610 0x7f882410f9c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.580+0000 7f88291bb640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8824058280 0x7f882410a5f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.580+0000 7f88291bb640 1 --2- 10.20.193.28:0/3205967423 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8824057970 0x7f882410a0b0 unknown :-1 s=CLOSED pgs=704 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:50.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.580+0000 7f88291bb640 1 -- 10.20.193.28:0/3205967423 >> 10.20.193.28:0/3205967423 conn(0x7f882405dbd0 msgr2=0x7f8824166770 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:50.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.580+0000 7f88291bb640 1 -- 10.20.193.28:0/3205967423 shutdown_connections 2026-02-25T11:54:50.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:50.580+0000 7f88291bb640 1 -- 10.20.193.28:0/3205967423 wait complete. 2026-02-25T11:54:50.681 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:54:50.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:50 trial167 ceph-mon[43392]: pgmap v424: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:54:50.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:50 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1956351962' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:54:50.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:50 trial119 ceph-mon[42710]: pgmap v424: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:54:50.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:50 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1956351962' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:54:50.865 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:51.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.040+0000 7fdb7affd640 1 -- 10.20.193.28:0/3047368507 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdb68038470 con 0x7fdb7c10d3d0 2026-02-25T11:54:51.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.040+0000 7fdb814ba640 1 -- 10.20.193.28:0/3047368507 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdb7c10d3d0 msgr2=0x7fdb7c10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:51.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.040+0000 7fdb814ba640 1 --2- 10.20.193.28:0/3047368507 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdb7c10d3d0 0x7fdb7c10d7a0 secure :-1 s=READY pgs=191 gs=1 cs=0 l=1 c_cookie=2106645755966027663 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdb68009950 tx=0x7fdb6802f190 comp rx=0 tx=0).stop 2026-02-25T11:54:51.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.040+0000 7fdb814ba640 1 -- 10.20.193.28:0/3047368507 shutdown_connections 2026-02-25T11:54:51.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.040+0000 7fdb814ba640 1 --2- 10.20.193.28:0/3047368507 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdb7c1135c0 0x7fdb7c173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.040+0000 7fdb814ba640 1 --2- 10.20.193.28:0/3047368507 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdb7c10ddc0 0x7fdb7c112e10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.040+0000 7fdb814ba640 1 --2- 10.20.193.28:0/3047368507 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdb7c10d3d0 0x7fdb7c10d7a0 unknown :-1 s=CLOSED pgs=191 gs=1 cs=0 l=1 c_cookie=2106645755966027663 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.040+0000 7fdb814ba640 1 -- 10.20.193.28:0/3047368507 >> 10.20.193.28:0/3047368507 conn(0x7fdb7c0eaca0 msgr2=0x7fdb7c10b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:51.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.040+0000 7fdb814ba640 1 -- 10.20.193.28:0/3047368507 shutdown_connections 2026-02-25T11:54:51.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.040+0000 7fdb814ba640 1 -- 10.20.193.28:0/3047368507 wait complete. 2026-02-25T11:54:51.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.041+0000 7fdb814ba640 1 Processor -- start 2026-02-25T11:54:51.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.041+0000 7fdb814ba640 1 -- start start 2026-02-25T11:54:51.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.041+0000 7fdb814ba640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdb7c10d3d0 0x7fdb7c0f5980 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.041+0000 7fdb814ba640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdb7c10ddc0 0x7fdb7c173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.041+0000 7fdb814ba640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdb7c1135c0 0x7fdb7c0f6080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.041+0000 7fdb814ba640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c0fb240 con 0x7fdb7c10d3d0 2026-02-25T11:54:51.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.041+0000 7fdb814ba640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c0fb3b0 con 0x7fdb7c10ddc0 2026-02-25T11:54:51.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.041+0000 7fdb814ba640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdb7c0fb570 con 0x7fdb7c1135c0 2026-02-25T11:54:51.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb7bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdb7c10d3d0 0x7fdb7c0f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb7b7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdb7c10ddc0 0x7fdb7c173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb7bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdb7c10d3d0 0x7fdb7c0f5980 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57588/0 (socket says 10.20.193.28:57588) 2026-02-25T11:54:51.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb7b7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdb7c10ddc0 0x7fdb7c173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:35168/0 (socket says 10.20.193.28:35168) 2026-02-25T11:54:51.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb7bfff640 1 -- 10.20.193.28:0/573536673 learned_addr learned my addr 10.20.193.28:0/573536673 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:51.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb80cb9640 1 --2- 10.20.193.28:0/573536673 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdb7c1135c0 0x7fdb7c0f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb7bfff640 1 -- 10.20.193.28:0/573536673 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdb7c1135c0 msgr2=0x7fdb7c0f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:51.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/573536673 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdb7c1135c0 0x7fdb7c0f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb7bfff640 1 -- 10.20.193.28:0/573536673 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdb7c10ddc0 msgr2=0x7fdb7c173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:51.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/573536673 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdb7c10ddc0 0x7fdb7c173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb7bfff640 1 -- 10.20.193.28:0/573536673 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb68009570 con 0x7fdb7c10d3d0 2026-02-25T11:54:51.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/573536673 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdb7c10d3d0 0x7fdb7c0f5980 secure :-1 s=READY pgs=706 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdb680098a0 tx=0x7fdb680028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:51.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb797fa640 1 -- 10.20.193.28:0/573536673 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdb6803d070 con 0x7fdb7c10d3d0 2026-02-25T11:54:51.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb797fa640 1 -- 10.20.193.28:0/573536673 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdb68005dd0 con 0x7fdb7c10d3d0 2026-02-25T11:54:51.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb797fa640 1 -- 10.20.193.28:0/573536673 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdb680417b0 con 0x7fdb7c10d3d0 2026-02-25T11:54:51.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb814ba640 1 -- 10.20.193.28:0/573536673 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7c0fb7f0 con 0x7fdb7c10d3d0 2026-02-25T11:54:51.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.042+0000 7fdb814ba640 1 -- 10.20.193.28:0/573536673 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb7c0fbc10 con 0x7fdb7c10d3d0 2026-02-25T11:54:51.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.046+0000 7fdb797fa640 1 -- 10.20.193.28:0/573536673 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fdb680386a0 con 0x7fdb7c10d3d0 2026-02-25T11:54:51.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.047+0000 7fdb797fa640 1 --2- 10.20.193.28:0/573536673 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdb500b56d0 0x7fdb500d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.047+0000 7fdb797fa640 1 -- 10.20.193.28:0/573536673 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fdb6810b140 con 0x7fdb7c10d3d0 2026-02-25T11:54:51.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.047+0000 7fdb7b7fe640 1 --2- 10.20.193.28:0/573536673 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdb500b56d0 0x7fdb500d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.048+0000 7fdb8473ee00 1 Processor -- start 2026-02-25T11:54:51.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.048+0000 7fdb8473ee00 1 -- start start 2026-02-25T11:54:51.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.048+0000 7fdb7b7fe640 1 --2- 10.20.193.28:0/573536673 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdb500b56d0 0x7fdb500d5b80 secure :-1 s=READY pgs=342 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdb7c0fc2d0 tx=0x7fdb6c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.048+0000 7fdb8473ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0cbf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.048+0000 7fdb8473ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0cc440 0x55fa3a110ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.048+0000 7fdb8473ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a1110e0 0x55fa3a1314c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.048+0000 7fdb8473ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55fa39fe18a0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.048+0000 7fdb8473ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55fa39fdd5d0 con 0x55fa3a0cc440 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.048+0000 7fdb8473ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55fa39fd7570 con 0x55fa3a1110e0 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.048+0000 7fdb7bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.048+0000 7fdb7bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0cbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57596/0 (socket says 10.20.193.28:57596) 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.048+0000 7fdb7bfff640 1 -- 10.20.193.28:0/2788832155 learned_addr learned my addr 10.20.193.28:0/2788832155 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.048+0000 7fdb80cb9640 1 --2- 10.20.193.28:0/2788832155 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a1110e0 0x55fa3a1314c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb7bfff640 1 -- 10.20.193.28:0/2788832155 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a1110e0 msgr2=0x55fa3a1314c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/2788832155 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a1110e0 0x55fa3a1314c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb7b7fe640 1 --2- 10.20.193.28:0/2788832155 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0cc440 0x55fa3a110ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb7bfff640 1 -- 10.20.193.28:0/2788832155 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0cc440 msgr2=0x55fa3a110ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/2788832155 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0cc440 0x55fa3a110ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb7bfff640 1 -- 10.20.193.28:0/2788832155 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa39f82d10 con 0x55fa3a0cbb30 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb7b7fe640 1 --2- 10.20.193.28:0/2788832155 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0cc440 0x55fa3a110ba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:54:51.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/2788832155 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0cbf00 secure :-1 s=READY pgs=707 gs=1 cs=0 l=1 c_cookie=12354780327832602000 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdb680d1550 tx=0x7fdb6802fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab74ff3fee724590 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:51.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb4cce3640 1 -- 10.20.193.28:0/2788832155 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdb680d0d70 con 0x55fa3a0cbb30 2026-02-25T11:54:51.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb4cce3640 1 -- 10.20.193.28:0/2788832155 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdb68005bb0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb4cce3640 1 -- 10.20.193.28:0/2788832155 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdb680cf550 con 0x55fa3a0cbb30 2026-02-25T11:54:51.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb8473ee00 1 -- 10.20.193.28:0/2788832155 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cbb30 msgr2=0x55fa3a0cbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:51.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/2788832155 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0cbf00 secure :-1 s=READY pgs=707 gs=1 cs=0 l=1 c_cookie=12354780327832602000 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdb680d1550 tx=0x7fdb6802fe20 comp rx=0 tx=0).stop 2026-02-25T11:54:51.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb8473ee00 1 -- 10.20.193.28:0/2788832155 shutdown_connections 2026-02-25T11:54:51.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/2788832155 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a1110e0 0x55fa3a1314c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/2788832155 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0cc440 0x55fa3a110ba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/2788832155 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0cbf00 unknown :-1 s=CLOSED pgs=707 gs=1 cs=0 l=1 c_cookie=12354780327832602000 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb8473ee00 1 -- 10.20.193.28:0/2788832155 >> 10.20.193.28:0/2788832155 conn(0x55fa39fea7a0 msgr2=0x55fa3a0ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:51.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb8473ee00 1 -- 10.20.193.28:0/2788832155 shutdown_connections 2026-02-25T11:54:51.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.049+0000 7fdb8473ee00 1 -- 10.20.193.28:0/2788832155 wait complete. 2026-02-25T11:54:51.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.050+0000 7fdb8473ee00 1 -- start start 2026-02-25T11:54:51.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.051+0000 7fdb8473ee00 1 Processor -- start 2026-02-25T11:54:51.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.051+0000 7fdb8473ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0eca50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.051+0000 7fdb8473ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cc440 0x55fa3a0ecf90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.051+0000 7fdb8473ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a1110e0 0x55fa3a0f1c70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.051+0000 7fdb8473ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55fa3a0f21b0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.051+0000 7fdb8473ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55fa3a0f2320 con 0x55fa3a1110e0 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.051+0000 7fdb8473ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55fa3a0f24e0 con 0x55fa3a0cc440 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.051+0000 7fdb80cb9640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a1110e0 0x55fa3a0f1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.051+0000 7fdb80cb9640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a1110e0 0x55fa3a0f1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:35180/0 (socket says 10.20.193.28:35180) 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.051+0000 7fdb80cb9640 1 -- 10.20.193.28:0/928506787 learned_addr learned my addr 10.20.193.28:0/928506787 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.051+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0eca50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.051+0000 7fdb7b7fe640 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cc440 0x55fa3a0ecf90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb7bfff640 1 -- 10.20.193.28:0/928506787 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cc440 msgr2=0x55fa3a0ecf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cc440 0x55fa3a0ecf90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb7bfff640 1 -- 10.20.193.28:0/928506787 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a1110e0 msgr2=0x55fa3a0f1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a1110e0 0x55fa3a0f1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb7bfff640 1 -- 10.20.193.28:0/928506787 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa39f5b3c0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb80cb9640 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a1110e0 0x55fa3a0f1c70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0eca50 secure :-1 s=READY pgs=708 gs=2 cs=0 l=1 c_cookie=16089129077400732875 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdb68108c90 tx=0x7fdb6802f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4811a6ebe104cb server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb3dffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb3dffb640 1 -- 10.20.193.28:0/928506787 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdb680cf770 con 0x55fa3a0cbb30 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb8473ee00 1 -- 10.20.193.28:0/928506787 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb7c0fbc10 con 0x55fa3a0cbb30 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb8473ee00 10 client.16836 _ll_register_callbacks cb 0x55fa39f58a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb3dffb640 1 -- 10.20.193.28:0/928506787 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdb680d71e0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb8473ee00 1 -- 10.20.193.28:0/928506787 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fa39c6d6f0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb3dffb640 1 -- 10.20.193.28:0/928506787 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdb680cd140 con 0x55fa3a0cbb30 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.052+0000 7fdb3dffb640 1 -- 10.20.193.28:0/928506787 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fdb6807de40 con 0x55fa3a0cbb30 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.053+0000 7fdb3dffb640 1 -- 10.20.193.28:0/928506787 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdb680d02d0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.053+0000 7fdb8473ee00 10 client.16836 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.053+0000 7fdb8473ee00 1 -- 10.20.193.28:0/928506787 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55fa39f6d0e0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.053+0000 7fdb3dffb640 1 -- 10.20.193.28:0/928506787 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fdb680d0430 con 0x55fa3a0cbb30 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.053+0000 7fdb8473ee00 10 client.16836 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.053+0000 7fdb8473ee00 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:51.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.053+0000 7fdb8473ee00 20 client.16836 populate_metadata read hostname 'trial028' 2026-02-25T11:54:51.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.053+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55fa3a0f6990 0x55fa3a0f6e00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.053+0000 7fdb8473ee00 4 client.16836 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:54:51.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.053+0000 7fdb8473ee00 1 -- 10.20.193.28:0/928506787 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa39fa9fc0 con 0x55fa3a0f6990 2026-02-25T11:54:51.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.053+0000 7fdb7b7fe640 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55fa3a0f6990 0x55fa3a0f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.054+0000 7fdb3dffb640 10 client.16836 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:51.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.054+0000 7fdb7b7fe640 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55fa3a0f6990 0x55fa3a0f6e00 crc :-1 s=READY pgs=124 gs=4 cs=0 l=0 c_cookie=165122344936505559 s_cookie=10791028553389471008 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24aa1e5da6b80d7 server_cookie=95c16f4bc5e07520 in_seq=0 out_seq=0 2026-02-25T11:54:51.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.054+0000 7fdb3dffb640 1 -- 10.20.193.28:0/928506787 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55fa39fa9fc0 con 0x55fa3a0f6990 2026-02-25T11:54:51.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.054+0000 7fdb3dffb640 10 client.16836 handle_command_reply: tid=0 2026-02-25T11:54:51.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.055+0000 7fdb8473ee00 1 client.16836 shutdown 2026-02-25T11:54:51.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.055+0000 7fdb8473ee00 1 -- 10.20.193.28:0/928506787 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cbb30 msgr2=0x55fa3a0eca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:51.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.055+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0eca50 secure :-1 s=READY pgs=708 gs=2 cs=0 l=1 c_cookie=16089129077400732875 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdb68108c90 tx=0x7fdb6802f190 comp rx=0 tx=0).stop 2026-02-25T11:54:51.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.055+0000 7fdb8473ee00 1 -- 10.20.193.28:0/928506787 shutdown_connections 2026-02-25T11:54:51.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.055+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55fa3a0f6990 0x55fa3a0f6e00 crc :-1 s=READY pgs=124 gs=4 cs=0 l=0 c_cookie=165122344936505559 s_cookie=10791028553389471008 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.055+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a1110e0 0x55fa3a0f1c70 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.055+0000 7fdb3dffb640 0 client.16836 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:51.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.055+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cc440 0x55fa3a0ecf90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.055+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/928506787 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0eca50 unknown :-1 s=CLOSED pgs=708 gs=2 cs=0 l=1 c_cookie=16089129077400732875 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.055+0000 7fdb8473ee00 1 -- 10.20.193.28:0/928506787 >> 10.20.193.28:0/928506787 conn(0x55fa39fea7a0 msgr2=0x55fa3a0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:51.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.055+0000 7fdb8473ee00 1 -- 10.20.193.28:0/928506787 shutdown_connections 2026-02-25T11:54:51.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.055+0000 7fdb8473ee00 1 -- 10.20.193.28:0/928506787 wait complete. 2026-02-25T11:54:51.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.055+0000 7fdb8473ee00 20 client.16836 trim_cache size 0 max 16384 2026-02-25T11:54:51.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.062+0000 7fdb8473ee00 1 Processor -- start 2026-02-25T11:54:51.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb8473ee00 1 -- start start 2026-02-25T11:54:51.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb8473ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0ed100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb8473ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0cc440 0x55fa3a0b6980 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb8473ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0f6990 0x55fa3a0aa320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb8473ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55fa39fd51a0 con 0x55fa3a0f6990 2026-02-25T11:54:51.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb8473ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55fa39c95700 con 0x55fa3a0cc440 2026-02-25T11:54:51.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb8473ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55fa3a0c4cc0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb80cb9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0f6990 0x55fa3a0aa320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb80cb9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0f6990 0x55fa3a0aa320 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57612/0 (socket says 10.20.193.28:57612) 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb80cb9640 1 -- 10.20.193.28:0/2471672482 learned_addr learned my addr 10.20.193.28:0/2471672482 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb7bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0ed100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb7b7fe640 1 --2- 10.20.193.28:0/2471672482 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0cc440 0x55fa3a0b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb7bfff640 1 -- 10.20.193.28:0/2471672482 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0cc440 msgr2=0x55fa3a0b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/2471672482 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0cc440 0x55fa3a0b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb7bfff640 1 -- 10.20.193.28:0/2471672482 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0f6990 msgr2=0x55fa3a0aa320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/2471672482 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0f6990 0x55fa3a0aa320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb7bfff640 1 -- 10.20.193.28:0/2471672482 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa39f67b60 con 0x55fa3a0cbb30 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.063+0000 7fdb80cb9640 1 --2- 10.20.193.28:0/2471672482 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0f6990 0x55fa3a0aa320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.064+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/2471672482 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0ed100 secure :-1 s=READY pgs=192 gs=1 cs=0 l=1 c_cookie=18034100910297893609 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdb680d1a90 tx=0x7fdb68095000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa45ff3aa9dacae9 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.064+0000 7fdb3ffff640 1 -- 10.20.193.28:0/2471672482 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdb680d97c0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.064+0000 7fdb3ffff640 1 -- 10.20.193.28:0/2471672482 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55fa39c6d6f0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.064+0000 7fdb3ffff640 1 -- 10.20.193.28:0/2471672482 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdb680d97c0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.064+0000 7fdb8473ee00 1 -- 10.20.193.28:0/2471672482 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cbb30 msgr2=0x55fa3a0ed100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:51.065 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.064+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/2471672482 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0ed100 secure :-1 s=READY pgs=192 gs=1 cs=0 l=1 c_cookie=18034100910297893609 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdb680d1a90 tx=0x7fdb68095000 comp rx=0 tx=0).stop 2026-02-25T11:54:51.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.064+0000 7fdb8473ee00 1 -- 10.20.193.28:0/2471672482 shutdown_connections 2026-02-25T11:54:51.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.064+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/2471672482 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0f6990 0x55fa3a0aa320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.064+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/2471672482 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0cc440 0x55fa3a0b6980 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.064+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/2471672482 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cbb30 0x55fa3a0ed100 unknown :-1 s=CLOSED pgs=192 gs=1 cs=0 l=1 c_cookie=18034100910297893609 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.064+0000 7fdb8473ee00 1 -- 10.20.193.28:0/2471672482 >> 10.20.193.28:0/2471672482 conn(0x55fa39fea7a0 msgr2=0x55fa3a0a9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:51.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.064+0000 7fdb8473ee00 1 -- 10.20.193.28:0/2471672482 shutdown_connections 2026-02-25T11:54:51.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.064+0000 7fdb8473ee00 1 -- 10.20.193.28:0/2471672482 wait complete. 2026-02-25T11:54:51.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.065+0000 7fdb8473ee00 1 -- start start 2026-02-25T11:54:51.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.065+0000 7fdb8473ee00 1 Processor -- start 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.065+0000 7fdb8473ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cbb30 0x55fa3a109300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.065+0000 7fdb8473ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cc440 0x55fa3a109880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.065+0000 7fdb8473ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0f6990 0x55fa3a0f0650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.065+0000 7fdb8473ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55fa3a0c4cc0 con 0x55fa3a0cc440 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.065+0000 7fdb8473ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55fa39fd51a0 con 0x55fa3a0f6990 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.065+0000 7fdb8473ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55fa3a0f0e00 con 0x55fa3a0cbb30 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.065+0000 7fdb7bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cbb30 0x55fa3a109300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb7bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cbb30 0x55fa3a109300 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44014/0 (socket says 10.20.193.28:44014) 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb80cb9640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0f6990 0x55fa3a0f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb7bfff640 1 -- 10.20.193.28:0/1312105681 learned_addr learned my addr 10.20.193.28:0/1312105681 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb7b7fe640 1 --2- 10.20.193.28:0/1312105681 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cc440 0x55fa3a109880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb7bfff640 1 -- 10.20.193.28:0/1312105681 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0f6990 msgr2=0x55fa3a0f0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/1312105681 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0f6990 0x55fa3a0f0650 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb7bfff640 1 -- 10.20.193.28:0/1312105681 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cc440 msgr2=0x55fa3a109880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:51.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/1312105681 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cc440 0x55fa3a109880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb7bfff640 1 -- 10.20.193.28:0/1312105681 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa39f6d0e0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb7bfff640 1 --2- 10.20.193.28:0/1312105681 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cbb30 0x55fa3a109300 secure :-1 s=READY pgs=193 gs=1 cs=0 l=1 c_cookie=11699241615241207230 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdb6802ffd0 tx=0x7fdb68038e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a25c0e4389eb5dbe server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb3dffb640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb3dffb640 1 -- 10.20.193.28:0/1312105681 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdb680d97c0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb3dffb640 1 -- 10.20.193.28:0/1312105681 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdb680d71e0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb8473ee00 1 -- 10.20.193.28:0/1312105681 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb70004890 con 0x55fa3a0cbb30 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb8473ee00 10 client.26551 _ll_register_callbacks cb 0x55fa39f58a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb8473ee00 1 -- 10.20.193.28:0/1312105681 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb68005bb0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.066+0000 7fdb3dffb640 1 -- 10.20.193.28:0/1312105681 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdb6807f0b0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.067+0000 7fdb3dffb640 1 -- 10.20.193.28:0/1312105681 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fdb6807de40 con 0x55fa3a0cbb30 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.067+0000 7fdb3dffb640 1 -- 10.20.193.28:0/1312105681 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdb6807f240 con 0x55fa3a0cbb30 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.067+0000 7fdb8473ee00 10 client.26551 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.067+0000 7fdb8473ee00 1 -- 10.20.193.28:0/1312105681 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55fa39f5b6f0 con 0x55fa3a0cbb30 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.067+0000 7fdb3dffb640 1 -- 10.20.193.28:0/1312105681 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fdb680d0860 con 0x55fa3a0cbb30 2026-02-25T11:54:51.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.067+0000 7fdb8473ee00 10 client.26551 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:51.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.067+0000 7fdb8473ee00 10 client.26551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:51.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.067+0000 7fdb8473ee00 20 client.26551 populate_metadata read hostname 'trial028' 2026-02-25T11:54:51.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.067+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/1312105681 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55fa3a1110e0 0x55fa3a1e56a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:51.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.067+0000 7fdb8473ee00 4 client.26551 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:54:51.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.067+0000 7fdb8473ee00 1 -- 10.20.193.28:0/1312105681 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa39c6af40 con 0x55fa3a1110e0 2026-02-25T11:54:51.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.067+0000 7fdb7b7fe640 1 --2- 10.20.193.28:0/1312105681 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55fa3a1110e0 0x55fa3a1e56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:51.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb7b7fe640 1 --2- 10.20.193.28:0/1312105681 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55fa3a1110e0 0x55fa3a1e56a0 crc :-1 s=READY pgs=125 gs=4 cs=0 l=0 c_cookie=10092789416385602615 s_cookie=18067663935032326128 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c10ca80147c6437 server_cookie=fabd3c9f50bcf7f0 in_seq=0 out_seq=0 2026-02-25T11:54:51.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb3dffb640 10 client.26551 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:51.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb3dffb640 1 -- 10.20.193.28:0/1312105681 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa39c6af40 con 0x55fa3a1110e0 2026-02-25T11:54:51.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb3dffb640 10 client.26551 handle_command_reply: tid=0 2026-02-25T11:54:51.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb8473ee00 1 client.26551 shutdown 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb8473ee00 1 -- 10.20.193.28:0/1312105681 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cbb30 msgr2=0x55fa3a109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/1312105681 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cbb30 0x55fa3a109300 secure :-1 s=READY pgs=193 gs=1 cs=0 l=1 c_cookie=11699241615241207230 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdb6802ffd0 tx=0x7fdb68038e10 comp rx=0 tx=0).stop 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb8473ee00 1 -- 10.20.193.28:0/1312105681 shutdown_connections 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/1312105681 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55fa3a1110e0 0x55fa3a1e56a0 crc :-1 s=READY pgs=125 gs=4 cs=0 l=0 c_cookie=10092789416385602615 s_cookie=18067663935032326128 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/1312105681 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55fa3a0f6990 0x55fa3a0f0650 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/1312105681 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55fa3a0cc440 0x55fa3a109880 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb8473ee00 1 --2- 10.20.193.28:0/1312105681 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55fa3a0cbb30 0x55fa3a109300 unknown :-1 s=CLOSED pgs=193 gs=1 cs=0 l=1 c_cookie=11699241615241207230 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb3dffb640 0 client.26551 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb8473ee00 1 -- 10.20.193.28:0/1312105681 >> 10.20.193.28:0/1312105681 conn(0x55fa39fea7a0 msgr2=0x55fa3a0aa400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb8473ee00 1 -- 10.20.193.28:0/1312105681 shutdown_connections 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb8473ee00 1 -- 10.20.193.28:0/1312105681 wait complete. 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.068+0000 7fdb8473ee00 20 client.26551 trim_cache size 0 max 16384 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.069+0000 7fdb814ba640 1 -- 10.20.193.28:0/573536673 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdb500b56d0 msgr2=0x7fdb500d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.069+0000 7fdb814ba640 1 --2- 10.20.193.28:0/573536673 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdb500b56d0 0x7fdb500d5b80 secure :-1 s=READY pgs=342 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdb7c0fc2d0 tx=0x7fdb6c005f90 comp rx=0 tx=0).stop 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.069+0000 7fdb814ba640 1 -- 10.20.193.28:0/573536673 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdb7c10d3d0 msgr2=0x7fdb7c0f5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:51.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.069+0000 7fdb814ba640 1 --2- 10.20.193.28:0/573536673 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdb7c10d3d0 0x7fdb7c0f5980 secure :-1 s=READY pgs=706 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdb680098a0 tx=0x7fdb680028d0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.069+0000 7fdb814ba640 1 -- 10.20.193.28:0/573536673 shutdown_connections 2026-02-25T11:54:51.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.069+0000 7fdb814ba640 1 --2- 10.20.193.28:0/573536673 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdb500b56d0 0x7fdb500d5b80 unknown :-1 s=CLOSED pgs=342 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.069+0000 7fdb814ba640 1 --2- 10.20.193.28:0/573536673 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdb7c1135c0 0x7fdb7c0f6080 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.069+0000 7fdb814ba640 1 --2- 10.20.193.28:0/573536673 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdb7c10ddc0 0x7fdb7c173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.069+0000 7fdb814ba640 1 --2- 10.20.193.28:0/573536673 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdb7c10d3d0 0x7fdb7c0f5980 unknown :-1 s=CLOSED pgs=706 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:51.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.069+0000 7fdb814ba640 1 -- 10.20.193.28:0/573536673 >> 10.20.193.28:0/573536673 conn(0x7fdb7c0eaca0 msgr2=0x7fdb7c110520 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:51.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.069+0000 7fdb814ba640 1 -- 10.20.193.28:0/573536673 shutdown_connections 2026-02-25T11:54:51.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:51.069+0000 7fdb814ba640 1 -- 10.20.193.28:0/573536673 wait complete. 2026-02-25T11:54:52.168 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:54:52.169 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:54:52.353 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:52.484 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:52 trial028 ceph-mon[39224]: pgmap v425: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:54:52.529 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.526+0000 7f3529b75640 1 -- 10.20.193.28:0/1608018593 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3524114760 msgr2=0x7f3524114b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:52.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.526+0000 7f3529b75640 1 --2- 10.20.193.28:0/1608018593 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3524114760 0x7f3524114b30 secure :-1 s=READY pgs=158 gs=1 cs=0 l=1 c_cookie=13006470393191916630 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3510009950 tx=0x7f351002f190 comp rx=0 tx=0).stop 2026-02-25T11:54:52.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.526+0000 7f3529b75640 1 -- 10.20.193.28:0/1608018593 shutdown_connections 2026-02-25T11:54:52.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.526+0000 7f3529b75640 1 --2- 10.20.193.28:0/1608018593 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f352415c0e0 0x7f352417c4e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.526+0000 7f3529b75640 1 --2- 10.20.193.28:0/1608018593 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3524115100 0x7f352415bb10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.526+0000 7f3529b75640 1 --2- 10.20.193.28:0/1608018593 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3524114760 0x7f3524114b30 unknown :-1 s=CLOSED pgs=158 gs=1 cs=0 l=1 c_cookie=13006470393191916630 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.526+0000 7f3529b75640 1 -- 10.20.193.28:0/1608018593 >> 10.20.193.28:0/1608018593 conn(0x7f352405c270 msgr2=0x7f352405c670 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:52.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.526+0000 7f3529b75640 1 -- 10.20.193.28:0/1608018593 shutdown_connections 2026-02-25T11:54:52.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.526+0000 7f3529b75640 1 -- 10.20.193.28:0/1608018593 wait complete. 2026-02-25T11:54:52.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.527+0000 7f3529b75640 1 Processor -- start 2026-02-25T11:54:52.530 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.527+0000 7f3529b75640 1 -- start start 2026-02-25T11:54:52.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.527+0000 7f3529b75640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3524114760 0x7f3524104ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.527+0000 7f3529b75640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3524115100 0x7f3524104ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.527+0000 7f3529b75640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f352415c0e0 0x7f352410a3c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.527+0000 7f3529b75640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f352410a9e0 con 0x7f3524115100 2026-02-25T11:54:52.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.527+0000 7f3529b75640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f352410ab50 con 0x7f3524114760 2026-02-25T11:54:52.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.527+0000 7f3529b75640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f352410ad10 con 0x7f352415c0e0 2026-02-25T11:54:52.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.527+0000 7f3522ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3524115100 0x7f3524104ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.527+0000 7f3522ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3524115100 0x7f3524104ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57622/0 (socket says 10.20.193.28:57622) 2026-02-25T11:54:52.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.527+0000 7f3522ffd640 1 -- 10.20.193.28:0/1605695983 learned_addr learned my addr 10.20.193.28:0/1605695983 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:52.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.527+0000 7f3523fff640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f352415c0e0 0x7f352410a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.528+0000 7f35237fe640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3524114760 0x7f3524104ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.532 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.528+0000 7f3522ffd640 1 -- 10.20.193.28:0/1605695983 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f352415c0e0 msgr2=0x7f352410a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:52.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.528+0000 7f3522ffd640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f352415c0e0 0x7f352410a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.528+0000 7f3522ffd640 1 -- 10.20.193.28:0/1605695983 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3524114760 msgr2=0x7f3524104ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:52.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.528+0000 7f3522ffd640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3524114760 0x7f3524104ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.528+0000 7f3522ffd640 1 -- 10.20.193.28:0/1605695983 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3510009570 con 0x7f3524115100 2026-02-25T11:54:52.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.528+0000 7f3522ffd640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3524115100 0x7f3524104ff0 secure :-1 s=READY pgs=709 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3514009480 tx=0x7f351400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:52.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.528+0000 7f3520ff9640 1 -- 10.20.193.28:0/1605695983 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f351400cd40 con 0x7f3524115100 2026-02-25T11:54:52.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.528+0000 7f3520ff9640 1 -- 10.20.193.28:0/1605695983 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3514004d10 con 0x7f3524115100 2026-02-25T11:54:52.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.528+0000 7f3529b75640 1 -- 10.20.193.28:0/1605695983 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352410aff0 con 0x7f3524115100 2026-02-25T11:54:52.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.528+0000 7f3529b75640 1 -- 10.20.193.28:0/1605695983 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35241559e0 con 0x7f3524115100 2026-02-25T11:54:52.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.529+0000 7f3520ff9640 1 -- 10.20.193.28:0/1605695983 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f35140106b0 con 0x7f3524115100 2026-02-25T11:54:52.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.531+0000 7f3520ff9640 1 -- 10.20.193.28:0/1605695983 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f351401f060 con 0x7f3524115100 2026-02-25T11:54:52.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.533+0000 7f3520ff9640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f34f80b56e0 0x7f34f80d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.533+0000 7f3520ff9640 1 -- 10.20.193.28:0/1605695983 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f3514014070 con 0x7f3524115100 2026-02-25T11:54:52.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.533+0000 7f35237fe640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f34f80b56e0 0x7f34f80d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.534+0000 7f35237fe640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f34f80b56e0 0x7f34f80d5b90 secure :-1 s=READY pgs=343 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f35100098a0 tx=0x7f351003a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:52.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.534+0000 7f352bb70e00 1 Processor -- start 2026-02-25T11:54:52.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.534+0000 7f352bb70e00 1 -- start start 2026-02-25T11:54:52.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f352bb70e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f41ff750 0x5633f41ffb20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f352bb70e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4200060 0x5633f4244610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f352bb70e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4244b50 0x5633f4264f30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f352bb70e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5633f4115d80 con 0x5633f4200060 2026-02-25T11:54:52.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f352bb70e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5633f4111ab0 con 0x5633f41ff750 2026-02-25T11:54:52.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f352bb70e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5633f410ba50 con 0x5633f4244b50 2026-02-25T11:54:52.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f3523fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4244b50 0x5633f4264f30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f35237fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f41ff750 0x5633f41ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f3523fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4244b50 0x5633f4264f30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44042/0 (socket says 10.20.193.28:44042) 2026-02-25T11:54:52.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f3523fff640 1 -- 10.20.193.28:0/3783637688 learned_addr learned my addr 10.20.193.28:0/3783637688 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:52.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f3522ffd640 1 --2- 10.20.193.28:0/3783637688 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4200060 0x5633f4244610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f3522ffd640 1 -- 10.20.193.28:0/3783637688 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4244b50 msgr2=0x5633f4264f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:52.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f3522ffd640 1 --2- 10.20.193.28:0/3783637688 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4244b50 0x5633f4264f30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f3522ffd640 1 -- 10.20.193.28:0/3783637688 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f41ff750 msgr2=0x5633f41ffb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:52.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f3522ffd640 1 --2- 10.20.193.28:0/3783637688 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f41ff750 0x5633f41ffb20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f3522ffd640 1 -- 10.20.193.28:0/3783637688 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f40b7120 con 0x5633f4200060 2026-02-25T11:54:52.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.535+0000 7f35237fe640 1 --2- 10.20.193.28:0/3783637688 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f41ff750 0x5633f41ffb20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:52.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.536+0000 7f3522ffd640 1 --2- 10.20.193.28:0/3783637688 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4200060 0x5633f4244610 secure :-1 s=READY pgs=710 gs=3 cs=0 l=1 c_cookie=9491488666733649299 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f35140e4790 tx=0x7f35140a8f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b88aa7824e1d93 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:52.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.536+0000 7f34e7fff640 1 -- 10.20.193.28:0/3783637688 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3514005140 con 0x5633f4200060 2026-02-25T11:54:52.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.536+0000 7f34e7fff640 1 -- 10.20.193.28:0/3783637688 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3514016ea0 con 0x5633f4200060 2026-02-25T11:54:52.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.536+0000 7f34e7fff640 1 -- 10.20.193.28:0/3783637688 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f35140acbc0 con 0x5633f4200060 2026-02-25T11:54:52.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.536+0000 7f352bb70e00 1 -- 10.20.193.28:0/3783637688 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4200060 msgr2=0x5633f4244610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:52.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.536+0000 7f352bb70e00 1 --2- 10.20.193.28:0/3783637688 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4200060 0x5633f4244610 secure :-1 s=READY pgs=710 gs=3 cs=0 l=1 c_cookie=9491488666733649299 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f35140e4790 tx=0x7f35140a8f40 comp rx=0 tx=0).stop 2026-02-25T11:54:52.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.536+0000 7f352bb70e00 1 -- 10.20.193.28:0/3783637688 shutdown_connections 2026-02-25T11:54:52.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.536+0000 7f352bb70e00 1 --2- 10.20.193.28:0/3783637688 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4244b50 0x5633f4264f30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.536+0000 7f352bb70e00 1 --2- 10.20.193.28:0/3783637688 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4200060 0x5633f4244610 unknown :-1 s=CLOSED pgs=710 gs=3 cs=0 l=1 c_cookie=9491488666733649299 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.536+0000 7f352bb70e00 1 --2- 10.20.193.28:0/3783637688 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f41ff750 0x5633f41ffb20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.536+0000 7f352bb70e00 1 -- 10.20.193.28:0/3783637688 >> 10.20.193.28:0/3783637688 conn(0x5633f41dd8f0 msgr2=0x5633f41fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:52.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.536+0000 7f352bb70e00 1 -- 10.20.193.28:0/3783637688 shutdown_connections 2026-02-25T11:54:52.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.536+0000 7f352bb70e00 1 -- 10.20.193.28:0/3783637688 wait complete. 2026-02-25T11:54:52.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.538+0000 7f352bb70e00 1 -- start start 2026-02-25T11:54:52.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.538+0000 7f352bb70e00 1 Processor -- start 2026-02-25T11:54:52.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.538+0000 7f352bb70e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f41ff750 0x5633f425c520 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.538+0000 7f352bb70e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4200060 0x5633f425ca60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.538+0000 7f352bb70e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4244b50 0x5633f4261a50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.538+0000 7f352bb70e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5633f4261f90 con 0x5633f4244b50 2026-02-25T11:54:52.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.538+0000 7f352bb70e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5633f4262100 con 0x5633f41ff750 2026-02-25T11:54:52.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.538+0000 7f352bb70e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5633f42622c0 con 0x5633f4200060 2026-02-25T11:54:52.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.538+0000 7f3523fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4244b50 0x5633f4261a50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.539+0000 7f3523fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4244b50 0x5633f4261a50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57648/0 (socket says 10.20.193.28:57648) 2026-02-25T11:54:52.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.539+0000 7f35237fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f41ff750 0x5633f425c520 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.539+0000 7f3523fff640 1 -- 10.20.193.28:0/1449457818 learned_addr learned my addr 10.20.193.28:0/1449457818 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:52.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.539+0000 7f3522ffd640 1 --2- 10.20.193.28:0/1449457818 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4200060 0x5633f425ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.539+0000 7f35237fe640 1 -- 10.20.193.28:0/1449457818 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4200060 msgr2=0x5633f425ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:52.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.539+0000 7f35237fe640 1 --2- 10.20.193.28:0/1449457818 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4200060 0x5633f425ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.539+0000 7f35237fe640 1 -- 10.20.193.28:0/1449457818 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4244b50 msgr2=0x5633f4261a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:52.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.539+0000 7f35237fe640 1 --2- 10.20.193.28:0/1449457818 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4244b50 0x5633f4261a50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.539+0000 7f35237fe640 1 -- 10.20.193.28:0/1449457818 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35241559e0 con 0x5633f41ff750 2026-02-25T11:54:52.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.539+0000 7f3523fff640 1 --2- 10.20.193.28:0/1449457818 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4244b50 0x5633f4261a50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:52.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.539+0000 7f35237fe640 1 --2- 10.20.193.28:0/1449457818 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f41ff750 0x5633f425c520 secure :-1 s=READY pgs=159 gs=1 cs=0 l=1 c_cookie=8631052557155529973 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3510005b30 tx=0x7f3510009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77c7a8924afb74f5 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:52.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.539+0000 7f34e57fa640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:54:52.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.539+0000 7f34e57fa640 1 -- 10.20.193.28:0/1449457818 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3510038ad0 con 0x5633f41ff750 2026-02-25T11:54:52.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f352bb70e00 1 -- 10.20.193.28:0/1449457818 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352417d160 con 0x5633f41ff750 2026-02-25T11:54:52.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f352bb70e00 10 client.26591 _ll_register_callbacks cb 0x5633f408ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:52.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f34e57fa640 1 -- 10.20.193.28:0/1449457818 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f351003e040 con 0x5633f41ff750 2026-02-25T11:54:52.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f352bb70e00 1 -- 10.20.193.28:0/1449457818 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5633f3e63530 con 0x5633f41ff750 2026-02-25T11:54:52.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f34e57fa640 1 -- 10.20.193.28:0/1449457818 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f35100503f0 con 0x5633f41ff750 2026-02-25T11:54:52.545 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f34e57fa640 1 -- 10.20.193.28:0/1449457818 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f3510050580 con 0x5633f41ff750 2026-02-25T11:54:52.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f34e57fa640 1 -- 10.20.193.28:0/1449457818 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3510044410 con 0x5633f41ff750 2026-02-25T11:54:52.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f352bb70e00 10 client.26591 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:52.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f352bb70e00 1 -- 10.20.193.28:0/1449457818 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5633f408f810 con 0x5633f41ff750 2026-02-25T11:54:52.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f34e57fa640 1 -- 10.20.193.28:0/1449457818 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f351004f070 con 0x5633f41ff750 2026-02-25T11:54:52.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f352bb70e00 10 client.26591 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:52.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f352bb70e00 10 client.26591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:52.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f352bb70e00 20 client.26591 populate_metadata read hostname 'trial028' 2026-02-25T11:54:52.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f352bb70e00 1 --2- 10.20.193.28:0/1449457818 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5633f4241e40 0x5633f42422e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f352bb70e00 4 client.26591 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:54:52.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.540+0000 7f352bb70e00 1 -- 10.20.193.28:0/1449457818 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633f40b8b50 con 0x5633f4241e40 2026-02-25T11:54:52.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.541+0000 7f3522ffd640 1 --2- 10.20.193.28:0/1449457818 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5633f4241e40 0x5633f42422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.541+0000 7f3522ffd640 1 --2- 10.20.193.28:0/1449457818 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5633f4241e40 0x5633f42422e0 crc :-1 s=READY pgs=126 gs=4 cs=0 l=0 c_cookie=1033219018131020246 s_cookie=7390405319196291584 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e56bb3906e099d6 server_cookie=668ffe7b54da3200 in_seq=0 out_seq=0 2026-02-25T11:54:52.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.541+0000 7f34e57fa640 10 client.26591 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:52.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.542+0000 7f34e57fa640 1 -- 10.20.193.28:0/1449457818 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5633f40b8b50 con 0x5633f4241e40 2026-02-25T11:54:52.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.542+0000 7f34e57fa640 10 client.26591 handle_command_reply: tid=0 2026-02-25T11:54:52.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.542+0000 7f352bb70e00 1 client.26591 shutdown 2026-02-25T11:54:52.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.542+0000 7f352bb70e00 1 -- 10.20.193.28:0/1449457818 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f41ff750 msgr2=0x5633f425c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:52.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.542+0000 7f352bb70e00 1 --2- 10.20.193.28:0/1449457818 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f41ff750 0x5633f425c520 secure :-1 s=READY pgs=159 gs=1 cs=0 l=1 c_cookie=8631052557155529973 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3510005b30 tx=0x7f3510009310 comp rx=0 tx=0).stop 2026-02-25T11:54:52.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.542+0000 7f352bb70e00 1 -- 10.20.193.28:0/1449457818 shutdown_connections 2026-02-25T11:54:52.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.542+0000 7f352bb70e00 1 --2- 10.20.193.28:0/1449457818 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5633f4241e40 0x5633f42422e0 crc :-1 s=READY pgs=126 gs=4 cs=0 l=0 c_cookie=1033219018131020246 s_cookie=7390405319196291584 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.542+0000 7f352bb70e00 1 --2- 10.20.193.28:0/1449457818 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4244b50 0x5633f4261a50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.542+0000 7f352bb70e00 1 --2- 10.20.193.28:0/1449457818 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4200060 0x5633f425ca60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.542+0000 7f352bb70e00 1 --2- 10.20.193.28:0/1449457818 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f41ff750 0x5633f425c520 unknown :-1 s=CLOSED pgs=159 gs=1 cs=0 l=1 c_cookie=8631052557155529973 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.542+0000 7f352bb70e00 1 -- 10.20.193.28:0/1449457818 >> 10.20.193.28:0/1449457818 conn(0x5633f41dd8f0 msgr2=0x5633f41fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:52.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.543+0000 7f34e57fa640 0 client.26591 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:52.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.543+0000 7f352bb70e00 1 -- 10.20.193.28:0/1449457818 shutdown_connections 2026-02-25T11:54:52.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.543+0000 7f352bb70e00 1 -- 10.20.193.28:0/1449457818 wait complete. 2026-02-25T11:54:52.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.543+0000 7f352bb70e00 20 client.26591 trim_cache size 0 max 16384 2026-02-25T11:54:52.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f352bb70e00 1 Processor -- start 2026-02-25T11:54:52.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f352bb70e00 1 -- start start 2026-02-25T11:54:52.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f352bb70e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f41ff750 0x5633f4260fc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f352bb70e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f4200060 0x5633f42587c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f352bb70e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4244b50 0x5633f4258d00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f352bb70e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5633f4109680 con 0x5633f41ff750 2026-02-25T11:54:52.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f352bb70e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5633f3dc9790 con 0x5633f4200060 2026-02-25T11:54:52.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f352bb70e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5633f41dd330 con 0x5633f4244b50 2026-02-25T11:54:52.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f35237fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f41ff750 0x5633f4260fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f35237fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f41ff750 0x5633f4260fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57664/0 (socket says 10.20.193.28:57664) 2026-02-25T11:54:52.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f35237fe640 1 -- 10.20.193.28:0/2242762999 learned_addr learned my addr 10.20.193.28:0/2242762999 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:52.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f3522ffd640 1 --2- 10.20.193.28:0/2242762999 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f4200060 0x5633f42587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f3523fff640 1 --2- 10.20.193.28:0/2242762999 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4244b50 0x5633f4258d00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f35237fe640 1 -- 10.20.193.28:0/2242762999 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4244b50 msgr2=0x5633f4258d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:52.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f35237fe640 1 --2- 10.20.193.28:0/2242762999 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4244b50 0x5633f4258d00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f35237fe640 1 -- 10.20.193.28:0/2242762999 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f4200060 msgr2=0x5633f42587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:52.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f35237fe640 1 --2- 10.20.193.28:0/2242762999 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f4200060 0x5633f42587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.552+0000 7f35237fe640 1 -- 10.20.193.28:0/2242762999 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f424eab0 con 0x5633f41ff750 2026-02-25T11:54:52.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.553+0000 7f35237fe640 1 --2- 10.20.193.28:0/2242762999 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f41ff750 0x5633f4260fc0 secure :-1 s=READY pgs=711 gs=1 cs=0 l=1 c_cookie=12917077285189595322 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3510009950 tx=0x7f351005a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b342ad5a6ffe30ba server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:52.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.553+0000 7f34e77fe640 1 -- 10.20.193.28:0/2242762999 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3510004250 con 0x5633f41ff750 2026-02-25T11:54:52.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.553+0000 7f34e77fe640 1 -- 10.20.193.28:0/2242762999 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3510048030 con 0x5633f41ff750 2026-02-25T11:54:52.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.553+0000 7f34e77fe640 1 -- 10.20.193.28:0/2242762999 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3510063880 con 0x5633f41ff750 2026-02-25T11:54:52.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.553+0000 7f352bb70e00 1 -- 10.20.193.28:0/2242762999 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f41ff750 msgr2=0x5633f4260fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:52.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.553+0000 7f352bb70e00 1 --2- 10.20.193.28:0/2242762999 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f41ff750 0x5633f4260fc0 secure :-1 s=READY pgs=711 gs=1 cs=0 l=1 c_cookie=12917077285189595322 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3510009950 tx=0x7f351005a000 comp rx=0 tx=0).stop 2026-02-25T11:54:52.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.553+0000 7f352bb70e00 1 -- 10.20.193.28:0/2242762999 shutdown_connections 2026-02-25T11:54:52.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.553+0000 7f352bb70e00 1 --2- 10.20.193.28:0/2242762999 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f4244b50 0x5633f4258d00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.553+0000 7f352bb70e00 1 --2- 10.20.193.28:0/2242762999 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f4200060 0x5633f42587c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.553+0000 7f352bb70e00 1 --2- 10.20.193.28:0/2242762999 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f41ff750 0x5633f4260fc0 unknown :-1 s=CLOSED pgs=711 gs=1 cs=0 l=1 c_cookie=12917077285189595322 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.553+0000 7f352bb70e00 1 -- 10.20.193.28:0/2242762999 >> 10.20.193.28:0/2242762999 conn(0x5633f41dd8f0 msgr2=0x5633f42608b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:52.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.553+0000 7f352bb70e00 1 -- 10.20.193.28:0/2242762999 shutdown_connections 2026-02-25T11:54:52.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.553+0000 7f352bb70e00 1 -- 10.20.193.28:0/2242762999 wait complete. 2026-02-25T11:54:52.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.554+0000 7f352bb70e00 1 -- start start 2026-02-25T11:54:52.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.554+0000 7f352bb70e00 1 Processor -- start 2026-02-25T11:54:52.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.554+0000 7f352bb70e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f41ff750 0x5633f41e3760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.554+0000 7f352bb70e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4200060 0x5633f41deff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.554+0000 7f352bb70e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f4244b50 0x5633f41df530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.554+0000 7f352bb70e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5633f4262400 con 0x5633f4200060 2026-02-25T11:54:52.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.554+0000 7f352bb70e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5633f4109680 con 0x5633f4244b50 2026-02-25T11:54:52.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.554+0000 7f352bb70e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5633f41dfbf0 con 0x5633f41ff750 2026-02-25T11:54:52.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.554+0000 7f3523fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f4244b50 0x5633f41df530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.554+0000 7f3522ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4200060 0x5633f41deff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.554+0000 7f3523fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f4244b50 0x5633f41df530 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:35236/0 (socket says 10.20.193.28:35236) 2026-02-25T11:54:52.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.554+0000 7f3523fff640 1 -- 10.20.193.28:0/284357229 learned_addr learned my addr 10.20.193.28:0/284357229 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:52.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.554+0000 7f35237fe640 1 --2- 10.20.193.28:0/284357229 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f41ff750 0x5633f41e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f3523fff640 1 -- 10.20.193.28:0/284357229 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f41ff750 msgr2=0x5633f41e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:52.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f3523fff640 1 --2- 10.20.193.28:0/284357229 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f41ff750 0x5633f41e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f3523fff640 1 -- 10.20.193.28:0/284357229 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4200060 msgr2=0x5633f41deff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:52.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f3523fff640 1 --2- 10.20.193.28:0/284357229 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4200060 0x5633f41deff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f3523fff640 1 -- 10.20.193.28:0/284357229 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f40a14f0 con 0x5633f4244b50 2026-02-25T11:54:52.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f3522ffd640 1 --2- 10.20.193.28:0/284357229 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4200060 0x5633f41deff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:54:52.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f3523fff640 1 --2- 10.20.193.28:0/284357229 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f4244b50 0x5633f41df530 secure :-1 s=READY pgs=160 gs=2 cs=0 l=1 c_cookie=12572740858405675075 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3518013920 tx=0x7f3518013df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae7b59328e4f6043 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:52.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f34e57fa640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:54:52.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f34e57fa640 1 -- 10.20.193.28:0/284357229 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3518021070 con 0x5633f4244b50 2026-02-25T11:54:52.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f34e57fa640 1 -- 10.20.193.28:0/284357229 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3518005df0 con 0x5633f4244b50 2026-02-25T11:54:52.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f34e57fa640 1 -- 10.20.193.28:0/284357229 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3518017040 con 0x5633f4244b50 2026-02-25T11:54:52.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f352bb70e00 1 -- 10.20.193.28:0/284357229 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633f408f810 con 0x5633f4244b50 2026-02-25T11:54:52.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f352bb70e00 10 client.26597 _ll_register_callbacks cb 0x5633f408ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:52.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.555+0000 7f352bb70e00 1 -- 10.20.193.28:0/284357229 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5633f3d9f220 con 0x5633f4244b50 2026-02-25T11:54:52.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.556+0000 7f34e57fa640 1 -- 10.20.193.28:0/284357229 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f3518003b30 con 0x5633f4244b50 2026-02-25T11:54:52.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.556+0000 7f34e57fa640 1 -- 10.20.193.28:0/284357229 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3518003d10 con 0x5633f4244b50 2026-02-25T11:54:52.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.556+0000 7f352bb70e00 10 client.26597 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:52.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.556+0000 7f352bb70e00 1 -- 10.20.193.28:0/284357229 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5633f409bf70 con 0x5633f4244b50 2026-02-25T11:54:52.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.556+0000 7f34e57fa640 1 -- 10.20.193.28:0/284357229 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f3518005850 con 0x5633f4244b50 2026-02-25T11:54:52.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.556+0000 7f352bb70e00 10 client.26597 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:52.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.556+0000 7f352bb70e00 10 client.26597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:52.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.556+0000 7f352bb70e00 20 client.26597 populate_metadata read hostname 'trial028' 2026-02-25T11:54:52.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.556+0000 7f352bb70e00 1 --2- 10.20.193.28:0/284357229 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5633f42f26e0 0x5633f4312b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:52.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.556+0000 7f352bb70e00 4 client.26597 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:54:52.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.556+0000 7f352bb70e00 1 -- 10.20.193.28:0/284357229 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f3514016ea0 con 0x5633f42f26e0 2026-02-25T11:54:52.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.556+0000 7f35237fe640 1 --2- 10.20.193.28:0/284357229 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5633f42f26e0 0x5633f4312b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:52.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.557+0000 7f35237fe640 1 --2- 10.20.193.28:0/284357229 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5633f42f26e0 0x5633f4312b90 crc :-1 s=READY pgs=127 gs=4 cs=0 l=0 c_cookie=2738523403054843049 s_cookie=8665125355547529244 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26013086982d84a9 server_cookie=7840b599dfa4101c in_seq=0 out_seq=0 2026-02-25T11:54:52.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.557+0000 7f34e57fa640 10 client.26597 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:52.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.557+0000 7f34e57fa640 1 -- 10.20.193.28:0/284357229 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f3514016ea0 con 0x5633f42f26e0 2026-02-25T11:54:52.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.557+0000 7f34e57fa640 10 client.26597 handle_command_reply: tid=0 2026-02-25T11:54:52.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.557+0000 7f352bb70e00 1 client.26597 shutdown 2026-02-25T11:54:52.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f352bb70e00 1 -- 10.20.193.28:0/284357229 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f4244b50 msgr2=0x5633f41df530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:52.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f352bb70e00 1 --2- 10.20.193.28:0/284357229 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f4244b50 0x5633f41df530 secure :-1 s=READY pgs=160 gs=2 cs=0 l=1 c_cookie=12572740858405675075 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3518013920 tx=0x7f3518013df0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.564 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:54:52.564 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:54:52.564 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "8c0edddc-5eeb-4011-a44d-a25962ab3520", 2026-02-25T11:54:52.564 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:54:52.564 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:54:52.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f352bb70e00 1 -- 10.20.193.28:0/284357229 shutdown_connections 2026-02-25T11:54:52.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f352bb70e00 1 --2- 10.20.193.28:0/284357229 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5633f42f26e0 0x5633f4312b90 crc :-1 s=READY pgs=127 gs=4 cs=0 l=0 c_cookie=2738523403054843049 s_cookie=8665125355547529244 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f352bb70e00 1 --2- 10.20.193.28:0/284357229 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5633f4244b50 0x5633f41df530 unknown :-1 s=CLOSED pgs=160 gs=2 cs=0 l=1 c_cookie=12572740858405675075 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f352bb70e00 1 --2- 10.20.193.28:0/284357229 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5633f4200060 0x5633f41deff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f352bb70e00 1 --2- 10.20.193.28:0/284357229 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5633f41ff750 0x5633f41e3760 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f352bb70e00 1 -- 10.20.193.28:0/284357229 >> 10.20.193.28:0/284357229 conn(0x5633f41dd8f0 msgr2=0x5633f4258fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:52.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f34e57fa640 0 client.26597 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:52.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f352bb70e00 1 -- 10.20.193.28:0/284357229 shutdown_connections 2026-02-25T11:54:52.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f352bb70e00 1 -- 10.20.193.28:0/284357229 wait complete. 2026-02-25T11:54:52.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f352bb70e00 20 client.26597 trim_cache size 0 max 16384 2026-02-25T11:54:52.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f3529b75640 1 -- 10.20.193.28:0/1605695983 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f34f80b56e0 msgr2=0x7f34f80d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:52.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f3529b75640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f34f80b56e0 0x7f34f80d5b90 secure :-1 s=READY pgs=343 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f35100098a0 tx=0x7f351003a040 comp rx=0 tx=0).stop 2026-02-25T11:54:52.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f3529b75640 1 -- 10.20.193.28:0/1605695983 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3524115100 msgr2=0x7f3524104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:52.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.558+0000 7f3529b75640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3524115100 0x7f3524104ff0 secure :-1 s=READY pgs=709 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3514009480 tx=0x7f351400ee60 comp rx=0 tx=0).stop 2026-02-25T11:54:52.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.559+0000 7f3529b75640 1 -- 10.20.193.28:0/1605695983 shutdown_connections 2026-02-25T11:54:52.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.559+0000 7f3529b75640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f34f80b56e0 0x7f34f80d5b90 unknown :-1 s=CLOSED pgs=343 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.559+0000 7f3529b75640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f352415c0e0 0x7f352410a3c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.559+0000 7f3529b75640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3524115100 0x7f3524104ff0 unknown :-1 s=CLOSED pgs=709 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.559+0000 7f3529b75640 1 --2- 10.20.193.28:0/1605695983 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3524114760 0x7f3524104ab0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:52.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.559+0000 7f3529b75640 1 -- 10.20.193.28:0/1605695983 >> 10.20.193.28:0/1605695983 conn(0x7f352405c270 msgr2=0x7f35240fb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:52.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.559+0000 7f3529b75640 1 -- 10.20.193.28:0/1605695983 shutdown_connections 2026-02-25T11:54:52.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:52.559+0000 7f3529b75640 1 -- 10.20.193.28:0/1605695983 wait complete. 2026-02-25T11:54:52.660 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:54:52.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:52 trial167 ceph-mon[43392]: pgmap v425: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:54:52.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:52 trial119 ceph-mon[42710]: pgmap v425: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:54:52.846 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:54:53.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.025+0000 7fcd2a78e640 1 -- 10.20.193.28:0/369738158 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd2410be70 msgr2=0x7fcd24112f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:53.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.025+0000 7fcd2a78e640 1 --2- 10.20.193.28:0/369738158 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd2410be70 0x7fcd24112f20 secure :-1 s=READY pgs=712 gs=1 cs=0 l=1 c_cookie=16171815526455428668 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd18009950 tx=0x7fcd1802f190 comp rx=0 tx=0).stop 2026-02-25T11:54:53.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.025+0000 7fcd2a78e640 1 -- 10.20.193.28:0/369738158 shutdown_connections 2026-02-25T11:54:53.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.025+0000 7fcd2a78e640 1 --2- 10.20.193.28:0/369738158 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcd241136a0 0x7fcd24174010 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.025+0000 7fcd2a78e640 1 --2- 10.20.193.28:0/369738158 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd2410be70 0x7fcd24112f20 unknown :-1 s=CLOSED pgs=712 gs=1 cs=0 l=1 c_cookie=16171815526455428668 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.025+0000 7fcd2a78e640 1 --2- 10.20.193.28:0/369738158 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcd2410b480 0x7fcd2410b850 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.025+0000 7fcd2a78e640 1 -- 10.20.193.28:0/369738158 >> 10.20.193.28:0/369738158 conn(0x7fcd240e8e70 msgr2=0x7fcd24109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:53.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.025+0000 7fcd2a78e640 1 -- 10.20.193.28:0/369738158 shutdown_connections 2026-02-25T11:54:53.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.025+0000 7fcd2a78e640 1 -- 10.20.193.28:0/369738158 wait complete. 2026-02-25T11:54:53.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.026+0000 7fcd2a78e640 1 Processor -- start 2026-02-25T11:54:53.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.026+0000 7fcd2a78e640 1 -- start start 2026-02-25T11:54:53.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.026+0000 7fcd2a78e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd2410b480 0x7fcd240f3bd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.026+0000 7fcd2a78e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcd2410be70 0x7fcd24173ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.026+0000 7fcd2a78e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcd241136a0 0x7fcd240f4370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.026+0000 7fcd2a78e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcd240f95a0 con 0x7fcd2410b480 2026-02-25T11:54:53.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.026+0000 7fcd2a78e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcd240f9710 con 0x7fcd2410be70 2026-02-25T11:54:53.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.026+0000 7fcd2a78e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcd240f98d0 con 0x7fcd241136a0 2026-02-25T11:54:53.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.026+0000 7fcd2978c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd2410b480 0x7fcd240f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.026+0000 7fcd2978c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd2410b480 0x7fcd240f3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57676/0 (socket says 10.20.193.28:57676) 2026-02-25T11:54:53.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.026+0000 7fcd2978c640 1 -- 10.20.193.28:0/870208532 learned_addr learned my addr 10.20.193.28:0/870208532 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:53.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.026+0000 7fcd28f8b640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcd2410be70 0x7fcd24173ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.026+0000 7fcd29f8d640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcd241136a0 0x7fcd240f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.027+0000 7fcd2978c640 1 -- 10.20.193.28:0/870208532 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcd241136a0 msgr2=0x7fcd240f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:53.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.027+0000 7fcd2978c640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcd241136a0 0x7fcd240f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.027+0000 7fcd2978c640 1 -- 10.20.193.28:0/870208532 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcd2410be70 msgr2=0x7fcd24173ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:53.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.027+0000 7fcd2978c640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcd2410be70 0x7fcd24173ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.027+0000 7fcd2978c640 1 -- 10.20.193.28:0/870208532 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd18009570 con 0x7fcd2410b480 2026-02-25T11:54:53.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.027+0000 7fcd2978c640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd2410b480 0x7fcd240f3bd0 secure :-1 s=READY pgs=713 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd140072a0 tx=0x7fcd1400ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:53.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.027+0000 7fcd127fc640 1 -- 10.20.193.28:0/870208532 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcd14004500 con 0x7fcd2410b480 2026-02-25T11:54:53.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.027+0000 7fcd127fc640 1 -- 10.20.193.28:0/870208532 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcd1400bd00 con 0x7fcd2410b480 2026-02-25T11:54:53.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.027+0000 7fcd127fc640 1 -- 10.20.193.28:0/870208532 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcd140106e0 con 0x7fcd2410b480 2026-02-25T11:54:53.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.027+0000 7fcd2a78e640 1 -- 10.20.193.28:0/870208532 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd240f9bb0 con 0x7fcd2410b480 2026-02-25T11:54:53.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.027+0000 7fcd2a78e640 1 -- 10.20.193.28:0/870208532 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd240fa0d0 con 0x7fcd2410b480 2026-02-25T11:54:53.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.031+0000 7fcd127fc640 1 -- 10.20.193.28:0/870208532 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fcd1400b840 con 0x7fcd2410b480 2026-02-25T11:54:53.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.032+0000 7fcd127fc640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcd000b5760 0x7fcd000d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.032+0000 7fcd127fc640 1 -- 10.20.193.28:0/870208532 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fcd140e63a0 con 0x7fcd2410b480 2026-02-25T11:54:53.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.033+0000 7fcd28f8b640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcd000b5760 0x7fcd000d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.033+0000 7fcd2da12e00 1 Processor -- start 2026-02-25T11:54:53.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.033+0000 7fcd2da12e00 1 -- start start 2026-02-25T11:54:53.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.033+0000 7fcd2da12e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a6464f4510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.033+0000 7fcd2da12e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a6465391b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.033+0000 7fcd2da12e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a646559ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.033+0000 7fcd2da12e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a64640a9b0 con 0x55a6464f4140 2026-02-25T11:54:53.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.033+0000 7fcd2da12e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a6464066e0 con 0x55a6464f4a50 2026-02-25T11:54:53.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.033+0000 7fcd2da12e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a646400840 con 0x55a6465396f0 2026-02-25T11:54:53.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd28f8b640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcd000b5760 0x7fcd000d5c10 secure :-1 s=READY pgs=344 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd240fa570 tx=0x7fcd1803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:53.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd2978c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a6464f4510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd2978c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a6464f4510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57688/0 (socket says 10.20.193.28:57688) 2026-02-25T11:54:53.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd2978c640 1 -- 10.20.193.28:0/531461471 learned_addr learned my addr 10.20.193.28:0/531461471 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:53.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd29f8d640 1 --2- 10.20.193.28:0/531461471 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a646559ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd28f8b640 1 --2- 10.20.193.28:0/531461471 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a6465391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd2978c640 1 -- 10.20.193.28:0/531461471 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 msgr2=0x55a646559ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:53.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd2978c640 1 --2- 10.20.193.28:0/531461471 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a646559ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd2978c640 1 -- 10.20.193.28:0/531461471 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 msgr2=0x55a6465391b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:53.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd2978c640 1 --2- 10.20.193.28:0/531461471 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a6465391b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd2978c640 1 -- 10.20.193.28:0/531461471 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6463abd80 con 0x55a6464f4140 2026-02-25T11:54:53.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd28f8b640 1 --2- 10.20.193.28:0/531461471 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a6465391b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:54:53.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd2978c640 1 --2- 10.20.193.28:0/531461471 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a6464f4510 secure :-1 s=READY pgs=714 gs=1 cs=0 l=1 c_cookie=359909696339949026 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd140ac7b0 tx=0x7fcd14002b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fea7f6418fb5e2 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:53.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fccf1ce5640 1 -- 10.20.193.28:0/531461471 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcd140ac880 con 0x55a6464f4140 2026-02-25T11:54:53.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fccf1ce5640 1 -- 10.20.193.28:0/531461471 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcd140ac1b0 con 0x55a6464f4140 2026-02-25T11:54:53.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fccf1ce5640 1 -- 10.20.193.28:0/531461471 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcd140b78f0 con 0x55a6464f4140 2026-02-25T11:54:53.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd2da12e00 1 -- 10.20.193.28:0/531461471 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 msgr2=0x55a6464f4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:53.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.034+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/531461471 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a6464f4510 secure :-1 s=READY pgs=714 gs=1 cs=0 l=1 c_cookie=359909696339949026 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd140ac7b0 tx=0x7fcd14002b60 comp rx=0 tx=0).stop 2026-02-25T11:54:53.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.035+0000 7fcd2da12e00 1 -- 10.20.193.28:0/531461471 shutdown_connections 2026-02-25T11:54:53.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.035+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/531461471 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a646559ad0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.035+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/531461471 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a6465391b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.035+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/531461471 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a6464f4510 unknown :-1 s=CLOSED pgs=714 gs=1 cs=0 l=1 c_cookie=359909696339949026 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.035+0000 7fcd2da12e00 1 -- 10.20.193.28:0/531461471 >> 10.20.193.28:0/531461471 conn(0x55a646413d40 msgr2=0x55a6464f2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:53.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.035+0000 7fcd2da12e00 1 -- 10.20.193.28:0/531461471 shutdown_connections 2026-02-25T11:54:53.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.035+0000 7fcd2da12e00 1 -- 10.20.193.28:0/531461471 wait complete. 2026-02-25T11:54:53.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.036+0000 7fcd2da12e00 1 -- start start 2026-02-25T11:54:53.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.036+0000 7fcd2da12e00 1 Processor -- start 2026-02-25T11:54:53.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.036+0000 7fcd2da12e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a6465513f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.036+0000 7fcd2da12e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a646551930 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.036+0000 7fcd2da12e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a6465536c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.036+0000 7fcd2da12e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a646553c30 con 0x55a6464f4140 2026-02-25T11:54:53.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.036+0000 7fcd2da12e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a646553da0 con 0x55a6464f4a50 2026-02-25T11:54:53.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.036+0000 7fcd2da12e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a646553f60 con 0x55a6465396f0 2026-02-25T11:54:53.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fcd2978c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a6465513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fcd2978c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a6465513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57700/0 (socket says 10.20.193.28:57700) 2026-02-25T11:54:53.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fcd2978c640 1 -- 10.20.193.28:0/3211710782 learned_addr learned my addr 10.20.193.28:0/3211710782 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:53.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fcd28f8b640 1 --2- 10.20.193.28:0/3211710782 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a646551930 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fcd29f8d640 1 --2- 10.20.193.28:0/3211710782 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a6465536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fcd2978c640 1 -- 10.20.193.28:0/3211710782 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 msgr2=0x55a6465536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:53.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fcd2978c640 1 --2- 10.20.193.28:0/3211710782 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a6465536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fcd2978c640 1 -- 10.20.193.28:0/3211710782 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 msgr2=0x55a646551930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:53.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fcd2978c640 1 --2- 10.20.193.28:0/3211710782 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a646551930 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fcd2978c640 1 -- 10.20.193.28:0/3211710782 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a646384350 con 0x55a6464f4140 2026-02-25T11:54:53.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fcd2978c640 1 --2- 10.20.193.28:0/3211710782 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a6465513f0 secure :-1 s=READY pgs=715 gs=1 cs=0 l=1 c_cookie=4212794088549593776 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd140a8f90 tx=0x7fcd140ac0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a76d9e737dd32b0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:53.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fccdaffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:53.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fccdaffd640 1 -- 10.20.193.28:0/3211710782 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcd140b75f0 con 0x55a6464f4140 2026-02-25T11:54:53.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.037+0000 7fccdaffd640 1 -- 10.20.193.28:0/3211710782 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcd1400bb20 con 0x55a6464f4140 2026-02-25T11:54:53.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.038+0000 7fccdaffd640 1 -- 10.20.193.28:0/3211710782 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcd140aaa30 con 0x55a6464f4140 2026-02-25T11:54:53.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.038+0000 7fcd2da12e00 1 -- 10.20.193.28:0/3211710782 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd240fa0d0 con 0x55a6464f4140 2026-02-25T11:54:53.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.038+0000 7fccdaffd640 1 -- 10.20.193.28:0/3211710782 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fcd140ae110 con 0x55a6464f4140 2026-02-25T11:54:53.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.038+0000 7fccdaffd640 10 client.16872 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:54:53.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.038+0000 7fcd2da12e00 10 client.16872 _ll_register_callbacks cb 0x55a6463819a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:53.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.038+0000 7fcd2da12e00 1 -- 10.20.193.28:0/3211710782 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a6460966f0 con 0x55a6464f4140 2026-02-25T11:54:53.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.038+0000 7fccdaffd640 1 -- 10.20.193.28:0/3211710782 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcd14002730 con 0x55a6464f4140 2026-02-25T11:54:53.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.038+0000 7fcd2da12e00 10 client.16872 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:53.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.038+0000 7fcd2da12e00 1 -- 10.20.193.28:0/3211710782 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55a646396150 con 0x55a6464f4140 2026-02-25T11:54:53.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.038+0000 7fccdaffd640 1 -- 10.20.193.28:0/3211710782 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fcd140abac0 con 0x55a6464f4140 2026-02-25T11:54:53.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.038+0000 7fcd2da12e00 10 client.16872 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:53.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.038+0000 7fcd2da12e00 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:53.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.039+0000 7fcd2da12e00 20 client.16872 populate_metadata read hostname 'trial028' 2026-02-25T11:54:53.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.039+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/3211710782 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a646536840 0x55a646536ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.039+0000 7fcd2da12e00 4 client.16872 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:54:53.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.039+0000 7fcd2da12e00 1 -- 10.20.193.28:0/3211710782 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6463d2910 con 0x55a646536840 2026-02-25T11:54:53.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.039+0000 7fcd28f8b640 1 --2- 10.20.193.28:0/3211710782 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a646536840 0x55a646536ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.040+0000 7fcd28f8b640 1 --2- 10.20.193.28:0/3211710782 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a646536840 0x55a646536ce0 crc :-1 s=READY pgs=128 gs=4 cs=0 l=0 c_cookie=8875137679850997166 s_cookie=476166344928090811 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b2ad2b2843405ae server_cookie=69baec4936f5abb in_seq=0 out_seq=0 2026-02-25T11:54:53.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.040+0000 7fccdaffd640 10 client.16872 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:53.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.040+0000 7fccdaffd640 1 -- 10.20.193.28:0/3211710782 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55a6463d2910 con 0x55a646536840 2026-02-25T11:54:53.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.040+0000 7fccdaffd640 10 client.16872 handle_command_reply: tid=0 2026-02-25T11:54:53.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.040+0000 7fcd2da12e00 1 client.16872 shutdown 2026-02-25T11:54:53.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.040+0000 7fcd2da12e00 1 -- 10.20.193.28:0/3211710782 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 msgr2=0x55a6465513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:53.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.040+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/3211710782 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a6465513f0 secure :-1 s=READY pgs=715 gs=1 cs=0 l=1 c_cookie=4212794088549593776 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd140a8f90 tx=0x7fcd140ac0c0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.041+0000 7fcd2da12e00 1 -- 10.20.193.28:0/3211710782 shutdown_connections 2026-02-25T11:54:53.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.041+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/3211710782 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a646536840 0x55a646536ce0 crc :-1 s=READY pgs=128 gs=4 cs=0 l=0 c_cookie=8875137679850997166 s_cookie=476166344928090811 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.041+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/3211710782 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a6465536c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.041+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/3211710782 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a646551930 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.041+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/3211710782 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a6465513f0 unknown :-1 s=CLOSED pgs=715 gs=1 cs=0 l=1 c_cookie=4212794088549593776 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.041+0000 7fccdaffd640 0 client.16872 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:53.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.041+0000 7fcd2da12e00 1 -- 10.20.193.28:0/3211710782 >> 10.20.193.28:0/3211710782 conn(0x55a646413d40 msgr2=0x55a6464f2230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:53.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.041+0000 7fcd2da12e00 1 -- 10.20.193.28:0/3211710782 shutdown_connections 2026-02-25T11:54:53.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.041+0000 7fcd2da12e00 1 -- 10.20.193.28:0/3211710782 wait complete. 2026-02-25T11:54:53.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.041+0000 7fcd2da12e00 20 client.16872 trim_cache size 0 max 16384 2026-02-25T11:54:53.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.050+0000 7fcd2da12e00 1 Processor -- start 2026-02-25T11:54:53.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.050+0000 7fcd2da12e00 1 -- start start 2026-02-25T11:54:53.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.050+0000 7fcd2da12e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a646553650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.050+0000 7fcd2da12e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a6464f3650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.050+0000 7fcd2da12e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a6464f3b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.050+0000 7fcd2da12e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a6463fe470 con 0x55a6464f4140 2026-02-25T11:54:53.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.050+0000 7fcd2da12e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a6460be700 con 0x55a6464f4a50 2026-02-25T11:54:53.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.050+0000 7fcd2da12e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a6464f4f30 con 0x55a6465396f0 2026-02-25T11:54:53.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.050+0000 7fcd2978c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a646553650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.050+0000 7fcd2978c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a646553650 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57712/0 (socket says 10.20.193.28:57712) 2026-02-25T11:54:53.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.050+0000 7fcd2978c640 1 -- 10.20.193.28:0/4168555395 learned_addr learned my addr 10.20.193.28:0/4168555395 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:53.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.050+0000 7fcd28f8b640 1 --2- 10.20.193.28:0/4168555395 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a6464f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.050+0000 7fcd29f8d640 1 --2- 10.20.193.28:0/4168555395 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a6464f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2978c640 1 -- 10.20.193.28:0/4168555395 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 msgr2=0x55a6464f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:53.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2978c640 1 --2- 10.20.193.28:0/4168555395 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a6464f3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2978c640 1 -- 10.20.193.28:0/4168555395 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 msgr2=0x55a6464f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:53.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2978c640 1 --2- 10.20.193.28:0/4168555395 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a6464f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2978c640 1 -- 10.20.193.28:0/4168555395 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a646390bd0 con 0x55a6464f4140 2026-02-25T11:54:53.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd28f8b640 1 --2- 10.20.193.28:0/4168555395 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a6464f3650 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:54:53.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2978c640 1 --2- 10.20.193.28:0/4168555395 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a646553650 secure :-1 s=READY pgs=716 gs=1 cs=0 l=1 c_cookie=6954929179341515233 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd140aca40 tx=0x7fcd140c8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6084df3d64a669e1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:53.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fccf14e4640 1 -- 10.20.193.28:0/4168555395 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcd140b4a20 con 0x55a6464f4140 2026-02-25T11:54:53.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fccf14e4640 1 -- 10.20.193.28:0/4168555395 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55a6460966f0 con 0x55a6464f4140 2026-02-25T11:54:53.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fccf14e4640 1 -- 10.20.193.28:0/4168555395 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcd14002730 con 0x55a6464f4140 2026-02-25T11:54:53.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2da12e00 1 -- 10.20.193.28:0/4168555395 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 msgr2=0x55a646553650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:53.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/4168555395 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a646553650 secure :-1 s=READY pgs=716 gs=1 cs=0 l=1 c_cookie=6954929179341515233 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd140aca40 tx=0x7fcd140c8000 comp rx=0 tx=0).stop 2026-02-25T11:54:53.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2da12e00 1 -- 10.20.193.28:0/4168555395 shutdown_connections 2026-02-25T11:54:53.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/4168555395 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a6464f3b90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/4168555395 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4a50 0x55a6464f3650 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/4168555395 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4140 0x55a646553650 unknown :-1 s=CLOSED pgs=716 gs=1 cs=0 l=1 c_cookie=6954929179341515233 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2da12e00 1 -- 10.20.193.28:0/4168555395 >> 10.20.193.28:0/4168555395 conn(0x55a646413d40 msgr2=0x55a646555f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:53.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2da12e00 1 -- 10.20.193.28:0/4168555395 shutdown_connections 2026-02-25T11:54:53.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.051+0000 7fcd2da12e00 1 -- 10.20.193.28:0/4168555395 wait complete. 2026-02-25T11:54:53.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.052+0000 7fcd2da12e00 1 -- start start 2026-02-25T11:54:53.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.052+0000 7fcd2da12e00 1 Processor -- start 2026-02-25T11:54:53.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.052+0000 7fcd2da12e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4140 0x55a646555c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.052+0000 7fcd2da12e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4a50 0x55a646533dd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd2da12e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a646534310 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd2da12e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a6464cf490 con 0x55a6464f4a50 2026-02-25T11:54:53.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd2da12e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a646534850 con 0x55a6464f4140 2026-02-25T11:54:53.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd2da12e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a6465349c0 con 0x55a6465396f0 2026-02-25T11:54:53.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd28f8b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4a50 0x55a646533dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd28f8b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4a50 0x55a646533dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:57724/0 (socket says 10.20.193.28:57724) 2026-02-25T11:54:53.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd28f8b640 1 -- 10.20.193.28:0/2180040492 learned_addr learned my addr 10.20.193.28:0/2180040492 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:54:53.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd2978c640 1 --2- 10.20.193.28:0/2180040492 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4140 0x55a646555c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd29f8d640 1 --2- 10.20.193.28:0/2180040492 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a646534310 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd28f8b640 1 -- 10.20.193.28:0/2180040492 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 msgr2=0x55a646534310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:53.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd28f8b640 1 --2- 10.20.193.28:0/2180040492 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a646534310 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd28f8b640 1 -- 10.20.193.28:0/2180040492 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4140 msgr2=0x55a646555c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:54:53.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd28f8b640 1 --2- 10.20.193.28:0/2180040492 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4140 0x55a646555c80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd28f8b640 1 -- 10.20.193.28:0/2180040492 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a646384680 con 0x55a6464f4a50 2026-02-25T11:54:53.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd28f8b640 1 --2- 10.20.193.28:0/2180040492 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4a50 0x55a646533dd0 secure :-1 s=READY pgs=717 gs=1 cs=0 l=1 c_cookie=5195102300553231194 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd1802f6a0 tx=0x7fcd18009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4818b6047c4a3b5a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:54:53.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fccdaffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:54:53.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fccdaffd640 1 -- 10.20.193.28:0/2180040492 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcd18057070 con 0x55a6464f4a50 2026-02-25T11:54:53.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fccdaffd640 1 -- 10.20.193.28:0/2180040492 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcd1804a040 con 0x55a6464f4a50 2026-02-25T11:54:53.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd2da12e00 1 -- 10.20.193.28:0/2180040492 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a646396150 con 0x55a6464f4a50 2026-02-25T11:54:53.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fccdaffd640 1 -- 10.20.193.28:0/2180040492 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcd18038920 con 0x55a6464f4a50 2026-02-25T11:54:53.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd2da12e00 10 client.16878 _ll_register_callbacks cb 0x55a6463819a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:54:53.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.053+0000 7fcd2da12e00 1 -- 10.20.193.28:0/2180040492 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd140ac1b0 con 0x55a6464f4a50 2026-02-25T11:54:53.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.054+0000 7fccdaffd640 1 -- 10.20.193.28:0/2180040492 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fcd18038ab0 con 0x55a6464f4a50 2026-02-25T11:54:53.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.054+0000 7fccdaffd640 1 -- 10.20.193.28:0/2180040492 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcd18038cb0 con 0x55a6464f4a50 2026-02-25T11:54:53.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.054+0000 7fcd2da12e00 10 client.16878 fetch_fsmap learned FSMap version 23 2026-02-25T11:54:53.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.054+0000 7fcd2da12e00 1 -- 10.20.193.28:0/2180040492 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55a6463963b0 con 0x55a6464f4a50 2026-02-25T11:54:53.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.054+0000 7fccdaffd640 1 -- 10.20.193.28:0/2180040492 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fcd1804f040 con 0x55a6464f4a50 2026-02-25T11:54:53.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.054+0000 7fcd2da12e00 10 client.16878 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:54:53.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.054+0000 7fcd2da12e00 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:54:53.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.054+0000 7fcd2da12e00 20 client.16878 populate_metadata read hostname 'trial028' 2026-02-25T11:54:53.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.054+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/2180040492 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a6464d6b50 0x55a6464d6ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:54:53.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.054+0000 7fcd2da12e00 4 client.16878 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:54:53.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.054+0000 7fcd2da12e00 1 -- 10.20.193.28:0/2180040492 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a646093f40 con 0x55a6464d6b50 2026-02-25T11:54:53.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.054+0000 7fcd2978c640 1 --2- 10.20.193.28:0/2180040492 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a6464d6b50 0x55a6464d6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:54:53.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.055+0000 7fcd2978c640 1 --2- 10.20.193.28:0/2180040492 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a6464d6b50 0x55a6464d6ff0 crc :-1 s=READY pgs=129 gs=4 cs=0 l=0 c_cookie=9651933686963203853 s_cookie=14354308514233883228 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85f28e8cd7116f0d server_cookie=c734bf8a93469a5c in_seq=0 out_seq=0 2026-02-25T11:54:53.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.055+0000 7fccdaffd640 10 client.16878 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:53.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.055+0000 7fccdaffd640 1 -- 10.20.193.28:0/2180040492 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x55a646093f40 con 0x55a6464d6b50 2026-02-25T11:54:53.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.055+0000 7fccdaffd640 10 client.16878 handle_command_reply: tid=0 2026-02-25T11:54:53.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.055+0000 7fcd2da12e00 1 client.16878 shutdown 2026-02-25T11:54:53.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2da12e00 1 -- 10.20.193.28:0/2180040492 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4a50 msgr2=0x55a646533dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:53.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/2180040492 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4a50 0x55a646533dd0 secure :-1 s=READY pgs=717 gs=1 cs=0 l=1 c_cookie=5195102300553231194 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd1802f6a0 tx=0x7fcd18009310 comp rx=0 tx=0).stop 2026-02-25T11:54:53.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2da12e00 1 -- 10.20.193.28:0/2180040492 shutdown_connections 2026-02-25T11:54:53.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/2180040492 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55a6464d6b50 0x55a6464d6ff0 crc :-1 s=READY pgs=129 gs=4 cs=0 l=0 c_cookie=9651933686963203853 s_cookie=14354308514233883228 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/2180040492 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a6465396f0 0x55a646534310 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/2180040492 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a6464f4a50 0x55a646533dd0 unknown :-1 s=CLOSED pgs=717 gs=1 cs=0 l=1 c_cookie=5195102300553231194 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2da12e00 1 --2- 10.20.193.28:0/2180040492 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a6464f4140 0x55a646555c80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2da12e00 1 -- 10.20.193.28:0/2180040492 >> 10.20.193.28:0/2180040492 conn(0x55a646413d40 msgr2=0x55a646550aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:53.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fccdaffd640 0 client.16878 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:54:53.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2da12e00 1 -- 10.20.193.28:0/2180040492 shutdown_connections 2026-02-25T11:54:53.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2da12e00 1 -- 10.20.193.28:0/2180040492 wait complete. 2026-02-25T11:54:53.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2da12e00 20 client.16878 trim_cache size 0 max 16384 2026-02-25T11:54:53.064 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:54:53.064 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:54:53.064 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:54:53.064 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:54:53.064 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.064 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "8c0edddc-5eeb-4011-a44d-a25962ab3520", 2026-02-25T11:54:53.065 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:54:53.065 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:54:53.065 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.065 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:54:53.065 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:54:53.065 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:54:53.065 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.065 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.065 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.065 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.065 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.065 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.066 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:54:53.066 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:54:53.066 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.066 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.066 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.066 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.066 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.066 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.066 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:54:53.066 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:54:53.066 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.066 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.066 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.067 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.067 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.067 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.067 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:54:53.067 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:54:53.067 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.067 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.067 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.067 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.067 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.067 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.067 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:54:53.068 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:54:53.068 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.068 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.068 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.068 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.068 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.068 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.068 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:54:53.068 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:54:53.068 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.068 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.068 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.069 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.069 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.069 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.069 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:54:53.069 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:54:53.069 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.069 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.069 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.069 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.069 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.069 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.069 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:54:53.069 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:54:53.070 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.070 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.070 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.070 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.070 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.070 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.070 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:54:53.070 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:54:53.070 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.070 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.070 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.070 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.071 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.071 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.071 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:54:53.071 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:54:53.071 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.071 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.071 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.071 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.071 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.071 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.071 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:54:53.071 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:54:53.071 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.072 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.072 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.072 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.072 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.072 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.072 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:54:53.072 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:54:53.072 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.072 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.072 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.072 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.072 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.073 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.073 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:54:53.073 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:54:53.073 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.073 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.073 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.073 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.073 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.073 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:54:53.073 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:54:53.073 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:54:53.073 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:54:53.073 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:54:53.074 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:54:53.074 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:54:53.074 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:54:53.074 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:54:53.074 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:54:53.074 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:54:53.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2a78e640 1 -- 10.20.193.28:0/870208532 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcd000b5760 msgr2=0x7fcd000d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:53.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2a78e640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcd000b5760 0x7fcd000d5c10 secure :-1 s=READY pgs=344 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd240fa570 tx=0x7fcd1803a040 comp rx=0 tx=0).stop 2026-02-25T11:54:53.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2a78e640 1 -- 10.20.193.28:0/870208532 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd2410b480 msgr2=0x7fcd240f3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:54:53.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.056+0000 7fcd2a78e640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd2410b480 0x7fcd240f3bd0 secure :-1 s=READY pgs=713 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcd140072a0 tx=0x7fcd1400ddf0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.057+0000 7fcd2a78e640 1 -- 10.20.193.28:0/870208532 shutdown_connections 2026-02-25T11:54:53.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.057+0000 7fcd2a78e640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcd000b5760 0x7fcd000d5c10 unknown :-1 s=CLOSED pgs=344 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.057+0000 7fcd2a78e640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcd241136a0 0x7fcd240f4370 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.057+0000 7fcd2a78e640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcd2410be70 0x7fcd24173ff0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.057+0000 7fcd2a78e640 1 --2- 10.20.193.28:0/870208532 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcd2410b480 0x7fcd240f3bd0 unknown :-1 s=CLOSED pgs=713 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:54:53.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.057+0000 7fcd2a78e640 1 -- 10.20.193.28:0/870208532 >> 10.20.193.28:0/870208532 conn(0x7fcd240e8e70 msgr2=0x7fcd2410cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:54:53.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.057+0000 7fcd2a78e640 1 -- 10.20.193.28:0/870208532 shutdown_connections 2026-02-25T11:54:53.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:54:53.057+0000 7fcd2a78e640 1 -- 10.20.193.28:0/870208532 wait complete. 2026-02-25T11:54:53.157 INFO:tasks.cephfs.filesystem:scrub status for tag:8c0edddc-5eeb-4011-a44d-a25962ab3520 - {'path': '/', 'tag': '8c0edddc-5eeb-4011-a44d-a25962ab3520', 'options': 'recursive,force'} 2026-02-25T11:54:53.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:53 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:54:53.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:53 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:54:53.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:53 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:54:53.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:53 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:54:53.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:53 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:54:53.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:53 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:54:53.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:53 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:54:53.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:53 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:54:53.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:53 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:54:53.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:53 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:54:53.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:53 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:54:53.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:53 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:54:54.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:54 trial167 ceph-mon[43392]: pgmap v426: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:54:54.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:54 trial119 ceph-mon[42710]: pgmap v426: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:54:54.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:54 trial028 ceph-mon[39224]: pgmap v426: 97 pgs: 97 active+clean; 5.2 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:54:55.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:55 trial167 ceph-mon[43392]: pgmap v427: 97 pgs: 97 active+clean; 5.3 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:54:55.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:55 trial119 ceph-mon[42710]: pgmap v427: 97 pgs: 97 active+clean; 5.3 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:54:55.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:55 trial028 ceph-mon[39224]: pgmap v427: 97 pgs: 97 active+clean; 5.3 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:54:57.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:57 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:54:57.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:57 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:54:57.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:57 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:54:58.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:54:58 trial167 ceph-mon[43392]: pgmap v428: 97 pgs: 97 active+clean; 5.3 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:58.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:54:58 trial119 ceph-mon[42710]: pgmap v428: 97 pgs: 97 active+clean; 5.3 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:54:58.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:54:58 trial028 ceph-mon[39224]: pgmap v428: 97 pgs: 97 active+clean; 5.3 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:55:00.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:00 trial167 ceph-mon[43392]: pgmap v429: 97 pgs: 97 active+clean; 5.3 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:55:00.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:00 trial119 ceph-mon[42710]: pgmap v429: 97 pgs: 97 active+clean; 5.3 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:55:00.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:00 trial028 ceph-mon[39224]: pgmap v429: 97 pgs: 97 active+clean; 5.3 MiB data, 344 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:55:02.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:02 trial167 ceph-mon[43392]: pgmap v430: 97 pgs: 97 active+clean; 5.3 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:55:02.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:02 trial119 ceph-mon[42710]: pgmap v430: 97 pgs: 97 active+clean; 5.3 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:55:02.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:02 trial028 ceph-mon[39224]: pgmap v430: 97 pgs: 97 active+clean; 5.3 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:55:04.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:04 trial167 ceph-mon[43392]: pgmap v431: 97 pgs: 97 active+clean; 5.3 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:55:04.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:04 trial119 ceph-mon[42710]: pgmap v431: 97 pgs: 97 active+clean; 5.3 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:55:04.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:04 trial028 ceph-mon[39224]: pgmap v431: 97 pgs: 97 active+clean; 5.3 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:55:06.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:06 trial167 ceph-mon[43392]: pgmap v432: 97 pgs: 97 active+clean; 5.4 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:06.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:06 trial119 ceph-mon[42710]: pgmap v432: 97 pgs: 97 active+clean; 5.4 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:06.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:06 trial028 ceph-mon[39224]: pgmap v432: 97 pgs: 97 active+clean; 5.4 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:08.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:08 trial167 ceph-mon[43392]: pgmap v433: 97 pgs: 97 active+clean; 5.4 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 11 op/s 2026-02-25T11:55:08.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:08 trial119 ceph-mon[42710]: pgmap v433: 97 pgs: 97 active+clean; 5.4 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 11 op/s 2026-02-25T11:55:08.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:08 trial028 ceph-mon[39224]: pgmap v433: 97 pgs: 97 active+clean; 5.4 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 11 op/s 2026-02-25T11:55:10.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:10 trial119 ceph-mon[42710]: pgmap v434: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:55:10.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:10 trial028 ceph-mon[39224]: pgmap v434: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:55:10.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:10 trial167 ceph-mon[43392]: pgmap v434: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:55:12.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:12 trial119 ceph-mon[42710]: pgmap v435: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:55:12.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:12 trial028 ceph-mon[39224]: pgmap v435: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:55:12.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:12 trial167 ceph-mon[43392]: pgmap v435: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:55:14.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:14 trial119 ceph-mon[42710]: pgmap v436: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:55:14.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:14 trial028 ceph-mon[39224]: pgmap v436: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:55:14.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:14 trial167 ceph-mon[43392]: pgmap v436: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:55:16.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:16 trial119 ceph-mon[42710]: pgmap v437: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:16.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:16 trial028 ceph-mon[39224]: pgmap v437: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:16.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:16 trial167 ceph-mon[43392]: pgmap v437: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:18.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:18 trial119 ceph-mon[42710]: pgmap v438: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:55:18.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:18 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:55:18.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:18 trial028 ceph-mon[39224]: pgmap v438: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:55:18.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:18 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:55:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:18 trial167 ceph-mon[43392]: pgmap v438: 97 pgs: 97 active+clean; 5.5 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:55:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:18 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:55:19.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:19 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:55:19.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:19 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:55:19.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:19 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:55:19.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:19 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:55:19.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:19 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:55:19.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:19 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:55:19.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:19 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:55:19.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:19 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:55:19.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:19 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:55:19.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:19 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:55:19.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:19 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:55:19.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:19 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:55:20.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:20 trial028 ceph-mon[39224]: pgmap v439: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:55:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:20 trial167 ceph-mon[43392]: pgmap v439: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:55:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:20 trial119 ceph-mon[42710]: pgmap v439: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:55:22.551 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:22 trial167 ceph-mon[43392]: pgmap v440: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:55:22.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:22 trial028 ceph-mon[39224]: pgmap v440: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:55:22.561 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:22 trial119 ceph-mon[42710]: pgmap v440: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:55:23.157 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:55:23.343 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:55:23.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.524+0000 7fec0aeab640 1 -- 10.20.193.28:0/648063338 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec0410ddc0 msgr2=0x7fec04112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:23.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.524+0000 7fec0aeab640 1 --2- 10.20.193.28:0/648063338 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec0410ddc0 0x7fec04112e10 secure :-1 s=READY pgs=718 gs=1 cs=0 l=1 c_cookie=12352089217704763332 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febf8009950 tx=0x7febf802f190 comp rx=0 tx=0).stop 2026-02-25T11:55:23.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.524+0000 7fec0aeab640 1 -- 10.20.193.28:0/648063338 shutdown_connections 2026-02-25T11:55:23.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.524+0000 7fec0aeab640 1 --2- 10.20.193.28:0/648063338 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec041135c0 0x7fec04173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.524+0000 7fec0aeab640 1 --2- 10.20.193.28:0/648063338 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec0410ddc0 0x7fec04112e10 unknown :-1 s=CLOSED pgs=718 gs=1 cs=0 l=1 c_cookie=12352089217704763332 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.524+0000 7fec0aeab640 1 --2- 10.20.193.28:0/648063338 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec0410d3d0 0x7fec0410d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.524+0000 7fec0aeab640 1 -- 10.20.193.28:0/648063338 >> 10.20.193.28:0/648063338 conn(0x7fec040eaca0 msgr2=0x7fec0410b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:23.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.524+0000 7fec0aeab640 1 -- 10.20.193.28:0/648063338 shutdown_connections 2026-02-25T11:55:23.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.524+0000 7fec0aeab640 1 -- 10.20.193.28:0/648063338 wait complete. 2026-02-25T11:55:23.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.525+0000 7fec0aeab640 1 Processor -- start 2026-02-25T11:55:23.526 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.525+0000 7fec0aeab640 1 -- start start 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.525+0000 7fec0aeab640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec0410d3d0 0x7fec040f5970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.525+0000 7fec0aeab640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec0410ddc0 0x7fec04173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.525+0000 7fec0aeab640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec041135c0 0x7fec040f6070 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.525+0000 7fec0aeab640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fec040fb280 con 0x7fec0410d3d0 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.525+0000 7fec0aeab640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fec040fb3f0 con 0x7fec0410ddc0 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.525+0000 7fec0aeab640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fec040fb5b0 con 0x7fec041135c0 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.525+0000 7fec09ea9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec0410d3d0 0x7fec040f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.525+0000 7fec096a8640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec0410ddc0 0x7fec04173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.525+0000 7fec09ea9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec0410d3d0 0x7fec040f5970 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40324/0 (socket says 10.20.193.28:40324) 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.525+0000 7fec09ea9640 1 -- 10.20.193.28:0/3342930680 learned_addr learned my addr 10.20.193.28:0/3342930680 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.525+0000 7fec0a6aa640 1 --2- 10.20.193.28:0/3342930680 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec041135c0 0x7fec040f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.526+0000 7fec09ea9640 1 -- 10.20.193.28:0/3342930680 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec041135c0 msgr2=0x7fec040f6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.526+0000 7fec09ea9640 1 --2- 10.20.193.28:0/3342930680 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec041135c0 0x7fec040f6070 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.526+0000 7fec09ea9640 1 -- 10.20.193.28:0/3342930680 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec0410ddc0 msgr2=0x7fec04173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.526+0000 7fec09ea9640 1 --2- 10.20.193.28:0/3342930680 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec0410ddc0 0x7fec04173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.526+0000 7fec09ea9640 1 -- 10.20.193.28:0/3342930680 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf8009570 con 0x7fec0410d3d0 2026-02-25T11:55:23.527 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.526+0000 7fec09ea9640 1 --2- 10.20.193.28:0/3342930680 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec0410d3d0 0x7fec040f5970 secure :-1 s=READY pgs=719 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febf400c9b0 tx=0x7febf400ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:23.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.526+0000 7febf2ffd640 1 -- 10.20.193.28:0/3342930680 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febf40044a0 con 0x7fec0410d3d0 2026-02-25T11:55:23.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.526+0000 7febf2ffd640 1 -- 10.20.193.28:0/3342930680 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7febf4007d80 con 0x7fec0410d3d0 2026-02-25T11:55:23.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.526+0000 7febf2ffd640 1 -- 10.20.193.28:0/3342930680 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febf400f6d0 con 0x7fec0410d3d0 2026-02-25T11:55:23.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.526+0000 7fec0aeab640 1 -- 10.20.193.28:0/3342930680 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec040fb890 con 0x7fec0410d3d0 2026-02-25T11:55:23.528 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.526+0000 7fec0aeab640 1 -- 10.20.193.28:0/3342930680 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec040fbd10 con 0x7fec0410d3d0 2026-02-25T11:55:23.531 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.529+0000 7febf2ffd640 1 -- 10.20.193.28:0/3342930680 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7febf40078c0 con 0x7fec0410d3d0 2026-02-25T11:55:23.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.531+0000 7febf2ffd640 1 --2- 10.20.193.28:0/3342930680 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7febe00b5760 0x7febe00d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.531+0000 7febf2ffd640 1 -- 10.20.193.28:0/3342930680 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7febf40e6a20 con 0x7fec0410d3d0 2026-02-25T11:55:23.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.531+0000 7fec096a8640 1 --2- 10.20.193.28:0/3342930680 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7febe00b5760 0x7febe00d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.533 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.532+0000 7fec0e12fe00 1 Processor -- start 2026-02-25T11:55:23.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.532+0000 7fec0e12fe00 1 -- start start 2026-02-25T11:55:23.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.532+0000 7fec096a8640 1 --2- 10.20.193.28:0/3342930680 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7febe00b5760 0x7febe00d5c10 secure :-1 s=READY pgs=345 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec040fc310 tx=0x7febf803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:23.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.532+0000 7fec0e12fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 0x56351960e510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.532+0000 7fec0e12fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960ea50 0x5635196531b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.532+0000 7fec0e12fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 0x563519673ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.532+0000 7fec0e12fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5635195249b0 con 0x56351960e140 2026-02-25T11:55:23.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.532+0000 7fec0e12fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5635195206e0 con 0x56351960ea50 2026-02-25T11:55:23.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.532+0000 7fec0e12fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56351951a840 con 0x5635196536f0 2026-02-25T11:55:23.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7fec09ea9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 0x56351960e510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7fec09ea9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 0x56351960e510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40338/0 (socket says 10.20.193.28:40338) 2026-02-25T11:55:23.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7fec09ea9640 1 -- 10.20.193.28:0/2896423527 learned_addr learned my addr 10.20.193.28:0/2896423527 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:23.534 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7fec096a8640 1 --2- 10.20.193.28:0/2896423527 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960ea50 0x5635196531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7fec0a6aa640 1 --2- 10.20.193.28:0/2896423527 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 0x563519673ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7fec09ea9640 1 -- 10.20.193.28:0/2896423527 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 msgr2=0x563519673ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7fec09ea9640 1 --2- 10.20.193.28:0/2896423527 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 0x563519673ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7fec09ea9640 1 -- 10.20.193.28:0/2896423527 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960ea50 msgr2=0x5635196531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7fec09ea9640 1 --2- 10.20.193.28:0/2896423527 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960ea50 0x5635196531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7fec09ea9640 1 -- 10.20.193.28:0/2896423527 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635194c5d80 con 0x56351960e140 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7fec09ea9640 1 --2- 10.20.193.28:0/2896423527 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 0x56351960e510 secure :-1 s=READY pgs=720 gs=1 cs=0 l=1 c_cookie=3174124140885902460 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febf4005c40 tx=0x7febf40ace50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0cc116c27f047c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7febd24e6640 1 -- 10.20.193.28:0/2896423527 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febf40026a0 con 0x56351960e140 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7febd24e6640 1 -- 10.20.193.28:0/2896423527 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7febf4007ba0 con 0x56351960e140 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.533+0000 7febd24e6640 1 -- 10.20.193.28:0/2896423527 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febf40d20b0 con 0x56351960e140 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.534+0000 7fec0e12fe00 1 -- 10.20.193.28:0/2896423527 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 msgr2=0x56351960e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.534+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/2896423527 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 0x56351960e510 secure :-1 s=READY pgs=720 gs=1 cs=0 l=1 c_cookie=3174124140885902460 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febf4005c40 tx=0x7febf40ace50 comp rx=0 tx=0).stop 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.534+0000 7fec0e12fe00 1 -- 10.20.193.28:0/2896423527 shutdown_connections 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.534+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/2896423527 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 0x563519673ad0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.534+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/2896423527 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960ea50 0x5635196531b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.534+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/2896423527 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 0x56351960e510 unknown :-1 s=CLOSED pgs=720 gs=1 cs=0 l=1 c_cookie=3174124140885902460 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.535 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.534+0000 7fec0e12fe00 1 -- 10.20.193.28:0/2896423527 >> 10.20.193.28:0/2896423527 conn(0x56351952dd40 msgr2=0x56351960ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:23.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.534+0000 7fec0e12fe00 1 -- 10.20.193.28:0/2896423527 shutdown_connections 2026-02-25T11:55:23.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.534+0000 7fec0e12fe00 1 -- 10.20.193.28:0/2896423527 wait complete. 2026-02-25T11:55:23.536 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.535+0000 7fec0e12fe00 1 -- start start 2026-02-25T11:55:23.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.535+0000 7fec0e12fe00 1 Processor -- start 2026-02-25T11:55:23.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.535+0000 7fec0e12fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56351960e140 0x56351966b420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.535+0000 7fec0e12fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 0x56351966b960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec0e12fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5635196536f0 0x563519670950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec0e12fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x563519670e90 con 0x56351960ea50 2026-02-25T11:55:23.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec0e12fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x563519671000 con 0x5635196536f0 2026-02-25T11:55:23.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec0e12fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5635196711c0 con 0x56351960e140 2026-02-25T11:55:23.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec096a8640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 0x56351966b960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec096a8640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 0x56351966b960 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40340/0 (socket says 10.20.193.28:40340) 2026-02-25T11:55:23.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec096a8640 1 -- 10.20.193.28:0/43918145 learned_addr learned my addr 10.20.193.28:0/43918145 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:23.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec09ea9640 1 --2- 10.20.193.28:0/43918145 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56351960e140 0x56351966b420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec096a8640 1 -- 10.20.193.28:0/43918145 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56351960e140 msgr2=0x56351966b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:23.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec096a8640 1 --2- 10.20.193.28:0/43918145 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56351960e140 0x56351966b420 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec096a8640 1 -- 10.20.193.28:0/43918145 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5635196536f0 msgr2=0x563519670950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:23.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec096a8640 1 --2- 10.20.193.28:0/43918145 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5635196536f0 0x563519670950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec096a8640 1 -- 10.20.193.28:0/43918145 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56351949e350 con 0x56351960ea50 2026-02-25T11:55:23.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec096a8640 1 --2- 10.20.193.28:0/43918145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 0x56351966b960 secure :-1 s=READY pgs=721 gs=1 cs=0 l=1 c_cookie=4214733180526249127 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febf8009950 tx=0x7febf802f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7dbd7f4051c4a7 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:23.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7febbb7fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:23.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7febbb7fe640 1 -- 10.20.193.28:0/43918145 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febf8050640 con 0x56351960ea50 2026-02-25T11:55:23.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec0e12fe00 1 -- 10.20.193.28:0/43918145 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec040fbd10 con 0x56351960ea50 2026-02-25T11:55:23.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.536+0000 7fec0e12fe00 10 client.16890 _ll_register_callbacks cb 0x56351949b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:23.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.537+0000 7febbb7fe640 1 -- 10.20.193.28:0/43918145 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7febf803e040 con 0x56351960ea50 2026-02-25T11:55:23.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.537+0000 7fec0e12fe00 1 -- 10.20.193.28:0/43918145 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5635191b06f0 con 0x56351960ea50 2026-02-25T11:55:23.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.537+0000 7febbb7fe640 1 -- 10.20.193.28:0/43918145 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febf8044690 con 0x56351960ea50 2026-02-25T11:55:23.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.537+0000 7febbb7fe640 1 -- 10.20.193.28:0/43918145 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7febf8044820 con 0x56351960ea50 2026-02-25T11:55:23.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.537+0000 7febbb7fe640 1 -- 10.20.193.28:0/43918145 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7febf8044a20 con 0x56351960ea50 2026-02-25T11:55:23.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.537+0000 7fec0e12fe00 10 client.16890 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:23.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.537+0000 7fec0e12fe00 1 -- 10.20.193.28:0/43918145 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5635194b0150 con 0x56351960ea50 2026-02-25T11:55:23.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.537+0000 7febbb7fe640 1 -- 10.20.193.28:0/43918145 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7febf8044bc0 con 0x56351960ea50 2026-02-25T11:55:23.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.537+0000 7fec0e12fe00 10 client.16890 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:23.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.538+0000 7fec0e12fe00 10 client.16890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:23.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.538+0000 7fec0e12fe00 20 client.16890 populate_metadata read hostname 'trial028' 2026-02-25T11:55:23.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.538+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/43918145 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563519673a00 0x563519651420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.538+0000 7fec0e12fe00 4 client.16890 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:55:23.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.538+0000 7fec0e12fe00 1 -- 10.20.193.28:0/43918145 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635194ec910 con 0x563519673a00 2026-02-25T11:55:23.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.538+0000 7fec09ea9640 1 --2- 10.20.193.28:0/43918145 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563519673a00 0x563519651420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.538+0000 7fec09ea9640 1 --2- 10.20.193.28:0/43918145 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563519673a00 0x563519651420 crc :-1 s=READY pgs=130 gs=4 cs=0 l=0 c_cookie=2924810312004657442 s_cookie=16322369634195460319 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2897037b826fe122 server_cookie=e284b4b3e2d7d0df in_seq=0 out_seq=0 2026-02-25T11:55:23.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.538+0000 7febbb7fe640 10 client.16890 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.539+0000 7febbb7fe640 1 -- 10.20.193.28:0/43918145 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5635194ec910 con 0x563519673a00 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.539+0000 7febbb7fe640 10 client.16890 handle_command_reply: tid=0 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.539+0000 7fec0e12fe00 1 client.16890 shutdown 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.539+0000 7fec0e12fe00 1 -- 10.20.193.28:0/43918145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 msgr2=0x56351966b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.539+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/43918145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 0x56351966b960 secure :-1 s=READY pgs=721 gs=1 cs=0 l=1 c_cookie=4214733180526249127 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febf8009950 tx=0x7febf802f190 comp rx=0 tx=0).stop 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.539+0000 7fec0e12fe00 1 -- 10.20.193.28:0/43918145 shutdown_connections 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.539+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/43918145 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563519673a00 0x563519651420 crc :-1 s=READY pgs=130 gs=4 cs=0 l=0 c_cookie=2924810312004657442 s_cookie=16322369634195460319 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.539+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/43918145 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5635196536f0 0x563519670950 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.539+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/43918145 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 0x56351966b960 unknown :-1 s=CLOSED pgs=721 gs=1 cs=0 l=1 c_cookie=4214733180526249127 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.539+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/43918145 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56351960e140 0x56351966b420 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.539+0000 7fec0e12fe00 1 -- 10.20.193.28:0/43918145 >> 10.20.193.28:0/43918145 conn(0x56351952dd40 msgr2=0x56351960c230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.539+0000 7febbb7fe640 0 client.16890 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.540+0000 7fec0e12fe00 1 -- 10.20.193.28:0/43918145 shutdown_connections 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.540+0000 7fec0e12fe00 1 -- 10.20.193.28:0/43918145 wait complete. 2026-02-25T11:55:23.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.540+0000 7fec0e12fe00 20 client.16890 trim_cache size 0 max 16384 2026-02-25T11:55:23.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.548+0000 7fec0e12fe00 1 Processor -- start 2026-02-25T11:55:23.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec0e12fe00 1 -- start start 2026-02-25T11:55:23.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec0e12fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960e140 0x5635196561e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec0e12fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 0x5635196685f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec0e12fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 0x563519668b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec0e12fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x563519518470 con 0x56351960ea50 2026-02-25T11:55:23.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec0e12fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5635191d8700 con 0x56351960e140 2026-02-25T11:55:23.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec0e12fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56351957f820 con 0x5635196536f0 2026-02-25T11:55:23.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec096a8640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 0x5635196685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec096a8640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 0x5635196685f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40346/0 (socket says 10.20.193.28:40346) 2026-02-25T11:55:23.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec096a8640 1 -- 10.20.193.28:0/697731571 learned_addr learned my addr 10.20.193.28:0/697731571 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:23.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec09ea9640 1 --2- 10.20.193.28:0/697731571 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960e140 0x5635196561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec0a6aa640 1 --2- 10.20.193.28:0/697731571 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 0x563519668b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec096a8640 1 -- 10.20.193.28:0/697731571 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 msgr2=0x563519668b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:23.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec096a8640 1 --2- 10.20.193.28:0/697731571 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 0x563519668b30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec096a8640 1 -- 10.20.193.28:0/697731571 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960e140 msgr2=0x5635196561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:23.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec096a8640 1 --2- 10.20.193.28:0/697731571 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960e140 0x5635196561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.549+0000 7fec096a8640 1 -- 10.20.193.28:0/697731571 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635194aabd0 con 0x56351960ea50 2026-02-25T11:55:23.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.550+0000 7fec096a8640 1 --2- 10.20.193.28:0/697731571 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 0x5635196685f0 secure :-1 s=READY pgs=722 gs=3 cs=0 l=1 c_cookie=10479063458241036843 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febf80098a0 tx=0x7febf805a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916d1cb1df3bc22b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:23.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.550+0000 7febd1ce5640 1 -- 10.20.193.28:0/697731571 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febf8005eb0 con 0x56351960ea50 2026-02-25T11:55:23.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.550+0000 7febd1ce5640 1 -- 10.20.193.28:0/697731571 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7febf803e040 con 0x56351960ea50 2026-02-25T11:55:23.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.550+0000 7febd1ce5640 1 -- 10.20.193.28:0/697731571 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febf8044aa0 con 0x56351960ea50 2026-02-25T11:55:23.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.550+0000 7fec0e12fe00 1 -- 10.20.193.28:0/697731571 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 msgr2=0x5635196685f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:23.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.550+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/697731571 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 0x5635196685f0 secure :-1 s=READY pgs=722 gs=3 cs=0 l=1 c_cookie=10479063458241036843 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febf80098a0 tx=0x7febf805a000 comp rx=0 tx=0).stop 2026-02-25T11:55:23.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.550+0000 7fec0e12fe00 1 -- 10.20.193.28:0/697731571 shutdown_connections 2026-02-25T11:55:23.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.550+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/697731571 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 0x563519668b30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.550+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/697731571 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960ea50 0x5635196685f0 unknown :-1 s=CLOSED pgs=722 gs=3 cs=0 l=1 c_cookie=10479063458241036843 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.550+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/697731571 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960e140 0x5635196561e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.550+0000 7fec0e12fe00 1 -- 10.20.193.28:0/697731571 >> 10.20.193.28:0/697731571 conn(0x56351952dd40 msgr2=0x563519655ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:23.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.551+0000 7fec0e12fe00 1 -- 10.20.193.28:0/697731571 shutdown_connections 2026-02-25T11:55:23.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.551+0000 7fec0e12fe00 1 -- 10.20.193.28:0/697731571 wait complete. 2026-02-25T11:55:23.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.551+0000 7fec0e12fe00 1 -- start start 2026-02-25T11:55:23.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.551+0000 7fec0e12fe00 1 Processor -- start 2026-02-25T11:55:23.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec0e12fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 0x5635195ed900 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec0e12fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960ea50 0x5635195ede40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec0e12fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 0x5635196b9590 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec0e12fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5635195e9490 con 0x56351960e140 2026-02-25T11:55:23.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec0e12fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56351960ef30 con 0x56351960ea50 2026-02-25T11:55:23.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec0e12fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5635195ee710 con 0x5635196536f0 2026-02-25T11:55:23.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec09ea9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 0x5635195ed900 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec096a8640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960ea50 0x5635195ede40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec09ea9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 0x5635195ed900 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40356/0 (socket says 10.20.193.28:40356) 2026-02-25T11:55:23.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec09ea9640 1 -- 10.20.193.28:0/428300926 learned_addr learned my addr 10.20.193.28:0/428300926 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:23.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec096a8640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960ea50 0x5635195ede40 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41094/0 (socket says 10.20.193.28:41094) 2026-02-25T11:55:23.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec0a6aa640 1 --2- 10.20.193.28:0/428300926 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 0x5635196b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec09ea9640 1 -- 10.20.193.28:0/428300926 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 msgr2=0x5635196b9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:23.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec09ea9640 1 --2- 10.20.193.28:0/428300926 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 0x5635196b9590 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.552+0000 7fec09ea9640 1 -- 10.20.193.28:0/428300926 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960ea50 msgr2=0x5635195ede40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7fec09ea9640 1 --2- 10.20.193.28:0/428300926 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960ea50 0x5635195ede40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7fec09ea9640 1 -- 10.20.193.28:0/428300926 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56351949e680 con 0x56351960e140 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7fec096a8640 1 --2- 10.20.193.28:0/428300926 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960ea50 0x5635195ede40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7fec09ea9640 1 --2- 10.20.193.28:0/428300926 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 0x5635195ed900 secure :-1 s=READY pgs=723 gs=2 cs=0 l=1 c_cookie=10874461462864160948 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febf4005c40 tx=0x7febf400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e9d915c9b168b4 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7febbb7fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7febbb7fe640 1 -- 10.20.193.28:0/428300926 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febf40b7ee0 con 0x56351960e140 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7febbb7fe640 1 -- 10.20.193.28:0/428300926 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7febf401e260 con 0x56351960e140 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7febbb7fe640 1 -- 10.20.193.28:0/428300926 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7febf40ada30 con 0x56351960e140 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7fec0e12fe00 1 -- 10.20.193.28:0/428300926 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635194b0150 con 0x56351960e140 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7febbb7fe640 1 -- 10.20.193.28:0/428300926 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7febf40adbb0 con 0x56351960e140 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7febbb7fe640 10 client.16896 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7fec0e12fe00 10 client.16896 _ll_register_callbacks cb 0x56351949b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7fec0e12fe00 1 -- 10.20.193.28:0/428300926 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7febf4007ba0 con 0x56351960e140 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7febbb7fe640 1 -- 10.20.193.28:0/428300926 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7febf40d20b0 con 0x56351960e140 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7fec0e12fe00 10 client.16896 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:23.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.553+0000 7fec0e12fe00 1 -- 10.20.193.28:0/428300926 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5635194b03b0 con 0x56351960e140 2026-02-25T11:55:23.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.554+0000 7febbb7fe640 1 -- 10.20.193.28:0/428300926 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7febf40ae790 con 0x56351960e140 2026-02-25T11:55:23.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.554+0000 7fec0e12fe00 10 client.16896 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:23.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.554+0000 7fec0e12fe00 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:23.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.554+0000 7fec0e12fe00 20 client.16896 populate_metadata read hostname 'trial028' 2026-02-25T11:55:23.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.554+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/428300926 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563519701450 0x563519721900 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:23.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.554+0000 7fec0e12fe00 4 client.16896 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:55:23.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.554+0000 7fec0e12fe00 1 -- 10.20.193.28:0/428300926 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5635191adf40 con 0x563519701450 2026-02-25T11:55:23.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.554+0000 7fec096a8640 1 --2- 10.20.193.28:0/428300926 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563519701450 0x563519721900 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:23.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.554+0000 7fec096a8640 1 --2- 10.20.193.28:0/428300926 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563519701450 0x563519721900 crc :-1 s=READY pgs=131 gs=4 cs=0 l=0 c_cookie=12522721370193859862 s_cookie=7444749211890578169 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc9a4bceb991516 server_cookie=67510ff6d89b72f9 in_seq=0 out_seq=0 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.554+0000 7febbb7fe640 10 client.16896 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7febbb7fe640 1 -- 10.20.193.28:0/428300926 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x5635191adf40 con 0x563519701450 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7febbb7fe640 10 client.16896 handle_command_reply: tid=0 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7fec0e12fe00 1 client.16896 shutdown 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7fec0e12fe00 1 -- 10.20.193.28:0/428300926 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 msgr2=0x5635195ed900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/428300926 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 0x5635195ed900 secure :-1 s=READY pgs=723 gs=2 cs=0 l=1 c_cookie=10874461462864160948 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febf4005c40 tx=0x7febf400a660 comp rx=0 tx=0).stop 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7fec0e12fe00 1 -- 10.20.193.28:0/428300926 shutdown_connections 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/428300926 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563519701450 0x563519721900 crc :-1 s=READY pgs=131 gs=4 cs=0 l=0 c_cookie=12522721370193859862 s_cookie=7444749211890578169 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/428300926 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5635196536f0 0x5635196b9590 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/428300926 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56351960ea50 0x5635195ede40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7fec0e12fe00 1 --2- 10.20.193.28:0/428300926 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56351960e140 0x5635195ed900 unknown :-1 s=CLOSED pgs=723 gs=2 cs=0 l=1 c_cookie=10874461462864160948 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7fec0e12fe00 1 -- 10.20.193.28:0/428300926 >> 10.20.193.28:0/428300926 conn(0x56351952dd40 msgr2=0x5635196675e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7febbb7fe640 0 client.16896 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7fec0e12fe00 1 -- 10.20.193.28:0/428300926 shutdown_connections 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.555+0000 7fec0e12fe00 1 -- 10.20.193.28:0/428300926 wait complete. 2026-02-25T11:55:23.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.556+0000 7fec0e12fe00 20 client.16896 trim_cache size 0 max 16384 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.559 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.560 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.561 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:55:23.562 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:55:23.563 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.556+0000 7fec0aeab640 1 -- 10.20.193.28:0/3342930680 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7febe00b5760 msgr2=0x7febe00d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.556+0000 7fec0aeab640 1 --2- 10.20.193.28:0/3342930680 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7febe00b5760 0x7febe00d5c10 secure :-1 s=READY pgs=345 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fec040fc310 tx=0x7febf803a040 comp rx=0 tx=0).stop 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.556+0000 7fec0aeab640 1 -- 10.20.193.28:0/3342930680 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec0410d3d0 msgr2=0x7fec040f5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:23.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.556+0000 7fec0aeab640 1 --2- 10.20.193.28:0/3342930680 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec0410d3d0 0x7fec040f5970 secure :-1 s=READY pgs=719 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7febf400c9b0 tx=0x7febf400ce80 comp rx=0 tx=0).stop 2026-02-25T11:55:23.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.556+0000 7fec0aeab640 1 -- 10.20.193.28:0/3342930680 shutdown_connections 2026-02-25T11:55:23.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.556+0000 7fec0aeab640 1 --2- 10.20.193.28:0/3342930680 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7febe00b5760 0x7febe00d5c10 unknown :-1 s=CLOSED pgs=345 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.556+0000 7fec0aeab640 1 --2- 10.20.193.28:0/3342930680 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fec041135c0 0x7fec040f6070 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.556+0000 7fec0aeab640 1 --2- 10.20.193.28:0/3342930680 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fec0410ddc0 0x7fec04173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.556+0000 7fec0aeab640 1 --2- 10.20.193.28:0/3342930680 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fec0410d3d0 0x7fec040f5970 unknown :-1 s=CLOSED pgs=719 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:23.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.556+0000 7fec0aeab640 1 -- 10.20.193.28:0/3342930680 >> 10.20.193.28:0/3342930680 conn(0x7fec040eaca0 msgr2=0x7fec0410b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:23.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.556+0000 7fec0aeab640 1 -- 10.20.193.28:0/3342930680 shutdown_connections 2026-02-25T11:55:23.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:23.556+0000 7fec0aeab640 1 -- 10.20.193.28:0/3342930680 wait complete. 2026-02-25T11:55:23.655 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:55:23.656 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:55:23.840 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:55:24.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.026+0000 7f751cb19640 1 -- 10.20.193.28:0/25320246 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7518115a30 msgr2=0x7f7518176400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.026+0000 7f751cb19640 1 --2- 10.20.193.28:0/25320246 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7518115a30 0x7f7518176400 secure :-1 s=READY pgs=724 gs=1 cs=0 l=1 c_cookie=1951427736806874446 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f750800aa30 tx=0x7f7508030270 comp rx=0 tx=0).stop 2026-02-25T11:55:24.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.026+0000 7f751cb19640 1 -- 10.20.193.28:0/25320246 shutdown_connections 2026-02-25T11:55:24.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.026+0000 7f751cb19640 1 --2- 10.20.193.28:0/25320246 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7518115a30 0x7f7518176400 unknown :-1 s=CLOSED pgs=724 gs=1 cs=0 l=1 c_cookie=1951427736806874446 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.028 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.026+0000 7f751cb19640 1 --2- 10.20.193.28:0/25320246 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f7518110230 0x7f7518115280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.026+0000 7f751cb19640 1 --2- 10.20.193.28:0/25320246 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f751810f840 0x7f751810fc10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.026+0000 7f751cb19640 1 -- 10.20.193.28:0/25320246 >> 10.20.193.28:0/25320246 conn(0x7f75180ed130 msgr2=0x7f751810d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:24.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.026+0000 7f751cb19640 1 -- 10.20.193.28:0/25320246 shutdown_connections 2026-02-25T11:55:24.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.026+0000 7f751cb19640 1 -- 10.20.193.28:0/25320246 wait complete. 2026-02-25T11:55:24.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.026+0000 7f751cb19640 1 Processor -- start 2026-02-25T11:55:24.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.026+0000 7f751cb19640 1 -- start start 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f751cb19640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f751810f840 0x7f75180f5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f751cb19640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7518110230 0x7f7518176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f751cb19640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7518115a30 0x7f75180f6230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f751cb19640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f75180fb3f0 con 0x7f7518110230 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f751cb19640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f75180fb560 con 0x7f7518115a30 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f751cb19640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75180fb720 con 0x7f751810f840 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f7515d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7518110230 0x7f7518176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f7515d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7518110230 0x7f7518176300 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40378/0 (socket says 10.20.193.28:40378) 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f7515d74640 1 -- 10.20.193.28:0/173590303 learned_addr learned my addr 10.20.193.28:0/173590303 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f7516575640 1 --2- 10.20.193.28:0/173590303 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f751810f840 0x7f75180f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f7515d74640 1 -- 10.20.193.28:0/173590303 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f751810f840 msgr2=0x7f75180f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f7515d74640 1 --2- 10.20.193.28:0/173590303 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f751810f840 0x7f75180f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f7515d74640 1 -- 10.20.193.28:0/173590303 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7518115a30 msgr2=0x7f75180f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f7515d74640 1 --2- 10.20.193.28:0/173590303 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7518115a30 0x7f75180f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f7515d74640 1 -- 10.20.193.28:0/173590303 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750800a650 con 0x7f7518110230 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.027+0000 7f7515d74640 1 --2- 10.20.193.28:0/173590303 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7518110230 0x7f7518176300 secure :-1 s=READY pgs=725 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7500009480 tx=0x7f750000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.028+0000 7f74ff7fe640 1 -- 10.20.193.28:0/173590303 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f750000cd40 con 0x7f7518110230 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.028+0000 7f74ff7fe640 1 -- 10.20.193.28:0/173590303 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f7500004d10 con 0x7f7518110230 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.028+0000 7f74ff7fe640 1 -- 10.20.193.28:0/173590303 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f75000106b0 con 0x7f7518110230 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.028+0000 7f751cb19640 1 -- 10.20.193.28:0/173590303 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75180fba00 con 0x7f7518110230 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.028+0000 7f751cb19640 1 -- 10.20.193.28:0/173590303 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75180fbe80 con 0x7f7518110230 2026-02-25T11:55:24.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.028+0000 7f751cb19640 1 -- 10.20.193.28:0/173590303 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74dc005300 con 0x7f7518110230 2026-02-25T11:55:24.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.031+0000 7f74ff7fe640 1 -- 10.20.193.28:0/173590303 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f750001f060 con 0x7f7518110230 2026-02-25T11:55:24.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.032+0000 7f74ff7fe640 1 --2- 10.20.193.28:0/173590303 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74f00b5750 0x7f74f00d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.032+0000 7f74ff7fe640 1 -- 10.20.193.28:0/173590303 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f7500014070 con 0x7f7518110230 2026-02-25T11:55:24.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.032+0000 7f7516575640 1 --2- 10.20.193.28:0/173590303 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74f00b5750 0x7f74f00d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.032+0000 7f74ff7fe640 1 -- 10.20.193.28:0/173590303 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f75000e71f0 con 0x7f7518110230 2026-02-25T11:55:24.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.033+0000 7f7516575640 1 --2- 10.20.193.28:0/173590303 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74f00b5750 0x7f74f00d5c00 secure :-1 s=READY pgs=346 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f750c004740 tx=0x7f750c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:24.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.174+0000 7f751cb19640 1 -- 10.20.193.28:0/173590303 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74dc005950 con 0x7f7518110230 2026-02-25T11:55:24.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.174+0000 7f74ff7fe640 1 -- 10.20.193.28:0/173590303 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f75000a9900 con 0x7f7518110230 2026-02-25T11:55:24.177 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:55:24.177 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:55:24.178 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:55:24.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.176+0000 7f751cb19640 1 -- 10.20.193.28:0/173590303 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74f00b5750 msgr2=0x7f74f00d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.176+0000 7f751cb19640 1 --2- 10.20.193.28:0/173590303 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74f00b5750 0x7f74f00d5c00 secure :-1 s=READY pgs=346 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f750c004740 tx=0x7f750c005d00 comp rx=0 tx=0).stop 2026-02-25T11:55:24.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.176+0000 7f751cb19640 1 -- 10.20.193.28:0/173590303 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7518110230 msgr2=0x7f7518176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.176+0000 7f751cb19640 1 --2- 10.20.193.28:0/173590303 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7518110230 0x7f7518176300 secure :-1 s=READY pgs=725 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7500009480 tx=0x7f750000ee60 comp rx=0 tx=0).stop 2026-02-25T11:55:24.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.176+0000 7f751cb19640 1 -- 10.20.193.28:0/173590303 shutdown_connections 2026-02-25T11:55:24.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.176+0000 7f751cb19640 1 --2- 10.20.193.28:0/173590303 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74f00b5750 0x7f74f00d5c00 unknown :-1 s=CLOSED pgs=346 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.176+0000 7f751cb19640 1 --2- 10.20.193.28:0/173590303 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7518115a30 0x7f75180f6230 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.176+0000 7f751cb19640 1 --2- 10.20.193.28:0/173590303 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7518110230 0x7f7518176300 unknown :-1 s=CLOSED pgs=725 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.176+0000 7f751cb19640 1 --2- 10.20.193.28:0/173590303 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f751810f840 0x7f75180f5b30 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.176+0000 7f751cb19640 1 -- 10.20.193.28:0/173590303 >> 10.20.193.28:0/173590303 conn(0x7f75180ed130 msgr2=0x7f751810dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:24.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.176+0000 7f751cb19640 1 -- 10.20.193.28:0/173590303 shutdown_connections 2026-02-25T11:55:24.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.176+0000 7f751cb19640 1 -- 10.20.193.28:0/173590303 wait complete. 2026-02-25T11:55:24.276 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:55:24.458 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:55:24.502 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:24 trial028 ceph-mon[39224]: pgmap v441: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:55:24.502 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:24 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/173590303' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:55:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.629+0000 7fd28b429640 1 -- 10.20.193.28:0/577870460 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd28410b880 msgr2=0x7fd28410bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.629+0000 7fd28b429640 1 --2- 10.20.193.28:0/577870460 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd28410b880 0x7fd28410bc50 secure :-1 s=READY pgs=726 gs=1 cs=0 l=1 c_cookie=11781192124480594511 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd274009950 tx=0x7fd27402f190 comp rx=0 tx=0).stop 2026-02-25T11:55:24.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.629+0000 7fd28b429640 1 -- 10.20.193.28:0/577870460 shutdown_connections 2026-02-25T11:55:24.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.629+0000 7fd28b429640 1 --2- 10.20.193.28:0/577870460 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd284110370 0x7fd284173f80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.629+0000 7fd28b429640 1 --2- 10.20.193.28:0/577870460 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd28410c270 0x7fd28410fbc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.629+0000 7fd28b429640 1 --2- 10.20.193.28:0/577870460 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd28410b880 0x7fd28410bc50 unknown :-1 s=CLOSED pgs=726 gs=1 cs=0 l=1 c_cookie=11781192124480594511 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.629+0000 7fd28b429640 1 -- 10.20.193.28:0/577870460 >> 10.20.193.28:0/577870460 conn(0x7fd2840e9290 msgr2=0x7fd2841096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:24.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.630+0000 7fd28b429640 1 -- 10.20.193.28:0/577870460 shutdown_connections 2026-02-25T11:55:24.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.630+0000 7fd28b429640 1 -- 10.20.193.28:0/577870460 wait complete. 2026-02-25T11:55:24.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.630+0000 7fd28b429640 1 Processor -- start 2026-02-25T11:55:24.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28b429640 1 -- start start 2026-02-25T11:55:24.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28b429640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd28410b880 0x7fd2840f3f70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28b429640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd28410c270 0x7fd284173e80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28b429640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd284110370 0x7fd2840f4670 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28b429640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd2840f9880 con 0x7fd28410c270 2026-02-25T11:55:24.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28b429640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd2840f99f0 con 0x7fd28410b880 2026-02-25T11:55:24.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28b429640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2840f9bb0 con 0x7fd284110370 2026-02-25T11:55:24.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28a427640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd28410b880 0x7fd2840f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28a427640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd28410b880 0x7fd2840f3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41118/0 (socket says 10.20.193.28:41118) 2026-02-25T11:55:24.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28a427640 1 -- 10.20.193.28:0/724272040 learned_addr learned my addr 10.20.193.28:0/724272040 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:24.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd289c26640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd28410c270 0x7fd284173e80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28ac28640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd284110370 0x7fd2840f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28a427640 1 -- 10.20.193.28:0/724272040 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd284110370 msgr2=0x7fd2840f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28a427640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd284110370 0x7fd2840f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28a427640 1 -- 10.20.193.28:0/724272040 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd28410c270 msgr2=0x7fd284173e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28a427640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd28410c270 0x7fd284173e80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.631+0000 7fd28a427640 1 -- 10.20.193.28:0/724272040 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd274009570 con 0x7fd28410b880 2026-02-25T11:55:24.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.632+0000 7fd28ac28640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd284110370 0x7fd2840f4670 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:55:24.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.632+0000 7fd28a427640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd28410b880 0x7fd2840f3f70 secure :-1 s=READY pgs=161 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd2740098a0 tx=0x7fd2740028d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:24.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.632+0000 7fd26b7fe640 1 -- 10.20.193.28:0/724272040 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd27403d070 con 0x7fd28410b880 2026-02-25T11:55:24.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.632+0000 7fd26b7fe640 1 -- 10.20.193.28:0/724272040 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd274002a40 con 0x7fd28410b880 2026-02-25T11:55:24.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.632+0000 7fd28b429640 1 -- 10.20.193.28:0/724272040 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2840f9e30 con 0x7fd28410b880 2026-02-25T11:55:24.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.632+0000 7fd26b7fe640 1 -- 10.20.193.28:0/724272040 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd274041820 con 0x7fd28410b880 2026-02-25T11:55:24.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.632+0000 7fd28b429640 1 -- 10.20.193.28:0/724272040 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2840fa2a0 con 0x7fd28410b880 2026-02-25T11:55:24.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.635+0000 7fd26b7fe640 1 -- 10.20.193.28:0/724272040 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fd2740419c0 con 0x7fd28410b880 2026-02-25T11:55:24.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.637+0000 7fd26b7fe640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd2600b56d0 0x7fd2600d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.637+0000 7fd26b7fe640 1 -- 10.20.193.28:0/724272040 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd27410b1d0 con 0x7fd28410b880 2026-02-25T11:55:24.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.637+0000 7fd289c26640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd2600b56d0 0x7fd2600d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28e6ade00 1 Processor -- start 2026-02-25T11:55:24.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28e6ade00 1 -- start start 2026-02-25T11:55:24.641 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28e6ade00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384bb30 0x5643d384bf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28e6ade00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384c440 0x5643d3890ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28e6ade00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d38b14c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28e6ade00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5643d37618a0 con 0x5643d384bb30 2026-02-25T11:55:24.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28e6ade00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5643d375d5d0 con 0x5643d38910e0 2026-02-25T11:55:24.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28e6ade00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5643d3757570 con 0x5643d384c440 2026-02-25T11:55:24.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd289c26640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd2600b56d0 0x7fd2600d5b80 secure :-1 s=READY pgs=347 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd2840fa910 tx=0x7fd278005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:24.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28a427640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384bb30 0x5643d384bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28a427640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384bb30 0x5643d384bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40414/0 (socket says 10.20.193.28:40414) 2026-02-25T11:55:24.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28a427640 1 -- 10.20.193.28:0/2647184791 learned_addr learned my addr 10.20.193.28:0/2647184791 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:24.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd289c26640 1 --2- 10.20.193.28:0/2647184791 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384c440 0x5643d3890ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28ac28640 1 --2- 10.20.193.28:0/2647184791 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d38b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28a427640 1 -- 10.20.193.28:0/2647184791 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384c440 msgr2=0x5643d3890ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:24.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28a427640 1 --2- 10.20.193.28:0/2647184791 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384c440 0x5643d3890ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28a427640 1 -- 10.20.193.28:0/2647184791 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 msgr2=0x5643d38b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:24.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28a427640 1 --2- 10.20.193.28:0/2647184791 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d38b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.638+0000 7fd28a427640 1 -- 10.20.193.28:0/2647184791 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d3702d10 con 0x5643d384bb30 2026-02-25T11:55:24.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.639+0000 7fd28a427640 1 --2- 10.20.193.28:0/2647184791 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384bb30 0x5643d384bf00 secure :-1 s=READY pgs=727 gs=1 cs=0 l=1 c_cookie=13222084375789867593 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd274009480 tx=0x7fd27402f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77e47afa9329249 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:24.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.639+0000 7fd268b0a640 1 -- 10.20.193.28:0/2647184791 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd2740d1f50 con 0x5643d384bb30 2026-02-25T11:55:24.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.639+0000 7fd268b0a640 1 -- 10.20.193.28:0/2647184791 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd274106e90 con 0x5643d384bb30 2026-02-25T11:55:24.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.639+0000 7fd268b0a640 1 -- 10.20.193.28:0/2647184791 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd2740dc720 con 0x5643d384bb30 2026-02-25T11:55:24.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.639+0000 7fd28e6ade00 1 -- 10.20.193.28:0/2647184791 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384bb30 msgr2=0x5643d384bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.639+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/2647184791 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384bb30 0x5643d384bf00 secure :-1 s=READY pgs=727 gs=1 cs=0 l=1 c_cookie=13222084375789867593 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd274009480 tx=0x7fd27402f190 comp rx=0 tx=0).stop 2026-02-25T11:55:24.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.639+0000 7fd28e6ade00 1 -- 10.20.193.28:0/2647184791 shutdown_connections 2026-02-25T11:55:24.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.639+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/2647184791 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d38b14c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.639+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/2647184791 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384c440 0x5643d3890ba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.639+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/2647184791 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384bb30 0x5643d384bf00 unknown :-1 s=CLOSED pgs=727 gs=1 cs=0 l=1 c_cookie=13222084375789867593 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.639+0000 7fd28e6ade00 1 -- 10.20.193.28:0/2647184791 >> 10.20.193.28:0/2647184791 conn(0x5643d376a7a0 msgr2=0x5643d384a410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:24.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.639+0000 7fd28e6ade00 1 -- 10.20.193.28:0/2647184791 shutdown_connections 2026-02-25T11:55:24.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.639+0000 7fd28e6ade00 1 -- 10.20.193.28:0/2647184791 wait complete. 2026-02-25T11:55:24.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.641+0000 7fd28e6ade00 1 -- start start 2026-02-25T11:55:24.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.641+0000 7fd28e6ade00 1 Processor -- start 2026-02-25T11:55:24.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.641+0000 7fd28e6ade00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384bb30 0x5643d3863d50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.641+0000 7fd28e6ade00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384c440 0x5643d3864290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.641+0000 7fd28e6ade00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d3869280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.641+0000 7fd28e6ade00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5643d38697c0 con 0x5643d384bb30 2026-02-25T11:55:24.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.641+0000 7fd28e6ade00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5643d3869930 con 0x5643d38910e0 2026-02-25T11:55:24.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.641+0000 7fd28e6ade00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5643d3869af0 con 0x5643d384c440 2026-02-25T11:55:24.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.641+0000 7fd289c26640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384c440 0x5643d3864290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.641+0000 7fd289c26640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384c440 0x5643d3864290 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:37770/0 (socket says 10.20.193.28:37770) 2026-02-25T11:55:24.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.641+0000 7fd289c26640 1 -- 10.20.193.28:0/719132553 learned_addr learned my addr 10.20.193.28:0/719132553 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:24.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.641+0000 7fd28a427640 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384bb30 0x5643d3863d50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd28ac28640 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d3869280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd28a427640 1 -- 10.20.193.28:0/719132553 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384c440 msgr2=0x5643d3864290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:24.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd28a427640 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384c440 0x5643d3864290 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd28a427640 1 -- 10.20.193.28:0/719132553 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 msgr2=0x5643d3869280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:24.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd28a427640 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d3869280 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd28a427640 1 -- 10.20.193.28:0/719132553 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d36db3c0 con 0x5643d384bb30 2026-02-25T11:55:24.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd28ac28640 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d3869280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:55:24.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd28a427640 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384bb30 0x5643d3863d50 secure :-1 s=READY pgs=728 gs=3 cs=0 l=1 c_cookie=12257883217586910842 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd274005b30 tx=0x7fd2740dc430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1cbfd7627d767a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:24.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd237fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:24.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd237fff640 1 -- 10.20.193.28:0/719132553 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd2740d1f50 con 0x5643d384bb30 2026-02-25T11:55:24.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd237fff640 1 -- 10.20.193.28:0/719132553 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd274048e30 con 0x5643d384bb30 2026-02-25T11:55:24.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd237fff640 1 -- 10.20.193.28:0/719132553 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd2740cf770 con 0x5643d384bb30 2026-02-25T11:55:24.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd28e6ade00 1 -- 10.20.193.28:0/719132553 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2840fa2a0 con 0x5643d384bb30 2026-02-25T11:55:24.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd237fff640 1 -- 10.20.193.28:0/719132553 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd2740d2f40 con 0x5643d384bb30 2026-02-25T11:55:24.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd237fff640 10 client.16914 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:55:24.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd28e6ade00 10 client.16914 _ll_register_callbacks cb 0x5643d36d8a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:24.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.642+0000 7fd28e6ade00 1 -- 10.20.193.28:0/719132553 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5643d33ed6f0 con 0x5643d384bb30 2026-02-25T11:55:24.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.643+0000 7fd237fff640 1 -- 10.20.193.28:0/719132553 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd2740d7270 con 0x5643d384bb30 2026-02-25T11:55:24.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.643+0000 7fd28e6ade00 10 client.16914 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:24.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.643+0000 7fd28e6ade00 1 -- 10.20.193.28:0/719132553 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5643d36ed0e0 con 0x5643d384bb30 2026-02-25T11:55:24.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.643+0000 7fd237fff640 1 -- 10.20.193.28:0/719132553 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd2740d0840 con 0x5643d384bb30 2026-02-25T11:55:24.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.643+0000 7fd28e6ade00 10 client.16914 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:24.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.643+0000 7fd28e6ade00 10 client.16914 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:55:24.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.643+0000 7fd28e6ade00 20 client.16914 populate_metadata read hostname 'trial028' 2026-02-25T11:55:24.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.643+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5643d388de50 0x5643d388e2f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.643+0000 7fd28e6ade00 4 client.16914 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:55:24.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.643+0000 7fd28e6ade00 1 -- 10.20.193.28:0/719132553 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643d3729fc0 con 0x5643d388de50 2026-02-25T11:55:24.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.643+0000 7fd289c26640 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5643d388de50 0x5643d388e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.644+0000 7fd289c26640 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5643d388de50 0x5643d388e2f0 crc :-1 s=READY pgs=49 gs=4 cs=0 l=0 c_cookie=4692882158002679864 s_cookie=13484695467346899421 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=412077757084b038 server_cookie=bb234315238831dd in_seq=0 out_seq=0 2026-02-25T11:55:24.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.644+0000 7fd237fff640 10 client.16914 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:55:24.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.644+0000 7fd237fff640 1 -- 10.20.193.28:0/719132553 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5643d3729fc0 con 0x5643d388de50 2026-02-25T11:55:24.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.644+0000 7fd237fff640 10 client.16914 handle_command_reply: tid=0 2026-02-25T11:55:24.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.644+0000 7fd28e6ade00 1 client.16914 shutdown 2026-02-25T11:55:24.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.645+0000 7fd28e6ade00 1 -- 10.20.193.28:0/719132553 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384bb30 msgr2=0x5643d3863d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.645+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384bb30 0x5643d3863d50 secure :-1 s=READY pgs=728 gs=3 cs=0 l=1 c_cookie=12257883217586910842 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd274005b30 tx=0x7fd2740dc430 comp rx=0 tx=0).stop 2026-02-25T11:55:24.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.645+0000 7fd28e6ade00 1 -- 10.20.193.28:0/719132553 shutdown_connections 2026-02-25T11:55:24.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.645+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5643d388de50 0x5643d388e2f0 crc :-1 s=READY pgs=49 gs=4 cs=0 l=0 c_cookie=4692882158002679864 s_cookie=13484695467346899421 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.645+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d3869280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.645+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384c440 0x5643d3864290 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.645+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/719132553 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384bb30 0x5643d3863d50 unknown :-1 s=CLOSED pgs=728 gs=3 cs=0 l=1 c_cookie=12257883217586910842 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.645+0000 7fd28e6ade00 1 -- 10.20.193.28:0/719132553 >> 10.20.193.28:0/719132553 conn(0x5643d376a7a0 msgr2=0x5643d384ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:24.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.645+0000 7fd237fff640 0 client.16914 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:55:24.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.645+0000 7fd28e6ade00 1 -- 10.20.193.28:0/719132553 shutdown_connections 2026-02-25T11:55:24.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.645+0000 7fd28e6ade00 1 -- 10.20.193.28:0/719132553 wait complete. 2026-02-25T11:55:24.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.645+0000 7fd28e6ade00 20 client.16914 trim_cache size 0 max 16384 2026-02-25T11:55:24.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.653+0000 7fd28e6ade00 1 Processor -- start 2026-02-25T11:55:24.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.653+0000 7fd28e6ade00 1 -- start start 2026-02-25T11:55:24.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.653+0000 7fd28e6ade00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384bb30 0x5643d38687f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.653+0000 7fd28e6ade00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384c440 0x5643d3861140 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.653+0000 7fd28e6ade00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d3861680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.653+0000 7fd28e6ade00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5643d37551a0 con 0x5643d384c440 2026-02-25T11:55:24.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.653+0000 7fd28e6ade00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5643d3415700 con 0x5643d38910e0 2026-02-25T11:55:24.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.653+0000 7fd28e6ade00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5643d375d5d0 con 0x5643d384bb30 2026-02-25T11:55:24.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.653+0000 7fd28ac28640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d3861680 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.653+0000 7fd28ac28640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d3861680 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41146/0 (socket says 10.20.193.28:41146) 2026-02-25T11:55:24.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.653+0000 7fd28ac28640 1 -- 10.20.193.28:0/292549648 learned_addr learned my addr 10.20.193.28:0/292549648 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:24.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.653+0000 7fd289c26640 1 --2- 10.20.193.28:0/292549648 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384c440 0x5643d3861140 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.653+0000 7fd28a427640 1 --2- 10.20.193.28:0/292549648 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384bb30 0x5643d38687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28ac28640 1 -- 10.20.193.28:0/292549648 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384bb30 msgr2=0x5643d38687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:24.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28ac28640 1 --2- 10.20.193.28:0/292549648 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384bb30 0x5643d38687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28ac28640 1 -- 10.20.193.28:0/292549648 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384c440 msgr2=0x5643d3861140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:24.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28ac28640 1 --2- 10.20.193.28:0/292549648 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384c440 0x5643d3861140 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28ac28640 1 -- 10.20.193.28:0/292549648 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d36e7b60 con 0x5643d38910e0 2026-02-25T11:55:24.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28ac28640 1 --2- 10.20.193.28:0/292549648 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d3861680 secure :-1 s=READY pgs=162 gs=3 cs=0 l=1 c_cookie=12603141905620425981 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd280005790 tx=0x7fd280013820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aee75ac9f2585cfd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:24.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd2524e6640 1 -- 10.20.193.28:0/292549648 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd280018070 con 0x5643d38910e0 2026-02-25T11:55:24.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd2524e6640 1 -- 10.20.193.28:0/292549648 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd28001d930 con 0x5643d38910e0 2026-02-25T11:55:24.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd2524e6640 1 -- 10.20.193.28:0/292549648 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd280011980 con 0x5643d38910e0 2026-02-25T11:55:24.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28e6ade00 1 -- 10.20.193.28:0/292549648 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 msgr2=0x5643d3861680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/292549648 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d3861680 secure :-1 s=READY pgs=162 gs=3 cs=0 l=1 c_cookie=12603141905620425981 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd280005790 tx=0x7fd280013820 comp rx=0 tx=0).stop 2026-02-25T11:55:24.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28e6ade00 1 -- 10.20.193.28:0/292549648 shutdown_connections 2026-02-25T11:55:24.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/292549648 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d3861680 unknown :-1 s=CLOSED pgs=162 gs=3 cs=0 l=1 c_cookie=12603141905620425981 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/292549648 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384c440 0x5643d3861140 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/292549648 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384bb30 0x5643d38687f0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28e6ade00 1 -- 10.20.193.28:0/292549648 >> 10.20.193.28:0/292549648 conn(0x5643d376a7a0 msgr2=0x5643d38680e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:24.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.654+0000 7fd28e6ade00 1 -- 10.20.193.28:0/292549648 shutdown_connections 2026-02-25T11:55:24.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.655+0000 7fd28e6ade00 1 -- 10.20.193.28:0/292549648 wait complete. 2026-02-25T11:55:24.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.655+0000 7fd28e6ade00 1 -- start start 2026-02-25T11:55:24.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.655+0000 7fd28e6ade00 1 Processor -- start 2026-02-25T11:55:24.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd28e6ade00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384bb30 0x5643d384b190 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd28e6ade00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384c440 0x5643d386a340 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd28e6ade00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d386a880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd28e6ade00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5643d3869390 con 0x5643d384c440 2026-02-25T11:55:24.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd28e6ade00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5643d386adf0 con 0x5643d38910e0 2026-02-25T11:55:24.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd28e6ade00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5643d386afb0 con 0x5643d384bb30 2026-02-25T11:55:24.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd289c26640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384c440 0x5643d386a340 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd28a427640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384bb30 0x5643d384b190 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd28a427640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384bb30 0x5643d384b190 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:37774/0 (socket says 10.20.193.28:37774) 2026-02-25T11:55:24.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd28a427640 1 -- 10.20.193.28:0/1028709815 learned_addr learned my addr 10.20.193.28:0/1028709815 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:24.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd28ac28640 1 --2- 10.20.193.28:0/1028709815 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d386a880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd289c26640 1 -- 10.20.193.28:0/1028709815 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384bb30 msgr2=0x5643d384b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:24.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd289c26640 1 --2- 10.20.193.28:0/1028709815 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384bb30 0x5643d384b190 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd289c26640 1 -- 10.20.193.28:0/1028709815 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 msgr2=0x5643d386a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:24.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd289c26640 1 --2- 10.20.193.28:0/1028709815 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d386a880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd289c26640 1 -- 10.20.193.28:0/1028709815 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643d36db6f0 con 0x5643d384c440 2026-02-25T11:55:24.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd28a427640 1 --2- 10.20.193.28:0/1028709815 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384bb30 0x5643d384b190 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:55:24.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd289c26640 1 --2- 10.20.193.28:0/1028709815 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384c440 0x5643d386a340 secure :-1 s=READY pgs=729 gs=1 cs=0 l=1 c_cookie=981970637817394924 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x5643d388e220 tx=0x7fd2780047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0a917a6f836ec server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:24.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.656+0000 7fd237fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:24.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd237fff640 1 -- 10.20.193.28:0/1028709815 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd278021040 con 0x5643d384c440 2026-02-25T11:55:24.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd237fff640 1 -- 10.20.193.28:0/1028709815 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5643d3729fc0 con 0x5643d384c440 2026-02-25T11:55:24.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd28e6ade00 1 -- 10.20.193.28:0/1028709815 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd280003860 con 0x5643d384c440 2026-02-25T11:55:24.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd237fff640 1 -- 10.20.193.28:0/1028709815 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd278028970 con 0x5643d384c440 2026-02-25T11:55:24.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd28e6ade00 10 client.16920 _ll_register_callbacks cb 0x5643d36d8a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:24.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd28e6ade00 1 -- 10.20.193.28:0/1028709815 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd274106e90 con 0x5643d384c440 2026-02-25T11:55:24.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd237fff640 1 -- 10.20.193.28:0/1028709815 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd278028b00 con 0x5643d384c440 2026-02-25T11:55:24.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd237fff640 1 -- 10.20.193.28:0/1028709815 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd278028ce0 con 0x5643d384c440 2026-02-25T11:55:24.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd28e6ade00 10 client.16920 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:24.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd28e6ade00 1 -- 10.20.193.28:0/1028709815 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5643d36ed340 con 0x5643d384c440 2026-02-25T11:55:24.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd237fff640 1 -- 10.20.193.28:0/1028709815 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd27802c650 con 0x5643d384c440 2026-02-25T11:55:24.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd28e6ade00 10 client.16920 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:24.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd28e6ade00 10 client.16920 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:55:24.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd28e6ade00 20 client.16920 populate_metadata read hostname 'trial028' 2026-02-25T11:55:24.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/1028709815 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5643d393e570 0x5643d395ea20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:24.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd28e6ade00 4 client.16920 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:55:24.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.657+0000 7fd28e6ade00 1 -- 10.20.193.28:0/1028709815 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643d33eaf40 con 0x5643d393e570 2026-02-25T11:55:24.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.658+0000 7fd28a427640 1 --2- 10.20.193.28:0/1028709815 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5643d393e570 0x5643d395ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:24.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.658+0000 7fd28a427640 1 --2- 10.20.193.28:0/1028709815 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5643d393e570 0x5643d395ea20 crc :-1 s=READY pgs=50 gs=4 cs=0 l=0 c_cookie=11144303761829501683 s_cookie=16616732353182222407 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9aa88539f0e59af3 server_cookie=e69a7e09372ac847 in_seq=0 out_seq=0 2026-02-25T11:55:24.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.658+0000 7fd237fff640 10 client.16920 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:55:24.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.658+0000 7fd237fff640 1 -- 10.20.193.28:0/1028709815 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643d33eaf40 con 0x5643d393e570 2026-02-25T11:55:24.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.658+0000 7fd237fff640 10 client.16920 handle_command_reply: tid=0 2026-02-25T11:55:24.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.658+0000 7fd28e6ade00 1 client.16920 shutdown 2026-02-25T11:55:24.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.658+0000 7fd28e6ade00 1 -- 10.20.193.28:0/1028709815 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384c440 msgr2=0x5643d386a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.658+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/1028709815 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384c440 0x5643d386a340 secure :-1 s=READY pgs=729 gs=1 cs=0 l=1 c_cookie=981970637817394924 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x5643d388e220 tx=0x7fd2780047e0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28e6ade00 1 -- 10.20.193.28:0/1028709815 shutdown_connections 2026-02-25T11:55:24.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/1028709815 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5643d393e570 0x5643d395ea20 crc :-1 s=READY pgs=50 gs=4 cs=0 l=0 c_cookie=11144303761829501683 s_cookie=16616732353182222407 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/1028709815 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5643d38910e0 0x5643d386a880 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/1028709815 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5643d384c440 0x5643d386a340 unknown :-1 s=CLOSED pgs=729 gs=1 cs=0 l=1 c_cookie=981970637817394924 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28e6ade00 1 --2- 10.20.193.28:0/1028709815 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5643d384bb30 0x5643d384b190 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd237fff640 0 client.16920 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:55:24.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28e6ade00 1 -- 10.20.193.28:0/1028709815 >> 10.20.193.28:0/1028709815 conn(0x5643d376a7a0 msgr2=0x5643d38688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:24.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28e6ade00 1 -- 10.20.193.28:0/1028709815 shutdown_connections 2026-02-25T11:55:24.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28e6ade00 1 -- 10.20.193.28:0/1028709815 wait complete. 2026-02-25T11:55:24.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28e6ade00 20 client.16920 trim_cache size 0 max 16384 2026-02-25T11:55:24.668 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:55:24.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28b429640 1 -- 10.20.193.28:0/724272040 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd2600b56d0 msgr2=0x7fd2600d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28b429640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd2600b56d0 0x7fd2600d5b80 secure :-1 s=READY pgs=347 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd2840fa910 tx=0x7fd278005f90 comp rx=0 tx=0).stop 2026-02-25T11:55:24.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28b429640 1 -- 10.20.193.28:0/724272040 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd28410b880 msgr2=0x7fd2840f3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:24.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28b429640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd28410b880 0x7fd2840f3f70 secure :-1 s=READY pgs=161 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd2740098a0 tx=0x7fd2740028d0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28b429640 1 -- 10.20.193.28:0/724272040 shutdown_connections 2026-02-25T11:55:24.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28b429640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd2600b56d0 0x7fd2600d5b80 unknown :-1 s=CLOSED pgs=347 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28b429640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd284110370 0x7fd2840f4670 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28b429640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd28410c270 0x7fd284173e80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28b429640 1 --2- 10.20.193.28:0/724272040 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd28410b880 0x7fd2840f3f70 unknown :-1 s=CLOSED pgs=161 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:24.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.659+0000 7fd28b429640 1 -- 10.20.193.28:0/724272040 >> 10.20.193.28:0/724272040 conn(0x7fd2840e9290 msgr2=0x7fd2841115e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:24.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.660+0000 7fd28b429640 1 -- 10.20.193.28:0/724272040 shutdown_connections 2026-02-25T11:55:24.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:24.660+0000 7fd28b429640 1 -- 10.20.193.28:0/724272040 wait complete. 2026-02-25T11:55:24.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:24 trial167 ceph-mon[43392]: pgmap v441: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:55:24.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:24 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/173590303' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:55:24.758 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:55:24.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:24 trial119 ceph-mon[42710]: pgmap v441: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:55:24.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:24 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/173590303' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:55:24.942 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:55:25.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.121+0000 7f9a9c72e640 1 -- 10.20.193.28:0/4123054604 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a94110130 msgr2=0x7f9a94115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.121+0000 7f9a9c72e640 1 --2- 10.20.193.28:0/4123054604 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a94110130 0x7f9a94115180 secure :-1 s=READY pgs=730 gs=1 cs=0 l=1 c_cookie=5213550774713432065 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a88009950 tx=0x7f9a8802f190 comp rx=0 tx=0).stop 2026-02-25T11:55:25.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.121+0000 7f9a9c72e640 1 -- 10.20.193.28:0/4123054604 shutdown_connections 2026-02-25T11:55:25.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.121+0000 7f9a9c72e640 1 --2- 10.20.193.28:0/4123054604 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9a94115930 0x7f9a94176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.121+0000 7f9a9c72e640 1 --2- 10.20.193.28:0/4123054604 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a94110130 0x7f9a94115180 unknown :-1 s=CLOSED pgs=730 gs=1 cs=0 l=1 c_cookie=5213550774713432065 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.121+0000 7f9a9c72e640 1 --2- 10.20.193.28:0/4123054604 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9a9410f740 0x7f9a9410fb10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.121+0000 7f9a9c72e640 1 -- 10.20.193.28:0/4123054604 >> 10.20.193.28:0/4123054604 conn(0x7f9a940ecff0 msgr2=0x7f9a9410d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:25.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.121+0000 7f9a9c72e640 1 -- 10.20.193.28:0/4123054604 shutdown_connections 2026-02-25T11:55:25.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.121+0000 7f9a9c72e640 1 -- 10.20.193.28:0/4123054604 wait complete. 2026-02-25T11:55:25.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a9c72e640 1 Processor -- start 2026-02-25T11:55:25.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a9c72e640 1 -- start start 2026-02-25T11:55:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a9c72e640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9a9410f740 0x7f9a940f5af0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a9c72e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a94110130 0x7f9a94176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a9c72e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9a94115930 0x7f9a940f6180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a9c72e640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9a940fb400 con 0x7f9a94110130 2026-02-25T11:55:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a9c72e640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9a940fb570 con 0x7f9a9410f740 2026-02-25T11:55:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a9c72e640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a940fb6e0 con 0x7f9a94115930 2026-02-25T11:55:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a99ca2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a94110130 0x7f9a94176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a99ca2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a94110130 0x7f9a94176200 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40448/0 (socket says 10.20.193.28:40448) 2026-02-25T11:55:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a99ca2640 1 -- 10.20.193.28:0/2992411048 learned_addr learned my addr 10.20.193.28:0/2992411048 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a9a4a3640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9a9410f740 0x7f9a940f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a9aca4640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9a94115930 0x7f9a940f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a99ca2640 1 -- 10.20.193.28:0/2992411048 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9a94115930 msgr2=0x7f9a940f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a99ca2640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9a94115930 0x7f9a940f6180 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a99ca2640 1 -- 10.20.193.28:0/2992411048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9a9410f740 msgr2=0x7f9a940f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a99ca2640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9a9410f740 0x7f9a940f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.122+0000 7f9a99ca2640 1 -- 10.20.193.28:0/2992411048 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a88009570 con 0x7f9a94110130 2026-02-25T11:55:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.123+0000 7f9a99ca2640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a94110130 0x7f9a94176200 secure :-1 s=READY pgs=731 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a880098a0 tx=0x7f9a880028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.123+0000 7f9a7b7fe640 1 -- 10.20.193.28:0/2992411048 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a8803d070 con 0x7f9a94110130 2026-02-25T11:55:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.123+0000 7f9a7b7fe640 1 -- 10.20.193.28:0/2992411048 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9a88002a70 con 0x7f9a94110130 2026-02-25T11:55:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.123+0000 7f9a7b7fe640 1 -- 10.20.193.28:0/2992411048 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a88041890 con 0x7f9a94110130 2026-02-25T11:55:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.123+0000 7f9a9c72e640 1 -- 10.20.193.28:0/2992411048 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a940fb960 con 0x7f9a94110130 2026-02-25T11:55:25.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.123+0000 7f9a9c72e640 1 -- 10.20.193.28:0/2992411048 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a940fbd80 con 0x7f9a94110130 2026-02-25T11:55:25.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.126+0000 7f9a7b7fe640 1 -- 10.20.193.28:0/2992411048 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f9a88041a20 con 0x7f9a94110130 2026-02-25T11:55:25.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.127+0000 7f9a7b7fe640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9a700b5760 0x7f9a700d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.128+0000 7f9a7b7fe640 1 -- 10.20.193.28:0/2992411048 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9a8810b000 con 0x7f9a94110130 2026-02-25T11:55:25.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.128+0000 7f9a9a4a3640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9a700b5760 0x7f9a700d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.128+0000 7f9a9a4a3640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9a700b5760 0x7f9a700d5c10 secure :-1 s=READY pgs=348 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a84004740 tx=0x7f9a8400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:25.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.128+0000 7f9a9e729e00 1 Processor -- start 2026-02-25T11:55:25.132 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a9e729e00 1 -- start start 2026-02-25T11:55:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a9e729e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37abb30 0x5607f37abf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a9e729e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 0x5607f37f0ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a9e729e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f38114c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a9e729e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5607f36c18a0 con 0x5607f37ac440 2026-02-25T11:55:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a9e729e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5607f36bd5d0 con 0x5607f37abb30 2026-02-25T11:55:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a9e729e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5607f36b7570 con 0x5607f37f10e0 2026-02-25T11:55:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a9a4a3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37abb30 0x5607f37abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a99ca2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 0x5607f37f0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a9a4a3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37abb30 0x5607f37abf00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41188/0 (socket says 10.20.193.28:41188) 2026-02-25T11:55:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a9a4a3640 1 -- 10.20.193.28:0/542663195 learned_addr learned my addr 10.20.193.28:0/542663195 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a9aca4640 1 --2- 10.20.193.28:0/542663195 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f38114c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a99ca2640 1 -- 10.20.193.28:0/542663195 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 msgr2=0x5607f38114c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a99ca2640 1 --2- 10.20.193.28:0/542663195 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f38114c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a99ca2640 1 -- 10.20.193.28:0/542663195 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37abb30 msgr2=0x5607f37abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a99ca2640 1 --2- 10.20.193.28:0/542663195 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37abb30 0x5607f37abf00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a99ca2640 1 -- 10.20.193.28:0/542663195 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f3662d10 con 0x5607f37ac440 2026-02-25T11:55:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.129+0000 7f9a9aca4640 1 --2- 10.20.193.28:0/542663195 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f38114c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:55:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.130+0000 7f9a99ca2640 1 --2- 10.20.193.28:0/542663195 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 0x5607f37f0ba0 secure :-1 s=READY pgs=732 gs=2 cs=0 l=1 c_cookie=14093126240628334450 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a88108be0 tx=0x7f9a88038f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c394d7a5b3635f72 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.130+0000 7f9a78cf3640 1 -- 10.20.193.28:0/542663195 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a88041d00 con 0x5607f37ac440 2026-02-25T11:55:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.130+0000 7f9a78cf3640 1 -- 10.20.193.28:0/542663195 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9a880d70e0 con 0x5607f37ac440 2026-02-25T11:55:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.130+0000 7f9a78cf3640 1 -- 10.20.193.28:0/542663195 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a880cf6d0 con 0x5607f37ac440 2026-02-25T11:55:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.130+0000 7f9a9e729e00 1 -- 10.20.193.28:0/542663195 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 msgr2=0x5607f37f0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.130+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/542663195 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 0x5607f37f0ba0 secure :-1 s=READY pgs=732 gs=2 cs=0 l=1 c_cookie=14093126240628334450 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a88108be0 tx=0x7f9a88038f30 comp rx=0 tx=0).stop 2026-02-25T11:55:25.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.130+0000 7f9a9e729e00 1 -- 10.20.193.28:0/542663195 shutdown_connections 2026-02-25T11:55:25.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.130+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/542663195 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f38114c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.130+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/542663195 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 0x5607f37f0ba0 unknown :-1 s=CLOSED pgs=732 gs=2 cs=0 l=1 c_cookie=14093126240628334450 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.130+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/542663195 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37abb30 0x5607f37abf00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.130+0000 7f9a9e729e00 1 -- 10.20.193.28:0/542663195 >> 10.20.193.28:0/542663195 conn(0x5607f36ca7a0 msgr2=0x5607f37aa410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:25.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.130+0000 7f9a9e729e00 1 -- 10.20.193.28:0/542663195 shutdown_connections 2026-02-25T11:55:25.135 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.130+0000 7f9a9e729e00 1 -- 10.20.193.28:0/542663195 wait complete. 2026-02-25T11:55:25.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.131+0000 7f9a9e729e00 1 -- start start 2026-02-25T11:55:25.138 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9e729e00 1 Processor -- start 2026-02-25T11:55:25.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9e729e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37abb30 0x5607f37c3b00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9e729e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 0x5607f37c4040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9e729e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f37c9030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9e729e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5607f37c9570 con 0x5607f37ac440 2026-02-25T11:55:25.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9e729e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5607f37c96e0 con 0x5607f37abb30 2026-02-25T11:55:25.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9e729e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5607f37c98a0 con 0x5607f37f10e0 2026-02-25T11:55:25.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a99ca2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 0x5607f37c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9aca4640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f37c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9aca4640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f37c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:37810/0 (socket says 10.20.193.28:37810) 2026-02-25T11:55:25.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9aca4640 1 -- 10.20.193.28:0/1530391436 learned_addr learned my addr 10.20.193.28:0/1530391436 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:25.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9a4a3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37abb30 0x5607f37c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9aca4640 1 -- 10.20.193.28:0/1530391436 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37abb30 msgr2=0x5607f37c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9aca4640 1 --2- 10.20.193.28:0/1530391436 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37abb30 0x5607f37c3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9aca4640 1 -- 10.20.193.28:0/1530391436 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 msgr2=0x5607f37c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9aca4640 1 --2- 10.20.193.28:0/1530391436 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 0x5607f37c4040 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.132+0000 7f9a9aca4640 1 -- 10.20.193.28:0/1530391436 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f363b3c0 con 0x5607f37f10e0 2026-02-25T11:55:25.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.133+0000 7f9a9aca4640 1 --2- 10.20.193.28:0/1530391436 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f37c9030 secure :-1 s=READY pgs=194 gs=2 cs=0 l=1 c_cookie=11831153664540185972 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a900056d0 tx=0x7f9a90003ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a430b392e1fa9574 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:25.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.133+0000 7f9a47fff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:55:25.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.133+0000 7f9a47fff640 1 -- 10.20.193.28:0/1530391436 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a90008e40 con 0x5607f37f10e0 2026-02-25T11:55:25.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.133+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1530391436 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a940fbd80 con 0x5607f37f10e0 2026-02-25T11:55:25.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.133+0000 7f9a9e729e00 10 client.26635 _ll_register_callbacks cb 0x5607f3638a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:25.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.133+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1530391436 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5607f334d6f0 con 0x5607f37f10e0 2026-02-25T11:55:25.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.133+0000 7f9a47fff640 1 -- 10.20.193.28:0/1530391436 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9a9001f070 con 0x5607f37f10e0 2026-02-25T11:55:25.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.133+0000 7f9a47fff640 1 -- 10.20.193.28:0/1530391436 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a900119b0 con 0x5607f37f10e0 2026-02-25T11:55:25.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.133+0000 7f9a47fff640 1 -- 10.20.193.28:0/1530391436 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9a90011b40 con 0x5607f37f10e0 2026-02-25T11:55:25.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.133+0000 7f9a47fff640 1 -- 10.20.193.28:0/1530391436 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9a90011d40 con 0x5607f37f10e0 2026-02-25T11:55:25.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.133+0000 7f9a9e729e00 10 client.26635 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:25.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.133+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1530391436 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5607f364d0e0 con 0x5607f37f10e0 2026-02-25T11:55:25.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.134+0000 7f9a47fff640 1 -- 10.20.193.28:0/1530391436 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9a9002f430 con 0x5607f37f10e0 2026-02-25T11:55:25.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.134+0000 7f9a9e729e00 10 client.26635 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:25.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.134+0000 7f9a9e729e00 10 client.26635 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:55:25.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.134+0000 7f9a9e729e00 20 client.26635 populate_metadata read hostname 'trial028' 2026-02-25T11:55:25.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.134+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1530391436 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5607f37ee400 0x5607f37ee8a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.134+0000 7f9a9e729e00 4 client.26635 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:55:25.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.134+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1530391436 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607f3689fc0 con 0x5607f37ee400 2026-02-25T11:55:25.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.134+0000 7f9a9a4a3640 1 --2- 10.20.193.28:0/1530391436 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5607f37ee400 0x5607f37ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.134+0000 7f9a9a4a3640 1 --2- 10.20.193.28:0/1530391436 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5607f37ee400 0x5607f37ee8a0 crc :-1 s=READY pgs=50 gs=4 cs=0 l=0 c_cookie=15291002283297670601 s_cookie=16630528216070702204 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4348d8fbba505c9 server_cookie=e6cb814cda01447c in_seq=0 out_seq=0 2026-02-25T11:55:25.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a47fff640 10 client.26635 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:55:25.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a47fff640 1 -- 10.20.193.28:0/1530391436 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5607f3689fc0 con 0x5607f37ee400 2026-02-25T11:55:25.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a47fff640 10 client.26635 handle_command_reply: tid=0 2026-02-25T11:55:25.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a9e729e00 1 client.26635 shutdown 2026-02-25T11:55:25.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1530391436 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 msgr2=0x5607f37c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1530391436 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f37c9030 secure :-1 s=READY pgs=194 gs=2 cs=0 l=1 c_cookie=11831153664540185972 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a900056d0 tx=0x7f9a90003ea0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1530391436 shutdown_connections 2026-02-25T11:55:25.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1530391436 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5607f37ee400 0x5607f37ee8a0 crc :-1 s=READY pgs=50 gs=4 cs=0 l=0 c_cookie=15291002283297670601 s_cookie=16630528216070702204 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1530391436 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f37c9030 unknown :-1 s=CLOSED pgs=194 gs=2 cs=0 l=1 c_cookie=11831153664540185972 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1530391436 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 0x5607f37c4040 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1530391436 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37abb30 0x5607f37c3b00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a47fff640 0 client.26635 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:55:25.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1530391436 >> 10.20.193.28:0/1530391436 conn(0x5607f36ca7a0 msgr2=0x5607f37aab20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:25.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1530391436 shutdown_connections 2026-02-25T11:55:25.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.135+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1530391436 wait complete. 2026-02-25T11:55:25.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.136+0000 7f9a9e729e00 20 client.26635 trim_cache size 0 max 16384 2026-02-25T11:55:25.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.143+0000 7f9a9e729e00 1 Processor -- start 2026-02-25T11:55:25.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.143+0000 7f9a9e729e00 1 -- start start 2026-02-25T11:55:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9e729e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37abb30 0x5607f37e9c20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9e729e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37ac440 0x5607f37aaf30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9e729e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f37f0190 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9e729e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5607f36b51a0 con 0x5607f37abb30 2026-02-25T11:55:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9e729e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5607f3375700 con 0x5607f37ac440 2026-02-25T11:55:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9e729e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5607f36bd5d0 con 0x5607f37f10e0 2026-02-25T11:55:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9a4a3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37abb30 0x5607f37e9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9a4a3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37abb30 0x5607f37e9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40470/0 (socket says 10.20.193.28:40470) 2026-02-25T11:55:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9a4a3640 1 -- 10.20.193.28:0/1180402270 learned_addr learned my addr 10.20.193.28:0/1180402270 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9aca4640 1 --2- 10.20.193.28:0/1180402270 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f37f0190 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a99ca2640 1 --2- 10.20.193.28:0/1180402270 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37ac440 0x5607f37aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9a4a3640 1 -- 10.20.193.28:0/1180402270 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 msgr2=0x5607f37f0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9a4a3640 1 --2- 10.20.193.28:0/1180402270 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f37f0190 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9a4a3640 1 -- 10.20.193.28:0/1180402270 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37ac440 msgr2=0x5607f37aaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9a4a3640 1 --2- 10.20.193.28:0/1180402270 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37ac440 0x5607f37aaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9a4a3640 1 -- 10.20.193.28:0/1180402270 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f3647b60 con 0x5607f37abb30 2026-02-25T11:55:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.144+0000 7f9a9a4a3640 1 --2- 10.20.193.28:0/1180402270 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37abb30 0x5607f37e9c20 secure :-1 s=READY pgs=733 gs=1 cs=0 l=1 c_cookie=16344037911620057540 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a84011650 tx=0x7f9a84013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d1afe2dda179c4 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.145+0000 7f9a624e6640 1 -- 10.20.193.28:0/1180402270 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a84024040 con 0x5607f37abb30 2026-02-25T11:55:25.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.145+0000 7f9a624e6640 1 -- 10.20.193.28:0/1180402270 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9a84015070 con 0x5607f37abb30 2026-02-25T11:55:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.145+0000 7f9a624e6640 1 -- 10.20.193.28:0/1180402270 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a8402a320 con 0x5607f37abb30 2026-02-25T11:55:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.145+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1180402270 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37abb30 msgr2=0x5607f37e9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.145+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1180402270 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37abb30 0x5607f37e9c20 secure :-1 s=READY pgs=733 gs=1 cs=0 l=1 c_cookie=16344037911620057540 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a84011650 tx=0x7f9a84013000 comp rx=0 tx=0).stop 2026-02-25T11:55:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.145+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1180402270 shutdown_connections 2026-02-25T11:55:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.145+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1180402270 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37f10e0 0x5607f37f0190 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.145+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1180402270 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37ac440 0x5607f37aaf30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.145+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1180402270 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37abb30 0x5607f37e9c20 unknown :-1 s=CLOSED pgs=733 gs=1 cs=0 l=1 c_cookie=16344037911620057540 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.145+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1180402270 >> 10.20.193.28:0/1180402270 conn(0x5607f36ca7a0 msgr2=0x5607f37e8500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.145+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1180402270 shutdown_connections 2026-02-25T11:55:25.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.145+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1180402270 wait complete. 2026-02-25T11:55:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.146+0000 7f9a9e729e00 1 -- start start 2026-02-25T11:55:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.146+0000 7f9a9e729e00 1 Processor -- start 2026-02-25T11:55:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.146+0000 7f9a9e729e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37abb30 0x5607f37cfd50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.146+0000 7f9a9e729e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 0x5607f37cb420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.146+0000 7f9a9e729e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37f10e0 0x5607f37cba50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.146+0000 7f9a9e729e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5607f37bce50 con 0x5607f37ac440 2026-02-25T11:55:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.146+0000 7f9a9e729e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5607f37cc150 con 0x5607f37f10e0 2026-02-25T11:55:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.146+0000 7f9a9e729e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5607f3856600 con 0x5607f37abb30 2026-02-25T11:55:25.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.146+0000 7f9a9a4a3640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37abb30 0x5607f37cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.146+0000 7f9a9a4a3640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37abb30 0x5607f37cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:37828/0 (socket says 10.20.193.28:37828) 2026-02-25T11:55:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a9a4a3640 1 -- 10.20.193.28:0/1770926185 learned_addr learned my addr 10.20.193.28:0/1770926185 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a9aca4640 1 --2- 10.20.193.28:0/1770926185 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37f10e0 0x5607f37cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a9a4a3640 1 -- 10.20.193.28:0/1770926185 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37f10e0 msgr2=0x5607f37cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a99ca2640 1 --2- 10.20.193.28:0/1770926185 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 0x5607f37cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a9a4a3640 1 --2- 10.20.193.28:0/1770926185 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37f10e0 0x5607f37cba50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a9a4a3640 1 -- 10.20.193.28:0/1770926185 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 msgr2=0x5607f37cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a9a4a3640 1 --2- 10.20.193.28:0/1770926185 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 0x5607f37cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a9a4a3640 1 -- 10.20.193.28:0/1770926185 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f363b6f0 con 0x5607f37abb30 2026-02-25T11:55:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a9a4a3640 1 --2- 10.20.193.28:0/1770926185 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37abb30 0x5607f37cfd50 secure :-1 s=READY pgs=195 gs=1 cs=0 l=1 c_cookie=8410192183705570730 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a8400b510 tx=0x7f9a84011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74b7013b62898daa server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a47fff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:55:25.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a47fff640 1 -- 10.20.193.28:0/1770926185 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a84024040 con 0x5607f37abb30 2026-02-25T11:55:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a47fff640 1 -- 10.20.193.28:0/1770926185 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9a8401edb0 con 0x5607f37abb30 2026-02-25T11:55:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1770926185 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a84011700 con 0x5607f37abb30 2026-02-25T11:55:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.147+0000 7f9a47fff640 1 -- 10.20.193.28:0/1770926185 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a840336a0 con 0x5607f37abb30 2026-02-25T11:55:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a47fff640 1 -- 10.20.193.28:0/1770926185 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9a84017070 con 0x5607f37abb30 2026-02-25T11:55:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a9e729e00 10 client.26641 _ll_register_callbacks cb 0x5607f3638a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1770926185 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a880d70e0 con 0x5607f37abb30 2026-02-25T11:55:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a47fff640 1 -- 10.20.193.28:0/1770926185 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9a84033830 con 0x5607f37abb30 2026-02-25T11:55:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a9e729e00 10 client.26641 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1770926185 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5607f364d340 con 0x5607f37abb30 2026-02-25T11:55:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a47fff640 1 -- 10.20.193.28:0/1770926185 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9a8403a020 con 0x5607f37abb30 2026-02-25T11:55:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a9e729e00 10 client.26641 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:25.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a9e729e00 10 client.26641 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:55:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a9e729e00 20 client.26641 populate_metadata read hostname 'trial028' 2026-02-25T11:55:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1770926185 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5607f389e4a0 0x5607f38be950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a9e729e00 4 client.26641 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:55:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1770926185 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607f334af40 con 0x5607f389e4a0 2026-02-25T11:55:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.148+0000 7f9a99ca2640 1 --2- 10.20.193.28:0/1770926185 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5607f389e4a0 0x5607f38be950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a47fff640 10 client.26641 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:55:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a99ca2640 1 --2- 10.20.193.28:0/1770926185 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5607f389e4a0 0x5607f38be950 crc :-1 s=READY pgs=51 gs=4 cs=0 l=0 c_cookie=450312473155320038 s_cookie=8102269427445091393 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63fd4cef859b8e6 server_cookie=70710b1dd95f4c41 in_seq=0 out_seq=0 2026-02-25T11:55:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a47fff640 1 -- 10.20.193.28:0/1770926185 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607f334af40 con 0x5607f389e4a0 2026-02-25T11:55:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a47fff640 10 client.26641 handle_command_reply: tid=0 2026-02-25T11:55:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a9e729e00 1 client.26641 shutdown 2026-02-25T11:55:25.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1770926185 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37abb30 msgr2=0x5607f37cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1770926185 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37abb30 0x5607f37cfd50 secure :-1 s=READY pgs=195 gs=1 cs=0 l=1 c_cookie=8410192183705570730 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a8400b510 tx=0x7f9a84011440 comp rx=0 tx=0).stop 2026-02-25T11:55:25.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1770926185 shutdown_connections 2026-02-25T11:55:25.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1770926185 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5607f389e4a0 0x5607f38be950 crc :-1 s=READY pgs=51 gs=4 cs=0 l=0 c_cookie=450312473155320038 s_cookie=8102269427445091393 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1770926185 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5607f37f10e0 0x5607f37cba50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1770926185 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5607f37ac440 0x5607f37cb420 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a9e729e00 1 --2- 10.20.193.28:0/1770926185 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5607f37abb30 0x5607f37cfd50 unknown :-1 s=CLOSED pgs=195 gs=1 cs=0 l=1 c_cookie=8410192183705570730 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1770926185 >> 10.20.193.28:0/1770926185 conn(0x5607f36ca7a0 msgr2=0x5607f37e8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:25.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a47fff640 0 client.26641 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:55:25.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1770926185 shutdown_connections 2026-02-25T11:55:25.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a9e729e00 1 -- 10.20.193.28:0/1770926185 wait complete. 2026-02-25T11:55:25.158 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:55:25.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.149+0000 7f9a9e729e00 20 client.26641 trim_cache size 0 max 16384 2026-02-25T11:55:25.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.150+0000 7f9a9c72e640 1 -- 10.20.193.28:0/2992411048 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9a700b5760 msgr2=0x7f9a700d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.150+0000 7f9a9c72e640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9a700b5760 0x7f9a700d5c10 secure :-1 s=READY pgs=348 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a84004740 tx=0x7f9a8400b410 comp rx=0 tx=0).stop 2026-02-25T11:55:25.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.150+0000 7f9a9c72e640 1 -- 10.20.193.28:0/2992411048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a94110130 msgr2=0x7f9a94176200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.150+0000 7f9a9c72e640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a94110130 0x7f9a94176200 secure :-1 s=READY pgs=731 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a880098a0 tx=0x7f9a880028d0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.150+0000 7f9a9c72e640 1 -- 10.20.193.28:0/2992411048 shutdown_connections 2026-02-25T11:55:25.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.150+0000 7f9a9c72e640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9a700b5760 0x7f9a700d5c10 unknown :-1 s=CLOSED pgs=348 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.161 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.150+0000 7f9a9c72e640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9a94115930 0x7f9a940f6180 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.150+0000 7f9a9c72e640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a94110130 0x7f9a94176200 unknown :-1 s=CLOSED pgs=731 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.150+0000 7f9a9c72e640 1 --2- 10.20.193.28:0/2992411048 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9a9410f740 0x7f9a940f5af0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.150+0000 7f9a9c72e640 1 -- 10.20.193.28:0/2992411048 >> 10.20.193.28:0/2992411048 conn(0x7f9a940ecff0 msgr2=0x7f9a94112f90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:25.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.150+0000 7f9a9c72e640 1 -- 10.20.193.28:0/2992411048 shutdown_connections 2026-02-25T11:55:25.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.150+0000 7f9a9c72e640 1 -- 10.20.193.28:0/2992411048 wait complete. 2026-02-25T11:55:25.253 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:55:25.436 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:55:25.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.609+0000 7f6fc3746640 1 -- 10.20.193.28:0/2881354862 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fbc110130 msgr2=0x7f6fbc115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.609+0000 7f6fc3746640 1 --2- 10.20.193.28:0/2881354862 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fbc110130 0x7f6fbc115180 secure :-1 s=READY pgs=734 gs=1 cs=0 l=1 c_cookie=11012337537536578304 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fb0009950 tx=0x7f6fb002f1d0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.609+0000 7f6fc3746640 1 -- 10.20.193.28:0/2881354862 shutdown_connections 2026-02-25T11:55:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.609+0000 7f6fc3746640 1 --2- 10.20.193.28:0/2881354862 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6fbc115930 0x7f6fbc176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.609+0000 7f6fc3746640 1 --2- 10.20.193.28:0/2881354862 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fbc110130 0x7f6fbc115180 unknown :-1 s=CLOSED pgs=734 gs=1 cs=0 l=1 c_cookie=11012337537536578304 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.609+0000 7f6fc3746640 1 --2- 10.20.193.28:0/2881354862 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6fbc10f740 0x7f6fbc10fb10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.609+0000 7f6fc3746640 1 -- 10.20.193.28:0/2881354862 >> 10.20.193.28:0/2881354862 conn(0x7f6fbc0ecff0 msgr2=0x7f6fbc10d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.609+0000 7f6fc3746640 1 -- 10.20.193.28:0/2881354862 shutdown_connections 2026-02-25T11:55:25.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.609+0000 7f6fc3746640 1 -- 10.20.193.28:0/2881354862 wait complete. 2026-02-25T11:55:25.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.610+0000 7f6fc3746640 1 Processor -- start 2026-02-25T11:55:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.610+0000 7f6fc3746640 1 -- start start 2026-02-25T11:55:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.610+0000 7f6fc3746640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6fbc10f740 0x7f6fbc0f5af0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.610+0000 7f6fc3746640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fbc110130 0x7f6fbc176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.610+0000 7f6fc3746640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6fbc115930 0x7f6fbc0f6180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.610+0000 7f6fc3746640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0fb400 con 0x7f6fbc110130 2026-02-25T11:55:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.610+0000 7f6fc3746640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0fb570 con 0x7f6fbc10f740 2026-02-25T11:55:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.610+0000 7f6fc3746640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0fb6e0 con 0x7f6fbc115930 2026-02-25T11:55:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.610+0000 7f6fc0cba640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fbc110130 0x7f6fbc176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.614 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.610+0000 7f6fc14bb640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6fbc10f740 0x7f6fbc0f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.610+0000 7f6fc0cba640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fbc110130 0x7f6fbc176200 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40500/0 (socket says 10.20.193.28:40500) 2026-02-25T11:55:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.610+0000 7f6fc0cba640 1 -- 10.20.193.28:0/3372619304 learned_addr learned my addr 10.20.193.28:0/3372619304 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.610+0000 7f6fc1cbc640 1 --2- 10.20.193.28:0/3372619304 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6fbc115930 0x7f6fbc0f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.611+0000 7f6fc0cba640 1 -- 10.20.193.28:0/3372619304 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6fbc115930 msgr2=0x7f6fbc0f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.611+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/3372619304 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6fbc115930 0x7f6fbc0f6180 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.611+0000 7f6fc0cba640 1 -- 10.20.193.28:0/3372619304 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6fbc10f740 msgr2=0x7f6fbc0f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.611+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/3372619304 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6fbc10f740 0x7f6fbc0f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.611+0000 7f6fc0cba640 1 -- 10.20.193.28:0/3372619304 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb0009570 con 0x7f6fbc110130 2026-02-25T11:55:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.611+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/3372619304 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fbc110130 0x7f6fbc176200 secure :-1 s=READY pgs=735 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fb00098a0 tx=0x7f6fb00028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.611+0000 7f6faa7fc640 1 -- 10.20.193.28:0/3372619304 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fb003d070 con 0x7f6fbc110130 2026-02-25T11:55:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.611+0000 7f6faa7fc640 1 -- 10.20.193.28:0/3372619304 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6fb0002a40 con 0x7f6fbc110130 2026-02-25T11:55:25.615 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.611+0000 7f6fc3746640 1 -- 10.20.193.28:0/3372619304 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fbc0fb960 con 0x7f6fbc110130 2026-02-25T11:55:25.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.611+0000 7f6faa7fc640 1 -- 10.20.193.28:0/3372619304 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fb0041720 con 0x7f6fbc110130 2026-02-25T11:55:25.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.611+0000 7f6fc3746640 1 -- 10.20.193.28:0/3372619304 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fbc0fbdd0 con 0x7f6fbc110130 2026-02-25T11:55:25.616 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.614+0000 7f6faa7fc640 1 -- 10.20.193.28:0/3372619304 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f6fb00386a0 con 0x7f6fbc110130 2026-02-25T11:55:25.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.616+0000 7f6faa7fc640 1 --2- 10.20.193.28:0/3372619304 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6f980b56e0 0x7f6f980d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.616+0000 7f6faa7fc640 1 -- 10.20.193.28:0/3372619304 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6fb010afe0 con 0x7f6fbc110130 2026-02-25T11:55:25.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.616+0000 7f6fc14bb640 1 --2- 10.20.193.28:0/3372619304 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6f980b56e0 0x7f6f980d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.616+0000 7f6fc14bb640 1 --2- 10.20.193.28:0/3372619304 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6f980b56e0 0x7f6f980d5b90 secure :-1 s=READY pgs=349 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fac004740 tx=0x7f6fac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:25.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.616+0000 7f6fc5741e00 1 Processor -- start 2026-02-25T11:55:25.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc5741e00 1 -- start start 2026-02-25T11:55:25.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc5741e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555797419b30 0x555797419f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc5741e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55579741a440 0x55579745eba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc5741e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579745f0e0 0x55579747f4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc5741e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55579732f8a0 con 0x55579745f0e0 2026-02-25T11:55:25.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc5741e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55579732b5d0 con 0x55579741a440 2026-02-25T11:55:25.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc5741e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555797325570 con 0x555797419b30 2026-02-25T11:55:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc1cbc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579745f0e0 0x55579747f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc1cbc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579745f0e0 0x55579747f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40516/0 (socket says 10.20.193.28:40516) 2026-02-25T11:55:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc1cbc640 1 -- 10.20.193.28:0/3859584755 learned_addr learned my addr 10.20.193.28:0/3859584755 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/3859584755 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55579741a440 0x55579745eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc14bb640 1 --2- 10.20.193.28:0/3859584755 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555797419b30 0x555797419f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc1cbc640 1 -- 10.20.193.28:0/3859584755 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555797419b30 msgr2=0x555797419f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc1cbc640 1 --2- 10.20.193.28:0/3859584755 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555797419b30 0x555797419f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc1cbc640 1 -- 10.20.193.28:0/3859584755 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55579741a440 msgr2=0x55579745eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc1cbc640 1 --2- 10.20.193.28:0/3859584755 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55579741a440 0x55579745eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc1cbc640 1 -- 10.20.193.28:0/3859584755 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557972d0d10 con 0x55579745f0e0 2026-02-25T11:55:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.617+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/3859584755 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55579741a440 0x55579745eba0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:55:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.618+0000 7f6fc1cbc640 1 --2- 10.20.193.28:0/3859584755 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579745f0e0 0x55579747f4c0 secure :-1 s=READY pgs=736 gs=3 cs=0 l=1 c_cookie=18242341497335025276 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fb80055e0 tx=0x7f6fb8003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd29d0f094058e7c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.618+0000 7f6f89ce5640 1 -- 10.20.193.28:0/3859584755 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fb8011430 con 0x55579745f0e0 2026-02-25T11:55:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.618+0000 7f6f89ce5640 1 -- 10.20.193.28:0/3859584755 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6fb8011a50 con 0x55579745f0e0 2026-02-25T11:55:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.618+0000 7f6f89ce5640 1 -- 10.20.193.28:0/3859584755 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fb801ab00 con 0x55579745f0e0 2026-02-25T11:55:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.618+0000 7f6fc5741e00 1 -- 10.20.193.28:0/3859584755 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579745f0e0 msgr2=0x55579747f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.618+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/3859584755 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579745f0e0 0x55579747f4c0 secure :-1 s=READY pgs=736 gs=3 cs=0 l=1 c_cookie=18242341497335025276 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fb80055e0 tx=0x7f6fb8003c30 comp rx=0 tx=0).stop 2026-02-25T11:55:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.618+0000 7f6fc5741e00 1 -- 10.20.193.28:0/3859584755 shutdown_connections 2026-02-25T11:55:25.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.618+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/3859584755 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579745f0e0 0x55579747f4c0 unknown :-1 s=CLOSED pgs=736 gs=3 cs=0 l=1 c_cookie=18242341497335025276 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.618+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/3859584755 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55579741a440 0x55579745eba0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.618+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/3859584755 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555797419b30 0x555797419f00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.618+0000 7f6fc5741e00 1 -- 10.20.193.28:0/3859584755 >> 10.20.193.28:0/3859584755 conn(0x5557973387a0 msgr2=0x555797418410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:25.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.618+0000 7f6fc5741e00 1 -- 10.20.193.28:0/3859584755 shutdown_connections 2026-02-25T11:55:25.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.618+0000 7f6fc5741e00 1 -- 10.20.193.28:0/3859584755 wait complete. 2026-02-25T11:55:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.619+0000 7f6fc5741e00 1 -- start start 2026-02-25T11:55:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.620+0000 7f6fc5741e00 1 Processor -- start 2026-02-25T11:55:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.620+0000 7f6fc5741e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555797419b30 0x555797431e30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.620+0000 7f6fc5741e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579741a440 0x555797432370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.620+0000 7f6fc5741e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55579745f0e0 0x555797437360 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.620+0000 7f6fc5741e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5557974378a0 con 0x55579741a440 2026-02-25T11:55:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.620+0000 7f6fc5741e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555797437a10 con 0x55579745f0e0 2026-02-25T11:55:25.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.620+0000 7f6fc5741e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555797437bd0 con 0x555797419b30 2026-02-25T11:55:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.620+0000 7f6fc0cba640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579741a440 0x555797432370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.620+0000 7f6fc14bb640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555797419b30 0x555797431e30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.620+0000 7f6fc14bb640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555797419b30 0x555797431e30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:37868/0 (socket says 10.20.193.28:37868) 2026-02-25T11:55:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.620+0000 7f6fc14bb640 1 -- 10.20.193.28:0/2161943832 learned_addr learned my addr 10.20.193.28:0/2161943832 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.620+0000 7f6fc1cbc640 1 --2- 10.20.193.28:0/2161943832 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55579745f0e0 0x555797437360 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6fc14bb640 1 -- 10.20.193.28:0/2161943832 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55579745f0e0 msgr2=0x555797437360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6fc14bb640 1 --2- 10.20.193.28:0/2161943832 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55579745f0e0 0x555797437360 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6fc14bb640 1 -- 10.20.193.28:0/2161943832 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579741a440 msgr2=0x555797432370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6fc14bb640 1 --2- 10.20.193.28:0/2161943832 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579741a440 0x555797432370 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6fc14bb640 1 -- 10.20.193.28:0/2161943832 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb8005250 con 0x555797419b30 2026-02-25T11:55:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/2161943832 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579741a440 0x555797432370 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:55:25.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6fc14bb640 1 --2- 10.20.193.28:0/2161943832 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555797419b30 0x555797431e30 secure :-1 s=READY pgs=196 gs=2 cs=0 l=1 c_cookie=3456488306685915330 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fac009230 tx=0x7f6fac013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ff7e9cd3a4e30c2 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6f72ffd640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:55:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6f72ffd640 1 -- 10.20.193.28:0/2161943832 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fac021040 con 0x555797419b30 2026-02-25T11:55:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6f72ffd640 1 -- 10.20.193.28:0/2161943832 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6fac019400 con 0x555797419b30 2026-02-25T11:55:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6fc5741e00 1 -- 10.20.193.28:0/2161943832 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fbc0fbdd0 con 0x555797419b30 2026-02-25T11:55:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6f72ffd640 1 -- 10.20.193.28:0/2161943832 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fac025660 con 0x555797419b30 2026-02-25T11:55:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6fc5741e00 10 client.26653 _ll_register_callbacks cb 0x5557972a6a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6fc5741e00 1 -- 10.20.193.28:0/2161943832 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555796fbb6f0 con 0x555797419b30 2026-02-25T11:55:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6f72ffd640 1 -- 10.20.193.28:0/2161943832 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6fac0257f0 con 0x555797419b30 2026-02-25T11:55:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6f72ffd640 1 -- 10.20.193.28:0/2161943832 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6fac025a10 con 0x555797419b30 2026-02-25T11:55:25.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6fc5741e00 10 client.26653 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:25.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.621+0000 7f6fc5741e00 1 -- 10.20.193.28:0/2161943832 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5557972a93c0 con 0x555797419b30 2026-02-25T11:55:25.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.622+0000 7f6f72ffd640 1 -- 10.20.193.28:0/2161943832 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6fac025bb0 con 0x555797419b30 2026-02-25T11:55:25.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.622+0000 7f6fc5741e00 10 client.26653 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:25.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.622+0000 7f6fc5741e00 10 client.26653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:25.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.622+0000 7f6fc5741e00 20 client.26653 populate_metadata read hostname 'trial028' 2026-02-25T11:55:25.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.622+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/2161943832 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55579745c720 0x55579745cbc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.622+0000 7f6fc5741e00 4 client.26653 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:55:25.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.622+0000 7f6fc5741e00 1 -- 10.20.193.28:0/2161943832 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557972f7fc0 con 0x55579745c720 2026-02-25T11:55:25.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.622+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/2161943832 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55579745c720 0x55579745cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.623+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/2161943832 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55579745c720 0x55579745cbc0 crc :-1 s=READY pgs=132 gs=4 cs=0 l=0 c_cookie=17669863330524739352 s_cookie=7170262161505174340 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f537f71485a0b718 server_cookie=6381e3723af0c344 in_seq=0 out_seq=0 2026-02-25T11:55:25.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.623+0000 7f6f72ffd640 10 client.26653 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:25.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.623+0000 7f6f72ffd640 1 -- 10.20.193.28:0/2161943832 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5557972f7fc0 con 0x55579745c720 2026-02-25T11:55:25.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.623+0000 7f6f72ffd640 10 client.26653 handle_command_reply: tid=0 2026-02-25T11:55:25.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.623+0000 7f6fc5741e00 1 client.26653 shutdown 2026-02-25T11:55:25.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.624+0000 7f6fc5741e00 1 -- 10.20.193.28:0/2161943832 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555797419b30 msgr2=0x555797431e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.624+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/2161943832 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555797419b30 0x555797431e30 secure :-1 s=READY pgs=196 gs=2 cs=0 l=1 c_cookie=3456488306685915330 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fac009230 tx=0x7f6fac013040 comp rx=0 tx=0).stop 2026-02-25T11:55:25.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.624+0000 7f6fc5741e00 1 -- 10.20.193.28:0/2161943832 shutdown_connections 2026-02-25T11:55:25.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.624+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/2161943832 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55579745c720 0x55579745cbc0 crc :-1 s=READY pgs=132 gs=4 cs=0 l=0 c_cookie=17669863330524739352 s_cookie=7170262161505174340 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.624+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/2161943832 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55579745f0e0 0x555797437360 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.624+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/2161943832 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579741a440 0x555797432370 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.624+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/2161943832 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x555797419b30 0x555797431e30 unknown :-1 s=CLOSED pgs=196 gs=2 cs=0 l=1 c_cookie=3456488306685915330 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.624+0000 7f6fc5741e00 1 -- 10.20.193.28:0/2161943832 >> 10.20.193.28:0/2161943832 conn(0x5557973387a0 msgr2=0x555797418b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:25.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.624+0000 7f6f72ffd640 0 client.26653 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:25.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.624+0000 7f6fc5741e00 1 -- 10.20.193.28:0/2161943832 shutdown_connections 2026-02-25T11:55:25.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.624+0000 7f6fc5741e00 1 -- 10.20.193.28:0/2161943832 wait complete. 2026-02-25T11:55:25.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.624+0000 7f6fc5741e00 20 client.26653 trim_cache size 0 max 16384 2026-02-25T11:55:25.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.631+0000 7f6fc5741e00 1 Processor -- start 2026-02-25T11:55:25.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.631+0000 7f6fc5741e00 1 -- start start 2026-02-25T11:55:25.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.631+0000 7f6fc5741e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555797419b30 0x55579745c720 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.631+0000 7f6fc5741e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579741a440 0x555797418d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.631+0000 7f6fc5741e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55579745f0e0 0x555797419260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.631+0000 7f6fc5741e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5557973231a0 con 0x55579741a440 2026-02-25T11:55:25.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.631+0000 7f6fc5741e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x555796fe3700 con 0x555797419b30 2026-02-25T11:55:25.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.631+0000 7f6fc5741e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x555797458ef0 con 0x55579745f0e0 2026-02-25T11:55:25.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6fc0cba640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579741a440 0x555797418d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6fc0cba640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579741a440 0x555797418d20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:40544/0 (socket says 10.20.193.28:40544) 2026-02-25T11:55:25.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6fc0cba640 1 -- 10.20.193.28:0/504044792 learned_addr learned my addr 10.20.193.28:0/504044792 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:25.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6fc14bb640 1 --2- 10.20.193.28:0/504044792 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555797419b30 0x55579745c720 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6fc0cba640 1 -- 10.20.193.28:0/504044792 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55579745f0e0 msgr2=0x555797419260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/504044792 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55579745f0e0 0x555797419260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6fc0cba640 1 -- 10.20.193.28:0/504044792 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555797419b30 msgr2=0x55579745c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/504044792 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555797419b30 0x55579745c720 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6fc0cba640 1 -- 10.20.193.28:0/504044792 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557972bb0e0 con 0x55579741a440 2026-02-25T11:55:25.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/504044792 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579741a440 0x555797418d20 secure :-1 s=READY pgs=737 gs=1 cs=0 l=1 c_cookie=17433848742167181551 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fb0002bb0 tx=0x7f6fb0108a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f1791000c3f8ef server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:25.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6f894e4640 1 -- 10.20.193.28:0/504044792 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fb00d11d0 con 0x55579741a440 2026-02-25T11:55:25.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6f894e4640 1 -- 10.20.193.28:0/504044792 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6fb00d0c10 con 0x55579741a440 2026-02-25T11:55:25.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6f894e4640 1 -- 10.20.193.28:0/504044792 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fb00dc290 con 0x55579741a440 2026-02-25T11:55:25.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6fc5741e00 1 -- 10.20.193.28:0/504044792 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579741a440 msgr2=0x555797418d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.632+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/504044792 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579741a440 0x555797418d20 secure :-1 s=READY pgs=737 gs=1 cs=0 l=1 c_cookie=17433848742167181551 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fb0002bb0 tx=0x7f6fb0108a30 comp rx=0 tx=0).stop 2026-02-25T11:55:25.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.633+0000 7f6fc5741e00 1 -- 10.20.193.28:0/504044792 shutdown_connections 2026-02-25T11:55:25.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.633+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/504044792 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55579745f0e0 0x555797419260 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.633+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/504044792 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579741a440 0x555797418d20 unknown :-1 s=CLOSED pgs=737 gs=1 cs=0 l=1 c_cookie=17433848742167181551 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.633+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/504044792 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555797419b30 0x55579745c720 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.633+0000 7f6fc5741e00 1 -- 10.20.193.28:0/504044792 >> 10.20.193.28:0/504044792 conn(0x5557973387a0 msgr2=0x555797423f60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:25.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.633+0000 7f6fc5741e00 1 -- 10.20.193.28:0/504044792 shutdown_connections 2026-02-25T11:55:25.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.633+0000 7f6fc5741e00 1 -- 10.20.193.28:0/504044792 wait complete. 2026-02-25T11:55:25.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.633+0000 7f6fc5741e00 1 -- start start 2026-02-25T11:55:25.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.633+0000 7f6fc5741e00 1 Processor -- start 2026-02-25T11:55:25.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.634+0000 7f6fc5741e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555797419b30 0x55579745d8b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.634+0000 7f6fc5741e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55579741a440 0x55579745ddf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.634+0000 7f6fc5741e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579745f0e0 0x55579745e330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.634+0000 7f6fc5741e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55579743b410 con 0x55579745f0e0 2026-02-25T11:55:25.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.634+0000 7f6fc5741e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55579743b580 con 0x555797419b30 2026-02-25T11:55:25.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.634+0000 7f6fc5741e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55579743b740 con 0x55579741a440 2026-02-25T11:55:25.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.634+0000 7f6fc14bb640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555797419b30 0x55579745d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.634+0000 7f6fc1cbc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579745f0e0 0x55579745e330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.634+0000 7f6fc14bb640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555797419b30 0x55579745d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:41270/0 (socket says 10.20.193.28:41270) 2026-02-25T11:55:25.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.634+0000 7f6fc14bb640 1 -- 10.20.193.28:0/1940971499 learned_addr learned my addr 10.20.193.28:0/1940971499 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:25.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.634+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/1940971499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55579741a440 0x55579745ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6fc14bb640 1 -- 10.20.193.28:0/1940971499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55579741a440 msgr2=0x55579745ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6fc14bb640 1 --2- 10.20.193.28:0/1940971499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55579741a440 0x55579745ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6fc14bb640 1 -- 10.20.193.28:0/1940971499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579745f0e0 msgr2=0x55579745e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:25.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6fc14bb640 1 --2- 10.20.193.28:0/1940971499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579745f0e0 0x55579745e330 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6fc14bb640 1 -- 10.20.193.28:0/1940971499 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557972b5b60 con 0x555797419b30 2026-02-25T11:55:25.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6fc14bb640 1 --2- 10.20.193.28:0/1940971499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555797419b30 0x55579745d8b0 secure :-1 s=READY pgs=163 gs=2 cs=0 l=1 c_cookie=1508343430199255480 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fac025ff0 tx=0x7f6fac029000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14eeb65bb45221b8 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:25.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6f72ffd640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:55:25.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6f72ffd640 1 -- 10.20.193.28:0/1940971499 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fac021070 con 0x555797419b30 2026-02-25T11:55:25.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6f72ffd640 1 -- 10.20.193.28:0/1940971499 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x555796fbb6f0 con 0x555797419b30 2026-02-25T11:55:25.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6fc5741e00 1 -- 10.20.193.28:0/1940971499 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fbc0fb960 con 0x555797419b30 2026-02-25T11:55:25.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6f72ffd640 1 -- 10.20.193.28:0/1940971499 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6fac00eae0 con 0x555797419b30 2026-02-25T11:55:25.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6fc5741e00 10 client.26681 _ll_register_callbacks cb 0x5557972a6a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:25.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6f72ffd640 1 -- 10.20.193.28:0/1940971499 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6fac00ec70 con 0x555797419b30 2026-02-25T11:55:25.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.635+0000 7f6fc5741e00 1 -- 10.20.193.28:0/1940971499 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fb8011a50 con 0x555797419b30 2026-02-25T11:55:25.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.636+0000 7f6f72ffd640 1 -- 10.20.193.28:0/1940971499 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6fac01bdc0 con 0x555797419b30 2026-02-25T11:55:25.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.636+0000 7f6fc5741e00 10 client.26681 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:25.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.636+0000 7f6fc5741e00 1 -- 10.20.193.28:0/1940971499 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5557972a96f0 con 0x555797419b30 2026-02-25T11:55:25.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.636+0000 7f6f72ffd640 1 -- 10.20.193.28:0/1940971499 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f6fac01b870 con 0x555797419b30 2026-02-25T11:55:25.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.636+0000 7f6fc5741e00 10 client.26681 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:25.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.636+0000 7f6fc5741e00 10 client.26681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:25.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.636+0000 7f6fc5741e00 20 client.26681 populate_metadata read hostname 'trial028' 2026-02-25T11:55:25.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.636+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/1940971499 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55579750bbb0 0x55579752c060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:25.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.636+0000 7f6fc5741e00 4 client.26681 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:55:25.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.636+0000 7f6fc5741e00 1 -- 10.20.193.28:0/1940971499 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555796fb8f40 con 0x55579750bbb0 2026-02-25T11:55:25.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.636+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/1940971499 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55579750bbb0 0x55579752c060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:25.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6fc0cba640 1 --2- 10.20.193.28:0/1940971499 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55579750bbb0 0x55579752c060 crc :-1 s=READY pgs=133 gs=4 cs=0 l=0 c_cookie=10033368096027449337 s_cookie=8170594495136964857 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b3daf1fd07417f9 server_cookie=7163c8675666a0f9 in_seq=0 out_seq=0 2026-02-25T11:55:25.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6f72ffd640 10 client.26681 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:25.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6f72ffd640 1 -- 10.20.193.28:0/1940971499 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555796fb8f40 con 0x55579750bbb0 2026-02-25T11:55:25.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6f72ffd640 10 client.26681 handle_command_reply: tid=0 2026-02-25T11:55:25.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6fc5741e00 1 client.26681 shutdown 2026-02-25T11:55:25.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6fc5741e00 1 -- 10.20.193.28:0/1940971499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555797419b30 msgr2=0x55579745d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/1940971499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555797419b30 0x55579745d8b0 secure :-1 s=READY pgs=163 gs=2 cs=0 l=1 c_cookie=1508343430199255480 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fac025ff0 tx=0x7f6fac029000 comp rx=0 tx=0).stop 2026-02-25T11:55:25.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6fc5741e00 1 -- 10.20.193.28:0/1940971499 shutdown_connections 2026-02-25T11:55:25.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/1940971499 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55579750bbb0 0x55579752c060 crc :-1 s=READY pgs=133 gs=4 cs=0 l=0 c_cookie=10033368096027449337 s_cookie=8170594495136964857 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/1940971499 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55579745f0e0 0x55579745e330 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/1940971499 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55579741a440 0x55579745ddf0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6fc5741e00 1 --2- 10.20.193.28:0/1940971499 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x555797419b30 0x55579745d8b0 unknown :-1 s=CLOSED pgs=163 gs=2 cs=0 l=1 c_cookie=1508343430199255480 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6fc5741e00 1 -- 10.20.193.28:0/1940971499 >> 10.20.193.28:0/1940971499 conn(0x5557973387a0 msgr2=0x55579745c720 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:25.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6f72ffd640 0 client.26681 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:25.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6fc5741e00 1 -- 10.20.193.28:0/1940971499 shutdown_connections 2026-02-25T11:55:25.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.637+0000 7f6fc5741e00 1 -- 10.20.193.28:0/1940971499 wait complete. 2026-02-25T11:55:25.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.638+0000 7f6fc5741e00 20 client.26681 trim_cache size 0 max 16384 2026-02-25T11:55:25.658 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:55:25.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.638+0000 7f6fc3746640 1 -- 10.20.193.28:0/3372619304 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6f980b56e0 msgr2=0x7f6f980d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.638+0000 7f6fc3746640 1 --2- 10.20.193.28:0/3372619304 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6f980b56e0 0x7f6f980d5b90 secure :-1 s=READY pgs=349 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fac004740 tx=0x7f6fac005f90 comp rx=0 tx=0).stop 2026-02-25T11:55:25.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.638+0000 7f6fc3746640 1 -- 10.20.193.28:0/3372619304 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fbc110130 msgr2=0x7f6fbc176200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:25.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.638+0000 7f6fc3746640 1 --2- 10.20.193.28:0/3372619304 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fbc110130 0x7f6fbc176200 secure :-1 s=READY pgs=735 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6fb00098a0 tx=0x7f6fb00028d0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.638+0000 7f6fc3746640 1 -- 10.20.193.28:0/3372619304 shutdown_connections 2026-02-25T11:55:25.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.638+0000 7f6fc3746640 1 --2- 10.20.193.28:0/3372619304 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6f980b56e0 0x7f6f980d5b90 unknown :-1 s=CLOSED pgs=349 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.638+0000 7f6fc3746640 1 --2- 10.20.193.28:0/3372619304 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6fbc115930 0x7f6fbc0f6180 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.638+0000 7f6fc3746640 1 --2- 10.20.193.28:0/3372619304 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6fbc110130 0x7f6fbc176200 unknown :-1 s=CLOSED pgs=735 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.638+0000 7f6fc3746640 1 --2- 10.20.193.28:0/3372619304 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6fbc10f740 0x7f6fbc0f5af0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:25.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.638+0000 7f6fc3746640 1 -- 10.20.193.28:0/3372619304 >> 10.20.193.28:0/3372619304 conn(0x7f6fbc0ecff0 msgr2=0x7f6fbc112f90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:25.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.638+0000 7f6fc3746640 1 -- 10.20.193.28:0/3372619304 shutdown_connections 2026-02-25T11:55:25.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:25.638+0000 7f6fc3746640 1 -- 10.20.193.28:0/3372619304 wait complete. 2026-02-25T11:55:26.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:26 trial028 ceph-mon[39224]: pgmap v442: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:26.740 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:55:26.740 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:55:26.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:26 trial167 ceph-mon[43392]: pgmap v442: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:26.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:26 trial119 ceph-mon[42710]: pgmap v442: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:26.925 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:55:27.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.094+0000 7f4c9292c640 1 -- 10.20.193.28:0/302168411 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4c8c115840 msgr2=0x7f4c8c176210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.094+0000 7f4c9292c640 1 --2- 10.20.193.28:0/302168411 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4c8c115840 0x7f4c8c176210 secure :-1 s=READY pgs=738 gs=1 cs=0 l=1 c_cookie=11303918277061333482 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4c8800aa30 tx=0x7f4c88030270 comp rx=0 tx=0).stop 2026-02-25T11:55:27.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.094+0000 7f4c9292c640 1 -- 10.20.193.28:0/302168411 shutdown_connections 2026-02-25T11:55:27.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.094+0000 7f4c9292c640 1 --2- 10.20.193.28:0/302168411 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4c8c115840 0x7f4c8c176210 unknown :-1 s=CLOSED pgs=738 gs=1 cs=0 l=1 c_cookie=11303918277061333482 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.094+0000 7f4c9292c640 1 --2- 10.20.193.28:0/302168411 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4c8c110040 0x7f4c8c115090 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.094+0000 7f4c9292c640 1 --2- 10.20.193.28:0/302168411 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4c8c10f650 0x7f4c8c10fa20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.094+0000 7f4c9292c640 1 -- 10.20.193.28:0/302168411 >> 10.20.193.28:0/302168411 conn(0x7f4c8c0ecf00 msgr2=0x7f4c8c10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:27.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.094+0000 7f4c9292c640 1 -- 10.20.193.28:0/302168411 shutdown_connections 2026-02-25T11:55:27.096 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.094+0000 7f4c9292c640 1 -- 10.20.193.28:0/302168411 wait complete. 2026-02-25T11:55:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.095+0000 7f4c9292c640 1 Processor -- start 2026-02-25T11:55:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.095+0000 7f4c9292c640 1 -- start start 2026-02-25T11:55:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.095+0000 7f4c9292c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4c8c10f650 0x7f4c8c0f5930 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.095+0000 7f4c9292c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4c8c110040 0x7f4c8c176110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.095+0000 7f4c9292c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4c8c115840 0x7f4c8c0f6030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9292c640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c0fb240 con 0x7f4c8c115840 2026-02-25T11:55:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9292c640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c0fb3b0 con 0x7f4c8c10f650 2026-02-25T11:55:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9292c640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c0fb570 con 0x7f4c8c110040 2026-02-25T11:55:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9212b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4c8c115840 0x7f4c8c0f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9212b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4c8c115840 0x7f4c8c0f6030 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47398/0 (socket says 10.20.193.28:47398) 2026-02-25T11:55:27.097 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9212b640 1 -- 10.20.193.28:0/1611403712 learned_addr learned my addr 10.20.193.28:0/1611403712 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c91129640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4c8c110040 0x7f4c8c176110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9192a640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4c8c10f650 0x7f4c8c0f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9212b640 1 -- 10.20.193.28:0/1611403712 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4c8c110040 msgr2=0x7f4c8c176110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9212b640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4c8c110040 0x7f4c8c176110 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9212b640 1 -- 10.20.193.28:0/1611403712 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4c8c10f650 msgr2=0x7f4c8c0f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9212b640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4c8c10f650 0x7f4c8c0f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9212b640 1 -- 10.20.193.28:0/1611403712 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8800a650 con 0x7f4c8c115840 2026-02-25T11:55:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9212b640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4c8c115840 0x7f4c8c0f6030 secure :-1 s=READY pgs=739 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4c8800a980 tx=0x7f4c880039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c7affd640 1 -- 10.20.193.28:0/1611403712 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4c8803e070 con 0x7f4c8c115840 2026-02-25T11:55:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c9292c640 1 -- 10.20.193.28:0/1611403712 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c0fb7f0 con 0x7f4c8c115840 2026-02-25T11:55:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c7affd640 1 -- 10.20.193.28:0/1611403712 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4c88003aa0 con 0x7f4c8c115840 2026-02-25T11:55:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.096+0000 7f4c7affd640 1 -- 10.20.193.28:0/1611403712 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4c88042a60 con 0x7f4c8c115840 2026-02-25T11:55:27.098 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.097+0000 7f4c9292c640 1 -- 10.20.193.28:0/1611403712 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c0fbc60 con 0x7f4c8c115840 2026-02-25T11:55:27.101 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.099+0000 7f4c7affd640 1 -- 10.20.193.28:0/1611403712 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f4c880050c0 con 0x7f4c8c115840 2026-02-25T11:55:27.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.101+0000 7f4c7affd640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4c680b5750 0x7f4c680d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.102 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.101+0000 7f4c7affd640 1 -- 10.20.193.28:0/1611403712 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4c8810ca40 con 0x7f4c8c115840 2026-02-25T11:55:27.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.101+0000 7f4c9192a640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4c680b5750 0x7f4c680d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.103 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.102+0000 7f4c9192a640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4c680b5750 0x7f4c680d5c00 secure :-1 s=READY pgs=350 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4c7c006fd0 tx=0x7f4c7c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.102+0000 7f4c95bb0e00 1 Processor -- start 2026-02-25T11:55:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.102+0000 7f4c95bb0e00 1 -- start start 2026-02-25T11:55:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c95bb0e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbc750 0x5575d5fbcb20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c95bb0e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbd060 0x5575d6001610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c95bb0e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d6001b50 0x5575d6021f30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c95bb0e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5575d5ed2d80 con 0x5575d6001b50 2026-02-25T11:55:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c95bb0e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5575d5eceab0 con 0x5575d5fbc750 2026-02-25T11:55:27.104 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c95bb0e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5575d5ec8a50 con 0x5575d5fbd060 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c9212b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d6001b50 0x5575d6021f30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c9212b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d6001b50 0x5575d6021f30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47408/0 (socket says 10.20.193.28:47408) 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c9212b640 1 -- 10.20.193.28:0/1583019284 learned_addr learned my addr 10.20.193.28:0/1583019284 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c9192a640 1 --2- 10.20.193.28:0/1583019284 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbc750 0x5575d5fbcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c9212b640 1 -- 10.20.193.28:0/1583019284 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbd060 msgr2=0x5575d6001610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c9212b640 1 --2- 10.20.193.28:0/1583019284 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbd060 0x5575d6001610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c9212b640 1 -- 10.20.193.28:0/1583019284 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbc750 msgr2=0x5575d5fbcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c9212b640 1 --2- 10.20.193.28:0/1583019284 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbc750 0x5575d5fbcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c9212b640 1 -- 10.20.193.28:0/1583019284 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575d5e74120 con 0x5575d6001b50 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.103+0000 7f4c9212b640 1 --2- 10.20.193.28:0/1583019284 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d6001b50 0x5575d6021f30 secure :-1 s=READY pgs=740 gs=2 cs=0 l=1 c_cookie=964713605498587489 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4c880424c0 tx=0x7f4c88030270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6359e9b5bac961 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.104+0000 7f4c5a4e6640 1 -- 10.20.193.28:0/1583019284 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4c880d2600 con 0x5575d6001b50 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.104+0000 7f4c5a4e6640 1 -- 10.20.193.28:0/1583019284 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4c880d3800 con 0x5575d6001b50 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.104+0000 7f4c5a4e6640 1 -- 10.20.193.28:0/1583019284 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4c8810a0b0 con 0x5575d6001b50 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.104+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/1583019284 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d6001b50 msgr2=0x5575d6021f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.105 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.104+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/1583019284 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d6001b50 0x5575d6021f30 secure :-1 s=READY pgs=740 gs=2 cs=0 l=1 c_cookie=964713605498587489 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4c880424c0 tx=0x7f4c88030270 comp rx=0 tx=0).stop 2026-02-25T11:55:27.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.104+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/1583019284 shutdown_connections 2026-02-25T11:55:27.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.104+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/1583019284 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d6001b50 0x5575d6021f30 unknown :-1 s=CLOSED pgs=740 gs=2 cs=0 l=1 c_cookie=964713605498587489 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.104+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/1583019284 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbd060 0x5575d6001610 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.104+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/1583019284 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbc750 0x5575d5fbcb20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.104+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/1583019284 >> 10.20.193.28:0/1583019284 conn(0x5575d5f9a8f0 msgr2=0x5575d5fbacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:27.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.104+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/1583019284 shutdown_connections 2026-02-25T11:55:27.106 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.104+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/1583019284 wait complete. 2026-02-25T11:55:27.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.105+0000 7f4c95bb0e00 1 -- start start 2026-02-25T11:55:27.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.105+0000 7f4c95bb0e00 1 Processor -- start 2026-02-25T11:55:27.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c95bb0e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d5fbc750 0x5575d6019540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c95bb0e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbd060 0x5575d6019a80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c95bb0e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d6001b50 0x5575d601ea70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c95bb0e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5575d601efb0 con 0x5575d5fbc750 2026-02-25T11:55:27.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c95bb0e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5575d601f120 con 0x5575d5fbd060 2026-02-25T11:55:27.107 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c95bb0e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5575d601f2e0 con 0x5575d6001b50 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c9192a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d5fbc750 0x5575d6019540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c9192a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d5fbc750 0x5575d6019540 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47422/0 (socket says 10.20.193.28:47422) 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c9192a640 1 -- 10.20.193.28:0/3471536522 learned_addr learned my addr 10.20.193.28:0/3471536522 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c9212b640 1 --2- 10.20.193.28:0/3471536522 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d6001b50 0x5575d601ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c91129640 1 --2- 10.20.193.28:0/3471536522 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbd060 0x5575d6019a80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c9192a640 1 -- 10.20.193.28:0/3471536522 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d6001b50 msgr2=0x5575d601ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c9192a640 1 --2- 10.20.193.28:0/3471536522 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d6001b50 0x5575d601ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c9192a640 1 -- 10.20.193.28:0/3471536522 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbd060 msgr2=0x5575d6019a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c9192a640 1 --2- 10.20.193.28:0/3471536522 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbd060 0x5575d6019a80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.106+0000 7f4c9192a640 1 -- 10.20.193.28:0/3471536522 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575d5e4c810 con 0x5575d5fbc750 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.107+0000 7f4c9192a640 1 --2- 10.20.193.28:0/3471536522 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d5fbc750 0x5575d6019540 secure :-1 s=READY pgs=741 gs=1 cs=0 l=1 c_cookie=12681058633761180445 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4c7c009190 tx=0x7f4c7c004880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=affc2ba3e93c4b1d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.107+0000 7f4c477fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.107+0000 7f4c477fe640 1 -- 10.20.193.28:0/3471536522 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4c7c012640 con 0x5575d5fbc750 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.107+0000 7f4c477fe640 1 -- 10.20.193.28:0/3471536522 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4c7c012c60 con 0x5575d5fbc750 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.107+0000 7f4c477fe640 1 -- 10.20.193.28:0/3471536522 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4c7c011d00 con 0x5575d5fbc750 2026-02-25T11:55:27.108 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.107+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3471536522 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c0fbc60 con 0x5575d5fbc750 2026-02-25T11:55:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.107+0000 7f4c477fe640 1 -- 10.20.193.28:0/3471536522 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4c7c02b3b0 con 0x5575d5fbc750 2026-02-25T11:55:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.107+0000 7f4c95bb0e00 10 client.16965 _ll_register_callbacks cb 0x5575d5e49e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.107+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3471536522 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5575d5c20530 con 0x5575d5fbc750 2026-02-25T11:55:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.107+0000 7f4c477fe640 1 -- 10.20.193.28:0/3471536522 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4c7c011e90 con 0x5575d5fbc750 2026-02-25T11:55:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.107+0000 7f4c95bb0e00 10 client.16965 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.107+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3471536522 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5575d5e5e4f0 con 0x5575d5fbc750 2026-02-25T11:55:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.108+0000 7f4c477fe640 1 -- 10.20.193.28:0/3471536522 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f4c7c01f070 con 0x5575d5fbc750 2026-02-25T11:55:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.108+0000 7f4c95bb0e00 10 client.16965 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.108+0000 7f4c95bb0e00 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.108+0000 7f4c95bb0e00 20 client.16965 populate_metadata read hostname 'trial028' 2026-02-25T11:55:27.109 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.108+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/3471536522 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5575d5ffeed0 0x5575d5fff370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.108+0000 7f4c95bb0e00 4 client.16965 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:55:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.108+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3471536522 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575d5e75b50 con 0x5575d5ffeed0 2026-02-25T11:55:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.108+0000 7f4c91129640 1 --2- 10.20.193.28:0/3471536522 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5575d5ffeed0 0x5575d5fff370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.109+0000 7f4c91129640 1 --2- 10.20.193.28:0/3471536522 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5575d5ffeed0 0x5575d5fff370 crc :-1 s=READY pgs=134 gs=4 cs=0 l=0 c_cookie=18359525157443763989 s_cookie=4733184178746575978 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feca22db96487f15 server_cookie=41afa5eedeb04c6a in_seq=0 out_seq=0 2026-02-25T11:55:27.110 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.109+0000 7f4c477fe640 10 client.16965 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.109+0000 7f4c477fe640 1 -- 10.20.193.28:0/3471536522 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5575d5e75b50 con 0x5575d5ffeed0 2026-02-25T11:55:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.109+0000 7f4c477fe640 10 client.16965 handle_command_reply: tid=0 2026-02-25T11:55:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.109+0000 7f4c95bb0e00 1 client.16965 shutdown 2026-02-25T11:55:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.110+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3471536522 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d5fbc750 msgr2=0x5575d6019540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.110+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/3471536522 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d5fbc750 0x5575d6019540 secure :-1 s=READY pgs=741 gs=1 cs=0 l=1 c_cookie=12681058633761180445 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4c7c009190 tx=0x7f4c7c004880 comp rx=0 tx=0).stop 2026-02-25T11:55:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.110+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3471536522 shutdown_connections 2026-02-25T11:55:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.110+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/3471536522 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5575d5ffeed0 0x5575d5fff370 crc :-1 s=READY pgs=134 gs=4 cs=0 l=0 c_cookie=18359525157443763989 s_cookie=4733184178746575978 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.110+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/3471536522 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d6001b50 0x5575d601ea70 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.110+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/3471536522 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbd060 0x5575d6019a80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.110+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/3471536522 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d5fbc750 0x5575d6019540 unknown :-1 s=CLOSED pgs=741 gs=1 cs=0 l=1 c_cookie=12681058633761180445 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.110+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3471536522 >> 10.20.193.28:0/3471536522 conn(0x5575d5f9a8f0 msgr2=0x5575d5fbacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:27.111 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.110+0000 7f4c477fe640 0 client.16965 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:27.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.110+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3471536522 shutdown_connections 2026-02-25T11:55:27.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.110+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3471536522 wait complete. 2026-02-25T11:55:27.112 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.110+0000 7f4c95bb0e00 20 client.16965 trim_cache size 0 max 16384 2026-02-25T11:55:27.120 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.119+0000 7f4c95bb0e00 1 Processor -- start 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.119+0000 7f4c95bb0e00 1 -- start start 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.119+0000 7f4c95bb0e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 0x5575d601dfe0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.119+0000 7f4c95bb0e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d5fbd060 0x5575d5ffc790 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.119+0000 7f4c95bb0e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d6001b50 0x5575d5ffccd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.119+0000 7f4c95bb0e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5575d5ec6680 con 0x5575d5fbd060 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.119+0000 7f4c95bb0e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5575d5b86790 con 0x5575d6001b50 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.119+0000 7f4c95bb0e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5575d5eceab0 con 0x5575d5fbc750 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.119+0000 7f4c9192a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 0x5575d601dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.119+0000 7f4c91129640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d5fbd060 0x5575d5ffc790 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c9192a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 0x5575d601dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48848/0 (socket says 10.20.193.28:48848) 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c9192a640 1 -- 10.20.193.28:0/4141202756 learned_addr learned my addr 10.20.193.28:0/4141202756 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c9212b640 1 --2- 10.20.193.28:0/4141202756 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d6001b50 0x5575d5ffccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c9192a640 1 -- 10.20.193.28:0/4141202756 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d6001b50 msgr2=0x5575d5ffccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c9192a640 1 --2- 10.20.193.28:0/4141202756 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d6001b50 0x5575d5ffccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c9192a640 1 -- 10.20.193.28:0/4141202756 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d5fbd060 msgr2=0x5575d5ffc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c9192a640 1 --2- 10.20.193.28:0/4141202756 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d5fbd060 0x5575d5ffc790 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.121 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c9192a640 1 -- 10.20.193.28:0/4141202756 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575d5e58f70 con 0x5575d5fbc750 2026-02-25T11:55:27.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c9192a640 1 --2- 10.20.193.28:0/4141202756 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 0x5575d601dfe0 secure :-1 s=READY pgs=197 gs=1 cs=0 l=1 c_cookie=17184765179022009458 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4c7c008010 tx=0x7f4c7c016030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee7c8ce1ac1be472 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:27.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c59ce5640 1 -- 10.20.193.28:0/4141202756 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4c7c002b40 con 0x5575d5fbc750 2026-02-25T11:55:27.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c59ce5640 1 -- 10.20.193.28:0/4141202756 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4c7c032060 con 0x5575d5fbc750 2026-02-25T11:55:27.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c59ce5640 1 -- 10.20.193.28:0/4141202756 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4c7c020810 con 0x5575d5fbc750 2026-02-25T11:55:27.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/4141202756 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 msgr2=0x5575d601dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.120+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/4141202756 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 0x5575d601dfe0 secure :-1 s=READY pgs=197 gs=1 cs=0 l=1 c_cookie=17184765179022009458 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4c7c008010 tx=0x7f4c7c016030 comp rx=0 tx=0).stop 2026-02-25T11:55:27.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.121+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/4141202756 shutdown_connections 2026-02-25T11:55:27.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.121+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/4141202756 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d6001b50 0x5575d5ffccd0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.121+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/4141202756 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d5fbd060 0x5575d5ffc790 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.121+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/4141202756 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 0x5575d601dfe0 unknown :-1 s=CLOSED pgs=197 gs=1 cs=0 l=1 c_cookie=17184765179022009458 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.121+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/4141202756 >> 10.20.193.28:0/4141202756 conn(0x5575d5f9a8f0 msgr2=0x5575d601d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:27.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.121+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/4141202756 shutdown_connections 2026-02-25T11:55:27.122 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.121+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/4141202756 wait complete. 2026-02-25T11:55:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.121+0000 7f4c95bb0e00 1 -- start start 2026-02-25T11:55:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.121+0000 7f4c95bb0e00 1 Processor -- start 2026-02-25T11:55:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c95bb0e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 0x5575d5f9acc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c95bb0e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbd060 0x5575d5f9b200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c95bb0e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d6001b50 0x5575d5f9b740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c95bb0e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5575d5f9fd60 con 0x5575d6001b50 2026-02-25T11:55:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c95bb0e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5575d5f9fed0 con 0x5575d5fbd060 2026-02-25T11:55:27.123 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c95bb0e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5575d5fa0090 con 0x5575d5fbc750 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c9192a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 0x5575d5f9acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c9192a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 0x5575d5f9acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48858/0 (socket says 10.20.193.28:48858) 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c9192a640 1 -- 10.20.193.28:0/3506098661 learned_addr learned my addr 10.20.193.28:0/3506098661 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c9212b640 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d6001b50 0x5575d5f9b740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c9192a640 1 -- 10.20.193.28:0/3506098661 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbd060 msgr2=0x5575d5f9b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c91129640 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbd060 0x5575d5f9b200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c9192a640 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbd060 0x5575d5f9b200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c9192a640 1 -- 10.20.193.28:0/3506098661 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d6001b50 msgr2=0x5575d5f9b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c9192a640 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d6001b50 0x5575d5f9b740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c9192a640 1 -- 10.20.193.28:0/3506098661 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575d5e4cb40 con 0x5575d5fbc750 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c9212b640 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d6001b50 0x5575d5f9b740 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.122+0000 7f4c91129640 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbd060 0x5575d5f9b200 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.123+0000 7f4c9192a640 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 0x5575d5f9acc0 secure :-1 s=READY pgs=198 gs=2 cs=0 l=1 c_cookie=509285333141340167 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4c7c011f90 tx=0x7f4c7c004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=711584feb34c007 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.123+0000 7f4c477fe640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:55:27.124 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.123+0000 7f4c477fe640 1 -- 10.20.193.28:0/3506098661 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4c7c005c90 con 0x5575d5fbc750 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.123+0000 7f4c477fe640 1 -- 10.20.193.28:0/3506098661 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5575d5c20530 con 0x5575d5fbc750 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.123+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3506098661 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575d5e5e4f0 con 0x5575d5fbc750 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.123+0000 7f4c477fe640 1 -- 10.20.193.28:0/3506098661 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4c7c020c20 con 0x5575d5fbc750 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.123+0000 7f4c95bb0e00 10 client.26668 _ll_register_callbacks cb 0x5575d5e49e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.123+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3506098661 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c880d3800 con 0x5575d5fbc750 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.123+0000 7f4c477fe640 1 -- 10.20.193.28:0/3506098661 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4c7c020db0 con 0x5575d5fbc750 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.123+0000 7f4c477fe640 1 -- 10.20.193.28:0/3506098661 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4c7c006040 con 0x5575d5fbc750 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.123+0000 7f4c95bb0e00 10 client.26668 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.123+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3506098661 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5575d5e5e750 con 0x5575d5fbc750 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.123+0000 7f4c477fe640 1 -- 10.20.193.28:0/3506098661 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f4c7c03e030 con 0x5575d5fbc750 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.124+0000 7f4c95bb0e00 10 client.26668 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.124+0000 7f4c95bb0e00 10 client.26668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.124+0000 7f4c95bb0e00 20 client.26668 populate_metadata read hostname 'trial028' 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.124+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5575d60afb40 0x5575d60cfff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.124+0000 7f4c95bb0e00 4 client.26668 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.124+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3506098661 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5575d5d29690 con 0x5575d60afb40 2026-02-25T11:55:27.125 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.124+0000 7f4c91129640 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5575d60afb40 0x5575d60cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.124+0000 7f4c91129640 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5575d60afb40 0x5575d60cfff0 crc :-1 s=READY pgs=135 gs=4 cs=0 l=0 c_cookie=9905599414775236808 s_cookie=17433079138134788617 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8977c22f84fec8c8 server_cookie=f1eebd1c99cc0a09 in_seq=0 out_seq=0 2026-02-25T11:55:27.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.124+0000 7f4c477fe640 10 client.26668 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:27.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c477fe640 1 -- 10.20.193.28:0/3506098661 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5575d5d29690 con 0x5575d60afb40 2026-02-25T11:55:27.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c477fe640 10 client.26668 handle_command_reply: tid=0 2026-02-25T11:55:27.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c95bb0e00 1 client.26668 shutdown 2026-02-25T11:55:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3506098661 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 msgr2=0x5575d5f9acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 0x5575d5f9acc0 secure :-1 s=READY pgs=198 gs=2 cs=0 l=1 c_cookie=509285333141340167 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4c7c011f90 tx=0x7f4c7c004770 comp rx=0 tx=0).stop 2026-02-25T11:55:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3506098661 shutdown_connections 2026-02-25T11:55:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5575d60afb40 0x5575d60cfff0 crc :-1 s=READY pgs=135 gs=4 cs=0 l=0 c_cookie=9905599414775236808 s_cookie=17433079138134788617 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5575d6001b50 0x5575d5f9b740 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5575d5fbd060 0x5575d5f9b200 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c95bb0e00 1 --2- 10.20.193.28:0/3506098661 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5575d5fbc750 0x5575d5f9acc0 unknown :-1 s=CLOSED pgs=198 gs=2 cs=0 l=1 c_cookie=509285333141340167 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3506098661 >> 10.20.193.28:0/3506098661 conn(0x5575d5f9a8f0 msgr2=0x5575d6015fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c477fe640 0 client.26668 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3506098661 shutdown_connections 2026-02-25T11:55:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.125+0000 7f4c95bb0e00 1 -- 10.20.193.28:0/3506098661 wait complete. 2026-02-25T11:55:27.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.126+0000 7f4c95bb0e00 20 client.26668 trim_cache size 0 max 16384 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "850bcfd3-be8a-4930-896c-1ed6e48b8fed", 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.126+0000 7f4c9292c640 1 -- 10.20.193.28:0/1611403712 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4c680b5750 msgr2=0x7f4c680d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.126+0000 7f4c9292c640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4c680b5750 0x7f4c680d5c00 secure :-1 s=READY pgs=350 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4c7c006fd0 tx=0x7f4c7c008040 comp rx=0 tx=0).stop 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.126+0000 7f4c9292c640 1 -- 10.20.193.28:0/1611403712 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4c8c115840 msgr2=0x7f4c8c0f6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.126+0000 7f4c9292c640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4c8c115840 0x7f4c8c0f6030 secure :-1 s=READY pgs=739 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4c8800a980 tx=0x7f4c880039b0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.126+0000 7f4c9292c640 1 -- 10.20.193.28:0/1611403712 shutdown_connections 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.126+0000 7f4c9292c640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4c680b5750 0x7f4c680d5c00 unknown :-1 s=CLOSED pgs=350 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.126+0000 7f4c9292c640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4c8c115840 0x7f4c8c0f6030 unknown :-1 s=CLOSED pgs=739 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.126+0000 7f4c9292c640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4c8c110040 0x7f4c8c176110 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.126+0000 7f4c9292c640 1 --2- 10.20.193.28:0/1611403712 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4c8c10f650 0x7f4c8c0f5930 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.126+0000 7f4c9292c640 1 -- 10.20.193.28:0/1611403712 >> 10.20.193.28:0/1611403712 conn(0x7f4c8c0ecf00 msgr2=0x7f4c8c112ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.126+0000 7f4c9292c640 1 -- 10.20.193.28:0/1611403712 shutdown_connections 2026-02-25T11:55:27.128 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.127+0000 7f4c9292c640 1 -- 10.20.193.28:0/1611403712 wait complete. 2026-02-25T11:55:27.228 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:55:27.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:27 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:55:27.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:27 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:55:27.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:27 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:55:27.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:27 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:55:27.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:27 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:55:27.415 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:55:27.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.598+0000 7fef248d2640 1 -- 10.20.193.28:0/3017175870 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fef20060110 msgr2=0x7fef200604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.598+0000 7fef248d2640 1 --2- 10.20.193.28:0/3017175870 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fef20060110 0x7fef200604e0 secure :-1 s=READY pgs=742 gs=1 cs=0 l=1 c_cookie=15629082023346400588 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fef14009950 tx=0x7fef1402f190 comp rx=0 tx=0).stop 2026-02-25T11:55:27.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.599+0000 7fef248d2640 1 -- 10.20.193.28:0/3017175870 shutdown_connections 2026-02-25T11:55:27.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.599+0000 7fef248d2640 1 --2- 10.20.193.28:0/3017175870 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fef201135c0 0x7fef20173f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.599+0000 7fef248d2640 1 --2- 10.20.193.28:0/3017175870 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fef20060b00 0x7fef20112e10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.599+0000 7fef248d2640 1 --2- 10.20.193.28:0/3017175870 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fef20060110 0x7fef200604e0 unknown :-1 s=CLOSED pgs=742 gs=1 cs=0 l=1 c_cookie=15629082023346400588 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.599+0000 7fef248d2640 1 -- 10.20.193.28:0/3017175870 >> 10.20.193.28:0/3017175870 conn(0x7fef200e9b40 msgr2=0x7fef20109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.599+0000 7fef248d2640 1 -- 10.20.193.28:0/3017175870 shutdown_connections 2026-02-25T11:55:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.599+0000 7fef248d2640 1 -- 10.20.193.28:0/3017175870 wait complete. 2026-02-25T11:55:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.599+0000 7fef248d2640 1 Processor -- start 2026-02-25T11:55:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef248d2640 1 -- start start 2026-02-25T11:55:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef248d2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fef20060110 0x7fef200f4810 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef248d2640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fef20060b00 0x7fef20173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.604 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef248d2640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fef201135c0 0x7fef200f4f10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef248d2640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fef200fa120 con 0x7fef20060110 2026-02-25T11:55:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef248d2640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fef200fa290 con 0x7fef20060b00 2026-02-25T11:55:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef248d2640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fef200fa450 con 0x7fef201135c0 2026-02-25T11:55:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef1ed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fef20060110 0x7fef200f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef1ed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fef20060110 0x7fef200f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47458/0 (socket says 10.20.193.28:47458) 2026-02-25T11:55:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef1ed76640 1 -- 10.20.193.28:0/1845158852 learned_addr learned my addr 10.20.193.28:0/1845158852 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef1f577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fef201135c0 0x7fef200f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef1e575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fef20060b00 0x7fef20173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef1ed76640 1 -- 10.20.193.28:0/1845158852 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fef201135c0 msgr2=0x7fef200f4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef1ed76640 1 --2- 10.20.193.28:0/1845158852 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fef201135c0 0x7fef200f4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef1ed76640 1 -- 10.20.193.28:0/1845158852 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fef20060b00 msgr2=0x7fef20173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef1ed76640 1 --2- 10.20.193.28:0/1845158852 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fef20060b00 0x7fef20173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.600+0000 7fef1ed76640 1 -- 10.20.193.28:0/1845158852 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef14009570 con 0x7fef20060110 2026-02-25T11:55:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.601+0000 7fef1ed76640 1 --2- 10.20.193.28:0/1845158852 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fef20060110 0x7fef200f4810 secure :-1 s=READY pgs=743 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fef140098a0 tx=0x7fef140028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.601+0000 7feefffff640 1 -- 10.20.193.28:0/1845158852 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fef1403d070 con 0x7fef20060110 2026-02-25T11:55:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.601+0000 7feefffff640 1 -- 10.20.193.28:0/1845158852 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fef14002a70 con 0x7fef20060110 2026-02-25T11:55:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.601+0000 7feefffff640 1 -- 10.20.193.28:0/1845158852 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fef14041890 con 0x7fef20060110 2026-02-25T11:55:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.601+0000 7fef248d2640 1 -- 10.20.193.28:0/1845158852 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef200fa6d0 con 0x7fef20060110 2026-02-25T11:55:27.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.601+0000 7fef248d2640 1 -- 10.20.193.28:0/1845158852 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef200faaf0 con 0x7fef20060110 2026-02-25T11:55:27.607 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.604+0000 7feefffff640 1 -- 10.20.193.28:0/1845158852 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fef14041a20 con 0x7fef20060110 2026-02-25T11:55:27.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.605+0000 7feefffff640 1 --2- 10.20.193.28:0/1845158852 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7feef40b56d0 0x7feef40d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.605+0000 7feefffff640 1 -- 10.20.193.28:0/1845158852 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fef1410afd0 con 0x7fef20060110 2026-02-25T11:55:27.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.605+0000 7fef1e575640 1 --2- 10.20.193.28:0/1845158852 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7feef40b56d0 0x7feef40d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.608 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.606+0000 7fef1e575640 1 --2- 10.20.193.28:0/1845158852 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7feef40b56d0 0x7feef40d5b80 secure :-1 s=READY pgs=351 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fef200fb1b0 tx=0x7fef0800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.607+0000 7fef270cee00 1 Processor -- start 2026-02-25T11:55:27.609 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.607+0000 7fef270cee00 1 -- start start 2026-02-25T11:55:27.610 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.607+0000 7fef270cee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x5617353a3510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.607+0000 7fef270cee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 0x5617353e81b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.607+0000 7fef270cee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353e86f0 0x561735408ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.607+0000 7fef270cee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5617352b99b0 con 0x5617353a3140 2026-02-25T11:55:27.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.607+0000 7fef270cee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5617352b56e0 con 0x5617353e86f0 2026-02-25T11:55:27.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.607+0000 7fef270cee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5617352af840 con 0x5617353a3a50 2026-02-25T11:55:27.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.607+0000 7fef1e575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 0x5617353e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.607+0000 7fef1ed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x5617353a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.607+0000 7fef1e575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 0x5617353e81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48888/0 (socket says 10.20.193.28:48888) 2026-02-25T11:55:27.611 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.607+0000 7fef1e575640 1 -- 10.20.193.28:0/2543444640 learned_addr learned my addr 10.20.193.28:0/2543444640 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:27.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.608+0000 7fef1f577640 1 --2- 10.20.193.28:0/2543444640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353e86f0 0x561735408ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.608+0000 7fef1e575640 1 -- 10.20.193.28:0/2543444640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353e86f0 msgr2=0x561735408ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.608+0000 7fef1e575640 1 --2- 10.20.193.28:0/2543444640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353e86f0 0x561735408ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.608+0000 7fef1e575640 1 -- 10.20.193.28:0/2543444640 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 msgr2=0x5617353a3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.608+0000 7fef1e575640 1 --2- 10.20.193.28:0/2543444640 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x5617353a3510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.608+0000 7fef1e575640 1 -- 10.20.193.28:0/2543444640 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56173525ad80 con 0x5617353a3a50 2026-02-25T11:55:27.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.608+0000 7fef1ed76640 1 --2- 10.20.193.28:0/2543444640 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x5617353a3510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:55:27.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.608+0000 7fef1e575640 1 --2- 10.20.193.28:0/2543444640 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 0x5617353e81b0 secure :-1 s=READY pgs=199 gs=2 cs=0 l=1 c_cookie=3608380536990815288 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fef08011440 tx=0x7fef08013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32138afaf9c55438 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:27.612 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.608+0000 7feeeb7fe640 1 -- 10.20.193.28:0/2543444640 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fef0801e640 con 0x5617353a3a50 2026-02-25T11:55:27.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.608+0000 7feeeb7fe640 1 -- 10.20.193.28:0/2543444640 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fef08015070 con 0x5617353a3a50 2026-02-25T11:55:27.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.608+0000 7feeeb7fe640 1 -- 10.20.193.28:0/2543444640 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fef0801cc60 con 0x5617353a3a50 2026-02-25T11:55:27.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.608+0000 7fef270cee00 1 -- 10.20.193.28:0/2543444640 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 msgr2=0x5617353e81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.608+0000 7fef270cee00 1 --2- 10.20.193.28:0/2543444640 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 0x5617353e81b0 secure :-1 s=READY pgs=199 gs=2 cs=0 l=1 c_cookie=3608380536990815288 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fef08011440 tx=0x7fef08013000 comp rx=0 tx=0).stop 2026-02-25T11:55:27.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.609+0000 7fef270cee00 1 -- 10.20.193.28:0/2543444640 shutdown_connections 2026-02-25T11:55:27.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.609+0000 7fef270cee00 1 --2- 10.20.193.28:0/2543444640 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353e86f0 0x561735408ad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.609+0000 7fef270cee00 1 --2- 10.20.193.28:0/2543444640 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 0x5617353e81b0 unknown :-1 s=CLOSED pgs=199 gs=2 cs=0 l=1 c_cookie=3608380536990815288 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.609+0000 7fef270cee00 1 --2- 10.20.193.28:0/2543444640 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x5617353a3510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.609+0000 7fef270cee00 1 -- 10.20.193.28:0/2543444640 >> 10.20.193.28:0/2543444640 conn(0x5617352c2d40 msgr2=0x5617353a1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:27.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.609+0000 7fef270cee00 1 -- 10.20.193.28:0/2543444640 shutdown_connections 2026-02-25T11:55:27.613 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.609+0000 7fef270cee00 1 -- 10.20.193.28:0/2543444640 wait complete. 2026-02-25T11:55:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.610+0000 7fef270cee00 1 -- start start 2026-02-25T11:55:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.610+0000 7fef270cee00 1 Processor -- start 2026-02-25T11:55:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef270cee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x561735400410 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.617 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef270cee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 0x561735400950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef270cee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353e86f0 0x561735405940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef270cee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x561735405e80 con 0x5617353a3140 2026-02-25T11:55:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef270cee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561735405ff0 con 0x5617353e86f0 2026-02-25T11:55:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef270cee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5617354061b0 con 0x5617353a3a50 2026-02-25T11:55:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef1ed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x561735400410 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef1e575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 0x561735400950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef1e575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 0x561735400950 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48894/0 (socket says 10.20.193.28:48894) 2026-02-25T11:55:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef1e575640 1 -- 10.20.193.28:0/2224614614 learned_addr learned my addr 10.20.193.28:0/2224614614 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef1f577640 1 --2- 10.20.193.28:0/2224614614 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353e86f0 0x561735405940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef1ed76640 1 -- 10.20.193.28:0/2224614614 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 msgr2=0x561735400950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.618 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef1ed76640 1 --2- 10.20.193.28:0/2224614614 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 0x561735400950 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef1ed76640 1 -- 10.20.193.28:0/2224614614 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353e86f0 msgr2=0x561735405940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef1ed76640 1 --2- 10.20.193.28:0/2224614614 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353e86f0 0x561735405940 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef1ed76640 1 -- 10.20.193.28:0/2224614614 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0800bd10 con 0x5617353a3140 2026-02-25T11:55:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.611+0000 7fef1e575640 1 --2- 10.20.193.28:0/2224614614 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 0x561735400950 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:55:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.612+0000 7fef1ed76640 1 --2- 10.20.193.28:0/2224614614 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x561735400410 secure :-1 s=READY pgs=744 gs=1 cs=0 l=1 c_cookie=4503078319773275687 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fef14108ac0 tx=0x7fef1402f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7e25df9dd95627 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.612+0000 7feee8ff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.612+0000 7feee8ff9640 1 -- 10.20.193.28:0/2224614614 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fef140d0c00 con 0x5617353a3140 2026-02-25T11:55:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.612+0000 7feee8ff9640 1 -- 10.20.193.28:0/2224614614 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fef14041d00 con 0x5617353a3140 2026-02-25T11:55:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.612+0000 7feee8ff9640 1 -- 10.20.193.28:0/2224614614 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fef140d2fc0 con 0x5617353a3140 2026-02-25T11:55:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.612+0000 7fef270cee00 1 -- 10.20.193.28:0/2224614614 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef200faaf0 con 0x5617353a3140 2026-02-25T11:55:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.612+0000 7fef270cee00 10 client.16980 _ll_register_callbacks cb 0x5617352309a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:27.619 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.612+0000 7fef270cee00 1 -- 10.20.193.28:0/2224614614 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561734f456f0 con 0x5617353a3140 2026-02-25T11:55:27.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.612+0000 7feee8ff9640 1 -- 10.20.193.28:0/2224614614 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fef140d7070 con 0x5617353a3140 2026-02-25T11:55:27.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.612+0000 7feee8ff9640 1 -- 10.20.193.28:0/2224614614 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fef140d9650 con 0x5617353a3140 2026-02-25T11:55:27.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.612+0000 7fef270cee00 10 client.16980 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:27.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.612+0000 7fef270cee00 1 -- 10.20.193.28:0/2224614614 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x561735233350 con 0x5617353a3140 2026-02-25T11:55:27.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.613+0000 7feee8ff9640 1 -- 10.20.193.28:0/2224614614 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fef14092050 con 0x5617353a3140 2026-02-25T11:55:27.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.613+0000 7fef270cee00 10 client.16980 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:27.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.613+0000 7fef270cee00 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:27.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.613+0000 7fef270cee00 20 client.16980 populate_metadata read hostname 'trial028' 2026-02-25T11:55:27.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.613+0000 7fef270cee00 1 --2- 10.20.193.28:0/2224614614 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x561735408800 0x5617353e6420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.613+0000 7fef270cee00 4 client.16980 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:55:27.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.613+0000 7fef270cee00 1 -- 10.20.193.28:0/2224614614 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561735281910 con 0x561735408800 2026-02-25T11:55:27.620 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.613+0000 7fef1e575640 1 --2- 10.20.193.28:0/2224614614 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x561735408800 0x5617353e6420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7fef1e575640 1 --2- 10.20.193.28:0/2224614614 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x561735408800 0x5617353e6420 crc :-1 s=READY pgs=136 gs=4 cs=0 l=0 c_cookie=2341346001031250032 s_cookie=16468926801207130368 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=207e21c86fbfc070 server_cookie=e48d61ab69dc7900 in_seq=0 out_seq=0 2026-02-25T11:55:27.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7feee8ff9640 10 client.16980 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:27.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7feee8ff9640 1 -- 10.20.193.28:0/2224614614 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x561735281910 con 0x561735408800 2026-02-25T11:55:27.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7feee8ff9640 10 client.16980 handle_command_reply: tid=0 2026-02-25T11:55:27.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7fef270cee00 1 client.16980 shutdown 2026-02-25T11:55:27.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7fef270cee00 1 -- 10.20.193.28:0/2224614614 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 msgr2=0x561735400410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7fef270cee00 1 --2- 10.20.193.28:0/2224614614 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x561735400410 secure :-1 s=READY pgs=744 gs=1 cs=0 l=1 c_cookie=4503078319773275687 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fef14108ac0 tx=0x7fef1402f190 comp rx=0 tx=0).stop 2026-02-25T11:55:27.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7fef270cee00 1 -- 10.20.193.28:0/2224614614 shutdown_connections 2026-02-25T11:55:27.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7fef270cee00 1 --2- 10.20.193.28:0/2224614614 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x561735408800 0x5617353e6420 crc :-1 s=READY pgs=136 gs=4 cs=0 l=0 c_cookie=2341346001031250032 s_cookie=16468926801207130368 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7fef270cee00 1 --2- 10.20.193.28:0/2224614614 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353e86f0 0x561735405940 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7fef270cee00 1 --2- 10.20.193.28:0/2224614614 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353a3a50 0x561735400950 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.621 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7fef270cee00 1 --2- 10.20.193.28:0/2224614614 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x561735400410 unknown :-1 s=CLOSED pgs=744 gs=1 cs=0 l=1 c_cookie=4503078319773275687 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7fef270cee00 1 -- 10.20.193.28:0/2224614614 >> 10.20.193.28:0/2224614614 conn(0x5617352c2d40 msgr2=0x5617353a1230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:27.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.614+0000 7feee8ff9640 0 client.16980 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:27.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.615+0000 7fef270cee00 1 -- 10.20.193.28:0/2224614614 shutdown_connections 2026-02-25T11:55:27.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.615+0000 7fef270cee00 1 -- 10.20.193.28:0/2224614614 wait complete. 2026-02-25T11:55:27.622 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.615+0000 7fef270cee00 20 client.16980 trim_cache size 0 max 16384 2026-02-25T11:55:27.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.624+0000 7fef270cee00 1 Processor -- start 2026-02-25T11:55:27.626 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.624+0000 7fef270cee00 1 -- start start 2026-02-25T11:55:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.624+0000 7fef270cee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353a3140 0x5617353eb1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.624+0000 7fef270cee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3a50 0x5617353fd5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.624+0000 7fef270cee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353e86f0 0x5617353fdb20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.624+0000 7fef270cee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5617352ad470 con 0x5617353a3a50 2026-02-25T11:55:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.624+0000 7fef270cee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561734f6d700 con 0x5617353a3140 2026-02-25T11:55:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.624+0000 7fef270cee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x561735406070 con 0x5617353e86f0 2026-02-25T11:55:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.624+0000 7fef1e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3a50 0x5617353fd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef1e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3a50 0x5617353fd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47488/0 (socket says 10.20.193.28:47488) 2026-02-25T11:55:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef1e575640 1 -- 10.20.193.28:0/1321123896 learned_addr learned my addr 10.20.193.28:0/1321123896 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:27.627 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef1f577640 1 --2- 10.20.193.28:0/1321123896 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353e86f0 0x5617353fdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef1ed76640 1 --2- 10.20.193.28:0/1321123896 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353a3140 0x5617353eb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef1e575640 1 -- 10.20.193.28:0/1321123896 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353e86f0 msgr2=0x5617353fdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef1e575640 1 --2- 10.20.193.28:0/1321123896 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353e86f0 0x5617353fdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef1e575640 1 -- 10.20.193.28:0/1321123896 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353a3140 msgr2=0x5617353eb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef1e575640 1 --2- 10.20.193.28:0/1321123896 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353a3140 0x5617353eb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef1e575640 1 -- 10.20.193.28:0/1321123896 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561735245150 con 0x5617353a3a50 2026-02-25T11:55:27.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef1f577640 1 --2- 10.20.193.28:0/1321123896 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353e86f0 0x5617353fdb20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:55:27.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef1ed76640 1 --2- 10.20.193.28:0/1321123896 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353a3140 0x5617353eb1e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:55:27.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef1e575640 1 --2- 10.20.193.28:0/1321123896 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3a50 0x5617353fd5e0 secure :-1 s=READY pgs=745 gs=2 cs=0 l=1 c_cookie=8113854361307056934 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fef08011440 tx=0x7fef0801c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709a338d74f60b26 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:27.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7feeeaffd640 1 -- 10.20.193.28:0/1321123896 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fef0802a040 con 0x5617353a3a50 2026-02-25T11:55:27.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7feeeaffd640 1 -- 10.20.193.28:0/1321123896 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fef0802b040 con 0x5617353a3a50 2026-02-25T11:55:27.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7feeeaffd640 1 -- 10.20.193.28:0/1321123896 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fef08004700 con 0x5617353a3a50 2026-02-25T11:55:27.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef270cee00 1 -- 10.20.193.28:0/1321123896 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3a50 msgr2=0x5617353fd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.625+0000 7fef270cee00 1 --2- 10.20.193.28:0/1321123896 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3a50 0x5617353fd5e0 secure :-1 s=READY pgs=745 gs=2 cs=0 l=1 c_cookie=8113854361307056934 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fef08011440 tx=0x7fef0801c7b0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.626+0000 7fef270cee00 1 -- 10.20.193.28:0/1321123896 shutdown_connections 2026-02-25T11:55:27.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.626+0000 7fef270cee00 1 --2- 10.20.193.28:0/1321123896 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353e86f0 0x5617353fdb20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.626+0000 7fef270cee00 1 --2- 10.20.193.28:0/1321123896 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3a50 0x5617353fd5e0 unknown :-1 s=CLOSED pgs=745 gs=2 cs=0 l=1 c_cookie=8113854361307056934 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.626+0000 7fef270cee00 1 --2- 10.20.193.28:0/1321123896 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353a3140 0x5617353eb1e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.626+0000 7fef270cee00 1 -- 10.20.193.28:0/1321123896 >> 10.20.193.28:0/1321123896 conn(0x5617352c2d40 msgr2=0x5617353eaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:27.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.626+0000 7fef270cee00 1 -- 10.20.193.28:0/1321123896 shutdown_connections 2026-02-25T11:55:27.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.626+0000 7fef270cee00 1 -- 10.20.193.28:0/1321123896 wait complete. 2026-02-25T11:55:27.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.626+0000 7fef270cee00 1 -- start start 2026-02-25T11:55:27.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.626+0000 7fef270cee00 1 Processor -- start 2026-02-25T11:55:27.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef270cee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x561735386ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef270cee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353a3a50 0x561735387010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef270cee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353e86f0 0x5617353e3230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef270cee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56173537e490 con 0x5617353a3140 2026-02-25T11:55:27.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef270cee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5617353e3930 con 0x5617353a3a50 2026-02-25T11:55:27.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef270cee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5617353e3af0 con 0x5617353e86f0 2026-02-25T11:55:27.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef1ed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x561735386ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef1ed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x561735386ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47492/0 (socket says 10.20.193.28:47492) 2026-02-25T11:55:27.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef1ed76640 1 -- 10.20.193.28:0/2760778113 learned_addr learned my addr 10.20.193.28:0/2760778113 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:27.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef1e575640 1 --2- 10.20.193.28:0/2760778113 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353a3a50 0x561735387010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef1f577640 1 --2- 10.20.193.28:0/2760778113 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353e86f0 0x5617353e3230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef1ed76640 1 -- 10.20.193.28:0/2760778113 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353e86f0 msgr2=0x5617353e3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef1ed76640 1 --2- 10.20.193.28:0/2760778113 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353e86f0 0x5617353e3230 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef1ed76640 1 -- 10.20.193.28:0/2760778113 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353a3a50 msgr2=0x561735387010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:27.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef1ed76640 1 --2- 10.20.193.28:0/2760778113 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353a3a50 0x561735387010 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef1ed76640 1 -- 10.20.193.28:0/2760778113 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56173523fbd0 con 0x5617353a3140 2026-02-25T11:55:27.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef1ed76640 1 --2- 10.20.193.28:0/2760778113 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x561735386ad0 secure :-1 s=READY pgs=746 gs=1 cs=0 l=1 c_cookie=5459526896112909674 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fef140419f0 tx=0x7fef14038f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc422c944803d6a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:27.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7feee8ff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:27.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7feee8ff9640 1 -- 10.20.193.28:0/2760778113 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fef140d22a0 con 0x5617353a3140 2026-02-25T11:55:27.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7feee8ff9640 1 -- 10.20.193.28:0/2760778113 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x561734f456f0 con 0x5617353a3140 2026-02-25T11:55:27.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7feee8ff9640 1 -- 10.20.193.28:0/2760778113 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fef1407f340 con 0x5617353a3140 2026-02-25T11:55:27.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.627+0000 7fef270cee00 1 -- 10.20.193.28:0/2760778113 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56173525ad80 con 0x5617353a3140 2026-02-25T11:55:27.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7fef270cee00 10 client.16986 _ll_register_callbacks cb 0x5617352309a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:27.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7fef270cee00 1 -- 10.20.193.28:0/2760778113 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef08015070 con 0x5617353a3140 2026-02-25T11:55:27.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7feee8ff9640 1 -- 10.20.193.28:0/2760778113 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fef140d7070 con 0x5617353a3140 2026-02-25T11:55:27.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7feee8ff9640 1 -- 10.20.193.28:0/2760778113 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fef140dc490 con 0x5617353a3140 2026-02-25T11:55:27.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7fef270cee00 10 client.16986 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:27.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7fef270cee00 1 -- 10.20.193.28:0/2760778113 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x561735233680 con 0x5617353a3140 2026-02-25T11:55:27.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7feee8ff9640 1 -- 10.20.193.28:0/2760778113 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fef14092020 con 0x5617353a3140 2026-02-25T11:55:27.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7fef270cee00 10 client.16986 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:27.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7fef270cee00 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:27.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7fef270cee00 20 client.16986 populate_metadata read hostname 'trial028' 2026-02-25T11:55:27.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7fef270cee00 1 --2- 10.20.193.28:0/2760778113 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x561735438180 0x5617354385f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:27.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7fef270cee00 4 client.16986 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:55:27.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7fef270cee00 1 -- 10.20.193.28:0/2760778113 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561734f42f40 con 0x561735438180 2026-02-25T11:55:27.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.628+0000 7fef1e575640 1 --2- 10.20.193.28:0/2760778113 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x561735438180 0x5617354385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:27.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.629+0000 7fef1e575640 1 --2- 10.20.193.28:0/2760778113 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x561735438180 0x5617354385f0 crc :-1 s=READY pgs=137 gs=4 cs=0 l=0 c_cookie=13717876757424108165 s_cookie=3123216259654136618 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be5fb03afed9f285 server_cookie=2b57e4a40168b32a in_seq=0 out_seq=0 2026-02-25T11:55:27.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.629+0000 7feee8ff9640 10 client.16986 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:27.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.629+0000 7feee8ff9640 1 -- 10.20.193.28:0/2760778113 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x561734f42f40 con 0x561735438180 2026-02-25T11:55:27.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.629+0000 7feee8ff9640 10 client.16986 handle_command_reply: tid=0 2026-02-25T11:55:27.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.629+0000 7fef270cee00 1 client.16986 shutdown 2026-02-25T11:55:27.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.629+0000 7fef270cee00 1 -- 10.20.193.28:0/2760778113 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 msgr2=0x561735386ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.629+0000 7fef270cee00 1 --2- 10.20.193.28:0/2760778113 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x561735386ad0 secure :-1 s=READY pgs=746 gs=1 cs=0 l=1 c_cookie=5459526896112909674 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fef140419f0 tx=0x7fef14038f30 comp rx=0 tx=0).stop 2026-02-25T11:55:27.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7fef270cee00 1 -- 10.20.193.28:0/2760778113 shutdown_connections 2026-02-25T11:55:27.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7fef270cee00 1 --2- 10.20.193.28:0/2760778113 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x561735438180 0x5617354385f0 crc :-1 s=READY pgs=137 gs=4 cs=0 l=0 c_cookie=13717876757424108165 s_cookie=3123216259654136618 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7fef270cee00 1 --2- 10.20.193.28:0/2760778113 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617353e86f0 0x5617353e3230 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7fef270cee00 1 --2- 10.20.193.28:0/2760778113 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617353a3a50 0x561735387010 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7feee8ff9640 0 client.16986 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:27.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7fef270cee00 1 --2- 10.20.193.28:0/2760778113 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617353a3140 0x561735386ad0 unknown :-1 s=CLOSED pgs=746 gs=1 cs=0 l=1 c_cookie=5459526896112909674 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7fef270cee00 1 -- 10.20.193.28:0/2760778113 >> 10.20.193.28:0/2760778113 conn(0x5617352c2d40 msgr2=0x5617354047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:27.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7fef270cee00 1 -- 10.20.193.28:0/2760778113 shutdown_connections 2026-02-25T11:55:27.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7fef270cee00 1 -- 10.20.193.28:0/2760778113 wait complete. 2026-02-25T11:55:27.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7fef270cee00 20 client.16986 trim_cache size 0 max 16384 2026-02-25T11:55:27.640 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:55:27.640 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:55:27.640 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:55:27.640 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:55:27.640 INFO:teuthology.orchestra.run.trial028.stdout: "850bcfd3-be8a-4930-896c-1ed6e48b8fed": { 2026-02-25T11:55:27.640 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:27.124813+0000", 2026-02-25T11:55:27.640 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.641 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.641 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.641 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.641 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.641 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.641 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:55:27.641 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:55:27.641 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.641 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.641 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.641 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.642 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.642 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.642 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:55:27.642 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:55:27.642 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.642 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.642 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.642 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.642 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.642 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.642 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:55:27.642 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:55:27.643 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.643 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.643 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.643 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.643 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.643 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.643 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:55:27.643 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:55:27.643 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.643 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.643 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.643 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.644 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.644 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.644 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:55:27.644 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:55:27.644 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.644 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.644 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.644 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.644 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.644 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.644 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:55:27.644 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:55:27.645 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.645 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.645 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.645 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.645 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.645 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.645 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:55:27.645 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:55:27.645 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.645 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.645 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.645 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.645 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.646 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.646 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:55:27.646 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:55:27.646 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.646 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.646 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.646 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.646 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.646 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.646 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:55:27.646 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:55:27.646 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.647 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.647 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.647 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.647 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.647 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.647 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:55:27.647 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:55:27.647 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.647 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.647 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.647 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.647 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.647 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.648 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:27.649 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:27.650 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:27.650 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:27.650 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:27.650 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:55:27.650 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:55:27.650 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:55:27.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7fef248d2640 1 -- 10.20.193.28:0/1845158852 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7feef40b56d0 msgr2=0x7feef40d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7fef248d2640 1 --2- 10.20.193.28:0/1845158852 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7feef40b56d0 0x7feef40d5b80 secure :-1 s=READY pgs=351 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fef200fb1b0 tx=0x7fef0800b410 comp rx=0 tx=0).stop 2026-02-25T11:55:27.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7fef248d2640 1 -- 10.20.193.28:0/1845158852 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fef20060110 msgr2=0x7fef200f4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:27.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.630+0000 7fef248d2640 1 --2- 10.20.193.28:0/1845158852 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fef20060110 0x7fef200f4810 secure :-1 s=READY pgs=743 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fef140098a0 tx=0x7fef140028d0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.631+0000 7fef248d2640 1 -- 10.20.193.28:0/1845158852 shutdown_connections 2026-02-25T11:55:27.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.631+0000 7fef248d2640 1 --2- 10.20.193.28:0/1845158852 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7feef40b56d0 0x7feef40d5b80 unknown :-1 s=CLOSED pgs=351 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.631+0000 7fef248d2640 1 --2- 10.20.193.28:0/1845158852 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fef201135c0 0x7fef200f4f10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.631+0000 7fef248d2640 1 --2- 10.20.193.28:0/1845158852 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fef20060b00 0x7fef20173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.631+0000 7fef248d2640 1 --2- 10.20.193.28:0/1845158852 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fef20060110 0x7fef200f4810 unknown :-1 s=CLOSED pgs=743 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:27.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.631+0000 7fef248d2640 1 -- 10.20.193.28:0/1845158852 >> 10.20.193.28:0/1845158852 conn(0x7fef200e9b40 msgr2=0x7fef2010fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:27.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.633+0000 7fef248d2640 1 -- 10.20.193.28:0/1845158852 shutdown_connections 2026-02-25T11:55:27.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:27.633+0000 7fef248d2640 1 -- 10.20.193.28:0/1845158852 wait complete. 2026-02-25T11:55:27.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:27 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:55:27.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:27 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:55:27.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:27 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:55:27.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:27 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:55:27.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:27 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:55:27.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:27 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:55:27.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:27 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:55:27.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:27 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:55:27.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:27 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:55:27.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:27 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:55:27.770 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:55:27.771 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:55:27.954 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:55:28.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.123+0000 7f4d60fe9640 1 -- 10.20.193.28:0/308046635 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d5c10b360 msgr2=0x7f4d5c10b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.123+0000 7f4d60fe9640 1 --2- 10.20.193.28:0/308046635 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d5c10b360 0x7f4d5c10b730 secure :-1 s=READY pgs=747 gs=1 cs=0 l=1 c_cookie=14747149554492482569 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4d48009950 tx=0x7f4d4802f190 comp rx=0 tx=0).stop 2026-02-25T11:55:28.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.123+0000 7f4d60fe9640 1 -- 10.20.193.28:0/308046635 shutdown_connections 2026-02-25T11:55:28.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.123+0000 7f4d60fe9640 1 --2- 10.20.193.28:0/308046635 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4d5c111370 0x7f4d5c111d50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.123+0000 7f4d60fe9640 1 --2- 10.20.193.28:0/308046635 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4d5c10bd50 0x7f4d5c110be0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.123+0000 7f4d60fe9640 1 --2- 10.20.193.28:0/308046635 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d5c10b360 0x7f4d5c10b730 unknown :-1 s=CLOSED pgs=747 gs=1 cs=0 l=1 c_cookie=14747149554492482569 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.126 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.123+0000 7f4d60fe9640 1 -- 10.20.193.28:0/308046635 >> 10.20.193.28:0/308046635 conn(0x7f4d5c0e8e70 msgr2=0x7f4d5c109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:28.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.123+0000 7f4d60fe9640 1 -- 10.20.193.28:0/308046635 shutdown_connections 2026-02-25T11:55:28.127 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.124+0000 7f4d60fe9640 1 -- 10.20.193.28:0/308046635 wait complete. 2026-02-25T11:55:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.125+0000 7f4d60fe9640 1 Processor -- start 2026-02-25T11:55:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.125+0000 7f4d60fe9640 1 -- start start 2026-02-25T11:55:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.125+0000 7f4d60fe9640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4d5c10b360 0x7f4d5c1236d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.125+0000 7f4d60fe9640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4d5c10bd50 0x7f4d5c123c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.125+0000 7f4d60fe9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d5c111370 0x7f4d5c128fe0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.125+0000 7f4d60fe9640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c129600 con 0x7f4d5c111370 2026-02-25T11:55:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.125+0000 7f4d60fe9640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c129770 con 0x7f4d5c10bd50 2026-02-25T11:55:28.129 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.125+0000 7f4d60fe9640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d5c129930 con 0x7f4d5c10b360 2026-02-25T11:55:28.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.125+0000 7f4d5bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d5c111370 0x7f4d5c128fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.125+0000 7f4d5bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d5c111370 0x7f4d5c128fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47520/0 (socket says 10.20.193.28:47520) 2026-02-25T11:55:28.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.125+0000 7f4d5bfff640 1 -- 10.20.193.28:0/192893221 learned_addr learned my addr 10.20.193.28:0/192893221 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:28.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.125+0000 7f4d5affd640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4d5c10bd50 0x7f4d5c123c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.125+0000 7f4d5b7fe640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4d5c10b360 0x7f4d5c1236d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.126+0000 7f4d5bfff640 1 -- 10.20.193.28:0/192893221 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4d5c10b360 msgr2=0x7f4d5c1236d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.126+0000 7f4d5bfff640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4d5c10b360 0x7f4d5c1236d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.126+0000 7f4d5bfff640 1 -- 10.20.193.28:0/192893221 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4d5c10bd50 msgr2=0x7f4d5c123c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.126+0000 7f4d5bfff640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4d5c10bd50 0x7f4d5c123c10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.126+0000 7f4d5bfff640 1 -- 10.20.193.28:0/192893221 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d48009570 con 0x7f4d5c111370 2026-02-25T11:55:28.130 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.126+0000 7f4d5bfff640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d5c111370 0x7f4d5c128fe0 secure :-1 s=READY pgs=748 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4d50006e90 tx=0x7f4d5000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:28.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.126+0000 7f4d58ff9640 1 -- 10.20.193.28:0/192893221 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4d50005500 con 0x7f4d5c111370 2026-02-25T11:55:28.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.126+0000 7f4d58ff9640 1 -- 10.20.193.28:0/192893221 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4d50005df0 con 0x7f4d5c111370 2026-02-25T11:55:28.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.126+0000 7f4d58ff9640 1 -- 10.20.193.28:0/192893221 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4d50003a00 con 0x7f4d5c111370 2026-02-25T11:55:28.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.126+0000 7f4d60fe9640 1 -- 10.20.193.28:0/192893221 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5c129c10 con 0x7f4d5c111370 2026-02-25T11:55:28.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.126+0000 7f4d60fe9640 1 -- 10.20.193.28:0/192893221 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d5c16a310 con 0x7f4d5c111370 2026-02-25T11:55:28.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.127+0000 7f4d60fe9640 1 -- 10.20.193.28:0/192893221 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d5c129e30 con 0x7f4d5c111370 2026-02-25T11:55:28.131 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.129+0000 7f4d58ff9640 1 -- 10.20.193.28:0/192893221 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f4d50003b60 con 0x7f4d5c111370 2026-02-25T11:55:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.131+0000 7f4d58ff9640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4d300b5760 0x7f4d300d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.131+0000 7f4d58ff9640 1 -- 10.20.193.28:0/192893221 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4d500e7a60 con 0x7f4d5c111370 2026-02-25T11:55:28.133 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.131+0000 7f4d5b7fe640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4d300b5760 0x7f4d300d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.131+0000 7f4d58ff9640 1 -- 10.20.193.28:0/192893221 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f4d50120940 con 0x7f4d5c111370 2026-02-25T11:55:28.134 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.132+0000 7f4d5b7fe640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4d300b5760 0x7f4d300d5c10 secure :-1 s=READY pgs=352 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4d480098a0 tx=0x7f4d4803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:28.272 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:28 trial028 ceph-mon[39224]: pgmap v443: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 11 op/s 2026-02-25T11:55:28.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.272+0000 7f4d60fe9640 1 -- 10.20.193.28:0/192893221 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d5c124830 con 0x7f4d5c111370 2026-02-25T11:55:28.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.273+0000 7f4d58ff9640 1 -- 10.20.193.28:0/192893221 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f4d500aa4e0 con 0x7f4d5c111370 2026-02-25T11:55:28.275 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:55:28.275 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:55:28.276 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:55:28.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.274+0000 7f4d60fe9640 1 -- 10.20.193.28:0/192893221 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4d300b5760 msgr2=0x7f4d300d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.274+0000 7f4d60fe9640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4d300b5760 0x7f4d300d5c10 secure :-1 s=READY pgs=352 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4d480098a0 tx=0x7f4d4803a040 comp rx=0 tx=0).stop 2026-02-25T11:55:28.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.274+0000 7f4d60fe9640 1 -- 10.20.193.28:0/192893221 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d5c111370 msgr2=0x7f4d5c128fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.274+0000 7f4d60fe9640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d5c111370 0x7f4d5c128fe0 secure :-1 s=READY pgs=748 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4d50006e90 tx=0x7f4d5000edf0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.274+0000 7f4d60fe9640 1 -- 10.20.193.28:0/192893221 shutdown_connections 2026-02-25T11:55:28.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.274+0000 7f4d60fe9640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4d300b5760 0x7f4d300d5c10 unknown :-1 s=CLOSED pgs=352 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.274+0000 7f4d60fe9640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4d5c111370 0x7f4d5c128fe0 unknown :-1 s=CLOSED pgs=748 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.274+0000 7f4d60fe9640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4d5c10bd50 0x7f4d5c123c10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.274+0000 7f4d60fe9640 1 --2- 10.20.193.28:0/192893221 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4d5c10b360 0x7f4d5c1236d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.274+0000 7f4d60fe9640 1 -- 10.20.193.28:0/192893221 >> 10.20.193.28:0/192893221 conn(0x7f4d5c0e8e70 msgr2=0x7f4d5c109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:28.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.274+0000 7f4d60fe9640 1 -- 10.20.193.28:0/192893221 shutdown_connections 2026-02-25T11:55:28.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.274+0000 7f4d60fe9640 1 -- 10.20.193.28:0/192893221 wait complete. 2026-02-25T11:55:28.379 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:55:28.563 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:55:28.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.739+0000 7f4f1e048640 1 -- 10.20.193.28:0/1442070442 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f18110130 msgr2=0x7f4f18115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.739+0000 7f4f1e048640 1 --2- 10.20.193.28:0/1442070442 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f18110130 0x7f4f18115180 secure :-1 s=READY pgs=749 gs=1 cs=0 l=1 c_cookie=6513871840429088378 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f08009950 tx=0x7f4f0802f190 comp rx=0 tx=0).stop 2026-02-25T11:55:28.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.739+0000 7f4f1e048640 1 -- 10.20.193.28:0/1442070442 shutdown_connections 2026-02-25T11:55:28.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.739+0000 7f4f1e048640 1 --2- 10.20.193.28:0/1442070442 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4f18115930 0x7f4f18176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.744 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.739+0000 7f4f1e048640 1 --2- 10.20.193.28:0/1442070442 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f18110130 0x7f4f18115180 unknown :-1 s=CLOSED pgs=749 gs=1 cs=0 l=1 c_cookie=6513871840429088378 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.739+0000 7f4f1e048640 1 --2- 10.20.193.28:0/1442070442 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4f1810f740 0x7f4f1810fb10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.739+0000 7f4f1e048640 1 -- 10.20.193.28:0/1442070442 >> 10.20.193.28:0/1442070442 conn(0x7f4f180ecff0 msgr2=0x7f4f1810d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:28.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.740+0000 7f4f1e048640 1 -- 10.20.193.28:0/1442070442 shutdown_connections 2026-02-25T11:55:28.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.740+0000 7f4f1e048640 1 -- 10.20.193.28:0/1442070442 wait complete. 2026-02-25T11:55:28.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.740+0000 7f4f1e048640 1 Processor -- start 2026-02-25T11:55:28.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.740+0000 7f4f1e048640 1 -- start start 2026-02-25T11:55:28.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.740+0000 7f4f1e048640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4f1810f740 0x7f4f180f5ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f1e048640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4f18110130 0x7f4f18176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f1e048640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f18115930 0x7f4f180f6160 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f1e048640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4f180fb3e0 con 0x7f4f18115930 2026-02-25T11:55:28.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f1e048640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4f180fb550 con 0x7f4f18110130 2026-02-25T11:55:28.745 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f1e048640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f180fb710 con 0x7f4f1810f740 2026-02-25T11:55:28.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f17fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f18115930 0x7f4f180f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f17fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f18115930 0x7f4f180f6160 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47538/0 (socket says 10.20.193.28:47538) 2026-02-25T11:55:28.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f17fff640 1 -- 10.20.193.28:0/428935494 learned_addr learned my addr 10.20.193.28:0/428935494 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:28.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f177fe640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4f1810f740 0x7f4f180f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f16ffd640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4f18110130 0x7f4f18176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f17fff640 1 -- 10.20.193.28:0/428935494 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4f1810f740 msgr2=0x7f4f180f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f17fff640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4f1810f740 0x7f4f180f5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f17fff640 1 -- 10.20.193.28:0/428935494 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4f18110130 msgr2=0x7f4f18176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f17fff640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4f18110130 0x7f4f18176200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f17fff640 1 -- 10.20.193.28:0/428935494 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f08009570 con 0x7f4f18115930 2026-02-25T11:55:28.746 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.741+0000 7f4f17fff640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f18115930 0x7f4f180f6160 secure :-1 s=READY pgs=750 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f0c006e90 tx=0x7f4f0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:28.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.742+0000 7f4f14ff9640 1 -- 10.20.193.28:0/428935494 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f0c005500 con 0x7f4f18115930 2026-02-25T11:55:28.747 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.742+0000 7f4f14ff9640 1 -- 10.20.193.28:0/428935494 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4f0c005df0 con 0x7f4f18115930 2026-02-25T11:55:28.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.742+0000 7f4f14ff9640 1 -- 10.20.193.28:0/428935494 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f0c003a00 con 0x7f4f18115930 2026-02-25T11:55:28.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.742+0000 7f4f1e048640 1 -- 10.20.193.28:0/428935494 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f180fb9f0 con 0x7f4f18115930 2026-02-25T11:55:28.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.742+0000 7f4f1e048640 1 -- 10.20.193.28:0/428935494 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f180fbe70 con 0x7f4f18115930 2026-02-25T11:55:28.748 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.745+0000 7f4f14ff9640 1 -- 10.20.193.28:0/428935494 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f4f0c003b60 con 0x7f4f18115930 2026-02-25T11:55:28.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.746+0000 7f4f14ff9640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4eec0b5760 0x7f4eec0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.746+0000 7f4f14ff9640 1 -- 10.20.193.28:0/428935494 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4f0c0e77b0 con 0x7f4f18115930 2026-02-25T11:55:28.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.747+0000 7f4f177fe640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4eec0b5760 0x7f4eec0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.747+0000 7f4f177fe640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4eec0b5760 0x7f4eec0d5c10 secure :-1 s=READY pgs=353 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f04004740 tx=0x7f4f0400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:28.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:28 trial167 ceph-mon[43392]: pgmap v443: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 11 op/s 2026-02-25T11:55:28.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.747+0000 7f4f20043e00 1 Processor -- start 2026-02-25T11:55:28.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f20043e00 1 -- start start 2026-02-25T11:55:28.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f20043e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 0x565210a70f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f20043e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a71440 0x565210ab5ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f20043e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210ab60e0 0x565210ad64c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f20043e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5652109868a0 con 0x565210a71440 2026-02-25T11:55:28.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f20043e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5652109825d0 con 0x565210a70b30 2026-02-25T11:55:28.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f20043e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56521097c570 con 0x565210ab60e0 2026-02-25T11:55:28.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f16ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a71440 0x565210ab5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f16ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a71440 0x565210ab5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47542/0 (socket says 10.20.193.28:47542) 2026-02-25T11:55:28.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f16ffd640 1 -- 10.20.193.28:0/2850147718 learned_addr learned my addr 10.20.193.28:0/2850147718 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:28.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f177fe640 1 --2- 10.20.193.28:0/2850147718 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 0x565210a70f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f17fff640 1 --2- 10.20.193.28:0/2850147718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210ab60e0 0x565210ad64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f16ffd640 1 -- 10.20.193.28:0/2850147718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210ab60e0 msgr2=0x565210ad64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:28.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f16ffd640 1 --2- 10.20.193.28:0/2850147718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210ab60e0 0x565210ad64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f16ffd640 1 -- 10.20.193.28:0/2850147718 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 msgr2=0x565210a70f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:28.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f16ffd640 1 --2- 10.20.193.28:0/2850147718 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 0x565210a70f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.748+0000 7f4f16ffd640 1 -- 10.20.193.28:0/2850147718 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565210927d10 con 0x565210a71440 2026-02-25T11:55:28.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.749+0000 7f4f16ffd640 1 --2- 10.20.193.28:0/2850147718 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a71440 0x565210ab5ba0 secure :-1 s=READY pgs=751 gs=3 cs=0 l=1 c_cookie=434385211772102387 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f08005c30 tx=0x7f4f08038560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6073f0c735be6f3 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:28.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.749+0000 7f4edbfff640 1 -- 10.20.193.28:0/2850147718 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f08002e30 con 0x565210a71440 2026-02-25T11:55:28.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.749+0000 7f4edbfff640 1 -- 10.20.193.28:0/2850147718 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4f080026a0 con 0x565210a71440 2026-02-25T11:55:28.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.749+0000 7f4edbfff640 1 -- 10.20.193.28:0/2850147718 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f08038920 con 0x565210a71440 2026-02-25T11:55:28.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.749+0000 7f4f20043e00 1 -- 10.20.193.28:0/2850147718 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a71440 msgr2=0x565210ab5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.749+0000 7f4f20043e00 1 --2- 10.20.193.28:0/2850147718 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a71440 0x565210ab5ba0 secure :-1 s=READY pgs=751 gs=3 cs=0 l=1 c_cookie=434385211772102387 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f08005c30 tx=0x7f4f08038560 comp rx=0 tx=0).stop 2026-02-25T11:55:28.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.749+0000 7f4f20043e00 1 -- 10.20.193.28:0/2850147718 shutdown_connections 2026-02-25T11:55:28.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.749+0000 7f4f20043e00 1 --2- 10.20.193.28:0/2850147718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210ab60e0 0x565210ad64c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.749+0000 7f4f20043e00 1 --2- 10.20.193.28:0/2850147718 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a71440 0x565210ab5ba0 unknown :-1 s=CLOSED pgs=751 gs=3 cs=0 l=1 c_cookie=434385211772102387 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.749+0000 7f4f20043e00 1 --2- 10.20.193.28:0/2850147718 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 0x565210a70f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.749+0000 7f4f20043e00 1 -- 10.20.193.28:0/2850147718 >> 10.20.193.28:0/2850147718 conn(0x56521098f7a0 msgr2=0x565210a6f410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:28.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.750+0000 7f4f20043e00 1 -- 10.20.193.28:0/2850147718 shutdown_connections 2026-02-25T11:55:28.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.750+0000 7f4f20043e00 1 -- 10.20.193.28:0/2850147718 wait complete. 2026-02-25T11:55:28.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.751+0000 7f4f20043e00 1 -- start start 2026-02-25T11:55:28.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.751+0000 7f4f20043e00 1 Processor -- start 2026-02-25T11:55:28.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.751+0000 7f4f20043e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a70b30 0x565210a88dc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.751+0000 7f4f20043e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 0x565210a89300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.751+0000 7f4f20043e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210ab60e0 0x565210a8e2f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.751+0000 7f4f20043e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x565210a8e830 con 0x565210a70b30 2026-02-25T11:55:28.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.751+0000 7f4f20043e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x565210a8e9a0 con 0x565210ab60e0 2026-02-25T11:55:28.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.751+0000 7f4f20043e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x565210a8eb60 con 0x565210a71440 2026-02-25T11:55:28.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4f177fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a70b30 0x565210a88dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4f177fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a70b30 0x565210a88dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47556/0 (socket says 10.20.193.28:47556) 2026-02-25T11:55:28.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4f177fe640 1 -- 10.20.193.28:0/1065179251 learned_addr learned my addr 10.20.193.28:0/1065179251 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:28.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4f17fff640 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210ab60e0 0x565210a8e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4f16ffd640 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 0x565210a89300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4f177fe640 1 -- 10.20.193.28:0/1065179251 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 msgr2=0x565210a89300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:28.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4f177fe640 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 0x565210a89300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4f177fe640 1 -- 10.20.193.28:0/1065179251 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210ab60e0 msgr2=0x565210a8e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:28.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4f177fe640 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210ab60e0 0x565210a8e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4f177fe640 1 -- 10.20.193.28:0/1065179251 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652109003c0 con 0x565210a70b30 2026-02-25T11:55:28.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4f17fff640 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210ab60e0 0x565210a8e2f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:55:28.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4f177fe640 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a70b30 0x565210a88dc0 secure :-1 s=READY pgs=752 gs=1 cs=0 l=1 c_cookie=616853525611175922 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f040109b0 tx=0x7f4f040047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f810368805ff2 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:28.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4ed97fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:28.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1065179251 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f0401de00 con 0x565210a70b30 2026-02-25T11:55:28.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1065179251 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4f0401e8c0 con 0x565210a70b30 2026-02-25T11:55:28.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1065179251 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f04026880 con 0x565210a70b30 2026-02-25T11:55:28.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.752+0000 7f4f20043e00 1 -- 10.20.193.28:0/1065179251 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f18176a30 con 0x565210a70b30 2026-02-25T11:55:28.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.753+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1065179251 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4f04016040 con 0x565210a70b30 2026-02-25T11:55:28.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.753+0000 7f4f20043e00 10 client.17004 _ll_register_callbacks cb 0x5652108fda10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:28.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.753+0000 7f4f20043e00 1 -- 10.20.193.28:0/1065179251 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5652106126f0 con 0x565210a70b30 2026-02-25T11:55:28.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.753+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1065179251 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4f0401ea30 con 0x565210a70b30 2026-02-25T11:55:28.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.753+0000 7f4f20043e00 10 client.17004 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:28.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.753+0000 7f4f20043e00 1 -- 10.20.193.28:0/1065179251 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5652109120e0 con 0x565210a70b30 2026-02-25T11:55:28.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.753+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1065179251 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f4f0401e400 con 0x565210a70b30 2026-02-25T11:55:28.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.753+0000 7f4f20043e00 10 client.17004 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:28.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.753+0000 7f4f20043e00 10 client.17004 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:55:28.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.753+0000 7f4f20043e00 20 client.17004 populate_metadata read hostname 'trial028' 2026-02-25T11:55:28.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.754+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565210ab3540 0x565210ab39e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.754+0000 7f4f20043e00 4 client.17004 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:55:28.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.754+0000 7f4f20043e00 1 -- 10.20.193.28:0/1065179251 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56521094efc0 con 0x565210ab3540 2026-02-25T11:55:28.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.754+0000 7f4f16ffd640 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565210ab3540 0x565210ab39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.755+0000 7f4f16ffd640 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565210ab3540 0x565210ab39e0 crc :-1 s=READY pgs=51 gs=4 cs=0 l=0 c_cookie=8167915415306663895 s_cookie=9237592643120520271 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=715a43cb74ca77d7 server_cookie=80328590e8f5dc4f in_seq=0 out_seq=0 2026-02-25T11:55:28.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.755+0000 7f4ed97fa640 10 client.17004 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:55:28.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.755+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1065179251 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x56521094efc0 con 0x565210ab3540 2026-02-25T11:55:28.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.755+0000 7f4ed97fa640 10 client.17004 handle_command_reply: tid=0 2026-02-25T11:55:28.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.755+0000 7f4f20043e00 1 client.17004 shutdown 2026-02-25T11:55:28.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.756+0000 7f4f20043e00 1 -- 10.20.193.28:0/1065179251 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a70b30 msgr2=0x565210a88dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.756+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a70b30 0x565210a88dc0 secure :-1 s=READY pgs=752 gs=1 cs=0 l=1 c_cookie=616853525611175922 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f040109b0 tx=0x7f4f040047e0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.756+0000 7f4f20043e00 1 -- 10.20.193.28:0/1065179251 shutdown_connections 2026-02-25T11:55:28.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.756+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565210ab3540 0x565210ab39e0 crc :-1 s=READY pgs=51 gs=4 cs=0 l=0 c_cookie=8167915415306663895 s_cookie=9237592643120520271 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.756+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210ab60e0 0x565210a8e2f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.756+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 0x565210a89300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.756+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1065179251 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210a70b30 0x565210a88dc0 unknown :-1 s=CLOSED pgs=752 gs=1 cs=0 l=1 c_cookie=616853525611175922 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.756+0000 7f4ed97fa640 0 client.17004 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:55:28.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.756+0000 7f4f20043e00 1 -- 10.20.193.28:0/1065179251 >> 10.20.193.28:0/1065179251 conn(0x56521098f7a0 msgr2=0x565210a6fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:28.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.756+0000 7f4f20043e00 1 -- 10.20.193.28:0/1065179251 shutdown_connections 2026-02-25T11:55:28.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.756+0000 7f4f20043e00 1 -- 10.20.193.28:0/1065179251 wait complete. 2026-02-25T11:55:28.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.756+0000 7f4f20043e00 20 client.17004 trim_cache size 0 max 16384 2026-02-25T11:55:28.763 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:28 trial119 ceph-mon[42710]: pgmap v443: 97 pgs: 97 active+clean; 5.6 MiB data, 345 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 11 op/s 2026-02-25T11:55:28.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.763+0000 7f4f20043e00 1 Processor -- start 2026-02-25T11:55:28.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.763+0000 7f4f20043e00 1 -- start start 2026-02-25T11:55:28.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.763+0000 7f4f20043e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 0x565210a8d860 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.763+0000 7f4f20043e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 0x565210aae7d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.763+0000 7f4f20043e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 0x565210aaed10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.763+0000 7f4f20043e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56521097a1a0 con 0x565210ab60e0 2026-02-25T11:55:28.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.763+0000 7f4f20043e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56521063a700 con 0x565210a70b30 2026-02-25T11:55:28.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.763+0000 7f4f20043e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x565210aafdb0 con 0x565210a71440 2026-02-25T11:55:28.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4f17fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 0x565210aaed10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4f17fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 0x565210aaed10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47558/0 (socket says 10.20.193.28:47558) 2026-02-25T11:55:28.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4f17fff640 1 -- 10.20.193.28:0/1269733961 learned_addr learned my addr 10.20.193.28:0/1269733961 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:28.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4f177fe640 1 --2- 10.20.193.28:0/1269733961 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 0x565210a8d860 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4f16ffd640 1 --2- 10.20.193.28:0/1269733961 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 0x565210aae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4f17fff640 1 -- 10.20.193.28:0/1269733961 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 msgr2=0x565210aae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:28.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4f17fff640 1 --2- 10.20.193.28:0/1269733961 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 0x565210aae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4f17fff640 1 -- 10.20.193.28:0/1269733961 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 msgr2=0x565210a8d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:28.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4f17fff640 1 --2- 10.20.193.28:0/1269733961 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 0x565210a8d860 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4f17fff640 1 -- 10.20.193.28:0/1269733961 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521090cb60 con 0x565210ab60e0 2026-02-25T11:55:28.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4f17fff640 1 --2- 10.20.193.28:0/1269733961 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 0x565210aaed10 secure :-1 s=READY pgs=753 gs=3 cs=0 l=1 c_cookie=12162107194920272387 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f0c00cfd0 tx=0x7f4f0c0ad640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c87c0e61720203 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:28.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4edb7fe640 1 -- 10.20.193.28:0/1269733961 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f0c0ae530 con 0x565210ab60e0 2026-02-25T11:55:28.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4edb7fe640 1 -- 10.20.193.28:0/1269733961 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4f0c0b3850 con 0x565210ab60e0 2026-02-25T11:55:28.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.764+0000 7f4edb7fe640 1 -- 10.20.193.28:0/1269733961 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f0c0ae7c0 con 0x565210ab60e0 2026-02-25T11:55:28.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.765+0000 7f4f20043e00 1 -- 10.20.193.28:0/1269733961 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 msgr2=0x565210aaed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.765+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1269733961 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 0x565210aaed10 secure :-1 s=READY pgs=753 gs=3 cs=0 l=1 c_cookie=12162107194920272387 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f0c00cfd0 tx=0x7f4f0c0ad640 comp rx=0 tx=0).stop 2026-02-25T11:55:28.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.765+0000 7f4f20043e00 1 -- 10.20.193.28:0/1269733961 shutdown_connections 2026-02-25T11:55:28.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.765+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1269733961 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 0x565210aaed10 unknown :-1 s=CLOSED pgs=753 gs=3 cs=0 l=1 c_cookie=12162107194920272387 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.765+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1269733961 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 0x565210aae7d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.765+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1269733961 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 0x565210a8d860 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.765+0000 7f4f20043e00 1 -- 10.20.193.28:0/1269733961 >> 10.20.193.28:0/1269733961 conn(0x56521098f7a0 msgr2=0x565210a8d150 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:28.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.765+0000 7f4f20043e00 1 -- 10.20.193.28:0/1269733961 shutdown_connections 2026-02-25T11:55:28.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.765+0000 7f4f20043e00 1 -- 10.20.193.28:0/1269733961 wait complete. 2026-02-25T11:55:28.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.765+0000 7f4f20043e00 1 -- start start 2026-02-25T11:55:28.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.765+0000 7f4f20043e00 1 Processor -- start 2026-02-25T11:55:28.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f20043e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 0x565210ab5970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f20043e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 0x565210a8f080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f20043e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 0x565210a8f5c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f20043e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x565210a8fb00 con 0x565210ab60e0 2026-02-25T11:55:28.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f20043e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x565210a8fc70 con 0x565210a70b30 2026-02-25T11:55:28.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f20043e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x565210a8fe30 con 0x565210a71440 2026-02-25T11:55:28.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f17fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 0x565210a8f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f17fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 0x565210a8f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47572/0 (socket says 10.20.193.28:47572) 2026-02-25T11:55:28.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f17fff640 1 -- 10.20.193.28:0/1939033542 learned_addr learned my addr 10.20.193.28:0/1939033542 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:28.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f177fe640 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 0x565210ab5970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f16ffd640 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 0x565210a8f080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f17fff640 1 -- 10.20.193.28:0/1939033542 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 msgr2=0x565210a8f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:28.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f17fff640 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 0x565210a8f080 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f17fff640 1 -- 10.20.193.28:0/1939033542 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 msgr2=0x565210ab5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:28.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f17fff640 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 0x565210ab5970 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f17fff640 1 -- 10.20.193.28:0/1939033542 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652109006f0 con 0x565210ab60e0 2026-02-25T11:55:28.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f16ffd640 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 0x565210a8f080 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:55:28.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.766+0000 7f4f177fe640 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 0x565210ab5970 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:55:28.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4f17fff640 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 0x565210a8f5c0 secure :-1 s=READY pgs=754 gs=1 cs=0 l=1 c_cookie=3002938302649993040 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f0c0050a0 tx=0x7f4f0c00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ac9479dbde0750 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:28.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4ed97fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:28.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1939033542 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f0c0aa050 con 0x565210ab60e0 2026-02-25T11:55:28.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4f20043e00 1 -- 10.20.193.28:0/1939033542 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f180fbe70 con 0x565210ab60e0 2026-02-25T11:55:28.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4f20043e00 10 client.17010 _ll_register_callbacks cb 0x5652108fda10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:28.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4f20043e00 1 -- 10.20.193.28:0/1939033542 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f080026a0 con 0x565210ab60e0 2026-02-25T11:55:28.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1939033542 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4f0c0ae530 con 0x565210ab60e0 2026-02-25T11:55:28.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1939033542 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4f0c0abd90 con 0x565210ab60e0 2026-02-25T11:55:28.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1939033542 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4f0c0d1470 con 0x565210ab60e0 2026-02-25T11:55:28.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1939033542 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4f0c0aea40 con 0x565210ab60e0 2026-02-25T11:55:28.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4f20043e00 10 client.17010 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:28.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4f20043e00 1 -- 10.20.193.28:0/1939033542 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x565210912340 con 0x565210ab60e0 2026-02-25T11:55:28.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1939033542 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f4f0c0b8850 con 0x565210ab60e0 2026-02-25T11:55:28.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4f20043e00 10 client.17010 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:28.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4f20043e00 10 client.17010 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:55:28.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4f20043e00 20 client.17010 populate_metadata read hostname 'trial028' 2026-02-25T11:55:28.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565210b74440 0x565210b948f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:28.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4f20043e00 4 client.17010 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:55:28.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.767+0000 7f4f20043e00 1 -- 10.20.193.28:0/1939033542 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56521060ff40 con 0x565210b74440 2026-02-25T11:55:28.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.768+0000 7f4f177fe640 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565210b74440 0x565210b948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:28.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.768+0000 7f4f177fe640 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565210b74440 0x565210b948f0 crc :-1 s=READY pgs=52 gs=4 cs=0 l=0 c_cookie=17052587352919450177 s_cookie=13419746083534804218 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eca6f5d958152241 server_cookie=ba3c83f64d0814fa in_seq=0 out_seq=0 2026-02-25T11:55:28.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.768+0000 7f4ed97fa640 10 client.17010 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:55:28.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4ed97fa640 1 -- 10.20.193.28:0/1939033542 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56521060ff40 con 0x565210b74440 2026-02-25T11:55:28.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4ed97fa640 10 client.17010 handle_command_reply: tid=0 2026-02-25T11:55:28.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f20043e00 1 client.17010 shutdown 2026-02-25T11:55:28.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f20043e00 1 -- 10.20.193.28:0/1939033542 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 msgr2=0x565210a8f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 0x565210a8f5c0 secure :-1 s=READY pgs=754 gs=1 cs=0 l=1 c_cookie=3002938302649993040 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f0c0050a0 tx=0x7f4f0c00ef30 comp rx=0 tx=0).stop 2026-02-25T11:55:28.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f20043e00 1 -- 10.20.193.28:0/1939033542 shutdown_connections 2026-02-25T11:55:28.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565210b74440 0x565210b948f0 crc :-1 s=READY pgs=52 gs=4 cs=0 l=0 c_cookie=17052587352919450177 s_cookie=13419746083534804218 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565210ab60e0 0x565210a8f5c0 unknown :-1 s=CLOSED pgs=754 gs=1 cs=0 l=1 c_cookie=3002938302649993040 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565210a71440 0x565210a8f080 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f20043e00 1 --2- 10.20.193.28:0/1939033542 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565210a70b30 0x565210ab5970 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4ed97fa640 0 client.17010 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:55:28.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f20043e00 1 -- 10.20.193.28:0/1939033542 >> 10.20.193.28:0/1939033542 conn(0x56521098f7a0 msgr2=0x565210a8d940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:28.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f20043e00 1 -- 10.20.193.28:0/1939033542 shutdown_connections 2026-02-25T11:55:28.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f20043e00 1 -- 10.20.193.28:0/1939033542 wait complete. 2026-02-25T11:55:28.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f20043e00 20 client.17010 trim_cache size 0 max 16384 2026-02-25T11:55:28.777 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:55:28.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f1e048640 1 -- 10.20.193.28:0/428935494 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4eec0b5760 msgr2=0x7f4eec0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f1e048640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4eec0b5760 0x7f4eec0d5c10 secure :-1 s=READY pgs=353 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f04004740 tx=0x7f4f0400a380 comp rx=0 tx=0).stop 2026-02-25T11:55:28.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f1e048640 1 -- 10.20.193.28:0/428935494 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f18115930 msgr2=0x7f4f180f6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:28.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.769+0000 7f4f1e048640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f18115930 0x7f4f180f6160 secure :-1 s=READY pgs=750 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4f0c006e90 tx=0x7f4f0c00edf0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.770+0000 7f4f1e048640 1 -- 10.20.193.28:0/428935494 shutdown_connections 2026-02-25T11:55:28.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.770+0000 7f4f1e048640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4eec0b5760 0x7f4eec0d5c10 unknown :-1 s=CLOSED pgs=353 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.770+0000 7f4f1e048640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4f18115930 0x7f4f180f6160 unknown :-1 s=CLOSED pgs=750 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.770+0000 7f4f1e048640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4f18110130 0x7f4f18176200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.770+0000 7f4f1e048640 1 --2- 10.20.193.28:0/428935494 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4f1810f740 0x7f4f180f5ad0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:28.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.770+0000 7f4f1e048640 1 -- 10.20.193.28:0/428935494 >> 10.20.193.28:0/428935494 conn(0x7f4f180ecff0 msgr2=0x7f4f18111f40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:28.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.770+0000 7f4f1e048640 1 -- 10.20.193.28:0/428935494 shutdown_connections 2026-02-25T11:55:28.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:28.770+0000 7f4f1e048640 1 -- 10.20.193.28:0/428935494 wait complete. 2026-02-25T11:55:28.872 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:55:29.056 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:55:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.223+0000 7f4cc7bac640 1 -- 10.20.193.28:0/665105462 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4cc0110130 msgr2=0x7f4cc0115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.223+0000 7f4cc7bac640 1 --2- 10.20.193.28:0/665105462 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4cc0110130 0x7f4cc0115180 secure :-1 s=READY pgs=755 gs=1 cs=0 l=1 c_cookie=16243090894694815182 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4cb4009950 tx=0x7f4cb402f190 comp rx=0 tx=0).stop 2026-02-25T11:55:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.224+0000 7f4cc7bac640 1 -- 10.20.193.28:0/665105462 shutdown_connections 2026-02-25T11:55:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.224+0000 7f4cc7bac640 1 --2- 10.20.193.28:0/665105462 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4cc0115930 0x7f4cc0176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.224+0000 7f4cc7bac640 1 --2- 10.20.193.28:0/665105462 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4cc0110130 0x7f4cc0115180 unknown :-1 s=CLOSED pgs=755 gs=1 cs=0 l=1 c_cookie=16243090894694815182 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.227 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.224+0000 7f4cc7bac640 1 --2- 10.20.193.28:0/665105462 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4cc010f740 0x7f4cc010fb10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.224+0000 7f4cc7bac640 1 -- 10.20.193.28:0/665105462 >> 10.20.193.28:0/665105462 conn(0x7f4cc00ecff0 msgr2=0x7f4cc010d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.224+0000 7f4cc7bac640 1 -- 10.20.193.28:0/665105462 shutdown_connections 2026-02-25T11:55:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.224+0000 7f4cc7bac640 1 -- 10.20.193.28:0/665105462 wait complete. 2026-02-25T11:55:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.225+0000 7f4cc7bac640 1 Processor -- start 2026-02-25T11:55:29.228 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.225+0000 7f4cc7bac640 1 -- start start 2026-02-25T11:55:29.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.225+0000 7f4cc7bac640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4cc010f740 0x7f4cc00f5b10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.225+0000 7f4cc7bac640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4cc0110130 0x7f4cc0176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.225+0000 7f4cc7bac640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4cc0115930 0x7f4cc00f61a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.225+0000 7f4cc7bac640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc00fb420 con 0x7f4cc0110130 2026-02-25T11:55:29.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.225+0000 7f4cc7bac640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc00fb590 con 0x7f4cc010f740 2026-02-25T11:55:29.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.225+0000 7f4cc7bac640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc00fb700 con 0x7f4cc0115930 2026-02-25T11:55:29.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4cc5120640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4cc0110130 0x7f4cc0176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4cc5921640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4cc010f740 0x7f4cc00f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.230 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4cc5921640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4cc010f740 0x7f4cc00f5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:35788/0 (socket says 10.20.193.28:35788) 2026-02-25T11:55:29.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4cc5921640 1 -- 10.20.193.28:0/327665433 learned_addr learned my addr 10.20.193.28:0/327665433 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:29.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4cc5120640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4cc0110130 0x7f4cc0176200 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47588/0 (socket says 10.20.193.28:47588) 2026-02-25T11:55:29.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4cc6122640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4cc0115930 0x7f4cc00f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4cc5921640 1 -- 10.20.193.28:0/327665433 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4cc0115930 msgr2=0x7f4cc00f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4cc5921640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4cc0115930 0x7f4cc00f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4cc5921640 1 -- 10.20.193.28:0/327665433 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4cc0110130 msgr2=0x7f4cc0176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4cc5921640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4cc0110130 0x7f4cc0176200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4cc5921640 1 -- 10.20.193.28:0/327665433 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cb4009570 con 0x7f4cc010f740 2026-02-25T11:55:29.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4cc5120640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4cc0110130 0x7f4cc0176200 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:55:29.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4cc5921640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4cc010f740 0x7f4cc00f5b10 secure :-1 s=READY pgs=164 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4cb00072a0 tx=0x7f4cb000ddf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:29.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4caeffd640 1 -- 10.20.193.28:0/327665433 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4cb0004500 con 0x7f4cc010f740 2026-02-25T11:55:29.231 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4caeffd640 1 -- 10.20.193.28:0/327665433 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4cb000bd00 con 0x7f4cc010f740 2026-02-25T11:55:29.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.226+0000 7f4caeffd640 1 -- 10.20.193.28:0/327665433 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4cb00106e0 con 0x7f4cc010f740 2026-02-25T11:55:29.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.227+0000 7f4cc7bac640 1 -- 10.20.193.28:0/327665433 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc00fb9e0 con 0x7f4cc010f740 2026-02-25T11:55:29.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.227+0000 7f4cc7bac640 1 -- 10.20.193.28:0/327665433 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc00fbf00 con 0x7f4cc010f740 2026-02-25T11:55:29.232 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.230+0000 7f4caeffd640 1 -- 10.20.193.28:0/327665433 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f4cb000b840 con 0x7f4cc010f740 2026-02-25T11:55:29.233 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.232+0000 7f4caeffd640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4c9c0b5760 0x7f4c9c0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.234 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.232+0000 7f4caeffd640 1 -- 10.20.193.28:0/327665433 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4cb00e69b0 con 0x7f4cc010f740 2026-02-25T11:55:29.234 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.232+0000 7f4cc9ba7e00 1 Processor -- start 2026-02-25T11:55:29.234 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.232+0000 7f4cc5120640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4c9c0b5760 0x7f4c9c0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.234 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc9ba7e00 1 -- start start 2026-02-25T11:55:29.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc9ba7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8bb30 0x55f71cf8bf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.236 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc9ba7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cf8c440 0x55f71cfd0ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc9ba7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cfd10e0 0x55f71cff14c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc9ba7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f71cea18a0 con 0x55f71cf8c440 2026-02-25T11:55:29.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc9ba7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f71ce9d5d0 con 0x55f71cf8bb30 2026-02-25T11:55:29.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc9ba7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f71ce97570 con 0x55f71cfd10e0 2026-02-25T11:55:29.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc5120640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cf8c440 0x55f71cfd0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc5921640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8bb30 0x55f71cf8bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc5120640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cf8c440 0x55f71cfd0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47602/0 (socket says 10.20.193.28:47602) 2026-02-25T11:55:29.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc5120640 1 -- 10.20.193.28:0/3047878826 learned_addr learned my addr 10.20.193.28:0/3047878826 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:29.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc5120640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4c9c0b5760 0x7f4c9c0d5c10 secure :-1 s=READY pgs=354 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4cc00fc4b0 tx=0x7f4cb403a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:29.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc6122640 1 --2- 10.20.193.28:0/3047878826 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cfd10e0 0x55f71cff14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.237 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc5120640 1 -- 10.20.193.28:0/3047878826 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cfd10e0 msgr2=0x55f71cff14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc5120640 1 --2- 10.20.193.28:0/3047878826 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cfd10e0 0x55f71cff14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc5120640 1 -- 10.20.193.28:0/3047878826 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8bb30 msgr2=0x55f71cf8bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.233+0000 7f4cc5120640 1 --2- 10.20.193.28:0/3047878826 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8bb30 0x55f71cf8bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.234+0000 7f4cc5120640 1 -- 10.20.193.28:0/3047878826 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f71ce42d10 con 0x55f71cf8c440 2026-02-25T11:55:29.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.234+0000 7f4cc5120640 1 --2- 10.20.193.28:0/3047878826 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cf8c440 0x55f71cfd0ba0 secure :-1 s=READY pgs=756 gs=2 cs=0 l=1 c_cookie=3432914715319304801 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4cb4005f00 tx=0x7f4cb4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa429bec6d18261 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:29.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.234+0000 7f4c8e4e6640 1 -- 10.20.193.28:0/3047878826 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4cb402fe50 con 0x55f71cf8c440 2026-02-25T11:55:29.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.234+0000 7f4c8e4e6640 1 -- 10.20.193.28:0/3047878826 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4cb403e040 con 0x55f71cf8c440 2026-02-25T11:55:29.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.234+0000 7f4c8e4e6640 1 -- 10.20.193.28:0/3047878826 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4cb404fba0 con 0x55f71cf8c440 2026-02-25T11:55:29.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.234+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3047878826 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cf8c440 msgr2=0x55f71cfd0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.234+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3047878826 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cf8c440 0x55f71cfd0ba0 secure :-1 s=READY pgs=756 gs=2 cs=0 l=1 c_cookie=3432914715319304801 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4cb4005f00 tx=0x7f4cb4040000 comp rx=0 tx=0).stop 2026-02-25T11:55:29.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.234+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3047878826 shutdown_connections 2026-02-25T11:55:29.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.234+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3047878826 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cfd10e0 0x55f71cff14c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.234+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3047878826 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cf8c440 0x55f71cfd0ba0 unknown :-1 s=CLOSED pgs=756 gs=2 cs=0 l=1 c_cookie=3432914715319304801 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.234+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3047878826 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8bb30 0x55f71cf8bf00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.234+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3047878826 >> 10.20.193.28:0/3047878826 conn(0x55f71ceaa7a0 msgr2=0x55f71cf8a410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:29.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.235+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3047878826 shutdown_connections 2026-02-25T11:55:29.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.235+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3047878826 wait complete. 2026-02-25T11:55:29.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.236+0000 7f4cc9ba7e00 1 -- start start 2026-02-25T11:55:29.243 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.236+0000 7f4cc9ba7e00 1 Processor -- start 2026-02-25T11:55:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.236+0000 7f4cc9ba7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 0x55f71cfa3d30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.236+0000 7f4cc9ba7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cf8c440 0x55f71cfa4270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.236+0000 7f4cc9ba7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cfd10e0 0x55f71cfa9260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.236+0000 7f4cc9ba7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f71cfa97a0 con 0x55f71cf8c440 2026-02-25T11:55:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.236+0000 7f4cc9ba7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f71cfa9910 con 0x55f71cfd10e0 2026-02-25T11:55:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.236+0000 7f4cc9ba7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f71cfa9ad0 con 0x55f71cf8bb30 2026-02-25T11:55:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.237+0000 7f4cc5921640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 0x55f71cfa3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.237+0000 7f4cc5921640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 0x55f71cfa3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:49042/0 (socket says 10.20.193.28:49042) 2026-02-25T11:55:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.237+0000 7f4cc5921640 1 -- 10.20.193.28:0/3763841142 learned_addr learned my addr 10.20.193.28:0/3763841142 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.237+0000 7f4cc6122640 1 --2- 10.20.193.28:0/3763841142 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cfd10e0 0x55f71cfa9260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.237+0000 7f4cc5120640 1 --2- 10.20.193.28:0/3763841142 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cf8c440 0x55f71cfa4270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.237+0000 7f4cc5921640 1 -- 10.20.193.28:0/3763841142 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cfd10e0 msgr2=0x55f71cfa9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.237+0000 7f4cc5921640 1 --2- 10.20.193.28:0/3763841142 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cfd10e0 0x55f71cfa9260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.237+0000 7f4cc5921640 1 -- 10.20.193.28:0/3763841142 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cf8c440 msgr2=0x55f71cfa4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.237+0000 7f4cc5921640 1 --2- 10.20.193.28:0/3763841142 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cf8c440 0x55f71cfa4270 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.237+0000 7f4cc5921640 1 -- 10.20.193.28:0/3763841142 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f71ce1b3c0 con 0x55f71cf8bb30 2026-02-25T11:55:29.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.237+0000 7f4cc5921640 1 --2- 10.20.193.28:0/3763841142 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 0x55f71cfa3d30 secure :-1 s=READY pgs=200 gs=1 cs=0 l=1 c_cookie=6463221584089166681 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4cb00e4540 tx=0x7f4cb00a9590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59b1f9c98045b359 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:29.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.237+0000 7f4c7b7fe640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:55:29.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.237+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3763841142 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4cb00e42d0 con 0x55f71cf8bb30 2026-02-25T11:55:29.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3763841142 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4cb00ac620 con 0x55f71cf8bb30 2026-02-25T11:55:29.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3763841142 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc0176a30 con 0x55f71cf8bb30 2026-02-25T11:55:29.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3763841142 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4cb00b7ca0 con 0x55f71cf8bb30 2026-02-25T11:55:29.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4cc9ba7e00 10 client.26719 _ll_register_callbacks cb 0x55f71ce18a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:29.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3763841142 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f71cb2d6f0 con 0x55f71cf8bb30 2026-02-25T11:55:29.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3763841142 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4cb00b2020 con 0x55f71cf8bb30 2026-02-25T11:55:29.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3763841142 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4cb00b7eb0 con 0x55f71cf8bb30 2026-02-25T11:55:29.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4cc9ba7e00 10 client.26719 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:29.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3763841142 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55f71ce2d0e0 con 0x55f71cf8bb30 2026-02-25T11:55:29.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3763841142 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f4cb00c4bd0 con 0x55f71cf8bb30 2026-02-25T11:55:29.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4cc9ba7e00 10 client.26719 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:29.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4cc9ba7e00 10 client.26719 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:55:29.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4cc9ba7e00 20 client.26719 populate_metadata read hostname 'trial028' 2026-02-25T11:55:29.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3763841142 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f71cfce610 0x55f71cfceab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4cc9ba7e00 4 client.26719 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:55:29.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.238+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3763841142 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f71ce69fc0 con 0x55f71cfce610 2026-02-25T11:55:29.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.239+0000 7f4cc5120640 1 --2- 10.20.193.28:0/3763841142 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f71cfce610 0x55f71cfceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.239+0000 7f4cc5120640 1 --2- 10.20.193.28:0/3763841142 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f71cfce610 0x55f71cfceab0 crc :-1 s=READY pgs=52 gs=4 cs=0 l=0 c_cookie=11697865621505867074 s_cookie=13582265301437030518 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2572ace081c7d42 server_cookie=bc7de6546944a076 in_seq=0 out_seq=0 2026-02-25T11:55:29.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.239+0000 7f4c7b7fe640 10 client.26719 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:55:29.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3763841142 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55f71ce69fc0 con 0x55f71cfce610 2026-02-25T11:55:29.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4c7b7fe640 10 client.26719 handle_command_reply: tid=0 2026-02-25T11:55:29.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4cc9ba7e00 1 client.26719 shutdown 2026-02-25T11:55:29.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3763841142 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 msgr2=0x55f71cfa3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3763841142 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 0x55f71cfa3d30 secure :-1 s=READY pgs=200 gs=1 cs=0 l=1 c_cookie=6463221584089166681 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4cb00e4540 tx=0x7f4cb00a9590 comp rx=0 tx=0).stop 2026-02-25T11:55:29.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3763841142 shutdown_connections 2026-02-25T11:55:29.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3763841142 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f71cfce610 0x55f71cfceab0 crc :-1 s=READY pgs=52 gs=4 cs=0 l=0 c_cookie=11697865621505867074 s_cookie=13582265301437030518 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3763841142 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cfd10e0 0x55f71cfa9260 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3763841142 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cf8c440 0x55f71cfa4270 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4c7b7fe640 0 client.26719 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:55:29.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3763841142 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 0x55f71cfa3d30 unknown :-1 s=CLOSED pgs=200 gs=1 cs=0 l=1 c_cookie=6463221584089166681 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3763841142 >> 10.20.193.28:0/3763841142 conn(0x55f71ceaa7a0 msgr2=0x55f71cf8ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:29.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3763841142 shutdown_connections 2026-02-25T11:55:29.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3763841142 wait complete. 2026-02-25T11:55:29.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.240+0000 7f4cc9ba7e00 20 client.26719 trim_cache size 0 max 16384 2026-02-25T11:55:29.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc9ba7e00 1 Processor -- start 2026-02-25T11:55:29.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc9ba7e00 1 -- start start 2026-02-25T11:55:29.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc9ba7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8bb30 0x55f71cfa1330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc9ba7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8c440 0x55f71cfa87d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc9ba7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cfd10e0 0x55f71cfc9740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc9ba7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f71cfcafa0 con 0x55f71cfd10e0 2026-02-25T11:55:29.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc9ba7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f71ce951a0 con 0x55f71cf8bb30 2026-02-25T11:55:29.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc9ba7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f71cb55700 con 0x55f71cf8c440 2026-02-25T11:55:29.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc6122640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cfd10e0 0x55f71cfc9740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc6122640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cfd10e0 0x55f71cfc9740 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47624/0 (socket says 10.20.193.28:47624) 2026-02-25T11:55:29.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc6122640 1 -- 10.20.193.28:0/3655332114 learned_addr learned my addr 10.20.193.28:0/3655332114 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:29.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc5120640 1 --2- 10.20.193.28:0/3655332114 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8c440 0x55f71cfa87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc5921640 1 --2- 10.20.193.28:0/3655332114 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8bb30 0x55f71cfa1330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc6122640 1 -- 10.20.193.28:0/3655332114 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8c440 msgr2=0x55f71cfa87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc6122640 1 --2- 10.20.193.28:0/3655332114 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8c440 0x55f71cfa87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc6122640 1 -- 10.20.193.28:0/3655332114 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8bb30 msgr2=0x55f71cfa1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc6122640 1 --2- 10.20.193.28:0/3655332114 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8bb30 0x55f71cfa1330 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.248+0000 7f4cc6122640 1 -- 10.20.193.28:0/3655332114 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f71ce27b60 con 0x55f71cfd10e0 2026-02-25T11:55:29.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.249+0000 7f4cc6122640 1 --2- 10.20.193.28:0/3655332114 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cfd10e0 0x55f71cfc9740 secure :-1 s=READY pgs=757 gs=1 cs=0 l=1 c_cookie=3413311611770459014 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4cbc004990 tx=0x7f4cbc00a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5e84d38f4b5386 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:29.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.249+0000 7f4c8dce5640 1 -- 10.20.193.28:0/3655332114 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4cbc018070 con 0x55f71cfd10e0 2026-02-25T11:55:29.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.249+0000 7f4c8dce5640 1 -- 10.20.193.28:0/3655332114 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4cbc019040 con 0x55f71cfd10e0 2026-02-25T11:55:29.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.249+0000 7f4c8dce5640 1 -- 10.20.193.28:0/3655332114 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4cbc0118e0 con 0x55f71cfd10e0 2026-02-25T11:55:29.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.249+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3655332114 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cfd10e0 msgr2=0x55f71cfc9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.249+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3655332114 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cfd10e0 0x55f71cfc9740 secure :-1 s=READY pgs=757 gs=1 cs=0 l=1 c_cookie=3413311611770459014 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4cbc004990 tx=0x7f4cbc00a310 comp rx=0 tx=0).stop 2026-02-25T11:55:29.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.249+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3655332114 shutdown_connections 2026-02-25T11:55:29.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.249+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3655332114 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cfd10e0 0x55f71cfc9740 unknown :-1 s=CLOSED pgs=757 gs=1 cs=0 l=1 c_cookie=3413311611770459014 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.249+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3655332114 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8c440 0x55f71cfa87d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.249+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3655332114 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8bb30 0x55f71cfa1330 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.249+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3655332114 >> 10.20.193.28:0/3655332114 conn(0x55f71ceaa7a0 msgr2=0x55f71cfa80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:29.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.249+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3655332114 shutdown_connections 2026-02-25T11:55:29.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.249+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3655332114 wait complete. 2026-02-25T11:55:29.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.250+0000 7f4cc9ba7e00 1 -- start start 2026-02-25T11:55:29.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.250+0000 7f4cc9ba7e00 1 Processor -- start 2026-02-25T11:55:29.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.250+0000 7f4cc9ba7e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 0x55f71cfaa4d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.250+0000 7f4cc9ba7e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8c440 0x55f71cfaaa10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.250+0000 7f4cc9ba7e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cfd10e0 0x55f71cfaf370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.250+0000 7f4cc9ba7e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f71cf66e80 con 0x55f71cfd10e0 2026-02-25T11:55:29.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.250+0000 7f4cc9ba7e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f71cfa43a0 con 0x55f71cf8c440 2026-02-25T11:55:29.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.250+0000 7f4cc9ba7e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f71cfaf8b0 con 0x55f71cf8bb30 2026-02-25T11:55:29.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc5921640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 0x55f71cfaa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc6122640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cfd10e0 0x55f71cfaf370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc5921640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 0x55f71cfaa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:49074/0 (socket says 10.20.193.28:49074) 2026-02-25T11:55:29.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc5921640 1 -- 10.20.193.28:0/3043335196 learned_addr learned my addr 10.20.193.28:0/3043335196 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:29.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc5120640 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8c440 0x55f71cfaaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc6122640 1 -- 10.20.193.28:0/3043335196 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 msgr2=0x55f71cfaa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc6122640 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 0x55f71cfaa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc6122640 1 -- 10.20.193.28:0/3043335196 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8c440 msgr2=0x55f71cfaaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc6122640 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8c440 0x55f71cfaaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.259 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc6122640 1 -- 10.20.193.28:0/3043335196 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f71ce1b6f0 con 0x55f71cfd10e0 2026-02-25T11:55:29.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc5921640 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 0x55f71cfaa4d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:55:29.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc5120640 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8c440 0x55f71cfaaa10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:55:29.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc6122640 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cfd10e0 0x55f71cfaf370 secure :-1 s=READY pgs=758 gs=1 cs=0 l=1 c_cookie=7227979936326359119 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4cbc004990 tx=0x7f4cbc011700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644ef174e462184f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:29.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4c7b7fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:29.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3043335196 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4cbc018040 con 0x55f71cfd10e0 2026-02-25T11:55:29.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3043335196 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4cbc011870 con 0x55f71cfd10e0 2026-02-25T11:55:29.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3043335196 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cbc005520 con 0x55f71cfd10e0 2026-02-25T11:55:29.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3043335196 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4cbc005d90 con 0x55f71cfd10e0 2026-02-25T11:55:29.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc9ba7e00 10 client.17028 _ll_register_callbacks cb 0x55f71ce18a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:29.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.251+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3043335196 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb403e040 con 0x55f71cfd10e0 2026-02-25T11:55:29.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.252+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3043335196 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f4cbc006050 con 0x55f71cfd10e0 2026-02-25T11:55:29.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.252+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3043335196 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4cbc006230 con 0x55f71cfd10e0 2026-02-25T11:55:29.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.252+0000 7f4cc9ba7e00 10 client.17028 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:29.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.252+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3043335196 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55f71ce2d340 con 0x55f71cfd10e0 2026-02-25T11:55:29.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.252+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3043335196 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f4cbc0063d0 con 0x55f71cfd10e0 2026-02-25T11:55:29.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.252+0000 7f4cc9ba7e00 10 client.17028 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:29.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.252+0000 7f4cc9ba7e00 10 client.17028 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:55:29.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.252+0000 7f4cc9ba7e00 20 client.17028 populate_metadata read hostname 'trial028' 2026-02-25T11:55:29.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.252+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f71d087470 0x55f71d0a7920 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.252+0000 7f4cc9ba7e00 4 client.17028 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:55:29.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.252+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3043335196 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f71cb2af40 con 0x55f71d087470 2026-02-25T11:55:29.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.252+0000 7f4cc5921640 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f71d087470 0x55f71d0a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4cc5921640 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f71d087470 0x55f71d0a7920 crc :-1 s=READY pgs=53 gs=4 cs=0 l=0 c_cookie=5920877008344713684 s_cookie=14207764225782429820 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=522b2e44d92a09d4 server_cookie=c52c1e499243b87c in_seq=0 out_seq=0 2026-02-25T11:55:29.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4c7b7fe640 10 client.17028 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:55:29.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4c7b7fe640 1 -- 10.20.193.28:0/3043335196 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f71cb2af40 con 0x55f71d087470 2026-02-25T11:55:29.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4c7b7fe640 10 client.17028 handle_command_reply: tid=0 2026-02-25T11:55:29.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4cc9ba7e00 1 client.17028 shutdown 2026-02-25T11:55:29.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3043335196 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cfd10e0 msgr2=0x55f71cfaf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cfd10e0 0x55f71cfaf370 secure :-1 s=READY pgs=758 gs=1 cs=0 l=1 c_cookie=7227979936326359119 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4cbc004990 tx=0x7f4cbc011700 comp rx=0 tx=0).stop 2026-02-25T11:55:29.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3043335196 shutdown_connections 2026-02-25T11:55:29.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f71d087470 0x55f71d0a7920 crc :-1 s=READY pgs=53 gs=4 cs=0 l=0 c_cookie=5920877008344713684 s_cookie=14207764225782429820 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f71cfd10e0 0x55f71cfaf370 unknown :-1 s=CLOSED pgs=758 gs=1 cs=0 l=1 c_cookie=7227979936326359119 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f71cf8c440 0x55f71cfaaa10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4cc9ba7e00 1 --2- 10.20.193.28:0/3043335196 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f71cf8bb30 0x55f71cfaa4d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4c7b7fe640 0 client.17028 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:55:29.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3043335196 >> 10.20.193.28:0/3043335196 conn(0x55f71ceaa7a0 msgr2=0x55f71cfa88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:29.263 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:55:29.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3043335196 shutdown_connections 2026-02-25T11:55:29.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4cc9ba7e00 1 -- 10.20.193.28:0/3043335196 wait complete. 2026-02-25T11:55:29.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.253+0000 7f4cc9ba7e00 20 client.17028 trim_cache size 0 max 16384 2026-02-25T11:55:29.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.254+0000 7f4cc7bac640 1 -- 10.20.193.28:0/327665433 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4c9c0b5760 msgr2=0x7f4c9c0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.254+0000 7f4cc7bac640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4c9c0b5760 0x7f4c9c0d5c10 secure :-1 s=READY pgs=354 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4cc00fc4b0 tx=0x7f4cb403a040 comp rx=0 tx=0).stop 2026-02-25T11:55:29.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.254+0000 7f4cc7bac640 1 -- 10.20.193.28:0/327665433 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4cc010f740 msgr2=0x7f4cc00f5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.254+0000 7f4cc7bac640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4cc010f740 0x7f4cc00f5b10 secure :-1 s=READY pgs=164 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4cb00072a0 tx=0x7f4cb000ddf0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.254+0000 7f4cc7bac640 1 -- 10.20.193.28:0/327665433 shutdown_connections 2026-02-25T11:55:29.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.254+0000 7f4cc7bac640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f4c9c0b5760 0x7f4c9c0d5c10 unknown :-1 s=CLOSED pgs=354 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.254+0000 7f4cc7bac640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4cc0115930 0x7f4cc00f61a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.254+0000 7f4cc7bac640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4cc0110130 0x7f4cc0176200 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.254+0000 7f4cc7bac640 1 --2- 10.20.193.28:0/327665433 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f4cc010f740 0x7f4cc00f5b10 unknown :-1 s=CLOSED pgs=164 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.254+0000 7f4cc7bac640 1 -- 10.20.193.28:0/327665433 >> 10.20.193.28:0/327665433 conn(0x7f4cc00ecff0 msgr2=0x7f4cc0111f40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:29.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.254+0000 7f4cc7bac640 1 -- 10.20.193.28:0/327665433 shutdown_connections 2026-02-25T11:55:29.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.254+0000 7f4cc7bac640 1 -- 10.20.193.28:0/327665433 wait complete. 2026-02-25T11:55:29.356 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:55:29.406 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:29 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/192893221' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:55:29.539 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:55:29.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.724+0000 7f0aeb127640 1 -- 10.20.193.28:0/1453257317 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0ae410ddc0 msgr2=0x7f0ae4112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.724+0000 7f0aeb127640 1 --2- 10.20.193.28:0/1453257317 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0ae410ddc0 0x7f0ae4112e10 secure :-1 s=READY pgs=759 gs=1 cs=0 l=1 c_cookie=17468823769924566936 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ad8009950 tx=0x7f0ad802f190 comp rx=0 tx=0).stop 2026-02-25T11:55:29.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.724+0000 7f0aeb127640 1 -- 10.20.193.28:0/1453257317 shutdown_connections 2026-02-25T11:55:29.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.724+0000 7f0aeb127640 1 --2- 10.20.193.28:0/1453257317 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0ae41135c0 0x7f0ae4173f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.724+0000 7f0aeb127640 1 --2- 10.20.193.28:0/1453257317 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0ae410ddc0 0x7f0ae4112e10 unknown :-1 s=CLOSED pgs=759 gs=1 cs=0 l=1 c_cookie=17468823769924566936 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.724+0000 7f0aeb127640 1 --2- 10.20.193.28:0/1453257317 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0ae410d3d0 0x7f0ae410d7a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.724+0000 7f0aeb127640 1 -- 10.20.193.28:0/1453257317 >> 10.20.193.28:0/1453257317 conn(0x7f0ae40eaca0 msgr2=0x7f0ae410b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:29.726 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.724+0000 7f0aeb127640 1 -- 10.20.193.28:0/1453257317 shutdown_connections 2026-02-25T11:55:29.727 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.724+0000 7f0aeb127640 1 -- 10.20.193.28:0/1453257317 wait complete. 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.724+0000 7f0aeb127640 1 Processor -- start 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aeb127640 1 -- start start 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aeb127640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0ae410d3d0 0x7f0ae40f59b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aeb127640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0ae410ddc0 0x7f0ae4173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aeb127640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0ae41135c0 0x7f0ae40f60b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aeb127640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae40fb2c0 con 0x7f0ae410d3d0 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aeb127640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae40fb430 con 0x7f0ae410ddc0 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aeb127640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae40fb5f0 con 0x7f0ae41135c0 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aea125640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0ae410d3d0 0x7f0ae40f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aea125640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0ae410d3d0 0x7f0ae40f59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47656/0 (socket says 10.20.193.28:47656) 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aea125640 1 -- 10.20.193.28:0/3148313958 learned_addr learned my addr 10.20.193.28:0/3148313958 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aea926640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0ae41135c0 0x7f0ae40f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0ae9924640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0ae410ddc0 0x7f0ae4173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aea125640 1 -- 10.20.193.28:0/3148313958 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0ae41135c0 msgr2=0x7f0ae40f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aea125640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0ae41135c0 0x7f0ae40f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aea125640 1 -- 10.20.193.28:0/3148313958 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0ae410ddc0 msgr2=0x7f0ae4173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aea125640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0ae410ddc0 0x7f0ae4173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.725+0000 7f0aea125640 1 -- 10.20.193.28:0/3148313958 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad8009570 con 0x7f0ae410d3d0 2026-02-25T11:55:29.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.726+0000 7f0aea125640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0ae410d3d0 0x7f0ae40f59b0 secure :-1 s=READY pgs=760 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ad4009480 tx=0x7f0ad400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:29.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.726+0000 7f0acb7fe640 1 -- 10.20.193.28:0/3148313958 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0ad400cd40 con 0x7f0ae410d3d0 2026-02-25T11:55:29.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.726+0000 7f0acb7fe640 1 -- 10.20.193.28:0/3148313958 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0ad4004d10 con 0x7f0ae410d3d0 2026-02-25T11:55:29.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.726+0000 7f0acb7fe640 1 -- 10.20.193.28:0/3148313958 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0ad40106b0 con 0x7f0ae410d3d0 2026-02-25T11:55:29.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.726+0000 7f0aeb127640 1 -- 10.20.193.28:0/3148313958 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae40fb8d0 con 0x7f0ae410d3d0 2026-02-25T11:55:29.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.726+0000 7f0aeb127640 1 -- 10.20.193.28:0/3148313958 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae40fbd50 con 0x7f0ae410d3d0 2026-02-25T11:55:29.730 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.729+0000 7f0acb7fe640 1 -- 10.20.193.28:0/3148313958 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f0ad401f060 con 0x7f0ae410d3d0 2026-02-25T11:55:29.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.730+0000 7f0acb7fe640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0ac00b57d0 0x7f0ac00d5c80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.730+0000 7f0acb7fe640 1 -- 10.20.193.28:0/3148313958 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f0ad4014070 con 0x7f0ae410d3d0 2026-02-25T11:55:29.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.730+0000 7f0ae9924640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0ac00b57d0 0x7f0ac00d5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.731+0000 7f0ae9924640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0ac00b57d0 0x7f0ac00d5c80 secure :-1 s=READY pgs=355 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ad80023d0 tx=0x7f0ad8031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:29.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.732+0000 7f0aee3abe00 1 Processor -- start 2026-02-25T11:55:29.734 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.732+0000 7f0aee3abe00 1 -- start start 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.732+0000 7f0aee3abe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af01f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.732+0000 7f0aee3abe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af02440 0x556e2af46ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.732+0000 7f0aee3abe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af470e0 0x556e2af674c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.732+0000 7f0aee3abe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x556e2ae178a0 con 0x556e2af02440 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.733+0000 7f0aee3abe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x556e2ae135d0 con 0x556e2af470e0 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.733+0000 7f0aee3abe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x556e2ae0d570 con 0x556e2af01b30 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.733+0000 7f0ae9924640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af02440 0x556e2af46ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.733+0000 7f0aea125640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af01f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.733+0000 7f0ae9924640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af02440 0x556e2af46ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47664/0 (socket says 10.20.193.28:47664) 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.733+0000 7f0ae9924640 1 -- 10.20.193.28:0/1228599010 learned_addr learned my addr 10.20.193.28:0/1228599010 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.733+0000 7f0aea926640 1 --2- 10.20.193.28:0/1228599010 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af470e0 0x556e2af674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.733+0000 7f0ae9924640 1 -- 10.20.193.28:0/1228599010 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 msgr2=0x556e2af01f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.733+0000 7f0ae9924640 1 --2- 10.20.193.28:0/1228599010 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af01f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.733+0000 7f0ae9924640 1 -- 10.20.193.28:0/1228599010 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af470e0 msgr2=0x556e2af674c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.733+0000 7f0ae9924640 1 --2- 10.20.193.28:0/1228599010 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af470e0 0x556e2af674c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.733+0000 7f0ae9924640 1 -- 10.20.193.28:0/1228599010 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e2adb8d10 con 0x556e2af02440 2026-02-25T11:55:29.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.733+0000 7f0ae9924640 1 --2- 10.20.193.28:0/1228599010 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af02440 0x556e2af46ba0 secure :-1 s=READY pgs=761 gs=2 cs=0 l=1 c_cookie=3163978678725167302 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ad8009950 tx=0x7f0ad8040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be8b5d84652c8c6 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:29.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.734+0000 7f0ac8884640 1 -- 10.20.193.28:0/1228599010 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0ad802f9f0 con 0x556e2af02440 2026-02-25T11:55:29.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.734+0000 7f0ac8884640 1 -- 10.20.193.28:0/1228599010 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0ad8038b60 con 0x556e2af02440 2026-02-25T11:55:29.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.734+0000 7f0ac8884640 1 -- 10.20.193.28:0/1228599010 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0ad804f7a0 con 0x556e2af02440 2026-02-25T11:55:29.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.734+0000 7f0aee3abe00 1 -- 10.20.193.28:0/1228599010 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af02440 msgr2=0x556e2af46ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.734+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/1228599010 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af02440 0x556e2af46ba0 secure :-1 s=READY pgs=761 gs=2 cs=0 l=1 c_cookie=3163978678725167302 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ad8009950 tx=0x7f0ad8040000 comp rx=0 tx=0).stop 2026-02-25T11:55:29.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.734+0000 7f0aee3abe00 1 -- 10.20.193.28:0/1228599010 shutdown_connections 2026-02-25T11:55:29.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.734+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/1228599010 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af470e0 0x556e2af674c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.734+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/1228599010 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af02440 0x556e2af46ba0 unknown :-1 s=CLOSED pgs=761 gs=2 cs=0 l=1 c_cookie=3163978678725167302 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.734+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/1228599010 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af01f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.734+0000 7f0aee3abe00 1 -- 10.20.193.28:0/1228599010 >> 10.20.193.28:0/1228599010 conn(0x556e2ae207a0 msgr2=0x556e2af00410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:29.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.734+0000 7f0aee3abe00 1 -- 10.20.193.28:0/1228599010 shutdown_connections 2026-02-25T11:55:29.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.734+0000 7f0aee3abe00 1 -- 10.20.193.28:0/1228599010 wait complete. 2026-02-25T11:55:29.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.735+0000 7f0aee3abe00 1 -- start start 2026-02-25T11:55:29.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.735+0000 7f0aee3abe00 1 Processor -- start 2026-02-25T11:55:29.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aee3abe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af22b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aee3abe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af02440 0x556e2af23060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aee3abe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2af24c20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aee3abe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x556e2af25160 con 0x556e2af470e0 2026-02-25T11:55:29.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aee3abe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x556e2af252d0 con 0x556e2af02440 2026-02-25T11:55:29.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aee3abe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x556e2af25490 con 0x556e2af01b30 2026-02-25T11:55:29.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aea926640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2af24c20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aea926640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2af24c20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47674/0 (socket says 10.20.193.28:47674) 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aea926640 1 -- 10.20.193.28:0/2766685240 learned_addr learned my addr 10.20.193.28:0/2766685240 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aea125640 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af22b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0ae9924640 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af02440 0x556e2af23060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aea926640 1 -- 10.20.193.28:0/2766685240 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 msgr2=0x556e2af22b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aea926640 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af22b20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aea926640 1 -- 10.20.193.28:0/2766685240 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af02440 msgr2=0x556e2af23060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aea926640 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af02440 0x556e2af23060 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aea926640 1 -- 10.20.193.28:0/2766685240 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e2ad913c0 con 0x556e2af470e0 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.736+0000 7f0aea125640 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af22b20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.737+0000 7f0aea926640 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2af24c20 secure :-1 s=READY pgs=762 gs=2 cs=0 l=1 c_cookie=15327682855914912616 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ae0003ae0 tx=0x7f0ae00055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b6de5910a73b68 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.737+0000 7f0a97fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.737+0000 7f0a97fff640 1 -- 10.20.193.28:0/2766685240 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0ae0012660 con 0x556e2af470e0 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.737+0000 7f0a97fff640 1 -- 10.20.193.28:0/2766685240 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0ae0008e40 con 0x556e2af470e0 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.737+0000 7f0a97fff640 1 -- 10.20.193.28:0/2766685240 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0ae0011da0 con 0x556e2af470e0 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.737+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2766685240 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae4174800 con 0x556e2af470e0 2026-02-25T11:55:29.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.737+0000 7f0a97fff640 1 -- 10.20.193.28:0/2766685240 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f0ae00127c0 con 0x556e2af470e0 2026-02-25T11:55:29.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.737+0000 7f0aee3abe00 10 client.17040 _ll_register_callbacks cb 0x556e2ad8ea10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:29.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.737+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2766685240 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556e2aaa36f0 con 0x556e2af470e0 2026-02-25T11:55:29.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.737+0000 7f0a97fff640 1 -- 10.20.193.28:0/2766685240 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0ae0012980 con 0x556e2af470e0 2026-02-25T11:55:29.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.737+0000 7f0aee3abe00 10 client.17040 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:29.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.737+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2766685240 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x556e2ada30e0 con 0x556e2af470e0 2026-02-25T11:55:29.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.738+0000 7f0a97fff640 1 -- 10.20.193.28:0/2766685240 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f0ae0012b20 con 0x556e2af470e0 2026-02-25T11:55:29.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.738+0000 7f0aee3abe00 10 client.17040 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:29.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.738+0000 7f0aee3abe00 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:29.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.738+0000 7f0aee3abe00 20 client.17040 populate_metadata read hostname 'trial028' 2026-02-25T11:55:29.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.738+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e2af2d290 0x556e2af2d730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.738+0000 7f0aee3abe00 4 client.17040 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:55:29.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.738+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2766685240 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e2addffc0 con 0x556e2af2d290 2026-02-25T11:55:29.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.738+0000 7f0aea125640 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e2af2d290 0x556e2af2d730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.739+0000 7f0aea125640 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e2af2d290 0x556e2af2d730 crc :-1 s=READY pgs=138 gs=4 cs=0 l=0 c_cookie=6468470058016614371 s_cookie=18289902595748109331 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c49f3f1054a7e3 server_cookie=fdd2c981bc0f9c13 in_seq=0 out_seq=0 2026-02-25T11:55:29.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.739+0000 7f0a97fff640 10 client.17040 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:29.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0a97fff640 1 -- 10.20.193.28:0/2766685240 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x556e2addffc0 con 0x556e2af2d290 2026-02-25T11:55:29.742 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0a97fff640 10 client.17040 handle_command_reply: tid=0 2026-02-25T11:55:29.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0aee3abe00 1 client.17040 shutdown 2026-02-25T11:55:29.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2766685240 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 msgr2=0x556e2af24c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2af24c20 secure :-1 s=READY pgs=762 gs=2 cs=0 l=1 c_cookie=15327682855914912616 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ae0003ae0 tx=0x7f0ae00055c0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2766685240 shutdown_connections 2026-02-25T11:55:29.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e2af2d290 0x556e2af2d730 crc :-1 s=READY pgs=138 gs=4 cs=0 l=0 c_cookie=6468470058016614371 s_cookie=18289902595748109331 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2af24c20 unknown :-1 s=CLOSED pgs=762 gs=2 cs=0 l=1 c_cookie=15327682855914912616 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af02440 0x556e2af23060 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2766685240 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af22b20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0a97fff640 0 client.17040 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:29.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2766685240 >> 10.20.193.28:0/2766685240 conn(0x556e2ae207a0 msgr2=0x556e2aee9520 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:29.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2766685240 shutdown_connections 2026-02-25T11:55:29.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2766685240 wait complete. 2026-02-25T11:55:29.743 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.740+0000 7f0aee3abe00 20 client.17040 trim_cache size 0 max 16384 2026-02-25T11:55:29.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:29 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/192893221' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:55:29.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.747+0000 7f0aee3abe00 1 Processor -- start 2026-02-25T11:55:29.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.747+0000 7f0aee3abe00 1 -- start start 2026-02-25T11:55:29.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aee3abe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af01b30 0x556e2aee0320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aee3abe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af02440 0x556e2aefeea0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aee3abe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2aeec540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.749 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aee3abe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x556e2ae0b1a0 con 0x556e2af470e0 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aee3abe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x556e2aacb700 con 0x556e2af01b30 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aee3abe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x556e2af23660 con 0x556e2af02440 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aea926640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2aeec540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0ae9924640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af02440 0x556e2aefeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aea926640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2aeec540 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47688/0 (socket says 10.20.193.28:47688) 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aea926640 1 -- 10.20.193.28:0/2394828600 learned_addr learned my addr 10.20.193.28:0/2394828600 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aea125640 1 --2- 10.20.193.28:0/2394828600 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af01b30 0x556e2aee0320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aea926640 1 -- 10.20.193.28:0/2394828600 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af02440 msgr2=0x556e2aefeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aea926640 1 --2- 10.20.193.28:0/2394828600 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af02440 0x556e2aefeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aea926640 1 -- 10.20.193.28:0/2394828600 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af01b30 msgr2=0x556e2aee0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aea926640 1 --2- 10.20.193.28:0/2394828600 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af01b30 0x556e2aee0320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.748+0000 7f0aea926640 1 -- 10.20.193.28:0/2394828600 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e2aef43b0 con 0x556e2af470e0 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.749+0000 7f0aea926640 1 --2- 10.20.193.28:0/2394828600 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2aeec540 secure :-1 s=READY pgs=763 gs=3 cs=0 l=1 c_cookie=2079147113596203354 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ae001c9d0 tx=0x7f0ae0025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cda9d48b5ae315a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.749+0000 7f0ab24e6640 1 -- 10.20.193.28:0/2394828600 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0ae0008e40 con 0x556e2af470e0 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.749+0000 7f0ab24e6640 1 -- 10.20.193.28:0/2394828600 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x556e2aaa36f0 con 0x556e2af470e0 2026-02-25T11:55:29.750 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.749+0000 7f0ab24e6640 1 -- 10.20.193.28:0/2394828600 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0ae0011c00 con 0x556e2af470e0 2026-02-25T11:55:29.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.749+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2394828600 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 msgr2=0x556e2aeec540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.749+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2394828600 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2aeec540 secure :-1 s=READY pgs=763 gs=3 cs=0 l=1 c_cookie=2079147113596203354 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ae001c9d0 tx=0x7f0ae0025000 comp rx=0 tx=0).stop 2026-02-25T11:55:29.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.749+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2394828600 shutdown_connections 2026-02-25T11:55:29.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.749+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2394828600 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2aeec540 unknown :-1 s=CLOSED pgs=763 gs=3 cs=0 l=1 c_cookie=2079147113596203354 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.749+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2394828600 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af02440 0x556e2aefeea0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.749+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2394828600 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af01b30 0x556e2aee0320 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.749+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2394828600 >> 10.20.193.28:0/2394828600 conn(0x556e2ae207a0 msgr2=0x556e2aedfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:29.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.749+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2394828600 shutdown_connections 2026-02-25T11:55:29.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.749+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2394828600 wait complete. 2026-02-25T11:55:29.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.750+0000 7f0aee3abe00 1 -- start start 2026-02-25T11:55:29.751 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.750+0000 7f0aee3abe00 1 Processor -- start 2026-02-25T11:55:29.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.750+0000 7f0aee3abe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af33c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.750+0000 7f0aee3abe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af02440 0x556e2af2e280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.750+0000 7f0aee3abe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2af2e970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.750+0000 7f0aee3abe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x556e2af2f070 con 0x556e2af470e0 2026-02-25T11:55:29.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.750+0000 7f0aee3abe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x556e2affb690 con 0x556e2af02440 2026-02-25T11:55:29.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.750+0000 7f0aee3abe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x556e2affb7d0 con 0x556e2af01b30 2026-02-25T11:55:29.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0aea125640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af33c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0aea125640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af33c10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:49140/0 (socket says 10.20.193.28:49140) 2026-02-25T11:55:29.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0aea125640 1 -- 10.20.193.28:0/2896584207 learned_addr learned my addr 10.20.193.28:0/2896584207 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:29.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0ae9924640 1 --2- 10.20.193.28:0/2896584207 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af02440 0x556e2af2e280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0aea926640 1 --2- 10.20.193.28:0/2896584207 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2af2e970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.752 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0aea125640 1 -- 10.20.193.28:0/2896584207 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af02440 msgr2=0x556e2af2e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0aea125640 1 --2- 10.20.193.28:0/2896584207 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af02440 0x556e2af2e280 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0aea125640 1 -- 10.20.193.28:0/2896584207 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 msgr2=0x556e2af2e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0aea125640 1 --2- 10.20.193.28:0/2896584207 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2af2e970 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0aea125640 1 -- 10.20.193.28:0/2896584207 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e2ad9db60 con 0x556e2af01b30 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0aea125640 1 --2- 10.20.193.28:0/2896584207 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af33c10 secure :-1 s=READY pgs=201 gs=1 cs=0 l=1 c_cookie=1540306311543548256 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ad400c6a0 tx=0x7f0ad40054e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1560446df0d86960 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0a97fff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0a97fff640 1 -- 10.20.193.28:0/2896584207 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0ad4072ea0 con 0x556e2af01b30 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2896584207 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae001ce70 con 0x556e2af01b30 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0a97fff640 1 -- 10.20.193.28:0/2896584207 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0ad40b5340 con 0x556e2af01b30 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0aee3abe00 10 client.26743 _ll_register_callbacks cb 0x556e2ad8ea10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0a97fff640 1 -- 10.20.193.28:0/2896584207 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0ad40c4ea0 con 0x556e2af01b30 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.751+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2896584207 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad8038b60 con 0x556e2af01b30 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.752+0000 7f0a97fff640 1 -- 10.20.193.28:0/2896584207 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f0ad40adcd0 con 0x556e2af01b30 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.752+0000 7f0a97fff640 1 -- 10.20.193.28:0/2896584207 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0ad40ade90 con 0x556e2af01b30 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.752+0000 7f0aee3abe00 10 client.26743 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:29.753 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.752+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2896584207 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x556e2ad916f0 con 0x556e2af01b30 2026-02-25T11:55:29.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.752+0000 7f0a97fff640 1 -- 10.20.193.28:0/2896584207 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f0ad4078020 con 0x556e2af01b30 2026-02-25T11:55:29.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.752+0000 7f0aee3abe00 10 client.26743 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:29.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.752+0000 7f0aee3abe00 10 client.26743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:29.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.752+0000 7f0aee3abe00 20 client.26743 populate_metadata read hostname 'trial028' 2026-02-25T11:55:29.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.752+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2896584207 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e2affd520 0x556e2b01d9d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:29.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.752+0000 7f0aee3abe00 4 client.26743 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:55:29.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.752+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2896584207 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e2aaa0f40 con 0x556e2affd520 2026-02-25T11:55:29.754 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.752+0000 7f0ae9924640 1 --2- 10.20.193.28:0/2896584207 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e2affd520 0x556e2b01d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:29.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.753+0000 7f0ae9924640 1 --2- 10.20.193.28:0/2896584207 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e2affd520 0x556e2b01d9d0 crc :-1 s=READY pgs=139 gs=4 cs=0 l=0 c_cookie=13933043501577717349 s_cookie=2790735467754301180 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15c1d3e879ac665 server_cookie=26baaf1f399426fc in_seq=0 out_seq=0 2026-02-25T11:55:29.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.753+0000 7f0a97fff640 10 client.26743 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:29.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0a97fff640 1 -- 10.20.193.28:0/2896584207 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e2aaa0f40 con 0x556e2affd520 2026-02-25T11:55:29.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0a97fff640 10 client.26743 handle_command_reply: tid=0 2026-02-25T11:55:29.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aee3abe00 1 client.26743 shutdown 2026-02-25T11:55:29.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2896584207 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 msgr2=0x556e2af33c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.755 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2896584207 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af33c10 secure :-1 s=READY pgs=201 gs=1 cs=0 l=1 c_cookie=1540306311543548256 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ad400c6a0 tx=0x7f0ad40054e0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2896584207 shutdown_connections 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2896584207 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x556e2affd520 0x556e2b01d9d0 crc :-1 s=READY pgs=139 gs=4 cs=0 l=0 c_cookie=13933043501577717349 s_cookie=2790735467754301180 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2896584207 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x556e2af470e0 0x556e2af2e970 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2896584207 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x556e2af02440 0x556e2af2e280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aee3abe00 1 --2- 10.20.193.28:0/2896584207 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x556e2af01b30 0x556e2af33c10 unknown :-1 s=CLOSED pgs=201 gs=1 cs=0 l=1 c_cookie=1540306311543548256 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0a97fff640 0 client.26743 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2896584207 >> 10.20.193.28:0/2896584207 conn(0x556e2ae207a0 msgr2=0x556e2aee0400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2896584207 shutdown_connections 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aee3abe00 1 -- 10.20.193.28:0/2896584207 wait complete. 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aee3abe00 20 client.26743 trim_cache size 0 max 16384 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aeb127640 1 -- 10.20.193.28:0/3148313958 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0ac00b57d0 msgr2=0x7f0ac00d5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aeb127640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0ac00b57d0 0x7f0ac00d5c80 secure :-1 s=READY pgs=355 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ad80023d0 tx=0x7f0ad8031040 comp rx=0 tx=0).stop 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aeb127640 1 -- 10.20.193.28:0/3148313958 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0ae410d3d0 msgr2=0x7f0ae40f59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.754+0000 7f0aeb127640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0ae410d3d0 0x7f0ae40f59b0 secure :-1 s=READY pgs=760 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0ad4009480 tx=0x7f0ad400ee60 comp rx=0 tx=0).stop 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.755+0000 7f0aeb127640 1 -- 10.20.193.28:0/3148313958 shutdown_connections 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.755+0000 7f0aeb127640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0ac00b57d0 0x7f0ac00d5c80 unknown :-1 s=CLOSED pgs=355 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.755+0000 7f0aeb127640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0ae41135c0 0x7f0ae40f60b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.755+0000 7f0aeb127640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0ae410ddc0 0x7f0ae4173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.755+0000 7f0aeb127640 1 --2- 10.20.193.28:0/3148313958 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0ae410d3d0 0x7f0ae40f59b0 unknown :-1 s=CLOSED pgs=760 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:29.756 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.755+0000 7f0aeb127640 1 -- 10.20.193.28:0/3148313958 >> 10.20.193.28:0/3148313958 conn(0x7f0ae40eaca0 msgr2=0x7f0ae410fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:29.757 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.755+0000 7f0aeb127640 1 -- 10.20.193.28:0/3148313958 shutdown_connections 2026-02-25T11:55:29.757 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:29.755+0000 7f0aeb127640 1 -- 10.20.193.28:0/3148313958 wait complete. 2026-02-25T11:55:29.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:29 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/192893221' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:55:30.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:30 trial028 ceph-mon[39224]: pgmap v444: 97 pgs: 97 active+clean; 5.7 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:55:30.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:30 trial167 ceph-mon[43392]: pgmap v444: 97 pgs: 97 active+clean; 5.7 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:55:30.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:30 trial119 ceph-mon[42710]: pgmap v444: 97 pgs: 97 active+clean; 5.7 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 14 op/s 2026-02-25T11:55:30.890 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:55:30.891 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:55:31.076 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:55:31.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.258+0000 7f0e2bfff640 1 -- 10.20.193.28:0/2845796410 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e2c115880 msgr2=0x7f0e2c176250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.258+0000 7f0e2bfff640 1 --2- 10.20.193.28:0/2845796410 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e2c115880 0x7f0e2c176250 secure :-1 s=READY pgs=764 gs=1 cs=0 l=1 c_cookie=15475063318765301505 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e1c00aa30 tx=0x7f0e1c030270 comp rx=0 tx=0).stop 2026-02-25T11:55:31.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.258+0000 7f0e2bfff640 1 -- 10.20.193.28:0/2845796410 shutdown_connections 2026-02-25T11:55:31.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.258+0000 7f0e2bfff640 1 --2- 10.20.193.28:0/2845796410 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e2c115880 0x7f0e2c176250 unknown :-1 s=CLOSED pgs=764 gs=1 cs=0 l=1 c_cookie=15475063318765301505 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.258+0000 7f0e2bfff640 1 --2- 10.20.193.28:0/2845796410 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0e2c110080 0x7f0e2c1150d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.258+0000 7f0e2bfff640 1 --2- 10.20.193.28:0/2845796410 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0e2c10f690 0x7f0e2c10fa60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.258+0000 7f0e2bfff640 1 -- 10.20.193.28:0/2845796410 >> 10.20.193.28:0/2845796410 conn(0x7f0e2c0ecf00 msgr2=0x7f0e2c10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:31.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.258+0000 7f0e2bfff640 1 -- 10.20.193.28:0/2845796410 shutdown_connections 2026-02-25T11:55:31.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.258+0000 7f0e2bfff640 1 -- 10.20.193.28:0/2845796410 wait complete. 2026-02-25T11:55:31.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.259+0000 7f0e2bfff640 1 Processor -- start 2026-02-25T11:55:31.260 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.259+0000 7f0e2bfff640 1 -- start start 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.259+0000 7f0e2bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0e2c10f690 0x7f0e2c0f5940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.259+0000 7f0e2bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0e2c110080 0x7f0e2c176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.259+0000 7f0e2bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e2c115880 0x7f0e2c0f6040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.259+0000 7f0e2bfff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c0fb200 con 0x7f0e2c115880 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.259+0000 7f0e2bfff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c0fb370 con 0x7f0e2c110080 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.259+0000 7f0e2bfff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e2c0fb530 con 0x7f0e2c10f690 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.259+0000 7f0e2b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e2c115880 0x7f0e2c0f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.259+0000 7f0e2b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e2c115880 0x7f0e2c0f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47718/0 (socket says 10.20.193.28:47718) 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.259+0000 7f0e2b7fe640 1 -- 10.20.193.28:0/2030314890 learned_addr learned my addr 10.20.193.28:0/2030314890 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e2affd640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0e2c10f690 0x7f0e2c0f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e2b7fe640 1 -- 10.20.193.28:0/2030314890 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0e2c10f690 msgr2=0x7f0e2c0f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e2a7fc640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0e2c110080 0x7f0e2c176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e2b7fe640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0e2c10f690 0x7f0e2c0f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e2b7fe640 1 -- 10.20.193.28:0/2030314890 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0e2c110080 msgr2=0x7f0e2c176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e2b7fe640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0e2c110080 0x7f0e2c176150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e2b7fe640 1 -- 10.20.193.28:0/2030314890 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c00a650 con 0x7f0e2c115880 2026-02-25T11:55:31.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e2a7fc640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0e2c110080 0x7f0e2c176150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:55:31.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e2b7fe640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e2c115880 0x7f0e2c0f6040 secure :-1 s=READY pgs=765 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e1c032fe0 tx=0x7f0e1c0039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:31.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e30a3e640 1 -- 10.20.193.28:0/2030314890 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0e1c03e070 con 0x7f0e2c115880 2026-02-25T11:55:31.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e30a3e640 1 -- 10.20.193.28:0/2030314890 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0e1c003e00 con 0x7f0e2c115880 2026-02-25T11:55:31.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e30a3e640 1 -- 10.20.193.28:0/2030314890 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0e1c042940 con 0x7f0e2c115880 2026-02-25T11:55:31.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e2bfff640 1 -- 10.20.193.28:0/2030314890 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e2c0fb7b0 con 0x7f0e2c115880 2026-02-25T11:55:31.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.260+0000 7f0e2bfff640 1 -- 10.20.193.28:0/2030314890 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2c0fbbd0 con 0x7f0e2c115880 2026-02-25T11:55:31.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.264+0000 7f0e30a3e640 1 -- 10.20.193.28:0/2030314890 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f0e1c0050c0 con 0x7f0e2c115880 2026-02-25T11:55:31.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.265+0000 7f0e30a3e640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0e000b5750 0x7f0e000d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.266+0000 7f0e30a3e640 1 -- 10.20.193.28:0/2030314890 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f0e1c10c690 con 0x7f0e2c115880 2026-02-25T11:55:31.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.266+0000 7f0e2affd640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0e000b5750 0x7f0e000d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.266+0000 7f0e33a3be00 1 Processor -- start 2026-02-25T11:55:31.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.266+0000 7f0e33a3be00 1 -- start start 2026-02-25T11:55:31.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.266+0000 7f0e2affd640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0e000b5750 0x7f0e000d5c00 secure :-1 s=READY pgs=356 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e20004740 tx=0x7f0e20004420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:31.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.266+0000 7f0e33a3be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617606e2750 0x5617606e2b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.266+0000 7f0e33a3be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x561760727610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.266+0000 7f0e33a3be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561760727b50 0x561760747f30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.266+0000 7f0e33a3be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5617605f8d80 con 0x5617606e3060 2026-02-25T11:55:31.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.266+0000 7f0e33a3be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5617605f4ab0 con 0x561760727b50 2026-02-25T11:55:31.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.266+0000 7f0e33a3be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5617605eea50 con 0x5617606e2750 2026-02-25T11:55:31.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0e2a7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x561760727610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0e2a7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x561760727610 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47724/0 (socket says 10.20.193.28:47724) 2026-02-25T11:55:31.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0e2a7fc640 1 -- 10.20.193.28:0/1524734207 learned_addr learned my addr 10.20.193.28:0/1524734207 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:31.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0e2affd640 1 --2- 10.20.193.28:0/1524734207 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617606e2750 0x5617606e2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0e2a7fc640 1 -- 10.20.193.28:0/1524734207 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617606e2750 msgr2=0x5617606e2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0e2a7fc640 1 --2- 10.20.193.28:0/1524734207 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617606e2750 0x5617606e2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0e2a7fc640 1 -- 10.20.193.28:0/1524734207 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561760727b50 msgr2=0x561760747f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0e2b7fe640 1 --2- 10.20.193.28:0/1524734207 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561760727b50 0x561760747f30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0e2a7fc640 1 --2- 10.20.193.28:0/1524734207 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561760727b50 0x561760747f30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0e2a7fc640 1 -- 10.20.193.28:0/1524734207 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176059a120 con 0x5617606e3060 2026-02-25T11:55:31.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0e2b7fe640 1 --2- 10.20.193.28:0/1524734207 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561760727b50 0x561760747f30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:55:31.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0e2a7fc640 1 --2- 10.20.193.28:0/1524734207 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x561760727610 secure :-1 s=READY pgs=766 gs=2 cs=0 l=1 c_cookie=45656924793811291 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e1400cc30 tx=0x7f0e1400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a234bb2fdf315b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:31.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0deffff640 1 -- 10.20.193.28:0/1524734207 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0e140040d0 con 0x5617606e3060 2026-02-25T11:55:31.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0deffff640 1 -- 10.20.193.28:0/1524734207 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0e14004230 con 0x5617606e3060 2026-02-25T11:55:31.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.267+0000 7f0deffff640 1 -- 10.20.193.28:0/1524734207 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0e14010820 con 0x5617606e3060 2026-02-25T11:55:31.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.268+0000 7f0e33a3be00 1 -- 10.20.193.28:0/1524734207 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 msgr2=0x561760727610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.268+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/1524734207 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x561760727610 secure :-1 s=READY pgs=766 gs=2 cs=0 l=1 c_cookie=45656924793811291 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e1400cc30 tx=0x7f0e1400e720 comp rx=0 tx=0).stop 2026-02-25T11:55:31.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.268+0000 7f0e33a3be00 1 -- 10.20.193.28:0/1524734207 shutdown_connections 2026-02-25T11:55:31.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.268+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/1524734207 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561760727b50 0x561760747f30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.268+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/1524734207 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x561760727610 unknown :-1 s=CLOSED pgs=766 gs=2 cs=0 l=1 c_cookie=45656924793811291 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.268+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/1524734207 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617606e2750 0x5617606e2b20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.268+0000 7f0e33a3be00 1 -- 10.20.193.28:0/1524734207 >> 10.20.193.28:0/1524734207 conn(0x5617606c08f0 msgr2=0x5617606e0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:31.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.268+0000 7f0e33a3be00 1 -- 10.20.193.28:0/1524734207 shutdown_connections 2026-02-25T11:55:31.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.268+0000 7f0e33a3be00 1 -- 10.20.193.28:0/1524734207 wait complete. 2026-02-25T11:55:31.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.269+0000 7f0e33a3be00 1 -- start start 2026-02-25T11:55:31.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.269+0000 7f0e33a3be00 1 Processor -- start 2026-02-25T11:55:31.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.270+0000 7f0e33a3be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617606e2750 0x5617606e3760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.270+0000 7f0e33a3be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x5617606e3ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.270+0000 7f0e33a3be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561760727b50 0x561760708610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.270+0000 7f0e33a3be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x561760708b80 con 0x5617606e3060 2026-02-25T11:55:31.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.270+0000 7f0e33a3be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561760708cf0 con 0x5617606e2750 2026-02-25T11:55:31.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.270+0000 7f0e33a3be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x561760708eb0 con 0x561760727b50 2026-02-25T11:55:31.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.270+0000 7f0e2a7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x5617606e3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.270+0000 7f0e2a7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x5617606e3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47728/0 (socket says 10.20.193.28:47728) 2026-02-25T11:55:31.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.270+0000 7f0e2a7fc640 1 -- 10.20.193.28:0/3119834683 learned_addr learned my addr 10.20.193.28:0/3119834683 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:31.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.270+0000 7f0e2b7fe640 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561760727b50 0x561760708610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.270+0000 7f0e2affd640 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617606e2750 0x5617606e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0e2b7fe640 1 -- 10.20.193.28:0/3119834683 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617606e2750 msgr2=0x5617606e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0e2b7fe640 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617606e2750 0x5617606e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0e2b7fe640 1 -- 10.20.193.28:0/3119834683 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 msgr2=0x5617606e3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0e2b7fe640 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x5617606e3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0e2b7fe640 1 -- 10.20.193.28:0/3119834683 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561760572810 con 0x561760727b50 2026-02-25T11:55:31.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0e2a7fc640 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x5617606e3ca0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:55:31.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0e2b7fe640 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561760727b50 0x561760708610 secure :-1 s=READY pgs=202 gs=3 cs=0 l=1 c_cookie=6753566812915715604 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e1c0d2c80 tx=0x7f0e1c0d2bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5db97d3c02b54e14 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:31.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0ded7fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:55:31.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0ded7fa640 1 -- 10.20.193.28:0/3119834683 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0e1c0dd8e0 con 0x561760727b50 2026-02-25T11:55:31.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0e33a3be00 1 -- 10.20.193.28:0/3119834683 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1400c850 con 0x561760727b50 2026-02-25T11:55:31.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0e33a3be00 10 client.26755 _ll_register_callbacks cb 0x56176056fe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:31.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0e33a3be00 1 -- 10.20.193.28:0/3119834683 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561760346530 con 0x561760727b50 2026-02-25T11:55:31.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0ded7fa640 1 -- 10.20.193.28:0/3119834683 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0e1c0d8730 con 0x561760727b50 2026-02-25T11:55:31.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0ded7fa640 1 -- 10.20.193.28:0/3119834683 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0e1c0d3920 con 0x561760727b50 2026-02-25T11:55:31.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0ded7fa640 1 -- 10.20.193.28:0/3119834683 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f0e1c0d09f0 con 0x561760727b50 2026-02-25T11:55:31.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0ded7fa640 1 -- 10.20.193.28:0/3119834683 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0e1c0d0c10 con 0x561760727b50 2026-02-25T11:55:31.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0e33a3be00 10 client.26755 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:31.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.271+0000 7f0e33a3be00 1 -- 10.20.193.28:0/3119834683 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5617605844f0 con 0x561760727b50 2026-02-25T11:55:31.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.272+0000 7f0ded7fa640 1 -- 10.20.193.28:0/3119834683 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f0e1c0d4400 con 0x561760727b50 2026-02-25T11:55:31.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.272+0000 7f0e33a3be00 10 client.26755 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:31.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.272+0000 7f0e33a3be00 10 client.26755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:31.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.272+0000 7f0e33a3be00 20 client.26755 populate_metadata read hostname 'trial028' 2026-02-25T11:55:31.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.272+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56176070d440 0x56176070d8e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.272+0000 7f0e33a3be00 4 client.26755 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:55:31.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.272+0000 7f0e33a3be00 1 -- 10.20.193.28:0/3119834683 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56176059bb50 con 0x56176070d440 2026-02-25T11:55:31.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.272+0000 7f0e2affd640 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56176070d440 0x56176070d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.273+0000 7f0e2affd640 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56176070d440 0x56176070d8e0 crc :-1 s=READY pgs=140 gs=4 cs=0 l=0 c_cookie=12213996730821678548 s_cookie=14919168431776666090 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a980d5506c6895d4 server_cookie=cf0b88a4c04511ea in_seq=0 out_seq=0 2026-02-25T11:55:31.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.273+0000 7f0ded7fa640 10 client.26755 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:31.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0ded7fa640 1 -- 10.20.193.28:0/3119834683 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x56176059bb50 con 0x56176070d440 2026-02-25T11:55:31.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0ded7fa640 10 client.26755 handle_command_reply: tid=0 2026-02-25T11:55:31.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0e33a3be00 1 client.26755 shutdown 2026-02-25T11:55:31.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0e33a3be00 1 -- 10.20.193.28:0/3119834683 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561760727b50 msgr2=0x561760708610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561760727b50 0x561760708610 secure :-1 s=READY pgs=202 gs=3 cs=0 l=1 c_cookie=6753566812915715604 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e1c0d2c80 tx=0x7f0e1c0d2bc0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0e33a3be00 1 -- 10.20.193.28:0/3119834683 shutdown_connections 2026-02-25T11:55:31.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56176070d440 0x56176070d8e0 crc :-1 s=READY pgs=140 gs=4 cs=0 l=0 c_cookie=12213996730821678548 s_cookie=14919168431776666090 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561760727b50 0x561760708610 unknown :-1 s=CLOSED pgs=202 gs=3 cs=0 l=1 c_cookie=6753566812915715604 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x5617606e3ca0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/3119834683 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617606e2750 0x5617606e3760 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0e33a3be00 1 -- 10.20.193.28:0/3119834683 >> 10.20.193.28:0/3119834683 conn(0x5617606c08f0 msgr2=0x5617606e0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:31.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0ded7fa640 0 client.26755 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:31.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0e33a3be00 1 -- 10.20.193.28:0/3119834683 shutdown_connections 2026-02-25T11:55:31.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0e33a3be00 1 -- 10.20.193.28:0/3119834683 wait complete. 2026-02-25T11:55:31.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.274+0000 7f0e33a3be00 20 client.26755 trim_cache size 0 max 16384 2026-02-25T11:55:31.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e33a3be00 1 Processor -- start 2026-02-25T11:55:31.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e33a3be00 1 -- start start 2026-02-25T11:55:31.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e33a3be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e2750 0x5617606e3c50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e33a3be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617606e3060 0x5617607450d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e33a3be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561760727b50 0x561760745610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e33a3be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5617605ec680 con 0x5617606e2750 2026-02-25T11:55:31.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e33a3be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5617602ac790 con 0x561760727b50 2026-02-25T11:55:31.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e33a3be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x561760740f80 con 0x5617606e3060 2026-02-25T11:55:31.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e2affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e2750 0x5617606e3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e2affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e2750 0x5617606e3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47744/0 (socket says 10.20.193.28:47744) 2026-02-25T11:55:31.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e2affd640 1 -- 10.20.193.28:0/1715005737 learned_addr learned my addr 10.20.193.28:0/1715005737 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:31.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e2a7fc640 1 --2- 10.20.193.28:0/1715005737 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617606e3060 0x5617607450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e2b7fe640 1 --2- 10.20.193.28:0/1715005737 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561760727b50 0x561760745610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e2affd640 1 -- 10.20.193.28:0/1715005737 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617606e3060 msgr2=0x5617607450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e2affd640 1 --2- 10.20.193.28:0/1715005737 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617606e3060 0x5617607450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e2affd640 1 -- 10.20.193.28:0/1715005737 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561760727b50 msgr2=0x561760745610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e2affd640 1 --2- 10.20.193.28:0/1715005737 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561760727b50 0x561760745610 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.284+0000 7f0e2affd640 1 -- 10.20.193.28:0/1715005737 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561760728580 con 0x5617606e2750 2026-02-25T11:55:31.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.285+0000 7f0e2affd640 1 --2- 10.20.193.28:0/1715005737 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e2750 0x5617606e3c50 secure :-1 s=READY pgs=767 gs=1 cs=0 l=1 c_cookie=16139774471799628980 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e200046f0 tx=0x7f0e2000b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffbff5eb288e4b4 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:31.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.285+0000 7f0def7fe640 1 -- 10.20.193.28:0/1715005737 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0e20025040 con 0x5617606e2750 2026-02-25T11:55:31.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.285+0000 7f0def7fe640 1 -- 10.20.193.28:0/1715005737 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0e2001fdd0 con 0x5617606e2750 2026-02-25T11:55:31.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.285+0000 7f0def7fe640 1 -- 10.20.193.28:0/1715005737 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0e200298b0 con 0x5617606e2750 2026-02-25T11:55:31.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.285+0000 7f0e33a3be00 1 -- 10.20.193.28:0/1715005737 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e2750 msgr2=0x5617606e3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.285+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/1715005737 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e2750 0x5617606e3c50 secure :-1 s=READY pgs=767 gs=1 cs=0 l=1 c_cookie=16139774471799628980 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e200046f0 tx=0x7f0e2000b000 comp rx=0 tx=0).stop 2026-02-25T11:55:31.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.285+0000 7f0e33a3be00 1 -- 10.20.193.28:0/1715005737 shutdown_connections 2026-02-25T11:55:31.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.285+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/1715005737 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x561760727b50 0x561760745610 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.285+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/1715005737 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5617606e3060 0x5617607450d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.285+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/1715005737 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e2750 0x5617606e3c50 unknown :-1 s=CLOSED pgs=767 gs=1 cs=0 l=1 c_cookie=16139774471799628980 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.285+0000 7f0e33a3be00 1 -- 10.20.193.28:0/1715005737 >> 10.20.193.28:0/1715005737 conn(0x5617606c08f0 msgr2=0x5617606e3540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:31.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.285+0000 7f0e33a3be00 1 -- 10.20.193.28:0/1715005737 shutdown_connections 2026-02-25T11:55:31.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.285+0000 7f0e33a3be00 1 -- 10.20.193.28:0/1715005737 wait complete. 2026-02-25T11:55:31.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.286+0000 7f0e33a3be00 1 -- start start 2026-02-25T11:55:31.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.286+0000 7f0e33a3be00 1 Processor -- start 2026-02-25T11:55:31.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.286+0000 7f0e33a3be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617606e2750 0x56176071cb80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.286+0000 7f0e33a3be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x56176071d0c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.286+0000 7f0e33a3be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561760727b50 0x56176071d600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.286+0000 7f0e33a3be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56176073fa80 con 0x5617606e3060 2026-02-25T11:55:31.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.286+0000 7f0e33a3be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x561760709e20 con 0x5617606e2750 2026-02-25T11:55:31.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.286+0000 7f0e33a3be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x561760651670 con 0x561760727b50 2026-02-25T11:55:31.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.286+0000 7f0e2a7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x56176071d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.286+0000 7f0e2a7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x56176071d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47752/0 (socket says 10.20.193.28:47752) 2026-02-25T11:55:31.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.286+0000 7f0e2a7fc640 1 -- 10.20.193.28:0/4114926987 learned_addr learned my addr 10.20.193.28:0/4114926987 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:31.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0e2affd640 1 --2- 10.20.193.28:0/4114926987 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617606e2750 0x56176071cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0e2b7fe640 1 --2- 10.20.193.28:0/4114926987 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561760727b50 0x56176071d600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0e2a7fc640 1 -- 10.20.193.28:0/4114926987 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561760727b50 msgr2=0x56176071d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0e2a7fc640 1 --2- 10.20.193.28:0/4114926987 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561760727b50 0x56176071d600 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0e2a7fc640 1 -- 10.20.193.28:0/4114926987 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617606e2750 msgr2=0x56176071cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0e2a7fc640 1 --2- 10.20.193.28:0/4114926987 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617606e2750 0x56176071cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0e2a7fc640 1 -- 10.20.193.28:0/4114926987 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176057ef70 con 0x5617606e3060 2026-02-25T11:55:31.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0e2a7fc640 1 --2- 10.20.193.28:0/4114926987 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x56176071d0c0 secure :-1 s=READY pgs=768 gs=1 cs=0 l=1 c_cookie=15814909195399203129 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e1400c530 tx=0x7f0e1400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db79d81f4ad40939 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:31.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0ded7fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:31.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0ded7fa640 1 -- 10.20.193.28:0/4114926987 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0e1401f030 con 0x5617606e3060 2026-02-25T11:55:31.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0ded7fa640 1 -- 10.20.193.28:0/4114926987 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0e14020040 con 0x5617606e3060 2026-02-25T11:55:31.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0ded7fa640 1 -- 10.20.193.28:0/4114926987 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0e14019450 con 0x5617606e3060 2026-02-25T11:55:31.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0e33a3be00 1 -- 10.20.193.28:0/4114926987 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e2000fa10 con 0x5617606e3060 2026-02-25T11:55:31.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0ded7fa640 1 -- 10.20.193.28:0/4114926987 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f0e140195b0 con 0x5617606e3060 2026-02-25T11:55:31.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0e33a3be00 10 client.17064 _ll_register_callbacks cb 0x56176056fe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:31.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0e33a3be00 1 -- 10.20.193.28:0/4114926987 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561760282220 con 0x5617606e3060 2026-02-25T11:55:31.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.287+0000 7f0ded7fa640 1 -- 10.20.193.28:0/4114926987 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0e140046e0 con 0x5617606e3060 2026-02-25T11:55:31.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.288+0000 7f0e33a3be00 10 client.17064 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:31.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.288+0000 7f0e33a3be00 1 -- 10.20.193.28:0/4114926987 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x561760572b40 con 0x5617606e3060 2026-02-25T11:55:31.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.288+0000 7f0ded7fa640 1 -- 10.20.193.28:0/4114926987 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f0e140048c0 con 0x5617606e3060 2026-02-25T11:55:31.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.288+0000 7f0e33a3be00 10 client.17064 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:31.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.288+0000 7f0e33a3be00 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:31.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.288+0000 7f0e33a3be00 20 client.17064 populate_metadata read hostname 'trial028' 2026-02-25T11:55:31.297 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.288+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/4114926987 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5617607be4c0 0x5617607be960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.288+0000 7f0e33a3be00 4 client.17064 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:55:31.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.288+0000 7f0e33a3be00 1 -- 10.20.193.28:0/4114926987 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f0e14004230 con 0x5617607be4c0 2026-02-25T11:55:31.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.288+0000 7f0e2affd640 1 --2- 10.20.193.28:0/4114926987 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5617607be4c0 0x5617607be960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.289+0000 7f0e2affd640 1 --2- 10.20.193.28:0/4114926987 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5617607be4c0 0x5617607be960 crc :-1 s=READY pgs=141 gs=4 cs=0 l=0 c_cookie=7344548603845335892 s_cookie=18396250537657957849 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ed140ac7cadb54 server_cookie=ff4c9c658030fdd9 in_seq=0 out_seq=0 2026-02-25T11:55:31.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.289+0000 7f0ded7fa640 10 client.17064 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:31.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.289+0000 7f0ded7fa640 1 -- 10.20.193.28:0/4114926987 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f0e14004230 con 0x5617607be4c0 2026-02-25T11:55:31.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.289+0000 7f0ded7fa640 10 client.17064 handle_command_reply: tid=0 2026-02-25T11:55:31.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.289+0000 7f0e33a3be00 1 client.17064 shutdown 2026-02-25T11:55:31.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e33a3be00 1 -- 10.20.193.28:0/4114926987 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 msgr2=0x56176071d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/4114926987 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x56176071d0c0 secure :-1 s=READY pgs=768 gs=1 cs=0 l=1 c_cookie=15814909195399203129 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e1400c530 tx=0x7f0e1400cf40 comp rx=0 tx=0).stop 2026-02-25T11:55:31.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e33a3be00 1 -- 10.20.193.28:0/4114926987 shutdown_connections 2026-02-25T11:55:31.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/4114926987 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5617607be4c0 0x5617607be960 crc :-1 s=READY pgs=141 gs=4 cs=0 l=0 c_cookie=7344548603845335892 s_cookie=18396250537657957849 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/4114926987 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x561760727b50 0x56176071d600 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/4114926987 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5617606e3060 0x56176071d0c0 unknown :-1 s=CLOSED pgs=768 gs=1 cs=0 l=1 c_cookie=15814909195399203129 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e33a3be00 1 --2- 10.20.193.28:0/4114926987 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5617606e2750 0x56176071cb80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e33a3be00 1 -- 10.20.193.28:0/4114926987 >> 10.20.193.28:0/4114926987 conn(0x5617606c08f0 msgr2=0x561760726ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:31.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0ded7fa640 0 client.17064 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:31.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e33a3be00 1 -- 10.20.193.28:0/4114926987 shutdown_connections 2026-02-25T11:55:31.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e33a3be00 1 -- 10.20.193.28:0/4114926987 wait complete. 2026-02-25T11:55:31.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e33a3be00 20 client.17064 trim_cache size 0 max 16384 2026-02-25T11:55:31.300 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:55:31.300 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:55:31.300 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "7b237934-6b95-43dd-b3a6-82b74134ec55", 2026-02-25T11:55:31.300 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:55:31.300 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:55:31.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e2bfff640 1 -- 10.20.193.28:0/2030314890 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0e000b5750 msgr2=0x7f0e000d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e2bfff640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0e000b5750 0x7f0e000d5c00 secure :-1 s=READY pgs=356 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e20004740 tx=0x7f0e20004420 comp rx=0 tx=0).stop 2026-02-25T11:55:31.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e2bfff640 1 -- 10.20.193.28:0/2030314890 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e2c115880 msgr2=0x7f0e2c0f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.290+0000 7f0e2bfff640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e2c115880 0x7f0e2c0f6040 secure :-1 s=READY pgs=765 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e1c032fe0 tx=0x7f0e1c0039b0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.291+0000 7f0e2bfff640 1 -- 10.20.193.28:0/2030314890 shutdown_connections 2026-02-25T11:55:31.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.291+0000 7f0e2bfff640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0e000b5750 0x7f0e000d5c00 unknown :-1 s=CLOSED pgs=356 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.291+0000 7f0e2bfff640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e2c115880 0x7f0e2c0f6040 unknown :-1 s=CLOSED pgs=765 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.291+0000 7f0e2bfff640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0e2c110080 0x7f0e2c176150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.291+0000 7f0e2bfff640 1 --2- 10.20.193.28:0/2030314890 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0e2c10f690 0x7f0e2c0f5940 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.291+0000 7f0e2bfff640 1 -- 10.20.193.28:0/2030314890 >> 10.20.193.28:0/2030314890 conn(0x7f0e2c0ecf00 msgr2=0x7f0e2c111e90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:31.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.291+0000 7f0e2bfff640 1 -- 10.20.193.28:0/2030314890 shutdown_connections 2026-02-25T11:55:31.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.291+0000 7f0e2bfff640 1 -- 10.20.193.28:0/2030314890 wait complete. 2026-02-25T11:55:31.391 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:55:31.574 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:55:31.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.753+0000 7f3f64fef640 1 -- 10.20.193.28:0/2033660977 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6010c9f0 msgr2=0x7f3f6010cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.753+0000 7f3f64fef640 1 --2- 10.20.193.28:0/2033660977 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6010c9f0 0x7f3f6010cdc0 secure :-1 s=READY pgs=769 gs=1 cs=0 l=1 c_cookie=17267025136634206292 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f4c009950 tx=0x7f3f4c02f190 comp rx=0 tx=0).stop 2026-02-25T11:55:31.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.754+0000 7f3f64fef640 1 -- 10.20.193.28:0/2033660977 shutdown_connections 2026-02-25T11:55:31.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.754+0000 7f3f64fef640 1 --2- 10.20.193.28:0/2033660977 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3f60112470 0x7f3f60172e40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.754+0000 7f3f64fef640 1 --2- 10.20.193.28:0/2033660977 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3f6010d3e0 0x7f3f60111cc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.754+0000 7f3f64fef640 1 --2- 10.20.193.28:0/2033660977 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6010c9f0 0x7f3f6010cdc0 unknown :-1 s=CLOSED pgs=769 gs=1 cs=0 l=1 c_cookie=17267025136634206292 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.754+0000 7f3f64fef640 1 -- 10.20.193.28:0/2033660977 >> 10.20.193.28:0/2033660977 conn(0x7f3f600e9290 msgr2=0x7f3f601096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:31.758 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.754+0000 7f3f64fef640 1 -- 10.20.193.28:0/2033660977 shutdown_connections 2026-02-25T11:55:31.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.754+0000 7f3f64fef640 1 -- 10.20.193.28:0/2033660977 wait complete. 2026-02-25T11:55:31.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.754+0000 7f3f64fef640 1 Processor -- start 2026-02-25T11:55:31.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f64fef640 1 -- start start 2026-02-25T11:55:31.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f64fef640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6010c9f0 0x7f3f600f5170 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f64fef640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3f6010d3e0 0x7f3f60172d40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f64fef640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3f60112470 0x7f3f600f5870 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f64fef640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3f600faa80 con 0x7f3f6010c9f0 2026-02-25T11:55:31.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f64fef640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3f600fabf0 con 0x7f3f6010d3e0 2026-02-25T11:55:31.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f64fef640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3f600fadb0 con 0x7f3f60112470 2026-02-25T11:55:31.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f5f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6010c9f0 0x7f3f600f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f5f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6010c9f0 0x7f3f600f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47764/0 (socket says 10.20.193.28:47764) 2026-02-25T11:55:31.759 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f5f7fe640 1 -- 10.20.193.28:0/489452593 learned_addr learned my addr 10.20.193.28:0/489452593 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:31.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f5effd640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3f6010d3e0 0x7f3f60172d40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f5ffff640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3f60112470 0x7f3f600f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f5f7fe640 1 -- 10.20.193.28:0/489452593 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3f60112470 msgr2=0x7f3f600f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3f60112470 0x7f3f600f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f5f7fe640 1 -- 10.20.193.28:0/489452593 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3f6010d3e0 msgr2=0x7f3f60172d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3f6010d3e0 0x7f3f60172d40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f5f7fe640 1 -- 10.20.193.28:0/489452593 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4c009570 con 0x7f3f6010c9f0 2026-02-25T11:55:31.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f5effd640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3f6010d3e0 0x7f3f60172d40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:55:31.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.755+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6010c9f0 0x7f3f600f5170 secure :-1 s=READY pgs=770 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f4c0098a0 tx=0x7f3f4c0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:31.760 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.756+0000 7f3f5cff9640 1 -- 10.20.193.28:0/489452593 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3f4c03d070 con 0x7f3f6010c9f0 2026-02-25T11:55:31.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.756+0000 7f3f5cff9640 1 -- 10.20.193.28:0/489452593 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3f4c002a70 con 0x7f3f6010c9f0 2026-02-25T11:55:31.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.756+0000 7f3f5cff9640 1 -- 10.20.193.28:0/489452593 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3f4c041890 con 0x7f3f6010c9f0 2026-02-25T11:55:31.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.756+0000 7f3f64fef640 1 -- 10.20.193.28:0/489452593 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f600fb030 con 0x7f3f6010c9f0 2026-02-25T11:55:31.761 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.756+0000 7f3f64fef640 1 -- 10.20.193.28:0/489452593 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f600fb450 con 0x7f3f6010c9f0 2026-02-25T11:55:31.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.758+0000 7f3f5cff9640 1 -- 10.20.193.28:0/489452593 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f3f4c041a20 con 0x7f3f6010c9f0 2026-02-25T11:55:31.762 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.760+0000 7f3f5cff9640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3f340b5750 0x7f3f340d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.760+0000 7f3f5cff9640 1 -- 10.20.193.28:0/489452593 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f3f4c10b280 con 0x7f3f6010c9f0 2026-02-25T11:55:31.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.760+0000 7f3f5effd640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3f340b5750 0x7f3f340d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.761+0000 7f3f5effd640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3f340b5750 0x7f3f340d5c00 secure :-1 s=READY pgs=357 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f50005970 tx=0x7f3f50005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:31.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.761+0000 7f3f68273e00 1 Processor -- start 2026-02-25T11:55:31.763 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.762+0000 7f3f68273e00 1 -- start start 2026-02-25T11:55:31.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.762+0000 7f3f68273e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3140 0x55ee5fba3510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.762+0000 7f3f68273e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fba3a50 0x55ee5fbe81b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.762+0000 7f3f68273e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fbe86f0 0x55ee5fc08ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.762+0000 7f3f68273e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ee5fab99b0 con 0x55ee5fbe86f0 2026-02-25T11:55:31.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.762+0000 7f3f68273e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ee5fab56e0 con 0x55ee5fba3140 2026-02-25T11:55:31.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.762+0000 7f3f68273e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ee5faaf840 con 0x55ee5fba3a50 2026-02-25T11:55:31.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.762+0000 7f3f5ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fbe86f0 0x55ee5fc08ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.762+0000 7f3f5ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fbe86f0 0x55ee5fc08ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47780/0 (socket says 10.20.193.28:47780) 2026-02-25T11:55:31.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.762+0000 7f3f5ffff640 1 -- 10.20.193.28:0/2869590518 learned_addr learned my addr 10.20.193.28:0/2869590518 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:31.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.762+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/2869590518 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3140 0x55ee5fba3510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.762+0000 7f3f5effd640 1 --2- 10.20.193.28:0/2869590518 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fba3a50 0x55ee5fbe81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.762+0000 7f3f5ffff640 1 -- 10.20.193.28:0/2869590518 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fba3a50 msgr2=0x55ee5fbe81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f5ffff640 1 --2- 10.20.193.28:0/2869590518 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fba3a50 0x55ee5fbe81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f5ffff640 1 -- 10.20.193.28:0/2869590518 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3140 msgr2=0x55ee5fba3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f5ffff640 1 --2- 10.20.193.28:0/2869590518 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3140 0x55ee5fba3510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f5ffff640 1 -- 10.20.193.28:0/2869590518 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee5fa5ad80 con 0x55ee5fbe86f0 2026-02-25T11:55:31.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f5ffff640 1 --2- 10.20.193.28:0/2869590518 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fbe86f0 0x55ee5fc08ad0 secure :-1 s=READY pgs=771 gs=1 cs=0 l=1 c_cookie=4765288891758303945 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f54006a50 tx=0x7f3f54003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4221b4ffdd5e5ac9 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:31.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f23fff640 1 -- 10.20.193.28:0/2869590518 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3f54010c20 con 0x55ee5fbe86f0 2026-02-25T11:55:31.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f23fff640 1 -- 10.20.193.28:0/2869590518 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3f54010d80 con 0x55ee5fbe86f0 2026-02-25T11:55:31.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f23fff640 1 -- 10.20.193.28:0/2869590518 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3f5401a810 con 0x55ee5fbe86f0 2026-02-25T11:55:31.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f68273e00 1 -- 10.20.193.28:0/2869590518 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fbe86f0 msgr2=0x55ee5fc08ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f68273e00 1 --2- 10.20.193.28:0/2869590518 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fbe86f0 0x55ee5fc08ad0 secure :-1 s=READY pgs=771 gs=1 cs=0 l=1 c_cookie=4765288891758303945 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f54006a50 tx=0x7f3f54003e70 comp rx=0 tx=0).stop 2026-02-25T11:55:31.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f68273e00 1 -- 10.20.193.28:0/2869590518 shutdown_connections 2026-02-25T11:55:31.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f68273e00 1 --2- 10.20.193.28:0/2869590518 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fbe86f0 0x55ee5fc08ad0 unknown :-1 s=CLOSED pgs=771 gs=1 cs=0 l=1 c_cookie=4765288891758303945 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f68273e00 1 --2- 10.20.193.28:0/2869590518 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fba3a50 0x55ee5fbe81b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f68273e00 1 --2- 10.20.193.28:0/2869590518 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3140 0x55ee5fba3510 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f68273e00 1 -- 10.20.193.28:0/2869590518 >> 10.20.193.28:0/2869590518 conn(0x55ee5fac2d40 msgr2=0x55ee5fba1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:31.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f68273e00 1 -- 10.20.193.28:0/2869590518 shutdown_connections 2026-02-25T11:55:31.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.763+0000 7f3f68273e00 1 -- 10.20.193.28:0/2869590518 wait complete. 2026-02-25T11:55:31.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.765+0000 7f3f68273e00 1 -- start start 2026-02-25T11:55:31.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.765+0000 7f3f68273e00 1 Processor -- start 2026-02-25T11:55:31.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.765+0000 7f3f68273e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3140 0x55ee5fc00460 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.765+0000 7f3f68273e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3a50 0x55ee5fc009a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.765+0000 7f3f68273e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fbe86f0 0x55ee5fc05990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.765+0000 7f3f68273e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ee5fc05ed0 con 0x55ee5fba3a50 2026-02-25T11:55:31.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.765+0000 7f3f68273e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ee5fc06040 con 0x55ee5fba3140 2026-02-25T11:55:31.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.765+0000 7f3f68273e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ee5fc06200 con 0x55ee5fbe86f0 2026-02-25T11:55:31.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.765+0000 7f3f5effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3a50 0x55ee5fc009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.765+0000 7f3f5effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3a50 0x55ee5fc009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47788/0 (socket says 10.20.193.28:47788) 2026-02-25T11:55:31.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.765+0000 7f3f5effd640 1 -- 10.20.193.28:0/3456468888 learned_addr learned my addr 10.20.193.28:0/3456468888 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:31.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f5ffff640 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fbe86f0 0x55ee5fc05990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3140 0x55ee5fc00460 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f5effd640 1 -- 10.20.193.28:0/3456468888 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fbe86f0 msgr2=0x55ee5fc05990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f5effd640 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fbe86f0 0x55ee5fc05990 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f5effd640 1 -- 10.20.193.28:0/3456468888 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3140 msgr2=0x55ee5fc00460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f5effd640 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3140 0x55ee5fc00460 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f5effd640 1 -- 10.20.193.28:0/3456468888 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee5fa33350 con 0x55ee5fba3a50 2026-02-25T11:55:31.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3140 0x55ee5fc00460 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:55:31.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f5effd640 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3a50 0x55ee5fc009a0 secure :-1 s=READY pgs=772 gs=1 cs=0 l=1 c_cookie=9407867893919314113 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f500027b0 tx=0x7f3f50004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828f760126dcf0c1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:31.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f217fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:31.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f217fa640 1 -- 10.20.193.28:0/3456468888 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3f5001d630 con 0x55ee5fba3a50 2026-02-25T11:55:31.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f217fa640 1 -- 10.20.193.28:0/3456468888 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3f50011e30 con 0x55ee5fba3a50 2026-02-25T11:55:31.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f68273e00 1 -- 10.20.193.28:0/3456468888 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f540055b0 con 0x55ee5fba3a50 2026-02-25T11:55:31.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f217fa640 1 -- 10.20.193.28:0/3456468888 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3f50010c80 con 0x55ee5fba3a50 2026-02-25T11:55:31.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.766+0000 7f3f68273e00 10 client.17076 _ll_register_callbacks cb 0x55ee5fa309a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:31.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.767+0000 7f3f217fa640 1 -- 10.20.193.28:0/3456468888 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f3f50010e10 con 0x55ee5fba3a50 2026-02-25T11:55:31.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.767+0000 7f3f68273e00 1 -- 10.20.193.28:0/3456468888 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee5f7456f0 con 0x55ee5fba3a50 2026-02-25T11:55:31.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.767+0000 7f3f217fa640 1 -- 10.20.193.28:0/3456468888 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3f500103d0 con 0x55ee5fba3a50 2026-02-25T11:55:31.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.767+0000 7f3f68273e00 10 client.17076 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:31.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.767+0000 7f3f68273e00 1 -- 10.20.193.28:0/3456468888 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55ee5fa45150 con 0x55ee5fba3a50 2026-02-25T11:55:31.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.767+0000 7f3f217fa640 1 -- 10.20.193.28:0/3456468888 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f3f5001a040 con 0x55ee5fba3a50 2026-02-25T11:55:31.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.767+0000 7f3f68273e00 10 client.17076 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:31.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.767+0000 7f3f68273e00 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:31.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.767+0000 7f3f68273e00 20 client.17076 populate_metadata read hostname 'trial028' 2026-02-25T11:55:31.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.767+0000 7f3f68273e00 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ee5fc08ab0 0x55ee5fbe6420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.767+0000 7f3f68273e00 4 client.17076 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:55:31.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.767+0000 7f3f68273e00 1 -- 10.20.193.28:0/3456468888 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee5fa81910 con 0x55ee5fc08ab0 2026-02-25T11:55:31.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.768+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ee5fc08ab0 0x55ee5fbe6420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.768+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ee5fc08ab0 0x55ee5fbe6420 crc :-1 s=READY pgs=142 gs=4 cs=0 l=0 c_cookie=3455127534433786038 s_cookie=1327796953170697441 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff3142fbf98d8b6 server_cookie=126d484b3f95dce1 in_seq=0 out_seq=0 2026-02-25T11:55:31.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.768+0000 7f3f217fa640 10 client.17076 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:31.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.769+0000 7f3f217fa640 1 -- 10.20.193.28:0/3456468888 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55ee5fa81910 con 0x55ee5fc08ab0 2026-02-25T11:55:31.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.769+0000 7f3f217fa640 10 client.17076 handle_command_reply: tid=0 2026-02-25T11:55:31.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.769+0000 7f3f68273e00 1 client.17076 shutdown 2026-02-25T11:55:31.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.769+0000 7f3f68273e00 1 -- 10.20.193.28:0/3456468888 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3a50 msgr2=0x55ee5fc009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.769+0000 7f3f68273e00 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3a50 0x55ee5fc009a0 secure :-1 s=READY pgs=772 gs=1 cs=0 l=1 c_cookie=9407867893919314113 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f500027b0 tx=0x7f3f50004770 comp rx=0 tx=0).stop 2026-02-25T11:55:31.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.769+0000 7f3f68273e00 1 -- 10.20.193.28:0/3456468888 shutdown_connections 2026-02-25T11:55:31.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.769+0000 7f3f68273e00 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ee5fc08ab0 0x55ee5fbe6420 crc :-1 s=READY pgs=142 gs=4 cs=0 l=0 c_cookie=3455127534433786038 s_cookie=1327796953170697441 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.769+0000 7f3f68273e00 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fbe86f0 0x55ee5fc05990 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.769+0000 7f3f68273e00 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3a50 0x55ee5fc009a0 unknown :-1 s=CLOSED pgs=772 gs=1 cs=0 l=1 c_cookie=9407867893919314113 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.769+0000 7f3f68273e00 1 --2- 10.20.193.28:0/3456468888 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3140 0x55ee5fc00460 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.769+0000 7f3f68273e00 1 -- 10.20.193.28:0/3456468888 >> 10.20.193.28:0/3456468888 conn(0x55ee5fac2d40 msgr2=0x55ee5fba1230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:31.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.769+0000 7f3f217fa640 0 client.17076 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:31.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.769+0000 7f3f68273e00 1 -- 10.20.193.28:0/3456468888 shutdown_connections 2026-02-25T11:55:31.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.770+0000 7f3f68273e00 1 -- 10.20.193.28:0/3456468888 wait complete. 2026-02-25T11:55:31.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.770+0000 7f3f68273e00 20 client.17076 trim_cache size 0 max 16384 2026-02-25T11:55:31.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.778+0000 7f3f68273e00 1 Processor -- start 2026-02-25T11:55:31.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f68273e00 1 -- start start 2026-02-25T11:55:31.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f68273e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3140 0x55ee5fbeb1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f68273e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3a50 0x55ee5fbe7a00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f68273e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fbe86f0 0x55ee5fbe7f40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.782 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f68273e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ee5faad470 con 0x55ee5fba3140 2026-02-25T11:55:31.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f68273e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ee5f76d700 con 0x55ee5fba3a50 2026-02-25T11:55:31.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f68273e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ee5fbf7da0 con 0x55ee5fbe86f0 2026-02-25T11:55:31.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f5f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3140 0x55ee5fbeb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f5f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3140 0x55ee5fbeb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:47792/0 (socket says 10.20.193.28:47792) 2026-02-25T11:55:31.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f5f7fe640 1 -- 10.20.193.28:0/1089917224 learned_addr learned my addr 10.20.193.28:0/1089917224 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:31.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f5ffff640 1 --2- 10.20.193.28:0/1089917224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fbe86f0 0x55ee5fbe7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f5effd640 1 --2- 10.20.193.28:0/1089917224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3a50 0x55ee5fbe7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f5f7fe640 1 -- 10.20.193.28:0/1089917224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fbe86f0 msgr2=0x55ee5fbe7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/1089917224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fbe86f0 0x55ee5fbe7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f5f7fe640 1 -- 10.20.193.28:0/1089917224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3a50 msgr2=0x55ee5fbe7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/1089917224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3a50 0x55ee5fbe7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f5f7fe640 1 -- 10.20.193.28:0/1089917224 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee5fa3fbd0 con 0x55ee5fba3140 2026-02-25T11:55:31.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f5effd640 1 --2- 10.20.193.28:0/1089917224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3a50 0x55ee5fbe7a00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:55:31.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.779+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/1089917224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3140 0x55ee5fbeb1e0 secure :-1 s=READY pgs=773 gs=1 cs=0 l=1 c_cookie=6472616926489135780 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f4c108dd0 tx=0x7f3f4c038f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d35acd1e90caa4 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:31.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.780+0000 7f3f237fe640 1 -- 10.20.193.28:0/1089917224 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3f4c0d2000 con 0x55ee5fba3140 2026-02-25T11:55:31.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.780+0000 7f3f237fe640 1 -- 10.20.193.28:0/1089917224 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3f4c0d7320 con 0x55ee5fba3140 2026-02-25T11:55:31.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.780+0000 7f3f237fe640 1 -- 10.20.193.28:0/1089917224 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3f4c0dc2b0 con 0x55ee5fba3140 2026-02-25T11:55:31.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.780+0000 7f3f68273e00 1 -- 10.20.193.28:0/1089917224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3140 msgr2=0x55ee5fbeb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.780+0000 7f3f68273e00 1 --2- 10.20.193.28:0/1089917224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3140 0x55ee5fbeb1e0 secure :-1 s=READY pgs=773 gs=1 cs=0 l=1 c_cookie=6472616926489135780 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f4c108dd0 tx=0x7f3f4c038f30 comp rx=0 tx=0).stop 2026-02-25T11:55:31.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.780+0000 7f3f68273e00 1 -- 10.20.193.28:0/1089917224 shutdown_connections 2026-02-25T11:55:31.784 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.780+0000 7f3f68273e00 1 --2- 10.20.193.28:0/1089917224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fbe86f0 0x55ee5fbe7f40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.780+0000 7f3f68273e00 1 --2- 10.20.193.28:0/1089917224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fba3a50 0x55ee5fbe7a00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.780+0000 7f3f68273e00 1 --2- 10.20.193.28:0/1089917224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3140 0x55ee5fbeb1e0 unknown :-1 s=CLOSED pgs=773 gs=1 cs=0 l=1 c_cookie=6472616926489135780 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.780+0000 7f3f68273e00 1 -- 10.20.193.28:0/1089917224 >> 10.20.193.28:0/1089917224 conn(0x55ee5fac2d40 msgr2=0x55ee5fbeaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:31.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.780+0000 7f3f68273e00 1 -- 10.20.193.28:0/1089917224 shutdown_connections 2026-02-25T11:55:31.785 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.780+0000 7f3f68273e00 1 -- 10.20.193.28:0/1089917224 wait complete. 2026-02-25T11:55:31.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.781+0000 7f3f68273e00 1 -- start start 2026-02-25T11:55:31.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.781+0000 7f3f68273e00 1 Processor -- start 2026-02-25T11:55:31.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.781+0000 7f3f68273e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3140 0x55ee5fb86f30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.781+0000 7f3f68273e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fba3a50 0x55ee5fb82410 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.781+0000 7f3f68273e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fbe86f0 0x55ee5fb82ac0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.781+0000 7f3f68273e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55ee5fb7e490 con 0x55ee5fba3140 2026-02-25T11:55:31.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.781+0000 7f3f68273e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55ee5fb831f0 con 0x55ee5fbe86f0 2026-02-25T11:55:31.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.781+0000 7f3f68273e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55ee5fc56fa0 con 0x55ee5fba3a50 2026-02-25T11:55:31.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.781+0000 7f3f5effd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fba3a50 0x55ee5fb82410 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.781+0000 7f3f5effd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fba3a50 0x55ee5fb82410 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:49250/0 (socket says 10.20.193.28:49250) 2026-02-25T11:55:31.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.781+0000 7f3f5effd640 1 -- 10.20.193.28:0/3496021112 learned_addr learned my addr 10.20.193.28:0/3496021112 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:55:31.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.781+0000 7f3f5ffff640 1 --2- 10.20.193.28:0/3496021112 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fbe86f0 0x55ee5fb82ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.781+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/3496021112 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3140 0x55ee5fb86f30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f5f7fe640 1 -- 10.20.193.28:0/3496021112 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fba3a50 msgr2=0x55ee5fb82410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/3496021112 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fba3a50 0x55ee5fb82410 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f5f7fe640 1 -- 10.20.193.28:0/3496021112 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fbe86f0 msgr2=0x55ee5fb82ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:55:31.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/3496021112 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fbe86f0 0x55ee5fb82ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f5f7fe640 1 -- 10.20.193.28:0/3496021112 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee5fa33680 con 0x55ee5fba3140 2026-02-25T11:55:31.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f5f7fe640 1 --2- 10.20.193.28:0/3496021112 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3140 0x55ee5fb86f30 secure :-1 s=READY pgs=774 gs=1 cs=0 l=1 c_cookie=10303022523063159820 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f4c0419f0 tx=0x7f3f4c0d10e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efbb065a0074c0c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:55:31.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f217fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:55:31.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f217fa640 1 -- 10.20.193.28:0/3496021112 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3f4c095070 con 0x55ee5fba3140 2026-02-25T11:55:31.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f217fa640 1 -- 10.20.193.28:0/3496021112 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f3f4c0dc740 con 0x55ee5fba3140 2026-02-25T11:55:31.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f217fa640 1 -- 10.20.193.28:0/3496021112 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3f4c0dc530 con 0x55ee5fba3140 2026-02-25T11:55:31.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f68273e00 1 -- 10.20.193.28:0/3496021112 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f600fb450 con 0x55ee5fba3140 2026-02-25T11:55:31.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f68273e00 10 client.17082 _ll_register_callbacks cb 0x55ee5fa309a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:55:31.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f68273e00 1 -- 10.20.193.28:0/3496021112 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f54010d80 con 0x55ee5fba3140 2026-02-25T11:55:31.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f217fa640 1 -- 10.20.193.28:0/3496021112 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f3f4c0d2000 con 0x55ee5fba3140 2026-02-25T11:55:31.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f217fa640 1 -- 10.20.193.28:0/3496021112 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3f4c0d2300 con 0x55ee5fba3140 2026-02-25T11:55:31.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f68273e00 10 client.17082 fetch_fsmap learned FSMap version 23 2026-02-25T11:55:31.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.782+0000 7f3f68273e00 1 -- 10.20.193.28:0/3496021112 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55ee5fa453b0 con 0x55ee5fba3140 2026-02-25T11:55:31.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.783+0000 7f3f217fa640 1 -- 10.20.193.28:0/3496021112 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f3f4c0cd870 con 0x55ee5fba3140 2026-02-25T11:55:31.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.783+0000 7f3f68273e00 10 client.17082 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:55:31.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.783+0000 7f3f68273e00 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:55:31.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.783+0000 7f3f68273e00 20 client.17082 populate_metadata read hostname 'trial028' 2026-02-25T11:55:31.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.783+0000 7f3f68273e00 1 --2- 10.20.193.28:0/3496021112 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ee5fc9eb00 0x55ee5fcbefb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:55:31.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.783+0000 7f3f68273e00 4 client.17082 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:55:31.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.783+0000 7f3f68273e00 1 -- 10.20.193.28:0/3496021112 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ee5f742f40 con 0x55ee5fc9eb00 2026-02-25T11:55:31.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.783+0000 7f3f5effd640 1 --2- 10.20.193.28:0/3496021112 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ee5fc9eb00 0x55ee5fcbefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:55:31.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.783+0000 7f3f5effd640 1 --2- 10.20.193.28:0/3496021112 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ee5fc9eb00 0x55ee5fcbefb0 crc :-1 s=READY pgs=143 gs=4 cs=0 l=0 c_cookie=14503112733377187446 s_cookie=5149232588529533946 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9456830b2644e76 server_cookie=4775bfc1e92b03fa in_seq=0 out_seq=0 2026-02-25T11:55:31.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.783+0000 7f3f217fa640 10 client.17082 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:31.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f217fa640 1 -- 10.20.193.28:0/3496021112 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x55ee5f742f40 con 0x55ee5fc9eb00 2026-02-25T11:55:31.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f217fa640 10 client.17082 handle_command_reply: tid=0 2026-02-25T11:55:31.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f68273e00 1 client.17082 shutdown 2026-02-25T11:55:31.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f68273e00 1 -- 10.20.193.28:0/3496021112 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3140 msgr2=0x55ee5fb86f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f68273e00 1 --2- 10.20.193.28:0/3496021112 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3140 0x55ee5fb86f30 secure :-1 s=READY pgs=774 gs=1 cs=0 l=1 c_cookie=10303022523063159820 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f4c0419f0 tx=0x7f3f4c0d10e0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f68273e00 1 -- 10.20.193.28:0/3496021112 shutdown_connections 2026-02-25T11:55:31.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f68273e00 1 --2- 10.20.193.28:0/3496021112 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55ee5fc9eb00 0x55ee5fcbefb0 crc :-1 s=READY pgs=143 gs=4 cs=0 l=0 c_cookie=14503112733377187446 s_cookie=5149232588529533946 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f68273e00 1 --2- 10.20.193.28:0/3496021112 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55ee5fbe86f0 0x55ee5fb82ac0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f68273e00 1 --2- 10.20.193.28:0/3496021112 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55ee5fba3a50 0x55ee5fb82410 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f68273e00 1 --2- 10.20.193.28:0/3496021112 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55ee5fba3140 0x55ee5fb86f30 unknown :-1 s=CLOSED pgs=774 gs=1 cs=0 l=1 c_cookie=10303022523063159820 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f217fa640 0 client.17082 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:55:31.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f68273e00 1 -- 10.20.193.28:0/3496021112 >> 10.20.193.28:0/3496021112 conn(0x55ee5fac2d40 msgr2=0x55ee5fbe69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:31.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f68273e00 1 -- 10.20.193.28:0/3496021112 shutdown_connections 2026-02-25T11:55:31.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.784+0000 7f3f68273e00 1 -- 10.20.193.28:0/3496021112 wait complete. 2026-02-25T11:55:31.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.785+0000 7f3f68273e00 20 client.17082 trim_cache size 0 max 16384 2026-02-25T11:55:31.793 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:55:31.794 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:55:31.794 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:55:31.794 INFO:teuthology.orchestra.run.trial028.stdout: "7b237934-6b95-43dd-b3a6-82b74134ec55": { 2026-02-25T11:55:31.794 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.794 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "7b237934-6b95-43dd-b3a6-82b74134ec55", 2026-02-25T11:55:31.794 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:55:31.794 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:55:31.794 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.794 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:55:31.795 INFO:teuthology.orchestra.run.trial028.stdout: "850bcfd3-be8a-4930-896c-1ed6e48b8fed": { 2026-02-25T11:55:31.795 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:27.124813+0000", 2026-02-25T11:55:31.795 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.795 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.795 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.795 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.795 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.795 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.795 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:55:31.795 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:55:31.795 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.795 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.796 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.796 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.796 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.796 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.796 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:55:31.796 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:55:31.796 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.796 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.796 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.796 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.796 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.796 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.797 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:55:31.797 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:55:31.797 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.797 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.797 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.797 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.797 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.797 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.797 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:55:31.797 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:55:31.797 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.797 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.798 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.798 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.798 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.798 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.798 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:55:31.798 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:55:31.798 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.798 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.798 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.799 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.799 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.799 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.799 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:55:31.799 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:55:31.799 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.799 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.799 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.799 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.799 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.799 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.799 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:55:31.800 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:55:31.800 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.800 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.800 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.800 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.800 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.800 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.800 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:55:31.800 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:55:31.800 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.800 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.800 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.801 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.801 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.801 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.801 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:55:31.801 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:55:31.801 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.801 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.801 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.801 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.801 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.801 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.801 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.802 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.803 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.804 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.804 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.804 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.804 INFO:teuthology.orchestra.run.trial028.stdout: "7b237934-6b95-43dd-b3a6-82b74134ec55": { 2026-02-25T11:55:31.804 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:31.289725+0000", 2026-02-25T11:55:31.804 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.804 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.804 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.804 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.804 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.804 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:55:31.804 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:55:31.804 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:55:31.805 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:55:31.805 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:55:31.805 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:55:31.805 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:55:31.805 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:55:31.805 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:55:31.805 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:55:31.805 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:55:31.805 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.785+0000 7f3f64fef640 1 -- 10.20.193.28:0/489452593 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3f340b5750 msgr2=0x7f3f340d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.805 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.785+0000 7f3f64fef640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3f340b5750 0x7f3f340d5c00 secure :-1 s=READY pgs=357 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f50005970 tx=0x7f3f50005900 comp rx=0 tx=0).stop 2026-02-25T11:55:31.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.785+0000 7f3f64fef640 1 -- 10.20.193.28:0/489452593 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6010c9f0 msgr2=0x7f3f600f5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:55:31.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.785+0000 7f3f64fef640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6010c9f0 0x7f3f600f5170 secure :-1 s=READY pgs=770 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3f4c0098a0 tx=0x7f3f4c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.785+0000 7f3f64fef640 1 -- 10.20.193.28:0/489452593 shutdown_connections 2026-02-25T11:55:31.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.785+0000 7f3f64fef640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f3f340b5750 0x7f3f340d5c00 unknown :-1 s=CLOSED pgs=357 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.785+0000 7f3f64fef640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f3f60112470 0x7f3f600f5870 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.785+0000 7f3f64fef640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f3f6010d3e0 0x7f3f60172d40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.785+0000 7f3f64fef640 1 --2- 10.20.193.28:0/489452593 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f3f6010c9f0 0x7f3f600f5170 unknown :-1 s=CLOSED pgs=770 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:55:31.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.785+0000 7f3f64fef640 1 -- 10.20.193.28:0/489452593 >> 10.20.193.28:0/489452593 conn(0x7f3f600e9290 msgr2=0x7f3f6010bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:55:31.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.785+0000 7f3f64fef640 1 -- 10.20.193.28:0/489452593 shutdown_connections 2026-02-25T11:55:31.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:55:31.785+0000 7f3f64fef640 1 -- 10.20.193.28:0/489452593 wait complete. 2026-02-25T11:55:31.886 INFO:tasks.cephfs.filesystem:scrub status for tag:7b237934-6b95-43dd-b3a6-82b74134ec55 - {'path': '/', 'tag': '7b237934-6b95-43dd-b3a6-82b74134ec55', 'options': 'recursive,force'} 2026-02-25T11:55:32.280 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:32 trial028 ceph-mon[39224]: pgmap v445: 97 pgs: 97 active+clean; 5.7 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:55:32.280 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:32 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:55:32.280 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:32 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:55:32.280 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:32 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:55:32.280 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:32 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:55:32.280 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:32 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:55:32.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:32 trial167 ceph-mon[43392]: pgmap v445: 97 pgs: 97 active+clean; 5.7 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:55:32.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:32 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:55:32.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:32 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:55:32.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:32 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:55:32.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:32 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:55:32.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:32 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:55:32.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:32 trial119 ceph-mon[42710]: pgmap v445: 97 pgs: 97 active+clean; 5.7 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:55:32.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:32 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:55:32.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:32 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:55:32.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:32 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:55:32.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:32 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:55:32.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:32 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:55:34.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:34 trial028 ceph-mon[39224]: pgmap v446: 97 pgs: 97 active+clean; 5.7 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 12 op/s 2026-02-25T11:55:34.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:34 trial167 ceph-mon[43392]: pgmap v446: 97 pgs: 97 active+clean; 5.7 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 12 op/s 2026-02-25T11:55:34.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:34 trial119 ceph-mon[42710]: pgmap v446: 97 pgs: 97 active+clean; 5.7 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 12 op/s 2026-02-25T11:55:36.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:36 trial028 ceph-mon[39224]: pgmap v447: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2026-02-25T11:55:36.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:36 trial167 ceph-mon[43392]: pgmap v447: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2026-02-25T11:55:36.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:36 trial119 ceph-mon[42710]: pgmap v447: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 21 KiB/s wr, 13 op/s 2026-02-25T11:55:38.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:38 trial028 ceph-mon[39224]: pgmap v448: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 11 op/s 2026-02-25T11:55:38.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:38 trial167 ceph-mon[43392]: pgmap v448: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 11 op/s 2026-02-25T11:55:38.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:38 trial119 ceph-mon[42710]: pgmap v448: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 11 op/s 2026-02-25T11:55:40.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:40 trial028 ceph-mon[39224]: pgmap v449: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:55:40.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:40 trial167 ceph-mon[43392]: pgmap v449: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:55:40.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:40 trial119 ceph-mon[42710]: pgmap v449: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:55:42.039 INFO:tasks.workunit.client.0.trial028.stdout:375/600: open fd = 3 2026-02-25T11:55:42.039 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.039 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.039 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.039 INFO:tasks.workunit.client.0.trial028.stdout:376/600: open fd = 3 2026-02-25T11:55:42.039 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.039 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.039 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.039 INFO:tasks.workunit.client.0.trial028.stdout:377/600: open fd = 3 2026-02-25T11:55:42.040 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.040 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.040 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.040 INFO:tasks.workunit.client.0.trial028.stdout:378/600: open fd = 3 2026-02-25T11:55:42.040 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.040 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.040 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.040 INFO:tasks.workunit.client.0.trial028.stdout:379/600: open fd = 3 2026-02-25T11:55:42.040 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.040 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.041 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.041 INFO:tasks.workunit.client.0.trial028.stdout:380/600: open fd = 3 2026-02-25T11:55:42.041 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.041 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.041 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.041 INFO:tasks.workunit.client.0.trial028.stdout:381/600: open fd = 3 2026-02-25T11:55:42.041 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.041 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.041 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.041 INFO:tasks.workunit.client.0.trial028.stdout:382/600: open fd = 3 2026-02-25T11:55:42.041 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.042 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.042 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.042 INFO:tasks.workunit.client.0.trial028.stdout:383/600: open fd = 3 2026-02-25T11:55:42.042 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.042 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.042 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.042 INFO:tasks.workunit.client.0.trial028.stdout:384/600: open fd = 3 2026-02-25T11:55:42.042 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.042 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.042 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.042 INFO:tasks.workunit.client.0.trial028.stdout:385/600: open fd = 3 2026-02-25T11:55:42.043 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.043 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.043 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.043 INFO:tasks.workunit.client.0.trial028.stdout:386/600: open fd = 3 2026-02-25T11:55:42.043 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.043 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.043 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.043 INFO:tasks.workunit.client.0.trial028.stdout:387/600: open fd = 3 2026-02-25T11:55:42.043 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.043 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.043 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.044 INFO:tasks.workunit.client.0.trial028.stdout:388/600: open fd = 3 2026-02-25T11:55:42.044 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.044 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.044 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.044 INFO:tasks.workunit.client.0.trial028.stdout:389/600: open fd = 3 2026-02-25T11:55:42.044 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.044 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.044 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.044 INFO:tasks.workunit.client.0.trial028.stdout:390/600: open fd = 3 2026-02-25T11:55:42.044 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.045 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.045 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.045 INFO:tasks.workunit.client.0.trial028.stdout:391/600: open fd = 3 2026-02-25T11:55:42.045 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.045 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.045 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.045 INFO:tasks.workunit.client.0.trial028.stdout:392/600: open fd = 3 2026-02-25T11:55:42.046 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.046 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.046 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.046 INFO:tasks.workunit.client.0.trial028.stdout:393/600: open fd = 3 2026-02-25T11:55:42.046 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.046 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.046 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.046 INFO:tasks.workunit.client.0.trial028.stdout:394/600: open fd = 3 2026-02-25T11:55:42.046 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.046 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.046 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.046 INFO:tasks.workunit.client.0.trial028.stdout:395/600: open fd = 3 2026-02-25T11:55:42.046 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.047 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.047 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.047 INFO:tasks.workunit.client.0.trial028.stdout:396/600: open fd = 3 2026-02-25T11:55:42.047 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.047 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.047 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.047 INFO:tasks.workunit.client.0.trial028.stdout:397/600: open fd = 3 2026-02-25T11:55:42.047 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.047 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.047 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.047 INFO:tasks.workunit.client.0.trial028.stdout:398/600: open fd = 3 2026-02-25T11:55:42.047 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.047 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.048 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.048 INFO:tasks.workunit.client.0.trial028.stdout:399/600: open fd = 3 2026-02-25T11:55:42.048 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.048 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.048 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.048 INFO:tasks.workunit.client.0.trial028.stdout:400/600: open fd = 3 2026-02-25T11:55:42.048 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.048 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.048 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.048 INFO:tasks.workunit.client.0.trial028.stdout:401/600: open fd = 3 2026-02-25T11:55:42.048 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.048 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.048 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.049 INFO:tasks.workunit.client.0.trial028.stdout:402/600: open fd = 3 2026-02-25T11:55:42.049 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.049 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.049 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.049 INFO:tasks.workunit.client.0.trial028.stdout:403/600: open fd = 3 2026-02-25T11:55:42.049 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.049 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.049 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.049 INFO:tasks.workunit.client.0.trial028.stdout:404/600: open fd = 3 2026-02-25T11:55:42.049 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.049 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.049 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.050 INFO:tasks.workunit.client.0.trial028.stdout:405/600: open fd = 3 2026-02-25T11:55:42.050 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.050 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.050 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.050 INFO:tasks.workunit.client.0.trial028.stdout:406/600: open fd = 3 2026-02-25T11:55:42.050 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.050 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.050 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.050 INFO:tasks.workunit.client.0.trial028.stdout:407/600: open fd = 3 2026-02-25T11:55:42.050 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.050 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.050 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.051 INFO:tasks.workunit.client.0.trial028.stdout:408/600: open fd = 3 2026-02-25T11:55:42.051 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.051 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.051 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.051 INFO:tasks.workunit.client.0.trial028.stdout:409/600: open fd = 3 2026-02-25T11:55:42.051 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.051 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.051 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.051 INFO:tasks.workunit.client.0.trial028.stdout:410/600: open fd = 3 2026-02-25T11:55:42.051 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.051 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.051 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.052 INFO:tasks.workunit.client.0.trial028.stdout:411/600: open fd = 3 2026-02-25T11:55:42.052 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.052 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.052 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.052 INFO:tasks.workunit.client.0.trial028.stdout:412/600: open fd = 3 2026-02-25T11:55:42.052 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.052 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.052 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.052 INFO:tasks.workunit.client.0.trial028.stdout:413/600: open fd = 3 2026-02-25T11:55:42.052 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.052 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.053 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.053 INFO:tasks.workunit.client.0.trial028.stdout:414/600: open fd = 3 2026-02-25T11:55:42.053 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.053 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.053 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.053 INFO:tasks.workunit.client.0.trial028.stdout:415/600: open fd = 3 2026-02-25T11:55:42.053 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.053 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.053 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.053 INFO:tasks.workunit.client.0.trial028.stdout:416/600: open fd = 3 2026-02-25T11:55:42.054 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.054 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.054 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.054 INFO:tasks.workunit.client.0.trial028.stdout:417/600: open fd = 3 2026-02-25T11:55:42.054 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.054 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.054 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.054 INFO:tasks.workunit.client.0.trial028.stdout:418/600: open fd = 3 2026-02-25T11:55:42.054 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.054 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.054 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.054 INFO:tasks.workunit.client.0.trial028.stdout:419/600: open fd = 3 2026-02-25T11:55:42.055 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.055 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.055 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.055 INFO:tasks.workunit.client.0.trial028.stdout:420/600: open fd = 3 2026-02-25T11:55:42.055 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.055 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.055 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.055 INFO:tasks.workunit.client.0.trial028.stdout:421/600: open fd = 3 2026-02-25T11:55:42.055 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.055 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.055 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.056 INFO:tasks.workunit.client.0.trial028.stdout:422/600: open fd = 3 2026-02-25T11:55:42.056 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.056 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.056 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.056 INFO:tasks.workunit.client.0.trial028.stdout:423/600: open fd = 3 2026-02-25T11:55:42.056 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.056 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.057 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.057 INFO:tasks.workunit.client.0.trial028.stdout:424/600: open fd = 3 2026-02-25T11:55:42.057 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.057 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.057 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.057 INFO:tasks.workunit.client.0.trial028.stdout:425/600: open fd = 3 2026-02-25T11:55:42.057 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.057 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.057 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.058 INFO:tasks.workunit.client.0.trial028.stdout:426/600: open fd = 3 2026-02-25T11:55:42.058 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.058 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.058 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.058 INFO:tasks.workunit.client.0.trial028.stdout:427/600: open fd = 3 2026-02-25T11:55:42.058 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.058 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.058 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.058 INFO:tasks.workunit.client.0.trial028.stdout:428/600: open fd = 3 2026-02-25T11:55:42.058 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.058 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.058 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.059 INFO:tasks.workunit.client.0.trial028.stdout:429/600: open fd = 3 2026-02-25T11:55:42.059 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.059 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.059 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.059 INFO:tasks.workunit.client.0.trial028.stdout:430/600: open fd = 3 2026-02-25T11:55:42.059 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.059 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.059 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.059 INFO:tasks.workunit.client.0.trial028.stdout:431/600: open fd = 3 2026-02-25T11:55:42.059 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.059 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.059 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.059 INFO:tasks.workunit.client.0.trial028.stdout:432/600: open fd = 3 2026-02-25T11:55:42.060 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.060 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.060 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.060 INFO:tasks.workunit.client.0.trial028.stdout:433/600: open fd = 3 2026-02-25T11:55:42.060 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.060 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.060 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.060 INFO:tasks.workunit.client.0.trial028.stdout:434/600: open fd = 3 2026-02-25T11:55:42.060 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.060 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.060 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.060 INFO:tasks.workunit.client.0.trial028.stdout:435/600: open fd = 3 2026-02-25T11:55:42.060 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.061 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.061 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.061 INFO:tasks.workunit.client.0.trial028.stdout:436/600: open fd = 3 2026-02-25T11:55:42.061 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.061 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:55:42.061 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:55:42.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:42 trial028 ceph-mon[39224]: pgmap v450: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:55:42.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:42 trial167 ceph-mon[43392]: pgmap v450: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:55:42.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:42 trial119 ceph-mon[42710]: pgmap v450: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:55:44.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:44 trial028 ceph-mon[39224]: pgmap v451: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:55:44.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:44 trial167 ceph-mon[43392]: pgmap v451: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:55:44.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:44 trial119 ceph-mon[42710]: pgmap v451: 97 pgs: 97 active+clean; 5.8 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:55:46.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:46 trial028 ceph-mon[39224]: pgmap v452: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:46.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:46 trial167 ceph-mon[43392]: pgmap v452: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:46.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:46 trial119 ceph-mon[42710]: pgmap v452: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:48.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:48 trial028 ceph-mon[39224]: pgmap v453: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 11 op/s 2026-02-25T11:55:48.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:48 trial167 ceph-mon[43392]: pgmap v453: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 11 op/s 2026-02-25T11:55:48.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:48 trial119 ceph-mon[42710]: pgmap v453: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 11 op/s 2026-02-25T11:55:50.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:50 trial028 ceph-mon[39224]: pgmap v454: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:55:50.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:50 trial167 ceph-mon[43392]: pgmap v454: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:55:50.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:50 trial119 ceph-mon[42710]: pgmap v454: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:55:52.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:52 trial028 ceph-mon[39224]: pgmap v455: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:55:52.561 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:52 trial119 ceph-mon[42710]: pgmap v455: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:55:52.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:52 trial167 ceph-mon[43392]: pgmap v455: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:55:54.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:54 trial028 ceph-mon[39224]: pgmap v456: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:55:54.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:54 trial167 ceph-mon[43392]: pgmap v456: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:55:54.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:54 trial119 ceph-mon[42710]: pgmap v456: 97 pgs: 97 active+clean; 5.9 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:55:56.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:56 trial028 ceph-mon[39224]: pgmap v457: 97 pgs: 97 active+clean; 6.0 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:56.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:56 trial167 ceph-mon[43392]: pgmap v457: 97 pgs: 97 active+clean; 6.0 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:56.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:56 trial119 ceph-mon[42710]: pgmap v457: 97 pgs: 97 active+clean; 6.0 MiB data, 346 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:55:58.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:55:58 trial028 ceph-mon[39224]: pgmap v458: 97 pgs: 97 active+clean; 6.0 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:55:58.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:55:58 trial167 ceph-mon[43392]: pgmap v458: 97 pgs: 97 active+clean; 6.0 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:55:58.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:55:58 trial119 ceph-mon[42710]: pgmap v458: 97 pgs: 97 active+clean; 6.0 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:00.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:00 trial167 ceph-mon[43392]: pgmap v459: 97 pgs: 97 active+clean; 6.0 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:56:00.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:00 trial119 ceph-mon[42710]: pgmap v459: 97 pgs: 97 active+clean; 6.0 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:56:00.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:00 trial028 ceph-mon[39224]: pgmap v459: 97 pgs: 97 active+clean; 6.0 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:56:01.887 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:56:02.076 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:02.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.243+0000 7fdec7398640 1 -- 10.20.193.28:0/3719902242 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdec010f690 msgr2=0x7fdec010fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.243+0000 7fdec7398640 1 --2- 10.20.193.28:0/3719902242 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdec010f690 0x7fdec010fa60 secure :-1 s=READY pgs=775 gs=1 cs=0 l=1 c_cookie=11512329618301984538 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdeb0009950 tx=0x7fdeb002f190 comp rx=0 tx=0).stop 2026-02-25T11:56:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.243+0000 7fdec7398640 1 -- 10.20.193.28:0/3719902242 shutdown_connections 2026-02-25T11:56:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.243+0000 7fdec7398640 1 --2- 10.20.193.28:0/3719902242 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdec0115880 0x7fdec0176250 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.243+0000 7fdec7398640 1 --2- 10.20.193.28:0/3719902242 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdec0110080 0x7fdec01150d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.243+0000 7fdec7398640 1 --2- 10.20.193.28:0/3719902242 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdec010f690 0x7fdec010fa60 unknown :-1 s=CLOSED pgs=775 gs=1 cs=0 l=1 c_cookie=11512329618301984538 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.243+0000 7fdec7398640 1 -- 10.20.193.28:0/3719902242 >> 10.20.193.28:0/3719902242 conn(0x7fdec00ecf00 msgr2=0x7fdec010d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.243+0000 7fdec7398640 1 -- 10.20.193.28:0/3719902242 shutdown_connections 2026-02-25T11:56:02.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.243+0000 7fdec7398640 1 -- 10.20.193.28:0/3719902242 wait complete. 2026-02-25T11:56:02.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.244+0000 7fdec7398640 1 Processor -- start 2026-02-25T11:56:02.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.244+0000 7fdec7398640 1 -- start start 2026-02-25T11:56:02.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.244+0000 7fdec7398640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdec010f690 0x7fdec011e090 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.244+0000 7fdec7398640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdec0110080 0x7fdec011e5d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.244+0000 7fdec7398640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdec0115880 0x7fdec01239a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.244+0000 7fdec7398640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdec0123f30 con 0x7fdec010f690 2026-02-25T11:56:02.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.244+0000 7fdec7398640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdec01240a0 con 0x7fdec0115880 2026-02-25T11:56:02.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.244+0000 7fdec7398640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdec0124260 con 0x7fdec0110080 2026-02-25T11:56:02.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdec6396640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdec010f690 0x7fdec011e090 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdec6396640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdec010f690 0x7fdec011e090 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38540/0 (socket says 10.20.193.28:38540) 2026-02-25T11:56:02.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdec6396640 1 -- 10.20.193.28:0/3937670043 learned_addr learned my addr 10.20.193.28:0/3937670043 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:02.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdec5b95640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdec0110080 0x7fdec011e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdec6b97640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdec0115880 0x7fdec01239a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdec6396640 1 -- 10.20.193.28:0/3937670043 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdec0110080 msgr2=0x7fdec011e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdec6396640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdec0110080 0x7fdec011e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdec6396640 1 -- 10.20.193.28:0/3937670043 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdec0115880 msgr2=0x7fdec01239a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdec6396640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdec0115880 0x7fdec01239a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdec6396640 1 -- 10.20.193.28:0/3937670043 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb0009570 con 0x7fdec010f690 2026-02-25T11:56:02.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdec6396640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdec010f690 0x7fdec011e090 secure :-1 s=READY pgs=776 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdeb0004820 tx=0x7fdeb00028b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:02.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdea77fe640 1 -- 10.20.193.28:0/3937670043 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdeb003d070 con 0x7fdec010f690 2026-02-25T11:56:02.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdea77fe640 1 -- 10.20.193.28:0/3937670043 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdeb002fe20 con 0x7fdec010f690 2026-02-25T11:56:02.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdec7398640 1 -- 10.20.193.28:0/3937670043 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec01244e0 con 0x7fdec010f690 2026-02-25T11:56:02.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.245+0000 7fdec7398640 1 -- 10.20.193.28:0/3937670043 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec0164b30 con 0x7fdec010f690 2026-02-25T11:56:02.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.246+0000 7fdea77fe640 1 -- 10.20.193.28:0/3937670043 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdeb0041960 con 0x7fdec010f690 2026-02-25T11:56:02.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.248+0000 7fdea77fe640 1 -- 10.20.193.28:0/3937670043 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fdeb0041b50 con 0x7fdec010f690 2026-02-25T11:56:02.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.250+0000 7fdea77fe640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fde9c0b5760 0x7fde9c0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.250+0000 7fdea77fe640 1 -- 10.20.193.28:0/3937670043 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fdeb010afe0 con 0x7fdec010f690 2026-02-25T11:56:02.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.250+0000 7fdec5b95640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fde9c0b5760 0x7fde9c0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.251+0000 7fdec5b95640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fde9c0b5760 0x7fde9c0d5c10 secure :-1 s=READY pgs=358 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdec0124a30 tx=0x7fdeb4009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:02.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.251+0000 7fdeca61ce00 1 Processor -- start 2026-02-25T11:56:02.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.251+0000 7fdeca61ce00 1 -- start start 2026-02-25T11:56:02.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.251+0000 7fdeca61ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694dfb50 0x55f7694dff20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.251+0000 7fdeca61ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694e0460 0x55f769524bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.251+0000 7fdeca61ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7695454e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.251+0000 7fdeca61ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f7693ed9b0 con 0x55f769525100 2026-02-25T11:56:02.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.251+0000 7fdeca61ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f7693e96e0 con 0x55f7694e0460 2026-02-25T11:56:02.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.251+0000 7fdeca61ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f7693e3840 con 0x55f7694dfb50 2026-02-25T11:56:02.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.252+0000 7fdec6396640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694dfb50 0x55f7694dff20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.256 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.252+0000 7fdec6396640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694dfb50 0x55f7694dff20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:54784/0 (socket says 10.20.193.28:54784) 2026-02-25T11:56:02.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.252+0000 7fdec6396640 1 -- 10.20.193.28:0/1872666820 learned_addr learned my addr 10.20.193.28:0/1872666820 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:02.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.252+0000 7fdec6b97640 1 --2- 10.20.193.28:0/1872666820 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7695454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.252+0000 7fdec5b95640 1 --2- 10.20.193.28:0/1872666820 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694e0460 0x55f769524bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.252+0000 7fdec6396640 1 -- 10.20.193.28:0/1872666820 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694e0460 msgr2=0x55f769524bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:02.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.252+0000 7fdec6396640 1 --2- 10.20.193.28:0/1872666820 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694e0460 0x55f769524bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.252+0000 7fdec6396640 1 -- 10.20.193.28:0/1872666820 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 msgr2=0x55f7695454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:02.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.252+0000 7fdec6396640 1 --2- 10.20.193.28:0/1872666820 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7695454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.252+0000 7fdec6396640 1 -- 10.20.193.28:0/1872666820 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f76938ed80 con 0x55f7694dfb50 2026-02-25T11:56:02.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.252+0000 7fdec6396640 1 --2- 10.20.193.28:0/1872666820 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694dfb50 0x55f7694dff20 secure :-1 s=READY pgs=203 gs=2 cs=0 l=1 c_cookie=17450054147809088568 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdeb0108bc0 tx=0x7fdeb0005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f22b0bcb077e1438 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:02.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.253+0000 7fdea4b0a640 1 -- 10.20.193.28:0/1872666820 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdeb00d0ba0 con 0x55f7694dfb50 2026-02-25T11:56:02.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.253+0000 7fdea4b0a640 1 -- 10.20.193.28:0/1872666820 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdeb00d1da0 con 0x55f7694dfb50 2026-02-25T11:56:02.257 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.253+0000 7fdea4b0a640 1 -- 10.20.193.28:0/1872666820 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdeb00d22b0 con 0x55f7694dfb50 2026-02-25T11:56:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.253+0000 7fdeca61ce00 1 -- 10.20.193.28:0/1872666820 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694dfb50 msgr2=0x55f7694dff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.253+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/1872666820 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694dfb50 0x55f7694dff20 secure :-1 s=READY pgs=203 gs=2 cs=0 l=1 c_cookie=17450054147809088568 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdeb0108bc0 tx=0x7fdeb0005ab0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.253+0000 7fdeca61ce00 1 -- 10.20.193.28:0/1872666820 shutdown_connections 2026-02-25T11:56:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.253+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/1872666820 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7695454e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.253+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/1872666820 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694e0460 0x55f769524bc0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.253+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/1872666820 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694dfb50 0x55f7694dff20 unknown :-1 s=CLOSED pgs=203 gs=2 cs=0 l=1 c_cookie=17450054147809088568 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.253+0000 7fdeca61ce00 1 -- 10.20.193.28:0/1872666820 >> 10.20.193.28:0/1872666820 conn(0x55f7693f6d40 msgr2=0x55f7694d5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.253+0000 7fdeca61ce00 1 -- 10.20.193.28:0/1872666820 shutdown_connections 2026-02-25T11:56:02.258 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.253+0000 7fdeca61ce00 1 -- 10.20.193.28:0/1872666820 wait complete. 2026-02-25T11:56:02.261 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.255+0000 7fdeca61ce00 1 -- start start 2026-02-25T11:56:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.255+0000 7fdeca61ce00 1 Processor -- start 2026-02-25T11:56:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.255+0000 7fdeca61ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694dfb50 0x55f7694c5e60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.255+0000 7fdeca61ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f7694e0460 0x55f7694c63a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.255+0000 7fdeca61ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f769525100 0x55f7694cb390 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.255+0000 7fdeca61ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f7694cb8d0 con 0x55f7694e0460 2026-02-25T11:56:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.255+0000 7fdeca61ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f7694cba40 con 0x55f7694dfb50 2026-02-25T11:56:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.255+0000 7fdeca61ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f7694cbc00 con 0x55f769525100 2026-02-25T11:56:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.255+0000 7fdec5b95640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f7694e0460 0x55f7694c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.255+0000 7fdec6396640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694dfb50 0x55f7694c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.255+0000 7fdec6b97640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f769525100 0x55f7694cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.255+0000 7fdec6396640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694dfb50 0x55f7694c5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:44978/0 (socket says 10.20.193.28:44978) 2026-02-25T11:56:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.255+0000 7fdec6396640 1 -- 10.20.193.28:0/765398320 learned_addr learned my addr 10.20.193.28:0/765398320 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.256+0000 7fdec6396640 1 -- 10.20.193.28:0/765398320 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f769525100 msgr2=0x55f7694cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.256+0000 7fdec6396640 1 --2- 10.20.193.28:0/765398320 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f769525100 0x55f7694cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.256+0000 7fdec6396640 1 -- 10.20.193.28:0/765398320 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f7694e0460 msgr2=0x55f7694c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.256+0000 7fdec6396640 1 --2- 10.20.193.28:0/765398320 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f7694e0460 0x55f7694c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.256+0000 7fdec6396640 1 -- 10.20.193.28:0/765398320 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f769367350 con 0x55f7694dfb50 2026-02-25T11:56:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.256+0000 7fdec6396640 1 --2- 10.20.193.28:0/765398320 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694dfb50 0x55f7694c5e60 secure :-1 s=READY pgs=165 gs=3 cs=0 l=1 c_cookie=11788781588652008755 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdeb00cd920 tx=0x7fdeb0041410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a39a2a652e44a533 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.256+0000 7fde73fff640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:56:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.256+0000 7fde73fff640 1 -- 10.20.193.28:0/765398320 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdeb00d70c0 con 0x55f7694dfb50 2026-02-25T11:56:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.256+0000 7fde73fff640 1 -- 10.20.193.28:0/765398320 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdeb00d96a0 con 0x55f7694dfb50 2026-02-25T11:56:02.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.256+0000 7fdeca61ce00 1 -- 10.20.193.28:0/765398320 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec0164b30 con 0x55f7694dfb50 2026-02-25T11:56:02.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.256+0000 7fde73fff640 1 -- 10.20.193.28:0/765398320 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdeb00cf600 con 0x55f7694dfb50 2026-02-25T11:56:02.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.256+0000 7fdeca61ce00 10 client.26801 _ll_register_callbacks cb 0x55f7693649a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:02.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.257+0000 7fde73fff640 1 -- 10.20.193.28:0/765398320 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fdeb00d2d90 con 0x55f7694dfb50 2026-02-25T11:56:02.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.257+0000 7fdeca61ce00 1 -- 10.20.193.28:0/765398320 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f7690796f0 con 0x55f7694dfb50 2026-02-25T11:56:02.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.257+0000 7fde73fff640 1 -- 10.20.193.28:0/765398320 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdeb0035320 con 0x55f7694dfb50 2026-02-25T11:56:02.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.257+0000 7fdeca61ce00 10 client.26801 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:02.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.257+0000 7fdeca61ce00 1 -- 10.20.193.28:0/765398320 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55f769379150 con 0x55f7694dfb50 2026-02-25T11:56:02.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.257+0000 7fde73fff640 1 -- 10.20.193.28:0/765398320 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fdeb00d2030 con 0x55f7694dfb50 2026-02-25T11:56:02.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.257+0000 7fdeca61ce00 10 client.26801 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:02.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.257+0000 7fdeca61ce00 10 client.26801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:02.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.257+0000 7fdeca61ce00 20 client.26801 populate_metadata read hostname 'trial028' 2026-02-25T11:56:02.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.257+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/765398320 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f769524800 0x55f7694cc500 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.257+0000 7fdeca61ce00 4 client.26801 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:56:02.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.257+0000 7fdeca61ce00 1 -- 10.20.193.28:0/765398320 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7693b5910 con 0x55f769524800 2026-02-25T11:56:02.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.257+0000 7fdec5b95640 1 --2- 10.20.193.28:0/765398320 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f769524800 0x55f7694cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.258+0000 7fdec5b95640 1 --2- 10.20.193.28:0/765398320 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f769524800 0x55f7694cc500 crc :-1 s=READY pgs=144 gs=4 cs=0 l=0 c_cookie=11746782653852559812 s_cookie=4233972770230880633 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a304f496289a29c4 server_cookie=3ac217cd4e49c979 in_seq=0 out_seq=0 2026-02-25T11:56:02.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.258+0000 7fde73fff640 10 client.26801 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:02.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.258+0000 7fde73fff640 1 -- 10.20.193.28:0/765398320 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55f7693b5910 con 0x55f769524800 2026-02-25T11:56:02.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.258+0000 7fde73fff640 10 client.26801 handle_command_reply: tid=0 2026-02-25T11:56:02.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.258+0000 7fdeca61ce00 1 client.26801 shutdown 2026-02-25T11:56:02.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.259+0000 7fdeca61ce00 1 -- 10.20.193.28:0/765398320 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694dfb50 msgr2=0x55f7694c5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.259+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/765398320 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694dfb50 0x55f7694c5e60 secure :-1 s=READY pgs=165 gs=3 cs=0 l=1 c_cookie=11788781588652008755 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdeb00cd920 tx=0x7fdeb0041410 comp rx=0 tx=0).stop 2026-02-25T11:56:02.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.259+0000 7fdeca61ce00 1 -- 10.20.193.28:0/765398320 shutdown_connections 2026-02-25T11:56:02.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.259+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/765398320 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f769524800 0x55f7694cc500 crc :-1 s=READY pgs=144 gs=4 cs=0 l=0 c_cookie=11746782653852559812 s_cookie=4233972770230880633 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.259+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/765398320 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f769525100 0x55f7694cb390 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.259+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/765398320 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f7694e0460 0x55f7694c63a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.259+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/765398320 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694dfb50 0x55f7694c5e60 unknown :-1 s=CLOSED pgs=165 gs=3 cs=0 l=1 c_cookie=11788781588652008755 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.259+0000 7fde73fff640 0 client.26801 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:02.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.259+0000 7fdeca61ce00 1 -- 10.20.193.28:0/765398320 >> 10.20.193.28:0/765398320 conn(0x55f7693f6d40 msgr2=0x55f7694d83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:02.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.259+0000 7fdeca61ce00 1 -- 10.20.193.28:0/765398320 shutdown_connections 2026-02-25T11:56:02.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.259+0000 7fdeca61ce00 1 -- 10.20.193.28:0/765398320 wait complete. 2026-02-25T11:56:02.266 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.259+0000 7fdeca61ce00 20 client.26801 trim_cache size 0 max 16384 2026-02-25T11:56:02.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.268+0000 7fdeca61ce00 1 Processor -- start 2026-02-25T11:56:02.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.268+0000 7fdeca61ce00 1 -- start start 2026-02-25T11:56:02.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.268+0000 7fdeca61ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694dfb50 0x55f7694b7970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.268+0000 7fdeca61ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694e0460 0x55f7694c3030 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.268+0000 7fdeca61ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7694c3570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.268+0000 7fdeca61ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f7694b8140 con 0x55f769525100 2026-02-25T11:56:02.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.268+0000 7fdeca61ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f7693e1470 con 0x55f7694e0460 2026-02-25T11:56:02.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.268+0000 7fdeca61ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f7690a1700 con 0x55f7694dfb50 2026-02-25T11:56:02.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.268+0000 7fdec6b97640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7694c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fdec6b97640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7694c3570 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38556/0 (socket says 10.20.193.28:38556) 2026-02-25T11:56:02.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fdec6b97640 1 -- 10.20.193.28:0/1840516572 learned_addr learned my addr 10.20.193.28:0/1840516572 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:02.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fdec6396640 1 --2- 10.20.193.28:0/1840516572 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694dfb50 0x55f7694b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fdec5b95640 1 --2- 10.20.193.28:0/1840516572 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694e0460 0x55f7694c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fdec6b97640 1 -- 10.20.193.28:0/1840516572 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694dfb50 msgr2=0x55f7694b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:02.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fdec6b97640 1 --2- 10.20.193.28:0/1840516572 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694dfb50 0x55f7694b7970 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fdec6b97640 1 -- 10.20.193.28:0/1840516572 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694e0460 msgr2=0x55f7694c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:02.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fdec6b97640 1 --2- 10.20.193.28:0/1840516572 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694e0460 0x55f7694c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fdec6b97640 1 -- 10.20.193.28:0/1840516572 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f769373bd0 con 0x55f769525100 2026-02-25T11:56:02.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fdec5b95640 1 --2- 10.20.193.28:0/1840516572 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694e0460 0x55f7694c3030 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:56:02.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fdec6b97640 1 --2- 10.20.193.28:0/1840516572 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7694c3570 secure :-1 s=READY pgs=777 gs=2 cs=0 l=1 c_cookie=2712089789912246331 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdebc00df00 tx=0x7fdebc005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a3474b6eaee83b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:02.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fde8e4e6640 1 -- 10.20.193.28:0/1840516572 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdebc017070 con 0x55f769525100 2026-02-25T11:56:02.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fde8e4e6640 1 -- 10.20.193.28:0/1840516572 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdebc005680 con 0x55f769525100 2026-02-25T11:56:02.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fde8e4e6640 1 -- 10.20.193.28:0/1840516572 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdebc01cd40 con 0x55f769525100 2026-02-25T11:56:02.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fdeca61ce00 1 -- 10.20.193.28:0/1840516572 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 msgr2=0x55f7694c3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.269+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/1840516572 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7694c3570 secure :-1 s=READY pgs=777 gs=2 cs=0 l=1 c_cookie=2712089789912246331 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdebc00df00 tx=0x7fdebc005420 comp rx=0 tx=0).stop 2026-02-25T11:56:02.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.270+0000 7fdeca61ce00 1 -- 10.20.193.28:0/1840516572 shutdown_connections 2026-02-25T11:56:02.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.270+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/1840516572 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7694c3570 unknown :-1 s=CLOSED pgs=777 gs=2 cs=0 l=1 c_cookie=2712089789912246331 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.270+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/1840516572 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694e0460 0x55f7694c3030 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.270+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/1840516572 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694dfb50 0x55f7694b7970 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.270+0000 7fdeca61ce00 1 -- 10.20.193.28:0/1840516572 >> 10.20.193.28:0/1840516572 conn(0x55f7693f6d40 msgr2=0x55f7694ca9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:02.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.270+0000 7fdeca61ce00 1 -- 10.20.193.28:0/1840516572 shutdown_connections 2026-02-25T11:56:02.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.270+0000 7fdeca61ce00 1 -- 10.20.193.28:0/1840516572 wait complete. 2026-02-25T11:56:02.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.270+0000 7fdeca61ce00 1 -- start start 2026-02-25T11:56:02.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.270+0000 7fdeca61ce00 1 Processor -- start 2026-02-25T11:56:02.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdeca61ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694dfb50 0x55f769596750 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdeca61ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694e0460 0x55f769593ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdeca61ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7695941e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdeca61ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f7694db6a0 con 0x55f769525100 2026-02-25T11:56:02.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdeca61ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f7694e0940 con 0x55f7694dfb50 2026-02-25T11:56:02.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdeca61ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f769594750 con 0x55f7694e0460 2026-02-25T11:56:02.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdec6b97640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7695941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdec6396640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694dfb50 0x55f769596750 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdec6396640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694dfb50 0x55f769596750 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:44990/0 (socket says 10.20.193.28:44990) 2026-02-25T11:56:02.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdec6396640 1 -- 10.20.193.28:0/171153432 learned_addr learned my addr 10.20.193.28:0/171153432 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:02.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdec5b95640 1 --2- 10.20.193.28:0/171153432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694e0460 0x55f769593ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdec6b97640 1 -- 10.20.193.28:0/171153432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694e0460 msgr2=0x55f769593ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:02.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdec6b97640 1 --2- 10.20.193.28:0/171153432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694e0460 0x55f769593ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdec6b97640 1 -- 10.20.193.28:0/171153432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694dfb50 msgr2=0x55f769596750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:02.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdec6b97640 1 --2- 10.20.193.28:0/171153432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694dfb50 0x55f769596750 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.271+0000 7fdec6b97640 1 -- 10.20.193.28:0/171153432 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f769367680 con 0x55f769525100 2026-02-25T11:56:02.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.272+0000 7fdec6b97640 1 --2- 10.20.193.28:0/171153432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7695941e0 secure :-1 s=READY pgs=778 gs=2 cs=0 l=1 c_cookie=9534002026676646192 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdebc003b50 tx=0x7fdebc01c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844f945477ea7130 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:02.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.272+0000 7fde73fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:02.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.272+0000 7fde73fff640 1 -- 10.20.193.28:0/171153432 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdebc017030 con 0x55f769525100 2026-02-25T11:56:02.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.272+0000 7fde73fff640 1 -- 10.20.193.28:0/171153432 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdebc01c6f0 con 0x55f769525100 2026-02-25T11:56:02.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.272+0000 7fde73fff640 1 -- 10.20.193.28:0/171153432 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdebc020040 con 0x55f769525100 2026-02-25T11:56:02.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.272+0000 7fdeca61ce00 1 -- 10.20.193.28:0/171153432 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdebc00db70 con 0x55f769525100 2026-02-25T11:56:02.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.272+0000 7fde73fff640 1 -- 10.20.193.28:0/171153432 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fdebc0243e0 con 0x55f769525100 2026-02-25T11:56:02.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.272+0000 7fde73fff640 10 client.17100 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:56:02.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.272+0000 7fdeca61ce00 10 client.17100 _ll_register_callbacks cb 0x55f7693649a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:02.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.272+0000 7fdeca61ce00 1 -- 10.20.193.28:0/171153432 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f76913b4a0 con 0x55f769525100 2026-02-25T11:56:02.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.273+0000 7fde73fff640 1 -- 10.20.193.28:0/171153432 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdebc011a70 con 0x55f769525100 2026-02-25T11:56:02.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.273+0000 7fdeca61ce00 10 client.17100 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:02.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.273+0000 7fdeca61ce00 1 -- 10.20.193.28:0/171153432 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55f7693793b0 con 0x55f769525100 2026-02-25T11:56:02.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.273+0000 7fde73fff640 1 -- 10.20.193.28:0/171153432 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fdebc011bd0 con 0x55f769525100 2026-02-25T11:56:02.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.273+0000 7fdeca61ce00 10 client.17100 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:02.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.273+0000 7fdeca61ce00 10 client.17100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:02.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.273+0000 7fdeca61ce00 20 client.17100 populate_metadata read hostname 'trial028' 2026-02-25T11:56:02.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.273+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/171153432 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f7694d0ae0 0x55f7694d0f80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.273+0000 7fdeca61ce00 4 client.17100 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:56:02.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.273+0000 7fdeca61ce00 1 -- 10.20.193.28:0/171153432 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fdeb00d1da0 con 0x55f7694d0ae0 2026-02-25T11:56:02.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.273+0000 7fdec6396640 1 --2- 10.20.193.28:0/171153432 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f7694d0ae0 0x55f7694d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.274+0000 7fdec6396640 1 --2- 10.20.193.28:0/171153432 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f7694d0ae0 0x55f7694d0f80 crc :-1 s=READY pgs=145 gs=4 cs=0 l=0 c_cookie=13175008193716590009 s_cookie=10919063846822940293 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d70825e35695b9 server_cookie=97884eb78db10e85 in_seq=0 out_seq=0 2026-02-25T11:56:02.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.274+0000 7fde73fff640 10 client.17100 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:02.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.274+0000 7fde73fff640 1 -- 10.20.193.28:0/171153432 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x7fdeb00d1da0 con 0x55f7694d0ae0 2026-02-25T11:56:02.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.274+0000 7fde73fff640 10 client.17100 handle_command_reply: tid=0 2026-02-25T11:56:02.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.274+0000 7fdeca61ce00 1 client.17100 shutdown 2026-02-25T11:56:02.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.274+0000 7fdeca61ce00 1 -- 10.20.193.28:0/171153432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 msgr2=0x55f7695941e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.274+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/171153432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7695941e0 secure :-1 s=READY pgs=778 gs=2 cs=0 l=1 c_cookie=9534002026676646192 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdebc003b50 tx=0x7fdebc01c650 comp rx=0 tx=0).stop 2026-02-25T11:56:02.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.274+0000 7fdeca61ce00 1 -- 10.20.193.28:0/171153432 shutdown_connections 2026-02-25T11:56:02.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.274+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/171153432 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f7694d0ae0 0x55f7694d0f80 crc :-1 s=READY pgs=145 gs=4 cs=0 l=0 c_cookie=13175008193716590009 s_cookie=10919063846822940293 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.274+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/171153432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f769525100 0x55f7695941e0 unknown :-1 s=CLOSED pgs=778 gs=2 cs=0 l=1 c_cookie=9534002026676646192 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.274+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/171153432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f7694e0460 0x55f769593ca0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.274+0000 7fdeca61ce00 1 --2- 10.20.193.28:0/171153432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f7694dfb50 0x55f769596750 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.274+0000 7fdeca61ce00 1 -- 10.20.193.28:0/171153432 >> 10.20.193.28:0/171153432 conn(0x55f7693f6d40 msgr2=0x55f7694c2020 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:02.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.275+0000 7fde73fff640 0 client.17100 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:02.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.275+0000 7fdeca61ce00 1 -- 10.20.193.28:0/171153432 shutdown_connections 2026-02-25T11:56:02.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.275+0000 7fdeca61ce00 1 -- 10.20.193.28:0/171153432 wait complete. 2026-02-25T11:56:02.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.275+0000 7fdeca61ce00 20 client.17100 trim_cache size 0 max 16384 2026-02-25T11:56:02.282 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:56:02.282 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:56:02.283 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:56:02.283 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:56:02.283 INFO:teuthology.orchestra.run.trial028.stdout: "850bcfd3-be8a-4930-896c-1ed6e48b8fed": { 2026-02-25T11:56:02.283 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:27.124813+0000", 2026-02-25T11:56:02.283 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.283 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.283 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.283 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.283 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.283 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.283 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:56:02.283 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:56:02.284 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.284 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.284 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.284 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.284 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.284 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.284 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:56:02.284 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:56:02.284 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.284 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.284 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.284 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.285 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.285 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.285 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:56:02.285 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:56:02.285 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.285 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.285 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.285 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.285 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.285 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.285 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:56:02.285 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:56:02.285 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.286 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.286 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.286 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.286 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.286 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.286 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:56:02.286 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:56:02.286 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.286 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.286 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.286 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.286 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.287 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.287 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:56:02.287 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:56:02.287 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.287 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.287 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.287 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.287 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.287 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.287 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:56:02.287 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:56:02.287 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.288 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.288 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.288 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.288 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.288 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.288 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:56:02.288 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:56:02.288 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.288 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.288 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.288 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.288 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.288 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.289 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.290 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.290 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.290 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:56:02.290 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:56:02.290 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.290 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.290 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.290 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.290 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.290 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.290 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:56:02.290 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:56:02.291 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.291 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.291 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.291 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.291 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.291 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.291 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:56:02.291 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:56:02.291 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.291 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.291 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.291 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.291 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.292 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.292 INFO:teuthology.orchestra.run.trial028.stdout: "7b237934-6b95-43dd-b3a6-82b74134ec55": { 2026-02-25T11:56:02.292 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:31.289725+0000", 2026-02-25T11:56:02.292 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.292 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.292 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.292 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.292 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.292 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:02.292 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:56:02.292 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:56:02.292 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:02.293 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:02.293 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:02.293 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:02.293 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:02.293 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:56:02.293 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:56:02.293 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:56:02.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.275+0000 7fdec7398640 1 -- 10.20.193.28:0/3937670043 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fde9c0b5760 msgr2=0x7fde9c0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.275+0000 7fdec7398640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fde9c0b5760 0x7fde9c0d5c10 secure :-1 s=READY pgs=358 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdec0124a30 tx=0x7fdeb4009190 comp rx=0 tx=0).stop 2026-02-25T11:56:02.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.275+0000 7fdec7398640 1 -- 10.20.193.28:0/3937670043 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdec010f690 msgr2=0x7fdec011e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.275+0000 7fdec7398640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdec010f690 0x7fdec011e090 secure :-1 s=READY pgs=776 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdeb0004820 tx=0x7fdeb00028b0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.277+0000 7fdec7398640 1 -- 10.20.193.28:0/3937670043 shutdown_connections 2026-02-25T11:56:02.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.277+0000 7fdec7398640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fde9c0b5760 0x7fde9c0d5c10 unknown :-1 s=CLOSED pgs=358 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.277+0000 7fdec7398640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdec0115880 0x7fdec01239a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.277+0000 7fdec7398640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdec0110080 0x7fdec011e5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.277+0000 7fdec7398640 1 --2- 10.20.193.28:0/3937670043 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdec010f690 0x7fdec011e090 unknown :-1 s=CLOSED pgs=776 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.277+0000 7fdec7398640 1 -- 10.20.193.28:0/3937670043 >> 10.20.193.28:0/3937670043 conn(0x7fdec00ecf00 msgr2=0x7fdec010d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:02.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.277+0000 7fdec7398640 1 -- 10.20.193.28:0/3937670043 shutdown_connections 2026-02-25T11:56:02.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.277+0000 7fdec7398640 1 -- 10.20.193.28:0/3937670043 wait complete. 2026-02-25T11:56:02.377 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:56:02.377 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:56:02.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:02 trial028 ceph-mon[39224]: pgmap v460: 97 pgs: 97 active+clean; 6.0 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:02.560 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:02.732 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.730+0000 7f776f348640 1 -- 10.20.193.28:0/499085764 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7768153900 msgr2=0x7f7768173d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.730+0000 7f776f348640 1 --2- 10.20.193.28:0/499085764 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7768153900 0x7f7768173d20 secure :-1 s=READY pgs=779 gs=2 cs=0 l=1 c_cookie=3887383450114094057 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f776400aa30 tx=0x7f7764030270 comp rx=0 tx=0).stop 2026-02-25T11:56:02.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.730+0000 7f776f348640 1 -- 10.20.193.28:0/499085764 shutdown_connections 2026-02-25T11:56:02.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.730+0000 7f776f348640 1 --2- 10.20.193.28:0/499085764 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f7768153900 0x7f7768173d20 unknown :-1 s=CLOSED pgs=779 gs=2 cs=0 l=1 c_cookie=3887383450114094057 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.730+0000 7f776f348640 1 --2- 10.20.193.28:0/499085764 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f776810ed40 0x7f77681533c0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.730+0000 7f776f348640 1 --2- 10.20.193.28:0/499085764 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f776810e350 0x7f776810e720 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.730+0000 7f776f348640 1 -- 10.20.193.28:0/499085764 >> 10.20.193.28:0/499085764 conn(0x7f77680e9d40 msgr2=0x7f776810a150 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:02.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.730+0000 7f776f348640 1 -- 10.20.193.28:0/499085764 shutdown_connections 2026-02-25T11:56:02.733 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.730+0000 7f776f348640 1 -- 10.20.193.28:0/499085764 wait complete. 2026-02-25T11:56:02.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.731+0000 7f776f348640 1 Processor -- start 2026-02-25T11:56:02.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.731+0000 7f776f348640 1 -- start start 2026-02-25T11:56:02.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.731+0000 7f776f348640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f776810e350 0x7f776810c1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.731+0000 7f776f348640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f776810ed40 0x7f776810c720 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.735 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.731+0000 7f776f348640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7768153900 0x7f776810cc60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.731+0000 7f776f348640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f776814f440 con 0x7f776810e350 2026-02-25T11:56:02.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.731+0000 7f776f348640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f776814f5b0 con 0x7f7768153900 2026-02-25T11:56:02.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.731+0000 7f776f348640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f776814f770 con 0x7f776810ed40 2026-02-25T11:56:02.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776c8bc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f776810ed40 0x7f776810c720 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776d0bd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f776810e350 0x7f776810c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776c8bc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f776810ed40 0x7f776810c720 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:54832/0 (socket says 10.20.193.28:54832) 2026-02-25T11:56:02.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776c8bc640 1 -- 10.20.193.28:0/2545314923 learned_addr learned my addr 10.20.193.28:0/2545314923 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:02.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776d8be640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7768153900 0x7f776810cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776c8bc640 1 -- 10.20.193.28:0/2545314923 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7768153900 msgr2=0x7f776810cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776c8bc640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7768153900 0x7f776810cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.736 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776c8bc640 1 -- 10.20.193.28:0/2545314923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f776810e350 msgr2=0x7f776810c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776c8bc640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f776810e350 0x7f776810c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776c8bc640 1 -- 10.20.193.28:0/2545314923 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776400a650 con 0x7f776810ed40 2026-02-25T11:56:02.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776d8be640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7768153900 0x7f776810cc60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:56:02.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776c8bc640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f776810ed40 0x7f776810c720 secure :-1 s=READY pgs=204 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f775c009480 tx=0x7f775c00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:02.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f77567fc640 1 -- 10.20.193.28:0/2545314923 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f775c00cd40 con 0x7f776810ed40 2026-02-25T11:56:02.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776f348640 1 -- 10.20.193.28:0/2545314923 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776814fa50 con 0x7f776810ed40 2026-02-25T11:56:02.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f77567fc640 1 -- 10.20.193.28:0/2545314923 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f775c004d10 con 0x7f776810ed40 2026-02-25T11:56:02.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f77567fc640 1 -- 10.20.193.28:0/2545314923 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f775c0106b0 con 0x7f776810ed40 2026-02-25T11:56:02.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.732+0000 7f776f348640 1 -- 10.20.193.28:0/2545314923 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776814ff20 con 0x7f776810ed40 2026-02-25T11:56:02.737 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.733+0000 7f776f348640 1 -- 10.20.193.28:0/2545314923 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7730005300 con 0x7f776810ed40 2026-02-25T11:56:02.738 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.735+0000 7f77567fc640 1 -- 10.20.193.28:0/2545314923 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f775c01f060 con 0x7f776810ed40 2026-02-25T11:56:02.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.737+0000 7f77567fc640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f77440b5760 0x7f77440d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:02.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.737+0000 7f77567fc640 1 -- 10.20.193.28:0/2545314923 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f775c014070 con 0x7f776810ed40 2026-02-25T11:56:02.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.737+0000 7f77567fc640 1 -- 10.20.193.28:0/2545314923 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f775c11f940 con 0x7f776810ed40 2026-02-25T11:56:02.739 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.737+0000 7f776d0bd640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f77440b5760 0x7f77440d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:02.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.738+0000 7f776d0bd640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f77440b5760 0x7f77440d5c10 secure :-1 s=READY pgs=359 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7758004740 tx=0x7f77580073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:02.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:02 trial167 ceph-mon[43392]: pgmap v460: 97 pgs: 97 active+clean; 6.0 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:02.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:02 trial119 ceph-mon[42710]: pgmap v460: 97 pgs: 97 active+clean; 6.0 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:02.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.884+0000 7f776f348640 1 -- 10.20.193.28:0/2545314923 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7730005950 con 0x7f776810ed40 2026-02-25T11:56:02.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.885+0000 7f77567fc640 1 -- 10.20.193.28:0/2545314923 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f775c0a95e0 con 0x7f776810ed40 2026-02-25T11:56:02.888 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:56:02.889 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:56:02.889 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:56:02.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.886+0000 7f776f348640 1 -- 10.20.193.28:0/2545314923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f77440b5760 msgr2=0x7f77440d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.886+0000 7f776f348640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f77440b5760 0x7f77440d5c10 secure :-1 s=READY pgs=359 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f7758004740 tx=0x7f77580073d0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.886+0000 7f776f348640 1 -- 10.20.193.28:0/2545314923 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f776810ed40 msgr2=0x7f776810c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:02.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.886+0000 7f776f348640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f776810ed40 0x7f776810c720 secure :-1 s=READY pgs=204 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f775c009480 tx=0x7f775c00ee60 comp rx=0 tx=0).stop 2026-02-25T11:56:02.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.886+0000 7f776f348640 1 -- 10.20.193.28:0/2545314923 shutdown_connections 2026-02-25T11:56:02.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.886+0000 7f776f348640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f77440b5760 0x7f77440d5c10 unknown :-1 s=CLOSED pgs=359 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.886+0000 7f776f348640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f7768153900 0x7f776810cc60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.886+0000 7f776f348640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f776810ed40 0x7f776810c720 unknown :-1 s=CLOSED pgs=204 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.886+0000 7f776f348640 1 --2- 10.20.193.28:0/2545314923 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f776810e350 0x7f776810c1e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:02.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.886+0000 7f776f348640 1 -- 10.20.193.28:0/2545314923 >> 10.20.193.28:0/2545314923 conn(0x7f77680e9d40 msgr2=0x7f776810a120 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:02.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.886+0000 7f776f348640 1 -- 10.20.193.28:0/2545314923 shutdown_connections 2026-02-25T11:56:02.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:02.886+0000 7f776f348640 1 -- 10.20.193.28:0/2545314923 wait complete. 2026-02-25T11:56:02.990 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:56:03.174 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:03.343 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.341+0000 7f6565369640 1 -- 10.20.193.28:0/1825922957 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f656010c250 msgr2=0x7f6560112e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.341+0000 7f6565369640 1 --2- 10.20.193.28:0/1825922957 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f656010c250 0x7f6560112e00 secure :-1 s=READY pgs=780 gs=3 cs=0 l=1 c_cookie=2543105493978354298 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6550009950 tx=0x7f655002f190 comp rx=0 tx=0).stop 2026-02-25T11:56:03.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.341+0000 7f6565369640 1 -- 10.20.193.28:0/1825922957 shutdown_connections 2026-02-25T11:56:03.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.341+0000 7f6565369640 1 --2- 10.20.193.28:0/1825922957 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f65601135b0 0x7f6560173f80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.341+0000 7f6565369640 1 --2- 10.20.193.28:0/1825922957 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f656010c250 0x7f6560112e00 unknown :-1 s=CLOSED pgs=780 gs=3 cs=0 l=1 c_cookie=2543105493978354298 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.341+0000 7f6565369640 1 --2- 10.20.193.28:0/1825922957 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f656010b860 0x7f656010bc30 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.341+0000 7f6565369640 1 -- 10.20.193.28:0/1825922957 >> 10.20.193.28:0/1825922957 conn(0x7f65600e9290 msgr2=0x7f65601096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:03.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.341+0000 7f6565369640 1 -- 10.20.193.28:0/1825922957 shutdown_connections 2026-02-25T11:56:03.344 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.341+0000 7f6565369640 1 -- 10.20.193.28:0/1825922957 wait complete. 2026-02-25T11:56:03.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6565369640 1 Processor -- start 2026-02-25T11:56:03.345 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6565369640 1 -- start start 2026-02-25T11:56:03.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6565369640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f656010b860 0x7f65600f3ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6565369640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f656010c250 0x7f6560173e80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6565369640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f65601135b0 0x7f65600f46f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6565369640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f65600f9900 con 0x7f65601135b0 2026-02-25T11:56:03.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6565369640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f65600f9a70 con 0x7f656010c250 2026-02-25T11:56:03.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6565369640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f65600f9c30 con 0x7f656010b860 2026-02-25T11:56:03.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6564b68640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f65601135b0 0x7f65600f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6564b68640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f65601135b0 0x7f65600f46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38620/0 (socket says 10.20.193.28:38620) 2026-02-25T11:56:03.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6564b68640 1 -- 10.20.193.28:0/4210091949 learned_addr learned my addr 10.20.193.28:0/4210091949 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:03.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f655f7fe640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f656010c250 0x7f6560173e80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f655ffff640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f656010b860 0x7f65600f3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6564b68640 1 -- 10.20.193.28:0/4210091949 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f656010b860 msgr2=0x7f65600f3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6564b68640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f656010b860 0x7f65600f3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6564b68640 1 -- 10.20.193.28:0/4210091949 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f656010c250 msgr2=0x7f6560173e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6564b68640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f656010c250 0x7f6560173e80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.342+0000 7f6564b68640 1 -- 10.20.193.28:0/4210091949 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6550009570 con 0x7f65601135b0 2026-02-25T11:56:03.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.343+0000 7f655ffff640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f656010b860 0x7f65600f3ff0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:56:03.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.343+0000 7f6564b68640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f65601135b0 0x7f65600f46f0 secure :-1 s=READY pgs=781 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6554006e90 tx=0x7f655400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:03.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.343+0000 7f655d7fa640 1 -- 10.20.193.28:0/4210091949 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6554005500 con 0x7f65601135b0 2026-02-25T11:56:03.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.343+0000 7f6565369640 1 -- 10.20.193.28:0/4210091949 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65600f9f10 con 0x7f65601135b0 2026-02-25T11:56:03.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.343+0000 7f6565369640 1 -- 10.20.193.28:0/4210091949 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65600fa430 con 0x7f65601135b0 2026-02-25T11:56:03.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.343+0000 7f655d7fa640 1 -- 10.20.193.28:0/4210091949 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6554005df0 con 0x7f65601135b0 2026-02-25T11:56:03.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.343+0000 7f655d7fa640 1 -- 10.20.193.28:0/4210091949 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6554003a00 con 0x7f65601135b0 2026-02-25T11:56:03.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.346+0000 7f655d7fa640 1 -- 10.20.193.28:0/4210091949 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f6554003b60 con 0x7f65601135b0 2026-02-25T11:56:03.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.347+0000 7f655d7fa640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65340b5750 0x7f65340d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.347+0000 7f655d7fa640 1 -- 10.20.193.28:0/4210091949 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f65540e79d0 con 0x7f65601135b0 2026-02-25T11:56:03.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.347+0000 7f655ffff640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65340b5750 0x7f65340d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.348+0000 7f655ffff640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65340b5750 0x7f65340d5c00 secure :-1 s=READY pgs=360 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f654c005e30 tx=0x7f654c005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:03.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.348+0000 7f65685ede00 1 Processor -- start 2026-02-25T11:56:03.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.348+0000 7f65685ede00 1 -- start start 2026-02-25T11:56:03.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f65685ede00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be2b30 0x55acc5be2f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f65685ede00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be3440 0x55acc5c27ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f65685ede00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5c280e0 0x55acc5c484c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f65685ede00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55acc5af88a0 con 0x55acc5c280e0 2026-02-25T11:56:03.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f65685ede00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55acc5af45d0 con 0x55acc5be2b30 2026-02-25T11:56:03.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f65685ede00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55acc5aee570 con 0x55acc5be3440 2026-02-25T11:56:03.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f6564b68640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5c280e0 0x55acc5c484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f6564b68640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5c280e0 0x55acc5c484c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38628/0 (socket says 10.20.193.28:38628) 2026-02-25T11:56:03.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f6564b68640 1 -- 10.20.193.28:0/70416198 learned_addr learned my addr 10.20.193.28:0/70416198 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:03.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f655f7fe640 1 --2- 10.20.193.28:0/70416198 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be3440 0x55acc5c27ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f655ffff640 1 --2- 10.20.193.28:0/70416198 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be2b30 0x55acc5be2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f6564b68640 1 -- 10.20.193.28:0/70416198 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be3440 msgr2=0x55acc5c27ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f6564b68640 1 --2- 10.20.193.28:0/70416198 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be3440 0x55acc5c27ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f6564b68640 1 -- 10.20.193.28:0/70416198 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be2b30 msgr2=0x55acc5be2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f6564b68640 1 --2- 10.20.193.28:0/70416198 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be2b30 0x55acc5be2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f6564b68640 1 -- 10.20.193.28:0/70416198 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc5a99d10 con 0x55acc5c280e0 2026-02-25T11:56:03.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f6564b68640 1 --2- 10.20.193.28:0/70416198 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5c280e0 0x55acc5c484c0 secure :-1 s=READY pgs=782 gs=1 cs=0 l=1 c_cookie=4120446642698890972 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f655400c880 tx=0x7f6554005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392ec463f604aedc server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:03.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f6530ce3640 1 -- 10.20.193.28:0/70416198 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f65540ae5b0 con 0x55acc5c280e0 2026-02-25T11:56:03.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f6530ce3640 1 -- 10.20.193.28:0/70416198 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f65540ae3d0 con 0x55acc5c280e0 2026-02-25T11:56:03.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.349+0000 7f6530ce3640 1 -- 10.20.193.28:0/70416198 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f65540e50c0 con 0x55acc5c280e0 2026-02-25T11:56:03.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.350+0000 7f65685ede00 1 -- 10.20.193.28:0/70416198 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5c280e0 msgr2=0x55acc5c484c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.354 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.350+0000 7f65685ede00 1 --2- 10.20.193.28:0/70416198 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5c280e0 0x55acc5c484c0 secure :-1 s=READY pgs=782 gs=1 cs=0 l=1 c_cookie=4120446642698890972 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f655400c880 tx=0x7f6554005000 comp rx=0 tx=0).stop 2026-02-25T11:56:03.355 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.350+0000 7f65685ede00 1 -- 10.20.193.28:0/70416198 shutdown_connections 2026-02-25T11:56:03.355 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.350+0000 7f65685ede00 1 --2- 10.20.193.28:0/70416198 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5c280e0 0x55acc5c484c0 unknown :-1 s=CLOSED pgs=782 gs=1 cs=0 l=1 c_cookie=4120446642698890972 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.355 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.350+0000 7f65685ede00 1 --2- 10.20.193.28:0/70416198 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be3440 0x55acc5c27ba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.355 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.350+0000 7f65685ede00 1 --2- 10.20.193.28:0/70416198 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be2b30 0x55acc5be2f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.355 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.350+0000 7f65685ede00 1 -- 10.20.193.28:0/70416198 >> 10.20.193.28:0/70416198 conn(0x55acc5b017a0 msgr2=0x55acc5be1410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:03.355 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.350+0000 7f65685ede00 1 -- 10.20.193.28:0/70416198 shutdown_connections 2026-02-25T11:56:03.355 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.350+0000 7f65685ede00 1 -- 10.20.193.28:0/70416198 wait complete. 2026-02-25T11:56:03.359 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.351+0000 7f65685ede00 1 -- start start 2026-02-25T11:56:03.359 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.351+0000 7f65685ede00 1 Processor -- start 2026-02-25T11:56:03.359 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f65685ede00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5be2b30 0x55acc5bfae80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.359 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f65685ede00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be3440 0x55acc5bfb3c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.359 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f65685ede00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5c280e0 0x55acc5c00490 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.359 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f65685ede00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55acc5c009d0 con 0x55acc5be2b30 2026-02-25T11:56:03.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f65685ede00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55acc5c00b40 con 0x55acc5c280e0 2026-02-25T11:56:03.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f65685ede00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55acc5c00d00 con 0x55acc5be3440 2026-02-25T11:56:03.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f655ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5be2b30 0x55acc5bfae80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f655ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5be2b30 0x55acc5bfae80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38630/0 (socket says 10.20.193.28:38630) 2026-02-25T11:56:03.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f655ffff640 1 -- 10.20.193.28:0/3148916725 learned_addr learned my addr 10.20.193.28:0/3148916725 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:03.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f655f7fe640 1 --2- 10.20.193.28:0/3148916725 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be3440 0x55acc5bfb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f6564b68640 1 --2- 10.20.193.28:0/3148916725 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5c280e0 0x55acc5c00490 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f655ffff640 1 -- 10.20.193.28:0/3148916725 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be3440 msgr2=0x55acc5bfb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f655ffff640 1 --2- 10.20.193.28:0/3148916725 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be3440 0x55acc5bfb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.360 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f655ffff640 1 -- 10.20.193.28:0/3148916725 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5c280e0 msgr2=0x55acc5c00490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f655ffff640 1 --2- 10.20.193.28:0/3148916725 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5c280e0 0x55acc5c00490 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f655ffff640 1 -- 10.20.193.28:0/3148916725 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc5a723c0 con 0x55acc5be2b30 2026-02-25T11:56:03.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f655ffff640 1 --2- 10.20.193.28:0/3148916725 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5be2b30 0x55acc5bfae80 secure :-1 s=READY pgs=783 gs=1 cs=0 l=1 c_cookie=12011754335890192725 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f654c002fd0 tx=0x7f654c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b252ed6ffe3955 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:03.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f6521ffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:03.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f6521ffb640 1 -- 10.20.193.28:0/3148916725 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f654c011e10 con 0x55acc5be2b30 2026-02-25T11:56:03.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f6521ffb640 1 -- 10.20.193.28:0/3148916725 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f654c01d8d0 con 0x55acc5be2b30 2026-02-25T11:56:03.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f6521ffb640 1 -- 10.20.193.28:0/3148916725 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f654c01a980 con 0x55acc5be2b30 2026-02-25T11:56:03.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.352+0000 7f65685ede00 1 -- 10.20.193.28:0/3148916725 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65600fa430 con 0x55acc5be2b30 2026-02-25T11:56:03.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f6521ffb640 1 -- 10.20.193.28:0/3148916725 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f654c01ab10 con 0x55acc5be2b30 2026-02-25T11:56:03.361 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f6521ffb640 10 client.17118 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:56:03.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f65685ede00 10 client.17118 _ll_register_callbacks cb 0x55acc5a6fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:03.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f65685ede00 1 -- 10.20.193.28:0/3148916725 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55acc57846f0 con 0x55acc5be2b30 2026-02-25T11:56:03.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f6521ffb640 1 -- 10.20.193.28:0/3148916725 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f654c01acd0 con 0x55acc5be2b30 2026-02-25T11:56:03.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f65685ede00 10 client.17118 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:03.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f65685ede00 1 -- 10.20.193.28:0/3148916725 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55acc5a840e0 con 0x55acc5be2b30 2026-02-25T11:56:03.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f6521ffb640 1 -- 10.20.193.28:0/3148916725 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f654c01d410 con 0x55acc5be2b30 2026-02-25T11:56:03.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f65685ede00 10 client.17118 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:03.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f65685ede00 10 client.17118 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:56:03.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f65685ede00 20 client.17118 populate_metadata read hostname 'trial028' 2026-02-25T11:56:03.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f65685ede00 1 --2- 10.20.193.28:0/3148916725 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55acc5c25860 0x55acc5c25d00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.362 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f65685ede00 4 client.17118 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:56:03.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.353+0000 7f65685ede00 1 -- 10.20.193.28:0/3148916725 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acc5ac0fc0 con 0x55acc5c25860 2026-02-25T11:56:03.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.354+0000 7f655f7fe640 1 --2- 10.20.193.28:0/3148916725 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55acc5c25860 0x55acc5c25d00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.354+0000 7f655f7fe640 1 --2- 10.20.193.28:0/3148916725 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55acc5c25860 0x55acc5c25d00 crc :-1 s=READY pgs=53 gs=4 cs=0 l=0 c_cookie=17741383260410748024 s_cookie=11151561733535973135 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6360e13d4d81c78 server_cookie=9ac24e502a403f0f in_seq=0 out_seq=0 2026-02-25T11:56:03.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.354+0000 7f6521ffb640 10 client.17118 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:56:03.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f6521ffb640 1 -- 10.20.193.28:0/3148916725 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55acc5ac0fc0 con 0x55acc5c25860 2026-02-25T11:56:03.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f6521ffb640 10 client.17118 handle_command_reply: tid=0 2026-02-25T11:56:03.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f65685ede00 1 client.17118 shutdown 2026-02-25T11:56:03.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f65685ede00 1 -- 10.20.193.28:0/3148916725 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5be2b30 msgr2=0x55acc5bfae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f65685ede00 1 --2- 10.20.193.28:0/3148916725 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5be2b30 0x55acc5bfae80 secure :-1 s=READY pgs=783 gs=1 cs=0 l=1 c_cookie=12011754335890192725 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f654c002fd0 tx=0x7f654c004770 comp rx=0 tx=0).stop 2026-02-25T11:56:03.363 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f65685ede00 1 -- 10.20.193.28:0/3148916725 shutdown_connections 2026-02-25T11:56:03.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f65685ede00 1 --2- 10.20.193.28:0/3148916725 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55acc5c25860 0x55acc5c25d00 crc :-1 s=READY pgs=53 gs=4 cs=0 l=0 c_cookie=17741383260410748024 s_cookie=11151561733535973135 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f65685ede00 1 --2- 10.20.193.28:0/3148916725 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5c280e0 0x55acc5c00490 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f65685ede00 1 --2- 10.20.193.28:0/3148916725 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be3440 0x55acc5bfb3c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f65685ede00 1 --2- 10.20.193.28:0/3148916725 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5be2b30 0x55acc5bfae80 unknown :-1 s=CLOSED pgs=783 gs=1 cs=0 l=1 c_cookie=12011754335890192725 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f65685ede00 1 -- 10.20.193.28:0/3148916725 >> 10.20.193.28:0/3148916725 conn(0x55acc5b017a0 msgr2=0x55acc5be1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:03.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f6521ffb640 0 client.17118 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:56:03.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f65685ede00 1 -- 10.20.193.28:0/3148916725 shutdown_connections 2026-02-25T11:56:03.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.355+0000 7f65685ede00 1 -- 10.20.193.28:0/3148916725 wait complete. 2026-02-25T11:56:03.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.356+0000 7f65685ede00 20 client.17118 trim_cache size 0 max 16384 2026-02-25T11:56:03.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.363+0000 7f65685ede00 1 Processor -- start 2026-02-25T11:56:03.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.363+0000 7f65685ede00 1 -- start start 2026-02-25T11:56:03.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.363+0000 7f65685ede00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be2b30 0x55acc5be1d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.363+0000 7f65685ede00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5be3440 0x55acc5be2260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.363+0000 7f65685ede00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5c280e0 0x55acc5c273d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.363+0000 7f65685ede00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55acc5aec1a0 con 0x55acc5be3440 2026-02-25T11:56:03.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.363+0000 7f65685ede00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55acc57ac700 con 0x55acc5be2b30 2026-02-25T11:56:03.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.363+0000 7f65685ede00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55acc5bfb9a0 con 0x55acc5c280e0 2026-02-25T11:56:03.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f655ffff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be2b30 0x55acc5be1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f655ffff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be2b30 0x55acc5be1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:45068/0 (socket says 10.20.193.28:45068) 2026-02-25T11:56:03.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f655ffff640 1 -- 10.20.193.28:0/2105563330 learned_addr learned my addr 10.20.193.28:0/2105563330 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:03.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f655f7fe640 1 --2- 10.20.193.28:0/2105563330 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5be3440 0x55acc5be2260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f6564b68640 1 --2- 10.20.193.28:0/2105563330 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5c280e0 0x55acc5c273d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f655ffff640 1 -- 10.20.193.28:0/2105563330 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5c280e0 msgr2=0x55acc5c273d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f655ffff640 1 --2- 10.20.193.28:0/2105563330 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5c280e0 0x55acc5c273d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f655ffff640 1 -- 10.20.193.28:0/2105563330 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5be3440 msgr2=0x55acc5be2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f655ffff640 1 --2- 10.20.193.28:0/2105563330 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5be3440 0x55acc5be2260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f655ffff640 1 -- 10.20.193.28:0/2105563330 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc5a7eb60 con 0x55acc5be2b30 2026-02-25T11:56:03.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f655ffff640 1 --2- 10.20.193.28:0/2105563330 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be2b30 0x55acc5be1d20 secure :-1 s=READY pgs=166 gs=1 cs=0 l=1 c_cookie=12540942621626112903 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f654c014040 tx=0x7f654c01f030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae0a60de9e949b87 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:03.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f6523fff640 1 -- 10.20.193.28:0/2105563330 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f654c03d070 con 0x55acc5be2b30 2026-02-25T11:56:03.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f6523fff640 1 -- 10.20.193.28:0/2105563330 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55acc57846f0 con 0x55acc5be2b30 2026-02-25T11:56:03.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.364+0000 7f6523fff640 1 -- 10.20.193.28:0/2105563330 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f654c011e90 con 0x55acc5be2b30 2026-02-25T11:56:03.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.365+0000 7f65685ede00 1 -- 10.20.193.28:0/2105563330 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be2b30 msgr2=0x55acc5be1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.365+0000 7f65685ede00 1 --2- 10.20.193.28:0/2105563330 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be2b30 0x55acc5be1d20 secure :-1 s=READY pgs=166 gs=1 cs=0 l=1 c_cookie=12540942621626112903 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f654c014040 tx=0x7f654c01f030 comp rx=0 tx=0).stop 2026-02-25T11:56:03.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.365+0000 7f65685ede00 1 -- 10.20.193.28:0/2105563330 shutdown_connections 2026-02-25T11:56:03.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.365+0000 7f65685ede00 1 --2- 10.20.193.28:0/2105563330 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5c280e0 0x55acc5c273d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.365+0000 7f65685ede00 1 --2- 10.20.193.28:0/2105563330 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5be3440 0x55acc5be2260 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.365+0000 7f65685ede00 1 --2- 10.20.193.28:0/2105563330 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be2b30 0x55acc5be1d20 unknown :-1 s=CLOSED pgs=166 gs=1 cs=0 l=1 c_cookie=12540942621626112903 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.365+0000 7f65685ede00 1 -- 10.20.193.28:0/2105563330 >> 10.20.193.28:0/2105563330 conn(0x55acc5b017a0 msgr2=0x55acc5be59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:03.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.365+0000 7f65685ede00 1 -- 10.20.193.28:0/2105563330 shutdown_connections 2026-02-25T11:56:03.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.365+0000 7f65685ede00 1 -- 10.20.193.28:0/2105563330 wait complete. 2026-02-25T11:56:03.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.366+0000 7f65685ede00 1 -- start start 2026-02-25T11:56:03.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.366+0000 7f65685ede00 1 Processor -- start 2026-02-25T11:56:03.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.366+0000 7f65685ede00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be2b30 0x55acc5c063c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.366+0000 7f65685ede00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be3440 0x55acc5c06900 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.366+0000 7f65685ede00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5c280e0 0x55acc5c06e40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.366+0000 7f65685ede00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55acc5bfb510 con 0x55acc5c280e0 2026-02-25T11:56:03.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.366+0000 7f65685ede00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55acc5c05430 con 0x55acc5be3440 2026-02-25T11:56:03.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.366+0000 7f65685ede00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55acc5c055f0 con 0x55acc5be2b30 2026-02-25T11:56:03.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.366+0000 7f655ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be2b30 0x55acc5c063c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.366+0000 7f655ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be2b30 0x55acc5c063c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:54892/0 (socket says 10.20.193.28:54892) 2026-02-25T11:56:03.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.366+0000 7f655ffff640 1 -- 10.20.193.28:0/1837143126 learned_addr learned my addr 10.20.193.28:0/1837143126 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:03.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.366+0000 7f655f7fe640 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be3440 0x55acc5c06900 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.366+0000 7f6564b68640 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5c280e0 0x55acc5c06e40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f655ffff640 1 -- 10.20.193.28:0/1837143126 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be3440 msgr2=0x55acc5c06900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f655ffff640 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be3440 0x55acc5c06900 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f655ffff640 1 -- 10.20.193.28:0/1837143126 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5c280e0 msgr2=0x55acc5c06e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f655ffff640 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5c280e0 0x55acc5c06e40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f655ffff640 1 -- 10.20.193.28:0/1837143126 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acc5a726f0 con 0x55acc5be2b30 2026-02-25T11:56:03.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f655f7fe640 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be3440 0x55acc5c06900 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:56:03.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f655ffff640 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be2b30 0x55acc5c063c0 secure :-1 s=READY pgs=205 gs=2 cs=0 l=1 c_cookie=9225387543340201513 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f654c0023d0 tx=0x7f654c01c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80072917ac1b3629 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:03.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f6521ffb640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:56:03.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f6521ffb640 1 -- 10.20.193.28:0/1837143126 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f654c03d070 con 0x55acc5be2b30 2026-02-25T11:56:03.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f6521ffb640 1 -- 10.20.193.28:0/1837143126 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f654c01dd40 con 0x55acc5be2b30 2026-02-25T11:56:03.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f6521ffb640 1 -- 10.20.193.28:0/1837143126 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f654c0053d0 con 0x55acc5be2b30 2026-02-25T11:56:03.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f65685ede00 1 -- 10.20.193.28:0/1837143126 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acc5a840e0 con 0x55acc5be2b30 2026-02-25T11:56:03.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f6521ffb640 1 -- 10.20.193.28:0/1837143126 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f654c005560 con 0x55acc5be2b30 2026-02-25T11:56:03.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f65685ede00 10 client.26818 _ll_register_callbacks cb 0x55acc5a6fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:03.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.367+0000 7f65685ede00 1 -- 10.20.193.28:0/1837143126 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65540ae3d0 con 0x55acc5be2b30 2026-02-25T11:56:03.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.368+0000 7f6521ffb640 1 -- 10.20.193.28:0/1837143126 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f654c011bf0 con 0x55acc5be2b30 2026-02-25T11:56:03.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.368+0000 7f65685ede00 10 client.26818 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:03.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.368+0000 7f65685ede00 1 -- 10.20.193.28:0/1837143126 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55acc5a84340 con 0x55acc5be2b30 2026-02-25T11:56:03.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.368+0000 7f6521ffb640 1 -- 10.20.193.28:0/1837143126 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f654c01d880 con 0x55acc5be2b30 2026-02-25T11:56:03.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.368+0000 7f65685ede00 10 client.26818 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:03.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.368+0000 7f65685ede00 10 client.26818 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:56:03.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.368+0000 7f65685ede00 20 client.26818 populate_metadata read hostname 'trial028' 2026-02-25T11:56:03.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.368+0000 7f65685ede00 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55acc5cd5680 0x55acc5cf5b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.368+0000 7f65685ede00 4 client.26818 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:56:03.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.368+0000 7f65685ede00 1 -- 10.20.193.28:0/1837143126 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55acc5781f40 con 0x55acc5cd5680 2026-02-25T11:56:03.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.368+0000 7f655f7fe640 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55acc5cd5680 0x55acc5cf5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f655f7fe640 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55acc5cd5680 0x55acc5cf5b30 crc :-1 s=READY pgs=54 gs=4 cs=0 l=0 c_cookie=17779856564258771628 s_cookie=3337346244276447890 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6bebd57ee0a92ac server_cookie=2e50a2ba2694ea92 in_seq=0 out_seq=0 2026-02-25T11:56:03.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f6521ffb640 10 client.26818 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:56:03.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f6521ffb640 1 -- 10.20.193.28:0/1837143126 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55acc5781f40 con 0x55acc5cd5680 2026-02-25T11:56:03.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f6521ffb640 10 client.26818 handle_command_reply: tid=0 2026-02-25T11:56:03.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f65685ede00 1 client.26818 shutdown 2026-02-25T11:56:03.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f65685ede00 1 -- 10.20.193.28:0/1837143126 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be2b30 msgr2=0x55acc5c063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f65685ede00 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be2b30 0x55acc5c063c0 secure :-1 s=READY pgs=205 gs=2 cs=0 l=1 c_cookie=9225387543340201513 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f654c0023d0 tx=0x7f654c01c930 comp rx=0 tx=0).stop 2026-02-25T11:56:03.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f65685ede00 1 -- 10.20.193.28:0/1837143126 shutdown_connections 2026-02-25T11:56:03.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f65685ede00 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55acc5cd5680 0x55acc5cf5b30 crc :-1 s=READY pgs=54 gs=4 cs=0 l=0 c_cookie=17779856564258771628 s_cookie=3337346244276447890 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f65685ede00 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55acc5c280e0 0x55acc5c06e40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f65685ede00 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55acc5be3440 0x55acc5c06900 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f65685ede00 1 --2- 10.20.193.28:0/1837143126 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55acc5be2b30 0x55acc5c063c0 unknown :-1 s=CLOSED pgs=205 gs=2 cs=0 l=1 c_cookie=9225387543340201513 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f65685ede00 1 -- 10.20.193.28:0/1837143126 >> 10.20.193.28:0/1837143126 conn(0x55acc5b017a0 msgr2=0x55acc5be0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:03.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.369+0000 7f6521ffb640 0 client.26818 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:56:03.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f65685ede00 1 -- 10.20.193.28:0/1837143126 shutdown_connections 2026-02-25T11:56:03.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f65685ede00 1 -- 10.20.193.28:0/1837143126 wait complete. 2026-02-25T11:56:03.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f65685ede00 20 client.26818 trim_cache size 0 max 16384 2026-02-25T11:56:03.378 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:56:03.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f6565369640 1 -- 10.20.193.28:0/4210091949 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65340b5750 msgr2=0x7f65340d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f6565369640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65340b5750 0x7f65340d5c00 secure :-1 s=READY pgs=360 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f654c005e30 tx=0x7f654c005da0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f6565369640 1 -- 10.20.193.28:0/4210091949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f65601135b0 msgr2=0x7f65600f46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f6565369640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f65601135b0 0x7f65600f46f0 secure :-1 s=READY pgs=781 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6554006e90 tx=0x7f655400edf0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f6565369640 1 -- 10.20.193.28:0/4210091949 shutdown_connections 2026-02-25T11:56:03.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f6565369640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f65340b5750 0x7f65340d5c00 unknown :-1 s=CLOSED pgs=360 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f6565369640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f65601135b0 0x7f65600f46f0 unknown :-1 s=CLOSED pgs=781 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f6565369640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f656010c250 0x7f6560173e80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f6565369640 1 --2- 10.20.193.28:0/4210091949 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f656010b860 0x7f65600f3ff0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f6565369640 1 -- 10.20.193.28:0/4210091949 >> 10.20.193.28:0/4210091949 conn(0x7f65600e9290 msgr2=0x7f656010fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:03.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f6565369640 1 -- 10.20.193.28:0/4210091949 shutdown_connections 2026-02-25T11:56:03.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.370+0000 7f6565369640 1 -- 10.20.193.28:0/4210091949 wait complete. 2026-02-25T11:56:03.474 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:03 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2545314923' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:56:03.507 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:56:03.692 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:03.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:03 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2545314923' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:56:03.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:03 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2545314923' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:56:03.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.868+0000 7f9be1c2a640 1 -- 10.20.193.28:0/2401149936 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9bdc10c290 msgr2=0x7f9bdc10c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.868+0000 7f9be1c2a640 1 --2- 10.20.193.28:0/2401149936 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9bdc10c290 0x7f9bdc10c660 secure :-1 s=READY pgs=784 gs=2 cs=0 l=1 c_cookie=3856018220293405246 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9bcc009950 tx=0x7f9bcc02f1b0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.868+0000 7f9be1c2a640 1 -- 10.20.193.28:0/2401149936 shutdown_connections 2026-02-25T11:56:03.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.868+0000 7f9be1c2a640 1 --2- 10.20.193.28:0/2401149936 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9bdc112480 0x7f9bdc172e50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.868+0000 7f9be1c2a640 1 --2- 10.20.193.28:0/2401149936 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9bdc10cc80 0x7f9bdc111cd0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.868+0000 7f9be1c2a640 1 --2- 10.20.193.28:0/2401149936 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9bdc10c290 0x7f9bdc10c660 unknown :-1 s=CLOSED pgs=784 gs=2 cs=0 l=1 c_cookie=3856018220293405246 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.868+0000 7f9be1c2a640 1 -- 10.20.193.28:0/2401149936 >> 10.20.193.28:0/2401149936 conn(0x7f9bdc0e9b40 msgr2=0x7f9bdc109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:03.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.868+0000 7f9be1c2a640 1 -- 10.20.193.28:0/2401149936 shutdown_connections 2026-02-25T11:56:03.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.868+0000 7f9be1c2a640 1 -- 10.20.193.28:0/2401149936 wait complete. 2026-02-25T11:56:03.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.869+0000 7f9be1c2a640 1 Processor -- start 2026-02-25T11:56:03.872 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.869+0000 7f9be1c2a640 1 -- start start 2026-02-25T11:56:03.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.869+0000 7f9be1c2a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9bdc10c290 0x7f9bdc175db0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.869+0000 7f9be1c2a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9bdc10cc80 0x7f9bdc1762f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.869+0000 7f9be1c2a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9bdc112480 0x7f9bdc176830 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.869+0000 7f9be1c2a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc162b80 con 0x7f9bdc112480 2026-02-25T11:56:03.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.869+0000 7f9be1c2a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc162cf0 con 0x7f9bdc10cc80 2026-02-25T11:56:03.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.869+0000 7f9be1c2a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc162eb0 con 0x7f9bdc10c290 2026-02-25T11:56:03.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9bd3fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9bdc10cc80 0x7f9bdc1762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9be1429640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9bdc112480 0x7f9bdc176830 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9be0c28640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9bdc10c290 0x7f9bdc175db0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9bd3fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9bdc10cc80 0x7f9bdc1762f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:45104/0 (socket says 10.20.193.28:45104) 2026-02-25T11:56:03.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9be1429640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9bdc112480 0x7f9bdc176830 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38666/0 (socket says 10.20.193.28:38666) 2026-02-25T11:56:03.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9bd3fff640 1 -- 10.20.193.28:0/3737966211 learned_addr learned my addr 10.20.193.28:0/3737966211 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:03.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9be0c28640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9bdc10c290 0x7f9bdc175db0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:54910/0 (socket says 10.20.193.28:54910) 2026-02-25T11:56:03.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9be1429640 1 -- 10.20.193.28:0/3737966211 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9bdc10c290 msgr2=0x7f9bdc175db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9be1429640 1 --2- 10.20.193.28:0/3737966211 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9bdc10c290 0x7f9bdc175db0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9be1429640 1 -- 10.20.193.28:0/3737966211 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9bdc10cc80 msgr2=0x7f9bdc1762f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9be1429640 1 --2- 10.20.193.28:0/3737966211 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9bdc10cc80 0x7f9bdc1762f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9be1429640 1 -- 10.20.193.28:0/3737966211 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bcc009570 con 0x7f9bdc112480 2026-02-25T11:56:03.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9be1429640 1 --2- 10.20.193.28:0/3737966211 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9bdc112480 0x7f9bdc176830 secure :-1 s=READY pgs=785 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9bd800f940 tx=0x7f9bd800fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:03.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9bd1ffb640 1 -- 10.20.193.28:0/3737966211 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9bd800dce0 con 0x7f9bdc112480 2026-02-25T11:56:03.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9bd1ffb640 1 -- 10.20.193.28:0/3737966211 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9bd800de40 con 0x7f9bdc112480 2026-02-25T11:56:03.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9bd1ffb640 1 -- 10.20.193.28:0/3737966211 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9bd80116b0 con 0x7f9bdc112480 2026-02-25T11:56:03.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.870+0000 7f9be1c2a640 1 -- 10.20.193.28:0/3737966211 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bdc163190 con 0x7f9bdc112480 2026-02-25T11:56:03.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.871+0000 7f9be1c2a640 1 -- 10.20.193.28:0/3737966211 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bdc163610 con 0x7f9bdc112480 2026-02-25T11:56:03.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.873+0000 7f9bd1ffb640 1 -- 10.20.193.28:0/3737966211 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f9bd8020060 con 0x7f9bdc112480 2026-02-25T11:56:03.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.875+0000 7f9bd1ffb640 1 --2- 10.20.193.28:0/3737966211 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9bb80b5760 0x7f9bb80d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.875+0000 7f9bd1ffb640 1 -- 10.20.193.28:0/3737966211 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9bd8015070 con 0x7f9bdc112480 2026-02-25T11:56:03.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9be0c28640 1 --2- 10.20.193.28:0/3737966211 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9bb80b5760 0x7f9bb80d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9be4eaee00 1 Processor -- start 2026-02-25T11:56:03.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9be4eaee00 1 -- start start 2026-02-25T11:56:03.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9be0c28640 1 --2- 10.20.193.28:0/3737966211 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9bb80b5760 0x7f9bb80d5c10 secure :-1 s=READY pgs=361 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9bcc0098a0 tx=0x7f9bcc03a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:03.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9be4eaee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea4b6f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9be4eaee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b7440 0x55a5ea4fbba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9be4eaee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea51c4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9be4eaee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a5ea3cc8a0 con 0x55a5ea4b6b30 2026-02-25T11:56:03.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9be4eaee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a5ea3c85d0 con 0x55a5ea4fc0e0 2026-02-25T11:56:03.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9be4eaee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a5ea3c2570 con 0x55a5ea4b7440 2026-02-25T11:56:03.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9be0c28640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea4b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9be0c28640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea4b6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38682/0 (socket says 10.20.193.28:38682) 2026-02-25T11:56:03.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9be0c28640 1 -- 10.20.193.28:0/1441816979 learned_addr learned my addr 10.20.193.28:0/1441816979 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:03.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.876+0000 7f9bd3fff640 1 --2- 10.20.193.28:0/1441816979 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b7440 0x55a5ea4fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be1429640 1 --2- 10.20.193.28:0/1441816979 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea51c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be0c28640 1 -- 10.20.193.28:0/1441816979 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b7440 msgr2=0x55a5ea4fbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be0c28640 1 --2- 10.20.193.28:0/1441816979 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b7440 0x55a5ea4fbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be0c28640 1 -- 10.20.193.28:0/1441816979 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4fc0e0 msgr2=0x55a5ea51c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be0c28640 1 --2- 10.20.193.28:0/1441816979 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea51c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be0c28640 1 -- 10.20.193.28:0/1441816979 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ea36dd10 con 0x55a5ea4b6b30 2026-02-25T11:56:03.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be0c28640 1 --2- 10.20.193.28:0/1441816979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea4b6f00 secure :-1 s=READY pgs=786 gs=1 cs=0 l=1 c_cookie=18243786118742341953 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9bcc004050 tx=0x7f9bcc02f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2ef2d0b5d6f941 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:03.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9ba94e4640 1 -- 10.20.193.28:0/1441816979 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9bcc038a40 con 0x55a5ea4b6b30 2026-02-25T11:56:03.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9ba94e4640 1 -- 10.20.193.28:0/1441816979 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9bcc03e040 con 0x55a5ea4b6b30 2026-02-25T11:56:03.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9ba94e4640 1 -- 10.20.193.28:0/1441816979 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9bcc04f6c0 con 0x55a5ea4b6b30 2026-02-25T11:56:03.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be4eaee00 1 -- 10.20.193.28:0/1441816979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 msgr2=0x55a5ea4b6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/1441816979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea4b6f00 secure :-1 s=READY pgs=786 gs=1 cs=0 l=1 c_cookie=18243786118742341953 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9bcc004050 tx=0x7f9bcc02f1b0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be4eaee00 1 -- 10.20.193.28:0/1441816979 shutdown_connections 2026-02-25T11:56:03.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/1441816979 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea51c4c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/1441816979 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b7440 0x55a5ea4fbba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/1441816979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea4b6f00 unknown :-1 s=CLOSED pgs=786 gs=1 cs=0 l=1 c_cookie=18243786118742341953 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.877+0000 7f9be4eaee00 1 -- 10.20.193.28:0/1441816979 >> 10.20.193.28:0/1441816979 conn(0x55a5ea3d57a0 msgr2=0x55a5ea4b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:03.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.878+0000 7f9be4eaee00 1 -- 10.20.193.28:0/1441816979 shutdown_connections 2026-02-25T11:56:03.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.878+0000 7f9be4eaee00 1 -- 10.20.193.28:0/1441816979 wait complete. 2026-02-25T11:56:03.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.879+0000 7f9be4eaee00 1 -- start start 2026-02-25T11:56:03.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.879+0000 7f9be4eaee00 1 Processor -- start 2026-02-25T11:56:03.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.879+0000 7f9be4eaee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea5139d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.879+0000 7f9be4eaee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b7440 0x55a5ea513f10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.879+0000 7f9be4eaee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea518f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be4eaee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a5ea519440 con 0x55a5ea4b6b30 2026-02-25T11:56:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be4eaee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a5ea5195b0 con 0x55a5ea4fc0e0 2026-02-25T11:56:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be4eaee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a5ea519770 con 0x55a5ea4b7440 2026-02-25T11:56:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be0c28640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea5139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be0c28640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea5139d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38698/0 (socket says 10.20.193.28:38698) 2026-02-25T11:56:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be0c28640 1 -- 10.20.193.28:0/3499291590 learned_addr learned my addr 10.20.193.28:0/3499291590 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9bd3fff640 1 --2- 10.20.193.28:0/3499291590 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b7440 0x55a5ea513f10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be1429640 1 --2- 10.20.193.28:0/3499291590 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea518f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be0c28640 1 -- 10.20.193.28:0/3499291590 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b7440 msgr2=0x55a5ea513f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.886 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be0c28640 1 --2- 10.20.193.28:0/3499291590 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b7440 0x55a5ea513f10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be0c28640 1 -- 10.20.193.28:0/3499291590 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4fc0e0 msgr2=0x55a5ea518f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be0c28640 1 --2- 10.20.193.28:0/3499291590 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea518f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be0c28640 1 -- 10.20.193.28:0/3499291590 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ea3463c0 con 0x55a5ea4b6b30 2026-02-25T11:56:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be0c28640 1 --2- 10.20.193.28:0/3499291590 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea5139d0 secure :-1 s=READY pgs=787 gs=1 cs=0 l=1 c_cookie=11956833403381633263 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9bcc04f7f0 tx=0x7f9bcc0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ef34a15f1650ef server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9b9a7fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3499291590 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9bcc04fa70 con 0x55a5ea4b6b30 2026-02-25T11:56:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3499291590 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bdc1736c0 con 0x55a5ea4b6b30 2026-02-25T11:56:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be4eaee00 10 client.17136 _ll_register_callbacks cb 0x55a5ea343a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3499291590 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a5ea0586f0 con 0x55a5ea4b6b30 2026-02-25T11:56:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3499291590 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9bcc009310 con 0x55a5ea4b6b30 2026-02-25T11:56:03.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.880+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3499291590 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9bcc044790 con 0x55a5ea4b6b30 2026-02-25T11:56:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.881+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3499291590 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9bcc044920 con 0x55a5ea4b6b30 2026-02-25T11:56:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.881+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3499291590 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9bcc038820 con 0x55a5ea4b6b30 2026-02-25T11:56:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.881+0000 7f9be4eaee00 10 client.17136 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.881+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3499291590 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55a5ea3580e0 con 0x55a5ea4b6b30 2026-02-25T11:56:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.881+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3499291590 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9bcc04fbd0 con 0x55a5ea4b6b30 2026-02-25T11:56:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.881+0000 7f9be4eaee00 10 client.17136 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.881+0000 7f9be4eaee00 10 client.17136 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:56:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.881+0000 7f9be4eaee00 20 client.17136 populate_metadata read hostname 'trial028' 2026-02-25T11:56:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.881+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/3499291590 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a5ea4f8f70 0x55a5ea4f9410 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.881+0000 7f9be4eaee00 4 client.17136 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:56:03.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.881+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3499291590 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5ea394fc0 con 0x55a5ea4f8f70 2026-02-25T11:56:03.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.881+0000 7f9bd3fff640 1 --2- 10.20.193.28:0/3499291590 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a5ea4f8f70 0x55a5ea4f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.882+0000 7f9bd3fff640 1 --2- 10.20.193.28:0/3499291590 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a5ea4f8f70 0x55a5ea4f9410 crc :-1 s=READY pgs=54 gs=4 cs=0 l=0 c_cookie=15444440419721252528 s_cookie=5609301952966072450 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d655acbbcc2b8eb0 server_cookie=4dd83e6828729882 in_seq=0 out_seq=0 2026-02-25T11:56:03.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.882+0000 7f9b9a7fc640 10 client.17136 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:56:03.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.882+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3499291590 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55a5ea394fc0 con 0x55a5ea4f8f70 2026-02-25T11:56:03.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.882+0000 7f9b9a7fc640 10 client.17136 handle_command_reply: tid=0 2026-02-25T11:56:03.890 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.883+0000 7f9be4eaee00 1 client.17136 shutdown 2026-02-25T11:56:03.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.883+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3499291590 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 msgr2=0x55a5ea5139d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.883+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/3499291590 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea5139d0 secure :-1 s=READY pgs=787 gs=1 cs=0 l=1 c_cookie=11956833403381633263 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9bcc04f7f0 tx=0x7f9bcc0028d0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.883+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3499291590 shutdown_connections 2026-02-25T11:56:03.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.883+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/3499291590 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a5ea4f8f70 0x55a5ea4f9410 crc :-1 s=READY pgs=54 gs=4 cs=0 l=0 c_cookie=15444440419721252528 s_cookie=5609301952966072450 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.883+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/3499291590 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea518f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.883+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/3499291590 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b7440 0x55a5ea513f10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.883+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/3499291590 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea5139d0 unknown :-1 s=CLOSED pgs=787 gs=1 cs=0 l=1 c_cookie=11956833403381633263 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.883+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3499291590 >> 10.20.193.28:0/3499291590 conn(0x55a5ea3d57a0 msgr2=0x55a5ea4b4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:03.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.883+0000 7f9b9a7fc640 0 client.17136 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:56:03.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.883+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3499291590 shutdown_connections 2026-02-25T11:56:03.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.883+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3499291590 wait complete. 2026-02-25T11:56:03.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.883+0000 7f9be4eaee00 20 client.17136 trim_cache size 0 max 16384 2026-02-25T11:56:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.890+0000 7f9be4eaee00 1 Processor -- start 2026-02-25T11:56:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.890+0000 7f9be4eaee00 1 -- start start 2026-02-25T11:56:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.890+0000 7f9be4eaee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b6b30 0x55a5ea518470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.890+0000 7f9be4eaee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4b7440 0x55a5ea4f6bb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.890+0000 7f9be4eaee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea4f70f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.890+0000 7f9be4eaee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a5ea3c01a0 con 0x55a5ea4fc0e0 2026-02-25T11:56:03.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.890+0000 7f9be4eaee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a5ea080700 con 0x55a5ea4b7440 2026-02-25T11:56:03.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.890+0000 7f9be4eaee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a5ea519630 con 0x55a5ea4b6b30 2026-02-25T11:56:03.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.891+0000 7f9be1429640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea4f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.891+0000 7f9be1429640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea4f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38706/0 (socket says 10.20.193.28:38706) 2026-02-25T11:56:03.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.891+0000 7f9be1429640 1 -- 10.20.193.28:0/2730477219 learned_addr learned my addr 10.20.193.28:0/2730477219 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:03.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.891+0000 7f9be0c28640 1 --2- 10.20.193.28:0/2730477219 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b6b30 0x55a5ea518470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.891+0000 7f9be1429640 1 -- 10.20.193.28:0/2730477219 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b6b30 msgr2=0x55a5ea518470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.891+0000 7f9be1429640 1 --2- 10.20.193.28:0/2730477219 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b6b30 0x55a5ea518470 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.891+0000 7f9bd3fff640 1 --2- 10.20.193.28:0/2730477219 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4b7440 0x55a5ea4f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.891+0000 7f9be1429640 1 -- 10.20.193.28:0/2730477219 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4b7440 msgr2=0x55a5ea4f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.894 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.891+0000 7f9be1429640 1 --2- 10.20.193.28:0/2730477219 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4b7440 0x55a5ea4f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.891+0000 7f9be1429640 1 -- 10.20.193.28:0/2730477219 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ea5062d0 con 0x55a5ea4fc0e0 2026-02-25T11:56:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.891+0000 7f9be0c28640 1 --2- 10.20.193.28:0/2730477219 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b6b30 0x55a5ea518470 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:56:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.891+0000 7f9bd3fff640 1 --2- 10.20.193.28:0/2730477219 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4b7440 0x55a5ea4f6bb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:56:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.891+0000 7f9be1429640 1 --2- 10.20.193.28:0/2730477219 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea4f70f0 secure :-1 s=READY pgs=788 gs=2 cs=0 l=1 c_cookie=1009806811423452667 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9bd80e56b0 tx=0x7f9bd80064b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e038df1deaa2dfb server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.892+0000 7f9ba8ce3640 1 -- 10.20.193.28:0/2730477219 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9bd80b3c00 con 0x55a5ea4fc0e0 2026-02-25T11:56:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.892+0000 7f9ba8ce3640 1 -- 10.20.193.28:0/2730477219 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9bd80b61e0 con 0x55a5ea4fc0e0 2026-02-25T11:56:03.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.892+0000 7f9ba8ce3640 1 -- 10.20.193.28:0/2730477219 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9bd80add00 con 0x55a5ea4fc0e0 2026-02-25T11:56:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.892+0000 7f9be4eaee00 1 -- 10.20.193.28:0/2730477219 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4fc0e0 msgr2=0x55a5ea4f70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.892+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/2730477219 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea4f70f0 secure :-1 s=READY pgs=788 gs=2 cs=0 l=1 c_cookie=1009806811423452667 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9bd80e56b0 tx=0x7f9bd80064b0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.892+0000 7f9be4eaee00 1 -- 10.20.193.28:0/2730477219 shutdown_connections 2026-02-25T11:56:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.892+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/2730477219 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea4f70f0 unknown :-1 s=CLOSED pgs=788 gs=2 cs=0 l=1 c_cookie=1009806811423452667 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.892+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/2730477219 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4b7440 0x55a5ea4f6bb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.892+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/2730477219 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4b6b30 0x55a5ea518470 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.892+0000 7f9be4eaee00 1 -- 10.20.193.28:0/2730477219 >> 10.20.193.28:0/2730477219 conn(0x55a5ea3d57a0 msgr2=0x55a5ea517d60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.892+0000 7f9be4eaee00 1 -- 10.20.193.28:0/2730477219 shutdown_connections 2026-02-25T11:56:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.892+0000 7f9be4eaee00 1 -- 10.20.193.28:0/2730477219 wait complete. 2026-02-25T11:56:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.893+0000 7f9be4eaee00 1 -- start start 2026-02-25T11:56:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.893+0000 7f9be4eaee00 1 Processor -- start 2026-02-25T11:56:03.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.893+0000 7f9be4eaee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea496de0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.893+0000 7f9be4eaee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4b7440 0x55a5ea497320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.893+0000 7f9be4eaee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea4f68b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.893+0000 7f9be4eaee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55a5ea497900 con 0x55a5ea4b6b30 2026-02-25T11:56:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.893+0000 7f9be4eaee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55a5ea497a70 con 0x55a5ea4b7440 2026-02-25T11:56:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.893+0000 7f9be4eaee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55a5ea4f6df0 con 0x55a5ea4fc0e0 2026-02-25T11:56:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.893+0000 7f9be0c28640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea496de0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9be0c28640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea496de0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38714/0 (socket says 10.20.193.28:38714) 2026-02-25T11:56:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9be0c28640 1 -- 10.20.193.28:0/3263007019 learned_addr learned my addr 10.20.193.28:0/3263007019 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9be1429640 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea4f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9bd3fff640 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4b7440 0x55a5ea497320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9be0c28640 1 -- 10.20.193.28:0/3263007019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4fc0e0 msgr2=0x55a5ea4f68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9be0c28640 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea4f68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9be0c28640 1 -- 10.20.193.28:0/3263007019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4b7440 msgr2=0x55a5ea497320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:03.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9be0c28640 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4b7440 0x55a5ea497320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9be0c28640 1 -- 10.20.193.28:0/3263007019 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ea352b60 con 0x55a5ea4b6b30 2026-02-25T11:56:03.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9bd3fff640 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4b7440 0x55a5ea497320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:56:03.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9be1429640 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea4f68b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:56:03.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9be0c28640 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea496de0 secure :-1 s=READY pgs=789 gs=1 cs=0 l=1 c_cookie=137598662711399270 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9bcc02feb0 tx=0x7f9bcc005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8d941347b5366 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:03.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9b9a7fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:03.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3263007019 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9bcc0647d0 con 0x55a5ea4b6b30 2026-02-25T11:56:03.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3263007019 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9bcc069060 con 0x55a5ea4b6b30 2026-02-25T11:56:03.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3263007019 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9bcc04f750 con 0x55a5ea4b6b30 2026-02-25T11:56:03.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3263007019 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bdc163610 con 0x55a5ea4b6b30 2026-02-25T11:56:03.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.894+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3263007019 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9bcc04f8e0 con 0x55a5ea4b6b30 2026-02-25T11:56:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9b9a7fc640 10 client.17142 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:56:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9be4eaee00 10 client.17142 _ll_register_callbacks cb 0x55a5ea343a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3263007019 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bcc03e040 con 0x55a5ea4b6b30 2026-02-25T11:56:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3263007019 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9bcc0028d0 con 0x55a5ea4b6b30 2026-02-25T11:56:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9be4eaee00 10 client.17142 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3263007019 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55a5ea3466f0 con 0x55a5ea4b6b30 2026-02-25T11:56:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3263007019 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9bcc04faa0 con 0x55a5ea4b6b30 2026-02-25T11:56:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9be4eaee00 10 client.17142 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9be4eaee00 10 client.17142 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:56:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9be4eaee00 20 client.17142 populate_metadata read hostname 'trial028' 2026-02-25T11:56:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a5ea498760 0x55a5ea5d1ac0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:03.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9be4eaee00 4 client.17142 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3263007019 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5ea055f40 con 0x55a5ea498760 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.895+0000 7f9bd3fff640 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a5ea498760 0x55a5ea5d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9bd3fff640 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a5ea498760 0x55a5ea5d1ac0 crc :-1 s=READY pgs=55 gs=4 cs=0 l=0 c_cookie=4511460125711862423 s_cookie=11444775769562844664 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e9bed14e3650a97 server_cookie=9ed402ece6ca8df8 in_seq=0 out_seq=0 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9b9a7fc640 10 client.17142 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9b9a7fc640 1 -- 10.20.193.28:0/3263007019 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5ea055f40 con 0x55a5ea498760 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9b9a7fc640 10 client.17142 handle_command_reply: tid=0 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9be4eaee00 1 client.17142 shutdown 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3263007019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 msgr2=0x55a5ea496de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea496de0 secure :-1 s=READY pgs=789 gs=1 cs=0 l=1 c_cookie=137598662711399270 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9bcc02feb0 tx=0x7f9bcc005cc0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3263007019 shutdown_connections 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55a5ea498760 0x55a5ea5d1ac0 crc :-1 s=READY pgs=55 gs=4 cs=0 l=0 c_cookie=4511460125711862423 s_cookie=11444775769562844664 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55a5ea4fc0e0 0x55a5ea4f68b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55a5ea4b7440 0x55a5ea497320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.904 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9be4eaee00 1 --2- 10.20.193.28:0/3263007019 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55a5ea4b6b30 0x55a5ea496de0 unknown :-1 s=CLOSED pgs=789 gs=1 cs=0 l=1 c_cookie=137598662711399270 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9b9a7fc640 0 client.17142 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:56:03.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3263007019 >> 10.20.193.28:0/3263007019 conn(0x55a5ea3d57a0 msgr2=0x55a5ea518550 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:03.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3263007019 shutdown_connections 2026-02-25T11:56:03.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9be4eaee00 1 -- 10.20.193.28:0/3263007019 wait complete. 2026-02-25T11:56:03.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.896+0000 7f9be4eaee00 20 client.17142 trim_cache size 0 max 16384 2026-02-25T11:56:03.905 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:56:03.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.897+0000 7f9be1c2a640 1 -- 10.20.193.28:0/3737966211 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9bb80b5760 msgr2=0x7f9bb80d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.897+0000 7f9be1c2a640 1 --2- 10.20.193.28:0/3737966211 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9bb80b5760 0x7f9bb80d5c10 secure :-1 s=READY pgs=361 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9bcc0098a0 tx=0x7f9bcc03a040 comp rx=0 tx=0).stop 2026-02-25T11:56:03.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.897+0000 7f9be1c2a640 1 -- 10.20.193.28:0/3737966211 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9bdc112480 msgr2=0x7f9bdc176830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:03.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.897+0000 7f9be1c2a640 1 --2- 10.20.193.28:0/3737966211 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9bdc112480 0x7f9bdc176830 secure :-1 s=READY pgs=785 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9bd800f940 tx=0x7f9bd800fe10 comp rx=0 tx=0).stop 2026-02-25T11:56:03.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.897+0000 7f9be1c2a640 1 -- 10.20.193.28:0/3737966211 shutdown_connections 2026-02-25T11:56:03.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.897+0000 7f9be1c2a640 1 --2- 10.20.193.28:0/3737966211 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9bb80b5760 0x7f9bb80d5c10 unknown :-1 s=CLOSED pgs=361 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.897+0000 7f9be1c2a640 1 --2- 10.20.193.28:0/3737966211 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9bdc112480 0x7f9bdc176830 unknown :-1 s=CLOSED pgs=785 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.897+0000 7f9be1c2a640 1 --2- 10.20.193.28:0/3737966211 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9bdc10cc80 0x7f9bdc1762f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.897+0000 7f9be1c2a640 1 --2- 10.20.193.28:0/3737966211 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9bdc10c290 0x7f9bdc175db0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:03.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.897+0000 7f9be1c2a640 1 -- 10.20.193.28:0/3737966211 >> 10.20.193.28:0/3737966211 conn(0x7f9bdc0e9b40 msgr2=0x7f9bdc109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:03.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.897+0000 7f9be1c2a640 1 -- 10.20.193.28:0/3737966211 shutdown_connections 2026-02-25T11:56:03.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:03.897+0000 7f9be1c2a640 1 -- 10.20.193.28:0/3737966211 wait complete. 2026-02-25T11:56:03.996 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:56:04.182 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:04.364 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.361+0000 7ffb7ffff640 1 -- 10.20.193.28:0/96518825 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb80119d30 msgr2=0x7ffb8017a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:04.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.361+0000 7ffb7ffff640 1 --2- 10.20.193.28:0/96518825 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb80119d30 0x7ffb8017a700 secure :-1 s=READY pgs=790 gs=2 cs=0 l=1 c_cookie=17446083537583781199 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb7000aa30 tx=0x7ffb70030270 comp rx=0 tx=0).stop 2026-02-25T11:56:04.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.361+0000 7ffb7ffff640 1 -- 10.20.193.28:0/96518825 shutdown_connections 2026-02-25T11:56:04.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.361+0000 7ffb7ffff640 1 --2- 10.20.193.28:0/96518825 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb80119d30 0x7ffb8017a700 unknown :-1 s=CLOSED pgs=790 gs=2 cs=0 l=1 c_cookie=17446083537583781199 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.361+0000 7ffb7ffff640 1 --2- 10.20.193.28:0/96518825 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffb80114a90 0x7ffb80119580 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.361+0000 7ffb7ffff640 1 --2- 10.20.193.28:0/96518825 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffb801140a0 0x7ffb80114470 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.361+0000 7ffb7ffff640 1 -- 10.20.193.28:0/96518825 >> 10.20.193.28:0/96518825 conn(0x7ffb80064c40 msgr2=0x7ffb80065040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:04.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.361+0000 7ffb7ffff640 1 -- 10.20.193.28:0/96518825 shutdown_connections 2026-02-25T11:56:04.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.361+0000 7ffb7ffff640 1 -- 10.20.193.28:0/96518825 wait complete. 2026-02-25T11:56:04.365 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.362+0000 7ffb7ffff640 1 Processor -- start 2026-02-25T11:56:04.366 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.362+0000 7ffb7ffff640 1 -- start start 2026-02-25T11:56:04.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.362+0000 7ffb7ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffb801140a0 0x7ffb80122520 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.362+0000 7ffb7ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb80114a90 0x7ffb80122a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.362+0000 7ffb7ffff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffb80119d30 0x7ffb80127e30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7ffff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80128450 con 0x7ffb80114a90 2026-02-25T11:56:04.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7ffff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffb801285c0 con 0x7ffb80119d30 2026-02-25T11:56:04.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7ffff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffb80128780 con 0x7ffb801140a0 2026-02-25T11:56:04.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7e7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb80114a90 0x7ffb80122a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7effd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffb801140a0 0x7ffb80122520 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7effd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffb801140a0 0x7ffb80122520 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:54954/0 (socket says 10.20.193.28:54954) 2026-02-25T11:56:04.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7effd640 1 -- 10.20.193.28:0/705188737 learned_addr learned my addr 10.20.193.28:0/705188737 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:04.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7f7fe640 1 --2- 10.20.193.28:0/705188737 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffb80119d30 0x7ffb80127e30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7effd640 1 -- 10.20.193.28:0/705188737 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffb80119d30 msgr2=0x7ffb80127e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:04.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7effd640 1 --2- 10.20.193.28:0/705188737 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffb80119d30 0x7ffb80127e30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7effd640 1 -- 10.20.193.28:0/705188737 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb80114a90 msgr2=0x7ffb80122a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:04.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7effd640 1 --2- 10.20.193.28:0/705188737 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb80114a90 0x7ffb80122a60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7effd640 1 -- 10.20.193.28:0/705188737 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7000a650 con 0x7ffb801140a0 2026-02-25T11:56:04.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb7effd640 1 --2- 10.20.193.28:0/705188737 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffb801140a0 0x7ffb80122520 secure :-1 s=READY pgs=206 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb74009480 tx=0x7ffb7400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:04.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.363+0000 7ffb5ffff640 1 -- 10.20.193.28:0/705188737 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb7400cd40 con 0x7ffb801140a0 2026-02-25T11:56:04.368 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.364+0000 7ffb5ffff640 1 -- 10.20.193.28:0/705188737 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ffb74004d10 con 0x7ffb801140a0 2026-02-25T11:56:04.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.364+0000 7ffb5ffff640 1 -- 10.20.193.28:0/705188737 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb740106b0 con 0x7ffb801140a0 2026-02-25T11:56:04.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.364+0000 7ffb7ffff640 1 -- 10.20.193.28:0/705188737 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb80128a60 con 0x7ffb801140a0 2026-02-25T11:56:04.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.364+0000 7ffb7ffff640 1 -- 10.20.193.28:0/705188737 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb80169160 con 0x7ffb801140a0 2026-02-25T11:56:04.369 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.367+0000 7ffb5ffff640 1 -- 10.20.193.28:0/705188737 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7ffb7401f060 con 0x7ffb801140a0 2026-02-25T11:56:04.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.368+0000 7ffb5ffff640 1 --2- 10.20.193.28:0/705188737 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffb540b5760 0x7ffb540d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.368+0000 7ffb5ffff640 1 -- 10.20.193.28:0/705188737 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ffb74014070 con 0x7ffb801140a0 2026-02-25T11:56:04.371 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.369+0000 7ffb7e7fc640 1 --2- 10.20.193.28:0/705188737 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffb540b5760 0x7ffb540d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.372 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.369+0000 7ffb7e7fc640 1 --2- 10.20.193.28:0/705188737 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffb540b5760 0x7ffb540d5c10 secure :-1 s=READY pgs=362 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb80128ec0 tx=0x7ffb68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:04.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.369+0000 7ffb872a1e00 1 Processor -- start 2026-02-25T11:56:04.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.369+0000 7ffb872a1e00 1 -- start start 2026-02-25T11:56:04.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.369+0000 7ffb872a1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf4978b30 0x55abf4978f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.369+0000 7ffb872a1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf49bdba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.369+0000 7ffb872a1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf49be0e0 0x55abf49de4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb872a1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55abf488e8a0 con 0x55abf4979440 2026-02-25T11:56:04.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb872a1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55abf488a5d0 con 0x55abf4978b30 2026-02-25T11:56:04.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb872a1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55abf4884570 con 0x55abf49be0e0 2026-02-25T11:56:04.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb7e7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf49bdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb7f7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf49be0e0 0x55abf49de4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.373 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb7effd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf4978b30 0x55abf4978f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb7f7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf49be0e0 0x55abf49de4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:54968/0 (socket says 10.20.193.28:54968) 2026-02-25T11:56:04.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb7f7fe640 1 -- 10.20.193.28:0/2335416488 learned_addr learned my addr 10.20.193.28:0/2335416488 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:04.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb7effd640 1 -- 10.20.193.28:0/2335416488 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf49be0e0 msgr2=0x55abf49de4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:04.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb7effd640 1 --2- 10.20.193.28:0/2335416488 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf49be0e0 0x55abf49de4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb7effd640 1 -- 10.20.193.28:0/2335416488 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 msgr2=0x55abf49bdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:04.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb7effd640 1 --2- 10.20.193.28:0/2335416488 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf49bdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb7effd640 1 -- 10.20.193.28:0/2335416488 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abf482fd10 con 0x55abf4978b30 2026-02-25T11:56:04.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb7e7fc640 1 --2- 10.20.193.28:0/2335416488 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf49bdba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:56:04.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.370+0000 7ffb7effd640 1 --2- 10.20.193.28:0/2335416488 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf4978b30 0x55abf4978f00 secure :-1 s=READY pgs=167 gs=1 cs=0 l=1 c_cookie=1112468341170460189 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb740a9420 tx=0x7ffb74009380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f70480feb55561d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:04.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.371+0000 7ffb4b7fe640 1 -- 10.20.193.28:0/2335416488 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb740ac650 con 0x55abf4978b30 2026-02-25T11:56:04.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.371+0000 7ffb4b7fe640 1 -- 10.20.193.28:0/2335416488 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ffb74016ea0 con 0x55abf4978b30 2026-02-25T11:56:04.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.371+0000 7ffb4b7fe640 1 -- 10.20.193.28:0/2335416488 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb74004e80 con 0x55abf4978b30 2026-02-25T11:56:04.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.371+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2335416488 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf4978b30 msgr2=0x55abf4978f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:04.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.371+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/2335416488 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf4978b30 0x55abf4978f00 secure :-1 s=READY pgs=167 gs=1 cs=0 l=1 c_cookie=1112468341170460189 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb740a9420 tx=0x7ffb74009380 comp rx=0 tx=0).stop 2026-02-25T11:56:04.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.371+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2335416488 shutdown_connections 2026-02-25T11:56:04.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.371+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/2335416488 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf49be0e0 0x55abf49de4c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.371+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/2335416488 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf49bdba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.371+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/2335416488 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf4978b30 0x55abf4978f00 unknown :-1 s=CLOSED pgs=167 gs=1 cs=0 l=1 c_cookie=1112468341170460189 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.371+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2335416488 >> 10.20.193.28:0/2335416488 conn(0x55abf48977a0 msgr2=0x55abf4977410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:04.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.371+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2335416488 shutdown_connections 2026-02-25T11:56:04.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.371+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2335416488 wait complete. 2026-02-25T11:56:04.377 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:04 trial028 ceph-mon[39224]: pgmap v461: 97 pgs: 97 active+clean; 6.0 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:56:04.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.372+0000 7ffb872a1e00 1 -- start start 2026-02-25T11:56:04.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.373+0000 7ffb872a1e00 1 Processor -- start 2026-02-25T11:56:04.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.373+0000 7ffb872a1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4978b30 0x55abf4990ed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.373+0000 7ffb872a1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf4979440 0x55abf4991410 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.373+0000 7ffb872a1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf49be0e0 0x55abf4996400 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.380 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.373+0000 7ffb872a1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55abf4996940 con 0x55abf4978b30 2026-02-25T11:56:04.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.373+0000 7ffb872a1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55abf4996ab0 con 0x55abf49be0e0 2026-02-25T11:56:04.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.373+0000 7ffb872a1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55abf4996c70 con 0x55abf4979440 2026-02-25T11:56:04.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.373+0000 7ffb7effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4978b30 0x55abf4990ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.373+0000 7ffb7effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4978b30 0x55abf4990ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38754/0 (socket says 10.20.193.28:38754) 2026-02-25T11:56:04.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.373+0000 7ffb7effd640 1 -- 10.20.193.28:0/3308742132 learned_addr learned my addr 10.20.193.28:0/3308742132 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:04.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.373+0000 7ffb7f7fe640 1 --2- 10.20.193.28:0/3308742132 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf49be0e0 0x55abf4996400 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb7effd640 1 -- 10.20.193.28:0/3308742132 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf4979440 msgr2=0x55abf4991410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:04.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb7effd640 1 --2- 10.20.193.28:0/3308742132 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf4979440 0x55abf4991410 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb7effd640 1 -- 10.20.193.28:0/3308742132 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf49be0e0 msgr2=0x55abf4996400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:04.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb7effd640 1 --2- 10.20.193.28:0/3308742132 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf49be0e0 0x55abf4996400 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.381 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb7effd640 1 -- 10.20.193.28:0/3308742132 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb80169160 con 0x55abf4978b30 2026-02-25T11:56:04.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb7f7fe640 1 --2- 10.20.193.28:0/3308742132 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf49be0e0 0x55abf4996400 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:56:04.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb7effd640 1 --2- 10.20.193.28:0/3308742132 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4978b30 0x55abf4990ed0 secure :-1 s=READY pgs=791 gs=1 cs=0 l=1 c_cookie=7987905018816221858 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb740ad0c0 tx=0x7ffb74009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edabd4afa19d2a2 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:04.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb48ff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:04.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb48ff9640 1 -- 10.20.193.28:0/3308742132 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb740b7d20 con 0x55abf4978b30 2026-02-25T11:56:04.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb48ff9640 1 -- 10.20.193.28:0/3308742132 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ffb740ac650 con 0x55abf4978b30 2026-02-25T11:56:04.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb48ff9640 1 -- 10.20.193.28:0/3308742132 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb740add60 con 0x55abf4978b30 2026-02-25T11:56:04.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3308742132 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb680098a0 con 0x55abf4978b30 2026-02-25T11:56:04.382 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb872a1e00 10 client.17154 _ll_register_callbacks cb 0x55abf4805a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:04.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3308742132 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55abf451a6f0 con 0x55abf4978b30 2026-02-25T11:56:04.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb48ff9640 1 -- 10.20.193.28:0/3308742132 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ffb740b7050 con 0x55abf4978b30 2026-02-25T11:56:04.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.374+0000 7ffb48ff9640 1 -- 10.20.193.28:0/3308742132 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ffb740aae30 con 0x55abf4978b30 2026-02-25T11:56:04.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.375+0000 7ffb872a1e00 10 client.17154 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:04.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.375+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3308742132 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55abf48083c0 con 0x55abf4978b30 2026-02-25T11:56:04.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.375+0000 7ffb48ff9640 1 -- 10.20.193.28:0/3308742132 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7ffb74072060 con 0x55abf4978b30 2026-02-25T11:56:04.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.375+0000 7ffb872a1e00 10 client.17154 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:04.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.375+0000 7ffb872a1e00 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:04.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.375+0000 7ffb872a1e00 20 client.17154 populate_metadata read hostname 'trial028' 2026-02-25T11:56:04.383 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.375+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/3308742132 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55abf49bb430 0x55abf49bb8d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.375+0000 7ffb872a1e00 4 client.17154 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:56:04.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.375+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3308742132 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abf4856fc0 con 0x55abf49bb430 2026-02-25T11:56:04.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.375+0000 7ffb7e7fc640 1 --2- 10.20.193.28:0/3308742132 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55abf49bb430 0x55abf49bb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.376+0000 7ffb7e7fc640 1 --2- 10.20.193.28:0/3308742132 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55abf49bb430 0x55abf49bb8d0 crc :-1 s=READY pgs=146 gs=4 cs=0 l=0 c_cookie=14919105477800886395 s_cookie=11645478554846976714 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf0b4f6322e8007b server_cookie=a19d0d0b95155aca in_seq=0 out_seq=0 2026-02-25T11:56:04.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.376+0000 7ffb48ff9640 10 client.17154 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:04.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb48ff9640 1 -- 10.20.193.28:0/3308742132 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55abf4856fc0 con 0x55abf49bb430 2026-02-25T11:56:04.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb48ff9640 10 client.17154 handle_command_reply: tid=0 2026-02-25T11:56:04.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb872a1e00 1 client.17154 shutdown 2026-02-25T11:56:04.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3308742132 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4978b30 msgr2=0x55abf4990ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:04.384 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/3308742132 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4978b30 0x55abf4990ed0 secure :-1 s=READY pgs=791 gs=1 cs=0 l=1 c_cookie=7987905018816221858 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb740ad0c0 tx=0x7ffb74009f70 comp rx=0 tx=0).stop 2026-02-25T11:56:04.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3308742132 shutdown_connections 2026-02-25T11:56:04.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/3308742132 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55abf49bb430 0x55abf49bb8d0 crc :-1 s=READY pgs=146 gs=4 cs=0 l=0 c_cookie=14919105477800886395 s_cookie=11645478554846976714 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/3308742132 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf49be0e0 0x55abf4996400 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/3308742132 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf4979440 0x55abf4991410 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/3308742132 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4978b30 0x55abf4990ed0 unknown :-1 s=CLOSED pgs=791 gs=1 cs=0 l=1 c_cookie=7987905018816221858 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3308742132 >> 10.20.193.28:0/3308742132 conn(0x55abf48977a0 msgr2=0x55abf4977b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:04.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb48ff9640 0 client.17154 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:04.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3308742132 shutdown_connections 2026-02-25T11:56:04.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3308742132 wait complete. 2026-02-25T11:56:04.385 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.377+0000 7ffb872a1e00 20 client.17154 trim_cache size 0 max 16384 2026-02-25T11:56:04.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.384+0000 7ffb872a1e00 1 Processor -- start 2026-02-25T11:56:04.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.384+0000 7ffb872a1e00 1 -- start start 2026-02-25T11:56:04.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.384+0000 7ffb872a1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf4978b30 0x55abf49912f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.384+0000 7ffb872a1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf4977d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.384+0000 7ffb872a1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf49be0e0 0x55abf4978260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.384+0000 7ffb872a1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55abf48821a0 con 0x55abf4979440 2026-02-25T11:56:04.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.384+0000 7ffb872a1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55abf4542700 con 0x55abf49be0e0 2026-02-25T11:56:04.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.384+0000 7ffb872a1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55abf49b7d10 con 0x55abf4978b30 2026-02-25T11:56:04.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.384+0000 7ffb7e7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf4977d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb7e7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf4977d20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38770/0 (socket says 10.20.193.28:38770) 2026-02-25T11:56:04.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb7e7fc640 1 -- 10.20.193.28:0/3576201477 learned_addr learned my addr 10.20.193.28:0/3576201477 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:04.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb7effd640 1 --2- 10.20.193.28:0/3576201477 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf4978b30 0x55abf49912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb7e7fc640 1 -- 10.20.193.28:0/3576201477 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf4978b30 msgr2=0x55abf49912f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:04.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb7e7fc640 1 --2- 10.20.193.28:0/3576201477 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf4978b30 0x55abf49912f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb7e7fc640 1 -- 10.20.193.28:0/3576201477 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf49be0e0 msgr2=0x55abf4978260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2026-02-25T11:56:04.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb7f7fe640 1 --2- 10.20.193.28:0/3576201477 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf49be0e0 0x55abf4978260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb7e7fc640 1 --2- 10.20.193.28:0/3576201477 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf49be0e0 0x55abf4978260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb7e7fc640 1 -- 10.20.193.28:0/3576201477 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abf481a0e0 con 0x55abf4979440 2026-02-25T11:56:04.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb7f7fe640 1 --2- 10.20.193.28:0/3576201477 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf49be0e0 0x55abf4978260 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:56:04.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb7e7fc640 1 --2- 10.20.193.28:0/3576201477 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf4977d20 secure :-1 s=READY pgs=792 gs=1 cs=0 l=1 c_cookie=12378798376630501286 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb68004480 tx=0x7ffb68014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abca5389b13e3fa6 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:04.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb4affd640 1 -- 10.20.193.28:0/3576201477 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb68021040 con 0x55abf4979440 2026-02-25T11:56:04.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb4affd640 1 -- 10.20.193.28:0/3576201477 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ffb68011e60 con 0x55abf4979440 2026-02-25T11:56:04.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.385+0000 7ffb4affd640 1 -- 10.20.193.28:0/3576201477 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb6801b870 con 0x55abf4979440 2026-02-25T11:56:04.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.386+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3576201477 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 msgr2=0x55abf4977d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:04.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.386+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/3576201477 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf4977d20 secure :-1 s=READY pgs=792 gs=1 cs=0 l=1 c_cookie=12378798376630501286 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb68004480 tx=0x7ffb68014000 comp rx=0 tx=0).stop 2026-02-25T11:56:04.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.386+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3576201477 shutdown_connections 2026-02-25T11:56:04.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.386+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/3576201477 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf49be0e0 0x55abf4978260 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.386+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/3576201477 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf4977d20 unknown :-1 s=CLOSED pgs=792 gs=1 cs=0 l=1 c_cookie=12378798376630501286 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.386+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/3576201477 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf4978b30 0x55abf49912f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.386+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3576201477 >> 10.20.193.28:0/3576201477 conn(0x55abf48977a0 msgr2=0x55abf497b930 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:04.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.386+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3576201477 shutdown_connections 2026-02-25T11:56:04.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.386+0000 7ffb872a1e00 1 -- 10.20.193.28:0/3576201477 wait complete. 2026-02-25T11:56:04.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.386+0000 7ffb872a1e00 1 -- start start 2026-02-25T11:56:04.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.386+0000 7ffb872a1e00 1 Processor -- start 2026-02-25T11:56:04.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb872a1e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf4978b30 0x55abf4997f80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb872a1e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf49984c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb872a1e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf49be0e0 0x55abf4a225f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb872a1e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55abf4a22b30 con 0x55abf4979440 2026-02-25T11:56:04.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb872a1e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55abf4a22ca0 con 0x55abf4978b30 2026-02-25T11:56:04.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb872a1e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55abf4a22e60 con 0x55abf49be0e0 2026-02-25T11:56:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb7e7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf49984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb7e7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf49984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38786/0 (socket says 10.20.193.28:38786) 2026-02-25T11:56:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb7e7fc640 1 -- 10.20.193.28:0/2663990881 learned_addr learned my addr 10.20.193.28:0/2663990881 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb7f7fe640 1 --2- 10.20.193.28:0/2663990881 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf49be0e0 0x55abf4a225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb7effd640 1 --2- 10.20.193.28:0/2663990881 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf4978b30 0x55abf4997f80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb7e7fc640 1 -- 10.20.193.28:0/2663990881 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf49be0e0 msgr2=0x55abf4a225f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb7e7fc640 1 --2- 10.20.193.28:0/2663990881 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf49be0e0 0x55abf4a225f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb7e7fc640 1 -- 10.20.193.28:0/2663990881 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf4978b30 msgr2=0x55abf4997f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb7e7fc640 1 --2- 10.20.193.28:0/2663990881 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf4978b30 0x55abf4997f80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.387+0000 7ffb7e7fc640 1 -- 10.20.193.28:0/2663990881 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abf4814b60 con 0x55abf4979440 2026-02-25T11:56:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.388+0000 7ffb7e7fc640 1 --2- 10.20.193.28:0/2663990881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf49984c0 secure :-1 s=READY pgs=793 gs=1 cs=0 l=1 c_cookie=6207713927958491166 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb68009f80 tx=0x7ffb68011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56263aed7735f01e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:04.396 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.388+0000 7ffb48ff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.388+0000 7ffb48ff9640 1 -- 10.20.193.28:0/2663990881 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb68021070 con 0x55abf4979440 2026-02-25T11:56:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.388+0000 7ffb48ff9640 1 -- 10.20.193.28:0/2663990881 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ffb6801b7e0 con 0x55abf4979440 2026-02-25T11:56:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.388+0000 7ffb48ff9640 1 -- 10.20.193.28:0/2663990881 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffb68004af0 con 0x55abf4979440 2026-02-25T11:56:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.388+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2663990881 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb68009a30 con 0x55abf4979440 2026-02-25T11:56:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.388+0000 7ffb872a1e00 10 client.17160 _ll_register_callbacks cb 0x55abf4805a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.388+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2663990881 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb74016ea0 con 0x55abf4979440 2026-02-25T11:56:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.388+0000 7ffb48ff9640 1 -- 10.20.193.28:0/2663990881 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ffb680284f0 con 0x55abf4979440 2026-02-25T11:56:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.388+0000 7ffb48ff9640 1 -- 10.20.193.28:0/2663990881 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ffb680286f0 con 0x55abf4979440 2026-02-25T11:56:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.388+0000 7ffb872a1e00 10 client.17160 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.388+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2663990881 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55abf48086f0 con 0x55abf4979440 2026-02-25T11:56:04.397 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.389+0000 7ffb48ff9640 1 -- 10.20.193.28:0/2663990881 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7ffb68038070 con 0x55abf4979440 2026-02-25T11:56:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.389+0000 7ffb872a1e00 10 client.17160 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.389+0000 7ffb872a1e00 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.389+0000 7ffb872a1e00 20 client.17160 populate_metadata read hostname 'trial028' 2026-02-25T11:56:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.389+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/2663990881 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55abf4a6add0 0x55abf4a8b280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.389+0000 7ffb872a1e00 4 client.17160 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:56:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.389+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2663990881 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abf4517f40 con 0x55abf4a6add0 2026-02-25T11:56:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.389+0000 7ffb7effd640 1 --2- 10.20.193.28:0/2663990881 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55abf4a6add0 0x55abf4a8b280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb7effd640 1 --2- 10.20.193.28:0/2663990881 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55abf4a6add0 0x55abf4a8b280 crc :-1 s=READY pgs=147 gs=4 cs=0 l=0 c_cookie=10971999659977618449 s_cookie=16364021311729140997 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98445f8f00ac3411 server_cookie=e318aeaec6898505 in_seq=0 out_seq=0 2026-02-25T11:56:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb48ff9640 10 client.17160 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb48ff9640 1 -- 10.20.193.28:0/2663990881 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abf4517f40 con 0x55abf4a6add0 2026-02-25T11:56:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb48ff9640 10 client.17160 handle_command_reply: tid=0 2026-02-25T11:56:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb872a1e00 1 client.17160 shutdown 2026-02-25T11:56:04.398 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2663990881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 msgr2=0x55abf49984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:04.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/2663990881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf49984c0 secure :-1 s=READY pgs=793 gs=1 cs=0 l=1 c_cookie=6207713927958491166 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb68009f80 tx=0x7ffb68011c40 comp rx=0 tx=0).stop 2026-02-25T11:56:04.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2663990881 shutdown_connections 2026-02-25T11:56:04.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/2663990881 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55abf4a6add0 0x55abf4a8b280 crc :-1 s=READY pgs=147 gs=4 cs=0 l=0 c_cookie=10971999659977618449 s_cookie=16364021311729140997 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/2663990881 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55abf49be0e0 0x55abf4a225f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.399 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:56:04.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/2663990881 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55abf4979440 0x55abf49984c0 unknown :-1 s=CLOSED pgs=793 gs=1 cs=0 l=1 c_cookie=6207713927958491166 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb872a1e00 1 --2- 10.20.193.28:0/2663990881 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55abf4978b30 0x55abf4997f80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2663990881 >> 10.20.193.28:0/2663990881 conn(0x55abf48977a0 msgr2=0x55abf4976d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:04.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb48ff9640 0 client.17160 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:04.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2663990881 shutdown_connections 2026-02-25T11:56:04.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb872a1e00 1 -- 10.20.193.28:0/2663990881 wait complete. 2026-02-25T11:56:04.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.390+0000 7ffb872a1e00 20 client.17160 trim_cache size 0 max 16384 2026-02-25T11:56:04.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.391+0000 7ffb7ffff640 1 -- 10.20.193.28:0/705188737 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffb540b5760 msgr2=0x7ffb540d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:04.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.391+0000 7ffb7ffff640 1 --2- 10.20.193.28:0/705188737 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffb540b5760 0x7ffb540d5c10 secure :-1 s=READY pgs=362 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb80128ec0 tx=0x7ffb68005f90 comp rx=0 tx=0).stop 2026-02-25T11:56:04.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.391+0000 7ffb7ffff640 1 -- 10.20.193.28:0/705188737 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffb801140a0 msgr2=0x7ffb80122520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:04.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.391+0000 7ffb7ffff640 1 --2- 10.20.193.28:0/705188737 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffb801140a0 0x7ffb80122520 secure :-1 s=READY pgs=206 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffb74009480 tx=0x7ffb7400ee60 comp rx=0 tx=0).stop 2026-02-25T11:56:04.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.391+0000 7ffb7ffff640 1 -- 10.20.193.28:0/705188737 shutdown_connections 2026-02-25T11:56:04.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.391+0000 7ffb7ffff640 1 --2- 10.20.193.28:0/705188737 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffb540b5760 0x7ffb540d5c10 unknown :-1 s=CLOSED pgs=362 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.391+0000 7ffb7ffff640 1 --2- 10.20.193.28:0/705188737 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffb80119d30 0x7ffb80127e30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.391+0000 7ffb7ffff640 1 --2- 10.20.193.28:0/705188737 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffb80114a90 0x7ffb80122a60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.391+0000 7ffb7ffff640 1 --2- 10.20.193.28:0/705188737 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffb801140a0 0x7ffb80122520 unknown :-1 s=CLOSED pgs=206 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:04.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.391+0000 7ffb7ffff640 1 -- 10.20.193.28:0/705188737 >> 10.20.193.28:0/705188737 conn(0x7ffb80064c40 msgr2=0x7ffb80116b30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:04.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.391+0000 7ffb7ffff640 1 -- 10.20.193.28:0/705188737 shutdown_connections 2026-02-25T11:56:04.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:04.391+0000 7ffb7ffff640 1 -- 10.20.193.28:0/705188737 wait complete. 2026-02-25T11:56:04.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:04 trial167 ceph-mon[43392]: pgmap v461: 97 pgs: 97 active+clean; 6.0 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:56:04.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:04 trial119 ceph-mon[42710]: pgmap v461: 97 pgs: 97 active+clean; 6.0 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:56:05.489 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:56:05.490 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:56:05.674 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:05.846 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.843+0000 7f985a296640 1 -- 10.20.193.28:0/2736177589 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f985410f840 msgr2=0x7f985410fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:05.847 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.843+0000 7f985a296640 1 --2- 10.20.193.28:0/2736177589 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f985410f840 0x7f985410fc10 secure :-1 s=READY pgs=794 gs=3 cs=0 l=1 c_cookie=6462817843266483610 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9840009950 tx=0x7f984002f1d0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.847 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.843+0000 7f985a296640 1 -- 10.20.193.28:0/2736177589 shutdown_connections 2026-02-25T11:56:05.847 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.843+0000 7f985a296640 1 --2- 10.20.193.28:0/2736177589 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9854115a30 0x7f9854176400 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.847 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.843+0000 7f985a296640 1 --2- 10.20.193.28:0/2736177589 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9854110230 0x7f9854115280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.847 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.843+0000 7f985a296640 1 --2- 10.20.193.28:0/2736177589 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f985410f840 0x7f985410fc10 unknown :-1 s=CLOSED pgs=794 gs=3 cs=0 l=1 c_cookie=6462817843266483610 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.847 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.843+0000 7f985a296640 1 -- 10.20.193.28:0/2736177589 >> 10.20.193.28:0/2736177589 conn(0x7f98540ed130 msgr2=0x7f985410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:05.847 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.843+0000 7f985a296640 1 -- 10.20.193.28:0/2736177589 shutdown_connections 2026-02-25T11:56:05.847 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.843+0000 7f985a296640 1 -- 10.20.193.28:0/2736177589 wait complete. 2026-02-25T11:56:05.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.844+0000 7f985a296640 1 Processor -- start 2026-02-25T11:56:05.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.844+0000 7f985a296640 1 -- start start 2026-02-25T11:56:05.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.844+0000 7f985a296640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f985410f840 0x7f985411e200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.844+0000 7f985a296640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9854110230 0x7f985411e740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.844+0000 7f985a296640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9854115a30 0x7f9854123ac0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.844+0000 7f985a296640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f98541240e0 con 0x7f9854110230 2026-02-25T11:56:05.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.844+0000 7f985a296640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9854124250 con 0x7f985410f840 2026-02-25T11:56:05.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.844+0000 7f985a296640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9854124410 con 0x7f9854115a30 2026-02-25T11:56:05.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.844+0000 7f98537fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9854110230 0x7f985411e740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.844+0000 7f98537fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9854110230 0x7f985411e740 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38796/0 (socket says 10.20.193.28:38796) 2026-02-25T11:56:05.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.844+0000 7f98537fe640 1 -- 10.20.193.28:0/875133219 learned_addr learned my addr 10.20.193.28:0/875133219 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:05.848 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.844+0000 7f9853fff640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f985410f840 0x7f985411e200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.844+0000 7f985880c640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9854115a30 0x7f9854123ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.845+0000 7f98537fe640 1 -- 10.20.193.28:0/875133219 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9854115a30 msgr2=0x7f9854123ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:05.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.845+0000 7f98537fe640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9854115a30 0x7f9854123ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.845+0000 7f98537fe640 1 -- 10.20.193.28:0/875133219 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f985410f840 msgr2=0x7f985411e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:05.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.845+0000 7f98537fe640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f985410f840 0x7f985411e200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.845+0000 7f98537fe640 1 -- 10.20.193.28:0/875133219 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9840009570 con 0x7f9854110230 2026-02-25T11:56:05.849 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.845+0000 7f98537fe640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9854110230 0x7f985411e740 secure :-1 s=READY pgs=795 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9844009480 tx=0x7f984400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:05.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.845+0000 7f98517fa640 1 -- 10.20.193.28:0/875133219 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f984400cd40 con 0x7f9854110230 2026-02-25T11:56:05.850 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.845+0000 7f985a296640 1 -- 10.20.193.28:0/875133219 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98541246f0 con 0x7f9854110230 2026-02-25T11:56:05.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.845+0000 7f985a296640 1 -- 10.20.193.28:0/875133219 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9854164df0 con 0x7f9854110230 2026-02-25T11:56:05.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.845+0000 7f98517fa640 1 -- 10.20.193.28:0/875133219 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9844004d10 con 0x7f9854110230 2026-02-25T11:56:05.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.845+0000 7f98517fa640 1 -- 10.20.193.28:0/875133219 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f984400cea0 con 0x7f9854110230 2026-02-25T11:56:05.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.848+0000 7f98517fa640 1 -- 10.20.193.28:0/875133219 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f984401f060 con 0x7f9854110230 2026-02-25T11:56:05.851 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.849+0000 7f98517fa640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f98280b56e0 0x7f98280d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.849+0000 7f98517fa640 1 -- 10.20.193.28:0/875133219 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9844014070 con 0x7f9854110230 2026-02-25T11:56:05.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.850+0000 7f9853fff640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f98280b56e0 0x7f98280d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.852 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.850+0000 7f9853fff640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f98280b56e0 0x7f98280d5b90 secure :-1 s=READY pgs=363 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9840005b30 tx=0x7f984003a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:05.853 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.850+0000 7f985c291e00 1 Processor -- start 2026-02-25T11:56:05.853 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.850+0000 7f985c291e00 1 -- start start 2026-02-25T11:56:05.853 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985c291e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 0x55f5c5793970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.853 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985c291e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c5793eb0 0x55f5c57d8460 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.853 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985c291e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c57d89a0 0x55f5c57f8d80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.853 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985c291e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f5c56a9d80 con 0x55f5c57d89a0 2026-02-25T11:56:05.853 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985c291e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f5c56a5ab0 con 0x55f5c5793eb0 2026-02-25T11:56:05.853 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985c291e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5c569fa50 con 0x55f5c57935a0 2026-02-25T11:56:05.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985880c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c57d89a0 0x55f5c57f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.854 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985880c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c57d89a0 0x55f5c57f8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38802/0 (socket says 10.20.193.28:38802) 2026-02-25T11:56:05.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985880c640 1 -- 10.20.193.28:0/2760726814 learned_addr learned my addr 10.20.193.28:0/2760726814 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:05.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f98537fe640 1 --2- 10.20.193.28:0/2760726814 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c5793eb0 0x55f5c57d8460 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f9853fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 0x55f5c5793970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985880c640 1 -- 10.20.193.28:0/2760726814 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 msgr2=0x55f5c5793970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:05.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985880c640 1 --2- 10.20.193.28:0/2760726814 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 0x55f5c5793970 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985880c640 1 -- 10.20.193.28:0/2760726814 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c5793eb0 msgr2=0x55f5c57d8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:05.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985880c640 1 --2- 10.20.193.28:0/2760726814 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c5793eb0 0x55f5c57d8460 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985880c640 1 -- 10.20.193.28:0/2760726814 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c564b120 con 0x55f5c57d89a0 2026-02-25T11:56:05.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.851+0000 7f985880c640 1 --2- 10.20.193.28:0/2760726814 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c57d89a0 0x55f5c57f8d80 secure :-1 s=READY pgs=796 gs=1 cs=0 l=1 c_cookie=10254728231487863446 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9848005660 tx=0x7f9848003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e501cfea200fa96 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:05.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.852+0000 7f9824ce3640 1 -- 10.20.193.28:0/2760726814 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f984800aea0 con 0x55f5c57d89a0 2026-02-25T11:56:05.855 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.852+0000 7f9824ce3640 1 -- 10.20.193.28:0/2760726814 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f98480118f0 con 0x55f5c57d89a0 2026-02-25T11:56:05.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.852+0000 7f9824ce3640 1 -- 10.20.193.28:0/2760726814 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9848010b20 con 0x55f5c57d89a0 2026-02-25T11:56:05.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.852+0000 7f985c291e00 1 -- 10.20.193.28:0/2760726814 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c57d89a0 msgr2=0x55f5c57f8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:05.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.852+0000 7f985c291e00 1 --2- 10.20.193.28:0/2760726814 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c57d89a0 0x55f5c57f8d80 secure :-1 s=READY pgs=796 gs=1 cs=0 l=1 c_cookie=10254728231487863446 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9848005660 tx=0x7f9848003c30 comp rx=0 tx=0).stop 2026-02-25T11:56:05.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.852+0000 7f985c291e00 1 -- 10.20.193.28:0/2760726814 shutdown_connections 2026-02-25T11:56:05.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.852+0000 7f985c291e00 1 --2- 10.20.193.28:0/2760726814 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c57d89a0 0x55f5c57f8d80 unknown :-1 s=CLOSED pgs=796 gs=1 cs=0 l=1 c_cookie=10254728231487863446 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.852+0000 7f985c291e00 1 --2- 10.20.193.28:0/2760726814 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c5793eb0 0x55f5c57d8460 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.852+0000 7f985c291e00 1 --2- 10.20.193.28:0/2760726814 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 0x55f5c5793970 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.852+0000 7f985c291e00 1 -- 10.20.193.28:0/2760726814 >> 10.20.193.28:0/2760726814 conn(0x55f5c57718f0 msgr2=0x55f5c5791cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:05.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.852+0000 7f985c291e00 1 -- 10.20.193.28:0/2760726814 shutdown_connections 2026-02-25T11:56:05.856 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.852+0000 7f985c291e00 1 -- 10.20.193.28:0/2760726814 wait complete. 2026-02-25T11:56:05.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.853+0000 7f985c291e00 1 -- start start 2026-02-25T11:56:05.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.854+0000 7f985c291e00 1 Processor -- start 2026-02-25T11:56:05.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.854+0000 7f985c291e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c57935a0 0x55f5c57f0310 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.854+0000 7f985c291e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c5793eb0 0x55f5c57f0850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.854+0000 7f985c291e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 0x55f5c57f5840 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.854+0000 7f985c291e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f5c57f5d80 con 0x55f5c57935a0 2026-02-25T11:56:05.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.854+0000 7f985c291e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f5c57f5ef0 con 0x55f5c57d89a0 2026-02-25T11:56:05.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.854+0000 7f985c291e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5c57f60b0 con 0x55f5c5793eb0 2026-02-25T11:56:05.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.854+0000 7f9853fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c57935a0 0x55f5c57f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.854+0000 7f985880c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 0x55f5c57f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.854+0000 7f98537fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c5793eb0 0x55f5c57f0850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.854+0000 7f985880c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 0x55f5c57f5840 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:45224/0 (socket says 10.20.193.28:45224) 2026-02-25T11:56:05.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.854+0000 7f985880c640 1 -- 10.20.193.28:0/3894269597 learned_addr learned my addr 10.20.193.28:0/3894269597 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:05.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f98537fe640 1 -- 10.20.193.28:0/3894269597 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 msgr2=0x55f5c57f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:05.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f98537fe640 1 --2- 10.20.193.28:0/3894269597 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 0x55f5c57f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f98537fe640 1 -- 10.20.193.28:0/3894269597 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c57935a0 msgr2=0x55f5c57f0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:05.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f98537fe640 1 --2- 10.20.193.28:0/3894269597 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c57935a0 0x55f5c57f0310 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f98537fe640 1 -- 10.20.193.28:0/3894269597 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c5623810 con 0x55f5c5793eb0 2026-02-25T11:56:05.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f985880c640 1 --2- 10.20.193.28:0/3894269597 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 0x55f5c57f5840 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:56:05.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f98537fe640 1 --2- 10.20.193.28:0/3894269597 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c5793eb0 0x55f5c57f0850 secure :-1 s=READY pgs=207 gs=3 cs=0 l=1 c_cookie=4632316189838020826 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f98440a94d0 tx=0x7f98440ac8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40494b08169610da server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:05.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f9815ffb640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:56:05.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f9815ffb640 1 -- 10.20.193.28:0/3894269597 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f98440b2c20 con 0x55f5c5793eb0 2026-02-25T11:56:05.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f9815ffb640 1 -- 10.20.193.28:0/3894269597 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f98440b5200 con 0x55f5c5793eb0 2026-02-25T11:56:05.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f9815ffb640 1 -- 10.20.193.28:0/3894269597 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f98440b7e30 con 0x55f5c5793eb0 2026-02-25T11:56:05.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f985c291e00 1 -- 10.20.193.28:0/3894269597 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98480052d0 con 0x55f5c5793eb0 2026-02-25T11:56:05.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f985c291e00 10 client.26860 _ll_register_callbacks cb 0x55f5c5620e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:05.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f985c291e00 1 -- 10.20.193.28:0/3894269597 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f5c53f7530 con 0x55f5c5793eb0 2026-02-25T11:56:05.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.855+0000 7f9815ffb640 1 -- 10.20.193.28:0/3894269597 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f98440050b0 con 0x55f5c5793eb0 2026-02-25T11:56:05.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.856+0000 7f9815ffb640 1 -- 10.20.193.28:0/3894269597 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f98440b8040 con 0x55f5c5793eb0 2026-02-25T11:56:05.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.856+0000 7f985c291e00 10 client.26860 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:05.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.856+0000 7f985c291e00 1 -- 10.20.193.28:0/3894269597 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55f5c56354f0 con 0x55f5c5793eb0 2026-02-25T11:56:05.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.856+0000 7f9815ffb640 1 -- 10.20.193.28:0/3894269597 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f98440adb90 con 0x55f5c5793eb0 2026-02-25T11:56:05.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.856+0000 7f985c291e00 10 client.26860 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:05.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.856+0000 7f985c291e00 10 client.26860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:05.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.856+0000 7f985c291e00 20 client.26860 populate_metadata read hostname 'trial028' 2026-02-25T11:56:05.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.856+0000 7f985c291e00 1 --2- 10.20.193.28:0/3894269597 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f5c57d5b70 0x55f5c57d6010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.856+0000 7f985c291e00 4 client.26860 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:56:05.863 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.856+0000 7f985c291e00 1 -- 10.20.193.28:0/3894269597 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5c564cb50 con 0x55f5c57d5b70 2026-02-25T11:56:05.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.856+0000 7f9853fff640 1 --2- 10.20.193.28:0/3894269597 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f5c57d5b70 0x55f5c57d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.857+0000 7f9853fff640 1 --2- 10.20.193.28:0/3894269597 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f5c57d5b70 0x55f5c57d6010 crc :-1 s=READY pgs=148 gs=4 cs=0 l=0 c_cookie=4555512444609592636 s_cookie=2562945726674720798 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f386e6ea6d37d3c server_cookie=2391698f310d581e in_seq=0 out_seq=0 2026-02-25T11:56:05.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.857+0000 7f9815ffb640 10 client.26860 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:05.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.857+0000 7f9815ffb640 1 -- 10.20.193.28:0/3894269597 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55f5c564cb50 con 0x55f5c57d5b70 2026-02-25T11:56:05.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.857+0000 7f9815ffb640 10 client.26860 handle_command_reply: tid=0 2026-02-25T11:56:05.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.858+0000 7f985c291e00 1 client.26860 shutdown 2026-02-25T11:56:05.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.858+0000 7f985c291e00 1 -- 10.20.193.28:0/3894269597 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c5793eb0 msgr2=0x55f5c57f0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:05.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.858+0000 7f985c291e00 1 --2- 10.20.193.28:0/3894269597 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c5793eb0 0x55f5c57f0850 secure :-1 s=READY pgs=207 gs=3 cs=0 l=1 c_cookie=4632316189838020826 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f98440a94d0 tx=0x7f98440ac8e0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.858+0000 7f985c291e00 1 -- 10.20.193.28:0/3894269597 shutdown_connections 2026-02-25T11:56:05.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.858+0000 7f985c291e00 1 --2- 10.20.193.28:0/3894269597 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f5c57d5b70 0x55f5c57d6010 crc :-1 s=READY pgs=148 gs=4 cs=0 l=0 c_cookie=4555512444609592636 s_cookie=2562945726674720798 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.858+0000 7f985c291e00 1 --2- 10.20.193.28:0/3894269597 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 0x55f5c57f5840 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.858+0000 7f985c291e00 1 --2- 10.20.193.28:0/3894269597 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c5793eb0 0x55f5c57f0850 unknown :-1 s=CLOSED pgs=207 gs=3 cs=0 l=1 c_cookie=4632316189838020826 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.858+0000 7f985c291e00 1 --2- 10.20.193.28:0/3894269597 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c57935a0 0x55f5c57f0310 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.858+0000 7f985c291e00 1 -- 10.20.193.28:0/3894269597 >> 10.20.193.28:0/3894269597 conn(0x55f5c57718f0 msgr2=0x55f5c5791cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:05.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.858+0000 7f9815ffb640 0 client.26860 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:05.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.858+0000 7f985c291e00 1 -- 10.20.193.28:0/3894269597 shutdown_connections 2026-02-25T11:56:05.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.858+0000 7f985c291e00 1 -- 10.20.193.28:0/3894269597 wait complete. 2026-02-25T11:56:05.865 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.858+0000 7f985c291e00 20 client.26860 trim_cache size 0 max 16384 2026-02-25T11:56:05.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.867+0000 7f985c291e00 1 Processor -- start 2026-02-25T11:56:05.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.867+0000 7f985c291e00 1 -- start start 2026-02-25T11:56:05.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f985c291e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 0x55f5c57edcd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f985c291e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c5793eb0 0x55f5c57d25f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f985c291e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 0x55f5c57d2b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f985c291e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f5c57f0ae0 con 0x55f5c5793eb0 2026-02-25T11:56:05.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f985c291e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f5c569d680 con 0x55f5c57d89a0 2026-02-25T11:56:05.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f985c291e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5c535d790 con 0x55f5c57935a0 2026-02-25T11:56:05.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f98537fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c5793eb0 0x55f5c57d25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.870 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f9853fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 0x55f5c57edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f98537fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c5793eb0 0x55f5c57d25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:38820/0 (socket says 10.20.193.28:38820) 2026-02-25T11:56:05.871 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f98537fe640 1 -- 10.20.193.28:0/2622995306 learned_addr learned my addr 10.20.193.28:0/2622995306 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:05.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f98537fe640 1 -- 10.20.193.28:0/2622995306 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 msgr2=0x55f5c57edcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:05.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f98537fe640 1 --2- 10.20.193.28:0/2622995306 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 0x55f5c57edcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f98537fe640 1 -- 10.20.193.28:0/2622995306 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 msgr2=0x55f5c57d2b30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2026-02-25T11:56:05.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f98537fe640 1 --2- 10.20.193.28:0/2622995306 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 0x55f5c57d2b30 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f98537fe640 1 -- 10.20.193.28:0/2622995306 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c562ff70 con 0x55f5c5793eb0 2026-02-25T11:56:05.874 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f98537fe640 1 --2- 10.20.193.28:0/2622995306 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c5793eb0 0x55f5c57d25f0 secure :-1 s=READY pgs=797 gs=1 cs=0 l=1 c_cookie=14661641779472480000 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f98440af190 tx=0x7f984406f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb789d84897bb700 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:05.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f9817fff640 1 -- 10.20.193.28:0/2622995306 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f98440050b0 con 0x55f5c5793eb0 2026-02-25T11:56:05.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f9817fff640 1 -- 10.20.193.28:0/2622995306 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55f5c53f7530 con 0x55f5c5793eb0 2026-02-25T11:56:05.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.868+0000 7f9817fff640 1 -- 10.20.193.28:0/2622995306 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f98440b5200 con 0x55f5c5793eb0 2026-02-25T11:56:05.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.869+0000 7f985c291e00 1 -- 10.20.193.28:0/2622995306 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c5793eb0 msgr2=0x55f5c57d25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:05.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.869+0000 7f985c291e00 1 --2- 10.20.193.28:0/2622995306 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c5793eb0 0x55f5c57d25f0 secure :-1 s=READY pgs=797 gs=1 cs=0 l=1 c_cookie=14661641779472480000 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f98440af190 tx=0x7f984406f000 comp rx=0 tx=0).stop 2026-02-25T11:56:05.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.869+0000 7f985c291e00 1 -- 10.20.193.28:0/2622995306 shutdown_connections 2026-02-25T11:56:05.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.869+0000 7f985c291e00 1 --2- 10.20.193.28:0/2622995306 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 0x55f5c57d2b30 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.869+0000 7f985c291e00 1 --2- 10.20.193.28:0/2622995306 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c5793eb0 0x55f5c57d25f0 unknown :-1 s=CLOSED pgs=797 gs=1 cs=0 l=1 c_cookie=14661641779472480000 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.869+0000 7f985c291e00 1 --2- 10.20.193.28:0/2622995306 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 0x55f5c57edcd0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.869+0000 7f985c291e00 1 -- 10.20.193.28:0/2622995306 >> 10.20.193.28:0/2622995306 conn(0x55f5c57718f0 msgr2=0x55f5c57ec5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:05.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.869+0000 7f985c291e00 1 -- 10.20.193.28:0/2622995306 shutdown_connections 2026-02-25T11:56:05.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.869+0000 7f985c291e00 1 -- 10.20.193.28:0/2622995306 wait complete. 2026-02-25T11:56:05.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.869+0000 7f985c291e00 1 -- start start 2026-02-25T11:56:05.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f985c291e00 1 Processor -- start 2026-02-25T11:56:05.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f985c291e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 0x55f5c5773540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f985c291e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c5793eb0 0x55f5c5773a80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f985c291e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 0x55f5c58271e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f985c291e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f5c57dd9b0 con 0x55f5c5793eb0 2026-02-25T11:56:05.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f985c291e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f5c576e370 con 0x55f5c57d89a0 2026-02-25T11:56:05.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f985c291e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f5c57d61a0 con 0x55f5c57935a0 2026-02-25T11:56:05.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f98537fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c5793eb0 0x55f5c5773a80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f9853fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 0x55f5c5773540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f9853fff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 0x55f5c5773540 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:55048/0 (socket says 10.20.193.28:55048) 2026-02-25T11:56:05.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f9853fff640 1 -- 10.20.193.28:0/36405252 learned_addr learned my addr 10.20.193.28:0/36405252 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:05.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f985880c640 1 --2- 10.20.193.28:0/36405252 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 0x55f5c58271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f98537fe640 1 -- 10.20.193.28:0/36405252 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 msgr2=0x55f5c5773540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:05.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f98537fe640 1 --2- 10.20.193.28:0/36405252 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 0x55f5c5773540 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f98537fe640 1 -- 10.20.193.28:0/36405252 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 msgr2=0x55f5c58271e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:05.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f98537fe640 1 --2- 10.20.193.28:0/36405252 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 0x55f5c58271e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.870+0000 7f98537fe640 1 -- 10.20.193.28:0/36405252 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c5623b40 con 0x55f5c5793eb0 2026-02-25T11:56:05.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f98537fe640 1 --2- 10.20.193.28:0/36405252 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c5793eb0 0x55f5c5773a80 secure :-1 s=READY pgs=798 gs=1 cs=0 l=1 c_cookie=10675908589676172065 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f98440a94d0 tx=0x7f98440a8c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9428724c86309321 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:05.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f9815ffb640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:05.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f9815ffb640 1 -- 10.20.193.28:0/36405252 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f98440044f0 con 0x55f5c5793eb0 2026-02-25T11:56:05.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f9815ffb640 1 -- 10.20.193.28:0/36405252 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f98440b5200 con 0x55f5c5793eb0 2026-02-25T11:56:05.877 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f9815ffb640 1 -- 10.20.193.28:0/36405252 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f98440adb90 con 0x55f5c5793eb0 2026-02-25T11:56:05.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f985c291e00 1 -- 10.20.193.28:0/36405252 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98480052d0 con 0x55f5c5793eb0 2026-02-25T11:56:05.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f985c291e00 10 client.17178 _ll_register_callbacks cb 0x55f5c5620e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:05.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f985c291e00 1 -- 10.20.193.28:0/36405252 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98480118f0 con 0x55f5c5793eb0 2026-02-25T11:56:05.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f9815ffb640 1 -- 10.20.193.28:0/36405252 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f98440a8850 con 0x55f5c5793eb0 2026-02-25T11:56:05.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f9815ffb640 1 -- 10.20.193.28:0/36405252 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f98440b7bb0 con 0x55f5c5793eb0 2026-02-25T11:56:05.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f985c291e00 10 client.17178 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:05.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f985c291e00 1 -- 10.20.193.28:0/36405252 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55f5c5635750 con 0x55f5c5793eb0 2026-02-25T11:56:05.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f9815ffb640 1 -- 10.20.193.28:0/36405252 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f98440aaee0 con 0x55f5c5793eb0 2026-02-25T11:56:05.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f985c291e00 10 client.17178 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:05.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f985c291e00 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:05.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f985c291e00 20 client.17178 populate_metadata read hostname 'trial028' 2026-02-25T11:56:05.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f985c291e00 1 --2- 10.20.193.28:0/36405252 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f5c58287e0 0x55f5c5828c50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:05.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f985c291e00 4 client.17178 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:56:05.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.871+0000 7f985c291e00 1 -- 10.20.193.28:0/36405252 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f5c5500690 con 0x55f5c58287e0 2026-02-25T11:56:05.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.872+0000 7f9853fff640 1 --2- 10.20.193.28:0/36405252 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f5c58287e0 0x55f5c5828c50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:05.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.872+0000 7f9853fff640 1 --2- 10.20.193.28:0/36405252 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f5c58287e0 0x55f5c5828c50 crc :-1 s=READY pgs=149 gs=4 cs=0 l=0 c_cookie=15056331600723667424 s_cookie=16861288034029484119 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f2d5d1a99f69e0 server_cookie=e9ff5421cbdf3857 in_seq=0 out_seq=0 2026-02-25T11:56:05.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.872+0000 7f9815ffb640 10 client.17178 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:05.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.873+0000 7f9815ffb640 1 -- 10.20.193.28:0/36405252 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f5c5500690 con 0x55f5c58287e0 2026-02-25T11:56:05.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.873+0000 7f9815ffb640 10 client.17178 handle_command_reply: tid=0 2026-02-25T11:56:05.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.873+0000 7f985c291e00 1 client.17178 shutdown 2026-02-25T11:56:05.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.873+0000 7f985c291e00 1 -- 10.20.193.28:0/36405252 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c5793eb0 msgr2=0x55f5c5773a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:05.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.873+0000 7f985c291e00 1 --2- 10.20.193.28:0/36405252 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c5793eb0 0x55f5c5773a80 secure :-1 s=READY pgs=798 gs=1 cs=0 l=1 c_cookie=10675908589676172065 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f98440a94d0 tx=0x7f98440a8c70 comp rx=0 tx=0).stop 2026-02-25T11:56:05.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.873+0000 7f985c291e00 1 -- 10.20.193.28:0/36405252 shutdown_connections 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.873+0000 7f985c291e00 1 --2- 10.20.193.28:0/36405252 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f5c58287e0 0x55f5c5828c50 crc :-1 s=READY pgs=149 gs=4 cs=0 l=0 c_cookie=15056331600723667424 s_cookie=16861288034029484119 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.873+0000 7f985c291e00 1 --2- 10.20.193.28:0/36405252 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f5c57d89a0 0x55f5c58271e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.873+0000 7f985c291e00 1 --2- 10.20.193.28:0/36405252 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f5c5793eb0 0x55f5c5773a80 unknown :-1 s=CLOSED pgs=798 gs=1 cs=0 l=1 c_cookie=10675908589676172065 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.873+0000 7f985c291e00 1 --2- 10.20.193.28:0/36405252 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f5c57935a0 0x55f5c5773540 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.873+0000 7f9815ffb640 0 client.17178 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.873+0000 7f985c291e00 1 -- 10.20.193.28:0/36405252 >> 10.20.193.28:0/36405252 conn(0x55f5c57718f0 msgr2=0x55f5c57ecda0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.873+0000 7f985c291e00 1 -- 10.20.193.28:0/36405252 shutdown_connections 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985c291e00 1 -- 10.20.193.28:0/36405252 wait complete. 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985c291e00 20 client.17178 trim_cache size 0 max 16384 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8", 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985a296640 1 -- 10.20.193.28:0/875133219 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f98280b56e0 msgr2=0x7f98280d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:05.881 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985a296640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f98280b56e0 0x7f98280d5b90 secure :-1 s=READY pgs=363 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9840005b30 tx=0x7f984003a040 comp rx=0 tx=0).stop 2026-02-25T11:56:05.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985a296640 1 -- 10.20.193.28:0/875133219 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9854110230 msgr2=0x7f985411e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:05.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985a296640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9854110230 0x7f985411e740 secure :-1 s=READY pgs=795 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9844009480 tx=0x7f984400ee60 comp rx=0 tx=0).stop 2026-02-25T11:56:05.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985a296640 1 -- 10.20.193.28:0/875133219 shutdown_connections 2026-02-25T11:56:05.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985a296640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f98280b56e0 0x7f98280d5b90 unknown :-1 s=CLOSED pgs=363 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985a296640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9854115a30 0x7f9854123ac0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985a296640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9854110230 0x7f985411e740 unknown :-1 s=CLOSED pgs=795 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985a296640 1 --2- 10.20.193.28:0/875133219 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f985410f840 0x7f985411e200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:05.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985a296640 1 -- 10.20.193.28:0/875133219 >> 10.20.193.28:0/875133219 conn(0x7f98540ed130 msgr2=0x7f985410d510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:05.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985a296640 1 -- 10.20.193.28:0/875133219 shutdown_connections 2026-02-25T11:56:05.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:05.874+0000 7f985a296640 1 -- 10.20.193.28:0/875133219 wait complete. 2026-02-25T11:56:06.010 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:56:06.193 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:06.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.367+0000 7f33affff640 1 -- 10.20.193.28:0/2000368294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f33b010df00 msgr2=0x7f33b010e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:06.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.367+0000 7f33affff640 1 --2- 10.20.193.28:0/2000368294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f33b010df00 0x7f33b010e2d0 secure :-1 s=READY pgs=799 gs=1 cs=0 l=1 c_cookie=10917797120058166036 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f33a4009950 tx=0x7f33a402f190 comp rx=0 tx=0).stop 2026-02-25T11:56:06.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.367+0000 7f33affff640 1 -- 10.20.193.28:0/2000368294 shutdown_connections 2026-02-25T11:56:06.370 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.367+0000 7f33affff640 1 --2- 10.20.193.28:0/2000368294 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f33b010c070 0x7f33b0175f40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.371 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.367+0000 7f33affff640 1 --2- 10.20.193.28:0/2000368294 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f33b010b5f0 0x7f33b010ba20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.371 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.367+0000 7f33affff640 1 --2- 10.20.193.28:0/2000368294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f33b010df00 0x7f33b010e2d0 unknown :-1 s=CLOSED pgs=799 gs=1 cs=0 l=1 c_cookie=10917797120058166036 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.371 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.367+0000 7f33affff640 1 -- 10.20.193.28:0/2000368294 >> 10.20.193.28:0/2000368294 conn(0x7f33b00e9290 msgr2=0x7f33b01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:06.371 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.367+0000 7f33affff640 1 -- 10.20.193.28:0/2000368294 shutdown_connections 2026-02-25T11:56:06.371 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.367+0000 7f33affff640 1 -- 10.20.193.28:0/2000368294 wait complete. 2026-02-25T11:56:06.374 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33affff640 1 Processor -- start 2026-02-25T11:56:06.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33affff640 1 -- start start 2026-02-25T11:56:06.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33affff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f33b010b5f0 0x7f33b00f4cc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33affff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f33b010c070 0x7f33b00f5200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33affff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f33b010df00 0x7f33b00fa5d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33affff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f33b00fabf0 con 0x7f33b010b5f0 2026-02-25T11:56:06.375 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33affff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f33b00fad60 con 0x7f33b010c070 2026-02-25T11:56:06.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33affff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f33b00faf20 con 0x7f33b010df00 2026-02-25T11:56:06.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33aeffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f33b010b5f0 0x7f33b00f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33aeffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f33b010b5f0 0x7f33b00f4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:33880/0 (socket says 10.20.193.28:33880) 2026-02-25T11:56:06.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33aeffd640 1 -- 10.20.193.28:0/435879509 learned_addr learned my addr 10.20.193.28:0/435879509 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:06.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33af7fe640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f33b010df00 0x7f33b00fa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.376 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f33b010c070 0x7f33b00f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33aeffd640 1 -- 10.20.193.28:0/435879509 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f33b010df00 msgr2=0x7f33b00fa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:06.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33aeffd640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f33b010df00 0x7f33b00fa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33aeffd640 1 -- 10.20.193.28:0/435879509 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f33b010c070 msgr2=0x7f33b00f5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:06.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33aeffd640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f33b010c070 0x7f33b00f5200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33aeffd640 1 -- 10.20.193.28:0/435879509 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a4009570 con 0x7f33b010b5f0 2026-02-25T11:56:06.377 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33af7fe640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f33b010df00 0x7f33b00fa5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:56:06.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.368+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f33b010c070 0x7f33b00f5200 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:56:06.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.369+0000 7f33aeffd640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f33b010b5f0 0x7f33b00f4cc0 secure :-1 s=READY pgs=800 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f33a40023b0 tx=0x7f33a40028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:06.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.369+0000 7f33b49f2640 1 -- 10.20.193.28:0/435879509 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f33a403d070 con 0x7f33b010b5f0 2026-02-25T11:56:06.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.369+0000 7f33b49f2640 1 -- 10.20.193.28:0/435879509 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f33a4002a40 con 0x7f33b010b5f0 2026-02-25T11:56:06.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.369+0000 7f33b49f2640 1 -- 10.20.193.28:0/435879509 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f33a40418c0 con 0x7f33b010b5f0 2026-02-25T11:56:06.378 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.369+0000 7f33affff640 1 -- 10.20.193.28:0/435879509 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b00fb1a0 con 0x7f33b010b5f0 2026-02-25T11:56:06.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.369+0000 7f33affff640 1 -- 10.20.193.28:0/435879509 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b0172f10 con 0x7f33b010b5f0 2026-02-25T11:56:06.379 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.372+0000 7f33b49f2640 1 -- 10.20.193.28:0/435879509 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f33a4048020 con 0x7f33b010b5f0 2026-02-25T11:56:06.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.374+0000 7f33b49f2640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f33840b5760 0x7f33840d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.374+0000 7f33b49f2640 1 -- 10.20.193.28:0/435879509 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f33a410ae40 con 0x7f33b010b5f0 2026-02-25T11:56:06.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.374+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f33840b5760 0x7f33840d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.374+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f33840b5760 0x7f33840d5c10 secure :-1 s=READY pgs=364 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3398005db0 tx=0x7f339800a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:06.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33b79efe00 1 Processor -- start 2026-02-25T11:56:06.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33b79efe00 1 -- start start 2026-02-25T11:56:06.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33b79efe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 0x55790f3cff20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33b79efe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f3d0460 0x55790f414bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33b79efe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f415100 0x55790f4354e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33b79efe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55790f2dd9b0 con 0x55790f415100 2026-02-25T11:56:06.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33b79efe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55790f2d96e0 con 0x55790f3d0460 2026-02-25T11:56:06.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33b79efe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55790f2d3840 con 0x55790f3cfb50 2026-02-25T11:56:06.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33af7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f415100 0x55790f4354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33aeffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 0x55790f3cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33aeffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 0x55790f3cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44744/0 (socket says 10.20.193.28:44744) 2026-02-25T11:56:06.389 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33aeffd640 1 -- 10.20.193.28:0/792151595 learned_addr learned my addr 10.20.193.28:0/792151595 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/792151595 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f3d0460 0x55790f414bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33aeffd640 1 -- 10.20.193.28:0/792151595 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f3d0460 msgr2=0x55790f414bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33aeffd640 1 --2- 10.20.193.28:0/792151595 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f3d0460 0x55790f414bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33aeffd640 1 -- 10.20.193.28:0/792151595 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f415100 msgr2=0x55790f4354e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33aeffd640 1 --2- 10.20.193.28:0/792151595 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f415100 0x55790f4354e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33aeffd640 1 -- 10.20.193.28:0/792151595 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55790f27ed80 con 0x55790f3cfb50 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.375+0000 7f33af7fe640 1 --2- 10.20.193.28:0/792151595 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f415100 0x55790f4354e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.376+0000 7f33aeffd640 1 --2- 10.20.193.28:0/792151595 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 0x55790f3cff20 secure :-1 s=READY pgs=208 gs=1 cs=0 l=1 c_cookie=2450110926783612221 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f33a4032040 tx=0x7f33a402f190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22008ae81b3d853d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.376+0000 7f3373fff640 1 -- 10.20.193.28:0/792151595 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f33a40d09c0 con 0x55790f3cfb50 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.376+0000 7f3373fff640 1 -- 10.20.193.28:0/792151595 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f33a40d1bc0 con 0x55790f3cfb50 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.376+0000 7f3373fff640 1 -- 10.20.193.28:0/792151595 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f33a40dbe70 con 0x55790f3cfb50 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.376+0000 7f33b79efe00 1 -- 10.20.193.28:0/792151595 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 msgr2=0x55790f3cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.376+0000 7f33b79efe00 1 --2- 10.20.193.28:0/792151595 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 0x55790f3cff20 secure :-1 s=READY pgs=208 gs=1 cs=0 l=1 c_cookie=2450110926783612221 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f33a4032040 tx=0x7f33a402f190 comp rx=0 tx=0).stop 2026-02-25T11:56:06.390 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.376+0000 7f33b79efe00 1 -- 10.20.193.28:0/792151595 shutdown_connections 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.376+0000 7f33b79efe00 1 --2- 10.20.193.28:0/792151595 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f415100 0x55790f4354e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.376+0000 7f33b79efe00 1 --2- 10.20.193.28:0/792151595 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f3d0460 0x55790f414bc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.376+0000 7f33b79efe00 1 --2- 10.20.193.28:0/792151595 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 0x55790f3cff20 unknown :-1 s=CLOSED pgs=208 gs=1 cs=0 l=1 c_cookie=2450110926783612221 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.376+0000 7f33b79efe00 1 -- 10.20.193.28:0/792151595 >> 10.20.193.28:0/792151595 conn(0x55790f2e6d40 msgr2=0x55790f3c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.376+0000 7f33b79efe00 1 -- 10.20.193.28:0/792151595 shutdown_connections 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.376+0000 7f33b79efe00 1 -- 10.20.193.28:0/792151595 wait complete. 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.378+0000 7f33b79efe00 1 -- start start 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.378+0000 7f33b79efe00 1 Processor -- start 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.378+0000 7f33b79efe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f3cfb50 0x55790f42cea0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.378+0000 7f33b79efe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3d0460 0x55790f42d3e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.378+0000 7f33b79efe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f415100 0x55790f4323d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.378+0000 7f33b79efe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55790f432910 con 0x55790f3d0460 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.378+0000 7f33b79efe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55790f432a80 con 0x55790f3cfb50 2026-02-25T11:56:06.391 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.378+0000 7f33b79efe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55790f432c40 con 0x55790f415100 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33ae7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3d0460 0x55790f42d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33ae7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3d0460 0x55790f42d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:33890/0 (socket says 10.20.193.28:33890) 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33ae7fc640 1 -- 10.20.193.28:0/3936776991 learned_addr learned my addr 10.20.193.28:0/3936776991 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33af7fe640 1 --2- 10.20.193.28:0/3936776991 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f415100 0x55790f4323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33aeffd640 1 --2- 10.20.193.28:0/3936776991 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f3cfb50 0x55790f42cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33ae7fc640 1 -- 10.20.193.28:0/3936776991 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f415100 msgr2=0x55790f4323d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/3936776991 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f415100 0x55790f4323d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33ae7fc640 1 -- 10.20.193.28:0/3936776991 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f3cfb50 msgr2=0x55790f42cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/3936776991 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f3cfb50 0x55790f42cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33ae7fc640 1 -- 10.20.193.28:0/3936776991 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b0172f10 con 0x55790f3d0460 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/3936776991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3d0460 0x55790f42d3e0 secure :-1 s=READY pgs=801 gs=1 cs=0 l=1 c_cookie=1678580264362428799 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f339800ed80 tx=0x7f3398004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174b83db604dc97f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33717fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33717fa640 1 -- 10.20.193.28:0/3936776991 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f339800fb30 con 0x55790f3d0460 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33717fa640 1 -- 10.20.193.28:0/3936776991 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f339800fc90 con 0x55790f3d0460 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33717fa640 1 -- 10.20.193.28:0/3936776991 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f339801c8a0 con 0x55790f3d0460 2026-02-25T11:56:06.392 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.379+0000 7f33b79efe00 1 -- 10.20.193.28:0/3936776991 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a0005580 con 0x55790f3d0460 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.380+0000 7f33717fa640 1 -- 10.20.193.28:0/3936776991 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f339801ca30 con 0x55790f3d0460 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.380+0000 7f33b79efe00 10 client.17190 _ll_register_callbacks cb 0x55790f2549a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.380+0000 7f33b79efe00 1 -- 10.20.193.28:0/3936776991 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55790ef696f0 con 0x55790f3d0460 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.380+0000 7f33717fa640 1 -- 10.20.193.28:0/3936776991 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f339800fe00 con 0x55790f3d0460 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.380+0000 7f33b79efe00 10 client.17190 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.380+0000 7f33b79efe00 1 -- 10.20.193.28:0/3936776991 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55790f257350 con 0x55790f3d0460 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.380+0000 7f33717fa640 1 -- 10.20.193.28:0/3936776991 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f339802f020 con 0x55790f3d0460 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.380+0000 7f33b79efe00 10 client.17190 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.380+0000 7f33b79efe00 10 client.17190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.380+0000 7f33b79efe00 20 client.17190 populate_metadata read hostname 'trial028' 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.380+0000 7f33b79efe00 1 --2- 10.20.193.28:0/3936776991 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55790f4354b0 0x55790f412e30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.380+0000 7f33b79efe00 4 client.17190 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.380+0000 7f33b79efe00 1 -- 10.20.193.28:0/3936776991 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55790f2a5910 con 0x55790f4354b0 2026-02-25T11:56:06.393 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.381+0000 7f33aeffd640 1 --2- 10.20.193.28:0/3936776991 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55790f4354b0 0x55790f412e30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.381+0000 7f33aeffd640 1 --2- 10.20.193.28:0/3936776991 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55790f4354b0 0x55790f412e30 crc :-1 s=READY pgs=150 gs=4 cs=0 l=0 c_cookie=16207954901040975182 s_cookie=2209243434082066136 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0ee391bec194d4e server_cookie=1ea8cf32bed35ed8 in_seq=0 out_seq=0 2026-02-25T11:56:06.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.381+0000 7f33717fa640 10 client.17190 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:06.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33717fa640 1 -- 10.20.193.28:0/3936776991 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55790f2a5910 con 0x55790f4354b0 2026-02-25T11:56:06.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33717fa640 10 client.17190 handle_command_reply: tid=0 2026-02-25T11:56:06.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33b79efe00 1 client.17190 shutdown 2026-02-25T11:56:06.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33b79efe00 1 -- 10.20.193.28:0/3936776991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3d0460 msgr2=0x55790f42d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:06.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33b79efe00 1 --2- 10.20.193.28:0/3936776991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3d0460 0x55790f42d3e0 secure :-1 s=READY pgs=801 gs=1 cs=0 l=1 c_cookie=1678580264362428799 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f339800ed80 tx=0x7f3398004770 comp rx=0 tx=0).stop 2026-02-25T11:56:06.394 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33b79efe00 1 -- 10.20.193.28:0/3936776991 shutdown_connections 2026-02-25T11:56:06.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33b79efe00 1 --2- 10.20.193.28:0/3936776991 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55790f4354b0 0x55790f412e30 crc :-1 s=READY pgs=150 gs=4 cs=0 l=0 c_cookie=16207954901040975182 s_cookie=2209243434082066136 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33b79efe00 1 --2- 10.20.193.28:0/3936776991 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f415100 0x55790f4323d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33b79efe00 1 --2- 10.20.193.28:0/3936776991 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3d0460 0x55790f42d3e0 unknown :-1 s=CLOSED pgs=801 gs=1 cs=0 l=1 c_cookie=1678580264362428799 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33b79efe00 1 --2- 10.20.193.28:0/3936776991 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f3cfb50 0x55790f42cea0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33b79efe00 1 -- 10.20.193.28:0/3936776991 >> 10.20.193.28:0/3936776991 conn(0x55790f2e6d40 msgr2=0x55790f3c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:06.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33717fa640 0 client.17190 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:06.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33b79efe00 1 -- 10.20.193.28:0/3936776991 shutdown_connections 2026-02-25T11:56:06.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33b79efe00 1 -- 10.20.193.28:0/3936776991 wait complete. 2026-02-25T11:56:06.395 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.382+0000 7f33b79efe00 20 client.17190 trim_cache size 0 max 16384 2026-02-25T11:56:06.399 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.391+0000 7f33b79efe00 1 Processor -- start 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.391+0000 7f33b79efe00 1 -- start start 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.391+0000 7f33b79efe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3cfb50 0x55790f417bf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.391+0000 7f33b79efe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3d0460 0x55790f419fd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.391+0000 7f33b79efe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f415100 0x55790f414410 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.391+0000 7f33b79efe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55790f2d1470 con 0x55790f3cfb50 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.391+0000 7f33b79efe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55790ef91700 con 0x55790f415100 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.391+0000 7f33b79efe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55790f2dd9b0 con 0x55790f3d0460 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.392+0000 7f33ae7fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3d0460 0x55790f419fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.392+0000 7f33ae7fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3d0460 0x55790f419fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44750/0 (socket says 10.20.193.28:44750) 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.392+0000 7f33ae7fc640 1 -- 10.20.193.28:0/2305305472 learned_addr learned my addr 10.20.193.28:0/2305305472 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.392+0000 7f33af7fe640 1 --2- 10.20.193.28:0/2305305472 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f415100 0x55790f414410 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.392+0000 7f33aeffd640 1 --2- 10.20.193.28:0/2305305472 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3cfb50 0x55790f417bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.392+0000 7f33ae7fc640 1 -- 10.20.193.28:0/2305305472 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f415100 msgr2=0x55790f414410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:06.400 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.392+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/2305305472 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f415100 0x55790f414410 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.392+0000 7f33ae7fc640 1 -- 10.20.193.28:0/2305305472 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3cfb50 msgr2=0x55790f417bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.392+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/2305305472 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3cfb50 0x55790f417bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.392+0000 7f33ae7fc640 1 -- 10.20.193.28:0/2305305472 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55790f269150 con 0x55790f3d0460 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.392+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/2305305472 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3d0460 0x55790f419fd0 secure :-1 s=READY pgs=209 gs=2 cs=0 l=1 c_cookie=3281470539508952375 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3398013040 tx=0x7f339801f030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d8a2011d8572d37 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.393+0000 7f33737fe640 1 -- 10.20.193.28:0/2305305472 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f33980105d0 con 0x55790f3d0460 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.393+0000 7f33737fe640 1 -- 10.20.193.28:0/2305305472 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55790ef696f0 con 0x55790f3d0460 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.393+0000 7f33737fe640 1 -- 10.20.193.28:0/2305305472 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f3398026420 con 0x55790f3d0460 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.393+0000 7f33b79efe00 1 -- 10.20.193.28:0/2305305472 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3d0460 msgr2=0x55790f419fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.393+0000 7f33b79efe00 1 --2- 10.20.193.28:0/2305305472 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3d0460 0x55790f419fd0 secure :-1 s=READY pgs=209 gs=2 cs=0 l=1 c_cookie=3281470539508952375 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3398013040 tx=0x7f339801f030 comp rx=0 tx=0).stop 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.393+0000 7f33b79efe00 1 -- 10.20.193.28:0/2305305472 shutdown_connections 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.393+0000 7f33b79efe00 1 --2- 10.20.193.28:0/2305305472 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f415100 0x55790f414410 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.393+0000 7f33b79efe00 1 --2- 10.20.193.28:0/2305305472 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3d0460 0x55790f419fd0 unknown :-1 s=CLOSED pgs=209 gs=2 cs=0 l=1 c_cookie=3281470539508952375 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.393+0000 7f33b79efe00 1 --2- 10.20.193.28:0/2305305472 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3cfb50 0x55790f417bf0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.401 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.393+0000 7f33b79efe00 1 -- 10.20.193.28:0/2305305472 >> 10.20.193.28:0/2305305472 conn(0x55790f2e6d40 msgr2=0x55790f4174e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:06.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.393+0000 7f33b79efe00 1 -- 10.20.193.28:0/2305305472 shutdown_connections 2026-02-25T11:56:06.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.393+0000 7f33b79efe00 1 -- 10.20.193.28:0/2305305472 wait complete. 2026-02-25T11:56:06.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.394+0000 7f33b79efe00 1 -- start start 2026-02-25T11:56:06.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.394+0000 7f33b79efe00 1 Processor -- start 2026-02-25T11:56:06.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.394+0000 7f33b79efe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 0x55790f3a5e30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.394+0000 7f33b79efe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3d0460 0x55790f3a6370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.394+0000 7f33b79efe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f415100 0x55790f459d00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.394+0000 7f33b79efe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55790f45a240 con 0x55790f3d0460 2026-02-25T11:56:06.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.394+0000 7f33b79efe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55790f45a3b0 con 0x55790f415100 2026-02-25T11:56:06.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.394+0000 7f33b79efe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55790f45a570 con 0x55790f3cfb50 2026-02-25T11:56:06.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.394+0000 7f33aeffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 0x55790f3a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.394+0000 7f33aeffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 0x55790f3a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:44754/0 (socket says 10.20.193.28:44754) 2026-02-25T11:56:06.402 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.394+0000 7f33aeffd640 1 -- 10.20.193.28:0/3968887405 learned_addr learned my addr 10.20.193.28:0/3968887405 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.394+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/3968887405 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3d0460 0x55790f3a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33aeffd640 1 -- 10.20.193.28:0/3968887405 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f415100 msgr2=0x55790f459d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33aeffd640 1 --2- 10.20.193.28:0/3968887405 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f415100 0x55790f459d00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33aeffd640 1 -- 10.20.193.28:0/3968887405 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3d0460 msgr2=0x55790f3a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33aeffd640 1 --2- 10.20.193.28:0/3968887405 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3d0460 0x55790f3a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33aeffd640 1 -- 10.20.193.28:0/3968887405 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55790f263bd0 con 0x55790f3cfb50 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/3968887405 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3d0460 0x55790f3a6370 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33aeffd640 1 --2- 10.20.193.28:0/3968887405 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 0x55790f3a5e30 secure :-1 s=READY pgs=210 gs=1 cs=0 l=1 c_cookie=4896653395014264924 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f33a40098a0 tx=0x7f33a40d6fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43f46851b609285c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33717fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33717fa640 1 -- 10.20.193.28:0/3968887405 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f33a40dc300 con 0x55790f3cfb50 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33717fa640 1 -- 10.20.193.28:0/3968887405 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f33a40d1e50 con 0x55790f3cfb50 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33717fa640 1 -- 10.20.193.28:0/3968887405 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f33a40cf1a0 con 0x55790f3cfb50 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33b79efe00 1 -- 10.20.193.28:0/3968887405 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55790f257350 con 0x55790f3cfb50 2026-02-25T11:56:06.403 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33b79efe00 10 client.26884 _ll_register_callbacks cb 0x55790f2549a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33b79efe00 1 -- 10.20.193.28:0/3968887405 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33a40d1bc0 con 0x55790f3cfb50 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33717fa640 1 -- 10.20.193.28:0/3968887405 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f33a40cf300 con 0x55790f3cfb50 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33717fa640 1 -- 10.20.193.28:0/3968887405 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f33a40d2bb0 con 0x55790f3cfb50 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33b79efe00 10 client.26884 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.395+0000 7f33b79efe00 1 -- 10.20.193.28:0/3968887405 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55790f257680 con 0x55790f3cfb50 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.396+0000 7f33717fa640 1 -- 10.20.193.28:0/3968887405 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f33a40ef020 con 0x55790f3cfb50 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.396+0000 7f33b79efe00 10 client.26884 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.396+0000 7f33b79efe00 10 client.26884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.396+0000 7f33b79efe00 20 client.26884 populate_metadata read hostname 'trial028' 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.396+0000 7f33b79efe00 1 --2- 10.20.193.28:0/3968887405 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55790f4c2400 0x55790f4e2880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.396+0000 7f33b79efe00 4 client.26884 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.396+0000 7f33b79efe00 1 -- 10.20.193.28:0/3968887405 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55790ef66f40 con 0x55790f4c2400 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.396+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/3968887405 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55790f4c2400 0x55790f4e2880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33ae7fc640 1 --2- 10.20.193.28:0/3968887405 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55790f4c2400 0x55790f4e2880 crc :-1 s=READY pgs=151 gs=4 cs=0 l=0 c_cookie=15537389144223824414 s_cookie=7874920849531088060 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d79fe51b2678ba1e server_cookie=6d4956ca1e9e2cbc in_seq=0 out_seq=0 2026-02-25T11:56:06.404 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33717fa640 10 client.26884 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33717fa640 1 -- 10.20.193.28:0/3968887405 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5114 (crc 0 0 0) 0x55790ef66f40 con 0x55790f4c2400 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33717fa640 10 client.26884 handle_command_reply: tid=0 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33b79efe00 1 client.26884 shutdown 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33b79efe00 1 -- 10.20.193.28:0/3968887405 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 msgr2=0x55790f3a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33b79efe00 1 --2- 10.20.193.28:0/3968887405 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 0x55790f3a5e30 secure :-1 s=READY pgs=210 gs=1 cs=0 l=1 c_cookie=4896653395014264924 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f33a40098a0 tx=0x7f33a40d6fd0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33b79efe00 1 -- 10.20.193.28:0/3968887405 shutdown_connections 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33b79efe00 1 --2- 10.20.193.28:0/3968887405 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55790f4c2400 0x55790f4e2880 crc :-1 s=READY pgs=151 gs=4 cs=0 l=0 c_cookie=15537389144223824414 s_cookie=7874920849531088060 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33b79efe00 1 --2- 10.20.193.28:0/3968887405 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55790f415100 0x55790f459d00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33b79efe00 1 --2- 10.20.193.28:0/3968887405 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55790f3d0460 0x55790f3a6370 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33b79efe00 1 --2- 10.20.193.28:0/3968887405 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55790f3cfb50 0x55790f3a5e30 unknown :-1 s=CLOSED pgs=210 gs=1 cs=0 l=1 c_cookie=4896653395014264924 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33717fa640 0 client.26884 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33b79efe00 1 -- 10.20.193.28:0/3968887405 >> 10.20.193.28:0/3968887405 conn(0x55790f2e6d40 msgr2=0x55790f413400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33b79efe00 1 -- 10.20.193.28:0/3968887405 shutdown_connections 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33b79efe00 1 -- 10.20.193.28:0/3968887405 wait complete. 2026-02-25T11:56:06.405 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.397+0000 7f33b79efe00 20 client.26884 trim_cache size 0 max 16384 2026-02-25T11:56:06.406 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:56:06.406 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:56:06.406 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:56:06.406 INFO:teuthology.orchestra.run.trial028.stdout: "0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8": { 2026-02-25T11:56:06.406 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.406 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8", 2026-02-25T11:56:06.407 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:56:06.407 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:56:06.407 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.407 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:56:06.407 INFO:teuthology.orchestra.run.trial028.stdout: "0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8": { 2026-02-25T11:56:06.407 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:05.873000+0000", 2026-02-25T11:56:06.407 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.407 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.407 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.407 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.407 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.407 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.407 INFO:teuthology.orchestra.run.trial028.stdout: "850bcfd3-be8a-4930-896c-1ed6e48b8fed": { 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:27.124813+0000", 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.408 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:56:06.409 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:56:06.410 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:56:06.411 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:56:06.412 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:56:06.413 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "7b237934-6b95-43dd-b3a6-82b74134ec55": { 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:31.289725+0000", 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.414 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.398+0000 7f33affff640 1 -- 10.20.193.28:0/435879509 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f33840b5760 msgr2=0x7f33840d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:06.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.398+0000 7f33affff640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f33840b5760 0x7f33840d5c10 secure :-1 s=READY pgs=364 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f3398005db0 tx=0x7f339800a250 comp rx=0 tx=0).stop 2026-02-25T11:56:06.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.398+0000 7f33affff640 1 -- 10.20.193.28:0/435879509 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f33b010b5f0 msgr2=0x7f33b00f4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:06.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.398+0000 7f33affff640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f33b010b5f0 0x7f33b00f4cc0 secure :-1 s=READY pgs=800 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f33a40023b0 tx=0x7f33a40028d0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.398+0000 7f33affff640 1 -- 10.20.193.28:0/435879509 shutdown_connections 2026-02-25T11:56:06.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.398+0000 7f33affff640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f33840b5760 0x7f33840d5c10 unknown :-1 s=CLOSED pgs=364 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.398+0000 7f33affff640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f33b010df00 0x7f33b00fa5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.398+0000 7f33affff640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f33b010c070 0x7f33b00f5200 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.398+0000 7f33affff640 1 --2- 10.20.193.28:0/435879509 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f33b010b5f0 0x7f33b00f4cc0 unknown :-1 s=CLOSED pgs=800 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:06.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.398+0000 7f33affff640 1 -- 10.20.193.28:0/435879509 >> 10.20.193.28:0/435879509 conn(0x7f33b00e9290 msgr2=0x7f33b00eb120 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:06.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.398+0000 7f33affff640 1 -- 10.20.193.28:0/435879509 shutdown_connections 2026-02-25T11:56:06.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:06.398+0000 7f33affff640 1 -- 10.20.193.28:0/435879509 wait complete. 2026-02-25T11:56:06.499 INFO:tasks.cephfs.filesystem:scrub status for tag:0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8 - {'path': '/', 'tag': '0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8', 'options': 'recursive,force'} 2026-02-25T11:56:06.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:06 trial167 ceph-mon[43392]: pgmap v462: 97 pgs: 97 active+clean; 6.1 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:56:06.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:06 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:56:06.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:06 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:56:06.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:06 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:56:06.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:06 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:56:06.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:06 trial119 ceph-mon[42710]: pgmap v462: 97 pgs: 97 active+clean; 6.1 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:56:06.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:06 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:56:06.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:06 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:56:06.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:06 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:56:06.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:06 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:56:06.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:06 trial028 ceph-mon[39224]: pgmap v462: 97 pgs: 97 active+clean; 6.1 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:56:06.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:06 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:56:06.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:06 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:56:06.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:06 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:56:06.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:06 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:56:07.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:07 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:56:07.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:07 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:56:07.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:07 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:56:08.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:08 trial167 ceph-mon[43392]: pgmap v463: 97 pgs: 97 active+clean; 6.1 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:08.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:08 trial119 ceph-mon[42710]: pgmap v463: 97 pgs: 97 active+clean; 6.1 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:08.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:08 trial028 ceph-mon[39224]: pgmap v463: 97 pgs: 97 active+clean; 6.1 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:10.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:10 trial167 ceph-mon[43392]: pgmap v464: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:56:10.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:10 trial119 ceph-mon[42710]: pgmap v464: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:56:10.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:10 trial028 ceph-mon[39224]: pgmap v464: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:56:12.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:12 trial167 ceph-mon[43392]: pgmap v465: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:12.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:12 trial119 ceph-mon[42710]: pgmap v465: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:12.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:12 trial028 ceph-mon[39224]: pgmap v465: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:14.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:14 trial167 ceph-mon[43392]: pgmap v466: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:56:14.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:14 trial119 ceph-mon[42710]: pgmap v466: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:56:14.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:14 trial028 ceph-mon[39224]: pgmap v466: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:56:16.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:16 trial167 ceph-mon[43392]: pgmap v467: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:56:16.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:16 trial119 ceph-mon[42710]: pgmap v467: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:56:16.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:16 trial028 ceph-mon[39224]: pgmap v467: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:56:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:18 trial167 ceph-mon[43392]: pgmap v468: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:18.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:18 trial119 ceph-mon[42710]: pgmap v468: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:18.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:18 trial028 ceph-mon[39224]: pgmap v468: 97 pgs: 97 active+clean; 6.2 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:19.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:19 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:56:19.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:19 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:56:19.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:19 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:56:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:20 trial167 ceph-mon[43392]: pgmap v469: 97 pgs: 97 active+clean; 6.3 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:56:20.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:56:20.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-25T11:56:20.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-25T11:56:20.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2026-02-25T11:56:20.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:56:20.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:56:20.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:56:20.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:56:20.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:56:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:20 trial119 ceph-mon[42710]: pgmap v469: 97 pgs: 97 active+clean; 6.3 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:56:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:56:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-25T11:56:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-25T11:56:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2026-02-25T11:56:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:56:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:56:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:56:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:56:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:56:20.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:20 trial028 ceph-mon[39224]: pgmap v469: 97 pgs: 97 active+clean; 6.3 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:56:20.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2026-02-25T11:56:20.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2026-02-25T11:56:20.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2026-02-25T11:56:20.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2026-02-25T11:56:20.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:56:20.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:56:20.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:56:20.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:56:20.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:56:21.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:21 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial028 to 16917M 2026-02-25T11:56:21.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:21 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial028 to 16917M 2026-02-25T11:56:21.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:21 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial028 to 16917M 2026-02-25T11:56:22.650 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:22 trial167 ceph-mon[43392]: pgmap v470: 97 pgs: 97 active+clean; 6.3 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:22.661 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:22 trial119 ceph-mon[42710]: pgmap v470: 97 pgs: 97 active+clean; 6.3 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:22.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:22 trial028 ceph-mon[39224]: pgmap v470: 97 pgs: 97 active+clean; 6.3 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:24.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:24 trial167 ceph-mon[43392]: pgmap v471: 97 pgs: 97 active+clean; 6.3 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:56:24.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:24 trial119 ceph-mon[42710]: pgmap v471: 97 pgs: 97 active+clean; 6.3 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:56:24.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:24 trial028 ceph-mon[39224]: pgmap v471: 97 pgs: 97 active+clean; 6.3 MiB data, 347 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:56:26.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:26 trial167 ceph-mon[43392]: pgmap v472: 97 pgs: 97 active+clean; 6.3 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:56:26.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:26 trial119 ceph-mon[42710]: pgmap v472: 97 pgs: 97 active+clean; 6.3 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:56:26.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:26 trial028 ceph-mon[39224]: pgmap v472: 97 pgs: 97 active+clean; 6.3 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:56:28.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:28 trial167 ceph-mon[43392]: pgmap v473: 97 pgs: 97 active+clean; 6.3 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:28.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:28 trial119 ceph-mon[42710]: pgmap v473: 97 pgs: 97 active+clean; 6.3 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:28.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:28 trial028 ceph-mon[39224]: pgmap v473: 97 pgs: 97 active+clean; 6.3 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:30.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:30 trial167 ceph-mon[43392]: pgmap v474: 97 pgs: 97 active+clean; 6.4 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:56:30.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:30 trial119 ceph-mon[42710]: pgmap v474: 97 pgs: 97 active+clean; 6.4 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:56:30.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:30 trial028 ceph-mon[39224]: pgmap v474: 97 pgs: 97 active+clean; 6.4 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:56:32.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:32 trial167 ceph-mon[43392]: pgmap v475: 97 pgs: 97 active+clean; 6.4 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:32.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:32 trial119 ceph-mon[42710]: pgmap v475: 97 pgs: 97 active+clean; 6.4 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:32.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:32 trial028 ceph-mon[39224]: pgmap v475: 97 pgs: 97 active+clean; 6.4 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:34.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:34 trial167 ceph-mon[43392]: pgmap v476: 97 pgs: 97 active+clean; 6.4 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:56:34.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:34 trial119 ceph-mon[42710]: pgmap v476: 97 pgs: 97 active+clean; 6.4 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:56:34.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:34 trial028 ceph-mon[39224]: pgmap v476: 97 pgs: 97 active+clean; 6.4 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:56:36.500 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:56:36.690 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:36.734 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:36 trial028 ceph-mon[39224]: pgmap v477: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:56:36.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:36 trial167 ceph-mon[43392]: pgmap v477: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:56:36.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:36 trial119 ceph-mon[42710]: pgmap v477: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:56:36.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.873+0000 7f2628dc0640 1 -- 10.20.193.28:0/3123081422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f262410ddc0 msgr2=0x7f2624112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:36.875 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.873+0000 7f2628dc0640 1 --2- 10.20.193.28:0/3123081422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f262410ddc0 0x7f2624112e10 secure :-1 s=READY pgs=802 gs=1 cs=0 l=1 c_cookie=17705236526341132273 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2614009950 tx=0x7f261402f190 comp rx=0 tx=0).stop 2026-02-25T11:56:36.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.873+0000 7f2628dc0640 1 -- 10.20.193.28:0/3123081422 shutdown_connections 2026-02-25T11:56:36.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.873+0000 7f2628dc0640 1 --2- 10.20.193.28:0/3123081422 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f26241135c0 0x7f2624173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.873+0000 7f2628dc0640 1 --2- 10.20.193.28:0/3123081422 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f262410ddc0 0x7f2624112e10 unknown :-1 s=CLOSED pgs=802 gs=1 cs=0 l=1 c_cookie=17705236526341132273 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.873+0000 7f2628dc0640 1 --2- 10.20.193.28:0/3123081422 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f262410d3d0 0x7f262410d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.873+0000 7f2628dc0640 1 -- 10.20.193.28:0/3123081422 >> 10.20.193.28:0/3123081422 conn(0x7f26240eaca0 msgr2=0x7f262410b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:36.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.873+0000 7f2628dc0640 1 -- 10.20.193.28:0/3123081422 shutdown_connections 2026-02-25T11:56:36.876 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.873+0000 7f2628dc0640 1 -- 10.20.193.28:0/3123081422 wait complete. 2026-02-25T11:56:36.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.874+0000 7f2628dc0640 1 Processor -- start 2026-02-25T11:56:36.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.874+0000 7f2628dc0640 1 -- start start 2026-02-25T11:56:36.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.874+0000 7f2628dc0640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f262410d3d0 0x7f26240f5990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.874+0000 7f2628dc0640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f262410ddc0 0x7f2624173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.874+0000 7f2628dc0640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f26241135c0 0x7f26240f6090 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.874+0000 7f2628dc0640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f26240fb2a0 con 0x7f262410d3d0 2026-02-25T11:56:36.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.874+0000 7f2628dc0640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f26240fb410 con 0x7f262410ddc0 2026-02-25T11:56:36.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.874+0000 7f2628dc0640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f26240fb5d0 con 0x7f26241135c0 2026-02-25T11:56:36.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.874+0000 7f26237fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f262410d3d0 0x7f26240f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.878 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.874+0000 7f26237fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f262410d3d0 0x7f26240f5990 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52556/0 (socket says 10.20.193.28:52556) 2026-02-25T11:56:36.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.874+0000 7f26237fe640 1 -- 10.20.193.28:0/1747852711 learned_addr learned my addr 10.20.193.28:0/1747852711 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:36.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.875+0000 7f2622ffd640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f262410ddc0 0x7f2624173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.875+0000 7f2623fff640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f26241135c0 0x7f26240f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.875+0000 7f26237fe640 1 -- 10.20.193.28:0/1747852711 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f26241135c0 msgr2=0x7f26240f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:36.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.875+0000 7f26237fe640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f26241135c0 0x7f26240f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.875+0000 7f26237fe640 1 -- 10.20.193.28:0/1747852711 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f262410ddc0 msgr2=0x7f2624173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:36.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.875+0000 7f26237fe640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f262410ddc0 0x7f2624173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.875+0000 7f26237fe640 1 -- 10.20.193.28:0/1747852711 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2614009570 con 0x7f262410d3d0 2026-02-25T11:56:36.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.875+0000 7f26237fe640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f262410d3d0 0x7f26240f5990 secure :-1 s=READY pgs=803 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f261000d950 tx=0x7f261000de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:36.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.875+0000 7f2620ff9640 1 -- 10.20.193.28:0/1747852711 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2610004540 con 0x7f262410d3d0 2026-02-25T11:56:36.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.875+0000 7f2620ff9640 1 -- 10.20.193.28:0/1747852711 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2610009ce0 con 0x7f262410d3d0 2026-02-25T11:56:36.879 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.875+0000 7f2620ff9640 1 -- 10.20.193.28:0/1747852711 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f26100106e0 con 0x7f262410d3d0 2026-02-25T11:56:36.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.875+0000 7f2628dc0640 1 -- 10.20.193.28:0/1747852711 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26240fb8b0 con 0x7f262410d3d0 2026-02-25T11:56:36.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.875+0000 7f2628dc0640 1 -- 10.20.193.28:0/1747852711 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26240fbd30 con 0x7f262410d3d0 2026-02-25T11:56:36.880 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.878+0000 7f2620ff9640 1 -- 10.20.193.28:0/1747852711 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f26100097c0 con 0x7f262410d3d0 2026-02-25T11:56:36.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.880+0000 7f2620ff9640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f25f80b5750 0x7f25f80d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.880+0000 7f2620ff9640 1 -- 10.20.193.28:0/1747852711 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f26100e67d0 con 0x7f262410d3d0 2026-02-25T11:56:36.882 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.880+0000 7f2622ffd640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f25f80b5750 0x7f25f80d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.880+0000 7f262c044e00 1 Processor -- start 2026-02-25T11:56:36.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f262c044e00 1 -- start start 2026-02-25T11:56:36.883 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f2622ffd640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f25f80b5750 0x7f25f80d5c00 secure :-1 s=READY pgs=365 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26240fc330 tx=0x7f261403a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:36.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f262c044e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a628140 0x55839a628510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f262c044e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628a50 0x55839a66d1b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f262c044e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a66d6f0 0x55839a68dad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f262c044e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55839a53e9b0 con 0x55839a628140 2026-02-25T11:56:36.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f262c044e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55839a53a6e0 con 0x55839a66d6f0 2026-02-25T11:56:36.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f262c044e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55839a534840 con 0x55839a628a50 2026-02-25T11:56:36.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f26237fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a628140 0x55839a628510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f26237fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a628140 0x55839a628510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52570/0 (socket says 10.20.193.28:52570) 2026-02-25T11:56:36.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f26237fe640 1 -- 10.20.193.28:0/4282987309 learned_addr learned my addr 10.20.193.28:0/4282987309 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:36.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f2622ffd640 1 --2- 10.20.193.28:0/4282987309 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628a50 0x55839a66d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f2623fff640 1 --2- 10.20.193.28:0/4282987309 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a66d6f0 0x55839a68dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f26237fe640 1 -- 10.20.193.28:0/4282987309 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628a50 msgr2=0x55839a66d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:36.884 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f26237fe640 1 --2- 10.20.193.28:0/4282987309 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628a50 0x55839a66d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f26237fe640 1 -- 10.20.193.28:0/4282987309 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a66d6f0 msgr2=0x55839a68dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:36.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f26237fe640 1 --2- 10.20.193.28:0/4282987309 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a66d6f0 0x55839a68dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.881+0000 7f26237fe640 1 -- 10.20.193.28:0/4282987309 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55839a4dfd80 con 0x55839a628140 2026-02-25T11:56:36.885 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.882+0000 7f26237fe640 1 --2- 10.20.193.28:0/4282987309 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a628140 0x55839a628510 secure :-1 s=READY pgs=804 gs=1 cs=0 l=1 c_cookie=12246481242173539089 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26100a9120 tx=0x7f26100a8ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f43dce1c3d0311 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:36.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.882+0000 7f25e7fff640 1 -- 10.20.193.28:0/4282987309 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2610009e50 con 0x55839a628140 2026-02-25T11:56:36.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.882+0000 7f25e7fff640 1 -- 10.20.193.28:0/4282987309 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f26100ad550 con 0x55839a628140 2026-02-25T11:56:36.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.882+0000 7f25e7fff640 1 -- 10.20.193.28:0/4282987309 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f26100960c0 con 0x55839a628140 2026-02-25T11:56:36.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.882+0000 7f262c044e00 1 -- 10.20.193.28:0/4282987309 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a628140 msgr2=0x55839a628510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:36.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.882+0000 7f262c044e00 1 --2- 10.20.193.28:0/4282987309 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a628140 0x55839a628510 secure :-1 s=READY pgs=804 gs=1 cs=0 l=1 c_cookie=12246481242173539089 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26100a9120 tx=0x7f26100a8ac0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.882+0000 7f262c044e00 1 -- 10.20.193.28:0/4282987309 shutdown_connections 2026-02-25T11:56:36.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.882+0000 7f262c044e00 1 --2- 10.20.193.28:0/4282987309 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a66d6f0 0x55839a68dad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.882+0000 7f262c044e00 1 --2- 10.20.193.28:0/4282987309 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628a50 0x55839a66d1b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.887 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.882+0000 7f262c044e00 1 --2- 10.20.193.28:0/4282987309 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a628140 0x55839a628510 unknown :-1 s=CLOSED pgs=804 gs=1 cs=0 l=1 c_cookie=12246481242173539089 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.882+0000 7f262c044e00 1 -- 10.20.193.28:0/4282987309 >> 10.20.193.28:0/4282987309 conn(0x55839a547d40 msgr2=0x55839a626a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.882+0000 7f262c044e00 1 -- 10.20.193.28:0/4282987309 shutdown_connections 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.882+0000 7f262c044e00 1 -- 10.20.193.28:0/4282987309 wait complete. 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.883+0000 7f262c044e00 1 -- start start 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.884+0000 7f262c044e00 1 Processor -- start 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.884+0000 7f262c044e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628140 0x55839a685410 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.884+0000 7f262c044e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628a50 0x55839a685950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.884+0000 7f262c044e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 0x55839a68a940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.884+0000 7f262c044e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55839a68ae80 con 0x55839a66d6f0 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.884+0000 7f262c044e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55839a68aff0 con 0x55839a628a50 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.884+0000 7f262c044e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55839a68b1b0 con 0x55839a628140 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.884+0000 7f26237fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628140 0x55839a685410 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.884+0000 7f26237fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628140 0x55839a685410 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34658/0 (socket says 10.20.193.28:34658) 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.884+0000 7f26237fe640 1 -- 10.20.193.28:0/2663102001 learned_addr learned my addr 10.20.193.28:0/2663102001 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.884+0000 7f2622ffd640 1 --2- 10.20.193.28:0/2663102001 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628a50 0x55839a685950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.888 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.884+0000 7f2623fff640 1 --2- 10.20.193.28:0/2663102001 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 0x55839a68a940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.885+0000 7f2622ffd640 1 -- 10.20.193.28:0/2663102001 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628140 msgr2=0x55839a685410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:36.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.885+0000 7f2622ffd640 1 --2- 10.20.193.28:0/2663102001 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628140 0x55839a685410 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.885+0000 7f2622ffd640 1 -- 10.20.193.28:0/2663102001 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 msgr2=0x55839a68a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:36.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.885+0000 7f2622ffd640 1 --2- 10.20.193.28:0/2663102001 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 0x55839a68a940 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.885+0000 7f2622ffd640 1 -- 10.20.193.28:0/2663102001 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55839a4b8350 con 0x55839a628a50 2026-02-25T11:56:36.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.885+0000 7f2622ffd640 1 --2- 10.20.193.28:0/2663102001 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628a50 0x55839a685950 secure :-1 s=READY pgs=168 gs=3 cs=0 l=1 c_cookie=8268992405401823172 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2614004820 tx=0x7f2614009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72c15cc81e60e7c4 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:36.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.885+0000 7f25e57fa640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:56:36.889 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.885+0000 7f25e57fa640 1 -- 10.20.193.28:0/2663102001 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2614044630 con 0x55839a628a50 2026-02-25T11:56:36.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.885+0000 7f262c044e00 1 -- 10.20.193.28:0/2663102001 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26240fbd30 con 0x55839a628a50 2026-02-25T11:56:36.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.885+0000 7f25e57fa640 1 -- 10.20.193.28:0/2663102001 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f261403e040 con 0x55839a628a50 2026-02-25T11:56:36.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.885+0000 7f25e57fa640 1 -- 10.20.193.28:0/2663102001 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2614050670 con 0x55839a628a50 2026-02-25T11:56:36.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.885+0000 7f262c044e00 10 client.26891 _ll_register_callbacks cb 0x55839a4b59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:36.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.885+0000 7f262c044e00 1 -- 10.20.193.28:0/2663102001 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55839a1ca6f0 con 0x55839a628a50 2026-02-25T11:56:36.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.886+0000 7f25e57fa640 1 -- 10.20.193.28:0/2663102001 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f2614044790 con 0x55839a628a50 2026-02-25T11:56:36.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.886+0000 7f25e57fa640 1 -- 10.20.193.28:0/2663102001 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2614044950 con 0x55839a628a50 2026-02-25T11:56:36.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.886+0000 7f262c044e00 10 client.26891 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:36.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.886+0000 7f262c044e00 1 -- 10.20.193.28:0/2663102001 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55839a4ca150 con 0x55839a628a50 2026-02-25T11:56:36.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.886+0000 7f25e57fa640 1 -- 10.20.193.28:0/2663102001 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f261404f070 con 0x55839a628a50 2026-02-25T11:56:36.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.886+0000 7f262c044e00 10 client.26891 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:36.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.886+0000 7f262c044e00 10 client.26891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:36.891 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.886+0000 7f262c044e00 20 client.26891 populate_metadata read hostname 'trial028' 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.886+0000 7f262c044e00 1 --2- 10.20.193.28:0/2663102001 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55839a68d930 0x55839a66b420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.886+0000 7f262c044e00 4 client.26891 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.886+0000 7f262c044e00 1 -- 10.20.193.28:0/2663102001 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55839a506910 con 0x55839a68d930 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.886+0000 7f26237fe640 1 --2- 10.20.193.28:0/2663102001 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55839a68d930 0x55839a66b420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.887+0000 7f26237fe640 1 --2- 10.20.193.28:0/2663102001 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55839a68d930 0x55839a66b420 crc :-1 s=READY pgs=152 gs=4 cs=0 l=0 c_cookie=197012831893044058 s_cookie=13276414728493798013 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bbee20752dbb5a server_cookie=b83f4cdaa6a3127d in_seq=0 out_seq=0 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.887+0000 7f25e57fa640 10 client.26891 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f25e57fa640 1 -- 10.20.193.28:0/2663102001 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55839a506910 con 0x55839a68d930 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f25e57fa640 10 client.26891 handle_command_reply: tid=0 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f262c044e00 1 client.26891 shutdown 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f262c044e00 1 -- 10.20.193.28:0/2663102001 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628a50 msgr2=0x55839a685950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f262c044e00 1 --2- 10.20.193.28:0/2663102001 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628a50 0x55839a685950 secure :-1 s=READY pgs=168 gs=3 cs=0 l=1 c_cookie=8268992405401823172 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2614004820 tx=0x7f2614009310 comp rx=0 tx=0).stop 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f262c044e00 1 -- 10.20.193.28:0/2663102001 shutdown_connections 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f262c044e00 1 --2- 10.20.193.28:0/2663102001 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55839a68d930 0x55839a66b420 crc :-1 s=READY pgs=152 gs=4 cs=0 l=0 c_cookie=197012831893044058 s_cookie=13276414728493798013 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f262c044e00 1 --2- 10.20.193.28:0/2663102001 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 0x55839a68a940 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.892 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f262c044e00 1 --2- 10.20.193.28:0/2663102001 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628a50 0x55839a685950 unknown :-1 s=CLOSED pgs=168 gs=3 cs=0 l=1 c_cookie=8268992405401823172 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f262c044e00 1 --2- 10.20.193.28:0/2663102001 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628140 0x55839a685410 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f25e57fa640 0 client.26891 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:36.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f262c044e00 1 -- 10.20.193.28:0/2663102001 >> 10.20.193.28:0/2663102001 conn(0x55839a547d40 msgr2=0x55839a626230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:36.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f262c044e00 1 -- 10.20.193.28:0/2663102001 shutdown_connections 2026-02-25T11:56:36.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f262c044e00 1 -- 10.20.193.28:0/2663102001 wait complete. 2026-02-25T11:56:36.893 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.888+0000 7f262c044e00 20 client.26891 trim_cache size 0 max 16384 2026-02-25T11:56:36.899 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.897+0000 7f262c044e00 1 Processor -- start 2026-02-25T11:56:36.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.897+0000 7f262c044e00 1 -- start start 2026-02-25T11:56:36.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.897+0000 7f262c044e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628140 0x55839a6701e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.897+0000 7f262c044e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628a50 0x55839a6ad960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.897+0000 7f262c044e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 0x55839a6adea0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f262c044e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55839a532470 con 0x55839a66d6f0 2026-02-25T11:56:36.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f262c044e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55839a1f2700 con 0x55839a628a50 2026-02-25T11:56:36.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f262c044e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55839a599820 con 0x55839a628140 2026-02-25T11:56:36.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f26237fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628140 0x55839a6701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f26237fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628140 0x55839a6701e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34668/0 (socket says 10.20.193.28:34668) 2026-02-25T11:56:36.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f26237fe640 1 -- 10.20.193.28:0/4182846058 learned_addr learned my addr 10.20.193.28:0/4182846058 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:36.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f2622ffd640 1 --2- 10.20.193.28:0/4182846058 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628a50 0x55839a6ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.900 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f2623fff640 1 --2- 10.20.193.28:0/4182846058 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 0x55839a6adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f26237fe640 1 -- 10.20.193.28:0/4182846058 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628a50 msgr2=0x55839a6ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:36.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f26237fe640 1 --2- 10.20.193.28:0/4182846058 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628a50 0x55839a6ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f26237fe640 1 -- 10.20.193.28:0/4182846058 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 msgr2=0x55839a6adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:36.901 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f26237fe640 1 --2- 10.20.193.28:0/4182846058 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 0x55839a6adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f26237fe640 1 -- 10.20.193.28:0/4182846058 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55839a677680 con 0x55839a628140 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f26237fe640 1 --2- 10.20.193.28:0/4182846058 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628140 0x55839a6701e0 secure :-1 s=READY pgs=211 gs=1 cs=0 l=1 c_cookie=6699497349304334695 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26100e4320 tx=0x7f2610002390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cf96557fef4b967 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.898+0000 7f25e77fe640 1 -- 10.20.193.28:0/4182846058 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f26100b4e50 con 0x55839a628140 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.899+0000 7f25e77fe640 1 -- 10.20.193.28:0/4182846058 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f26100b7c90 con 0x55839a628140 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.899+0000 7f25e77fe640 1 -- 10.20.193.28:0/4182846058 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f26100960b0 con 0x55839a628140 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.899+0000 7f262c044e00 1 -- 10.20.193.28:0/4182846058 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628140 msgr2=0x55839a6701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.899+0000 7f262c044e00 1 --2- 10.20.193.28:0/4182846058 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628140 0x55839a6701e0 secure :-1 s=READY pgs=211 gs=1 cs=0 l=1 c_cookie=6699497349304334695 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26100e4320 tx=0x7f2610002390 comp rx=0 tx=0).stop 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.899+0000 7f262c044e00 1 -- 10.20.193.28:0/4182846058 shutdown_connections 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.899+0000 7f262c044e00 1 --2- 10.20.193.28:0/4182846058 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 0x55839a6adea0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.899+0000 7f262c044e00 1 --2- 10.20.193.28:0/4182846058 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628a50 0x55839a6ad960 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.899+0000 7f262c044e00 1 --2- 10.20.193.28:0/4182846058 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628140 0x55839a6701e0 unknown :-1 s=CLOSED pgs=211 gs=1 cs=0 l=1 c_cookie=6699497349304334695 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.899+0000 7f262c044e00 1 -- 10.20.193.28:0/4182846058 >> 10.20.193.28:0/4182846058 conn(0x55839a547d40 msgr2=0x55839a66fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.899+0000 7f262c044e00 1 -- 10.20.193.28:0/4182846058 shutdown_connections 2026-02-25T11:56:36.902 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.899+0000 7f262c044e00 1 -- 10.20.193.28:0/4182846058 wait complete. 2026-02-25T11:56:36.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.900+0000 7f262c044e00 1 -- start start 2026-02-25T11:56:36.903 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.900+0000 7f262c044e00 1 Processor -- start 2026-02-25T11:56:36.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.900+0000 7f262c044e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628140 0x55839a693bd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.900+0000 7f262c044e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628a50 0x55839a694150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.900+0000 7f262c044e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 0x55839a627120 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.900+0000 7f262c044e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55839a603490 con 0x55839a66d6f0 2026-02-25T11:56:36.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.900+0000 7f262c044e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55839a68b970 con 0x55839a628140 2026-02-25T11:56:36.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.900+0000 7f262c044e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55839a628f30 con 0x55839a628a50 2026-02-25T11:56:36.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.900+0000 7f2622ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628a50 0x55839a694150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.900+0000 7f2623fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 0x55839a627120 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.900+0000 7f2622ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628a50 0x55839a694150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34682/0 (socket says 10.20.193.28:34682) 2026-02-25T11:56:36.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.900+0000 7f2622ffd640 1 -- 10.20.193.28:0/124720239 learned_addr learned my addr 10.20.193.28:0/124720239 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:36.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.900+0000 7f26237fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628140 0x55839a693bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f2623fff640 1 -- 10.20.193.28:0/124720239 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628a50 msgr2=0x55839a694150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f2623fff640 1 --2- 10.20.193.28:0/124720239 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628a50 0x55839a694150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f2623fff640 1 -- 10.20.193.28:0/124720239 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628140 msgr2=0x55839a693bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f2623fff640 1 --2- 10.20.193.28:0/124720239 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628140 0x55839a693bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f2623fff640 1 -- 10.20.193.28:0/124720239 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55839a4c4bd0 con 0x55839a66d6f0 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f2623fff640 1 --2- 10.20.193.28:0/124720239 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 0x55839a627120 secure :-1 s=READY pgs=805 gs=3 cs=0 l=1 c_cookie=6833510536669184874 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2618005820 tx=0x7f2618014b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed581a0ab0e436a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f25e57fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f25e57fa640 1 -- 10.20.193.28:0/124720239 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2618012ae0 con 0x55839a66d6f0 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f25e57fa640 1 -- 10.20.193.28:0/124720239 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f261801a040 con 0x55839a66d6f0 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f25e57fa640 1 -- 10.20.193.28:0/124720239 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f261800f660 con 0x55839a66d6f0 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f262c044e00 1 -- 10.20.193.28:0/124720239 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26100b2040 con 0x55839a66d6f0 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f262c044e00 10 client.17214 _ll_register_callbacks cb 0x55839a4b59a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f262c044e00 1 -- 10.20.193.28:0/124720239 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26100ad550 con 0x55839a66d6f0 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.901+0000 7f25e57fa640 1 -- 10.20.193.28:0/124720239 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f2618012c40 con 0x55839a66d6f0 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.902+0000 7f25e57fa640 1 -- 10.20.193.28:0/124720239 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f261801e3f0 con 0x55839a66d6f0 2026-02-25T11:56:36.906 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.902+0000 7f262c044e00 10 client.17214 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.902+0000 7f262c044e00 1 -- 10.20.193.28:0/124720239 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55839a4b8680 con 0x55839a66d6f0 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.902+0000 7f25e57fa640 1 -- 10.20.193.28:0/124720239 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f261801e590 con 0x55839a66d6f0 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.902+0000 7f262c044e00 10 client.17214 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.902+0000 7f262c044e00 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.902+0000 7f262c044e00 20 client.17214 populate_metadata read hostname 'trial028' 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.902+0000 7f262c044e00 1 --2- 10.20.193.28:0/124720239 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55839a723b10 0x55839a743f90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.902+0000 7f262c044e00 4 client.17214 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.902+0000 7f262c044e00 1 -- 10.20.193.28:0/124720239 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55839a1c7f40 con 0x55839a723b10 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.902+0000 7f26237fe640 1 --2- 10.20.193.28:0/124720239 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55839a723b10 0x55839a743f90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f26237fe640 1 --2- 10.20.193.28:0/124720239 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55839a723b10 0x55839a743f90 crc :-1 s=READY pgs=153 gs=4 cs=0 l=0 c_cookie=12151458161083278741 s_cookie=6048011017313012981 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a2a6d0fa63e595 server_cookie=53eed9f9fc974cf5 in_seq=0 out_seq=0 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f25e57fa640 10 client.17214 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f25e57fa640 1 -- 10.20.193.28:0/124720239 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x55839a1c7f40 con 0x55839a723b10 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f25e57fa640 10 client.17214 handle_command_reply: tid=0 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f262c044e00 1 client.17214 shutdown 2026-02-25T11:56:36.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f262c044e00 1 -- 10.20.193.28:0/124720239 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 msgr2=0x55839a627120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:36.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f262c044e00 1 --2- 10.20.193.28:0/124720239 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 0x55839a627120 secure :-1 s=READY pgs=805 gs=3 cs=0 l=1 c_cookie=6833510536669184874 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2618005820 tx=0x7f2618014b80 comp rx=0 tx=0).stop 2026-02-25T11:56:36.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f262c044e00 1 -- 10.20.193.28:0/124720239 shutdown_connections 2026-02-25T11:56:36.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f262c044e00 1 --2- 10.20.193.28:0/124720239 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55839a723b10 0x55839a743f90 crc :-1 s=READY pgs=153 gs=4 cs=0 l=0 c_cookie=12151458161083278741 s_cookie=6048011017313012981 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f262c044e00 1 --2- 10.20.193.28:0/124720239 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55839a66d6f0 0x55839a627120 unknown :-1 s=CLOSED pgs=805 gs=3 cs=0 l=1 c_cookie=6833510536669184874 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f262c044e00 1 --2- 10.20.193.28:0/124720239 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55839a628a50 0x55839a694150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f262c044e00 1 --2- 10.20.193.28:0/124720239 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55839a628140 0x55839a693bd0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f262c044e00 1 -- 10.20.193.28:0/124720239 >> 10.20.193.28:0/124720239 conn(0x55839a547d40 msgr2=0x55839a6ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:36.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f25e57fa640 0 client.17214 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:36.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f262c044e00 1 -- 10.20.193.28:0/124720239 shutdown_connections 2026-02-25T11:56:36.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f262c044e00 1 -- 10.20.193.28:0/124720239 wait complete. 2026-02-25T11:56:36.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.903+0000 7f262c044e00 20 client.17214 trim_cache size 0 max 16384 2026-02-25T11:56:36.909 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:56:36.909 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:56:36.909 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:56:36.909 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:56:36.909 INFO:teuthology.orchestra.run.trial028.stdout: "0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8": { 2026-02-25T11:56:36.909 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:05.873000+0000", 2026-02-25T11:56:36.909 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.909 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "850bcfd3-be8a-4930-896c-1ed6e48b8fed": { 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:27.124813+0000", 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.910 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.911 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.912 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.913 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.914 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.915 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "7b237934-6b95-43dd-b3a6-82b74134ec55": { 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:31.289725+0000", 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:56:36.916 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.904+0000 7f2628dc0640 1 -- 10.20.193.28:0/1747852711 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f25f80b5750 msgr2=0x7f25f80d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.904+0000 7f2628dc0640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f25f80b5750 0x7f25f80d5c00 secure :-1 s=READY pgs=365 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f26240fc330 tx=0x7f261403a040 comp rx=0 tx=0).stop 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.904+0000 7f2628dc0640 1 -- 10.20.193.28:0/1747852711 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f262410d3d0 msgr2=0x7f26240f5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.904+0000 7f2628dc0640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f262410d3d0 0x7f26240f5990 secure :-1 s=READY pgs=803 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f261000d950 tx=0x7f261000de20 comp rx=0 tx=0).stop 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.904+0000 7f2628dc0640 1 -- 10.20.193.28:0/1747852711 shutdown_connections 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.904+0000 7f2628dc0640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f25f80b5750 0x7f25f80d5c00 unknown :-1 s=CLOSED pgs=365 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.904+0000 7f2628dc0640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f26241135c0 0x7f26240f6090 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.904+0000 7f2628dc0640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f262410ddc0 0x7f2624173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.904+0000 7f2628dc0640 1 --2- 10.20.193.28:0/1747852711 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f262410d3d0 0x7f26240f5990 unknown :-1 s=CLOSED pgs=803 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:36.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.904+0000 7f2628dc0640 1 -- 10.20.193.28:0/1747852711 >> 10.20.193.28:0/1747852711 conn(0x7f26240eaca0 msgr2=0x7f262410b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:36.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.904+0000 7f2628dc0640 1 -- 10.20.193.28:0/1747852711 shutdown_connections 2026-02-25T11:56:36.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:36.904+0000 7f2628dc0640 1 -- 10.20.193.28:0/1747852711 wait complete. 2026-02-25T11:56:37.042 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:56:37.043 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:56:37.227 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:37.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.407+0000 7f5ca95fa640 1 -- 10.20.193.28:0/4019502061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ca410c990 msgr2=0x7f5ca410cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:37.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.407+0000 7f5ca95fa640 1 --2- 10.20.193.28:0/4019502061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ca410c990 0x7f5ca410cd60 secure :-1 s=READY pgs=806 gs=1 cs=0 l=1 c_cookie=16339021535246585990 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5c90009950 tx=0x7f5c9002f190 comp rx=0 tx=0).stop 2026-02-25T11:56:37.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.407+0000 7f5ca95fa640 1 -- 10.20.193.28:0/4019502061 shutdown_connections 2026-02-25T11:56:37.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.407+0000 7f5ca95fa640 1 --2- 10.20.193.28:0/4019502061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ca4112470 0x7f5ca4172e40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:37.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.407+0000 7f5ca95fa640 1 --2- 10.20.193.28:0/4019502061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ca410d380 0x7f5ca4111cc0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:37.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.407+0000 7f5ca95fa640 1 --2- 10.20.193.28:0/4019502061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ca410c990 0x7f5ca410cd60 unknown :-1 s=CLOSED pgs=806 gs=1 cs=0 l=1 c_cookie=16339021535246585990 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:37.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.407+0000 7f5ca95fa640 1 -- 10.20.193.28:0/4019502061 >> 10.20.193.28:0/4019502061 conn(0x7f5ca40e9290 msgr2=0x7f5ca41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:37.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.407+0000 7f5ca95fa640 1 -- 10.20.193.28:0/4019502061 shutdown_connections 2026-02-25T11:56:37.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.407+0000 7f5ca95fa640 1 -- 10.20.193.28:0/4019502061 wait complete. 2026-02-25T11:56:37.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.408+0000 7f5ca95fa640 1 Processor -- start 2026-02-25T11:56:37.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.408+0000 7f5ca95fa640 1 -- start start 2026-02-25T11:56:37.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.408+0000 7f5ca95fa640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ca410c990 0x7f5ca41737d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:37.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.408+0000 7f5ca95fa640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ca410d380 0x7f5ca4172d40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:37.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.408+0000 7f5ca95fa640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ca4112470 0x7f5ca4173d10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:37.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.408+0000 7f5ca95fa640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca4174250 con 0x7f5ca410c990 2026-02-25T11:56:37.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.408+0000 7f5ca95fa640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca41743c0 con 0x7f5ca4112470 2026-02-25T11:56:37.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.408+0000 7f5ca95fa640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca4174580 con 0x7f5ca410d380 2026-02-25T11:56:37.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.408+0000 7f5ca3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ca410c990 0x7f5ca41737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:37.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.408+0000 7f5ca3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ca410c990 0x7f5ca41737d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52614/0 (socket says 10.20.193.28:52614) 2026-02-25T11:56:37.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca3fff640 1 -- 10.20.193.28:0/1189376437 learned_addr learned my addr 10.20.193.28:0/1189376437 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:37.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca37fe640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ca410d380 0x7f5ca4172d40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:37.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca8df9640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ca4112470 0x7f5ca4173d10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:37.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca3fff640 1 -- 10.20.193.28:0/1189376437 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ca410d380 msgr2=0x7f5ca4172d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:37.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca3fff640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ca410d380 0x7f5ca4172d40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:37.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca3fff640 1 -- 10.20.193.28:0/1189376437 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ca4112470 msgr2=0x7f5ca4173d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:37.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca3fff640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ca4112470 0x7f5ca4173d10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:37.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca3fff640 1 -- 10.20.193.28:0/1189376437 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c90009570 con 0x7f5ca410c990 2026-02-25T11:56:37.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca3fff640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ca410c990 0x7f5ca41737d0 secure :-1 s=READY pgs=807 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5c900098a0 tx=0x7f5c900028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:37.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca17fa640 1 -- 10.20.193.28:0/1189376437 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5c9003d070 con 0x7f5ca410c990 2026-02-25T11:56:37.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca17fa640 1 -- 10.20.193.28:0/1189376437 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5c90002a70 con 0x7f5ca410c990 2026-02-25T11:56:37.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca95fa640 1 -- 10.20.193.28:0/1189376437 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca40f9f80 con 0x7f5ca410c990 2026-02-25T11:56:37.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca17fa640 1 -- 10.20.193.28:0/1189376437 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5c90041890 con 0x7f5ca410c990 2026-02-25T11:56:37.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.409+0000 7f5ca95fa640 1 -- 10.20.193.28:0/1189376437 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca40fa440 con 0x7f5ca410c990 2026-02-25T11:56:37.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.410+0000 7f5ca95fa640 1 -- 10.20.193.28:0/1189376437 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca405fb90 con 0x7f5ca410c990 2026-02-25T11:56:37.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.412+0000 7f5ca17fa640 1 -- 10.20.193.28:0/1189376437 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f5c90041a20 con 0x7f5ca410c990 2026-02-25T11:56:37.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.414+0000 7f5ca17fa640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5c780b5750 0x7f5c780d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:37.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.414+0000 7f5ca17fa640 1 -- 10.20.193.28:0/1189376437 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f5c9010b370 con 0x7f5ca410c990 2026-02-25T11:56:37.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.414+0000 7f5ca37fe640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5c780b5750 0x7f5c780d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:37.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.414+0000 7f5ca17fa640 1 -- 10.20.193.28:0/1189376437 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f5c9010b7f0 con 0x7f5ca410c990 2026-02-25T11:56:37.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.414+0000 7f5ca37fe640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5c780b5750 0x7f5c780d5c00 secure :-1 s=READY pgs=366 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ca4110d80 tx=0x7f5c9400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:37.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.556+0000 7f5ca95fa640 1 -- 10.20.193.28:0/1189376437 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ca4174ad0 con 0x7f5ca410c990 2026-02-25T11:56:37.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.557+0000 7f5ca17fa640 1 -- 10.20.193.28:0/1189376437 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f5c900cddf0 con 0x7f5ca410c990 2026-02-25T11:56:37.559 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:56:37.560 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:56:37.561 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:56:37.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.558+0000 7f5ca95fa640 1 -- 10.20.193.28:0/1189376437 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5c780b5750 msgr2=0x7f5c780d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:37.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.558+0000 7f5ca95fa640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5c780b5750 0x7f5c780d5c00 secure :-1 s=READY pgs=366 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5ca4110d80 tx=0x7f5c9400b410 comp rx=0 tx=0).stop 2026-02-25T11:56:37.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.558+0000 7f5ca95fa640 1 -- 10.20.193.28:0/1189376437 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ca410c990 msgr2=0x7f5ca41737d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:37.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.558+0000 7f5ca95fa640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ca410c990 0x7f5ca41737d0 secure :-1 s=READY pgs=807 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5c900098a0 tx=0x7f5c900028d0 comp rx=0 tx=0).stop 2026-02-25T11:56:37.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.558+0000 7f5ca95fa640 1 -- 10.20.193.28:0/1189376437 shutdown_connections 2026-02-25T11:56:37.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.558+0000 7f5ca95fa640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5c780b5750 0x7f5c780d5c00 unknown :-1 s=CLOSED pgs=366 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:37.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.558+0000 7f5ca95fa640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5ca4112470 0x7f5ca4173d10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:37.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.558+0000 7f5ca95fa640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5ca410d380 0x7f5ca4172d40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:37.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.558+0000 7f5ca95fa640 1 --2- 10.20.193.28:0/1189376437 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5ca410c990 0x7f5ca41737d0 unknown :-1 s=CLOSED pgs=807 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:37.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.558+0000 7f5ca95fa640 1 -- 10.20.193.28:0/1189376437 >> 10.20.193.28:0/1189376437 conn(0x7f5ca40e9290 msgr2=0x7f5ca410e680 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:37.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.559+0000 7f5ca95fa640 1 -- 10.20.193.28:0/1189376437 shutdown_connections 2026-02-25T11:56:37.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:37.559+0000 7f5ca95fa640 1 -- 10.20.193.28:0/1189376437 wait complete. 2026-02-25T11:56:37.658 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:56:37.844 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:38.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.016+0000 7f95224aa640 1 -- 10.20.193.28:0/276512718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f951c10eb80 msgr2=0x7f951c153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.016+0000 7f95224aa640 1 --2- 10.20.193.28:0/276512718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f951c10eb80 0x7f951c153200 secure :-1 s=READY pgs=212 gs=3 cs=0 l=1 c_cookie=5528526753544675669 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9510009890 tx=0x7f951002f120 comp rx=0 tx=0).stop 2026-02-25T11:56:38.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.016+0000 7f95224aa640 1 -- 10.20.193.28:0/276512718 shutdown_connections 2026-02-25T11:56:38.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.016+0000 7f95224aa640 1 --2- 10.20.193.28:0/276512718 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f951c153740 0x7f951c173b60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.016+0000 7f95224aa640 1 --2- 10.20.193.28:0/276512718 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f951c10eb80 0x7f951c153200 unknown :-1 s=CLOSED pgs=212 gs=3 cs=0 l=1 c_cookie=5528526753544675669 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.016+0000 7f95224aa640 1 --2- 10.20.193.28:0/276512718 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f951c10e190 0x7f951c10e560 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.016+0000 7f95224aa640 1 -- 10.20.193.28:0/276512718 >> 10.20.193.28:0/276512718 conn(0x7f951c0e9b40 msgr2=0x7f951c109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:38.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.016+0000 7f95224aa640 1 -- 10.20.193.28:0/276512718 shutdown_connections 2026-02-25T11:56:38.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.016+0000 7f95224aa640 1 -- 10.20.193.28:0/276512718 wait complete. 2026-02-25T11:56:38.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.017+0000 7f95224aa640 1 Processor -- start 2026-02-25T11:56:38.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.017+0000 7f95224aa640 1 -- start start 2026-02-25T11:56:38.020 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.017+0000 7f95224aa640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f951c10e190 0x7f951c162210 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.017+0000 7f95224aa640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f951c10eb80 0x7f951c162750 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.017+0000 7f95224aa640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f951c153740 0x7f951c167b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.017+0000 7f95224aa640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f951c168140 con 0x7f951c153740 2026-02-25T11:56:38.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.017+0000 7f95224aa640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f951c1682b0 con 0x7f951c10e190 2026-02-25T11:56:38.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.017+0000 7f95224aa640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f951c168470 con 0x7f951c10eb80 2026-02-25T11:56:38.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.017+0000 7f9520ca7640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f951c10eb80 0x7f951c162750 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.017+0000 7f9520ca7640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f951c10eb80 0x7f951c162750 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34710/0 (socket says 10.20.193.28:34710) 2026-02-25T11:56:38.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.017+0000 7f9520ca7640 1 -- 10.20.193.28:0/3430364079 learned_addr learned my addr 10.20.193.28:0/3430364079 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:38.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.017+0000 7f9521ca9640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f951c153740 0x7f951c167b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.021 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.017+0000 7f95214a8640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f951c10e190 0x7f951c162210 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.022 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.018+0000 7f9520ca7640 1 -- 10.20.193.28:0/3430364079 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f951c10e190 msgr2=0x7f951c162210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.018+0000 7f9520ca7640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f951c10e190 0x7f951c162210 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.018+0000 7f9520ca7640 1 -- 10.20.193.28:0/3430364079 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f951c153740 msgr2=0x7f951c167b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.018+0000 7f9520ca7640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f951c153740 0x7f951c167b20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.018+0000 7f9520ca7640 1 -- 10.20.193.28:0/3430364079 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95100094b0 con 0x7f951c10eb80 2026-02-25T11:56:38.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.018+0000 7f9520ca7640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f951c10eb80 0x7f951c162750 secure :-1 s=READY pgs=213 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9510004660 tx=0x7f9510004690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:38.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.018+0000 7f950a7fc640 1 -- 10.20.193.28:0/3430364079 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f951003d070 con 0x7f951c10eb80 2026-02-25T11:56:38.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.018+0000 7f95224aa640 1 -- 10.20.193.28:0/3430364079 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951c1686f0 con 0x7f951c10eb80 2026-02-25T11:56:38.023 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.018+0000 7f950a7fc640 1 -- 10.20.193.28:0/3430364079 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9510041410 con 0x7f951c10eb80 2026-02-25T11:56:38.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.018+0000 7f950a7fc640 1 -- 10.20.193.28:0/3430364079 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9510037650 con 0x7f951c10eb80 2026-02-25T11:56:38.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.018+0000 7f95224aa640 1 -- 10.20.193.28:0/3430364079 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f951c14f360 con 0x7f951c10eb80 2026-02-25T11:56:38.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.021+0000 7f950a7fc640 1 -- 10.20.193.28:0/3430364079 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f9510041960 con 0x7f951c10eb80 2026-02-25T11:56:38.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.023+0000 7f950a7fc640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f94f80b5760 0x7f94f80d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.023+0000 7f950a7fc640 1 -- 10.20.193.28:0/3430364079 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f951010aef0 con 0x7f951c10eb80 2026-02-25T11:56:38.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.023+0000 7f95214a8640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f94f80b5760 0x7f94f80d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f95214a8640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f94f80b5760 0x7f94f80d5c10 secure :-1 s=READY pgs=367 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f950c004740 tx=0x7f950c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:38.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f952572ee00 1 Processor -- start 2026-02-25T11:56:38.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f952572ee00 1 -- start start 2026-02-25T11:56:38.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f952572ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a2b30 0x5592de8a2f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f952572ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a3440 0x5592de8e7ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f952572ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de9084c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f952572ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5592de7b88a0 con 0x5592de8e80e0 2026-02-25T11:56:38.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f952572ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5592de7b45d0 con 0x5592de8a2b30 2026-02-25T11:56:38.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f952572ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5592de7ae570 con 0x5592de8a3440 2026-02-25T11:56:38.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f9521ca9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de9084c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f9521ca9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de9084c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52628/0 (socket says 10.20.193.28:52628) 2026-02-25T11:56:38.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f9521ca9640 1 -- 10.20.193.28:0/1034942038 learned_addr learned my addr 10.20.193.28:0/1034942038 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:38.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f95214a8640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a2b30 0x5592de8a2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.024+0000 7f9520ca7640 1 --2- 10.20.193.28:0/1034942038 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a3440 0x5592de8e7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f9521ca9640 1 -- 10.20.193.28:0/1034942038 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a3440 msgr2=0x5592de8e7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f9521ca9640 1 --2- 10.20.193.28:0/1034942038 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a3440 0x5592de8e7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f9521ca9640 1 -- 10.20.193.28:0/1034942038 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a2b30 msgr2=0x5592de8a2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f9521ca9640 1 --2- 10.20.193.28:0/1034942038 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a2b30 0x5592de8a2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f9521ca9640 1 -- 10.20.193.28:0/1034942038 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592de759d10 con 0x5592de8e80e0 2026-02-25T11:56:38.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f9521ca9640 1 --2- 10.20.193.28:0/1034942038 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de9084c0 secure :-1 s=READY pgs=808 gs=3 cs=0 l=1 c_cookie=7703145447258263867 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f95180055b0 tx=0x7f9518003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae711f8a4ff113b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:38.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f94e9ce5640 1 -- 10.20.193.28:0/1034942038 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9518003e30 con 0x5592de8e80e0 2026-02-25T11:56:38.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f94e9ce5640 1 -- 10.20.193.28:0/1034942038 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f951800ace0 con 0x5592de8e80e0 2026-02-25T11:56:38.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f94e9ce5640 1 -- 10.20.193.28:0/1034942038 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f95180117b0 con 0x5592de8e80e0 2026-02-25T11:56:38.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f952572ee00 1 -- 10.20.193.28:0/1034942038 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 msgr2=0x5592de9084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f952572ee00 1 --2- 10.20.193.28:0/1034942038 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de9084c0 secure :-1 s=READY pgs=808 gs=3 cs=0 l=1 c_cookie=7703145447258263867 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f95180055b0 tx=0x7f9518003c30 comp rx=0 tx=0).stop 2026-02-25T11:56:38.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f952572ee00 1 -- 10.20.193.28:0/1034942038 shutdown_connections 2026-02-25T11:56:38.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f952572ee00 1 --2- 10.20.193.28:0/1034942038 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de9084c0 unknown :-1 s=CLOSED pgs=808 gs=3 cs=0 l=1 c_cookie=7703145447258263867 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f952572ee00 1 --2- 10.20.193.28:0/1034942038 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a3440 0x5592de8e7ba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f952572ee00 1 --2- 10.20.193.28:0/1034942038 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a2b30 0x5592de8a2f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f952572ee00 1 -- 10.20.193.28:0/1034942038 >> 10.20.193.28:0/1034942038 conn(0x5592de7c17a0 msgr2=0x5592de8a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:38.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.025+0000 7f952572ee00 1 -- 10.20.193.28:0/1034942038 shutdown_connections 2026-02-25T11:56:38.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.026+0000 7f952572ee00 1 -- 10.20.193.28:0/1034942038 wait complete. 2026-02-25T11:56:38.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.027+0000 7f952572ee00 1 -- start start 2026-02-25T11:56:38.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.027+0000 7f952572ee00 1 Processor -- start 2026-02-25T11:56:38.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.027+0000 7f952572ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a2b30 0x5592de8baf40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.027+0000 7f952572ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a3440 0x5592de8bb480 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.027+0000 7f952572ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8c0470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.027+0000 7f952572ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5592de8c09b0 con 0x5592de8e80e0 2026-02-25T11:56:38.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.027+0000 7f952572ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5592de8c0b20 con 0x5592de8a2b30 2026-02-25T11:56:38.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.027+0000 7f952572ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5592de8c0ce0 con 0x5592de8a3440 2026-02-25T11:56:38.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f9521ca9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8c0470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f9521ca9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8c0470 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52642/0 (socket says 10.20.193.28:52642) 2026-02-25T11:56:38.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f95214a8640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a2b30 0x5592de8baf40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f9521ca9640 1 -- 10.20.193.28:0/3216285856 learned_addr learned my addr 10.20.193.28:0/3216285856 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:38.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f9520ca7640 1 --2- 10.20.193.28:0/3216285856 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a3440 0x5592de8bb480 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f9521ca9640 1 -- 10.20.193.28:0/3216285856 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a3440 msgr2=0x5592de8bb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f9521ca9640 1 --2- 10.20.193.28:0/3216285856 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a3440 0x5592de8bb480 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f9521ca9640 1 -- 10.20.193.28:0/3216285856 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a2b30 msgr2=0x5592de8baf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f9521ca9640 1 --2- 10.20.193.28:0/3216285856 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a2b30 0x5592de8baf40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f9521ca9640 1 -- 10.20.193.28:0/3216285856 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592de7323c0 con 0x5592de8e80e0 2026-02-25T11:56:38.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f9520ca7640 1 --2- 10.20.193.28:0/3216285856 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a3440 0x5592de8bb480 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:56:38.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f94d6ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:38.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f9521ca9640 1 --2- 10.20.193.28:0/3216285856 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8c0470 secure :-1 s=READY pgs=809 gs=2 cs=0 l=1 c_cookie=4692767932453584794 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f951800aad0 tx=0x7f9518012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41200f923b3d7f9a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:38.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f94d6ffd640 1 -- 10.20.193.28:0/3216285856 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f951801bd30 con 0x5592de8e80e0 2026-02-25T11:56:38.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f952572ee00 1 -- 10.20.193.28:0/3216285856 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9518005250 con 0x5592de8e80e0 2026-02-25T11:56:38.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f952572ee00 10 client.17232 _ll_register_callbacks cb 0x5592de72fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:38.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f952572ee00 1 -- 10.20.193.28:0/3216285856 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5592de4446f0 con 0x5592de8e80e0 2026-02-25T11:56:38.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f94d6ffd640 1 -- 10.20.193.28:0/3216285856 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f951801bd30 con 0x5592de8e80e0 2026-02-25T11:56:38.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.028+0000 7f94d6ffd640 1 -- 10.20.193.28:0/3216285856 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9518005c90 con 0x5592de8e80e0 2026-02-25T11:56:38.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.029+0000 7f94d6ffd640 1 -- 10.20.193.28:0/3216285856 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9518005e20 con 0x5592de8e80e0 2026-02-25T11:56:38.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.029+0000 7f94d6ffd640 1 -- 10.20.193.28:0/3216285856 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9518006070 con 0x5592de8e80e0 2026-02-25T11:56:38.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.029+0000 7f952572ee00 10 client.17232 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:38.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.029+0000 7f952572ee00 1 -- 10.20.193.28:0/3216285856 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5592de7440e0 con 0x5592de8e80e0 2026-02-25T11:56:38.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.029+0000 7f94d6ffd640 1 -- 10.20.193.28:0/3216285856 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f951801b440 con 0x5592de8e80e0 2026-02-25T11:56:38.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.029+0000 7f952572ee00 10 client.17232 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:38.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.029+0000 7f952572ee00 10 client.17232 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:56:38.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.029+0000 7f952572ee00 20 client.17232 populate_metadata read hostname 'trial028' 2026-02-25T11:56:38.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.029+0000 7f952572ee00 1 --2- 10.20.193.28:0/3216285856 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5592de8e5610 0x5592de8e5ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.029+0000 7f952572ee00 4 client.17232 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:56:38.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.029+0000 7f952572ee00 1 -- 10.20.193.28:0/3216285856 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592de780fc0 con 0x5592de8e5610 2026-02-25T11:56:38.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.029+0000 7f95214a8640 1 --2- 10.20.193.28:0/3216285856 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5592de8e5610 0x5592de8e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.030+0000 7f95214a8640 1 --2- 10.20.193.28:0/3216285856 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5592de8e5610 0x5592de8e5ab0 crc :-1 s=READY pgs=55 gs=4 cs=0 l=0 c_cookie=7358437625303483635 s_cookie=9681477366097048621 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=661e6c08961544f3 server_cookie=865b845eab32f42d in_seq=0 out_seq=0 2026-02-25T11:56:38.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.030+0000 7f94d6ffd640 10 client.17232 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:56:38.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.030+0000 7f94d6ffd640 1 -- 10.20.193.28:0/3216285856 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5592de780fc0 con 0x5592de8e5610 2026-02-25T11:56:38.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.030+0000 7f94d6ffd640 10 client.17232 handle_command_reply: tid=0 2026-02-25T11:56:38.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.030+0000 7f952572ee00 1 client.17232 shutdown 2026-02-25T11:56:38.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.031+0000 7f952572ee00 1 -- 10.20.193.28:0/3216285856 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 msgr2=0x5592de8c0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.031+0000 7f952572ee00 1 --2- 10.20.193.28:0/3216285856 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8c0470 secure :-1 s=READY pgs=809 gs=2 cs=0 l=1 c_cookie=4692767932453584794 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f951800aad0 tx=0x7f9518012b80 comp rx=0 tx=0).stop 2026-02-25T11:56:38.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.031+0000 7f952572ee00 1 -- 10.20.193.28:0/3216285856 shutdown_connections 2026-02-25T11:56:38.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.031+0000 7f952572ee00 1 --2- 10.20.193.28:0/3216285856 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5592de8e5610 0x5592de8e5ab0 crc :-1 s=READY pgs=55 gs=4 cs=0 l=0 c_cookie=7358437625303483635 s_cookie=9681477366097048621 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.031+0000 7f952572ee00 1 --2- 10.20.193.28:0/3216285856 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8c0470 unknown :-1 s=CLOSED pgs=809 gs=2 cs=0 l=1 c_cookie=4692767932453584794 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.031+0000 7f952572ee00 1 --2- 10.20.193.28:0/3216285856 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a3440 0x5592de8bb480 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.031+0000 7f952572ee00 1 --2- 10.20.193.28:0/3216285856 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a2b30 0x5592de8baf40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.031+0000 7f94d6ffd640 0 client.17232 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:56:38.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.031+0000 7f952572ee00 1 -- 10.20.193.28:0/3216285856 >> 10.20.193.28:0/3216285856 conn(0x5592de7c17a0 msgr2=0x5592de8a1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:38.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.031+0000 7f952572ee00 1 -- 10.20.193.28:0/3216285856 shutdown_connections 2026-02-25T11:56:38.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.031+0000 7f952572ee00 1 -- 10.20.193.28:0/3216285856 wait complete. 2026-02-25T11:56:38.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.031+0000 7f952572ee00 20 client.17232 trim_cache size 0 max 16384 2026-02-25T11:56:38.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.038+0000 7f952572ee00 1 Processor -- start 2026-02-25T11:56:38.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.038+0000 7f952572ee00 1 -- start start 2026-02-25T11:56:38.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.038+0000 7f952572ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a2b30 0x5592de8b8540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.038+0000 7f952572ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a3440 0x5592de8e73a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.038+0000 7f952572ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8e78e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.038+0000 7f952572ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5592de7ac1a0 con 0x5592de8e80e0 2026-02-25T11:56:38.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.038+0000 7f952572ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5592de46c700 con 0x5592de8a3440 2026-02-25T11:56:38.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.038+0000 7f952572ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5592de8c0580 con 0x5592de8a2b30 2026-02-25T11:56:38.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f95214a8640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a2b30 0x5592de8b8540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f95214a8640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a2b30 0x5592de8b8540 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34734/0 (socket says 10.20.193.28:34734) 2026-02-25T11:56:38.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f95214a8640 1 -- 10.20.193.28:0/3160638421 learned_addr learned my addr 10.20.193.28:0/3160638421 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:38.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f9520ca7640 1 --2- 10.20.193.28:0/3160638421 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a3440 0x5592de8e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f9521ca9640 1 --2- 10.20.193.28:0/3160638421 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f95214a8640 1 -- 10.20.193.28:0/3160638421 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a3440 msgr2=0x5592de8e73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f95214a8640 1 --2- 10.20.193.28:0/3160638421 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a3440 0x5592de8e73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f95214a8640 1 -- 10.20.193.28:0/3160638421 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 msgr2=0x5592de8e78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f95214a8640 1 --2- 10.20.193.28:0/3160638421 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8e78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f95214a8640 1 -- 10.20.193.28:0/3160638421 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592de73eb60 con 0x5592de8a2b30 2026-02-25T11:56:38.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f95214a8640 1 --2- 10.20.193.28:0/3160638421 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a2b30 0x5592de8b8540 secure :-1 s=READY pgs=214 gs=1 cs=0 l=1 c_cookie=14007292346189522495 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x5592de8e6360 tx=0x7f950c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c263e62ccfaca63f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:38.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f94e94e4640 1 -- 10.20.193.28:0/3160638421 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f950c021040 con 0x5592de8a2b30 2026-02-25T11:56:38.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f94e94e4640 1 -- 10.20.193.28:0/3160638421 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f950c022040 con 0x5592de8a2b30 2026-02-25T11:56:38.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.039+0000 7f94e94e4640 1 -- 10.20.193.28:0/3160638421 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f950c003640 con 0x5592de8a2b30 2026-02-25T11:56:38.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.040+0000 7f952572ee00 1 -- 10.20.193.28:0/3160638421 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a2b30 msgr2=0x5592de8b8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.040+0000 7f952572ee00 1 --2- 10.20.193.28:0/3160638421 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a2b30 0x5592de8b8540 secure :-1 s=READY pgs=214 gs=1 cs=0 l=1 c_cookie=14007292346189522495 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x5592de8e6360 tx=0x7f950c014000 comp rx=0 tx=0).stop 2026-02-25T11:56:38.044 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.040+0000 7f952572ee00 1 -- 10.20.193.28:0/3160638421 shutdown_connections 2026-02-25T11:56:38.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.040+0000 7f952572ee00 1 --2- 10.20.193.28:0/3160638421 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8e78e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.040+0000 7f952572ee00 1 --2- 10.20.193.28:0/3160638421 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a3440 0x5592de8e73a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.040+0000 7f952572ee00 1 --2- 10.20.193.28:0/3160638421 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a2b30 0x5592de8b8540 unknown :-1 s=CLOSED pgs=214 gs=1 cs=0 l=1 c_cookie=14007292346189522495 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.040+0000 7f952572ee00 1 -- 10.20.193.28:0/3160638421 >> 10.20.193.28:0/3160638421 conn(0x5592de7c17a0 msgr2=0x5592de8a0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:38.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.040+0000 7f952572ee00 1 -- 10.20.193.28:0/3160638421 shutdown_connections 2026-02-25T11:56:38.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.040+0000 7f952572ee00 1 -- 10.20.193.28:0/3160638421 wait complete. 2026-02-25T11:56:38.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.041+0000 7f952572ee00 1 -- start start 2026-02-25T11:56:38.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.041+0000 7f952572ee00 1 Processor -- start 2026-02-25T11:56:38.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.041+0000 7f952572ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a2b30 0x5592de8c29a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.041+0000 7f952572ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a3440 0x5592de8c2ee0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.041+0000 7f952572ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8c3420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.045 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.041+0000 7f952572ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5592de94d600 con 0x5592de8e80e0 2026-02-25T11:56:38.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.041+0000 7f952572ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5592de94d770 con 0x5592de8a3440 2026-02-25T11:56:38.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.041+0000 7f952572ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5592de94d930 con 0x5592de8a2b30 2026-02-25T11:56:38.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.041+0000 7f9521ca9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8c3420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.041+0000 7f95214a8640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a2b30 0x5592de8c29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.041+0000 7f9521ca9640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8c3420 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52658/0 (socket says 10.20.193.28:52658) 2026-02-25T11:56:38.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.041+0000 7f9521ca9640 1 -- 10.20.193.28:0/327424299 learned_addr learned my addr 10.20.193.28:0/327424299 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:38.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f9520ca7640 1 --2- 10.20.193.28:0/327424299 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a3440 0x5592de8c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f9521ca9640 1 -- 10.20.193.28:0/327424299 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a2b30 msgr2=0x5592de8c29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f9521ca9640 1 --2- 10.20.193.28:0/327424299 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a2b30 0x5592de8c29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f9521ca9640 1 -- 10.20.193.28:0/327424299 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a3440 msgr2=0x5592de8c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f9521ca9640 1 --2- 10.20.193.28:0/327424299 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a3440 0x5592de8c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f9521ca9640 1 -- 10.20.193.28:0/327424299 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592de7326f0 con 0x5592de8e80e0 2026-02-25T11:56:38.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f9521ca9640 1 --2- 10.20.193.28:0/327424299 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8c3420 secure :-1 s=READY pgs=810 gs=1 cs=0 l=1 c_cookie=14558646463730045882 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f95180055b0 tx=0x7f9518003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0ab3d2e12053ba server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:38.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f94d6ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:38.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f94d6ffd640 1 -- 10.20.193.28:0/327424299 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f951800ae50 con 0x5592de8e80e0 2026-02-25T11:56:38.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f94d6ffd640 1 -- 10.20.193.28:0/327424299 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5592de4446f0 con 0x5592de8e80e0 2026-02-25T11:56:38.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f94d6ffd640 1 -- 10.20.193.28:0/327424299 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f95180248c0 con 0x5592de8e80e0 2026-02-25T11:56:38.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f952572ee00 1 -- 10.20.193.28:0/327424299 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f950c003aa0 con 0x5592de8e80e0 2026-02-25T11:56:38.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f94d6ffd640 1 -- 10.20.193.28:0/327424299 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9518005e20 con 0x5592de8e80e0 2026-02-25T11:56:38.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f952572ee00 10 client.17238 _ll_register_callbacks cb 0x5592de72fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:38.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.042+0000 7f952572ee00 1 -- 10.20.193.28:0/327424299 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f951800ace0 con 0x5592de8e80e0 2026-02-25T11:56:38.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.043+0000 7f94d6ffd640 1 -- 10.20.193.28:0/327424299 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f951801b440 con 0x5592de8e80e0 2026-02-25T11:56:38.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.043+0000 7f952572ee00 10 client.17238 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:38.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.043+0000 7f952572ee00 1 -- 10.20.193.28:0/327424299 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5592de744340 con 0x5592de8e80e0 2026-02-25T11:56:38.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.043+0000 7f94d6ffd640 1 -- 10.20.193.28:0/327424299 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9518040030 con 0x5592de8e80e0 2026-02-25T11:56:38.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.043+0000 7f952572ee00 10 client.17238 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:38.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.043+0000 7f952572ee00 10 client.17238 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:56:38.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.043+0000 7f952572ee00 20 client.17238 populate_metadata read hostname 'trial028' 2026-02-25T11:56:38.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.043+0000 7f952572ee00 1 --2- 10.20.193.28:0/327424299 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5592de995470 0x5592de9b5920 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.043+0000 7f952572ee00 4 client.17238 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:56:38.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.043+0000 7f952572ee00 1 -- 10.20.193.28:0/327424299 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5592de441f40 con 0x5592de995470 2026-02-25T11:56:38.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.043+0000 7f95214a8640 1 --2- 10.20.193.28:0/327424299 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5592de995470 0x5592de9b5920 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f95214a8640 1 --2- 10.20.193.28:0/327424299 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5592de995470 0x5592de9b5920 crc :-1 s=READY pgs=56 gs=4 cs=0 l=0 c_cookie=12787011442030161516 s_cookie=7500884401285343739 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b174972875102e6c server_cookie=68187e9f1fe1a9fb in_seq=0 out_seq=0 2026-02-25T11:56:38.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f94d6ffd640 10 client.17238 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:56:38.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f94d6ffd640 1 -- 10.20.193.28:0/327424299 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5592de441f40 con 0x5592de995470 2026-02-25T11:56:38.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f94d6ffd640 10 client.17238 handle_command_reply: tid=0 2026-02-25T11:56:38.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f952572ee00 1 client.17238 shutdown 2026-02-25T11:56:38.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f952572ee00 1 -- 10.20.193.28:0/327424299 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 msgr2=0x5592de8c3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f952572ee00 1 --2- 10.20.193.28:0/327424299 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8c3420 secure :-1 s=READY pgs=810 gs=1 cs=0 l=1 c_cookie=14558646463730045882 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f95180055b0 tx=0x7f9518003c30 comp rx=0 tx=0).stop 2026-02-25T11:56:38.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f952572ee00 1 -- 10.20.193.28:0/327424299 shutdown_connections 2026-02-25T11:56:38.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f952572ee00 1 --2- 10.20.193.28:0/327424299 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5592de995470 0x5592de9b5920 crc :-1 s=READY pgs=56 gs=4 cs=0 l=0 c_cookie=12787011442030161516 s_cookie=7500884401285343739 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f952572ee00 1 --2- 10.20.193.28:0/327424299 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5592de8e80e0 0x5592de8c3420 unknown :-1 s=CLOSED pgs=810 gs=1 cs=0 l=1 c_cookie=14558646463730045882 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f952572ee00 1 --2- 10.20.193.28:0/327424299 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5592de8a3440 0x5592de8c2ee0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f952572ee00 1 --2- 10.20.193.28:0/327424299 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5592de8a2b30 0x5592de8c29a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f94d6ffd640 0 client.17238 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:56:38.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f952572ee00 1 -- 10.20.193.28:0/327424299 >> 10.20.193.28:0/327424299 conn(0x5592de7c17a0 msgr2=0x5592de8a1500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:38.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f952572ee00 1 -- 10.20.193.28:0/327424299 shutdown_connections 2026-02-25T11:56:38.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f952572ee00 1 -- 10.20.193.28:0/327424299 wait complete. 2026-02-25T11:56:38.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.044+0000 7f952572ee00 20 client.17238 trim_cache size 0 max 16384 2026-02-25T11:56:38.053 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:56:38.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.045+0000 7f95224aa640 1 -- 10.20.193.28:0/3430364079 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f94f80b5760 msgr2=0x7f94f80d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.045+0000 7f95224aa640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f94f80b5760 0x7f94f80d5c10 secure :-1 s=READY pgs=367 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f950c004740 tx=0x7f950c009310 comp rx=0 tx=0).stop 2026-02-25T11:56:38.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.045+0000 7f95224aa640 1 -- 10.20.193.28:0/3430364079 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f951c10eb80 msgr2=0x7f951c162750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.045+0000 7f95224aa640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f951c10eb80 0x7f951c162750 secure :-1 s=READY pgs=213 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9510004660 tx=0x7f9510004690 comp rx=0 tx=0).stop 2026-02-25T11:56:38.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.045+0000 7f95224aa640 1 -- 10.20.193.28:0/3430364079 shutdown_connections 2026-02-25T11:56:38.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.045+0000 7f95224aa640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f94f80b5760 0x7f94f80d5c10 unknown :-1 s=CLOSED pgs=367 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.045+0000 7f95224aa640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f951c153740 0x7f951c167b20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.045+0000 7f95224aa640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f951c10eb80 0x7f951c162750 unknown :-1 s=CLOSED pgs=213 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.045+0000 7f95224aa640 1 --2- 10.20.193.28:0/3430364079 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f951c10e190 0x7f951c162210 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.045+0000 7f95224aa640 1 -- 10.20.193.28:0/3430364079 >> 10.20.193.28:0/3430364079 conn(0x7f951c0e9b40 msgr2=0x7f951c109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:38.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.045+0000 7f95224aa640 1 -- 10.20.193.28:0/3430364079 shutdown_connections 2026-02-25T11:56:38.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.045+0000 7f95224aa640 1 -- 10.20.193.28:0/3430364079 wait complete. 2026-02-25T11:56:38.179 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:56:38.362 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:38.404 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:38 trial028 ceph-mon[39224]: pgmap v478: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:38.404 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:38 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1189376437' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:56:38.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.549+0000 7f06eddb8640 1 -- 10.20.193.28:0/2017771505 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f06e81135c0 msgr2=0x7f06e8173f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.549+0000 7f06eddb8640 1 --2- 10.20.193.28:0/2017771505 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f06e81135c0 0x7f06e8173f90 secure :-1 s=READY pgs=811 gs=1 cs=0 l=1 c_cookie=9835383865926962101 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f06e400aa30 tx=0x7f06e4030270 comp rx=0 tx=0).stop 2026-02-25T11:56:38.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.549+0000 7f06eddb8640 1 -- 10.20.193.28:0/2017771505 shutdown_connections 2026-02-25T11:56:38.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.549+0000 7f06eddb8640 1 --2- 10.20.193.28:0/2017771505 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f06e81135c0 0x7f06e8173f90 unknown :-1 s=CLOSED pgs=811 gs=1 cs=0 l=1 c_cookie=9835383865926962101 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.549+0000 7f06eddb8640 1 --2- 10.20.193.28:0/2017771505 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f06e810ddc0 0x7f06e8112e10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.549+0000 7f06eddb8640 1 --2- 10.20.193.28:0/2017771505 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f06e810d3d0 0x7f06e810d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.549+0000 7f06eddb8640 1 -- 10.20.193.28:0/2017771505 >> 10.20.193.28:0/2017771505 conn(0x7f06e80eaca0 msgr2=0x7f06e810b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:38.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.549+0000 7f06eddb8640 1 -- 10.20.193.28:0/2017771505 shutdown_connections 2026-02-25T11:56:38.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.549+0000 7f06eddb8640 1 -- 10.20.193.28:0/2017771505 wait complete. 2026-02-25T11:56:38.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.549+0000 7f06eddb8640 1 Processor -- start 2026-02-25T11:56:38.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.549+0000 7f06eddb8640 1 -- start start 2026-02-25T11:56:38.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06eddb8640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f06e810d3d0 0x7f06e80f5910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06eddb8640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f06e810ddc0 0x7f06e8173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06eddb8640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f06e81135c0 0x7f06e80f6010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06eddb8640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f06e80fb220 con 0x7f06e810ddc0 2026-02-25T11:56:38.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06eddb8640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f06e80fb390 con 0x7f06e81135c0 2026-02-25T11:56:38.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06eddb8640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06e80fb550 con 0x7f06e810d3d0 2026-02-25T11:56:38.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06ecdb6640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f06e810d3d0 0x7f06e80f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06ecdb6640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f06e810d3d0 0x7f06e80f5910 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34762/0 (socket says 10.20.193.28:34762) 2026-02-25T11:56:38.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06ecdb6640 1 -- 10.20.193.28:0/2124083971 learned_addr learned my addr 10.20.193.28:0/2124083971 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:38.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06dffff640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f06e810ddc0 0x7f06e8173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06ed5b7640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f06e81135c0 0x7f06e80f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06ecdb6640 1 -- 10.20.193.28:0/2124083971 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f06e81135c0 msgr2=0x7f06e80f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f06e81135c0 0x7f06e80f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06ecdb6640 1 -- 10.20.193.28:0/2124083971 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f06e810ddc0 msgr2=0x7f06e8173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f06e810ddc0 0x7f06e8173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.550+0000 7f06ecdb6640 1 -- 10.20.193.28:0/2124083971 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e400a650 con 0x7f06e810d3d0 2026-02-25T11:56:38.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.551+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f06e810d3d0 0x7f06e80f5910 secure :-1 s=READY pgs=215 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f06d80072a0 tx=0x7f06d800ddf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:38.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.551+0000 7f06ddffb640 1 -- 10.20.193.28:0/2124083971 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f06d8004500 con 0x7f06e810d3d0 2026-02-25T11:56:38.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.551+0000 7f06ddffb640 1 -- 10.20.193.28:0/2124083971 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f06d800bd00 con 0x7f06e810d3d0 2026-02-25T11:56:38.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.551+0000 7f06ddffb640 1 -- 10.20.193.28:0/2124083971 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f06d80106e0 con 0x7f06e810d3d0 2026-02-25T11:56:38.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.551+0000 7f06eddb8640 1 -- 10.20.193.28:0/2124083971 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e80fb830 con 0x7f06e810d3d0 2026-02-25T11:56:38.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.551+0000 7f06eddb8640 1 -- 10.20.193.28:0/2124083971 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e80fbcb0 con 0x7f06e810d3d0 2026-02-25T11:56:38.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.554+0000 7f06ddffb640 1 -- 10.20.193.28:0/2124083971 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f06d800b840 con 0x7f06e810d3d0 2026-02-25T11:56:38.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.555+0000 7f06ddffb640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f06c40b5820 0x7f06c40d5cd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.556+0000 7f06ddffb640 1 -- 10.20.193.28:0/2124083971 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f06d80e6b80 con 0x7f06e810d3d0 2026-02-25T11:56:38.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.556+0000 7f06dffff640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f06c40b5820 0x7f06c40d5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.556+0000 7f06f103ce00 1 Processor -- start 2026-02-25T11:56:38.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.556+0000 7f06f103ce00 1 -- start start 2026-02-25T11:56:38.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.556+0000 7f06dffff640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f06c40b5820 0x7f06c40d5cd0 secure :-1 s=READY pgs=368 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f06e80fc2b0 tx=0x7f06d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:38.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.556+0000 7f06f103ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25dbf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.556+0000 7f06f103ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 0x55cbb2620ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.556+0000 7f06f103ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb26414c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.556+0000 7f06f103ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55cbb24f18a0 con 0x55cbb25dbb30 2026-02-25T11:56:38.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.556+0000 7f06f103ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55cbb24ed5d0 con 0x55cbb26210e0 2026-02-25T11:56:38.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.556+0000 7f06f103ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55cbb24e7570 con 0x55cbb25dc440 2026-02-25T11:56:38.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06ecdb6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25dbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06dffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 0x55cbb2620ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06ecdb6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25dbf00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52704/0 (socket says 10.20.193.28:52704) 2026-02-25T11:56:38.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06ecdb6640 1 -- 10.20.193.28:0/392172558 learned_addr learned my addr 10.20.193.28:0/392172558 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:38.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06ed5b7640 1 --2- 10.20.193.28:0/392172558 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb26414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06ecdb6640 1 -- 10.20.193.28:0/392172558 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 msgr2=0x55cbb2620ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/392172558 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 0x55cbb2620ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06ecdb6640 1 -- 10.20.193.28:0/392172558 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 msgr2=0x55cbb26414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/392172558 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb26414c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06ecdb6640 1 -- 10.20.193.28:0/392172558 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb2492d10 con 0x55cbb25dbb30 2026-02-25T11:56:38.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/392172558 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25dbf00 secure :-1 s=READY pgs=812 gs=1 cs=0 l=1 c_cookie=5852858897925107487 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f06d80040a0 tx=0x7f06d8002390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513988286b4f6f1f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:38.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06b54e4640 1 -- 10.20.193.28:0/392172558 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f06d80ac920 con 0x55cbb25dbb30 2026-02-25T11:56:38.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06b54e4640 1 -- 10.20.193.28:0/392172558 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f06d80ac740 con 0x55cbb25dbb30 2026-02-25T11:56:38.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.557+0000 7f06b54e4640 1 -- 10.20.193.28:0/392172558 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f06d80b2cf0 con 0x55cbb25dbb30 2026-02-25T11:56:38.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.558+0000 7f06f103ce00 1 -- 10.20.193.28:0/392172558 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 msgr2=0x55cbb25dbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.558+0000 7f06f103ce00 1 --2- 10.20.193.28:0/392172558 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25dbf00 secure :-1 s=READY pgs=812 gs=1 cs=0 l=1 c_cookie=5852858897925107487 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f06d80040a0 tx=0x7f06d8002390 comp rx=0 tx=0).stop 2026-02-25T11:56:38.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.558+0000 7f06f103ce00 1 -- 10.20.193.28:0/392172558 shutdown_connections 2026-02-25T11:56:38.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.558+0000 7f06f103ce00 1 --2- 10.20.193.28:0/392172558 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb26414c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.558+0000 7f06f103ce00 1 --2- 10.20.193.28:0/392172558 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 0x55cbb2620ba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.558+0000 7f06f103ce00 1 --2- 10.20.193.28:0/392172558 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25dbf00 unknown :-1 s=CLOSED pgs=812 gs=1 cs=0 l=1 c_cookie=5852858897925107487 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.558+0000 7f06f103ce00 1 -- 10.20.193.28:0/392172558 >> 10.20.193.28:0/392172558 conn(0x55cbb24fa7a0 msgr2=0x55cbb25da410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:38.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.558+0000 7f06f103ce00 1 -- 10.20.193.28:0/392172558 shutdown_connections 2026-02-25T11:56:38.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.558+0000 7f06f103ce00 1 -- 10.20.193.28:0/392172558 wait complete. 2026-02-25T11:56:38.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.559+0000 7f06f103ce00 1 -- start start 2026-02-25T11:56:38.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.559+0000 7f06f103ce00 1 Processor -- start 2026-02-25T11:56:38.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06f103ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dbb30 0x55cbb25f3fa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06f103ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dc440 0x55cbb25f44e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06f103ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb25f94d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06f103ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55cbb25f9a10 con 0x55cbb25dc440 2026-02-25T11:56:38.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06f103ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55cbb25f9b80 con 0x55cbb26210e0 2026-02-25T11:56:38.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06f103ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55cbb25f9d40 con 0x55cbb25dbb30 2026-02-25T11:56:38.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06dffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dc440 0x55cbb25f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06dffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dc440 0x55cbb25f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52712/0 (socket says 10.20.193.28:52712) 2026-02-25T11:56:38.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06dffff640 1 -- 10.20.193.28:0/834403952 learned_addr learned my addr 10.20.193.28:0/834403952 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:38.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/834403952 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dbb30 0x55cbb25f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06ed5b7640 1 --2- 10.20.193.28:0/834403952 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb25f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06dffff640 1 -- 10.20.193.28:0/834403952 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dbb30 msgr2=0x55cbb25f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06dffff640 1 --2- 10.20.193.28:0/834403952 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dbb30 0x55cbb25f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06dffff640 1 -- 10.20.193.28:0/834403952 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 msgr2=0x55cbb25f94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06dffff640 1 --2- 10.20.193.28:0/834403952 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb25f94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06dffff640 1 -- 10.20.193.28:0/834403952 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb246b3c0 con 0x55cbb25dc440 2026-02-25T11:56:38.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06dffff640 1 --2- 10.20.193.28:0/834403952 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dc440 0x55cbb25f44e0 secure :-1 s=READY pgs=813 gs=1 cs=0 l=1 c_cookie=11903542458077025691 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f06d0002920 tx=0x7f06d00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a531e0cc4676159b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:38.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.560+0000 7f06a67fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:38.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06a67fc640 1 -- 10.20.193.28:0/834403952 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f06d001fdf0 con 0x55cbb25dc440 2026-02-25T11:56:38.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06a67fc640 1 -- 10.20.193.28:0/834403952 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f06d0009e90 con 0x55cbb25dc440 2026-02-25T11:56:38.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06a67fc640 1 -- 10.20.193.28:0/834403952 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f06d00277b0 con 0x55cbb25dc440 2026-02-25T11:56:38.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06f103ce00 1 -- 10.20.193.28:0/834403952 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e80fbcb0 con 0x55cbb25dc440 2026-02-25T11:56:38.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06a67fc640 1 -- 10.20.193.28:0/834403952 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f06d0027940 con 0x55cbb25dc440 2026-02-25T11:56:38.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06f103ce00 10 client.17250 _ll_register_callbacks cb 0x55cbb2468a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:38.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06f103ce00 1 -- 10.20.193.28:0/834403952 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cbb217d6f0 con 0x55cbb25dc440 2026-02-25T11:56:38.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06a67fc640 1 -- 10.20.193.28:0/834403952 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f06d0011db0 con 0x55cbb25dc440 2026-02-25T11:56:38.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06f103ce00 10 client.17250 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:38.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06f103ce00 1 -- 10.20.193.28:0/834403952 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55cbb247d0e0 con 0x55cbb25dc440 2026-02-25T11:56:38.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06a67fc640 1 -- 10.20.193.28:0/834403952 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f06d0019030 con 0x55cbb25dc440 2026-02-25T11:56:38.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06f103ce00 10 client.17250 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:38.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06f103ce00 10 client.17250 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:56:38.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06f103ce00 20 client.17250 populate_metadata read hostname 'trial028' 2026-02-25T11:56:38.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.561+0000 7f06f103ce00 1 --2- 10.20.193.28:0/834403952 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55cbb261e770 0x55cbb261ec10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.562+0000 7f06f103ce00 4 client.17250 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:56:38.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.562+0000 7f06f103ce00 1 -- 10.20.193.28:0/834403952 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbb24b9fc0 con 0x55cbb261e770 2026-02-25T11:56:38.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.562+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/834403952 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55cbb261e770 0x55cbb261ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.562+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/834403952 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55cbb261e770 0x55cbb261ec10 crc :-1 s=READY pgs=56 gs=4 cs=0 l=0 c_cookie=11218046107721947708 s_cookie=10781194897557025991 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9bae817fccb85a3c server_cookie=959e7fa35b23f4c7 in_seq=0 out_seq=0 2026-02-25T11:56:38.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.562+0000 7f06a67fc640 10 client.17250 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:56:38.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06a67fc640 1 -- 10.20.193.28:0/834403952 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55cbb24b9fc0 con 0x55cbb261e770 2026-02-25T11:56:38.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06a67fc640 10 client.17250 handle_command_reply: tid=0 2026-02-25T11:56:38.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06f103ce00 1 client.17250 shutdown 2026-02-25T11:56:38.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06f103ce00 1 -- 10.20.193.28:0/834403952 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dc440 msgr2=0x55cbb25f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06f103ce00 1 --2- 10.20.193.28:0/834403952 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dc440 0x55cbb25f44e0 secure :-1 s=READY pgs=813 gs=1 cs=0 l=1 c_cookie=11903542458077025691 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f06d0002920 tx=0x7f06d00047e0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06f103ce00 1 -- 10.20.193.28:0/834403952 shutdown_connections 2026-02-25T11:56:38.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06f103ce00 1 --2- 10.20.193.28:0/834403952 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55cbb261e770 0x55cbb261ec10 crc :-1 s=READY pgs=56 gs=4 cs=0 l=0 c_cookie=11218046107721947708 s_cookie=10781194897557025991 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06f103ce00 1 --2- 10.20.193.28:0/834403952 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb25f94d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06f103ce00 1 --2- 10.20.193.28:0/834403952 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dc440 0x55cbb25f44e0 unknown :-1 s=CLOSED pgs=813 gs=1 cs=0 l=1 c_cookie=11903542458077025691 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06f103ce00 1 --2- 10.20.193.28:0/834403952 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dbb30 0x55cbb25f3fa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06f103ce00 1 -- 10.20.193.28:0/834403952 >> 10.20.193.28:0/834403952 conn(0x55cbb24fa7a0 msgr2=0x55cbb25dab20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:38.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06a67fc640 0 client.17250 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:56:38.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06f103ce00 1 -- 10.20.193.28:0/834403952 shutdown_connections 2026-02-25T11:56:38.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.563+0000 7f06f103ce00 1 -- 10.20.193.28:0/834403952 wait complete. 2026-02-25T11:56:38.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.564+0000 7f06f103ce00 20 client.17250 trim_cache size 0 max 16384 2026-02-25T11:56:38.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.571+0000 7f06f103ce00 1 Processor -- start 2026-02-25T11:56:38.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.571+0000 7f06f103ce00 1 -- start start 2026-02-25T11:56:38.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.571+0000 7f06f103ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25dad20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.571+0000 7f06f103ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 0x55cbb25db260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.571+0000 7f06f103ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb26202e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.571+0000 7f06f103ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55cbb24e51a0 con 0x55cbb25dbb30 2026-02-25T11:56:38.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.571+0000 7f06f103ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55cbb21a5700 con 0x55cbb26210e0 2026-02-25T11:56:38.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.571+0000 7f06f103ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55cbb25b6e80 con 0x55cbb25dc440 2026-02-25T11:56:38.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.571+0000 7f06ecdb6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25dad20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.571+0000 7f06ecdb6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25dad20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52728/0 (socket says 10.20.193.28:52728) 2026-02-25T11:56:38.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.571+0000 7f06ecdb6640 1 -- 10.20.193.28:0/1556824880 learned_addr learned my addr 10.20.193.28:0/1556824880 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:38.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.571+0000 7f06dffff640 1 --2- 10.20.193.28:0/1556824880 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 0x55cbb25db260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06ed5b7640 1 --2- 10.20.193.28:0/1556824880 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb26202e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06ecdb6640 1 -- 10.20.193.28:0/1556824880 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 msgr2=0x55cbb25db260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/1556824880 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 0x55cbb25db260 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06ecdb6640 1 -- 10.20.193.28:0/1556824880 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 msgr2=0x55cbb26202e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/1556824880 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb26202e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06ecdb6640 1 -- 10.20.193.28:0/1556824880 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb25e68c0 con 0x55cbb25dbb30 2026-02-25T11:56:38.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/1556824880 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25dad20 secure :-1 s=READY pgs=814 gs=1 cs=0 l=1 c_cookie=5173136602905878479 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f06d80e4760 tx=0x7f06d8002390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47caac54f6c69fcf server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:38.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06b4ce3640 1 -- 10.20.193.28:0/1556824880 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f06d8078040 con 0x55cbb25dbb30 2026-02-25T11:56:38.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06b4ce3640 1 -- 10.20.193.28:0/1556824880 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55cbb24b9fc0 con 0x55cbb25dbb30 2026-02-25T11:56:38.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06b4ce3640 1 -- 10.20.193.28:0/1556824880 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f06d80b8080 con 0x55cbb25dbb30 2026-02-25T11:56:38.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06f103ce00 1 -- 10.20.193.28:0/1556824880 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 msgr2=0x55cbb25dad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06f103ce00 1 --2- 10.20.193.28:0/1556824880 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25dad20 secure :-1 s=READY pgs=814 gs=1 cs=0 l=1 c_cookie=5173136602905878479 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f06d80e4760 tx=0x7f06d8002390 comp rx=0 tx=0).stop 2026-02-25T11:56:38.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06f103ce00 1 -- 10.20.193.28:0/1556824880 shutdown_connections 2026-02-25T11:56:38.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06f103ce00 1 --2- 10.20.193.28:0/1556824880 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb26202e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06f103ce00 1 --2- 10.20.193.28:0/1556824880 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 0x55cbb25db260 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06f103ce00 1 --2- 10.20.193.28:0/1556824880 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25dad20 unknown :-1 s=CLOSED pgs=814 gs=1 cs=0 l=1 c_cookie=5173136602905878479 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06f103ce00 1 -- 10.20.193.28:0/1556824880 >> 10.20.193.28:0/1556824880 conn(0x55cbb24fa7a0 msgr2=0x55cbb25de9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:38.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06f103ce00 1 -- 10.20.193.28:0/1556824880 shutdown_connections 2026-02-25T11:56:38.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.572+0000 7f06f103ce00 1 -- 10.20.193.28:0/1556824880 wait complete. 2026-02-25T11:56:38.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.573+0000 7f06f103ce00 1 -- start start 2026-02-25T11:56:38.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.573+0000 7f06f103ce00 1 Processor -- start 2026-02-25T11:56:38.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.573+0000 7f06f103ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25fbb70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.573+0000 7f06f103ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 0x55cbb25fc0b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.573+0000 7f06f103ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb25fc5f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.573+0000 7f06f103ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55cbb21a5700 con 0x55cbb25dbb30 2026-02-25T11:56:38.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.573+0000 7f06f103ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2685650 con 0x55cbb26210e0 2026-02-25T11:56:38.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.573+0000 7f06f103ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55cbb2685810 con 0x55cbb25dc440 2026-02-25T11:56:38.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06ecdb6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25fbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06ecdb6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25fbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52734/0 (socket says 10.20.193.28:52734) 2026-02-25T11:56:38.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06ecdb6640 1 -- 10.20.193.28:0/1593401516 learned_addr learned my addr 10.20.193.28:0/1593401516 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:38.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06dffff640 1 --2- 10.20.193.28:0/1593401516 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 0x55cbb25fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06ecdb6640 1 -- 10.20.193.28:0/1593401516 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 msgr2=0x55cbb25fc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/1593401516 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 0x55cbb25fc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06ecdb6640 1 -- 10.20.193.28:0/1593401516 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 msgr2=0x55cbb25fc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:38.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/1593401516 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb25fc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06ecdb6640 1 -- 10.20.193.28:0/1593401516 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbb2477b60 con 0x55cbb25dbb30 2026-02-25T11:56:38.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06ecdb6640 1 --2- 10.20.193.28:0/1593401516 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25fbb70 secure :-1 s=READY pgs=815 gs=1 cs=0 l=1 c_cookie=5393878445068317581 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f06d800b630 tx=0x7f06d8002b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adae7de27eef38d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:38.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06a67fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:38.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06a67fc640 1 -- 10.20.193.28:0/1593401516 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f06d8078050 con 0x55cbb25dbb30 2026-02-25T11:56:38.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06a67fc640 1 -- 10.20.193.28:0/1593401516 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f06d8002e30 con 0x55cbb25dbb30 2026-02-25T11:56:38.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06f103ce00 1 -- 10.20.193.28:0/1593401516 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbb2492d10 con 0x55cbb25dbb30 2026-02-25T11:56:38.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06a67fc640 1 -- 10.20.193.28:0/1593401516 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f06d80b2c60 con 0x55cbb25dbb30 2026-02-25T11:56:38.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06f103ce00 10 client.17256 _ll_register_callbacks cb 0x55cbb2468a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:38.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06f103ce00 1 -- 10.20.193.28:0/1593401516 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d80ac740 con 0x55cbb25dbb30 2026-02-25T11:56:38.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.574+0000 7f06a67fc640 1 -- 10.20.193.28:0/1593401516 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f06d80adbd0 con 0x55cbb25dbb30 2026-02-25T11:56:38.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.575+0000 7f06a67fc640 1 -- 10.20.193.28:0/1593401516 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f06d80addf0 con 0x55cbb25dbb30 2026-02-25T11:56:38.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.575+0000 7f06f103ce00 10 client.17256 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:38.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.575+0000 7f06f103ce00 1 -- 10.20.193.28:0/1593401516 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55cbb246b6f0 con 0x55cbb25dbb30 2026-02-25T11:56:38.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.575+0000 7f06a67fc640 1 -- 10.20.193.28:0/1593401516 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f06d80b7bf0 con 0x55cbb25dbb30 2026-02-25T11:56:38.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.575+0000 7f06f103ce00 10 client.17256 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:38.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.575+0000 7f06f103ce00 10 client.17256 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:56:38.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.575+0000 7f06f103ce00 20 client.17256 populate_metadata read hostname 'trial028' 2026-02-25T11:56:38.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.575+0000 7f06f103ce00 1 --2- 10.20.193.28:0/1593401516 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55cbb2686f60 0x55cbb26aeb60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:38.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.575+0000 7f06f103ce00 4 client.17256 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:56:38.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.575+0000 7f06f103ce00 1 -- 10.20.193.28:0/1593401516 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cbb217af40 con 0x55cbb2686f60 2026-02-25T11:56:38.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.575+0000 7f06dffff640 1 --2- 10.20.193.28:0/1593401516 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55cbb2686f60 0x55cbb26aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:38.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06dffff640 1 --2- 10.20.193.28:0/1593401516 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55cbb2686f60 0x55cbb26aeb60 crc :-1 s=READY pgs=57 gs=4 cs=0 l=0 c_cookie=8923255165566497597 s_cookie=7091626566177729520 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7bd5c54bafc2b33d server_cookie=626a84c9f4bf2bf0 in_seq=0 out_seq=0 2026-02-25T11:56:38.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06a67fc640 10 client.17256 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:56:38.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06a67fc640 1 -- 10.20.193.28:0/1593401516 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cbb217af40 con 0x55cbb2686f60 2026-02-25T11:56:38.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06a67fc640 10 client.17256 handle_command_reply: tid=0 2026-02-25T11:56:38.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06f103ce00 1 client.17256 shutdown 2026-02-25T11:56:38.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06f103ce00 1 -- 10.20.193.28:0/1593401516 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 msgr2=0x55cbb25fbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06f103ce00 1 --2- 10.20.193.28:0/1593401516 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25fbb70 secure :-1 s=READY pgs=815 gs=1 cs=0 l=1 c_cookie=5393878445068317581 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f06d800b630 tx=0x7f06d8002b60 comp rx=0 tx=0).stop 2026-02-25T11:56:38.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06f103ce00 1 -- 10.20.193.28:0/1593401516 shutdown_connections 2026-02-25T11:56:38.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06f103ce00 1 --2- 10.20.193.28:0/1593401516 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55cbb2686f60 0x55cbb26aeb60 crc :-1 s=READY pgs=57 gs=4 cs=0 l=0 c_cookie=8923255165566497597 s_cookie=7091626566177729520 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06f103ce00 1 --2- 10.20.193.28:0/1593401516 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cbb26210e0 0x55cbb25fc5f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06f103ce00 1 --2- 10.20.193.28:0/1593401516 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cbb25dc440 0x55cbb25fc0b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06f103ce00 1 --2- 10.20.193.28:0/1593401516 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cbb25dbb30 0x55cbb25fbb70 unknown :-1 s=CLOSED pgs=815 gs=1 cs=0 l=1 c_cookie=5393878445068317581 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06a67fc640 0 client.17256 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:56:38.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06f103ce00 1 -- 10.20.193.28:0/1593401516 >> 10.20.193.28:0/1593401516 conn(0x55cbb24fa7a0 msgr2=0x55cbb25d9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:38.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06f103ce00 1 -- 10.20.193.28:0/1593401516 shutdown_connections 2026-02-25T11:56:38.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06f103ce00 1 -- 10.20.193.28:0/1593401516 wait complete. 2026-02-25T11:56:38.587 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.576+0000 7f06f103ce00 20 client.17256 trim_cache size 0 max 16384 2026-02-25T11:56:38.587 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:56:38.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.577+0000 7f06eddb8640 1 -- 10.20.193.28:0/2124083971 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f06c40b5820 msgr2=0x7f06c40d5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.577+0000 7f06eddb8640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f06c40b5820 0x7f06c40d5cd0 secure :-1 s=READY pgs=368 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f06e80fc2b0 tx=0x7f06d0005f90 comp rx=0 tx=0).stop 2026-02-25T11:56:38.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.577+0000 7f06eddb8640 1 -- 10.20.193.28:0/2124083971 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f06e810d3d0 msgr2=0x7f06e80f5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:38.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.577+0000 7f06eddb8640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f06e810d3d0 0x7f06e80f5910 secure :-1 s=READY pgs=215 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f06d80072a0 tx=0x7f06d800ddf0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.577+0000 7f06eddb8640 1 -- 10.20.193.28:0/2124083971 shutdown_connections 2026-02-25T11:56:38.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.577+0000 7f06eddb8640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f06c40b5820 0x7f06c40d5cd0 unknown :-1 s=CLOSED pgs=368 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.577+0000 7f06eddb8640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f06e81135c0 0x7f06e80f6010 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.577+0000 7f06eddb8640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f06e810ddc0 0x7f06e8173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.577+0000 7f06eddb8640 1 --2- 10.20.193.28:0/2124083971 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f06e810d3d0 0x7f06e80f5910 unknown :-1 s=CLOSED pgs=215 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:38.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.577+0000 7f06eddb8640 1 -- 10.20.193.28:0/2124083971 >> 10.20.193.28:0/2124083971 conn(0x7f06e80eaca0 msgr2=0x7f06e810b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:38.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.577+0000 7f06eddb8640 1 -- 10.20.193.28:0/2124083971 shutdown_connections 2026-02-25T11:56:38.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:38.577+0000 7f06eddb8640 1 -- 10.20.193.28:0/2124083971 wait complete. 2026-02-25T11:56:38.679 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:56:38.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:38 trial167 ceph-mon[43392]: pgmap v478: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:38.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:38 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1189376437' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:56:38.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:38 trial119 ceph-mon[42710]: pgmap v478: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:38.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:38 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1189376437' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:56:38.861 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:39.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.054+0000 7fb8fb2e5640 1 -- 10.20.193.28:0/1630743954 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8f410ddc0 msgr2=0x7fb8f4112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:39.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.054+0000 7fb8fb2e5640 1 --2- 10.20.193.28:0/1630743954 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8f410ddc0 0x7fb8f4112e10 secure :-1 s=READY pgs=816 gs=1 cs=0 l=1 c_cookie=13372638545807412381 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e8009950 tx=0x7fb8e802f190 comp rx=0 tx=0).stop 2026-02-25T11:56:39.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.054+0000 7fb8fb2e5640 1 -- 10.20.193.28:0/1630743954 shutdown_connections 2026-02-25T11:56:39.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.054+0000 7fb8fb2e5640 1 --2- 10.20.193.28:0/1630743954 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb8f41135c0 0x7fb8f4173f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.054+0000 7fb8fb2e5640 1 --2- 10.20.193.28:0/1630743954 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8f410ddc0 0x7fb8f4112e10 unknown :-1 s=CLOSED pgs=816 gs=1 cs=0 l=1 c_cookie=13372638545807412381 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.054+0000 7fb8fb2e5640 1 --2- 10.20.193.28:0/1630743954 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb8f410d3d0 0x7fb8f410d7a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.054+0000 7fb8fb2e5640 1 -- 10.20.193.28:0/1630743954 >> 10.20.193.28:0/1630743954 conn(0x7fb8f40eaca0 msgr2=0x7fb8f410b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:39.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.054+0000 7fb8fb2e5640 1 -- 10.20.193.28:0/1630743954 shutdown_connections 2026-02-25T11:56:39.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.054+0000 7fb8fb2e5640 1 -- 10.20.193.28:0/1630743954 wait complete. 2026-02-25T11:56:39.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.055+0000 7fb8fb2e5640 1 Processor -- start 2026-02-25T11:56:39.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.055+0000 7fb8fb2e5640 1 -- start start 2026-02-25T11:56:39.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.055+0000 7fb8fb2e5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8f410d3d0 0x7fb8f40f59a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.055+0000 7fb8fb2e5640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb8f410ddc0 0x7fb8f4173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.055+0000 7fb8fb2e5640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb8f41135c0 0x7fb8f40f60a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.055+0000 7fb8fb2e5640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f40fb2b0 con 0x7fb8f410d3d0 2026-02-25T11:56:39.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.055+0000 7fb8fb2e5640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f40fb420 con 0x7fb8f41135c0 2026-02-25T11:56:39.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.055+0000 7fb8fb2e5640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f40fb5e0 con 0x7fb8f410ddc0 2026-02-25T11:56:39.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.056+0000 7fb8f9ae2640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb8f410ddc0 0x7fb8f4173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.056+0000 7fb8f9ae2640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb8f410ddc0 0x7fb8f4173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34788/0 (socket says 10.20.193.28:34788) 2026-02-25T11:56:39.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.056+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/1101565657 learned_addr learned my addr 10.20.193.28:0/1101565657 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:39.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.056+0000 7fb8faae4640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb8f41135c0 0x7fb8f40f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.056+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/1101565657 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb8f41135c0 msgr2=0x7fb8f40f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:39.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.056+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb8f41135c0 0x7fb8f40f60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.056+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/1101565657 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8f410d3d0 msgr2=0x7fb8f40f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:39.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.056+0000 7fb8fa2e3640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8f410d3d0 0x7fb8f40f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.056+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8f410d3d0 0x7fb8f40f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.056+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/1101565657 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e8009570 con 0x7fb8f410ddc0 2026-02-25T11:56:39.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.056+0000 7fb8faae4640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb8f41135c0 0x7fb8f40f60a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:56:39.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.056+0000 7fb8fa2e3640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8f410d3d0 0x7fb8f40f59a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:56:39.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.056+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb8f410ddc0 0x7fb8f4173e90 secure :-1 s=READY pgs=216 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e80098a0 tx=0x7fb8e80028d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:39.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.057+0000 7fb8db7fe640 1 -- 10.20.193.28:0/1101565657 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb8e803d070 con 0x7fb8f410ddc0 2026-02-25T11:56:39.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.057+0000 7fb8fb2e5640 1 -- 10.20.193.28:0/1101565657 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f40fb860 con 0x7fb8f410ddc0 2026-02-25T11:56:39.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.057+0000 7fb8fb2e5640 1 -- 10.20.193.28:0/1101565657 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f40fbd20 con 0x7fb8f410ddc0 2026-02-25T11:56:39.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.057+0000 7fb8db7fe640 1 -- 10.20.193.28:0/1101565657 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb8e8002a40 con 0x7fb8f410ddc0 2026-02-25T11:56:39.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.057+0000 7fb8db7fe640 1 -- 10.20.193.28:0/1101565657 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb8e8041690 con 0x7fb8f410ddc0 2026-02-25T11:56:39.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.059+0000 7fb8db7fe640 1 -- 10.20.193.28:0/1101565657 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fb8e80386a0 con 0x7fb8f410ddc0 2026-02-25T11:56:39.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.061+0000 7fb8db7fe640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb8d00b5750 0x7fb8d00d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.061+0000 7fb8db7fe640 1 -- 10.20.193.28:0/1101565657 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb8e810b1c0 con 0x7fb8f410ddc0 2026-02-25T11:56:39.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.061+0000 7fb8fa2e3640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb8d00b5750 0x7fb8d00d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.062+0000 7fb8fa2e3640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb8d00b5750 0x7fb8d00d5c00 secure :-1 s=READY pgs=369 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e4004740 tx=0x7fb8e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:39.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.062+0000 7fb8fe569e00 1 Processor -- start 2026-02-25T11:56:39.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.062+0000 7fb8fe569e00 1 -- start start 2026-02-25T11:56:39.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fe569e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c47b30 0x55b429c47f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fe569e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c48440 0x55b429c8cba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fe569e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c8d0e0 0x55b429cad4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fe569e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b429b5d8a0 con 0x55b429c47b30 2026-02-25T11:56:39.066 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fe569e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b429b595d0 con 0x55b429c48440 2026-02-25T11:56:39.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fe569e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b429b53570 con 0x55b429c8d0e0 2026-02-25T11:56:39.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fa2e3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c47b30 0x55b429c47f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fa2e3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c47b30 0x55b429c47f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52760/0 (socket says 10.20.193.28:52760) 2026-02-25T11:56:39.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fa2e3640 1 -- 10.20.193.28:0/1591593924 learned_addr learned my addr 10.20.193.28:0/1591593924 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:39.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8faae4640 1 --2- 10.20.193.28:0/1591593924 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c8d0e0 0x55b429cad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/1591593924 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c48440 0x55b429c8cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fa2e3640 1 -- 10.20.193.28:0/1591593924 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c8d0e0 msgr2=0x55b429cad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:39.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fa2e3640 1 --2- 10.20.193.28:0/1591593924 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c8d0e0 0x55b429cad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fa2e3640 1 -- 10.20.193.28:0/1591593924 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c48440 msgr2=0x55b429c8cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:39.067 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fa2e3640 1 --2- 10.20.193.28:0/1591593924 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c48440 0x55b429c8cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fa2e3640 1 -- 10.20.193.28:0/1591593924 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b429afed10 con 0x55b429c47b30 2026-02-25T11:56:39.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.063+0000 7fb8fa2e3640 1 --2- 10.20.193.28:0/1591593924 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c47b30 0x55b429c47f00 secure :-1 s=READY pgs=817 gs=1 cs=0 l=1 c_cookie=12448006788568760376 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e4005d90 tx=0x7fb8e4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc03438ca4ac038 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:39.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.064+0000 7fb8d8b0a640 1 -- 10.20.193.28:0/1591593924 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb8e4018040 con 0x55b429c47b30 2026-02-25T11:56:39.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.064+0000 7fb8d8b0a640 1 -- 10.20.193.28:0/1591593924 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb8e40045a0 con 0x55b429c47b30 2026-02-25T11:56:39.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.064+0000 7fb8d8b0a640 1 -- 10.20.193.28:0/1591593924 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb8e40127f0 con 0x55b429c47b30 2026-02-25T11:56:39.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.064+0000 7fb8fe569e00 1 -- 10.20.193.28:0/1591593924 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c47b30 msgr2=0x55b429c47f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:39.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.064+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/1591593924 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c47b30 0x55b429c47f00 secure :-1 s=READY pgs=817 gs=1 cs=0 l=1 c_cookie=12448006788568760376 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e4005d90 tx=0x7fb8e4013000 comp rx=0 tx=0).stop 2026-02-25T11:56:39.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.064+0000 7fb8fe569e00 1 -- 10.20.193.28:0/1591593924 shutdown_connections 2026-02-25T11:56:39.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.064+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/1591593924 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c8d0e0 0x55b429cad4c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.064+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/1591593924 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c48440 0x55b429c8cba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.064+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/1591593924 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c47b30 0x55b429c47f00 unknown :-1 s=CLOSED pgs=817 gs=1 cs=0 l=1 c_cookie=12448006788568760376 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.068 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.064+0000 7fb8fe569e00 1 -- 10.20.193.28:0/1591593924 >> 10.20.193.28:0/1591593924 conn(0x55b429b667a0 msgr2=0x55b429c46410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:39.069 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.064+0000 7fb8fe569e00 1 -- 10.20.193.28:0/1591593924 shutdown_connections 2026-02-25T11:56:39.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.064+0000 7fb8fe569e00 1 -- 10.20.193.28:0/1591593924 wait complete. 2026-02-25T11:56:39.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.065+0000 7fb8fe569e00 1 -- start start 2026-02-25T11:56:39.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.065+0000 7fb8fe569e00 1 Processor -- start 2026-02-25T11:56:39.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.066+0000 7fb8fe569e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c47b30 0x55b429c6be30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.066+0000 7fb8fe569e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c48440 0x55b429c6c370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.066+0000 7fb8fe569e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c8d0e0 0x55b429c6ac50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.066+0000 7fb8fe569e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b429c6b1c0 con 0x55b429c48440 2026-02-25T11:56:39.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.066+0000 7fb8fe569e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b429c6b330 con 0x55b429c8d0e0 2026-02-25T11:56:39.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.066+0000 7fb8fa2e3640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c47b30 0x55b429c6be30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.066+0000 7fb8fa2e3640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c47b30 0x55b429c6be30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34800/0 (socket says 10.20.193.28:34800) 2026-02-25T11:56:39.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.066+0000 7fb8fa2e3640 1 -- 10.20.193.28:0/411509432 learned_addr learned my addr 10.20.193.28:0/411509432 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:39.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.066+0000 7fb8fe569e00 1 -- 10.20.193.28:0/411509432 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b429c6b4f0 con 0x55b429c47b30 2026-02-25T11:56:39.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.067+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c48440 0x55b429c6c370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.067+0000 7fb8faae4640 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c8d0e0 0x55b429c6ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.067+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/411509432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c47b30 msgr2=0x55b429c6be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:39.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.067+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c47b30 0x55b429c6be30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.067+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/411509432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c8d0e0 msgr2=0x55b429c6ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:39.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.067+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c8d0e0 0x55b429c6ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.067+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/411509432 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b429ad73c0 con 0x55b429c48440 2026-02-25T11:56:39.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.067+0000 7fb8fa2e3640 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c47b30 0x55b429c6be30 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:56:39.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.067+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c48440 0x55b429c6c370 secure :-1 s=READY pgs=818 gs=2 cs=0 l=1 c_cookie=11690045406263260721 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e8032040 tx=0x7fb8e80d15f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23b625c48823631 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:39.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.067+0000 7fb8a7fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:39.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.068+0000 7fb8a7fff640 1 -- 10.20.193.28:0/411509432 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb8e80d7070 con 0x55b429c48440 2026-02-25T11:56:39.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.068+0000 7fb8fe569e00 1 -- 10.20.193.28:0/411509432 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e4009c40 con 0x55b429c48440 2026-02-25T11:56:39.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.068+0000 7fb8fe569e00 10 client.17265 _ll_register_callbacks cb 0x55b429ad4a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:39.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.068+0000 7fb8a7fff640 1 -- 10.20.193.28:0/411509432 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb8e8048e80 con 0x55b429c48440 2026-02-25T11:56:39.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.068+0000 7fb8a7fff640 1 -- 10.20.193.28:0/411509432 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb8e80dc470 con 0x55b429c48440 2026-02-25T11:56:39.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.068+0000 7fb8fe569e00 1 -- 10.20.193.28:0/411509432 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b4297e96f0 con 0x55b429c48440 2026-02-25T11:56:39.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.068+0000 7fb8a7fff640 1 -- 10.20.193.28:0/411509432 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb8e8096640 con 0x55b429c48440 2026-02-25T11:56:39.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.068+0000 7fb8a7fff640 1 -- 10.20.193.28:0/411509432 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb8e8096840 con 0x55b429c48440 2026-02-25T11:56:39.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.068+0000 7fb8fe569e00 10 client.17265 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:39.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.068+0000 7fb8fe569e00 1 -- 10.20.193.28:0/411509432 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55b429ae90e0 con 0x55b429c48440 2026-02-25T11:56:39.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.068+0000 7fb8a7fff640 1 -- 10.20.193.28:0/411509432 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb8e8096150 con 0x55b429c48440 2026-02-25T11:56:39.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.069+0000 7fb8fe569e00 10 client.17265 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:39.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.069+0000 7fb8fe569e00 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:39.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.069+0000 7fb8fe569e00 20 client.17265 populate_metadata read hostname 'trial028' 2026-02-25T11:56:39.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.069+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b429c72d70 0x55b429c731e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.069+0000 7fb8fe569e00 4 client.17265 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:56:39.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.069+0000 7fb8fe569e00 1 -- 10.20.193.28:0/411509432 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b429b25fc0 con 0x55b429c72d70 2026-02-25T11:56:39.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.069+0000 7fb8fa2e3640 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b429c72d70 0x55b429c731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.069+0000 7fb8fa2e3640 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b429c72d70 0x55b429c731e0 crc :-1 s=READY pgs=154 gs=4 cs=0 l=0 c_cookie=7705294985689328212 s_cookie=12861061069467443211 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aeeb4f70f9c6a54 server_cookie=b27baae6e036600b in_seq=0 out_seq=0 2026-02-25T11:56:39.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.069+0000 7fb8a7fff640 10 client.17265 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:39.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.070+0000 7fb8a7fff640 1 -- 10.20.193.28:0/411509432 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55b429b25fc0 con 0x55b429c72d70 2026-02-25T11:56:39.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.070+0000 7fb8a7fff640 10 client.17265 handle_command_reply: tid=0 2026-02-25T11:56:39.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.070+0000 7fb8fe569e00 1 client.17265 shutdown 2026-02-25T11:56:39.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.070+0000 7fb8fe569e00 1 -- 10.20.193.28:0/411509432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c48440 msgr2=0x55b429c6c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:39.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.070+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c48440 0x55b429c6c370 secure :-1 s=READY pgs=818 gs=2 cs=0 l=1 c_cookie=11690045406263260721 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e8032040 tx=0x7fb8e80d15f0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.070+0000 7fb8fe569e00 1 -- 10.20.193.28:0/411509432 shutdown_connections 2026-02-25T11:56:39.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.070+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b429c72d70 0x55b429c731e0 crc :-1 s=READY pgs=154 gs=4 cs=0 l=0 c_cookie=7705294985689328212 s_cookie=12861061069467443211 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.070+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c8d0e0 0x55b429c6ac50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.070+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c48440 0x55b429c6c370 unknown :-1 s=CLOSED pgs=818 gs=2 cs=0 l=1 c_cookie=11690045406263260721 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.070+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/411509432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c47b30 0x55b429c6be30 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.070+0000 7fb8a7fff640 0 client.17265 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:39.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.070+0000 7fb8fe569e00 1 -- 10.20.193.28:0/411509432 >> 10.20.193.28:0/411509432 conn(0x55b429b667a0 msgr2=0x55b429c2f520 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:39.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.071+0000 7fb8fe569e00 1 -- 10.20.193.28:0/411509432 shutdown_connections 2026-02-25T11:56:39.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.071+0000 7fb8fe569e00 1 -- 10.20.193.28:0/411509432 wait complete. 2026-02-25T11:56:39.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.071+0000 7fb8fe569e00 20 client.17265 trim_cache size 0 max 16384 2026-02-25T11:56:39.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8fe569e00 1 Processor -- start 2026-02-25T11:56:39.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8fe569e00 1 -- start start 2026-02-25T11:56:39.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8fe569e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c47b30 0x55b429c26320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8fe569e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c48440 0x55b429c32540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8fe569e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c8d0e0 0x55b429c32a80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8fe569e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b429b511a0 con 0x55b429c48440 2026-02-25T11:56:39.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8fe569e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b429811700 con 0x55b429c47b30 2026-02-25T11:56:39.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8fe569e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b429c35310 con 0x55b429c8d0e0 2026-02-25T11:56:39.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8f9ae2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c48440 0x55b429c32540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8fa2e3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c47b30 0x55b429c26320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8f9ae2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c48440 0x55b429c32540 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52784/0 (socket says 10.20.193.28:52784) 2026-02-25T11:56:39.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/2951030393 learned_addr learned my addr 10.20.193.28:0/2951030393 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:39.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8fa2e3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c47b30 0x55b429c26320 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:39690/0 (socket says 10.20.193.28:39690) 2026-02-25T11:56:39.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8faae4640 1 --2- 10.20.193.28:0/2951030393 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c8d0e0 0x55b429c32a80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/2951030393 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c8d0e0 msgr2=0x55b429c32a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:39.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/2951030393 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c8d0e0 0x55b429c32a80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/2951030393 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c47b30 msgr2=0x55b429c26320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:39.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/2951030393 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c47b30 0x55b429c26320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/2951030393 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b429ae3b60 con 0x55b429c48440 2026-02-25T11:56:39.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.079+0000 7fb8fa2e3640 1 --2- 10.20.193.28:0/2951030393 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c47b30 0x55b429c26320 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:56:39.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.080+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/2951030393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c48440 0x55b429c32540 secure :-1 s=READY pgs=819 gs=3 cs=0 l=1 c_cookie=17764780612857682202 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e8002bb0 tx=0x7fb8e809a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6892dd87620d91a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:39.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.080+0000 7fb8c24e6640 1 -- 10.20.193.28:0/2951030393 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb8e8048e80 con 0x55b429c48440 2026-02-25T11:56:39.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.080+0000 7fb8c24e6640 1 -- 10.20.193.28:0/2951030393 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55b4297e96f0 con 0x55b429c48440 2026-02-25T11:56:39.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.080+0000 7fb8c24e6640 1 -- 10.20.193.28:0/2951030393 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb8e80dc680 con 0x55b429c48440 2026-02-25T11:56:39.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.080+0000 7fb8fe569e00 1 -- 10.20.193.28:0/2951030393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c48440 msgr2=0x55b429c32540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:39.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.080+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/2951030393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c48440 0x55b429c32540 secure :-1 s=READY pgs=819 gs=3 cs=0 l=1 c_cookie=17764780612857682202 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e8002bb0 tx=0x7fb8e809a000 comp rx=0 tx=0).stop 2026-02-25T11:56:39.084 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.080+0000 7fb8fe569e00 1 -- 10.20.193.28:0/2951030393 shutdown_connections 2026-02-25T11:56:39.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.080+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/2951030393 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c8d0e0 0x55b429c32a80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.080+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/2951030393 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c48440 0x55b429c32540 unknown :-1 s=CLOSED pgs=819 gs=3 cs=0 l=1 c_cookie=17764780612857682202 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.080+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/2951030393 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c47b30 0x55b429c26320 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.080+0000 7fb8fe569e00 1 -- 10.20.193.28:0/2951030393 >> 10.20.193.28:0/2951030393 conn(0x55b429b667a0 msgr2=0x55b429c25c10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:39.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.080+0000 7fb8fe569e00 1 -- 10.20.193.28:0/2951030393 shutdown_connections 2026-02-25T11:56:39.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.080+0000 7fb8fe569e00 1 -- 10.20.193.28:0/2951030393 wait complete. 2026-02-25T11:56:39.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.081+0000 7fb8fe569e00 1 -- start start 2026-02-25T11:56:39.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.081+0000 7fb8fe569e00 1 Processor -- start 2026-02-25T11:56:39.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.081+0000 7fb8fe569e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c47b30 0x55b429c802d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.081+0000 7fb8fe569e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c48440 0x55b429c80810 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.081+0000 7fb8fe569e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c8d0e0 0x55b429c85170 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.081+0000 7fb8fe569e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55b429c6f110 con 0x55b429c47b30 2026-02-25T11:56:39.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.081+0000 7fb8fe569e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55b429c85710 con 0x55b429c8d0e0 2026-02-25T11:56:39.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.081+0000 7fb8fe569e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55b429c858d0 con 0x55b429c48440 2026-02-25T11:56:39.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.081+0000 7fb8fa2e3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c47b30 0x55b429c802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8fa2e3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c47b30 0x55b429c802d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52788/0 (socket says 10.20.193.28:52788) 2026-02-25T11:56:39.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8fa2e3640 1 -- 10.20.193.28:0/712161460 learned_addr learned my addr 10.20.193.28:0/712161460 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:39.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/712161460 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c48440 0x55b429c80810 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8faae4640 1 --2- 10.20.193.28:0/712161460 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c8d0e0 0x55b429c85170 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/712161460 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c8d0e0 msgr2=0x55b429c85170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:39.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/712161460 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c8d0e0 0x55b429c85170 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/712161460 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c47b30 msgr2=0x55b429c802d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:39.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/712161460 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c47b30 0x55b429c802d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8f9ae2640 1 -- 10.20.193.28:0/712161460 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b429ad76f0 con 0x55b429c48440 2026-02-25T11:56:39.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8f9ae2640 1 --2- 10.20.193.28:0/712161460 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c48440 0x55b429c80810 secure :-1 s=READY pgs=217 gs=2 cs=0 l=1 c_cookie=4425375182027593409 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e8005b30 tx=0x7fb8e80d15f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d6a17480a9596c1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:39.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8a7fff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:56:39.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8a7fff640 1 -- 10.20.193.28:0/712161460 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb8e802fe20 con 0x55b429c48440 2026-02-25T11:56:39.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8a7fff640 1 -- 10.20.193.28:0/712161460 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb8e80d9840 con 0x55b429c48440 2026-02-25T11:56:39.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8a7fff640 1 -- 10.20.193.28:0/712161460 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb8e80dc1f0 con 0x55b429c48440 2026-02-25T11:56:39.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8fe569e00 1 -- 10.20.193.28:0/712161460 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e809c020 con 0x55b429c48440 2026-02-25T11:56:39.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.082+0000 7fb8a7fff640 1 -- 10.20.193.28:0/712161460 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb8e80dc350 con 0x55b429c48440 2026-02-25T11:56:39.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.083+0000 7fb8fe569e00 10 client.26959 _ll_register_callbacks cb 0x55b429ad4a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:39.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.083+0000 7fb8fe569e00 1 -- 10.20.193.28:0/712161460 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8e40045a0 con 0x55b429c48440 2026-02-25T11:56:39.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.083+0000 7fb8a7fff640 1 -- 10.20.193.28:0/712161460 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb8e80d7260 con 0x55b429c48440 2026-02-25T11:56:39.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.083+0000 7fb8fe569e00 10 client.26959 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:39.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.083+0000 7fb8fe569e00 1 -- 10.20.193.28:0/712161460 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55b429ae9340 con 0x55b429c48440 2026-02-25T11:56:39.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.083+0000 7fb8a7fff640 1 -- 10.20.193.28:0/712161460 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb8e80cf520 con 0x55b429c48440 2026-02-25T11:56:39.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.083+0000 7fb8fe569e00 10 client.26959 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:39.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.083+0000 7fb8fe569e00 10 client.26959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:39.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.083+0000 7fb8fe569e00 20 client.26959 populate_metadata read hostname 'trial028' 2026-02-25T11:56:39.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.083+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/712161460 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b429d41ef0 0x55b429d623a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:39.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.083+0000 7fb8fe569e00 4 client.26959 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:56:39.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.083+0000 7fb8fe569e00 1 -- 10.20.193.28:0/712161460 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b4297e6f40 con 0x55b429d41ef0 2026-02-25T11:56:39.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.083+0000 7fb8fa2e3640 1 --2- 10.20.193.28:0/712161460 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b429d41ef0 0x55b429d623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:39.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8fa2e3640 1 --2- 10.20.193.28:0/712161460 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b429d41ef0 0x55b429d623a0 crc :-1 s=READY pgs=155 gs=4 cs=0 l=0 c_cookie=8060541137832749940 s_cookie=17763220782346046175 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fdccb7551f1ab74 server_cookie=f683a3301e928adf in_seq=0 out_seq=0 2026-02-25T11:56:39.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8a7fff640 10 client.26959 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:39.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8a7fff640 1 -- 10.20.193.28:0/712161460 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b4297e6f40 con 0x55b429d41ef0 2026-02-25T11:56:39.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8a7fff640 10 client.26959 handle_command_reply: tid=0 2026-02-25T11:56:39.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8fe569e00 1 client.26959 shutdown 2026-02-25T11:56:39.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8fe569e00 1 -- 10.20.193.28:0/712161460 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c48440 msgr2=0x55b429c80810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:39.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/712161460 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c48440 0x55b429c80810 secure :-1 s=READY pgs=217 gs=2 cs=0 l=1 c_cookie=4425375182027593409 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e8005b30 tx=0x7fb8e80d15f0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8fe569e00 1 -- 10.20.193.28:0/712161460 shutdown_connections 2026-02-25T11:56:39.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/712161460 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55b429d41ef0 0x55b429d623a0 crc :-1 s=READY pgs=155 gs=4 cs=0 l=0 c_cookie=8060541137832749940 s_cookie=17763220782346046175 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/712161460 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55b429c8d0e0 0x55b429c85170 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/712161460 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55b429c48440 0x55b429c80810 unknown :-1 s=CLOSED pgs=217 gs=2 cs=0 l=1 c_cookie=4425375182027593409 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8fe569e00 1 --2- 10.20.193.28:0/712161460 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55b429c47b30 0x55b429c802d0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8fe569e00 1 -- 10.20.193.28:0/712161460 >> 10.20.193.28:0/712161460 conn(0x55b429b667a0 msgr2=0x55b429c26400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:39.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8a7fff640 0 client.26959 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:39.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8fe569e00 1 -- 10.20.193.28:0/712161460 shutdown_connections 2026-02-25T11:56:39.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.084+0000 7fb8fe569e00 1 -- 10.20.193.28:0/712161460 wait complete. 2026-02-25T11:56:39.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.085+0000 7fb8fe569e00 20 client.26959 trim_cache size 0 max 16384 2026-02-25T11:56:39.093 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:56:39.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.085+0000 7fb8fb2e5640 1 -- 10.20.193.28:0/1101565657 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb8d00b5750 msgr2=0x7fb8d00d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:39.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.085+0000 7fb8fb2e5640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb8d00b5750 0x7fb8d00d5c00 secure :-1 s=READY pgs=369 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e4004740 tx=0x7fb8e4005f90 comp rx=0 tx=0).stop 2026-02-25T11:56:39.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.085+0000 7fb8fb2e5640 1 -- 10.20.193.28:0/1101565657 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb8f410ddc0 msgr2=0x7fb8f4173e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:39.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.085+0000 7fb8fb2e5640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb8f410ddc0 0x7fb8f4173e90 secure :-1 s=READY pgs=216 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb8e80098a0 tx=0x7fb8e80028d0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.085+0000 7fb8fb2e5640 1 -- 10.20.193.28:0/1101565657 shutdown_connections 2026-02-25T11:56:39.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.085+0000 7fb8fb2e5640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb8d00b5750 0x7fb8d00d5c00 unknown :-1 s=CLOSED pgs=369 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.085+0000 7fb8fb2e5640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb8f41135c0 0x7fb8f40f60a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.085+0000 7fb8fb2e5640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb8f410ddc0 0x7fb8f4173e90 unknown :-1 s=CLOSED pgs=216 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.085+0000 7fb8fb2e5640 1 --2- 10.20.193.28:0/1101565657 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb8f410d3d0 0x7fb8f40f59a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:39.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.085+0000 7fb8fb2e5640 1 -- 10.20.193.28:0/1101565657 >> 10.20.193.28:0/1101565657 conn(0x7fb8f40eaca0 msgr2=0x7fb8f410c150 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:39.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.085+0000 7fb8fb2e5640 1 -- 10.20.193.28:0/1101565657 shutdown_connections 2026-02-25T11:56:39.095 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:39.085+0000 7fb8fb2e5640 1 -- 10.20.193.28:0/1101565657 wait complete. 2026-02-25T11:56:39.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:39 trial167 ceph-mon[43392]: pgmap v479: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:56:39.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:39 trial119 ceph-mon[42710]: pgmap v479: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:56:39.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:39 trial028 ceph-mon[39224]: pgmap v479: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:56:40.186 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:56:40.187 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:56:40.373 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:40.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.546+0000 7f8b47577640 1 -- 10.20.193.28:0/4273088317 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b48115880 msgr2=0x7f8b48176250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:40.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.546+0000 7f8b47577640 1 --2- 10.20.193.28:0/4273088317 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b48115880 0x7f8b48176250 secure :-1 s=READY pgs=820 gs=1 cs=0 l=1 c_cookie=7626586581506524880 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b3800aa30 tx=0x7f8b38030280 comp rx=0 tx=0).stop 2026-02-25T11:56:40.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.546+0000 7f8b47577640 1 -- 10.20.193.28:0/4273088317 shutdown_connections 2026-02-25T11:56:40.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.546+0000 7f8b47577640 1 --2- 10.20.193.28:0/4273088317 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b48115880 0x7f8b48176250 unknown :-1 s=CLOSED pgs=820 gs=1 cs=0 l=1 c_cookie=7626586581506524880 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.546+0000 7f8b47577640 1 --2- 10.20.193.28:0/4273088317 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8b48110080 0x7f8b481150d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.546+0000 7f8b47577640 1 --2- 10.20.193.28:0/4273088317 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4810f690 0x7f8b4810fa60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.546+0000 7f8b47577640 1 -- 10.20.193.28:0/4273088317 >> 10.20.193.28:0/4273088317 conn(0x7f8b480ecf00 msgr2=0x7f8b4810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:40.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.546+0000 7f8b47577640 1 -- 10.20.193.28:0/4273088317 shutdown_connections 2026-02-25T11:56:40.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.546+0000 7f8b47577640 1 -- 10.20.193.28:0/4273088317 wait complete. 2026-02-25T11:56:40.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.547+0000 7f8b47577640 1 Processor -- start 2026-02-25T11:56:40.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.547+0000 7f8b47577640 1 -- start start 2026-02-25T11:56:40.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b47577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4810f690 0x7f8b4811e010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b47577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8b48110080 0x7f8b4811e550 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b47577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b48115880 0x7f8b48123920 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b47577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48123f40 con 0x7f8b48115880 2026-02-25T11:56:40.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b47577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8b481240b0 con 0x7f8b4810f690 2026-02-25T11:56:40.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b47577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b48124270 con 0x7f8b48110080 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b46575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4810f690 0x7f8b4811e010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b46575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4810f690 0x7f8b4811e010 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:39728/0 (socket says 10.20.193.28:39728) 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b46575640 1 -- 10.20.193.28:0/478800241 learned_addr learned my addr 10.20.193.28:0/478800241 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b45d74640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8b48110080 0x7f8b4811e550 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b46d76640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b48115880 0x7f8b48123920 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b46575640 1 -- 10.20.193.28:0/478800241 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8b48110080 msgr2=0x7f8b4811e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b46575640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8b48110080 0x7f8b4811e550 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b46575640 1 -- 10.20.193.28:0/478800241 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b48115880 msgr2=0x7f8b48123920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b46575640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b48115880 0x7f8b48123920 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b46575640 1 -- 10.20.193.28:0/478800241 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3800a650 con 0x7f8b4810f690 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.548+0000 7f8b46575640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4810f690 0x7f8b4811e010 secure :-1 s=READY pgs=169 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b3c009480 tx=0x7f8b3c00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.549+0000 7f8b2f7fe640 1 -- 10.20.193.28:0/478800241 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8b3c00ccc0 con 0x7f8b4810f690 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.549+0000 7f8b2f7fe640 1 -- 10.20.193.28:0/478800241 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8b3c00ce20 con 0x7f8b4810f690 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.549+0000 7f8b2f7fe640 1 -- 10.20.193.28:0/478800241 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8b3c0106b0 con 0x7f8b4810f690 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.549+0000 7f8b47577640 1 -- 10.20.193.28:0/478800241 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b48124550 con 0x7f8b4810f690 2026-02-25T11:56:40.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.549+0000 7f8b47577640 1 -- 10.20.193.28:0/478800241 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b48164c50 con 0x7f8b4810f690 2026-02-25T11:56:40.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.552+0000 7f8b2f7fe640 1 -- 10.20.193.28:0/478800241 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f8b3c01f060 con 0x7f8b4810f690 2026-02-25T11:56:40.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.554+0000 7f8b2f7fe640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8b200b5760 0x7f8b200d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.554+0000 7f8b2f7fe640 1 -- 10.20.193.28:0/478800241 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8b3c014070 con 0x7f8b4810f690 2026-02-25T11:56:40.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.554+0000 7f8b45d74640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8b200b5760 0x7f8b200d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.555+0000 7f8b45d74640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8b200b5760 0x7f8b200d5c10 secure :-1 s=READY pgs=370 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b481249b0 tx=0x7f8b30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:40.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.555+0000 7f8b4ee2fe00 1 Processor -- start 2026-02-25T11:56:40.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.555+0000 7f8b4ee2fe00 1 -- start start 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.555+0000 7f8b4ee2fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6684750 0x558cf6684b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.555+0000 7f8b4ee2fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf6685060 0x558cf66c9610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.555+0000 7f8b4ee2fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf66c9b50 0x558cf66e9f30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.555+0000 7f8b4ee2fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558cf659ad80 con 0x558cf6685060 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.555+0000 7f8b4ee2fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558cf6596ab0 con 0x558cf66c9b50 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.555+0000 7f8b4ee2fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558cf6590a50 con 0x558cf6684750 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.555+0000 7f8b45d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf6685060 0x558cf66c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.555+0000 7f8b45d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf6685060 0x558cf66c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52820/0 (socket says 10.20.193.28:52820) 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.555+0000 7f8b45d74640 1 -- 10.20.193.28:0/171563872 learned_addr learned my addr 10.20.193.28:0/171563872 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.556+0000 7f8b46d76640 1 --2- 10.20.193.28:0/171563872 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf66c9b50 0x558cf66e9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.556+0000 7f8b46575640 1 --2- 10.20.193.28:0/171563872 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6684750 0x558cf6684b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.556+0000 7f8b45d74640 1 -- 10.20.193.28:0/171563872 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6684750 msgr2=0x558cf6684b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.556+0000 7f8b45d74640 1 --2- 10.20.193.28:0/171563872 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6684750 0x558cf6684b20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.556+0000 7f8b45d74640 1 -- 10.20.193.28:0/171563872 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf66c9b50 msgr2=0x558cf66e9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.556+0000 7f8b45d74640 1 --2- 10.20.193.28:0/171563872 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf66c9b50 0x558cf66e9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.556+0000 7f8b45d74640 1 -- 10.20.193.28:0/171563872 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cf653c120 con 0x558cf6685060 2026-02-25T11:56:40.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.556+0000 7f8b45d74640 1 --2- 10.20.193.28:0/171563872 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf6685060 0x558cf66c9610 secure :-1 s=READY pgs=821 gs=1 cs=0 l=1 c_cookie=14263953081213515860 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b30004200 tx=0x7f8b30014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f3bdc0f291f854 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:40.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.556+0000 7f8b2cff9640 1 -- 10.20.193.28:0/171563872 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8b30011e60 con 0x558cf6685060 2026-02-25T11:56:40.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.556+0000 7f8b2cff9640 1 -- 10.20.193.28:0/171563872 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8b30009d70 con 0x558cf6685060 2026-02-25T11:56:40.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.556+0000 7f8b2cff9640 1 -- 10.20.193.28:0/171563872 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8b3001b8a0 con 0x558cf6685060 2026-02-25T11:56:40.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.557+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/171563872 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf6685060 msgr2=0x558cf66c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:40.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.557+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/171563872 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf6685060 0x558cf66c9610 secure :-1 s=READY pgs=821 gs=1 cs=0 l=1 c_cookie=14263953081213515860 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b30004200 tx=0x7f8b30014000 comp rx=0 tx=0).stop 2026-02-25T11:56:40.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.557+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/171563872 shutdown_connections 2026-02-25T11:56:40.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.557+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/171563872 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf66c9b50 0x558cf66e9f30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.557+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/171563872 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf6685060 0x558cf66c9610 unknown :-1 s=CLOSED pgs=821 gs=1 cs=0 l=1 c_cookie=14263953081213515860 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.557+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/171563872 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6684750 0x558cf6684b20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.557+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/171563872 >> 10.20.193.28:0/171563872 conn(0x558cf66628f0 msgr2=0x558cf6682cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:40.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.557+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/171563872 shutdown_connections 2026-02-25T11:56:40.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.557+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/171563872 wait complete. 2026-02-25T11:56:40.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.559+0000 7f8b4ee2fe00 1 -- start start 2026-02-25T11:56:40.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.559+0000 7f8b4ee2fe00 1 Processor -- start 2026-02-25T11:56:40.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.559+0000 7f8b4ee2fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6684750 0x558cf66856f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.559+0000 7f8b4ee2fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6685060 0x558cf6685c30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.559+0000 7f8b4ee2fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf66c9b50 0x558cf668a600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.559+0000 7f8b4ee2fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558cf668ab40 con 0x558cf66c9b50 2026-02-25T11:56:40.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.559+0000 7f8b4ee2fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558cf668acb0 con 0x558cf6685060 2026-02-25T11:56:40.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.559+0000 7f8b4ee2fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558cf668ae70 con 0x558cf6684750 2026-02-25T11:56:40.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.560+0000 7f8b45d74640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6685060 0x558cf6685c30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.560+0000 7f8b45d74640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6685060 0x558cf6685c30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:39750/0 (socket says 10.20.193.28:39750) 2026-02-25T11:56:40.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.560+0000 7f8b45d74640 1 -- 10.20.193.28:0/2198984859 learned_addr learned my addr 10.20.193.28:0/2198984859 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:40.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.560+0000 7f8b46575640 1 --2- 10.20.193.28:0/2198984859 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6684750 0x558cf66856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.560+0000 7f8b46d76640 1 --2- 10.20.193.28:0/2198984859 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf66c9b50 0x558cf668a600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.560+0000 7f8b45d74640 1 -- 10.20.193.28:0/2198984859 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6684750 msgr2=0x558cf66856f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.560+0000 7f8b45d74640 1 --2- 10.20.193.28:0/2198984859 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6684750 0x558cf66856f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.560+0000 7f8b45d74640 1 -- 10.20.193.28:0/2198984859 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf66c9b50 msgr2=0x558cf668a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.560+0000 7f8b45d74640 1 --2- 10.20.193.28:0/2198984859 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf66c9b50 0x558cf668a600 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.560+0000 7f8b45d74640 1 -- 10.20.193.28:0/2198984859 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cf6514810 con 0x558cf6685060 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.560+0000 7f8b45d74640 1 --2- 10.20.193.28:0/2198984859 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6685060 0x558cf6685c30 secure :-1 s=READY pgs=170 gs=2 cs=0 l=1 c_cookie=8935926665937733866 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b3001bc90 tx=0x7f8b3001bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c02c9f54628c8ea server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.560+0000 7f8af7fff640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.561+0000 7f8af7fff640 1 -- 10.20.193.28:0/2198984859 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8b3001ae00 con 0x558cf6685060 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.561+0000 7f8af7fff640 1 -- 10.20.193.28:0/2198984859 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8b3001a460 con 0x558cf6685060 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.561+0000 7f8af7fff640 1 -- 10.20.193.28:0/2198984859 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8b30004980 con 0x558cf6685060 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.561+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2198984859 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b30009940 con 0x558cf6685060 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.561+0000 7f8af7fff640 1 -- 10.20.193.28:0/2198984859 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8b30004b10 con 0x558cf6685060 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.561+0000 7f8b4ee2fe00 10 client.26963 _ll_register_callbacks cb 0x558cf6511e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:40.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.561+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2198984859 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558cf62e8530 con 0x558cf6685060 2026-02-25T11:56:40.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.561+0000 7f8af7fff640 1 -- 10.20.193.28:0/2198984859 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8b30004cd0 con 0x558cf6685060 2026-02-25T11:56:40.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.561+0000 7f8b4ee2fe00 10 client.26963 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:40.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.561+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2198984859 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x558cf65264f0 con 0x558cf6685060 2026-02-25T11:56:40.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.561+0000 7f8af7fff640 1 -- 10.20.193.28:0/2198984859 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f8b30005260 con 0x558cf6685060 2026-02-25T11:56:40.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.562+0000 7f8b4ee2fe00 10 client.26963 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:40.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.562+0000 7f8b4ee2fe00 10 client.26963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:40.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.562+0000 7f8b4ee2fe00 20 client.26963 populate_metadata read hostname 'trial028' 2026-02-25T11:56:40.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.562+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/2198984859 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558cf66ce770 0x558cf66cec10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.562+0000 7f8b4ee2fe00 4 client.26963 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:56:40.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.562+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2198984859 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cf653db50 con 0x558cf66ce770 2026-02-25T11:56:40.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.562+0000 7f8b46575640 1 --2- 10.20.193.28:0/2198984859 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558cf66ce770 0x558cf66cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.562+0000 7f8b46575640 1 --2- 10.20.193.28:0/2198984859 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558cf66ce770 0x558cf66cec10 crc :-1 s=READY pgs=156 gs=4 cs=0 l=0 c_cookie=4153641843555716845 s_cookie=2071728225385787706 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a4b33ff1564eed server_cookie=1cc041d83145313a in_seq=0 out_seq=0 2026-02-25T11:56:40.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.562+0000 7f8af7fff640 10 client.26963 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:40.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.563+0000 7f8af7fff640 1 -- 10.20.193.28:0/2198984859 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x558cf653db50 con 0x558cf66ce770 2026-02-25T11:56:40.563 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.563+0000 7f8af7fff640 10 client.26963 handle_command_reply: tid=0 2026-02-25T11:56:40.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.563+0000 7f8b4ee2fe00 1 client.26963 shutdown 2026-02-25T11:56:40.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.563+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2198984859 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6685060 msgr2=0x558cf6685c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:40.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.563+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/2198984859 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6685060 0x558cf6685c30 secure :-1 s=READY pgs=170 gs=2 cs=0 l=1 c_cookie=8935926665937733866 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b3001bc90 tx=0x7f8b3001bcc0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.563+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2198984859 shutdown_connections 2026-02-25T11:56:40.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.563+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/2198984859 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558cf66ce770 0x558cf66cec10 crc :-1 s=READY pgs=156 gs=4 cs=0 l=0 c_cookie=4153641843555716845 s_cookie=2071728225385787706 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.563+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/2198984859 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf66c9b50 0x558cf668a600 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.563+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/2198984859 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6685060 0x558cf6685c30 unknown :-1 s=CLOSED pgs=170 gs=2 cs=0 l=1 c_cookie=8935926665937733866 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.563+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/2198984859 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6684750 0x558cf66856f0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.563+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2198984859 >> 10.20.193.28:0/2198984859 conn(0x558cf66628f0 msgr2=0x558cf6682cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:40.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.564+0000 7f8af7fff640 0 client.26963 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:40.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.564+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2198984859 shutdown_connections 2026-02-25T11:56:40.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.564+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2198984859 wait complete. 2026-02-25T11:56:40.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.564+0000 7f8b4ee2fe00 20 client.26963 trim_cache size 0 max 16384 2026-02-25T11:56:40.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b4ee2fe00 1 Processor -- start 2026-02-25T11:56:40.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b4ee2fe00 1 -- start start 2026-02-25T11:56:40.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b4ee2fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6684750 0x558cf6685540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b4ee2fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6685060 0x558cf6685a80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b4ee2fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf66c9b50 0x558cf66e7220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b4ee2fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558cf658e680 con 0x558cf66c9b50 2026-02-25T11:56:40.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b4ee2fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558cf624e790 con 0x558cf6684750 2026-02-25T11:56:40.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b4ee2fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558cf6686230 con 0x558cf6685060 2026-02-25T11:56:40.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b46d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf66c9b50 0x558cf66e7220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b46d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf66c9b50 0x558cf66e7220 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52830/0 (socket says 10.20.193.28:52830) 2026-02-25T11:56:40.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b46d76640 1 -- 10.20.193.28:0/1283030130 learned_addr learned my addr 10.20.193.28:0/1283030130 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:40.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b45d74640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6685060 0x558cf6685a80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b46575640 1 --2- 10.20.193.28:0/1283030130 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6684750 0x558cf6685540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b46d76640 1 -- 10.20.193.28:0/1283030130 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6685060 msgr2=0x558cf6685a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:40.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b46d76640 1 --2- 10.20.193.28:0/1283030130 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6685060 0x558cf6685a80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b46d76640 1 -- 10.20.193.28:0/1283030130 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6684750 msgr2=0x558cf6685540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:40.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b46d76640 1 --2- 10.20.193.28:0/1283030130 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6684750 0x558cf6685540 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.573+0000 7f8b46d76640 1 -- 10.20.193.28:0/1283030130 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cf66dc500 con 0x558cf66c9b50 2026-02-25T11:56:40.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.574+0000 7f8b46d76640 1 --2- 10.20.193.28:0/1283030130 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf66c9b50 0x558cf66e7220 secure :-1 s=READY pgs=822 gs=3 cs=0 l=1 c_cookie=8601817748716856095 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b38006db0 tx=0x7f8b3803b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775fcbab06eb231f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:40.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.574+0000 7f8b124e6640 1 -- 10.20.193.28:0/1283030130 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8b3803e040 con 0x558cf66c9b50 2026-02-25T11:56:40.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.574+0000 7f8b124e6640 1 -- 10.20.193.28:0/1283030130 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8b38030ca0 con 0x558cf66c9b50 2026-02-25T11:56:40.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.574+0000 7f8b124e6640 1 -- 10.20.193.28:0/1283030130 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8b38046460 con 0x558cf66c9b50 2026-02-25T11:56:40.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.574+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/1283030130 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf66c9b50 msgr2=0x558cf66e7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:40.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.574+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/1283030130 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf66c9b50 0x558cf66e7220 secure :-1 s=READY pgs=822 gs=3 cs=0 l=1 c_cookie=8601817748716856095 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b38006db0 tx=0x7f8b3803b040 comp rx=0 tx=0).stop 2026-02-25T11:56:40.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.574+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/1283030130 shutdown_connections 2026-02-25T11:56:40.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.574+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/1283030130 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf66c9b50 0x558cf66e7220 unknown :-1 s=CLOSED pgs=822 gs=3 cs=0 l=1 c_cookie=8601817748716856095 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.574+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/1283030130 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf6685060 0x558cf6685a80 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.574+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/1283030130 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6684750 0x558cf6685540 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.574+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/1283030130 >> 10.20.193.28:0/1283030130 conn(0x558cf66628f0 msgr2=0x558cf66c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:40.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.574+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/1283030130 shutdown_connections 2026-02-25T11:56:40.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.575+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/1283030130 wait complete. 2026-02-25T11:56:40.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.575+0000 7f8b4ee2fe00 1 -- start start 2026-02-25T11:56:40.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.575+0000 7f8b4ee2fe00 1 Processor -- start 2026-02-25T11:56:40.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b4ee2fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf6684750 0x558cf668c090 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b4ee2fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6685060 0x558cf668c5d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b4ee2fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf66c9b50 0x558cf6786a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b4ee2fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558cf6786fa0 con 0x558cf6684750 2026-02-25T11:56:40.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b4ee2fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558cf67870e0 con 0x558cf6685060 2026-02-25T11:56:40.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b4ee2fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558cf6787220 con 0x558cf66c9b50 2026-02-25T11:56:40.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b45d74640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6685060 0x558cf668c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b46d76640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf66c9b50 0x558cf6786a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b46d76640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf66c9b50 0x558cf6786a60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34872/0 (socket says 10.20.193.28:34872) 2026-02-25T11:56:40.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b46d76640 1 -- 10.20.193.28:0/2022320107 learned_addr learned my addr 10.20.193.28:0/2022320107 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:40.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b46575640 1 --2- 10.20.193.28:0/2022320107 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf6684750 0x558cf668c090 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b46d76640 1 -- 10.20.193.28:0/2022320107 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6685060 msgr2=0x558cf668c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:40.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b46d76640 1 --2- 10.20.193.28:0/2022320107 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6685060 0x558cf668c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b46d76640 1 -- 10.20.193.28:0/2022320107 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf6684750 msgr2=0x558cf668c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:40.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b46d76640 1 --2- 10.20.193.28:0/2022320107 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf6684750 0x558cf668c090 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b46d76640 1 -- 10.20.193.28:0/2022320107 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cf6520f70 con 0x558cf66c9b50 2026-02-25T11:56:40.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b46575640 1 --2- 10.20.193.28:0/2022320107 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf6684750 0x558cf668c090 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:56:40.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8b46d76640 1 --2- 10.20.193.28:0/2022320107 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf66c9b50 0x558cf6786a60 secure :-1 s=READY pgs=218 gs=1 cs=0 l=1 c_cookie=17329068076153022805 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b38030e30 tx=0x7f8b38006eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f07d379a17a9a955 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:40.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.576+0000 7f8af7fff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:56:40.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.577+0000 7f8af7fff640 1 -- 10.20.193.28:0/2022320107 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8b38046460 con 0x558cf66c9b50 2026-02-25T11:56:40.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.577+0000 7f8af7fff640 1 -- 10.20.193.28:0/2022320107 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8b38005270 con 0x558cf66c9b50 2026-02-25T11:56:40.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.577+0000 7f8af7fff640 1 -- 10.20.193.28:0/2022320107 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8b38046870 con 0x558cf66c9b50 2026-02-25T11:56:40.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.577+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2022320107 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b48176ac0 con 0x558cf66c9b50 2026-02-25T11:56:40.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.577+0000 7f8b4ee2fe00 10 client.26977 _ll_register_callbacks cb 0x558cf6511e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:40.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.577+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2022320107 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b30009d70 con 0x558cf66c9b50 2026-02-25T11:56:40.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.577+0000 7f8af7fff640 1 -- 10.20.193.28:0/2022320107 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8b38039d90 con 0x558cf66c9b50 2026-02-25T11:56:40.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.577+0000 7f8af7fff640 1 -- 10.20.193.28:0/2022320107 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8b38003780 con 0x558cf66c9b50 2026-02-25T11:56:40.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.577+0000 7f8b4ee2fe00 10 client.26977 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:40.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.577+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2022320107 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x558cf6514b40 con 0x558cf66c9b50 2026-02-25T11:56:40.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.578+0000 7f8af7fff640 1 -- 10.20.193.28:0/2022320107 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f8b38044040 con 0x558cf66c9b50 2026-02-25T11:56:40.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.578+0000 7f8b4ee2fe00 10 client.26977 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:40.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.578+0000 7f8b4ee2fe00 10 client.26977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:40.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.578+0000 7f8b4ee2fe00 20 client.26977 populate_metadata read hostname 'trial028' 2026-02-25T11:56:40.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.578+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/2022320107 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558cf6788d70 0x558cf67a9220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:40.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.578+0000 7f8b4ee2fe00 4 client.26977 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:56:40.584 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.578+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2022320107 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558cf63f1690 con 0x558cf6788d70 2026-02-25T11:56:40.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.578+0000 7f8b46575640 1 --2- 10.20.193.28:0/2022320107 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558cf6788d70 0x558cf67a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:40.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8b46575640 1 --2- 10.20.193.28:0/2022320107 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558cf6788d70 0x558cf67a9220 crc :-1 s=READY pgs=157 gs=4 cs=0 l=0 c_cookie=1296457345177896567 s_cookie=1625198024518546485 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11fdf11625c64677 server_cookie=168ddcfe1d183435 in_seq=0 out_seq=0 2026-02-25T11:56:40.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8af7fff640 10 client.26977 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:40.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8af7fff640 1 -- 10.20.193.28:0/2022320107 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558cf63f1690 con 0x558cf6788d70 2026-02-25T11:56:40.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8af7fff640 10 client.26977 handle_command_reply: tid=0 2026-02-25T11:56:40.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8b4ee2fe00 1 client.26977 shutdown 2026-02-25T11:56:40.585 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2022320107 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf66c9b50 msgr2=0x558cf6786a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:40.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/2022320107 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf66c9b50 0x558cf6786a60 secure :-1 s=READY pgs=218 gs=1 cs=0 l=1 c_cookie=17329068076153022805 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b38030e30 tx=0x7f8b38006eb0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2022320107 shutdown_connections 2026-02-25T11:56:40.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/2022320107 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x558cf6788d70 0x558cf67a9220 crc :-1 s=READY pgs=157 gs=4 cs=0 l=0 c_cookie=1296457345177896567 s_cookie=1625198024518546485 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/2022320107 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558cf66c9b50 0x558cf6786a60 unknown :-1 s=CLOSED pgs=218 gs=1 cs=0 l=1 c_cookie=17329068076153022805 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/2022320107 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558cf6685060 0x558cf668c5d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8b4ee2fe00 1 --2- 10.20.193.28:0/2022320107 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558cf6684750 0x558cf668c090 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8af7fff640 0 client.26977 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:40.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2022320107 >> 10.20.193.28:0/2022320107 conn(0x558cf66628f0 msgr2=0x558cf6685540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:40.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2022320107 shutdown_connections 2026-02-25T11:56:40.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.579+0000 7f8b4ee2fe00 1 -- 10.20.193.28:0/2022320107 wait complete. 2026-02-25T11:56:40.586 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.580+0000 7f8b4ee2fe00 20 client.26977 trim_cache size 0 max 16384 2026-02-25T11:56:40.587 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:56:40.587 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:56:40.587 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "91e989b7-1f1f-41fb-9047-642867de91e6", 2026-02-25T11:56:40.587 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:56:40.587 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:56:40.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.580+0000 7f8b47577640 1 -- 10.20.193.28:0/478800241 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8b200b5760 msgr2=0x7f8b200d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:40.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.580+0000 7f8b47577640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8b200b5760 0x7f8b200d5c10 secure :-1 s=READY pgs=370 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b481249b0 tx=0x7f8b30005f90 comp rx=0 tx=0).stop 2026-02-25T11:56:40.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.580+0000 7f8b47577640 1 -- 10.20.193.28:0/478800241 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4810f690 msgr2=0x7f8b4811e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:40.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.580+0000 7f8b47577640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4810f690 0x7f8b4811e010 secure :-1 s=READY pgs=169 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8b3c009480 tx=0x7f8b3c00ee60 comp rx=0 tx=0).stop 2026-02-25T11:56:40.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.580+0000 7f8b47577640 1 -- 10.20.193.28:0/478800241 shutdown_connections 2026-02-25T11:56:40.588 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.580+0000 7f8b47577640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8b200b5760 0x7f8b200d5c10 unknown :-1 s=CLOSED pgs=370 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.580+0000 7f8b47577640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8b48115880 0x7f8b48123920 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.580+0000 7f8b47577640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8b48110080 0x7f8b4811e550 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.580+0000 7f8b47577640 1 --2- 10.20.193.28:0/478800241 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8b4810f690 0x7f8b4811e010 unknown :-1 s=CLOSED pgs=169 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:40.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.580+0000 7f8b47577640 1 -- 10.20.193.28:0/478800241 >> 10.20.193.28:0/478800241 conn(0x7f8b480ecf00 msgr2=0x7f8b4810d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:40.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.580+0000 7f8b47577640 1 -- 10.20.193.28:0/478800241 shutdown_connections 2026-02-25T11:56:40.589 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:40.580+0000 7f8b47577640 1 -- 10.20.193.28:0/478800241 wait complete. 2026-02-25T11:56:40.682 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:56:40.866 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:56:41.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.045+0000 7f6074a24640 1 -- 10.20.193.28:0/4152694210 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6070115300 msgr2=0x7f6070119720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:41.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.045+0000 7f6074a24640 1 --2- 10.20.193.28:0/4152694210 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6070115300 0x7f6070119720 secure :-1 s=READY pgs=823 gs=1 cs=0 l=1 c_cookie=6107456935141354921 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6058009950 tx=0x7f605802f190 comp rx=0 tx=0).stop 2026-02-25T11:56:41.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.045+0000 7f6074a24640 1 -- 10.20.193.28:0/4152694210 shutdown_connections 2026-02-25T11:56:41.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.045+0000 7f6074a24640 1 --2- 10.20.193.28:0/4152694210 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6070119eb0 0x7f607011a350 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.045+0000 7f6074a24640 1 --2- 10.20.193.28:0/4152694210 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6070115300 0x7f6070119720 unknown :-1 s=CLOSED pgs=823 gs=1 cs=0 l=1 c_cookie=6107456935141354921 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.045+0000 7f6074a24640 1 --2- 10.20.193.28:0/4152694210 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6070114960 0x7f6070114d30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.045+0000 7f6074a24640 1 -- 10.20.193.28:0/4152694210 >> 10.20.193.28:0/4152694210 conn(0x7f607005be90 msgr2=0x7f607005c290 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:41.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.045+0000 7f6074a24640 1 -- 10.20.193.28:0/4152694210 shutdown_connections 2026-02-25T11:56:41.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.045+0000 7f6074a24640 1 -- 10.20.193.28:0/4152694210 wait complete. 2026-02-25T11:56:41.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.045+0000 7f6074a24640 1 Processor -- start 2026-02-25T11:56:41.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f6074a24640 1 -- start start 2026-02-25T11:56:41.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f6074a24640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6070114960 0x7f607012c380 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f6074a24640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6070119eb0 0x7f607012c8c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f6074a24640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6070131c90 0x7f6070132130 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f6074a24640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f60701326a0 con 0x7f6070119eb0 2026-02-25T11:56:41.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f6074a24640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6070132810 con 0x7f6070131c90 2026-02-25T11:56:41.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f6074a24640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f60701329d0 con 0x7f6070114960 2026-02-25T11:56:41.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f606e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6070119eb0 0x7f607012c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f606e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6070119eb0 0x7f607012c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52850/0 (socket says 10.20.193.28:52850) 2026-02-25T11:56:41.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f606e575640 1 -- 10.20.193.28:0/1191713222 learned_addr learned my addr 10.20.193.28:0/1191713222 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:41.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f606ed76640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6070114960 0x7f607012c380 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f606f577640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6070131c90 0x7f6070132130 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f606e575640 1 -- 10.20.193.28:0/1191713222 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6070114960 msgr2=0x7f607012c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:41.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f606e575640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6070114960 0x7f607012c380 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f606e575640 1 -- 10.20.193.28:0/1191713222 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6070131c90 msgr2=0x7f6070132130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:41.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f606e575640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6070131c90 0x7f6070132130 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.046+0000 7f606e575640 1 -- 10.20.193.28:0/1191713222 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6058009570 con 0x7f6070119eb0 2026-02-25T11:56:41.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.047+0000 7f606e575640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6070119eb0 0x7f607012c8c0 secure :-1 s=READY pgs=824 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60580098a0 tx=0x7f60580028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:41.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.047+0000 7f604ffff640 1 -- 10.20.193.28:0/1191713222 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f605803d070 con 0x7f6070119eb0 2026-02-25T11:56:41.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.047+0000 7f604ffff640 1 -- 10.20.193.28:0/1191713222 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6058002a70 con 0x7f6070119eb0 2026-02-25T11:56:41.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.047+0000 7f604ffff640 1 -- 10.20.193.28:0/1191713222 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6058041890 con 0x7f6070119eb0 2026-02-25T11:56:41.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.047+0000 7f6074a24640 1 -- 10.20.193.28:0/1191713222 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6070172e20 con 0x7f6070119eb0 2026-02-25T11:56:41.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.047+0000 7f6074a24640 1 -- 10.20.193.28:0/1191713222 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6070173240 con 0x7f6070119eb0 2026-02-25T11:56:41.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.050+0000 7f604ffff640 1 -- 10.20.193.28:0/1191713222 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f6058041a20 con 0x7f6070119eb0 2026-02-25T11:56:41.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.052+0000 7f604ffff640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60440b5760 0x7f60440d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.052+0000 7f604ffff640 1 -- 10.20.193.28:0/1191713222 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f605810b130 con 0x7f6070119eb0 2026-02-25T11:56:41.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.052+0000 7f606ed76640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60440b5760 0x7f60440d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.052+0000 7f606ed76640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60440b5760 0x7f60440d5c10 secure :-1 s=READY pgs=371 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6064004740 tx=0x7f60640073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:41.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f6077220e00 1 Processor -- start 2026-02-25T11:56:41.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f6077220e00 1 -- start start 2026-02-25T11:56:41.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f6077220e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baba140 0x560e0baba510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f6077220e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0babaa50 0x560e0baff1b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f6077220e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0baff6f0 0x560e0bb1fad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f6077220e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x560e0b9d09b0 con 0x560e0baff6f0 2026-02-25T11:56:41.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f6077220e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560e0b9cc6e0 con 0x560e0baba140 2026-02-25T11:56:41.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f6077220e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560e0b9c6840 con 0x560e0babaa50 2026-02-25T11:56:41.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f606f577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0baff6f0 0x560e0bb1fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f606e575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0babaa50 0x560e0baff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f606f577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0baff6f0 0x560e0bb1fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52864/0 (socket says 10.20.193.28:52864) 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f606f577640 1 -- 10.20.193.28:0/888930806 learned_addr learned my addr 10.20.193.28:0/888930806 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f606ed76640 1 --2- 10.20.193.28:0/888930806 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baba140 0x560e0baba510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f606f577640 1 -- 10.20.193.28:0/888930806 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0babaa50 msgr2=0x560e0baff1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f606f577640 1 --2- 10.20.193.28:0/888930806 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0babaa50 0x560e0baff1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f606f577640 1 -- 10.20.193.28:0/888930806 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baba140 msgr2=0x560e0baba510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f606f577640 1 --2- 10.20.193.28:0/888930806 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baba140 0x560e0baba510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.053+0000 7f606f577640 1 -- 10.20.193.28:0/888930806 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e0b971d80 con 0x560e0baff6f0 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.054+0000 7f606f577640 1 --2- 10.20.193.28:0/888930806 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0baff6f0 0x560e0bb1fad0 secure :-1 s=READY pgs=825 gs=3 cs=0 l=1 c_cookie=3270631194181797063 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6060006a50 tx=0x7f6060003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d639dbe188484c7 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.054+0000 7f603b7fe640 1 -- 10.20.193.28:0/888930806 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6060010c20 con 0x560e0baff6f0 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.054+0000 7f603b7fe640 1 -- 10.20.193.28:0/888930806 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6060010d80 con 0x560e0baff6f0 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.054+0000 7f603b7fe640 1 -- 10.20.193.28:0/888930806 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f606001a810 con 0x560e0baff6f0 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.054+0000 7f6077220e00 1 -- 10.20.193.28:0/888930806 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0baff6f0 msgr2=0x560e0bb1fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.054+0000 7f6077220e00 1 --2- 10.20.193.28:0/888930806 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0baff6f0 0x560e0bb1fad0 secure :-1 s=READY pgs=825 gs=3 cs=0 l=1 c_cookie=3270631194181797063 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6060006a50 tx=0x7f6060003e70 comp rx=0 tx=0).stop 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.054+0000 7f6077220e00 1 -- 10.20.193.28:0/888930806 shutdown_connections 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.054+0000 7f6077220e00 1 --2- 10.20.193.28:0/888930806 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0baff6f0 0x560e0bb1fad0 unknown :-1 s=CLOSED pgs=825 gs=3 cs=0 l=1 c_cookie=3270631194181797063 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.054+0000 7f6077220e00 1 --2- 10.20.193.28:0/888930806 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0babaa50 0x560e0baff1b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.054+0000 7f6077220e00 1 --2- 10.20.193.28:0/888930806 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baba140 0x560e0baba510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.054+0000 7f6077220e00 1 -- 10.20.193.28:0/888930806 >> 10.20.193.28:0/888930806 conn(0x560e0b9d9d40 msgr2=0x560e0bab8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:41.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.054+0000 7f6077220e00 1 -- 10.20.193.28:0/888930806 shutdown_connections 2026-02-25T11:56:41.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.054+0000 7f6077220e00 1 -- 10.20.193.28:0/888930806 wait complete. 2026-02-25T11:56:41.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.056+0000 7f6077220e00 1 -- start start 2026-02-25T11:56:41.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.056+0000 7f6077220e00 1 Processor -- start 2026-02-25T11:56:41.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.056+0000 7f6077220e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baba140 0x560e0bb17430 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.056+0000 7f6077220e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0babaa50 0x560e0bb17970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.056+0000 7f6077220e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0baff6f0 0x560e0bb1c960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.056+0000 7f6077220e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x560e0bb1cea0 con 0x560e0baff6f0 2026-02-25T11:56:41.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.056+0000 7f6077220e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560e0bb1d010 con 0x560e0babaa50 2026-02-25T11:56:41.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.056+0000 7f6077220e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560e0bb1d1d0 con 0x560e0baba140 2026-02-25T11:56:41.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.056+0000 7f606ed76640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baba140 0x560e0bb17430 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.056+0000 7f606ed76640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baba140 0x560e0bb17430 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:34920/0 (socket says 10.20.193.28:34920) 2026-02-25T11:56:41.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.056+0000 7f606ed76640 1 -- 10.20.193.28:0/689589757 learned_addr learned my addr 10.20.193.28:0/689589757 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:41.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f606e575640 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0babaa50 0x560e0bb17970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f606f577640 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0baff6f0 0x560e0bb1c960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f606ed76640 1 -- 10.20.193.28:0/689589757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0babaa50 msgr2=0x560e0bb17970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:41.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f606ed76640 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0babaa50 0x560e0bb17970 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f606ed76640 1 -- 10.20.193.28:0/689589757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0baff6f0 msgr2=0x560e0bb1c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:41.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f606ed76640 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0baff6f0 0x560e0bb1c960 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f606ed76640 1 -- 10.20.193.28:0/689589757 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e0b94a350 con 0x560e0baba140 2026-02-25T11:56:41.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f606f577640 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0baff6f0 0x560e0bb1c960 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:56:41.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f606ed76640 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baba140 0x560e0bb17430 secure :-1 s=READY pgs=219 gs=1 cs=0 l=1 c_cookie=3677256421368913240 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60640079d0 tx=0x7f60640047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33083d3b77b7dd58 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:41.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f6038ff9640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f6038ff9640 1 -- 10.20.193.28:0/689589757 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f606401d630 con 0x560e0baba140 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f6077220e00 1 -- 10.20.193.28:0/689589757 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60600055b0 con 0x560e0baba140 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f6077220e00 10 client.26989 _ll_register_callbacks cb 0x560e0b9479a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f6077220e00 1 -- 10.20.193.28:0/689589757 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e0b65c6f0 con 0x560e0baba140 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.057+0000 7f6038ff9640 1 -- 10.20.193.28:0/689589757 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6064015070 con 0x560e0baba140 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.058+0000 7f6038ff9640 1 -- 10.20.193.28:0/689589757 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f606401caf0 con 0x560e0baba140 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.058+0000 7f6038ff9640 1 -- 10.20.193.28:0/689589757 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f606401cc80 con 0x560e0baba140 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.058+0000 7f6038ff9640 1 -- 10.20.193.28:0/689589757 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f606401ce80 con 0x560e0baba140 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.058+0000 7f6077220e00 10 client.26989 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.058+0000 7f6077220e00 1 -- 10.20.193.28:0/689589757 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x560e0b95c150 con 0x560e0baba140 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.058+0000 7f6038ff9640 1 -- 10.20.193.28:0/689589757 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f606401d790 con 0x560e0baba140 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.058+0000 7f6077220e00 10 client.26989 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.058+0000 7f6077220e00 10 client.26989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.058+0000 7f6077220e00 20 client.26989 populate_metadata read hostname 'trial028' 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.058+0000 7f6077220e00 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560e0bb1fa80 0x560e0bafd420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.058+0000 7f6077220e00 4 client.26989 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:56:41.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.058+0000 7f6077220e00 1 -- 10.20.193.28:0/689589757 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e0b998910 con 0x560e0bb1fa80 2026-02-25T11:56:41.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.059+0000 7f606e575640 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560e0bb1fa80 0x560e0bafd420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.059+0000 7f606e575640 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560e0bb1fa80 0x560e0bafd420 crc :-1 s=READY pgs=158 gs=4 cs=0 l=0 c_cookie=1174465568994634144 s_cookie=3004908924228299337 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=104c8a3660da29a0 server_cookie=29b394bef3076a49 in_seq=0 out_seq=0 2026-02-25T11:56:41.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.059+0000 7f6038ff9640 10 client.26989 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:41.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6038ff9640 1 -- 10.20.193.28:0/689589757 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x560e0b998910 con 0x560e0bb1fa80 2026-02-25T11:56:41.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6038ff9640 10 client.26989 handle_command_reply: tid=0 2026-02-25T11:56:41.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6077220e00 1 client.26989 shutdown 2026-02-25T11:56:41.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6077220e00 1 -- 10.20.193.28:0/689589757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baba140 msgr2=0x560e0bb17430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:41.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6077220e00 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baba140 0x560e0bb17430 secure :-1 s=READY pgs=219 gs=1 cs=0 l=1 c_cookie=3677256421368913240 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60640079d0 tx=0x7f60640047e0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6077220e00 1 -- 10.20.193.28:0/689589757 shutdown_connections 2026-02-25T11:56:41.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6077220e00 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560e0bb1fa80 0x560e0bafd420 crc :-1 s=READY pgs=158 gs=4 cs=0 l=0 c_cookie=1174465568994634144 s_cookie=3004908924228299337 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6077220e00 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0baff6f0 0x560e0bb1c960 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6077220e00 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0babaa50 0x560e0bb17970 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6077220e00 1 --2- 10.20.193.28:0/689589757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baba140 0x560e0bb17430 unknown :-1 s=CLOSED pgs=219 gs=1 cs=0 l=1 c_cookie=3677256421368913240 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6038ff9640 0 client.26989 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:41.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6077220e00 1 -- 10.20.193.28:0/689589757 >> 10.20.193.28:0/689589757 conn(0x560e0b9d9d40 msgr2=0x560e0bab8230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:41.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6077220e00 1 -- 10.20.193.28:0/689589757 shutdown_connections 2026-02-25T11:56:41.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6077220e00 1 -- 10.20.193.28:0/689589757 wait complete. 2026-02-25T11:56:41.062 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.060+0000 7f6077220e00 20 client.26989 trim_cache size 0 max 16384 2026-02-25T11:56:41.070 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.069+0000 7f6077220e00 1 Processor -- start 2026-02-25T11:56:41.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f6077220e00 1 -- start start 2026-02-25T11:56:41.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f6077220e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baba140 0x560e0bb021e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f6077220e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0babaa50 0x560e0bb14600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f6077220e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baff6f0 0x560e0bb14b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f6077220e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x560e0b9c4470 con 0x560e0babaa50 2026-02-25T11:56:41.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f6077220e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560e0b684700 con 0x560e0baba140 2026-02-25T11:56:41.071 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f6077220e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560e0baba750 con 0x560e0baff6f0 2026-02-25T11:56:41.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f606e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0babaa50 0x560e0bb14600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f606f577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baff6f0 0x560e0bb14b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.072 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f606e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0babaa50 0x560e0bb14600 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52882/0 (socket says 10.20.193.28:52882) 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f606e575640 1 -- 10.20.193.28:0/2688331378 learned_addr learned my addr 10.20.193.28:0/2688331378 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f606ed76640 1 --2- 10.20.193.28:0/2688331378 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baba140 0x560e0bb021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f606f577640 1 -- 10.20.193.28:0/2688331378 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baba140 msgr2=0x560e0bb021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f606f577640 1 --2- 10.20.193.28:0/2688331378 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baba140 0x560e0bb021e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f606f577640 1 -- 10.20.193.28:0/2688331378 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0babaa50 msgr2=0x560e0bb14600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f606f577640 1 --2- 10.20.193.28:0/2688331378 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0babaa50 0x560e0bb14600 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f606f577640 1 -- 10.20.193.28:0/2688331378 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e0b956bd0 con 0x560e0baff6f0 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.070+0000 7f606ed76640 1 --2- 10.20.193.28:0/2688331378 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baba140 0x560e0bb021e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.071+0000 7f606f577640 1 --2- 10.20.193.28:0/2688331378 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baff6f0 0x560e0bb14b40 secure :-1 s=READY pgs=220 gs=2 cs=0 l=1 c_cookie=3067545050617278028 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6060010730 tx=0x7f6060005090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a921bf8928cee4c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.071+0000 7f603affd640 1 -- 10.20.193.28:0/2688331378 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6060010c20 con 0x560e0baff6f0 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.071+0000 7f603affd640 1 -- 10.20.193.28:0/2688331378 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6060005c80 con 0x560e0baff6f0 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.071+0000 7f603affd640 1 -- 10.20.193.28:0/2688331378 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6060012440 con 0x560e0baff6f0 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.071+0000 7f6077220e00 1 -- 10.20.193.28:0/2688331378 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baff6f0 msgr2=0x560e0bb14b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.071+0000 7f6077220e00 1 --2- 10.20.193.28:0/2688331378 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baff6f0 0x560e0bb14b40 secure :-1 s=READY pgs=220 gs=2 cs=0 l=1 c_cookie=3067545050617278028 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6060010730 tx=0x7f6060005090 comp rx=0 tx=0).stop 2026-02-25T11:56:41.073 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.071+0000 7f6077220e00 1 -- 10.20.193.28:0/2688331378 shutdown_connections 2026-02-25T11:56:41.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.071+0000 7f6077220e00 1 --2- 10.20.193.28:0/2688331378 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baff6f0 0x560e0bb14b40 unknown :-1 s=CLOSED pgs=220 gs=2 cs=0 l=1 c_cookie=3067545050617278028 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.071+0000 7f6077220e00 1 --2- 10.20.193.28:0/2688331378 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0babaa50 0x560e0bb14600 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.071+0000 7f6077220e00 1 --2- 10.20.193.28:0/2688331378 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baba140 0x560e0bb021e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.071+0000 7f6077220e00 1 -- 10.20.193.28:0/2688331378 >> 10.20.193.28:0/2688331378 conn(0x560e0b9d9d40 msgr2=0x560e0bb01ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:41.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.071+0000 7f6077220e00 1 -- 10.20.193.28:0/2688331378 shutdown_connections 2026-02-25T11:56:41.074 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.071+0000 7f6077220e00 1 -- 10.20.193.28:0/2688331378 wait complete. 2026-02-25T11:56:41.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.072+0000 7f6077220e00 1 -- start start 2026-02-25T11:56:41.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.072+0000 7f6077220e00 1 Processor -- start 2026-02-25T11:56:41.075 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.072+0000 7f6077220e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baba140 0x560e0ba9da00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.072+0000 7f6077220e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0babaa50 0x560e0ba9df40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.072+0000 7f6077220e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baff6f0 0x560e0bb14300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.072+0000 7f6077220e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x560e0bb1f330 con 0x560e0babaa50 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.072+0000 7f6077220e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560e0bb14840 con 0x560e0baff6f0 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.072+0000 7f6077220e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560e0bb14a00 con 0x560e0baba140 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f606e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0babaa50 0x560e0ba9df40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f606f577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baff6f0 0x560e0bb14300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f606f577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baff6f0 0x560e0bb14300 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:39842/0 (socket says 10.20.193.28:39842) 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f606f577640 1 -- 10.20.193.28:0/3140121793 learned_addr learned my addr 10.20.193.28:0/3140121793 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f606ed76640 1 --2- 10.20.193.28:0/3140121793 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baba140 0x560e0ba9da00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f606e575640 1 -- 10.20.193.28:0/3140121793 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baba140 msgr2=0x560e0ba9da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f606e575640 1 --2- 10.20.193.28:0/3140121793 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baba140 0x560e0ba9da00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f606e575640 1 -- 10.20.193.28:0/3140121793 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baff6f0 msgr2=0x560e0bb14300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f606e575640 1 --2- 10.20.193.28:0/3140121793 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baff6f0 0x560e0bb14300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f606e575640 1 -- 10.20.193.28:0/3140121793 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e0b94a680 con 0x560e0babaa50 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f606e575640 1 --2- 10.20.193.28:0/3140121793 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0babaa50 0x560e0ba9df40 secure :-1 s=READY pgs=826 gs=1 cs=0 l=1 c_cookie=625905328068644093 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6058009480 tx=0x7f6058009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afa994668e4cfd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f6038ff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f6038ff9640 1 -- 10.20.193.28:0/3140121793 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f60580d2240 con 0x560e0babaa50 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f6077220e00 1 -- 10.20.193.28:0/3140121793 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e0b971d80 con 0x560e0babaa50 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f6077220e00 10 client.17304 _ll_register_callbacks cb 0x560e0b9479a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f6077220e00 1 -- 10.20.193.28:0/3140121793 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6060010d80 con 0x560e0babaa50 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f6038ff9640 1 -- 10.20.193.28:0/3140121793 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f60580f5070 con 0x560e0babaa50 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.073+0000 7f6038ff9640 1 -- 10.20.193.28:0/3140121793 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f60580d3390 con 0x560e0babaa50 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.074+0000 7f6038ff9640 1 -- 10.20.193.28:0/3140121793 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f60580dd5c0 con 0x560e0babaa50 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.074+0000 7f6038ff9640 1 -- 10.20.193.28:0/3140121793 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f60580da780 con 0x560e0babaa50 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.074+0000 7f6077220e00 10 client.17304 fetch_fsmap learned FSMap version 23 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.074+0000 7f6077220e00 1 -- 10.20.193.28:0/3140121793 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x560e0b95c3b0 con 0x560e0babaa50 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.074+0000 7f6038ff9640 1 -- 10.20.193.28:0/3140121793 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f60580d0460 con 0x560e0babaa50 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.074+0000 7f6077220e00 10 client.17304 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.074+0000 7f6077220e00 10 client.17304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.074+0000 7f6077220e00 20 client.17304 populate_metadata read hostname 'trial028' 2026-02-25T11:56:41.077 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.074+0000 7f6077220e00 1 --2- 10.20.193.28:0/3140121793 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560e0bbacd10 0x560e0bbcd1c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:56:41.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.074+0000 7f6077220e00 4 client.17304 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:56:41.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.074+0000 7f6077220e00 1 -- 10.20.193.28:0/3140121793 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e0b659f40 con 0x560e0bbacd10 2026-02-25T11:56:41.078 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.074+0000 7f606ed76640 1 --2- 10.20.193.28:0/3140121793 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560e0bbacd10 0x560e0bbcd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.075+0000 7f606ed76640 1 --2- 10.20.193.28:0/3140121793 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560e0bbacd10 0x560e0bbcd1c0 crc :-1 s=READY pgs=159 gs=4 cs=0 l=0 c_cookie=16712707446993370315 s_cookie=8497339611829047336 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ef76e01510bccb server_cookie=75ec9d5b2aed1028 in_seq=0 out_seq=0 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.075+0000 7f6038ff9640 10 client.17304 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.075+0000 7f6038ff9640 1 -- 10.20.193.28:0/3140121793 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x560e0b659f40 con 0x560e0bbacd10 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.075+0000 7f6038ff9640 10 client.17304 handle_command_reply: tid=0 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.075+0000 7f6077220e00 1 client.17304 shutdown 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.075+0000 7f6077220e00 1 -- 10.20.193.28:0/3140121793 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0babaa50 msgr2=0x560e0ba9df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.075+0000 7f6077220e00 1 --2- 10.20.193.28:0/3140121793 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0babaa50 0x560e0ba9df40 secure :-1 s=READY pgs=826 gs=1 cs=0 l=1 c_cookie=625905328068644093 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6058009480 tx=0x7f6058009310 comp rx=0 tx=0).stop 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6077220e00 1 -- 10.20.193.28:0/3140121793 shutdown_connections 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6077220e00 1 --2- 10.20.193.28:0/3140121793 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560e0bbacd10 0x560e0bbcd1c0 crc :-1 s=READY pgs=159 gs=4 cs=0 l=0 c_cookie=16712707446993370315 s_cookie=8497339611829047336 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6077220e00 1 --2- 10.20.193.28:0/3140121793 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560e0baff6f0 0x560e0bb14300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6077220e00 1 --2- 10.20.193.28:0/3140121793 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560e0babaa50 0x560e0ba9df40 unknown :-1 s=CLOSED pgs=826 gs=1 cs=0 l=1 c_cookie=625905328068644093 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6077220e00 1 --2- 10.20.193.28:0/3140121793 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560e0baba140 0x560e0ba9da00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6077220e00 1 -- 10.20.193.28:0/3140121793 >> 10.20.193.28:0/3140121793 conn(0x560e0b9d9d40 msgr2=0x560e0bb1b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6038ff9640 0 client.17304 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:56:41.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6077220e00 1 -- 10.20.193.28:0/3140121793 shutdown_connections 2026-02-25T11:56:41.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6077220e00 1 -- 10.20.193.28:0/3140121793 wait complete. 2026-02-25T11:56:41.080 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6077220e00 20 client.17304 trim_cache size 0 max 16384 2026-02-25T11:56:41.080 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:56:41.080 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:56:41.080 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:56:41.080 INFO:teuthology.orchestra.run.trial028.stdout: "91e989b7-1f1f-41fb-9047-642867de91e6": { 2026-02-25T11:56:41.080 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.080 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "91e989b7-1f1f-41fb-9047-642867de91e6", 2026-02-25T11:56:41.080 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:56:41.080 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: "0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8": { 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:05.873000+0000", 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: "850bcfd3-be8a-4930-896c-1ed6e48b8fed": { 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:27.124813+0000", 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.081 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:56:41.082 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.083 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.084 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.085 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "91e989b7-1f1f-41fb-9047-642867de91e6": { 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:40.578257+0000", 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.086 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.087 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.088 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "7b237934-6b95-43dd-b3a6-82b74134ec55": { 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:31.289725+0000", 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:56:41.089 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:56:41.090 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:56:41.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6074a24640 1 -- 10.20.193.28:0/1191713222 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60440b5760 msgr2=0x7f60440d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:41.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6074a24640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60440b5760 0x7f60440d5c10 secure :-1 s=READY pgs=371 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6064004740 tx=0x7f60640073d0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6074a24640 1 -- 10.20.193.28:0/1191713222 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6070119eb0 msgr2=0x7f607012c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:56:41.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6074a24640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6070119eb0 0x7f607012c8c0 secure :-1 s=READY pgs=824 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f60580098a0 tx=0x7f60580028d0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6074a24640 1 -- 10.20.193.28:0/1191713222 shutdown_connections 2026-02-25T11:56:41.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6074a24640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f60440b5760 0x7f60440d5c10 unknown :-1 s=CLOSED pgs=371 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6074a24640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6070131c90 0x7f6070132130 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6074a24640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6070119eb0 0x7f607012c8c0 unknown :-1 s=CLOSED pgs=824 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6074a24640 1 --2- 10.20.193.28:0/1191713222 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6070114960 0x7f607012c380 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:56:41.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.076+0000 7f6074a24640 1 -- 10.20.193.28:0/1191713222 >> 10.20.193.28:0/1191713222 conn(0x7f607005be90 msgr2=0x7f60701167d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:56:41.090 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.077+0000 7f6074a24640 1 -- 10.20.193.28:0/1191713222 shutdown_connections 2026-02-25T11:56:41.091 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:56:41.077+0000 7f6074a24640 1 -- 10.20.193.28:0/1191713222 wait complete. 2026-02-25T11:56:41.177 INFO:tasks.cephfs.filesystem:scrub status for tag:91e989b7-1f1f-41fb-9047-642867de91e6 - {'path': '/', 'tag': '91e989b7-1f1f-41fb-9047-642867de91e6', 'options': 'recursive,force'} 2026-02-25T11:56:41.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:41 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:56:41.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:41 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:56:41.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:41 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:56:41.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:41 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:56:41.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:41 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:56:41.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:41 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:56:41.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:41 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:56:41.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:41 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:56:41.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:41 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:56:41.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:41 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:56:41.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:41 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:56:41.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:41 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:56:42.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:42 trial167 ceph-mon[43392]: pgmap v480: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:42.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:42 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:56:42.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:42 trial119 ceph-mon[42710]: pgmap v480: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:42.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:42 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:56:42.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:42 trial028 ceph-mon[39224]: pgmap v480: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:42.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:42 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:56:44.342 INFO:tasks.workunit.client.0.trial028.stdout:437/600: open fd = 3 2026-02-25T11:56:44.342 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.342 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.342 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.342 INFO:tasks.workunit.client.0.trial028.stdout:438/600: open fd = 3 2026-02-25T11:56:44.342 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.342 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.343 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.343 INFO:tasks.workunit.client.0.trial028.stdout:439/600: open fd = 3 2026-02-25T11:56:44.343 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.343 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.343 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.343 INFO:tasks.workunit.client.0.trial028.stdout:440/600: open fd = 3 2026-02-25T11:56:44.343 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.343 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.343 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.343 INFO:tasks.workunit.client.0.trial028.stdout:441/600: open fd = 3 2026-02-25T11:56:44.343 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.343 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.344 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.344 INFO:tasks.workunit.client.0.trial028.stdout:442/600: open fd = 3 2026-02-25T11:56:44.344 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.344 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.344 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.344 INFO:tasks.workunit.client.0.trial028.stdout:443/600: open fd = 3 2026-02-25T11:56:44.344 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.344 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.344 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.344 INFO:tasks.workunit.client.0.trial028.stdout:444/600: open fd = 3 2026-02-25T11:56:44.344 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.344 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.344 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.345 INFO:tasks.workunit.client.0.trial028.stdout:445/600: open fd = 3 2026-02-25T11:56:44.345 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.345 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.345 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.345 INFO:tasks.workunit.client.0.trial028.stdout:446/600: open fd = 3 2026-02-25T11:56:44.345 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.345 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.345 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.345 INFO:tasks.workunit.client.0.trial028.stdout:447/600: open fd = 3 2026-02-25T11:56:44.345 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.345 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.345 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.345 INFO:tasks.workunit.client.0.trial028.stdout:448/600: open fd = 3 2026-02-25T11:56:44.346 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.346 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.346 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.346 INFO:tasks.workunit.client.0.trial028.stdout:449/600: open fd = 3 2026-02-25T11:56:44.346 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.346 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.346 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.346 INFO:tasks.workunit.client.0.trial028.stdout:450/600: open fd = 3 2026-02-25T11:56:44.346 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.346 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.346 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.346 INFO:tasks.workunit.client.0.trial028.stdout:451/600: open fd = 3 2026-02-25T11:56:44.346 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.347 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.347 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.347 INFO:tasks.workunit.client.0.trial028.stdout:452/600: open fd = 3 2026-02-25T11:56:44.347 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.347 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.347 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.347 INFO:tasks.workunit.client.0.trial028.stdout:453/600: open fd = 3 2026-02-25T11:56:44.347 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.347 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.347 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.347 INFO:tasks.workunit.client.0.trial028.stdout:454/600: open fd = 3 2026-02-25T11:56:44.347 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.348 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.348 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.348 INFO:tasks.workunit.client.0.trial028.stdout:455/600: open fd = 3 2026-02-25T11:56:44.348 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.348 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.348 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.348 INFO:tasks.workunit.client.0.trial028.stdout:456/600: open fd = 3 2026-02-25T11:56:44.348 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.348 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.348 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.348 INFO:tasks.workunit.client.0.trial028.stdout:457/600: open fd = 3 2026-02-25T11:56:44.348 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.348 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:458/600: open fd = 3 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:459/600: open fd = 3 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:460/600: open fd = 3 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.349 INFO:tasks.workunit.client.0.trial028.stdout:461/600: open fd = 3 2026-02-25T11:56:44.350 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.350 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.350 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.350 INFO:tasks.workunit.client.0.trial028.stdout:462/600: open fd = 3 2026-02-25T11:56:44.350 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.350 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.350 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.350 INFO:tasks.workunit.client.0.trial028.stdout:463/600: open fd = 3 2026-02-25T11:56:44.350 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.350 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.350 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.350 INFO:tasks.workunit.client.0.trial028.stdout:464/600: open fd = 3 2026-02-25T11:56:44.350 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.351 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.351 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.351 INFO:tasks.workunit.client.0.trial028.stdout:465/600: open fd = 3 2026-02-25T11:56:44.351 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.351 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.351 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.351 INFO:tasks.workunit.client.0.trial028.stdout:466/600: open fd = 3 2026-02-25T11:56:44.351 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.351 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.351 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.351 INFO:tasks.workunit.client.0.trial028.stdout:467/600: open fd = 3 2026-02-25T11:56:44.351 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.351 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.352 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.352 INFO:tasks.workunit.client.0.trial028.stdout:468/600: open fd = 3 2026-02-25T11:56:44.352 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.352 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.352 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.352 INFO:tasks.workunit.client.0.trial028.stdout:469/600: open fd = 3 2026-02-25T11:56:44.352 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.352 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.352 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.352 INFO:tasks.workunit.client.0.trial028.stdout:470/600: open fd = 3 2026-02-25T11:56:44.352 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.352 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.352 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.353 INFO:tasks.workunit.client.0.trial028.stdout:471/600: open fd = 3 2026-02-25T11:56:44.353 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.353 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.353 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.353 INFO:tasks.workunit.client.0.trial028.stdout:472/600: open fd = 3 2026-02-25T11:56:44.353 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.353 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.353 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.353 INFO:tasks.workunit.client.0.trial028.stdout:473/600: open fd = 3 2026-02-25T11:56:44.353 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.353 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.353 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.353 INFO:tasks.workunit.client.0.trial028.stdout:474/600: open fd = 3 2026-02-25T11:56:44.354 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.354 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.354 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.354 INFO:tasks.workunit.client.0.trial028.stdout:475/600: open fd = 3 2026-02-25T11:56:44.354 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.354 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.354 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.354 INFO:tasks.workunit.client.0.trial028.stdout:476/600: open fd = 3 2026-02-25T11:56:44.354 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.354 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.354 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.354 INFO:tasks.workunit.client.0.trial028.stdout:477/600: open fd = 3 2026-02-25T11:56:44.354 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.355 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.355 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.355 INFO:tasks.workunit.client.0.trial028.stdout:478/600: open fd = 3 2026-02-25T11:56:44.355 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.355 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.355 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.355 INFO:tasks.workunit.client.0.trial028.stdout:479/600: open fd = 3 2026-02-25T11:56:44.355 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.355 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.355 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.355 INFO:tasks.workunit.client.0.trial028.stdout:480/600: open fd = 3 2026-02-25T11:56:44.356 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.356 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.356 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.356 INFO:tasks.workunit.client.0.trial028.stdout:481/600: open fd = 3 2026-02-25T11:56:44.356 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.356 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.356 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.356 INFO:tasks.workunit.client.0.trial028.stdout:482/600: open fd = 3 2026-02-25T11:56:44.356 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.356 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.356 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.356 INFO:tasks.workunit.client.0.trial028.stdout:483/600: open fd = 3 2026-02-25T11:56:44.357 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.357 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.357 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.357 INFO:tasks.workunit.client.0.trial028.stdout:484/600: open fd = 3 2026-02-25T11:56:44.357 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.357 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.357 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.357 INFO:tasks.workunit.client.0.trial028.stdout:485/600: open fd = 3 2026-02-25T11:56:44.357 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.357 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.357 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.357 INFO:tasks.workunit.client.0.trial028.stdout:486/600: open fd = 3 2026-02-25T11:56:44.357 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.358 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.358 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.358 INFO:tasks.workunit.client.0.trial028.stdout:487/600: open fd = 3 2026-02-25T11:56:44.358 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.358 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.358 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.358 INFO:tasks.workunit.client.0.trial028.stdout:488/600: open fd = 3 2026-02-25T11:56:44.358 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.358 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.358 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.358 INFO:tasks.workunit.client.0.trial028.stdout:489/600: open fd = 3 2026-02-25T11:56:44.358 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.358 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:490/600: open fd = 3 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:491/600: open fd = 3 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:492/600: open fd = 3 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.359 INFO:tasks.workunit.client.0.trial028.stdout:493/600: open fd = 3 2026-02-25T11:56:44.360 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.360 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.360 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.360 INFO:tasks.workunit.client.0.trial028.stdout:494/600: open fd = 3 2026-02-25T11:56:44.360 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.360 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.360 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.360 INFO:tasks.workunit.client.0.trial028.stdout:495/600: open fd = 3 2026-02-25T11:56:44.360 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.360 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.360 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.360 INFO:tasks.workunit.client.0.trial028.stdout:496/600: open fd = 3 2026-02-25T11:56:44.360 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.361 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.361 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.361 INFO:tasks.workunit.client.0.trial028.stdout:497/600: open fd = 3 2026-02-25T11:56:44.361 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.361 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.361 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.361 INFO:tasks.workunit.client.0.trial028.stdout:498/600: open fd = 3 2026-02-25T11:56:44.361 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.361 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:56:44.361 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:56:44.499 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:44 trial167 ceph-mon[43392]: pgmap v481: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:56:44.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:44 trial119 ceph-mon[42710]: pgmap v481: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:56:44.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:44 trial028 ceph-mon[39224]: pgmap v481: 97 pgs: 97 active+clean; 6.5 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:56:46.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:46 trial119 ceph-mon[42710]: pgmap v482: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:56:46.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:46 trial028 ceph-mon[39224]: pgmap v482: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:56:46.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:46 trial167 ceph-mon[43392]: pgmap v482: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:56:48.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:48 trial119 ceph-mon[42710]: pgmap v483: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:48.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:48 trial028 ceph-mon[39224]: pgmap v483: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:48.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:48 trial167 ceph-mon[43392]: pgmap v483: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:50.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:50 trial119 ceph-mon[42710]: pgmap v484: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:56:50.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:50 trial028 ceph-mon[39224]: pgmap v484: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:56:50.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:50 trial167 ceph-mon[43392]: pgmap v484: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:56:52.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:52 trial119 ceph-mon[42710]: pgmap v485: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:52.551 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:52 trial167 ceph-mon[43392]: pgmap v485: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:52.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:52 trial028 ceph-mon[39224]: pgmap v485: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:56:54.509 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:54 trial119 ceph-mon[42710]: pgmap v486: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:56:54.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:54 trial028 ceph-mon[39224]: pgmap v486: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:56:54.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:54 trial167 ceph-mon[43392]: pgmap v486: 97 pgs: 97 active+clean; 6.6 MiB data, 348 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:56:56.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:56 trial028 ceph-mon[39224]: pgmap v487: 97 pgs: 97 active+clean; 6.7 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:56:56.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:56 trial167 ceph-mon[43392]: pgmap v487: 97 pgs: 97 active+clean; 6.7 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:56:56.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:56 trial119 ceph-mon[42710]: pgmap v487: 97 pgs: 97 active+clean; 6.7 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:56:58.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:56:58 trial028 ceph-mon[39224]: pgmap v488: 97 pgs: 97 active+clean; 6.7 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:58.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:56:58 trial167 ceph-mon[43392]: pgmap v488: 97 pgs: 97 active+clean; 6.7 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:56:58.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:56:58 trial119 ceph-mon[42710]: pgmap v488: 97 pgs: 97 active+clean; 6.7 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:00.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:00 trial028 ceph-mon[39224]: pgmap v489: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:00.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:00 trial167 ceph-mon[43392]: pgmap v489: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:00.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:00 trial119 ceph-mon[42710]: pgmap v489: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:02.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:02 trial028 ceph-mon[39224]: pgmap v490: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:57:02.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:02 trial167 ceph-mon[43392]: pgmap v490: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:57:02.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:02 trial119 ceph-mon[42710]: pgmap v490: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:57:04.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:04 trial028 ceph-mon[39224]: pgmap v491: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:57:04.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:04 trial167 ceph-mon[43392]: pgmap v491: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:57:04.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:04 trial119 ceph-mon[42710]: pgmap v491: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:57:06.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:06 trial028 ceph-mon[39224]: pgmap v492: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:57:06.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:06 trial167 ceph-mon[43392]: pgmap v492: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:57:06.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:06 trial119 ceph-mon[42710]: pgmap v492: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:57:08.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:08 trial028 ceph-mon[39224]: pgmap v493: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:08.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:08 trial167 ceph-mon[43392]: pgmap v493: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:08.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:08 trial119 ceph-mon[42710]: pgmap v493: 97 pgs: 97 active+clean; 6.8 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:10.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:10 trial028 ceph-mon[39224]: pgmap v494: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:10.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:10 trial167 ceph-mon[43392]: pgmap v494: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:10.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:10 trial119 ceph-mon[42710]: pgmap v494: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:11.178 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:57:11.365 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:11.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.540+0000 7f5a75ac4640 1 -- 10.20.193.28:0/3216085720 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a701140a0 msgr2=0x7f5a70114470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:11.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.540+0000 7f5a75ac4640 1 --2- 10.20.193.28:0/3216085720 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a701140a0 0x7f5a70114470 secure :-1 s=READY pgs=827 gs=1 cs=0 l=1 c_cookie=5210404203361983468 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a60009950 tx=0x7f5a6002f190 comp rx=0 tx=0).stop 2026-02-25T11:57:11.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.540+0000 7f5a75ac4640 1 -- 10.20.193.28:0/3216085720 shutdown_connections 2026-02-25T11:57:11.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.540+0000 7f5a75ac4640 1 --2- 10.20.193.28:0/3216085720 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a70119d30 0x7f5a7017a700 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.540+0000 7f5a75ac4640 1 --2- 10.20.193.28:0/3216085720 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5a70114a90 0x7f5a70119580 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.540+0000 7f5a75ac4640 1 --2- 10.20.193.28:0/3216085720 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a701140a0 0x7f5a70114470 unknown :-1 s=CLOSED pgs=827 gs=1 cs=0 l=1 c_cookie=5210404203361983468 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.540+0000 7f5a75ac4640 1 -- 10.20.193.28:0/3216085720 >> 10.20.193.28:0/3216085720 conn(0x7f5a70064c40 msgr2=0x7f5a70065040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:11.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.541+0000 7f5a75ac4640 1 -- 10.20.193.28:0/3216085720 shutdown_connections 2026-02-25T11:57:11.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.541+0000 7f5a75ac4640 1 -- 10.20.193.28:0/3216085720 wait complete. 2026-02-25T11:57:11.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.541+0000 7f5a75ac4640 1 Processor -- start 2026-02-25T11:57:11.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.541+0000 7f5a75ac4640 1 -- start start 2026-02-25T11:57:11.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a75ac4640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a701140a0 0x7f5a70122580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a75ac4640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5a70114a90 0x7f5a70122ac0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a75ac4640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a70119d30 0x7f5a70127e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a75ac4640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5a701284b0 con 0x7f5a70119d30 2026-02-25T11:57:11.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a75ac4640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70128620 con 0x7f5a701140a0 2026-02-25T11:57:11.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a75ac4640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5a701287e0 con 0x7f5a70114a90 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a74ac2640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a701140a0 0x7f5a70122580 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a74ac2640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a701140a0 0x7f5a70122580 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:54512/0 (socket says 10.20.193.28:54512) 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a74ac2640 1 -- 10.20.193.28:0/715897154 learned_addr learned my addr 10.20.193.28:0/715897154 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a752c3640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a70119d30 0x7f5a70127e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a67fff640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5a70114a90 0x7f5a70122ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a74ac2640 1 -- 10.20.193.28:0/715897154 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5a70114a90 msgr2=0x7f5a70122ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a74ac2640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5a70114a90 0x7f5a70122ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a74ac2640 1 -- 10.20.193.28:0/715897154 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a70119d30 msgr2=0x7f5a70127e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a74ac2640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a70119d30 0x7f5a70127e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a74ac2640 1 -- 10.20.193.28:0/715897154 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a60009570 con 0x7f5a701140a0 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.542+0000 7f5a752c3640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a70119d30 0x7f5a70127e90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.543+0000 7f5a74ac2640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a701140a0 0x7f5a70122580 secure :-1 s=READY pgs=171 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a600098a0 tx=0x7f5a600028d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.543+0000 7f5a65ffb640 1 -- 10.20.193.28:0/715897154 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a6003d070 con 0x7f5a701140a0 2026-02-25T11:57:11.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.543+0000 7f5a75ac4640 1 -- 10.20.193.28:0/715897154 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a70128a60 con 0x7f5a701140a0 2026-02-25T11:57:11.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.543+0000 7f5a65ffb640 1 -- 10.20.193.28:0/715897154 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5a60002a70 con 0x7f5a701140a0 2026-02-25T11:57:11.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.543+0000 7f5a65ffb640 1 -- 10.20.193.28:0/715897154 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a600418c0 con 0x7f5a701140a0 2026-02-25T11:57:11.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.543+0000 7f5a75ac4640 1 -- 10.20.193.28:0/715897154 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a701691c0 con 0x7f5a701140a0 2026-02-25T11:57:11.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.546+0000 7f5a65ffb640 1 -- 10.20.193.28:0/715897154 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f5a60041a50 con 0x7f5a701140a0 2026-02-25T11:57:11.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.548+0000 7f5a65ffb640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5a4c0b57e0 0x7f5a4c0d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.548+0000 7f5a65ffb640 1 -- 10.20.193.28:0/715897154 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f5a6010b050 con 0x7f5a701140a0 2026-02-25T11:57:11.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.548+0000 7f5a67fff640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5a4c0b57e0 0x7f5a4c0d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.549+0000 7f5a67fff640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5a4c0b57e0 0x7f5a4c0d5c90 secure :-1 s=READY pgs=372 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a70128f20 tx=0x7f5a5800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:11.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.549+0000 7f5a78d48e00 1 Processor -- start 2026-02-25T11:57:11.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.549+0000 7f5a78d48e00 1 -- start start 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.549+0000 7f5a78d48e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345140 0x55efc8345510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.549+0000 7f5a78d48e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345a50 0x55efc838a1b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.549+0000 7f5a78d48e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc838a6f0 0x55efc83aaad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.549+0000 7f5a78d48e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55efc825b9b0 con 0x55efc8345a50 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.549+0000 7f5a78d48e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55efc82576e0 con 0x55efc8345140 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.549+0000 7f5a78d48e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55efc8251840 con 0x55efc838a6f0 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.549+0000 7f5a67fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345a50 0x55efc838a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.549+0000 7f5a74ac2640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345140 0x55efc8345510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.550+0000 7f5a74ac2640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345140 0x55efc8345510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:54516/0 (socket says 10.20.193.28:54516) 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.550+0000 7f5a74ac2640 1 -- 10.20.193.28:0/2850079362 learned_addr learned my addr 10.20.193.28:0/2850079362 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.550+0000 7f5a752c3640 1 --2- 10.20.193.28:0/2850079362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc838a6f0 0x55efc83aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.550+0000 7f5a74ac2640 1 -- 10.20.193.28:0/2850079362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc838a6f0 msgr2=0x55efc83aaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.550+0000 7f5a74ac2640 1 --2- 10.20.193.28:0/2850079362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc838a6f0 0x55efc83aaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.550+0000 7f5a74ac2640 1 -- 10.20.193.28:0/2850079362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345a50 msgr2=0x55efc838a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.550+0000 7f5a74ac2640 1 --2- 10.20.193.28:0/2850079362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345a50 0x55efc838a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.550+0000 7f5a74ac2640 1 -- 10.20.193.28:0/2850079362 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc81fcd80 con 0x55efc8345140 2026-02-25T11:57:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.550+0000 7f5a752c3640 1 --2- 10.20.193.28:0/2850079362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc838a6f0 0x55efc83aaad0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:57:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.550+0000 7f5a74ac2640 1 --2- 10.20.193.28:0/2850079362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345140 0x55efc8345510 secure :-1 s=READY pgs=172 gs=1 cs=0 l=1 c_cookie=18147504217983958488 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a60009480 tx=0x7f5a60005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbd8e2efc8d63dd8 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.550+0000 7f5a3d4e4640 1 -- 10.20.193.28:0/2850079362 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a600d0c80 con 0x55efc8345140 2026-02-25T11:57:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.550+0000 7f5a3d4e4640 1 -- 10.20.193.28:0/2850079362 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5a60041d30 con 0x55efc8345140 2026-02-25T11:57:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.550+0000 7f5a3d4e4640 1 -- 10.20.193.28:0/2850079362 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a600cf460 con 0x55efc8345140 2026-02-25T11:57:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.551+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2850079362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345140 msgr2=0x55efc8345510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.551+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/2850079362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345140 0x55efc8345510 secure :-1 s=READY pgs=172 gs=1 cs=0 l=1 c_cookie=18147504217983958488 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a60009480 tx=0x7f5a60005ab0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.551+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2850079362 shutdown_connections 2026-02-25T11:57:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.551+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/2850079362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc838a6f0 0x55efc83aaad0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.551+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/2850079362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345a50 0x55efc838a1b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.551+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/2850079362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345140 0x55efc8345510 unknown :-1 s=CLOSED pgs=172 gs=1 cs=0 l=1 c_cookie=18147504217983958488 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.551+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2850079362 >> 10.20.193.28:0/2850079362 conn(0x55efc8264d40 msgr2=0x55efc8343a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:11.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.551+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2850079362 shutdown_connections 2026-02-25T11:57:11.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.551+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2850079362 wait complete. 2026-02-25T11:57:11.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.552+0000 7f5a78d48e00 1 -- start start 2026-02-25T11:57:11.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.552+0000 7f5a78d48e00 1 Processor -- start 2026-02-25T11:57:11.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a78d48e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc8345140 0x55efc83a2440 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a78d48e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345a50 0x55efc83a2980 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a78d48e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc838a6f0 0x55efc83a7970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a78d48e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55efc83a7eb0 con 0x55efc838a6f0 2026-02-25T11:57:11.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a78d48e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55efc83a8020 con 0x55efc8345a50 2026-02-25T11:57:11.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a78d48e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55efc83a81e0 con 0x55efc8345140 2026-02-25T11:57:11.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a752c3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc838a6f0 0x55efc83a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a752c3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc838a6f0 0x55efc83a7970 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58784/0 (socket says 10.20.193.28:58784) 2026-02-25T11:57:11.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a752c3640 1 -- 10.20.193.28:0/106024533 learned_addr learned my addr 10.20.193.28:0/106024533 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:11.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a74ac2640 1 --2- 10.20.193.28:0/106024533 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc8345140 0x55efc83a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a67fff640 1 --2- 10.20.193.28:0/106024533 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345a50 0x55efc83a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a752c3640 1 -- 10.20.193.28:0/106024533 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc8345140 msgr2=0x55efc83a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:11.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a752c3640 1 --2- 10.20.193.28:0/106024533 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc8345140 0x55efc83a2440 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a752c3640 1 -- 10.20.193.28:0/106024533 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345a50 msgr2=0x55efc83a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:11.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a752c3640 1 --2- 10.20.193.28:0/106024533 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345a50 0x55efc83a2980 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.553+0000 7f5a752c3640 1 -- 10.20.193.28:0/106024533 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a701691c0 con 0x55efc838a6f0 2026-02-25T11:57:11.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.554+0000 7f5a752c3640 1 --2- 10.20.193.28:0/106024533 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc838a6f0 0x55efc83a7970 secure :-1 s=READY pgs=828 gs=3 cs=0 l=1 c_cookie=249039352553346183 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a6c0085f0 tx=0x7f5a6c0117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374c3f8c0091487 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.554+0000 7f5a2e7fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.554+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/106024533 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a6c017070 con 0x55efc838a6f0 2026-02-25T11:57:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.554+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/106024533 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5a6c003c40 con 0x55efc838a6f0 2026-02-25T11:57:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.554+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/106024533 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a6c01c8b0 con 0x55efc838a6f0 2026-02-25T11:57:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.554+0000 7f5a78d48e00 1 -- 10.20.193.28:0/106024533 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c008c30 con 0x55efc838a6f0 2026-02-25T11:57:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.554+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/106024533 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f5a6c01ca40 con 0x55efc838a6f0 2026-02-25T11:57:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.554+0000 7f5a2e7fc640 10 client.17316 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:57:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.554+0000 7f5a78d48e00 10 client.17316 _ll_register_callbacks cb 0x55efc81d29a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.554+0000 7f5a78d48e00 1 -- 10.20.193.28:0/106024533 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55efc7ee76f0 con 0x55efc838a6f0 2026-02-25T11:57:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.555+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/106024533 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5a6c01cc40 con 0x55efc838a6f0 2026-02-25T11:57:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.555+0000 7f5a78d48e00 10 client.17316 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.555+0000 7f5a78d48e00 1 -- 10.20.193.28:0/106024533 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55efc81d5350 con 0x55efc838a6f0 2026-02-25T11:57:11.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.555+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/106024533 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f5a6c005330 con 0x55efc838a6f0 2026-02-25T11:57:11.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.555+0000 7f5a78d48e00 10 client.17316 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:11.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.555+0000 7f5a78d48e00 10 client.17316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:11.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.555+0000 7f5a78d48e00 20 client.17316 populate_metadata read hostname 'trial028' 2026-02-25T11:57:11.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.555+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/106024533 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55efc83aa920 0x55efc8388420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.555+0000 7f5a78d48e00 4 client.17316 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:57:11.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.555+0000 7f5a78d48e00 1 -- 10.20.193.28:0/106024533 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efc8223910 con 0x55efc83aa920 2026-02-25T11:57:11.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.555+0000 7f5a74ac2640 1 --2- 10.20.193.28:0/106024533 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55efc83aa920 0x55efc8388420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.556+0000 7f5a74ac2640 1 --2- 10.20.193.28:0/106024533 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55efc83aa920 0x55efc8388420 crc :-1 s=READY pgs=160 gs=4 cs=0 l=0 c_cookie=10780576333674010712 s_cookie=9698847283924668454 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=959c4d0eba9f5858 server_cookie=86993a37d1179026 in_seq=0 out_seq=0 2026-02-25T11:57:11.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.556+0000 7f5a2e7fc640 10 client.17316 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:11.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.557+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/106024533 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55efc8223910 con 0x55efc83aa920 2026-02-25T11:57:11.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.557+0000 7f5a2e7fc640 10 client.17316 handle_command_reply: tid=0 2026-02-25T11:57:11.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.557+0000 7f5a78d48e00 1 client.17316 shutdown 2026-02-25T11:57:11.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.557+0000 7f5a78d48e00 1 -- 10.20.193.28:0/106024533 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc838a6f0 msgr2=0x55efc83a7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:11.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.557+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/106024533 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc838a6f0 0x55efc83a7970 secure :-1 s=READY pgs=828 gs=3 cs=0 l=1 c_cookie=249039352553346183 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a6c0085f0 tx=0x7f5a6c0117a0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.557+0000 7f5a78d48e00 1 -- 10.20.193.28:0/106024533 shutdown_connections 2026-02-25T11:57:11.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.557+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/106024533 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55efc83aa920 0x55efc8388420 crc :-1 s=READY pgs=160 gs=4 cs=0 l=0 c_cookie=10780576333674010712 s_cookie=9698847283924668454 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.557+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/106024533 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc838a6f0 0x55efc83a7970 unknown :-1 s=CLOSED pgs=828 gs=3 cs=0 l=1 c_cookie=249039352553346183 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.557+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/106024533 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345a50 0x55efc83a2980 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.557+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/106024533 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc8345140 0x55efc83a2440 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.557+0000 7f5a78d48e00 1 -- 10.20.193.28:0/106024533 >> 10.20.193.28:0/106024533 conn(0x55efc8264d40 msgr2=0x55efc8343230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:11.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.557+0000 7f5a2e7fc640 0 client.17316 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:11.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.558+0000 7f5a78d48e00 1 -- 10.20.193.28:0/106024533 shutdown_connections 2026-02-25T11:57:11.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.558+0000 7f5a78d48e00 1 -- 10.20.193.28:0/106024533 wait complete. 2026-02-25T11:57:11.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.558+0000 7f5a78d48e00 20 client.17316 trim_cache size 0 max 16384 2026-02-25T11:57:11.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.567+0000 7f5a78d48e00 1 Processor -- start 2026-02-25T11:57:11.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.567+0000 7f5a78d48e00 1 -- start start 2026-02-25T11:57:11.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.567+0000 7f5a78d48e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345140 0x55efc838d1e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.567+0000 7f5a78d48e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345a50 0x55efc839f610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.567+0000 7f5a78d48e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc838a6f0 0x55efc839fb50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.567+0000 7f5a78d48e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55efc824f470 con 0x55efc8345a50 2026-02-25T11:57:11.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.567+0000 7f5a78d48e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55efc7f0f700 con 0x55efc8345140 2026-02-25T11:57:11.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.567+0000 7f5a78d48e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55efc825b9b0 con 0x55efc838a6f0 2026-02-25T11:57:11.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a67fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345a50 0x55efc839f610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a67fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345a50 0x55efc839f610 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58788/0 (socket says 10.20.193.28:58788) 2026-02-25T11:57:11.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a67fff640 1 -- 10.20.193.28:0/1899258170 learned_addr learned my addr 10.20.193.28:0/1899258170 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:11.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a74ac2640 1 --2- 10.20.193.28:0/1899258170 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345140 0x55efc838d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a752c3640 1 --2- 10.20.193.28:0/1899258170 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc838a6f0 0x55efc839fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a67fff640 1 -- 10.20.193.28:0/1899258170 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc838a6f0 msgr2=0x55efc839fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:11.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a67fff640 1 --2- 10.20.193.28:0/1899258170 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc838a6f0 0x55efc839fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a67fff640 1 -- 10.20.193.28:0/1899258170 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345140 msgr2=0x55efc838d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:11.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a67fff640 1 --2- 10.20.193.28:0/1899258170 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345140 0x55efc838d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a67fff640 1 -- 10.20.193.28:0/1899258170 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc81e7150 con 0x55efc8345a50 2026-02-25T11:57:11.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a67fff640 1 --2- 10.20.193.28:0/1899258170 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345a50 0x55efc839f610 secure :-1 s=READY pgs=829 gs=1 cs=0 l=1 c_cookie=10662865290403990647 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a58004710 tx=0x7f5a58013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fa1b7cc25f4c77 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:11.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a3cce3640 1 -- 10.20.193.28:0/1899258170 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a58024070 con 0x55efc8345a50 2026-02-25T11:57:11.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a3cce3640 1 -- 10.20.193.28:0/1899258170 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5a58015070 con 0x55efc8345a50 2026-02-25T11:57:11.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.568+0000 7f5a3cce3640 1 -- 10.20.193.28:0/1899258170 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a58028d20 con 0x55efc8345a50 2026-02-25T11:57:11.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.569+0000 7f5a78d48e00 1 -- 10.20.193.28:0/1899258170 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345a50 msgr2=0x55efc839f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:11.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.569+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/1899258170 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345a50 0x55efc839f610 secure :-1 s=READY pgs=829 gs=1 cs=0 l=1 c_cookie=10662865290403990647 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a58004710 tx=0x7f5a58013000 comp rx=0 tx=0).stop 2026-02-25T11:57:11.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.569+0000 7f5a78d48e00 1 -- 10.20.193.28:0/1899258170 shutdown_connections 2026-02-25T11:57:11.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.569+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/1899258170 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc838a6f0 0x55efc839fb50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.569+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/1899258170 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345a50 0x55efc839f610 unknown :-1 s=CLOSED pgs=829 gs=1 cs=0 l=1 c_cookie=10662865290403990647 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.569+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/1899258170 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc8345140 0x55efc838d1e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.569+0000 7f5a78d48e00 1 -- 10.20.193.28:0/1899258170 >> 10.20.193.28:0/1899258170 conn(0x55efc8264d40 msgr2=0x55efc838cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:11.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.569+0000 7f5a78d48e00 1 -- 10.20.193.28:0/1899258170 shutdown_connections 2026-02-25T11:57:11.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.569+0000 7f5a78d48e00 1 -- 10.20.193.28:0/1899258170 wait complete. 2026-02-25T11:57:11.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.569+0000 7f5a78d48e00 1 -- start start 2026-02-25T11:57:11.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a78d48e00 1 Processor -- start 2026-02-25T11:57:11.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a78d48e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345140 0x55efc8324870 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a78d48e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc8345a50 0x55efc8324db0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a78d48e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc838a6f0 0x55efc8325330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a78d48e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55efc8320490 con 0x55efc8345140 2026-02-25T11:57:11.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a78d48e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55efc8399d80 con 0x55efc838a6f0 2026-02-25T11:57:11.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a78d48e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55efc83f0610 con 0x55efc8345a50 2026-02-25T11:57:11.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a74ac2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345140 0x55efc8324870 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a74ac2640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345140 0x55efc8324870 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58790/0 (socket says 10.20.193.28:58790) 2026-02-25T11:57:11.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a74ac2640 1 -- 10.20.193.28:0/2496966538 learned_addr learned my addr 10.20.193.28:0/2496966538 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:11.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a74ac2640 1 -- 10.20.193.28:0/2496966538 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc8345a50 msgr2=0x55efc8324db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2026-02-25T11:57:11.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a74ac2640 1 --2- 10.20.193.28:0/2496966538 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc8345a50 0x55efc8324db0 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a74ac2640 1 -- 10.20.193.28:0/2496966538 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc838a6f0 msgr2=0x55efc8325330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:11.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a74ac2640 1 --2- 10.20.193.28:0/2496966538 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc838a6f0 0x55efc8325330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.570+0000 7f5a74ac2640 1 -- 10.20.193.28:0/2496966538 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efc81d5680 con 0x55efc8345140 2026-02-25T11:57:11.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a74ac2640 1 --2- 10.20.193.28:0/2496966538 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345140 0x55efc8324870 secure :-1 s=READY pgs=830 gs=1 cs=0 l=1 c_cookie=13665696621761385238 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a60002940 tx=0x7f5a6002f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda64eac6c57d316 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:11.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a2e7fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:11.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/2496966538 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a600dc2a0 con 0x55efc8345140 2026-02-25T11:57:11.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/2496966538 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5a600d96d0 con 0x55efc8345140 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/2496966538 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5a600d3000 con 0x55efc8345140 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2496966538 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efc81e1bd0 con 0x55efc8345140 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a78d48e00 10 client.17322 _ll_register_callbacks cb 0x55efc81d29a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2496966538 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a60041d30 con 0x55efc8345140 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/2496966538 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f5a600ccdd0 con 0x55efc8345140 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/2496966538 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f5a600ccfb0 con 0x55efc8345140 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a78d48e00 10 client.17322 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2496966538 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55efc81e73b0 con 0x55efc8345140 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/2496966538 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f5a600eb020 con 0x55efc8345140 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a78d48e00 10 client.17322 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a78d48e00 10 client.17322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a78d48e00 20 client.17322 populate_metadata read hostname 'trial028' 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/2496966538 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55efc84381b0 0x55efc8458660 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a78d48e00 4 client.17322 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.571+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2496966538 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55efc7ee4f40 con 0x55efc84381b0 2026-02-25T11:57:11.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.572+0000 7f5a67fff640 1 --2- 10.20.193.28:0/2496966538 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55efc84381b0 0x55efc8458660 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:11.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.572+0000 7f5a67fff640 1 --2- 10.20.193.28:0/2496966538 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55efc84381b0 0x55efc8458660 crc :-1 s=READY pgs=161 gs=3 cs=0 l=0 c_cookie=4766798460021837507 s_cookie=12746049502987026702 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=422711f19d7b0ec3 server_cookie=b0e3107dd316010e in_seq=0 out_seq=0 2026-02-25T11:57:11.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.572+0000 7f5a2e7fc640 10 client.17322 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:11.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a2e7fc640 1 -- 10.20.193.28:0/2496966538 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x55efc7ee4f40 con 0x55efc84381b0 2026-02-25T11:57:11.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a2e7fc640 10 client.17322 handle_command_reply: tid=0 2026-02-25T11:57:11.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a78d48e00 1 client.17322 shutdown 2026-02-25T11:57:11.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2496966538 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345140 msgr2=0x55efc8324870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:11.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/2496966538 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345140 0x55efc8324870 secure :-1 s=READY pgs=830 gs=1 cs=0 l=1 c_cookie=13665696621761385238 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a60002940 tx=0x7f5a6002f190 comp rx=0 tx=0).stop 2026-02-25T11:57:11.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2496966538 shutdown_connections 2026-02-25T11:57:11.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/2496966538 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55efc84381b0 0x55efc8458660 crc :-1 s=READY pgs=161 gs=3 cs=0 l=0 c_cookie=4766798460021837507 s_cookie=12746049502987026702 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/2496966538 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55efc838a6f0 0x55efc8325330 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/2496966538 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55efc8345a50 0x55efc8324db0 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a78d48e00 1 --2- 10.20.193.28:0/2496966538 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55efc8345140 0x55efc8324870 unknown :-1 s=CLOSED pgs=830 gs=1 cs=0 l=1 c_cookie=13665696621761385238 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a2e7fc640 0 client.17322 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:11.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2496966538 >> 10.20.193.28:0/2496966538 conn(0x55efc8264d40 msgr2=0x55efc839e600 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:11.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2496966538 shutdown_connections 2026-02-25T11:57:11.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a78d48e00 1 -- 10.20.193.28:0/2496966538 wait complete. 2026-02-25T11:57:11.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.573+0000 7f5a78d48e00 20 client.17322 trim_cache size 0 max 16384 2026-02-25T11:57:11.575 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8": { 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:05.873000+0000", 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "850bcfd3-be8a-4930-896c-1ed6e48b8fed": { 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:27.124813+0000", 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.576 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.577 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:57:11.578 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.579 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "91e989b7-1f1f-41fb-9047-642867de91e6": { 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:40.578257+0000", 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.580 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.581 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "7b237934-6b95-43dd-b3a6-82b74134ec55": { 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:31.289725+0000", 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:57:11.582 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:57:11.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.574+0000 7f5a75ac4640 1 -- 10.20.193.28:0/715897154 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5a4c0b57e0 msgr2=0x7f5a4c0d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:11.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.574+0000 7f5a75ac4640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5a4c0b57e0 0x7f5a4c0d5c90 secure :-1 s=READY pgs=372 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a70128f20 tx=0x7f5a5800b410 comp rx=0 tx=0).stop 2026-02-25T11:57:11.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.574+0000 7f5a75ac4640 1 -- 10.20.193.28:0/715897154 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a701140a0 msgr2=0x7f5a70122580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:11.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.574+0000 7f5a75ac4640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a701140a0 0x7f5a70122580 secure :-1 s=READY pgs=171 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5a600098a0 tx=0x7f5a600028d0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.575+0000 7f5a75ac4640 1 -- 10.20.193.28:0/715897154 shutdown_connections 2026-02-25T11:57:11.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.575+0000 7f5a75ac4640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5a4c0b57e0 0x7f5a4c0d5c90 unknown :-1 s=CLOSED pgs=372 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.575+0000 7f5a75ac4640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5a70119d30 0x7f5a70127e90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.575+0000 7f5a75ac4640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5a70114a90 0x7f5a70122ac0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.575+0000 7f5a75ac4640 1 --2- 10.20.193.28:0/715897154 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5a701140a0 0x7f5a70122580 unknown :-1 s=CLOSED pgs=171 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:11.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.575+0000 7f5a75ac4640 1 -- 10.20.193.28:0/715897154 >> 10.20.193.28:0/715897154 conn(0x7f5a70064c40 msgr2=0x7f5a70117b80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:11.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.575+0000 7f5a75ac4640 1 -- 10.20.193.28:0/715897154 shutdown_connections 2026-02-25T11:57:11.583 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:11.575+0000 7f5a75ac4640 1 -- 10.20.193.28:0/715897154 wait complete. 2026-02-25T11:57:11.675 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:57:11.675 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:57:11.859 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:12.030 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.029+0000 7fb7b5e79640 1 -- 10.20.193.28:0/1605834238 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb7b010c250 msgr2=0x7fb7b0112e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.029+0000 7fb7b5e79640 1 --2- 10.20.193.28:0/1605834238 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb7b010c250 0x7fb7b0112e00 secure :-1 s=READY pgs=831 gs=1 cs=0 l=1 c_cookie=14625720316052900302 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb798009950 tx=0x7fb79802f190 comp rx=0 tx=0).stop 2026-02-25T11:57:12.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.029+0000 7fb7b5e79640 1 -- 10.20.193.28:0/1605834238 shutdown_connections 2026-02-25T11:57:12.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.029+0000 7fb7b5e79640 1 --2- 10.20.193.28:0/1605834238 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb7b01135b0 0x7fb7b0173f80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.029+0000 7fb7b5e79640 1 --2- 10.20.193.28:0/1605834238 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb7b010c250 0x7fb7b0112e00 unknown :-1 s=CLOSED pgs=831 gs=1 cs=0 l=1 c_cookie=14625720316052900302 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.029+0000 7fb7b5e79640 1 --2- 10.20.193.28:0/1605834238 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb7b010b860 0x7fb7b010bc30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.029+0000 7fb7b5e79640 1 -- 10.20.193.28:0/1605834238 >> 10.20.193.28:0/1605834238 conn(0x7fb7b00e9290 msgr2=0x7fb7b01096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:12.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.029+0000 7fb7b5e79640 1 -- 10.20.193.28:0/1605834238 shutdown_connections 2026-02-25T11:57:12.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.029+0000 7fb7b5e79640 1 -- 10.20.193.28:0/1605834238 wait complete. 2026-02-25T11:57:12.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.030+0000 7fb7b5e79640 1 Processor -- start 2026-02-25T11:57:12.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.030+0000 7fb7b5e79640 1 -- start start 2026-02-25T11:57:12.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.030+0000 7fb7b5e79640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb7b010b860 0x7fb7b00f1650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.030+0000 7fb7b5e79640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb7b010c250 0x7fb7b0173e80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.030+0000 7fb7b5e79640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb7b01135b0 0x7fb7b00ed920 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.030+0000 7fb7b5e79640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b00ee020 con 0x7fb7b010b860 2026-02-25T11:57:12.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.030+0000 7fb7b5e79640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b00ee190 con 0x7fb7b01135b0 2026-02-25T11:57:12.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.030+0000 7fb7b5e79640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b00ee350 con 0x7fb7b010c250 2026-02-25T11:57:12.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7b4e77640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb7b010b860 0x7fb7b00f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7b4e77640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb7b010b860 0x7fb7b00f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58806/0 (socket says 10.20.193.28:58806) 2026-02-25T11:57:12.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7b4e77640 1 -- 10.20.193.28:0/2365409140 learned_addr learned my addr 10.20.193.28:0/2365409140 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:12.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7b5678640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb7b01135b0 0x7fb7b00ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7a7fff640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb7b010c250 0x7fb7b0173e80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7b4e77640 1 -- 10.20.193.28:0/2365409140 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb7b010c250 msgr2=0x7fb7b0173e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7b4e77640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb7b010c250 0x7fb7b0173e80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7b4e77640 1 -- 10.20.193.28:0/2365409140 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb7b01135b0 msgr2=0x7fb7b00ed920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7b4e77640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb7b01135b0 0x7fb7b00ed920 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7b4e77640 1 -- 10.20.193.28:0/2365409140 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb798009570 con 0x7fb7b010b860 2026-02-25T11:57:12.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7b5678640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb7b01135b0 0x7fb7b00ed920 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:57:12.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7b4e77640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb7b010b860 0x7fb7b00f1650 secure :-1 s=READY pgs=832 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb7a000c9b0 tx=0x7fb7a000ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:12.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7a5ffb640 1 -- 10.20.193.28:0/2365409140 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb7a00044a0 con 0x7fb7b010b860 2026-02-25T11:57:12.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7a5ffb640 1 -- 10.20.193.28:0/2365409140 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fb7a0007d80 con 0x7fb7b010b860 2026-02-25T11:57:12.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.031+0000 7fb7a5ffb640 1 -- 10.20.193.28:0/2365409140 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fb7a000f6d0 con 0x7fb7b010b860 2026-02-25T11:57:12.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.032+0000 7fb7b5e79640 1 -- 10.20.193.28:0/2365409140 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b00ee630 con 0x7fb7b010b860 2026-02-25T11:57:12.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.032+0000 7fb7b5e79640 1 -- 10.20.193.28:0/2365409140 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b00fe900 con 0x7fb7b010b860 2026-02-25T11:57:12.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.032+0000 7fb77f7fe640 1 -- 10.20.193.28:0/2365409140 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7b005fa30 con 0x7fb7b010b860 2026-02-25T11:57:12.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.034+0000 7fb7a5ffb640 1 -- 10.20.193.28:0/2365409140 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fb7a00078c0 con 0x7fb7b010b860 2026-02-25T11:57:12.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.036+0000 7fb7a5ffb640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb78c0b56e0 0x7fb78c0d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.036+0000 7fb7a5ffb640 1 -- 10.20.193.28:0/2365409140 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fb7a00e6e40 con 0x7fb7b010b860 2026-02-25T11:57:12.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.036+0000 7fb7a5ffb640 1 -- 10.20.193.28:0/2365409140 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fb7a00e72c0 con 0x7fb7b010b860 2026-02-25T11:57:12.037 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.036+0000 7fb7a7fff640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb78c0b56e0 0x7fb78c0d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.037+0000 7fb7a7fff640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb78c0b56e0 0x7fb78c0d5b90 secure :-1 s=READY pgs=373 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb7b00f13d0 tx=0x7fb79803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:12.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.176+0000 7fb77f7fe640 1 -- 10.20.193.28:0/2365409140 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7b010bc30 con 0x7fb7b010b860 2026-02-25T11:57:12.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.177+0000 7fb7a5ffb640 1 -- 10.20.193.28:0/2365409140 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7fb7a00a9810 con 0x7fb7b010b860 2026-02-25T11:57:12.178 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:57:12.179 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:57:12.180 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:57:12.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.178+0000 7fb77f7fe640 1 -- 10.20.193.28:0/2365409140 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb78c0b56e0 msgr2=0x7fb78c0d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.178+0000 7fb77f7fe640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb78c0b56e0 0x7fb78c0d5b90 secure :-1 s=READY pgs=373 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb7b00f13d0 tx=0x7fb79803a040 comp rx=0 tx=0).stop 2026-02-25T11:57:12.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.178+0000 7fb77f7fe640 1 -- 10.20.193.28:0/2365409140 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb7b010b860 msgr2=0x7fb7b00f1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.178+0000 7fb77f7fe640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb7b010b860 0x7fb7b00f1650 secure :-1 s=READY pgs=832 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fb7a000c9b0 tx=0x7fb7a000ce80 comp rx=0 tx=0).stop 2026-02-25T11:57:12.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.179+0000 7fb77f7fe640 1 -- 10.20.193.28:0/2365409140 shutdown_connections 2026-02-25T11:57:12.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.179+0000 7fb77f7fe640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fb78c0b56e0 0x7fb78c0d5b90 unknown :-1 s=CLOSED pgs=373 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.179+0000 7fb77f7fe640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fb7b01135b0 0x7fb7b00ed920 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.179+0000 7fb77f7fe640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fb7b010c250 0x7fb7b0173e80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.179+0000 7fb77f7fe640 1 --2- 10.20.193.28:0/2365409140 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fb7b010b860 0x7fb7b00f1650 unknown :-1 s=CLOSED pgs=832 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.179+0000 7fb77f7fe640 1 -- 10.20.193.28:0/2365409140 >> 10.20.193.28:0/2365409140 conn(0x7fb7b00e9290 msgr2=0x7fb7b0109e50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:12.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.179+0000 7fb77f7fe640 1 -- 10.20.193.28:0/2365409140 shutdown_connections 2026-02-25T11:57:12.181 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.179+0000 7fb77f7fe640 1 -- 10.20.193.28:0/2365409140 wait complete. 2026-02-25T11:57:12.277 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:57:12.463 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:12.505 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:12 trial028 ceph-mon[39224]: pgmap v495: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:57:12.505 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:12 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2365409140' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:57:12.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.646+0000 7f56ab577640 1 -- 10.20.193.28:0/1819565722 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56ac113610 msgr2=0x7f56ac173fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.646+0000 7f56ab577640 1 --2- 10.20.193.28:0/1819565722 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56ac113610 0x7f56ac173fe0 secure :-1 s=READY pgs=833 gs=3 cs=0 l=1 c_cookie=15431098237778490317 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f569c00aa30 tx=0x7f569c030270 comp rx=0 tx=0).stop 2026-02-25T11:57:12.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.647+0000 7f56ab577640 1 -- 10.20.193.28:0/1819565722 shutdown_connections 2026-02-25T11:57:12.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.647+0000 7f56ab577640 1 --2- 10.20.193.28:0/1819565722 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56ac113610 0x7f56ac173fe0 unknown :-1 s=CLOSED pgs=833 gs=3 cs=0 l=1 c_cookie=15431098237778490317 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.647+0000 7f56ab577640 1 --2- 10.20.193.28:0/1819565722 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56ac10de10 0x7f56ac112e60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.647+0000 7f56ab577640 1 --2- 10.20.193.28:0/1819565722 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f56ac10d420 0x7f56ac10d7f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.647+0000 7f56ab577640 1 -- 10.20.193.28:0/1819565722 >> 10.20.193.28:0/1819565722 conn(0x7f56ac0eacb0 msgr2=0x7f56ac10b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:12.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.647+0000 7f56ab577640 1 -- 10.20.193.28:0/1819565722 shutdown_connections 2026-02-25T11:57:12.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.647+0000 7f56ab577640 1 -- 10.20.193.28:0/1819565722 wait complete. 2026-02-25T11:57:12.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.647+0000 7f56ab577640 1 Processor -- start 2026-02-25T11:57:12.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56ab577640 1 -- start start 2026-02-25T11:57:12.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56ab577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f56ac10d420 0x7f56ac0f59b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56ab577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56ac10de10 0x7f56ac173ee0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56ab577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56ac113610 0x7f56ac0f60b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56ab577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac0fb2c0 con 0x7f56ac113610 2026-02-25T11:57:12.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56ab577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac0fb430 con 0x7f56ac10d420 2026-02-25T11:57:12.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56ab577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac0fb5f0 con 0x7f56ac10de10 2026-02-25T11:57:12.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56aad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56ac113610 0x7f56ac0f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56aad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56ac113610 0x7f56ac0f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58816/0 (socket says 10.20.193.28:58816) 2026-02-25T11:57:12.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56aad76640 1 -- 10.20.193.28:0/2920647117 learned_addr learned my addr 10.20.193.28:0/2920647117 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:12.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56aa575640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f56ac10d420 0x7f56ac0f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56a9d74640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56ac10de10 0x7f56ac173ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56aad76640 1 -- 10.20.193.28:0/2920647117 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56ac10de10 msgr2=0x7f56ac173ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56aad76640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56ac10de10 0x7f56ac173ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56aad76640 1 -- 10.20.193.28:0/2920647117 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f56ac10d420 msgr2=0x7f56ac0f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56aad76640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f56ac10d420 0x7f56ac0f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56aad76640 1 -- 10.20.193.28:0/2920647117 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c00a650 con 0x7f56ac113610 2026-02-25T11:57:12.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.648+0000 7f56aad76640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56ac113610 0x7f56ac0f60b0 secure :-1 s=READY pgs=834 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f569c006a50 tx=0x7f569c030ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:12.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.649+0000 7f56937fe640 1 -- 10.20.193.28:0/2920647117 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f569c03e070 con 0x7f56ac113610 2026-02-25T11:57:12.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.649+0000 7f56937fe640 1 -- 10.20.193.28:0/2920647117 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f569c039de0 con 0x7f56ac113610 2026-02-25T11:57:12.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.649+0000 7f56937fe640 1 -- 10.20.193.28:0/2920647117 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f569c042820 con 0x7f56ac113610 2026-02-25T11:57:12.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.649+0000 7f56ab577640 1 -- 10.20.193.28:0/2920647117 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ac0fb870 con 0x7f56ac113610 2026-02-25T11:57:12.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.649+0000 7f56ab577640 1 -- 10.20.193.28:0/2920647117 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ac0fbc90 con 0x7f56ac113610 2026-02-25T11:57:12.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.652+0000 7f56937fe640 1 -- 10.20.193.28:0/2920647117 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f569c039920 con 0x7f56ac113610 2026-02-25T11:57:12.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.653+0000 7f56937fe640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56840b5750 0x7f56840d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.653+0000 7f56937fe640 1 -- 10.20.193.28:0/2920647117 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f569c10c480 con 0x7f56ac113610 2026-02-25T11:57:12.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.654+0000 7f56aa575640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56840b5750 0x7f56840d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.654+0000 7f56aa575640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56840b5750 0x7f56840d5c00 secure :-1 s=READY pgs=374 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a0004740 tx=0x7f56a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:12.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.654+0000 7f56b2d7de00 1 Processor -- start 2026-02-25T11:57:12.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.654+0000 7f56b2d7de00 1 -- start start 2026-02-25T11:57:12.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56b2d7de00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d8b30 0x5564731d8f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56b2d7de00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 0x55647321dba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56b2d7de00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 0x55647323e4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.657 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56b2d7de00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5564730ee8a0 con 0x55647321e0e0 2026-02-25T11:57:12.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56b2d7de00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5564730ea5d0 con 0x5564731d9440 2026-02-25T11:57:12.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56b2d7de00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5564730e4570 con 0x5564731d8b30 2026-02-25T11:57:12.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56aad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 0x55647323e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56aad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 0x55647323e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58830/0 (socket says 10.20.193.28:58830) 2026-02-25T11:57:12.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56aad76640 1 -- 10.20.193.28:0/1231091355 learned_addr learned my addr 10.20.193.28:0/1231091355 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:12.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56aa575640 1 --2- 10.20.193.28:0/1231091355 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d8b30 0x5564731d8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56aad76640 1 -- 10.20.193.28:0/1231091355 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d8b30 msgr2=0x5564731d8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:12.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56a9d74640 1 --2- 10.20.193.28:0/1231091355 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 0x55647321dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56aad76640 1 --2- 10.20.193.28:0/1231091355 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d8b30 0x5564731d8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56aad76640 1 -- 10.20.193.28:0/1231091355 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 msgr2=0x55647321dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:12.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56aad76640 1 --2- 10.20.193.28:0/1231091355 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 0x55647321dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.658 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56aad76640 1 -- 10.20.193.28:0/1231091355 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55647308fd10 con 0x55647321e0e0 2026-02-25T11:57:12.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56a9d74640 1 --2- 10.20.193.28:0/1231091355 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 0x55647321dba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:57:12.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.655+0000 7f56aad76640 1 --2- 10.20.193.28:0/1231091355 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 0x55647323e4c0 secure :-1 s=READY pgs=835 gs=1 cs=0 l=1 c_cookie=14986774703149380953 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f569c039f50 tx=0x7f569c00a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffbb8306e077559 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:12.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.656+0000 7f5690ff9640 1 -- 10.20.193.28:0/1231091355 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f569c039c00 con 0x55647321e0e0 2026-02-25T11:57:12.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.656+0000 7f5690ff9640 1 -- 10.20.193.28:0/1231091355 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f569c109040 con 0x55647321e0e0 2026-02-25T11:57:12.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.656+0000 7f5690ff9640 1 -- 10.20.193.28:0/1231091355 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f569c10c0d0 con 0x55647321e0e0 2026-02-25T11:57:12.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.656+0000 7f56b2d7de00 1 -- 10.20.193.28:0/1231091355 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 msgr2=0x55647323e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.656+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/1231091355 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 0x55647323e4c0 secure :-1 s=READY pgs=835 gs=1 cs=0 l=1 c_cookie=14986774703149380953 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f569c039f50 tx=0x7f569c00a390 comp rx=0 tx=0).stop 2026-02-25T11:57:12.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.656+0000 7f56b2d7de00 1 -- 10.20.193.28:0/1231091355 shutdown_connections 2026-02-25T11:57:12.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.656+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/1231091355 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 0x55647323e4c0 unknown :-1 s=CLOSED pgs=835 gs=1 cs=0 l=1 c_cookie=14986774703149380953 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.656+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/1231091355 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 0x55647321dba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.659 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.656+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/1231091355 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d8b30 0x5564731d8f00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.656+0000 7f56b2d7de00 1 -- 10.20.193.28:0/1231091355 >> 10.20.193.28:0/1231091355 conn(0x5564730f77a0 msgr2=0x5564731d7410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:12.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.656+0000 7f56b2d7de00 1 -- 10.20.193.28:0/1231091355 shutdown_connections 2026-02-25T11:57:12.660 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.656+0000 7f56b2d7de00 1 -- 10.20.193.28:0/1231091355 wait complete. 2026-02-25T11:57:12.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.658+0000 7f56b2d7de00 1 -- start start 2026-02-25T11:57:12.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.658+0000 7f56b2d7de00 1 Processor -- start 2026-02-25T11:57:12.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.658+0000 7f56b2d7de00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5564731d8b30 0x5564731f0c40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.658+0000 7f56b2d7de00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d9440 0x5564731f1180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.658+0000 7f56b2d7de00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55647321e0e0 0x5564731f6170 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.658+0000 7f56b2d7de00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5564731f66b0 con 0x5564731d8b30 2026-02-25T11:57:12.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.658+0000 7f56b2d7de00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5564731f6820 con 0x55647321e0e0 2026-02-25T11:57:12.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.658+0000 7f56b2d7de00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5564731f69e0 con 0x5564731d9440 2026-02-25T11:57:12.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.658+0000 7f56aa575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5564731d8b30 0x5564731f0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.658+0000 7f56aad76640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55647321e0e0 0x5564731f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.658+0000 7f56aad76640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55647321e0e0 0x5564731f6170 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:54618/0 (socket says 10.20.193.28:54618) 2026-02-25T11:57:12.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.658+0000 7f56aad76640 1 -- 10.20.193.28:0/934336608 learned_addr learned my addr 10.20.193.28:0/934336608 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:12.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.658+0000 7f56a9d74640 1 --2- 10.20.193.28:0/934336608 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d9440 0x5564731f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.659+0000 7f56aa575640 1 -- 10.20.193.28:0/934336608 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d9440 msgr2=0x5564731f1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:12.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.659+0000 7f56aa575640 1 --2- 10.20.193.28:0/934336608 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d9440 0x5564731f1180 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.659+0000 7f56aa575640 1 -- 10.20.193.28:0/934336608 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55647321e0e0 msgr2=0x5564731f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:12.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.659+0000 7f56aa575640 1 --2- 10.20.193.28:0/934336608 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55647321e0e0 0x5564731f6170 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.659+0000 7f56aa575640 1 -- 10.20.193.28:0/934336608 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564730683c0 con 0x5564731d8b30 2026-02-25T11:57:12.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.659+0000 7f56aa575640 1 --2- 10.20.193.28:0/934336608 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5564731d8b30 0x5564731f0c40 secure :-1 s=READY pgs=836 gs=1 cs=0 l=1 c_cookie=16622767430755648979 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a0009e50 tx=0x7f56a00047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6afeee8361f85d3 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:12.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.659+0000 7f5657fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:12.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.659+0000 7f5657fff640 1 -- 10.20.193.28:0/934336608 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56a0022070 con 0x5564731d8b30 2026-02-25T11:57:12.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.659+0000 7f5657fff640 1 -- 10.20.193.28:0/934336608 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f56a001db60 con 0x5564731d8b30 2026-02-25T11:57:12.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.659+0000 7f5657fff640 1 -- 10.20.193.28:0/934336608 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56a001ac90 con 0x5564731d8b30 2026-02-25T11:57:12.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.659+0000 7f56b2d7de00 1 -- 10.20.193.28:0/934336608 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ac0fbc90 con 0x5564731d8b30 2026-02-25T11:57:12.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.660+0000 7f5657fff640 1 -- 10.20.193.28:0/934336608 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f56a001ae20 con 0x5564731d8b30 2026-02-25T11:57:12.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.660+0000 7f56b2d7de00 10 client.17340 _ll_register_callbacks cb 0x556473065a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:12.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.660+0000 7f56b2d7de00 1 -- 10.20.193.28:0/934336608 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556472d7a6f0 con 0x5564731d8b30 2026-02-25T11:57:12.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.660+0000 7f5657fff640 1 -- 10.20.193.28:0/934336608 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f56a001d8b0 con 0x5564731d8b30 2026-02-25T11:57:12.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.660+0000 7f56b2d7de00 10 client.17340 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:12.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.660+0000 7f56b2d7de00 1 -- 10.20.193.28:0/934336608 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55647307a0e0 con 0x5564731d8b30 2026-02-25T11:57:12.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.660+0000 7f5657fff640 1 -- 10.20.193.28:0/934336608 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f56a001dcd0 con 0x5564731d8b30 2026-02-25T11:57:12.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.660+0000 7f56b2d7de00 10 client.17340 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:12.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.660+0000 7f56b2d7de00 10 client.17340 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:57:12.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.660+0000 7f56b2d7de00 20 client.17340 populate_metadata read hostname 'trial028' 2026-02-25T11:57:12.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.660+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/934336608 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55647321b4f0 0x55647321b990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.660+0000 7f56b2d7de00 4 client.17340 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:57:12.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.660+0000 7f56b2d7de00 1 -- 10.20.193.28:0/934336608 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564730b6fc0 con 0x55647321b4f0 2026-02-25T11:57:12.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.661+0000 7f56a9d74640 1 --2- 10.20.193.28:0/934336608 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55647321b4f0 0x55647321b990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.661+0000 7f56a9d74640 1 --2- 10.20.193.28:0/934336608 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55647321b4f0 0x55647321b990 crc :-1 s=READY pgs=57 gs=4 cs=0 l=0 c_cookie=7497855109228651184 s_cookie=1581352209264410364 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=680dbb7f1aebe2b0 server_cookie=15f21774b9fbfefc in_seq=0 out_seq=0 2026-02-25T11:57:12.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.661+0000 7f5657fff640 10 client.17340 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:57:12.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.662+0000 7f5657fff640 1 -- 10.20.193.28:0/934336608 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5564730b6fc0 con 0x55647321b4f0 2026-02-25T11:57:12.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.662+0000 7f5657fff640 10 client.17340 handle_command_reply: tid=0 2026-02-25T11:57:12.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.662+0000 7f56b2d7de00 1 client.17340 shutdown 2026-02-25T11:57:12.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.662+0000 7f56b2d7de00 1 -- 10.20.193.28:0/934336608 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5564731d8b30 msgr2=0x5564731f0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.662+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/934336608 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5564731d8b30 0x5564731f0c40 secure :-1 s=READY pgs=836 gs=1 cs=0 l=1 c_cookie=16622767430755648979 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a0009e50 tx=0x7f56a00047e0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.662+0000 7f56b2d7de00 1 -- 10.20.193.28:0/934336608 shutdown_connections 2026-02-25T11:57:12.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.662+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/934336608 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x55647321b4f0 0x55647321b990 crc :-1 s=READY pgs=57 gs=4 cs=0 l=0 c_cookie=7497855109228651184 s_cookie=1581352209264410364 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.662+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/934336608 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55647321e0e0 0x5564731f6170 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.662+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/934336608 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d9440 0x5564731f1180 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.662+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/934336608 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5564731d8b30 0x5564731f0c40 unknown :-1 s=CLOSED pgs=836 gs=1 cs=0 l=1 c_cookie=16622767430755648979 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.662+0000 7f56b2d7de00 1 -- 10.20.193.28:0/934336608 >> 10.20.193.28:0/934336608 conn(0x5564730f77a0 msgr2=0x5564731d7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:12.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.662+0000 7f5657fff640 0 client.17340 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:57:12.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.662+0000 7f56b2d7de00 1 -- 10.20.193.28:0/934336608 shutdown_connections 2026-02-25T11:57:12.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.663+0000 7f56b2d7de00 1 -- 10.20.193.28:0/934336608 wait complete. 2026-02-25T11:57:12.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.663+0000 7f56b2d7de00 20 client.17340 trim_cache size 0 max 16384 2026-02-25T11:57:12.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.670+0000 7f56b2d7de00 1 Processor -- start 2026-02-25T11:57:12.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56b2d7de00 1 -- start start 2026-02-25T11:57:12.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56b2d7de00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5564731d8b30 0x5564731ee010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56b2d7de00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 0x5564731ee550 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56b2d7de00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55647321e0e0 0x55647321d280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56b2d7de00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5564730e21a0 con 0x5564731d8b30 2026-02-25T11:57:12.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56b2d7de00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x556472da2700 con 0x5564731d9440 2026-02-25T11:57:12.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56b2d7de00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5564731e9f90 con 0x55647321e0e0 2026-02-25T11:57:12.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56aa575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5564731d8b30 0x5564731ee010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56aa575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5564731d8b30 0x5564731ee010 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58850/0 (socket says 10.20.193.28:58850) 2026-02-25T11:57:12.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56aa575640 1 -- 10.20.193.28:0/2036733306 learned_addr learned my addr 10.20.193.28:0/2036733306 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:12.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56aad76640 1 --2- 10.20.193.28:0/2036733306 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55647321e0e0 0x55647321d280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56a9d74640 1 --2- 10.20.193.28:0/2036733306 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 0x5564731ee550 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56aa575640 1 -- 10.20.193.28:0/2036733306 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55647321e0e0 msgr2=0x55647321d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:12.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56aa575640 1 --2- 10.20.193.28:0/2036733306 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55647321e0e0 0x55647321d280 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56aa575640 1 -- 10.20.193.28:0/2036733306 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 msgr2=0x5564731ee550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:12.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56aa575640 1 --2- 10.20.193.28:0/2036733306 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 0x5564731ee550 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56aa575640 1 -- 10.20.193.28:0/2036733306 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556473074b60 con 0x5564731d8b30 2026-02-25T11:57:12.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.671+0000 7f56aa575640 1 --2- 10.20.193.28:0/2036733306 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5564731d8b30 0x5564731ee010 secure :-1 s=READY pgs=837 gs=1 cs=0 l=1 c_cookie=12748490663389623441 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a00093e0 tx=0x7f56a002b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ebbcb6c5b8f491 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:12.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.672+0000 7f56764e6640 1 -- 10.20.193.28:0/2036733306 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56a0022040 con 0x5564731d8b30 2026-02-25T11:57:12.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.672+0000 7f56764e6640 1 -- 10.20.193.28:0/2036733306 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x556472d7a6f0 con 0x5564731d8b30 2026-02-25T11:57:12.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.672+0000 7f56764e6640 1 -- 10.20.193.28:0/2036733306 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56a001a790 con 0x5564731d8b30 2026-02-25T11:57:12.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.672+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2036733306 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5564731d8b30 msgr2=0x5564731ee010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.672+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/2036733306 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5564731d8b30 0x5564731ee010 secure :-1 s=READY pgs=837 gs=1 cs=0 l=1 c_cookie=12748490663389623441 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a00093e0 tx=0x7f56a002b000 comp rx=0 tx=0).stop 2026-02-25T11:57:12.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.672+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2036733306 shutdown_connections 2026-02-25T11:57:12.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.672+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/2036733306 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55647321e0e0 0x55647321d280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.672+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/2036733306 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 0x5564731ee550 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.672+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/2036733306 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5564731d8b30 0x5564731ee010 unknown :-1 s=CLOSED pgs=837 gs=1 cs=0 l=1 c_cookie=12748490663389623441 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.672+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2036733306 >> 10.20.193.28:0/2036733306 conn(0x5564730f77a0 msgr2=0x5564731f57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:12.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.672+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2036733306 shutdown_connections 2026-02-25T11:57:12.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.672+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2036733306 wait complete. 2026-02-25T11:57:12.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.673+0000 7f56b2d7de00 1 -- start start 2026-02-25T11:57:12.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.673+0000 7f56b2d7de00 1 Processor -- start 2026-02-25T11:57:12.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.673+0000 7f56b2d7de00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d8b30 0x5564731f7230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.673+0000 7f56b2d7de00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 0x5564731f7770 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.673+0000 7f56b2d7de00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 0x5564731f7cf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.673+0000 7f56b2d7de00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5564731fc4b0 con 0x55647321e0e0 2026-02-25T11:57:12.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.673+0000 7f56b2d7de00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5564731fc620 con 0x5564731d9440 2026-02-25T11:57:12.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.673+0000 7f56b2d7de00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5564731fc7e0 con 0x5564731d8b30 2026-02-25T11:57:12.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56aad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 0x5564731f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56aad76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 0x5564731f7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58860/0 (socket says 10.20.193.28:58860) 2026-02-25T11:57:12.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56aad76640 1 -- 10.20.193.28:0/2897883113 learned_addr learned my addr 10.20.193.28:0/2897883113 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:12.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56a9d74640 1 --2- 10.20.193.28:0/2897883113 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 0x5564731f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56aad76640 1 -- 10.20.193.28:0/2897883113 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d8b30 msgr2=0x5564731f7230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2026-02-25T11:57:12.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56aad76640 1 --2- 10.20.193.28:0/2897883113 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d8b30 0x5564731f7230 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56aad76640 1 -- 10.20.193.28:0/2897883113 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 msgr2=0x5564731f7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:12.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56aad76640 1 --2- 10.20.193.28:0/2897883113 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 0x5564731f7770 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56aad76640 1 -- 10.20.193.28:0/2897883113 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564730686f0 con 0x55647321e0e0 2026-02-25T11:57:12.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56aad76640 1 --2- 10.20.193.28:0/2897883113 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 0x5564731f7cf0 secure :-1 s=READY pgs=838 gs=2 cs=0 l=1 c_cookie=10714405986331777876 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f569c0ce810 tx=0x7f569c0d86a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b1377a2a9fdb54 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:12.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f5657fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:12.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f5657fff640 1 -- 10.20.193.28:0/2897883113 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f569c039c00 con 0x55647321e0e0 2026-02-25T11:57:12.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f5657fff640 1 -- 10.20.193.28:0/2897883113 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f569c0d8520 con 0x55647321e0e0 2026-02-25T11:57:12.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2897883113 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55647307a0e0 con 0x55647321e0e0 2026-02-25T11:57:12.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f5657fff640 1 -- 10.20.193.28:0/2897883113 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f569c097b40 con 0x55647321e0e0 2026-02-25T11:57:12.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56b2d7de00 10 client.17346 _ll_register_callbacks cb 0x556473065a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:12.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2897883113 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f569c109040 con 0x55647321e0e0 2026-02-25T11:57:12.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f5657fff640 1 -- 10.20.193.28:0/2897883113 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f569c0d3490 con 0x55647321e0e0 2026-02-25T11:57:12.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f5657fff640 1 -- 10.20.193.28:0/2897883113 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f569c0d3690 con 0x55647321e0e0 2026-02-25T11:57:12.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56b2d7de00 10 client.17346 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:12.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.674+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2897883113 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55647307a340 con 0x55647321e0e0 2026-02-25T11:57:12.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.675+0000 7f5657fff640 1 -- 10.20.193.28:0/2897883113 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f569c0f60c0 con 0x55647321e0e0 2026-02-25T11:57:12.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.675+0000 7f56b2d7de00 10 client.17346 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:12.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.675+0000 7f56b2d7de00 10 client.17346 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:57:12.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.675+0000 7f56b2d7de00 20 client.17346 populate_metadata read hostname 'trial028' 2026-02-25T11:57:12.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.675+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/2897883113 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5564732cb5d0 0x5564732eba80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:12.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.675+0000 7f56b2d7de00 4 client.17346 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:57:12.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.675+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2897883113 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556472d77f40 con 0x5564732cb5d0 2026-02-25T11:57:12.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.675+0000 7f56aa575640 1 --2- 10.20.193.28:0/2897883113 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5564732cb5d0 0x5564732eba80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:12.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f56aa575640 1 --2- 10.20.193.28:0/2897883113 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5564732cb5d0 0x5564732eba80 crc :-1 s=READY pgs=58 gs=3 cs=0 l=0 c_cookie=3025376611459556536 s_cookie=17501985124026736270 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29fc4bffbdd264b8 server_cookie=f2e38abda5c8de8e in_seq=0 out_seq=0 2026-02-25T11:57:12.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f5657fff640 10 client.17346 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:57:12.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f5657fff640 1 -- 10.20.193.28:0/2897883113 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556472d77f40 con 0x5564732cb5d0 2026-02-25T11:57:12.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f5657fff640 10 client.17346 handle_command_reply: tid=0 2026-02-25T11:57:12.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f56b2d7de00 1 client.17346 shutdown 2026-02-25T11:57:12.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2897883113 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 msgr2=0x5564731f7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/2897883113 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 0x5564731f7cf0 secure :-1 s=READY pgs=838 gs=2 cs=0 l=1 c_cookie=10714405986331777876 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f569c0ce810 tx=0x7f569c0d86a0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2897883113 shutdown_connections 2026-02-25T11:57:12.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/2897883113 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x5564732cb5d0 0x5564732eba80 crc :-1 s=READY pgs=58 gs=3 cs=0 l=0 c_cookie=3025376611459556536 s_cookie=17501985124026736270 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/2897883113 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55647321e0e0 0x5564731f7cf0 unknown :-1 s=CLOSED pgs=838 gs=2 cs=0 l=1 c_cookie=10714405986331777876 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/2897883113 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5564731d9440 0x5564731f7770 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f56b2d7de00 1 --2- 10.20.193.28:0/2897883113 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5564731d8b30 0x5564731f7230 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f5657fff640 0 client.17346 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:57:12.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2897883113 >> 10.20.193.28:0/2897883113 conn(0x5564730f77a0 msgr2=0x5564731f3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:12.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.676+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2897883113 shutdown_connections 2026-02-25T11:57:12.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56b2d7de00 1 -- 10.20.193.28:0/2897883113 wait complete. 2026-02-25T11:57:12.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56b2d7de00 20 client.17346 trim_cache size 0 max 16384 2026-02-25T11:57:12.684 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:57:12.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56ab577640 1 -- 10.20.193.28:0/2920647117 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56840b5750 msgr2=0x7f56840d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56ab577640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56840b5750 0x7f56840d5c00 secure :-1 s=READY pgs=374 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56a0004740 tx=0x7f56a0005f90 comp rx=0 tx=0).stop 2026-02-25T11:57:12.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56ab577640 1 -- 10.20.193.28:0/2920647117 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56ac113610 msgr2=0x7f56ac0f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:12.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56ab577640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56ac113610 0x7f56ac0f60b0 secure :-1 s=READY pgs=834 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f569c006a50 tx=0x7f569c030ce0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56ab577640 1 -- 10.20.193.28:0/2920647117 shutdown_connections 2026-02-25T11:57:12.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56ab577640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56840b5750 0x7f56840d5c00 unknown :-1 s=CLOSED pgs=374 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56ab577640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f56ac113610 0x7f56ac0f60b0 unknown :-1 s=CLOSED pgs=834 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56ab577640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f56ac10de10 0x7f56ac173ee0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56ab577640 1 --2- 10.20.193.28:0/2920647117 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f56ac10d420 0x7f56ac0f59b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:12.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56ab577640 1 -- 10.20.193.28:0/2920647117 >> 10.20.193.28:0/2920647117 conn(0x7f56ac0eacb0 msgr2=0x7f56ac10b870 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:12.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56ab577640 1 -- 10.20.193.28:0/2920647117 shutdown_connections 2026-02-25T11:57:12.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:12.677+0000 7f56ab577640 1 -- 10.20.193.28:0/2920647117 wait complete. 2026-02-25T11:57:12.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:12 trial167 ceph-mon[43392]: pgmap v495: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:57:12.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:12 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2365409140' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:57:12.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:12 trial119 ceph-mon[42710]: pgmap v495: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:57:12.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:12 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2365409140' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:57:12.778 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:57:12.961 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:13.139 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.139+0000 7f2234a1f640 1 -- 10.20.193.28:0/3426289875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2230110130 msgr2=0x7f2230115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.139+0000 7f2234a1f640 1 --2- 10.20.193.28:0/3426289875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2230110130 0x7f2230115180 secure :-1 s=READY pgs=839 gs=1 cs=0 l=1 c_cookie=9556904659444189148 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2218009950 tx=0x7f221802f190 comp rx=0 tx=0).stop 2026-02-25T11:57:13.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.139+0000 7f2234a1f640 1 -- 10.20.193.28:0/3426289875 shutdown_connections 2026-02-25T11:57:13.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.139+0000 7f2234a1f640 1 --2- 10.20.193.28:0/3426289875 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2230115930 0x7f2230176300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.139+0000 7f2234a1f640 1 --2- 10.20.193.28:0/3426289875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2230110130 0x7f2230115180 unknown :-1 s=CLOSED pgs=839 gs=1 cs=0 l=1 c_cookie=9556904659444189148 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.139+0000 7f2234a1f640 1 --2- 10.20.193.28:0/3426289875 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f223010f740 0x7f223010fb10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.139+0000 7f2234a1f640 1 -- 10.20.193.28:0/3426289875 >> 10.20.193.28:0/3426289875 conn(0x7f22300ecff0 msgr2=0x7f223010d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:13.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.139+0000 7f2234a1f640 1 -- 10.20.193.28:0/3426289875 shutdown_connections 2026-02-25T11:57:13.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.139+0000 7f2234a1f640 1 -- 10.20.193.28:0/3426289875 wait complete. 2026-02-25T11:57:13.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.140+0000 7f2234a1f640 1 Processor -- start 2026-02-25T11:57:13.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.140+0000 7f2234a1f640 1 -- start start 2026-02-25T11:57:13.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f2234a1f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f223010f740 0x7f22300f5ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f2234a1f640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2230110130 0x7f2230176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f2234a1f640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2230115930 0x7f22300f6160 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f2234a1f640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f22300fb3e0 con 0x7f223010f740 2026-02-25T11:57:13.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f2234a1f640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f22300fb550 con 0x7f2230115930 2026-02-25T11:57:13.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f2234a1f640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f22300fb6c0 con 0x7f2230110130 2026-02-25T11:57:13.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f222e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f223010f740 0x7f22300f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f222e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f223010f740 0x7f22300f5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58876/0 (socket says 10.20.193.28:58876) 2026-02-25T11:57:13.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f222e575640 1 -- 10.20.193.28:0/3529857245 learned_addr learned my addr 10.20.193.28:0/3529857245 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:13.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f222dd74640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2230110130 0x7f2230176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f222ed76640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2230115930 0x7f22300f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f222e575640 1 -- 10.20.193.28:0/3529857245 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2230110130 msgr2=0x7f2230176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f222e575640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2230110130 0x7f2230176200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f222e575640 1 -- 10.20.193.28:0/3529857245 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2230115930 msgr2=0x7f22300f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f222e575640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2230115930 0x7f22300f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.142 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f222e575640 1 -- 10.20.193.28:0/3529857245 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2218009570 con 0x7f223010f740 2026-02-25T11:57:13.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.141+0000 7f222e575640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f223010f740 0x7f22300f5ad0 secure :-1 s=READY pgs=840 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f222400d950 tx=0x7f222400de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:13.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.142+0000 7f22177fe640 1 -- 10.20.193.28:0/3529857245 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2224004540 con 0x7f223010f740 2026-02-25T11:57:13.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.142+0000 7f22177fe640 1 -- 10.20.193.28:0/3529857245 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2224009ce0 con 0x7f223010f740 2026-02-25T11:57:13.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.142+0000 7f2234a1f640 1 -- 10.20.193.28:0/3529857245 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22300fb9a0 con 0x7f223010f740 2026-02-25T11:57:13.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.142+0000 7f2234a1f640 1 -- 10.20.193.28:0/3529857245 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22300fbe70 con 0x7f223010f740 2026-02-25T11:57:13.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.142+0000 7f22177fe640 1 -- 10.20.193.28:0/3529857245 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f22240106e0 con 0x7f223010f740 2026-02-25T11:57:13.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.145+0000 7f22177fe640 1 -- 10.20.193.28:0/3529857245 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f22240097c0 con 0x7f223010f740 2026-02-25T11:57:13.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.146+0000 7f22177fe640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f22080b56e0 0x7f22080d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.146+0000 7f22177fe640 1 -- 10.20.193.28:0/3529857245 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f22240e6e50 con 0x7f223010f740 2026-02-25T11:57:13.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.147+0000 7f222dd74640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f22080b56e0 0x7f22080d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.147+0000 7f222dd74640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f22080b56e0 0x7f22080d5b90 secure :-1 s=READY pgs=375 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f22180023d0 tx=0x7f2218031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:13.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.147+0000 7f2236a1ae00 1 Processor -- start 2026-02-25T11:57:13.148 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.147+0000 7f2236a1ae00 1 -- start start 2026-02-25T11:57:13.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f2236a1ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d78760c540 0x55d78760c910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f2236a1ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 0x55d7876515b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f2236a1ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d787651af0 0x55d787671ed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f2236a1ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d7875198a0 con 0x55d78760c540 2026-02-25T11:57:13.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f2236a1ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d7875155d0 con 0x55d78760ce50 2026-02-25T11:57:13.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f2236a1ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d78750f570 con 0x55d787651af0 2026-02-25T11:57:13.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f222e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d78760c540 0x55d78760c910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f222e575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d78760c540 0x55d78760c910 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58892/0 (socket says 10.20.193.28:58892) 2026-02-25T11:57:13.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f222e575640 1 -- 10.20.193.28:0/1579454846 learned_addr learned my addr 10.20.193.28:0/1579454846 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:13.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f222ed76640 1 --2- 10.20.193.28:0/1579454846 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d787651af0 0x55d787671ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f222dd74640 1 --2- 10.20.193.28:0/1579454846 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 0x55d7876515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f222e575640 1 -- 10.20.193.28:0/1579454846 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d787651af0 msgr2=0x55d787671ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f222e575640 1 --2- 10.20.193.28:0/1579454846 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d787651af0 0x55d787671ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f222e575640 1 -- 10.20.193.28:0/1579454846 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 msgr2=0x55d7876515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f222e575640 1 --2- 10.20.193.28:0/1579454846 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 0x55d7876515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f222e575640 1 -- 10.20.193.28:0/1579454846 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7874bad10 con 0x55d78760c540 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f222dd74640 1 --2- 10.20.193.28:0/1579454846 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 0x55d7876515b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.148+0000 7f222e575640 1 --2- 10.20.193.28:0/1579454846 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d78760c540 0x55d78760c910 secure :-1 s=READY pgs=841 gs=1 cs=0 l=1 c_cookie=4620943270105778163 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f22240a97a0 tx=0x7f2224009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4020e36bdde307f3 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.149+0000 7f2214ff9640 1 -- 10.20.193.28:0/1579454846 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f22240ac9d0 con 0x55d78760c540 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.149+0000 7f2214ff9640 1 -- 10.20.193.28:0/1579454846 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f22240d0e90 con 0x55d78760c540 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.149+0000 7f2214ff9640 1 -- 10.20.193.28:0/1579454846 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2224009aa0 con 0x55d78760c540 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.149+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1579454846 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d78760c540 msgr2=0x55d78760c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.149+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/1579454846 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d78760c540 0x55d78760c910 secure :-1 s=READY pgs=841 gs=1 cs=0 l=1 c_cookie=4620943270105778163 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f22240a97a0 tx=0x7f2224009f90 comp rx=0 tx=0).stop 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.149+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1579454846 shutdown_connections 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.149+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/1579454846 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d787651af0 0x55d787671ed0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.149+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/1579454846 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 0x55d7876515b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.149+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/1579454846 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d78760c540 0x55d78760c910 unknown :-1 s=CLOSED pgs=841 gs=1 cs=0 l=1 c_cookie=4620943270105778163 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.149+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1579454846 >> 10.20.193.28:0/1579454846 conn(0x55d7875227a0 msgr2=0x55d78760ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.149+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1579454846 shutdown_connections 2026-02-25T11:57:13.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.149+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1579454846 wait complete. 2026-02-25T11:57:13.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.151+0000 7f2236a1ae00 1 -- start start 2026-02-25T11:57:13.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.151+0000 7f2236a1ae00 1 Processor -- start 2026-02-25T11:57:13.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.151+0000 7f2236a1ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d7876248e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.151+0000 7f2236a1ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d78760ce50 0x55d787624e20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.151+0000 7f2236a1ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d787651af0 0x55d787629e10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.151+0000 7f2236a1ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d78762a350 con 0x55d78760ce50 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.151+0000 7f2236a1ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d78762a4c0 con 0x55d787651af0 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.151+0000 7f2236a1ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d78762a680 con 0x55d78760c540 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.151+0000 7f222dd74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d78760ce50 0x55d787624e20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.151+0000 7f222e575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d7876248e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.151+0000 7f222e575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d7876248e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:60662/0 (socket says 10.20.193.28:60662) 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.151+0000 7f222e575640 1 -- 10.20.193.28:0/1200164416 learned_addr learned my addr 10.20.193.28:0/1200164416 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.151+0000 7f222ed76640 1 --2- 10.20.193.28:0/1200164416 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d787651af0 0x55d787629e10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.152+0000 7f222e575640 1 -- 10.20.193.28:0/1200164416 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d787651af0 msgr2=0x55d787629e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.152+0000 7f222e575640 1 --2- 10.20.193.28:0/1200164416 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d787651af0 0x55d787629e10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.152+0000 7f222e575640 1 -- 10.20.193.28:0/1200164416 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d78760ce50 msgr2=0x55d787624e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.152+0000 7f222e575640 1 --2- 10.20.193.28:0/1200164416 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d78760ce50 0x55d787624e20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.153 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.152+0000 7f222e575640 1 -- 10.20.193.28:0/1200164416 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7874933c0 con 0x55d78760c540 2026-02-25T11:57:13.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.152+0000 7f222e575640 1 --2- 10.20.193.28:0/1200164416 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d7876248e0 secure :-1 s=READY pgs=221 gs=2 cs=0 l=1 c_cookie=8644814994199458825 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f222405b270 tx=0x7f2224005ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77f88d6f5e3ab409 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:13.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.152+0000 7f21dbfff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:57:13.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.152+0000 7f21dbfff640 1 -- 10.20.193.28:0/1200164416 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f22240b8310 con 0x55d78760c540 2026-02-25T11:57:13.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.152+0000 7f21dbfff640 1 -- 10.20.193.28:0/1200164416 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f22240ac9d0 con 0x55d78760c540 2026-02-25T11:57:13.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.152+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1200164416 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22300fbe70 con 0x55d78760c540 2026-02-25T11:57:13.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.152+0000 7f21dbfff640 1 -- 10.20.193.28:0/1200164416 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f22240ae0e0 con 0x55d78760c540 2026-02-25T11:57:13.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.153+0000 7f2236a1ae00 10 client.27040 _ll_register_callbacks cb 0x55d787490a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:13.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.153+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1200164416 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d7871a56f0 con 0x55d78760c540 2026-02-25T11:57:13.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.153+0000 7f21dbfff640 1 -- 10.20.193.28:0/1200164416 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f22240b54d0 con 0x55d78760c540 2026-02-25T11:57:13.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.153+0000 7f21dbfff640 1 -- 10.20.193.28:0/1200164416 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f22240c8070 con 0x55d78760c540 2026-02-25T11:57:13.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.153+0000 7f2236a1ae00 10 client.27040 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:13.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.153+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1200164416 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55d7874a50e0 con 0x55d78760c540 2026-02-25T11:57:13.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.153+0000 7f21dbfff640 1 -- 10.20.193.28:0/1200164416 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f22240cb020 con 0x55d78760c540 2026-02-25T11:57:13.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.153+0000 7f2236a1ae00 10 client.27040 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:13.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.153+0000 7f2236a1ae00 10 client.27040 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:57:13.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.153+0000 7f2236a1ae00 20 client.27040 populate_metadata read hostname 'trial028' 2026-02-25T11:57:13.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.153+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/1200164416 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55d78764ecc0 0x55d78764f160 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.153+0000 7f2236a1ae00 4 client.27040 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:57:13.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.153+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1200164416 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7874e1fc0 con 0x55d78764ecc0 2026-02-25T11:57:13.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.154+0000 7f222dd74640 1 --2- 10.20.193.28:0/1200164416 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55d78764ecc0 0x55d78764f160 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.154+0000 7f222dd74640 1 --2- 10.20.193.28:0/1200164416 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55d78764ecc0 0x55d78764f160 crc :-1 s=READY pgs=58 gs=4 cs=0 l=0 c_cookie=13925615643257909520 s_cookie=8481612570503064687 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c141b9a57f02b110 server_cookie=75b4bdb20352ac6f in_seq=0 out_seq=0 2026-02-25T11:57:13.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.154+0000 7f21dbfff640 10 client.27040 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:57:13.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f21dbfff640 1 -- 10.20.193.28:0/1200164416 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55d7874e1fc0 con 0x55d78764ecc0 2026-02-25T11:57:13.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f21dbfff640 10 client.27040 handle_command_reply: tid=0 2026-02-25T11:57:13.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f2236a1ae00 1 client.27040 shutdown 2026-02-25T11:57:13.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1200164416 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 msgr2=0x55d7876248e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/1200164416 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d7876248e0 secure :-1 s=READY pgs=221 gs=2 cs=0 l=1 c_cookie=8644814994199458825 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f222405b270 tx=0x7f2224005ca0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1200164416 shutdown_connections 2026-02-25T11:57:13.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/1200164416 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55d78764ecc0 0x55d78764f160 crc :-1 s=READY pgs=58 gs=4 cs=0 l=0 c_cookie=13925615643257909520 s_cookie=8481612570503064687 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/1200164416 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d787651af0 0x55d787629e10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/1200164416 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d78760ce50 0x55d787624e20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/1200164416 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d7876248e0 unknown :-1 s=CLOSED pgs=221 gs=2 cs=0 l=1 c_cookie=8644814994199458825 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f21dbfff640 0 client.27040 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:57:13.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1200164416 >> 10.20.193.28:0/1200164416 conn(0x55d7875227a0 msgr2=0x55d78760b610 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:13.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1200164416 shutdown_connections 2026-02-25T11:57:13.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f2236a1ae00 1 -- 10.20.193.28:0/1200164416 wait complete. 2026-02-25T11:57:13.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.155+0000 7f2236a1ae00 20 client.27040 trim_cache size 0 max 16384 2026-02-25T11:57:13.163 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.162+0000 7f2236a1ae00 1 Processor -- start 2026-02-25T11:57:13.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.162+0000 7f2236a1ae00 1 -- start start 2026-02-25T11:57:13.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.162+0000 7f2236a1ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d787621cd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.162+0000 7f2236a1ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 0x55d787622210 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.162+0000 7f2236a1ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d787651af0 0x55d7876247f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f2236a1ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d78750d1a0 con 0x55d787651af0 2026-02-25T11:57:13.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f2236a1ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d7871cd700 con 0x55d78760ce50 2026-02-25T11:57:13.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f2236a1ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d7875778d0 con 0x55d78760c540 2026-02-25T11:57:13.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f222ed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d787651af0 0x55d7876247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f222e575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d787621cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f222e575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d787621cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:60664/0 (socket says 10.20.193.28:60664) 2026-02-25T11:57:13.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f222e575640 1 -- 10.20.193.28:0/3613443337 learned_addr learned my addr 10.20.193.28:0/3613443337 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:13.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f222dd74640 1 --2- 10.20.193.28:0/3613443337 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 0x55d787622210 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f222e575640 1 -- 10.20.193.28:0/3613443337 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 msgr2=0x55d787622210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f222e575640 1 --2- 10.20.193.28:0/3613443337 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 0x55d787622210 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f222e575640 1 -- 10.20.193.28:0/3613443337 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d787651af0 msgr2=0x55d7876247f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f222e575640 1 --2- 10.20.193.28:0/3613443337 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d787651af0 0x55d7876247f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f222e575640 1 -- 10.20.193.28:0/3613443337 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d78749fb60 con 0x55d78760c540 2026-02-25T11:57:13.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.163+0000 7f222ed76640 1 --2- 10.20.193.28:0/3613443337 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d787651af0 0x55d7876247f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:57:13.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.164+0000 7f222e575640 1 --2- 10.20.193.28:0/3613443337 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d787621cd0 secure :-1 s=READY pgs=222 gs=1 cs=0 l=1 c_cookie=7638391032864812887 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f22240ad440 tx=0x7f22240e6030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a01042ca96faf57 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:13.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.164+0000 7f21fa4e6640 1 -- 10.20.193.28:0/3613443337 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f22240b8310 con 0x55d78760c540 2026-02-25T11:57:13.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.164+0000 7f21fa4e6640 1 -- 10.20.193.28:0/3613443337 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55d7871a56f0 con 0x55d78760c540 2026-02-25T11:57:13.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.164+0000 7f21fa4e6640 1 -- 10.20.193.28:0/3613443337 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f22240a9440 con 0x55d78760c540 2026-02-25T11:57:13.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.164+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3613443337 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 msgr2=0x55d787621cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.164+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/3613443337 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d787621cd0 secure :-1 s=READY pgs=222 gs=1 cs=0 l=1 c_cookie=7638391032864812887 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f22240ad440 tx=0x7f22240e6030 comp rx=0 tx=0).stop 2026-02-25T11:57:13.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.164+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3613443337 shutdown_connections 2026-02-25T11:57:13.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.164+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/3613443337 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d787651af0 0x55d7876247f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.164+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/3613443337 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 0x55d787622210 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.164+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/3613443337 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d787621cd0 unknown :-1 s=CLOSED pgs=222 gs=1 cs=0 l=1 c_cookie=7638391032864812887 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.164+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3613443337 >> 10.20.193.28:0/3613443337 conn(0x55d7875227a0 msgr2=0x55d78760f230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:13.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.164+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3613443337 shutdown_connections 2026-02-25T11:57:13.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.164+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3613443337 wait complete. 2026-02-25T11:57:13.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f2236a1ae00 1 -- start start 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f2236a1ae00 1 Processor -- start 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f2236a1ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d787628680 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f2236a1ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 0x55d787628bc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f2236a1ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d787651af0 0x55d787629100 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f2236a1ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55d78762ce60 con 0x55d787651af0 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f2236a1ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55d78762cfd0 con 0x55d78760ce50 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f2236a1ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55d78762d190 con 0x55d78760c540 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f222ed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d787651af0 0x55d787629100 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f222e575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d787628680 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f222ed76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d787651af0 0x55d787629100 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58910/0 (socket says 10.20.193.28:58910) 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f222ed76640 1 -- 10.20.193.28:0/3860522508 learned_addr learned my addr 10.20.193.28:0/3860522508 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f222e575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d787628680 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:60674/0 (socket says 10.20.193.28:60674) 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.165+0000 7f222dd74640 1 --2- 10.20.193.28:0/3860522508 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 0x55d787628bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.166+0000 7f222e575640 1 -- 10.20.193.28:0/3860522508 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 msgr2=0x55d787628bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.166+0000 7f222e575640 1 --2- 10.20.193.28:0/3860522508 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 0x55d787628bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.166+0000 7f222e575640 1 -- 10.20.193.28:0/3860522508 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d787651af0 msgr2=0x55d787629100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.166+0000 7f222e575640 1 --2- 10.20.193.28:0/3860522508 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d787651af0 0x55d787629100 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.166+0000 7f222e575640 1 -- 10.20.193.28:0/3860522508 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7874a50e0 con 0x55d78760c540 2026-02-25T11:57:13.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.166+0000 7f222ed76640 1 --2- 10.20.193.28:0/3860522508 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d787651af0 0x55d787629100 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:57:13.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.166+0000 7f222e575640 1 --2- 10.20.193.28:0/3860522508 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d787628680 secure :-1 s=READY pgs=223 gs=1 cs=0 l=1 c_cookie=4803987148426296558 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f22240aa440 tx=0x7f2224002390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42ab30dbdf7dbcee server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:13.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.166+0000 7f21dbfff640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:57:13.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.166+0000 7f21dbfff640 1 -- 10.20.193.28:0/3860522508 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f22240c8070 con 0x55d78760c540 2026-02-25T11:57:13.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.166+0000 7f21dbfff640 1 -- 10.20.193.28:0/3860522508 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f22240b8310 con 0x55d78760c540 2026-02-25T11:57:13.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.166+0000 7f21dbfff640 1 -- 10.20.193.28:0/3860522508 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f22240725e0 con 0x55d78760c540 2026-02-25T11:57:13.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.166+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3860522508 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2220003780 con 0x55d78760c540 2026-02-25T11:57:13.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f21dbfff640 1 -- 10.20.193.28:0/3860522508 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f22240adbd0 con 0x55d78760c540 2026-02-25T11:57:13.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f2236a1ae00 10 client.27046 _ll_register_callbacks cb 0x55d787490a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:13.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3860522508 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22240d0e90 con 0x55d78760c540 2026-02-25T11:57:13.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f21dbfff640 1 -- 10.20.193.28:0/3860522508 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f22240ac9d0 con 0x55d78760c540 2026-02-25T11:57:13.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f2236a1ae00 10 client.27046 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:13.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3860522508 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55d7874936f0 con 0x55d78760c540 2026-02-25T11:57:13.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f21dbfff640 1 -- 10.20.193.28:0/3860522508 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f22240cb020 con 0x55d78760c540 2026-02-25T11:57:13.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f2236a1ae00 10 client.27046 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:13.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f2236a1ae00 10 client.27046 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:57:13.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f2236a1ae00 20 client.27046 populate_metadata read hostname 'trial028' 2026-02-25T11:57:13.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/3860522508 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55d78762e740 0x55d78771e580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f2236a1ae00 4 client.27046 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:57:13.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3860522508 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7871a2f40 con 0x55d78762e740 2026-02-25T11:57:13.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.167+0000 7f222dd74640 1 --2- 10.20.193.28:0/3860522508 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55d78762e740 0x55d78771e580 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f222dd74640 1 --2- 10.20.193.28:0/3860522508 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55d78762e740 0x55d78771e580 crc :-1 s=READY pgs=59 gs=4 cs=0 l=0 c_cookie=3773184935895235578 s_cookie=15754165008319433268 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=345d0bb53fe3fbfa server_cookie=daa2099b1dc4ca34 in_seq=0 out_seq=0 2026-02-25T11:57:13.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f21dbfff640 10 client.27046 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:57:13.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f21dbfff640 1 -- 10.20.193.28:0/3860522508 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7871a2f40 con 0x55d78762e740 2026-02-25T11:57:13.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f21dbfff640 10 client.27046 handle_command_reply: tid=0 2026-02-25T11:57:13.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f2236a1ae00 1 client.27046 shutdown 2026-02-25T11:57:13.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3860522508 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 msgr2=0x55d787628680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/3860522508 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d787628680 secure :-1 s=READY pgs=223 gs=1 cs=0 l=1 c_cookie=4803987148426296558 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f22240aa440 tx=0x7f2224002390 comp rx=0 tx=0).stop 2026-02-25T11:57:13.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3860522508 shutdown_connections 2026-02-25T11:57:13.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/3860522508 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55d78762e740 0x55d78771e580 crc :-1 s=READY pgs=59 gs=4 cs=0 l=0 c_cookie=3773184935895235578 s_cookie=15754165008319433268 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/3860522508 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55d787651af0 0x55d787629100 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/3860522508 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55d78760ce50 0x55d787628bc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f2236a1ae00 1 --2- 10.20.193.28:0/3860522508 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55d78760c540 0x55d787628680 unknown :-1 s=CLOSED pgs=223 gs=1 cs=0 l=1 c_cookie=4803987148426296558 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f21dbfff640 0 client.27046 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:57:13.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3860522508 >> 10.20.193.28:0/3860522508 conn(0x55d7875227a0 msgr2=0x55d7876247f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:13.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3860522508 shutdown_connections 2026-02-25T11:57:13.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.168+0000 7f2236a1ae00 1 -- 10.20.193.28:0/3860522508 wait complete. 2026-02-25T11:57:13.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.169+0000 7f2236a1ae00 20 client.27046 trim_cache size 0 max 16384 2026-02-25T11:57:13.176 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:57:13.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.169+0000 7f2234a1f640 1 -- 10.20.193.28:0/3529857245 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f22080b56e0 msgr2=0x7f22080d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.169+0000 7f2234a1f640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f22080b56e0 0x7f22080d5b90 secure :-1 s=READY pgs=375 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f22180023d0 tx=0x7f2218031040 comp rx=0 tx=0).stop 2026-02-25T11:57:13.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.169+0000 7f2234a1f640 1 -- 10.20.193.28:0/3529857245 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f223010f740 msgr2=0x7f22300f5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.169+0000 7f2234a1f640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f223010f740 0x7f22300f5ad0 secure :-1 s=READY pgs=840 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f222400d950 tx=0x7f222400de20 comp rx=0 tx=0).stop 2026-02-25T11:57:13.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.169+0000 7f2234a1f640 1 -- 10.20.193.28:0/3529857245 shutdown_connections 2026-02-25T11:57:13.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.169+0000 7f2234a1f640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f22080b56e0 0x7f22080d5b90 unknown :-1 s=CLOSED pgs=375 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.169+0000 7f2234a1f640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2230115930 0x7f22300f6160 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.169+0000 7f2234a1f640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2230110130 0x7f2230176200 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.169+0000 7f2234a1f640 1 --2- 10.20.193.28:0/3529857245 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f223010f740 0x7f22300f5ad0 unknown :-1 s=CLOSED pgs=840 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.169+0000 7f2234a1f640 1 -- 10.20.193.28:0/3529857245 >> 10.20.193.28:0/3529857245 conn(0x7f22300ecff0 msgr2=0x7f2230111f40 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:13.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.169+0000 7f2234a1f640 1 -- 10.20.193.28:0/3529857245 shutdown_connections 2026-02-25T11:57:13.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.169+0000 7f2234a1f640 1 -- 10.20.193.28:0/3529857245 wait complete. 2026-02-25T11:57:13.263 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:57:13.448 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:13.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.635+0000 7f9c4b81d640 1 -- 10.20.193.28:0/2989818468 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9c44110130 msgr2=0x7f9c44115180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.635+0000 7f9c4b81d640 1 --2- 10.20.193.28:0/2989818468 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9c44110130 0x7f9c44115180 secure :-1 s=READY pgs=842 gs=1 cs=0 l=1 c_cookie=7080051706240389002 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9c38009950 tx=0x7f9c3802f190 comp rx=0 tx=0).stop 2026-02-25T11:57:13.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.635+0000 7f9c4b81d640 1 -- 10.20.193.28:0/2989818468 shutdown_connections 2026-02-25T11:57:13.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.635+0000 7f9c4b81d640 1 --2- 10.20.193.28:0/2989818468 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9c44115930 0x7f9c44176300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.635+0000 7f9c4b81d640 1 --2- 10.20.193.28:0/2989818468 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9c44110130 0x7f9c44115180 unknown :-1 s=CLOSED pgs=842 gs=1 cs=0 l=1 c_cookie=7080051706240389002 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.635+0000 7f9c4b81d640 1 --2- 10.20.193.28:0/2989818468 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9c4410f740 0x7f9c4410fb10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.635+0000 7f9c4b81d640 1 -- 10.20.193.28:0/2989818468 >> 10.20.193.28:0/2989818468 conn(0x7f9c440ecff0 msgr2=0x7f9c4410d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:13.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.635+0000 7f9c4b81d640 1 -- 10.20.193.28:0/2989818468 shutdown_connections 2026-02-25T11:57:13.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.635+0000 7f9c4b81d640 1 -- 10.20.193.28:0/2989818468 wait complete. 2026-02-25T11:57:13.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.636+0000 7f9c4b81d640 1 Processor -- start 2026-02-25T11:57:13.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.636+0000 7f9c4b81d640 1 -- start start 2026-02-25T11:57:13.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.636+0000 7f9c4b81d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9c4410f740 0x7f9c4410b380 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.636+0000 7f9c4b81d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9c44110130 0x7f9c44176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.636+0000 7f9c4b81d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9c44115930 0x7f9c4410ba10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.636+0000 7f9c4b81d640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9c440f16e0 con 0x7f9c44115930 2026-02-25T11:57:13.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.636+0000 7f9c4b81d640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9c440f1850 con 0x7f9c4410f740 2026-02-25T11:57:13.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.636+0000 7f9c4b81d640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9c440f19c0 con 0x7f9c44110130 2026-02-25T11:57:13.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.636+0000 7f9c49592640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9c4410f740 0x7f9c4410b380 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.636+0000 7f9c49592640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9c4410f740 0x7f9c4410b380 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:54722/0 (socket says 10.20.193.28:54722) 2026-02-25T11:57:13.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.636+0000 7f9c49592640 1 -- 10.20.193.28:0/1086931888 learned_addr learned my addr 10.20.193.28:0/1086931888 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:13.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.636+0000 7f9c48d91640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9c44110130 0x7f9c44176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.637+0000 7f9c49d93640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9c44115930 0x7f9c4410ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.637+0000 7f9c48d91640 1 -- 10.20.193.28:0/1086931888 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9c4410f740 msgr2=0x7f9c4410b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.637+0000 7f9c48d91640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9c4410f740 0x7f9c4410b380 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.637+0000 7f9c48d91640 1 -- 10.20.193.28:0/1086931888 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9c44115930 msgr2=0x7f9c4410ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.637+0000 7f9c48d91640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9c44115930 0x7f9c4410ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.637+0000 7f9c48d91640 1 -- 10.20.193.28:0/1086931888 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c38009570 con 0x7f9c44110130 2026-02-25T11:57:13.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.637+0000 7f9c49d93640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9c44115930 0x7f9c4410ba10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:57:13.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.638+0000 7f9c48d91640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9c44110130 0x7f9c44176200 secure :-1 s=READY pgs=224 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9c38009950 tx=0x7f9c380028d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:13.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.639+0000 7f9c327fc640 1 -- 10.20.193.28:0/1086931888 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9c3803d070 con 0x7f9c44110130 2026-02-25T11:57:13.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.639+0000 7f9c327fc640 1 -- 10.20.193.28:0/1086931888 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9c380043b0 con 0x7f9c44110130 2026-02-25T11:57:13.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.639+0000 7f9c327fc640 1 -- 10.20.193.28:0/1086931888 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9c38041720 con 0x7f9c44110130 2026-02-25T11:57:13.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.639+0000 7f9c4b81d640 1 -- 10.20.193.28:0/1086931888 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c440f1c40 con 0x7f9c44110130 2026-02-25T11:57:13.640 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.639+0000 7f9c4b81d640 1 -- 10.20.193.28:0/1086931888 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c440f2060 con 0x7f9c44110130 2026-02-25T11:57:13.642 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.641+0000 7f9c327fc640 1 -- 10.20.193.28:0/1086931888 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f9c380386a0 con 0x7f9c44110130 2026-02-25T11:57:13.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.643+0000 7f9c327fc640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9c200b5760 0x7f9c200d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.643+0000 7f9c327fc640 1 -- 10.20.193.28:0/1086931888 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9c3810afa0 con 0x7f9c44110130 2026-02-25T11:57:13.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.644+0000 7f9c49592640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9c200b5760 0x7f9c200d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.644+0000 7f9c49592640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9c200b5760 0x7f9c200d5c10 secure :-1 s=READY pgs=376 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9c340095b0 tx=0x7f9c34005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:13.645 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.645+0000 7f9c4d818e00 1 Processor -- start 2026-02-25T11:57:13.646 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.645+0000 7f9c4d818e00 1 -- start start 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.645+0000 7f9c4d818e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dedb30 0x563595dedf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.645+0000 7f9c4d818e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 0x563595e32ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.645+0000 7f9c4d818e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 0x563595e534c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.645+0000 7f9c4d818e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x563595d038a0 con 0x563595dee440 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.645+0000 7f9c4d818e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x563595cff5d0 con 0x563595dedb30 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.645+0000 7f9c4d818e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x563595cf9570 con 0x563595e330e0 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c49592640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dedb30 0x563595dedf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c48d91640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 0x563595e32ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c49d93640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 0x563595e534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c48d91640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 0x563595e32ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58940/0 (socket says 10.20.193.28:58940) 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c48d91640 1 -- 10.20.193.28:0/1492497596 learned_addr learned my addr 10.20.193.28:0/1492497596 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c48d91640 1 -- 10.20.193.28:0/1492497596 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 msgr2=0x563595e534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c48d91640 1 --2- 10.20.193.28:0/1492497596 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 0x563595e534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c48d91640 1 -- 10.20.193.28:0/1492497596 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dedb30 msgr2=0x563595dedf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c48d91640 1 --2- 10.20.193.28:0/1492497596 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dedb30 0x563595dedf00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c48d91640 1 -- 10.20.193.28:0/1492497596 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563595ca4d10 con 0x563595dee440 2026-02-25T11:57:13.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c48d91640 1 --2- 10.20.193.28:0/1492497596 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 0x563595e32ba0 secure :-1 s=READY pgs=843 gs=1 cs=0 l=1 c_cookie=11363083999768053441 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9c38004030 tx=0x7f9c3802fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db1c8b18f1986c1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:13.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c11ce5640 1 -- 10.20.193.28:0/1492497596 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9c380d0db0 con 0x563595dee440 2026-02-25T11:57:13.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c11ce5640 1 -- 10.20.193.28:0/1492497596 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9c380d0bd0 con 0x563595dee440 2026-02-25T11:57:13.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.646+0000 7f9c11ce5640 1 -- 10.20.193.28:0/1492497596 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9c380cf3b0 con 0x563595dee440 2026-02-25T11:57:13.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.647+0000 7f9c4d818e00 1 -- 10.20.193.28:0/1492497596 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 msgr2=0x563595e32ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.647+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/1492497596 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 0x563595e32ba0 secure :-1 s=READY pgs=843 gs=1 cs=0 l=1 c_cookie=11363083999768053441 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9c38004030 tx=0x7f9c3802fe20 comp rx=0 tx=0).stop 2026-02-25T11:57:13.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.647+0000 7f9c4d818e00 1 -- 10.20.193.28:0/1492497596 shutdown_connections 2026-02-25T11:57:13.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.647+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/1492497596 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 0x563595e534c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.647+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/1492497596 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 0x563595e32ba0 unknown :-1 s=CLOSED pgs=843 gs=1 cs=0 l=1 c_cookie=11363083999768053441 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.647+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/1492497596 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dedb30 0x563595dedf00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.647+0000 7f9c4d818e00 1 -- 10.20.193.28:0/1492497596 >> 10.20.193.28:0/1492497596 conn(0x563595d0c7a0 msgr2=0x563595dec410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:13.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.647+0000 7f9c4d818e00 1 -- 10.20.193.28:0/1492497596 shutdown_connections 2026-02-25T11:57:13.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.647+0000 7f9c4d818e00 1 -- 10.20.193.28:0/1492497596 wait complete. 2026-02-25T11:57:13.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.648+0000 7f9c4d818e00 1 -- start start 2026-02-25T11:57:13.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.649+0000 7f9c4d818e00 1 Processor -- start 2026-02-25T11:57:13.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.649+0000 7f9c4d818e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595dedb30 0x563595e05b10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c4d818e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 0x563595e06050 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c4d818e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595e330e0 0x563595e0b040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c4d818e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x563595e0b580 con 0x563595dee440 2026-02-25T11:57:13.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c4d818e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x563595e0b6f0 con 0x563595e330e0 2026-02-25T11:57:13.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c4d818e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x563595e0b8b0 con 0x563595dedb30 2026-02-25T11:57:13.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c49592640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595dedb30 0x563595e05b10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c48d91640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 0x563595e06050 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c48d91640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 0x563595e06050 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58944/0 (socket says 10.20.193.28:58944) 2026-02-25T11:57:13.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c48d91640 1 -- 10.20.193.28:0/312001927 learned_addr learned my addr 10.20.193.28:0/312001927 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:13.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c49d93640 1 --2- 10.20.193.28:0/312001927 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595e330e0 0x563595e0b040 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c48d91640 1 -- 10.20.193.28:0/312001927 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595dedb30 msgr2=0x563595e05b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c48d91640 1 --2- 10.20.193.28:0/312001927 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595dedb30 0x563595e05b10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c48d91640 1 -- 10.20.193.28:0/312001927 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595e330e0 msgr2=0x563595e0b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c48d91640 1 --2- 10.20.193.28:0/312001927 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595e330e0 0x563595e0b040 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c48d91640 1 -- 10.20.193.28:0/312001927 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563595c7d3c0 con 0x563595dee440 2026-02-25T11:57:13.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c49d93640 1 --2- 10.20.193.28:0/312001927 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595e330e0 0x563595e0b040 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:57:13.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9c48d91640 1 --2- 10.20.193.28:0/312001927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 0x563595e06050 secure :-1 s=READY pgs=844 gs=1 cs=0 l=1 c_cookie=9842531834070212275 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9c380d1360 tx=0x7f9c3802f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8897b28e1c3a9eb3 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:13.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.650+0000 7f9bfeffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:13.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.651+0000 7f9bfeffd640 1 -- 10.20.193.28:0/312001927 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9c38038980 con 0x563595dee440 2026-02-25T11:57:13.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.651+0000 7f9c4d818e00 1 -- 10.20.193.28:0/312001927 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c440f2060 con 0x563595dee440 2026-02-25T11:57:13.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.651+0000 7f9c4d818e00 10 client.17376 _ll_register_callbacks cb 0x563595c7aa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:13.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.651+0000 7f9c4d818e00 1 -- 10.20.193.28:0/312001927 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56359598f6f0 con 0x563595dee440 2026-02-25T11:57:13.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.651+0000 7f9bfeffd640 1 -- 10.20.193.28:0/312001927 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9c380d7040 con 0x563595dee440 2026-02-25T11:57:13.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.651+0000 7f9bfeffd640 1 -- 10.20.193.28:0/312001927 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9c380ccf80 con 0x563595dee440 2026-02-25T11:57:13.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.651+0000 7f9bfeffd640 1 -- 10.20.193.28:0/312001927 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9c380d9620 con 0x563595dee440 2026-02-25T11:57:13.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.651+0000 7f9bfeffd640 1 -- 10.20.193.28:0/312001927 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9c380d2d10 con 0x563595dee440 2026-02-25T11:57:13.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.651+0000 7f9c4d818e00 10 client.17376 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:13.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.651+0000 7f9c4d818e00 1 -- 10.20.193.28:0/312001927 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x563595c8f0e0 con 0x563595dee440 2026-02-25T11:57:13.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.652+0000 7f9bfeffd640 1 -- 10.20.193.28:0/312001927 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9c380d06c0 con 0x563595dee440 2026-02-25T11:57:13.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.652+0000 7f9c4d818e00 10 client.17376 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:13.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.652+0000 7f9c4d818e00 10 client.17376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:13.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.652+0000 7f9c4d818e00 20 client.17376 populate_metadata read hostname 'trial028' 2026-02-25T11:57:13.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.652+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/312001927 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563595e2ff60 0x563595e303d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.652+0000 7f9c4d818e00 4 client.17376 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:57:13.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.652+0000 7f9c4d818e00 1 -- 10.20.193.28:0/312001927 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563595ccbfc0 con 0x563595e2ff60 2026-02-25T11:57:13.654 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.652+0000 7f9c49592640 1 --2- 10.20.193.28:0/312001927 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563595e2ff60 0x563595e303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.653+0000 7f9bfeffd640 10 client.17376 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:13.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.653+0000 7f9c49592640 1 --2- 10.20.193.28:0/312001927 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563595e2ff60 0x563595e303d0 crc :-1 s=READY pgs=162 gs=4 cs=0 l=0 c_cookie=13530909962715843078 s_cookie=12914816613610088227 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc772ebd778ca06 server_cookie=b33aa548d18ae723 in_seq=0 out_seq=0 2026-02-25T11:57:13.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.653+0000 7f9bfeffd640 1 -- 10.20.193.28:0/312001927 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x563595ccbfc0 con 0x563595e2ff60 2026-02-25T11:57:13.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.653+0000 7f9bfeffd640 10 client.17376 handle_command_reply: tid=0 2026-02-25T11:57:13.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.653+0000 7f9c4d818e00 1 client.17376 shutdown 2026-02-25T11:57:13.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.654+0000 7f9c4d818e00 1 -- 10.20.193.28:0/312001927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 msgr2=0x563595e06050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.654+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/312001927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 0x563595e06050 secure :-1 s=READY pgs=844 gs=1 cs=0 l=1 c_cookie=9842531834070212275 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9c380d1360 tx=0x7f9c3802f190 comp rx=0 tx=0).stop 2026-02-25T11:57:13.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.654+0000 7f9c4d818e00 1 -- 10.20.193.28:0/312001927 shutdown_connections 2026-02-25T11:57:13.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.654+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/312001927 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563595e2ff60 0x563595e303d0 crc :-1 s=READY pgs=162 gs=4 cs=0 l=0 c_cookie=13530909962715843078 s_cookie=12914816613610088227 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.654+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/312001927 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595e330e0 0x563595e0b040 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.654+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/312001927 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dee440 0x563595e06050 unknown :-1 s=CLOSED pgs=844 gs=1 cs=0 l=1 c_cookie=9842531834070212275 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.654+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/312001927 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595dedb30 0x563595e05b10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.654+0000 7f9c4d818e00 1 -- 10.20.193.28:0/312001927 >> 10.20.193.28:0/312001927 conn(0x563595d0c7a0 msgr2=0x563595decb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:13.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.654+0000 7f9bfeffd640 0 client.17376 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:13.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.654+0000 7f9c4d818e00 1 -- 10.20.193.28:0/312001927 shutdown_connections 2026-02-25T11:57:13.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.654+0000 7f9c4d818e00 1 -- 10.20.193.28:0/312001927 wait complete. 2026-02-25T11:57:13.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.654+0000 7f9c4d818e00 20 client.17376 trim_cache size 0 max 16384 2026-02-25T11:57:13.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.661+0000 7f9c4d818e00 1 Processor -- start 2026-02-25T11:57:13.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.661+0000 7f9c4d818e00 1 -- start start 2026-02-25T11:57:13.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.661+0000 7f9c4d818e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 0x563595e30cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.661+0000 7f9c4d818e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dee440 0x563595e311f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.661+0000 7f9c4d818e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 0x563595e31730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.661+0000 7f9c4d818e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x563595cf71a0 con 0x563595dedb30 2026-02-25T11:57:13.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.661+0000 7f9c4d818e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5635959b7700 con 0x563595dee440 2026-02-25T11:57:13.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.661+0000 7f9c4d818e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x563595cff5d0 con 0x563595e330e0 2026-02-25T11:57:13.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.661+0000 7f9c49592640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 0x563595e30cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c49592640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 0x563595e30cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58952/0 (socket says 10.20.193.28:58952) 2026-02-25T11:57:13.662 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c49592640 1 -- 10.20.193.28:0/4290310965 learned_addr learned my addr 10.20.193.28:0/4290310965 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c49d93640 1 --2- 10.20.193.28:0/4290310965 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 0x563595e31730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c48d91640 1 --2- 10.20.193.28:0/4290310965 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dee440 0x563595e311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c49592640 1 -- 10.20.193.28:0/4290310965 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 msgr2=0x563595e31730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c49592640 1 --2- 10.20.193.28:0/4290310965 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 0x563595e31730 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c49592640 1 -- 10.20.193.28:0/4290310965 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dee440 msgr2=0x563595e311f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c49592640 1 --2- 10.20.193.28:0/4290310965 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dee440 0x563595e311f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c49592640 1 -- 10.20.193.28:0/4290310965 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563595c89b60 con 0x563595dedb30 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c49592640 1 --2- 10.20.193.28:0/4290310965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 0x563595e30cb0 secure :-1 s=READY pgs=845 gs=1 cs=0 l=1 c_cookie=7381359040109988529 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9c34003fd0 tx=0x7f9c34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666fdaf0595fcab1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c114e4640 1 -- 10.20.193.28:0/4290310965 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9c34021070 con 0x563595dedb30 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c114e4640 1 -- 10.20.193.28:0/4290310965 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9c34022040 con 0x563595dedb30 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c114e4640 1 -- 10.20.193.28:0/4290310965 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9c3401c410 con 0x563595dedb30 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c4d818e00 1 -- 10.20.193.28:0/4290310965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 msgr2=0x563595e30cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.662+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/4290310965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 0x563595e30cb0 secure :-1 s=READY pgs=845 gs=1 cs=0 l=1 c_cookie=7381359040109988529 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9c34003fd0 tx=0x7f9c34014000 comp rx=0 tx=0).stop 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.663+0000 7f9c4d818e00 1 -- 10.20.193.28:0/4290310965 shutdown_connections 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.663+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/4290310965 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 0x563595e31730 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.663 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.663+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/4290310965 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dee440 0x563595e311f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.663+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/4290310965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 0x563595e30cb0 unknown :-1 s=CLOSED pgs=845 gs=1 cs=0 l=1 c_cookie=7381359040109988529 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.663+0000 7f9c4d818e00 1 -- 10.20.193.28:0/4290310965 >> 10.20.193.28:0/4290310965 conn(0x563595d0c7a0 msgr2=0x563595debd10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:13.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.663+0000 7f9c4d818e00 1 -- 10.20.193.28:0/4290310965 shutdown_connections 2026-02-25T11:57:13.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.663+0000 7f9c4d818e00 1 -- 10.20.193.28:0/4290310965 wait complete. 2026-02-25T11:57:13.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.664+0000 7f9c4d818e00 1 -- start start 2026-02-25T11:57:13.664 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.664+0000 7f9c4d818e00 1 Processor -- start 2026-02-25T11:57:13.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c4d818e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 0x563595e0e820 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c4d818e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dee440 0x563595e0ed60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c4d818e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 0x563595e0f2c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c4d818e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x563595e97790 con 0x563595dedb30 2026-02-25T11:57:13.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c49592640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 0x563595e0e820 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c49592640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 0x563595e0e820 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58958/0 (socket says 10.20.193.28:58958) 2026-02-25T11:57:13.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c49592640 1 -- 10.20.193.28:0/3438224734 learned_addr learned my addr 10.20.193.28:0/3438224734 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:13.665 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c4d818e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x563595e97900 con 0x563595dee440 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c4d818e00 1 -- 10.20.193.28:0/3438224734 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x563595e97ac0 con 0x563595e330e0 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c48d91640 1 --2- 10.20.193.28:0/3438224734 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dee440 0x563595e0ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c49592640 1 -- 10.20.193.28:0/3438224734 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 msgr2=0x563595e0f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c49592640 1 --2- 10.20.193.28:0/3438224734 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 0x563595e0f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c49592640 1 -- 10.20.193.28:0/3438224734 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dee440 msgr2=0x563595e0ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c49592640 1 --2- 10.20.193.28:0/3438224734 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dee440 0x563595e0ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c49592640 1 -- 10.20.193.28:0/3438224734 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563595c7d6f0 con 0x563595dedb30 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9c49592640 1 --2- 10.20.193.28:0/3438224734 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 0x563595e0e820 secure :-1 s=READY pgs=846 gs=1 cs=0 l=1 c_cookie=428548597600540263 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9c340096c0 tx=0x7f9c34005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f282add0f8d667 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.665+0000 7f9bfeffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.666+0000 7f9bfeffd640 1 -- 10.20.193.28:0/3438224734 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9c34021070 con 0x563595dedb30 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.666+0000 7f9c4d818e00 1 -- 10.20.193.28:0/3438224734 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c34011440 con 0x563595dedb30 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.666+0000 7f9c4d818e00 10 client.17382 _ll_register_callbacks cb 0x563595c7aa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.666+0000 7f9c4d818e00 1 -- 10.20.193.28:0/3438224734 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c380d0bd0 con 0x563595dedb30 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.666+0000 7f9bfeffd640 1 -- 10.20.193.28:0/3438224734 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9c3402c860 con 0x563595dedb30 2026-02-25T11:57:13.666 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.666+0000 7f9bfeffd640 1 -- 10.20.193.28:0/3438224734 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9c3401cca0 con 0x563595dedb30 2026-02-25T11:57:13.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.666+0000 7f9bfeffd640 1 -- 10.20.193.28:0/3438224734 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9c3401ce30 con 0x563595dedb30 2026-02-25T11:57:13.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.666+0000 7f9bfeffd640 1 -- 10.20.193.28:0/3438224734 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9c3401c5a0 con 0x563595dedb30 2026-02-25T11:57:13.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.666+0000 7f9c4d818e00 10 client.17382 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:13.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.666+0000 7f9c4d818e00 1 -- 10.20.193.28:0/3438224734 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x563595c8f340 con 0x563595dedb30 2026-02-25T11:57:13.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.667+0000 7f9bfeffd640 1 -- 10.20.193.28:0/3438224734 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9c34033070 con 0x563595dedb30 2026-02-25T11:57:13.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.667+0000 7f9c4d818e00 10 client.17382 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:13.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.667+0000 7f9c4d818e00 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:13.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.667+0000 7f9c4d818e00 20 client.17382 populate_metadata read hostname 'trial028' 2026-02-25T11:57:13.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.667+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/3438224734 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563595edfb60 0x563595efffe0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:13.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.667+0000 7f9c4d818e00 4 client.17382 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:57:13.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.667+0000 7f9c4d818e00 1 -- 10.20.193.28:0/3438224734 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56359598cf40 con 0x563595edfb60 2026-02-25T11:57:13.667 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.667+0000 7f9c48d91640 1 --2- 10.20.193.28:0/3438224734 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563595edfb60 0x563595efffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:13.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.667+0000 7f9c48d91640 1 --2- 10.20.193.28:0/3438224734 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563595edfb60 0x563595efffe0 crc :-1 s=READY pgs=163 gs=4 cs=0 l=0 c_cookie=10504758915347874359 s_cookie=11906961171423397519 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91c86693b06d8237 server_cookie=a53e0619810f868f in_seq=0 out_seq=0 2026-02-25T11:57:13.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.667+0000 7f9bfeffd640 10 client.17382 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:13.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9bfeffd640 1 -- 10.20.193.28:0/3438224734 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56359598cf40 con 0x563595edfb60 2026-02-25T11:57:13.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9bfeffd640 10 client.17382 handle_command_reply: tid=0 2026-02-25T11:57:13.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9c4d818e00 1 client.17382 shutdown 2026-02-25T11:57:13.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9c4d818e00 1 -- 10.20.193.28:0/3438224734 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 msgr2=0x563595e0e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/3438224734 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 0x563595e0e820 secure :-1 s=READY pgs=846 gs=1 cs=0 l=1 c_cookie=428548597600540263 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9c340096c0 tx=0x7f9c34005780 comp rx=0 tx=0).stop 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9c4d818e00 1 -- 10.20.193.28:0/3438224734 shutdown_connections 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/3438224734 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x563595edfb60 0x563595efffe0 crc :-1 s=READY pgs=163 gs=4 cs=0 l=0 c_cookie=10504758915347874359 s_cookie=11906961171423397519 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/3438224734 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x563595e330e0 0x563595e0f2c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/3438224734 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x563595dee440 0x563595e0ed60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9c4d818e00 1 --2- 10.20.193.28:0/3438224734 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x563595dedb30 0x563595e0e820 unknown :-1 s=CLOSED pgs=846 gs=1 cs=0 l=1 c_cookie=428548597600540263 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9c4d818e00 1 -- 10.20.193.28:0/3438224734 >> 10.20.193.28:0/3438224734 conn(0x563595d0c7a0 msgr2=0x563595dec500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9bfeffd640 0 client.17382 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9c4d818e00 1 -- 10.20.193.28:0/3438224734 shutdown_connections 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9c4d818e00 1 -- 10.20.193.28:0/3438224734 wait complete. 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.668+0000 7f9c4d818e00 20 client.17382 trim_cache size 0 max 16384 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.669+0000 7f9c4b81d640 1 -- 10.20.193.28:0/1086931888 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9c200b5760 msgr2=0x7f9c200d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.669+0000 7f9c4b81d640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9c200b5760 0x7f9c200d5c10 secure :-1 s=READY pgs=376 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9c340095b0 tx=0x7f9c34005f90 comp rx=0 tx=0).stop 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.669+0000 7f9c4b81d640 1 -- 10.20.193.28:0/1086931888 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9c44110130 msgr2=0x7f9c44176200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.669+0000 7f9c4b81d640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9c44110130 0x7f9c44176200 secure :-1 s=READY pgs=224 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9c38009950 tx=0x7f9c380028d0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.669+0000 7f9c4b81d640 1 -- 10.20.193.28:0/1086931888 shutdown_connections 2026-02-25T11:57:13.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.669+0000 7f9c4b81d640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9c200b5760 0x7f9c200d5c10 unknown :-1 s=CLOSED pgs=376 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.669+0000 7f9c4b81d640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9c44115930 0x7f9c4410ba10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.669+0000 7f9c4b81d640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9c44110130 0x7f9c44176200 unknown :-1 s=CLOSED pgs=224 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.669+0000 7f9c4b81d640 1 --2- 10.20.193.28:0/1086931888 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9c4410f740 0x7f9c4410b380 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:13.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.669+0000 7f9c4b81d640 1 -- 10.20.193.28:0/1086931888 >> 10.20.193.28:0/1086931888 conn(0x7f9c440ecff0 msgr2=0x7f9c44112f90 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:13.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.669+0000 7f9c4b81d640 1 -- 10.20.193.28:0/1086931888 shutdown_connections 2026-02-25T11:57:13.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:13.669+0000 7f9c4b81d640 1 -- 10.20.193.28:0/1086931888 wait complete. 2026-02-25T11:57:14.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:14 trial028 ceph-mon[39224]: pgmap v496: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:57:14.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:14 trial167 ceph-mon[43392]: pgmap v496: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:57:14.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:14 trial119 ceph-mon[42710]: pgmap v496: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:57:14.768 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:57:14.769 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:57:14.951 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:15.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.139+0000 7f72d622b640 1 -- 10.20.193.28:0/1002650932 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f72d01140a0 msgr2=0x7f72d0114470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.140 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.139+0000 7f72d622b640 1 --2- 10.20.193.28:0/1002650932 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f72d01140a0 0x7f72d0114470 secure :-1 s=READY pgs=847 gs=1 cs=0 l=1 c_cookie=7439233601791763191 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f72c0009950 tx=0x7f72c002f190 comp rx=0 tx=0).stop 2026-02-25T11:57:15.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.139+0000 7f72d622b640 1 -- 10.20.193.28:0/1002650932 shutdown_connections 2026-02-25T11:57:15.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.139+0000 7f72d622b640 1 --2- 10.20.193.28:0/1002650932 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f72d0119d30 0x7f72d017a700 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.139+0000 7f72d622b640 1 --2- 10.20.193.28:0/1002650932 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f72d0114a90 0x7f72d0119580 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.139+0000 7f72d622b640 1 --2- 10.20.193.28:0/1002650932 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f72d01140a0 0x7f72d0114470 unknown :-1 s=CLOSED pgs=847 gs=1 cs=0 l=1 c_cookie=7439233601791763191 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.139+0000 7f72d622b640 1 -- 10.20.193.28:0/1002650932 >> 10.20.193.28:0/1002650932 conn(0x7f72d0064c40 msgr2=0x7f72d0065040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:15.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.139+0000 7f72d622b640 1 -- 10.20.193.28:0/1002650932 shutdown_connections 2026-02-25T11:57:15.141 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.139+0000 7f72d622b640 1 -- 10.20.193.28:0/1002650932 wait complete. 2026-02-25T11:57:15.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.140+0000 7f72d622b640 1 Processor -- start 2026-02-25T11:57:15.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.140+0000 7f72d622b640 1 -- start start 2026-02-25T11:57:15.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.140+0000 7f72d622b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f72d01140a0 0x7f72d0122510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.140+0000 7f72d622b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f72d0114a90 0x7f72d0122a50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.140+0000 7f72d622b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f72d0119d30 0x7f72d0127e20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.140+0000 7f72d622b640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0128440 con 0x7f72d0119d30 2026-02-25T11:57:15.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.140+0000 7f72d622b640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01285b0 con 0x7f72d01140a0 2026-02-25T11:57:15.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.140+0000 7f72d622b640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0128770 con 0x7f72d0114a90 2026-02-25T11:57:15.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.140+0000 7f72d5a2a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f72d0119d30 0x7f72d0127e20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.140+0000 7f72d5a2a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f72d0119d30 0x7f72d0127e20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:58984/0 (socket says 10.20.193.28:58984) 2026-02-25T11:57:15.143 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.140+0000 7f72d5a2a640 1 -- 10.20.193.28:0/3699174710 learned_addr learned my addr 10.20.193.28:0/3699174710 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:15.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.140+0000 7f72d5229640 1 --2- 10.20.193.28:0/3699174710 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f72d01140a0 0x7f72d0122510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.141+0000 7f72d5a2a640 1 -- 10.20.193.28:0/3699174710 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f72d0114a90 msgr2=0x7f72d0122a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.141+0000 7f72d5a2a640 1 --2- 10.20.193.28:0/3699174710 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f72d0114a90 0x7f72d0122a50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.141+0000 7f72d5a2a640 1 -- 10.20.193.28:0/3699174710 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f72d01140a0 msgr2=0x7f72d0122510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.141+0000 7f72d5a2a640 1 --2- 10.20.193.28:0/3699174710 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f72d01140a0 0x7f72d0122510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.141+0000 7f72d5a2a640 1 -- 10.20.193.28:0/3699174710 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c0009570 con 0x7f72d0119d30 2026-02-25T11:57:15.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.141+0000 7f72d5a2a640 1 --2- 10.20.193.28:0/3699174710 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f72d0119d30 0x7f72d0127e20 secure :-1 s=READY pgs=848 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f72cc00f8d0 tx=0x7f72cc00fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:15.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.141+0000 7f72be7fc640 1 -- 10.20.193.28:0/3699174710 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f72cc00dce0 con 0x7f72d0119d30 2026-02-25T11:57:15.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.141+0000 7f72be7fc640 1 -- 10.20.193.28:0/3699174710 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f72cc00de40 con 0x7f72d0119d30 2026-02-25T11:57:15.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.141+0000 7f72be7fc640 1 -- 10.20.193.28:0/3699174710 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f72cc0116b0 con 0x7f72d0119d30 2026-02-25T11:57:15.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.141+0000 7f72d622b640 1 -- 10.20.193.28:0/3699174710 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d0128a50 con 0x7f72d0119d30 2026-02-25T11:57:15.144 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.141+0000 7f72d622b640 1 -- 10.20.193.28:0/3699174710 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d0169150 con 0x7f72d0119d30 2026-02-25T11:57:15.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.144+0000 7f72be7fc640 1 -- 10.20.193.28:0/3699174710 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f72cc020060 con 0x7f72d0119d30 2026-02-25T11:57:15.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.145+0000 7f72be7fc640 1 --2- 10.20.193.28:0/3699174710 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f72ac0b56e0 0x7f72ac0d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.145+0000 7f72be7fc640 1 -- 10.20.193.28:0/3699174710 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f72cc015070 con 0x7f72d0119d30 2026-02-25T11:57:15.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.146+0000 7f72d5229640 1 --2- 10.20.193.28:0/3699174710 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f72ac0b56e0 0x7f72ac0d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.146+0000 7f72d5229640 1 --2- 10.20.193.28:0/3699174710 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f72ac0b56e0 0x7f72ac0d5b90 secure :-1 s=READY pgs=377 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f72c00023d0 tx=0x7f72c0031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:15.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.147+0000 7f72d94afe00 1 Processor -- start 2026-02-25T11:57:15.147 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.147+0000 7f72d94afe00 1 -- start start 2026-02-25T11:57:15.149 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.147+0000 7f72d94afe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc26750 0x55f2cdc26b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.147+0000 7f72d94afe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc6b610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.147+0000 7f72d94afe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc8bf30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.147+0000 7f72d94afe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdb3cd80 con 0x55f2cdc27060 2026-02-25T11:57:15.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.147+0000 7f72d94afe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdb38ab0 con 0x55f2cdc26750 2026-02-25T11:57:15.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.147+0000 7f72d94afe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdb32a50 con 0x55f2cdc6bb50 2026-02-25T11:57:15.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.147+0000 7f72d5229640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc26750 0x55f2cdc26b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.147+0000 7f72d5229640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc26750 0x55f2cdc26b20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:54780/0 (socket says 10.20.193.28:54780) 2026-02-25T11:57:15.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.147+0000 7f72d5229640 1 -- 10.20.193.28:0/1695627081 learned_addr learned my addr 10.20.193.28:0/1695627081 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:15.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.147+0000 7f72d5a2a640 1 --2- 10.20.193.28:0/1695627081 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc8bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.148+0000 7f72d4a28640 1 --2- 10.20.193.28:0/1695627081 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc6b610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.148+0000 7f72d5229640 1 -- 10.20.193.28:0/1695627081 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc6bb50 msgr2=0x55f2cdc8bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.150 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.148+0000 7f72d5229640 1 --2- 10.20.193.28:0/1695627081 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc8bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.148+0000 7f72d5229640 1 -- 10.20.193.28:0/1695627081 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 msgr2=0x55f2cdc6b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.148+0000 7f72d5229640 1 --2- 10.20.193.28:0/1695627081 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc6b610 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.148+0000 7f72d5229640 1 -- 10.20.193.28:0/1695627081 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2cdade120 con 0x55f2cdc26750 2026-02-25T11:57:15.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.148+0000 7f72d5229640 1 --2- 10.20.193.28:0/1695627081 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc26750 0x55f2cdc26b20 secure :-1 s=READY pgs=173 gs=1 cs=0 l=1 c_cookie=4382738098140981021 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f72c0004790 tx=0x7f72c0040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cd29d145b43a71d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:15.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.148+0000 7f729dce5640 1 -- 10.20.193.28:0/1695627081 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f72c002fc10 con 0x55f2cdc26750 2026-02-25T11:57:15.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.148+0000 7f729dce5640 1 -- 10.20.193.28:0/1695627081 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f72c00029d0 con 0x55f2cdc26750 2026-02-25T11:57:15.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.148+0000 7f729dce5640 1 -- 10.20.193.28:0/1695627081 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f72c004f7a0 con 0x55f2cdc26750 2026-02-25T11:57:15.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.149+0000 7f72d94afe00 1 -- 10.20.193.28:0/1695627081 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc26750 msgr2=0x55f2cdc26b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.149+0000 7f72d94afe00 1 --2- 10.20.193.28:0/1695627081 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc26750 0x55f2cdc26b20 secure :-1 s=READY pgs=173 gs=1 cs=0 l=1 c_cookie=4382738098140981021 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f72c0004790 tx=0x7f72c0040000 comp rx=0 tx=0).stop 2026-02-25T11:57:15.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.149+0000 7f72d94afe00 1 -- 10.20.193.28:0/1695627081 shutdown_connections 2026-02-25T11:57:15.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.149+0000 7f72d94afe00 1 --2- 10.20.193.28:0/1695627081 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc8bf30 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.149+0000 7f72d94afe00 1 --2- 10.20.193.28:0/1695627081 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc6b610 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.151 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.149+0000 7f72d94afe00 1 --2- 10.20.193.28:0/1695627081 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc26750 0x55f2cdc26b20 unknown :-1 s=CLOSED pgs=173 gs=1 cs=0 l=1 c_cookie=4382738098140981021 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.149+0000 7f72d94afe00 1 -- 10.20.193.28:0/1695627081 >> 10.20.193.28:0/1695627081 conn(0x55f2cdc048f0 msgr2=0x55f2cdc24cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:15.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.149+0000 7f72d94afe00 1 -- 10.20.193.28:0/1695627081 shutdown_connections 2026-02-25T11:57:15.152 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.149+0000 7f72d94afe00 1 -- 10.20.193.28:0/1695627081 wait complete. 2026-02-25T11:57:15.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.150+0000 7f72d94afe00 1 -- start start 2026-02-25T11:57:15.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.150+0000 7f72d94afe00 1 Processor -- start 2026-02-25T11:57:15.154 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d94afe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc26750 0x55f2cdc83480 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d94afe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc27060 0x55f2cdc839c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d94afe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc889b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d94afe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdc88ef0 con 0x55f2cdc26750 2026-02-25T11:57:15.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d94afe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdc89060 con 0x55f2cdc6bb50 2026-02-25T11:57:15.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d94afe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdc89220 con 0x55f2cdc27060 2026-02-25T11:57:15.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d5229640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc26750 0x55f2cdc83480 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d5229640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc26750 0x55f2cdc83480 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59004/0 (socket says 10.20.193.28:59004) 2026-02-25T11:57:15.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d5229640 1 -- 10.20.193.28:0/4072186986 learned_addr learned my addr 10.20.193.28:0/4072186986 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:15.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d4a28640 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc27060 0x55f2cdc839c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d5a2a640 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d5a2a640 1 -- 10.20.193.28:0/4072186986 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc27060 msgr2=0x55f2cdc839c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.155 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d5a2a640 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc27060 0x55f2cdc839c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d5a2a640 1 -- 10.20.193.28:0/4072186986 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc26750 msgr2=0x55f2cdc83480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d5a2a640 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc26750 0x55f2cdc83480 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.151+0000 7f72d5a2a640 1 -- 10.20.193.28:0/4072186986 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2cdab6810 con 0x55f2cdc6bb50 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.152+0000 7f72d5229640 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc26750 0x55f2cdc83480 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.152+0000 7f72d5a2a640 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc889b0 secure :-1 s=READY pgs=174 gs=3 cs=0 l=1 c_cookie=10927930302811310940 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f72cc0aa640 tx=0x7f72cc0b02b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97a7ceb6354b8b5c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.152+0000 7f7286ffd640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.152+0000 7f7286ffd640 1 -- 10.20.193.28:0/4072186986 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f72cc0aef20 con 0x55f2cdc6bb50 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.152+0000 7f7286ffd640 1 -- 10.20.193.28:0/4072186986 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f72cc0aea70 con 0x55f2cdc6bb50 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.152+0000 7f7286ffd640 1 -- 10.20.193.28:0/4072186986 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f72cc0b8fa0 con 0x55f2cdc6bb50 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.152+0000 7f72d94afe00 1 -- 10.20.193.28:0/4072186986 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d017af70 con 0x55f2cdc6bb50 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.152+0000 7f7286ffd640 1 -- 10.20.193.28:0/4072186986 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f72cc073200 con 0x55f2cdc6bb50 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.152+0000 7f7286ffd640 10 client.27068 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.152+0000 7f72d94afe00 10 client.27068 _ll_register_callbacks cb 0x55f2cdab3e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:15.156 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.153+0000 7f72d94afe00 1 -- 10.20.193.28:0/4072186986 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f2cd88a530 con 0x55f2cdc6bb50 2026-02-25T11:57:15.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.153+0000 7f7286ffd640 1 -- 10.20.193.28:0/4072186986 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f72cc0b6370 con 0x55f2cdc6bb50 2026-02-25T11:57:15.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.153+0000 7f72d94afe00 10 client.27068 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:15.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.153+0000 7f72d94afe00 1 -- 10.20.193.28:0/4072186986 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55f2cdac84f0 con 0x55f2cdc6bb50 2026-02-25T11:57:15.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.153+0000 7f7286ffd640 1 -- 10.20.193.28:0/4072186986 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f72cc0ac050 con 0x55f2cdc6bb50 2026-02-25T11:57:15.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.153+0000 7f72d94afe00 10 client.27068 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:15.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.153+0000 7f72d94afe00 10 client.27068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:15.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.153+0000 7f72d94afe00 20 client.27068 populate_metadata read hostname 'trial028' 2026-02-25T11:57:15.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.153+0000 7f72d94afe00 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f2cdc68db0 0x55f2cdc69250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.153+0000 7f72d94afe00 4 client.27068 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:57:15.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.153+0000 7f72d94afe00 1 -- 10.20.193.28:0/4072186986 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2cdadfb50 con 0x55f2cdc68db0 2026-02-25T11:57:15.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.153+0000 7f72d5229640 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f2cdc68db0 0x55f2cdc69250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.154+0000 7f72d5229640 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f2cdc68db0 0x55f2cdc69250 crc :-1 s=READY pgs=164 gs=4 cs=0 l=0 c_cookie=8368070714677757254 s_cookie=16834775783715299447 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74215bfaa3b78546 server_cookie=e9a123617db2e077 in_seq=0 out_seq=0 2026-02-25T11:57:15.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.154+0000 7f7286ffd640 10 client.27068 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:15.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f7286ffd640 1 -- 10.20.193.28:0/4072186986 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55f2cdadfb50 con 0x55f2cdc68db0 2026-02-25T11:57:15.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f7286ffd640 10 client.27068 handle_command_reply: tid=0 2026-02-25T11:57:15.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f72d94afe00 1 client.27068 shutdown 2026-02-25T11:57:15.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f72d94afe00 1 -- 10.20.193.28:0/4072186986 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc6bb50 msgr2=0x55f2cdc889b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f72d94afe00 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc889b0 secure :-1 s=READY pgs=174 gs=3 cs=0 l=1 c_cookie=10927930302811310940 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f72cc0aa640 tx=0x7f72cc0b02b0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f72d94afe00 1 -- 10.20.193.28:0/4072186986 shutdown_connections 2026-02-25T11:57:15.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f72d94afe00 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f2cdc68db0 0x55f2cdc69250 crc :-1 s=READY pgs=164 gs=4 cs=0 l=0 c_cookie=8368070714677757254 s_cookie=16834775783715299447 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f72d94afe00 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc889b0 unknown :-1 s=CLOSED pgs=174 gs=3 cs=0 l=1 c_cookie=10927930302811310940 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f72d94afe00 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc27060 0x55f2cdc839c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f72d94afe00 1 --2- 10.20.193.28:0/4072186986 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc26750 0x55f2cdc83480 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f72d94afe00 1 -- 10.20.193.28:0/4072186986 >> 10.20.193.28:0/4072186986 conn(0x55f2cdc048f0 msgr2=0x55f2cdc24cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:15.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f7286ffd640 0 client.27068 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:15.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f72d94afe00 1 -- 10.20.193.28:0/4072186986 shutdown_connections 2026-02-25T11:57:15.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f72d94afe00 1 -- 10.20.193.28:0/4072186986 wait complete. 2026-02-25T11:57:15.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.155+0000 7f72d94afe00 20 client.27068 trim_cache size 0 max 16384 2026-02-25T11:57:15.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.164+0000 7f72d94afe00 1 Processor -- start 2026-02-25T11:57:15.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.164+0000 7f72d94afe00 1 -- start start 2026-02-25T11:57:15.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d94afe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc26750 0x55f2cdc80730 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d94afe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc80c70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d94afe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc6a920 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d94afe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdb30680 con 0x55f2cdc27060 2026-02-25T11:57:15.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d94afe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f2cd7f0790 con 0x55f2cdc6bb50 2026-02-25T11:57:15.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d94afe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdc04660 con 0x55f2cdc26750 2026-02-25T11:57:15.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d4a28640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc80c70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d4a28640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc80c70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59012/0 (socket says 10.20.193.28:59012) 2026-02-25T11:57:15.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d4a28640 1 -- 10.20.193.28:0/3232534362 learned_addr learned my addr 10.20.193.28:0/3232534362 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:15.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d5229640 1 --2- 10.20.193.28:0/3232534362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc26750 0x55f2cdc80730 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d5a2a640 1 --2- 10.20.193.28:0/3232534362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc6a920 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d4a28640 1 -- 10.20.193.28:0/3232534362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc26750 msgr2=0x55f2cdc80730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d4a28640 1 --2- 10.20.193.28:0/3232534362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc26750 0x55f2cdc80730 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d4a28640 1 -- 10.20.193.28:0/3232534362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc6bb50 msgr2=0x55f2cdc6a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d4a28640 1 --2- 10.20.193.28:0/3232534362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc6a920 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d4a28640 1 -- 10.20.193.28:0/3232534362 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2cdc75a10 con 0x55f2cdc27060 2026-02-25T11:57:15.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d5a2a640 1 --2- 10.20.193.28:0/3232534362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc6a920 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:57:15.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.165+0000 7f72d4a28640 1 --2- 10.20.193.28:0/3232534362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc80c70 secure :-1 s=READY pgs=849 gs=1 cs=0 l=1 c_cookie=15908869341075177561 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f72c4005900 tx=0x7f72c4002c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc7a8607ce38059 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:15.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.166+0000 7f729d4e4640 1 -- 10.20.193.28:0/3232534362 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f72c4016070 con 0x55f2cdc27060 2026-02-25T11:57:15.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.166+0000 7f729d4e4640 1 -- 10.20.193.28:0/3232534362 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f72c4009c50 con 0x55f2cdc27060 2026-02-25T11:57:15.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.166+0000 7f729d4e4640 1 -- 10.20.193.28:0/3232534362 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f72c4011750 con 0x55f2cdc27060 2026-02-25T11:57:15.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.166+0000 7f72d94afe00 1 -- 10.20.193.28:0/3232534362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 msgr2=0x55f2cdc80c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.166+0000 7f72d94afe00 1 --2- 10.20.193.28:0/3232534362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc80c70 secure :-1 s=READY pgs=849 gs=1 cs=0 l=1 c_cookie=15908869341075177561 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f72c4005900 tx=0x7f72c4002c70 comp rx=0 tx=0).stop 2026-02-25T11:57:15.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.166+0000 7f72d94afe00 1 -- 10.20.193.28:0/3232534362 shutdown_connections 2026-02-25T11:57:15.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.166+0000 7f72d94afe00 1 --2- 10.20.193.28:0/3232534362 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc6a920 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.166+0000 7f72d94afe00 1 --2- 10.20.193.28:0/3232534362 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc80c70 unknown :-1 s=CLOSED pgs=849 gs=1 cs=0 l=1 c_cookie=15908869341075177561 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.166+0000 7f72d94afe00 1 --2- 10.20.193.28:0/3232534362 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc26750 0x55f2cdc80730 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.166+0000 7f72d94afe00 1 -- 10.20.193.28:0/3232534362 >> 10.20.193.28:0/3232534362 conn(0x55f2cdc048f0 msgr2=0x55f2cdc6df30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:15.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.166+0000 7f72d94afe00 1 -- 10.20.193.28:0/3232534362 shutdown_connections 2026-02-25T11:57:15.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.166+0000 7f72d94afe00 1 -- 10.20.193.28:0/3232534362 wait complete. 2026-02-25T11:57:15.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.167+0000 7f72d94afe00 1 -- start start 2026-02-25T11:57:15.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.167+0000 7f72d94afe00 1 Processor -- start 2026-02-25T11:57:15.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.167+0000 7f72d94afe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc26750 0x55f2cdc0a770 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.167+0000 7f72d94afe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc05c50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.167+0000 7f72d94afe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc06190 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.167+0000 7f72d94afe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdc067a0 con 0x55f2cdc27060 2026-02-25T11:57:15.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.167+0000 7f72d94afe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdc06910 con 0x55f2cdc26750 2026-02-25T11:57:15.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.167+0000 7f72d94afe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f2cdc06ad0 con 0x55f2cdc6bb50 2026-02-25T11:57:15.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.167+0000 7f72d4a28640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc05c50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.167+0000 7f72d4a28640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc05c50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59020/0 (socket says 10.20.193.28:59020) 2026-02-25T11:57:15.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.167+0000 7f72d4a28640 1 -- 10.20.193.28:0/3279318483 learned_addr learned my addr 10.20.193.28:0/3279318483 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:15.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f72d5229640 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc26750 0x55f2cdc0a770 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f72d5a2a640 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc06190 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f72d4a28640 1 -- 10.20.193.28:0/3279318483 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc6bb50 msgr2=0x55f2cdc06190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f72d4a28640 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc06190 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f72d4a28640 1 -- 10.20.193.28:0/3279318483 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc26750 msgr2=0x55f2cdc0a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f72d4a28640 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc26750 0x55f2cdc0a770 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f72d4a28640 1 -- 10.20.193.28:0/3279318483 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2cdac2f70 con 0x55f2cdc27060 2026-02-25T11:57:15.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f72d5a2a640 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc06190 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:57:15.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f72d4a28640 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc05c50 secure :-1 s=READY pgs=850 gs=1 cs=0 l=1 c_cookie=9843162342350747140 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f72c40027a0 tx=0x7f72c401bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8899efffc2517604 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:15.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f7286ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:15.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f7286ffd640 1 -- 10.20.193.28:0/3279318483 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f72c4016030 con 0x55f2cdc27060 2026-02-25T11:57:15.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f7286ffd640 1 -- 10.20.193.28:0/3279318483 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f72c4020040 con 0x55f2cdc27060 2026-02-25T11:57:15.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f72d94afe00 1 -- 10.20.193.28:0/3279318483 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c4004420 con 0x55f2cdc27060 2026-02-25T11:57:15.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f7286ffd640 1 -- 10.20.193.28:0/3279318483 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f72c401f050 con 0x55f2cdc27060 2026-02-25T11:57:15.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f72d94afe00 10 client.17400 _ll_register_callbacks cb 0x55f2cdab3e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:15.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.168+0000 7f72d94afe00 1 -- 10.20.193.28:0/3279318483 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f2cd7c6220 con 0x55f2cdc27060 2026-02-25T11:57:15.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.169+0000 7f7286ffd640 1 -- 10.20.193.28:0/3279318483 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f72c4024690 con 0x55f2cdc27060 2026-02-25T11:57:15.175 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.169+0000 7f7286ffd640 1 -- 10.20.193.28:0/3279318483 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f72c4024850 con 0x55f2cdc27060 2026-02-25T11:57:15.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.169+0000 7f72d94afe00 10 client.17400 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:15.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.169+0000 7f72d94afe00 1 -- 10.20.193.28:0/3279318483 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55f2cdab6b40 con 0x55f2cdc27060 2026-02-25T11:57:15.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.169+0000 7f7286ffd640 1 -- 10.20.193.28:0/3279318483 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f72c4026cd0 con 0x55f2cdc27060 2026-02-25T11:57:15.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.169+0000 7f72d94afe00 10 client.17400 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:15.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.169+0000 7f72d94afe00 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:15.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.169+0000 7f72d94afe00 20 client.17400 populate_metadata read hostname 'trial028' 2026-02-25T11:57:15.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.169+0000 7f72d94afe00 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f2cdd198c0 0x55f2cdd39d70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.169+0000 7f72d94afe00 4 client.17400 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:57:15.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.169+0000 7f72d94afe00 1 -- 10.20.193.28:0/3279318483 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f72c00029d0 con 0x55f2cdd198c0 2026-02-25T11:57:15.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.169+0000 7f72d5229640 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f2cdd198c0 0x55f2cdd39d70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.170+0000 7f72d5229640 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f2cdd198c0 0x55f2cdd39d70 crc :-1 s=READY pgs=165 gs=4 cs=0 l=0 c_cookie=14101926665272759447 s_cookie=7820870977566339870 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b41b963c311497 server_cookie=6c8950b7a400f31e in_seq=0 out_seq=0 2026-02-25T11:57:15.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.170+0000 7f7286ffd640 10 client.17400 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:15.176 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f7286ffd640 1 -- 10.20.193.28:0/3279318483 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f72c00029d0 con 0x55f2cdd198c0 2026-02-25T11:57:15.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f7286ffd640 10 client.17400 handle_command_reply: tid=0 2026-02-25T11:57:15.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f72d94afe00 1 client.17400 shutdown 2026-02-25T11:57:15.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f72d94afe00 1 -- 10.20.193.28:0/3279318483 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 msgr2=0x55f2cdc05c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f72d94afe00 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc05c50 secure :-1 s=READY pgs=850 gs=1 cs=0 l=1 c_cookie=9843162342350747140 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f72c40027a0 tx=0x7f72c401bba0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f72d94afe00 1 -- 10.20.193.28:0/3279318483 shutdown_connections 2026-02-25T11:57:15.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f72d94afe00 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55f2cdd198c0 0x55f2cdd39d70 crc :-1 s=READY pgs=165 gs=4 cs=0 l=0 c_cookie=14101926665272759447 s_cookie=7820870977566339870 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f72d94afe00 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f2cdc6bb50 0x55f2cdc06190 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f72d94afe00 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f2cdc27060 0x55f2cdc05c50 unknown :-1 s=CLOSED pgs=850 gs=1 cs=0 l=1 c_cookie=9843162342350747140 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f72d94afe00 1 --2- 10.20.193.28:0/3279318483 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f2cdc26750 0x55f2cdc0a770 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f72d94afe00 1 -- 10.20.193.28:0/3279318483 >> 10.20.193.28:0/3279318483 conn(0x55f2cdc048f0 msgr2=0x55f2cdc7f720 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:15.177 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f7286ffd640 0 client.17400 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:15.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f72d94afe00 1 -- 10.20.193.28:0/3279318483 shutdown_connections 2026-02-25T11:57:15.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f72d94afe00 1 -- 10.20.193.28:0/3279318483 wait complete. 2026-02-25T11:57:15.178 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.171+0000 7f72d94afe00 20 client.17400 trim_cache size 0 max 16384 2026-02-25T11:57:15.178 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:57:15.178 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:57:15.178 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "516f576f-224f-4d5c-9352-d469d781f97c", 2026-02-25T11:57:15.178 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:57:15.179 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:57:15.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.172+0000 7f72d622b640 1 -- 10.20.193.28:0/3699174710 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f72ac0b56e0 msgr2=0x7f72ac0d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.172+0000 7f72d622b640 1 --2- 10.20.193.28:0/3699174710 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f72ac0b56e0 0x7f72ac0d5b90 secure :-1 s=READY pgs=377 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f72c00023d0 tx=0x7f72c0031040 comp rx=0 tx=0).stop 2026-02-25T11:57:15.179 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.172+0000 7f72d622b640 1 -- 10.20.193.28:0/3699174710 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f72d0119d30 msgr2=0x7f72d0127e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.172+0000 7f72d622b640 1 --2- 10.20.193.28:0/3699174710 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f72d0119d30 0x7f72d0127e20 secure :-1 s=READY pgs=848 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f72cc00f8d0 tx=0x7f72cc00fda0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.172+0000 7f72d622b640 1 -- 10.20.193.28:0/3699174710 shutdown_connections 2026-02-25T11:57:15.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.172+0000 7f72d622b640 1 --2- 10.20.193.28:0/3699174710 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f72ac0b56e0 0x7f72ac0d5b90 unknown :-1 s=CLOSED pgs=377 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.172+0000 7f72d622b640 1 --2- 10.20.193.28:0/3699174710 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f72d0119d30 0x7f72d0127e20 unknown :-1 s=CLOSED pgs=848 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.172+0000 7f72d622b640 1 --2- 10.20.193.28:0/3699174710 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f72d0114a90 0x7f72d0122a50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.172+0000 7f72d622b640 1 --2- 10.20.193.28:0/3699174710 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f72d01140a0 0x7f72d0122510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.172+0000 7f72d622b640 1 -- 10.20.193.28:0/3699174710 >> 10.20.193.28:0/3699174710 conn(0x7f72d0064c40 msgr2=0x7f72d0115b00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:15.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.172+0000 7f72d622b640 1 -- 10.20.193.28:0/3699174710 shutdown_connections 2026-02-25T11:57:15.180 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.172+0000 7f72d622b640 1 -- 10.20.193.28:0/3699174710 wait complete. 2026-02-25T11:57:15.306 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:57:15.358 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:15 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:57:15.358 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:15 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:57:15.358 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:15 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:57:15.358 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:15 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:57:15.491 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:15.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.667+0000 7fcac9f4a640 1 -- 10.20.193.28:0/3828675868 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcac410c8f0 msgr2=0x7fcac41557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.667+0000 7fcac9f4a640 1 --2- 10.20.193.28:0/3828675868 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcac410c8f0 0x7fcac41557d0 secure :-1 s=READY pgs=851 gs=1 cs=0 l=1 c_cookie=2026915652672636921 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcab4009950 tx=0x7fcab402f190 comp rx=0 tx=0).stop 2026-02-25T11:57:15.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.667+0000 7fcac9f4a640 1 -- 10.20.193.28:0/3828675868 shutdown_connections 2026-02-25T11:57:15.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.667+0000 7fcac9f4a640 1 --2- 10.20.193.28:0/3828675868 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcac4155d10 0x7fcac41760f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.667+0000 7fcac9f4a640 1 --2- 10.20.193.28:0/3828675868 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcac410c8f0 0x7fcac41557d0 unknown :-1 s=CLOSED pgs=851 gs=1 cs=0 l=1 c_cookie=2026915652672636921 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.667+0000 7fcac9f4a640 1 --2- 10.20.193.28:0/3828675868 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcac410bfe0 0x7fcac410c3b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.667+0000 7fcac9f4a640 1 -- 10.20.193.28:0/3828675868 >> 10.20.193.28:0/3828675868 conn(0x7fcac40e9c80 msgr2=0x7fcac410a090 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:15.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.667+0000 7fcac9f4a640 1 -- 10.20.193.28:0/3828675868 shutdown_connections 2026-02-25T11:57:15.669 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.667+0000 7fcac9f4a640 1 -- 10.20.193.28:0/3828675868 wait complete. 2026-02-25T11:57:15.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.668+0000 7fcac9f4a640 1 Processor -- start 2026-02-25T11:57:15.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.668+0000 7fcac9f4a640 1 -- start start 2026-02-25T11:57:15.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.668+0000 7fcac9f4a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcac410bfe0 0x7fcac4161570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.668+0000 7fcac9f4a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcac410c8f0 0x7fcac4161ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.668+0000 7fcac9f4a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcac4155d10 0x7fcac4166e30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.668+0000 7fcac9f4a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4167450 con 0x7fcac410bfe0 2026-02-25T11:57:15.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.668+0000 7fcac9f4a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcac41675c0 con 0x7fcac410c8f0 2026-02-25T11:57:15.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.668+0000 7fcac9f4a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcac4167780 con 0x7fcac4155d10 2026-02-25T11:57:15.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.668+0000 7fcac37fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcac410bfe0 0x7fcac4161570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.668+0000 7fcac2ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcac410c8f0 0x7fcac4161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac2ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcac410c8f0 0x7fcac4161ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:54830/0 (socket says 10.20.193.28:54830) 2026-02-25T11:57:15.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac37fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcac410bfe0 0x7fcac4161570 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59034/0 (socket says 10.20.193.28:59034) 2026-02-25T11:57:15.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac2ffd640 1 -- 10.20.193.28:0/1189002738 learned_addr learned my addr 10.20.193.28:0/1189002738 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:15.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac3fff640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcac4155d10 0x7fcac4166e30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac2ffd640 1 -- 10.20.193.28:0/1189002738 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcac4155d10 msgr2=0x7fcac4166e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac2ffd640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcac4155d10 0x7fcac4166e30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac2ffd640 1 -- 10.20.193.28:0/1189002738 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcac410bfe0 msgr2=0x7fcac4161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac2ffd640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcac410bfe0 0x7fcac4161570 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac2ffd640 1 -- 10.20.193.28:0/1189002738 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab4009570 con 0x7fcac410c8f0 2026-02-25T11:57:15.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac37fe640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcac410bfe0 0x7fcac4161570 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:57:15.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac2ffd640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcac410c8f0 0x7fcac4161ab0 secure :-1 s=READY pgs=175 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcab40098a0 tx=0x7fcab40028d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:15.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac0ff9640 1 -- 10.20.193.28:0/1189002738 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcab403d070 con 0x7fcac410c8f0 2026-02-25T11:57:15.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac9f4a640 1 -- 10.20.193.28:0/1189002738 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac4167a00 con 0x7fcac410c8f0 2026-02-25T11:57:15.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac0ff9640 1 -- 10.20.193.28:0/1189002738 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcab402fe20 con 0x7fcac410c8f0 2026-02-25T11:57:15.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac0ff9640 1 -- 10.20.193.28:0/1189002738 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcab4041790 con 0x7fcac410c8f0 2026-02-25T11:57:15.673 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.669+0000 7fcac9f4a640 1 -- 10.20.193.28:0/1189002738 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac4152940 con 0x7fcac410c8f0 2026-02-25T11:57:15.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.672+0000 7fcac0ff9640 1 -- 10.20.193.28:0/1189002738 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fcab40386a0 con 0x7fcac410c8f0 2026-02-25T11:57:15.678 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.674+0000 7fcac0ff9640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca980b5760 0x7fca980d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.674+0000 7fcac0ff9640 1 -- 10.20.193.28:0/1189002738 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fcab410b150 con 0x7fcac410c8f0 2026-02-25T11:57:15.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.674+0000 7fcac37fe640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca980b5760 0x7fca980d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.675+0000 7fcac37fe640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca980b5760 0x7fca980d5c10 secure :-1 s=READY pgs=378 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcab0004740 tx=0x7fcab0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:15.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.675+0000 7fcacbf45e00 1 Processor -- start 2026-02-25T11:57:15.679 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.675+0000 7fcacbf45e00 1 -- start start 2026-02-25T11:57:15.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.675+0000 7fcacbf45e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0b140 0x560bd7a0b510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.675+0000 7fcacbf45e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a0ba50 0x560bd7a501b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.675+0000 7fcacbf45e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a506f0 0x560bd7a70ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.675+0000 7fcacbf45e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x560bd79219b0 con 0x560bd7a506f0 2026-02-25T11:57:15.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.675+0000 7fcacbf45e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560bd791d6e0 con 0x560bd7a0b140 2026-02-25T11:57:15.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.675+0000 7fcacbf45e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560bd7917840 con 0x560bd7a0ba50 2026-02-25T11:57:15.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fcac37fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0b140 0x560bd7a0b510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fcac3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a506f0 0x560bd7a70ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fcac37fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0b140 0x560bd7a0b510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:54836/0 (socket says 10.20.193.28:54836) 2026-02-25T11:57:15.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fcac37fe640 1 -- 10.20.193.28:0/4248757828 learned_addr learned my addr 10.20.193.28:0/4248757828 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:15.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fcac2ffd640 1 --2- 10.20.193.28:0/4248757828 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a0ba50 0x560bd7a501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fcac3fff640 1 -- 10.20.193.28:0/4248757828 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a0ba50 msgr2=0x560bd7a501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fcac3fff640 1 --2- 10.20.193.28:0/4248757828 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a0ba50 0x560bd7a501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fcac3fff640 1 -- 10.20.193.28:0/4248757828 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0b140 msgr2=0x560bd7a0b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fcac3fff640 1 --2- 10.20.193.28:0/4248757828 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0b140 0x560bd7a0b510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fcac3fff640 1 -- 10.20.193.28:0/4248757828 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd78c2d80 con 0x560bd7a506f0 2026-02-25T11:57:15.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fcac3fff640 1 --2- 10.20.193.28:0/4248757828 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a506f0 0x560bd7a70ad0 secure :-1 s=READY pgs=852 gs=2 cs=0 l=1 c_cookie=2695969294290750481 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcab8005820 tx=0x7fcab80117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256a01ca0f8f8c11 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:15.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fca87fff640 1 -- 10.20.193.28:0/4248757828 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcab8008e90 con 0x560bd7a506f0 2026-02-25T11:57:15.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fca87fff640 1 -- 10.20.193.28:0/4248757828 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcab8005590 con 0x560bd7a506f0 2026-02-25T11:57:15.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fca87fff640 1 -- 10.20.193.28:0/4248757828 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcab8012730 con 0x560bd7a506f0 2026-02-25T11:57:15.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fcacbf45e00 1 -- 10.20.193.28:0/4248757828 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a506f0 msgr2=0x560bd7a70ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.676+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/4248757828 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a506f0 0x560bd7a70ad0 secure :-1 s=READY pgs=852 gs=2 cs=0 l=1 c_cookie=2695969294290750481 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcab8005820 tx=0x7fcab80117a0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.677+0000 7fcacbf45e00 1 -- 10.20.193.28:0/4248757828 shutdown_connections 2026-02-25T11:57:15.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.677+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/4248757828 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a506f0 0x560bd7a70ad0 unknown :-1 s=CLOSED pgs=852 gs=2 cs=0 l=1 c_cookie=2695969294290750481 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.677+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/4248757828 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a0ba50 0x560bd7a501b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.677+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/4248757828 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0b140 0x560bd7a0b510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.677+0000 7fcacbf45e00 1 -- 10.20.193.28:0/4248757828 >> 10.20.193.28:0/4248757828 conn(0x560bd792ad40 msgr2=0x560bd7a09a20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:15.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.677+0000 7fcacbf45e00 1 -- 10.20.193.28:0/4248757828 shutdown_connections 2026-02-25T11:57:15.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.677+0000 7fcacbf45e00 1 -- 10.20.193.28:0/4248757828 wait complete. 2026-02-25T11:57:15.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.678+0000 7fcacbf45e00 1 -- start start 2026-02-25T11:57:15.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.678+0000 7fcacbf45e00 1 Processor -- start 2026-02-25T11:57:15.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcacbf45e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0b140 0x560bd7a68350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.688 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcacbf45e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0ba50 0x560bd7a68890 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcacbf45e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a506f0 0x560bd7a6d880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcacbf45e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x560bd7a6ddc0 con 0x560bd7a0ba50 2026-02-25T11:57:15.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcacbf45e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560bd7a6df30 con 0x560bd7a0b140 2026-02-25T11:57:15.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcacbf45e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560bd7a6e0f0 con 0x560bd7a506f0 2026-02-25T11:57:15.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcac2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0ba50 0x560bd7a68890 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcac37fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0b140 0x560bd7a68350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcac2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0ba50 0x560bd7a68890 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59042/0 (socket says 10.20.193.28:59042) 2026-02-25T11:57:15.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcac2ffd640 1 -- 10.20.193.28:0/1142865626 learned_addr learned my addr 10.20.193.28:0/1142865626 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:15.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcac3fff640 1 --2- 10.20.193.28:0/1142865626 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a506f0 0x560bd7a6d880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcac2ffd640 1 -- 10.20.193.28:0/1142865626 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a506f0 msgr2=0x560bd7a6d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcac2ffd640 1 --2- 10.20.193.28:0/1142865626 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a506f0 0x560bd7a6d880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.689 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcac2ffd640 1 -- 10.20.193.28:0/1142865626 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0b140 msgr2=0x560bd7a68350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcac2ffd640 1 --2- 10.20.193.28:0/1142865626 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0b140 0x560bd7a68350 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcac2ffd640 1 -- 10.20.193.28:0/1142865626 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd789b350 con 0x560bd7a0ba50 2026-02-25T11:57:15.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcac2ffd640 1 --2- 10.20.193.28:0/1142865626 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0ba50 0x560bd7a68890 secure :-1 s=READY pgs=853 gs=2 cs=0 l=1 c_cookie=12660352075759501538 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcab4004480 tx=0x7fcab4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb29b22af0640e2 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:15.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fca857fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:15.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fca857fa640 1 -- 10.20.193.28:0/1142865626 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcab40d0d80 con 0x560bd7a0ba50 2026-02-25T11:57:15.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fca857fa640 1 -- 10.20.193.28:0/1142865626 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcab40d71f0 con 0x560bd7a0ba50 2026-02-25T11:57:15.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fca857fa640 1 -- 10.20.193.28:0/1142865626 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcab40d3140 con 0x560bd7a0ba50 2026-02-25T11:57:15.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.679+0000 7fcacbf45e00 1 -- 10.20.193.28:0/1142865626 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab80087c0 con 0x560bd7a0ba50 2026-02-25T11:57:15.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fca857fa640 1 -- 10.20.193.28:0/1142865626 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fcab40f6040 con 0x560bd7a0ba50 2026-02-25T11:57:15.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fca857fa640 10 client.17412 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:57:15.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fcacbf45e00 10 client.17412 _ll_register_callbacks cb 0x560bd78989a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:15.690 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fcacbf45e00 1 -- 10.20.193.28:0/1142865626 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560bd75ad6f0 con 0x560bd7a0ba50 2026-02-25T11:57:15.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fca857fa640 1 -- 10.20.193.28:0/1142865626 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcab40d97d0 con 0x560bd7a0ba50 2026-02-25T11:57:15.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fcacbf45e00 10 client.17412 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:15.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fcacbf45e00 1 -- 10.20.193.28:0/1142865626 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x560bd78ad150 con 0x560bd7a0ba50 2026-02-25T11:57:15.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fca857fa640 1 -- 10.20.193.28:0/1142865626 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fcab40d0870 con 0x560bd7a0ba50 2026-02-25T11:57:15.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fcacbf45e00 10 client.17412 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:15.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fcacbf45e00 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:15.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fcacbf45e00 20 client.17412 populate_metadata read hostname 'trial028' 2026-02-25T11:57:15.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/1142865626 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560bd7a4d930 0x560bd7a4ddd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fcacbf45e00 4 client.17412 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:57:15.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.680+0000 7fcacbf45e00 1 -- 10.20.193.28:0/1142865626 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bd78e9910 con 0x560bd7a4d930 2026-02-25T11:57:15.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.681+0000 7fcac37fe640 1 --2- 10.20.193.28:0/1142865626 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560bd7a4d930 0x560bd7a4ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.691 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.681+0000 7fcac37fe640 1 --2- 10.20.193.28:0/1142865626 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560bd7a4d930 0x560bd7a4ddd0 crc :-1 s=READY pgs=166 gs=4 cs=0 l=0 c_cookie=4770319205409732271 s_cookie=15842836088014397404 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4233940b07abb6af server_cookie=dbdd0f7bacd65fdc in_seq=0 out_seq=0 2026-02-25T11:57:15.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.681+0000 7fca857fa640 10 client.17412 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:15.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fca857fa640 1 -- 10.20.193.28:0/1142865626 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x560bd78e9910 con 0x560bd7a4d930 2026-02-25T11:57:15.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fca857fa640 10 client.17412 handle_command_reply: tid=0 2026-02-25T11:57:15.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fcacbf45e00 1 client.17412 shutdown 2026-02-25T11:57:15.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fcacbf45e00 1 -- 10.20.193.28:0/1142865626 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0ba50 msgr2=0x560bd7a68890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/1142865626 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0ba50 0x560bd7a68890 secure :-1 s=READY pgs=853 gs=2 cs=0 l=1 c_cookie=12660352075759501538 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcab4004480 tx=0x7fcab4009310 comp rx=0 tx=0).stop 2026-02-25T11:57:15.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fcacbf45e00 1 -- 10.20.193.28:0/1142865626 shutdown_connections 2026-02-25T11:57:15.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/1142865626 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560bd7a4d930 0x560bd7a4ddd0 crc :-1 s=READY pgs=166 gs=4 cs=0 l=0 c_cookie=4770319205409732271 s_cookie=15842836088014397404 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/1142865626 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a506f0 0x560bd7a6d880 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/1142865626 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0ba50 0x560bd7a68890 unknown :-1 s=CLOSED pgs=853 gs=2 cs=0 l=1 c_cookie=12660352075759501538 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/1142865626 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0b140 0x560bd7a68350 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.692 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fcacbf45e00 1 -- 10.20.193.28:0/1142865626 >> 10.20.193.28:0/1142865626 conn(0x560bd792ad40 msgr2=0x560bd7a09230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:15.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fca857fa640 0 client.17412 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:15.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fcacbf45e00 1 -- 10.20.193.28:0/1142865626 shutdown_connections 2026-02-25T11:57:15.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fcacbf45e00 1 -- 10.20.193.28:0/1142865626 wait complete. 2026-02-25T11:57:15.693 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.682+0000 7fcacbf45e00 20 client.17412 trim_cache size 0 max 16384 2026-02-25T11:57:15.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcacbf45e00 1 Processor -- start 2026-02-25T11:57:15.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcacbf45e00 1 -- start start 2026-02-25T11:57:15.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcacbf45e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 0x560bd7a531e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcacbf45e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a0ba50 0x560bd7a65520 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcacbf45e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a506f0 0x560bd7a65a60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcacbf45e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x560bd7915470 con 0x560bd7a0b140 2026-02-25T11:57:15.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcacbf45e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560bd75d5700 con 0x560bd7a506f0 2026-02-25T11:57:15.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcacbf45e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560bd797f6e0 con 0x560bd7a0ba50 2026-02-25T11:57:15.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcac37fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 0x560bd7a531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcac37fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 0x560bd7a531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59056/0 (socket says 10.20.193.28:59056) 2026-02-25T11:57:15.696 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcac37fe640 1 -- 10.20.193.28:0/2250880058 learned_addr learned my addr 10.20.193.28:0/2250880058 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:15.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcac2ffd640 1 --2- 10.20.193.28:0/2250880058 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a0ba50 0x560bd7a65520 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcac3fff640 1 --2- 10.20.193.28:0/2250880058 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a506f0 0x560bd7a65a60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcac37fe640 1 -- 10.20.193.28:0/2250880058 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a0ba50 msgr2=0x560bd7a65520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcac37fe640 1 --2- 10.20.193.28:0/2250880058 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a0ba50 0x560bd7a65520 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcac37fe640 1 -- 10.20.193.28:0/2250880058 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a506f0 msgr2=0x560bd7a65a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcac37fe640 1 --2- 10.20.193.28:0/2250880058 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a506f0 0x560bd7a65a60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.692+0000 7fcac37fe640 1 -- 10.20.193.28:0/2250880058 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd7a5a5c0 con 0x560bd7a0b140 2026-02-25T11:57:15.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.693+0000 7fcac37fe640 1 --2- 10.20.193.28:0/2250880058 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 0x560bd7a531e0 secure :-1 s=READY pgs=854 gs=1 cs=0 l=1 c_cookie=15504644568419571508 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcab0009e50 tx=0x7fcab0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72b90167d569734 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:15.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.693+0000 7fca877fe640 1 -- 10.20.193.28:0/2250880058 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcab0021070 con 0x560bd7a0b140 2026-02-25T11:57:15.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.693+0000 7fca877fe640 1 -- 10.20.193.28:0/2250880058 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcab0022040 con 0x560bd7a0b140 2026-02-25T11:57:15.697 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.693+0000 7fcacbf45e00 1 -- 10.20.193.28:0/2250880058 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 msgr2=0x560bd7a531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.693+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/2250880058 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 0x560bd7a531e0 secure :-1 s=READY pgs=854 gs=1 cs=0 l=1 c_cookie=15504644568419571508 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcab0009e50 tx=0x7fcab0014000 comp rx=0 tx=0).stop 2026-02-25T11:57:15.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.693+0000 7fcacbf45e00 1 -- 10.20.193.28:0/2250880058 shutdown_connections 2026-02-25T11:57:15.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.693+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/2250880058 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a506f0 0x560bd7a65a60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.693+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/2250880058 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a0ba50 0x560bd7a65520 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.693+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/2250880058 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 0x560bd7a531e0 unknown :-1 s=CLOSED pgs=854 gs=1 cs=0 l=1 c_cookie=15504644568419571508 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.693+0000 7fcacbf45e00 1 -- 10.20.193.28:0/2250880058 >> 10.20.193.28:0/2250880058 conn(0x560bd792ad40 msgr2=0x560bd7a52ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:15.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.693+0000 7fcacbf45e00 1 -- 10.20.193.28:0/2250880058 shutdown_connections 2026-02-25T11:57:15.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.693+0000 7fcacbf45e00 1 -- 10.20.193.28:0/2250880058 wait complete. 2026-02-25T11:57:15.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.694+0000 7fcacbf45e00 1 -- start start 2026-02-25T11:57:15.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.694+0000 7fcacbf45e00 1 Processor -- start 2026-02-25T11:57:15.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.694+0000 7fcacbf45e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 0x560bd79eeba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.694+0000 7fcacbf45e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0ba50 0x560bd79ea080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.698 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.694+0000 7fcacbf45e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a506f0 0x560bd79ea5c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.694+0000 7fcacbf45e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x560bd79eab00 con 0x560bd7a0b140 2026-02-25T11:57:15.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.694+0000 7fcacbf45e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x560bd79eac70 con 0x560bd7a0ba50 2026-02-25T11:57:15.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.694+0000 7fcacbf45e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x560bd79eae30 con 0x560bd7a506f0 2026-02-25T11:57:15.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fcac2ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0ba50 0x560bd79ea080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fcac37fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 0x560bd79eeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fcac2ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0ba50 0x560bd79ea080 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:54872/0 (socket says 10.20.193.28:54872) 2026-02-25T11:57:15.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fcac37fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 0x560bd79eeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:59060/0 (socket says 10.20.193.28:59060) 2026-02-25T11:57:15.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fcac3fff640 1 --2- 10.20.193.28:0/243492966 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a506f0 0x560bd79ea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.699 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fcac2ffd640 1 -- 10.20.193.28:0/243492966 learned_addr learned my addr 10.20.193.28:0/243492966 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:15.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fcac37fe640 1 -- 10.20.193.28:0/243492966 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a506f0 msgr2=0x560bd79ea5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fcac37fe640 1 --2- 10.20.193.28:0/243492966 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a506f0 0x560bd79ea5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fcac37fe640 1 -- 10.20.193.28:0/243492966 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0ba50 msgr2=0x560bd79ea080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:15.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fcac37fe640 1 --2- 10.20.193.28:0/243492966 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0ba50 0x560bd79ea080 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fcac37fe640 1 -- 10.20.193.28:0/243492966 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd78a7bd0 con 0x560bd7a0b140 2026-02-25T11:57:15.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fcac37fe640 1 --2- 10.20.193.28:0/243492966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 0x560bd79eeba0 secure :-1 s=READY pgs=855 gs=1 cs=0 l=1 c_cookie=14526076123227563484 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcab0009e50 tx=0x7fcab00091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c996fd45894325dc server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:15.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fca857fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:15.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fca857fa640 1 -- 10.20.193.28:0/243492966 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcab0021040 con 0x560bd7a0b140 2026-02-25T11:57:15.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.695+0000 7fca857fa640 1 -- 10.20.193.28:0/243492966 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcab001b750 con 0x560bd7a0b140 2026-02-25T11:57:15.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fca857fa640 1 -- 10.20.193.28:0/243492966 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcab0035c50 con 0x560bd7a0b140 2026-02-25T11:57:15.702 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fcacbf45e00 1 -- 10.20.193.28:0/243492966 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab0003d50 con 0x560bd7a0b140 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fca857fa640 1 -- 10.20.193.28:0/243492966 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fcab0035de0 con 0x560bd7a0b140 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fca857fa640 10 client.17418 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fcacbf45e00 10 client.17418 _ll_register_callbacks cb 0x560bd78989a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fcacbf45e00 1 -- 10.20.193.28:0/243492966 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcab8005590 con 0x560bd7a0b140 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fca857fa640 1 -- 10.20.193.28:0/243492966 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fcab0035450 con 0x560bd7a0b140 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fcacbf45e00 10 client.17418 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fcacbf45e00 1 -- 10.20.193.28:0/243492966 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x560bd789b680 con 0x560bd7a0b140 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fca857fa640 1 -- 10.20.193.28:0/243492966 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fcab0033030 con 0x560bd7a0b140 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fcacbf45e00 10 client.17418 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fcacbf45e00 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fcacbf45e00 20 client.17418 populate_metadata read hostname 'trial028' 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/243492966 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560bd7aa0920 0x560bd7aa0d90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fcacbf45e00 4 client.17418 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:57:15.703 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.696+0000 7fcacbf45e00 1 -- 10.20.193.28:0/243492966 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560bd75aaf40 con 0x560bd7aa0920 2026-02-25T11:57:15.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.697+0000 7fcac2ffd640 1 --2- 10.20.193.28:0/243492966 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560bd7aa0920 0x560bd7aa0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:15.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.697+0000 7fcac2ffd640 1 --2- 10.20.193.28:0/243492966 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560bd7aa0920 0x560bd7aa0d90 crc :-1 s=READY pgs=167 gs=4 cs=0 l=0 c_cookie=3611881140971961365 s_cookie=2101752296075984474 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321ffac2da90fc15 server_cookie=1d2aec93e5787e5a in_seq=0 out_seq=0 2026-02-25T11:57:15.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.697+0000 7fca857fa640 10 client.17418 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:15.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.697+0000 7fca857fa640 1 -- 10.20.193.28:0/243492966 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5682 (crc 0 0 0) 0x560bd75aaf40 con 0x560bd7aa0920 2026-02-25T11:57:15.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.697+0000 7fca857fa640 10 client.17418 handle_command_reply: tid=0 2026-02-25T11:57:15.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.697+0000 7fcacbf45e00 1 client.17418 shutdown 2026-02-25T11:57:15.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcacbf45e00 1 -- 10.20.193.28:0/243492966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 msgr2=0x560bd79eeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/243492966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 0x560bd79eeba0 secure :-1 s=READY pgs=855 gs=1 cs=0 l=1 c_cookie=14526076123227563484 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcab0009e50 tx=0x7fcab00091e0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcacbf45e00 1 -- 10.20.193.28:0/243492966 shutdown_connections 2026-02-25T11:57:15.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/243492966 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x560bd7aa0920 0x560bd7aa0d90 crc :-1 s=READY pgs=167 gs=4 cs=0 l=0 c_cookie=3611881140971961365 s_cookie=2101752296075984474 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/243492966 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x560bd7a506f0 0x560bd79ea5c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.704 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/243492966 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x560bd7a0ba50 0x560bd79ea080 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.705 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:57:15.705 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:57:15.705 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:57:15.705 INFO:teuthology.orchestra.run.trial028.stdout: "516f576f-224f-4d5c-9352-d469d781f97c": { 2026-02-25T11:57:15.705 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.705 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "516f576f-224f-4d5c-9352-d469d781f97c", 2026-02-25T11:57:15.705 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:57:15.705 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:57:15.705 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.705 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:57:15.705 INFO:teuthology.orchestra.run.trial028.stdout: "0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8": { 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:05.873000+0000", 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "850bcfd3-be8a-4930-896c-1ed6e48b8fed": { 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:27.124813+0000", 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.706 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.707 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "516f576f-224f-4d5c-9352-d469d781f97c": { 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:57:15.169491+0000", 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.708 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.709 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.710 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "91e989b7-1f1f-41fb-9047-642867de91e6": { 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:40.578257+0000", 2026-02-25T11:57:15.711 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.712 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.713 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "7b237934-6b95-43dd-b3a6-82b74134ec55": { 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:31.289725+0000", 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:15.714 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:15.715 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:57:15.715 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:57:15.715 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:57:15.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcacbf45e00 1 --2- 10.20.193.28:0/243492966 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x560bd7a0b140 0x560bd79eeba0 unknown :-1 s=CLOSED pgs=855 gs=1 cs=0 l=1 c_cookie=14526076123227563484 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fca857fa640 0 client.17418 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:15.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcacbf45e00 1 -- 10.20.193.28:0/243492966 >> 10.20.193.28:0/243492966 conn(0x560bd792ad40 msgr2=0x560bd7a64510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:15.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcacbf45e00 1 -- 10.20.193.28:0/243492966 shutdown_connections 2026-02-25T11:57:15.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcacbf45e00 1 -- 10.20.193.28:0/243492966 wait complete. 2026-02-25T11:57:15.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcacbf45e00 20 client.17418 trim_cache size 0 max 16384 2026-02-25T11:57:15.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcac9f4a640 1 -- 10.20.193.28:0/1189002738 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca980b5760 msgr2=0x7fca980d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcac9f4a640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca980b5760 0x7fca980d5c10 secure :-1 s=READY pgs=378 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcab0004740 tx=0x7fcab0005f90 comp rx=0 tx=0).stop 2026-02-25T11:57:15.715 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcac9f4a640 1 -- 10.20.193.28:0/1189002738 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcac410c8f0 msgr2=0x7fcac4161ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:15.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcac9f4a640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcac410c8f0 0x7fcac4161ab0 secure :-1 s=READY pgs=175 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcab40098a0 tx=0x7fcab40028d0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcac9f4a640 1 -- 10.20.193.28:0/1189002738 shutdown_connections 2026-02-25T11:57:15.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcac9f4a640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fca980b5760 0x7fca980d5c10 unknown :-1 s=CLOSED pgs=378 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcac9f4a640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcac4155d10 0x7fcac4166e30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcac9f4a640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcac410c8f0 0x7fcac4161ab0 unknown :-1 s=CLOSED pgs=175 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcac9f4a640 1 --2- 10.20.193.28:0/1189002738 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcac410bfe0 0x7fcac4161570 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:15.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcac9f4a640 1 -- 10.20.193.28:0/1189002738 >> 10.20.193.28:0/1189002738 conn(0x7fcac40e9c80 msgr2=0x7fcac410a060 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:15.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcac9f4a640 1 -- 10.20.193.28:0/1189002738 shutdown_connections 2026-02-25T11:57:15.716 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:15.698+0000 7fcac9f4a640 1 -- 10.20.193.28:0/1189002738 wait complete. 2026-02-25T11:57:15.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:15 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:57:15.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:15 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:57:15.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:15 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:57:15.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:15 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:57:15.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:15 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:57:15.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:15 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:57:15.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:15 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:57:15.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:15 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:57:15.796 INFO:tasks.cephfs.filesystem:scrub status for tag:516f576f-224f-4d5c-9352-d469d781f97c - {'path': '/', 'tag': '516f576f-224f-4d5c-9352-d469d781f97c', 'options': 'recursive,force'} 2026-02-25T11:57:16.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:16 trial028 ceph-mon[39224]: pgmap v497: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:57:16.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:16 trial167 ceph-mon[43392]: pgmap v497: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:57:16.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:16 trial119 ceph-mon[42710]: pgmap v497: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:57:17.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:17 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:57:17.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:17 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:57:17.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:17 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:57:18.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:18 trial028 ceph-mon[39224]: pgmap v498: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:18 trial167 ceph-mon[43392]: pgmap v498: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:18.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:18 trial119 ceph-mon[42710]: pgmap v498: 97 pgs: 97 active+clean; 6.9 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: pgmap v499: 97 pgs: 97 active+clean; 7.0 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:57:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:57:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.560 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:57:20.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:57:20.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:57:20.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:57:20.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:57:20.561 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:20 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: pgmap v499: 97 pgs: 97 active+clean; 7.0 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:57:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:57:20.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:20 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:57:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: pgmap v499: 97 pgs: 97 active+clean; 7.0 MiB data, 349 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:57:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:57:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2026-02-25T11:57:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:57:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:57:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:57:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:57:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:57:20.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:20 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:57:21.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:21 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial119 to 16917M 2026-02-25T11:57:21.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:21 trial028 ceph-mon[39224]: Adjusting osd_memory_target on trial167 to 16917M 2026-02-25T11:57:21.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:21 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial119 to 16917M 2026-02-25T11:57:21.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:21 trial167 ceph-mon[43392]: Adjusting osd_memory_target on trial167 to 16917M 2026-02-25T11:57:21.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:21 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial119 to 16917M 2026-02-25T11:57:21.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:21 trial119 ceph-mon[42710]: Adjusting osd_memory_target on trial167 to 16917M 2026-02-25T11:57:22.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:22 trial028 ceph-mon[39224]: pgmap v500: 97 pgs: 97 active+clean; 7.0 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:22.651 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:22 trial167 ceph-mon[43392]: pgmap v500: 97 pgs: 97 active+clean; 7.0 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:22.660 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:22 trial119 ceph-mon[42710]: pgmap v500: 97 pgs: 97 active+clean; 7.0 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:24.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:24 trial028 ceph-mon[39224]: pgmap v501: 97 pgs: 97 active+clean; 7.0 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:57:24.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:24 trial167 ceph-mon[43392]: pgmap v501: 97 pgs: 97 active+clean; 7.0 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:57:24.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:24 trial119 ceph-mon[42710]: pgmap v501: 97 pgs: 97 active+clean; 7.0 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:57:26.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:26 trial028 ceph-mon[39224]: pgmap v502: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:57:26.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:26 trial167 ceph-mon[43392]: pgmap v502: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:57:26.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:26 trial119 ceph-mon[42710]: pgmap v502: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:57:28.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:28 trial028 ceph-mon[39224]: pgmap v503: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:28.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:28 trial167 ceph-mon[43392]: pgmap v503: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:28.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:28 trial119 ceph-mon[42710]: pgmap v503: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:30.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:30 trial028 ceph-mon[39224]: pgmap v504: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:30.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:30 trial167 ceph-mon[43392]: pgmap v504: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:30.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:30 trial119 ceph-mon[42710]: pgmap v504: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:32.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:32 trial028 ceph-mon[39224]: pgmap v505: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:57:32.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:32 trial167 ceph-mon[43392]: pgmap v505: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:57:32.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:32 trial119 ceph-mon[42710]: pgmap v505: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 10 op/s 2026-02-25T11:57:34.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:34 trial167 ceph-mon[43392]: pgmap v506: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:57:34.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:34 trial119 ceph-mon[42710]: pgmap v506: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:57:34.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:34 trial028 ceph-mon[39224]: pgmap v506: 97 pgs: 97 active+clean; 7.1 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:57:36.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:36 trial167 ceph-mon[43392]: pgmap v507: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:57:36.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:36 trial119 ceph-mon[42710]: pgmap v507: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:57:36.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:36 trial028 ceph-mon[39224]: pgmap v507: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:57:38.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:38 trial167 ceph-mon[43392]: pgmap v508: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:38.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:38 trial119 ceph-mon[42710]: pgmap v508: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:38.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:38 trial028 ceph-mon[39224]: pgmap v508: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:40.612 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:40 trial028 ceph-mon[39224]: pgmap v509: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:40.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:40 trial167 ceph-mon[43392]: pgmap v509: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:40.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:40 trial119 ceph-mon[42710]: pgmap v509: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:42.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:42 trial167 ceph-mon[43392]: pgmap v510: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:57:42.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:42 trial119 ceph-mon[42710]: pgmap v510: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:57:42.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:42 trial028 ceph-mon[39224]: pgmap v510: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:57:44.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:44 trial167 ceph-mon[43392]: pgmap v511: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:57:44.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:44 trial119 ceph-mon[42710]: pgmap v511: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:57:44.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:44 trial028 ceph-mon[39224]: pgmap v511: 97 pgs: 97 active+clean; 7.2 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:57:45.797 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:57:45.984 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:46.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.156+0000 7f4914b77640 1 -- 10.20.193.28:0/2596585133 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4910153730 msgr2=0x7f4910173b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.156+0000 7f4914b77640 1 --2- 10.20.193.28:0/2596585133 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4910153730 0x7f4910173b50 secure :-1 s=READY pgs=856 gs=1 cs=0 l=1 c_cookie=4868404572753645066 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f490400aa30 tx=0x7f4904030270 comp rx=0 tx=0).stop 2026-02-25T11:57:46.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.156+0000 7f4914b77640 1 -- 10.20.193.28:0/2596585133 shutdown_connections 2026-02-25T11:57:46.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.156+0000 7f4914b77640 1 --2- 10.20.193.28:0/2596585133 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f4910153730 0x7f4910173b50 unknown :-1 s=CLOSED pgs=856 gs=1 cs=0 l=1 c_cookie=4868404572753645066 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.156+0000 7f4914b77640 1 --2- 10.20.193.28:0/2596585133 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f491010eb70 0x7f49101531f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.157 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.156+0000 7f4914b77640 1 --2- 10.20.193.28:0/2596585133 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f491010e180 0x7f491010e550 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.156+0000 7f4914b77640 1 -- 10.20.193.28:0/2596585133 >> 10.20.193.28:0/2596585133 conn(0x7f49100e9b50 msgr2=0x7f4910109f60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:46.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.156+0000 7f4914b77640 1 -- 10.20.193.28:0/2596585133 shutdown_connections 2026-02-25T11:57:46.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.156+0000 7f4914b77640 1 -- 10.20.193.28:0/2596585133 wait complete. 2026-02-25T11:57:46.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.157+0000 7f4914b77640 1 Processor -- start 2026-02-25T11:57:46.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.157+0000 7f4914b77640 1 -- start start 2026-02-25T11:57:46.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.157+0000 7f4914b77640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f491010e180 0x7f49101621a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.157+0000 7f4914b77640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f491010eb70 0x7f49101626e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.157+0000 7f4914b77640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4910153730 0x7f4910167ab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.157+0000 7f4914b77640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4910168040 con 0x7f491010e180 2026-02-25T11:57:46.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.157+0000 7f4914b77640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f49101681b0 con 0x7f491010eb70 2026-02-25T11:57:46.158 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.157+0000 7f4914b77640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4910168370 con 0x7f4910153730 2026-02-25T11:57:46.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f491010e180 0x7f49101621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f491010e180 0x7f49101621a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44358/0 (socket says 10.20.193.28:44358) 2026-02-25T11:57:46.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490f7fe640 1 -- 10.20.193.28:0/3780606823 learned_addr learned my addr 10.20.193.28:0/3780606823 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:46.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490ffff640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4910153730 0x7f4910167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490effd640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f491010eb70 0x7f49101626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490f7fe640 1 -- 10.20.193.28:0/3780606823 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4910153730 msgr2=0x7f4910167ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490f7fe640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4910153730 0x7f4910167ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.159 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490f7fe640 1 -- 10.20.193.28:0/3780606823 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f491010eb70 msgr2=0x7f49101626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490f7fe640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f491010eb70 0x7f49101626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490f7fe640 1 -- 10.20.193.28:0/3780606823 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490400a650 con 0x7f491010e180 2026-02-25T11:57:46.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490f7fe640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f491010e180 0x7f49101621a0 secure :-1 s=READY pgs=857 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f48fc005db0 tx=0x7f48fc00ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:46.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490cff9640 1 -- 10.20.193.28:0/3780606823 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f48fc004500 con 0x7f491010e180 2026-02-25T11:57:46.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490cff9640 1 -- 10.20.193.28:0/3780606823 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f48fc004d10 con 0x7f491010e180 2026-02-25T11:57:46.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f490cff9640 1 -- 10.20.193.28:0/3780606823 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f48fc005280 con 0x7f491010e180 2026-02-25T11:57:46.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.158+0000 7f4914b77640 1 -- 10.20.193.28:0/3780606823 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4910168650 con 0x7f491010e180 2026-02-25T11:57:46.160 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.159+0000 7f4914b77640 1 -- 10.20.193.28:0/3780606823 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4910168930 con 0x7f491010e180 2026-02-25T11:57:46.162 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.161+0000 7f490cff9640 1 -- 10.20.193.28:0/3780606823 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f48fc020020 con 0x7f491010e180 2026-02-25T11:57:46.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.163+0000 7f490cff9640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f48e40b5760 0x7f48e40d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.163+0000 7f490cff9640 1 -- 10.20.193.28:0/3780606823 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f48fc0e6620 con 0x7f491010e180 2026-02-25T11:57:46.164 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.163+0000 7f490effd640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f48e40b5760 0x7f48e40d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.164+0000 7f490effd640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f48e40b5760 0x7f48e40d5c10 secure :-1 s=READY pgs=379 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4900005fd0 tx=0x7f490000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:46.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.164+0000 7f4917dfbe00 1 Processor -- start 2026-02-25T11:57:46.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.164+0000 7f4917dfbe00 1 -- start start 2026-02-25T11:57:46.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.164+0000 7f4917dfbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191140 0x55e0a9191510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.164+0000 7f4917dfbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191a50 0x55e0a91d61b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.165 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.164+0000 7f4917dfbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a91d66f0 0x55e0a91f6ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.164+0000 7f4917dfbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e0a90a79b0 con 0x55e0a9191a50 2026-02-25T11:57:46.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.164+0000 7f4917dfbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e0a90a36e0 con 0x55e0a91d66f0 2026-02-25T11:57:46.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.164+0000 7f4917dfbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e0a909d840 con 0x55e0a9191140 2026-02-25T11:57:46.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.164+0000 7f490effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191a50 0x55e0a91d61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.164+0000 7f490effd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191a50 0x55e0a91d61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44372/0 (socket says 10.20.193.28:44372) 2026-02-25T11:57:46.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f490effd640 1 -- 10.20.193.28:0/408266734 learned_addr learned my addr 10.20.193.28:0/408266734 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:46.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f490f7fe640 1 --2- 10.20.193.28:0/408266734 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191140 0x55e0a9191510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f490ffff640 1 --2- 10.20.193.28:0/408266734 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a91d66f0 0x55e0a91f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f490effd640 1 -- 10.20.193.28:0/408266734 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191140 msgr2=0x55e0a9191510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:46.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f490effd640 1 --2- 10.20.193.28:0/408266734 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191140 0x55e0a9191510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f490effd640 1 -- 10.20.193.28:0/408266734 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a91d66f0 msgr2=0x55e0a91f6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:46.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f490effd640 1 --2- 10.20.193.28:0/408266734 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a91d66f0 0x55e0a91f6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.166 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f490effd640 1 -- 10.20.193.28:0/408266734 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a9048d80 con 0x55e0a9191a50 2026-02-25T11:57:46.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f490effd640 1 --2- 10.20.193.28:0/408266734 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191a50 0x55e0a91d61b0 secure :-1 s=READY pgs=858 gs=1 cs=0 l=1 c_cookie=7465430478779422095 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f490000ee70 tx=0x7f4900011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679a89778705958f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:46.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f48d3fff640 1 -- 10.20.193.28:0/408266734 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4900004560 con 0x55e0a9191a50 2026-02-25T11:57:46.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f48d3fff640 1 -- 10.20.193.28:0/408266734 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f4900002e30 con 0x55e0a9191a50 2026-02-25T11:57:46.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f48d3fff640 1 -- 10.20.193.28:0/408266734 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f4900010b70 con 0x55e0a9191a50 2026-02-25T11:57:46.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f4917dfbe00 1 -- 10.20.193.28:0/408266734 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191a50 msgr2=0x55e0a91d61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.165+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/408266734 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191a50 0x55e0a91d61b0 secure :-1 s=READY pgs=858 gs=1 cs=0 l=1 c_cookie=7465430478779422095 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f490000ee70 tx=0x7f4900011000 comp rx=0 tx=0).stop 2026-02-25T11:57:46.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.166+0000 7f4917dfbe00 1 -- 10.20.193.28:0/408266734 shutdown_connections 2026-02-25T11:57:46.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.166+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/408266734 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a91d66f0 0x55e0a91f6ad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.167 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.166+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/408266734 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191a50 0x55e0a91d61b0 unknown :-1 s=CLOSED pgs=858 gs=1 cs=0 l=1 c_cookie=7465430478779422095 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.166+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/408266734 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191140 0x55e0a9191510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.166+0000 7f4917dfbe00 1 -- 10.20.193.28:0/408266734 >> 10.20.193.28:0/408266734 conn(0x55e0a90b0d40 msgr2=0x55e0a918fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:46.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.166+0000 7f4917dfbe00 1 -- 10.20.193.28:0/408266734 shutdown_connections 2026-02-25T11:57:46.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.166+0000 7f4917dfbe00 1 -- 10.20.193.28:0/408266734 wait complete. 2026-02-25T11:57:46.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.167+0000 7f4917dfbe00 1 -- start start 2026-02-25T11:57:46.168 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.167+0000 7f4917dfbe00 1 Processor -- start 2026-02-25T11:57:46.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f4917dfbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 0x55e0a91ee4e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f4917dfbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191a50 0x55e0a91eea20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f4917dfbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a91d66f0 0x55e0a91f3a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f4917dfbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e0a91f3f50 con 0x55e0a9191140 2026-02-25T11:57:46.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f4917dfbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e0a91f40c0 con 0x55e0a91d66f0 2026-02-25T11:57:46.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f4917dfbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e0a91f4280 con 0x55e0a9191a50 2026-02-25T11:57:46.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f490f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 0x55e0a91ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f490f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 0x55e0a91ee4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44380/0 (socket says 10.20.193.28:44380) 2026-02-25T11:57:46.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f490f7fe640 1 -- 10.20.193.28:0/1546365321 learned_addr learned my addr 10.20.193.28:0/1546365321 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:46.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f490effd640 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191a50 0x55e0a91eea20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.169 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f490ffff640 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a91d66f0 0x55e0a91f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f490f7fe640 1 -- 10.20.193.28:0/1546365321 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191a50 msgr2=0x55e0a91eea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:46.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f490f7fe640 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191a50 0x55e0a91eea20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f490f7fe640 1 -- 10.20.193.28:0/1546365321 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a91d66f0 msgr2=0x55e0a91f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:46.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f490f7fe640 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a91d66f0 0x55e0a91f3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.168+0000 7f490f7fe640 1 -- 10.20.193.28:0/1546365321 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a9021350 con 0x55e0a9191140 2026-02-25T11:57:46.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.169+0000 7f490ffff640 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a91d66f0 0x55e0a91f3a10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:57:46.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.169+0000 7f490f7fe640 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 0x55e0a91ee4e0 secure :-1 s=READY pgs=859 gs=1 cs=0 l=1 c_cookie=7138966300244118335 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f48fc0a9c10 tx=0x7f48fc0a8ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6312b406b5890b3f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:46.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.169+0000 7f48d17fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:46.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.169+0000 7f48d17fa640 1 -- 10.20.193.28:0/1546365321 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f48fc0ac760 con 0x55e0a9191140 2026-02-25T11:57:46.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.169+0000 7f48d17fa640 1 -- 10.20.193.28:0/1546365321 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f48fc0b2070 con 0x55e0a9191140 2026-02-25T11:57:46.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.169+0000 7f48d17fa640 1 -- 10.20.193.28:0/1546365321 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f48fc0ad850 con 0x55e0a9191140 2026-02-25T11:57:46.170 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.169+0000 7f4917dfbe00 1 -- 10.20.193.28:0/1546365321 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490000eb10 con 0x55e0a9191140 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.169+0000 7f4917dfbe00 10 client.17430 _ll_register_callbacks cb 0x55e0a901e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.169+0000 7f4917dfbe00 1 -- 10.20.193.28:0/1546365321 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e0a8d336f0 con 0x55e0a9191140 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.169+0000 7f48d17fa640 1 -- 10.20.193.28:0/1546365321 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f48fc0a8040 con 0x55e0a9191140 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.170+0000 7f48d17fa640 1 -- 10.20.193.28:0/1546365321 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f48fc009940 con 0x55e0a9191140 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.170+0000 7f4917dfbe00 10 client.17430 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.170+0000 7f4917dfbe00 1 -- 10.20.193.28:0/1546365321 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55e0a9033150 con 0x55e0a9191140 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.170+0000 7f48d17fa640 1 -- 10.20.193.28:0/1546365321 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f48fc0b7650 con 0x55e0a9191140 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.170+0000 7f4917dfbe00 10 client.17430 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.170+0000 7f4917dfbe00 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.170+0000 7f4917dfbe00 20 client.17430 populate_metadata read hostname 'trial028' 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.170+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e0a91f69f0 0x55e0a91d4420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.170+0000 7f4917dfbe00 4 client.17430 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.170+0000 7f4917dfbe00 1 -- 10.20.193.28:0/1546365321 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0a906f910 con 0x55e0a91f69f0 2026-02-25T11:57:46.171 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.170+0000 7f490effd640 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e0a91f69f0 0x55e0a91d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.171+0000 7f490effd640 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e0a91f69f0 0x55e0a91d4420 crc :-1 s=READY pgs=168 gs=4 cs=0 l=0 c_cookie=3555774103160634172 s_cookie=639771555956966888 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3158a5b5206c3b3c server_cookie=8e0ecd72a36c5e8 in_seq=0 out_seq=0 2026-02-25T11:57:46.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.171+0000 7f48d17fa640 10 client.17430 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:46.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.172+0000 7f48d17fa640 1 -- 10.20.193.28:0/1546365321 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55e0a906f910 con 0x55e0a91f69f0 2026-02-25T11:57:46.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.172+0000 7f48d17fa640 10 client.17430 handle_command_reply: tid=0 2026-02-25T11:57:46.172 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.172+0000 7f4917dfbe00 1 client.17430 shutdown 2026-02-25T11:57:46.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.172+0000 7f4917dfbe00 1 -- 10.20.193.28:0/1546365321 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 msgr2=0x55e0a91ee4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.172+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 0x55e0a91ee4e0 secure :-1 s=READY pgs=859 gs=1 cs=0 l=1 c_cookie=7138966300244118335 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f48fc0a9c10 tx=0x7f48fc0a8ec0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.172+0000 7f4917dfbe00 1 -- 10.20.193.28:0/1546365321 shutdown_connections 2026-02-25T11:57:46.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.172+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e0a91f69f0 0x55e0a91d4420 crc :-1 s=READY pgs=168 gs=4 cs=0 l=0 c_cookie=3555774103160634172 s_cookie=639771555956966888 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.172+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a91d66f0 0x55e0a91f3a10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.172+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191a50 0x55e0a91eea20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.172+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/1546365321 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 0x55e0a91ee4e0 unknown :-1 s=CLOSED pgs=859 gs=1 cs=0 l=1 c_cookie=7138966300244118335 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.172+0000 7f4917dfbe00 1 -- 10.20.193.28:0/1546365321 >> 10.20.193.28:0/1546365321 conn(0x55e0a90b0d40 msgr2=0x55e0a918f230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:46.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.172+0000 7f48d17fa640 0 client.17430 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:46.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.172+0000 7f4917dfbe00 1 -- 10.20.193.28:0/1546365321 shutdown_connections 2026-02-25T11:57:46.173 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.173+0000 7f4917dfbe00 1 -- 10.20.193.28:0/1546365321 wait complete. 2026-02-25T11:57:46.174 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.173+0000 7f4917dfbe00 20 client.17430 trim_cache size 0 max 16384 2026-02-25T11:57:46.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.181+0000 7f4917dfbe00 1 Processor -- start 2026-02-25T11:57:46.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.181+0000 7f4917dfbe00 1 -- start start 2026-02-25T11:57:46.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f4917dfbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 0x55e0a91d9140 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.182 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f4917dfbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a9191a50 0x55e0a91dff80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f4917dfbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a91d66f0 0x55e0a91d1300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f4917dfbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e0a909b470 con 0x55e0a9191140 2026-02-25T11:57:46.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f4917dfbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e0a8d5b700 con 0x55e0a9191a50 2026-02-25T11:57:46.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f4917dfbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e0a91ef000 con 0x55e0a91d66f0 2026-02-25T11:57:46.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f490f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 0x55e0a91d9140 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f490f7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 0x55e0a91d9140 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44384/0 (socket says 10.20.193.28:44384) 2026-02-25T11:57:46.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f490f7fe640 1 -- 10.20.193.28:0/106037943 learned_addr learned my addr 10.20.193.28:0/106037943 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:46.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f490f7fe640 1 -- 10.20.193.28:0/106037943 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a91d66f0 msgr2=0x55e0a91d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:46.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f490f7fe640 1 --2- 10.20.193.28:0/106037943 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a91d66f0 0x55e0a91d1300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f490f7fe640 1 -- 10.20.193.28:0/106037943 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a9191a50 msgr2=0x55e0a91dff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:46.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f490f7fe640 1 --2- 10.20.193.28:0/106037943 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a9191a50 0x55e0a91dff80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.183 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f490f7fe640 1 -- 10.20.193.28:0/106037943 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a91e0750 con 0x55e0a9191140 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f490f7fe640 1 --2- 10.20.193.28:0/106037943 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 0x55e0a91d9140 secure :-1 s=READY pgs=860 gs=1 cs=0 l=1 c_cookie=4093386541396607405 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f48fc020960 tx=0x7f48fc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cea15f284f41ad server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f48d37fe640 1 -- 10.20.193.28:0/106037943 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f48fc009940 con 0x55e0a9191140 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f48d37fe640 1 -- 10.20.193.28:0/106037943 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f48fc0b2040 con 0x55e0a9191140 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.182+0000 7f48d37fe640 1 -- 10.20.193.28:0/106037943 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f48fc0b7ae0 con 0x55e0a9191140 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.183+0000 7f4917dfbe00 1 -- 10.20.193.28:0/106037943 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 msgr2=0x55e0a91d9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.183+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/106037943 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 0x55e0a91d9140 secure :-1 s=READY pgs=860 gs=1 cs=0 l=1 c_cookie=4093386541396607405 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f48fc020960 tx=0x7f48fc01a000 comp rx=0 tx=0).stop 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.183+0000 7f4917dfbe00 1 -- 10.20.193.28:0/106037943 shutdown_connections 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.183+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/106037943 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a91d66f0 0x55e0a91d1300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.183+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/106037943 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a9191a50 0x55e0a91dff80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.183+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/106037943 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a9191140 0x55e0a91d9140 unknown :-1 s=CLOSED pgs=860 gs=1 cs=0 l=1 c_cookie=4093386541396607405 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.183+0000 7f4917dfbe00 1 -- 10.20.193.28:0/106037943 >> 10.20.193.28:0/106037943 conn(0x55e0a90b0d40 msgr2=0x55e0a91f3060 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.183+0000 7f4917dfbe00 1 -- 10.20.193.28:0/106037943 shutdown_connections 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.183+0000 7f4917dfbe00 1 -- 10.20.193.28:0/106037943 wait complete. 2026-02-25T11:57:46.184 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.184+0000 7f4917dfbe00 1 -- start start 2026-02-25T11:57:46.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.184+0000 7f4917dfbe00 1 Processor -- start 2026-02-25T11:57:46.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.184+0000 7f4917dfbe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191140 0x55e0a916f220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.184+0000 7f4917dfbe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a9191a50 0x55e0a916f760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.184+0000 7f4917dfbe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a91d66f0 0x55e0a916fca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.184+0000 7f4917dfbe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55e0a916c490 con 0x55e0a91d66f0 2026-02-25T11:57:46.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.184+0000 7f4917dfbe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55e0a923d530 con 0x55e0a9191a50 2026-02-25T11:57:46.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.184+0000 7f4917dfbe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55e0a923d6a0 con 0x55e0a9191140 2026-02-25T11:57:46.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.184+0000 7f490ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a91d66f0 0x55e0a916fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.184+0000 7f490f7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191140 0x55e0a916f220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.184+0000 7f490f7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191140 0x55e0a916f220 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48338/0 (socket says 10.20.193.28:48338) 2026-02-25T11:57:46.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.184+0000 7f490f7fe640 1 -- 10.20.193.28:0/2205476413 learned_addr learned my addr 10.20.193.28:0/2205476413 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:46.185 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.184+0000 7f490effd640 1 --2- 10.20.193.28:0/2205476413 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a9191a50 0x55e0a916f760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f490f7fe640 1 -- 10.20.193.28:0/2205476413 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a9191a50 msgr2=0x55e0a916f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:46.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f490f7fe640 1 --2- 10.20.193.28:0/2205476413 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a9191a50 0x55e0a916f760 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f490f7fe640 1 -- 10.20.193.28:0/2205476413 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a91d66f0 msgr2=0x55e0a916fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:46.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f490f7fe640 1 --2- 10.20.193.28:0/2205476413 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a91d66f0 0x55e0a916fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f490f7fe640 1 -- 10.20.193.28:0/2205476413 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0a902dbd0 con 0x55e0a9191140 2026-02-25T11:57:46.186 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f490f7fe640 1 --2- 10.20.193.28:0/2205476413 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191140 0x55e0a916f220 secure :-1 s=READY pgs=225 gs=2 cs=0 l=1 c_cookie=17982167915429803281 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f48fc009610 tx=0x7f48fc009c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f98d7e720a9eb511 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f48d17fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f48d17fa640 1 -- 10.20.193.28:0/2205476413 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f48fc0ad850 con 0x55e0a9191140 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f4917dfbe00 1 -- 10.20.193.28:0/2205476413 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0a9033150 con 0x55e0a9191140 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f4917dfbe00 10 client.27109 _ll_register_callbacks cb 0x55e0a901e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f4917dfbe00 1 -- 10.20.193.28:0/2205476413 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e0a8df54a0 con 0x55e0a9191140 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f48d17fa640 1 -- 10.20.193.28:0/2205476413 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55e0a8d336f0 con 0x55e0a9191140 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f48d17fa640 1 -- 10.20.193.28:0/2205476413 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f48fc0abf10 con 0x55e0a9191140 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f48d17fa640 1 -- 10.20.193.28:0/2205476413 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f48fc0b26c0 con 0x55e0a9191140 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f48d17fa640 1 -- 10.20.193.28:0/2205476413 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f48fc0ac540 con 0x55e0a9191140 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f4917dfbe00 10 client.27109 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.185+0000 7f4917dfbe00 1 -- 10.20.193.28:0/2205476413 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55e0a9021680 con 0x55e0a9191140 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.186+0000 7f48d17fa640 1 -- 10.20.193.28:0/2205476413 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f48fc071b30 con 0x55e0a9191140 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.186+0000 7f4917dfbe00 10 client.27109 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.186+0000 7f4917dfbe00 10 client.27109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.186+0000 7f4917dfbe00 20 client.27109 populate_metadata read hostname 'trial028' 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.186+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/2205476413 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e0a9283e00 0x55e0a92a42b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.186+0000 7f4917dfbe00 4 client.27109 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:57:46.187 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.186+0000 7f4917dfbe00 1 -- 10.20.193.28:0/2205476413 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f4900002e30 con 0x55e0a9283e00 2026-02-25T11:57:46.188 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.186+0000 7f490effd640 1 --2- 10.20.193.28:0/2205476413 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e0a9283e00 0x55e0a92a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f490effd640 1 --2- 10.20.193.28:0/2205476413 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e0a9283e00 0x55e0a92a42b0 crc :-1 s=READY pgs=169 gs=4 cs=0 l=0 c_cookie=10490873475920499026 s_cookie=759837148788813749 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919711d7e3cf1152 server_cookie=a8b7bdc6cb3efb5 in_seq=0 out_seq=0 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f48d17fa640 10 client.27109 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f48d17fa640 1 -- 10.20.193.28:0/2205476413 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5486 (crc 0 0 0) 0x7f4900002e30 con 0x55e0a9283e00 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f48d17fa640 10 client.27109 handle_command_reply: tid=0 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f4917dfbe00 1 client.27109 shutdown 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f4917dfbe00 1 -- 10.20.193.28:0/2205476413 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191140 msgr2=0x55e0a916f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/2205476413 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191140 0x55e0a916f220 secure :-1 s=READY pgs=225 gs=2 cs=0 l=1 c_cookie=17982167915429803281 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f48fc009610 tx=0x7f48fc009c40 comp rx=0 tx=0).stop 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f4917dfbe00 1 -- 10.20.193.28:0/2205476413 shutdown_connections 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/2205476413 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55e0a9283e00 0x55e0a92a42b0 crc :-1 s=READY pgs=169 gs=4 cs=0 l=0 c_cookie=10490873475920499026 s_cookie=759837148788813749 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/2205476413 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55e0a91d66f0 0x55e0a916fca0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/2205476413 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55e0a9191a50 0x55e0a916f760 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f4917dfbe00 1 --2- 10.20.193.28:0/2205476413 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55e0a9191140 0x55e0a916f220 unknown :-1 s=CLOSED pgs=225 gs=2 cs=0 l=1 c_cookie=17982167915429803281 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f48d17fa640 0 client.27109 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f4917dfbe00 1 -- 10.20.193.28:0/2205476413 >> 10.20.193.28:0/2205476413 conn(0x55e0a90b0d40 msgr2=0x55e0a91eb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f4917dfbe00 1 -- 10.20.193.28:0/2205476413 shutdown_connections 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f4917dfbe00 1 -- 10.20.193.28:0/2205476413 wait complete. 2026-02-25T11:57:46.189 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.187+0000 7f4917dfbe00 20 client.27109 trim_cache size 0 max 16384 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8": { 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:05.873000+0000", 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "850bcfd3-be8a-4930-896c-1ed6e48b8fed": { 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:27.124813+0000", 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:57:46.190 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:57:46.191 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "516f576f-224f-4d5c-9352-d469d781f97c": { 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:57:15.169491+0000", 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:57:46.192 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.193 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.194 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "91e989b7-1f1f-41fb-9047-642867de91e6": { 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:40.578257+0000", 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.195 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:57:46.196 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "7b237934-6b95-43dd-b3a6-82b74134ec55": { 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:31.289725+0000", 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.197 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.188+0000 7f4914b77640 1 -- 10.20.193.28:0/3780606823 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f48e40b5760 msgr2=0x7f48e40d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.188+0000 7f4914b77640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f48e40b5760 0x7f48e40d5c10 secure :-1 s=READY pgs=379 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f4900005fd0 tx=0x7f490000a560 comp rx=0 tx=0).stop 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.188+0000 7f4914b77640 1 -- 10.20.193.28:0/3780606823 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f491010e180 msgr2=0x7f49101621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.188+0000 7f4914b77640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f491010e180 0x7f49101621a0 secure :-1 s=READY pgs=857 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f48fc005db0 tx=0x7f48fc00ddf0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.188+0000 7f4914b77640 1 -- 10.20.193.28:0/3780606823 shutdown_connections 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.188+0000 7f4914b77640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f48e40b5760 0x7f48e40d5c10 unknown :-1 s=CLOSED pgs=379 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.188+0000 7f4914b77640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f4910153730 0x7f4910167ab0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.188+0000 7f4914b77640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f491010eb70 0x7f49101626e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.198 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.188+0000 7f4914b77640 1 --2- 10.20.193.28:0/3780606823 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f491010e180 0x7f49101621a0 unknown :-1 s=CLOSED pgs=857 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.188+0000 7f4914b77640 1 -- 10.20.193.28:0/3780606823 >> 10.20.193.28:0/3780606823 conn(0x7f49100e9b50 msgr2=0x7f4910109f30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:46.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.188+0000 7f4914b77640 1 -- 10.20.193.28:0/3780606823 shutdown_connections 2026-02-25T11:57:46.199 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.188+0000 7f4914b77640 1 -- 10.20.193.28:0/3780606823 wait complete. 2026-02-25T11:57:46.289 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:57:46.289 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:57:46.473 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:46.517 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:46 trial028 ceph-mon[39224]: pgmap v512: 97 pgs: 97 active+clean; 7.3 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:57:46.659 INFO:tasks.workunit.client.0.trial028.stdout:499/600: open fd = 3 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:500/600: open fd = 3 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:501/600: open fd = 3 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:502/600: open fd = 3 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:503/600: open fd = 3 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:504/600: open fd = 3 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:505/600: open fd = 3 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.660 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:506/600: open fd = 3 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:507/600: open fd = 3 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:508/600: open fd = 3 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:509/600: open fd = 3 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:510/600: open fd = 3 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:511/600: open fd = 3 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:512/600: open fd = 3 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:513/600: open fd = 3 2026-02-25T11:57:46.661 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:514/600: open fd = 3 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:515/600: open fd = 3 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:516/600: open fd = 3 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:517/600: open fd = 3 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:518/600: open fd = 3 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:519/600: open fd = 3 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:520/600: open fd = 3 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.662 INFO:tasks.workunit.client.0.trial028.stdout:521/600: open fd = 3 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:522/600: open fd = 3 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:523/600: open fd = 3 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:524/600: open fd = 3 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:525/600: open fd = 3 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:526/600: open fd = 3 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:527/600: open fd = 3 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:528/600: open fd = 3 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.663 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:529/600: open fd = 3 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:530/600: open fd = 3 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:531/600: open fd = 3 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:532/600: open fd = 3 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:533/600: open fd = 3 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:534/600: open fd = 3 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:535/600: open fd = 3 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:536/600: open fd = 3 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.664 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:537/600: open fd = 3 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:538/600: open fd = 3 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:539/600: open fd = 3 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:540/600: open fd = 3 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:541/600: open fd = 3 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:542/600: open fd = 3 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:543/600: open fd = 3 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:544/600: open fd = 3 2026-02-25T11:57:46.665 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:545/600: open fd = 3 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:546/600: open fd = 3 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:547/600: open fd = 3 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:548/600: open fd = 3 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:549/600: open fd = 3 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:550/600: open fd = 3 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.666 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:551/600: open fd = 3 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:552/600: open fd = 3 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:553/600: open fd = 3 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:554/600: open fd = 3 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:555/600: open fd = 3 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:556/600: open fd = 3 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.667 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:557/600: open fd = 3 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:558/600: open fd = 3 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:559/600: open fd = 3 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:560/600: open fd = 3 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:57:46.668 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:57:46.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.661+0000 7f20356ee640 1 -- 10.20.193.28:0/3153724744 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f203010bee0 msgr2=0x7f203010c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.661+0000 7f20356ee640 1 --2- 10.20.193.28:0/3153724744 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f203010bee0 0x7f203010c2b0 secure :-1 s=READY pgs=226 gs=1 cs=0 l=1 c_cookie=15954800216059293508 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f201c009950 tx=0x7f201c02f1d0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.661+0000 7f20356ee640 1 -- 10.20.193.28:0/3153724744 shutdown_connections 2026-02-25T11:57:46.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.661+0000 7f20356ee640 1 --2- 10.20.193.28:0/3153724744 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2030155c10 0x7f2030175ff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.661+0000 7f20356ee640 1 --2- 10.20.193.28:0/3153724744 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f203010c7f0 0x7f20301556d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.661+0000 7f20356ee640 1 --2- 10.20.193.28:0/3153724744 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f203010bee0 0x7f203010c2b0 unknown :-1 s=CLOSED pgs=226 gs=1 cs=0 l=1 c_cookie=15954800216059293508 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.661+0000 7f20356ee640 1 -- 10.20.193.28:0/3153724744 >> 10.20.193.28:0/3153724744 conn(0x7f20300e9b40 msgr2=0x7f2030109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:46.670 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.661+0000 7f20356ee640 1 -- 10.20.193.28:0/3153724744 shutdown_connections 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.661+0000 7f20356ee640 1 -- 10.20.193.28:0/3153724744 wait complete. 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.662+0000 7f20356ee640 1 Processor -- start 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.662+0000 7f20356ee640 1 -- start start 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.662+0000 7f20356ee640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f203010bee0 0x7f20301613b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.662+0000 7f20356ee640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f203010c7f0 0x7f20301618f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.662+0000 7f20356ee640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2030155c10 0x7f2030166cc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.662+0000 7f20356ee640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f20301672e0 con 0x7f2030155c10 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.662+0000 7f20356ee640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2030167450 con 0x7f203010c7f0 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.662+0000 7f20356ee640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2030167610 con 0x7f203010bee0 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.662+0000 7f202f7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f203010c7f0 0x7f20301618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.662+0000 7f202ffff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f203010bee0 0x7f20301613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.662+0000 7f202f7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f203010c7f0 0x7f20301618f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:49014/0 (socket says 10.20.193.28:49014) 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.662+0000 7f202f7fe640 1 -- 10.20.193.28:0/509608218 learned_addr learned my addr 10.20.193.28:0/509608218 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.663+0000 7f2034eed640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2030155c10 0x7f2030166cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.663+0000 7f202ffff640 1 -- 10.20.193.28:0/509608218 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f203010c7f0 msgr2=0x7f20301618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.663+0000 7f202ffff640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f203010c7f0 0x7f20301618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.663+0000 7f202ffff640 1 -- 10.20.193.28:0/509608218 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2030155c10 msgr2=0x7f2030166cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.663+0000 7f202ffff640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2030155c10 0x7f2030166cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.671 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.663+0000 7f202ffff640 1 -- 10.20.193.28:0/509608218 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201c009570 con 0x7f203010bee0 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.663+0000 7f2034eed640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2030155c10 0x7f2030166cc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.663+0000 7f202ffff640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f203010bee0 0x7f20301613b0 secure :-1 s=READY pgs=227 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f201c009950 tx=0x7f201c0028d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.663+0000 7f202d7fa640 1 -- 10.20.193.28:0/509608218 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f201c03d070 con 0x7f203010bee0 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.663+0000 7f202d7fa640 1 -- 10.20.193.28:0/509608218 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f201c0043b0 con 0x7f203010bee0 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.663+0000 7f20356ee640 1 -- 10.20.193.28:0/509608218 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2030167890 con 0x7f203010bee0 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.663+0000 7f202d7fa640 1 -- 10.20.193.28:0/509608218 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f201c041720 con 0x7f203010bee0 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.663+0000 7f20356ee640 1 -- 10.20.193.28:0/509608218 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2030152840 con 0x7f203010bee0 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.664+0000 7f20356ee640 1 -- 10.20.193.28:0/509608218 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2030060420 con 0x7f203010bee0 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.666+0000 7f202d7fa640 1 -- 10.20.193.28:0/509608218 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f201c0386a0 con 0x7f203010bee0 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.667+0000 7f202d7fa640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f20040b56e0 0x7f20040d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.668+0000 7f202d7fa640 1 -- 10.20.193.28:0/509608218 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f201c10b3b0 con 0x7f203010bee0 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.668+0000 7f202f7fe640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f20040b56e0 0x7f20040d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.668+0000 7f202d7fa640 1 -- 10.20.193.28:0/509608218 <== mon.1 v2:10.20.193.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f201c10b8a0 con 0x7f203010bee0 2026-02-25T11:57:46.672 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.668+0000 7f202f7fe640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f20040b56e0 0x7f20040d5b90 secure :-1 s=READY pgs=380 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2030167d50 tx=0x7f2020005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:46.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:46 trial167 ceph-mon[43392]: pgmap v512: 97 pgs: 97 active+clean; 7.3 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:57:46.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:46 trial119 ceph-mon[42710]: pgmap v512: 97 pgs: 97 active+clean; 7.3 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:57:46.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.810+0000 7f20356ee640 1 -- 10.20.193.28:0/509608218 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f203010c2b0 con 0x7f203010bee0 2026-02-25T11:57:46.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.810+0000 7f202d7fa640 1 -- 10.20.193.28:0/509608218 <== mon.1 v2:10.20.193.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f201c0cddc0 con 0x7f203010bee0 2026-02-25T11:57:46.813 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:57:46.813 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:57:46.814 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:57:46.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.812+0000 7f20356ee640 1 -- 10.20.193.28:0/509608218 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f20040b56e0 msgr2=0x7f20040d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.812+0000 7f20356ee640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f20040b56e0 0x7f20040d5b90 secure :-1 s=READY pgs=380 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2030167d50 tx=0x7f2020005f90 comp rx=0 tx=0).stop 2026-02-25T11:57:46.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.812+0000 7f20356ee640 1 -- 10.20.193.28:0/509608218 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f203010bee0 msgr2=0x7f20301613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:46.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.812+0000 7f20356ee640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f203010bee0 0x7f20301613b0 secure :-1 s=READY pgs=227 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f201c009950 tx=0x7f201c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.812+0000 7f20356ee640 1 -- 10.20.193.28:0/509608218 shutdown_connections 2026-02-25T11:57:46.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.812+0000 7f20356ee640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f20040b56e0 0x7f20040d5b90 unknown :-1 s=CLOSED pgs=380 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.812+0000 7f20356ee640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2030155c10 0x7f2030166cc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.812+0000 7f20356ee640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f203010c7f0 0x7f20301618f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.812+0000 7f20356ee640 1 --2- 10.20.193.28:0/509608218 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f203010bee0 0x7f20301613b0 unknown :-1 s=CLOSED pgs=227 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:46.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.812+0000 7f20356ee640 1 -- 10.20.193.28:0/509608218 >> 10.20.193.28:0/509608218 conn(0x7f20300e9b40 msgr2=0x7f2030109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:46.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.812+0000 7f20356ee640 1 -- 10.20.193.28:0/509608218 shutdown_connections 2026-02-25T11:57:46.816 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:46.812+0000 7f20356ee640 1 -- 10.20.193.28:0/509608218 wait complete. 2026-02-25T11:57:46.912 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:57:47.093 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:47.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.267+0000 7fe931d69640 1 -- 10.20.193.28:0/1212662972 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe92c061090 msgr2=0x7fe92c061460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.267+0000 7fe931d69640 1 --2- 10.20.193.28:0/1212662972 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe92c061090 0x7fe92c061460 secure :-1 s=READY pgs=861 gs=1 cs=0 l=1 c_cookie=16424946548129618735 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe918009950 tx=0x7fe91802f190 comp rx=0 tx=0).stop 2026-02-25T11:57:47.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.268+0000 7fe931d69640 1 -- 10.20.193.28:0/1212662972 shutdown_connections 2026-02-25T11:57:47.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.268+0000 7fe931d69640 1 --2- 10.20.193.28:0/1212662972 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe92c155bd0 0x7fe92c175fb0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.268+0000 7fe931d69640 1 --2- 10.20.193.28:0/1212662972 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe92c0619a0 0x7fe92c155690 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.268+0000 7fe931d69640 1 --2- 10.20.193.28:0/1212662972 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe92c061090 0x7fe92c061460 unknown :-1 s=CLOSED pgs=861 gs=1 cs=0 l=1 c_cookie=16424946548129618735 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.268+0000 7fe931d69640 1 -- 10.20.193.28:0/1212662972 >> 10.20.193.28:0/1212662972 conn(0x7fe92c0eadc0 msgr2=0x7fe92c10b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:47.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.268+0000 7fe931d69640 1 -- 10.20.193.28:0/1212662972 shutdown_connections 2026-02-25T11:57:47.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.268+0000 7fe931d69640 1 -- 10.20.193.28:0/1212662972 wait complete. 2026-02-25T11:57:47.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.269+0000 7fe931d69640 1 Processor -- start 2026-02-25T11:57:47.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.269+0000 7fe931d69640 1 -- start start 2026-02-25T11:57:47.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.269+0000 7fe931d69640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe92c061090 0x7fe92c1624a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.269+0000 7fe931d69640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe92c0619a0 0x7fe92c1629e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.269+0000 7fe931d69640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe92c155bd0 0x7fe92c167db0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.269+0000 7fe931d69640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1683d0 con 0x7fe92c0619a0 2026-02-25T11:57:47.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.269+0000 7fe931d69640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c168540 con 0x7fe92c155bd0 2026-02-25T11:57:47.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.269+0000 7fe931d69640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c168700 con 0x7fe92c061090 2026-02-25T11:57:47.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.270+0000 7fe92b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe92c061090 0x7fe92c1624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.270+0000 7fe92b7fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe92c061090 0x7fe92c1624a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48374/0 (socket says 10.20.193.28:48374) 2026-02-25T11:57:47.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.270+0000 7fe92b7fe640 1 -- 10.20.193.28:0/2444726796 learned_addr learned my addr 10.20.193.28:0/2444726796 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:47.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.270+0000 7fe92affd640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe92c0619a0 0x7fe92c1629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.270+0000 7fe92bfff640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe92c155bd0 0x7fe92c167db0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.270+0000 7fe92b7fe640 1 -- 10.20.193.28:0/2444726796 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe92c155bd0 msgr2=0x7fe92c167db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.270+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe92c155bd0 0x7fe92c167db0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.270+0000 7fe92b7fe640 1 -- 10.20.193.28:0/2444726796 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe92c0619a0 msgr2=0x7fe92c1629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.270+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe92c0619a0 0x7fe92c1629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.270+0000 7fe92b7fe640 1 -- 10.20.193.28:0/2444726796 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe918009570 con 0x7fe92c061090 2026-02-25T11:57:47.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.270+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe92c061090 0x7fe92c1624a0 secure :-1 s=READY pgs=228 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe9180098a0 tx=0x7fe9180028d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:47.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.271+0000 7fe928ff9640 1 -- 10.20.193.28:0/2444726796 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe91803d070 con 0x7fe92c061090 2026-02-25T11:57:47.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.271+0000 7fe928ff9640 1 -- 10.20.193.28:0/2444726796 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe918002a70 con 0x7fe92c061090 2026-02-25T11:57:47.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.271+0000 7fe928ff9640 1 -- 10.20.193.28:0/2444726796 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe918041890 con 0x7fe92c061090 2026-02-25T11:57:47.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.271+0000 7fe931d69640 1 -- 10.20.193.28:0/2444726796 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c168980 con 0x7fe92c061090 2026-02-25T11:57:47.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.271+0000 7fe931d69640 1 -- 10.20.193.28:0/2444726796 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c151650 con 0x7fe92c061090 2026-02-25T11:57:47.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.274+0000 7fe928ff9640 1 -- 10.20.193.28:0/2444726796 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fe918041a20 con 0x7fe92c061090 2026-02-25T11:57:47.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.275+0000 7fe928ff9640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe9000b5760 0x7fe9000d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.275+0000 7fe928ff9640 1 -- 10.20.193.28:0/2444726796 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fe91810af70 con 0x7fe92c061090 2026-02-25T11:57:47.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.275+0000 7fe92affd640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe9000b5760 0x7fe9000d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.276+0000 7fe92affd640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe9000b5760 0x7fe9000d5c10 secure :-1 s=READY pgs=381 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe92c168e40 tx=0x7fe91c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:47.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.276+0000 7fe933d64e00 1 Processor -- start 2026-02-25T11:57:47.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.276+0000 7fe933d64e00 1 -- start start 2026-02-25T11:57:47.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe933d64e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b8f7f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe933d64e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b8f8440 0x557e3b93cba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe933d64e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b93d0e0 0x557e3b95d4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe933d64e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x557e3b80d8a0 con 0x557e3b8f7b30 2026-02-25T11:57:47.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe933d64e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x557e3b8095d0 con 0x557e3b93d0e0 2026-02-25T11:57:47.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe933d64e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x557e3b803570 con 0x557e3b8f8440 2026-02-25T11:57:47.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe92b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b8f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe92affd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b8f8440 0x557e3b93cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe92bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b93d0e0 0x557e3b95d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe92affd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b8f8440 0x557e3b93cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48386/0 (socket says 10.20.193.28:48386) 2026-02-25T11:57:47.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe92affd640 1 -- 10.20.193.28:0/681084784 learned_addr learned my addr 10.20.193.28:0/681084784 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:47.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe92b7fe640 1 -- 10.20.193.28:0/681084784 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b8f8440 msgr2=0x557e3b93cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/681084784 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b8f8440 0x557e3b93cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe92b7fe640 1 -- 10.20.193.28:0/681084784 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b93d0e0 msgr2=0x557e3b95d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/681084784 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b93d0e0 0x557e3b95d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe92b7fe640 1 -- 10.20.193.28:0/681084784 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e3b7aed10 con 0x557e3b8f7b30 2026-02-25T11:57:47.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe92bfff640 1 --2- 10.20.193.28:0/681084784 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b93d0e0 0x557e3b95d4c0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:57:47.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/681084784 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b8f7f00 secure :-1 s=READY pgs=862 gs=3 cs=0 l=1 c_cookie=14276650862695295943 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe918004030 tx=0x7fe918038f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c620da51953a2bc7 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:47.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.277+0000 7fe8effff640 1 -- 10.20.193.28:0/681084784 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe918041d00 con 0x557e3b8f7b30 2026-02-25T11:57:47.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.278+0000 7fe8effff640 1 -- 10.20.193.28:0/681084784 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe9180d1cf0 con 0x557e3b8f7b30 2026-02-25T11:57:47.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.278+0000 7fe8effff640 1 -- 10.20.193.28:0/681084784 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe9180d1f80 con 0x557e3b8f7b30 2026-02-25T11:57:47.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.278+0000 7fe933d64e00 1 -- 10.20.193.28:0/681084784 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 msgr2=0x557e3b8f7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.278+0000 7fe933d64e00 1 --2- 10.20.193.28:0/681084784 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b8f7f00 secure :-1 s=READY pgs=862 gs=3 cs=0 l=1 c_cookie=14276650862695295943 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe918004030 tx=0x7fe918038f30 comp rx=0 tx=0).stop 2026-02-25T11:57:47.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.278+0000 7fe933d64e00 1 -- 10.20.193.28:0/681084784 shutdown_connections 2026-02-25T11:57:47.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.278+0000 7fe933d64e00 1 --2- 10.20.193.28:0/681084784 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b93d0e0 0x557e3b95d4c0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.278+0000 7fe933d64e00 1 --2- 10.20.193.28:0/681084784 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b8f8440 0x557e3b93cba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.278+0000 7fe933d64e00 1 --2- 10.20.193.28:0/681084784 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b8f7f00 unknown :-1 s=CLOSED pgs=862 gs=3 cs=0 l=1 c_cookie=14276650862695295943 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.278+0000 7fe933d64e00 1 -- 10.20.193.28:0/681084784 >> 10.20.193.28:0/681084784 conn(0x557e3b8167a0 msgr2=0x557e3b8f6410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:47.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.278+0000 7fe933d64e00 1 -- 10.20.193.28:0/681084784 shutdown_connections 2026-02-25T11:57:47.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.278+0000 7fe933d64e00 1 -- 10.20.193.28:0/681084784 wait complete. 2026-02-25T11:57:47.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.279+0000 7fe933d64e00 1 -- start start 2026-02-25T11:57:47.284 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.279+0000 7fe933d64e00 1 Processor -- start 2026-02-25T11:57:47.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe933d64e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b90fc30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe933d64e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b8f8440 0x557e3b910170 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe933d64e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b93d0e0 0x557e3b915200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe933d64e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x557e3b915740 con 0x557e3b8f7b30 2026-02-25T11:57:47.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe933d64e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x557e3b9158b0 con 0x557e3b8f8440 2026-02-25T11:57:47.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe933d64e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x557e3b915a70 con 0x557e3b93d0e0 2026-02-25T11:57:47.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe92b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b90fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe92bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b93d0e0 0x557e3b915200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe92bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b93d0e0 0x557e3b915200 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48390/0 (socket says 10.20.193.28:48390) 2026-02-25T11:57:47.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe92bfff640 1 -- 10.20.193.28:0/311909054 learned_addr learned my addr 10.20.193.28:0/311909054 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:47.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe92affd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b8f8440 0x557e3b910170 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe92b7fe640 1 -- 10.20.193.28:0/311909054 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b93d0e0 msgr2=0x557e3b915200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/311909054 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b93d0e0 0x557e3b915200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe92b7fe640 1 -- 10.20.193.28:0/311909054 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b8f8440 msgr2=0x557e3b910170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/311909054 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b8f8440 0x557e3b910170 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.280+0000 7fe92b7fe640 1 -- 10.20.193.28:0/311909054 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e3b7873c0 con 0x557e3b8f7b30 2026-02-25T11:57:47.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.281+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/311909054 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b90fc30 secure :-1 s=READY pgs=863 gs=1 cs=0 l=1 c_cookie=3595001164635208023 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe918032040 tx=0x7fe91802f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e4028319327957 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:47.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.281+0000 7fe8ed7fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:47.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.281+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/311909054 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe918041d00 con 0x557e3b8f7b30 2026-02-25T11:57:47.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.281+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/311909054 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe9180d95f0 con 0x557e3b8f7b30 2026-02-25T11:57:47.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.281+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/311909054 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe9180cbae0 con 0x557e3b8f7b30 2026-02-25T11:57:47.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.281+0000 7fe933d64e00 1 -- 10.20.193.28:0/311909054 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c151650 con 0x557e3b8f7b30 2026-02-25T11:57:47.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.281+0000 7fe933d64e00 10 client.17454 _ll_register_callbacks cb 0x557e3b784a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:47.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.281+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/311909054 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fe9180ccc40 con 0x557e3b8f7b30 2026-02-25T11:57:47.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.281+0000 7fe933d64e00 1 -- 10.20.193.28:0/311909054 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557e3b4996f0 con 0x557e3b8f7b30 2026-02-25T11:57:47.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.281+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/311909054 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe9180dc430 con 0x557e3b8f7b30 2026-02-25T11:57:47.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.281+0000 7fe933d64e00 10 client.17454 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:47.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.281+0000 7fe933d64e00 1 -- 10.20.193.28:0/311909054 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x557e3b7990e0 con 0x557e3b8f7b30 2026-02-25T11:57:47.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.282+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/311909054 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe9180cce40 con 0x557e3b8f7b30 2026-02-25T11:57:47.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.282+0000 7fe933d64e00 10 client.17454 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:47.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.282+0000 7fe933d64e00 10 client.17454 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:57:47.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.282+0000 7fe933d64e00 20 client.17454 populate_metadata read hostname 'trial028' 2026-02-25T11:57:47.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.282+0000 7fe933d64e00 1 --2- 10.20.193.28:0/311909054 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x557e3b939ff0 0x557e3b93a490 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.282+0000 7fe933d64e00 4 client.17454 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:57:47.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.282+0000 7fe933d64e00 1 -- 10.20.193.28:0/311909054 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e3b7d5fc0 con 0x557e3b939ff0 2026-02-25T11:57:47.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.282+0000 7fe92affd640 1 --2- 10.20.193.28:0/311909054 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x557e3b939ff0 0x557e3b93a490 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.282+0000 7fe92affd640 1 --2- 10.20.193.28:0/311909054 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x557e3b939ff0 0x557e3b93a490 crc :-1 s=READY pgs=59 gs=4 cs=0 l=0 c_cookie=11054845013328333977 s_cookie=938952883705466044 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=996ab2f7f26f0c99 server_cookie=d07d4ac4617ccbc in_seq=0 out_seq=0 2026-02-25T11:57:47.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.282+0000 7fe8ed7fa640 10 client.17454 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:57:47.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/311909054 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x557e3b7d5fc0 con 0x557e3b939ff0 2026-02-25T11:57:47.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe8ed7fa640 10 client.17454 handle_command_reply: tid=0 2026-02-25T11:57:47.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe933d64e00 1 client.17454 shutdown 2026-02-25T11:57:47.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe933d64e00 1 -- 10.20.193.28:0/311909054 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 msgr2=0x557e3b90fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe933d64e00 1 --2- 10.20.193.28:0/311909054 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b90fc30 secure :-1 s=READY pgs=863 gs=1 cs=0 l=1 c_cookie=3595001164635208023 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe918032040 tx=0x7fe91802f190 comp rx=0 tx=0).stop 2026-02-25T11:57:47.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe933d64e00 1 -- 10.20.193.28:0/311909054 shutdown_connections 2026-02-25T11:57:47.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe933d64e00 1 --2- 10.20.193.28:0/311909054 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x557e3b939ff0 0x557e3b93a490 crc :-1 s=READY pgs=59 gs=4 cs=0 l=0 c_cookie=11054845013328333977 s_cookie=938952883705466044 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe933d64e00 1 --2- 10.20.193.28:0/311909054 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b93d0e0 0x557e3b915200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe933d64e00 1 --2- 10.20.193.28:0/311909054 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b8f8440 0x557e3b910170 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe933d64e00 1 --2- 10.20.193.28:0/311909054 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b90fc30 unknown :-1 s=CLOSED pgs=863 gs=1 cs=0 l=1 c_cookie=3595001164635208023 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe933d64e00 1 -- 10.20.193.28:0/311909054 >> 10.20.193.28:0/311909054 conn(0x557e3b8167a0 msgr2=0x557e3b8f6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:47.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe8ed7fa640 0 client.17454 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:57:47.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe933d64e00 1 -- 10.20.193.28:0/311909054 shutdown_connections 2026-02-25T11:57:47.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.283+0000 7fe933d64e00 1 -- 10.20.193.28:0/311909054 wait complete. 2026-02-25T11:57:47.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.284+0000 7fe933d64e00 20 client.17454 trim_cache size 0 max 16384 2026-02-25T11:57:47.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.290+0000 7fe933d64e00 1 Processor -- start 2026-02-25T11:57:47.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe933d64e00 1 -- start start 2026-02-25T11:57:47.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe933d64e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b910160 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe933d64e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b8f8440 0x557e3b90d270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe933d64e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b93d0e0 0x557e3b8f6d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe933d64e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x557e3b8011a0 con 0x557e3b8f7b30 2026-02-25T11:57:47.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe933d64e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x557e3b4c1700 con 0x557e3b93d0e0 2026-02-25T11:57:47.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe933d64e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x557e3b910750 con 0x557e3b8f8440 2026-02-25T11:57:47.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe92b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b910160 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe92b7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b910160 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44444/0 (socket says 10.20.193.28:44444) 2026-02-25T11:57:47.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe92b7fe640 1 -- 10.20.193.28:0/3623158823 learned_addr learned my addr 10.20.193.28:0/3623158823 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:47.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe92affd640 1 --2- 10.20.193.28:0/3623158823 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b8f8440 0x557e3b90d270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe92bfff640 1 --2- 10.20.193.28:0/3623158823 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b93d0e0 0x557e3b8f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe92b7fe640 1 -- 10.20.193.28:0/3623158823 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b8f8440 msgr2=0x557e3b90d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/3623158823 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b8f8440 0x557e3b90d270 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe92b7fe640 1 -- 10.20.193.28:0/3623158823 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b93d0e0 msgr2=0x557e3b8f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/3623158823 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b93d0e0 0x557e3b8f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.291+0000 7fe92b7fe640 1 -- 10.20.193.28:0/3623158823 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e3b793b60 con 0x557e3b8f7b30 2026-02-25T11:57:47.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.292+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/3623158823 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b910160 secure :-1 s=READY pgs=864 gs=1 cs=0 l=1 c_cookie=4093976406083568792 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe9180d1720 tx=0x7fe918093000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d0b9d9bb710498 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:47.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.292+0000 7fe8ef7fe640 1 -- 10.20.193.28:0/3623158823 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe9180dc430 con 0x557e3b8f7b30 2026-02-25T11:57:47.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.292+0000 7fe8ef7fe640 1 -- 10.20.193.28:0/3623158823 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x557e3b4996f0 con 0x557e3b8f7b30 2026-02-25T11:57:47.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.292+0000 7fe8ef7fe640 1 -- 10.20.193.28:0/3623158823 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe9180d2ce0 con 0x557e3b8f7b30 2026-02-25T11:57:47.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.292+0000 7fe933d64e00 1 -- 10.20.193.28:0/3623158823 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 msgr2=0x557e3b910160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.292+0000 7fe933d64e00 1 --2- 10.20.193.28:0/3623158823 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b910160 secure :-1 s=READY pgs=864 gs=1 cs=0 l=1 c_cookie=4093976406083568792 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe9180d1720 tx=0x7fe918093000 comp rx=0 tx=0).stop 2026-02-25T11:57:47.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.292+0000 7fe933d64e00 1 -- 10.20.193.28:0/3623158823 shutdown_connections 2026-02-25T11:57:47.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.292+0000 7fe933d64e00 1 --2- 10.20.193.28:0/3623158823 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b93d0e0 0x557e3b8f6d20 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.292+0000 7fe933d64e00 1 --2- 10.20.193.28:0/3623158823 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b8f8440 0x557e3b90d270 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.292+0000 7fe933d64e00 1 --2- 10.20.193.28:0/3623158823 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f7b30 0x557e3b910160 unknown :-1 s=CLOSED pgs=864 gs=1 cs=0 l=1 c_cookie=4093976406083568792 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.298 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.292+0000 7fe933d64e00 1 -- 10.20.193.28:0/3623158823 >> 10.20.193.28:0/3623158823 conn(0x557e3b8167a0 msgr2=0x557e3b90fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:47.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.292+0000 7fe933d64e00 1 -- 10.20.193.28:0/3623158823 shutdown_connections 2026-02-25T11:57:47.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.292+0000 7fe933d64e00 1 -- 10.20.193.28:0/3623158823 wait complete. 2026-02-25T11:57:47.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.293+0000 7fe933d64e00 1 -- start start 2026-02-25T11:57:47.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.293+0000 7fe933d64e00 1 Processor -- start 2026-02-25T11:57:47.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.293+0000 7fe933d64e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b8f7b30 0x557e3b90f330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.293+0000 7fe933d64e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f8440 0x557e3b90f870 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.293+0000 7fe933d64e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b93d0e0 0x557e3b91a300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.293+0000 7fe933d64e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x557e3b91a840 con 0x557e3b8f8440 2026-02-25T11:57:47.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.293+0000 7fe933d64e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x557e3b91a9b0 con 0x557e3b8f7b30 2026-02-25T11:57:47.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.293+0000 7fe933d64e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x557e3b91ab70 con 0x557e3b93d0e0 2026-02-25T11:57:47.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.293+0000 7fe92affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f8440 0x557e3b90f870 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.293+0000 7fe92affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f8440 0x557e3b90f870 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44452/0 (socket says 10.20.193.28:44452) 2026-02-25T11:57:47.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.293+0000 7fe92affd640 1 -- 10.20.193.28:0/3183886177 learned_addr learned my addr 10.20.193.28:0/3183886177 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:47.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.293+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/3183886177 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b8f7b30 0x557e3b90f330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.293+0000 7fe92bfff640 1 --2- 10.20.193.28:0/3183886177 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b93d0e0 0x557e3b91a300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe92affd640 1 -- 10.20.193.28:0/3183886177 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b93d0e0 msgr2=0x557e3b91a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe92affd640 1 --2- 10.20.193.28:0/3183886177 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b93d0e0 0x557e3b91a300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe92affd640 1 -- 10.20.193.28:0/3183886177 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b8f7b30 msgr2=0x557e3b90f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe92affd640 1 --2- 10.20.193.28:0/3183886177 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b8f7b30 0x557e3b90f330 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe92affd640 1 -- 10.20.193.28:0/3183886177 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e3b7876f0 con 0x557e3b8f8440 2026-02-25T11:57:47.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe92affd640 1 --2- 10.20.193.28:0/3183886177 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f8440 0x557e3b90f870 secure :-1 s=READY pgs=865 gs=1 cs=0 l=1 c_cookie=17407123998830656176 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe91c004740 tx=0x7fe91c0047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f192870ccf7a7ab0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:47.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe8ed7fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:47.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/3183886177 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe91c024040 con 0x557e3b8f8440 2026-02-25T11:57:47.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/3183886177 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fe91c015070 con 0x557e3b8f8440 2026-02-25T11:57:47.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/3183886177 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fe91c035bd0 con 0x557e3b8f8440 2026-02-25T11:57:47.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe933d64e00 1 -- 10.20.193.28:0/3183886177 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e3b7990e0 con 0x557e3b8f8440 2026-02-25T11:57:47.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/3183886177 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fe91c017070 con 0x557e3b8f8440 2026-02-25T11:57:47.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe8ed7fa640 10 client.17460 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:57:47.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe933d64e00 10 client.17460 _ll_register_callbacks cb 0x557e3b784a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:47.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.294+0000 7fe933d64e00 1 -- 10.20.193.28:0/3183886177 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9180d1cf0 con 0x557e3b8f8440 2026-02-25T11:57:47.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.295+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/3183886177 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe91c011e60 con 0x557e3b8f8440 2026-02-25T11:57:47.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.295+0000 7fe933d64e00 10 client.17460 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:47.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.295+0000 7fe933d64e00 1 -- 10.20.193.28:0/3183886177 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x557e3b799340 con 0x557e3b8f8440 2026-02-25T11:57:47.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.295+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/3183886177 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe91c02c780 con 0x557e3b8f8440 2026-02-25T11:57:47.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.295+0000 7fe933d64e00 10 client.17460 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:47.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.295+0000 7fe933d64e00 10 client.17460 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:57:47.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.295+0000 7fe933d64e00 20 client.17460 populate_metadata read hostname 'trial028' 2026-02-25T11:57:47.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.295+0000 7fe933d64e00 1 --2- 10.20.193.28:0/3183886177 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x557e3b9fb1b0 0x557e3ba1b630 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.295+0000 7fe933d64e00 4 client.17460 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:57:47.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.295+0000 7fe933d64e00 1 -- 10.20.193.28:0/3183886177 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e3b496f40 con 0x557e3b9fb1b0 2026-02-25T11:57:47.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.295+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/3183886177 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x557e3b9fb1b0 0x557e3ba1b630 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe92b7fe640 1 --2- 10.20.193.28:0/3183886177 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x557e3b9fb1b0 0x557e3ba1b630 crc :-1 s=READY pgs=60 gs=4 cs=0 l=0 c_cookie=18223926519886873493 s_cookie=10353562788724862957 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fce8649d86ad8395 server_cookie=8faf3e8035a5efed in_seq=0 out_seq=0 2026-02-25T11:57:47.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe8ed7fa640 10 client.17460 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:57:47.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe8ed7fa640 1 -- 10.20.193.28:0/3183886177 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e3b496f40 con 0x557e3b9fb1b0 2026-02-25T11:57:47.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe8ed7fa640 10 client.17460 handle_command_reply: tid=0 2026-02-25T11:57:47.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe933d64e00 1 client.17460 shutdown 2026-02-25T11:57:47.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe933d64e00 1 -- 10.20.193.28:0/3183886177 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f8440 msgr2=0x557e3b90f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe933d64e00 1 --2- 10.20.193.28:0/3183886177 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f8440 0x557e3b90f870 secure :-1 s=READY pgs=865 gs=1 cs=0 l=1 c_cookie=17407123998830656176 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe91c004740 tx=0x7fe91c0047e0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe933d64e00 1 -- 10.20.193.28:0/3183886177 shutdown_connections 2026-02-25T11:57:47.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe933d64e00 1 --2- 10.20.193.28:0/3183886177 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x557e3b9fb1b0 0x557e3ba1b630 crc :-1 s=READY pgs=60 gs=4 cs=0 l=0 c_cookie=18223926519886873493 s_cookie=10353562788724862957 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe933d64e00 1 --2- 10.20.193.28:0/3183886177 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x557e3b93d0e0 0x557e3b91a300 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe933d64e00 1 --2- 10.20.193.28:0/3183886177 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x557e3b8f8440 0x557e3b90f870 unknown :-1 s=CLOSED pgs=865 gs=1 cs=0 l=1 c_cookie=17407123998830656176 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe933d64e00 1 --2- 10.20.193.28:0/3183886177 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x557e3b8f7b30 0x557e3b90f330 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe8ed7fa640 0 client.17460 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:57:47.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe933d64e00 1 -- 10.20.193.28:0/3183886177 >> 10.20.193.28:0/3183886177 conn(0x557e3b8167a0 msgr2=0x557e3b913790 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:47.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe933d64e00 1 -- 10.20.193.28:0/3183886177 shutdown_connections 2026-02-25T11:57:47.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe933d64e00 1 -- 10.20.193.28:0/3183886177 wait complete. 2026-02-25T11:57:47.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.296+0000 7fe933d64e00 20 client.17460 trim_cache size 0 max 16384 2026-02-25T11:57:47.305 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:57:47.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.297+0000 7fe931d69640 1 -- 10.20.193.28:0/2444726796 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe9000b5760 msgr2=0x7fe9000d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.297+0000 7fe931d69640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe9000b5760 0x7fe9000d5c10 secure :-1 s=READY pgs=381 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe92c168e40 tx=0x7fe91c00b410 comp rx=0 tx=0).stop 2026-02-25T11:57:47.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.297+0000 7fe931d69640 1 -- 10.20.193.28:0/2444726796 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe92c061090 msgr2=0x7fe92c1624a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.297+0000 7fe931d69640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe92c061090 0x7fe92c1624a0 secure :-1 s=READY pgs=228 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fe9180098a0 tx=0x7fe9180028d0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.297+0000 7fe931d69640 1 -- 10.20.193.28:0/2444726796 shutdown_connections 2026-02-25T11:57:47.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.297+0000 7fe931d69640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fe9000b5760 0x7fe9000d5c10 unknown :-1 s=CLOSED pgs=381 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.297+0000 7fe931d69640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fe92c155bd0 0x7fe92c167db0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.297+0000 7fe931d69640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fe92c0619a0 0x7fe92c1629e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.297+0000 7fe931d69640 1 --2- 10.20.193.28:0/2444726796 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fe92c061090 0x7fe92c1624a0 unknown :-1 s=CLOSED pgs=228 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.297+0000 7fe931d69640 1 -- 10.20.193.28:0/2444726796 >> 10.20.193.28:0/2444726796 conn(0x7fe92c0eadc0 msgr2=0x7fe92c10b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:47.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.297+0000 7fe931d69640 1 -- 10.20.193.28:0/2444726796 shutdown_connections 2026-02-25T11:57:47.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.297+0000 7fe931d69640 1 -- 10.20.193.28:0/2444726796 wait complete. 2026-02-25T11:57:47.394 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:47 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/509608218' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:57:47.397 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:57:47.581 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:47.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:47 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/509608218' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:57:47.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:47 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/509608218' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:57:47.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.776+0000 7f2818c14640 1 -- 10.20.193.28:0/2052867279 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f281410ddc0 msgr2=0x7f2814112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.776+0000 7f2818c14640 1 --2- 10.20.193.28:0/2052867279 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f281410ddc0 0x7f2814112e10 secure :-1 s=READY pgs=866 gs=1 cs=0 l=1 c_cookie=4228504705792095075 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2804009950 tx=0x7f280402f190 comp rx=0 tx=0).stop 2026-02-25T11:57:47.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.776+0000 7f2818c14640 1 -- 10.20.193.28:0/2052867279 shutdown_connections 2026-02-25T11:57:47.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.776+0000 7f2818c14640 1 --2- 10.20.193.28:0/2052867279 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f28141135c0 0x7f2814173f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.776+0000 7f2818c14640 1 --2- 10.20.193.28:0/2052867279 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f281410ddc0 0x7f2814112e10 unknown :-1 s=CLOSED pgs=866 gs=1 cs=0 l=1 c_cookie=4228504705792095075 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.776+0000 7f2818c14640 1 --2- 10.20.193.28:0/2052867279 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f281410d3d0 0x7f281410d7a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.776+0000 7f2818c14640 1 -- 10.20.193.28:0/2052867279 >> 10.20.193.28:0/2052867279 conn(0x7f28140eaca0 msgr2=0x7f281410b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:47.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.776+0000 7f2818c14640 1 -- 10.20.193.28:0/2052867279 shutdown_connections 2026-02-25T11:57:47.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.776+0000 7f2818c14640 1 -- 10.20.193.28:0/2052867279 wait complete. 2026-02-25T11:57:47.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.777+0000 7f2818c14640 1 Processor -- start 2026-02-25T11:57:47.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.777+0000 7f2818c14640 1 -- start start 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.777+0000 7f2818c14640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f281410d3d0 0x7f28140f5a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.777+0000 7f2818c14640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f281410ddc0 0x7f2814173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.777+0000 7f2818c14640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f28141135c0 0x7f28140f6110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.777+0000 7f2818c14640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f28140fb320 con 0x7f281410ddc0 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.777+0000 7f2818c14640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f28140fb490 con 0x7f281410d3d0 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.777+0000 7f2818c14640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f28140fb650 con 0x7f28141135c0 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.777+0000 7f2812ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f281410ddc0 0x7f2814173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.777+0000 7f2812ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f281410ddc0 0x7f2814173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44466/0 (socket says 10.20.193.28:44466) 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.777+0000 7f2812ffd640 1 -- 10.20.193.28:0/3136996062 learned_addr learned my addr 10.20.193.28:0/3136996062 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.777+0000 7f28137fe640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f281410d3d0 0x7f28140f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.778+0000 7f2813fff640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f28141135c0 0x7f28140f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.778+0000 7f2812ffd640 1 -- 10.20.193.28:0/3136996062 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f28141135c0 msgr2=0x7f28140f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.778+0000 7f2812ffd640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f28141135c0 0x7f28140f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.778+0000 7f2812ffd640 1 -- 10.20.193.28:0/3136996062 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f281410d3d0 msgr2=0x7f28140f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.778+0000 7f2812ffd640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f281410d3d0 0x7f28140f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.778+0000 7f2812ffd640 1 -- 10.20.193.28:0/3136996062 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280800a590 con 0x7f281410ddc0 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.778+0000 7f2813fff640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f28141135c0 0x7f28140f6110 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:57:47.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.778+0000 7f2812ffd640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f281410ddc0 0x7f2814173e90 secure :-1 s=READY pgs=867 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f28040098a0 tx=0x7f28040028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:47.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.778+0000 7f2810ff9640 1 -- 10.20.193.28:0/3136996062 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f280403d070 con 0x7f281410ddc0 2026-02-25T11:57:47.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.778+0000 7f2810ff9640 1 -- 10.20.193.28:0/3136996062 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f2804002a40 con 0x7f281410ddc0 2026-02-25T11:57:47.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.778+0000 7f2810ff9640 1 -- 10.20.193.28:0/3136996062 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2804041720 con 0x7f281410ddc0 2026-02-25T11:57:47.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.778+0000 7f2818c14640 1 -- 10.20.193.28:0/3136996062 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2804009570 con 0x7f281410ddc0 2026-02-25T11:57:47.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.778+0000 7f2818c14640 1 -- 10.20.193.28:0/3136996062 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28140fbb60 con 0x7f281410ddc0 2026-02-25T11:57:47.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.781+0000 7f2810ff9640 1 -- 10.20.193.28:0/3136996062 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f28040386a0 con 0x7f281410ddc0 2026-02-25T11:57:47.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.782+0000 7f2810ff9640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f27e80b5750 0x7f27e80d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.783+0000 7f2810ff9640 1 -- 10.20.193.28:0/3136996062 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f280410af10 con 0x7f281410ddc0 2026-02-25T11:57:47.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.783+0000 7f28137fe640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f27e80b5750 0x7f27e80d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.783 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.783+0000 7f28137fe640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f27e80b5750 0x7f27e80d5c00 secure :-1 s=READY pgs=382 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2800004740 tx=0x7f2800005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:47.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.784+0000 7f281be98e00 1 Processor -- start 2026-02-25T11:57:47.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.784+0000 7f281be98e00 1 -- start start 2026-02-25T11:57:47.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.784+0000 7f281be98e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c0876c7b30 0x55c0876c7f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.784+0000 7f281be98e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c8440 0x55c08770cba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.784+0000 7f281be98e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c08770d0e0 0x55c08772d4c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.784+0000 7f281be98e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55c0875dd8a0 con 0x55c0876c8440 2026-02-25T11:57:47.786 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.784+0000 7f281be98e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55c0875d95d0 con 0x55c0876c7b30 2026-02-25T11:57:47.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.784+0000 7f281be98e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55c0875d3570 con 0x55c08770d0e0 2026-02-25T11:57:47.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.784+0000 7f28137fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c0876c7b30 0x55c0876c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.784+0000 7f28137fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c0876c7b30 0x55c0876c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:49074/0 (socket says 10.20.193.28:49074) 2026-02-25T11:57:47.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.784+0000 7f28137fe640 1 -- 10.20.193.28:0/2827222308 learned_addr learned my addr 10.20.193.28:0/2827222308 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:47.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.784+0000 7f2813fff640 1 --2- 10.20.193.28:0/2827222308 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c08770d0e0 0x55c08772d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.785+0000 7f2812ffd640 1 --2- 10.20.193.28:0/2827222308 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c8440 0x55c08770cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.785+0000 7f2812ffd640 1 -- 10.20.193.28:0/2827222308 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c08770d0e0 msgr2=0x55c08772d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.787 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.785+0000 7f2812ffd640 1 --2- 10.20.193.28:0/2827222308 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c08770d0e0 0x55c08772d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.785+0000 7f2812ffd640 1 -- 10.20.193.28:0/2827222308 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c0876c7b30 msgr2=0x55c0876c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.785+0000 7f2812ffd640 1 --2- 10.20.193.28:0/2827222308 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c0876c7b30 0x55c0876c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.785+0000 7f2812ffd640 1 -- 10.20.193.28:0/2827222308 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c08757ed10 con 0x55c0876c8440 2026-02-25T11:57:47.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.785+0000 7f28137fe640 1 --2- 10.20.193.28:0/2827222308 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c0876c7b30 0x55c0876c7f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:57:47.788 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.785+0000 7f2812ffd640 1 --2- 10.20.193.28:0/2827222308 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c8440 0x55c08770cba0 secure :-1 s=READY pgs=868 gs=3 cs=0 l=1 c_cookie=14786345232395419472 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2804004030 tx=0x7f2804038db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd33a6a5bf383f50 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:47.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.785+0000 7f27d7fff640 1 -- 10.20.193.28:0/2827222308 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2804108ea0 con 0x55c0876c8440 2026-02-25T11:57:47.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.785+0000 7f27d7fff640 1 -- 10.20.193.28:0/2827222308 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f28040d0a90 con 0x55c0876c8440 2026-02-25T11:57:47.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.785+0000 7f27d7fff640 1 -- 10.20.193.28:0/2827222308 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f28040d10b0 con 0x55c0876c8440 2026-02-25T11:57:47.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.786+0000 7f281be98e00 1 -- 10.20.193.28:0/2827222308 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c8440 msgr2=0x55c08770cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.786+0000 7f281be98e00 1 --2- 10.20.193.28:0/2827222308 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c8440 0x55c08770cba0 secure :-1 s=READY pgs=868 gs=3 cs=0 l=1 c_cookie=14786345232395419472 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2804004030 tx=0x7f2804038db0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.786+0000 7f281be98e00 1 -- 10.20.193.28:0/2827222308 shutdown_connections 2026-02-25T11:57:47.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.786+0000 7f281be98e00 1 --2- 10.20.193.28:0/2827222308 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c08770d0e0 0x55c08772d4c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.786+0000 7f281be98e00 1 --2- 10.20.193.28:0/2827222308 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c8440 0x55c08770cba0 unknown :-1 s=CLOSED pgs=868 gs=3 cs=0 l=1 c_cookie=14786345232395419472 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.786+0000 7f281be98e00 1 --2- 10.20.193.28:0/2827222308 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c0876c7b30 0x55c0876c7f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.786+0000 7f281be98e00 1 -- 10.20.193.28:0/2827222308 >> 10.20.193.28:0/2827222308 conn(0x55c0875e67a0 msgr2=0x55c0876c6410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:47.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.786+0000 7f281be98e00 1 -- 10.20.193.28:0/2827222308 shutdown_connections 2026-02-25T11:57:47.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.786+0000 7f281be98e00 1 -- 10.20.193.28:0/2827222308 wait complete. 2026-02-25T11:57:47.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.787+0000 7f281be98e00 1 -- start start 2026-02-25T11:57:47.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.787+0000 7f281be98e00 1 Processor -- start 2026-02-25T11:57:47.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.787+0000 7f281be98e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c7b30 0x55c0876dfd00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.787+0000 7f281be98e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c0876c8440 0x55c0876e0240 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.787+0000 7f281be98e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c08770d0e0 0x55c0876e5230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.787+0000 7f281be98e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55c0876e5770 con 0x55c08770d0e0 2026-02-25T11:57:47.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.787+0000 7f281be98e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55c0876e58e0 con 0x55c0876c8440 2026-02-25T11:57:47.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.787+0000 7f281be98e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55c0876e5aa0 con 0x55c0876c7b30 2026-02-25T11:57:47.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.788+0000 7f2813fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c08770d0e0 0x55c0876e5230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.788+0000 7f2813fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c08770d0e0 0x55c0876e5230 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44492/0 (socket says 10.20.193.28:44492) 2026-02-25T11:57:47.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.788+0000 7f2813fff640 1 -- 10.20.193.28:0/3573602231 learned_addr learned my addr 10.20.193.28:0/3573602231 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:47.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.788+0000 7f2812ffd640 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c0876c8440 0x55c0876e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.788+0000 7f28137fe640 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c7b30 0x55c0876dfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.788+0000 7f2812ffd640 1 -- 10.20.193.28:0/3573602231 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c7b30 msgr2=0x55c0876dfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.788+0000 7f2812ffd640 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c7b30 0x55c0876dfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.788+0000 7f2812ffd640 1 -- 10.20.193.28:0/3573602231 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c08770d0e0 msgr2=0x55c0876e5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.788+0000 7f2812ffd640 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c08770d0e0 0x55c0876e5230 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.788+0000 7f2812ffd640 1 -- 10.20.193.28:0/3573602231 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28140fbb60 con 0x55c0876c8440 2026-02-25T11:57:47.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.788+0000 7f2813fff640 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c08770d0e0 0x55c0876e5230 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:57:47.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.788+0000 7f2812ffd640 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c0876c8440 0x55c0876e0240 secure :-1 s=READY pgs=176 gs=3 cs=0 l=1 c_cookie=5975641261367689107 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2804004030 tx=0x7f28040d1060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52edbe1120fcff93 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:47.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.788+0000 7f27d57fa640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:57:47.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f27d57fa640 1 -- 10.20.193.28:0/3573602231 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2804108ea0 con 0x55c0876c8440 2026-02-25T11:57:47.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f27d57fa640 1 -- 10.20.193.28:0/3573602231 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f28040d9590 con 0x55c0876c8440 2026-02-25T11:57:47.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f281be98e00 1 -- 10.20.193.28:0/3573602231 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2800009960 con 0x55c0876c8440 2026-02-25T11:57:47.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f27d57fa640 1 -- 10.20.193.28:0/3573602231 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f28040d21a0 con 0x55c0876c8440 2026-02-25T11:57:47.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f281be98e00 10 client.27140 _ll_register_callbacks cb 0x55c087554a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:47.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f281be98e00 1 -- 10.20.193.28:0/3573602231 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c0872696f0 con 0x55c0876c8440 2026-02-25T11:57:47.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f27d57fa640 1 -- 10.20.193.28:0/3573602231 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f2804092020 con 0x55c0876c8440 2026-02-25T11:57:47.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f27d57fa640 1 -- 10.20.193.28:0/3573602231 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2804106e00 con 0x55c0876c8440 2026-02-25T11:57:47.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f281be98e00 10 client.27140 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:47.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f281be98e00 1 -- 10.20.193.28:0/3573602231 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55c0875573c0 con 0x55c0876c8440 2026-02-25T11:57:47.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f27d57fa640 1 -- 10.20.193.28:0/3573602231 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f28040963c0 con 0x55c0876c8440 2026-02-25T11:57:47.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f281be98e00 10 client.27140 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:47.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f281be98e00 10 client.27140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:57:47.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f281be98e00 20 client.27140 populate_metadata read hostname 'trial028' 2026-02-25T11:57:47.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f281be98e00 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c08770a060 0x55c08770a500 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f281be98e00 4 client.27140 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:57:47.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.789+0000 7f281be98e00 1 -- 10.20.193.28:0/3573602231 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0875a5fc0 con 0x55c08770a060 2026-02-25T11:57:47.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.790+0000 7f28137fe640 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c08770a060 0x55c08770a500 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.790+0000 7f28137fe640 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c08770a060 0x55c08770a500 crc :-1 s=READY pgs=60 gs=4 cs=0 l=0 c_cookie=7489698264700018463 s_cookie=675011731300300083 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67f0c0e3be4ddf1f server_cookie=95e1f97df603533 in_seq=0 out_seq=0 2026-02-25T11:57:47.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.790+0000 7f27d57fa640 10 client.27140 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:57:47.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f27d57fa640 1 -- 10.20.193.28:0/3573602231 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55c0875a5fc0 con 0x55c08770a060 2026-02-25T11:57:47.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f27d57fa640 10 client.27140 handle_command_reply: tid=0 2026-02-25T11:57:47.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f281be98e00 1 client.27140 shutdown 2026-02-25T11:57:47.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f281be98e00 1 -- 10.20.193.28:0/3573602231 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c0876c8440 msgr2=0x55c0876e0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f281be98e00 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c0876c8440 0x55c0876e0240 secure :-1 s=READY pgs=176 gs=3 cs=0 l=1 c_cookie=5975641261367689107 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2804004030 tx=0x7f28040d1060 comp rx=0 tx=0).stop 2026-02-25T11:57:47.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f281be98e00 1 -- 10.20.193.28:0/3573602231 shutdown_connections 2026-02-25T11:57:47.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f281be98e00 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c08770a060 0x55c08770a500 crc :-1 s=READY pgs=60 gs=4 cs=0 l=0 c_cookie=7489698264700018463 s_cookie=675011731300300083 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f281be98e00 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c08770d0e0 0x55c0876e5230 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f27d57fa640 0 client.27140 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:57:47.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f281be98e00 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c0876c8440 0x55c0876e0240 unknown :-1 s=CLOSED pgs=176 gs=3 cs=0 l=1 c_cookie=5975641261367689107 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f281be98e00 1 --2- 10.20.193.28:0/3573602231 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c7b30 0x55c0876dfd00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f281be98e00 1 -- 10.20.193.28:0/3573602231 >> 10.20.193.28:0/3573602231 conn(0x55c0875e67a0 msgr2=0x55c0876c6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:47.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f281be98e00 1 -- 10.20.193.28:0/3573602231 shutdown_connections 2026-02-25T11:57:47.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.791+0000 7f281be98e00 1 -- 10.20.193.28:0/3573602231 wait complete. 2026-02-25T11:57:47.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.792+0000 7f281be98e00 20 client.27140 trim_cache size 0 max 16384 2026-02-25T11:57:47.805 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.798+0000 7f281be98e00 1 Processor -- start 2026-02-25T11:57:47.805 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.798+0000 7f281be98e00 1 -- start start 2026-02-25T11:57:47.805 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f281be98e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c7b30 0x55c0876d1e50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.805 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f281be98e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c8440 0x55c0876c6c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.805 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f281be98e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c08770a060 0x55c0876c7150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.805 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f281be98e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55c0875d11a0 con 0x55c0876c8440 2026-02-25T11:57:47.805 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f281be98e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55c087291700 con 0x55c08770a060 2026-02-25T11:57:47.805 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f281be98e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55c0875d95d0 con 0x55c0876c7b30 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f2812ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c8440 0x55c0876c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f2812ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c8440 0x55c0876c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44502/0 (socket says 10.20.193.28:44502) 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f2812ffd640 1 -- 10.20.193.28:0/4063387438 learned_addr learned my addr 10.20.193.28:0/4063387438 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f2813fff640 1 --2- 10.20.193.28:0/4063387438 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c08770a060 0x55c0876c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f28137fe640 1 --2- 10.20.193.28:0/4063387438 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c7b30 0x55c0876d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f2812ffd640 1 -- 10.20.193.28:0/4063387438 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c7b30 msgr2=0x55c0876d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f2812ffd640 1 --2- 10.20.193.28:0/4063387438 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c7b30 0x55c0876d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f2812ffd640 1 -- 10.20.193.28:0/4063387438 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c08770a060 msgr2=0x55c0876c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f2812ffd640 1 --2- 10.20.193.28:0/4063387438 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c08770a060 0x55c0876c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f2812ffd640 1 -- 10.20.193.28:0/4063387438 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0876d2620 con 0x55c0876c8440 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.799+0000 7f2812ffd640 1 --2- 10.20.193.28:0/4063387438 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c8440 0x55c0876c6c10 secure :-1 s=READY pgs=869 gs=1 cs=0 l=1 c_cookie=15578829932918002780 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f28040ce500 tx=0x7f28040eb000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8331f48a7ccd45c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.800+0000 7f27d77fe640 1 -- 10.20.193.28:0/4063387438 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2804108ea0 con 0x55c0876c8440 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.800+0000 7f27d77fe640 1 -- 10.20.193.28:0/4063387438 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55c0872696f0 con 0x55c0876c8440 2026-02-25T11:57:47.806 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.800+0000 7f27d77fe640 1 -- 10.20.193.28:0/4063387438 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f28040dc1e0 con 0x55c0876c8440 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.800+0000 7f281be98e00 1 -- 10.20.193.28:0/4063387438 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c8440 msgr2=0x55c0876c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.800+0000 7f281be98e00 1 --2- 10.20.193.28:0/4063387438 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c8440 0x55c0876c6c10 secure :-1 s=READY pgs=869 gs=1 cs=0 l=1 c_cookie=15578829932918002780 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f28040ce500 tx=0x7f28040eb000 comp rx=0 tx=0).stop 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.800+0000 7f281be98e00 1 -- 10.20.193.28:0/4063387438 shutdown_connections 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.800+0000 7f281be98e00 1 --2- 10.20.193.28:0/4063387438 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c08770a060 0x55c0876c7150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.800+0000 7f281be98e00 1 --2- 10.20.193.28:0/4063387438 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c8440 0x55c0876c6c10 unknown :-1 s=CLOSED pgs=869 gs=1 cs=0 l=1 c_cookie=15578829932918002780 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.800+0000 7f281be98e00 1 --2- 10.20.193.28:0/4063387438 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c7b30 0x55c0876d1e50 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.800+0000 7f281be98e00 1 -- 10.20.193.28:0/4063387438 >> 10.20.193.28:0/4063387438 conn(0x55c0875e67a0 msgr2=0x55c0876c6500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.800+0000 7f281be98e00 1 -- 10.20.193.28:0/4063387438 shutdown_connections 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.800+0000 7f281be98e00 1 -- 10.20.193.28:0/4063387438 wait complete. 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f281be98e00 1 -- start start 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f281be98e00 1 Processor -- start 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f281be98e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c7b30 0x55c087751790 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f281be98e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c8440 0x55c087751cd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f281be98e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c08770a060 0x55c0876eb490 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.807 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f281be98e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55c0876a2e80 con 0x55c0876c7b30 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f281be98e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55c0875dd8a0 con 0x55c08770a060 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f281be98e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55c0876eba80 con 0x55c0876c8440 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f28137fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c7b30 0x55c087751790 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f28137fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c7b30 0x55c087751790 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44510/0 (socket says 10.20.193.28:44510) 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f28137fe640 1 -- 10.20.193.28:0/2059782076 learned_addr learned my addr 10.20.193.28:0/2059782076 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f2812ffd640 1 --2- 10.20.193.28:0/2059782076 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c8440 0x55c087751cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f2813fff640 1 --2- 10.20.193.28:0/2059782076 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c08770a060 0x55c0876eb490 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f28137fe640 1 -- 10.20.193.28:0/2059782076 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c8440 msgr2=0x55c087751cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f28137fe640 1 --2- 10.20.193.28:0/2059782076 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c8440 0x55c087751cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f28137fe640 1 -- 10.20.193.28:0/2059782076 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c08770a060 msgr2=0x55c0876eb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f28137fe640 1 --2- 10.20.193.28:0/2059782076 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c08770a060 0x55c0876eb490 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.801+0000 7f28137fe640 1 -- 10.20.193.28:0/2059782076 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0876cdaf0 con 0x55c0876c7b30 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.802+0000 7f28137fe640 1 --2- 10.20.193.28:0/2059782076 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c7b30 0x55c087751790 secure :-1 s=READY pgs=870 gs=3 cs=0 l=1 c_cookie=13583079463956322749 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2800012770 tx=0x7f2800014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc80cace650519bd server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.802+0000 7f27d57fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:47.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.802+0000 7f27d57fa640 1 -- 10.20.193.28:0/2059782076 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2800029040 con 0x55c0876c7b30 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.802+0000 7f27d57fa640 1 -- 10.20.193.28:0/2059782076 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f280001ae20 con 0x55c0876c7b30 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.802+0000 7f281be98e00 1 -- 10.20.193.28:0/2059782076 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0875573c0 con 0x55c0876c7b30 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.802+0000 7f281be98e00 10 client.17478 _ll_register_callbacks cb 0x55c087554a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.802+0000 7f281be98e00 1 -- 10.20.193.28:0/2059782076 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28040d0a90 con 0x55c0876c7b30 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.802+0000 7f27d57fa640 1 -- 10.20.193.28:0/2059782076 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2800030690 con 0x55c0876c7b30 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.802+0000 7f27d57fa640 1 -- 10.20.193.28:0/2059782076 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f2800034e00 con 0x55c0876c7b30 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.802+0000 7f27d57fa640 1 -- 10.20.193.28:0/2059782076 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f2800030820 con 0x55c0876c7b30 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.802+0000 7f281be98e00 10 client.17478 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.802+0000 7f281be98e00 1 -- 10.20.193.28:0/2059782076 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55c0875690e0 con 0x55c0876c7b30 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.803+0000 7f27d57fa640 1 -- 10.20.193.28:0/2059782076 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f280001a960 con 0x55c0876c7b30 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.803+0000 7f281be98e00 10 client.17478 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.803+0000 7f281be98e00 10 client.17478 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.803+0000 7f281be98e00 20 client.17478 populate_metadata read hostname 'trial028' 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.803+0000 7f281be98e00 1 --2- 10.20.193.28:0/2059782076 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c08770d0e0 0x55c0877d9e60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.803+0000 7f281be98e00 4 client.17478 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.803+0000 7f281be98e00 1 -- 10.20.193.28:0/2059782076 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c087266f40 con 0x55c08770d0e0 2026-02-25T11:57:47.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.803+0000 7f2812ffd640 1 --2- 10.20.193.28:0/2059782076 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c08770d0e0 0x55c0877d9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.803+0000 7f2812ffd640 1 --2- 10.20.193.28:0/2059782076 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c08770d0e0 0x55c0877d9e60 crc :-1 s=READY pgs=61 gs=4 cs=0 l=0 c_cookie=1611728755646476187 s_cookie=8870621402645551603 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=165e02c392972f9b server_cookie=7b1ac72ac9aa59f3 in_seq=0 out_seq=0 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.803+0000 7f27d57fa640 10 client.17478 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f27d57fa640 1 -- 10.20.193.28:0/2059782076 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c087266f40 con 0x55c08770d0e0 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f27d57fa640 10 client.17478 handle_command_reply: tid=0 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f281be98e00 1 client.17478 shutdown 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f281be98e00 1 -- 10.20.193.28:0/2059782076 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c7b30 msgr2=0x55c087751790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f281be98e00 1 --2- 10.20.193.28:0/2059782076 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c7b30 0x55c087751790 secure :-1 s=READY pgs=870 gs=3 cs=0 l=1 c_cookie=13583079463956322749 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2800012770 tx=0x7f2800014000 comp rx=0 tx=0).stop 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f281be98e00 1 -- 10.20.193.28:0/2059782076 shutdown_connections 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f281be98e00 1 --2- 10.20.193.28:0/2059782076 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55c08770d0e0 0x55c0877d9e60 crc :-1 s=READY pgs=61 gs=4 cs=0 l=0 c_cookie=1611728755646476187 s_cookie=8870621402645551603 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f281be98e00 1 --2- 10.20.193.28:0/2059782076 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55c08770a060 0x55c0876eb490 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f281be98e00 1 --2- 10.20.193.28:0/2059782076 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55c0876c8440 0x55c087751cd0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f281be98e00 1 --2- 10.20.193.28:0/2059782076 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55c0876c7b30 0x55c087751790 unknown :-1 s=CLOSED pgs=870 gs=3 cs=0 l=1 c_cookie=13583079463956322749 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f281be98e00 1 -- 10.20.193.28:0/2059782076 >> 10.20.193.28:0/2059782076 conn(0x55c0875e67a0 msgr2=0x55c0876c6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f27d57fa640 0 client.17478 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f281be98e00 1 -- 10.20.193.28:0/2059782076 shutdown_connections 2026-02-25T11:57:47.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f281be98e00 1 -- 10.20.193.28:0/2059782076 wait complete. 2026-02-25T11:57:47.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f281be98e00 20 client.17478 trim_cache size 0 max 16384 2026-02-25T11:57:47.811 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:57:47.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f2818c14640 1 -- 10.20.193.28:0/3136996062 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f27e80b5750 msgr2=0x7f27e80d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f2818c14640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f27e80b5750 0x7f27e80d5c00 secure :-1 s=READY pgs=382 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2800004740 tx=0x7f2800005f90 comp rx=0 tx=0).stop 2026-02-25T11:57:47.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f2818c14640 1 -- 10.20.193.28:0/3136996062 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f281410ddc0 msgr2=0x7f2814173e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:47.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f2818c14640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f281410ddc0 0x7f2814173e90 secure :-1 s=READY pgs=867 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f28040098a0 tx=0x7f28040028d0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f2818c14640 1 -- 10.20.193.28:0/3136996062 shutdown_connections 2026-02-25T11:57:47.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f2818c14640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f27e80b5750 0x7f27e80d5c00 unknown :-1 s=CLOSED pgs=382 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f2818c14640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f28141135c0 0x7f28140f6110 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f2818c14640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f281410ddc0 0x7f2814173e90 unknown :-1 s=CLOSED pgs=867 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f2818c14640 1 --2- 10.20.193.28:0/3136996062 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f281410d3d0 0x7f28140f5a10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:47.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.804+0000 7f2818c14640 1 -- 10.20.193.28:0/3136996062 >> 10.20.193.28:0/3136996062 conn(0x7f28140eaca0 msgr2=0x7f2814110c20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:47.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.805+0000 7f2818c14640 1 -- 10.20.193.28:0/3136996062 shutdown_connections 2026-02-25T11:57:47.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:47.805+0000 7f2818c14640 1 -- 10.20.193.28:0/3136996062 wait complete. 2026-02-25T11:57:47.906 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:57:48.091 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:48.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.273+0000 7fdda3577640 1 -- 10.20.193.28:0/1968290223 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdda4110080 msgr2=0x7fdda41150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:48.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.273+0000 7fdda3577640 1 --2- 10.20.193.28:0/1968290223 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdda4110080 0x7fdda41150d0 secure :-1 s=READY pgs=177 gs=2 cs=0 l=1 c_cookie=17448534005253341240 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd8c009950 tx=0x7fdd8c02f190 comp rx=0 tx=0).stop 2026-02-25T11:57:48.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.274+0000 7fdda3577640 1 -- 10.20.193.28:0/1968290223 shutdown_connections 2026-02-25T11:57:48.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.274+0000 7fdda3577640 1 --2- 10.20.193.28:0/1968290223 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdda4115880 0x7fdda4176250 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.274+0000 7fdda3577640 1 --2- 10.20.193.28:0/1968290223 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdda4110080 0x7fdda41150d0 unknown :-1 s=CLOSED pgs=177 gs=2 cs=0 l=1 c_cookie=17448534005253341240 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.274+0000 7fdda3577640 1 --2- 10.20.193.28:0/1968290223 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdda410f690 0x7fdda410fa60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.274+0000 7fdda3577640 1 -- 10.20.193.28:0/1968290223 >> 10.20.193.28:0/1968290223 conn(0x7fdda40ecf00 msgr2=0x7fdda410d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:48.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.274+0000 7fdda3577640 1 -- 10.20.193.28:0/1968290223 shutdown_connections 2026-02-25T11:57:48.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.274+0000 7fdda3577640 1 -- 10.20.193.28:0/1968290223 wait complete. 2026-02-25T11:57:48.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.274+0000 7fdda3577640 1 Processor -- start 2026-02-25T11:57:48.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda3577640 1 -- start start 2026-02-25T11:57:48.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda3577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdda410f690 0x7fdda411e0e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda3577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdda4110080 0x7fdda411e620 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda3577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdda4115880 0x7fdda41239f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda3577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4124010 con 0x7fdda4110080 2026-02-25T11:57:48.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda3577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4124180 con 0x7fdda4115880 2026-02-25T11:57:48.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda3577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdda4124340 con 0x7fdda410f690 2026-02-25T11:57:48.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda1d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdda4110080 0x7fdda411e620 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda2d76640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdda4115880 0x7fdda41239f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda2d76640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdda4115880 0x7fdda41239f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:49124/0 (socket says 10.20.193.28:49124) 2026-02-25T11:57:48.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda2575640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdda410f690 0x7fdda411e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.278 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda2d76640 1 -- 10.20.193.28:0/2792543962 learned_addr learned my addr 10.20.193.28:0/2792543962 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:48.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda2d76640 1 -- 10.20.193.28:0/2792543962 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdda410f690 msgr2=0x7fdda411e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:48.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda2d76640 1 --2- 10.20.193.28:0/2792543962 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdda410f690 0x7fdda411e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda2d76640 1 -- 10.20.193.28:0/2792543962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdda4110080 msgr2=0x7fdda411e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:48.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda2d76640 1 --2- 10.20.193.28:0/2792543962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdda4110080 0x7fdda411e620 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.275+0000 7fdda2d76640 1 -- 10.20.193.28:0/2792543962 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c009570 con 0x7fdda4115880 2026-02-25T11:57:48.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.276+0000 7fdda2d76640 1 --2- 10.20.193.28:0/2792543962 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdda4115880 0x7fdda41239f0 secure :-1 s=READY pgs=178 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd9400f8d0 tx=0x7fdd9400fda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:48.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.276+0000 7fdd8b7fe640 1 -- 10.20.193.28:0/2792543962 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd9400dce0 con 0x7fdda4115880 2026-02-25T11:57:48.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.276+0000 7fdd8b7fe640 1 -- 10.20.193.28:0/2792543962 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdd9400de40 con 0x7fdda4115880 2026-02-25T11:57:48.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.276+0000 7fdd8b7fe640 1 -- 10.20.193.28:0/2792543962 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd940116b0 con 0x7fdda4115880 2026-02-25T11:57:48.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.276+0000 7fdda3577640 1 -- 10.20.193.28:0/2792543962 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdda4124620 con 0x7fdda4115880 2026-02-25T11:57:48.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.276+0000 7fdda3577640 1 -- 10.20.193.28:0/2792543962 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda4063320 con 0x7fdda4115880 2026-02-25T11:57:48.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.279+0000 7fdd8b7fe640 1 -- 10.20.193.28:0/2792543962 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fdd94020060 con 0x7fdda4115880 2026-02-25T11:57:48.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.281+0000 7fdd8b7fe640 1 --2- 10.20.193.28:0/2792543962 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdd7c0b5760 0x7fdd7c0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.281+0000 7fdd8b7fe640 1 -- 10.20.193.28:0/2792543962 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fdd94015070 con 0x7fdda4115880 2026-02-25T11:57:48.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fdda2575640 1 --2- 10.20.193.28:0/2792543962 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdd7c0b5760 0x7fdd7c0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fddaaf7ae00 1 Processor -- start 2026-02-25T11:57:48.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fddaaf7ae00 1 -- start start 2026-02-25T11:57:48.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fddaaf7ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f73f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fddaaf7ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f74440 0x5587f0fb8ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fddaaf7ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 0x5587f0fd94c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fddaaf7ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5587f0e898a0 con 0x5587f0f74440 2026-02-25T11:57:48.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fddaaf7ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5587f0e855d0 con 0x5587f0f73b30 2026-02-25T11:57:48.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fddaaf7ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5587f0e7f570 con 0x5587f0fb90e0 2026-02-25T11:57:48.285 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fdda2575640 1 --2- 10.20.193.28:0/2792543962 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdd7c0b5760 0x7fdd7c0d5c10 secure :-1 s=READY pgs=383 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd98004740 tx=0x7fdd98005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:48.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fdda2575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f73f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fdda2d76640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 0x5587f0fd94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fdda2575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f73f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:49132/0 (socket says 10.20.193.28:49132) 2026-02-25T11:57:48.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.282+0000 7fdda2575640 1 -- 10.20.193.28:0/2137272963 learned_addr learned my addr 10.20.193.28:0/2137272963 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:48.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.283+0000 7fdda1d74640 1 --2- 10.20.193.28:0/2137272963 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f74440 0x5587f0fb8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.283+0000 7fdda2d76640 1 -- 10.20.193.28:0/2137272963 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 msgr2=0x5587f0f73f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:48.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.283+0000 7fdda2d76640 1 --2- 10.20.193.28:0/2137272963 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f73f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.283+0000 7fdda2d76640 1 -- 10.20.193.28:0/2137272963 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f74440 msgr2=0x5587f0fb8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:48.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.283+0000 7fdda2d76640 1 --2- 10.20.193.28:0/2137272963 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f74440 0x5587f0fb8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.283+0000 7fdda2d76640 1 -- 10.20.193.28:0/2137272963 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587f0e2ad10 con 0x5587f0fb90e0 2026-02-25T11:57:48.286 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.283+0000 7fdda2575640 1 --2- 10.20.193.28:0/2137272963 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f73f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:57:48.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.283+0000 7fdda2d76640 1 --2- 10.20.193.28:0/2137272963 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 0x5587f0fd94c0 secure :-1 s=READY pgs=229 gs=2 cs=0 l=1 c_cookie=10631599088271691025 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd940aa420 tx=0x7fdd940adfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=938b070acacd0d11 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:48.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.283+0000 7fdd88ff9640 1 -- 10.20.193.28:0/2137272963 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd940068a0 con 0x5587f0fb90e0 2026-02-25T11:57:48.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.283+0000 7fdd88ff9640 1 -- 10.20.193.28:0/2137272963 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdd940ae850 con 0x5587f0fb90e0 2026-02-25T11:57:48.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.283+0000 7fdd88ff9640 1 -- 10.20.193.28:0/2137272963 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd940adc70 con 0x5587f0fb90e0 2026-02-25T11:57:48.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.284+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2137272963 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 msgr2=0x5587f0fd94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:48.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.284+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/2137272963 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 0x5587f0fd94c0 secure :-1 s=READY pgs=229 gs=2 cs=0 l=1 c_cookie=10631599088271691025 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd940aa420 tx=0x7fdd940adfd0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.284+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2137272963 shutdown_connections 2026-02-25T11:57:48.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.284+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/2137272963 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 0x5587f0fd94c0 unknown :-1 s=CLOSED pgs=229 gs=2 cs=0 l=1 c_cookie=10631599088271691025 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.284+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/2137272963 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f74440 0x5587f0fb8ba0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.284+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/2137272963 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f73f00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.284+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2137272963 >> 10.20.193.28:0/2137272963 conn(0x5587f0e927a0 msgr2=0x5587f0f72410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:48.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.284+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2137272963 shutdown_connections 2026-02-25T11:57:48.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.284+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2137272963 wait complete. 2026-02-25T11:57:48.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.285+0000 7fddaaf7ae00 1 -- start start 2026-02-25T11:57:48.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.286+0000 7fddaaf7ae00 1 Processor -- start 2026-02-25T11:57:48.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.286+0000 7fddaaf7ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f8bf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.286+0000 7fddaaf7ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0f74440 0x5587f0f8c440 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.286+0000 7fddaaf7ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0fb90e0 0x5587f0f91540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.286+0000 7fddaaf7ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5587f0f91a80 con 0x5587f0fb90e0 2026-02-25T11:57:48.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.286+0000 7fddaaf7ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5587f0f91bf0 con 0x5587f0f73b30 2026-02-25T11:57:48.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.286+0000 7fddaaf7ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5587f0f91db0 con 0x5587f0f74440 2026-02-25T11:57:48.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.286+0000 7fdda2d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0fb90e0 0x5587f0f91540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.286+0000 7fdda2d76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0fb90e0 0x5587f0f91540 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44546/0 (socket says 10.20.193.28:44546) 2026-02-25T11:57:48.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.286+0000 7fdda2d76640 1 -- 10.20.193.28:0/2679788622 learned_addr learned my addr 10.20.193.28:0/2679788622 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:48.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.286+0000 7fdda1d74640 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0f74440 0x5587f0f8c440 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.286+0000 7fdda2575640 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f8bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.292 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fdda2d76640 1 -- 10.20.193.28:0/2679788622 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0f74440 msgr2=0x5587f0f8c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:48.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fdda2d76640 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0f74440 0x5587f0f8c440 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fdda2d76640 1 -- 10.20.193.28:0/2679788622 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 msgr2=0x5587f0f8bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:48.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fdda2d76640 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f8bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fdda2d76640 1 -- 10.20.193.28:0/2679788622 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdda4063320 con 0x5587f0fb90e0 2026-02-25T11:57:48.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fdda2575640 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f8bf00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:57:48.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fdda2d76640 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0fb90e0 0x5587f0f91540 secure :-1 s=READY pgs=871 gs=1 cs=0 l=1 c_cookie=5279675137628448006 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd940a9730 tx=0x7fdd940aa2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49452c9093ce2506 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:48.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fdd53fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:48.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fdd53fff640 1 -- 10.20.193.28:0/2679788622 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd940b8d20 con 0x5587f0fb90e0 2026-02-25T11:57:48.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2679788622 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd98009dc0 con 0x5587f0fb90e0 2026-02-25T11:57:48.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fddaaf7ae00 10 client.17490 _ll_register_callbacks cb 0x5587f0e00a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:48.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2679788622 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5587f0b156f0 con 0x5587f0fb90e0 2026-02-25T11:57:48.293 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fdd53fff640 1 -- 10.20.193.28:0/2679788622 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdd940b3b70 con 0x5587f0fb90e0 2026-02-25T11:57:48.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fdd53fff640 1 -- 10.20.193.28:0/2679788622 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd940abe30 con 0x5587f0fb90e0 2026-02-25T11:57:48.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fdd53fff640 1 -- 10.20.193.28:0/2679788622 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fdd940abf90 con 0x5587f0fb90e0 2026-02-25T11:57:48.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fdd53fff640 1 -- 10.20.193.28:0/2679788622 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdd940af840 con 0x5587f0fb90e0 2026-02-25T11:57:48.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fddaaf7ae00 10 client.17490 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:48.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.287+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2679788622 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5587f0e033c0 con 0x5587f0fb90e0 2026-02-25T11:57:48.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.288+0000 7fdd53fff640 1 -- 10.20.193.28:0/2679788622 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fdd940cd070 con 0x5587f0fb90e0 2026-02-25T11:57:48.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.288+0000 7fddaaf7ae00 10 client.17490 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:48.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.288+0000 7fddaaf7ae00 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:48.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.288+0000 7fddaaf7ae00 20 client.17490 populate_metadata read hostname 'trial028' 2026-02-25T11:57:48.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.288+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5587f0fb6570 0x5587f0fb6a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.288+0000 7fddaaf7ae00 4 client.17490 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:57:48.294 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.288+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2679788622 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587f0e51fc0 con 0x5587f0fb6570 2026-02-25T11:57:48.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.288+0000 7fdda2575640 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5587f0fb6570 0x5587f0fb6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.289+0000 7fdda2575640 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5587f0fb6570 0x5587f0fb6a10 crc :-1 s=READY pgs=170 gs=4 cs=0 l=0 c_cookie=962904235434312862 s_cookie=11125923932521280957 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5cec4ce9c8049e server_cookie=9a6738de73f2b5bd in_seq=0 out_seq=0 2026-02-25T11:57:48.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.289+0000 7fdd53fff640 10 client.17490 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:48.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.289+0000 7fdd53fff640 1 -- 10.20.193.28:0/2679788622 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5587f0e51fc0 con 0x5587f0fb6570 2026-02-25T11:57:48.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.289+0000 7fdd53fff640 10 client.17490 handle_command_reply: tid=0 2026-02-25T11:57:48.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.289+0000 7fddaaf7ae00 1 client.17490 shutdown 2026-02-25T11:57:48.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.290+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2679788622 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0fb90e0 msgr2=0x5587f0f91540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:48.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.290+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0fb90e0 0x5587f0f91540 secure :-1 s=READY pgs=871 gs=1 cs=0 l=1 c_cookie=5279675137628448006 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd940a9730 tx=0x7fdd940aa2f0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.290+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2679788622 shutdown_connections 2026-02-25T11:57:48.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.290+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5587f0fb6570 0x5587f0fb6a10 crc :-1 s=READY pgs=170 gs=4 cs=0 l=0 c_cookie=962904235434312862 s_cookie=11125923932521280957 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.295 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.290+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0fb90e0 0x5587f0f91540 unknown :-1 s=CLOSED pgs=871 gs=1 cs=0 l=1 c_cookie=5279675137628448006 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.290+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0f74440 0x5587f0f8c440 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.290+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/2679788622 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f8bf00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.290+0000 7fdd53fff640 0 client.17490 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:48.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.290+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2679788622 >> 10.20.193.28:0/2679788622 conn(0x5587f0e927a0 msgr2=0x5587f0f72b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:48.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.290+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2679788622 shutdown_connections 2026-02-25T11:57:48.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.290+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/2679788622 wait complete. 2026-02-25T11:57:48.296 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.290+0000 7fddaaf7ae00 20 client.17490 trim_cache size 0 max 16384 2026-02-25T11:57:48.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.298+0000 7fddaaf7ae00 1 Processor -- start 2026-02-25T11:57:48.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.298+0000 7fddaaf7ae00 1 -- start start 2026-02-25T11:57:48.299 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.298+0000 7fddaaf7ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f89340 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.298+0000 7fddaaf7ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f74440 0x5587f0f89880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.298+0000 7fddaaf7ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 0x5587f0fb8300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.298+0000 7fddaaf7ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5587f0e7d1a0 con 0x5587f0f74440 2026-02-25T11:57:48.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.298+0000 7fddaaf7ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5587f0b3d700 con 0x5587f0f73b30 2026-02-25T11:57:48.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.298+0000 7fddaaf7ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5587f0f91ef0 con 0x5587f0fb90e0 2026-02-25T11:57:48.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.298+0000 7fdda1d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f74440 0x5587f0f89880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.298+0000 7fdda2575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f89340 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.298+0000 7fdda2575640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f89340 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:49152/0 (socket says 10.20.193.28:49152) 2026-02-25T11:57:48.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.298+0000 7fdda2575640 1 -- 10.20.193.28:0/4024552167 learned_addr learned my addr 10.20.193.28:0/4024552167 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:48.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.299+0000 7fdda2d76640 1 --2- 10.20.193.28:0/4024552167 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 0x5587f0fb8300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.299+0000 7fdda2575640 1 -- 10.20.193.28:0/4024552167 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 msgr2=0x5587f0fb8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:48.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.299+0000 7fdda2575640 1 --2- 10.20.193.28:0/4024552167 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 0x5587f0fb8300 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.299+0000 7fdda2575640 1 -- 10.20.193.28:0/4024552167 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f74440 msgr2=0x5587f0f89880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:48.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.299+0000 7fdda2575640 1 --2- 10.20.193.28:0/4024552167 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f74440 0x5587f0f89880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.299+0000 7fdda2575640 1 -- 10.20.193.28:0/4024552167 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587f0e150e0 con 0x5587f0f73b30 2026-02-25T11:57:48.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.299+0000 7fdda1d74640 1 --2- 10.20.193.28:0/4024552167 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f74440 0x5587f0f89880 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:57:48.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.299+0000 7fdda2575640 1 --2- 10.20.193.28:0/4024552167 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f89340 secure :-1 s=READY pgs=179 gs=2 cs=0 l=1 c_cookie=6381783655290438223 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x5587f0fb72c0 tx=0x7fdd98014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5890a66c1efb024f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:48.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.299+0000 7fdd6e4e6640 1 -- 10.20.193.28:0/4024552167 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd98021040 con 0x5587f0f73b30 2026-02-25T11:57:48.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.299+0000 7fdd6e4e6640 1 -- 10.20.193.28:0/4024552167 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdd980045d0 con 0x5587f0f73b30 2026-02-25T11:57:48.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.299+0000 7fdd6e4e6640 1 -- 10.20.193.28:0/4024552167 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd9800e720 con 0x5587f0f73b30 2026-02-25T11:57:48.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.300+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/4024552167 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 msgr2=0x5587f0f89340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:48.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.300+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/4024552167 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f89340 secure :-1 s=READY pgs=179 gs=2 cs=0 l=1 c_cookie=6381783655290438223 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x5587f0fb72c0 tx=0x7fdd98014000 comp rx=0 tx=0).stop 2026-02-25T11:57:48.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.300+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/4024552167 shutdown_connections 2026-02-25T11:57:48.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.300+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/4024552167 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 0x5587f0fb8300 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.300+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/4024552167 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f74440 0x5587f0f89880 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.300+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/4024552167 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f73b30 0x5587f0f89340 unknown :-1 s=CLOSED pgs=179 gs=2 cs=0 l=1 c_cookie=6381783655290438223 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.300+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/4024552167 >> 10.20.193.28:0/4024552167 conn(0x5587f0e927a0 msgr2=0x5587f0f8a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:48.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.300+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/4024552167 shutdown_connections 2026-02-25T11:57:48.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.300+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/4024552167 wait complete. 2026-02-25T11:57:48.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.300+0000 7fddaaf7ae00 1 -- start start 2026-02-25T11:57:48.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fddaaf7ae00 1 Processor -- start 2026-02-25T11:57:48.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fddaaf7ae00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f73b30 0x5587f0f98050 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fddaaf7ae00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f74440 0x5587f0f96b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fddaaf7ae00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 0x5587f0f97080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fddaaf7ae00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5587f0f97690 con 0x5587f0f73b30 2026-02-25T11:57:48.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fddaaf7ae00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5587f0f97800 con 0x5587f0f74440 2026-02-25T11:57:48.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fddaaf7ae00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5587f0f979c0 con 0x5587f0fb90e0 2026-02-25T11:57:48.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fdda2575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f73b30 0x5587f0f98050 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fdda2575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f73b30 0x5587f0f98050 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44568/0 (socket says 10.20.193.28:44568) 2026-02-25T11:57:48.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fdda2575640 1 -- 10.20.193.28:0/3323267286 learned_addr learned my addr 10.20.193.28:0/3323267286 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:48.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fdda1d74640 1 --2- 10.20.193.28:0/3323267286 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f74440 0x5587f0f96b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fdda2d76640 1 --2- 10.20.193.28:0/3323267286 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 0x5587f0f97080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fdda2575640 1 -- 10.20.193.28:0/3323267286 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 msgr2=0x5587f0f97080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:48.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fdda2575640 1 --2- 10.20.193.28:0/3323267286 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 0x5587f0f97080 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fdda2575640 1 -- 10.20.193.28:0/3323267286 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f74440 msgr2=0x5587f0f96b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:48.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fdda2575640 1 --2- 10.20.193.28:0/3323267286 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f74440 0x5587f0f96b40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.301+0000 7fdda2575640 1 -- 10.20.193.28:0/3323267286 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9801b3f0 con 0x5587f0f73b30 2026-02-25T11:57:48.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.302+0000 7fdda2575640 1 --2- 10.20.193.28:0/3323267286 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f73b30 0x5587f0f98050 secure :-1 s=READY pgs=872 gs=1 cs=0 l=1 c_cookie=8167046639703351421 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd98003f60 tx=0x7fdd98003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71572da5df90e07d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:48.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.302+0000 7fdd53fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:48.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.302+0000 7fdd53fff640 1 -- 10.20.193.28:0/3323267286 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd98004330 con 0x5587f0f73b30 2026-02-25T11:57:48.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.302+0000 7fdd53fff640 1 -- 10.20.193.28:0/3323267286 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fdd9800e6b0 con 0x5587f0f73b30 2026-02-25T11:57:48.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.302+0000 7fdd53fff640 1 -- 10.20.193.28:0/3323267286 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fdd980297e0 con 0x5587f0f73b30 2026-02-25T11:57:48.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.302+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/3323267286 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdda4176ac0 con 0x5587f0f73b30 2026-02-25T11:57:48.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.302+0000 7fdd53fff640 1 -- 10.20.193.28:0/3323267286 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fdd98021040 con 0x5587f0f73b30 2026-02-25T11:57:48.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.302+0000 7fddaaf7ae00 10 client.17496 _ll_register_callbacks cb 0x5587f0e00a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:48.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.302+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/3323267286 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd940ae850 con 0x5587f0f73b30 2026-02-25T11:57:48.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.302+0000 7fdd53fff640 1 -- 10.20.193.28:0/3323267286 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fdd9800e8b0 con 0x5587f0f73b30 2026-02-25T11:57:48.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.302+0000 7fddaaf7ae00 10 client.17496 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:48.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.302+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/3323267286 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5587f0e0fb60 con 0x5587f0f73b30 2026-02-25T11:57:48.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.303+0000 7fdd53fff640 1 -- 10.20.193.28:0/3323267286 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fdd9803b050 con 0x5587f0f73b30 2026-02-25T11:57:48.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.303+0000 7fddaaf7ae00 10 client.17496 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:48.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.303+0000 7fddaaf7ae00 10 client.17496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:48.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.303+0000 7fddaaf7ae00 20 client.17496 populate_metadata read hostname 'trial028' 2026-02-25T11:57:48.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.303+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/3323267286 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5587f1066370 0x5587f1086820 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:48.309 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.303+0000 7fddaaf7ae00 4 client.17496 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:57:48.309 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.303+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/3323267286 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587f0b12f40 con 0x5587f1066370 2026-02-25T11:57:48.309 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.303+0000 7fdda1d74640 1 --2- 10.20.193.28:0/3323267286 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5587f1066370 0x5587f1086820 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:48.310 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fdda1d74640 1 --2- 10.20.193.28:0/3323267286 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5587f1066370 0x5587f1086820 crc :-1 s=READY pgs=171 gs=4 cs=0 l=0 c_cookie=6744599823841155021 s_cookie=12653214823643511834 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d99a1ce2c36d7cd server_cookie=af993fd7adfa7c1a in_seq=0 out_seq=0 2026-02-25T11:57:48.310 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fdd53fff640 10 client.17496 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:48.310 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fdd53fff640 1 -- 10.20.193.28:0/3323267286 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587f0b12f40 con 0x5587f1066370 2026-02-25T11:57:48.310 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fdd53fff640 10 client.17496 handle_command_reply: tid=0 2026-02-25T11:57:48.310 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fddaaf7ae00 1 client.17496 shutdown 2026-02-25T11:57:48.310 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/3323267286 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f73b30 msgr2=0x5587f0f98050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:48.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/3323267286 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f73b30 0x5587f0f98050 secure :-1 s=READY pgs=872 gs=1 cs=0 l=1 c_cookie=8167046639703351421 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd98003f60 tx=0x7fdd98003f90 comp rx=0 tx=0).stop 2026-02-25T11:57:48.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/3323267286 shutdown_connections 2026-02-25T11:57:48.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/3323267286 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5587f1066370 0x5587f1086820 crc :-1 s=READY pgs=171 gs=4 cs=0 l=0 c_cookie=6744599823841155021 s_cookie=12653214823643511834 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/3323267286 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5587f0fb90e0 0x5587f0f97080 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/3323267286 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5587f0f74440 0x5587f0f96b40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fddaaf7ae00 1 --2- 10.20.193.28:0/3323267286 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5587f0f73b30 0x5587f0f98050 unknown :-1 s=CLOSED pgs=872 gs=1 cs=0 l=1 c_cookie=8167046639703351421 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/3323267286 >> 10.20.193.28:0/3323267286 conn(0x5587f0e927a0 msgr2=0x5587f0f88330 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:48.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fdd53fff640 0 client.17496 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:48.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/3323267286 shutdown_connections 2026-02-25T11:57:48.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fddaaf7ae00 1 -- 10.20.193.28:0/3323267286 wait complete. 2026-02-25T11:57:48.311 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.304+0000 7fddaaf7ae00 20 client.17496 trim_cache size 0 max 16384 2026-02-25T11:57:48.312 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:57:48.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.305+0000 7fdda3577640 1 -- 10.20.193.28:0/2792543962 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdd7c0b5760 msgr2=0x7fdd7c0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:48.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.305+0000 7fdda3577640 1 --2- 10.20.193.28:0/2792543962 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdd7c0b5760 0x7fdd7c0d5c10 secure :-1 s=READY pgs=383 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd98004740 tx=0x7fdd98005f90 comp rx=0 tx=0).stop 2026-02-25T11:57:48.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.305+0000 7fdda3577640 1 -- 10.20.193.28:0/2792543962 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdda4115880 msgr2=0x7fdda41239f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:48.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.305+0000 7fdda3577640 1 --2- 10.20.193.28:0/2792543962 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdda4115880 0x7fdda41239f0 secure :-1 s=READY pgs=178 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fdd9400f8d0 tx=0x7fdd9400fda0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.305+0000 7fdda3577640 1 -- 10.20.193.28:0/2792543962 shutdown_connections 2026-02-25T11:57:48.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.305+0000 7fdda3577640 1 --2- 10.20.193.28:0/2792543962 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fdd7c0b5760 0x7fdd7c0d5c10 unknown :-1 s=CLOSED pgs=383 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.305+0000 7fdda3577640 1 --2- 10.20.193.28:0/2792543962 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fdda4115880 0x7fdda41239f0 unknown :-1 s=CLOSED pgs=178 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.312 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.305+0000 7fdda3577640 1 --2- 10.20.193.28:0/2792543962 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fdda4110080 0x7fdda411e620 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.313 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.305+0000 7fdda3577640 1 --2- 10.20.193.28:0/2792543962 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fdda410f690 0x7fdda411e0e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:48.313 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.305+0000 7fdda3577640 1 -- 10.20.193.28:0/2792543962 >> 10.20.193.28:0/2792543962 conn(0x7fdda40ecf00 msgr2=0x7fdda410d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:48.313 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.305+0000 7fdda3577640 1 -- 10.20.193.28:0/2792543962 shutdown_connections 2026-02-25T11:57:48.313 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:48.305+0000 7fdda3577640 1 -- 10.20.193.28:0/2792543962 wait complete. 2026-02-25T11:57:48.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:48 trial028 ceph-mon[39224]: pgmap v513: 97 pgs: 97 active+clean; 7.3 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:48.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:48 trial167 ceph-mon[43392]: pgmap v513: 97 pgs: 97 active+clean; 7.3 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:48.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:48 trial119 ceph-mon[42710]: pgmap v513: 97 pgs: 97 active+clean; 7.3 MiB data, 350 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:49.405 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:57:49.406 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:57:49.587 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:49.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.764+0000 7f277ec59640 1 -- 10.20.193.28:0/3414135163 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2778110080 msgr2=0x7f27781150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:49.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.764+0000 7f277ec59640 1 --2- 10.20.193.28:0/3414135163 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2778110080 0x7f27781150d0 secure :-1 s=READY pgs=873 gs=1 cs=0 l=1 c_cookie=2847425008621036876 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f276c009950 tx=0x7f276c02f190 comp rx=0 tx=0).stop 2026-02-25T11:57:49.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.764+0000 7f277ec59640 1 -- 10.20.193.28:0/3414135163 shutdown_connections 2026-02-25T11:57:49.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.764+0000 7f277ec59640 1 --2- 10.20.193.28:0/3414135163 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2778115880 0x7f2778176250 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.764+0000 7f277ec59640 1 --2- 10.20.193.28:0/3414135163 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2778110080 0x7f27781150d0 unknown :-1 s=CLOSED pgs=873 gs=1 cs=0 l=1 c_cookie=2847425008621036876 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.764+0000 7f277ec59640 1 --2- 10.20.193.28:0/3414135163 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f277810f690 0x7f277810fa60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.764+0000 7f277ec59640 1 -- 10.20.193.28:0/3414135163 >> 10.20.193.28:0/3414135163 conn(0x7f27780ecf00 msgr2=0x7f277810d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:49.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.764+0000 7f277ec59640 1 -- 10.20.193.28:0/3414135163 shutdown_connections 2026-02-25T11:57:49.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.764+0000 7f277ec59640 1 -- 10.20.193.28:0/3414135163 wait complete. 2026-02-25T11:57:49.765 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.765+0000 7f277ec59640 1 Processor -- start 2026-02-25T11:57:49.766 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.765+0000 7f277ec59640 1 -- start start 2026-02-25T11:57:49.767 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.765+0000 7f277ec59640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f277810f690 0x7f277811e080 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.765+0000 7f277ec59640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2778110080 0x7f277811e5c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.765+0000 7f277ec59640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2778115880 0x7f2778123990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.765+0000 7f277ec59640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2778123fb0 con 0x7f2778110080 2026-02-25T11:57:49.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.765+0000 7f277ec59640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2778124120 con 0x7f2778115880 2026-02-25T11:57:49.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.765+0000 7f277ec59640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f27781242e0 con 0x7f277810f690 2026-02-25T11:57:49.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f277d456640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2778110080 0x7f277811e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f277d456640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2778110080 0x7f277811e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44578/0 (socket says 10.20.193.28:44578) 2026-02-25T11:57:49.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f277d456640 1 -- 10.20.193.28:0/177846979 learned_addr learned my addr 10.20.193.28:0/177846979 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:49.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f277e458640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2778115880 0x7f2778123990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f277dc57640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f277810f690 0x7f277811e080 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.768 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f277d456640 1 -- 10.20.193.28:0/177846979 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f277810f690 msgr2=0x7f277811e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:49.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f277d456640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f277810f690 0x7f277811e080 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f277d456640 1 -- 10.20.193.28:0/177846979 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2778115880 msgr2=0x7f2778123990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:49.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f277d456640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2778115880 0x7f2778123990 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f277d456640 1 -- 10.20.193.28:0/177846979 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276c009570 con 0x7f2778110080 2026-02-25T11:57:49.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f277d456640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2778110080 0x7f277811e5c0 secure :-1 s=READY pgs=874 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f276c0098a0 tx=0x7f276c0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:49.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f2766ffd640 1 -- 10.20.193.28:0/177846979 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f276c03d070 con 0x7f2778110080 2026-02-25T11:57:49.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f277ec59640 1 -- 10.20.193.28:0/177846979 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2778124560 con 0x7f2778110080 2026-02-25T11:57:49.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f2766ffd640 1 -- 10.20.193.28:0/177846979 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f276c002a40 con 0x7f2778110080 2026-02-25T11:57:49.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f2766ffd640 1 -- 10.20.193.28:0/177846979 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f276c041820 con 0x7f2778110080 2026-02-25T11:57:49.769 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.766+0000 7f277ec59640 1 -- 10.20.193.28:0/177846979 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2778164cc0 con 0x7f2778110080 2026-02-25T11:57:49.770 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.769+0000 7f2766ffd640 1 -- 10.20.193.28:0/177846979 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f276c0419c0 con 0x7f2778110080 2026-02-25T11:57:49.771 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.771+0000 7f2766ffd640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f27540b5760 0x7f27540d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.771+0000 7f277dc57640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f27540b5760 0x7f27540d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.771+0000 7f2766ffd640 1 -- 10.20.193.28:0/177846979 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f276c10b1b0 con 0x7f2778110080 2026-02-25T11:57:49.772 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.772+0000 7f277dc57640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f27540b5760 0x7f27540d5c10 secure :-1 s=READY pgs=384 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2768005970 tx=0x7f2768005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:49.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.773+0000 7f2781edde00 1 Processor -- start 2026-02-25T11:57:49.773 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.773+0000 7f2781edde00 1 -- start start 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.773+0000 7f2781edde00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9585750 0x5611b9585b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.773+0000 7f2781edde00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9586060 0x5611b95ca610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.773+0000 7f2781edde00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b95cab50 0x5611b95eaf30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.773+0000 7f2781edde00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5611b949bd80 con 0x5611b9586060 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.773+0000 7f2781edde00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5611b9497ab0 con 0x5611b9585750 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.773+0000 7f2781edde00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5611b9491a50 con 0x5611b95cab50 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.773+0000 7f277d456640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9586060 0x5611b95ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.773+0000 7f277d456640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9586060 0x5611b95ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44582/0 (socket says 10.20.193.28:44582) 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.773+0000 7f277d456640 1 -- 10.20.193.28:0/873080428 learned_addr learned my addr 10.20.193.28:0/873080428 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.773+0000 7f277dc57640 1 --2- 10.20.193.28:0/873080428 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9585750 0x5611b9585b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.774+0000 7f277d456640 1 -- 10.20.193.28:0/873080428 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b95cab50 msgr2=0x5611b95eaf30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.774+0000 7f277d456640 1 --2- 10.20.193.28:0/873080428 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b95cab50 0x5611b95eaf30 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.774+0000 7f277d456640 1 -- 10.20.193.28:0/873080428 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9585750 msgr2=0x5611b9585b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.774+0000 7f277d456640 1 --2- 10.20.193.28:0/873080428 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9585750 0x5611b9585b20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.774+0000 7f277d456640 1 -- 10.20.193.28:0/873080428 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611b943d120 con 0x5611b9586060 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.774+0000 7f277d456640 1 --2- 10.20.193.28:0/873080428 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9586060 0x5611b95ca610 secure :-1 s=READY pgs=875 gs=1 cs=0 l=1 c_cookie=15739327633481169877 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f276c009480 tx=0x7f276c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6d531795227bd5 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.774+0000 7f27464e6640 1 -- 10.20.193.28:0/873080428 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f276c0d0d30 con 0x5611b9586060 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.774+0000 7f27464e6640 1 -- 10.20.193.28:0/873080428 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f276c10a070 con 0x5611b9586060 2026-02-25T11:57:49.774 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.774+0000 7f27464e6640 1 -- 10.20.193.28:0/873080428 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f276c0d21c0 con 0x5611b9586060 2026-02-25T11:57:49.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.774+0000 7f2781edde00 1 -- 10.20.193.28:0/873080428 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9586060 msgr2=0x5611b95ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:49.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.774+0000 7f2781edde00 1 --2- 10.20.193.28:0/873080428 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9586060 0x5611b95ca610 secure :-1 s=READY pgs=875 gs=1 cs=0 l=1 c_cookie=15739327633481169877 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f276c009480 tx=0x7f276c009310 comp rx=0 tx=0).stop 2026-02-25T11:57:49.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.775+0000 7f2781edde00 1 -- 10.20.193.28:0/873080428 shutdown_connections 2026-02-25T11:57:49.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.775+0000 7f2781edde00 1 --2- 10.20.193.28:0/873080428 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b95cab50 0x5611b95eaf30 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.775+0000 7f2781edde00 1 --2- 10.20.193.28:0/873080428 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9586060 0x5611b95ca610 unknown :-1 s=CLOSED pgs=875 gs=1 cs=0 l=1 c_cookie=15739327633481169877 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.775+0000 7f2781edde00 1 --2- 10.20.193.28:0/873080428 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9585750 0x5611b9585b20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.775+0000 7f2781edde00 1 -- 10.20.193.28:0/873080428 >> 10.20.193.28:0/873080428 conn(0x5611b95638f0 msgr2=0x5611b9583cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:49.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.775+0000 7f2781edde00 1 -- 10.20.193.28:0/873080428 shutdown_connections 2026-02-25T11:57:49.775 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.775+0000 7f2781edde00 1 -- 10.20.193.28:0/873080428 wait complete. 2026-02-25T11:57:49.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.776+0000 7f2781edde00 1 -- start start 2026-02-25T11:57:49.776 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.776+0000 7f2781edde00 1 Processor -- start 2026-02-25T11:57:49.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.776+0000 7f2781edde00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9585750 0x5611b95e2470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.776+0000 7f2781edde00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9586060 0x5611b95e29b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.776+0000 7f2781edde00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b95cab50 0x5611b95e79a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.776+0000 7f2781edde00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5611b95e7ee0 con 0x5611b9585750 2026-02-25T11:57:49.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.776+0000 7f2781edde00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5611b95e8050 con 0x5611b9586060 2026-02-25T11:57:49.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.776+0000 7f2781edde00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5611b95e8210 con 0x5611b95cab50 2026-02-25T11:57:49.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.777+0000 7f277d456640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9586060 0x5611b95e29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.777+0000 7f277d456640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9586060 0x5611b95e29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:49210/0 (socket says 10.20.193.28:49210) 2026-02-25T11:57:49.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.777+0000 7f277d456640 1 -- 10.20.193.28:0/3318000294 learned_addr learned my addr 10.20.193.28:0/3318000294 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:49.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.777+0000 7f277e458640 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b95cab50 0x5611b95e79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.777+0000 7f277dc57640 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9585750 0x5611b95e2470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.777 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.777+0000 7f277d456640 1 -- 10.20.193.28:0/3318000294 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b95cab50 msgr2=0x5611b95e79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.777+0000 7f277d456640 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b95cab50 0x5611b95e79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.777+0000 7f277d456640 1 -- 10.20.193.28:0/3318000294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9585750 msgr2=0x5611b95e2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.777+0000 7f277d456640 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9585750 0x5611b95e2470 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.777+0000 7f277d456640 1 -- 10.20.193.28:0/3318000294 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611b9415810 con 0x5611b9586060 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.777+0000 7f277dc57640 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9585750 0x5611b95e2470 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.777+0000 7f277d456640 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9586060 0x5611b95e29b0 secure :-1 s=READY pgs=180 gs=2 cs=0 l=1 c_cookie=4516494596924167757 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f276c0d14c0 tx=0x7f276c0dc4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eadcfe80fa2d64d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.777+0000 7f27337fe640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f27337fe640 1 -- 10.20.193.28:0/3318000294 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f276c0d7250 con 0x5611b9586060 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f27337fe640 1 -- 10.20.193.28:0/3318000294 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f276c0dc670 con 0x5611b9586060 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f27337fe640 1 -- 10.20.193.28:0/3318000294 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f276c0cf510 con 0x5611b9586060 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f2781edde00 1 -- 10.20.193.28:0/3318000294 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2778164cc0 con 0x5611b9586060 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f27337fe640 1 -- 10.20.193.28:0/3318000294 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f276c0cf6a0 con 0x5611b9586060 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f2781edde00 10 client.27176 _ll_register_callbacks cb 0x5611b9412e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f2781edde00 1 -- 10.20.193.28:0/3318000294 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5611b91e9530 con 0x5611b9586060 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f27337fe640 1 -- 10.20.193.28:0/3318000294 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f276c048e30 con 0x5611b9586060 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f2781edde00 10 client.27176 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:49.778 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f2781edde00 1 -- 10.20.193.28:0/3318000294 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5611b94274f0 con 0x5611b9586060 2026-02-25T11:57:49.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f27337fe640 1 -- 10.20.193.28:0/3318000294 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f276c0d2f20 con 0x5611b9586060 2026-02-25T11:57:49.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f2781edde00 10 client.27176 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:49.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f2781edde00 10 client.27176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:49.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f2781edde00 20 client.27176 populate_metadata read hostname 'trial028' 2026-02-25T11:57:49.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f2781edde00 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5611b95c77a0 0x5611b95c7c40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f2781edde00 4 client.27176 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:57:49.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.778+0000 7f2781edde00 1 -- 10.20.193.28:0/3318000294 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611b943eb50 con 0x5611b95c77a0 2026-02-25T11:57:49.779 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.779+0000 7f277dc57640 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5611b95c77a0 0x5611b95c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.779+0000 7f277dc57640 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5611b95c77a0 0x5611b95c7c40 crc :-1 s=READY pgs=172 gs=4 cs=0 l=0 c_cookie=18274559216115591009 s_cookie=15927389406457996899 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd9c46c8c7692761 server_cookie=dd0974473b3fa663 in_seq=0 out_seq=0 2026-02-25T11:57:49.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.779+0000 7f27337fe640 10 client.27176 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:49.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.780+0000 7f27337fe640 1 -- 10.20.193.28:0/3318000294 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5611b943eb50 con 0x5611b95c77a0 2026-02-25T11:57:49.780 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.780+0000 7f27337fe640 10 client.27176 handle_command_reply: tid=0 2026-02-25T11:57:49.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.780+0000 7f2781edde00 1 client.27176 shutdown 2026-02-25T11:57:49.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.780+0000 7f2781edde00 1 -- 10.20.193.28:0/3318000294 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9586060 msgr2=0x5611b95e29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:49.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.780+0000 7f2781edde00 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9586060 0x5611b95e29b0 secure :-1 s=READY pgs=180 gs=2 cs=0 l=1 c_cookie=4516494596924167757 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f276c0d14c0 tx=0x7f276c0dc4e0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.780+0000 7f2781edde00 1 -- 10.20.193.28:0/3318000294 shutdown_connections 2026-02-25T11:57:49.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.780+0000 7f2781edde00 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5611b95c77a0 0x5611b95c7c40 crc :-1 s=READY pgs=172 gs=4 cs=0 l=0 c_cookie=18274559216115591009 s_cookie=15927389406457996899 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.780+0000 7f2781edde00 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b95cab50 0x5611b95e79a0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.780+0000 7f2781edde00 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9586060 0x5611b95e29b0 unknown :-1 s=CLOSED pgs=180 gs=2 cs=0 l=1 c_cookie=4516494596924167757 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.780+0000 7f2781edde00 1 --2- 10.20.193.28:0/3318000294 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9585750 0x5611b95e2470 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.780+0000 7f27337fe640 0 client.27176 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:49.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.780+0000 7f2781edde00 1 -- 10.20.193.28:0/3318000294 >> 10.20.193.28:0/3318000294 conn(0x5611b95638f0 msgr2=0x5611b9583cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:49.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.781+0000 7f2781edde00 1 -- 10.20.193.28:0/3318000294 shutdown_connections 2026-02-25T11:57:49.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.781+0000 7f2781edde00 1 -- 10.20.193.28:0/3318000294 wait complete. 2026-02-25T11:57:49.781 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.781+0000 7f2781edde00 20 client.27176 trim_cache size 0 max 16384 2026-02-25T11:57:49.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.790+0000 7f2781edde00 1 Processor -- start 2026-02-25T11:57:49.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.790+0000 7f2781edde00 1 -- start start 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.790+0000 7f2781edde00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9585750 0x5611b95dfe30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.790+0000 7f2781edde00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9586060 0x5611b95e18f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.790+0000 7f2781edde00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b95cab50 0x5611b95e1e30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.790+0000 7f2781edde00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5611b948f680 con 0x5611b9585750 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.790+0000 7f2781edde00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5611b914f790 con 0x5611b9586060 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.790+0000 7f2781edde00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5611b9497ab0 con 0x5611b95cab50 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.790+0000 7f277dc57640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9585750 0x5611b95dfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.790+0000 7f277dc57640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9585750 0x5611b95dfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44590/0 (socket says 10.20.193.28:44590) 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.790+0000 7f277dc57640 1 -- 10.20.193.28:0/4132336228 learned_addr learned my addr 10.20.193.28:0/4132336228 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f277d456640 1 --2- 10.20.193.28:0/4132336228 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9586060 0x5611b95e18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f277dc57640 1 -- 10.20.193.28:0/4132336228 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b95cab50 msgr2=0x5611b95e1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f277dc57640 1 --2- 10.20.193.28:0/4132336228 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b95cab50 0x5611b95e1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f277dc57640 1 -- 10.20.193.28:0/4132336228 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9586060 msgr2=0x5611b95e18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f277dc57640 1 --2- 10.20.193.28:0/4132336228 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9586060 0x5611b95e18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f277dc57640 1 -- 10.20.193.28:0/4132336228 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611b9421f70 con 0x5611b9585750 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f277d456640 1 --2- 10.20.193.28:0/4132336228 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9586060 0x5611b95e18f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f277dc57640 1 --2- 10.20.193.28:0/4132336228 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9585750 0x5611b95dfe30 secure :-1 s=READY pgs=876 gs=1 cs=0 l=1 c_cookie=17060179200816350445 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f276801f510 tx=0x7f276800b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc1ee982d2a84ed server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f2745ce5640 1 -- 10.20.193.28:0/4132336228 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2768025040 con 0x5611b9585750 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f2745ce5640 1 -- 10.20.193.28:0/4132336228 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f276801fdc0 con 0x5611b9585750 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f2745ce5640 1 -- 10.20.193.28:0/4132336228 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f2768029710 con 0x5611b9585750 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f2781edde00 1 -- 10.20.193.28:0/4132336228 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9585750 msgr2=0x5611b95dfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f2781edde00 1 --2- 10.20.193.28:0/4132336228 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9585750 0x5611b95dfe30 secure :-1 s=READY pgs=876 gs=1 cs=0 l=1 c_cookie=17060179200816350445 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f276801f510 tx=0x7f276800b000 comp rx=0 tx=0).stop 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f2781edde00 1 -- 10.20.193.28:0/4132336228 shutdown_connections 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f2781edde00 1 --2- 10.20.193.28:0/4132336228 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b95cab50 0x5611b95e1e30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f2781edde00 1 --2- 10.20.193.28:0/4132336228 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b9586060 0x5611b95e18f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f2781edde00 1 --2- 10.20.193.28:0/4132336228 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9585750 0x5611b95dfe30 unknown :-1 s=CLOSED pgs=876 gs=1 cs=0 l=1 c_cookie=17060179200816350445 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f2781edde00 1 -- 10.20.193.28:0/4132336228 >> 10.20.193.28:0/4132336228 conn(0x5611b95638f0 msgr2=0x5611b95de710 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.791+0000 7f2781edde00 1 -- 10.20.193.28:0/4132336228 shutdown_connections 2026-02-25T11:57:49.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.792+0000 7f2781edde00 1 -- 10.20.193.28:0/4132336228 wait complete. 2026-02-25T11:57:49.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.792+0000 7f2781edde00 1 -- start start 2026-02-25T11:57:49.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.792+0000 7f2781edde00 1 Processor -- start 2026-02-25T11:57:49.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f2781edde00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b9585750 0x5611b9569ac0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f2781edde00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9586060 0x5611b9565010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f2781edde00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b95cab50 0x5611b95657a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f2781edde00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5611b9560370 con 0x5611b9586060 2026-02-25T11:57:49.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f2781edde00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5611b95e7ab0 con 0x5611b95cab50 2026-02-25T11:57:49.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f2781edde00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5611b9639140 con 0x5611b9585750 2026-02-25T11:57:49.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f277e458640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b95cab50 0x5611b95657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f277d456640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9586060 0x5611b9565010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f277dc57640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b9585750 0x5611b9569ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f277e458640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b95cab50 0x5611b95657a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:49226/0 (socket says 10.20.193.28:49226) 2026-02-25T11:57:49.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f277e458640 1 -- 10.20.193.28:0/2048113146 learned_addr learned my addr 10.20.193.28:0/2048113146 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:49.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f277d456640 1 -- 10.20.193.28:0/2048113146 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b9585750 msgr2=0x5611b9569ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:49.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f277d456640 1 --2- 10.20.193.28:0/2048113146 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b9585750 0x5611b9569ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f277d456640 1 -- 10.20.193.28:0/2048113146 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b95cab50 msgr2=0x5611b95657a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:49.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f277d456640 1 --2- 10.20.193.28:0/2048113146 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b95cab50 0x5611b95657a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.793+0000 7f277d456640 1 -- 10.20.193.28:0/2048113146 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611b9415b40 con 0x5611b9586060 2026-02-25T11:57:49.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.794+0000 7f277d456640 1 --2- 10.20.193.28:0/2048113146 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9586060 0x5611b9565010 secure :-1 s=READY pgs=877 gs=2 cs=0 l=1 c_cookie=3605595690615703623 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f276c0d1cb0 tx=0x7f276c02ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3209a62d66d40847 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.794+0000 7f27337fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.794+0000 7f27337fe640 1 -- 10.20.193.28:0/2048113146 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f276c0d1ce0 con 0x5611b9586060 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.794+0000 7f27337fe640 1 -- 10.20.193.28:0/2048113146 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f276c0b1060 con 0x5611b9586060 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.794+0000 7f27337fe640 1 -- 10.20.193.28:0/2048113146 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f276c0966c0 con 0x5611b9586060 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.794+0000 7f2781edde00 1 -- 10.20.193.28:0/2048113146 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f276800fe40 con 0x5611b9586060 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.794+0000 7f2781edde00 10 client.17514 _ll_register_callbacks cb 0x5611b9412e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.794+0000 7f2781edde00 1 -- 10.20.193.28:0/2048113146 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f276c10a070 con 0x5611b9586060 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.794+0000 7f27337fe640 1 -- 10.20.193.28:0/2048113146 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f276c0d7250 con 0x5611b9586060 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.794+0000 7f27337fe640 1 -- 10.20.193.28:0/2048113146 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f276c048e30 con 0x5611b9586060 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.794+0000 7f2781edde00 10 client.17514 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.794+0000 7f2781edde00 1 -- 10.20.193.28:0/2048113146 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5611b9427750 con 0x5611b9586060 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.794+0000 7f27337fe640 1 -- 10.20.193.28:0/2048113146 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f276c0cf510 con 0x5611b9586060 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.795+0000 7f2781edde00 10 client.17514 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.795+0000 7f2781edde00 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.795+0000 7f2781edde00 20 client.17514 populate_metadata read hostname 'trial028' 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.795+0000 7f2781edde00 1 --2- 10.20.193.28:0/2048113146 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5611b96809a0 0x5611b96a0e50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.795+0000 7f2781edde00 4 client.17514 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:57:49.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.795+0000 7f2781edde00 1 -- 10.20.193.28:0/2048113146 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5611b92f2690 con 0x5611b96809a0 2026-02-25T11:57:49.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.795+0000 7f277dc57640 1 --2- 10.20.193.28:0/2048113146 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5611b96809a0 0x5611b96a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:49.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.796+0000 7f277dc57640 1 --2- 10.20.193.28:0/2048113146 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5611b96809a0 0x5611b96a0e50 crc :-1 s=READY pgs=173 gs=4 cs=0 l=0 c_cookie=11690399041605374396 s_cookie=8132023250235764476 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a23ca3fd6d7491bc server_cookie=70dac00f8f852efc in_seq=0 out_seq=0 2026-02-25T11:57:49.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.796+0000 7f27337fe640 10 client.17514 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:49.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.796+0000 7f27337fe640 1 -- 10.20.193.28:0/2048113146 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5611b92f2690 con 0x5611b96809a0 2026-02-25T11:57:49.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.796+0000 7f27337fe640 10 client.17514 handle_command_reply: tid=0 2026-02-25T11:57:49.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.796+0000 7f2781edde00 1 client.17514 shutdown 2026-02-25T11:57:49.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.797+0000 7f2781edde00 1 -- 10.20.193.28:0/2048113146 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9586060 msgr2=0x5611b9565010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:49.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.797+0000 7f2781edde00 1 --2- 10.20.193.28:0/2048113146 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9586060 0x5611b9565010 secure :-1 s=READY pgs=877 gs=2 cs=0 l=1 c_cookie=3605595690615703623 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f276c0d1cb0 tx=0x7f276c02ff40 comp rx=0 tx=0).stop 2026-02-25T11:57:49.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.797+0000 7f2781edde00 1 -- 10.20.193.28:0/2048113146 shutdown_connections 2026-02-25T11:57:49.797 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.797+0000 7f2781edde00 1 --2- 10.20.193.28:0/2048113146 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5611b96809a0 0x5611b96a0e50 crc :-1 s=READY pgs=173 gs=4 cs=0 l=0 c_cookie=11690399041605374396 s_cookie=8132023250235764476 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.797+0000 7f2781edde00 1 --2- 10.20.193.28:0/2048113146 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5611b95cab50 0x5611b95657a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.797+0000 7f2781edde00 1 --2- 10.20.193.28:0/2048113146 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5611b9586060 0x5611b9565010 unknown :-1 s=CLOSED pgs=877 gs=2 cs=0 l=1 c_cookie=3605595690615703623 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.797+0000 7f2781edde00 1 --2- 10.20.193.28:0/2048113146 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5611b9585750 0x5611b9569ac0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.797+0000 7f27337fe640 0 client.17514 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:49.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.797+0000 7f2781edde00 1 -- 10.20.193.28:0/2048113146 >> 10.20.193.28:0/2048113146 conn(0x5611b95638f0 msgr2=0x5611b95def00 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:49.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.797+0000 7f2781edde00 1 -- 10.20.193.28:0/2048113146 shutdown_connections 2026-02-25T11:57:49.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.797+0000 7f2781edde00 1 -- 10.20.193.28:0/2048113146 wait complete. 2026-02-25T11:57:49.798 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.797+0000 7f2781edde00 20 client.17514 trim_cache size 0 max 16384 2026-02-25T11:57:49.798 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:57:49.798 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:57:49.798 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "52c38620-9fde-44bc-acbe-a9a9e3452e53", 2026-02-25T11:57:49.798 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:57:49.798 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:57:49.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.798+0000 7f277ec59640 1 -- 10.20.193.28:0/177846979 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f27540b5760 msgr2=0x7f27540d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:49.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.798+0000 7f277ec59640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f27540b5760 0x7f27540d5c10 secure :-1 s=READY pgs=384 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f2768005970 tx=0x7f2768005900 comp rx=0 tx=0).stop 2026-02-25T11:57:49.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.798+0000 7f277ec59640 1 -- 10.20.193.28:0/177846979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2778110080 msgr2=0x7f277811e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:49.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.798+0000 7f277ec59640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2778110080 0x7f277811e5c0 secure :-1 s=READY pgs=874 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f276c0098a0 tx=0x7f276c0028d0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.798+0000 7f277ec59640 1 -- 10.20.193.28:0/177846979 shutdown_connections 2026-02-25T11:57:49.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.798+0000 7f277ec59640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f27540b5760 0x7f27540d5c10 unknown :-1 s=CLOSED pgs=384 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.798+0000 7f277ec59640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f2778115880 0x7f2778123990 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.798+0000 7f277ec59640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2778110080 0x7f277811e5c0 unknown :-1 s=CLOSED pgs=874 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.798+0000 7f277ec59640 1 --2- 10.20.193.28:0/177846979 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f277810f690 0x7f277811e080 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:49.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.798+0000 7f277ec59640 1 -- 10.20.193.28:0/177846979 >> 10.20.193.28:0/177846979 conn(0x7f27780ecf00 msgr2=0x7f277810d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:49.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.798+0000 7f277ec59640 1 -- 10.20.193.28:0/177846979 shutdown_connections 2026-02-25T11:57:49.799 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:49.798+0000 7f277ec59640 1 -- 10.20.193.28:0/177846979 wait complete. 2026-02-25T11:57:49.898 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:57:50.081 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:57:50.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.265+0000 7fd97f18a640 1 -- 10.20.193.28:0/2710511724 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd97810ddc0 msgr2=0x7fd978112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:50.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.265+0000 7fd97f18a640 1 --2- 10.20.193.28:0/2710511724 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd97810ddc0 0x7fd978112e10 secure :-1 s=READY pgs=878 gs=1 cs=0 l=1 c_cookie=11132535004076242989 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd96c009950 tx=0x7fd96c02f1d0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.266+0000 7fd97f18a640 1 -- 10.20.193.28:0/2710511724 shutdown_connections 2026-02-25T11:57:50.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.266+0000 7fd97f18a640 1 --2- 10.20.193.28:0/2710511724 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd9781135c0 0x7fd978173f90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.266+0000 7fd97f18a640 1 --2- 10.20.193.28:0/2710511724 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd97810ddc0 0x7fd978112e10 unknown :-1 s=CLOSED pgs=878 gs=1 cs=0 l=1 c_cookie=11132535004076242989 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.266+0000 7fd97f18a640 1 --2- 10.20.193.28:0/2710511724 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd97810d3d0 0x7fd97810d7a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.266+0000 7fd97f18a640 1 -- 10.20.193.28:0/2710511724 >> 10.20.193.28:0/2710511724 conn(0x7fd9780eaca0 msgr2=0x7fd97810b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:50.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.266+0000 7fd97f18a640 1 -- 10.20.193.28:0/2710511724 shutdown_connections 2026-02-25T11:57:50.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.266+0000 7fd97f18a640 1 -- 10.20.193.28:0/2710511724 wait complete. 2026-02-25T11:57:50.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.266+0000 7fd97f18a640 1 Processor -- start 2026-02-25T11:57:50.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97f18a640 1 -- start start 2026-02-25T11:57:50.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97f18a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd97810d3d0 0x7fd9780f5970 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97f18a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd97810ddc0 0x7fd978173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97f18a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd9781135c0 0x7fd9780f6070 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97f18a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd9780fb280 con 0x7fd9781135c0 2026-02-25T11:57:50.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97f18a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd9780fb3f0 con 0x7fd97810ddc0 2026-02-25T11:57:50.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97f18a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd9780fb5b0 con 0x7fd97810d3d0 2026-02-25T11:57:50.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97e989640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd9781135c0 0x7fd9780f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97e989640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd9781135c0 0x7fd9780f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44618/0 (socket says 10.20.193.28:44618) 2026-02-25T11:57:50.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97e989640 1 -- 10.20.193.28:0/2705011232 learned_addr learned my addr 10.20.193.28:0/2705011232 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:50.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97e188640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd97810d3d0 0x7fd9780f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97d987640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd97810ddc0 0x7fd978173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97e989640 1 -- 10.20.193.28:0/2705011232 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd97810d3d0 msgr2=0x7fd9780f5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:50.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97e989640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd97810d3d0 0x7fd9780f5970 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97e989640 1 -- 10.20.193.28:0/2705011232 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd97810ddc0 msgr2=0x7fd978173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:50.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97e989640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd97810ddc0 0x7fd978173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.267+0000 7fd97e989640 1 -- 10.20.193.28:0/2705011232 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c009570 con 0x7fd9781135c0 2026-02-25T11:57:50.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.268+0000 7fd97e989640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd9781135c0 0x7fd9780f6070 secure :-1 s=READY pgs=879 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd97400f8d0 tx=0x7fd97400fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:50.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.268+0000 7fd95f7fe640 1 -- 10.20.193.28:0/2705011232 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd97400dce0 con 0x7fd9781135c0 2026-02-25T11:57:50.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.268+0000 7fd95f7fe640 1 -- 10.20.193.28:0/2705011232 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd97400de40 con 0x7fd9781135c0 2026-02-25T11:57:50.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.268+0000 7fd95f7fe640 1 -- 10.20.193.28:0/2705011232 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd9740116b0 con 0x7fd9781135c0 2026-02-25T11:57:50.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.268+0000 7fd97f18a640 1 -- 10.20.193.28:0/2705011232 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9780fb890 con 0x7fd9781135c0 2026-02-25T11:57:50.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.268+0000 7fd97f18a640 1 -- 10.20.193.28:0/2705011232 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9780fbd10 con 0x7fd9781135c0 2026-02-25T11:57:50.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.271+0000 7fd95f7fe640 1 -- 10.20.193.28:0/2705011232 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fd974020060 con 0x7fd9781135c0 2026-02-25T11:57:50.273 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.273+0000 7fd95f7fe640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd9540b5750 0x7fd9540d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.273+0000 7fd95f7fe640 1 -- 10.20.193.28:0/2705011232 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd974015070 con 0x7fd9781135c0 2026-02-25T11:57:50.274 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.273+0000 7fd97e188640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd9540b5750 0x7fd9540d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.273+0000 7fd98240ee00 1 Processor -- start 2026-02-25T11:57:50.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.273+0000 7fd98240ee00 1 -- start start 2026-02-25T11:57:50.275 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd97e188640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd9540b5750 0x7fd9540d5c00 secure :-1 s=READY pgs=385 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd968004740 tx=0x7fd968005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:50.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd98240ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977d140 0x5652b977d510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd98240ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977da50 0x5652b97c21b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd98240ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b97e2ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd98240ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5652b96939b0 con 0x5652b977da50 2026-02-25T11:57:50.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd98240ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5652b968f6e0 con 0x5652b97c26f0 2026-02-25T11:57:50.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd98240ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5652b9689840 con 0x5652b977d140 2026-02-25T11:57:50.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd97e188640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977d140 0x5652b977d510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd97e188640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977d140 0x5652b977d510 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48644/0 (socket says 10.20.193.28:48644) 2026-02-25T11:57:50.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd97e188640 1 -- 10.20.193.28:0/517728771 learned_addr learned my addr 10.20.193.28:0/517728771 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:50.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd97d987640 1 --2- 10.20.193.28:0/517728771 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977da50 0x5652b97c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.276 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd97e989640 1 --2- 10.20.193.28:0/517728771 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b97e2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd97e188640 1 -- 10.20.193.28:0/517728771 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 msgr2=0x5652b97e2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:50.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd97e188640 1 --2- 10.20.193.28:0/517728771 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b97e2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.274+0000 7fd97e188640 1 -- 10.20.193.28:0/517728771 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977da50 msgr2=0x5652b97c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:50.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd97e188640 1 --2- 10.20.193.28:0/517728771 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977da50 0x5652b97c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd97e188640 1 -- 10.20.193.28:0/517728771 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652b9634d80 con 0x5652b977d140 2026-02-25T11:57:50.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd97e989640 1 --2- 10.20.193.28:0/517728771 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b97e2ad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:57:50.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd97d987640 1 --2- 10.20.193.28:0/517728771 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977da50 0x5652b97c21b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:57:50.277 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd97e188640 1 --2- 10.20.193.28:0/517728771 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977d140 0x5652b977d510 secure :-1 s=READY pgs=230 gs=1 cs=0 l=1 c_cookie=4021513293998552102 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd968009480 tx=0x7fd968013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37cf4908b44b7426 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:50.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd95c884640 1 -- 10.20.193.28:0/517728771 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd96801d460 con 0x5652b977d140 2026-02-25T11:57:50.279 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd95c884640 1 -- 10.20.193.28:0/517728771 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd96801da80 con 0x5652b977d140 2026-02-25T11:57:50.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd95c884640 1 -- 10.20.193.28:0/517728771 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd96801aa40 con 0x5652b977d140 2026-02-25T11:57:50.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd98240ee00 1 -- 10.20.193.28:0/517728771 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977d140 msgr2=0x5652b977d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:50.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd98240ee00 1 --2- 10.20.193.28:0/517728771 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977d140 0x5652b977d510 secure :-1 s=READY pgs=230 gs=1 cs=0 l=1 c_cookie=4021513293998552102 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd968009480 tx=0x7fd968013000 comp rx=0 tx=0).stop 2026-02-25T11:57:50.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd98240ee00 1 -- 10.20.193.28:0/517728771 shutdown_connections 2026-02-25T11:57:50.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd98240ee00 1 --2- 10.20.193.28:0/517728771 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b97e2ad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd98240ee00 1 --2- 10.20.193.28:0/517728771 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977da50 0x5652b97c21b0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd98240ee00 1 --2- 10.20.193.28:0/517728771 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977d140 0x5652b977d510 unknown :-1 s=CLOSED pgs=230 gs=1 cs=0 l=1 c_cookie=4021513293998552102 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd98240ee00 1 -- 10.20.193.28:0/517728771 >> 10.20.193.28:0/517728771 conn(0x5652b969cd40 msgr2=0x5652b977ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:50.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd98240ee00 1 -- 10.20.193.28:0/517728771 shutdown_connections 2026-02-25T11:57:50.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.275+0000 7fd98240ee00 1 -- 10.20.193.28:0/517728771 wait complete. 2026-02-25T11:57:50.280 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd98240ee00 1 -- start start 2026-02-25T11:57:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd98240ee00 1 Processor -- start 2026-02-25T11:57:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd98240ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 0x5652b977e2b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd98240ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977da50 0x5652b977e7f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd98240ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b9781210 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd98240ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5652b9781750 con 0x5652b977d140 2026-02-25T11:57:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd98240ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5652b97818c0 con 0x5652b97c26f0 2026-02-25T11:57:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd98240ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5652b9781a80 con 0x5652b977da50 2026-02-25T11:57:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd97e188640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 0x5652b977e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd97d987640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977da50 0x5652b977e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd97d987640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977da50 0x5652b977e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48652/0 (socket says 10.20.193.28:48652) 2026-02-25T11:57:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd97d987640 1 -- 10.20.193.28:0/1190302111 learned_addr learned my addr 10.20.193.28:0/1190302111 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:50.281 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd97e188640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 0x5652b977e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44638/0 (socket says 10.20.193.28:44638) 2026-02-25T11:57:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.277+0000 7fd97e989640 1 --2- 10.20.193.28:0/1190302111 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b9781210 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd97e188640 1 -- 10.20.193.28:0/1190302111 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977da50 msgr2=0x5652b977e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd97e188640 1 --2- 10.20.193.28:0/1190302111 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977da50 0x5652b977e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd97e188640 1 -- 10.20.193.28:0/1190302111 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 msgr2=0x5652b9781210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd97e188640 1 --2- 10.20.193.28:0/1190302111 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b9781210 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd97e188640 1 -- 10.20.193.28:0/1190302111 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd968009880 con 0x5652b977d140 2026-02-25T11:57:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd97e989640 1 --2- 10.20.193.28:0/1190302111 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b9781210 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:57:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd97e188640 1 --2- 10.20.193.28:0/1190302111 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 0x5652b977e2b0 secure :-1 s=READY pgs=880 gs=1 cs=0 l=1 c_cookie=3102796929838063267 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd968009480 tx=0x7fd96801beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0f595e4d2b9ea3 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd92bfff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:50.282 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd92bfff640 1 -- 10.20.193.28:0/1190302111 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd96802c070 con 0x5652b977d140 2026-02-25T11:57:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd98240ee00 1 -- 10.20.193.28:0/1190302111 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd978174800 con 0x5652b977d140 2026-02-25T11:57:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd92bfff640 1 -- 10.20.193.28:0/1190302111 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd968009ec0 con 0x5652b977d140 2026-02-25T11:57:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd92bfff640 1 -- 10.20.193.28:0/1190302111 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd968027960 con 0x5652b977d140 2026-02-25T11:57:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd98240ee00 10 client.17526 _ll_register_callbacks cb 0x5652b960a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd98240ee00 1 -- 10.20.193.28:0/1190302111 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5652b931f6f0 con 0x5652b977d140 2026-02-25T11:57:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.278+0000 7fd92bfff640 1 -- 10.20.193.28:0/1190302111 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd968027af0 con 0x5652b977d140 2026-02-25T11:57:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.279+0000 7fd92bfff640 1 -- 10.20.193.28:0/1190302111 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd968027cb0 con 0x5652b977d140 2026-02-25T11:57:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.279+0000 7fd98240ee00 10 client.17526 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:50.283 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.279+0000 7fd98240ee00 1 -- 10.20.193.28:0/1190302111 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5652b960d350 con 0x5652b977d140 2026-02-25T11:57:50.287 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.279+0000 7fd92bfff640 1 -- 10.20.193.28:0/1190302111 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd968025070 con 0x5652b977d140 2026-02-25T11:57:50.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.279+0000 7fd98240ee00 10 client.17526 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:50.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.279+0000 7fd98240ee00 10 client.17526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:50.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.279+0000 7fd98240ee00 20 client.17526 populate_metadata read hostname 'trial028' 2026-02-25T11:57:50.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.279+0000 7fd98240ee00 1 --2- 10.20.193.28:0/1190302111 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5652b97a8750 0x5652b97a8bf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.288 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.279+0000 7fd98240ee00 4 client.17526 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:57:50.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.279+0000 7fd98240ee00 1 -- 10.20.193.28:0/1190302111 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652b965b910 con 0x5652b97a8750 2026-02-25T11:57:50.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.279+0000 7fd97d987640 1 --2- 10.20.193.28:0/1190302111 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5652b97a8750 0x5652b97a8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.280+0000 7fd97d987640 1 --2- 10.20.193.28:0/1190302111 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5652b97a8750 0x5652b97a8bf0 crc :-1 s=READY pgs=174 gs=4 cs=0 l=0 c_cookie=15307227100196172285 s_cookie=332396055826960965 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d46e31f24b6cf1fd server_cookie=49ce873a04e9245 in_seq=0 out_seq=0 2026-02-25T11:57:50.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.280+0000 7fd92bfff640 10 client.17526 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:50.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.281+0000 7fd92bfff640 1 -- 10.20.193.28:0/1190302111 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5652b965b910 con 0x5652b97a8750 2026-02-25T11:57:50.289 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.281+0000 7fd92bfff640 10 client.17526 handle_command_reply: tid=0 2026-02-25T11:57:50.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.281+0000 7fd98240ee00 1 client.17526 shutdown 2026-02-25T11:57:50.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.281+0000 7fd98240ee00 1 -- 10.20.193.28:0/1190302111 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 msgr2=0x5652b977e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:50.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.281+0000 7fd98240ee00 1 --2- 10.20.193.28:0/1190302111 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 0x5652b977e2b0 secure :-1 s=READY pgs=880 gs=1 cs=0 l=1 c_cookie=3102796929838063267 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd968009480 tx=0x7fd96801beb0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.281+0000 7fd98240ee00 1 -- 10.20.193.28:0/1190302111 shutdown_connections 2026-02-25T11:57:50.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.281+0000 7fd98240ee00 1 --2- 10.20.193.28:0/1190302111 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5652b97a8750 0x5652b97a8bf0 crc :-1 s=READY pgs=174 gs=4 cs=0 l=0 c_cookie=15307227100196172285 s_cookie=332396055826960965 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.290 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.281+0000 7fd98240ee00 1 --2- 10.20.193.28:0/1190302111 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b9781210 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.281+0000 7fd98240ee00 1 --2- 10.20.193.28:0/1190302111 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977da50 0x5652b977e7f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.281+0000 7fd98240ee00 1 --2- 10.20.193.28:0/1190302111 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 0x5652b977e2b0 unknown :-1 s=CLOSED pgs=880 gs=1 cs=0 l=1 c_cookie=3102796929838063267 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.281+0000 7fd98240ee00 1 -- 10.20.193.28:0/1190302111 >> 10.20.193.28:0/1190302111 conn(0x5652b969cd40 msgr2=0x5652b977b230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:50.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.281+0000 7fd92bfff640 0 client.17526 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:50.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.282+0000 7fd98240ee00 1 -- 10.20.193.28:0/1190302111 shutdown_connections 2026-02-25T11:57:50.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.282+0000 7fd98240ee00 1 -- 10.20.193.28:0/1190302111 wait complete. 2026-02-25T11:57:50.291 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.282+0000 7fd98240ee00 20 client.17526 trim_cache size 0 max 16384 2026-02-25T11:57:50.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.290+0000 7fd98240ee00 1 Processor -- start 2026-02-25T11:57:50.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.290+0000 7fd98240ee00 1 -- start start 2026-02-25T11:57:50.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd98240ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977d140 0x5652b97cff50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd98240ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977da50 0x5652b97e1d90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd98240ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b97e22d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.300 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd98240ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5652b9687470 con 0x5652b977da50 2026-02-25T11:57:50.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd98240ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5652b9347700 con 0x5652b97c26f0 2026-02-25T11:57:50.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd98240ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5652b977edd0 con 0x5652b977d140 2026-02-25T11:57:50.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd97d987640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977da50 0x5652b97e1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd97d987640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977da50 0x5652b97e1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44642/0 (socket says 10.20.193.28:44642) 2026-02-25T11:57:50.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd97d987640 1 -- 10.20.193.28:0/1674875456 learned_addr learned my addr 10.20.193.28:0/1674875456 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:50.301 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd97e188640 1 --2- 10.20.193.28:0/1674875456 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977d140 0x5652b97cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd97d987640 1 -- 10.20.193.28:0/1674875456 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977d140 msgr2=0x5652b97cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:50.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd97d987640 1 --2- 10.20.193.28:0/1674875456 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977d140 0x5652b97cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd97d987640 1 -- 10.20.193.28:0/1674875456 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 msgr2=0x5652b97e22d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2026-02-25T11:57:50.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd97e989640 1 --2- 10.20.193.28:0/1674875456 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b97e22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd97d987640 1 --2- 10.20.193.28:0/1674875456 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b97e22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.302 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd97d987640 1 -- 10.20.193.28:0/1674875456 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652b961f150 con 0x5652b977da50 2026-02-25T11:57:50.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.291+0000 7fd97d987640 1 --2- 10.20.193.28:0/1674875456 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977da50 0x5652b97e1d90 secure :-1 s=READY pgs=881 gs=1 cs=0 l=1 c_cookie=13327355842295074030 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd96c004820 tx=0x7fd96c031040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f44786f7398cee server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:50.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.292+0000 7fd9464e6640 1 -- 10.20.193.28:0/1674875456 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd96c002e70 con 0x5652b977da50 2026-02-25T11:57:50.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.292+0000 7fd9464e6640 1 -- 10.20.193.28:0/1674875456 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5652b965b910 con 0x5652b977da50 2026-02-25T11:57:50.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.292+0000 7fd9464e6640 1 -- 10.20.193.28:0/1674875456 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd96c04c6f0 con 0x5652b977da50 2026-02-25T11:57:50.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.292+0000 7fd98240ee00 1 -- 10.20.193.28:0/1674875456 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977da50 msgr2=0x5652b97e1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:50.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.292+0000 7fd98240ee00 1 --2- 10.20.193.28:0/1674875456 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977da50 0x5652b97e1d90 secure :-1 s=READY pgs=881 gs=1 cs=0 l=1 c_cookie=13327355842295074030 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd96c004820 tx=0x7fd96c031040 comp rx=0 tx=0).stop 2026-02-25T11:57:50.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.292+0000 7fd98240ee00 1 -- 10.20.193.28:0/1674875456 shutdown_connections 2026-02-25T11:57:50.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.292+0000 7fd98240ee00 1 --2- 10.20.193.28:0/1674875456 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b97c26f0 0x5652b97e22d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.292+0000 7fd98240ee00 1 --2- 10.20.193.28:0/1674875456 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977da50 0x5652b97e1d90 unknown :-1 s=CLOSED pgs=881 gs=1 cs=0 l=1 c_cookie=13327355842295074030 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.303 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.292+0000 7fd98240ee00 1 --2- 10.20.193.28:0/1674875456 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b977d140 0x5652b97cff50 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.292+0000 7fd98240ee00 1 -- 10.20.193.28:0/1674875456 >> 10.20.193.28:0/1674875456 conn(0x5652b969cd40 msgr2=0x5652b977e100 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:50.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.292+0000 7fd98240ee00 1 -- 10.20.193.28:0/1674875456 shutdown_connections 2026-02-25T11:57:50.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.292+0000 7fd98240ee00 1 -- 10.20.193.28:0/1674875456 wait complete. 2026-02-25T11:57:50.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.293+0000 7fd98240ee00 1 -- start start 2026-02-25T11:57:50.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.293+0000 7fd98240ee00 1 Processor -- start 2026-02-25T11:57:50.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.293+0000 7fd98240ee00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 0x5652b977cae0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.293+0000 7fd98240ee00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b977da50 0x5652b977b320 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.293+0000 7fd98240ee00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b97c26f0 0x5652b977b9f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.293+0000 7fd98240ee00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5652b9758490 con 0x5652b977d140 2026-02-25T11:57:50.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.293+0000 7fd98240ee00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5652b97da750 con 0x5652b977da50 2026-02-25T11:57:50.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.293+0000 7fd98240ee00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5652b977c1a0 con 0x5652b97c26f0 2026-02-25T11:57:50.304 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.293+0000 7fd97e188640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 0x5652b977cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.293+0000 7fd97d987640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b977da50 0x5652b977b320 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.293+0000 7fd97e188640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 0x5652b977cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:44650/0 (socket says 10.20.193.28:44650) 2026-02-25T11:57:50.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.293+0000 7fd97e188640 1 -- 10.20.193.28:0/645570594 learned_addr learned my addr 10.20.193.28:0/645570594 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:57:50.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.293+0000 7fd97e989640 1 --2- 10.20.193.28:0/645570594 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b97c26f0 0x5652b977b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd97e188640 1 -- 10.20.193.28:0/645570594 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b97c26f0 msgr2=0x5652b977b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:50.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd97e188640 1 --2- 10.20.193.28:0/645570594 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b97c26f0 0x5652b977b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd97e188640 1 -- 10.20.193.28:0/645570594 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b977da50 msgr2=0x5652b977b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:57:50.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd97e188640 1 --2- 10.20.193.28:0/645570594 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b977da50 0x5652b977b320 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd97e188640 1 -- 10.20.193.28:0/645570594 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652b9619bd0 con 0x5652b977d140 2026-02-25T11:57:50.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd97e188640 1 --2- 10.20.193.28:0/645570594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 0x5652b977cae0 secure :-1 s=READY pgs=882 gs=1 cs=0 l=1 c_cookie=4305549084888783273 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd968009850 tx=0x7fd968038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc06214b7cdd9a9 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:57:50.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd92bfff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:57:50.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd92bfff640 1 -- 10.20.193.28:0/645570594 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd96802c040 con 0x5652b977d140 2026-02-25T11:57:50.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd92bfff640 1 -- 10.20.193.28:0/645570594 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5652b931f6f0 con 0x5652b977d140 2026-02-25T11:57:50.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd98240ee00 1 -- 10.20.193.28:0/645570594 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96c002c10 con 0x5652b977d140 2026-02-25T11:57:50.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd92bfff640 1 -- 10.20.193.28:0/645570594 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd968010c00 con 0x5652b977d140 2026-02-25T11:57:50.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd98240ee00 10 client.17532 _ll_register_callbacks cb 0x5652b960a9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:57:50.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd98240ee00 1 -- 10.20.193.28:0/645570594 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd96801da80 con 0x5652b977d140 2026-02-25T11:57:50.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd92bfff640 1 -- 10.20.193.28:0/645570594 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd968010d90 con 0x5652b977d140 2026-02-25T11:57:50.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd92bfff640 1 -- 10.20.193.28:0/645570594 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd96801a7a0 con 0x5652b977d140 2026-02-25T11:57:50.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd98240ee00 10 client.17532 fetch_fsmap learned FSMap version 23 2026-02-25T11:57:50.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.294+0000 7fd98240ee00 1 -- 10.20.193.28:0/645570594 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5652b960d680 con 0x5652b977d140 2026-02-25T11:57:50.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.295+0000 7fd92bfff640 1 -- 10.20.193.28:0/645570594 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd968025040 con 0x5652b977d140 2026-02-25T11:57:50.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.295+0000 7fd98240ee00 10 client.17532 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:57:50.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.295+0000 7fd98240ee00 10 client.17532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:57:50.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.295+0000 7fd98240ee00 20 client.17532 populate_metadata read hostname 'trial028' 2026-02-25T11:57:50.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.295+0000 7fd98240ee00 1 --2- 10.20.193.28:0/645570594 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5652b97bb360 0x5652b97bb7d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.295+0000 7fd98240ee00 4 client.17532 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.295+0000 7fd98240ee00 1 -- 10.20.193.28:0/645570594 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5652b931cf40 con 0x5652b97bb360 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.295+0000 7fd97d987640 1 --2- 10.20.193.28:0/645570594 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5652b97bb360 0x5652b97bb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd97d987640 1 --2- 10.20.193.28:0/645570594 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5652b97bb360 0x5652b97bb7d0 crc :-1 s=READY pgs=175 gs=4 cs=0 l=0 c_cookie=974354024945253854 s_cookie=1333660406802904353 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8599d2c6557dde server_cookie=12821d12ee7d7521 in_seq=0 out_seq=0 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd92bfff640 10 client.17532 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd92bfff640 1 -- 10.20.193.28:0/645570594 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5966 (crc 0 0 0) 0x5652b931cf40 con 0x5652b97bb360 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd92bfff640 10 client.17532 handle_command_reply: tid=0 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd98240ee00 1 client.17532 shutdown 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd98240ee00 1 -- 10.20.193.28:0/645570594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 msgr2=0x5652b977cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd98240ee00 1 --2- 10.20.193.28:0/645570594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 0x5652b977cae0 secure :-1 s=READY pgs=882 gs=1 cs=0 l=1 c_cookie=4305549084888783273 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd968009850 tx=0x7fd968038000 comp rx=0 tx=0).stop 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd98240ee00 1 -- 10.20.193.28:0/645570594 shutdown_connections 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd98240ee00 1 --2- 10.20.193.28:0/645570594 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5652b97bb360 0x5652b97bb7d0 crc :-1 s=READY pgs=175 gs=4 cs=0 l=0 c_cookie=974354024945253854 s_cookie=1333660406802904353 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd98240ee00 1 --2- 10.20.193.28:0/645570594 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5652b97c26f0 0x5652b977b9f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.307 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd98240ee00 1 --2- 10.20.193.28:0/645570594 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5652b977da50 0x5652b977b320 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd98240ee00 1 --2- 10.20.193.28:0/645570594 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5652b977d140 0x5652b977cae0 unknown :-1 s=CLOSED pgs=882 gs=1 cs=0 l=1 c_cookie=4305549084888783273 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd98240ee00 1 -- 10.20.193.28:0/645570594 >> 10.20.193.28:0/645570594 conn(0x5652b969cd40 msgr2=0x5652b97c2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:50.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd92bfff640 0 client.17532 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:57:50.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd98240ee00 1 -- 10.20.193.28:0/645570594 shutdown_connections 2026-02-25T11:57:50.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.296+0000 7fd98240ee00 1 -- 10.20.193.28:0/645570594 wait complete. 2026-02-25T11:57:50.308 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.297+0000 7fd98240ee00 20 client.17532 trim_cache size 0 max 16384 2026-02-25T11:57:50.308 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: "52c38620-9fde-44bc-acbe-a9a9e3452e53": { 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "52c38620-9fde-44bc-acbe-a9a9e3452e53", 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: "52c38620-9fde-44bc-acbe-a9a9e3452e53": { 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:57:49.795729+0000", 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.309 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: "0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8": { 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:05.873000+0000", 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: "850bcfd3-be8a-4930-896c-1ed6e48b8fed": { 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:27.124813+0000", 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.310 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:57:50.311 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "516f576f-224f-4d5c-9352-d469d781f97c": { 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:57:15.169491+0000", 2026-02-25T11:57:50.312 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.313 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:57:50.314 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:57:50.315 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "91e989b7-1f1f-41fb-9047-642867de91e6": { 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:40.578257+0000", 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:57:50.316 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.317 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "7b237934-6b95-43dd-b3a6-82b74134ec55": { 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:31.289725+0000", 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.318 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.298+0000 7fd97f18a640 1 -- 10.20.193.28:0/2705011232 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd9540b5750 msgr2=0x7fd9540d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.298+0000 7fd97f18a640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd9540b5750 0x7fd9540d5c00 secure :-1 s=READY pgs=385 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd968004740 tx=0x7fd968005f90 comp rx=0 tx=0).stop 2026-02-25T11:57:50.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.298+0000 7fd97f18a640 1 -- 10.20.193.28:0/2705011232 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd9781135c0 msgr2=0x7fd9780f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:57:50.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.298+0000 7fd97f18a640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd9781135c0 0x7fd9780f6070 secure :-1 s=READY pgs=879 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd97400f8d0 tx=0x7fd97400fda0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.298+0000 7fd97f18a640 1 -- 10.20.193.28:0/2705011232 shutdown_connections 2026-02-25T11:57:50.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.298+0000 7fd97f18a640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd9540b5750 0x7fd9540d5c00 unknown :-1 s=CLOSED pgs=385 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.298+0000 7fd97f18a640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd9781135c0 0x7fd9780f6070 unknown :-1 s=CLOSED pgs=879 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.298+0000 7fd97f18a640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd97810ddc0 0x7fd978173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.298+0000 7fd97f18a640 1 --2- 10.20.193.28:0/2705011232 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd97810d3d0 0x7fd9780f5970 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:57:50.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.298+0000 7fd97f18a640 1 -- 10.20.193.28:0/2705011232 >> 10.20.193.28:0/2705011232 conn(0x7fd9780eaca0 msgr2=0x7fd97810b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:57:50.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.298+0000 7fd97f18a640 1 -- 10.20.193.28:0/2705011232 shutdown_connections 2026-02-25T11:57:50.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:57:50.298+0000 7fd97f18a640 1 -- 10.20.193.28:0/2705011232 wait complete. 2026-02-25T11:57:50.386 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:50 trial028 ceph-mon[39224]: pgmap v514: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:50.386 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:50 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:57:50.387 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:50 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:57:50.387 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:50 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:57:50.387 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:50 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:57:50.435 INFO:tasks.cephfs.filesystem:scrub status for tag:52c38620-9fde-44bc-acbe-a9a9e3452e53 - {'path': '/', 'tag': '52c38620-9fde-44bc-acbe-a9a9e3452e53', 'options': 'recursive,force'} 2026-02-25T11:57:50.580 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:50 trial119 ceph-mon[42710]: pgmap v514: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:50.580 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:50 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:57:50.580 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:50 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:57:50.580 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:50 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:57:50.580 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:50 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:57:50.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:50 trial167 ceph-mon[43392]: pgmap v514: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:57:50.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:50 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:57:50.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:50 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:57:50.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:50 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:57:50.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:50 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:57:52.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:52 trial167 ceph-mon[43392]: pgmap v515: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:57:52.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:52 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:57:52.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:52 trial119 ceph-mon[42710]: pgmap v515: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:57:52.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:52 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:57:52.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:52 trial028 ceph-mon[39224]: pgmap v515: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:57:52.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:52 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:57:54.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:54 trial167 ceph-mon[43392]: pgmap v516: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:57:54.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:54 trial119 ceph-mon[42710]: pgmap v516: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:57:54.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:54 trial028 ceph-mon[39224]: pgmap v516: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:57:56.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:56 trial167 ceph-mon[43392]: pgmap v517: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:57:56.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:56 trial119 ceph-mon[42710]: pgmap v517: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:57:56.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:56 trial028 ceph-mon[39224]: pgmap v517: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:57:58.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:57:58 trial167 ceph-mon[43392]: pgmap v518: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:58.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:57:58 trial119 ceph-mon[42710]: pgmap v518: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:57:58.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:57:58 trial028 ceph-mon[39224]: pgmap v518: 97 pgs: 97 active+clean; 7.4 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:58:00.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:00 trial167 ceph-mon[43392]: pgmap v519: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:58:00.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:00 trial119 ceph-mon[42710]: pgmap v519: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:58:00.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:00 trial028 ceph-mon[39224]: pgmap v519: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 24 KiB/s wr, 14 op/s 2026-02-25T11:58:02.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:02 trial167 ceph-mon[43392]: pgmap v520: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:58:02.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:02 trial119 ceph-mon[42710]: pgmap v520: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:58:02.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:02 trial028 ceph-mon[39224]: pgmap v520: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:58:04.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:04 trial167 ceph-mon[43392]: pgmap v521: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:58:04.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:04 trial119 ceph-mon[42710]: pgmap v521: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:58:04.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:04 trial028 ceph-mon[39224]: pgmap v521: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 15 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:58:06.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:06 trial167 ceph-mon[43392]: pgmap v522: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:58:06.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:06 trial119 ceph-mon[42710]: pgmap v522: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:58:06.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:06 trial028 ceph-mon[39224]: pgmap v522: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 13 op/s 2026-02-25T11:58:08.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:08 trial167 ceph-mon[43392]: pgmap v523: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:58:08.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:08 trial119 ceph-mon[42710]: pgmap v523: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:58:08.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:08 trial028 ceph-mon[39224]: pgmap v523: 97 pgs: 97 active+clean; 7.5 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:58:10.612 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:10 trial028 ceph-mon[39224]: pgmap v524: 97 pgs: 97 active+clean; 7.6 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:58:10.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:10 trial167 ceph-mon[43392]: pgmap v524: 97 pgs: 97 active+clean; 7.6 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:58:10.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:10 trial119 ceph-mon[42710]: pgmap v524: 97 pgs: 97 active+clean; 7.6 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:58:12.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:12 trial167 ceph-mon[43392]: pgmap v525: 97 pgs: 97 active+clean; 7.6 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:58:12.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:12 trial119 ceph-mon[42710]: pgmap v525: 97 pgs: 97 active+clean; 7.6 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:58:12.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:12 trial028 ceph-mon[39224]: pgmap v525: 97 pgs: 97 active+clean; 7.6 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:58:14.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:14 trial167 ceph-mon[43392]: pgmap v526: 97 pgs: 97 active+clean; 7.6 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:58:14.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:14 trial119 ceph-mon[42710]: pgmap v526: 97 pgs: 97 active+clean; 7.6 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:58:14.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:14 trial028 ceph-mon[39224]: pgmap v526: 97 pgs: 97 active+clean; 7.6 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 17 KiB/s wr, 12 op/s 2026-02-25T11:58:16.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:16 trial167 ceph-mon[43392]: pgmap v527: 97 pgs: 97 active+clean; 7.7 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:58:16.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:16 trial119 ceph-mon[42710]: pgmap v527: 97 pgs: 97 active+clean; 7.7 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:58:16.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:16 trial028 ceph-mon[39224]: pgmap v527: 97 pgs: 97 active+clean; 7.7 MiB data, 351 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:58:18.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:18 trial167 ceph-mon[43392]: pgmap v528: 97 pgs: 97 active+clean; 7.7 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:58:18.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:18 trial119 ceph-mon[42710]: pgmap v528: 97 pgs: 97 active+clean; 7.7 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:58:18.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:18 trial028 ceph-mon[39224]: pgmap v528: 97 pgs: 97 active+clean; 7.7 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:58:20.436 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:58:20.622 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:58:20.666 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:20 trial028 ceph-mon[39224]: pgmap v529: 97 pgs: 97 active+clean; 7.7 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:58:20.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:20 trial167 ceph-mon[43392]: pgmap v529: 97 pgs: 97 active+clean; 7.7 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:58:20.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:20 trial119 ceph-mon[42710]: pgmap v529: 97 pgs: 97 active+clean; 7.7 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 19 KiB/s rd, 23 KiB/s wr, 14 op/s 2026-02-25T11:58:20.800 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.799+0000 7fecbe6e7640 1 -- 10.20.193.28:0/4214113520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fecb81135c0 msgr2=0x7fecb8173f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:20.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.799+0000 7fecbe6e7640 1 --2- 10.20.193.28:0/4214113520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fecb81135c0 0x7fecb8173f90 secure :-1 s=READY pgs=883 gs=3 cs=0 l=1 c_cookie=12744521283682058835 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fecb400a970 tx=0x7fecb40301b0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.800+0000 7fecbe6e7640 1 -- 10.20.193.28:0/4214113520 shutdown_connections 2026-02-25T11:58:20.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.800+0000 7fecbe6e7640 1 --2- 10.20.193.28:0/4214113520 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fecb81135c0 0x7fecb8173f90 unknown :-1 s=CLOSED pgs=883 gs=3 cs=0 l=1 c_cookie=12744521283682058835 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.800+0000 7fecbe6e7640 1 --2- 10.20.193.28:0/4214113520 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fecb810ddc0 0x7fecb8112e10 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.800+0000 7fecbe6e7640 1 --2- 10.20.193.28:0/4214113520 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fecb810d3d0 0x7fecb810d7a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.800+0000 7fecbe6e7640 1 -- 10.20.193.28:0/4214113520 >> 10.20.193.28:0/4214113520 conn(0x7fecb80eaca0 msgr2=0x7fecb810b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:20.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.800+0000 7fecbe6e7640 1 -- 10.20.193.28:0/4214113520 shutdown_connections 2026-02-25T11:58:20.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.800+0000 7fecbe6e7640 1 -- 10.20.193.28:0/4214113520 wait complete. 2026-02-25T11:58:20.801 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.801+0000 7fecbe6e7640 1 Processor -- start 2026-02-25T11:58:20.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.801+0000 7fecbe6e7640 1 -- start start 2026-02-25T11:58:20.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.801+0000 7fecbe6e7640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fecb810d3d0 0x7fecb80f5960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.801+0000 7fecbe6e7640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fecb810ddc0 0x7fecb8173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.801+0000 7fecbe6e7640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fecb81135c0 0x7fecb80f6060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.801+0000 7fecbe6e7640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80fb270 con 0x7fecb81135c0 2026-02-25T11:58:20.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.801+0000 7fecbe6e7640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80fb3e0 con 0x7fecb810d3d0 2026-02-25T11:58:20.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.801+0000 7fecbe6e7640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fecb80fb5a0 con 0x7fecb810ddc0 2026-02-25T11:58:20.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.802+0000 7fecbcee4640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fecb810ddc0 0x7fecb8173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.802+0000 7fecbcee4640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fecb810ddc0 0x7fecb8173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52496/0 (socket says 10.20.193.28:52496) 2026-02-25T11:58:20.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.802+0000 7fecbcee4640 1 -- 10.20.193.28:0/2577331634 learned_addr learned my addr 10.20.193.28:0/2577331634 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:20.802 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.802+0000 7fecbdee6640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fecb81135c0 0x7fecb80f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.802+0000 7fecbd6e5640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fecb810d3d0 0x7fecb80f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.802+0000 7fecbcee4640 1 -- 10.20.193.28:0/2577331634 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fecb810d3d0 msgr2=0x7fecb80f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:20.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.803+0000 7fecbcee4640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fecb810d3d0 0x7fecb80f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.803+0000 7fecbcee4640 1 -- 10.20.193.28:0/2577331634 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fecb81135c0 msgr2=0x7fecb80f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:20.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.803+0000 7fecbcee4640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fecb81135c0 0x7fecb80f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.803+0000 7fecbcee4640 1 -- 10.20.193.28:0/2577331634 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb400a590 con 0x7fecb810ddc0 2026-02-25T11:58:20.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.803+0000 7fecbdee6640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fecb81135c0 0x7fecb80f6060 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:58:20.803 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.803+0000 7fecbcee4640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fecb810ddc0 0x7fecb8173e90 secure :-1 s=READY pgs=231 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fecac009f90 tx=0x7fecac00ede0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:20.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.803+0000 7feca67fc640 1 -- 10.20.193.28:0/2577331634 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fecac00cce0 con 0x7fecb810ddc0 2026-02-25T11:58:20.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.803+0000 7fecbe6e7640 1 -- 10.20.193.28:0/2577331634 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb80fb880 con 0x7fecb810ddc0 2026-02-25T11:58:20.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.803+0000 7feca67fc640 1 -- 10.20.193.28:0/2577331634 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fecac00ce40 con 0x7fecb810ddc0 2026-02-25T11:58:20.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.803+0000 7feca67fc640 1 -- 10.20.193.28:0/2577331634 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fecac01f6c0 con 0x7fecb810ddc0 2026-02-25T11:58:20.804 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.803+0000 7fecbe6e7640 1 -- 10.20.193.28:0/2577331634 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb80fbda0 con 0x7fecb810ddc0 2026-02-25T11:58:20.808 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.807+0000 7feca67fc640 1 -- 10.20.193.28:0/2577331634 <== mon.1 v2:10.20.193.167:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fecac0040d0 con 0x7fecb810ddc0 2026-02-25T11:58:20.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.808+0000 7feca67fc640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec940b56d0 0x7fec940d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.808+0000 7feca67fc640 1 -- 10.20.193.28:0/2577331634 <== mon.1 v2:10.20.193.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fecac0ef650 con 0x7fecb810ddc0 2026-02-25T11:58:20.809 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.809+0000 7fecbd6e5640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec940b56d0 0x7fec940d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.809+0000 7fecc196be00 1 Processor -- start 2026-02-25T11:58:20.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.809+0000 7fecc196be00 1 -- start start 2026-02-25T11:58:20.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.809+0000 7fecbd6e5640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec940b56d0 0x7fec940d5b80 secure :-1 s=READY pgs=391 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feca8004740 tx=0x7feca8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:20.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecc196be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3a7e140 0x55cef3a7e510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecc196be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 0x55cef3ac31b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecc196be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3ac36f0 0x55cef3ae3ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecc196be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55cef39949b0 con 0x55cef3ac36f0 2026-02-25T11:58:20.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecc196be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55cef39906e0 con 0x55cef3a7ea50 2026-02-25T11:58:20.810 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecc196be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55cef398a840 con 0x55cef3a7e140 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecbdee6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3ac36f0 0x55cef3ae3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecbd6e5640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3a7e140 0x55cef3a7e510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecbdee6640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3ac36f0 0x55cef3ae3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52150/0 (socket says 10.20.193.28:52150) 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecbdee6640 1 -- 10.20.193.28:0/1860592987 learned_addr learned my addr 10.20.193.28:0/1860592987 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecbdee6640 1 -- 10.20.193.28:0/1860592987 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3a7e140 msgr2=0x55cef3a7e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecbdee6640 1 --2- 10.20.193.28:0/1860592987 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3a7e140 0x55cef3a7e510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecbdee6640 1 -- 10.20.193.28:0/1860592987 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 msgr2=0x55cef3ac31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecbcee4640 1 --2- 10.20.193.28:0/1860592987 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 0x55cef3ac31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecbdee6640 1 --2- 10.20.193.28:0/1860592987 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 0x55cef3ac31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecbdee6640 1 -- 10.20.193.28:0/1860592987 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cef3935d80 con 0x55cef3ac36f0 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecbcee4640 1 --2- 10.20.193.28:0/1860592987 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 0x55cef3ac31b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.810+0000 7fecbdee6640 1 --2- 10.20.193.28:0/1860592987 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3ac36f0 0x55cef3ae3ad0 secure :-1 s=READY pgs=884 gs=1 cs=0 l=1 c_cookie=1996550372077947536 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fecb4006f50 tx=0x7fecb4006f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb52bfc37e14a90 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.811+0000 7fec85ce5640 1 -- 10.20.193.28:0/1860592987 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fecb403e070 con 0x55cef3ac36f0 2026-02-25T11:58:20.811 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.811+0000 7fec85ce5640 1 -- 10.20.193.28:0/1860592987 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fecb403f040 con 0x55cef3ac36f0 2026-02-25T11:58:20.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.811+0000 7fec85ce5640 1 -- 10.20.193.28:0/1860592987 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fecb4005320 con 0x55cef3ac36f0 2026-02-25T11:58:20.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.811+0000 7fecc196be00 1 -- 10.20.193.28:0/1860592987 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3ac36f0 msgr2=0x55cef3ae3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:20.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.811+0000 7fecc196be00 1 --2- 10.20.193.28:0/1860592987 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3ac36f0 0x55cef3ae3ad0 secure :-1 s=READY pgs=884 gs=1 cs=0 l=1 c_cookie=1996550372077947536 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fecb4006f50 tx=0x7fecb4006f80 comp rx=0 tx=0).stop 2026-02-25T11:58:20.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.811+0000 7fecc196be00 1 -- 10.20.193.28:0/1860592987 shutdown_connections 2026-02-25T11:58:20.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.811+0000 7fecc196be00 1 --2- 10.20.193.28:0/1860592987 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3ac36f0 0x55cef3ae3ad0 unknown :-1 s=CLOSED pgs=884 gs=1 cs=0 l=1 c_cookie=1996550372077947536 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.811+0000 7fecc196be00 1 --2- 10.20.193.28:0/1860592987 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 0x55cef3ac31b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.811+0000 7fecc196be00 1 --2- 10.20.193.28:0/1860592987 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3a7e140 0x55cef3a7e510 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.811+0000 7fecc196be00 1 -- 10.20.193.28:0/1860592987 >> 10.20.193.28:0/1860592987 conn(0x55cef399dd40 msgr2=0x55cef3a7ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:20.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.812+0000 7fecc196be00 1 -- 10.20.193.28:0/1860592987 shutdown_connections 2026-02-25T11:58:20.812 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.812+0000 7fecc196be00 1 -- 10.20.193.28:0/1860592987 wait complete. 2026-02-25T11:58:20.813 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.813+0000 7fecc196be00 1 -- start start 2026-02-25T11:58:20.813 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.813+0000 7fecc196be00 1 Processor -- start 2026-02-25T11:58:20.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.813+0000 7fecc196be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 0x55cef3adb480 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.813+0000 7fecc196be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 0x55cef3adb9c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.813+0000 7fecc196be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3ac36f0 0x55cef3add750 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.813+0000 7fecc196be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55cef3addcc0 con 0x55cef3a7e140 2026-02-25T11:58:20.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.813+0000 7fecc196be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55cef3adde30 con 0x55cef3a7ea50 2026-02-25T11:58:20.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.813+0000 7fecc196be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55cef3addff0 con 0x55cef3ac36f0 2026-02-25T11:58:20.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fecbd6e5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 0x55cef3adb480 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fecbd6e5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 0x55cef3adb480 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52154/0 (socket says 10.20.193.28:52154) 2026-02-25T11:58:20.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fecbd6e5640 1 -- 10.20.193.28:0/3994107095 learned_addr learned my addr 10.20.193.28:0/3994107095 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:20.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fecbdee6640 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3ac36f0 0x55cef3add750 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.814 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fecbcee4640 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 0x55cef3adb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fecbdee6640 1 -- 10.20.193.28:0/3994107095 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 msgr2=0x55cef3adb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fecbdee6640 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 0x55cef3adb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fecbdee6640 1 -- 10.20.193.28:0/3994107095 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 msgr2=0x55cef3adb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fecbdee6640 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 0x55cef3adb480 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fecbdee6640 1 -- 10.20.193.28:0/3994107095 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cef390e350 con 0x55cef3ac36f0 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fecbd6e5640 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 0x55cef3adb480 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fecbdee6640 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3ac36f0 0x55cef3add750 secure :-1 s=READY pgs=232 gs=3 cs=0 l=1 c_cookie=14298321714972936311 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fecb40306c0 tx=0x7fecb400a310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c66dd7d811374877 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fec72ffd640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.814+0000 7fec72ffd640 1 -- 10.20.193.28:0/3994107095 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fecb403e040 con 0x55cef3ac36f0 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fecc196be00 1 -- 10.20.193.28:0/3994107095 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb8174800 con 0x55cef3ac36f0 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fec72ffd640 1 -- 10.20.193.28:0/3994107095 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fecb4006a40 con 0x55cef3ac36f0 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fec72ffd640 1 -- 10.20.193.28:0/3994107095 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fecb4044bd0 con 0x55cef3ac36f0 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fecc196be00 10 client.27208 _ll_register_callbacks cb 0x55cef390b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:20.815 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fecc196be00 1 -- 10.20.193.28:0/3994107095 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cef36206f0 con 0x55cef3ac36f0 2026-02-25T11:58:20.816 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fec72ffd640 1 -- 10.20.193.28:0/3994107095 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fecb4044d60 con 0x55cef3ac36f0 2026-02-25T11:58:20.816 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fec72ffd640 1 -- 10.20.193.28:0/3994107095 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fecb404d430 con 0x55cef3ac36f0 2026-02-25T11:58:20.816 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fecc196be00 10 client.27208 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:20.816 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fecc196be00 1 -- 10.20.193.28:0/3994107095 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55cef3920150 con 0x55cef3ac36f0 2026-02-25T11:58:20.816 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fec72ffd640 1 -- 10.20.193.28:0/3994107095 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fecb4044460 con 0x55cef3ac36f0 2026-02-25T11:58:20.816 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fecc196be00 10 client.27208 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:20.816 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fecc196be00 10 client.27208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:58:20.816 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fecc196be00 20 client.27208 populate_metadata read hostname 'trial028' 2026-02-25T11:58:20.816 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fecc196be00 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cef3ac0980 0x55cef3ac0e20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.816 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fecc196be00 4 client.27208 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:58:20.816 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.815+0000 7fecc196be00 1 -- 10.20.193.28:0/3994107095 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cef395c910 con 0x55cef3ac0980 2026-02-25T11:58:20.816 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.816+0000 7fecbd6e5640 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cef3ac0980 0x55cef3ac0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.817 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.816+0000 7fecbd6e5640 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cef3ac0980 0x55cef3ac0e20 crc :-1 s=READY pgs=176 gs=4 cs=0 l=0 c_cookie=276307407062012811 s_cookie=14033326749735391972 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d5a41f6ebd938b server_cookie=c2c06453bef712e4 in_seq=0 out_seq=0 2026-02-25T11:58:20.817 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.816+0000 7fec72ffd640 10 client.27208 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:20.817 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fec72ffd640 1 -- 10.20.193.28:0/3994107095 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55cef395c910 con 0x55cef3ac0980 2026-02-25T11:58:20.817 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fec72ffd640 10 client.27208 handle_command_reply: tid=0 2026-02-25T11:58:20.817 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fecc196be00 1 client.27208 shutdown 2026-02-25T11:58:20.818 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fecc196be00 1 -- 10.20.193.28:0/3994107095 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3ac36f0 msgr2=0x55cef3add750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:20.818 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fecc196be00 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3ac36f0 0x55cef3add750 secure :-1 s=READY pgs=232 gs=3 cs=0 l=1 c_cookie=14298321714972936311 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fecb40306c0 tx=0x7fecb400a310 comp rx=0 tx=0).stop 2026-02-25T11:58:20.818 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fecc196be00 1 -- 10.20.193.28:0/3994107095 shutdown_connections 2026-02-25T11:58:20.818 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fecc196be00 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cef3ac0980 0x55cef3ac0e20 crc :-1 s=READY pgs=176 gs=4 cs=0 l=0 c_cookie=276307407062012811 s_cookie=14033326749735391972 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.818 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fecc196be00 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3ac36f0 0x55cef3add750 unknown :-1 s=CLOSED pgs=232 gs=3 cs=0 l=1 c_cookie=14298321714972936311 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.818 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fecc196be00 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 0x55cef3adb9c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.818 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fecc196be00 1 --2- 10.20.193.28:0/3994107095 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 0x55cef3adb480 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.818 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fec72ffd640 0 client.27208 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:20.818 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fecc196be00 1 -- 10.20.193.28:0/3994107095 >> 10.20.193.28:0/3994107095 conn(0x55cef399dd40 msgr2=0x55cef3a7c230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:20.818 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fecc196be00 1 -- 10.20.193.28:0/3994107095 shutdown_connections 2026-02-25T11:58:20.818 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fecc196be00 1 -- 10.20.193.28:0/3994107095 wait complete. 2026-02-25T11:58:20.818 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.817+0000 7fecc196be00 20 client.27208 trim_cache size 0 max 16384 2026-02-25T11:58:20.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecc196be00 1 Processor -- start 2026-02-25T11:58:20.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecc196be00 1 -- start start 2026-02-25T11:58:20.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecc196be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 0x55cef3ae1880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecc196be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 0x55cef3ad85f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecc196be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3ac36f0 0x55cef3ad8b30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecc196be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55cef3988470 con 0x55cef3a7e140 2026-02-25T11:58:20.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecc196be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55cef3648700 con 0x55cef3a7ea50 2026-02-25T11:58:20.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecc196be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55cef3a7e750 con 0x55cef3ac36f0 2026-02-25T11:58:20.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecbd6e5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 0x55cef3ae1880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecbcee4640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 0x55cef3ad85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecbd6e5640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 0x55cef3ae1880 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52156/0 (socket says 10.20.193.28:52156) 2026-02-25T11:58:20.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecbd6e5640 1 -- 10.20.193.28:0/3303447636 learned_addr learned my addr 10.20.193.28:0/3303447636 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:20.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecbdee6640 1 --2- 10.20.193.28:0/3303447636 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3ac36f0 0x55cef3ad8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecbd6e5640 1 -- 10.20.193.28:0/3303447636 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3ac36f0 msgr2=0x55cef3ad8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecbd6e5640 1 --2- 10.20.193.28:0/3303447636 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3ac36f0 0x55cef3ad8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecbd6e5640 1 -- 10.20.193.28:0/3303447636 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 msgr2=0x55cef3ad85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecbd6e5640 1 --2- 10.20.193.28:0/3303447636 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 0x55cef3ad85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.827+0000 7fecbd6e5640 1 -- 10.20.193.28:0/3303447636 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cef391abd0 con 0x55cef3a7e140 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.828+0000 7fecbd6e5640 1 --2- 10.20.193.28:0/3303447636 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 0x55cef3ae1880 secure :-1 s=READY pgs=885 gs=1 cs=0 l=1 c_cookie=17907958649251346925 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x55cef3adab30 tx=0x7feca8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f885d982d502b9ed server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.828+0000 7fec854e4640 1 -- 10.20.193.28:0/3303447636 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feca8011de0 con 0x55cef3a7e140 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.828+0000 7fec854e4640 1 -- 10.20.193.28:0/3303447636 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7feca8023040 con 0x55cef3a7e140 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.828+0000 7fec854e4640 1 -- 10.20.193.28:0/3303447636 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feca8011de0 con 0x55cef3a7e140 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.828+0000 7fecc196be00 1 -- 10.20.193.28:0/3303447636 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 msgr2=0x55cef3ae1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.828+0000 7fecc196be00 1 --2- 10.20.193.28:0/3303447636 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 0x55cef3ae1880 secure :-1 s=READY pgs=885 gs=1 cs=0 l=1 c_cookie=17907958649251346925 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x55cef3adab30 tx=0x7feca8013000 comp rx=0 tx=0).stop 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.828+0000 7fecc196be00 1 -- 10.20.193.28:0/3303447636 shutdown_connections 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.828+0000 7fecc196be00 1 --2- 10.20.193.28:0/3303447636 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3ac36f0 0x55cef3ad8b30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.828+0000 7fecc196be00 1 --2- 10.20.193.28:0/3303447636 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3a7ea50 0x55cef3ad85f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.828+0000 7fecc196be00 1 --2- 10.20.193.28:0/3303447636 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7e140 0x55cef3ae1880 unknown :-1 s=CLOSED pgs=885 gs=1 cs=0 l=1 c_cookie=17907958649251346925 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.828+0000 7fecc196be00 1 -- 10.20.193.28:0/3303447636 >> 10.20.193.28:0/3303447636 conn(0x55cef399dd40 msgr2=0x55cef3ad9680 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.829+0000 7fecc196be00 1 -- 10.20.193.28:0/3303447636 shutdown_connections 2026-02-25T11:58:20.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.829+0000 7fecc196be00 1 -- 10.20.193.28:0/3303447636 wait complete. 2026-02-25T11:58:20.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.829+0000 7fecc196be00 1 -- start start 2026-02-25T11:58:20.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.829+0000 7fecc196be00 1 Processor -- start 2026-02-25T11:58:20.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecc196be00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3a7e140 0x55cef3ac16a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecc196be00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7ea50 0x55cef3ac1be0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecc196be00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3ac36f0 0x55cef3a5f250 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecc196be00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55cef3ac23d0 con 0x55cef3a7ea50 2026-02-25T11:58:20.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecc196be00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55cef3ac2540 con 0x55cef3ac36f0 2026-02-25T11:58:20.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecc196be00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55cef3a5f7e0 con 0x55cef3a7e140 2026-02-25T11:58:20.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecbcee4640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7ea50 0x55cef3ac1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecbcee4640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7ea50 0x55cef3ac1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52170/0 (socket says 10.20.193.28:52170) 2026-02-25T11:58:20.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecbcee4640 1 -- 10.20.193.28:0/2238842550 learned_addr learned my addr 10.20.193.28:0/2238842550 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:20.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecbd6e5640 1 --2- 10.20.193.28:0/2238842550 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3a7e140 0x55cef3ac16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecbdee6640 1 --2- 10.20.193.28:0/2238842550 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3ac36f0 0x55cef3a5f250 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecbd6e5640 1 -- 10.20.193.28:0/2238842550 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3ac36f0 msgr2=0x55cef3a5f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:20.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecbd6e5640 1 --2- 10.20.193.28:0/2238842550 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3ac36f0 0x55cef3a5f250 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecbd6e5640 1 -- 10.20.193.28:0/2238842550 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7ea50 msgr2=0x55cef3ac1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:20.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecbd6e5640 1 --2- 10.20.193.28:0/2238842550 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7ea50 0x55cef3ac1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.830+0000 7fecbd6e5640 1 -- 10.20.193.28:0/2238842550 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cef39203b0 con 0x55cef3a7e140 2026-02-25T11:58:20.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.831+0000 7fecbd6e5640 1 --2- 10.20.193.28:0/2238842550 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3a7e140 0x55cef3ac16a0 secure :-1 s=READY pgs=233 gs=1 cs=0 l=1 c_cookie=2408799387551698113 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feca802ad80 tx=0x7feca8005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=216dc647d6328cc1 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:20.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.831+0000 7fec72ffd640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:58:20.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.831+0000 7fec72ffd640 1 -- 10.20.193.28:0/2238842550 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feca8011de0 con 0x55cef3a7e140 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.831+0000 7fecc196be00 1 -- 10.20.193.28:0/2238842550 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cef390e680 con 0x55cef3a7e140 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.831+0000 7fec72ffd640 1 -- 10.20.193.28:0/2238842550 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7feca802a9c0 con 0x55cef3a7e140 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.831+0000 7fecc196be00 10 client.27214 _ll_register_callbacks cb 0x55cef390b9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.831+0000 7fecc196be00 1 -- 10.20.193.28:0/2238842550 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb403f040 con 0x55cef3a7e140 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.831+0000 7fec72ffd640 1 -- 10.20.193.28:0/2238842550 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7feca8011de0 con 0x55cef3a7e140 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.831+0000 7fec72ffd640 1 -- 10.20.193.28:0/2238842550 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7feca801dc20 con 0x55cef3a7e140 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.831+0000 7fec72ffd640 1 -- 10.20.193.28:0/2238842550 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7feca8011de0 con 0x55cef3a7e140 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.832+0000 7fecc196be00 10 client.27214 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.832+0000 7fecc196be00 1 -- 10.20.193.28:0/2238842550 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55cef3a60dd0 con 0x55cef3a7e140 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.832+0000 7fec72ffd640 1 -- 10.20.193.28:0/2238842550 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7feca8028040 con 0x55cef3a7e140 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.832+0000 7fecc196be00 10 client.27214 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.832+0000 7fecc196be00 10 client.27214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.832+0000 7fecc196be00 20 client.27214 populate_metadata read hostname 'trial028' 2026-02-25T11:58:20.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.832+0000 7fecc196be00 1 --2- 10.20.193.28:0/2238842550 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cef3a613f0 0x55cef3a61890 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:20.833 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.832+0000 7fecc196be00 4 client.27214 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:58:20.833 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.832+0000 7fecc196be00 1 -- 10.20.193.28:0/2238842550 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cef361df40 con 0x55cef3a613f0 2026-02-25T11:58:20.833 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.832+0000 7fecbcee4640 1 --2- 10.20.193.28:0/2238842550 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cef3a613f0 0x55cef3a61890 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:20.833 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.833+0000 7fecbcee4640 1 --2- 10.20.193.28:0/2238842550 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cef3a613f0 0x55cef3a61890 crc :-1 s=READY pgs=177 gs=4 cs=0 l=0 c_cookie=14183104731484171734 s_cookie=15486689778843543513 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d4829ba446add6 server_cookie=d6ebc64d46a467d9 in_seq=0 out_seq=0 2026-02-25T11:58:20.833 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.833+0000 7fec72ffd640 10 client.27214 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:20.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.833+0000 7fec72ffd640 1 -- 10.20.193.28:0/2238842550 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5770 (crc 0 0 0) 0x55cef361df40 con 0x55cef3a613f0 2026-02-25T11:58:20.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.833+0000 7fec72ffd640 10 client.27214 handle_command_reply: tid=0 2026-02-25T11:58:20.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.833+0000 7fecc196be00 1 client.27214 shutdown 2026-02-25T11:58:20.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.833+0000 7fecc196be00 1 -- 10.20.193.28:0/2238842550 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3a7e140 msgr2=0x55cef3ac16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:20.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.833+0000 7fecc196be00 1 --2- 10.20.193.28:0/2238842550 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3a7e140 0x55cef3ac16a0 secure :-1 s=READY pgs=233 gs=1 cs=0 l=1 c_cookie=2408799387551698113 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feca802ad80 tx=0x7feca8005780 comp rx=0 tx=0).stop 2026-02-25T11:58:20.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fecc196be00 1 -- 10.20.193.28:0/2238842550 shutdown_connections 2026-02-25T11:58:20.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fecc196be00 1 --2- 10.20.193.28:0/2238842550 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x55cef3a613f0 0x55cef3a61890 crc :-1 s=READY pgs=177 gs=4 cs=0 l=0 c_cookie=14183104731484171734 s_cookie=15486689778843543513 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fecc196be00 1 --2- 10.20.193.28:0/2238842550 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55cef3ac36f0 0x55cef3a5f250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fecc196be00 1 --2- 10.20.193.28:0/2238842550 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55cef3a7ea50 0x55cef3ac1be0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fecc196be00 1 --2- 10.20.193.28:0/2238842550 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55cef3a7e140 0x55cef3ac16a0 unknown :-1 s=CLOSED pgs=233 gs=1 cs=0 l=1 c_cookie=2408799387551698113 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fec72ffd640 0 client.27214 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:20.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fecc196be00 1 -- 10.20.193.28:0/2238842550 >> 10.20.193.28:0/2238842550 conn(0x55cef399dd40 msgr2=0x55cef3ad9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:20.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fecc196be00 1 -- 10.20.193.28:0/2238842550 shutdown_connections 2026-02-25T11:58:20.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fecc196be00 1 -- 10.20.193.28:0/2238842550 wait complete. 2026-02-25T11:58:20.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fecc196be00 20 client.27214 trim_cache size 0 max 16384 2026-02-25T11:58:20.836 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:58:20.836 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:58:20.836 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:58:20.836 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:58:20.836 INFO:teuthology.orchestra.run.trial028.stdout: "52c38620-9fde-44bc-acbe-a9a9e3452e53": { 2026-02-25T11:58:20.836 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:57:49.795729+0000", 2026-02-25T11:58:20.836 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.836 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.836 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.836 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.836 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8": { 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:05.873000+0000", 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "850bcfd3-be8a-4930-896c-1ed6e48b8fed": { 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:27.124813+0000", 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.837 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "516f576f-224f-4d5c-9352-d469d781f97c": { 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:57:15.169491+0000", 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:58:20.838 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.839 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "91e989b7-1f1f-41fb-9047-642867de91e6": { 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:40.578257+0000", 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.840 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "7b237934-6b95-43dd-b3a6-82b74134ec55": { 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:31.289725+0000", 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:20.841 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fecbe6e7640 1 -- 10.20.193.28:0/2577331634 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec940b56d0 msgr2=0x7fec940d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fecbe6e7640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec940b56d0 0x7fec940d5b80 secure :-1 s=READY pgs=391 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7feca8004740 tx=0x7feca8005f90 comp rx=0 tx=0).stop 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fecbe6e7640 1 -- 10.20.193.28:0/2577331634 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fecb810ddc0 msgr2=0x7fecb8173e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.834+0000 7fecbe6e7640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fecb810ddc0 0x7fecb8173e90 secure :-1 s=READY pgs=231 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fecac009f90 tx=0x7fecac00ede0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.835+0000 7fecbe6e7640 1 -- 10.20.193.28:0/2577331634 shutdown_connections 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.835+0000 7fecbe6e7640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fec940b56d0 0x7fec940d5b80 unknown :-1 s=CLOSED pgs=391 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.835+0000 7fecbe6e7640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fecb81135c0 0x7fecb80f6060 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.835+0000 7fecbe6e7640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fecb810ddc0 0x7fecb8173e90 unknown :-1 s=CLOSED pgs=231 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.835+0000 7fecbe6e7640 1 --2- 10.20.193.28:0/2577331634 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fecb810d3d0 0x7fecb80f5960 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.835+0000 7fecbe6e7640 1 -- 10.20.193.28:0/2577331634 >> 10.20.193.28:0/2577331634 conn(0x7fecb80eaca0 msgr2=0x7fecb810b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.836+0000 7fecbe6e7640 1 -- 10.20.193.28:0/2577331634 shutdown_connections 2026-02-25T11:58:20.842 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:20.836+0000 7fecbe6e7640 1 -- 10.20.193.28:0/2577331634 wait complete. 2026-02-25T11:58:20.937 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:58:20.938 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:58:21.128 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:58:21.318 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.317+0000 7ff9c61ee640 1 -- 10.20.193.28:0/4205528258 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9c010f840 msgr2=0x7ff9c010fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.317+0000 7ff9c61ee640 1 --2- 10.20.193.28:0/4205528258 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9c010f840 0x7ff9c010fc10 secure :-1 s=READY pgs=886 gs=1 cs=0 l=1 c_cookie=8262051860032344168 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff9ac009950 tx=0x7ff9ac02f190 comp rx=0 tx=0).stop 2026-02-25T11:58:21.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.317+0000 7ff9c61ee640 1 -- 10.20.193.28:0/4205528258 shutdown_connections 2026-02-25T11:58:21.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.317+0000 7ff9c61ee640 1 --2- 10.20.193.28:0/4205528258 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff9c0115a30 0x7ff9c0176400 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.317+0000 7ff9c61ee640 1 --2- 10.20.193.28:0/4205528258 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff9c0110230 0x7ff9c0115280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.317+0000 7ff9c61ee640 1 --2- 10.20.193.28:0/4205528258 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9c010f840 0x7ff9c010fc10 unknown :-1 s=CLOSED pgs=886 gs=1 cs=0 l=1 c_cookie=8262051860032344168 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.317+0000 7ff9c61ee640 1 -- 10.20.193.28:0/4205528258 >> 10.20.193.28:0/4205528258 conn(0x7ff9c00ed130 msgr2=0x7ff9c010d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:21.319 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.317+0000 7ff9c61ee640 1 -- 10.20.193.28:0/4205528258 shutdown_connections 2026-02-25T11:58:21.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.317+0000 7ff9c61ee640 1 -- 10.20.193.28:0/4205528258 wait complete. 2026-02-25T11:58:21.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.317+0000 7ff9c61ee640 1 Processor -- start 2026-02-25T11:58:21.320 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.318+0000 7ff9c61ee640 1 -- start start 2026-02-25T11:58:21.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.318+0000 7ff9c61ee640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9c010f840 0x7ff9c011e190 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.318+0000 7ff9c61ee640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff9c0110230 0x7ff9c011e6d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.318+0000 7ff9c61ee640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff9c0115a30 0x7ff9c0123aa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.318+0000 7ff9c61ee640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01240c0 con 0x7ff9c010f840 2026-02-25T11:58:21.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.318+0000 7ff9c61ee640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c0124230 con 0x7ff9c0110230 2026-02-25T11:58:21.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.318+0000 7ff9c61ee640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c01243f0 con 0x7ff9c0115a30 2026-02-25T11:58:21.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.318+0000 7ff9bf7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9c010f840 0x7ff9c011e190 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.318+0000 7ff9bf7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9c010f840 0x7ff9c011e190 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52184/0 (socket says 10.20.193.28:52184) 2026-02-25T11:58:21.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.318+0000 7ff9bf7fe640 1 -- 10.20.193.28:0/2404303061 learned_addr learned my addr 10.20.193.28:0/2404303061 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:21.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.318+0000 7ff9beffd640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff9c0110230 0x7ff9c011e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.318+0000 7ff9bffff640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff9c0115a30 0x7ff9c0123aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.319+0000 7ff9bf7fe640 1 -- 10.20.193.28:0/2404303061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff9c0115a30 msgr2=0x7ff9c0123aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.319+0000 7ff9bf7fe640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff9c0115a30 0x7ff9c0123aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.319+0000 7ff9bf7fe640 1 -- 10.20.193.28:0/2404303061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff9c0110230 msgr2=0x7ff9c011e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.319+0000 7ff9bf7fe640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff9c0110230 0x7ff9c011e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.319+0000 7ff9bf7fe640 1 -- 10.20.193.28:0/2404303061 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9ac009570 con 0x7ff9c010f840 2026-02-25T11:58:21.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.319+0000 7ff9bf7fe640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9c010f840 0x7ff9c011e190 secure :-1 s=READY pgs=887 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff9ac0098a0 tx=0x7ff9ac0028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:21.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.319+0000 7ff9bcff9640 1 -- 10.20.193.28:0/2404303061 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff9ac03d070 con 0x7ff9c010f840 2026-02-25T11:58:21.323 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.319+0000 7ff9bcff9640 1 -- 10.20.193.28:0/2404303061 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ff9ac002a70 con 0x7ff9c010f840 2026-02-25T11:58:21.324 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.319+0000 7ff9bcff9640 1 -- 10.20.193.28:0/2404303061 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ff9ac041890 con 0x7ff9c010f840 2026-02-25T11:58:21.324 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.319+0000 7ff9c61ee640 1 -- 10.20.193.28:0/2404303061 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c0124670 con 0x7ff9c010f840 2026-02-25T11:58:21.324 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.319+0000 7ff9c61ee640 1 -- 10.20.193.28:0/2404303061 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c0164dd0 con 0x7ff9c010f840 2026-02-25T11:58:21.324 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.320+0000 7ff9c61ee640 1 -- 10.20.193.28:0/2404303061 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c0110b20 con 0x7ff9c010f840 2026-02-25T11:58:21.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.322+0000 7ff9bcff9640 1 -- 10.20.193.28:0/2404303061 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7ff9ac041a20 con 0x7ff9c010f840 2026-02-25T11:58:21.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.324+0000 7ff9bcff9640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff9940b5760 0x7ff9940d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.324+0000 7ff9bcff9640 1 -- 10.20.193.28:0/2404303061 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ff9ac10b420 con 0x7ff9c010f840 2026-02-25T11:58:21.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.324+0000 7ff9beffd640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff9940b5760 0x7ff9940d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.324+0000 7ff9bcff9640 1 -- 10.20.193.28:0/2404303061 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7ff9ac10b8a0 con 0x7ff9c010f840 2026-02-25T11:58:21.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.324+0000 7ff9beffd640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff9940b5760 0x7ff9940d5c10 secure :-1 s=READY pgs=392 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff9c0124b30 tx=0x7ff9b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:21.467 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:21 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:58:21.467 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:21 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:58:21.467 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:21 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:58:21.467 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:21 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:58:21.468 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:21 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:58:21.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.466+0000 7ff9c61ee640 1 -- 10.20.193.28:0/2404303061 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9c011f2f0 con 0x7ff9c010f840 2026-02-25T11:58:21.468 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:58:21.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.468+0000 7ff9bcff9640 1 -- 10.20.193.28:0/2404303061 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7ff9ac0cddf0 con 0x7ff9c010f840 2026-02-25T11:58:21.469 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:58:21.469 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:58:21.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.469+0000 7ff9c61ee640 1 -- 10.20.193.28:0/2404303061 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff9940b5760 msgr2=0x7ff9940d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.469+0000 7ff9c61ee640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff9940b5760 0x7ff9940d5c10 secure :-1 s=READY pgs=392 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff9c0124b30 tx=0x7ff9b000b410 comp rx=0 tx=0).stop 2026-02-25T11:58:21.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.469+0000 7ff9c61ee640 1 -- 10.20.193.28:0/2404303061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9c010f840 msgr2=0x7ff9c011e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.469+0000 7ff9c61ee640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9c010f840 0x7ff9c011e190 secure :-1 s=READY pgs=887 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ff9ac0098a0 tx=0x7ff9ac0028d0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.469+0000 7ff9c61ee640 1 -- 10.20.193.28:0/2404303061 shutdown_connections 2026-02-25T11:58:21.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.469+0000 7ff9c61ee640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ff9940b5760 0x7ff9940d5c10 unknown :-1 s=CLOSED pgs=392 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.469+0000 7ff9c61ee640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ff9c0115a30 0x7ff9c0123aa0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.469+0000 7ff9c61ee640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ff9c0110230 0x7ff9c011e6d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.469+0000 7ff9c61ee640 1 --2- 10.20.193.28:0/2404303061 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ff9c010f840 0x7ff9c011e190 unknown :-1 s=CLOSED pgs=887 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.469+0000 7ff9c61ee640 1 -- 10.20.193.28:0/2404303061 >> 10.20.193.28:0/2404303061 conn(0x7ff9c00ed130 msgr2=0x7ff9c010d510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:21.470 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.469+0000 7ff9c61ee640 1 -- 10.20.193.28:0/2404303061 shutdown_connections 2026-02-25T11:58:21.471 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.470+0000 7ff9c61ee640 1 -- 10.20.193.28:0/2404303061 wait complete. 2026-02-25T11:58:21.569 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:58:21.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:21 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:58:21.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:21 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:58:21.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:21 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:58:21.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:21 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:58:21.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:21 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:58:21.754 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:58:21.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:21 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2026-02-25T11:58:21.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:21 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2026-02-25T11:58:21.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:21 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2026-02-25T11:58:21.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:21 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' 2026-02-25T11:58:21.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:21 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2026-02-25T11:58:21.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.925+0000 7fd3b936a640 1 -- 10.20.193.28:0/1708514757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd3b41115d0 msgr2=0x7fd3b4111a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.925+0000 7fd3b936a640 1 --2- 10.20.193.28:0/1708514757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd3b41115d0 0x7fd3b4111a70 secure :-1 s=READY pgs=234 gs=1 cs=0 l=1 c_cookie=2039165920830000434 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd3a400a970 tx=0x7fd3a40301b0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.925+0000 7fd3b936a640 1 -- 10.20.193.28:0/1708514757 shutdown_connections 2026-02-25T11:58:21.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.925+0000 7fd3b936a640 1 --2- 10.20.193.28:0/1708514757 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd3b41115d0 0x7fd3b4111a70 unknown :-1 s=CLOSED pgs=234 gs=1 cs=0 l=1 c_cookie=2039165920830000434 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.925+0000 7fd3b936a640 1 --2- 10.20.193.28:0/1708514757 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd3b410c9e0 0x7fd3b4110fb0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.925+0000 7fd3b936a640 1 --2- 10.20.193.28:0/1708514757 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd3b410bff0 0x7fd3b410c3c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.925+0000 7fd3b936a640 1 -- 10.20.193.28:0/1708514757 >> 10.20.193.28:0/1708514757 conn(0x7fd3b40e9cc0 msgr2=0x7fd3b410a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:21.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.925+0000 7fd3b936a640 1 -- 10.20.193.28:0/1708514757 shutdown_connections 2026-02-25T11:58:21.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.925+0000 7fd3b936a640 1 -- 10.20.193.28:0/1708514757 wait complete. 2026-02-25T11:58:21.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.926+0000 7fd3b936a640 1 Processor -- start 2026-02-25T11:58:21.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.926+0000 7fd3b936a640 1 -- start start 2026-02-25T11:58:21.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b936a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd3b410bff0 0x7fd3b41229e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b936a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd3b410c9e0 0x7fd3b4122f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b936a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd3b41115d0 0x7fd3b41282f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b936a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4128910 con 0x7fd3b410c9e0 2026-02-25T11:58:21.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b936a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4128a80 con 0x7fd3b410bff0 2026-02-25T11:58:21.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b936a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b4128c40 con 0x7fd3b41115d0 2026-02-25T11:58:21.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b27fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd3b410c9e0 0x7fd3b4122f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b27fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd3b410c9e0 0x7fd3b4122f20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52206/0 (socket says 10.20.193.28:52206) 2026-02-25T11:58:21.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b27fc640 1 -- 10.20.193.28:0/2865133557 learned_addr learned my addr 10.20.193.28:0/2865133557 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:21.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b2ffd640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd3b410bff0 0x7fd3b41229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b37fe640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd3b41115d0 0x7fd3b41282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b27fc640 1 -- 10.20.193.28:0/2865133557 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd3b41115d0 msgr2=0x7fd3b41282f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd3b41115d0 0x7fd3b41282f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b27fc640 1 -- 10.20.193.28:0/2865133557 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd3b410bff0 msgr2=0x7fd3b41229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd3b410bff0 0x7fd3b41229e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b27fc640 1 -- 10.20.193.28:0/2865133557 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a400a590 con 0x7fd3b410c9e0 2026-02-25T11:58:21.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd3b410c9e0 0x7fd3b4122f20 secure :-1 s=READY pgs=888 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd39c00e9c0 tx=0x7fd39c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:21.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.927+0000 7fd393fff640 1 -- 10.20.193.28:0/2865133557 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd39c00cd70 con 0x7fd3b410c9e0 2026-02-25T11:58:21.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.928+0000 7fd393fff640 1 -- 10.20.193.28:0/2865133557 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd39c004d10 con 0x7fd3b410c9e0 2026-02-25T11:58:21.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.928+0000 7fd393fff640 1 -- 10.20.193.28:0/2865133557 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd39c0106b0 con 0x7fd3b410c9e0 2026-02-25T11:58:21.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.928+0000 7fd3b936a640 1 -- 10.20.193.28:0/2865133557 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b4128f20 con 0x7fd3b410c9e0 2026-02-25T11:58:21.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.928+0000 7fd3b936a640 1 -- 10.20.193.28:0/2865133557 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b4169620 con 0x7fd3b410c9e0 2026-02-25T11:58:21.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.931+0000 7fd393fff640 1 -- 10.20.193.28:0/2865133557 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fd39c01f060 con 0x7fd3b410c9e0 2026-02-25T11:58:21.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.932+0000 7fd393fff640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd3880b5760 0x7fd3880d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.933+0000 7fd393fff640 1 -- 10.20.193.28:0/2865133557 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd39c014070 con 0x7fd3b410c9e0 2026-02-25T11:58:21.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.933+0000 7fd3b2ffd640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd3880b5760 0x7fd3880d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.933+0000 7fd3bb365e00 1 Processor -- start 2026-02-25T11:58:21.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.933+0000 7fd3bb365e00 1 -- start start 2026-02-25T11:58:21.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.933+0000 7fd3b2ffd640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd3880b5760 0x7fd3880d5c10 secure :-1 s=READY pgs=393 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd3a8004740 tx=0x7fd3a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:21.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.933+0000 7fd3bb365e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96ccb30 0x558fc96ccf00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.933+0000 7fd3bb365e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 0x558fc9711ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.933+0000 7fd3bb365e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 0x558fc97324c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.933+0000 7fd3bb365e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558fc95e28a0 con 0x558fc96ccb30 2026-02-25T11:58:21.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.933+0000 7fd3bb365e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558fc95de5d0 con 0x558fc97120e0 2026-02-25T11:58:21.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.933+0000 7fd3bb365e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558fc95d8570 con 0x558fc96cd440 2026-02-25T11:58:21.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.934+0000 7fd3b27fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 0x558fc9711ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.934+0000 7fd3b27fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 0x558fc9711ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52582/0 (socket says 10.20.193.28:52582) 2026-02-25T11:58:21.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.934+0000 7fd3b27fc640 1 -- 10.20.193.28:0/1250710212 learned_addr learned my addr 10.20.193.28:0/1250710212 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:21.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.934+0000 7fd3b2ffd640 1 --2- 10.20.193.28:0/1250710212 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96ccb30 0x558fc96ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.934+0000 7fd3b37fe640 1 --2- 10.20.193.28:0/1250710212 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 0x558fc97324c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.934+0000 7fd3b27fc640 1 -- 10.20.193.28:0/1250710212 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 msgr2=0x558fc97324c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:21.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.934+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/1250710212 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 0x558fc97324c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.934+0000 7fd3b27fc640 1 -- 10.20.193.28:0/1250710212 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96ccb30 msgr2=0x558fc96ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:21.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.934+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/1250710212 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96ccb30 0x558fc96ccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.934+0000 7fd3b27fc640 1 -- 10.20.193.28:0/1250710212 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc9583d10 con 0x558fc96cd440 2026-02-25T11:58:21.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.934+0000 7fd3b2ffd640 1 --2- 10.20.193.28:0/1250710212 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96ccb30 0x558fc96ccf00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:58:21.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.934+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/1250710212 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 0x558fc9711ba0 secure :-1 s=READY pgs=235 gs=2 cs=0 l=1 c_cookie=1376754317723397972 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd39c0a8dc0 tx=0x7fd39c0acf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=131b36c1f2e36f54 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:21.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.935+0000 7fd37f7fe640 1 -- 10.20.193.28:0/1250710212 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd39c0b2bc0 con 0x558fc96cd440 2026-02-25T11:58:21.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.935+0000 7fd37f7fe640 1 -- 10.20.193.28:0/1250710212 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd39c0b51a0 con 0x558fc96cd440 2026-02-25T11:58:21.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.935+0000 7fd37f7fe640 1 -- 10.20.193.28:0/1250710212 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd39c0a9110 con 0x558fc96cd440 2026-02-25T11:58:21.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.935+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1250710212 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 msgr2=0x558fc9711ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.935+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/1250710212 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 0x558fc9711ba0 secure :-1 s=READY pgs=235 gs=2 cs=0 l=1 c_cookie=1376754317723397972 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd39c0a8dc0 tx=0x7fd39c0acf50 comp rx=0 tx=0).stop 2026-02-25T11:58:21.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.935+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1250710212 shutdown_connections 2026-02-25T11:58:21.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.935+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/1250710212 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 0x558fc97324c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.935+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/1250710212 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 0x558fc9711ba0 unknown :-1 s=CLOSED pgs=235 gs=2 cs=0 l=1 c_cookie=1376754317723397972 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.935+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/1250710212 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96ccb30 0x558fc96ccf00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.935+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1250710212 >> 10.20.193.28:0/1250710212 conn(0x558fc95eb7a0 msgr2=0x558fc96cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:21.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.935+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1250710212 shutdown_connections 2026-02-25T11:58:21.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.935+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1250710212 wait complete. 2026-02-25T11:58:21.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.936+0000 7fd3bb365e00 1 -- start start 2026-02-25T11:58:21.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.937+0000 7fd3bb365e00 1 Processor -- start 2026-02-25T11:58:21.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.937+0000 7fd3bb365e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc96ccb30 0x558fc96e4d50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.937+0000 7fd3bb365e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96cd440 0x558fc96e5290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.937+0000 7fd3bb365e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc97120e0 0x558fc96ea280 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.937+0000 7fd3bb365e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558fc96ea7c0 con 0x558fc96cd440 2026-02-25T11:58:21.944 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.937+0000 7fd3bb365e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558fc96ea930 con 0x558fc96ccb30 2026-02-25T11:58:21.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd3bb365e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558fc96eaaf0 con 0x558fc97120e0 2026-02-25T11:58:21.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd3b27fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96cd440 0x558fc96e5290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd3b2ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc96ccb30 0x558fc96e4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd3b2ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc96ccb30 0x558fc96e4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:59028/0 (socket says 10.20.193.28:59028) 2026-02-25T11:58:21.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd3b2ffd640 1 -- 10.20.193.28:0/2528117364 learned_addr learned my addr 10.20.193.28:0/2528117364 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:21.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd3b37fe640 1 --2- 10.20.193.28:0/2528117364 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc97120e0 0x558fc96ea280 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd3b2ffd640 1 -- 10.20.193.28:0/2528117364 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc97120e0 msgr2=0x558fc96ea280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:21.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd3b2ffd640 1 --2- 10.20.193.28:0/2528117364 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc97120e0 0x558fc96ea280 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.945 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd3b2ffd640 1 -- 10.20.193.28:0/2528117364 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96cd440 msgr2=0x558fc96e5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:21.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd3b2ffd640 1 --2- 10.20.193.28:0/2528117364 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96cd440 0x558fc96e5290 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd3b2ffd640 1 -- 10.20.193.28:0/2528117364 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b4169620 con 0x558fc96ccb30 2026-02-25T11:58:21.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/2528117364 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96cd440 0x558fc96e5290 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:58:21.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd3b2ffd640 1 --2- 10.20.193.28:0/2528117364 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc96ccb30 0x558fc96e4d50 secure :-1 s=READY pgs=181 gs=2 cs=0 l=1 c_cookie=4509445332196738369 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd3a8004480 tx=0x7fd3a8013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e94c4a337a14541 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:21.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.938+0000 7fd37cff9640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:58:21.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.939+0000 7fd37cff9640 1 -- 10.20.193.28:0/2528117364 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd3a8021040 con 0x558fc96ccb30 2026-02-25T11:58:21.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.939+0000 7fd37cff9640 1 -- 10.20.193.28:0/2528117364 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd3a801bd70 con 0x558fc96ccb30 2026-02-25T11:58:21.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.939+0000 7fd37cff9640 1 -- 10.20.193.28:0/2528117364 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd3a80258d0 con 0x558fc96ccb30 2026-02-25T11:58:21.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.939+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2528117364 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a8009900 con 0x558fc96ccb30 2026-02-25T11:58:21.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.939+0000 7fd37cff9640 1 -- 10.20.193.28:0/2528117364 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd3a8025a60 con 0x558fc96ccb30 2026-02-25T11:58:21.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.939+0000 7fd37cff9640 10 client.27227 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:58:21.946 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.939+0000 7fd3bb365e00 10 client.27227 _ll_register_callbacks cb 0x558fc9559a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:21.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.939+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2528117364 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558fc926e6f0 con 0x558fc96ccb30 2026-02-25T11:58:21.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.939+0000 7fd37cff9640 1 -- 10.20.193.28:0/2528117364 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd3a8005020 con 0x558fc96ccb30 2026-02-25T11:58:21.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.939+0000 7fd3bb365e00 10 client.27227 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:21.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.939+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2528117364 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x558fc955c3c0 con 0x558fc96ccb30 2026-02-25T11:58:21.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.940+0000 7fd37cff9640 1 -- 10.20.193.28:0/2528117364 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd3a80051c0 con 0x558fc96ccb30 2026-02-25T11:58:21.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.940+0000 7fd3bb365e00 10 client.27227 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:21.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.940+0000 7fd3bb365e00 10 client.27227 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:58:21.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.940+0000 7fd3bb365e00 20 client.27227 populate_metadata read hostname 'trial028' 2026-02-25T11:58:21.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.940+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/2528117364 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x558fc970f600 0x558fc970faa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.947 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.940+0000 7fd3bb365e00 4 client.27227 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:58:21.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.940+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2528117364 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fc95aafc0 con 0x558fc970f600 2026-02-25T11:58:21.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.940+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/2528117364 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x558fc970f600 0x558fc970faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.941+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/2528117364 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x558fc970f600 0x558fc970faa0 crc :-1 s=READY pgs=61 gs=4 cs=0 l=0 c_cookie=7827927790757861553 s_cookie=3899682883146828552 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ca262d9ff87d8b1 server_cookie=361e74ebbaa34b08 in_seq=0 out_seq=0 2026-02-25T11:58:21.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.941+0000 7fd37cff9640 10 client.27227 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:58:21.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.941+0000 7fd37cff9640 1 -- 10.20.193.28:0/2528117364 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x558fc95aafc0 con 0x558fc970f600 2026-02-25T11:58:21.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.941+0000 7fd37cff9640 10 client.27227 handle_command_reply: tid=0 2026-02-25T11:58:21.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.941+0000 7fd3bb365e00 1 client.27227 shutdown 2026-02-25T11:58:21.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.941+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2528117364 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc96ccb30 msgr2=0x558fc96e4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.941+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/2528117364 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc96ccb30 0x558fc96e4d50 secure :-1 s=READY pgs=181 gs=2 cs=0 l=1 c_cookie=4509445332196738369 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd3a8004480 tx=0x7fd3a8013040 comp rx=0 tx=0).stop 2026-02-25T11:58:21.948 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.942+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2528117364 shutdown_connections 2026-02-25T11:58:21.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.942+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/2528117364 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x558fc970f600 0x558fc970faa0 crc :-1 s=READY pgs=61 gs=4 cs=0 l=0 c_cookie=7827927790757861553 s_cookie=3899682883146828552 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.942+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/2528117364 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc97120e0 0x558fc96ea280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.942+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/2528117364 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96cd440 0x558fc96e5290 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.942+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/2528117364 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc96ccb30 0x558fc96e4d50 unknown :-1 s=CLOSED pgs=181 gs=2 cs=0 l=1 c_cookie=4509445332196738369 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.942+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2528117364 >> 10.20.193.28:0/2528117364 conn(0x558fc95eb7a0 msgr2=0x558fc96cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:21.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.942+0000 7fd37cff9640 0 client.27227 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:58:21.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.942+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2528117364 shutdown_connections 2026-02-25T11:58:21.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.942+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2528117364 wait complete. 2026-02-25T11:58:21.949 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.942+0000 7fd3bb365e00 20 client.27227 trim_cache size 0 max 16384 2026-02-25T11:58:21.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.949+0000 7fd3bb365e00 1 Processor -- start 2026-02-25T11:58:21.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.949+0000 7fd3bb365e00 1 -- start start 2026-02-25T11:58:21.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3bb365e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96ccb30 0x558fc96cbd20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3bb365e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96cd440 0x558fc96cc260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3bb365e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 0x558fc970a760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3bb365e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558fc95d61a0 con 0x558fc96cd440 2026-02-25T11:58:21.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3bb365e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558fc9296700 con 0x558fc97120e0 2026-02-25T11:58:21.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3bb365e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558fc96d9d80 con 0x558fc96ccb30 2026-02-25T11:58:21.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3b27fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96cd440 0x558fc96cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3b27fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96cd440 0x558fc96cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52230/0 (socket says 10.20.193.28:52230) 2026-02-25T11:58:21.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3b27fc640 1 -- 10.20.193.28:0/2229084306 learned_addr learned my addr 10.20.193.28:0/2229084306 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:21.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3b2ffd640 1 --2- 10.20.193.28:0/2229084306 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96ccb30 0x558fc96cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3b27fc640 1 -- 10.20.193.28:0/2229084306 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96ccb30 msgr2=0x558fc96cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:21.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/2229084306 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96ccb30 0x558fc96cbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3b37fe640 1 --2- 10.20.193.28:0/2229084306 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 0x558fc970a760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3b27fc640 1 -- 10.20.193.28:0/2229084306 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 msgr2=0x558fc970a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:21.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/2229084306 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 0x558fc970a760 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3b27fc640 1 -- 10.20.193.28:0/2229084306 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc956e0e0 con 0x558fc96cd440 2026-02-25T11:58:21.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3b37fe640 1 --2- 10.20.193.28:0/2229084306 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 0x558fc970a760 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:58:21.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/2229084306 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96cd440 0x558fc96cc260 secure :-1 s=READY pgs=889 gs=1 cs=0 l=1 c_cookie=9986121597840744776 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd39c0a9470 tx=0x7fd39c096040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a95d4af3b276148 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:21.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd37effd640 1 -- 10.20.193.28:0/2229084306 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd39c0b2bc0 con 0x558fc96cd440 2026-02-25T11:58:21.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.950+0000 7fd37effd640 1 -- 10.20.193.28:0/2229084306 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd39c0b7fe0 con 0x558fc96cd440 2026-02-25T11:58:21.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.951+0000 7fd37effd640 1 -- 10.20.193.28:0/2229084306 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd39c0051d0 con 0x558fc96cd440 2026-02-25T11:58:21.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.951+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2229084306 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96cd440 msgr2=0x558fc96cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.951+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/2229084306 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96cd440 0x558fc96cc260 secure :-1 s=READY pgs=889 gs=1 cs=0 l=1 c_cookie=9986121597840744776 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd39c0a9470 tx=0x7fd39c096040 comp rx=0 tx=0).stop 2026-02-25T11:58:21.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.951+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2229084306 shutdown_connections 2026-02-25T11:58:21.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.951+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/2229084306 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 0x558fc970a760 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.951+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/2229084306 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96cd440 0x558fc96cc260 unknown :-1 s=CLOSED pgs=889 gs=1 cs=0 l=1 c_cookie=9986121597840744776 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.951+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/2229084306 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96ccb30 0x558fc96cbd20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.951+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2229084306 >> 10.20.193.28:0/2229084306 conn(0x558fc95eb7a0 msgr2=0x558fc96cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:21.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.951+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2229084306 shutdown_connections 2026-02-25T11:58:21.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.951+0000 7fd3bb365e00 1 -- 10.20.193.28:0/2229084306 wait complete. 2026-02-25T11:58:21.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.951+0000 7fd3bb365e00 1 -- start start 2026-02-25T11:58:21.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3bb365e00 1 Processor -- start 2026-02-25T11:58:21.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3bb365e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96ccb30 0x558fc9711660 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3bb365e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 0x558fc9711ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3bb365e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 0x558fc96eb010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3bb365e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x558fc96eb550 con 0x558fc96ccb30 2026-02-25T11:58:21.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3bb365e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x558fc96eb6c0 con 0x558fc97120e0 2026-02-25T11:58:21.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3bb365e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x558fc96eb880 con 0x558fc96cd440 2026-02-25T11:58:21.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3b27fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 0x558fc9711ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3b2ffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96ccb30 0x558fc9711660 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3b27fc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 0x558fc9711ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52612/0 (socket says 10.20.193.28:52612) 2026-02-25T11:58:21.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3b27fc640 1 -- 10.20.193.28:0/1458074431 learned_addr learned my addr 10.20.193.28:0/1458074431 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:21.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3b37fe640 1 --2- 10.20.193.28:0/1458074431 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 0x558fc96eb010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.959 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3b27fc640 1 -- 10.20.193.28:0/1458074431 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 msgr2=0x558fc96eb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:21.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/1458074431 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 0x558fc96eb010 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.952+0000 7fd3b27fc640 1 -- 10.20.193.28:0/1458074431 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96ccb30 msgr2=0x558fc9711660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:21.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.953+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/1458074431 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96ccb30 0x558fc9711660 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.953+0000 7fd3b27fc640 1 -- 10.20.193.28:0/1458074431 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc9568b60 con 0x558fc96cd440 2026-02-25T11:58:21.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.953+0000 7fd3b2ffd640 1 --2- 10.20.193.28:0/1458074431 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96ccb30 0x558fc9711660 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:58:21.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.953+0000 7fd3b27fc640 1 --2- 10.20.193.28:0/1458074431 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 0x558fc9711ba0 secure :-1 s=READY pgs=236 gs=1 cs=0 l=1 c_cookie=13597878946149076149 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd39c0a9470 tx=0x7fd39c0e46f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb55edb5bfa8cb5 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:21.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.953+0000 7fd37cff9640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:58:21.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.953+0000 7fd37cff9640 1 -- 10.20.193.28:0/1458074431 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd39c0051b0 con 0x558fc96cd440 2026-02-25T11:58:21.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.953+0000 7fd37cff9640 1 -- 10.20.193.28:0/1458074431 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd39c0ac880 con 0x558fc96cd440 2026-02-25T11:58:21.960 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.953+0000 7fd37cff9640 1 -- 10.20.193.28:0/1458074431 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd39c0aae80 con 0x558fc96cd440 2026-02-25T11:58:21.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.953+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1458074431 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fc9583d10 con 0x558fc96cd440 2026-02-25T11:58:21.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.953+0000 7fd37cff9640 1 -- 10.20.193.28:0/1458074431 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd39c0ab010 con 0x558fc96cd440 2026-02-25T11:58:21.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.953+0000 7fd3bb365e00 10 client.27238 _ll_register_callbacks cb 0x558fc9559a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:21.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.953+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1458074431 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd39c0b51a0 con 0x558fc96cd440 2026-02-25T11:58:21.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.954+0000 7fd37cff9640 1 -- 10.20.193.28:0/1458074431 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd39c0b2bc0 con 0x558fc96cd440 2026-02-25T11:58:21.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.954+0000 7fd3bb365e00 10 client.27238 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:21.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.954+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1458074431 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x558fc955c6f0 con 0x558fc96cd440 2026-02-25T11:58:21.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.954+0000 7fd37cff9640 1 -- 10.20.193.28:0/1458074431 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd39c0d2040 con 0x558fc96cd440 2026-02-25T11:58:21.961 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.954+0000 7fd3bb365e00 10 client.27238 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:21.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.954+0000 7fd3bb365e00 10 client.27238 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:58:21.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.954+0000 7fd3bb365e00 20 client.27238 populate_metadata read hostname 'trial028' 2026-02-25T11:58:21.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.954+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/1458074431 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x558fc96f0c80 0x558fc97dec20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:21.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.954+0000 7fd3bb365e00 4 client.27238 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:58:21.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.954+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1458074431 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558fc926bf40 con 0x558fc96f0c80 2026-02-25T11:58:21.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.954+0000 7fd3b2ffd640 1 --2- 10.20.193.28:0/1458074431 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x558fc96f0c80 0x558fc97dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:21.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd3b2ffd640 1 --2- 10.20.193.28:0/1458074431 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x558fc96f0c80 0x558fc97dec20 crc :-1 s=READY pgs=62 gs=4 cs=0 l=0 c_cookie=8880142391002024826 s_cookie=17631549862987948891 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b3c9a74a2904b7a server_cookie=f4afd92f36fbe35b in_seq=0 out_seq=0 2026-02-25T11:58:21.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd37cff9640 10 client.27238 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:58:21.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd37cff9640 1 -- 10.20.193.28:0/1458074431 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558fc926bf40 con 0x558fc96f0c80 2026-02-25T11:58:21.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd37cff9640 10 client.27238 handle_command_reply: tid=0 2026-02-25T11:58:21.962 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd3bb365e00 1 client.27238 shutdown 2026-02-25T11:58:21.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1458074431 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 msgr2=0x558fc9711ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/1458074431 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 0x558fc9711ba0 secure :-1 s=READY pgs=236 gs=1 cs=0 l=1 c_cookie=13597878946149076149 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd39c0a9470 tx=0x7fd39c0e46f0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1458074431 shutdown_connections 2026-02-25T11:58:21.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/1458074431 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x558fc96f0c80 0x558fc97dec20 crc :-1 s=READY pgs=62 gs=4 cs=0 l=0 c_cookie=8880142391002024826 s_cookie=17631549862987948891 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/1458074431 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x558fc97120e0 0x558fc96eb010 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/1458074431 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x558fc96cd440 0x558fc9711ba0 unknown :-1 s=CLOSED pgs=236 gs=1 cs=0 l=1 c_cookie=13597878946149076149 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd3bb365e00 1 --2- 10.20.193.28:0/1458074431 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x558fc96ccb30 0x558fc9711660 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1458074431 >> 10.20.193.28:0/1458074431 conn(0x558fc95eb7a0 msgr2=0x558fc96e90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:21.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd37cff9640 0 client.27238 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:58:21.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1458074431 shutdown_connections 2026-02-25T11:58:21.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd3bb365e00 1 -- 10.20.193.28:0/1458074431 wait complete. 2026-02-25T11:58:21.963 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.955+0000 7fd3bb365e00 20 client.27238 trim_cache size 0 max 16384 2026-02-25T11:58:21.964 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:58:21.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.956+0000 7fd3b936a640 1 -- 10.20.193.28:0/2865133557 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd3880b5760 msgr2=0x7fd3880d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.956+0000 7fd3b936a640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd3880b5760 0x7fd3880d5c10 secure :-1 s=READY pgs=393 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd3a8004740 tx=0x7fd3a8005f90 comp rx=0 tx=0).stop 2026-02-25T11:58:21.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.956+0000 7fd3b936a640 1 -- 10.20.193.28:0/2865133557 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd3b410c9e0 msgr2=0x7fd3b4122f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:21.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.956+0000 7fd3b936a640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd3b410c9e0 0x7fd3b4122f20 secure :-1 s=READY pgs=888 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd39c00e9c0 tx=0x7fd39c00ee90 comp rx=0 tx=0).stop 2026-02-25T11:58:21.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.956+0000 7fd3b936a640 1 -- 10.20.193.28:0/2865133557 shutdown_connections 2026-02-25T11:58:21.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.956+0000 7fd3b936a640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd3880b5760 0x7fd3880d5c10 unknown :-1 s=CLOSED pgs=393 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.956+0000 7fd3b936a640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd3b41115d0 0x7fd3b41282f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.956+0000 7fd3b936a640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd3b410c9e0 0x7fd3b4122f20 unknown :-1 s=CLOSED pgs=888 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.956+0000 7fd3b936a640 1 --2- 10.20.193.28:0/2865133557 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd3b410bff0 0x7fd3b41229e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:21.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.956+0000 7fd3b936a640 1 -- 10.20.193.28:0/2865133557 >> 10.20.193.28:0/2865133557 conn(0x7fd3b40e9cc0 msgr2=0x7fd3b410a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:21.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.956+0000 7fd3b936a640 1 -- 10.20.193.28:0/2865133557 shutdown_connections 2026-02-25T11:58:21.965 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:21.956+0000 7fd3b936a640 1 -- 10.20.193.28:0/2865133557 wait complete. 2026-02-25T11:58:22.054 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:58:22.236 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:58:22.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.414+0000 7fd83ab27640 1 -- 10.20.193.28:0/657329965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd83410f690 msgr2=0x7fd83410fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.414+0000 7fd83ab27640 1 --2- 10.20.193.28:0/657329965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd83410f690 0x7fd83410fa60 secure :-1 s=READY pgs=890 gs=1 cs=0 l=1 c_cookie=3390829470329132529 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd824009950 tx=0x7fd82402f190 comp rx=0 tx=0).stop 2026-02-25T11:58:22.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.414+0000 7fd83ab27640 1 -- 10.20.193.28:0/657329965 shutdown_connections 2026-02-25T11:58:22.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.414+0000 7fd83ab27640 1 --2- 10.20.193.28:0/657329965 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd834115880 0x7fd834176250 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.414+0000 7fd83ab27640 1 --2- 10.20.193.28:0/657329965 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd834110080 0x7fd8341150d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.414+0000 7fd83ab27640 1 --2- 10.20.193.28:0/657329965 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd83410f690 0x7fd83410fa60 unknown :-1 s=CLOSED pgs=890 gs=1 cs=0 l=1 c_cookie=3390829470329132529 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.414+0000 7fd83ab27640 1 -- 10.20.193.28:0/657329965 >> 10.20.193.28:0/657329965 conn(0x7fd8340ecf00 msgr2=0x7fd83410d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:22.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.415+0000 7fd83ab27640 1 -- 10.20.193.28:0/657329965 shutdown_connections 2026-02-25T11:58:22.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.415+0000 7fd83ab27640 1 -- 10.20.193.28:0/657329965 wait complete. 2026-02-25T11:58:22.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.415+0000 7fd83ab27640 1 Processor -- start 2026-02-25T11:58:22.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.415+0000 7fd83ab27640 1 -- start start 2026-02-25T11:58:22.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.415+0000 7fd83ab27640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd83410f690 0x7fd8340f5960 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.415+0000 7fd83ab27640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd834110080 0x7fd834176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.415+0000 7fd83ab27640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd834115880 0x7fd8340f6060 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd83ab27640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd8340fb270 con 0x7fd83410f690 2026-02-25T11:58:22.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd83ab27640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd8340fb3e0 con 0x7fd834110080 2026-02-25T11:58:22.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd83ab27640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8340fb5a0 con 0x7fd834115880 2026-02-25T11:58:22.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd839b25640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd83410f690 0x7fd8340f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd839b25640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd83410f690 0x7fd8340f5960 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52254/0 (socket says 10.20.193.28:52254) 2026-02-25T11:58:22.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd839b25640 1 -- 10.20.193.28:0/3180561311 learned_addr learned my addr 10.20.193.28:0/3180561311 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:22.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd839324640 1 --2- 10.20.193.28:0/3180561311 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd834110080 0x7fd834176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd839b25640 1 -- 10.20.193.28:0/3180561311 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd834115880 msgr2=0x7fd8340f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd839b25640 1 --2- 10.20.193.28:0/3180561311 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd834115880 0x7fd8340f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd839b25640 1 -- 10.20.193.28:0/3180561311 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd834110080 msgr2=0x7fd834176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd839b25640 1 --2- 10.20.193.28:0/3180561311 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd834110080 0x7fd834176150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd839b25640 1 -- 10.20.193.28:0/3180561311 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd824009570 con 0x7fd83410f690 2026-02-25T11:58:22.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd839b25640 1 --2- 10.20.193.28:0/3180561311 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd83410f690 0x7fd8340f5960 secure :-1 s=READY pgs=891 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd8240098a0 tx=0x7fd8240028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:22.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd822ffd640 1 -- 10.20.193.28:0/3180561311 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd82403d070 con 0x7fd83410f690 2026-02-25T11:58:22.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd822ffd640 1 -- 10.20.193.28:0/3180561311 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd824002a70 con 0x7fd83410f690 2026-02-25T11:58:22.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd83ab27640 1 -- 10.20.193.28:0/3180561311 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8340fb820 con 0x7fd83410f690 2026-02-25T11:58:22.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd822ffd640 1 -- 10.20.193.28:0/3180561311 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd824041890 con 0x7fd83410f690 2026-02-25T11:58:22.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.416+0000 7fd83ab27640 1 -- 10.20.193.28:0/3180561311 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8340fbc90 con 0x7fd83410f690 2026-02-25T11:58:22.421 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.419+0000 7fd822ffd640 1 -- 10.20.193.28:0/3180561311 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fd824041a20 con 0x7fd83410f690 2026-02-25T11:58:22.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.421+0000 7fd822ffd640 1 --2- 10.20.193.28:0/3180561311 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd8100b56d0 0x7fd8100d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.421+0000 7fd822ffd640 1 -- 10.20.193.28:0/3180561311 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd82410b2b0 con 0x7fd83410f690 2026-02-25T11:58:22.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.421+0000 7fd839324640 1 --2- 10.20.193.28:0/3180561311 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd8100b56d0 0x7fd8100d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.422+0000 7fd839324640 1 --2- 10.20.193.28:0/3180561311 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd8100b56d0 0x7fd8100d5b80 secure :-1 s=READY pgs=394 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd828005970 tx=0x7fd828005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:22.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.422+0000 7fd83ddabe00 1 Processor -- start 2026-02-25T11:58:22.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.422+0000 7fd83ddabe00 1 -- start start 2026-02-25T11:58:22.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.422+0000 7fd83ddabe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2e540 0x5594c6a2e910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.422+0000 7fd83ddabe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a2ee50 0x5594c6a735b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.422+0000 7fd83ddabe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a73af0 0x5594c6a93ed0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.422+0000 7fd83ddabe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5594c693b8a0 con 0x5594c6a2ee50 2026-02-25T11:58:22.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.422+0000 7fd83ddabe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5594c69375d0 con 0x5594c6a73af0 2026-02-25T11:58:22.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.422+0000 7fd83ddabe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5594c6931570 con 0x5594c6a2e540 2026-02-25T11:58:22.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.422+0000 7fd839324640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a2ee50 0x5594c6a735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.423+0000 7fd839324640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a2ee50 0x5594c6a735b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52258/0 (socket says 10.20.193.28:52258) 2026-02-25T11:58:22.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.423+0000 7fd839324640 1 -- 10.20.193.28:0/1146718308 learned_addr learned my addr 10.20.193.28:0/1146718308 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:22.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.423+0000 7fd839b25640 1 --2- 10.20.193.28:0/1146718308 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2e540 0x5594c6a2e910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.423+0000 7fd83a326640 1 --2- 10.20.193.28:0/1146718308 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a73af0 0x5594c6a93ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.423+0000 7fd839324640 1 -- 10.20.193.28:0/1146718308 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2e540 msgr2=0x5594c6a2e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.423+0000 7fd839324640 1 --2- 10.20.193.28:0/1146718308 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2e540 0x5594c6a2e910 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.423+0000 7fd839324640 1 -- 10.20.193.28:0/1146718308 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a73af0 msgr2=0x5594c6a93ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.423+0000 7fd839324640 1 --2- 10.20.193.28:0/1146718308 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a73af0 0x5594c6a93ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.423+0000 7fd839324640 1 -- 10.20.193.28:0/1146718308 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c68dcd10 con 0x5594c6a2ee50 2026-02-25T11:58:22.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.423+0000 7fd839324640 1 --2- 10.20.193.28:0/1146718308 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a2ee50 0x5594c6a735b0 secure :-1 s=READY pgs=892 gs=1 cs=0 l=1 c_cookie=5695655619824560325 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd82800fd10 tx=0x7fd828012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0b089bfe99a8c5 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:22.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.423+0000 7fd8024e6640 1 -- 10.20.193.28:0/1146718308 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd828011e60 con 0x5594c6a2ee50 2026-02-25T11:58:22.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.423+0000 7fd8024e6640 1 -- 10.20.193.28:0/1146718308 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd82801e920 con 0x5594c6a2ee50 2026-02-25T11:58:22.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.423+0000 7fd8024e6640 1 -- 10.20.193.28:0/1146718308 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd828010a00 con 0x5594c6a2ee50 2026-02-25T11:58:22.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.424+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1146718308 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a2ee50 msgr2=0x5594c6a735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.424+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/1146718308 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a2ee50 0x5594c6a735b0 secure :-1 s=READY pgs=892 gs=1 cs=0 l=1 c_cookie=5695655619824560325 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd82800fd10 tx=0x7fd828012000 comp rx=0 tx=0).stop 2026-02-25T11:58:22.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.424+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1146718308 shutdown_connections 2026-02-25T11:58:22.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.424+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/1146718308 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a73af0 0x5594c6a93ed0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.424+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/1146718308 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a2ee50 0x5594c6a735b0 unknown :-1 s=CLOSED pgs=892 gs=1 cs=0 l=1 c_cookie=5695655619824560325 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.424+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/1146718308 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2e540 0x5594c6a2e910 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.424+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1146718308 >> 10.20.193.28:0/1146718308 conn(0x5594c69447a0 msgr2=0x5594c6a24410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:22.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.424+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1146718308 shutdown_connections 2026-02-25T11:58:22.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.424+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1146718308 wait complete. 2026-02-25T11:58:22.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.425+0000 7fd83ddabe00 1 -- start start 2026-02-25T11:58:22.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.425+0000 7fd83ddabe00 1 Processor -- start 2026-02-25T11:58:22.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.425+0000 7fd83ddabe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2e540 0x5594c6a46900 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.425+0000 7fd83ddabe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2ee50 0x5594c6a46e40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.425+0000 7fd83ddabe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a4bea0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd83ddabe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5594c6a4c3e0 con 0x5594c6a73af0 2026-02-25T11:58:22.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd83ddabe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5594c6a4c550 con 0x5594c6a2ee50 2026-02-25T11:58:22.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd83ddabe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5594c6a4c710 con 0x5594c6a2e540 2026-02-25T11:58:22.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd83a326640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a4bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd83a326640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a4bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52260/0 (socket says 10.20.193.28:52260) 2026-02-25T11:58:22.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd839324640 1 --2- 10.20.193.28:0/1777425040 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2ee50 0x5594c6a46e40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd839b25640 1 --2- 10.20.193.28:0/1777425040 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2e540 0x5594c6a46900 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd83a326640 1 -- 10.20.193.28:0/1777425040 learned_addr learned my addr 10.20.193.28:0/1777425040 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:22.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd83a326640 1 -- 10.20.193.28:0/1777425040 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2e540 msgr2=0x5594c6a46900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd83a326640 1 --2- 10.20.193.28:0/1777425040 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2e540 0x5594c6a46900 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd83a326640 1 -- 10.20.193.28:0/1777425040 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2ee50 msgr2=0x5594c6a46e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd83a326640 1 --2- 10.20.193.28:0/1777425040 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2ee50 0x5594c6a46e40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd83a326640 1 -- 10.20.193.28:0/1777425040 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c68b53c0 con 0x5594c6a73af0 2026-02-25T11:58:22.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd83a326640 1 --2- 10.20.193.28:0/1777425040 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a4bea0 secure :-1 s=READY pgs=893 gs=2 cs=0 l=1 c_cookie=16063970441719667646 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd830003bd0 tx=0x7fd830010730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deeeb00195135bbe server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:22.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.426+0000 7fd7ef7fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:58:22.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.427+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/1777425040 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd830012660 con 0x5594c6a73af0 2026-02-25T11:58:22.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.427+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/1777425040 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd830012c80 con 0x5594c6a73af0 2026-02-25T11:58:22.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.427+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/1777425040 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd830011d50 con 0x5594c6a73af0 2026-02-25T11:58:22.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.427+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1777425040 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd82800f9b0 con 0x5594c6a73af0 2026-02-25T11:58:22.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.427+0000 7fd83ddabe00 10 client.17586 _ll_register_callbacks cb 0x5594c68b2a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:22.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.427+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1777425040 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5594c65c76f0 con 0x5594c6a73af0 2026-02-25T11:58:22.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.427+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/1777425040 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd8300127c0 con 0x5594c6a73af0 2026-02-25T11:58:22.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.427+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/1777425040 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd830012980 con 0x5594c6a73af0 2026-02-25T11:58:22.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.427+0000 7fd83ddabe00 10 client.17586 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:22.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.427+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1777425040 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5594c68c70e0 con 0x5594c6a73af0 2026-02-25T11:58:22.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.427+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/1777425040 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd830024050 con 0x5594c6a73af0 2026-02-25T11:58:22.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.428+0000 7fd83ddabe00 10 client.17586 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:22.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.428+0000 7fd83ddabe00 10 client.17586 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:58:22.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.428+0000 7fd83ddabe00 20 client.17586 populate_metadata read hostname 'trial028' 2026-02-25T11:58:22.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.428+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/1777425040 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5594c6a71220 0x5594c6a716c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.428+0000 7fd83ddabe00 4 client.17586 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:58:22.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.428+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1777425040 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594c6903fc0 con 0x5594c6a71220 2026-02-25T11:58:22.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.428+0000 7fd839b25640 1 --2- 10.20.193.28:0/1777425040 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5594c6a71220 0x5594c6a716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.428+0000 7fd839b25640 1 --2- 10.20.193.28:0/1777425040 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5594c6a71220 0x5594c6a716c0 crc :-1 s=READY pgs=62 gs=4 cs=0 l=0 c_cookie=2409225586934135464 s_cookie=11966243930757305525 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=216f49e81cfdbaa8 server_cookie=a610a374847694b5 in_seq=0 out_seq=0 2026-02-25T11:58:22.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.428+0000 7fd7ef7fe640 10 client.17586 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:58:22.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/1777425040 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5594c6903fc0 con 0x5594c6a71220 2026-02-25T11:58:22.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd7ef7fe640 10 client.17586 handle_command_reply: tid=0 2026-02-25T11:58:22.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd83ddabe00 1 client.17586 shutdown 2026-02-25T11:58:22.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1777425040 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 msgr2=0x5594c6a4bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/1777425040 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a4bea0 secure :-1 s=READY pgs=893 gs=2 cs=0 l=1 c_cookie=16063970441719667646 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd830003bd0 tx=0x7fd830010730 comp rx=0 tx=0).stop 2026-02-25T11:58:22.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1777425040 shutdown_connections 2026-02-25T11:58:22.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/1777425040 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5594c6a71220 0x5594c6a716c0 crc :-1 s=READY pgs=62 gs=4 cs=0 l=0 c_cookie=2409225586934135464 s_cookie=11966243930757305525 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/1777425040 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a4bea0 unknown :-1 s=CLOSED pgs=893 gs=2 cs=0 l=1 c_cookie=16063970441719667646 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/1777425040 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2ee50 0x5594c6a46e40 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/1777425040 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2e540 0x5594c6a46900 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd7ef7fe640 0 client.17586 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:58:22.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1777425040 >> 10.20.193.28:0/1777425040 conn(0x5594c69447a0 msgr2=0x5594c6a24b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:22.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1777425040 shutdown_connections 2026-02-25T11:58:22.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd83ddabe00 1 -- 10.20.193.28:0/1777425040 wait complete. 2026-02-25T11:58:22.435 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.429+0000 7fd83ddabe00 20 client.17586 trim_cache size 0 max 16384 2026-02-25T11:58:22.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.436+0000 7fd83ddabe00 1 Processor -- start 2026-02-25T11:58:22.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.436+0000 7fd83ddabe00 1 -- start start 2026-02-25T11:58:22.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.436+0000 7fd83ddabe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2e540 0x5594c6a72d90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.436+0000 7fd83ddabe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2ee50 0x5594c6a732d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.436+0000 7fd83ddabe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a6b370 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.436+0000 7fd83ddabe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5594c692f1a0 con 0x5594c6a73af0 2026-02-25T11:58:22.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.436+0000 7fd83ddabe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5594c65ef700 con 0x5594c6a2e540 2026-02-25T11:58:22.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.436+0000 7fd83ddabe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5594c69bf780 con 0x5594c6a2ee50 2026-02-25T11:58:22.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.436+0000 7fd839b25640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2e540 0x5594c6a72d90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.436+0000 7fd83a326640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a6b370 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.436+0000 7fd839b25640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2e540 0x5594c6a72d90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:59102/0 (socket says 10.20.193.28:59102) 2026-02-25T11:58:22.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.436+0000 7fd839b25640 1 -- 10.20.193.28:0/962599477 learned_addr learned my addr 10.20.193.28:0/962599477 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:22.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.437+0000 7fd839324640 1 --2- 10.20.193.28:0/962599477 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2ee50 0x5594c6a732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.437+0000 7fd83a326640 1 -- 10.20.193.28:0/962599477 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2ee50 msgr2=0x5594c6a732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.437+0000 7fd83a326640 1 --2- 10.20.193.28:0/962599477 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2ee50 0x5594c6a732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.437+0000 7fd83a326640 1 -- 10.20.193.28:0/962599477 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2e540 msgr2=0x5594c6a72d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.437+0000 7fd83a326640 1 --2- 10.20.193.28:0/962599477 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2e540 0x5594c6a72d90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.437+0000 7fd83a326640 1 -- 10.20.193.28:0/962599477 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c68c1b60 con 0x5594c6a73af0 2026-02-25T11:58:22.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.437+0000 7fd839b25640 1 --2- 10.20.193.28:0/962599477 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2e540 0x5594c6a72d90 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:58:22.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.437+0000 7fd83a326640 1 --2- 10.20.193.28:0/962599477 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a6b370 secure :-1 s=READY pgs=894 gs=2 cs=0 l=1 c_cookie=4708973224277967802 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd830026f70 tx=0x7fd830025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4159a232c1ea03ba server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:22.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.437+0000 7fd801ce5640 1 -- 10.20.193.28:0/962599477 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd830012c10 con 0x5594c6a73af0 2026-02-25T11:58:22.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.437+0000 7fd801ce5640 1 -- 10.20.193.28:0/962599477 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5594c65c76f0 con 0x5594c6a73af0 2026-02-25T11:58:22.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.437+0000 7fd801ce5640 1 -- 10.20.193.28:0/962599477 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd830005930 con 0x5594c6a73af0 2026-02-25T11:58:22.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.437+0000 7fd83ddabe00 1 -- 10.20.193.28:0/962599477 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 msgr2=0x5594c6a6b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.437+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/962599477 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a6b370 secure :-1 s=READY pgs=894 gs=2 cs=0 l=1 c_cookie=4708973224277967802 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd830026f70 tx=0x7fd830025000 comp rx=0 tx=0).stop 2026-02-25T11:58:22.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.438+0000 7fd83ddabe00 1 -- 10.20.193.28:0/962599477 shutdown_connections 2026-02-25T11:58:22.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.438+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/962599477 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a6b370 unknown :-1 s=CLOSED pgs=894 gs=2 cs=0 l=1 c_cookie=4708973224277967802 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.438+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/962599477 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2ee50 0x5594c6a732d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.438+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/962599477 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2e540 0x5594c6a72d90 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.438+0000 7fd83ddabe00 1 -- 10.20.193.28:0/962599477 >> 10.20.193.28:0/962599477 conn(0x5594c69447a0 msgr2=0x5594c6a23d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:22.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.438+0000 7fd83ddabe00 1 -- 10.20.193.28:0/962599477 shutdown_connections 2026-02-25T11:58:22.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.438+0000 7fd83ddabe00 1 -- 10.20.193.28:0/962599477 wait complete. 2026-02-25T11:58:22.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.438+0000 7fd83ddabe00 1 -- start start 2026-02-25T11:58:22.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.438+0000 7fd83ddabe00 1 Processor -- start 2026-02-25T11:58:22.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd83ddabe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2e540 0x5594c6a4e570 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd83ddabe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2ee50 0x5594c6a4eab0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd83ddabe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a4eff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd83ddabe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5594c6ad9010 con 0x5594c6a73af0 2026-02-25T11:58:22.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd83ddabe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5594c6ad9150 con 0x5594c6a2e540 2026-02-25T11:58:22.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd83ddabe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5594c6ad9310 con 0x5594c6a2ee50 2026-02-25T11:58:22.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd83a326640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a4eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd839b25640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2e540 0x5594c6a4e570 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd839b25640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2e540 0x5594c6a4e570 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:59106/0 (socket says 10.20.193.28:59106) 2026-02-25T11:58:22.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd839b25640 1 -- 10.20.193.28:0/970973644 learned_addr learned my addr 10.20.193.28:0/970973644 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:22.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd839324640 1 --2- 10.20.193.28:0/970973644 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2ee50 0x5594c6a4eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd83a326640 1 -- 10.20.193.28:0/970973644 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2ee50 msgr2=0x5594c6a4eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd83a326640 1 --2- 10.20.193.28:0/970973644 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2ee50 0x5594c6a4eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd83a326640 1 -- 10.20.193.28:0/970973644 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2e540 msgr2=0x5594c6a4e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd83a326640 1 --2- 10.20.193.28:0/970973644 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2e540 0x5594c6a4e570 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.439+0000 7fd83a326640 1 -- 10.20.193.28:0/970973644 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594c68c70e0 con 0x5594c6a73af0 2026-02-25T11:58:22.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.440+0000 7fd83a326640 1 --2- 10.20.193.28:0/970973644 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a4eff0 secure :-1 s=READY pgs=895 gs=1 cs=0 l=1 c_cookie=2735498101053992168 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd830026f40 tx=0x7fd830026e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f6710795142ce8 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:22.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.440+0000 7fd7ef7fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:58:22.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.440+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/970973644 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd830005930 con 0x5594c6a73af0 2026-02-25T11:58:22.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.440+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/970973644 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd830012c10 con 0x5594c6a73af0 2026-02-25T11:58:22.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.440+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/970973644 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd830012440 con 0x5594c6a73af0 2026-02-25T11:58:22.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.440+0000 7fd83ddabe00 1 -- 10.20.193.28:0/970973644 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8340fbc90 con 0x5594c6a73af0 2026-02-25T11:58:22.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.440+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/970973644 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd8300125d0 con 0x5594c6a73af0 2026-02-25T11:58:22.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.440+0000 7fd7ef7fe640 10 client.17592 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:58:22.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.440+0000 7fd83ddabe00 10 client.17592 _ll_register_callbacks cb 0x5594c68b2a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:22.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.440+0000 7fd83ddabe00 1 -- 10.20.193.28:0/970973644 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd82801e920 con 0x5594c6a73af0 2026-02-25T11:58:22.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.440+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/970973644 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd83000eda0 con 0x5594c6a73af0 2026-02-25T11:58:22.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.440+0000 7fd83ddabe00 10 client.17592 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:22.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.440+0000 7fd83ddabe00 1 -- 10.20.193.28:0/970973644 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5594c68b56f0 con 0x5594c6a73af0 2026-02-25T11:58:22.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.441+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/970973644 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd830024070 con 0x5594c6a73af0 2026-02-25T11:58:22.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.441+0000 7fd83ddabe00 10 client.17592 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:22.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.441+0000 7fd83ddabe00 10 client.17592 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:58:22.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.441+0000 7fd83ddabe00 20 client.17592 populate_metadata read hostname 'trial028' 2026-02-25T11:58:22.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.441+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/970973644 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5594c6b20cd0 0x5594c6b41180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.441+0000 7fd83ddabe00 4 client.17592 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:58:22.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.441+0000 7fd83ddabe00 1 -- 10.20.193.28:0/970973644 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5594c65c4f40 con 0x5594c6b20cd0 2026-02-25T11:58:22.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.441+0000 7fd839b25640 1 --2- 10.20.193.28:0/970973644 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5594c6b20cd0 0x5594c6b41180 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.441+0000 7fd839b25640 1 --2- 10.20.193.28:0/970973644 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5594c6b20cd0 0x5594c6b41180 crc :-1 s=READY pgs=63 gs=4 cs=0 l=0 c_cookie=8328116954082357762 s_cookie=13442626704422703924 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73936a3ed7fcb202 server_cookie=ba8dcdc3fd4b9334 in_seq=0 out_seq=0 2026-02-25T11:58:22.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.441+0000 7fd7ef7fe640 10 client.17592 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:58:22.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd7ef7fe640 1 -- 10.20.193.28:0/970973644 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5594c65c4f40 con 0x5594c6b20cd0 2026-02-25T11:58:22.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd7ef7fe640 10 client.17592 handle_command_reply: tid=0 2026-02-25T11:58:22.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ddabe00 1 client.17592 shutdown 2026-02-25T11:58:22.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ddabe00 1 -- 10.20.193.28:0/970973644 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 msgr2=0x5594c6a4eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/970973644 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a4eff0 secure :-1 s=READY pgs=895 gs=1 cs=0 l=1 c_cookie=2735498101053992168 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd830026f40 tx=0x7fd830026e90 comp rx=0 tx=0).stop 2026-02-25T11:58:22.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ddabe00 1 -- 10.20.193.28:0/970973644 shutdown_connections 2026-02-25T11:58:22.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/970973644 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x5594c6b20cd0 0x5594c6b41180 crc :-1 s=READY pgs=63 gs=4 cs=0 l=0 c_cookie=8328116954082357762 s_cookie=13442626704422703924 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/970973644 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5594c6a73af0 0x5594c6a4eff0 unknown :-1 s=CLOSED pgs=895 gs=1 cs=0 l=1 c_cookie=2735498101053992168 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/970973644 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5594c6a2ee50 0x5594c6a4eab0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ddabe00 1 --2- 10.20.193.28:0/970973644 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5594c6a2e540 0x5594c6a4e570 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd7ef7fe640 0 client.17592 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:58:22.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ddabe00 1 -- 10.20.193.28:0/970973644 >> 10.20.193.28:0/970973644 conn(0x5594c69447a0 msgr2=0x5594c6a24500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:22.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ddabe00 1 -- 10.20.193.28:0/970973644 shutdown_connections 2026-02-25T11:58:22.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ddabe00 1 -- 10.20.193.28:0/970973644 wait complete. 2026-02-25T11:58:22.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ddabe00 20 client.17592 trim_cache size 0 max 16384 2026-02-25T11:58:22.452 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:58:22.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ab27640 1 -- 10.20.193.28:0/3180561311 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd8100b56d0 msgr2=0x7fd8100d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ab27640 1 --2- 10.20.193.28:0/3180561311 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd8100b56d0 0x7fd8100d5b80 secure :-1 s=READY pgs=394 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd828005970 tx=0x7fd828005900 comp rx=0 tx=0).stop 2026-02-25T11:58:22.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ab27640 1 -- 10.20.193.28:0/3180561311 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd83410f690 msgr2=0x7fd8340f5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.442+0000 7fd83ab27640 1 --2- 10.20.193.28:0/3180561311 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd83410f690 0x7fd8340f5960 secure :-1 s=READY pgs=891 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd8240098a0 tx=0x7fd8240028d0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.443+0000 7fd83ab27640 1 -- 10.20.193.28:0/3180561311 shutdown_connections 2026-02-25T11:58:22.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.443+0000 7fd83ab27640 1 --2- 10.20.193.28:0/3180561311 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd8100b56d0 0x7fd8100d5b80 unknown :-1 s=CLOSED pgs=394 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.443+0000 7fd83ab27640 1 --2- 10.20.193.28:0/3180561311 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd834115880 0x7fd8340f6060 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.443+0000 7fd83ab27640 1 --2- 10.20.193.28:0/3180561311 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd834110080 0x7fd834176150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.443+0000 7fd83ab27640 1 --2- 10.20.193.28:0/3180561311 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd83410f690 0x7fd8340f5960 unknown :-1 s=CLOSED pgs=891 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.454 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.443+0000 7fd83ab27640 1 -- 10.20.193.28:0/3180561311 >> 10.20.193.28:0/3180561311 conn(0x7fd8340ecf00 msgr2=0x7fd834111b30 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:22.454 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.443+0000 7fd83ab27640 1 -- 10.20.193.28:0/3180561311 shutdown_connections 2026-02-25T11:58:22.454 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.443+0000 7fd83ab27640 1 -- 10.20.193.28:0/3180561311 wait complete. 2026-02-25T11:58:22.543 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:58:22.594 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:22 trial028 ceph-mon[39224]: pgmap v530: 97 pgs: 97 active+clean; 7.7 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:58:22.594 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:22 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2404303061' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:58:22.651 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:22 trial167 ceph-mon[43392]: pgmap v530: 97 pgs: 97 active+clean; 7.7 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:58:22.651 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:22 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2404303061' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:58:22.660 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:22 trial119 ceph-mon[42710]: pgmap v530: 97 pgs: 97 active+clean; 7.7 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 16 KiB/s wr, 10 op/s 2026-02-25T11:58:22.661 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:22 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2404303061' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:58:22.726 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:58:22.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.906+0000 7efd5f6aa640 1 -- 10.20.193.28:0/2413690356 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efd58115930 msgr2=0x7efd58176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.906+0000 7efd5f6aa640 1 --2- 10.20.193.28:0/2413690356 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efd58115930 0x7efd58176300 secure :-1 s=READY pgs=896 gs=1 cs=0 l=1 c_cookie=5124490561493849358 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efd5400aa30 tx=0x7efd54030270 comp rx=0 tx=0).stop 2026-02-25T11:58:22.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.906+0000 7efd5f6aa640 1 -- 10.20.193.28:0/2413690356 shutdown_connections 2026-02-25T11:58:22.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.906+0000 7efd5f6aa640 1 --2- 10.20.193.28:0/2413690356 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efd58115930 0x7efd58176300 unknown :-1 s=CLOSED pgs=896 gs=1 cs=0 l=1 c_cookie=5124490561493849358 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.906+0000 7efd5f6aa640 1 --2- 10.20.193.28:0/2413690356 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7efd58110130 0x7efd58115180 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.906+0000 7efd5f6aa640 1 --2- 10.20.193.28:0/2413690356 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efd5810f740 0x7efd5810fb10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.906+0000 7efd5f6aa640 1 -- 10.20.193.28:0/2413690356 >> 10.20.193.28:0/2413690356 conn(0x7efd580ecff0 msgr2=0x7efd5810d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:22.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.906+0000 7efd5f6aa640 1 -- 10.20.193.28:0/2413690356 shutdown_connections 2026-02-25T11:58:22.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.906+0000 7efd5f6aa640 1 -- 10.20.193.28:0/2413690356 wait complete. 2026-02-25T11:58:22.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5f6aa640 1 Processor -- start 2026-02-25T11:58:22.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5f6aa640 1 -- start start 2026-02-25T11:58:22.910 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5f6aa640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efd5810f740 0x7efd5811e150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.910 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5f6aa640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efd58110130 0x7efd5811e690 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5f6aa640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7efd58115930 0x7efd58123a10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5f6aa640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7efd58124030 con 0x7efd5810f740 2026-02-25T11:58:22.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5f6aa640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7efd581241a0 con 0x7efd58110130 2026-02-25T11:58:22.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5f6aa640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd58124360 con 0x7efd58115930 2026-02-25T11:58:22.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5d41f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efd5810f740 0x7efd5811e150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5d41f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efd5810f740 0x7efd5811e150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52300/0 (socket says 10.20.193.28:52300) 2026-02-25T11:58:22.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5d41f640 1 -- 10.20.193.28:0/1220567928 learned_addr learned my addr 10.20.193.28:0/1220567928 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:22.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5cc1e640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efd58110130 0x7efd5811e690 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5dc20640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7efd58115930 0x7efd58123a10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5d41f640 1 -- 10.20.193.28:0/1220567928 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7efd58115930 msgr2=0x7efd58123a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5d41f640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7efd58115930 0x7efd58123a10 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.907+0000 7efd5d41f640 1 -- 10.20.193.28:0/1220567928 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efd58110130 msgr2=0x7efd5811e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.908+0000 7efd5d41f640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efd58110130 0x7efd5811e690 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.908+0000 7efd5d41f640 1 -- 10.20.193.28:0/1220567928 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd5400a650 con 0x7efd5810f740 2026-02-25T11:58:22.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.908+0000 7efd5d41f640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efd5810f740 0x7efd5811e150 secure :-1 s=READY pgs=897 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efd4800d950 tx=0x7efd4800de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:22.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.908+0000 7efd467fc640 1 -- 10.20.193.28:0/1220567928 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efd48004540 con 0x7efd5810f740 2026-02-25T11:58:22.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.908+0000 7efd467fc640 1 -- 10.20.193.28:0/1220567928 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7efd48009ce0 con 0x7efd5810f740 2026-02-25T11:58:22.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.908+0000 7efd467fc640 1 -- 10.20.193.28:0/1220567928 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efd480106c0 con 0x7efd5810f740 2026-02-25T11:58:22.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.908+0000 7efd5f6aa640 1 -- 10.20.193.28:0/1220567928 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd58124640 con 0x7efd5810f740 2026-02-25T11:58:22.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.908+0000 7efd5f6aa640 1 -- 10.20.193.28:0/1220567928 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd58164d40 con 0x7efd5810f740 2026-02-25T11:58:22.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.911+0000 7efd467fc640 1 -- 10.20.193.28:0/1220567928 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7efd480097c0 con 0x7efd5810f740 2026-02-25T11:58:22.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.913+0000 7efd467fc640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efd340b5760 0x7efd340d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.913+0000 7efd467fc640 1 -- 10.20.193.28:0/1220567928 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7efd480e6d50 con 0x7efd5810f740 2026-02-25T11:58:22.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.913+0000 7efd5cc1e640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efd340b5760 0x7efd340d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd616a5e00 1 Processor -- start 2026-02-25T11:58:22.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd5cc1e640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efd340b5760 0x7efd340d5c10 secure :-1 s=READY pgs=395 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efd58124aa0 tx=0x7efd4c004420 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:22.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd616a5e00 1 -- start start 2026-02-25T11:58:22.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd616a5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dfb2b30 0x56313dfb2f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd616a5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dff7ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd616a5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dff80e0 0x56313e0184c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd616a5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56313dec88a0 con 0x56313dfb2b30 2026-02-25T11:58:22.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd616a5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56313dec45d0 con 0x56313dff80e0 2026-02-25T11:58:22.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd616a5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56313debe570 con 0x56313dfb3440 2026-02-25T11:58:22.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd5d41f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dfb2b30 0x56313dfb2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd5d41f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dfb2b30 0x56313dfb2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52316/0 (socket says 10.20.193.28:52316) 2026-02-25T11:58:22.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd5d41f640 1 -- 10.20.193.28:0/1194218408 learned_addr learned my addr 10.20.193.28:0/1194218408 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:22.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd5d41f640 1 -- 10.20.193.28:0/1194218408 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 msgr2=0x56313dff7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd5d41f640 1 --2- 10.20.193.28:0/1194218408 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dff7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd5d41f640 1 -- 10.20.193.28:0/1194218408 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dff80e0 msgr2=0x56313e0184c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2026-02-25T11:58:22.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd5d41f640 1 --2- 10.20.193.28:0/1194218408 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dff80e0 0x56313e0184c0 unknown :-1 s=START_CONNECT pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.914+0000 7efd5d41f640 1 -- 10.20.193.28:0/1194218408 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56313de69d10 con 0x56313dfb2b30 2026-02-25T11:58:22.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.915+0000 7efd5d41f640 1 --2- 10.20.193.28:0/1194218408 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dfb2b30 0x56313dfb2f00 secure :-1 s=READY pgs=898 gs=1 cs=0 l=1 c_cookie=10895395518473828863 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efd480e4880 tx=0x7efd480a8fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9734387f6e03f9ff server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:22.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.915+0000 7efd25ce5640 1 -- 10.20.193.28:0/1194218408 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efd48009e50 con 0x56313dfb2b30 2026-02-25T11:58:22.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.915+0000 7efd25ce5640 1 -- 10.20.193.28:0/1194218408 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7efd48002850 con 0x56313dfb2b30 2026-02-25T11:58:22.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.915+0000 7efd25ce5640 1 -- 10.20.193.28:0/1194218408 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efd48009e50 con 0x56313dfb2b30 2026-02-25T11:58:22.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.915+0000 7efd616a5e00 1 -- 10.20.193.28:0/1194218408 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dfb2b30 msgr2=0x56313dfb2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.915+0000 7efd616a5e00 1 --2- 10.20.193.28:0/1194218408 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dfb2b30 0x56313dfb2f00 secure :-1 s=READY pgs=898 gs=1 cs=0 l=1 c_cookie=10895395518473828863 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efd480e4880 tx=0x7efd480a8fb0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.915+0000 7efd616a5e00 1 -- 10.20.193.28:0/1194218408 shutdown_connections 2026-02-25T11:58:22.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.915+0000 7efd616a5e00 1 --2- 10.20.193.28:0/1194218408 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dff80e0 0x56313e0184c0 unknown :-1 s=CLOSED pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.915+0000 7efd616a5e00 1 --2- 10.20.193.28:0/1194218408 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dff7ba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.915+0000 7efd616a5e00 1 --2- 10.20.193.28:0/1194218408 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dfb2b30 0x56313dfb2f00 unknown :-1 s=CLOSED pgs=898 gs=1 cs=0 l=1 c_cookie=10895395518473828863 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.915+0000 7efd616a5e00 1 -- 10.20.193.28:0/1194218408 >> 10.20.193.28:0/1194218408 conn(0x56313ded17a0 msgr2=0x56313dfb1410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:22.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.915+0000 7efd616a5e00 1 -- 10.20.193.28:0/1194218408 shutdown_connections 2026-02-25T11:58:22.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.915+0000 7efd616a5e00 1 -- 10.20.193.28:0/1194218408 wait complete. 2026-02-25T11:58:22.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.917+0000 7efd616a5e00 1 -- start start 2026-02-25T11:58:22.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.917+0000 7efd616a5e00 1 Processor -- start 2026-02-25T11:58:22.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.917+0000 7efd616a5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dfb2b30 0x56313dfd37c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.917+0000 7efd616a5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfd3d00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.917+0000 7efd616a5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dff80e0 0x56313dfd8cf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.917+0000 7efd616a5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56313dfd9230 con 0x56313dff80e0 2026-02-25T11:58:22.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.917+0000 7efd616a5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56313dfd93a0 con 0x56313dfb2b30 2026-02-25T11:58:22.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.917+0000 7efd616a5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56313dfd9560 con 0x56313dfb3440 2026-02-25T11:58:22.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd5d41f640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dfb2b30 0x56313dfd37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd5d41f640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dfb2b30 0x56313dfd37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:59160/0 (socket says 10.20.193.28:59160) 2026-02-25T11:58:22.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd5d41f640 1 -- 10.20.193.28:0/938169047 learned_addr learned my addr 10.20.193.28:0/938169047 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:22.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd5cc1e640 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfd3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd5dc20640 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dff80e0 0x56313dfd8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd5d41f640 1 -- 10.20.193.28:0/938169047 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 msgr2=0x56313dfd3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd5d41f640 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfd3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd5d41f640 1 -- 10.20.193.28:0/938169047 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dff80e0 msgr2=0x56313dfd8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd5d41f640 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dff80e0 0x56313dfd8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd5d41f640 1 -- 10.20.193.28:0/938169047 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56313de423c0 con 0x56313dfb2b30 2026-02-25T11:58:22.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd5dc20640 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dff80e0 0x56313dfd8cf0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:58:22.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd5d41f640 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dfb2b30 0x56313dfd37c0 secure :-1 s=READY pgs=182 gs=1 cs=0 l=1 c_cookie=2385000802562593162 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efd480ad0f0 tx=0x7efd4800bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21193997f9c3d18a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:22.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd0effd640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:58:22.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd0effd640 1 -- 10.20.193.28:0/938169047 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efd480a9530 con 0x56313dfb2b30 2026-02-25T11:58:22.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.918+0000 7efd616a5e00 1 -- 10.20.193.28:0/938169047 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd58164d40 con 0x56313dfb2b30 2026-02-25T11:58:22.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd616a5e00 10 client.27257 _ll_register_callbacks cb 0x56313de3fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:22.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd616a5e00 1 -- 10.20.193.28:0/938169047 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56313db546f0 con 0x56313dfb2b30 2026-02-25T11:58:22.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd0effd640 1 -- 10.20.193.28:0/938169047 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7efd48009e50 con 0x56313dfb2b30 2026-02-25T11:58:22.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd0effd640 1 -- 10.20.193.28:0/938169047 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efd480722a0 con 0x56313dfb2b30 2026-02-25T11:58:22.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd0effd640 1 -- 10.20.193.28:0/938169047 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7efd48072400 con 0x56313dfb2b30 2026-02-25T11:58:22.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd0effd640 1 -- 10.20.193.28:0/938169047 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efd480ab0f0 con 0x56313dfb2b30 2026-02-25T11:58:22.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd616a5e00 10 client.27257 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:22.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd616a5e00 1 -- 10.20.193.28:0/938169047 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x56313de540e0 con 0x56313dfb2b30 2026-02-25T11:58:22.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd0effd640 1 -- 10.20.193.28:0/938169047 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7efd480e4d20 con 0x56313dfb2b30 2026-02-25T11:58:22.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd616a5e00 10 client.27257 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:22.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd616a5e00 10 client.27257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:58:22.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd616a5e00 20 client.27257 populate_metadata read hostname 'trial028' 2026-02-25T11:58:22.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd616a5e00 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56313dfddc90 0x56313dfde130 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd616a5e00 4 client.27257 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:58:22.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.919+0000 7efd616a5e00 1 -- 10.20.193.28:0/938169047 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56313de90fc0 con 0x56313dfddc90 2026-02-25T11:58:22.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.920+0000 7efd5cc1e640 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56313dfddc90 0x56313dfde130 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.920+0000 7efd5cc1e640 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56313dfddc90 0x56313dfde130 crc :-1 s=READY pgs=178 gs=4 cs=0 l=0 c_cookie=3141300388887464703 s_cookie=3348784742230023814 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b98240f746f0aff server_cookie=2e7945fafdde5e86 in_seq=0 out_seq=0 2026-02-25T11:58:22.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.920+0000 7efd0effd640 10 client.27257 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:22.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.921+0000 7efd0effd640 1 -- 10.20.193.28:0/938169047 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x56313de90fc0 con 0x56313dfddc90 2026-02-25T11:58:22.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.921+0000 7efd0effd640 10 client.27257 handle_command_reply: tid=0 2026-02-25T11:58:22.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.921+0000 7efd616a5e00 1 client.27257 shutdown 2026-02-25T11:58:22.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.921+0000 7efd616a5e00 1 -- 10.20.193.28:0/938169047 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dfb2b30 msgr2=0x56313dfd37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.921+0000 7efd616a5e00 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dfb2b30 0x56313dfd37c0 secure :-1 s=READY pgs=182 gs=1 cs=0 l=1 c_cookie=2385000802562593162 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efd480ad0f0 tx=0x7efd4800bf60 comp rx=0 tx=0).stop 2026-02-25T11:58:22.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.921+0000 7efd616a5e00 1 -- 10.20.193.28:0/938169047 shutdown_connections 2026-02-25T11:58:22.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.921+0000 7efd616a5e00 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56313dfddc90 0x56313dfde130 crc :-1 s=READY pgs=178 gs=4 cs=0 l=0 c_cookie=3141300388887464703 s_cookie=3348784742230023814 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.921+0000 7efd616a5e00 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dff80e0 0x56313dfd8cf0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.921+0000 7efd616a5e00 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfd3d00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.921+0000 7efd616a5e00 1 --2- 10.20.193.28:0/938169047 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dfb2b30 0x56313dfd37c0 unknown :-1 s=CLOSED pgs=182 gs=1 cs=0 l=1 c_cookie=2385000802562593162 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.921+0000 7efd616a5e00 1 -- 10.20.193.28:0/938169047 >> 10.20.193.28:0/938169047 conn(0x56313ded17a0 msgr2=0x56313dfb1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:22.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.921+0000 7efd0effd640 0 client.27257 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:22.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.922+0000 7efd616a5e00 1 -- 10.20.193.28:0/938169047 shutdown_connections 2026-02-25T11:58:22.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.922+0000 7efd616a5e00 1 -- 10.20.193.28:0/938169047 wait complete. 2026-02-25T11:58:22.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.922+0000 7efd616a5e00 20 client.27257 trim_cache size 0 max 16384 2026-02-25T11:58:22.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.928+0000 7efd616a5e00 1 Processor -- start 2026-02-25T11:58:22.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd616a5e00 1 -- start start 2026-02-25T11:58:22.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd616a5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dfb2b30 0x56313dff7b00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd616a5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfb1f30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd616a5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dff80e0 0x56313dfc8120 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd616a5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56313debc1a0 con 0x56313dff80e0 2026-02-25T11:58:22.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd616a5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56313db7c700 con 0x56313dfb2b30 2026-02-25T11:58:22.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd616a5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56313dfbfc80 con 0x56313dfb3440 2026-02-25T11:58:22.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd5dc20640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dff80e0 0x56313dfc8120 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd5dc20640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dff80e0 0x56313dfc8120 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52332/0 (socket says 10.20.193.28:52332) 2026-02-25T11:58:22.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd5dc20640 1 -- 10.20.193.28:0/3169232859 learned_addr learned my addr 10.20.193.28:0/3169232859 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:22.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd5d41f640 1 --2- 10.20.193.28:0/3169232859 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dfb2b30 0x56313dff7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd5cc1e640 1 --2- 10.20.193.28:0/3169232859 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfb1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd5d41f640 1 -- 10.20.193.28:0/3169232859 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 msgr2=0x56313dfb1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd5d41f640 1 --2- 10.20.193.28:0/3169232859 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfb1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd5d41f640 1 -- 10.20.193.28:0/3169232859 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dff80e0 msgr2=0x56313dfc8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd5d41f640 1 --2- 10.20.193.28:0/3169232859 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dff80e0 0x56313dfc8120 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd5d41f640 1 -- 10.20.193.28:0/3169232859 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56313de4eb60 con 0x56313dfb2b30 2026-02-25T11:58:22.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.929+0000 7efd5dc20640 1 --2- 10.20.193.28:0/3169232859 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dff80e0 0x56313dfc8120 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:58:22.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.930+0000 7efd5d41f640 1 --2- 10.20.193.28:0/3169232859 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dfb2b30 0x56313dff7b00 secure :-1 s=READY pgs=183 gs=1 cs=0 l=1 c_cookie=16542608151644209834 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efd480d2370 tx=0x7efd480cb000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5932677a93c02aa server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:22.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.930+0000 7efd254e4640 1 -- 10.20.193.28:0/3169232859 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efd480adb60 con 0x56313dfb2b30 2026-02-25T11:58:22.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.930+0000 7efd254e4640 1 -- 10.20.193.28:0/3169232859 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x56313db546f0 con 0x56313dfb2b30 2026-02-25T11:58:22.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.930+0000 7efd254e4640 1 -- 10.20.193.28:0/3169232859 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efd48009b10 con 0x56313dfb2b30 2026-02-25T11:58:22.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.930+0000 7efd616a5e00 1 -- 10.20.193.28:0/3169232859 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dfb2b30 msgr2=0x56313dff7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.930+0000 7efd616a5e00 1 --2- 10.20.193.28:0/3169232859 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dfb2b30 0x56313dff7b00 secure :-1 s=READY pgs=183 gs=1 cs=0 l=1 c_cookie=16542608151644209834 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efd480d2370 tx=0x7efd480cb000 comp rx=0 tx=0).stop 2026-02-25T11:58:22.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.930+0000 7efd616a5e00 1 -- 10.20.193.28:0/3169232859 shutdown_connections 2026-02-25T11:58:22.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.930+0000 7efd616a5e00 1 --2- 10.20.193.28:0/3169232859 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dff80e0 0x56313dfc8120 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.930+0000 7efd616a5e00 1 --2- 10.20.193.28:0/3169232859 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfb1f30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.930+0000 7efd616a5e00 1 --2- 10.20.193.28:0/3169232859 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dfb2b30 0x56313dff7b00 unknown :-1 s=CLOSED pgs=183 gs=1 cs=0 l=1 c_cookie=16542608151644209834 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.930+0000 7efd616a5e00 1 -- 10.20.193.28:0/3169232859 >> 10.20.193.28:0/3169232859 conn(0x56313ded17a0 msgr2=0x56313dff73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:22.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.930+0000 7efd616a5e00 1 -- 10.20.193.28:0/3169232859 shutdown_connections 2026-02-25T11:58:22.935 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.930+0000 7efd616a5e00 1 -- 10.20.193.28:0/3169232859 wait complete. 2026-02-25T11:58:22.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.931+0000 7efd616a5e00 1 -- start start 2026-02-25T11:58:22.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.931+0000 7efd616a5e00 1 Processor -- start 2026-02-25T11:58:22.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.931+0000 7efd616a5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dfb2b30 0x56313dfeb070 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.931+0000 7efd616a5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfeb5b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.931+0000 7efd616a5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dff80e0 0x56313dff00c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.931+0000 7efd616a5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56313dfd4860 con 0x56313dfb2b30 2026-02-25T11:58:22.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.931+0000 7efd616a5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56313dfebf30 con 0x56313dff80e0 2026-02-25T11:58:22.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.931+0000 7efd616a5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56313dff0630 con 0x56313dfb3440 2026-02-25T11:58:22.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.932+0000 7efd5cc1e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfeb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.932+0000 7efd5cc1e640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfeb5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:52710/0 (socket says 10.20.193.28:52710) 2026-02-25T11:58:22.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.932+0000 7efd5cc1e640 1 -- 10.20.193.28:0/2103950050 learned_addr learned my addr 10.20.193.28:0/2103950050 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:22.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.932+0000 7efd5dc20640 1 --2- 10.20.193.28:0/2103950050 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dff80e0 0x56313dff00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.932+0000 7efd5d41f640 1 --2- 10.20.193.28:0/2103950050 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dfb2b30 0x56313dfeb070 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.932+0000 7efd5cc1e640 1 -- 10.20.193.28:0/2103950050 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dff80e0 msgr2=0x56313dff00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.932+0000 7efd5cc1e640 1 --2- 10.20.193.28:0/2103950050 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dff80e0 0x56313dff00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.932+0000 7efd5cc1e640 1 -- 10.20.193.28:0/2103950050 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dfb2b30 msgr2=0x56313dfeb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:22.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.932+0000 7efd5cc1e640 1 --2- 10.20.193.28:0/2103950050 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dfb2b30 0x56313dfeb070 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.932+0000 7efd5cc1e640 1 -- 10.20.193.28:0/2103950050 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56313de540e0 con 0x56313dfb3440 2026-02-25T11:58:22.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.932+0000 7efd5cc1e640 1 --2- 10.20.193.28:0/2103950050 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfeb5b0 secure :-1 s=READY pgs=237 gs=1 cs=0 l=1 c_cookie=14583259992928591150 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efd4c0046f0 tx=0x7efd4c0047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca6225b2b9ac9d2e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:22.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.932+0000 7efd0effd640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:58:22.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd0effd640 1 -- 10.20.193.28:0/2103950050 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efd4c01faa0 con 0x56313dfb3440 2026-02-25T11:58:22.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd0effd640 1 -- 10.20.193.28:0/2103950050 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7efd4c01fc00 con 0x56313dfb3440 2026-02-25T11:58:22.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd616a5e00 1 -- 10.20.193.28:0/2103950050 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd58176a30 con 0x56313dfb3440 2026-02-25T11:58:22.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd0effd640 1 -- 10.20.193.28:0/2103950050 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efd4c031650 con 0x56313dfb3440 2026-02-25T11:58:22.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd616a5e00 10 client.27262 _ll_register_callbacks cb 0x56313de3fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:22.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd0effd640 1 -- 10.20.193.28:0/2103950050 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7efd4c011440 con 0x56313dfb3440 2026-02-25T11:58:22.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd616a5e00 1 -- 10.20.193.28:0/2103950050 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd48002850 con 0x56313dfb3440 2026-02-25T11:58:22.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd0effd640 1 -- 10.20.193.28:0/2103950050 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7efd4c011600 con 0x56313dfb3440 2026-02-25T11:58:22.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd616a5e00 10 client.27262 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:22.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd616a5e00 1 -- 10.20.193.28:0/2103950050 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x56313de426f0 con 0x56313dfb3440 2026-02-25T11:58:22.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd0effd640 1 -- 10.20.193.28:0/2103950050 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7efd4c03a020 con 0x56313dfb3440 2026-02-25T11:58:22.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd616a5e00 10 client.27262 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:22.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd616a5e00 10 client.27262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:58:22.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd616a5e00 20 client.27262 populate_metadata read hostname 'trial028' 2026-02-25T11:58:22.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd616a5e00 1 --2- 10.20.193.28:0/2103950050 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56313e0ad930 0x56313e0cdde0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:22.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd616a5e00 4 client.27262 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:58:22.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.933+0000 7efd616a5e00 1 -- 10.20.193.28:0/2103950050 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56313db51f40 con 0x56313e0ad930 2026-02-25T11:58:22.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.934+0000 7efd5d41f640 1 --2- 10.20.193.28:0/2103950050 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56313e0ad930 0x56313e0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:22.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.934+0000 7efd5d41f640 1 --2- 10.20.193.28:0/2103950050 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56313e0ad930 0x56313e0cdde0 crc :-1 s=READY pgs=179 gs=4 cs=0 l=0 c_cookie=7993607156509400316 s_cookie=726002291299506425 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eeeff5b5dc35cfc server_cookie=a13473cc706d8f9 in_seq=0 out_seq=0 2026-02-25T11:58:22.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.934+0000 7efd0effd640 10 client.27262 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:22.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.934+0000 7efd0effd640 1 -- 10.20.193.28:0/2103950050 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56313db51f40 con 0x56313e0ad930 2026-02-25T11:58:22.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.934+0000 7efd0effd640 10 client.27262 handle_command_reply: tid=0 2026-02-25T11:58:22.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.934+0000 7efd616a5e00 1 client.27262 shutdown 2026-02-25T11:58:22.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd616a5e00 1 -- 10.20.193.28:0/2103950050 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 msgr2=0x56313dfeb5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd616a5e00 1 --2- 10.20.193.28:0/2103950050 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfeb5b0 secure :-1 s=READY pgs=237 gs=1 cs=0 l=1 c_cookie=14583259992928591150 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efd4c0046f0 tx=0x7efd4c0047e0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd616a5e00 1 -- 10.20.193.28:0/2103950050 shutdown_connections 2026-02-25T11:58:22.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd616a5e00 1 --2- 10.20.193.28:0/2103950050 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x56313e0ad930 0x56313e0cdde0 crc :-1 s=READY pgs=179 gs=4 cs=0 l=0 c_cookie=7993607156509400316 s_cookie=726002291299506425 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd616a5e00 1 --2- 10.20.193.28:0/2103950050 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56313dff80e0 0x56313dff00c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd616a5e00 1 --2- 10.20.193.28:0/2103950050 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56313dfb3440 0x56313dfeb5b0 unknown :-1 s=CLOSED pgs=237 gs=1 cs=0 l=1 c_cookie=14583259992928591150 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd616a5e00 1 --2- 10.20.193.28:0/2103950050 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56313dfb2b30 0x56313dfeb070 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd616a5e00 1 -- 10.20.193.28:0/2103950050 >> 10.20.193.28:0/2103950050 conn(0x56313ded17a0 msgr2=0x56313dfd3500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:22.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd0effd640 0 client.27262 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:22.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd616a5e00 1 -- 10.20.193.28:0/2103950050 shutdown_connections 2026-02-25T11:58:22.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd616a5e00 1 -- 10.20.193.28:0/2103950050 wait complete. 2026-02-25T11:58:22.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd616a5e00 20 client.27262 trim_cache size 0 max 16384 2026-02-25T11:58:22.941 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:58:22.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd5f6aa640 1 -- 10.20.193.28:0/1220567928 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efd340b5760 msgr2=0x7efd340d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd5f6aa640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efd340b5760 0x7efd340d5c10 secure :-1 s=READY pgs=395 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efd58124aa0 tx=0x7efd4c004420 comp rx=0 tx=0).stop 2026-02-25T11:58:22.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd5f6aa640 1 -- 10.20.193.28:0/1220567928 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efd5810f740 msgr2=0x7efd5811e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:22.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd5f6aa640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efd5810f740 0x7efd5811e150 secure :-1 s=READY pgs=897 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efd4800d950 tx=0x7efd4800de20 comp rx=0 tx=0).stop 2026-02-25T11:58:22.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd5f6aa640 1 -- 10.20.193.28:0/1220567928 shutdown_connections 2026-02-25T11:58:22.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd5f6aa640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efd340b5760 0x7efd340d5c10 unknown :-1 s=CLOSED pgs=395 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd5f6aa640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7efd58115930 0x7efd58123a10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd5f6aa640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7efd58110130 0x7efd5811e690 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd5f6aa640 1 --2- 10.20.193.28:0/1220567928 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7efd5810f740 0x7efd5811e150 unknown :-1 s=CLOSED pgs=897 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:22.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd5f6aa640 1 -- 10.20.193.28:0/1220567928 >> 10.20.193.28:0/1220567928 conn(0x7efd580ecff0 msgr2=0x7efd5810d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:22.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.935+0000 7efd5f6aa640 1 -- 10.20.193.28:0/1220567928 shutdown_connections 2026-02-25T11:58:22.943 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:22.936+0000 7efd5f6aa640 1 -- 10.20.193.28:0/1220567928 wait complete. 2026-02-25T11:58:24.037 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2026-02-25T11:58:24.038 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub start / force,recursive 2026-02-25T11:58:24.224 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:58:24.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.403+0000 7f07eb577640 1 -- 10.20.193.28:0/774320186 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07ec110080 msgr2=0x7f07ec1150d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.408 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.403+0000 7f07eb577640 1 --2- 10.20.193.28:0/774320186 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07ec110080 0x7f07ec1150d0 secure :-1 s=READY pgs=899 gs=1 cs=0 l=1 c_cookie=7279303015295741593 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07d4009950 tx=0x7f07d402f190 comp rx=0 tx=0).stop 2026-02-25T11:58:24.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.404+0000 7f07eb577640 1 -- 10.20.193.28:0/774320186 shutdown_connections 2026-02-25T11:58:24.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.404+0000 7f07eb577640 1 --2- 10.20.193.28:0/774320186 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f07ec115880 0x7f07ec176250 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.404+0000 7f07eb577640 1 --2- 10.20.193.28:0/774320186 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07ec110080 0x7f07ec1150d0 unknown :-1 s=CLOSED pgs=899 gs=1 cs=0 l=1 c_cookie=7279303015295741593 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.404+0000 7f07eb577640 1 --2- 10.20.193.28:0/774320186 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f07ec10f690 0x7f07ec10fa60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.404+0000 7f07eb577640 1 -- 10.20.193.28:0/774320186 >> 10.20.193.28:0/774320186 conn(0x7f07ec0ecf00 msgr2=0x7f07ec10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:24.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.404+0000 7f07eb577640 1 -- 10.20.193.28:0/774320186 shutdown_connections 2026-02-25T11:58:24.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.404+0000 7f07eb577640 1 -- 10.20.193.28:0/774320186 wait complete. 2026-02-25T11:58:24.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.404+0000 7f07eb577640 1 Processor -- start 2026-02-25T11:58:24.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.404+0000 7f07eb577640 1 -- start start 2026-02-25T11:58:24.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07eb577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07ec10f690 0x7f07ec11e0a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.409 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07eb577640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f07ec110080 0x7f07ec11e5e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07eb577640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f07ec115880 0x7f07ec1239b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07eb577640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec123fd0 con 0x7f07ec10f690 2026-02-25T11:58:24.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07eb577640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec124140 con 0x7f07ec115880 2026-02-25T11:58:24.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07eb577640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec124300 con 0x7f07ec110080 2026-02-25T11:58:24.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07ea575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07ec10f690 0x7f07ec11e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07ea575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07ec10f690 0x7f07ec11e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52374/0 (socket says 10.20.193.28:52374) 2026-02-25T11:58:24.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07ea575640 1 -- 10.20.193.28:0/3965589364 learned_addr learned my addr 10.20.193.28:0/3965589364 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:24.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07e9d74640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f07ec110080 0x7f07ec11e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07ead76640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f07ec115880 0x7f07ec1239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07ea575640 1 -- 10.20.193.28:0/3965589364 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f07ec110080 msgr2=0x7f07ec11e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.410 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07ea575640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f07ec110080 0x7f07ec11e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07ea575640 1 -- 10.20.193.28:0/3965589364 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f07ec115880 msgr2=0x7f07ec1239b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07ea575640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f07ec115880 0x7f07ec1239b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07ea575640 1 -- 10.20.193.28:0/3965589364 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d4009570 con 0x7f07ec10f690 2026-02-25T11:58:24.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07ead76640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f07ec115880 0x7f07ec1239b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:58:24.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.405+0000 7f07ea575640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07ec10f690 0x7f07ec11e0a0 secure :-1 s=READY pgs=900 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07e000d950 tx=0x7f07e000de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:24.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.406+0000 7f07d37fe640 1 -- 10.20.193.28:0/3965589364 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07e0004540 con 0x7f07ec10f690 2026-02-25T11:58:24.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.406+0000 7f07d37fe640 1 -- 10.20.193.28:0/3965589364 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f07e0009ce0 con 0x7f07ec10f690 2026-02-25T11:58:24.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.406+0000 7f07d37fe640 1 -- 10.20.193.28:0/3965589364 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07e00106e0 con 0x7f07ec10f690 2026-02-25T11:58:24.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.406+0000 7f07eb577640 1 -- 10.20.193.28:0/3965589364 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ec1245e0 con 0x7f07ec10f690 2026-02-25T11:58:24.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.406+0000 7f07eb577640 1 -- 10.20.193.28:0/3965589364 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ec164ce0 con 0x7f07ec10f690 2026-02-25T11:58:24.412 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.410+0000 7f07d37fe640 1 -- 10.20.193.28:0/3965589364 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f07e0009820 con 0x7f07ec10f690 2026-02-25T11:58:24.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.411+0000 7f07d37fe640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07c40b56e0 0x7f07c40d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07d37fe640 1 -- 10.20.193.28:0/3965589364 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f07e00e6440 con 0x7f07ec10f690 2026-02-25T11:58:24.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07f2a0ce00 1 Processor -- start 2026-02-25T11:58:24.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07f2a0ce00 1 -- start start 2026-02-25T11:58:24.414 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07e9d74640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07c40b56e0 0x7f07c40d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07f2a0ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec2e530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07f2a0ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2ea70 0x5609fec73020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07f2a0ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec73560 0x5609fec93940 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.415 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07f2a0ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5609feb3bd80 con 0x5609fec2e160 2026-02-25T11:58:24.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07f2a0ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5609feb37ab0 con 0x5609fec73560 2026-02-25T11:58:24.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07f2a0ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5609feb31a50 con 0x5609fec2ea70 2026-02-25T11:58:24.416 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07ea575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec2e530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07ea575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec2e530 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52390/0 (socket says 10.20.193.28:52390) 2026-02-25T11:58:24.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07ea575640 1 -- 10.20.193.28:0/4233067641 learned_addr learned my addr 10.20.193.28:0/4233067641 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:24.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.412+0000 7f07ead76640 1 --2- 10.20.193.28:0/4233067641 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec73560 0x5609fec93940 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.413+0000 7f07e9d74640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07c40b56e0 0x7f07c40d5b90 secure :-1 s=READY pgs=396 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07d40023d0 tx=0x7f07d4031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:24.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.413+0000 7f07e9d74640 1 --2- 10.20.193.28:0/4233067641 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2ea70 0x5609fec73020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.413+0000 7f07ea575640 1 -- 10.20.193.28:0/4233067641 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2ea70 msgr2=0x5609fec73020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.413+0000 7f07ea575640 1 --2- 10.20.193.28:0/4233067641 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2ea70 0x5609fec73020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.413+0000 7f07ea575640 1 -- 10.20.193.28:0/4233067641 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec73560 msgr2=0x5609fec93940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.413+0000 7f07ea575640 1 --2- 10.20.193.28:0/4233067641 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec73560 0x5609fec93940 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.413+0000 7f07ea575640 1 -- 10.20.193.28:0/4233067641 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609feadd120 con 0x5609fec2e160 2026-02-25T11:58:24.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.413+0000 7f07e9d74640 1 --2- 10.20.193.28:0/4233067641 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2ea70 0x5609fec73020 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:58:24.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.413+0000 7f07ea575640 1 --2- 10.20.193.28:0/4233067641 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec2e530 secure :-1 s=READY pgs=901 gs=1 cs=0 l=1 c_cookie=8248430921793263486 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07e0005aa0 tx=0x7f07e00027c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72785038a7d5d77e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:24.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.413+0000 7f07d0ff9640 1 -- 10.20.193.28:0/4233067641 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07e00abfc0 con 0x5609fec2e160 2026-02-25T11:58:24.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.413+0000 7f07d0ff9640 1 -- 10.20.193.28:0/4233067641 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f07e0009e50 con 0x5609fec2e160 2026-02-25T11:58:24.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.413+0000 7f07d0ff9640 1 -- 10.20.193.28:0/4233067641 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07e00b7990 con 0x5609fec2e160 2026-02-25T11:58:24.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.414+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/4233067641 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 msgr2=0x5609fec2e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.414+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/4233067641 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec2e530 secure :-1 s=READY pgs=901 gs=1 cs=0 l=1 c_cookie=8248430921793263486 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07e0005aa0 tx=0x7f07e00027c0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.414+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/4233067641 shutdown_connections 2026-02-25T11:58:24.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.414+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/4233067641 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec73560 0x5609fec93940 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.414+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/4233067641 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2ea70 0x5609fec73020 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.414+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/4233067641 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec2e530 unknown :-1 s=CLOSED pgs=901 gs=1 cs=0 l=1 c_cookie=8248430921793263486 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.414+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/4233067641 >> 10.20.193.28:0/4233067641 conn(0x5609fec038f0 msgr2=0x5609fec23cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:24.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.414+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/4233067641 shutdown_connections 2026-02-25T11:58:24.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.414+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/4233067641 wait complete. 2026-02-25T11:58:24.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.415+0000 7f07f2a0ce00 1 -- start start 2026-02-25T11:58:24.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.415+0000 7f07f2a0ce00 1 Processor -- start 2026-02-25T11:58:24.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07f2a0ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec8aeb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07f2a0ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec2ea70 0x5609fec8b3f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07f2a0ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec73560 0x5609fec903e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07f2a0ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5609fec90920 con 0x5609fec2e160 2026-02-25T11:58:24.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07f2a0ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5609fec90a90 con 0x5609fec2ea70 2026-02-25T11:58:24.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07f2a0ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5609fec90c50 con 0x5609fec73560 2026-02-25T11:58:24.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07ea575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec8aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07ea575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec8aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52404/0 (socket says 10.20.193.28:52404) 2026-02-25T11:58:24.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07ea575640 1 -- 10.20.193.28:0/3139485784 learned_addr learned my addr 10.20.193.28:0/3139485784 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:24.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07e9d74640 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec2ea70 0x5609fec8b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07ead76640 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec73560 0x5609fec903e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07ea575640 1 -- 10.20.193.28:0/3139485784 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec73560 msgr2=0x5609fec903e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07ea575640 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec73560 0x5609fec903e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07ea575640 1 -- 10.20.193.28:0/3139485784 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec2ea70 msgr2=0x5609fec8b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07ea575640 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec2ea70 0x5609fec8b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07ea575640 1 -- 10.20.193.28:0/3139485784 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609feab5810 con 0x5609fec2e160 2026-02-25T11:58:24.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07ead76640 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec73560 0x5609fec903e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:58:24.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f07ea575640 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec8aeb0 secure :-1 s=READY pgs=902 gs=1 cs=0 l=1 c_cookie=15042971538692150341 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07e00a8860 tx=0x7f07e0002b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c35ee9d20e9445 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:24.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.416+0000 7f079bfff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:58:24.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f079bfff640 1 -- 10.20.193.28:0/3139485784 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07e00b24e0 con 0x5609fec2e160 2026-02-25T11:58:24.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f079bfff640 1 -- 10.20.193.28:0/3139485784 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f07e00b4ac0 con 0x5609fec2e160 2026-02-25T11:58:24.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f079bfff640 1 -- 10.20.193.28:0/3139485784 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07e00b7470 con 0x5609fec2e160 2026-02-25T11:58:24.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/3139485784 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ec164ce0 con 0x5609fec2e160 2026-02-25T11:58:24.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f079bfff640 1 -- 10.20.193.28:0/3139485784 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f07e00b7600 con 0x5609fec2e160 2026-02-25T11:58:24.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f079bfff640 10 client.17619 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:58:24.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f07f2a0ce00 10 client.17619 _ll_register_callbacks cb 0x5609feab2e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:24.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/3139485784 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5609fe889530 con 0x5609fec2e160 2026-02-25T11:58:24.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f079bfff640 1 -- 10.20.193.28:0/3139485784 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f07e00ad450 con 0x5609fec2e160 2026-02-25T11:58:24.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f07f2a0ce00 10 client.17619 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:24.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/3139485784 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5609feac74f0 con 0x5609fec2e160 2026-02-25T11:58:24.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f079bfff640 1 -- 10.20.193.28:0/3139485784 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f07e00d00a0 con 0x5609fec2e160 2026-02-25T11:58:24.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f07f2a0ce00 10 client.17619 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:24.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f07f2a0ce00 10 client.17619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:58:24.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f07f2a0ce00 20 client.17619 populate_metadata read hostname 'trial028' 2026-02-25T11:58:24.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5609fec704b0 0x5609fec70950 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f07f2a0ce00 4 client.17619 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:58:24.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.417+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/3139485784 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609feadeb50 con 0x5609fec704b0 2026-02-25T11:58:24.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.418+0000 7f07e9d74640 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5609fec704b0 0x5609fec70950 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.418+0000 7f07e9d74640 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5609fec704b0 0x5609fec70950 crc :-1 s=READY pgs=180 gs=4 cs=0 l=0 c_cookie=14731771683855188614 s_cookie=126645757560289318 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc71c44b5058c286 server_cookie=1c1efa53fb0f826 in_seq=0 out_seq=0 2026-02-25T11:58:24.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.418+0000 7f079bfff640 10 client.17619 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:24.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f079bfff640 1 -- 10.20.193.28:0/3139485784 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5609feadeb50 con 0x5609fec704b0 2026-02-25T11:58:24.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f079bfff640 10 client.17619 handle_command_reply: tid=0 2026-02-25T11:58:24.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f07f2a0ce00 1 client.17619 shutdown 2026-02-25T11:58:24.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/3139485784 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 msgr2=0x5609fec8aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec8aeb0 secure :-1 s=READY pgs=902 gs=1 cs=0 l=1 c_cookie=15042971538692150341 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07e00a8860 tx=0x7f07e0002b20 comp rx=0 tx=0).stop 2026-02-25T11:58:24.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/3139485784 shutdown_connections 2026-02-25T11:58:24.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5609fec704b0 0x5609fec70950 crc :-1 s=READY pgs=180 gs=4 cs=0 l=0 c_cookie=14731771683855188614 s_cookie=126645757560289318 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec73560 0x5609fec903e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec2ea70 0x5609fec8b3f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/3139485784 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec8aeb0 unknown :-1 s=CLOSED pgs=902 gs=1 cs=0 l=1 c_cookie=15042971538692150341 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f079bfff640 0 client.17619 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:24.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/3139485784 >> 10.20.193.28:0/3139485784 conn(0x5609fec038f0 msgr2=0x5609fec23cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:24.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/3139485784 shutdown_connections 2026-02-25T11:58:24.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/3139485784 wait complete. 2026-02-25T11:58:24.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.419+0000 7f07f2a0ce00 20 client.17619 trim_cache size 0 max 16384 2026-02-25T11:58:24.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.428+0000 7f07f2a0ce00 1 Processor -- start 2026-02-25T11:58:24.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.428+0000 7f07f2a0ce00 1 -- start start 2026-02-25T11:58:24.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.428+0000 7f07f2a0ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2e160 0x5609fec88160 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.428+0000 7f07f2a0ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec2ea70 0x5609fec886a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.428+0000 7f07f2a0ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec73560 0x5609fec6e150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.428+0000 7f07f2a0ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5609feb2f680 con 0x5609fec73560 2026-02-25T11:58:24.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.428+0000 7f07f2a0ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5609fe7ef790 con 0x5609fec2ea70 2026-02-25T11:58:24.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.428+0000 7f07f2a0ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5609fec84010 con 0x5609fec2e160 2026-02-25T11:58:24.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.429+0000 7f07ead76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec73560 0x5609fec6e150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.429+0000 7f07ead76640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec73560 0x5609fec6e150 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52412/0 (socket says 10.20.193.28:52412) 2026-02-25T11:58:24.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.429+0000 7f07ead76640 1 -- 10.20.193.28:0/1744646994 learned_addr learned my addr 10.20.193.28:0/1744646994 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:24.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.429+0000 7f07ea575640 1 --2- 10.20.193.28:0/1744646994 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2e160 0x5609fec88160 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.429+0000 7f07e9d74640 1 --2- 10.20.193.28:0/1744646994 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec2ea70 0x5609fec886a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.429+0000 7f07ead76640 1 -- 10.20.193.28:0/1744646994 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2e160 msgr2=0x5609fec88160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.436 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.429+0000 7f07ead76640 1 --2- 10.20.193.28:0/1744646994 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2e160 0x5609fec88160 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.429+0000 7f07ead76640 1 -- 10.20.193.28:0/1744646994 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec2ea70 msgr2=0x5609fec886a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.429+0000 7f07ead76640 1 --2- 10.20.193.28:0/1744646994 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec2ea70 0x5609fec886a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.429+0000 7f07ead76640 1 -- 10.20.193.28:0/1744646994 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609feac1f70 con 0x5609fec73560 2026-02-25T11:58:24.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.429+0000 7f07ead76640 1 --2- 10.20.193.28:0/1744646994 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec73560 0x5609fec6e150 secure :-1 s=READY pgs=903 gs=2 cs=0 l=1 c_cookie=5820316247293466071 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07dc003bc0 tx=0x7f07dc014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c5eaca205c95d7 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:24.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.429+0000 7f07b64e6640 1 -- 10.20.193.28:0/1744646994 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07dc003d80 con 0x5609fec73560 2026-02-25T11:58:24.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.430+0000 7f07b64e6640 1 -- 10.20.193.28:0/1744646994 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f07dc012910 con 0x5609fec73560 2026-02-25T11:58:24.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.430+0000 7f07b64e6640 1 -- 10.20.193.28:0/1744646994 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07dc010920 con 0x5609fec73560 2026-02-25T11:58:24.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.430+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/1744646994 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec73560 msgr2=0x5609fec6e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.430+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/1744646994 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec73560 0x5609fec6e150 secure :-1 s=READY pgs=903 gs=2 cs=0 l=1 c_cookie=5820316247293466071 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07dc003bc0 tx=0x7f07dc014830 comp rx=0 tx=0).stop 2026-02-25T11:58:24.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.430+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/1744646994 shutdown_connections 2026-02-25T11:58:24.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.430+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/1744646994 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec73560 0x5609fec6e150 unknown :-1 s=CLOSED pgs=903 gs=2 cs=0 l=1 c_cookie=5820316247293466071 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.430+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/1744646994 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec2ea70 0x5609fec886a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.430+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/1744646994 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2e160 0x5609fec88160 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.430+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/1744646994 >> 10.20.193.28:0/1744646994 conn(0x5609fec038f0 msgr2=0x5609fec75940 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:24.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.430+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/1744646994 shutdown_connections 2026-02-25T11:58:24.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.430+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/1744646994 wait complete. 2026-02-25T11:58:24.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.431+0000 7f07f2a0ce00 1 -- start start 2026-02-25T11:58:24.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.431+0000 7f07f2a0ce00 1 Processor -- start 2026-02-25T11:58:24.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.431+0000 7f07f2a0ce00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec03cc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.431+0000 7f07f2a0ce00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2ea70 0x5609fec04200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.431+0000 7f07f2a0ce00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec73560 0x5609fec04740 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.431+0000 7f07f2a0ce00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5609fec82b10 con 0x5609fec2e160 2026-02-25T11:58:24.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.431+0000 7f07f2a0ce00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5609fec6ea50 con 0x5609fec73560 2026-02-25T11:58:24.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.431+0000 7f07f2a0ce00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5609fec6ec10 con 0x5609fec2ea70 2026-02-25T11:58:24.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07ea575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec03cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07ea575640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec03cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52424/0 (socket says 10.20.193.28:52424) 2026-02-25T11:58:24.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07ea575640 1 -- 10.20.193.28:0/943801230 learned_addr learned my addr 10.20.193.28:0/943801230 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:24.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07ead76640 1 --2- 10.20.193.28:0/943801230 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec73560 0x5609fec04740 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07e9d74640 1 --2- 10.20.193.28:0/943801230 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2ea70 0x5609fec04200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07ea575640 1 -- 10.20.193.28:0/943801230 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2ea70 msgr2=0x5609fec04200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07ea575640 1 --2- 10.20.193.28:0/943801230 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2ea70 0x5609fec04200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07ea575640 1 -- 10.20.193.28:0/943801230 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec73560 msgr2=0x5609fec04740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07ea575640 1 --2- 10.20.193.28:0/943801230 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec73560 0x5609fec04740 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07ea575640 1 -- 10.20.193.28:0/943801230 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609feab5b40 con 0x5609fec2e160 2026-02-25T11:58:24.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07ea575640 1 --2- 10.20.193.28:0/943801230 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec03cc0 secure :-1 s=READY pgs=904 gs=3 cs=0 l=1 c_cookie=200862595825359758 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07e00e3f90 tx=0x7f07e0002b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c99b7778d55b8e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:24.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f079bfff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:58:24.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f079bfff640 1 -- 10.20.193.28:0/943801230 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07e00b24e0 con 0x5609fec2e160 2026-02-25T11:58:24.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f079bfff640 1 -- 10.20.193.28:0/943801230 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5609fe889530 con 0x5609fec2e160 2026-02-25T11:58:24.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f079bfff640 1 -- 10.20.193.28:0/943801230 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07e00b76d0 con 0x5609fec2e160 2026-02-25T11:58:24.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/943801230 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07dc003860 con 0x5609fec2e160 2026-02-25T11:58:24.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f079bfff640 1 -- 10.20.193.28:0/943801230 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f07e00d0170 con 0x5609fec2e160 2026-02-25T11:58:24.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f079bfff640 10 client.17625 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:58:24.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07f2a0ce00 10 client.17625 _ll_register_callbacks cb 0x5609feab2e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:24.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.432+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/943801230 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07e0009e50 con 0x5609fec2e160 2026-02-25T11:58:24.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.433+0000 7f079bfff640 1 -- 10.20.193.28:0/943801230 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f07e00b7970 con 0x5609fec2e160 2026-02-25T11:58:24.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.433+0000 7f07f2a0ce00 10 client.17625 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:24.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.433+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/943801230 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5609feac7750 con 0x5609fec2e160 2026-02-25T11:58:24.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.433+0000 7f079bfff640 1 -- 10.20.193.28:0/943801230 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f07e008f070 con 0x5609fec2e160 2026-02-25T11:58:24.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.433+0000 7f07f2a0ce00 10 client.17625 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:24.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.433+0000 7f07f2a0ce00 10 client.17625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:58:24.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.433+0000 7f07f2a0ce00 20 client.17625 populate_metadata read hostname 'trial028' 2026-02-25T11:58:24.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.433+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/943801230 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5609fed20bc0 0x5609fed41070 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.433+0000 7f07f2a0ce00 4 client.17625 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2026-02-25T11:58:24.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.433+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/943801230 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5609fe992690 con 0x5609fed20bc0 2026-02-25T11:58:24.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.433+0000 7f07e9d74640 1 --2- 10.20.193.28:0/943801230 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5609fed20bc0 0x5609fed41070 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.434+0000 7f07e9d74640 1 --2- 10.20.193.28:0/943801230 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5609fed20bc0 0x5609fed41070 crc :-1 s=READY pgs=181 gs=4 cs=0 l=0 c_cookie=535811067610869226 s_cookie=3101237041343931815 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f9553bfab2dea server_cookie=2b09cea87596a5a7 in_seq=0 out_seq=0 2026-02-25T11:58:24.442 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.434+0000 7f079bfff640 10 client.17625 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:24.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f079bfff640 1 -- 10.20.193.28:0/943801230 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5609fe992690 con 0x5609fed20bc0 2026-02-25T11:58:24.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f079bfff640 10 client.17625 handle_command_reply: tid=0 2026-02-25T11:58:24.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f07f2a0ce00 1 client.17625 shutdown 2026-02-25T11:58:24.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/943801230 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 msgr2=0x5609fec03cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/943801230 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec03cc0 secure :-1 s=READY pgs=904 gs=3 cs=0 l=1 c_cookie=200862595825359758 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07e00e3f90 tx=0x7f07e0002b90 comp rx=0 tx=0).stop 2026-02-25T11:58:24.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/943801230 shutdown_connections 2026-02-25T11:58:24.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/943801230 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5609fed20bc0 0x5609fed41070 crc :-1 s=READY pgs=181 gs=4 cs=0 l=0 c_cookie=535811067610869226 s_cookie=3101237041343931815 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/943801230 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5609fec73560 0x5609fec04740 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/943801230 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5609fec2ea70 0x5609fec04200 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f07f2a0ce00 1 --2- 10.20.193.28:0/943801230 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5609fec2e160 0x5609fec03cc0 unknown :-1 s=CLOSED pgs=904 gs=3 cs=0 l=1 c_cookie=200862595825359758 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f079bfff640 0 client.17625 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:24.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/943801230 >> 10.20.193.28:0/943801230 conn(0x5609fec038f0 msgr2=0x5609fec8f240 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:24.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/943801230 shutdown_connections 2026-02-25T11:58:24.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f07f2a0ce00 1 -- 10.20.193.28:0/943801230 wait complete. 2026-02-25T11:58:24.444 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:58:24.444 INFO:teuthology.orchestra.run.trial028.stdout: "return_code": 0, 2026-02-25T11:58:24.445 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_tag": "b897dba9-e832-4e59-932d-6585e9a0071e", 2026-02-25T11:58:24.445 INFO:teuthology.orchestra.run.trial028.stdout: "mode": "asynchronous" 2026-02-25T11:58:24.445 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:58:24.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.435+0000 7f07f2a0ce00 20 client.17625 trim_cache size 0 max 16384 2026-02-25T11:58:24.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.436+0000 7f07eb577640 1 -- 10.20.193.28:0/3965589364 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07c40b56e0 msgr2=0x7f07c40d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.436+0000 7f07eb577640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07c40b56e0 0x7f07c40d5b90 secure :-1 s=READY pgs=396 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07d40023d0 tx=0x7f07d4031040 comp rx=0 tx=0).stop 2026-02-25T11:58:24.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.436+0000 7f07eb577640 1 -- 10.20.193.28:0/3965589364 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07ec10f690 msgr2=0x7f07ec11e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.436+0000 7f07eb577640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07ec10f690 0x7f07ec11e0a0 secure :-1 s=READY pgs=900 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07e000d950 tx=0x7f07e000de20 comp rx=0 tx=0).stop 2026-02-25T11:58:24.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.436+0000 7f07eb577640 1 -- 10.20.193.28:0/3965589364 shutdown_connections 2026-02-25T11:58:24.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.436+0000 7f07eb577640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07c40b56e0 0x7f07c40d5b90 unknown :-1 s=CLOSED pgs=396 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.436+0000 7f07eb577640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f07ec115880 0x7f07ec1239b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.436+0000 7f07eb577640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f07ec110080 0x7f07ec11e5e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.436+0000 7f07eb577640 1 --2- 10.20.193.28:0/3965589364 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07ec10f690 0x7f07ec11e0a0 unknown :-1 s=CLOSED pgs=900 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.436+0000 7f07eb577640 1 -- 10.20.193.28:0/3965589364 >> 10.20.193.28:0/3965589364 conn(0x7f07ec0ecf00 msgr2=0x7f07ec10d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:24.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.436+0000 7f07eb577640 1 -- 10.20.193.28:0/3965589364 shutdown_connections 2026-02-25T11:58:24.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.436+0000 7f07eb577640 1 -- 10.20.193.28:0/3965589364 wait complete. 2026-02-25T11:58:24.537 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:58:24.721 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:58:24.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:24 trial167 ceph-mon[43392]: pgmap v531: 97 pgs: 97 active+clean; 7.7 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:58:24.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:24 trial119 ceph-mon[42710]: pgmap v531: 97 pgs: 97 active+clean; 7.7 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:58:24.766 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:24 trial028 ceph-mon[39224]: pgmap v531: 97 pgs: 97 active+clean; 7.7 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 18 KiB/s wr, 12 op/s 2026-02-25T11:58:24.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.903+0000 7f96d4870640 1 -- 10.20.193.28:0/3703323346 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f96d0119d30 msgr2=0x7f96d017a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.903+0000 7f96d4870640 1 --2- 10.20.193.28:0/3703323346 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f96d0119d30 0x7f96d017a700 secure :-1 s=READY pgs=184 gs=2 cs=0 l=1 c_cookie=7977398347079275294 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96c400a970 tx=0x7f96c40301b0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.903+0000 7f96d4870640 1 -- 10.20.193.28:0/3703323346 shutdown_connections 2026-02-25T11:58:24.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.903+0000 7f96d4870640 1 --2- 10.20.193.28:0/3703323346 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f96d0119d30 0x7f96d017a700 unknown :-1 s=CLOSED pgs=184 gs=2 cs=0 l=1 c_cookie=7977398347079275294 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.903+0000 7f96d4870640 1 --2- 10.20.193.28:0/3703323346 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f96d0114a90 0x7f96d0119580 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.903+0000 7f96d4870640 1 --2- 10.20.193.28:0/3703323346 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96d01140a0 0x7f96d0114470 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.903+0000 7f96d4870640 1 -- 10.20.193.28:0/3703323346 >> 10.20.193.28:0/3703323346 conn(0x7f96d0064c40 msgr2=0x7f96d0065040 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:24.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.903+0000 7f96d4870640 1 -- 10.20.193.28:0/3703323346 shutdown_connections 2026-02-25T11:58:24.905 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.903+0000 7f96d4870640 1 -- 10.20.193.28:0/3703323346 wait complete. 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96d4870640 1 Processor -- start 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96d4870640 1 -- start start 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96d4870640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f96d01140a0 0x7f96d01224b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96d4870640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96d0114a90 0x7f96d01229f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96d4870640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f96d0119d30 0x7f96d0127dc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96d4870640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f96d01283e0 con 0x7f96d0114a90 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96d4870640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f96d0128550 con 0x7f96d01140a0 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96d4870640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96d0128710 con 0x7f96d0119d30 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96ceffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96d0114a90 0x7f96d01229f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96ceffd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96d0114a90 0x7f96d01229f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52450/0 (socket says 10.20.193.28:52450) 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96ceffd640 1 -- 10.20.193.28:0/3822311628 learned_addr learned my addr 10.20.193.28:0/3822311628 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96cf7fe640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f96d01140a0 0x7f96d01224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96cffff640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f96d0119d30 0x7f96d0127dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.907 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.904+0000 7f96ceffd640 1 -- 10.20.193.28:0/3822311628 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f96d0119d30 msgr2=0x7f96d0127dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.905+0000 7f96ceffd640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f96d0119d30 0x7f96d0127dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.905+0000 7f96ceffd640 1 -- 10.20.193.28:0/3822311628 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f96d01140a0 msgr2=0x7f96d01224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.905+0000 7f96ceffd640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f96d01140a0 0x7f96d01224b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.905+0000 7f96ceffd640 1 -- 10.20.193.28:0/3822311628 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c400a590 con 0x7f96d0114a90 2026-02-25T11:58:24.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.905+0000 7f96ceffd640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96d0114a90 0x7f96d01229f0 secure :-1 s=READY pgs=905 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96c0005940 tx=0x7f96c000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:24.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.905+0000 7f96ccff9640 1 -- 10.20.193.28:0/3822311628 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f96c000cd10 con 0x7f96d0114a90 2026-02-25T11:58:24.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.905+0000 7f96d4870640 1 -- 10.20.193.28:0/3822311628 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d01289f0 con 0x7f96d0114a90 2026-02-25T11:58:24.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.905+0000 7f96ccff9640 1 -- 10.20.193.28:0/3822311628 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f96c000ce70 con 0x7f96d0114a90 2026-02-25T11:58:24.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.905+0000 7f96d4870640 1 -- 10.20.193.28:0/3822311628 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d01690f0 con 0x7f96d0114a90 2026-02-25T11:58:24.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.906+0000 7f96ccff9640 1 -- 10.20.193.28:0/3822311628 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f96c0005240 con 0x7f96d0114a90 2026-02-25T11:58:24.910 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.908+0000 7f96ccff9640 1 -- 10.20.193.28:0/3822311628 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f96c00053a0 con 0x7f96d0114a90 2026-02-25T11:58:24.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.910+0000 7f96ccff9640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f96a40b56e0 0x7f96a40d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.910+0000 7f96ccff9640 1 -- 10.20.193.28:0/3822311628 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f96c0014070 con 0x7f96d0114a90 2026-02-25T11:58:24.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.910+0000 7f96d7af4e00 1 Processor -- start 2026-02-25T11:58:24.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.910+0000 7f96cf7fe640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f96a40b56e0 0x7f96a40d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.910+0000 7f96d7af4e00 1 -- start start 2026-02-25T11:58:24.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96d7af4e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f9111140 0x5568f9111510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96d7af4e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111a50 0x5568f91561b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96d7af4e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f91566f0 0x5568f9176ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96d7af4e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5568f90279b0 con 0x5568f9111140 2026-02-25T11:58:24.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96d7af4e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5568f90236e0 con 0x5568f9111a50 2026-02-25T11:58:24.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96d7af4e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5568f901d840 con 0x5568f91566f0 2026-02-25T11:58:24.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96ceffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111a50 0x5568f91561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96cf7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f9111140 0x5568f9111510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96ceffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111a50 0x5568f91561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:59258/0 (socket says 10.20.193.28:59258) 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96ceffd640 1 -- 10.20.193.28:0/1337966432 learned_addr learned my addr 10.20.193.28:0/1337966432 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96cf7fe640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f96a40b56e0 0x7f96a40d5b90 secure :-1 s=READY pgs=397 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96bc004740 tx=0x7f96bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96cffff640 1 --2- 10.20.193.28:0/1337966432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f91566f0 0x5568f9176ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96cf7fe640 1 -- 10.20.193.28:0/1337966432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f91566f0 msgr2=0x5568f9176ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96cf7fe640 1 --2- 10.20.193.28:0/1337966432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f91566f0 0x5568f9176ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96cf7fe640 1 -- 10.20.193.28:0/1337966432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111a50 msgr2=0x5568f91561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96cf7fe640 1 --2- 10.20.193.28:0/1337966432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111a50 0x5568f91561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96cf7fe640 1 -- 10.20.193.28:0/1337966432 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568f8fc8d80 con 0x5568f9111140 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.911+0000 7f96cf7fe640 1 --2- 10.20.193.28:0/1337966432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f9111140 0x5568f9111510 secure :-1 s=READY pgs=906 gs=2 cs=0 l=1 c_cookie=5641906541805779292 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96bc004220 tx=0x7f96bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4c141b9049395c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.912+0000 7f9693fff640 1 -- 10.20.193.28:0/1337966432 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f96bc011e60 con 0x5568f9111140 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.912+0000 7f9693fff640 1 -- 10.20.193.28:0/1337966432 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f96bc009d20 con 0x5568f9111140 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.912+0000 7f9693fff640 1 -- 10.20.193.28:0/1337966432 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f96bc01a480 con 0x5568f9111140 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.912+0000 7f96d7af4e00 1 -- 10.20.193.28:0/1337966432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f9111140 msgr2=0x5568f9111510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.912+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/1337966432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f9111140 0x5568f9111510 secure :-1 s=READY pgs=906 gs=2 cs=0 l=1 c_cookie=5641906541805779292 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96bc004220 tx=0x7f96bc014000 comp rx=0 tx=0).stop 2026-02-25T11:58:24.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.912+0000 7f96d7af4e00 1 -- 10.20.193.28:0/1337966432 shutdown_connections 2026-02-25T11:58:24.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.912+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/1337966432 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f91566f0 0x5568f9176ad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.912+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/1337966432 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111a50 0x5568f91561b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.912+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/1337966432 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f9111140 0x5568f9111510 unknown :-1 s=CLOSED pgs=906 gs=2 cs=0 l=1 c_cookie=5641906541805779292 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.912+0000 7f96d7af4e00 1 -- 10.20.193.28:0/1337966432 >> 10.20.193.28:0/1337966432 conn(0x5568f9030d40 msgr2=0x5568f910fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:24.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.912+0000 7f96d7af4e00 1 -- 10.20.193.28:0/1337966432 shutdown_connections 2026-02-25T11:58:24.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.912+0000 7f96d7af4e00 1 -- 10.20.193.28:0/1337966432 wait complete. 2026-02-25T11:58:24.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.913+0000 7f96d7af4e00 1 -- start start 2026-02-25T11:58:24.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.914+0000 7f96d7af4e00 1 Processor -- start 2026-02-25T11:58:24.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.914+0000 7f96d7af4e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111140 0x5568f916e360 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.914+0000 7f96d7af4e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f9111a50 0x5568f916e8a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.914+0000 7f96d7af4e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f91566f0 0x5568f9173890 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.914+0000 7f96d7af4e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5568f9173dd0 con 0x5568f91566f0 2026-02-25T11:58:24.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.914+0000 7f96d7af4e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5568f9173f40 con 0x5568f9111140 2026-02-25T11:58:24.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.914+0000 7f96d7af4e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5568f9174100 con 0x5568f9111a50 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.914+0000 7f96cf7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111140 0x5568f916e360 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.914+0000 7f96cf7fe640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111140 0x5568f916e360 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:59260/0 (socket says 10.20.193.28:59260) 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.914+0000 7f96cf7fe640 1 -- 10.20.193.28:0/2489948033 learned_addr learned my addr 10.20.193.28:0/2489948033 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.914+0000 7f96ceffd640 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f9111a50 0x5568f916e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.914+0000 7f96cffff640 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f91566f0 0x5568f9173890 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96cf7fe640 1 -- 10.20.193.28:0/2489948033 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f9111a50 msgr2=0x5568f916e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96cf7fe640 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f9111a50 0x5568f916e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96cf7fe640 1 -- 10.20.193.28:0/2489948033 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f91566f0 msgr2=0x5568f9173890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96cf7fe640 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f91566f0 0x5568f9173890 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96cf7fe640 1 -- 10.20.193.28:0/2489948033 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568f8fa1350 con 0x5568f9111140 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96ceffd640 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f9111a50 0x5568f916e8a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96cf7fe640 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111140 0x5568f916e360 secure :-1 s=READY pgs=185 gs=1 cs=0 l=1 c_cookie=13311433769272950827 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96bc009f80 tx=0x7f96bc004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8bbb67c74a7402b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96917fa640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96917fa640 1 -- 10.20.193.28:0/2489948033 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f96bc004ba0 con 0x5568f9111140 2026-02-25T11:58:24.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2489948033 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96bc0098d0 con 0x5568f9111140 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96d7af4e00 10 client.27293 _ll_register_callbacks cb 0x5568f8f9e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2489948033 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5568f8cb36f0 con 0x5568f9111140 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96917fa640 1 -- 10.20.193.28:0/2489948033 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5568f8cb36f0 con 0x5568f9111140 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96917fa640 1 -- 10.20.193.28:0/2489948033 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f96bc036530 con 0x5568f9111140 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.915+0000 7f96917fa640 1 -- 10.20.193.28:0/2489948033 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f96bc0366c0 con 0x5568f9111140 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.916+0000 7f96917fa640 1 -- 10.20.193.28:0/2489948033 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f96bc036900 con 0x5568f9111140 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.916+0000 7f96d7af4e00 10 client.27293 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.916+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2489948033 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5568f8fb3150 con 0x5568f9111140 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.916+0000 7f96917fa640 1 -- 10.20.193.28:0/2489948033 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f96bc034060 con 0x5568f9111140 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.916+0000 7f96d7af4e00 10 client.27293 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.916+0000 7f96d7af4e00 10 client.27293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.916+0000 7f96d7af4e00 20 client.27293 populate_metadata read hostname 'trial028' 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.916+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568f9176920 0x5568f9154420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.916+0000 7f96d7af4e00 4 client.27293 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:58:24.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.916+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2489948033 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568f8fef910 con 0x5568f9176920 2026-02-25T11:58:24.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.916+0000 7f96ceffd640 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568f9176920 0x5568f9154420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.917+0000 7f96ceffd640 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568f9176920 0x5568f9154420 crc :-1 s=READY pgs=182 gs=4 cs=0 l=0 c_cookie=5569340081261461977 s_cookie=13680155329992237980 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d4a454bd8ef41d9 server_cookie=bdd9accac45aa39c in_seq=0 out_seq=0 2026-02-25T11:58:24.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.917+0000 7f96917fa640 10 client.27293 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:24.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.918+0000 7f96917fa640 1 -- 10.20.193.28:0/2489948033 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5568f8fef910 con 0x5568f9176920 2026-02-25T11:58:24.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.918+0000 7f96917fa640 10 client.27293 handle_command_reply: tid=0 2026-02-25T11:58:24.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.918+0000 7f96d7af4e00 1 client.27293 shutdown 2026-02-25T11:58:24.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.918+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2489948033 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111140 msgr2=0x5568f916e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.918+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111140 0x5568f916e360 secure :-1 s=READY pgs=185 gs=1 cs=0 l=1 c_cookie=13311433769272950827 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96bc009f80 tx=0x7f96bc004770 comp rx=0 tx=0).stop 2026-02-25T11:58:24.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.919+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2489948033 shutdown_connections 2026-02-25T11:58:24.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.919+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568f9176920 0x5568f9154420 crc :-1 s=READY pgs=182 gs=4 cs=0 l=0 c_cookie=5569340081261461977 s_cookie=13680155329992237980 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.919+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f91566f0 0x5568f9173890 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.919+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f9111a50 0x5568f916e8a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.919+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/2489948033 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111140 0x5568f916e360 unknown :-1 s=CLOSED pgs=185 gs=1 cs=0 l=1 c_cookie=13311433769272950827 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.919+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2489948033 >> 10.20.193.28:0/2489948033 conn(0x5568f9030d40 msgr2=0x5568f910f230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:24.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.919+0000 7f96917fa640 0 client.27293 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:24.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.919+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2489948033 shutdown_connections 2026-02-25T11:58:24.921 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.919+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2489948033 wait complete. 2026-02-25T11:58:24.922 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.919+0000 7f96d7af4e00 20 client.27293 trim_cache size 0 max 16384 2026-02-25T11:58:24.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.928+0000 7f96d7af4e00 1 Processor -- start 2026-02-25T11:58:24.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.928+0000 7f96d7af4e00 1 -- start start 2026-02-25T11:58:24.930 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.928+0000 7f96d7af4e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111140 0x5568f91591e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.928+0000 7f96d7af4e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f9111a50 0x5568f916b530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.928+0000 7f96d7af4e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f91566f0 0x5568f916ba70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.928+0000 7f96d7af4e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5568f901b470 con 0x5568f91566f0 2026-02-25T11:58:24.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.928+0000 7f96d7af4e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5568f8cdb700 con 0x5568f9111140 2026-02-25T11:58:24.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.928+0000 7f96d7af4e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5568f916ee80 con 0x5568f9111a50 2026-02-25T11:58:24.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96cffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f91566f0 0x5568f916ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96cffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f91566f0 0x5568f916ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52472/0 (socket says 10.20.193.28:52472) 2026-02-25T11:58:24.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96cffff640 1 -- 10.20.193.28:0/408499278 learned_addr learned my addr 10.20.193.28:0/408499278 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:24.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96ceffd640 1 --2- 10.20.193.28:0/408499278 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f9111a50 0x5568f916b530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96cf7fe640 1 --2- 10.20.193.28:0/408499278 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111140 0x5568f91591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96cffff640 1 -- 10.20.193.28:0/408499278 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f9111a50 msgr2=0x5568f916b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96cffff640 1 --2- 10.20.193.28:0/408499278 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f9111a50 0x5568f916b530 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96cffff640 1 -- 10.20.193.28:0/408499278 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111140 msgr2=0x5568f91591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96cffff640 1 --2- 10.20.193.28:0/408499278 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111140 0x5568f91591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96cffff640 1 -- 10.20.193.28:0/408499278 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568f8fadbd0 con 0x5568f91566f0 2026-02-25T11:58:24.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96cffff640 1 --2- 10.20.193.28:0/408499278 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f91566f0 0x5568f916ba70 secure :-1 s=READY pgs=907 gs=3 cs=0 l=1 c_cookie=3547961067012059004 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96c4006e20 tx=0x7f96c4032040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313ce3cae42b6f7c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:24.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96937fe640 1 -- 10.20.193.28:0/408499278 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f96c4030cf0 con 0x5568f91566f0 2026-02-25T11:58:24.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96937fe640 1 -- 10.20.193.28:0/408499278 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f96c403f040 con 0x5568f91566f0 2026-02-25T11:58:24.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.929+0000 7f96937fe640 1 -- 10.20.193.28:0/408499278 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f96c4003a70 con 0x5568f91566f0 2026-02-25T11:58:24.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.930+0000 7f96d7af4e00 1 -- 10.20.193.28:0/408499278 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f91566f0 msgr2=0x5568f916ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.930+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/408499278 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f91566f0 0x5568f916ba70 secure :-1 s=READY pgs=907 gs=3 cs=0 l=1 c_cookie=3547961067012059004 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96c4006e20 tx=0x7f96c4032040 comp rx=0 tx=0).stop 2026-02-25T11:58:24.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.930+0000 7f96d7af4e00 1 -- 10.20.193.28:0/408499278 shutdown_connections 2026-02-25T11:58:24.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.930+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/408499278 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f91566f0 0x5568f916ba70 unknown :-1 s=CLOSED pgs=907 gs=3 cs=0 l=1 c_cookie=3547961067012059004 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.930+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/408499278 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f9111a50 0x5568f916b530 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.930+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/408499278 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111140 0x5568f91591e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.930+0000 7f96d7af4e00 1 -- 10.20.193.28:0/408499278 >> 10.20.193.28:0/408499278 conn(0x5568f9030d40 msgr2=0x5568f9158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:24.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.930+0000 7f96d7af4e00 1 -- 10.20.193.28:0/408499278 shutdown_connections 2026-02-25T11:58:24.934 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.930+0000 7f96d7af4e00 1 -- 10.20.193.28:0/408499278 wait complete. 2026-02-25T11:58:24.936 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.931+0000 7f96d7af4e00 1 -- start start 2026-02-25T11:58:24.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.931+0000 7f96d7af4e00 1 Processor -- start 2026-02-25T11:58:24.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.931+0000 7f96d7af4e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f9111140 0x5568f9151990 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.931+0000 7f96d7af4e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111a50 0x5568f90ef220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.931+0000 7f96d7af4e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f91566f0 0x5568f90ef760 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.931+0000 7f96d7af4e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5568f90ec490 con 0x5568f9111140 2026-02-25T11:58:24.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.931+0000 7f96d7af4e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5568f91bd6e0 con 0x5568f9111a50 2026-02-25T11:58:24.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.931+0000 7f96d7af4e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5568f90efca0 con 0x5568f91566f0 2026-02-25T11:58:24.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.931+0000 7f96cf7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f9111140 0x5568f9151990 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.931+0000 7f96cf7fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f9111140 0x5568f9151990 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:52480/0 (socket says 10.20.193.28:52480) 2026-02-25T11:58:24.937 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.931+0000 7f96cf7fe640 1 -- 10.20.193.28:0/2128486876 learned_addr learned my addr 10.20.193.28:0/2128486876 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:24.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.931+0000 7f96cffff640 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f91566f0 0x5568f90ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.931+0000 7f96ceffd640 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111a50 0x5568f90ef220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96cf7fe640 1 -- 10.20.193.28:0/2128486876 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f91566f0 msgr2=0x5568f90ef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96cf7fe640 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f91566f0 0x5568f90ef760 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96cf7fe640 1 -- 10.20.193.28:0/2128486876 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111a50 msgr2=0x5568f90ef220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:24.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96cf7fe640 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111a50 0x5568f90ef220 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96cf7fe640 1 -- 10.20.193.28:0/2128486876 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568f8fa1680 con 0x5568f9111140 2026-02-25T11:58:24.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96cffff640 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f91566f0 0x5568f90ef760 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:58:24.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96cf7fe640 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f9111140 0x5568f9151990 secure :-1 s=READY pgs=908 gs=1 cs=0 l=1 c_cookie=16827125882438636424 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96bc016040 tx=0x7f96bc009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e985f5d60d895388 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:24.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96917fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:58:24.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96917fa640 1 -- 10.20.193.28:0/2128486876 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f96bc03b070 con 0x5568f9111140 2026-02-25T11:58:24.938 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96917fa640 1 -- 10.20.193.28:0/2128486876 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f96bc004f20 con 0x5568f9111140 2026-02-25T11:58:24.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96917fa640 1 -- 10.20.193.28:0/2128486876 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f96bc01a7e0 con 0x5568f9111140 2026-02-25T11:58:24.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2128486876 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d017af70 con 0x5568f9111140 2026-02-25T11:58:24.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96917fa640 1 -- 10.20.193.28:0/2128486876 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f96bc01a940 con 0x5568f9111140 2026-02-25T11:58:24.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96d7af4e00 10 client.17643 _ll_register_callbacks cb 0x5568f8f9e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:24.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2128486876 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96bc009d20 con 0x5568f9111140 2026-02-25T11:58:24.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96917fa640 1 -- 10.20.193.28:0/2128486876 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f96bc004770 con 0x5568f9111140 2026-02-25T11:58:24.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96d7af4e00 10 client.17643 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:24.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.932+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2128486876 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5568f8fb33b0 con 0x5568f9111140 2026-02-25T11:58:24.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.933+0000 7f96917fa640 1 -- 10.20.193.28:0/2128486876 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f96bc00e3e0 con 0x5568f9111140 2026-02-25T11:58:24.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.933+0000 7f96d7af4e00 10 client.17643 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:24.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.933+0000 7f96d7af4e00 10 client.17643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:58:24.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.933+0000 7f96d7af4e00 20 client.17643 populate_metadata read hostname 'trial028' 2026-02-25T11:58:24.939 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.933+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568f9203d90 0x5568f9224240 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:24.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.933+0000 7f96d7af4e00 4 client.17643 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:58:24.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.933+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2128486876 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5568f8cb0f40 con 0x5568f9203d90 2026-02-25T11:58:24.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.933+0000 7f96ceffd640 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568f9203d90 0x5568f9224240 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:24.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96ceffd640 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568f9203d90 0x5568f9224240 crc :-1 s=READY pgs=183 gs=4 cs=0 l=0 c_cookie=17270675236711887511 s_cookie=2669296084522595747 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efadc39fb358f297 server_cookie=250b3ea5764f75a3 in_seq=0 out_seq=0 2026-02-25T11:58:24.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96917fa640 10 client.17643 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:24.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96917fa640 1 -- 10.20.193.28:0/2128486876 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6250 (crc 0 0 0) 0x5568f8cb0f40 con 0x5568f9203d90 2026-02-25T11:58:24.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96917fa640 10 client.17643 handle_command_reply: tid=0 2026-02-25T11:58:24.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96d7af4e00 1 client.17643 shutdown 2026-02-25T11:58:24.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2128486876 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f9111140 msgr2=0x5568f9151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f9111140 0x5568f9151990 secure :-1 s=READY pgs=908 gs=1 cs=0 l=1 c_cookie=16827125882438636424 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96bc016040 tx=0x7f96bc009710 comp rx=0 tx=0).stop 2026-02-25T11:58:24.940 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2128486876 shutdown_connections 2026-02-25T11:58:24.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5568f9203d90 0x5568f9224240 crc :-1 s=READY pgs=183 gs=4 cs=0 l=0 c_cookie=17270675236711887511 s_cookie=2669296084522595747 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5568f91566f0 0x5568f90ef760 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5568f9111a50 0x5568f90ef220 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.941 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96917fa640 0 client.17643 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:24.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96d7af4e00 1 --2- 10.20.193.28:0/2128486876 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5568f9111140 0x5568f9151990 unknown :-1 s=CLOSED pgs=908 gs=1 cs=0 l=1 c_cookie=16827125882438636424 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.934+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2128486876 >> 10.20.193.28:0/2128486876 conn(0x5568f9030d40 msgr2=0x5568f91726f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:24.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2128486876 shutdown_connections 2026-02-25T11:58:24.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d7af4e00 1 -- 10.20.193.28:0/2128486876 wait complete. 2026-02-25T11:58:24.942 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d7af4e00 20 client.17643 trim_cache size 0 max 16384 2026-02-25T11:58:24.942 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:58:24.942 INFO:teuthology.orchestra.run.trial028.stdout: "status": "IDLE (waiting for more scrubs)", 2026-02-25T11:58:24.942 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": { 2026-02-25T11:58:24.943 INFO:teuthology.orchestra.run.trial028.stdout: "b897dba9-e832-4e59-932d-6585e9a0071e": { 2026-02-25T11:58:24.943 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.943 INFO:teuthology.orchestra.run.trial028.stdout: "tag": "b897dba9-e832-4e59-932d-6585e9a0071e", 2026-02-25T11:58:24.943 INFO:teuthology.orchestra.run.trial028.stdout: "options": "recursive,force" 2026-02-25T11:58:24.943 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:24.943 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.943 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:58:24.943 INFO:teuthology.orchestra.run.trial028.stdout: "b897dba9-e832-4e59-932d-6585e9a0071e": { 2026-02-25T11:58:24.943 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:58:24.433973+0000", 2026-02-25T11:58:24.943 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.943 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.944 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.944 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.944 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.944 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.944 INFO:teuthology.orchestra.run.trial028.stdout: "52c38620-9fde-44bc-acbe-a9a9e3452e53": { 2026-02-25T11:58:24.944 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:57:49.795729+0000", 2026-02-25T11:58:24.944 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.944 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.944 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.944 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.944 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.944 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.945 INFO:teuthology.orchestra.run.trial028.stdout: "0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8": { 2026-02-25T11:58:24.945 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:05.873000+0000", 2026-02-25T11:58:24.945 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.945 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.945 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.945 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.945 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.945 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.945 INFO:teuthology.orchestra.run.trial028.stdout: "850bcfd3-be8a-4930-896c-1ed6e48b8fed": { 2026-02-25T11:58:24.945 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:27.124813+0000", 2026-02-25T11:58:24.945 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.945 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.946 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.946 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.946 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.946 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.946 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:58:24.946 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:58:24.946 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.946 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.946 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.946 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.946 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.946 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.946 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:58:24.947 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:58:24.947 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.947 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.947 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.947 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.947 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.947 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.947 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:58:24.947 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:58:24.947 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.947 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.947 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.948 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.948 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.948 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.948 INFO:teuthology.orchestra.run.trial028.stdout: "516f576f-224f-4d5c-9352-d469d781f97c": { 2026-02-25T11:58:24.948 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:57:15.169491+0000", 2026-02-25T11:58:24.948 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.948 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.948 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.948 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.948 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.948 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.948 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:58:24.948 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:58:24.949 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.949 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.949 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.949 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.949 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.949 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.949 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:58:24.949 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:58:24.949 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.949 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.949 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.950 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.950 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.950 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.950 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:58:24.950 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:58:24.950 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.950 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.950 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.950 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.950 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.950 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.950 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:58:24.950 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:58:24.951 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.951 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.951 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.951 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.951 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.951 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.951 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:58:24.951 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:58:24.951 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.951 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.952 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.952 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.952 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.952 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.952 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:58:24.952 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:58:24.952 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.952 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.952 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.952 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.952 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.952 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.953 INFO:teuthology.orchestra.run.trial028.stdout: "91e989b7-1f1f-41fb-9047-642867de91e6": { 2026-02-25T11:58:24.953 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:40.578257+0000", 2026-02-25T11:58:24.953 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.953 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.953 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.953 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.953 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.953 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.953 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:58:24.953 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:58:24.953 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.954 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.955 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.956 INFO:teuthology.orchestra.run.trial028.stdout: "7b237934-6b95-43dd-b3a6-82b74134ec55": { 2026-02-25T11:58:24.956 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:31.289725+0000", 2026-02-25T11:58:24.956 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.956 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.956 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.956 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.956 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.956 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:24.956 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:58:24.956 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:58:24.956 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:24.956 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:24.957 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:24.957 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:24.957 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:24.957 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:24.957 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:24.957 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:58:24.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d4870640 1 -- 10.20.193.28:0/3822311628 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f96a40b56e0 msgr2=0x7f96a40d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d4870640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f96a40b56e0 0x7f96a40d5b90 secure :-1 s=READY pgs=397 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96bc004740 tx=0x7f96bc005f90 comp rx=0 tx=0).stop 2026-02-25T11:58:24.957 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d4870640 1 -- 10.20.193.28:0/3822311628 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96d0114a90 msgr2=0x7f96d01229f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:24.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d4870640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96d0114a90 0x7f96d01229f0 secure :-1 s=READY pgs=905 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f96c0005940 tx=0x7f96c000ee60 comp rx=0 tx=0).stop 2026-02-25T11:58:24.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d4870640 1 -- 10.20.193.28:0/3822311628 shutdown_connections 2026-02-25T11:58:24.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d4870640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f96a40b56e0 0x7f96a40d5b90 unknown :-1 s=CLOSED pgs=397 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d4870640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f96d0119d30 0x7f96d0127dc0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d4870640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f96d0114a90 0x7f96d01229f0 unknown :-1 s=CLOSED pgs=905 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d4870640 1 --2- 10.20.193.28:0/3822311628 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f96d01140a0 0x7f96d01224b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:24.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d4870640 1 -- 10.20.193.28:0/3822311628 >> 10.20.193.28:0/3822311628 conn(0x7f96d0064c40 msgr2=0x7f96d0065fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:24.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d4870640 1 -- 10.20.193.28:0/3822311628 shutdown_connections 2026-02-25T11:58:24.958 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:24.935+0000 7f96d4870640 1 -- 10.20.193.28:0/3822311628 wait complete. 2026-02-25T11:58:25.037 INFO:tasks.cephfs.filesystem:scrub status for tag:b897dba9-e832-4e59-932d-6585e9a0071e - {'path': '/', 'tag': 'b897dba9-e832-4e59-932d-6585e9a0071e', 'options': 'recursive,force'} 2026-02-25T11:58:25.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:25 trial167 ceph-mon[43392]: scrub queued for path: / 2026-02-25T11:58:25.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:25 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:58:25.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:25 trial167 ceph-mon[43392]: scrub summary: active paths [/] 2026-02-25T11:58:25.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:25 trial167 ceph-mon[43392]: scrub summary: idle+waiting paths [/] 2026-02-25T11:58:25.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:25 trial119 ceph-mon[42710]: scrub queued for path: / 2026-02-25T11:58:25.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:25 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:58:25.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:25 trial119 ceph-mon[42710]: scrub summary: active paths [/] 2026-02-25T11:58:25.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:25 trial119 ceph-mon[42710]: scrub summary: idle+waiting paths [/] 2026-02-25T11:58:25.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:25 trial028 ceph-mon[39224]: scrub queued for path: / 2026-02-25T11:58:25.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:25 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:58:25.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:25 trial028 ceph-mon[39224]: scrub summary: active paths [/] 2026-02-25T11:58:25.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:25 trial028 ceph-mon[39224]: scrub summary: idle+waiting paths [/] 2026-02-25T11:58:26.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:26 trial167 ceph-mon[43392]: pgmap v532: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:58:26.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:26 trial119 ceph-mon[42710]: pgmap v532: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:58:26.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:26 trial028 ceph-mon[39224]: pgmap v532: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 18 KiB/s rd, 22 KiB/s wr, 13 op/s 2026-02-25T11:58:26.859 INFO:tasks.workunit.client.0.trial028.stdout:561/600: open fd = 3 2026-02-25T11:58:26.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.859 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.859 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:562/600: open fd = 3 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:563/600: open fd = 3 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:564/600: open fd = 3 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:565/600: open fd = 3 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:566/600: open fd = 3 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:567/600: open fd = 3 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:568/600: open fd = 3 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:569/600: open fd = 3 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:570/600: open fd = 3 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.860 INFO:tasks.workunit.client.0.trial028.stdout:571/600: open fd = 3 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:572/600: open fd = 3 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:573/600: open fd = 3 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:574/600: open fd = 3 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:575/600: open fd = 3 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:576/600: open fd = 3 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:577/600: open fd = 3 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:578/600: open fd = 3 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:579/600: open fd = 3 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:580/600: open fd = 3 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.861 INFO:tasks.workunit.client.0.trial028.stdout:581/600: open fd = 3 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:582/600: open fd = 3 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:583/600: open fd = 3 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:584/600: open fd = 3 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:585/600: open fd = 3 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:586/600: open fd = 3 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:587/600: open fd = 3 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:588/600: open fd = 3 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:589/600: open fd = 3 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:590/600: open fd = 3 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.862 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:591/600: open fd = 3 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:592/600: open fd = 3 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:593/600: open fd = 3 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:594/600: open fd = 3 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:595/600: open fd = 3 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:596/600: open fd = 3 2026-02-25T11:58:26.863 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:597/600: open fd = 3 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:598/600: open fd = 3 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:599/600: open fd = 3 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:600/600: open fd = 3 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:write ret = 32 2026-02-25T11:58:26.864 INFO:tasks.workunit.client.0.trial028.stdout:pread ret = 64 2026-02-25T11:58:26.865 INFO:tasks.workunit.client.0.trial028.stdout:OK 2026-02-25T11:58:26.865 INFO:tasks.workunit.client.0.trial028.stderr:+ echo OK 2026-02-25T11:58:26.865 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-25T11:58:26.866 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-25T11:58:26.933 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2026-02-25T11:58:26.934 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-25T11:58:27.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:27 trial167 ceph-mon[43392]: scrub summary: idle 2026-02-25T11:58:27.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:27 trial167 ceph-mon[43392]: pgmap v533: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:58:27.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:27 trial119 ceph-mon[42710]: scrub summary: idle 2026-02-25T11:58:27.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:27 trial119 ceph-mon[42710]: pgmap v533: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:58:27.809 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:27 trial028 ceph-mon[39224]: scrub summary: idle 2026-02-25T11:58:27.810 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:27 trial028 ceph-mon[39224]: pgmap v533: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 14 KiB/s rd, 16 KiB/s wr, 11 op/s 2026-02-25T11:58:28.113 DEBUG:teuthology.parallel:result is None 2026-02-25T11:58:28.113 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-25T11:58:28.177 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-25T11:58:28.177 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2026-02-25T11:58:28.187 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2026-02-25T11:58:30.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:30 trial028 ceph-mon[39224]: pgmap v534: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 13 op/s 2026-02-25T11:58:30.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:30 trial167 ceph-mon[43392]: pgmap v534: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 13 op/s 2026-02-25T11:58:30.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:30 trial119 ceph-mon[42710]: pgmap v534: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 17 KiB/s rd, 21 KiB/s wr, 13 op/s 2026-02-25T11:58:32.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:32 trial028 ceph-mon[39224]: pgmap v535: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 11 KiB/s rd, 14 KiB/s wr, 10 op/s 2026-02-25T11:58:32.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:32 trial167 ceph-mon[43392]: pgmap v535: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 11 KiB/s rd, 14 KiB/s wr, 10 op/s 2026-02-25T11:58:32.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:32 trial119 ceph-mon[42710]: pgmap v535: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 11 KiB/s rd, 14 KiB/s wr, 10 op/s 2026-02-25T11:58:34.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:34 trial028 ceph-mon[39224]: pgmap v536: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 11 op/s 2026-02-25T11:58:34.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:34 trial167 ceph-mon[43392]: pgmap v536: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 11 op/s 2026-02-25T11:58:34.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:34 trial119 ceph-mon[42710]: pgmap v536: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 11 op/s 2026-02-25T11:58:36.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:36 trial028 ceph-mon[39224]: pgmap v537: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 11 op/s 2026-02-25T11:58:36.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:36 trial167 ceph-mon[43392]: pgmap v537: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 11 op/s 2026-02-25T11:58:36.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:36 trial119 ceph-mon[42710]: pgmap v537: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 13 KiB/s rd, 14 KiB/s wr, 11 op/s 2026-02-25T11:58:38.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:38 trial028 ceph-mon[39224]: pgmap v538: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 8.7 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2026-02-25T11:58:38.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:38 trial167 ceph-mon[43392]: pgmap v538: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 8.7 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2026-02-25T11:58:38.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:38 trial119 ceph-mon[42710]: pgmap v538: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 8.7 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2026-02-25T11:58:40.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:40 trial028 ceph-mon[39224]: pgmap v539: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 8.3 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2026-02-25T11:58:40.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:40 trial167 ceph-mon[43392]: pgmap v539: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 8.3 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2026-02-25T11:58:40.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:40 trial119 ceph-mon[42710]: pgmap v539: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 8.3 KiB/s rd, 8.2 KiB/s wr, 10 op/s 2026-02-25T11:58:42.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:42 trial028 ceph-mon[39224]: pgmap v540: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2026-02-25T11:58:42.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:42 trial167 ceph-mon[43392]: pgmap v540: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2026-02-25T11:58:42.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:42 trial119 ceph-mon[42710]: pgmap v540: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 4.3 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2026-02-25T11:58:44.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:44 trial028 ceph-mon[39224]: pgmap v541: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2026-02-25T11:58:44.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:44 trial167 ceph-mon[43392]: pgmap v541: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2026-02-25T11:58:44.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:44 trial119 ceph-mon[42710]: pgmap v541: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 5.6 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2026-02-25T11:58:46.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:46 trial028 ceph-mon[39224]: pgmap v542: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 4.5 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2026-02-25T11:58:46.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:46 trial167 ceph-mon[43392]: pgmap v542: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 4.5 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2026-02-25T11:58:46.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:46 trial119 ceph-mon[42710]: pgmap v542: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 4.5 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2026-02-25T11:58:48.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:48 trial028 ceph-mon[39224]: pgmap v543: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2026-02-25T11:58:48.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:48 trial167 ceph-mon[43392]: pgmap v543: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2026-02-25T11:58:48.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:48 trial119 ceph-mon[42710]: pgmap v543: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2026-02-25T11:58:50.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:50 trial028 ceph-mon[39224]: pgmap v544: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2026-02-25T11:58:50.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:50 trial167 ceph-mon[43392]: pgmap v544: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2026-02-25T11:58:50.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:50 trial119 ceph-mon[42710]: pgmap v544: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 6 op/s 2026-02-25T11:58:52.552 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:52 trial167 ceph-mon[43392]: pgmap v545: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2026-02-25T11:58:52.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:52 trial028 ceph-mon[39224]: pgmap v545: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2026-02-25T11:58:52.561 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:52 trial119 ceph-mon[42710]: pgmap v545: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2026-02-25T11:58:54.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:54 trial028 ceph-mon[39224]: pgmap v546: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2026-02-25T11:58:54.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:54 trial167 ceph-mon[43392]: pgmap v546: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2026-02-25T11:58:54.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:54 trial119 ceph-mon[42710]: pgmap v546: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 4.1 KiB/s rd, 341 B/s wr, 6 op/s 2026-02-25T11:58:55.038 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 scrub status 2026-02-25T11:58:55.227 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:58:55.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.414+0000 7f562bad8640 1 -- 10.20.193.28:0/4004379935 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f562410b540 msgr2=0x7f562410b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:55.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.414+0000 7f562bad8640 1 --2- 10.20.193.28:0/4004379935 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f562410b540 0x7f562410b910 secure :-1 s=READY pgs=909 gs=1 cs=0 l=1 c_cookie=12440224347859139099 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5614009950 tx=0x7f561402f190 comp rx=0 tx=0).stop 2026-02-25T11:58:55.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.414+0000 7f562bad8640 1 -- 10.20.193.28:0/4004379935 shutdown_connections 2026-02-25T11:58:55.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.414+0000 7f562bad8640 1 --2- 10.20.193.28:0/4004379935 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5624114400 0x7f5624175f30 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.414+0000 7f562bad8640 1 --2- 10.20.193.28:0/4004379935 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f562410bee0 0x7f5624113c80 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.414+0000 7f562bad8640 1 --2- 10.20.193.28:0/4004379935 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f562410b540 0x7f562410b910 unknown :-1 s=CLOSED pgs=909 gs=1 cs=0 l=1 c_cookie=12440224347859139099 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.414+0000 7f562bad8640 1 -- 10.20.193.28:0/4004379935 >> 10.20.193.28:0/4004379935 conn(0x7f56240e8fb0 msgr2=0x7f56241093c0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:55.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.415+0000 7f562bad8640 1 -- 10.20.193.28:0/4004379935 shutdown_connections 2026-02-25T11:58:55.417 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.415+0000 7f562bad8640 1 -- 10.20.193.28:0/4004379935 wait complete. 2026-02-25T11:58:55.418 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.415+0000 7f562bad8640 1 Processor -- start 2026-02-25T11:58:55.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.415+0000 7f562bad8640 1 -- start start 2026-02-25T11:58:55.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562bad8640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f562410b540 0x7f56240f1980 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562bad8640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f562410bee0 0x7f5624175ea0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562bad8640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5624114400 0x7f56240f2140 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562bad8640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f56240f7320 con 0x7f562410bee0 2026-02-25T11:58:55.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562bad8640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56240f7490 con 0x7f5624114400 2026-02-25T11:58:55.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562bad8640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56240f7650 con 0x7f562410b540 2026-02-25T11:58:55.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562904c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f562410bee0 0x7f5624175ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.419 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562984d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f562410b540 0x7f56240f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562904c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f562410bee0 0x7f5624175ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:32946/0 (socket says 10.20.193.28:32946) 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562904c640 1 -- 10.20.193.28:0/3672594850 learned_addr learned my addr 10.20.193.28:0/3672594850 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562a04e640 1 --2- 10.20.193.28:0/3672594850 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5624114400 0x7f56240f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562904c640 1 -- 10.20.193.28:0/3672594850 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f562410b540 msgr2=0x7f56240f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562904c640 1 --2- 10.20.193.28:0/3672594850 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f562410b540 0x7f56240f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562904c640 1 -- 10.20.193.28:0/3672594850 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5624114400 msgr2=0x7f56240f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562904c640 1 --2- 10.20.193.28:0/3672594850 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5624114400 0x7f56240f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562904c640 1 -- 10.20.193.28:0/3672594850 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5614009570 con 0x7f562410bee0 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.416+0000 7f562904c640 1 --2- 10.20.193.28:0/3672594850 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f562410bee0 0x7f5624175ea0 secure :-1 s=READY pgs=910 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f561800d950 tx=0x7f561800de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.417+0000 7f5612ffd640 1 -- 10.20.193.28:0/3672594850 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5618004500 con 0x7f562410bee0 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.417+0000 7f5612ffd640 1 -- 10.20.193.28:0/3672594850 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f561800bd00 con 0x7f562410bee0 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.417+0000 7f5612ffd640 1 -- 10.20.193.28:0/3672594850 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56180106e0 con 0x7f562410bee0 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.417+0000 7f562bad8640 1 -- 10.20.193.28:0/3672594850 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56240f7930 con 0x7f562410bee0 2026-02-25T11:58:55.420 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.417+0000 7f562bad8640 1 -- 10.20.193.28:0/3672594850 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56240f7db0 con 0x7f562410bee0 2026-02-25T11:58:55.422 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.420+0000 7f5612ffd640 1 -- 10.20.193.28:0/3672594850 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f561800b840 con 0x7f562410bee0 2026-02-25T11:58:55.423 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.422+0000 7f5612ffd640 1 --2- 10.20.193.28:0/3672594850 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56000b5760 0x7f56000d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.422+0000 7f5612ffd640 1 -- 10.20.193.28:0/3672594850 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f56180e64b0 con 0x7f562410bee0 2026-02-25T11:58:55.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.422+0000 7f562984d640 1 --2- 10.20.193.28:0/3672594850 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56000b5760 0x7f56000d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.424 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.423+0000 7f562984d640 1 --2- 10.20.193.28:0/3672594850 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56000b5760 0x7f56000d5c10 secure :-1 s=READY pgs=399 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5614005b30 tx=0x7f561403a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:55.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.423+0000 7f562dad3e00 1 Processor -- start 2026-02-25T11:58:55.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.423+0000 7f562dad3e00 1 -- start start 2026-02-25T11:58:55.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.423+0000 7f562dad3e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1f140 0x562156d1f510 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.423+0000 7f562dad3e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1fa50 0x562156d641b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.423+0000 7f562dad3e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d646f0 0x562156d84ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.423+0000 7f562dad3e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562156c359b0 con 0x562156d646f0 2026-02-25T11:58:55.425 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.423+0000 7f562dad3e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562156c316e0 con 0x562156d1f140 2026-02-25T11:58:55.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.423+0000 7f562dad3e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562156c2b840 con 0x562156d1fa50 2026-02-25T11:58:55.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f562a04e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d646f0 0x562156d84ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f562984d640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1f140 0x562156d1f510 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f562a04e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d646f0 0x562156d84ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:32958/0 (socket says 10.20.193.28:32958) 2026-02-25T11:58:55.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f562a04e640 1 -- 10.20.193.28:0/4146781752 learned_addr learned my addr 10.20.193.28:0/4146781752 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:55.426 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f562904c640 1 --2- 10.20.193.28:0/4146781752 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1fa50 0x562156d641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f562a04e640 1 -- 10.20.193.28:0/4146781752 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1fa50 msgr2=0x562156d641b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:55.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f562a04e640 1 --2- 10.20.193.28:0/4146781752 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1fa50 0x562156d641b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f562a04e640 1 -- 10.20.193.28:0/4146781752 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1f140 msgr2=0x562156d1f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:55.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f562a04e640 1 --2- 10.20.193.28:0/4146781752 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1f140 0x562156d1f510 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f562a04e640 1 -- 10.20.193.28:0/4146781752 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562156bd6d80 con 0x562156d646f0 2026-02-25T11:58:55.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f562a04e640 1 --2- 10.20.193.28:0/4146781752 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d646f0 0x562156d84ad0 secure :-1 s=READY pgs=911 gs=2 cs=0 l=1 c_cookie=15164532818344585766 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56200055b0 tx=0x7f5620003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2733e40ce2eaa26 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:55.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f55f24e6640 1 -- 10.20.193.28:0/4146781752 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5620003e30 con 0x562156d646f0 2026-02-25T11:58:55.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f55f24e6640 1 -- 10.20.193.28:0/4146781752 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f562000ace0 con 0x562156d646f0 2026-02-25T11:58:55.427 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.424+0000 7f55f24e6640 1 -- 10.20.193.28:0/4146781752 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f562000f760 con 0x562156d646f0 2026-02-25T11:58:55.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.425+0000 7f562dad3e00 1 -- 10.20.193.28:0/4146781752 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d646f0 msgr2=0x562156d84ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:55.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.425+0000 7f562dad3e00 1 --2- 10.20.193.28:0/4146781752 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d646f0 0x562156d84ad0 secure :-1 s=READY pgs=911 gs=2 cs=0 l=1 c_cookie=15164532818344585766 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56200055b0 tx=0x7f5620003c30 comp rx=0 tx=0).stop 2026-02-25T11:58:55.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.425+0000 7f562dad3e00 1 -- 10.20.193.28:0/4146781752 shutdown_connections 2026-02-25T11:58:55.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.425+0000 7f562dad3e00 1 --2- 10.20.193.28:0/4146781752 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d646f0 0x562156d84ad0 unknown :-1 s=CLOSED pgs=911 gs=2 cs=0 l=1 c_cookie=15164532818344585766 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.425+0000 7f562dad3e00 1 --2- 10.20.193.28:0/4146781752 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1fa50 0x562156d641b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.425+0000 7f562dad3e00 1 --2- 10.20.193.28:0/4146781752 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1f140 0x562156d1f510 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.425+0000 7f562dad3e00 1 -- 10.20.193.28:0/4146781752 >> 10.20.193.28:0/4146781752 conn(0x562156c3ed40 msgr2=0x562156d1da20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:55.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.425+0000 7f562dad3e00 1 -- 10.20.193.28:0/4146781752 shutdown_connections 2026-02-25T11:58:55.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.425+0000 7f562dad3e00 1 -- 10.20.193.28:0/4146781752 wait complete. 2026-02-25T11:58:55.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.426+0000 7f562dad3e00 1 -- start start 2026-02-25T11:58:55.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.426+0000 7f562dad3e00 1 Processor -- start 2026-02-25T11:58:55.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562dad3e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1f140 0x562156d7c350 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562dad3e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1fa50 0x562156d7c890 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562dad3e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d646f0 0x562156d81880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562dad3e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562156d81dc0 con 0x562156d646f0 2026-02-25T11:58:55.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562dad3e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562156d81f30 con 0x562156d1fa50 2026-02-25T11:58:55.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562dad3e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562156d820f0 con 0x562156d1f140 2026-02-25T11:58:55.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562984d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1f140 0x562156d7c350 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562904c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1fa50 0x562156d7c890 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562904c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1fa50 0x562156d7c890 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:44874/0 (socket says 10.20.193.28:44874) 2026-02-25T11:58:55.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562984d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1f140 0x562156d7c350 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:37442/0 (socket says 10.20.193.28:37442) 2026-02-25T11:58:55.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562904c640 1 -- 10.20.193.28:0/984330366 learned_addr learned my addr 10.20.193.28:0/984330366 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:55.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562a04e640 1 --2- 10.20.193.28:0/984330366 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d646f0 0x562156d81880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562904c640 1 -- 10.20.193.28:0/984330366 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1f140 msgr2=0x562156d7c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:55.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562904c640 1 --2- 10.20.193.28:0/984330366 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1f140 0x562156d7c350 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562904c640 1 -- 10.20.193.28:0/984330366 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d646f0 msgr2=0x562156d81880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:55.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562904c640 1 --2- 10.20.193.28:0/984330366 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d646f0 0x562156d81880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562904c640 1 -- 10.20.193.28:0/984330366 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562156baf350 con 0x562156d1fa50 2026-02-25T11:58:55.430 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.427+0000 7f562a04e640 1 --2- 10.20.193.28:0/984330366 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d646f0 0x562156d81880 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:58:55.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f562904c640 1 --2- 10.20.193.28:0/984330366 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1fa50 0x562156d7c890 secure :-1 s=READY pgs=186 gs=1 cs=0 l=1 c_cookie=17525784419838491164 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f561800b630 tx=0x7f56180a9000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3381813028c7a1c server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:55.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f55df7fe640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:58:55.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f55df7fe640 1 -- 10.20.193.28:0/984330366 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56180e62f0 con 0x562156d1fa50 2026-02-25T11:58:55.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f55df7fe640 1 -- 10.20.193.28:0/984330366 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f56180ac0e0 con 0x562156d1fa50 2026-02-25T11:58:55.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f562dad3e00 1 -- 10.20.193.28:0/984330366 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5620005250 con 0x562156d1fa50 2026-02-25T11:58:55.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f55df7fe640 1 -- 10.20.193.28:0/984330366 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56180b7760 con 0x562156d1fa50 2026-02-25T11:58:55.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f562dad3e00 10 client.27308 _ll_register_callbacks cb 0x562156bac9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:55.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f562dad3e00 1 -- 10.20.193.28:0/984330366 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5621568c16f0 con 0x562156d1fa50 2026-02-25T11:58:55.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f55df7fe640 1 -- 10.20.193.28:0/984330366 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f56180b7970 con 0x562156d1fa50 2026-02-25T11:58:55.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f55df7fe640 1 -- 10.20.193.28:0/984330366 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f56180aa8c0 con 0x562156d1fa50 2026-02-25T11:58:55.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f562dad3e00 10 client.27308 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f562dad3e00 1 -- 10.20.193.28:0/984330366 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x562156bc1150 con 0x562156d1fa50 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f55df7fe640 1 -- 10.20.193.28:0/984330366 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f56180ae220 con 0x562156d1fa50 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f562dad3e00 10 client.27308 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f562dad3e00 10 client.27308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f562dad3e00 20 client.27308 populate_metadata read hostname 'trial028' 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.428+0000 7f562dad3e00 1 --2- 10.20.193.28:0/984330366 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562156d847e0 0x562156d62470 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.429+0000 7f562dad3e00 4 client.27308 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.429+0000 7f562dad3e00 1 -- 10.20.193.28:0/984330366 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562156bfd910 con 0x562156d847e0 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.429+0000 7f562984d640 1 --2- 10.20.193.28:0/984330366 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562156d847e0 0x562156d62470 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.429+0000 7f562984d640 1 --2- 10.20.193.28:0/984330366 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562156d847e0 0x562156d62470 crc :-1 s=READY pgs=184 gs=4 cs=0 l=0 c_cookie=13084204925246721315 s_cookie=16508422414055870034 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5946f0e71bf4d23 server_cookie=e519b2b85fe58e52 in_seq=0 out_seq=0 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.429+0000 7f55df7fe640 10 client.27308 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f55df7fe640 1 -- 10.20.193.28:0/984330366 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x562156bfd910 con 0x562156d847e0 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f55df7fe640 10 client.27308 handle_command_reply: tid=0 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f562dad3e00 1 client.27308 shutdown 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f562dad3e00 1 -- 10.20.193.28:0/984330366 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1fa50 msgr2=0x562156d7c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:55.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f562dad3e00 1 --2- 10.20.193.28:0/984330366 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1fa50 0x562156d7c890 secure :-1 s=READY pgs=186 gs=1 cs=0 l=1 c_cookie=17525784419838491164 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f561800b630 tx=0x7f56180a9000 comp rx=0 tx=0).stop 2026-02-25T11:58:55.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f562dad3e00 1 -- 10.20.193.28:0/984330366 shutdown_connections 2026-02-25T11:58:55.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f562dad3e00 1 --2- 10.20.193.28:0/984330366 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562156d847e0 0x562156d62470 crc :-1 s=READY pgs=184 gs=4 cs=0 l=0 c_cookie=13084204925246721315 s_cookie=16508422414055870034 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f562dad3e00 1 --2- 10.20.193.28:0/984330366 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d646f0 0x562156d81880 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f562dad3e00 1 --2- 10.20.193.28:0/984330366 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1fa50 0x562156d7c890 unknown :-1 s=CLOSED pgs=186 gs=1 cs=0 l=1 c_cookie=17525784419838491164 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f562dad3e00 1 --2- 10.20.193.28:0/984330366 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1f140 0x562156d7c350 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f562dad3e00 1 -- 10.20.193.28:0/984330366 >> 10.20.193.28:0/984330366 conn(0x562156c3ed40 msgr2=0x562156d1d230 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:55.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f55df7fe640 0 client.27308 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:55.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f562dad3e00 1 -- 10.20.193.28:0/984330366 shutdown_connections 2026-02-25T11:58:55.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f562dad3e00 1 -- 10.20.193.28:0/984330366 wait complete. 2026-02-25T11:58:55.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.430+0000 7f562dad3e00 20 client.27308 trim_cache size 0 max 16384 2026-02-25T11:58:55.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.440+0000 7f562dad3e00 1 Processor -- start 2026-02-25T11:58:55.441 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.440+0000 7f562dad3e00 1 -- start start 2026-02-25T11:58:55.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.440+0000 7f562dad3e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1f140 0x562156d671e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.440+0000 7f562dad3e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 0x562156d69540 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.440+0000 7f562dad3e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d646f0 0x562156d639f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.440+0000 7f562dad3e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562156c29470 con 0x562156d1fa50 2026-02-25T11:58:55.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.440+0000 7f562dad3e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5621568e9700 con 0x562156d646f0 2026-02-25T11:58:55.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.440+0000 7f562dad3e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562156c936e0 con 0x562156d1f140 2026-02-25T11:58:55.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.440+0000 7f562904c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 0x562156d69540 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.440+0000 7f562904c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 0x562156d69540 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:32966/0 (socket says 10.20.193.28:32966) 2026-02-25T11:58:55.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.440+0000 7f562904c640 1 -- 10.20.193.28:0/2176915228 learned_addr learned my addr 10.20.193.28:0/2176915228 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:55.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.440+0000 7f562984d640 1 --2- 10.20.193.28:0/2176915228 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1f140 0x562156d671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.443 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562904c640 1 -- 10.20.193.28:0/2176915228 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1f140 msgr2=0x562156d671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:55.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562a04e640 1 --2- 10.20.193.28:0/2176915228 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d646f0 0x562156d639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562904c640 1 --2- 10.20.193.28:0/2176915228 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1f140 0x562156d671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562904c640 1 -- 10.20.193.28:0/2176915228 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d646f0 msgr2=0x562156d639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:55.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562904c640 1 --2- 10.20.193.28:0/2176915228 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d646f0 0x562156d639f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562904c640 1 -- 10.20.193.28:0/2176915228 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562156bbbbd0 con 0x562156d1fa50 2026-02-25T11:58:55.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562a04e640 1 --2- 10.20.193.28:0/2176915228 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d646f0 0x562156d639f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:58:55.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562904c640 1 --2- 10.20.193.28:0/2176915228 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 0x562156d69540 secure :-1 s=READY pgs=912 gs=2 cs=0 l=1 c_cookie=17908610445703872379 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56180aeac0 tx=0x7f5618070000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8882a5105077f7b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:55.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f55f1ce5640 1 -- 10.20.193.28:0/2176915228 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56180b7740 con 0x562156d1fa50 2026-02-25T11:58:55.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f55f1ce5640 1 -- 10.20.193.28:0/2176915228 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5621568c16f0 con 0x562156d1fa50 2026-02-25T11:58:55.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f55f1ce5640 1 -- 10.20.193.28:0/2176915228 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56180b7970 con 0x562156d1fa50 2026-02-25T11:58:55.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562dad3e00 1 -- 10.20.193.28:0/2176915228 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 msgr2=0x562156d69540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:55.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562dad3e00 1 --2- 10.20.193.28:0/2176915228 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 0x562156d69540 secure :-1 s=READY pgs=912 gs=2 cs=0 l=1 c_cookie=17908610445703872379 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56180aeac0 tx=0x7f5618070000 comp rx=0 tx=0).stop 2026-02-25T11:58:55.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562dad3e00 1 -- 10.20.193.28:0/2176915228 shutdown_connections 2026-02-25T11:58:55.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562dad3e00 1 --2- 10.20.193.28:0/2176915228 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d646f0 0x562156d639f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562dad3e00 1 --2- 10.20.193.28:0/2176915228 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 0x562156d69540 unknown :-1 s=CLOSED pgs=912 gs=2 cs=0 l=1 c_cookie=17908610445703872379 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562dad3e00 1 --2- 10.20.193.28:0/2176915228 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d1f140 0x562156d671e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562dad3e00 1 -- 10.20.193.28:0/2176915228 >> 10.20.193.28:0/2176915228 conn(0x562156c3ed40 msgr2=0x562156d66ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:55.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562dad3e00 1 -- 10.20.193.28:0/2176915228 shutdown_connections 2026-02-25T11:58:55.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.441+0000 7f562dad3e00 1 -- 10.20.193.28:0/2176915228 wait complete. 2026-02-25T11:58:55.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.442+0000 7f562dad3e00 1 -- start start 2026-02-25T11:58:55.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.442+0000 7f562dad3e00 1 Processor -- start 2026-02-25T11:58:55.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.442+0000 7f562dad3e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1f140 0x562156cfd830 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.442+0000 7f562dad3e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 0x562156cfdd70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.442+0000 7f562dad3e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d646f0 0x562156dc93e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.442+0000 7f562dad3e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562156cfe4d0 con 0x562156d1fa50 2026-02-25T11:58:55.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.442+0000 7f562dad3e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562156cfe640 con 0x562156d1f140 2026-02-25T11:58:55.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.442+0000 7f562dad3e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562156dc9920 con 0x562156d646f0 2026-02-25T11:58:55.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f562904c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 0x562156cfdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f562904c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 0x562156cfdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:32976/0 (socket says 10.20.193.28:32976) 2026-02-25T11:58:55.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f562904c640 1 -- 10.20.193.28:0/1574113194 learned_addr learned my addr 10.20.193.28:0/1574113194 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:55.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f562984d640 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1f140 0x562156cfd830 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f562a04e640 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d646f0 0x562156dc93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f562904c640 1 -- 10.20.193.28:0/1574113194 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d646f0 msgr2=0x562156dc93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:55.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f562904c640 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d646f0 0x562156dc93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f562904c640 1 -- 10.20.193.28:0/1574113194 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1f140 msgr2=0x562156cfd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:55.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f562904c640 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1f140 0x562156cfd830 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f562904c640 1 -- 10.20.193.28:0/1574113194 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562156baf680 con 0x562156d1fa50 2026-02-25T11:58:55.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f562a04e640 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d646f0 0x562156dc93e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:58:55.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f562904c640 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 0x562156cfdd70 secure :-1 s=READY pgs=913 gs=2 cs=0 l=1 c_cookie=13236008676385440414 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56180ac8c0 tx=0x7f56180b2460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7afbfc387f1429e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:55.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f55df7fe640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:58:55.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f55df7fe640 1 -- 10.20.193.28:0/1574113194 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f561800bb20 con 0x562156d1fa50 2026-02-25T11:58:55.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f55df7fe640 1 -- 10.20.193.28:0/1574113194 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f56180960c0 con 0x562156d1fa50 2026-02-25T11:58:55.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f55df7fe640 1 -- 10.20.193.28:0/1574113194 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f56180b7970 con 0x562156d1fa50 2026-02-25T11:58:55.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f562dad3e00 1 -- 10.20.193.28:0/1574113194 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56180c8020 con 0x562156d1fa50 2026-02-25T11:58:55.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.443+0000 7f55df7fe640 1 -- 10.20.193.28:0/1574113194 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f56180d0120 con 0x562156d1fa50 2026-02-25T11:58:55.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f55df7fe640 10 client.17661 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:58:55.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f562dad3e00 10 client.17661 _ll_register_callbacks cb 0x562156bac9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:55.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f562dad3e00 1 -- 10.20.193.28:0/1574113194 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f562000ace0 con 0x562156d1fa50 2026-02-25T11:58:55.450 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f55df7fe640 1 -- 10.20.193.28:0/1574113194 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f56180d02e0 con 0x562156d1fa50 2026-02-25T11:58:55.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f562dad3e00 10 client.17661 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:55.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f562dad3e00 1 -- 10.20.193.28:0/1574113194 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x562156bc13b0 con 0x562156d1fa50 2026-02-25T11:58:55.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f55df7fe640 1 -- 10.20.193.28:0/1574113194 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f56180e6090 con 0x562156d1fa50 2026-02-25T11:58:55.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f562dad3e00 10 client.17661 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:55.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f562dad3e00 10 client.17661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:58:55.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f562dad3e00 20 client.17661 populate_metadata read hostname 'trial028' 2026-02-25T11:58:55.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f562dad3e00 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562156dcadf0 0x562156e31ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f562dad3e00 4 client.17661 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2026-02-25T11:58:55.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f562dad3e00 1 -- 10.20.193.28:0/1574113194 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5621568bef40 con 0x562156dcadf0 2026-02-25T11:58:55.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.444+0000 7f562984d640 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562156dcadf0 0x562156e31ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.451 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f562984d640 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562156dcadf0 0x562156e31ad0 crc :-1 s=READY pgs=185 gs=4 cs=0 l=0 c_cookie=16655908772301113612 s_cookie=13362842857782993210 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e725acc81852090c server_cookie=b9725ac79f85613a in_seq=0 out_seq=0 2026-02-25T11:58:55.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f55df7fe640 10 client.17661 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:55.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f55df7fe640 1 -- 10.20.193.28:0/1574113194 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6054 (crc 0 0 0) 0x5621568bef40 con 0x562156dcadf0 2026-02-25T11:58:55.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f55df7fe640 10 client.17661 handle_command_reply: tid=0 2026-02-25T11:58:55.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f562dad3e00 1 client.17661 shutdown 2026-02-25T11:58:55.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f562dad3e00 1 -- 10.20.193.28:0/1574113194 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 msgr2=0x562156cfdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:55.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f562dad3e00 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 0x562156cfdd70 secure :-1 s=READY pgs=913 gs=2 cs=0 l=1 c_cookie=13236008676385440414 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f56180ac8c0 tx=0x7f56180b2460 comp rx=0 tx=0).stop 2026-02-25T11:58:55.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f562dad3e00 1 -- 10.20.193.28:0/1574113194 shutdown_connections 2026-02-25T11:58:55.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f562dad3e00 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x562156dcadf0 0x562156e31ad0 crc :-1 s=READY pgs=185 gs=4 cs=0 l=0 c_cookie=16655908772301113612 s_cookie=13362842857782993210 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f562dad3e00 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562156d646f0 0x562156dc93e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f562dad3e00 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562156d1fa50 0x562156cfdd70 unknown :-1 s=CLOSED pgs=913 gs=2 cs=0 l=1 c_cookie=13236008676385440414 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f562dad3e00 1 --2- 10.20.193.28:0/1574113194 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562156d1f140 0x562156cfd830 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.452 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f55df7fe640 0 client.17661 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:55.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f562dad3e00 1 -- 10.20.193.28:0/1574113194 >> 10.20.193.28:0/1574113194 conn(0x562156c3ed40 msgr2=0x562156d629e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:55.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f562dad3e00 1 -- 10.20.193.28:0/1574113194 shutdown_connections 2026-02-25T11:58:55.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f562dad3e00 1 -- 10.20.193.28:0/1574113194 wait complete. 2026-02-25T11:58:55.453 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.445+0000 7f562dad3e00 20 client.17661 trim_cache size 0 max 16384 2026-02-25T11:58:55.453 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:58:55.453 INFO:teuthology.orchestra.run.trial028.stdout: "status": "no active scrubs running", 2026-02-25T11:58:55.454 INFO:teuthology.orchestra.run.trial028.stdout: "scrubs": {}, 2026-02-25T11:58:55.454 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_stats": { 2026-02-25T11:58:55.454 INFO:teuthology.orchestra.run.trial028.stdout: "b897dba9-e832-4e59-932d-6585e9a0071e": { 2026-02-25T11:58:55.454 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:58:24.433973+0000", 2026-02-25T11:58:55.454 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.454 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.454 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.454 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.454 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.454 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.454 INFO:teuthology.orchestra.run.trial028.stdout: "52c38620-9fde-44bc-acbe-a9a9e3452e53": { 2026-02-25T11:58:55.455 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:57:49.795729+0000", 2026-02-25T11:58:55.455 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.455 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.455 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.455 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.455 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.455 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.455 INFO:teuthology.orchestra.run.trial028.stdout: "0f8be1a9-2d86-40aa-bd10-cf2eb0a6f5e8": { 2026-02-25T11:58:55.455 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:05.873000+0000", 2026-02-25T11:58:55.455 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.455 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.456 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.456 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.456 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.456 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.456 INFO:teuthology.orchestra.run.trial028.stdout: "850bcfd3-be8a-4930-896c-1ed6e48b8fed": { 2026-02-25T11:58:55.456 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:27.124813+0000", 2026-02-25T11:58:55.456 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.456 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.456 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.456 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.456 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.457 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.457 INFO:teuthology.orchestra.run.trial028.stdout: "8c0edddc-5eeb-4011-a44d-a25962ab3520": { 2026-02-25T11:58:55.457 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:52.557532+0000", 2026-02-25T11:58:55.457 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.457 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.457 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.457 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.457 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.457 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.457 INFO:teuthology.orchestra.run.trial028.stdout: "feaa75b0-9031-483c-96ae-588afa74d2df": { 2026-02-25T11:58:55.457 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:27.125455+0000", 2026-02-25T11:58:55.457 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.458 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.458 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.458 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.458 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.458 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.458 INFO:teuthology.orchestra.run.trial028.stdout: "5a89f4dc-b298-4cb1-a5e4-ef808ea49090": { 2026-02-25T11:58:55.458 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:40.669019+0000", 2026-02-25T11:58:55.458 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.458 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.458 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.458 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.458 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.459 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.459 INFO:teuthology.orchestra.run.trial028.stdout: "516f576f-224f-4d5c-9352-d469d781f97c": { 2026-02-25T11:58:55.459 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:57:15.169491+0000", 2026-02-25T11:58:55.459 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.459 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.459 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.459 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.459 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.459 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.459 INFO:teuthology.orchestra.run.trial028.stdout: "df724da3-1c68-4141-ad99-f8332cf2f6c6": { 2026-02-25T11:58:55.459 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:48:31.196377+0000", 2026-02-25T11:58:55.459 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.459 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: "a45f08eb-b06d-4dcb-8fdc-f2ced26d8847": { 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:08.595727+0000", 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: "23922697-db31-470c-b97e-bd685d22642e": { 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:59.259181+0000", 2026-02-25T11:58:55.460 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.461 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.461 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.461 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.461 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.461 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.461 INFO:teuthology.orchestra.run.trial028.stdout: "92296b1b-2ea3-4dd6-a647-6be0790774fe": { 2026-02-25T11:58:55.461 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:15.281327+0000", 2026-02-25T11:58:55.461 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.461 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.461 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.461 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.461 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.461 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.462 INFO:teuthology.orchestra.run.trial028.stdout: "0134923a-7ba8-4d64-89bf-665ec8547ef9": { 2026-02-25T11:58:55.462 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:49:05.953702+0000", 2026-02-25T11:58:55.462 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.462 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.462 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.462 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.462 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.462 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.462 INFO:teuthology.orchestra.run.trial028.stdout: "36aaf888-2505-43bd-b817-3eafabc59fb7": { 2026-02-25T11:58:55.462 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:50:50.023618+0000", 2026-02-25T11:58:55.462 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.462 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.462 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.463 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.463 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.463 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.463 INFO:teuthology.orchestra.run.trial028.stdout: "91e989b7-1f1f-41fb-9047-642867de91e6": { 2026-02-25T11:58:55.463 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:56:40.578257+0000", 2026-02-25T11:58:55.463 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.463 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.463 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.463 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.463 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.463 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.463 INFO:teuthology.orchestra.run.trial028.stdout: "11ba92dd-980a-495c-a63e-d2ded1de1c9f": { 2026-02-25T11:58:55.463 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:51:24.603903+0000", 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: "3b1cc8e2-c9db-4e9d-9750-434809f320a2": { 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:47:52.382126+0000", 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.464 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "0d9e5421-597c-4231-88a4-8abf309c266b": { 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:53:43.148993+0000", 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "3c6fdafc-4d02-40d2-962e-24e4d16fa603": { 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:52:33.956455+0000", 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.465 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "7b237934-6b95-43dd-b3a6-82b74134ec55": { 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:55:31.289725+0000", 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "641fdd6e-6811-41fc-b473-48f98e83e9f2": { 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "start_time": "2026-02-25T11:54:17.867254+0000", 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "path": "/", 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_passed": 0, 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_failed": 0, 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_skipped": 0 2026-02-25T11:58:55.466 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:55.467 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:55.467 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:58:55.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.446+0000 7f562bad8640 1 -- 10.20.193.28:0/3672594850 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56000b5760 msgr2=0x7f56000d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:55.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.446+0000 7f562bad8640 1 --2- 10.20.193.28:0/3672594850 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56000b5760 0x7f56000d5c10 secure :-1 s=READY pgs=399 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5614005b30 tx=0x7f561403a040 comp rx=0 tx=0).stop 2026-02-25T11:58:55.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.446+0000 7f562bad8640 1 -- 10.20.193.28:0/3672594850 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f562410bee0 msgr2=0x7f5624175ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:55.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.446+0000 7f562bad8640 1 --2- 10.20.193.28:0/3672594850 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f562410bee0 0x7f5624175ea0 secure :-1 s=READY pgs=910 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f561800d950 tx=0x7f561800de20 comp rx=0 tx=0).stop 2026-02-25T11:58:55.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.446+0000 7f562bad8640 1 -- 10.20.193.28:0/3672594850 shutdown_connections 2026-02-25T11:58:55.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.446+0000 7f562bad8640 1 --2- 10.20.193.28:0/3672594850 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f56000b5760 0x7f56000d5c10 unknown :-1 s=CLOSED pgs=399 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.446+0000 7f562bad8640 1 --2- 10.20.193.28:0/3672594850 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5624114400 0x7f56240f2140 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.446+0000 7f562bad8640 1 --2- 10.20.193.28:0/3672594850 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f562410bee0 0x7f5624175ea0 unknown :-1 s=CLOSED pgs=910 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.446+0000 7f562bad8640 1 --2- 10.20.193.28:0/3672594850 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f562410b540 0x7f56240f1980 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.446+0000 7f562bad8640 1 -- 10.20.193.28:0/3672594850 >> 10.20.193.28:0/3672594850 conn(0x7f56240e8fb0 msgr2=0x7f562410d270 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:55.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.448+0000 7f562bad8640 1 -- 10.20.193.28:0/3672594850 shutdown_connections 2026-02-25T11:58:55.469 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.448+0000 7f562bad8640 1 -- 10.20.193.28:0/3672594850 wait complete. 2026-02-25T11:58:55.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2026-02-25T11:58:55.547 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:58:55.730 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:58:55.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.907+0000 7f74d06ce640 1 -- 10.20.193.28:0/2127381547 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f74c810d580 msgr2=0x7f74c810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:55.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.907+0000 7f74d06ce640 1 --2- 10.20.193.28:0/2127381547 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f74c810d580 0x7f74c810d950 secure :-1 s=READY pgs=914 gs=2 cs=0 l=1 c_cookie=6155521785620735516 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f74b8009950 tx=0x7f74b802f190 comp rx=0 tx=0).stop 2026-02-25T11:58:55.909 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.907+0000 7f74d06ce640 1 -- 10.20.193.28:0/2127381547 shutdown_connections 2026-02-25T11:58:55.910 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.907+0000 7f74d06ce640 1 --2- 10.20.193.28:0/2127381547 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f74c8113770 0x7f74c8174140 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.910 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.907+0000 7f74d06ce640 1 --2- 10.20.193.28:0/2127381547 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f74c810df70 0x7f74c8112fc0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.910 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.907+0000 7f74d06ce640 1 --2- 10.20.193.28:0/2127381547 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f74c810d580 0x7f74c810d950 unknown :-1 s=CLOSED pgs=914 gs=2 cs=0 l=1 c_cookie=6155521785620735516 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.910 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.907+0000 7f74d06ce640 1 -- 10.20.193.28:0/2127381547 >> 10.20.193.28:0/2127381547 conn(0x7f74c80eae70 msgr2=0x7f74c810b280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:55.910 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.907+0000 7f74d06ce640 1 -- 10.20.193.28:0/2127381547 shutdown_connections 2026-02-25T11:58:55.910 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.907+0000 7f74d06ce640 1 -- 10.20.193.28:0/2127381547 wait complete. 2026-02-25T11:58:55.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.908+0000 7f74d06ce640 1 Processor -- start 2026-02-25T11:58:55.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.908+0000 7f74d06ce640 1 -- start start 2026-02-25T11:58:55.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.908+0000 7f74d06ce640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f74c810d580 0x7f74c80f5e80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.908+0000 7f74d06ce640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f74c810df70 0x7f74c8174040 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74d06ce640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f74c8113770 0x7f74c80f6580 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74d06ce640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f74c80fb790 con 0x7f74c8113770 2026-02-25T11:58:55.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74d06ce640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f74c80fb900 con 0x7f74c810d580 2026-02-25T11:58:55.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74d06ce640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f74c80fbac0 con 0x7f74c810df70 2026-02-25T11:58:55.911 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74cec44640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f74c8113770 0x7f74c80f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74cec44640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f74c8113770 0x7f74c80f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34446/0 (socket says 10.20.193.28:34446) 2026-02-25T11:58:55.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74cec44640 1 -- 10.20.193.28:0/1592343195 learned_addr learned my addr 10.20.193.28:0/1592343195 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:55.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74cdc42640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f74c810df70 0x7f74c8174040 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74ce443640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f74c810d580 0x7f74c80f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74cec44640 1 -- 10.20.193.28:0/1592343195 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f74c810df70 msgr2=0x7f74c8174040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:55.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74cec44640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f74c810df70 0x7f74c8174040 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74cec44640 1 -- 10.20.193.28:0/1592343195 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f74c810d580 msgr2=0x7f74c80f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:55.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74cec44640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f74c810d580 0x7f74c80f5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:55.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74cec44640 1 -- 10.20.193.28:0/1592343195 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc0094b0 con 0x7f74c8113770 2026-02-25T11:58:55.912 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74cec44640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f74c8113770 0x7f74c80f6580 secure :-1 s=READY pgs=915 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f74c400f8d0 tx=0x7f74c400fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:55.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.909+0000 7f74af7fe640 1 -- 10.20.193.28:0/1592343195 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f74c400dce0 con 0x7f74c8113770 2026-02-25T11:58:55.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.910+0000 7f74af7fe640 1 -- 10.20.193.28:0/1592343195 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f74c400de40 con 0x7f74c8113770 2026-02-25T11:58:55.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.910+0000 7f74af7fe640 1 -- 10.20.193.28:0/1592343195 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f74c40116b0 con 0x7f74c8113770 2026-02-25T11:58:55.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.910+0000 7f74d06ce640 1 -- 10.20.193.28:0/1592343195 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b8009570 con 0x7f74c8113770 2026-02-25T11:58:55.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.910+0000 7f74d06ce640 1 -- 10.20.193.28:0/1592343195 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c80fc090 con 0x7f74c8113770 2026-02-25T11:58:55.913 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.910+0000 7f74d06ce640 1 -- 10.20.193.28:0/1592343195 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c8060c10 con 0x7f74c8113770 2026-02-25T11:58:55.914 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.912+0000 7f74af7fe640 1 -- 10.20.193.28:0/1592343195 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f74c4020060 con 0x7f74c8113770 2026-02-25T11:58:55.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.914+0000 7f74af7fe640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74a40b5750 0x7f74a40d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:55.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.914+0000 7f74af7fe640 1 -- 10.20.193.28:0/1592343195 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f74c4015070 con 0x7f74c8113770 2026-02-25T11:58:55.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.914+0000 7f74ce443640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74a40b5750 0x7f74a40d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:55.915 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.914+0000 7f74af7fe640 1 -- 10.20.193.28:0/1592343195 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f74c40aa6a0 con 0x7f74c8113770 2026-02-25T11:58:55.916 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:55.914+0000 7f74ce443640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74a40b5750 0x7f74a40d5c00 secure :-1 s=READY pgs=400 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f74b80098a0 tx=0x7f74b803a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:56.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.055+0000 7f74d06ce640 1 -- 10.20.193.28:0/1592343195 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74c810e860 con 0x7f74c8113770 2026-02-25T11:58:56.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.055+0000 7f74af7fe640 1 -- 10.20.193.28:0/1592343195 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f74c40ad850 con 0x7f74c8113770 2026-02-25T11:58:56.058 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:58:56.058 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:58:56.059 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:58:56.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.057+0000 7f74d06ce640 1 -- 10.20.193.28:0/1592343195 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74a40b5750 msgr2=0x7f74a40d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:56.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.057+0000 7f74d06ce640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74a40b5750 0x7f74a40d5c00 secure :-1 s=READY pgs=400 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f74b80098a0 tx=0x7f74b803a040 comp rx=0 tx=0).stop 2026-02-25T11:58:56.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.057+0000 7f74d06ce640 1 -- 10.20.193.28:0/1592343195 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f74c8113770 msgr2=0x7f74c80f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:56.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.057+0000 7f74d06ce640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f74c8113770 0x7f74c80f6580 secure :-1 s=READY pgs=915 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f74c400f8d0 tx=0x7f74c400fda0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.057+0000 7f74d06ce640 1 -- 10.20.193.28:0/1592343195 shutdown_connections 2026-02-25T11:58:56.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.057+0000 7f74d06ce640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f74a40b5750 0x7f74a40d5c00 unknown :-1 s=CLOSED pgs=400 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.057+0000 7f74d06ce640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f74c8113770 0x7f74c80f6580 unknown :-1 s=CLOSED pgs=915 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.057+0000 7f74d06ce640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f74c810df70 0x7f74c8174040 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.057+0000 7f74d06ce640 1 --2- 10.20.193.28:0/1592343195 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f74c810d580 0x7f74c80f5e80 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.057+0000 7f74d06ce640 1 -- 10.20.193.28:0/1592343195 >> 10.20.193.28:0/1592343195 conn(0x7f74c80eae70 msgr2=0x7f74c8110ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:56.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.057+0000 7f74d06ce640 1 -- 10.20.193.28:0/1592343195 shutdown_connections 2026-02-25T11:58:56.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.057+0000 7f74d06ce640 1 -- 10.20.193.28:0/1592343195 wait complete. 2026-02-25T11:58:56.161 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:2 damage ls 2026-02-25T11:58:56.345 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:58:56.391 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:56 trial028 ceph-mon[39224]: pgmap v547: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2026-02-25T11:58:56.391 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:56 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1592343195' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:58:56.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.533+0000 7f1cedbda640 1 -- 10.20.193.28:0/2249356102 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ce8115930 msgr2=0x7f1ce8176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:56.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.533+0000 7f1cedbda640 1 --2- 10.20.193.28:0/2249356102 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ce8115930 0x7f1ce8176300 secure :-1 s=READY pgs=916 gs=1 cs=0 l=1 c_cookie=17048794927249229577 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cdc00aa30 tx=0x7f1cdc030270 comp rx=0 tx=0).stop 2026-02-25T11:58:56.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.533+0000 7f1cedbda640 1 -- 10.20.193.28:0/2249356102 shutdown_connections 2026-02-25T11:58:56.537 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.533+0000 7f1cedbda640 1 --2- 10.20.193.28:0/2249356102 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ce8115930 0x7f1ce8176300 unknown :-1 s=CLOSED pgs=916 gs=1 cs=0 l=1 c_cookie=17048794927249229577 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.533+0000 7f1cedbda640 1 --2- 10.20.193.28:0/2249356102 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1ce8110130 0x7f1ce8115180 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.533+0000 7f1cedbda640 1 --2- 10.20.193.28:0/2249356102 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1ce810f740 0x7f1ce810fb10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.533+0000 7f1cedbda640 1 -- 10.20.193.28:0/2249356102 >> 10.20.193.28:0/2249356102 conn(0x7f1ce80ecff0 msgr2=0x7f1ce810d400 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:56.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.533+0000 7f1cedbda640 1 -- 10.20.193.28:0/2249356102 shutdown_connections 2026-02-25T11:58:56.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.533+0000 7f1cedbda640 1 -- 10.20.193.28:0/2249356102 wait complete. 2026-02-25T11:58:56.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1cedbda640 1 Processor -- start 2026-02-25T11:58:56.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1cedbda640 1 -- start start 2026-02-25T11:58:56.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1cedbda640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1ce810f740 0x7f1ce805b530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1cedbda640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1ce8110130 0x7f1ce8176200 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1cedbda640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ce8115930 0x7f1ce805bbc0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.538 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1cedbda640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce80ff7e0 con 0x7f1ce8115930 2026-02-25T11:58:56.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1cedbda640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce80ff950 con 0x7f1ce810f740 2026-02-25T11:58:56.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1cedbda640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce80ffac0 con 0x7f1ce8110130 2026-02-25T11:58:56.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1ce7fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ce8115930 0x7f1ce805bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1ce7fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ce8115930 0x7f1ce805bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34472/0 (socket says 10.20.193.28:34472) 2026-02-25T11:58:56.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1ce7fff640 1 -- 10.20.193.28:0/4018834154 learned_addr learned my addr 10.20.193.28:0/4018834154 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:56.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1ce6ffd640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1ce8110130 0x7f1ce8176200 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1ce77fe640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1ce810f740 0x7f1ce805b530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1ce7fff640 1 -- 10.20.193.28:0/4018834154 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1ce8110130 msgr2=0x7f1ce8176200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:56.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1ce7fff640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1ce8110130 0x7f1ce8176200 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1ce7fff640 1 -- 10.20.193.28:0/4018834154 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1ce810f740 msgr2=0x7f1ce805b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:56.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1ce7fff640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1ce810f740 0x7f1ce805b530 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.539 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.534+0000 7f1ce7fff640 1 -- 10.20.193.28:0/4018834154 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc00a650 con 0x7f1ce8115930 2026-02-25T11:58:56.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.535+0000 7f1ce7fff640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ce8115930 0x7f1ce805bbc0 secure :-1 s=READY pgs=917 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cdc006a50 tx=0x7f1cdc030ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:56.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.535+0000 7f1ce4ff9640 1 -- 10.20.193.28:0/4018834154 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cdc03e070 con 0x7f1ce8115930 2026-02-25T11:58:56.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.535+0000 7f1ce4ff9640 1 -- 10.20.193.28:0/4018834154 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1cdc039de0 con 0x7f1ce8115930 2026-02-25T11:58:56.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.535+0000 7f1ce4ff9640 1 -- 10.20.193.28:0/4018834154 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cdc042820 con 0x7f1ce8115930 2026-02-25T11:58:56.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.535+0000 7f1cedbda640 1 -- 10.20.193.28:0/4018834154 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce80ffd40 con 0x7f1ce8115930 2026-02-25T11:58:56.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.535+0000 7f1cedbda640 1 -- 10.20.193.28:0/4018834154 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce8100160 con 0x7f1ce8115930 2026-02-25T11:58:56.540 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.538+0000 7f1ce4ff9640 1 -- 10.20.193.28:0/4018834154 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f1cdc039920 con 0x7f1ce8115930 2026-02-25T11:58:56.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.539+0000 7f1ce4ff9640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1cbc0b5760 0x7f1cbc0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.541 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.539+0000 7f1ce4ff9640 1 -- 10.20.193.28:0/4018834154 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1cdc10c4e0 con 0x7f1ce8115930 2026-02-25T11:58:56.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.540+0000 7f1ce77fe640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1cbc0b5760 0x7f1cbc0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.542 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.540+0000 7f1ce77fe640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1cbc0b5760 0x7f1cbc0d5c10 secure :-1 s=READY pgs=401 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cd4004740 tx=0x7f1cd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:56.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.541+0000 7f1cefbd5e00 1 Processor -- start 2026-02-25T11:58:56.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.541+0000 7f1cefbd5e00 1 -- start start 2026-02-25T11:58:56.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.541+0000 7f1cefbd5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333b9ff00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.541+0000 7f1cefbd5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333ba0440 0x565333be4ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.541+0000 7f1cefbd5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333be50e0 0x565333c054c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.541+0000 7f1cefbd5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x565333ab58a0 con 0x565333b9fb30 2026-02-25T11:58:56.543 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.541+0000 7f1cefbd5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x565333ab15d0 con 0x565333ba0440 2026-02-25T11:58:56.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.541+0000 7f1cefbd5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x565333aab570 con 0x565333be50e0 2026-02-25T11:58:56.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.541+0000 7f1ce77fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333b9ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.541+0000 7f1ce77fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333b9ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34480/0 (socket says 10.20.193.28:34480) 2026-02-25T11:58:56.544 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.541+0000 7f1ce77fe640 1 -- 10.20.193.28:0/463340867 learned_addr learned my addr 10.20.193.28:0/463340867 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:56.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.541+0000 7f1ce6ffd640 1 --2- 10.20.193.28:0/463340867 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333ba0440 0x565333be4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.541+0000 7f1ce7fff640 1 --2- 10.20.193.28:0/463340867 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333be50e0 0x565333c054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.542+0000 7f1ce6ffd640 1 -- 10.20.193.28:0/463340867 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333be50e0 msgr2=0x565333c054c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:56.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.542+0000 7f1ce6ffd640 1 --2- 10.20.193.28:0/463340867 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333be50e0 0x565333c054c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.542+0000 7f1ce6ffd640 1 -- 10.20.193.28:0/463340867 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 msgr2=0x565333b9ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:56.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.542+0000 7f1ce6ffd640 1 --2- 10.20.193.28:0/463340867 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333b9ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.542+0000 7f1ce6ffd640 1 -- 10.20.193.28:0/463340867 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565333a56d10 con 0x565333ba0440 2026-02-25T11:58:56.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.542+0000 7f1ce77fe640 1 --2- 10.20.193.28:0/463340867 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333b9ff00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:58:56.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.542+0000 7f1ce6ffd640 1 --2- 10.20.193.28:0/463340867 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333ba0440 0x565333be4ba0 secure :-1 s=READY pgs=187 gs=2 cs=0 l=1 c_cookie=10336992359545940006 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cd8004740 tx=0x7f1cd8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f745fc884a9a826 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:56.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.542+0000 7f1cbbfff640 1 -- 10.20.193.28:0/463340867 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cd8009e80 con 0x565333ba0440 2026-02-25T11:58:56.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.542+0000 7f1cbbfff640 1 -- 10.20.193.28:0/463340867 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1cd8004850 con 0x565333ba0440 2026-02-25T11:58:56.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.542+0000 7f1cbbfff640 1 -- 10.20.193.28:0/463340867 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cd80054b0 con 0x565333ba0440 2026-02-25T11:58:56.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.542+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/463340867 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333ba0440 msgr2=0x565333be4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:56.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.542+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/463340867 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333ba0440 0x565333be4ba0 secure :-1 s=READY pgs=187 gs=2 cs=0 l=1 c_cookie=10336992359545940006 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cd8004740 tx=0x7f1cd8011810 comp rx=0 tx=0).stop 2026-02-25T11:58:56.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.543+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/463340867 shutdown_connections 2026-02-25T11:58:56.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.543+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/463340867 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333be50e0 0x565333c054c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.543+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/463340867 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333ba0440 0x565333be4ba0 unknown :-1 s=CLOSED pgs=187 gs=2 cs=0 l=1 c_cookie=10336992359545940006 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.543+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/463340867 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333b9ff00 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.543+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/463340867 >> 10.20.193.28:0/463340867 conn(0x565333abe7a0 msgr2=0x565333b9e410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:56.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.543+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/463340867 shutdown_connections 2026-02-25T11:58:56.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.543+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/463340867 wait complete. 2026-02-25T11:58:56.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.544+0000 7f1cefbd5e00 1 -- start start 2026-02-25T11:58:56.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.544+0000 7f1cefbd5e00 1 Processor -- start 2026-02-25T11:58:56.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.544+0000 7f1cefbd5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333b9fb30 0x565333bb7d30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.544+0000 7f1cefbd5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333ba0440 0x565333bb8270 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.544+0000 7f1cefbd5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333be50e0 0x565333bbd260 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1cefbd5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x565333bbd7a0 con 0x565333be50e0 2026-02-25T11:58:56.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1cefbd5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x565333bbd910 con 0x565333ba0440 2026-02-25T11:58:56.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1cefbd5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x565333bbdad0 con 0x565333b9fb30 2026-02-25T11:58:56.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1ce6ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333ba0440 0x565333bb8270 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1ce77fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333b9fb30 0x565333bb7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1ce7fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333be50e0 0x565333bbd260 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1ce77fe640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333b9fb30 0x565333bb7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48838/0 (socket says 10.20.193.28:48838) 2026-02-25T11:58:56.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1ce77fe640 1 -- 10.20.193.28:0/2877747751 learned_addr learned my addr 10.20.193.28:0/2877747751 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:56.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1ce6ffd640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333ba0440 0x565333bb8270 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:57150/0 (socket says 10.20.193.28:57150) 2026-02-25T11:58:56.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1ce7fff640 1 -- 10.20.193.28:0/2877747751 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333b9fb30 msgr2=0x565333bb7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:56.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1ce7fff640 1 --2- 10.20.193.28:0/2877747751 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333b9fb30 0x565333bb7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1ce7fff640 1 -- 10.20.193.28:0/2877747751 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333ba0440 msgr2=0x565333bb8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:56.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1ce7fff640 1 --2- 10.20.193.28:0/2877747751 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333ba0440 0x565333bb8270 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1ce7fff640 1 -- 10.20.193.28:0/2877747751 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd8009850 con 0x565333be50e0 2026-02-25T11:58:56.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1ce7fff640 1 --2- 10.20.193.28:0/2877747751 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333be50e0 0x565333bbd260 secure :-1 s=READY pgs=918 gs=2 cs=0 l=1 c_cookie=18219837258114037421 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cdc109fd0 tx=0x7f1cdc0ce8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd9dd74135076ad server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:56.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.545+0000 7f1cb97fa640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:58:56.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cb97fa640 1 -- 10.20.193.28:0/2877747751 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cdc10c310 con 0x565333be50e0 2026-02-25T11:58:56.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cb97fa640 1 -- 10.20.193.28:0/2877747751 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1cdc039c00 con 0x565333be50e0 2026-02-25T11:58:56.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cb97fa640 1 -- 10.20.193.28:0/2877747751 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cdc0dd510 con 0x565333be50e0 2026-02-25T11:58:56.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2877747751 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd4009970 con 0x565333be50e0 2026-02-25T11:58:56.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cb97fa640 1 -- 10.20.193.28:0/2877747751 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1cdc0dd670 con 0x565333be50e0 2026-02-25T11:58:56.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cb97fa640 10 client.17679 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:58:56.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cefbd5e00 10 client.17679 _ll_register_callbacks cb 0x565333a2ca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:56.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2877747751 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5653337416f0 con 0x565333be50e0 2026-02-25T11:58:56.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cb97fa640 1 -- 10.20.193.28:0/2877747751 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1cdc0dd9a0 con 0x565333be50e0 2026-02-25T11:58:56.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cefbd5e00 10 client.17679 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:56.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2877747751 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x565333a2f3c0 con 0x565333be50e0 2026-02-25T11:58:56.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cb97fa640 1 -- 10.20.193.28:0/2877747751 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f1cdc036320 con 0x565333be50e0 2026-02-25T11:58:56.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cefbd5e00 10 client.17679 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:56.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.546+0000 7f1cefbd5e00 10 client.17679 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:58:56.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.547+0000 7f1cefbd5e00 20 client.17679 populate_metadata read hostname 'trial028' 2026-02-25T11:58:56.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.547+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/2877747751 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565333be2260 0x565333be2700 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.547+0000 7f1cefbd5e00 4 client.17679 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:58:56.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.547+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2877747751 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565333a7dfc0 con 0x565333be2260 2026-02-25T11:58:56.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.547+0000 7f1ce77fe640 1 --2- 10.20.193.28:0/2877747751 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565333be2260 0x565333be2700 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.548+0000 7f1ce77fe640 1 --2- 10.20.193.28:0/2877747751 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565333be2260 0x565333be2700 crc :-1 s=READY pgs=63 gs=4 cs=0 l=0 c_cookie=15480365405041575146 s_cookie=16074091873350333734 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6d54e5140bfc4ea server_cookie=df12a565060a3526 in_seq=0 out_seq=0 2026-02-25T11:58:56.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.548+0000 7f1cb97fa640 10 client.17679 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:58:56.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.548+0000 7f1cb97fa640 1 -- 10.20.193.28:0/2877747751 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x565333a7dfc0 con 0x565333be2260 2026-02-25T11:58:56.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.548+0000 7f1cb97fa640 10 client.17679 handle_command_reply: tid=0 2026-02-25T11:58:56.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.548+0000 7f1cefbd5e00 1 client.17679 shutdown 2026-02-25T11:58:56.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.548+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2877747751 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333be50e0 msgr2=0x565333bbd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:56.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.548+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/2877747751 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333be50e0 0x565333bbd260 secure :-1 s=READY pgs=918 gs=2 cs=0 l=1 c_cookie=18219837258114037421 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cdc109fd0 tx=0x7f1cdc0ce8a0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.549+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2877747751 shutdown_connections 2026-02-25T11:58:56.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.549+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/2877747751 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565333be2260 0x565333be2700 crc :-1 s=READY pgs=63 gs=4 cs=0 l=0 c_cookie=15480365405041575146 s_cookie=16074091873350333734 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.549+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/2877747751 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333be50e0 0x565333bbd260 unknown :-1 s=CLOSED pgs=918 gs=2 cs=0 l=1 c_cookie=18219837258114037421 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.549+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/2877747751 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333ba0440 0x565333bb8270 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.549+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/2877747751 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333b9fb30 0x565333bb7d30 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.549+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2877747751 >> 10.20.193.28:0/2877747751 conn(0x565333abe7a0 msgr2=0x565333b9eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:56.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.549+0000 7f1cb97fa640 0 client.17679 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:58:56.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.549+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2877747751 shutdown_connections 2026-02-25T11:58:56.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.549+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2877747751 wait complete. 2026-02-25T11:58:56.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.549+0000 7f1cefbd5e00 20 client.17679 trim_cache size 0 max 16384 2026-02-25T11:58:56.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.556+0000 7f1cefbd5e00 1 Processor -- start 2026-02-25T11:58:56.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1cefbd5e00 1 -- start start 2026-02-25T11:58:56.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1cefbd5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333bb58d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1cefbd5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 0x565333ba9f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1cefbd5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333be50e0 0x565333be2fe0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1cefbd5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x565333aa91a0 con 0x565333b9fb30 2026-02-25T11:58:56.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1cefbd5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x565333769700 con 0x565333be50e0 2026-02-25T11:58:56.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1cefbd5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x565333ba58f0 con 0x565333ba0440 2026-02-25T11:58:56.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1ce6ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 0x565333ba9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1ce6ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 0x565333ba9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48846/0 (socket says 10.20.193.28:48846) 2026-02-25T11:58:56.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1ce7fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333be50e0 0x565333be2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1ce77fe640 1 --2- 10.20.193.28:0/2173066076 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333bb58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1ce6ffd640 1 -- 10.20.193.28:0/2173066076 learned_addr learned my addr 10.20.193.28:0/2173066076 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:56.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1ce6ffd640 1 -- 10.20.193.28:0/2173066076 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333be50e0 msgr2=0x565333be2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:56.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1ce6ffd640 1 --2- 10.20.193.28:0/2173066076 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333be50e0 0x565333be2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1ce6ffd640 1 -- 10.20.193.28:0/2173066076 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 msgr2=0x565333bb58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:56.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1ce6ffd640 1 --2- 10.20.193.28:0/2173066076 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333bb58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.557+0000 7f1ce6ffd640 1 -- 10.20.193.28:0/2173066076 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565333baa6d0 con 0x565333ba0440 2026-02-25T11:58:56.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.558+0000 7f1ce6ffd640 1 --2- 10.20.193.28:0/2173066076 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 0x565333ba9f00 secure :-1 s=READY pgs=238 gs=3 cs=0 l=1 c_cookie=81701029816761366 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cd8009820 tx=0x7f1cd8009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12242a75c02fc16 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:56.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.558+0000 7f1cbb7fe640 1 -- 10.20.193.28:0/2173066076 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cd8004a80 con 0x565333ba0440 2026-02-25T11:58:56.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.558+0000 7f1cbb7fe640 1 -- 10.20.193.28:0/2173066076 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1cd801e040 con 0x565333ba0440 2026-02-25T11:58:56.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.558+0000 7f1cbb7fe640 1 -- 10.20.193.28:0/2173066076 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cd80056a0 con 0x565333ba0440 2026-02-25T11:58:56.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.558+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2173066076 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 msgr2=0x565333ba9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:56.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.558+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/2173066076 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 0x565333ba9f00 secure :-1 s=READY pgs=238 gs=3 cs=0 l=1 c_cookie=81701029816761366 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cd8009820 tx=0x7f1cd8009e80 comp rx=0 tx=0).stop 2026-02-25T11:58:56.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.558+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2173066076 shutdown_connections 2026-02-25T11:58:56.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.558+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/2173066076 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333be50e0 0x565333be2fe0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.558+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/2173066076 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 0x565333ba9f00 unknown :-1 s=CLOSED pgs=238 gs=3 cs=0 l=1 c_cookie=81701029816761366 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.558+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/2173066076 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333bb58d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.558+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2173066076 >> 10.20.193.28:0/2173066076 conn(0x565333abe7a0 msgr2=0x565333bb41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:56.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.558+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2173066076 shutdown_connections 2026-02-25T11:58:56.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.558+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/2173066076 wait complete. 2026-02-25T11:58:56.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.559+0000 7f1cefbd5e00 1 -- start start 2026-02-25T11:58:56.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.559+0000 7f1cefbd5e00 1 Processor -- start 2026-02-25T11:58:56.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.559+0000 7f1cefbd5e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333bc3bb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.559+0000 7f1cefbd5e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 0x565333bc4110 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.559+0000 7f1cefbd5e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333be50e0 0x565333bc13d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.559+0000 7f1cefbd5e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x565333bc1ad0 con 0x565333b9fb30 2026-02-25T11:58:56.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.559+0000 7f1cefbd5e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x565333bc1c40 con 0x565333be50e0 2026-02-25T11:58:56.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.559+0000 7f1cefbd5e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x565333bc1e00 con 0x565333ba0440 2026-02-25T11:58:56.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1ce77fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333bc3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1ce6ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 0x565333bc4110 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1ce6ffd640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 0x565333bc4110 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48850/0 (socket says 10.20.193.28:48850) 2026-02-25T11:58:56.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1ce7fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333be50e0 0x565333bc13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1ce6ffd640 1 -- 10.20.193.28:0/4189790729 learned_addr learned my addr 10.20.193.28:0/4189790729 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:56.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1ce6ffd640 1 -- 10.20.193.28:0/4189790729 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333be50e0 msgr2=0x565333bc13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:56.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1ce6ffd640 1 --2- 10.20.193.28:0/4189790729 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333be50e0 0x565333bc13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1ce6ffd640 1 -- 10.20.193.28:0/4189790729 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 msgr2=0x565333bc3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:56.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1ce6ffd640 1 --2- 10.20.193.28:0/4189790729 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333bc3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1ce6ffd640 1 -- 10.20.193.28:0/4189790729 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565333a410e0 con 0x565333ba0440 2026-02-25T11:58:56.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1ce77fe640 1 --2- 10.20.193.28:0/4189790729 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333bc3bb0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:58:56.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1ce6ffd640 1 --2- 10.20.193.28:0/4189790729 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 0x565333bc4110 secure :-1 s=READY pgs=239 gs=2 cs=0 l=1 c_cookie=8192966882125122612 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cd800a010 tx=0x7f1cd8005630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b343f8999b8c34 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:56.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1cb97fa640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:58:56.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.560+0000 7f1cb97fa640 1 -- 10.20.193.28:0/4189790729 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cd8004e80 con 0x565333ba0440 2026-02-25T11:58:56.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/4189790729 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565333a56d10 con 0x565333ba0440 2026-02-25T11:58:56.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cefbd5e00 10 client.27328 _ll_register_callbacks cb 0x565333a2ca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:56.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/4189790729 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd8004850 con 0x565333ba0440 2026-02-25T11:58:56.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cb97fa640 1 -- 10.20.193.28:0/4189790729 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f1cd80042d0 con 0x565333ba0440 2026-02-25T11:58:56.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cb97fa640 1 -- 10.20.193.28:0/4189790729 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f1cd8022680 con 0x565333ba0440 2026-02-25T11:58:56.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cb97fa640 1 -- 10.20.193.28:0/4189790729 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f1cd8004440 con 0x565333ba0440 2026-02-25T11:58:56.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cb97fa640 1 -- 10.20.193.28:0/4189790729 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1cd80026a0 con 0x565333ba0440 2026-02-25T11:58:56.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cefbd5e00 10 client.27328 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:56.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/4189790729 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x565333a3bb60 con 0x565333ba0440 2026-02-25T11:58:56.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cb97fa640 1 -- 10.20.193.28:0/4189790729 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f1cd8029070 con 0x565333ba0440 2026-02-25T11:58:56.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cefbd5e00 10 client.27328 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:56.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cefbd5e00 10 client.27328 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2026-02-25T11:58:56.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cefbd5e00 20 client.27328 populate_metadata read hostname 'trial028' 2026-02-25T11:58:56.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/4189790729 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565333c91d30 0x565333cb21e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:56.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cefbd5e00 4 client.27328 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:58:56.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.561+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/4189790729 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56533373ef40 con 0x565333c91d30 2026-02-25T11:58:56.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.562+0000 7f1ce77fe640 1 --2- 10.20.193.28:0/4189790729 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565333c91d30 0x565333cb21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:56.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.562+0000 7f1ce77fe640 1 --2- 10.20.193.28:0/4189790729 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565333c91d30 0x565333cb21e0 crc :-1 s=READY pgs=64 gs=4 cs=0 l=0 c_cookie=10483381690807206293 s_cookie=3864470260234307347 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=917c741abd65c195 server_cookie=35a15b3a1253c713 in_seq=0 out_seq=0 2026-02-25T11:58:56.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.562+0000 7f1cb97fa640 10 client.27328 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:58:56.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.562+0000 7f1cb97fa640 1 -- 10.20.193.28:0/4189790729 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56533373ef40 con 0x565333c91d30 2026-02-25T11:58:56.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.562+0000 7f1cb97fa640 10 client.27328 handle_command_reply: tid=0 2026-02-25T11:58:56.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.562+0000 7f1cefbd5e00 1 client.27328 shutdown 2026-02-25T11:58:56.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.562+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/4189790729 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 msgr2=0x565333bc4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:56.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.562+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/4189790729 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 0x565333bc4110 secure :-1 s=READY pgs=239 gs=2 cs=0 l=1 c_cookie=8192966882125122612 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cd800a010 tx=0x7f1cd8005630 comp rx=0 tx=0).stop 2026-02-25T11:58:56.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/4189790729 shutdown_connections 2026-02-25T11:58:56.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/4189790729 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x565333c91d30 0x565333cb21e0 crc :-1 s=READY pgs=64 gs=4 cs=0 l=0 c_cookie=10483381690807206293 s_cookie=3864470260234307347 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/4189790729 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x565333be50e0 0x565333bc13d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/4189790729 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x565333ba0440 0x565333bc4110 unknown :-1 s=CLOSED pgs=239 gs=2 cs=0 l=1 c_cookie=8192966882125122612 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cefbd5e00 1 --2- 10.20.193.28:0/4189790729 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x565333b9fb30 0x565333bc3bb0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/4189790729 >> 10.20.193.28:0/4189790729 conn(0x565333abe7a0 msgr2=0x565333bb49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:56.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cb97fa640 0 client.27328 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:58:56.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/4189790729 shutdown_connections 2026-02-25T11:58:56.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cefbd5e00 1 -- 10.20.193.28:0/4189790729 wait complete. 2026-02-25T11:58:56.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cefbd5e00 20 client.27328 trim_cache size 0 max 16384 2026-02-25T11:58:56.572 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:58:56.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cedbda640 1 -- 10.20.193.28:0/4018834154 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1cbc0b5760 msgr2=0x7f1cbc0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:56.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cedbda640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1cbc0b5760 0x7f1cbc0d5c10 secure :-1 s=READY pgs=401 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cd4004740 tx=0x7f1cd4005f90 comp rx=0 tx=0).stop 2026-02-25T11:58:56.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cedbda640 1 -- 10.20.193.28:0/4018834154 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ce8115930 msgr2=0x7f1ce805bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:56.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.563+0000 7f1cedbda640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ce8115930 0x7f1ce805bbc0 secure :-1 s=READY pgs=917 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f1cdc006a50 tx=0x7f1cdc030ce0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.564+0000 7f1cedbda640 1 -- 10.20.193.28:0/4018834154 shutdown_connections 2026-02-25T11:58:56.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.564+0000 7f1cedbda640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f1cbc0b5760 0x7f1cbc0d5c10 unknown :-1 s=CLOSED pgs=401 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.564+0000 7f1cedbda640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f1ce8115930 0x7f1ce805bbc0 unknown :-1 s=CLOSED pgs=917 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.564+0000 7f1cedbda640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f1ce8110130 0x7f1ce8176200 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.564+0000 7f1cedbda640 1 --2- 10.20.193.28:0/4018834154 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f1ce810f740 0x7f1ce805b530 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:56.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.564+0000 7f1cedbda640 1 -- 10.20.193.28:0/4018834154 >> 10.20.193.28:0/4018834154 conn(0x7f1ce80ecff0 msgr2=0x7f1ce810dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:56.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.564+0000 7f1cedbda640 1 -- 10.20.193.28:0/4018834154 shutdown_connections 2026-02-25T11:58:56.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:56.564+0000 7f1cedbda640 1 -- 10.20.193.28:0/4018834154 wait complete. 2026-02-25T11:58:56.665 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:1 damage ls 2026-02-25T11:58:56.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:56 trial167 ceph-mon[43392]: pgmap v547: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2026-02-25T11:58:56.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:56 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1592343195' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:58:56.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:56 trial119 ceph-mon[42710]: pgmap v547: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2026-02-25T11:58:56.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:56 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1592343195' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:58:56.850 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:58:57.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.022+0000 7f05c1f1b640 1 -- 10.20.193.28:0/744311922 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05bc111370 msgr2=0x7f05bc111d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.022+0000 7f05c1f1b640 1 --2- 10.20.193.28:0/744311922 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05bc111370 0x7f05bc111d50 secure :-1 s=READY pgs=919 gs=1 cs=0 l=1 c_cookie=18114411330491214894 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05b800aa30 tx=0x7f05b8030270 comp rx=0 tx=0).stop 2026-02-25T11:58:57.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.023+0000 7f05c1f1b640 1 -- 10.20.193.28:0/744311922 shutdown_connections 2026-02-25T11:58:57.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.023+0000 7f05c1f1b640 1 --2- 10.20.193.28:0/744311922 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05bc111370 0x7f05bc111d50 unknown :-1 s=CLOSED pgs=919 gs=1 cs=0 l=1 c_cookie=18114411330491214894 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.023+0000 7f05c1f1b640 1 --2- 10.20.193.28:0/744311922 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f05bc10bd50 0x7f05bc110be0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.023+0000 7f05c1f1b640 1 --2- 10.20.193.28:0/744311922 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f05bc10b360 0x7f05bc10b730 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.023+0000 7f05c1f1b640 1 -- 10.20.193.28:0/744311922 >> 10.20.193.28:0/744311922 conn(0x7f05bc0e8e70 msgr2=0x7f05bc109280 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.023+0000 7f05c1f1b640 1 -- 10.20.193.28:0/744311922 shutdown_connections 2026-02-25T11:58:57.024 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.023+0000 7f05c1f1b640 1 -- 10.20.193.28:0/744311922 wait complete. 2026-02-25T11:58:57.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.023+0000 7f05c1f1b640 1 Processor -- start 2026-02-25T11:58:57.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.023+0000 7f05c1f1b640 1 -- start start 2026-02-25T11:58:57.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05c1f1b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f05bc10b360 0x7f05bc1235e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05c1f1b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05bc10bd50 0x7f05bc123b20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05c1f1b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f05bc111370 0x7f05bc128ef0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05c1f1b640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc129510 con 0x7f05bc10bd50 2026-02-25T11:58:57.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05c1f1b640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc129680 con 0x7f05bc111370 2026-02-25T11:58:57.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05c1f1b640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f05bc129840 con 0x7f05bc10b360 2026-02-25T11:58:57.025 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05b3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05bc10bd50 0x7f05bc123b20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05b3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05bc10bd50 0x7f05bc123b20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34510/0 (socket says 10.20.193.28:34510) 2026-02-25T11:58:57.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05b3fff640 1 -- 10.20.193.28:0/172574404 learned_addr learned my addr 10.20.193.28:0/172574404 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:57.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05c0f19640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f05bc10b360 0x7f05bc1235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05c171a640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f05bc111370 0x7f05bc128ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05b3fff640 1 -- 10.20.193.28:0/172574404 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f05bc10b360 msgr2=0x7f05bc1235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05b3fff640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f05bc10b360 0x7f05bc1235e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05b3fff640 1 -- 10.20.193.28:0/172574404 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f05bc111370 msgr2=0x7f05bc128ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05b3fff640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f05bc111370 0x7f05bc128ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05b3fff640 1 -- 10.20.193.28:0/172574404 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b800a650 con 0x7f05bc10bd50 2026-02-25T11:58:57.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.024+0000 7f05b3fff640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05bc10bd50 0x7f05bc123b20 secure :-1 s=READY pgs=920 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05a400e9c0 tx=0x7f05a400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.025+0000 7f05b1ffb640 1 -- 10.20.193.28:0/172574404 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f05a400ccb0 con 0x7f05bc10bd50 2026-02-25T11:58:57.026 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.025+0000 7f05c1f1b640 1 -- 10.20.193.28:0/172574404 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05bc129b20 con 0x7f05bc10bd50 2026-02-25T11:58:57.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.025+0000 7f05b1ffb640 1 -- 10.20.193.28:0/172574404 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f05a400ce10 con 0x7f05bc10bd50 2026-02-25T11:58:57.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.025+0000 7f05b1ffb640 1 -- 10.20.193.28:0/172574404 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f05a40106b0 con 0x7f05bc10bd50 2026-02-25T11:58:57.027 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.025+0000 7f05c1f1b640 1 -- 10.20.193.28:0/172574404 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05bc16a220 con 0x7f05bc10bd50 2026-02-25T11:58:57.029 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.028+0000 7f05b1ffb640 1 -- 10.20.193.28:0/172574404 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f05a4004080 con 0x7f05bc10bd50 2026-02-25T11:58:57.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.029+0000 7f05b1ffb640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f05980b5760 0x7f05980d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.029+0000 7f05b1ffb640 1 -- 10.20.193.28:0/172574404 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f05a4014070 con 0x7f05bc10bd50 2026-02-25T11:58:57.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.029+0000 7f05c0f19640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f05980b5760 0x7f05980d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.031 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.030+0000 7f05c0f19640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f05980b5760 0x7f05980d5c10 secure :-1 s=READY pgs=402 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05ac004740 tx=0x7f05ac005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.030+0000 7f05c519fe00 1 Processor -- start 2026-02-25T11:58:57.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.030+0000 7f05c519fe00 1 -- start start 2026-02-25T11:58:57.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c519fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 0x55f712b3ff00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c519fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b40440 0x55f712b84ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c519fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b850e0 0x55f712ba54c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.032 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c519fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f712a558a0 con 0x55f712b850e0 2026-02-25T11:58:57.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c519fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f712a515d0 con 0x55f712b3fb30 2026-02-25T11:58:57.033 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c519fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f712a4b570 con 0x55f712b40440 2026-02-25T11:58:57.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c171a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b850e0 0x55f712ba54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c171a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b850e0 0x55f712ba54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34524/0 (socket says 10.20.193.28:34524) 2026-02-25T11:58:57.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c171a640 1 -- 10.20.193.28:0/4277255062 learned_addr learned my addr 10.20.193.28:0/4277255062 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:57.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c0f19640 1 --2- 10.20.193.28:0/4277255062 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 0x55f712b3ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.034 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05b3fff640 1 --2- 10.20.193.28:0/4277255062 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b40440 0x55f712b84ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c171a640 1 -- 10.20.193.28:0/4277255062 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b40440 msgr2=0x55f712b84ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c171a640 1 --2- 10.20.193.28:0/4277255062 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b40440 0x55f712b84ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c171a640 1 -- 10.20.193.28:0/4277255062 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 msgr2=0x55f712b3ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c171a640 1 --2- 10.20.193.28:0/4277255062 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 0x55f712b3ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c171a640 1 -- 10.20.193.28:0/4277255062 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7129f6d10 con 0x55f712b850e0 2026-02-25T11:58:57.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05b3fff640 1 --2- 10.20.193.28:0/4277255062 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b40440 0x55f712b84ba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:58:57.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.031+0000 7f05c0f19640 1 --2- 10.20.193.28:0/4277255062 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 0x55f712b3ff00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:58:57.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.032+0000 7f05c171a640 1 --2- 10.20.193.28:0/4277255062 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b850e0 0x55f712ba54c0 secure :-1 s=READY pgs=921 gs=1 cs=0 l=1 c_cookie=6532414686496696334 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05b8006a00 tx=0x7f05b8006cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa7cc8c122e280e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.032+0000 7f05894e4640 1 -- 10.20.193.28:0/4277255062 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f05b803e040 con 0x55f712b850e0 2026-02-25T11:58:57.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.032+0000 7f05894e4640 1 -- 10.20.193.28:0/4277255062 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f05b8030ad0 con 0x55f712b850e0 2026-02-25T11:58:57.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.032+0000 7f05894e4640 1 -- 10.20.193.28:0/4277255062 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f05b8039e70 con 0x55f712b850e0 2026-02-25T11:58:57.035 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.032+0000 7f05c519fe00 1 -- 10.20.193.28:0/4277255062 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b850e0 msgr2=0x55f712ba54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.032+0000 7f05c519fe00 1 --2- 10.20.193.28:0/4277255062 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b850e0 0x55f712ba54c0 secure :-1 s=READY pgs=921 gs=1 cs=0 l=1 c_cookie=6532414686496696334 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05b8006a00 tx=0x7f05b8006cb0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.032+0000 7f05c519fe00 1 -- 10.20.193.28:0/4277255062 shutdown_connections 2026-02-25T11:58:57.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.032+0000 7f05c519fe00 1 --2- 10.20.193.28:0/4277255062 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b850e0 0x55f712ba54c0 unknown :-1 s=CLOSED pgs=921 gs=1 cs=0 l=1 c_cookie=6532414686496696334 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.032+0000 7f05c519fe00 1 --2- 10.20.193.28:0/4277255062 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b40440 0x55f712b84ba0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.032+0000 7f05c519fe00 1 --2- 10.20.193.28:0/4277255062 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 0x55f712b3ff00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.032+0000 7f05c519fe00 1 -- 10.20.193.28:0/4277255062 >> 10.20.193.28:0/4277255062 conn(0x55f712a5e7a0 msgr2=0x55f712b3e410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.032+0000 7f05c519fe00 1 -- 10.20.193.28:0/4277255062 shutdown_connections 2026-02-25T11:58:57.036 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.032+0000 7f05c519fe00 1 -- 10.20.193.28:0/4277255062 wait complete. 2026-02-25T11:58:57.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.033+0000 7f05c519fe00 1 -- start start 2026-02-25T11:58:57.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.034+0000 7f05c519fe00 1 Processor -- start 2026-02-25T11:58:57.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.034+0000 7f05c519fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 0x55f712b57e80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.034+0000 7f05c519fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b40440 0x55f712b583c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.038 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.034+0000 7f05c519fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b850e0 0x55f712b5d420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.034+0000 7f05c519fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f712b5d960 con 0x55f712b40440 2026-02-25T11:58:57.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.034+0000 7f05c519fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f712b5dad0 con 0x55f712b3fb30 2026-02-25T11:58:57.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.034+0000 7f05c519fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f712b5dc90 con 0x55f712b850e0 2026-02-25T11:58:57.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.034+0000 7f05c171a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b850e0 0x55f712b5d420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.034+0000 7f05c171a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b850e0 0x55f712b5d420 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48896/0 (socket says 10.20.193.28:48896) 2026-02-25T11:58:57.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.034+0000 7f05c171a640 1 -- 10.20.193.28:0/307437348 learned_addr learned my addr 10.20.193.28:0/307437348 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:57.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.034+0000 7f05b3fff640 1 --2- 10.20.193.28:0/307437348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b40440 0x55f712b583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.034+0000 7f05c0f19640 1 --2- 10.20.193.28:0/307437348 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 0x55f712b57e80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f05b3fff640 1 -- 10.20.193.28:0/307437348 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b850e0 msgr2=0x55f712b5d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f05b3fff640 1 --2- 10.20.193.28:0/307437348 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b850e0 0x55f712b5d420 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.039 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f05b3fff640 1 -- 10.20.193.28:0/307437348 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 msgr2=0x55f712b57e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f05b3fff640 1 --2- 10.20.193.28:0/307437348 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 0x55f712b57e80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f05b3fff640 1 -- 10.20.193.28:0/307437348 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7129cf3c0 con 0x55f712b40440 2026-02-25T11:58:57.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f05b3fff640 1 --2- 10.20.193.28:0/307437348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b40440 0x55f712b583c0 secure :-1 s=READY pgs=922 gs=1 cs=0 l=1 c_cookie=11216025380663072015 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05a40e46a0 tx=0x7f05a40ac730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba753a89e30150f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f057a7fc640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:58:57.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f057a7fc640 1 -- 10.20.193.28:0/307437348 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f05a40b2c50 con 0x55f712b40440 2026-02-25T11:58:57.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f05c519fe00 1 -- 10.20.193.28:0/307437348 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05bc112610 con 0x55f712b40440 2026-02-25T11:58:57.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f05c519fe00 10 client.17697 _ll_register_callbacks cb 0x55f7129cca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:57.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f05c519fe00 1 -- 10.20.193.28:0/307437348 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f7126e16f0 con 0x55f712b40440 2026-02-25T11:58:57.040 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f057a7fc640 1 -- 10.20.193.28:0/307437348 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f05a40b5230 con 0x55f712b40440 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f057a7fc640 1 -- 10.20.193.28:0/307437348 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f05a40adbc0 con 0x55f712b40440 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f057a7fc640 1 -- 10.20.193.28:0/307437348 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f05a40add40 con 0x55f712b40440 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f057a7fc640 1 -- 10.20.193.28:0/307437348 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f05a40b7be0 con 0x55f712b40440 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f05c519fe00 10 client.17697 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.035+0000 7f05c519fe00 1 -- 10.20.193.28:0/307437348 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55f7129e10e0 con 0x55f712b40440 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.036+0000 7f057a7fc640 1 -- 10.20.193.28:0/307437348 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f05a40720c0 con 0x55f712b40440 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.036+0000 7f05c519fe00 10 client.17697 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.036+0000 7f05c519fe00 10 client.17697 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.036+0000 7f05c519fe00 20 client.17697 populate_metadata read hostname 'trial028' 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.036+0000 7f05c519fe00 1 --2- 10.20.193.28:0/307437348 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f712b825b0 0x55f712b82a20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.036+0000 7f05c519fe00 4 client.17697 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.036+0000 7f05c519fe00 1 -- 10.20.193.28:0/307437348 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f712a1dfc0 con 0x55f712b825b0 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.036+0000 7f05c0f19640 1 --2- 10.20.193.28:0/307437348 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f712b825b0 0x55f712b82a20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.041 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.036+0000 7f05c0f19640 1 --2- 10.20.193.28:0/307437348 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f712b825b0 0x55f712b82a20 crc :-1 s=READY pgs=64 gs=4 cs=0 l=0 c_cookie=15427597109807233708 s_cookie=10332343319567472271 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d619d5d51be73aac server_cookie=8f63db818c33168f in_seq=0 out_seq=0 2026-02-25T11:58:57.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.036+0000 7f057a7fc640 10 client.17697 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:58:57.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.037+0000 7f057a7fc640 1 -- 10.20.193.28:0/307437348 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x55f712a1dfc0 con 0x55f712b825b0 2026-02-25T11:58:57.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.037+0000 7f057a7fc640 10 client.17697 handle_command_reply: tid=0 2026-02-25T11:58:57.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.037+0000 7f05c519fe00 1 client.17697 shutdown 2026-02-25T11:58:57.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.037+0000 7f05c519fe00 1 -- 10.20.193.28:0/307437348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b40440 msgr2=0x55f712b583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.042 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.037+0000 7f05c519fe00 1 --2- 10.20.193.28:0/307437348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b40440 0x55f712b583c0 secure :-1 s=READY pgs=922 gs=1 cs=0 l=1 c_cookie=11216025380663072015 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05a40e46a0 tx=0x7f05a40ac730 comp rx=0 tx=0).stop 2026-02-25T11:58:57.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.037+0000 7f05c519fe00 1 -- 10.20.193.28:0/307437348 shutdown_connections 2026-02-25T11:58:57.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.037+0000 7f05c519fe00 1 --2- 10.20.193.28:0/307437348 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f712b825b0 0x55f712b82a20 crc :-1 s=READY pgs=64 gs=4 cs=0 l=0 c_cookie=15427597109807233708 s_cookie=10332343319567472271 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.037+0000 7f05c519fe00 1 --2- 10.20.193.28:0/307437348 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b850e0 0x55f712b5d420 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.037+0000 7f057a7fc640 0 client.17697 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:58:57.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.037+0000 7f05c519fe00 1 --2- 10.20.193.28:0/307437348 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b40440 0x55f712b583c0 unknown :-1 s=CLOSED pgs=922 gs=1 cs=0 l=1 c_cookie=11216025380663072015 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.037+0000 7f05c519fe00 1 --2- 10.20.193.28:0/307437348 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 0x55f712b57e80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.037+0000 7f05c519fe00 1 -- 10.20.193.28:0/307437348 >> 10.20.193.28:0/307437348 conn(0x55f712a5e7a0 msgr2=0x55f712b3eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.038+0000 7f05c519fe00 1 -- 10.20.193.28:0/307437348 shutdown_connections 2026-02-25T11:58:57.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.038+0000 7f05c519fe00 1 -- 10.20.193.28:0/307437348 wait complete. 2026-02-25T11:58:57.043 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.038+0000 7f05c519fe00 20 client.17697 trim_cache size 0 max 16384 2026-02-25T11:58:57.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.045+0000 7f05c519fe00 1 Processor -- start 2026-02-25T11:58:57.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.045+0000 7f05c519fe00 1 -- start start 2026-02-25T11:58:57.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.045+0000 7f05c519fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 0x55f712b55480 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.045+0000 7f05c519fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b40440 0x55f712b7d5c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.045+0000 7f05c519fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b825b0 0x55f712b7db00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.045+0000 7f05c519fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f712a491a0 con 0x55f712b40440 2026-02-25T11:58:57.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.045+0000 7f05c519fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f712709700 con 0x55f712b3fb30 2026-02-25T11:58:57.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.045+0000 7f05c519fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f712b5ddd0 con 0x55f712b825b0 2026-02-25T11:58:57.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.045+0000 7f05b3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b40440 0x55f712b7d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.045+0000 7f05b3fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b40440 0x55f712b7d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34556/0 (socket says 10.20.193.28:34556) 2026-02-25T11:58:57.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.045+0000 7f05b3fff640 1 -- 10.20.193.28:0/1266754779 learned_addr learned my addr 10.20.193.28:0/1266754779 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:57.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.045+0000 7f05c0f19640 1 --2- 10.20.193.28:0/1266754779 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 0x55f712b55480 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.045+0000 7f05c171a640 1 --2- 10.20.193.28:0/1266754779 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b825b0 0x55f712b7db00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f05b3fff640 1 -- 10.20.193.28:0/1266754779 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b825b0 msgr2=0x55f712b7db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f05b3fff640 1 --2- 10.20.193.28:0/1266754779 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b825b0 0x55f712b7db00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f05b3fff640 1 -- 10.20.193.28:0/1266754779 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 msgr2=0x55f712b55480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f05b3fff640 1 --2- 10.20.193.28:0/1266754779 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 0x55f712b55480 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f05b3fff640 1 -- 10.20.193.28:0/1266754779 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7129dbb60 con 0x55f712b40440 2026-02-25T11:58:57.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f05b3fff640 1 --2- 10.20.193.28:0/1266754779 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b40440 0x55f712b7d5c0 secure :-1 s=READY pgs=923 gs=3 cs=0 l=1 c_cookie=5627867805206050201 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05a4016700 tx=0x7f05a40054b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1a33f37c29f599 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f0588ce3640 1 -- 10.20.193.28:0/1266754779 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f05a40b2c50 con 0x55f712b40440 2026-02-25T11:58:57.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f0588ce3640 1 -- 10.20.193.28:0/1266754779 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x55f7126e16f0 con 0x55f712b40440 2026-02-25T11:58:57.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f0588ce3640 1 -- 10.20.193.28:0/1266754779 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f05a4072230 con 0x55f712b40440 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f05c519fe00 1 -- 10.20.193.28:0/1266754779 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b40440 msgr2=0x55f712b7d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f05c519fe00 1 --2- 10.20.193.28:0/1266754779 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b40440 0x55f712b7d5c0 secure :-1 s=READY pgs=923 gs=3 cs=0 l=1 c_cookie=5627867805206050201 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05a4016700 tx=0x7f05a40054b0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f05c519fe00 1 -- 10.20.193.28:0/1266754779 shutdown_connections 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f05c519fe00 1 --2- 10.20.193.28:0/1266754779 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b825b0 0x55f712b7db00 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f05c519fe00 1 --2- 10.20.193.28:0/1266754779 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b40440 0x55f712b7d5c0 unknown :-1 s=CLOSED pgs=923 gs=3 cs=0 l=1 c_cookie=5627867805206050201 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f05c519fe00 1 --2- 10.20.193.28:0/1266754779 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b3fb30 0x55f712b55480 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.046+0000 7f05c519fe00 1 -- 10.20.193.28:0/1266754779 >> 10.20.193.28:0/1266754779 conn(0x55f712a5e7a0 msgr2=0x55f712b3dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.047+0000 7f05c519fe00 1 -- 10.20.193.28:0/1266754779 shutdown_connections 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.047+0000 7f05c519fe00 1 -- 10.20.193.28:0/1266754779 wait complete. 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.047+0000 7f05c519fe00 1 -- start start 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.047+0000 7f05c519fe00 1 Processor -- start 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.047+0000 7f05c519fe00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b3fb30 0x55f712b63e80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.047+0000 7f05c519fe00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b40440 0x55f712b5f4d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.049 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.047+0000 7f05c519fe00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b825b0 0x55f712b5fb50 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.047+0000 7f05c519fe00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x55f712b7f030 con 0x55f712b3fb30 2026-02-25T11:58:57.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.047+0000 7f05c519fe00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x55f712b60280 con 0x55f712b40440 2026-02-25T11:58:57.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.047+0000 7f05c519fe00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55f712bea600 con 0x55f712b825b0 2026-02-25T11:58:57.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.048+0000 7f05c0f19640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b3fb30 0x55f712b63e80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.048+0000 7f05c0f19640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b3fb30 0x55f712b63e80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34568/0 (socket says 10.20.193.28:34568) 2026-02-25T11:58:57.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.048+0000 7f05c0f19640 1 -- 10.20.193.28:0/2351461078 learned_addr learned my addr 10.20.193.28:0/2351461078 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:57.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.048+0000 7f05b3fff640 1 --2- 10.20.193.28:0/2351461078 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b40440 0x55f712b5f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.048+0000 7f05c171a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b825b0 0x55f712b5fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.048+0000 7f05b3fff640 1 -- 10.20.193.28:0/2351461078 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b825b0 msgr2=0x55f712b5fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.048+0000 7f05b3fff640 1 --2- 10.20.193.28:0/2351461078 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b825b0 0x55f712b5fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.048+0000 7f05b3fff640 1 -- 10.20.193.28:0/2351461078 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b3fb30 msgr2=0x55f712b63e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.048+0000 7f05b3fff640 1 --2- 10.20.193.28:0/2351461078 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b3fb30 0x55f712b63e80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.050 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.048+0000 7f05b3fff640 1 -- 10.20.193.28:0/2351461078 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7129cf6f0 con 0x55f712b40440 2026-02-25T11:58:57.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.049+0000 7f05b3fff640 1 --2- 10.20.193.28:0/2351461078 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b40440 0x55f712b5f4d0 secure :-1 s=READY pgs=188 gs=2 cs=0 l=1 c_cookie=1691074548701282602 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05a40e4700 tx=0x7f05a4002390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1777e757a40b412a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.049+0000 7f057a7fc640 10 client.0 ms_handle_connect on v2:10.20.193.119:3300/0 2026-02-25T11:58:57.051 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.049+0000 7f057a7fc640 1 -- 10.20.193.28:0/2351461078 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f05a4072210 con 0x55f712b40440 2026-02-25T11:58:57.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.049+0000 7f057a7fc640 1 -- 10.20.193.28:0/2351461078 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f05a40b5230 con 0x55f712b40440 2026-02-25T11:58:57.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.049+0000 7f057a7fc640 1 -- 10.20.193.28:0/2351461078 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f05a40d2070 con 0x55f712b40440 2026-02-25T11:58:57.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.049+0000 7f05c519fe00 1 -- 10.20.193.28:0/2351461078 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a406f070 con 0x55f712b40440 2026-02-25T11:58:57.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.049+0000 7f057a7fc640 1 -- 10.20.193.28:0/2351461078 <== mon.2 v2:10.20.193.119:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f05a40d21d0 con 0x55f712b40440 2026-02-25T11:58:57.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.049+0000 7f057a7fc640 10 client.27353 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:58:57.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.049+0000 7f05c519fe00 10 client.27353 _ll_register_callbacks cb 0x55f7129cca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:57.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.049+0000 7f05c519fe00 1 -- 10.20.193.28:0/2351461078 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b8030ad0 con 0x55f712b40440 2026-02-25T11:58:57.052 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.049+0000 7f057a7fc640 1 -- 10.20.193.28:0/2351461078 <== mon.2 v2:10.20.193.119:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f05a40b8070 con 0x55f712b40440 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.049+0000 7f05c519fe00 10 client.27353 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.049+0000 7f05c519fe00 1 -- 10.20.193.28:0/2351461078 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({fsmap=23}) -- 0x55f7129e1340 con 0x55f712b40440 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.050+0000 7f057a7fc640 1 -- 10.20.193.28:0/2351461078 <== mon.2 v2:10.20.193.119:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f05a40b7be0 con 0x55f712b40440 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.050+0000 7f05c519fe00 10 client.27353 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.050+0000 7f05c519fe00 10 client.27353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.g 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.050+0000 7f05c519fe00 20 client.27353 populate_metadata read hostname 'trial028' 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.050+0000 7f05c519fe00 1 --2- 10.20.193.28:0/2351461078 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f712b850e0 0x55f712c51d60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.050+0000 7f05c519fe00 4 client.27353 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.050+0000 7f05c519fe00 1 -- 10.20.193.28:0/2351461078 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f7126def40 con 0x55f712b850e0 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.050+0000 7f05c0f19640 1 --2- 10.20.193.28:0/2351461078 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f712b850e0 0x55f712c51d60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.050+0000 7f05c0f19640 1 --2- 10.20.193.28:0/2351461078 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f712b850e0 0x55f712c51d60 crc :-1 s=READY pgs=65 gs=4 cs=0 l=0 c_cookie=16368360284854685521 s_cookie=1985549523979224359 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e32818f4ae600f51 server_cookie=1b8e16c5acba4527 in_seq=0 out_seq=0 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.050+0000 7f057a7fc640 10 client.27353 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.050+0000 7f057a7fc640 1 -- 10.20.193.28:0/2351461078 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f7126def40 con 0x55f712b850e0 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.050+0000 7f057a7fc640 10 client.27353 handle_command_reply: tid=0 2026-02-25T11:58:57.053 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.050+0000 7f05c519fe00 1 client.27353 shutdown 2026-02-25T11:58:57.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c519fe00 1 -- 10.20.193.28:0/2351461078 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b40440 msgr2=0x55f712b5f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c519fe00 1 --2- 10.20.193.28:0/2351461078 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b40440 0x55f712b5f4d0 secure :-1 s=READY pgs=188 gs=2 cs=0 l=1 c_cookie=1691074548701282602 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05a40e4700 tx=0x7f05a4002390 comp rx=0 tx=0).stop 2026-02-25T11:58:57.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c519fe00 1 -- 10.20.193.28:0/2351461078 shutdown_connections 2026-02-25T11:58:57.054 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c519fe00 1 --2- 10.20.193.28:0/2351461078 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x55f712b850e0 0x55f712c51d60 crc :-1 s=READY pgs=65 gs=4 cs=0 l=0 c_cookie=16368360284854685521 s_cookie=1985549523979224359 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c519fe00 1 --2- 10.20.193.28:0/2351461078 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x55f712b825b0 0x55f712b5fb50 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c519fe00 1 --2- 10.20.193.28:0/2351461078 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x55f712b40440 0x55f712b5f4d0 unknown :-1 s=CLOSED pgs=188 gs=2 cs=0 l=1 c_cookie=1691074548701282602 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c519fe00 1 --2- 10.20.193.28:0/2351461078 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x55f712b3fb30 0x55f712b63e80 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f057a7fc640 0 client.27353 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:58:57.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c519fe00 1 -- 10.20.193.28:0/2351461078 >> 10.20.193.28:0/2351461078 conn(0x55f712a5e7a0 msgr2=0x55f712b3e500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c519fe00 1 -- 10.20.193.28:0/2351461078 shutdown_connections 2026-02-25T11:58:57.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c519fe00 1 -- 10.20.193.28:0/2351461078 wait complete. 2026-02-25T11:58:57.055 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c519fe00 20 client.27353 trim_cache size 0 max 16384 2026-02-25T11:58:57.055 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:58:57.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c1f1b640 1 -- 10.20.193.28:0/172574404 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f05980b5760 msgr2=0x7f05980d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c1f1b640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f05980b5760 0x7f05980d5c10 secure :-1 s=READY pgs=402 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05ac004740 tx=0x7f05ac005d00 comp rx=0 tx=0).stop 2026-02-25T11:58:57.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c1f1b640 1 -- 10.20.193.28:0/172574404 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05bc10bd50 msgr2=0x7f05bc123b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.051+0000 7f05c1f1b640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05bc10bd50 0x7f05bc123b20 secure :-1 s=READY pgs=920 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f05a400e9c0 tx=0x7f05a400ee90 comp rx=0 tx=0).stop 2026-02-25T11:58:57.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.052+0000 7f05c1f1b640 1 -- 10.20.193.28:0/172574404 shutdown_connections 2026-02-25T11:58:57.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.052+0000 7f05c1f1b640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f05980b5760 0x7f05980d5c10 unknown :-1 s=CLOSED pgs=402 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.052+0000 7f05c1f1b640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f05bc111370 0x7f05bc128ef0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.052+0000 7f05c1f1b640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f05bc10bd50 0x7f05bc123b20 unknown :-1 s=CLOSED pgs=920 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.056 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.052+0000 7f05c1f1b640 1 --2- 10.20.193.28:0/172574404 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f05bc10b360 0x7f05bc1235e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.052+0000 7f05c1f1b640 1 -- 10.20.193.28:0/172574404 >> 10.20.193.28:0/172574404 conn(0x7f05bc0e8e70 msgr2=0x7f05bc109250 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.052+0000 7f05c1f1b640 1 -- 10.20.193.28:0/172574404 shutdown_connections 2026-02-25T11:58:57.057 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.052+0000 7f05c1f1b640 1 -- 10.20.193.28:0/172574404 wait complete. 2026-02-25T11:58:57.190 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph tell mds.1:0 damage ls 2026-02-25T11:58:57.373 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:58:57.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.544+0000 7f9a35aef640 1 -- 10.20.193.28:0/76837851 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a3010ddc0 msgr2=0x7f9a30112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.544+0000 7f9a35aef640 1 --2- 10.20.193.28:0/76837851 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a3010ddc0 0x7f9a30112e10 secure :-1 s=READY pgs=924 gs=1 cs=0 l=1 c_cookie=8745372753045056338 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a18009950 tx=0x7f9a1802f190 comp rx=0 tx=0).stop 2026-02-25T11:58:57.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.544+0000 7f9a35aef640 1 -- 10.20.193.28:0/76837851 shutdown_connections 2026-02-25T11:58:57.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.544+0000 7f9a35aef640 1 --2- 10.20.193.28:0/76837851 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9a301135c0 0x7f9a30173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.544+0000 7f9a35aef640 1 --2- 10.20.193.28:0/76837851 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a3010ddc0 0x7f9a30112e10 unknown :-1 s=CLOSED pgs=924 gs=1 cs=0 l=1 c_cookie=8745372753045056338 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.544+0000 7f9a35aef640 1 --2- 10.20.193.28:0/76837851 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9a3010d3d0 0x7f9a3010d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.544+0000 7f9a35aef640 1 -- 10.20.193.28:0/76837851 >> 10.20.193.28:0/76837851 conn(0x7f9a300eaca0 msgr2=0x7f9a3010b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.544+0000 7f9a35aef640 1 -- 10.20.193.28:0/76837851 shutdown_connections 2026-02-25T11:58:57.546 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.544+0000 7f9a35aef640 1 -- 10.20.193.28:0/76837851 wait complete. 2026-02-25T11:58:57.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.545+0000 7f9a35aef640 1 Processor -- start 2026-02-25T11:58:57.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.545+0000 7f9a35aef640 1 -- start start 2026-02-25T11:58:57.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.545+0000 7f9a35aef640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9a3010d3d0 0x7f9a300f3720 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.545+0000 7f9a35aef640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a3010ddc0 0x7f9a30173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.545+0000 7f9a35aef640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9a301135c0 0x7f9a300f3e20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.545+0000 7f9a35aef640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9a300f8fe0 con 0x7f9a3010ddc0 2026-02-25T11:58:57.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.545+0000 7f9a35aef640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9a300f9150 con 0x7f9a3010d3d0 2026-02-25T11:58:57.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.545+0000 7f9a35aef640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a300f9310 con 0x7f9a301135c0 2026-02-25T11:58:57.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.545+0000 7f9a27fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a3010ddc0 0x7f9a30173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.545+0000 7f9a27fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a3010ddc0 0x7f9a30173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34586/0 (socket says 10.20.193.28:34586) 2026-02-25T11:58:57.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.545+0000 7f9a27fff640 1 -- 10.20.193.28:0/3889206166 learned_addr learned my addr 10.20.193.28:0/3889206166 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:57.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.546+0000 7f9a352ee640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9a301135c0 0x7f9a300f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.546+0000 7f9a34aed640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9a3010d3d0 0x7f9a300f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.546+0000 7f9a27fff640 1 -- 10.20.193.28:0/3889206166 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9a301135c0 msgr2=0x7f9a300f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.546+0000 7f9a27fff640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9a301135c0 0x7f9a300f3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.546+0000 7f9a27fff640 1 -- 10.20.193.28:0/3889206166 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9a3010d3d0 msgr2=0x7f9a300f3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.546+0000 7f9a27fff640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9a3010d3d0 0x7f9a300f3720 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.546+0000 7f9a27fff640 1 -- 10.20.193.28:0/3889206166 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a18009570 con 0x7f9a3010ddc0 2026-02-25T11:58:57.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.546+0000 7f9a27fff640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a3010ddc0 0x7f9a30173e90 secure :-1 s=READY pgs=925 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a180098a0 tx=0x7f9a180028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.546+0000 7f9a25ffb640 1 -- 10.20.193.28:0/3889206166 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a1803d070 con 0x7f9a3010ddc0 2026-02-25T11:58:57.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.546+0000 7f9a25ffb640 1 -- 10.20.193.28:0/3889206166 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9a18002a70 con 0x7f9a3010ddc0 2026-02-25T11:58:57.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.546+0000 7f9a25ffb640 1 -- 10.20.193.28:0/3889206166 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a18041890 con 0x7f9a3010ddc0 2026-02-25T11:58:57.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.546+0000 7f9a35aef640 1 -- 10.20.193.28:0/3889206166 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a300f9590 con 0x7f9a3010ddc0 2026-02-25T11:58:57.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.546+0000 7f9a35aef640 1 -- 10.20.193.28:0/3889206166 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a300f99b0 con 0x7f9a3010ddc0 2026-02-25T11:58:57.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.549+0000 7f9a25ffb640 1 -- 10.20.193.28:0/3889206166 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f9a18041a20 con 0x7f9a3010ddc0 2026-02-25T11:58:57.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.551+0000 7f9a25ffb640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9a0c0b5750 0x7f9a0c0d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.552 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.551+0000 7f9a34aed640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9a0c0b5750 0x7f9a0c0d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.551+0000 7f9a25ffb640 1 -- 10.20.193.28:0/3889206166 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9a1810b110 con 0x7f9a3010ddc0 2026-02-25T11:58:57.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.551+0000 7f9a34aed640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9a0c0b5750 0x7f9a0c0d5c00 secure :-1 s=READY pgs=403 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a20004740 tx=0x7f9a2000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.552+0000 7f9a38d73e00 1 Processor -- start 2026-02-25T11:58:57.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.552+0000 7f9a38d73e00 1 -- start start 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.552+0000 7f9a38d73e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e2b30 0x5572090e2f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.552+0000 7f9a38d73e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e3440 0x557209127ba0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.552+0000 7f9a38d73e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572091280e0 0x5572091484c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.552+0000 7f9a38d73e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x557208ff88a0 con 0x5572090e3440 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.552+0000 7f9a38d73e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x557208ff45d0 con 0x5572090e2b30 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.552+0000 7f9a38d73e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x557208fee570 con 0x5572091280e0 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.552+0000 7f9a27fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e3440 0x557209127ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.553+0000 7f9a27fff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e3440 0x557209127ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34592/0 (socket says 10.20.193.28:34592) 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.553+0000 7f9a27fff640 1 -- 10.20.193.28:0/1597991778 learned_addr learned my addr 10.20.193.28:0/1597991778 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.553+0000 7f9a34aed640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e2b30 0x5572090e2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.553+0000 7f9a352ee640 1 --2- 10.20.193.28:0/1597991778 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572091280e0 0x5572091484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.553+0000 7f9a27fff640 1 -- 10.20.193.28:0/1597991778 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572091280e0 msgr2=0x5572091484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.553+0000 7f9a27fff640 1 --2- 10.20.193.28:0/1597991778 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572091280e0 0x5572091484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.553+0000 7f9a27fff640 1 -- 10.20.193.28:0/1597991778 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e2b30 msgr2=0x5572090e2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.553+0000 7f9a27fff640 1 --2- 10.20.193.28:0/1597991778 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e2b30 0x5572090e2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.553+0000 7f9a27fff640 1 -- 10.20.193.28:0/1597991778 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557208f99d10 con 0x5572090e3440 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.553+0000 7f9a27fff640 1 --2- 10.20.193.28:0/1597991778 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e3440 0x557209127ba0 secure :-1 s=READY pgs=926 gs=1 cs=0 l=1 c_cookie=12165956379027465777 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a18108c00 tx=0x7f9a18005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d628de64d46631 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.553+0000 7f99fd4e4640 1 -- 10.20.193.28:0/1597991778 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a18041d00 con 0x5572090e3440 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.553+0000 7f99fd4e4640 1 -- 10.20.193.28:0/1597991778 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9a180d71b0 con 0x5572090e3440 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.553+0000 7f99fd4e4640 1 -- 10.20.193.28:0/1597991778 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a180cf7a0 con 0x5572090e3440 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.554+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1597991778 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e3440 msgr2=0x557209127ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.554+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1597991778 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e3440 0x557209127ba0 secure :-1 s=READY pgs=926 gs=1 cs=0 l=1 c_cookie=12165956379027465777 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a18108c00 tx=0x7f9a18005ab0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.554+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1597991778 shutdown_connections 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.554+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1597991778 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572091280e0 0x5572091484c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.554+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1597991778 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e3440 0x557209127ba0 unknown :-1 s=CLOSED pgs=926 gs=1 cs=0 l=1 c_cookie=12165956379027465777 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.554+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1597991778 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e2b30 0x5572090e2f00 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.554+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1597991778 >> 10.20.193.28:0/1597991778 conn(0x5572090017a0 msgr2=0x5572090e1410 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.554+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1597991778 shutdown_connections 2026-02-25T11:58:57.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.554+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1597991778 wait complete. 2026-02-25T11:58:57.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.555+0000 7f9a38d73e00 1 -- start start 2026-02-25T11:58:57.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.555+0000 7f9a38d73e00 1 Processor -- start 2026-02-25T11:58:57.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a38d73e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e2b30 0x5572090facf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a38d73e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e3440 0x5572090fb230 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.557 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a38d73e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572091280e0 0x557209100220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a38d73e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x557209100760 con 0x5572090e2b30 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a38d73e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5572091008d0 con 0x5572090e3440 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a38d73e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x557209100a90 con 0x5572091280e0 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a34aed640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e2b30 0x5572090facf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a352ee640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572091280e0 0x557209100220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a352ee640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572091280e0 0x557209100220 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48956/0 (socket says 10.20.193.28:48956) 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a352ee640 1 -- 10.20.193.28:0/1666054307 learned_addr learned my addr 10.20.193.28:0/1666054307 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a27fff640 1 --2- 10.20.193.28:0/1666054307 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e3440 0x5572090fb230 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a352ee640 1 -- 10.20.193.28:0/1666054307 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e3440 msgr2=0x5572090fb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a352ee640 1 --2- 10.20.193.28:0/1666054307 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e3440 0x5572090fb230 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a352ee640 1 -- 10.20.193.28:0/1666054307 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e2b30 msgr2=0x5572090facf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a352ee640 1 --2- 10.20.193.28:0/1666054307 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e2b30 0x5572090facf0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a352ee640 1 -- 10.20.193.28:0/1666054307 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557208f723c0 con 0x5572091280e0 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.556+0000 7f9a34aed640 1 --2- 10.20.193.28:0/1666054307 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e2b30 0x5572090facf0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.557+0000 7f9a352ee640 1 --2- 10.20.193.28:0/1666054307 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572091280e0 0x557209100220 secure :-1 s=READY pgs=240 gs=2 cs=0 l=1 c_cookie=4687427000041633642 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a2c012490 tx=0x7f9a2c012960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=410d1605800e776a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.557+0000 7f99ee7fc640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.557+0000 7f99ee7fc640 1 -- 10.20.193.28:0/1666054307 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a2c017070 con 0x5572091280e0 2026-02-25T11:58:57.558 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.557+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1666054307 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a300f99b0 con 0x5572091280e0 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.557+0000 7f99ee7fc640 1 -- 10.20.193.28:0/1666054307 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9a2c005500 con 0x5572091280e0 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.557+0000 7f99ee7fc640 1 -- 10.20.193.28:0/1666054307 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a2c011cf0 con 0x5572091280e0 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.557+0000 7f9a38d73e00 10 client.27358 _ll_register_callbacks cb 0x557208f6fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.557+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1666054307 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557208c846f0 con 0x5572091280e0 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.557+0000 7f99ee7fc640 1 -- 10.20.193.28:0/1666054307 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9a2c011e80 con 0x5572091280e0 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.557+0000 7f99ee7fc640 1 -- 10.20.193.28:0/1666054307 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9a2c0114f0 con 0x5572091280e0 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.557+0000 7f9a38d73e00 10 client.27358 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.557+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1666054307 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x557208f840e0 con 0x5572091280e0 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.558+0000 7f99ee7fc640 1 -- 10.20.193.28:0/1666054307 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9a2c025050 con 0x5572091280e0 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.558+0000 7f9a38d73e00 10 client.27358 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.558+0000 7f9a38d73e00 10 client.27358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.558+0000 7f9a38d73e00 20 client.27358 populate_metadata read hostname 'trial028' 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.558+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1666054307 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5572091255a0 0x557209125a40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.558+0000 7f9a38d73e00 4 client.27358 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:58:57.559 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.558+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1666054307 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557208fc0fc0 con 0x5572091255a0 2026-02-25T11:58:57.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.558+0000 7f9a34aed640 1 --2- 10.20.193.28:0/1666054307 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5572091255a0 0x557209125a40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.559+0000 7f99ee7fc640 10 client.27358 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:57.560 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.559+0000 7f9a34aed640 1 --2- 10.20.193.28:0/1666054307 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5572091255a0 0x557209125a40 crc :-1 s=READY pgs=186 gs=4 cs=0 l=0 c_cookie=5680617395711429203 s_cookie=2221039295438542143 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ed59b6c9848ca53 server_cookie=1ed2b778bccc553f in_seq=0 out_seq=0 2026-02-25T11:58:57.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.559+0000 7f99ee7fc640 1 -- 10.20.193.28:0/1666054307 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x557208fc0fc0 con 0x5572091255a0 2026-02-25T11:58:57.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.559+0000 7f99ee7fc640 10 client.27358 handle_command_reply: tid=0 2026-02-25T11:58:57.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.560+0000 7f9a38d73e00 1 client.27358 shutdown 2026-02-25T11:58:57.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.560+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1666054307 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572091280e0 msgr2=0x557209100220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.560+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1666054307 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572091280e0 0x557209100220 secure :-1 s=READY pgs=240 gs=2 cs=0 l=1 c_cookie=4687427000041633642 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a2c012490 tx=0x7f9a2c012960 comp rx=0 tx=0).stop 2026-02-25T11:58:57.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.560+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1666054307 shutdown_connections 2026-02-25T11:58:57.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.560+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1666054307 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5572091255a0 0x557209125a40 crc :-1 s=READY pgs=186 gs=4 cs=0 l=0 c_cookie=5680617395711429203 s_cookie=2221039295438542143 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.560+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1666054307 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572091280e0 0x557209100220 unknown :-1 s=CLOSED pgs=240 gs=2 cs=0 l=1 c_cookie=4687427000041633642 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.560+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1666054307 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e3440 0x5572090fb230 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.560+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1666054307 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e2b30 0x5572090facf0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.561 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.560+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1666054307 >> 10.20.193.28:0/1666054307 conn(0x5572090017a0 msgr2=0x5572090e1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.560+0000 7f99ee7fc640 0 client.27358 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:57.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.560+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1666054307 shutdown_connections 2026-02-25T11:58:57.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.560+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1666054307 wait complete. 2026-02-25T11:58:57.562 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.560+0000 7f9a38d73e00 20 client.27358 trim_cache size 0 max 16384 2026-02-25T11:58:57.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.567+0000 7f9a38d73e00 1 Processor -- start 2026-02-25T11:58:57.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.567+0000 7f9a38d73e00 1 -- start start 2026-02-25T11:58:57.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.567+0000 7f9a38d73e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 0x557209127330 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.567+0000 7f9a38d73e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e3440 0x557209127870 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a38d73e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572091280e0 0x55720911f6f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a38d73e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x557208fec1a0 con 0x5572090e3440 2026-02-25T11:58:57.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a38d73e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x557208cac700 con 0x5572091280e0 2026-02-25T11:58:57.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a38d73e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5572090abe30 con 0x5572090e2b30 2026-02-25T11:58:57.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a34aed640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 0x557209127330 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a34aed640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 0x557209127330 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48970/0 (socket says 10.20.193.28:48970) 2026-02-25T11:58:57.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a34aed640 1 -- 10.20.193.28:0/1839713369 learned_addr learned my addr 10.20.193.28:0/1839713369 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:57.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a352ee640 1 --2- 10.20.193.28:0/1839713369 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572091280e0 0x55720911f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a27fff640 1 --2- 10.20.193.28:0/1839713369 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e3440 0x557209127870 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a34aed640 1 -- 10.20.193.28:0/1839713369 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572091280e0 msgr2=0x55720911f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a34aed640 1 --2- 10.20.193.28:0/1839713369 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572091280e0 0x55720911f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a34aed640 1 -- 10.20.193.28:0/1839713369 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e3440 msgr2=0x557209127870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a34aed640 1 --2- 10.20.193.28:0/1839713369 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e3440 0x557209127870 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a34aed640 1 -- 10.20.193.28:0/1839713369 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572090ed5f0 con 0x5572090e2b30 2026-02-25T11:58:57.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a27fff640 1 --2- 10.20.193.28:0/1839713369 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e3440 0x557209127870 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:58:57.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.568+0000 7f9a34aed640 1 --2- 10.20.193.28:0/1839713369 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 0x557209127330 secure :-1 s=READY pgs=241 gs=1 cs=0 l=1 c_cookie=11007974913988544891 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a200023d0 tx=0x7f9a20013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98c42edc8cb2c97b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.569+0000 7f99fcce3640 1 -- 10.20.193.28:0/1839713369 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a20024040 con 0x5572090e2b30 2026-02-25T11:58:57.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.569+0000 7f99fcce3640 1 -- 10.20.193.28:0/1839713369 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9a20015070 con 0x5572090e2b30 2026-02-25T11:58:57.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.569+0000 7f99fcce3640 1 -- 10.20.193.28:0/1839713369 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a2002a320 con 0x5572090e2b30 2026-02-25T11:58:57.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.569+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1839713369 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 msgr2=0x557209127330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.569+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1839713369 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 0x557209127330 secure :-1 s=READY pgs=241 gs=1 cs=0 l=1 c_cookie=11007974913988544891 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a200023d0 tx=0x7f9a20013000 comp rx=0 tx=0).stop 2026-02-25T11:58:57.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.569+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1839713369 shutdown_connections 2026-02-25T11:58:57.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.569+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1839713369 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572091280e0 0x55720911f6f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.569+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1839713369 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572090e3440 0x557209127870 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.569+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/1839713369 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 0x557209127330 unknown :-1 s=CLOSED pgs=241 gs=1 cs=0 l=1 c_cookie=11007974913988544891 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.569+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1839713369 >> 10.20.193.28:0/1839713369 conn(0x5572090017a0 msgr2=0x5572090e0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.571 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.569+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1839713369 shutdown_connections 2026-02-25T11:58:57.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.569+0000 7f9a38d73e00 1 -- 10.20.193.28:0/1839713369 wait complete. 2026-02-25T11:58:57.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.570+0000 7f9a38d73e00 1 -- start start 2026-02-25T11:58:57.572 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.570+0000 7f9a38d73e00 1 Processor -- start 2026-02-25T11:58:57.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.570+0000 7f9a38d73e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 0x557209106ec0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.570+0000 7f9a38d73e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e3440 0x5572091025e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.570+0000 7f9a38d73e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572091280e0 0x557209102b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.570+0000 7f9a38d73e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x557209103240 con 0x5572091280e0 2026-02-25T11:58:57.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.570+0000 7f9a38d73e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5572091033b0 con 0x5572090e3440 2026-02-25T11:58:57.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.570+0000 7f9a38d73e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x55720918d600 con 0x5572090e2b30 2026-02-25T11:58:57.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.571+0000 7f9a34aed640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 0x557209106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.571+0000 7f9a34aed640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 0x557209106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:48974/0 (socket says 10.20.193.28:48974) 2026-02-25T11:58:57.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.571+0000 7f9a27fff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e3440 0x5572091025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.571+0000 7f9a34aed640 1 -- 10.20.193.28:0/802518573 learned_addr learned my addr 10.20.193.28:0/802518573 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:57.573 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.571+0000 7f9a352ee640 1 --2- 10.20.193.28:0/802518573 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572091280e0 0x557209102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.574 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.571+0000 7f9a34aed640 1 -- 10.20.193.28:0/802518573 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e3440 msgr2=0x5572091025e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.571+0000 7f9a34aed640 1 --2- 10.20.193.28:0/802518573 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e3440 0x5572091025e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.571+0000 7f9a34aed640 1 -- 10.20.193.28:0/802518573 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572091280e0 msgr2=0x557209102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:58:57.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.571+0000 7f9a34aed640 1 --2- 10.20.193.28:0/802518573 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572091280e0 0x557209102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.571+0000 7f9a34aed640 1 -- 10.20.193.28:0/802518573 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a20011620 con 0x5572090e2b30 2026-02-25T11:58:57.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.571+0000 7f9a34aed640 1 --2- 10.20.193.28:0/802518573 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 0x557209106ec0 secure :-1 s=READY pgs=242 gs=1 cs=0 l=1 c_cookie=11809132612921156729 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a20010c40 tx=0x7f9a20004ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3e2778b42a56479 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.571+0000 7f99ee7fc640 10 client.0 ms_handle_connect on v2:10.20.193.167:3300/0 2026-02-25T11:58:57.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f99ee7fc640 1 -- 10.20.193.28:0/802518573 <== mon.1 v2:10.20.193.167:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a20024030 con 0x5572090e2b30 2026-02-25T11:58:57.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f99ee7fc640 1 -- 10.20.193.28:0/802518573 <== mon.1 v2:10.20.193.167:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f9a20005790 con 0x5572090e2b30 2026-02-25T11:58:57.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f99ee7fc640 1 -- 10.20.193.28:0/802518573 <== mon.1 v2:10.20.193.167:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9a20034460 con 0x5572090e2b30 2026-02-25T11:58:57.575 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f9a38d73e00 1 -- 10.20.193.28:0/802518573 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557208f99d10 con 0x5572090e2b30 2026-02-25T11:58:57.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f9a38d73e00 10 client.27364 _ll_register_callbacks cb 0x557208f6fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:58:57.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f9a38d73e00 1 -- 10.20.193.28:0/802518573 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a180d71b0 con 0x5572090e2b30 2026-02-25T11:58:57.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f99ee7fc640 1 -- 10.20.193.28:0/802518573 <== mon.1 v2:10.20.193.167:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9a20017070 con 0x5572090e2b30 2026-02-25T11:58:57.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f99ee7fc640 1 -- 10.20.193.28:0/802518573 <== mon.1 v2:10.20.193.167:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9a200052d0 con 0x5572090e2b30 2026-02-25T11:58:57.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f9a38d73e00 10 client.27364 fetch_fsmap learned FSMap version 23 2026-02-25T11:58:57.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f9a38d73e00 1 -- 10.20.193.28:0/802518573 --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_subscribe({fsmap=23}) -- 0x557208f7eb60 con 0x5572090e2b30 2026-02-25T11:58:57.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f99ee7fc640 1 -- 10.20.193.28:0/802518573 <== mon.1 v2:10.20.193.167:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9a20032040 con 0x5572090e2b30 2026-02-25T11:58:57.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f9a38d73e00 10 client.27364 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:58:57.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f9a38d73e00 10 client.27364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2026-02-25T11:58:57.576 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f9a38d73e00 20 client.27364 populate_metadata read hostname 'trial028' 2026-02-25T11:58:57.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/802518573 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5572091d5460 0x5572091f5910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f9a38d73e00 4 client.27364 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2026-02-25T11:58:57.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.572+0000 7f9a38d73e00 1 -- 10.20.193.28:0/802518573 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557208c81f40 con 0x5572091d5460 2026-02-25T11:58:57.577 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.573+0000 7f9a27fff640 1 --2- 10.20.193.28:0/802518573 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5572091d5460 0x5572091f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.573+0000 7f9a27fff640 1 --2- 10.20.193.28:0/802518573 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5572091d5460 0x5572091f5910 crc :-1 s=READY pgs=187 gs=4 cs=0 l=0 c_cookie=799658681338820672 s_cookie=10620308954681928318 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b18f5557ac1f840 server_cookie=9362eab9b8b76a7e in_seq=0 out_seq=0 2026-02-25T11:58:57.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.573+0000 7f99ee7fc640 10 client.27364 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:57.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.573+0000 7f99ee7fc640 1 -- 10.20.193.28:0/802518573 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557208c81f40 con 0x5572091d5460 2026-02-25T11:58:57.578 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.573+0000 7f99ee7fc640 10 client.27364 handle_command_reply: tid=0 2026-02-25T11:58:57.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.573+0000 7f9a38d73e00 1 client.27364 shutdown 2026-02-25T11:58:57.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.573+0000 7f9a38d73e00 1 -- 10.20.193.28:0/802518573 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 msgr2=0x557209106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.573+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/802518573 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 0x557209106ec0 secure :-1 s=READY pgs=242 gs=1 cs=0 l=1 c_cookie=11809132612921156729 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a20010c40 tx=0x7f9a20004ec0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a38d73e00 1 -- 10.20.193.28:0/802518573 shutdown_connections 2026-02-25T11:58:57.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/802518573 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5572091d5460 0x5572091f5910 crc :-1 s=READY pgs=187 gs=4 cs=0 l=0 c_cookie=799658681338820672 s_cookie=10620308954681928318 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/802518573 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5572091280e0 0x557209102b40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/802518573 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5572090e3440 0x5572091025e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a38d73e00 1 --2- 10.20.193.28:0/802518573 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5572090e2b30 0x557209106ec0 unknown :-1 s=CLOSED pgs=242 gs=1 cs=0 l=1 c_cookie=11809132612921156729 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.579 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f99ee7fc640 0 client.27364 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:58:57.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a38d73e00 1 -- 10.20.193.28:0/802518573 >> 10.20.193.28:0/802518573 conn(0x5572090017a0 msgr2=0x5572090e1500 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a38d73e00 1 -- 10.20.193.28:0/802518573 shutdown_connections 2026-02-25T11:58:57.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a38d73e00 1 -- 10.20.193.28:0/802518573 wait complete. 2026-02-25T11:58:57.580 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a38d73e00 20 client.27364 trim_cache size 0 max 16384 2026-02-25T11:58:57.580 INFO:teuthology.orchestra.run.trial028.stdout:[] 2026-02-25T11:58:57.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a35aef640 1 -- 10.20.193.28:0/3889206166 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9a0c0b5750 msgr2=0x7f9a0c0d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a35aef640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9a0c0b5750 0x7f9a0c0d5c00 secure :-1 s=READY pgs=403 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a20004740 tx=0x7f9a2000b410 comp rx=0 tx=0).stop 2026-02-25T11:58:57.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a35aef640 1 -- 10.20.193.28:0/3889206166 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a3010ddc0 msgr2=0x7f9a30173e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a35aef640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a3010ddc0 0x7f9a30173e90 secure :-1 s=READY pgs=925 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9a180098a0 tx=0x7f9a180028d0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.581 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a35aef640 1 -- 10.20.193.28:0/3889206166 shutdown_connections 2026-02-25T11:58:57.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a35aef640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f9a0c0b5750 0x7f9a0c0d5c00 unknown :-1 s=CLOSED pgs=403 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a35aef640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f9a301135c0 0x7f9a300f3e20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a35aef640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f9a3010ddc0 0x7f9a30173e90 unknown :-1 s=CLOSED pgs=925 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a35aef640 1 --2- 10.20.193.28:0/3889206166 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f9a3010d3d0 0x7f9a300f3720 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a35aef640 1 -- 10.20.193.28:0/3889206166 >> 10.20.193.28:0/3889206166 conn(0x7f9a300eaca0 msgr2=0x7f9a3010b860 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a35aef640 1 -- 10.20.193.28:0/3889206166 shutdown_connections 2026-02-25T11:58:57.582 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.574+0000 7f9a35aef640 1 -- 10.20.193.28:0/3889206166 wait complete. 2026-02-25T11:58:57.678 INFO:tasks.fwd_scrub:done joining 2026-02-25T11:58:57.679 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2026-02-25T11:58:57.703 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T11:58:57.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.825+0000 7eff0accc640 1 -- 10.20.193.28:0/3179382020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff04060e40 msgr2=0x7eff040612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.825+0000 7eff0accc640 1 --2- 10.20.193.28:0/3179382020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff04060e40 0x7eff040612b0 secure :-1 s=READY pgs=927 gs=1 cs=0 l=1 c_cookie=14915451593827406990 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efeec009950 tx=0x7efeec02f190 comp rx=0 tx=0).stop 2026-02-25T11:58:57.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.825+0000 7eff0accc640 1 -- 10.20.193.28:0/3179382020 shutdown_connections 2026-02-25T11:58:57.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.825+0000 7eff0accc640 1 --2- 10.20.193.28:0/3179382020 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff04150f80 0x7eff04171360 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.827 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.825+0000 7eff0accc640 1 --2- 10.20.193.28:0/3179382020 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff04060e40 0x7eff040612b0 unknown :-1 s=CLOSED pgs=927 gs=1 cs=0 l=1 c_cookie=14915451593827406990 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.825+0000 7eff0accc640 1 --2- 10.20.193.28:0/3179382020 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff04062270 0x7eff04060900 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.825+0000 7eff0accc640 1 -- 10.20.193.28:0/3179382020 >> 10.20.193.28:0/3179382020 conn(0x7eff040ea5f0 msgr2=0x7eff0410aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.825+0000 7eff0accc640 1 -- 10.20.193.28:0/3179382020 shutdown_connections 2026-02-25T11:58:57.828 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.825+0000 7eff0accc640 1 -- 10.20.193.28:0/3179382020 wait complete. 2026-02-25T11:58:57.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7eff0accc640 1 Processor -- start 2026-02-25T11:58:57.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7eff0accc640 1 -- start start 2026-02-25T11:58:57.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7eff0accc640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff04060e40 0x7eff0415c860 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7eff0accc640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff04062270 0x7eff0415ceb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.830 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7eff0accc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff04150f80 0x7eff0415da60 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7eff0accc640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7eff0415e1a0 con 0x7eff04150f80 2026-02-25T11:58:57.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7eff0accc640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7eff0415e310 con 0x7eff04062270 2026-02-25T11:58:57.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7eff0accc640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff0415e4d0 con 0x7eff04060e40 2026-02-25T11:58:57.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7eff09242640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff04150f80 0x7eff0415da60 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7efefbfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff04062270 0x7eff0415ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7efefbfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff04062270 0x7eff0415ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:57302/0 (socket says 10.20.193.28:57302) 2026-02-25T11:58:57.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7eff09242640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff04150f80 0x7eff0415da60 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34642/0 (socket says 10.20.193.28:34642) 2026-02-25T11:58:57.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7efefbfff640 1 -- 10.20.193.28:0/2773554657 learned_addr learned my addr 10.20.193.28:0/2773554657 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:57.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.826+0000 7eff08a41640 1 --2- 10.20.193.28:0/2773554657 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff04060e40 0x7eff0415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.827+0000 7efefbfff640 1 -- 10.20.193.28:0/2773554657 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff04060e40 msgr2=0x7eff0415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.827+0000 7efefbfff640 1 --2- 10.20.193.28:0/2773554657 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff04060e40 0x7eff0415c860 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.827+0000 7efefbfff640 1 -- 10.20.193.28:0/2773554657 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff04150f80 msgr2=0x7eff0415da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.827+0000 7efefbfff640 1 --2- 10.20.193.28:0/2773554657 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff04150f80 0x7eff0415da60 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.827+0000 7efefbfff640 1 -- 10.20.193.28:0/2773554657 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeec009570 con 0x7eff04062270 2026-02-25T11:58:57.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.827+0000 7efefbfff640 1 --2- 10.20.193.28:0/2773554657 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff04062270 0x7eff0415ceb0 secure :-1 s=READY pgs=189 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efeec005b30 tx=0x7efeec0028d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.827+0000 7efef9ffb640 1 -- 10.20.193.28:0/2773554657 <== mon.2 v2:10.20.193.119:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efeec03d070 con 0x7eff04062270 2026-02-25T11:58:57.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.827+0000 7efef9ffb640 1 -- 10.20.193.28:0/2773554657 <== mon.2 v2:10.20.193.119:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7efeec02fc10 con 0x7eff04062270 2026-02-25T11:58:57.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.827+0000 7eff0accc640 1 -- 10.20.193.28:0/2773554657 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0415e750 con 0x7eff04062270 2026-02-25T11:58:57.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.827+0000 7efef9ffb640 1 -- 10.20.193.28:0/2773554657 <== mon.2 v2:10.20.193.119:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7efeec041820 con 0x7eff04062270 2026-02-25T11:58:57.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.827+0000 7eff0accc640 1 -- 10.20.193.28:0/2773554657 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0415ec70 con 0x7eff04062270 2026-02-25T11:58:57.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.828+0000 7eff0accc640 1 -- 10.20.193.28:0/2773554657 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff04175530 con 0x7eff04062270 2026-02-25T11:58:57.833 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.830+0000 7efef9ffb640 1 -- 10.20.193.28:0/2773554657 <== mon.2 v2:10.20.193.119:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7efeec0386a0 con 0x7eff04062270 2026-02-25T11:58:57.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.832+0000 7efef9ffb640 1 --2- 10.20.193.28:0/2773554657 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efee00b56e0 0x7efee00d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:57.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.832+0000 7efef9ffb640 1 -- 10.20.193.28:0/2773554657 <== mon.2 v2:10.20.193.119:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7efeec10b720 con 0x7eff04062270 2026-02-25T11:58:57.834 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.832+0000 7eff08a41640 1 --2- 10.20.193.28:0/2773554657 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efee00b56e0 0x7efee00d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:57.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.833+0000 7eff08a41640 1 --2- 10.20.193.28:0/2773554657 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efee00b56e0 0x7efee00d5b90 secure :-1 s=READY pgs=404 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efef4004740 tx=0x7efef4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:57.835 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.833+0000 7efef9ffb640 1 -- 10.20.193.28:0/2773554657 <== mon.2 v2:10.20.193.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7efeec0ce1a0 con 0x7eff04062270 2026-02-25T11:58:57.950 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.949+0000 7eff0accc640 1 -- 10.20.193.28:0/2773554657 --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff04060900 con 0x7eff04062270 2026-02-25T11:58:57.951 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.950+0000 7efef9ffb640 1 -- 10.20.193.28:0/2773554657 <== mon.2 v2:10.20.193.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22689 (secure 0 0 0) 0x7efeec0d1350 con 0x7eff04062270 2026-02-25T11:58:57.951 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:58:57.951 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":88,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","created":"2026-02-25T11:42:51.962844+0000","modified":"2026-02-25T11:48:18.148313+0000","last_up_change":"2026-02-25T11:46:45.156629+0000","last_in_change":"2026-02-25T11:46:31.585002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":38,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T11:44:43.266336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-25T11:47:11.236575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-25T11:47:12.196775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf596df7-d1fb-4d10-8d4c-14a35210bab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6803","nonce":668944205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6805","nonce":668944205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6809","nonce":668944205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6807","nonce":668944205}]},"public_addr":"10.20.193.28:6803/668944205","cluster_addr":"10.20.193.28:6805/668944205","heartbeat_back_addr":"10.20.193.28:6809/668944205","heartbeat_front_addr":"10.20.193.28:6807/668944205","state":["exists","up"]},{"osd":1,"uuid":"185719c0-7c2d-431f-9925-88bbee86a98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6800","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6801","nonce":3486566242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6802","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6803","nonce":3486566242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6806","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6807","nonce":3486566242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6804","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6805","nonce":3486566242}]},"public_addr":"10.20.193.119:6801/3486566242","cluster_addr":"10.20.193.119:6803/3486566242","heartbeat_back_addr":"10.20.193.119:6807/3486566242","heartbeat_front_addr":"10.20.193.119:6805/3486566242","state":["exists","up"]},{"osd":2,"uuid":"4de13b90-1c3c-4614-9fef-2d7315626433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6800","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6801","nonce":484781559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6802","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6803","nonce":484781559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6806","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6807","nonce":484781559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6804","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6805","nonce":484781559}]},"public_addr":"10.20.193.167:6801/484781559","cluster_addr":"10.20.193.167:6803/484781559","heartbeat_back_addr":"10.20.193.167:6807/484781559","heartbeat_front_addr":"10.20.193.167:6805/484781559","state":["exists","up"]},{"osd":3,"uuid":"45d9ea8e-5a77-4738-96da-ada98515fc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6811","nonce":2452541436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6813","nonce":2452541436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6817","nonce":2452541436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6815","nonce":2452541436}]},"public_addr":"10.20.193.28:6811/2452541436","cluster_addr":"10.20.193.28:6813/2452541436","heartbeat_back_addr":"10.20.193.28:6817/2452541436","heartbeat_front_addr":"10.20.193.28:6815/2452541436","state":["exists","up"]},{"osd":4,"uuid":"c9c3f512-665b-44b2-a6f0-f3356a4b661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6808","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6809","nonce":135971996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6810","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6811","nonce":135971996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6814","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6815","nonce":135971996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6812","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6813","nonce":135971996}]},"public_addr":"10.20.193.119:6809/135971996","cluster_addr":"10.20.193.119:6811/135971996","heartbeat_back_addr":"10.20.193.119:6815/135971996","heartbeat_front_addr":"10.20.193.119:6813/135971996","state":["exists","up"]},{"osd":5,"uuid":"777c3819-5c90-4b2b-a545-c024884c2eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6808","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6809","nonce":4203420884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6810","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6811","nonce":4203420884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6814","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6815","nonce":4203420884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6812","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6813","nonce":4203420884}]},"public_addr":"10.20.193.167:6809/4203420884","cluster_addr":"10.20.193.167:6811/4203420884","heartbeat_back_addr":"10.20.193.167:6815/4203420884","heartbeat_front_addr":"10.20.193.167:6813/4203420884","state":["exists","up"]},{"osd":6,"uuid":"bbd9de64-b9fe-499e-b451-c16828be2c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6819","nonce":621484999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6821","nonce":621484999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6825","nonce":621484999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6823","nonce":621484999}]},"public_addr":"10.20.193.28:6819/621484999","cluster_addr":"10.20.193.28:6821/621484999","heartbeat_back_addr":"10.20.193.28:6825/621484999","heartbeat_front_addr":"10.20.193.28:6823/621484999","state":["exists","up"]},{"osd":7,"uuid":"546d9f28-597b-454c-8456-fcc2d46c69f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6816","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6817","nonce":2724541747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6818","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6819","nonce":2724541747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6822","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6823","nonce":2724541747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6820","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6821","nonce":2724541747}]},"public_addr":"10.20.193.119:6817/2724541747","cluster_addr":"10.20.193.119:6819/2724541747","heartbeat_back_addr":"10.20.193.119:6823/2724541747","heartbeat_front_addr":"10.20.193.119:6821/2724541747","state":["exists","up"]},{"osd":8,"uuid":"a1a7cea6-3586-4bb2-aa4c-bf6f7799a501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6816","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6817","nonce":1328734317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6818","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6819","nonce":1328734317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6822","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6823","nonce":1328734317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6820","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6821","nonce":1328734317}]},"public_addr":"10.20.193.167:6817/1328734317","cluster_addr":"10.20.193.167:6819/1328734317","heartbeat_back_addr":"10.20.193.167:6823/1328734317","heartbeat_front_addr":"10.20.193.167:6821/1328734317","state":["exists","up"]},{"osd":9,"uuid":"03fef134-6946-43d5-9c26-dc0e59fffe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6827","nonce":354500625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6829","nonce":354500625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6833","nonce":354500625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6831","nonce":354500625}]},"public_addr":"10.20.193.28:6827/354500625","cluster_addr":"10.20.193.28:6829/354500625","heartbeat_back_addr":"10.20.193.28:6833/354500625","heartbeat_front_addr":"10.20.193.28:6831/354500625","state":["exists","up"]},{"osd":10,"uuid":"b46c29ef-285c-4184-be28-d99b76bc0e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6824","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6825","nonce":2996581116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6826","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6827","nonce":2996581116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6830","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6831","nonce":2996581116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6828","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6829","nonce":2996581116}]},"public_addr":"10.20.193.119:6825/2996581116","cluster_addr":"10.20.193.119:6827/2996581116","heartbeat_back_addr":"10.20.193.119:6831/2996581116","heartbeat_front_addr":"10.20.193.119:6829/2996581116","state":["exists","up"]},{"osd":11,"uuid":"f6696d8c-5de7-4fd9-b693-524f204e4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6824","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6825","nonce":2182906700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6826","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6827","nonce":2182906700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6830","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6831","nonce":2182906700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6828","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6829","nonce":2182906700}]},"public_addr":"10.20.193.167:6825/2182906700","cluster_addr":"10.20.193.167:6827/2182906700","heartbeat_back_addr":"10.20.193.167:6831/2182906700","heartbeat_front_addr":"10.20.193.167:6829/2182906700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:13.410235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:27.254773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:40.956615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:54.529866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:08.306772+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:21.899498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:35.500407+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:49.094965+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:02.790138+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:16.286937+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:29.922722+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:43.396028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.28:0/1467902196":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6800/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/2274270913":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/60087422":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:6800/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/1725332841":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/3082630924":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/2391715078":"2026-02-26T11:43:04.102029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T11:58:57.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.951+0000 7eff0accc640 1 -- 10.20.193.28:0/2773554657 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efee00b56e0 msgr2=0x7efee00d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.952 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.951+0000 7eff0accc640 1 --2- 10.20.193.28:0/2773554657 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efee00b56e0 0x7efee00d5b90 secure :-1 s=READY pgs=404 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efef4004740 tx=0x7efef4005f90 comp rx=0 tx=0).stop 2026-02-25T11:58:57.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.951+0000 7eff0accc640 1 -- 10.20.193.28:0/2773554657 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff04062270 msgr2=0x7eff0415ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:57.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.951+0000 7eff0accc640 1 --2- 10.20.193.28:0/2773554657 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff04062270 0x7eff0415ceb0 secure :-1 s=READY pgs=189 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7efeec005b30 tx=0x7efeec0028d0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.951+0000 7eff0accc640 1 -- 10.20.193.28:0/2773554657 shutdown_connections 2026-02-25T11:58:57.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.951+0000 7eff0accc640 1 --2- 10.20.193.28:0/2773554657 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7efee00b56e0 0x7efee00d5b90 unknown :-1 s=CLOSED pgs=404 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.951+0000 7eff0accc640 1 --2- 10.20.193.28:0/2773554657 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7eff04150f80 0x7eff0415da60 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.951+0000 7eff0accc640 1 --2- 10.20.193.28:0/2773554657 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7eff04062270 0x7eff0415ceb0 unknown :-1 s=CLOSED pgs=189 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.951+0000 7eff0accc640 1 --2- 10.20.193.28:0/2773554657 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7eff04060e40 0x7eff0415c860 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:57.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.951+0000 7eff0accc640 1 -- 10.20.193.28:0/2773554657 >> 10.20.193.28:0/2773554657 conn(0x7eff040ea5f0 msgr2=0x7eff0410a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:57.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.951+0000 7eff0accc640 1 -- 10.20.193.28:0/2773554657 shutdown_connections 2026-02-25T11:58:57.953 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:57.951+0000 7eff0accc640 1 -- 10.20.193.28:0/2773554657 wait complete. 2026-02-25T11:58:57.964 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2026-02-25T11:44:43.266336+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2026-02-25T11:47:11.236575+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2026-02-25T11:47:12.196775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'nonprimary_shards': '{}', 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2026-02-25T11:58:57.965 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2026-02-25T11:58:58.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.085+0000 7f923bfff640 1 -- 10.20.193.28:0/2393830404 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f923c0600f0 msgr2=0x7f923c12db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.085+0000 7f923bfff640 1 --2- 10.20.193.28:0/2393830404 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f923c0600f0 0x7f923c12db80 secure :-1 s=READY pgs=928 gs=1 cs=0 l=1 c_cookie=14962378705009222320 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9230009950 tx=0x7f923002f190 comp rx=0 tx=0).stop 2026-02-25T11:58:58.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.085+0000 7f923bfff640 1 -- 10.20.193.28:0/2393830404 shutdown_connections 2026-02-25T11:58:58.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.085+0000 7f923bfff640 1 --2- 10.20.193.28:0/2393830404 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f923c14eb60 0x7f923c16f010 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.085+0000 7f923bfff640 1 --2- 10.20.193.28:0/2393830404 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f923c12e0c0 0x7f923c14e4a0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.085+0000 7f923bfff640 1 --2- 10.20.193.28:0/2393830404 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f923c0600f0 0x7f923c12db80 unknown :-1 s=CLOSED pgs=928 gs=1 cs=0 l=1 c_cookie=14962378705009222320 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.085+0000 7f923bfff640 1 -- 10.20.193.28:0/2393830404 >> 10.20.193.28:0/2393830404 conn(0x7f923c0e94a0 msgr2=0x7f923c1098b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:58.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.085+0000 7f923bfff640 1 -- 10.20.193.28:0/2393830404 shutdown_connections 2026-02-25T11:58:58.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.085+0000 7f923bfff640 1 -- 10.20.193.28:0/2393830404 wait complete. 2026-02-25T11:58:58.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.086+0000 7f923bfff640 1 Processor -- start 2026-02-25T11:58:58.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.086+0000 7f923bfff640 1 -- start start 2026-02-25T11:58:58.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.086+0000 7f923bfff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f923c0600f0 0x7f923c14e4a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.086+0000 7f923bfff640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f923c12e0c0 0x7f923c109880 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.086+0000 7f923bfff640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f923c14eb60 0x7f923c119e40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.086+0000 7f923bfff640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f923c11a5d0 con 0x7f923c0600f0 2026-02-25T11:58:58.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.086+0000 7f923bfff640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f923c11a740 con 0x7f923c12e0c0 2026-02-25T11:58:58.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.086+0000 7f923bfff640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f923c11a900 con 0x7f923c14eb60 2026-02-25T11:58:58.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f923affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f923c0600f0 0x7f923c14e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f923affd640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f923c0600f0 0x7f923c14e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34672/0 (socket says 10.20.193.28:34672) 2026-02-25T11:58:58.088 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f923affd640 1 -- 10.20.193.28:0/2397047127 learned_addr learned my addr 10.20.193.28:0/2397047127 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f923a7fc640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f923c12e0c0 0x7f923c109880 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f923b7fe640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f923c14eb60 0x7f923c119e40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f923affd640 1 -- 10.20.193.28:0/2397047127 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f923c14eb60 msgr2=0x7f923c119e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f923affd640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f923c14eb60 0x7f923c119e40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f923affd640 1 -- 10.20.193.28:0/2397047127 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f923c12e0c0 msgr2=0x7f923c109880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f923affd640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f923c12e0c0 0x7f923c109880 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f923affd640 1 -- 10.20.193.28:0/2397047127 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9230009570 con 0x7f923c0600f0 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f923affd640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f923c0600f0 0x7f923c14e4a0 secure :-1 s=READY pgs=929 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f92300023b0 tx=0x7f92300028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f92409fc640 1 -- 10.20.193.28:0/2397047127 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f923003d070 con 0x7f923c0600f0 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f92409fc640 1 -- 10.20.193.28:0/2397047127 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f923002fe20 con 0x7f923c0600f0 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f92409fc640 1 -- 10.20.193.28:0/2397047127 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9230041a60 con 0x7f923c0600f0 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f923bfff640 1 -- 10.20.193.28:0/2397047127 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f923c11aff0 con 0x7f923c0600f0 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.087+0000 7f923bfff640 1 -- 10.20.193.28:0/2397047127 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f923c11b510 con 0x7f923c0600f0 2026-02-25T11:58:58.089 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.088+0000 7f923bfff640 1 -- 10.20.193.28:0/2397047127 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f923c061810 con 0x7f923c0600f0 2026-02-25T11:58:58.092 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.090+0000 7f92409fc640 1 -- 10.20.193.28:0/2397047127 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f92300386a0 con 0x7f923c0600f0 2026-02-25T11:58:58.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.092+0000 7f92409fc640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f92100b5760 0x7f92100d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.092+0000 7f92409fc640 1 -- 10.20.193.28:0/2397047127 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f923010bef0 con 0x7f923c0600f0 2026-02-25T11:58:58.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.092+0000 7f92409fc640 1 -- 10.20.193.28:0/2397047127 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f923010c3e0 con 0x7f923c0600f0 2026-02-25T11:58:58.093 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.092+0000 7f923a7fc640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f92100b5760 0x7f92100d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.094 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.093+0000 7f923a7fc640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f92100b5760 0x7f92100d5c10 secure :-1 s=READY pgs=406 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9224005970 tx=0x7f9224005900 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:58.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.207+0000 7f923bfff640 1 -- 10.20.193.28:0/2397047127 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f923c172910 con 0x7f923c0600f0 2026-02-25T11:58:58.210 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.208+0000 7f92409fc640 1 -- 10.20.193.28:0/2397047127 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f92300ce900 con 0x7f923c0600f0 2026-02-25T11:58:58.210 INFO:teuthology.orchestra.run.trial028.stdout:pg_num: 1 2026-02-25T11:58:58.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.209+0000 7f923bfff640 1 -- 10.20.193.28:0/2397047127 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f92100b5760 msgr2=0x7f92100d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.209+0000 7f923bfff640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f92100b5760 0x7f92100d5c10 secure :-1 s=READY pgs=406 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9224005970 tx=0x7f9224005900 comp rx=0 tx=0).stop 2026-02-25T11:58:58.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.209+0000 7f923bfff640 1 -- 10.20.193.28:0/2397047127 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f923c0600f0 msgr2=0x7f923c14e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.209+0000 7f923bfff640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f923c0600f0 0x7f923c14e4a0 secure :-1 s=READY pgs=929 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f92300023b0 tx=0x7f92300028d0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.209+0000 7f923bfff640 1 -- 10.20.193.28:0/2397047127 shutdown_connections 2026-02-25T11:58:58.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.209+0000 7f923bfff640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f92100b5760 0x7f92100d5c10 unknown :-1 s=CLOSED pgs=406 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.209+0000 7f923bfff640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f923c14eb60 0x7f923c119e40 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.209+0000 7f923bfff640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f923c12e0c0 0x7f923c109880 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.209+0000 7f923bfff640 1 --2- 10.20.193.28:0/2397047127 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f923c0600f0 0x7f923c14e4a0 unknown :-1 s=CLOSED pgs=929 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.209+0000 7f923bfff640 1 -- 10.20.193.28:0/2397047127 >> 10.20.193.28:0/2397047127 conn(0x7f923c0e94a0 msgr2=0x7f923c110900 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:58.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.209+0000 7f923bfff640 1 -- 10.20.193.28:0/2397047127 shutdown_connections 2026-02-25T11:58:58.213 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.209+0000 7f923bfff640 1 -- 10.20.193.28:0/2397047127 wait complete. 2026-02-25T11:58:58.223 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2026-02-25T11:58:58.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.342+0000 7ffa69243640 1 -- 10.20.193.28:0/245204196 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa64110280 msgr2=0x7ffa641106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.346 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.342+0000 7ffa69243640 1 --2- 10.20.193.28:0/245204196 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa64110280 0x7ffa641106f0 secure :-1 s=READY pgs=930 gs=1 cs=0 l=1 c_cookie=8054098756733371964 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa54009950 tx=0x7ffa5402f190 comp rx=0 tx=0).stop 2026-02-25T11:58:58.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.342+0000 7ffa69243640 1 -- 10.20.193.28:0/245204196 shutdown_connections 2026-02-25T11:58:58.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.342+0000 7ffa69243640 1 --2- 10.20.193.28:0/245204196 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffa64110c30 0x7ffa641715c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.342+0000 7ffa69243640 1 --2- 10.20.193.28:0/245204196 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa64110280 0x7ffa641106f0 unknown :-1 s=CLOSED pgs=930 gs=1 cs=0 l=1 c_cookie=8054098756733371964 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.342+0000 7ffa69243640 1 --2- 10.20.193.28:0/245204196 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffa6410f0f0 0x7ffa6410f4e0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.342+0000 7ffa69243640 1 -- 10.20.193.28:0/245204196 >> 10.20.193.28:0/245204196 conn(0x7ffa640ec860 msgr2=0x7ffa6410cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:58.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.342+0000 7ffa69243640 1 -- 10.20.193.28:0/245204196 shutdown_connections 2026-02-25T11:58:58.347 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.342+0000 7ffa69243640 1 -- 10.20.193.28:0/245204196 wait complete. 2026-02-25T11:58:58.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa69243640 1 Processor -- start 2026-02-25T11:58:58.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa69243640 1 -- start start 2026-02-25T11:58:58.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa69243640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffa6410f0f0 0x7ffa640f8850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa69243640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa64110280 0x7ffa641715c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa69243640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffa64110c30 0x7ffa640f90b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa69243640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffa640f9ca0 con 0x7ffa64110280 2026-02-25T11:58:58.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa69243640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffa640f9e10 con 0x7ffa6410f0f0 2026-02-25T11:58:58.348 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa69243640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffa640f9fd0 con 0x7ffa64110c30 2026-02-25T11:58:58.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa637fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa64110280 0x7ffa641715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa637fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa64110280 0x7ffa641715c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34692/0 (socket says 10.20.193.28:34692) 2026-02-25T11:58:58.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa637fe640 1 -- 10.20.193.28:0/3697335868 learned_addr learned my addr 10.20.193.28:0/3697335868 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:58.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa63fff640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffa6410f0f0 0x7ffa640f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa68a42640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffa64110c30 0x7ffa640f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa637fe640 1 -- 10.20.193.28:0/3697335868 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffa64110c30 msgr2=0x7ffa640f90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa637fe640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffa64110c30 0x7ffa640f90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.349 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa637fe640 1 -- 10.20.193.28:0/3697335868 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffa6410f0f0 msgr2=0x7ffa640f8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa637fe640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffa6410f0f0 0x7ffa640f8850 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.343+0000 7ffa637fe640 1 -- 10.20.193.28:0/3697335868 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa54009570 con 0x7ffa64110280 2026-02-25T11:58:58.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.344+0000 7ffa637fe640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa64110280 0x7ffa641715c0 secure :-1 s=READY pgs=931 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa54005b30 tx=0x7ffa540028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:58.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.344+0000 7ffa617fa640 1 -- 10.20.193.28:0/3697335868 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffa5403d070 con 0x7ffa64110280 2026-02-25T11:58:58.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.344+0000 7ffa69243640 1 -- 10.20.193.28:0/3697335868 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa640fa250 con 0x7ffa64110280 2026-02-25T11:58:58.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.344+0000 7ffa617fa640 1 -- 10.20.193.28:0/3697335868 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7ffa54005dd0 con 0x7ffa64110280 2026-02-25T11:58:58.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.344+0000 7ffa617fa640 1 -- 10.20.193.28:0/3697335868 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7ffa54041960 con 0x7ffa64110280 2026-02-25T11:58:58.350 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.344+0000 7ffa69243640 1 -- 10.20.193.28:0/3697335868 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa640fa710 con 0x7ffa64110280 2026-02-25T11:58:58.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.345+0000 7ffa69243640 1 -- 10.20.193.28:0/3697335868 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa641754e0 con 0x7ffa64110280 2026-02-25T11:58:58.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.347+0000 7ffa617fa640 1 -- 10.20.193.28:0/3697335868 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7ffa54041af0 con 0x7ffa64110280 2026-02-25T11:58:58.352 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.348+0000 7ffa617fa640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffa380b57e0 0x7ffa380d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.348+0000 7ffa617fa640 1 -- 10.20.193.28:0/3697335868 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7ffa5410b3f0 con 0x7ffa64110280 2026-02-25T11:58:58.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.348+0000 7ffa617fa640 1 -- 10.20.193.28:0/3697335868 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7ffa5410b7f0 con 0x7ffa64110280 2026-02-25T11:58:58.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.349+0000 7ffa63fff640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffa380b57e0 0x7ffa380d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.353 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.349+0000 7ffa63fff640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffa380b57e0 0x7ffa380d5c90 secure :-1 s=READY pgs=407 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa50004740 tx=0x7ffa50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:58.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.463+0000 7ffa69243640 1 -- 10.20.193.28:0/3697335868 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ffa64174ba0 con 0x7ffa64110280 2026-02-25T11:58:58.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.463+0000 7ffa617fa640 1 -- 10.20.193.28:0/3697335868 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7ffa540cddc0 con 0x7ffa64110280 2026-02-25T11:58:58.466 INFO:teuthology.orchestra.run.trial028.stdout:pg_num: 32 2026-02-25T11:58:58.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.464+0000 7ffa69243640 1 -- 10.20.193.28:0/3697335868 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffa380b57e0 msgr2=0x7ffa380d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.464+0000 7ffa69243640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffa380b57e0 0x7ffa380d5c90 secure :-1 s=READY pgs=407 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa50004740 tx=0x7ffa50005f90 comp rx=0 tx=0).stop 2026-02-25T11:58:58.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.464+0000 7ffa69243640 1 -- 10.20.193.28:0/3697335868 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa64110280 msgr2=0x7ffa641715c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.464+0000 7ffa69243640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa64110280 0x7ffa641715c0 secure :-1 s=READY pgs=931 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7ffa54005b30 tx=0x7ffa540028d0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.464+0000 7ffa69243640 1 -- 10.20.193.28:0/3697335868 shutdown_connections 2026-02-25T11:58:58.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.464+0000 7ffa69243640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7ffa380b57e0 0x7ffa380d5c90 unknown :-1 s=CLOSED pgs=407 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.467 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.464+0000 7ffa69243640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7ffa64110c30 0x7ffa640f90b0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.464+0000 7ffa69243640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7ffa64110280 0x7ffa641715c0 unknown :-1 s=CLOSED pgs=931 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.464+0000 7ffa69243640 1 --2- 10.20.193.28:0/3697335868 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7ffa6410f0f0 0x7ffa640f8850 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.464+0000 7ffa69243640 1 -- 10.20.193.28:0/3697335868 >> 10.20.193.28:0/3697335868 conn(0x7ffa640ec860 msgr2=0x7ffa640eeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:58.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.464+0000 7ffa69243640 1 -- 10.20.193.28:0/3697335868 shutdown_connections 2026-02-25T11:58:58.468 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.464+0000 7ffa69243640 1 -- 10.20.193.28:0/3697335868 wait complete. 2026-02-25T11:58:58.476 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:58 trial028 ceph-mon[39224]: pgmap v548: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.2 KiB/s rd, 6 op/s 2026-02-25T11:58:58.476 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:58 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2773554657' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:58:58.476 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:58 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2397047127' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2026-02-25T11:58:58.478 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2026-02-25T11:58:58.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.597+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/4025630329 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b9810f100 msgr2=0x7f0b9810f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.597+0000 7f0b9ca5a640 1 --2- 10.20.193.28:0/4025630329 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b9810f100 0x7f0b9810f4f0 secure :-1 s=READY pgs=932 gs=1 cs=0 l=1 c_cookie=5198213128907123253 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0b8c009950 tx=0x7f0b8c02f190 comp rx=0 tx=0).stop 2026-02-25T11:58:58.599 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.597+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/4025630329 shutdown_connections 2026-02-25T11:58:58.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.597+0000 7f0b9ca5a640 1 --2- 10.20.193.28:0/4025630329 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0b98110c40 0x7f0b981715d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.597+0000 7f0b9ca5a640 1 --2- 10.20.193.28:0/4025630329 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0b98110290 0x7f0b98110700 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.597+0000 7f0b9ca5a640 1 --2- 10.20.193.28:0/4025630329 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b9810f100 0x7f0b9810f4f0 unknown :-1 s=CLOSED pgs=932 gs=1 cs=0 l=1 c_cookie=5198213128907123253 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.597+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/4025630329 >> 10.20.193.28:0/4025630329 conn(0x7f0b980ec830 msgr2=0x7f0b9810cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:58.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.597+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/4025630329 shutdown_connections 2026-02-25T11:58:58.600 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.597+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/4025630329 wait complete. 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.598+0000 7f0b9ca5a640 1 Processor -- start 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.598+0000 7f0b9ca5a640 1 -- start start 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.598+0000 7f0b9ca5a640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0b9810f100 0x7f0b980f8850 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.598+0000 7f0b9ca5a640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0b98110290 0x7f0b981715d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.598+0000 7f0b9ca5a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b98110c40 0x7f0b980f90b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.598+0000 7f0b9ca5a640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0b980f9ca0 con 0x7f0b98110c40 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.598+0000 7f0b9ca5a640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0b980f9e10 con 0x7f0b98110290 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.598+0000 7f0b9ca5a640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0b980f9fd0 con 0x7f0b9810f100 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b97577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b98110c40 0x7f0b980f90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b97577640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b98110c40 0x7f0b980f90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34696/0 (socket says 10.20.193.28:34696) 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b97577640 1 -- 10.20.193.28:0/1106655556 learned_addr learned my addr 10.20.193.28:0/1106655556 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b96d76640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0b9810f100 0x7f0b980f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b96575640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0b98110290 0x7f0b981715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b97577640 1 -- 10.20.193.28:0/1106655556 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0b9810f100 msgr2=0x7f0b980f8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.602 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b97577640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0b9810f100 0x7f0b980f8850 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b97577640 1 -- 10.20.193.28:0/1106655556 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0b98110290 msgr2=0x7f0b981715d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b97577640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0b98110290 0x7f0b981715d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b97577640 1 -- 10.20.193.28:0/1106655556 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8c009570 con 0x7f0b98110c40 2026-02-25T11:58:58.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b97577640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b98110c40 0x7f0b980f90b0 secure :-1 s=READY pgs=933 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0b88006e90 tx=0x7f0b8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:58.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b77fff640 1 -- 10.20.193.28:0/1106655556 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0b88005500 con 0x7f0b98110c40 2026-02-25T11:58:58.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b77fff640 1 -- 10.20.193.28:0/1106655556 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0b88005df0 con 0x7f0b98110c40 2026-02-25T11:58:58.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b77fff640 1 -- 10.20.193.28:0/1106655556 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0b88003a00 con 0x7f0b98110c40 2026-02-25T11:58:58.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/1106655556 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b980fa2b0 con 0x7f0b98110c40 2026-02-25T11:58:58.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.599+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/1106655556 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b980fa730 con 0x7f0b98110c40 2026-02-25T11:58:58.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.600+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/1106655556 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b98175490 con 0x7f0b98110c40 2026-02-25T11:58:58.603 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.602+0000 7f0b77fff640 1 -- 10.20.193.28:0/1106655556 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f0b88003b60 con 0x7f0b98110c40 2026-02-25T11:58:58.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.604+0000 7f0b77fff640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0b6c0b5760 0x7f0b6c0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.604+0000 7f0b77fff640 1 -- 10.20.193.28:0/1106655556 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f0b880e7bf0 con 0x7f0b98110c40 2026-02-25T11:58:58.605 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.604+0000 7f0b77fff640 1 -- 10.20.193.28:0/1106655556 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f0b88120940 con 0x7f0b98110c40 2026-02-25T11:58:58.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.604+0000 7f0b96d76640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0b6c0b5760 0x7f0b6c0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.606 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.604+0000 7f0b96d76640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0b6c0b5760 0x7f0b6c0d5c10 secure :-1 s=READY pgs=408 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0b8c0023d0 tx=0x7f0b8c031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:58.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.720+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/1106655556 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0b9805ed00 con 0x7f0b98110c40 2026-02-25T11:58:58.722 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.720+0000 7f0b77fff640 1 -- 10.20.193.28:0/1106655556 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f0b880aa5c0 con 0x7f0b98110c40 2026-02-25T11:58:58.722 INFO:teuthology.orchestra.run.trial028.stdout:pg_num: 64 2026-02-25T11:58:58.723 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.721+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/1106655556 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0b6c0b5760 msgr2=0x7f0b6c0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.721+0000 7f0b9ca5a640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0b6c0b5760 0x7f0b6c0d5c10 secure :-1 s=READY pgs=408 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0b8c0023d0 tx=0x7f0b8c031040 comp rx=0 tx=0).stop 2026-02-25T11:58:58.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.721+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/1106655556 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b98110c40 msgr2=0x7f0b980f90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.721+0000 7f0b9ca5a640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b98110c40 0x7f0b980f90b0 secure :-1 s=READY pgs=933 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0b88006e90 tx=0x7f0b8800edf0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.721+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/1106655556 shutdown_connections 2026-02-25T11:58:58.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.721+0000 7f0b9ca5a640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0b6c0b5760 0x7f0b6c0d5c10 unknown :-1 s=CLOSED pgs=408 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.721+0000 7f0b9ca5a640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0b98110c40 0x7f0b980f90b0 unknown :-1 s=CLOSED pgs=933 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.721+0000 7f0b9ca5a640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0b98110290 0x7f0b981715d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.721+0000 7f0b9ca5a640 1 --2- 10.20.193.28:0/1106655556 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0b9810f100 0x7f0b980f8850 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.721+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/1106655556 >> 10.20.193.28:0/1106655556 conn(0x7f0b980ec830 msgr2=0x7f0b980eee60 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:58.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.721+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/1106655556 shutdown_connections 2026-02-25T11:58:58.724 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.721+0000 7f0b9ca5a640 1 -- 10.20.193.28:0/1106655556 wait complete. 2026-02-25T11:58:58.735 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2026-02-25T11:58:58.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:58 trial167 ceph-mon[43392]: pgmap v548: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.2 KiB/s rd, 6 op/s 2026-02-25T11:58:58.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:58 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2773554657' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:58:58.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:58 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2397047127' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2026-02-25T11:58:58.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:58 trial119 ceph-mon[42710]: pgmap v548: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.2 KiB/s rd, 6 op/s 2026-02-25T11:58:58.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:58 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2773554657' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:58:58.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:58 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2397047127' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2026-02-25T11:58:58.857 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.853+0000 7fcc2771c640 1 -- 10.20.193.28:0/3155671415 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcc2010dbd0 msgr2=0x7fcc2010b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.853+0000 7fcc2771c640 1 --2- 10.20.193.28:0/3155671415 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcc2010dbd0 0x7fcc2010b000 secure :-1 s=READY pgs=934 gs=3 cs=0 l=1 c_cookie=13248872502602263263 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcc0c009950 tx=0x7fcc0c02f190 comp rx=0 tx=0).stop 2026-02-25T11:58:58.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.853+0000 7fcc2771c640 1 -- 10.20.193.28:0/3155671415 shutdown_connections 2026-02-25T11:58:58.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.853+0000 7fcc2771c640 1 --2- 10.20.193.28:0/3155671415 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcc2010b540 0x7fcc2010b9c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.853+0000 7fcc2771c640 1 --2- 10.20.193.28:0/3155671415 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcc2010dbd0 0x7fcc2010b000 unknown :-1 s=CLOSED pgs=934 gs=3 cs=0 l=1 c_cookie=13248872502602263263 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.853+0000 7fcc2771c640 1 --2- 10.20.193.28:0/3155671415 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcc2010d250 0x7fcc2010d640 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.853+0000 7fcc2771c640 1 -- 10.20.193.28:0/3155671415 >> 10.20.193.28:0/3155671415 conn(0x7fcc200e8c00 msgr2=0x7fcc20109010 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:58.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.853+0000 7fcc2771c640 1 -- 10.20.193.28:0/3155671415 shutdown_connections 2026-02-25T11:58:58.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.853+0000 7fcc2771c640 1 -- 10.20.193.28:0/3155671415 wait complete. 2026-02-25T11:58:58.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.854+0000 7fcc2771c640 1 Processor -- start 2026-02-25T11:58:58.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.854+0000 7fcc2771c640 1 -- start start 2026-02-25T11:58:58.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.854+0000 7fcc2771c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcc2010b540 0x7fcc200f4650 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.854+0000 7fcc2771c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcc2010d250 0x7fcc200f4ca0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.854+0000 7fcc2771c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcc2010dbd0 0x7fcc200f58a0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.854+0000 7fcc2771c640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcc200f5fe0 con 0x7fcc2010b540 2026-02-25T11:58:58.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.854+0000 7fcc2771c640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcc200f6150 con 0x7fcc2010d250 2026-02-25T11:58:58.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.854+0000 7fcc2771c640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcc200f6310 con 0x7fcc2010dbd0 2026-02-25T11:58:58.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.854+0000 7fcc2671a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcc2010b540 0x7fcc200f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc2671a640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcc2010b540 0x7fcc200f4650 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34712/0 (socket says 10.20.193.28:34712) 2026-02-25T11:58:58.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc2671a640 1 -- 10.20.193.28:0/4211371917 learned_addr learned my addr 10.20.193.28:0/4211371917 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:58.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc26f1b640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcc2010dbd0 0x7fcc200f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc25f19640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcc2010d250 0x7fcc200f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc2671a640 1 -- 10.20.193.28:0/4211371917 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcc2010dbd0 msgr2=0x7fcc200f58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc2671a640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcc2010dbd0 0x7fcc200f58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc2671a640 1 -- 10.20.193.28:0/4211371917 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcc2010d250 msgr2=0x7fcc200f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:58.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc2671a640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcc2010d250 0x7fcc200f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:58.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc2671a640 1 -- 10.20.193.28:0/4211371917 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c009570 con 0x7fcc2010b540 2026-02-25T11:58:58.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc2671a640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcc2010b540 0x7fcc200f4650 secure :-1 s=READY pgs=935 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcc18009480 tx=0x7fcc1800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:58.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc0b7fe640 1 -- 10.20.193.28:0/4211371917 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcc1800cd40 con 0x7fcc2010b540 2026-02-25T11:58:58.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc0b7fe640 1 -- 10.20.193.28:0/4211371917 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fcc18004d10 con 0x7fcc2010b540 2026-02-25T11:58:58.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc0b7fe640 1 -- 10.20.193.28:0/4211371917 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fcc180106b0 con 0x7fcc2010b540 2026-02-25T11:58:58.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc2771c640 1 -- 10.20.193.28:0/4211371917 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc200f65f0 con 0x7fcc2010b540 2026-02-25T11:58:58.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.855+0000 7fcc2771c640 1 -- 10.20.193.28:0/4211371917 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc200f6b10 con 0x7fcc2010b540 2026-02-25T11:58:58.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.856+0000 7fcc2771c640 1 -- 10.20.193.28:0/4211371917 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc20060810 con 0x7fcc2010b540 2026-02-25T11:58:58.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.858+0000 7fcc0b7fe640 1 -- 10.20.193.28:0/4211371917 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fcc1801f060 con 0x7fcc2010b540 2026-02-25T11:58:58.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.860+0000 7fcc0b7fe640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcbfc0b5760 0x7fcbfc0d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:58.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.860+0000 7fcc0b7fe640 1 -- 10.20.193.28:0/4211371917 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fcc18014070 con 0x7fcc2010b540 2026-02-25T11:58:58.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.860+0000 7fcc0b7fe640 1 -- 10.20.193.28:0/4211371917 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fcc1811f960 con 0x7fcc2010b540 2026-02-25T11:58:58.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.860+0000 7fcc25f19640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcbfc0b5760 0x7fcbfc0d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:58.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:58.860+0000 7fcc25f19640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcbfc0b5760 0x7fcbfc0d5c10 secure :-1 s=READY pgs=409 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcc0c0023d0 tx=0x7fcc0c031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:59.006 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.005+0000 7fcc2771c640 1 -- 10.20.193.28:0/4211371917 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fcc20175510 con 0x7fcc2010b540 2026-02-25T11:58:59.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.012+0000 7fcc0b7fe640 1 -- 10.20.193.28:0/4211371917 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v17) ==== 81+0+827130 (secure 0 0 0) 0x7fcc180a9750 con 0x7fcc2010b540 2026-02-25T11:58:59.020 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:58:59.020 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:58:59.020 INFO:teuthology.orchestra.run.trial028.stdout: "epoch": 17, 2026-02-25T11:58:59.020 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: "active_gid": 14152, 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: "active_name": "x", 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: "active_addrs": { 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: "addrvec": [ 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: "type": "v2", 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: "addr": "10.20.193.28:6800", 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: "nonce": 3140694734 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: "type": "v1", 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: "addr": "10.20.193.28:6801", 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: "nonce": 3140694734 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: ] 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: "active_addr": "10.20.193.28:6801/3140694734", 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: "active_change": "2026-02-25T11:43:17.037590+0000", 2026-02-25T11:58:59.021 INFO:teuthology.orchestra.run.trial028.stdout: "active_mgr_features": 4544132024016699391, 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "available": true, 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "standbys": [ 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "gid": 24152, 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "name": "z", 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "mgr_features": 4544132024016699391, 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "available_modules": [ 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "name": "alerts", 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "interval": { 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "name": "interval", 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.022 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How frequently to reexamine health status", 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.023 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.024 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.025 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.026 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.026 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.026 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.026 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.026 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.026 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.026 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.026 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.026 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_destination": { 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_destination", 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.027 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_from_name": { 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_from_name", 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "Ceph", 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Email From: name", 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_host": { 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_host", 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.028 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "SMTP server", 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_password": { 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_password", 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Password to authenticate with", 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.029 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_port": { 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_port", 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "465", 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "SMTP port", 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_sender": { 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_sender", 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.030 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "SMTP envelope sender", 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_ssl": { 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_ssl", 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.031 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Use SSL to connect to SMTP server", 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_user": { 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_user", 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "User to authenticate as", 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.032 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "name": "balancer", 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "active": { 2026-02-25T11:58:59.033 INFO:teuthology.orchestra.run.trial028.stdout: "name": "active", 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "automatically balance PGs across cluster", 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "begin_time": { 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "name": "begin_time", 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0000", 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.034 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "beginning time of day to automatically balance", 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "This is a time of day in the format HHMM.", 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "begin_weekday": { 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "name": "begin_weekday", 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "type": "uint", 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "min": "0", 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "max": "6", 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.035 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "crush_compat_max_iterations": { 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crush_compat_max_iterations", 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "type": "uint", 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "25", 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1", 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "max": "250", 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "maximum number of iterations to attempt optimization", 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "crush_compat_metrics": { 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crush_compat_metrics", 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "pgs,objects,bytes", 2026-02-25T11:58:59.036 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "metrics with which to calculate OSD utilization", 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "crush_compat_step": { 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crush_compat_step", 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0.5", 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "min": "0.001", 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "max": "0.999", 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "aggressiveness of optimization", 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.037 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "end_time": { 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "name": "end_time", 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "2359", 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "ending time of day to automatically balance", 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "This is a time of day in the format HHMM.", 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "end_weekday": { 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "name": "end_weekday", 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "type": "uint", 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.038 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "min": "0", 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "max": "6", 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.039 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.040 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.041 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.042 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "min_score": { 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "name": "min_score", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "minimum score, below which no optimization is attempted", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "mode": { 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mode", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "upmap", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.043 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "crush-compat", 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "none", 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "read", 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "upmap", 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "upmap-read" 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Balancer mode", 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "pool_ids": { 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pool_ids", 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "pools which the automatic balancing will be limited to", 2026-02-25T11:58:59.044 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how frequently to wake up and attempt optimization", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.045 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "update_pg_upmap_activity": { 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "name": "update_pg_upmap_activity", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Updates pg_upmap activity stats to be used in `balancer status detail`", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "upmap_max_deviation": { 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "name": "upmap_max_deviation", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.046 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1", 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "deviation below which no optimization is attempted", 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "upmap_max_optimizations": { 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "name": "upmap_max_optimizations", 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "type": "uint", 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "maximum upmap optimizations to make per attempt", 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.047 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cephadm", 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "agent_down_multiplier": { 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "name": "agent_down_multiplier", 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "3.0", 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.048 INFO:teuthology.orchestra.run.trial028.stdout: "agent_refresh_rate": { 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "name": "agent_refresh_rate", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "20", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How often the agent on each host will gather and send metadata", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "agent_starting_port": { 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "name": "agent_starting_port", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "4721", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound", 2026-02-25T11:58:59.049 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "allow_ptrace": { 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "name": "allow_ptrace", 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.", 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "autotune_interval": { 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "name": "autotune_interval", 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.050 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How frequently to autotune daemon memory", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "autotune_memory_target_ratio": { 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "name": "autotune_memory_target_ratio", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0.7", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "cephadm_log_destination": { 2026-02-25T11:58:59.051 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cephadm_log_destination", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "file", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "file,syslog", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "syslog" 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Destination for cephadm command persistent logging", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_automated_rotation_enabled": { 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_automated_rotation_enabled", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.052 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "This flag controls whether cephadm automatically rotates certificates upon expiration.", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_check_debug_mode": { 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_check_debug_mode", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_check_period": { 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_check_period", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "min": "0", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "max": "30", 2026-02-25T11:58:59.053 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Specifies how often (in days) the certificate should be checked for validity.", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_duration_days": { 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_duration_days", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1095", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "min": "90", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "max": "3650", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Specifies the duration of self-signed certificates generated and signed by the cephadm root CA", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_renewal_threshold_days": { 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_renewal_threshold_days", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "30", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "min": "10", 2026-02-25T11:58:59.054 INFO:teuthology.orchestra.run.trial028.stdout: "max": "90", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Specifies the lead time in days to initiate certificate renewal before expiration.", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "cgroups_split": { 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cgroups_split", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently Podman only)", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "config_checks_enabled": { 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "name": "config_checks_enabled", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.055 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Enable or disable cephadm configuration analysis", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "config_dashboard": { 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "name": "config_dashboard", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Manage configs like API endpoints in Dashboard.", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_alertmanager": { 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_alertmanager", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.28.1", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.056 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Alertmanager container image", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_alloy": { 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_alloy", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "docker.io/grafana/alloy:latest", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Alloy container image", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_base": { 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_base", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/ceph", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Container image name, without the tag", 2026-02-25T11:58:59.057 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_elasticsearch": { 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_elasticsearch", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Elasticsearch container image", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_grafana": { 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_grafana", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/grafana:12.3.1", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Grafana container image", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.058 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_haproxy": { 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_haproxy", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Haproxy container image", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_jaeger_agent": { 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_jaeger_agent", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Jaeger agent container image", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_jaeger_collector": { 2026-02-25T11:58:59.059 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_jaeger_collector", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Jaeger collector container image", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_jaeger_query": { 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_jaeger_query", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Jaeger query container image", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.060 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_keepalived": { 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_keepalived", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Keepalived container image", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_loki": { 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_loki", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Loki container image", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.061 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_nginx": { 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_nginx", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Nginx container image", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_node_exporter": { 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_node_exporter", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.9.1", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Node exporter container image", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_nvmeof": { 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_nvmeof", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.062 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/nvmeof:1.7", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Nvmeof container image", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_oauth2_proxy": { 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_oauth2_proxy", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Oauth2 proxy container image", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_prometheus": { 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_prometheus", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/prometheus/prometheus:v3.6.0", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Prometheus container image", 2026-02-25T11:58:59.063 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_promtail": { 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_promtail", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Promtail container image", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_samba": { 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_samba", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Samba container image", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_samba_metrics": { 2026-02-25T11:58:59.064 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_samba_metrics", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/samba.org/samba-metrics:devbuilds-centos-amd64", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Samba metrics container image", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_snmp_gateway": { 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_snmp_gateway", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Snmp gateway container image", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "container_init": { 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_init", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.065 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Run podman/docker with `--init`", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "daemon_cache_timeout": { 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "name": "daemon_cache_timeout", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Seconds for which to cache service (daemon) inventory", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "default_cephadm_command_timeout": { 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "name": "default_cephadm_command_timeout", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "900", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.066 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "default_registry": { 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "name": "default_registry", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "device_cache_timeout": { 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "name": "device_cache_timeout", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1800", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Seconds for which to cache device inventory", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "device_enhanced_scan": { 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "name": "device_enhanced_scan", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.067 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Use libstoragemgmt during device scans", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "facts_cache_timeout": { 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "name": "facts_cache_timeout", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Seconds for which to cache host facts data", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "grafana_dashboards_path": { 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "name": "grafana_dashboards_path", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Location of dashboards to include in Grafana", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "host_check_interval": { 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "name": "host_check_interval", 2026-02-25T11:58:59.068 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How frequently to perform a host check", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "hw_monitoring": { 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "name": "hw_monitoring", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Whether the hardware monitoring daemon be deployed on every host?", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "inventory_list_all": { 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "name": "inventory_list_all", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.069 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "log_refresh_metadata": { 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_refresh_metadata", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.070 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Log to the \"cephadm\" cluster log channel\"", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.071 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "manage_etc_ceph_ceph_conf": { 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "name": "manage_etc_ceph_ceph_conf", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on hosts.", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "*", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2026-02-25T11:58:59.072 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "max_count_per_host": { 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_count_per_host", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Max number of daemons per service per host: guard against mistakes and runaways", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "max_osd_draining_count": { 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_osd_draining_count", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Max number of OSDs that will be drained simultaneously when OSDs are removed", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "max_parallel_osd_upgrades": { 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_parallel_osd_upgrades", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "16", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.073 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Maximum number of OSD daemons upgraded in parallel.", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "migration_current": { 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "name": "migration_current", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Internal - do not modify", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "mode": { 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mode", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "root", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "cephadm-package", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "root" 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Mode for remote execution of cephadm", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "oob_default_addr": { 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "name": "oob_default_addr", 2026-02-25T11:58:59.074 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "169.254.1.1", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Default IP address for RedFish API (OOB management).", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "prometheus_alerts_path": { 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "name": "prometheus_alerts_path", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Location of alerts to include in Prometheus Alertmanager", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "registry_insecure": { 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "name": "registry_insecure", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.075 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "registry_password": { 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "name": "registry_password", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "registry_url": { 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "name": "registry_url", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Registry URL for login purposes. This is not the default registry.", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "registry_username": { 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "name": "registry_username", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.076 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "secure_monitoring_stack": { 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "name": "secure_monitoring_stack", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Enable TLS security for all monitoring stack daemons", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "service_discovery_port": { 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "name": "service_discovery_port", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8765", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Cephadm service discovery port", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.077 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "ssh_config_file": { 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssh_config_file", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Customized SSH config file to distribute to managed hosts", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "ssh_keepalive_count_max": { 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssh_keepalive_count_max", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "3", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How many sequential SSH connection liveness check failures before the host is marked offline", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "ssh_keepalive_interval": { 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssh_keepalive_interval", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "7", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How often SSH connections are checked for liveness", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.078 INFO:teuthology.orchestra.run.trial028.stdout: "stray_daemon_check_interval": { 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "name": "stray_daemon_check_interval", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1800", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How frequently cephadm should check for the presence of stray daemons", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "use_agent": { 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "name": "use_agent", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Use the cephadm agent on each host to gather and send metadata", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "use_repo_digest": { 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "name": "use_repo_digest", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Automatically convert image tags to image digest to ensure that all daemons use the same image", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.079 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "warn_on_failed_host_check": { 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_on_failed_host_check", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Raise a health warning if the host check fails", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "warn_on_stray_daemons": { 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_on_stray_daemons", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Raise a health warning if daemons are detected that are not managed by cephadm", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "warn_on_stray_hosts": { 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_on_stray_hosts", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Raise a health warning if daemons are detected on a host that is not managed by cephadm", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.080 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crash", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.081 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "retain_interval": { 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "name": "retain_interval", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "31536000", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how long to retain crashes before pruning them", 2026-02-25T11:58:59.082 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "warn_recent_interval": { 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_recent_interval", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1209600", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "time interval in which to warn about recent crashes", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "name": "dashboard", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.083 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "ALERTMANAGER_API_HOST": { 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ALERTMANAGER_API_HOST", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "AUDIT_API_ENABLED": { 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "name": "AUDIT_API_ENABLED", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.084 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "AUDIT_API_LOG_PAYLOAD": { 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "ENABLE_BROWSABLE_API": { 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ENABLE_BROWSABLE_API", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_CEPHFS": { 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2026-02-25T11:58:59.085 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_ISCSI": { 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_MIRRORING": { 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_NFS": { 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_NFS", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.086 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_RBD": { 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_RBD", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_RGW": { 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_RGW", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.087 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_API_PASSWORD": { 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_API_PASSWORD", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "admin", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_API_SSL_VERIFY": { 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_API_URL": { 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_API_URL", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.088 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_API_USERNAME": { 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_API_USERNAME", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "admin", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_FRONTEND_API_URL": { 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "ISCSI_API_SSL_VERIFICATION": { 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.089 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "ISSUE_TRACKER_API_KEY": { 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ISSUE_TRACKER_API_KEY", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "MANAGED_BY_CLUSTERS": { 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "name": "MANAGED_BY_CLUSTERS", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "[]", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "MULTICLUSTER_CONFIG": { 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "name": "MULTICLUSTER_CONFIG", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "{}", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.090 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "PROMETHEUS_API_HOST": { 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PROMETHEUS_API_HOST", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "PROM_ALERT_CREDENTIAL_CACHE_TTL": { 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PROM_ALERT_CREDENTIAL_CACHE_TTL", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.091 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.092 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.093 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_ENABLED": { 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_ENABLED", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.094 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_MIN_LENGTH": { 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "REST_REQUESTS_TIMEOUT": { 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "name": "REST_REQUESTS_TIMEOUT", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "45", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_API_ACCESS_KEY": { 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_API_ACCESS_KEY", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.095 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_API_ADMIN_RESOURCE": { 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "admin", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_API_SECRET_KEY": { 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_API_SECRET_KEY", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_API_SSL_VERIFY": { 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_API_SSL_VERIFY", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_HOSTNAME_PER_DAEMON": { 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_HOSTNAME_PER_DAEMON", 2026-02-25T11:58:59.096 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "UNSAFE_TLS_v1_2": { 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "name": "UNSAFE_TLS_v1_2", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "USER_PWD_EXPIRATION_SPAN": { 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.097 INFO:teuthology.orchestra.run.trial028.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "cross_origin_url": { 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cross_origin_url", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "crt_file": { 2026-02-25T11:58:59.098 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crt_file", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "crypto_caller": { 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crypto_caller", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "debug": { 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "name": "debug", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Enable/disable debug options", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "jwt_token_ttl": { 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "name": "jwt_token_ttl", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.099 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "28800", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "key_file": { 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "name": "key_file", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.100 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.101 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "motd": { 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "name": "motd", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "The message of the day", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "redirect_resolve_ip_addr": { 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "name": "redirect_resolve_ip_addr", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "server_addr": { 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "name": "server_addr", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "::", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.102 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "server_port": { 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "name": "server_port", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8080", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "ssl": { 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssl", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.103 INFO:teuthology.orchestra.run.trial028.stdout: "ssl_server_port": { 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssl_server_port", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8443", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "sso_oauth2": { 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sso_oauth2", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "standby_behaviour": { 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "name": "standby_behaviour", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "redirect", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "redirect" 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.104 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "standby_error_status_code": { 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "name": "standby_error_status_code", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "500", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "min": "400", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "max": "599", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "url_prefix": { 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "name": "url_prefix", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "name": "devicehealth", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "enable_monitoring": { 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "name": "enable_monitoring", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.105 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "monitor device health metrics", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.106 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "mark_out_threshold": { 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mark_out_threshold", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "2419200", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "automatically mark OSD if it may fail before this long", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "pool_name": { 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pool_name", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.107 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "device_health_metrics", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "name of pool in which to store device health metrics", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "retention_period": { 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "name": "retention_period", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "15552000", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how long to retain device health metrics", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "scrape_frequency": { 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "name": "scrape_frequency", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "86400", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how frequently to scrape device health metrics", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "self_heal": { 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "name": "self_heal", 2026-02-25T11:58:59.108 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "preemptively heal cluster around devices that may fail", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how frequently to wake up and check device health", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "warn_threshold": { 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_threshold", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "7257600", 2026-02-25T11:58:59.109 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "raise health warning if OSD may fail before this long", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "name": "diskprediction_local", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.110 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "predict_interval": { 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "name": "predict_interval", 2026-02-25T11:58:59.111 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "86400", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "predictor_model": { 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "name": "predictor_model", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "prophetstor", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.112 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "name": "influx", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": false, 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "influxdb python module not found", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "batch_size": { 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "name": "batch_size", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5000", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "database": { 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "name": "database", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "ceph", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "hostname": { 2026-02-25T11:58:59.113 INFO:teuthology.orchestra.run.trial028.stdout: "name": "hostname", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "InfluxDB server hostname", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "interval": { 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "name": "interval", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "30", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "min": "5", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.114 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.115 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "password": { 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "name": "password", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "password of InfluxDB server user", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "port": { 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "name": "port", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8086", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "InfluxDB server port", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.116 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "ssl": { 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssl", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "false", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "threads": { 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "name": "threads", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "max": "32", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "username": { 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "name": "username", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "username of InfluxDB server user", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "verify_ssl": { 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "name": "verify_ssl", 2026-02-25T11:58:59.117 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "true", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "name": "insights", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.118 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.119 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "name": "iostat", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.120 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.121 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "name": "k8sevents", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_event_retention_days": { 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ceph_event_retention_days", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "7", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Days to hold ceph event information within local cache", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "config_check_secs": { 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "name": "config_check_secs", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.122 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "min": "10", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.123 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.124 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "name": "localpool", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "failure_domain": { 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "name": "failure_domain", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "host", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "failure domain for any created local pool", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.125 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.126 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "min_size": { 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "name": "min_size", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "default min_size for any created local pool", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "num_rep": { 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "name": "num_rep", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "3", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "default replica count for any created local pool", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "pg_num": { 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pg_num", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.127 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "128", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "default pg_num for any created local pool", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "prefix": { 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "name": "prefix", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "name prefix for any created local pool", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "subtree": { 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "name": "subtree", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "rack", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.128 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "CRUSH level for which to create a local pool", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mds_autoscaler", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.129 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.130 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mirroring", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.131 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.132 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "name": "nfs", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.133 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.134 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "name": "orchestrator", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "fail_fs": { 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "name": "fail_fs", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.135 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.136 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "orchestrator": { 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "name": "orchestrator", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "cephadm", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "rook", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "test_orchestrator" 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Orchestrator backend", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "name": "osd_perf_query", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.137 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.138 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "name": "osd_support", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.139 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.140 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pg_autoscaler", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.141 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.142 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.143 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "threshold": { 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "name": "threshold", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "3.0", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1.0", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "scaling threshold", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "name": "progress", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "allow_pg_recovery_event": { 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "name": "allow_pg_recovery_event", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "allow the module to show pg recovery progress", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "enabled": { 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "name": "enabled", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.144 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.145 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "max_completed_events": { 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_completed_events", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "50", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "number of past completed events to remember", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.146 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how long the module is going to sleep", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "name": "prometheus", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "cache": { 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cache", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.147 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "exclude_perf_counters": { 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "name": "exclude_perf_counters", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Do not include perf-counters in the metrics output", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "healthcheck_history_max_entries": { 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "name": "healthcheck_history_max_entries", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1000", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Maximum number of health check history entries to keep", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.148 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.149 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "rbd_stats_pools": { 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rbd_stats_pools", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "rbd_stats_pools_refresh_interval": { 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rbd_stats_pools_refresh_interval", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "300", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "scrape_interval": { 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "name": "scrape_interval", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "15.0", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.150 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "server_addr": { 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "name": "server_addr", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "::", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "server_port": { 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "name": "server_port", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "9283", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "the port on which the module listens for HTTP requests", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "stale_cache_strategy": { 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "name": "stale_cache_strategy", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.151 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "log", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "standby_behaviour": { 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "name": "standby_behaviour", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "default", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "default", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "error" 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "standby_error_status_code": { 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "name": "standby_error_status_code", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "500", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "min": "400", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "max": "599", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.152 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rbd_support", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.153 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "max_concurrent_snap_create": { 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_concurrent_snap_create", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "mirror_snapshot_schedule": { 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mirror_snapshot_schedule", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.154 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "trash_purge_schedule": { 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "name": "trash_purge_schedule", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rgw", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.155 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.156 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "secondary_zone_period_retry_limit": { 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "name": "secondary_zone_period_retry_limit", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "RGW module period update retry limit for secondary site", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.157 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rook", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.158 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "prometheus_tls_secret_name": { 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "name": "prometheus_tls_secret_name", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "rook-ceph-prometheus-server-tls", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.159 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "name of tls secret in k8s for prometheus", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "secure_monitoring_stack": { 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "name": "secure_monitoring_stack", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "storage_class": { 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "name": "storage_class", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "local", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "storage class name for LSO-discovered PVs", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.160 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "name": "selftest", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.161 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "roption1": { 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "name": "roption1", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.162 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "roption2": { 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "name": "roption2", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "xyz", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption1": { 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption1", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption2": { 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption2", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.163 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption3": { 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption3", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption4": { 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption4", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption5": { 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption5", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption6": { 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption6", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.164 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption7": { 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption7", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "max": "42", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "testkey": { 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "name": "testkey", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.165 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "testlkey": { 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "name": "testlkey", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "testnewline": { 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "name": "testnewline", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smb", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "internal_store_backend": { 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "name": "internal_store_backend", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "set internal store backend. for develoment and testing only", 2026-02-25T11:58:59.166 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.167 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "update_orchestration": { 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "name": "update_orchestration", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.168 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "automatically update orchestration when smb resources are changed", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "name": "snap_schedule", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "allow_m_granularity": { 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "name": "allow_m_granularity", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "allow minute scheduled snapshots", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "dump_on_update": { 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "name": "dump_on_update", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "dump database to debug log on update", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.169 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.170 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "name": "stats", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.171 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.172 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "name": "status", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.173 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.174 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "name": "telegraf", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "address": { 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "name": "address", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.175 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "interval": { 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "name": "interval", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "15", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.176 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.177 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "name": "telemetry", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "channel_basic": { 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_basic", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share basic cluster information (size, version)", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "channel_crash": { 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_crash", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "channel_device": { 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_device", 2026-02-25T11:58:59.178 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "channel_ident": { 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_ident", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "channel_perf": { 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_perf", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share various performance metrics of a cluster", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "contact": { 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "name": "contact", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.179 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "description": { 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "name": "description", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "device_url": { 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "name": "device_url", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "https://telemetry.ceph.com/device", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "enabled": { 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "name": "enabled", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.180 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "interval": { 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "name": "interval", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "24", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "min": "8", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "last_opt_revision": { 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "name": "last_opt_revision", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "leaderboard": { 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "name": "leaderboard", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.181 INFO:teuthology.orchestra.run.trial028.stdout: "leaderboard_description": { 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "name": "leaderboard_description", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.182 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "organization": { 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "name": "organization", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.183 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "proxy": { 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "name": "proxy", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "url": { 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "name": "url", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "https://telemetry.ceph.com/report", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.184 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "name": "test_orchestrator", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.185 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.186 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "name": "volumes", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.187 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "max_concurrent_clones": { 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_concurrent_clones", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "4", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Number of asynchronous cloner threads", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "pause_cloning": { 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pause_cloning", 2026-02-25T11:58:59.188 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Pause asynchronous cloner threads", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "pause_purging": { 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pause_purging", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Pause asynchronous subvolume purge threads", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "periodic_async_work": { 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "name": "periodic_async_work", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Periodically check for async work", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "snapshot_clone_delay": { 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "name": "snapshot_clone_delay", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.189 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "snapshot_clone_no_wait": { 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "name": "snapshot_clone_no_wait", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: ] 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "gid": 24233, 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "name": "y", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "mgr_features": 4544132024016699391, 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "available_modules": [ 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "name": "alerts", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.190 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "interval": { 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "name": "interval", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How frequently to reexamine health status", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.191 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_destination": { 2026-02-25T11:58:59.192 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_destination", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_from_name": { 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_from_name", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "Ceph", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Email From: name", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_host": { 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_host", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "SMTP server", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.193 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_password": { 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_password", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Password to authenticate with", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_port": { 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_port", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "465", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "SMTP port", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_sender": { 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_sender", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "SMTP envelope sender", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_ssl": { 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_ssl", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.194 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Use SSL to connect to SMTP server", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_user": { 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_user", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "User to authenticate as", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "name": "balancer", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "active": { 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "name": "active", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.195 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "automatically balance PGs across cluster", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "begin_time": { 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "name": "begin_time", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0000", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "beginning time of day to automatically balance", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "This is a time of day in the format HHMM.", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "begin_weekday": { 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "name": "begin_weekday", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "type": "uint", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "min": "0", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "max": "6", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "crush_compat_max_iterations": { 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crush_compat_max_iterations", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "type": "uint", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "25", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1", 2026-02-25T11:58:59.196 INFO:teuthology.orchestra.run.trial028.stdout: "max": "250", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "maximum number of iterations to attempt optimization", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "crush_compat_metrics": { 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crush_compat_metrics", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "pgs,objects,bytes", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "metrics with which to calculate OSD utilization", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "crush_compat_step": { 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crush_compat_step", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0.5", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "min": "0.001", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "max": "0.999", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "aggressiveness of optimization", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "end_time": { 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "name": "end_time", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "2359", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "ending time of day to automatically balance", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "This is a time of day in the format HHMM.", 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.197 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "end_weekday": { 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "name": "end_weekday", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "type": "uint", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "min": "0", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "max": "6", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.198 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "min_score": { 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "name": "min_score", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.199 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "minimum score, below which no optimization is attempted", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "mode": { 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mode", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "upmap", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "crush-compat", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "none", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "read", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "upmap", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "upmap-read" 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Balancer mode", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "pool_ids": { 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pool_ids", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "pools which the automatic balancing will be limited to", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.200 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how frequently to wake up and attempt optimization", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "update_pg_upmap_activity": { 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "name": "update_pg_upmap_activity", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Updates pg_upmap activity stats to be used in `balancer status detail`", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "upmap_max_deviation": { 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "name": "upmap_max_deviation", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.201 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "deviation below which no optimization is attempted", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "upmap_max_optimizations": { 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "name": "upmap_max_optimizations", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "type": "uint", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "maximum upmap optimizations to make per attempt", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cephadm", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "agent_down_multiplier": { 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "name": "agent_down_multiplier", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "3.0", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "agent_refresh_rate": { 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "name": "agent_refresh_rate", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.202 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "20", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How often the agent on each host will gather and send metadata", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "agent_starting_port": { 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "name": "agent_starting_port", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "4721", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "allow_ptrace": { 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "name": "allow_ptrace", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "autotune_interval": { 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "name": "autotune_interval", 2026-02-25T11:58:59.203 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How frequently to autotune daemon memory", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "autotune_memory_target_ratio": { 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "name": "autotune_memory_target_ratio", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0.7", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "cephadm_log_destination": { 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cephadm_log_destination", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "file", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "file,syslog", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "syslog" 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Destination for cephadm command persistent logging", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_automated_rotation_enabled": { 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_automated_rotation_enabled", 2026-02-25T11:58:59.204 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "This flag controls whether cephadm automatically rotates certificates upon expiration.", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_check_debug_mode": { 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_check_debug_mode", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_check_period": { 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_check_period", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "min": "0", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "max": "30", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Specifies how often (in days) the certificate should be checked for validity.", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_duration_days": { 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_duration_days", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1095", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "min": "90", 2026-02-25T11:58:59.205 INFO:teuthology.orchestra.run.trial028.stdout: "max": "3650", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Specifies the duration of self-signed certificates generated and signed by the cephadm root CA", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_renewal_threshold_days": { 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_renewal_threshold_days", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "30", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "min": "10", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "max": "90", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Specifies the lead time in days to initiate certificate renewal before expiration.", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "cgroups_split": { 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cgroups_split", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently Podman only)", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "config_checks_enabled": { 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "name": "config_checks_enabled", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Enable or disable cephadm configuration analysis", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.206 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "config_dashboard": { 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "name": "config_dashboard", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Manage configs like API endpoints in Dashboard.", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_alertmanager": { 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_alertmanager", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.28.1", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Alertmanager container image", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_alloy": { 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_alloy", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "docker.io/grafana/alloy:latest", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Alloy container image", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_base": { 2026-02-25T11:58:59.207 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_base", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/ceph", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Container image name, without the tag", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_elasticsearch": { 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_elasticsearch", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Elasticsearch container image", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_grafana": { 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_grafana", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/grafana:12.3.1", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Grafana container image", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_haproxy": { 2026-02-25T11:58:59.208 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_haproxy", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Haproxy container image", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_jaeger_agent": { 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_jaeger_agent", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Jaeger agent container image", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_jaeger_collector": { 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_jaeger_collector", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Jaeger collector container image", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_jaeger_query": { 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_jaeger_query", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.209 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Jaeger query container image", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_keepalived": { 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_keepalived", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Keepalived container image", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_loki": { 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_loki", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Loki container image", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_nginx": { 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_nginx", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Nginx container image", 2026-02-25T11:58:59.210 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_node_exporter": { 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_node_exporter", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.9.1", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Node exporter container image", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_nvmeof": { 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_nvmeof", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/nvmeof:1.7", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Nvmeof container image", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_oauth2_proxy": { 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_oauth2_proxy", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Oauth2 proxy container image", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.211 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_prometheus": { 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_prometheus", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/prometheus/prometheus:v3.6.0", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Prometheus container image", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_promtail": { 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_promtail", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Promtail container image", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_samba": { 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_samba", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Samba container image", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_samba_metrics": { 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_samba_metrics", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.212 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/samba.org/samba-metrics:devbuilds-centos-amd64", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Samba metrics container image", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_snmp_gateway": { 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_snmp_gateway", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Snmp gateway container image", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "container_init": { 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_init", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Run podman/docker with `--init`", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "daemon_cache_timeout": { 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "name": "daemon_cache_timeout", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.213 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Seconds for which to cache service (daemon) inventory", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "default_cephadm_command_timeout": { 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "name": "default_cephadm_command_timeout", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "900", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "default_registry": { 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "name": "default_registry", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "device_cache_timeout": { 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "name": "device_cache_timeout", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1800", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.214 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Seconds for which to cache device inventory", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "device_enhanced_scan": { 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "name": "device_enhanced_scan", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Use libstoragemgmt during device scans", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "facts_cache_timeout": { 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "name": "facts_cache_timeout", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Seconds for which to cache host facts data", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "grafana_dashboards_path": { 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "name": "grafana_dashboards_path", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Location of dashboards to include in Grafana", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.215 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "host_check_interval": { 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "name": "host_check_interval", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How frequently to perform a host check", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "hw_monitoring": { 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "name": "hw_monitoring", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Whether the hardware monitoring daemon be deployed on every host?", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "inventory_list_all": { 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "name": "inventory_list_all", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.216 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "log_refresh_metadata": { 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_refresh_metadata", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Log to the \"cephadm\" cluster log channel\"", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.217 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "manage_etc_ceph_ceph_conf": { 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "name": "manage_etc_ceph_ceph_conf", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on hosts.", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.218 INFO:teuthology.orchestra.run.trial028.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "*", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "max_count_per_host": { 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_count_per_host", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Max number of daemons per service per host: guard against mistakes and runaways", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "max_osd_draining_count": { 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_osd_draining_count", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Max number of OSDs that will be drained simultaneously when OSDs are removed", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "max_parallel_osd_upgrades": { 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_parallel_osd_upgrades", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.219 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "16", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Maximum number of OSD daemons upgraded in parallel.", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "migration_current": { 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "name": "migration_current", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Internal - do not modify", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "mode": { 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mode", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "root", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "cephadm-package", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "root" 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Mode for remote execution of cephadm", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "oob_default_addr": { 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "name": "oob_default_addr", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "169.254.1.1", 2026-02-25T11:58:59.220 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Default IP address for RedFish API (OOB management).", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "prometheus_alerts_path": { 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "name": "prometheus_alerts_path", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Location of alerts to include in Prometheus Alertmanager", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "registry_insecure": { 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "name": "registry_insecure", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "registry_password": { 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "name": "registry_password", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.221 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "registry_url": { 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "name": "registry_url", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Registry URL for login purposes. This is not the default registry.", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "registry_username": { 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "name": "registry_username", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "secure_monitoring_stack": { 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "name": "secure_monitoring_stack", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Enable TLS security for all monitoring stack daemons", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "service_discovery_port": { 2026-02-25T11:58:59.222 INFO:teuthology.orchestra.run.trial028.stdout: "name": "service_discovery_port", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8765", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Cephadm service discovery port", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "ssh_config_file": { 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssh_config_file", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Customized SSH config file to distribute to managed hosts", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "ssh_keepalive_count_max": { 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssh_keepalive_count_max", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.223 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "3", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How many sequential SSH connection liveness check failures before the host is marked offline", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "ssh_keepalive_interval": { 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssh_keepalive_interval", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "7", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How often SSH connections are checked for liveness", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "stray_daemon_check_interval": { 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "name": "stray_daemon_check_interval", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1800", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How frequently cephadm should check for the presence of stray daemons", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.224 INFO:teuthology.orchestra.run.trial028.stdout: "use_agent": { 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "name": "use_agent", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Use the cephadm agent on each host to gather and send metadata", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "use_repo_digest": { 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "name": "use_repo_digest", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Automatically convert image tags to image digest to ensure that all daemons use the same image", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "warn_on_failed_host_check": { 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_on_failed_host_check", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Raise a health warning if the host check fails", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "warn_on_stray_daemons": { 2026-02-25T11:58:59.225 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_on_stray_daemons", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Raise a health warning if daemons are detected that are not managed by cephadm", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "warn_on_stray_hosts": { 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_on_stray_hosts", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Raise a health warning if daemons are detected on a host that is not managed by cephadm", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crash", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.226 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.227 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "retain_interval": { 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "name": "retain_interval", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "31536000", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how long to retain crashes before pruning them", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "warn_recent_interval": { 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_recent_interval", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.228 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1209600", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "time interval in which to warn about recent crashes", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "name": "dashboard", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "ALERTMANAGER_API_HOST": { 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ALERTMANAGER_API_HOST", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.229 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "AUDIT_API_ENABLED": { 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "name": "AUDIT_API_ENABLED", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "AUDIT_API_LOG_PAYLOAD": { 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "ENABLE_BROWSABLE_API": { 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ENABLE_BROWSABLE_API", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.230 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_CEPHFS": { 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_ISCSI": { 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.231 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_MIRRORING": { 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_NFS": { 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_NFS", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_RBD": { 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_RBD", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_RGW": { 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_RGW", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.232 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_API_PASSWORD": { 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_API_PASSWORD", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "admin", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_API_SSL_VERIFY": { 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.233 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_API_URL": { 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_API_URL", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_API_USERNAME": { 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_API_USERNAME", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "admin", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_FRONTEND_API_URL": { 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.234 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "ISCSI_API_SSL_VERIFICATION": { 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "ISSUE_TRACKER_API_KEY": { 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ISSUE_TRACKER_API_KEY", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.235 INFO:teuthology.orchestra.run.trial028.stdout: "MANAGED_BY_CLUSTERS": { 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "name": "MANAGED_BY_CLUSTERS", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "[]", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "MULTICLUSTER_CONFIG": { 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "name": "MULTICLUSTER_CONFIG", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "{}", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "PROMETHEUS_API_HOST": { 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PROMETHEUS_API_HOST", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.236 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "PROM_ALERT_CREDENTIAL_CACHE_TTL": { 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PROM_ALERT_CREDENTIAL_CACHE_TTL", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.237 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2026-02-25T11:58:59.238 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_ENABLED": { 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_ENABLED", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2026-02-25T11:58:59.239 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_MIN_LENGTH": { 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "REST_REQUESTS_TIMEOUT": { 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "name": "REST_REQUESTS_TIMEOUT", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "45", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.240 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_API_ACCESS_KEY": { 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_API_ACCESS_KEY", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_API_ADMIN_RESOURCE": { 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "admin", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_API_SECRET_KEY": { 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_API_SECRET_KEY", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_API_SSL_VERIFY": { 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_API_SSL_VERIFY", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.241 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_HOSTNAME_PER_DAEMON": { 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_HOSTNAME_PER_DAEMON", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "UNSAFE_TLS_v1_2": { 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "name": "UNSAFE_TLS_v1_2", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "USER_PWD_EXPIRATION_SPAN": { 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.242 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "cross_origin_url": { 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cross_origin_url", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.243 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "crt_file": { 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crt_file", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "crypto_caller": { 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crypto_caller", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "debug": { 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "name": "debug", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.244 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Enable/disable debug options", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "jwt_token_ttl": { 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "name": "jwt_token_ttl", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "28800", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "key_file": { 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "name": "key_file", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.245 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.246 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "motd": { 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "name": "motd", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "The message of the day", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "redirect_resolve_ip_addr": { 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "name": "redirect_resolve_ip_addr", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "server_addr": { 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "name": "server_addr", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.247 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "::", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "server_port": { 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "name": "server_port", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8080", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "ssl": { 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssl", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.248 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "ssl_server_port": { 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssl_server_port", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8443", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "sso_oauth2": { 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sso_oauth2", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "standby_behaviour": { 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "name": "standby_behaviour", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "redirect", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "redirect" 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.249 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "standby_error_status_code": { 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "name": "standby_error_status_code", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "500", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "min": "400", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "max": "599", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "url_prefix": { 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "name": "url_prefix", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "name": "devicehealth", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "enable_monitoring": { 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "name": "enable_monitoring", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.250 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "monitor device health metrics", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.251 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "mark_out_threshold": { 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mark_out_threshold", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "2419200", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "automatically mark OSD if it may fail before this long", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "pool_name": { 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pool_name", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "device_health_metrics", 2026-02-25T11:58:59.252 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "name of pool in which to store device health metrics", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "retention_period": { 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "name": "retention_period", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "15552000", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how long to retain device health metrics", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "scrape_frequency": { 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "name": "scrape_frequency", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "86400", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how frequently to scrape device health metrics", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "self_heal": { 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "name": "self_heal", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "preemptively heal cluster around devices that may fail", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.253 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how frequently to wake up and check device health", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "warn_threshold": { 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_threshold", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "7257600", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "raise health warning if OSD may fail before this long", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.254 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "name": "diskprediction_local", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.255 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "predict_interval": { 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "name": "predict_interval", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "86400", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.256 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "predictor_model": { 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "name": "predictor_model", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "prophetstor", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "name": "influx", 2026-02-25T11:58:59.257 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": false, 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "influxdb python module not found", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "batch_size": { 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "name": "batch_size", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5000", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "database": { 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "name": "database", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "ceph", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "hostname": { 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "name": "hostname", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "InfluxDB server hostname", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.258 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "interval": { 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "name": "interval", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "30", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "min": "5", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.259 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "password": { 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "name": "password", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.260 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "password of InfluxDB server user", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "port": { 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "name": "port", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8086", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "InfluxDB server port", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "ssl": { 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssl", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "false", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.261 INFO:teuthology.orchestra.run.trial028.stdout: "threads": { 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "name": "threads", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "max": "32", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "username": { 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "name": "username", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "username of InfluxDB server user", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "verify_ssl": { 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "name": "verify_ssl", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "true", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "name": "insights", 2026-02-25T11:58:59.262 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.263 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "name": "iostat", 2026-02-25T11:58:59.264 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.265 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "name": "k8sevents", 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.266 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_event_retention_days": { 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ceph_event_retention_days", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "7", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Days to hold ceph event information within local cache", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "config_check_secs": { 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "name": "config_check_secs", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "min": "10", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.267 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.268 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "name": "localpool", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "failure_domain": { 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "name": "failure_domain", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "host", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "failure domain for any created local pool", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.269 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.270 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "min_size": { 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "name": "min_size", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "default min_size for any created local pool", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "num_rep": { 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "name": "num_rep", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "3", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "default replica count for any created local pool", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "pg_num": { 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pg_num", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.271 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "128", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "default pg_num for any created local pool", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "prefix": { 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "name": "prefix", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "name prefix for any created local pool", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "subtree": { 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "name": "subtree", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "rack", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.272 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "CRUSH level for which to create a local pool", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mds_autoscaler", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.273 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.274 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mirroring", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.275 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.276 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "name": "nfs", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.277 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.278 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "name": "orchestrator", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "fail_fs": { 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "name": "fail_fs", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.279 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.280 INFO:teuthology.orchestra.run.trial028.stdout: "orchestrator": { 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "name": "orchestrator", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "cephadm", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "rook", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "test_orchestrator" 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Orchestrator backend", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "name": "osd_perf_query", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.281 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.282 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "name": "osd_support", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.283 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.284 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pg_autoscaler", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.285 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.286 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "threshold": { 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "name": "threshold", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.287 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "3.0", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1.0", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "scaling threshold", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "name": "progress", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "allow_pg_recovery_event": { 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "name": "allow_pg_recovery_event", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "allow the module to show pg recovery progress", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "enabled": { 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "name": "enabled", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.288 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.289 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "max_completed_events": { 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_completed_events", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "50", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "number of past completed events to remember", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how long the module is going to sleep", 2026-02-25T11:58:59.290 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "name": "prometheus", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "cache": { 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cache", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "exclude_perf_counters": { 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "name": "exclude_perf_counters", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.291 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Do not include perf-counters in the metrics output", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "healthcheck_history_max_entries": { 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "name": "healthcheck_history_max_entries", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1000", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Maximum number of health check history entries to keep", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.292 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "rbd_stats_pools": { 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rbd_stats_pools", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.293 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "rbd_stats_pools_refresh_interval": { 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rbd_stats_pools_refresh_interval", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "300", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "scrape_interval": { 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "name": "scrape_interval", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "15.0", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "server_addr": { 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "name": "server_addr", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "::", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.294 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "server_port": { 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "name": "server_port", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "9283", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "the port on which the module listens for HTTP requests", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "stale_cache_strategy": { 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "name": "stale_cache_strategy", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "log", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.295 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "standby_behaviour": { 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "name": "standby_behaviour", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "default", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "default", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "error" 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "standby_error_status_code": { 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "name": "standby_error_status_code", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "500", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "min": "400", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "max": "599", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rbd_support", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.296 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.297 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "max_concurrent_snap_create": { 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_concurrent_snap_create", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "mirror_snapshot_schedule": { 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mirror_snapshot_schedule", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.298 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "trash_purge_schedule": { 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "name": "trash_purge_schedule", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rgw", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.299 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.300 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "secondary_zone_period_retry_limit": { 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "name": "secondary_zone_period_retry_limit", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "RGW module period update retry limit for secondary site", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rook", 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.301 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.302 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "prometheus_tls_secret_name": { 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "name": "prometheus_tls_secret_name", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "rook-ceph-prometheus-server-tls", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "name of tls secret in k8s for prometheus", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "secure_monitoring_stack": { 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "name": "secure_monitoring_stack", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.303 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "storage_class": { 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "name": "storage_class", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "local", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "storage class name for LSO-discovered PVs", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "name": "selftest", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.304 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.305 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "roption1": { 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "name": "roption1", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "roption2": { 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "name": "roption2", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "xyz", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.306 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption1": { 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption1", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption2": { 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption2", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption3": { 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption3", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption4": { 2026-02-25T11:58:59.307 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption4", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption5": { 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption5", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption6": { 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption6", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.308 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption7": { 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption7", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "max": "42", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "testkey": { 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "name": "testkey", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "testlkey": { 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "name": "testlkey", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.309 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "testnewline": { 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "name": "testnewline", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smb", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "internal_store_backend": { 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "name": "internal_store_backend", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "set internal store backend. for develoment and testing only", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.310 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.311 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "update_orchestration": { 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "name": "update_orchestration", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "automatically update orchestration when smb resources are changed", 2026-02-25T11:58:59.312 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "name": "snap_schedule", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "allow_m_granularity": { 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "name": "allow_m_granularity", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "allow minute scheduled snapshots", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "dump_on_update": { 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "name": "dump_on_update", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "dump database to debug log on update", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.313 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.314 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "name": "stats", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.315 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.316 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "name": "status", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.317 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.318 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "name": "telegraf", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "address": { 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "name": "address", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.319 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "interval": { 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "name": "interval", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "15", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.320 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.321 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "name": "telemetry", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "channel_basic": { 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_basic", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share basic cluster information (size, version)", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "channel_crash": { 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_crash", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.322 INFO:teuthology.orchestra.run.trial028.stdout: "channel_device": { 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_device", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "channel_ident": { 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_ident", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "channel_perf": { 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_perf", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share various performance metrics of a cluster", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "contact": { 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "name": "contact", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.323 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "description": { 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "name": "description", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "device_url": { 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "name": "device_url", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "https://telemetry.ceph.com/device", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "enabled": { 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "name": "enabled", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.324 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "interval": { 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "name": "interval", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "24", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "min": "8", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "last_opt_revision": { 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "name": "last_opt_revision", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "leaderboard": { 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "name": "leaderboard", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.325 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "leaderboard_description": { 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "name": "leaderboard_description", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.326 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "organization": { 2026-02-25T11:58:59.327 INFO:teuthology.orchestra.run.trial028.stdout: "name": "organization", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "proxy": { 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "name": "proxy", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "url": { 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "name": "url", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "https://telemetry.ceph.com/report", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.328 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "name": "test_orchestrator", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.329 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.330 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "name": "volumes", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.331 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "max_concurrent_clones": { 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_concurrent_clones", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "4", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Number of asynchronous cloner threads", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.332 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "pause_cloning": { 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pause_cloning", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Pause asynchronous cloner threads", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "pause_purging": { 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pause_purging", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Pause asynchronous subvolume purge threads", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "periodic_async_work": { 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "name": "periodic_async_work", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Periodically check for async work", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "snapshot_clone_delay": { 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "name": "snapshot_clone_delay", 2026-02-25T11:58:59.333 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "snapshot_clone_no_wait": { 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "name": "snapshot_clone_no_wait", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: ] 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "modules": [ 2026-02-25T11:58:59.334 INFO:teuthology.orchestra.run.trial028.stdout: "cephadm", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "dashboard", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "iostat", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "nfs" 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "available_modules": [ 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "name": "alerts", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "interval": { 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "name": "interval", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How frequently to reexamine health status", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.335 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.336 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_destination": { 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_destination", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_from_name": { 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_from_name", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "Ceph", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Email From: name", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_host": { 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_host", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.337 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "SMTP server", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_password": { 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_password", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Password to authenticate with", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_port": { 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_port", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "465", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "SMTP port", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_sender": { 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_sender", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.338 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "SMTP envelope sender", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_ssl": { 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_ssl", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Use SSL to connect to SMTP server", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "smtp_user": { 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smtp_user", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "User to authenticate as", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.339 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "name": "balancer", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "active": { 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "name": "active", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "automatically balance PGs across cluster", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "begin_time": { 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "name": "begin_time", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0000", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "beginning time of day to automatically balance", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "This is a time of day in the format HHMM.", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "begin_weekday": { 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "name": "begin_weekday", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "type": "uint", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "min": "0", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "max": "6", 2026-02-25T11:58:59.340 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "crush_compat_max_iterations": { 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crush_compat_max_iterations", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "type": "uint", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "25", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "max": "250", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "maximum number of iterations to attempt optimization", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "crush_compat_metrics": { 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crush_compat_metrics", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "pgs,objects,bytes", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "metrics with which to calculate OSD utilization", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "crush_compat_step": { 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crush_compat_step", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0.5", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "min": "0.001", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "max": "0.999", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "aggressiveness of optimization", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.341 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "end_time": { 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "name": "end_time", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "2359", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "ending time of day to automatically balance", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "This is a time of day in the format HHMM.", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "end_weekday": { 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "name": "end_weekday", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "type": "uint", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "min": "0", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "max": "6", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.342 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.343 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "min_score": { 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "name": "min_score", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "minimum score, below which no optimization is attempted", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "mode": { 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mode", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "upmap", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "crush-compat", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "none", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "read", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "upmap", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "upmap-read" 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Balancer mode", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "pool_ids": { 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pool_ids", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.344 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "pools which the automatic balancing will be limited to", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how frequently to wake up and attempt optimization", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "update_pg_upmap_activity": { 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "name": "update_pg_upmap_activity", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.345 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Updates pg_upmap activity stats to be used in `balancer status detail`", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "upmap_max_deviation": { 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "name": "upmap_max_deviation", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "deviation below which no optimization is attempted", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "upmap_max_optimizations": { 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "name": "upmap_max_optimizations", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "type": "uint", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "maximum upmap optimizations to make per attempt", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cephadm", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "agent_down_multiplier": { 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "name": "agent_down_multiplier", 2026-02-25T11:58:59.346 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "3.0", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "agent_refresh_rate": { 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "name": "agent_refresh_rate", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "20", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How often the agent on each host will gather and send metadata", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "agent_starting_port": { 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "name": "agent_starting_port", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "4721", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "allow_ptrace": { 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "name": "allow_ptrace", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.347 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "autotune_interval": { 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "name": "autotune_interval", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How frequently to autotune daemon memory", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "autotune_memory_target_ratio": { 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "name": "autotune_memory_target_ratio", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0.7", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "cephadm_log_destination": { 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cephadm_log_destination", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.348 INFO:teuthology.orchestra.run.trial028.stdout: "file", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "file,syslog", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "syslog" 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Destination for cephadm command persistent logging", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_automated_rotation_enabled": { 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_automated_rotation_enabled", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "This flag controls whether cephadm automatically rotates certificates upon expiration.", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_check_debug_mode": { 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_check_debug_mode", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_check_period": { 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_check_period", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "min": "0", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "max": "30", 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.349 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Specifies how often (in days) the certificate should be checked for validity.", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_duration_days": { 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_duration_days", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1095", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "min": "90", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "max": "3650", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Specifies the duration of self-signed certificates generated and signed by the cephadm root CA", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "certificate_renewal_threshold_days": { 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "name": "certificate_renewal_threshold_days", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "30", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "min": "10", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "max": "90", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Specifies the lead time in days to initiate certificate renewal before expiration.", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "cgroups_split": { 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cgroups_split", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently Podman only)", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.350 INFO:teuthology.orchestra.run.trial028.stdout: "config_checks_enabled": { 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "name": "config_checks_enabled", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Enable or disable cephadm configuration analysis", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "config_dashboard": { 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "name": "config_dashboard", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Manage configs like API endpoints in Dashboard.", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_alertmanager": { 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_alertmanager", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.28.1", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Alertmanager container image", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_alloy": { 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_alloy", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.351 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "docker.io/grafana/alloy:latest", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Alloy container image", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_base": { 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_base", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/ceph", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Container image name, without the tag", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_elasticsearch": { 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_elasticsearch", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Elasticsearch container image", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_grafana": { 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_grafana", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/grafana:12.3.1", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.352 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Grafana container image", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_haproxy": { 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_haproxy", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Haproxy container image", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_jaeger_agent": { 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_jaeger_agent", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Jaeger agent container image", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_jaeger_collector": { 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_jaeger_collector", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Jaeger collector container image", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.353 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_jaeger_query": { 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_jaeger_query", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Jaeger query container image", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_keepalived": { 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_keepalived", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Keepalived container image", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_loki": { 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_loki", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Loki container image", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_nginx": { 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_nginx", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.354 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Nginx container image", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_node_exporter": { 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_node_exporter", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.9.1", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Node exporter container image", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_nvmeof": { 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_nvmeof", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/ceph/nvmeof:1.7", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Nvmeof container image", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_oauth2_proxy": { 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_oauth2_proxy", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.355 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Oauth2 proxy container image", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_prometheus": { 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_prometheus", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/prometheus/prometheus:v3.6.0", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Prometheus container image", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_promtail": { 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_promtail", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Promtail container image", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_samba": { 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_samba", 2026-02-25T11:58:59.356 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Samba container image", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_samba_metrics": { 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_samba_metrics", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io/samba.org/samba-metrics:devbuilds-centos-amd64", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Samba metrics container image", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "container_image_snmp_gateway": { 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_image_snmp_gateway", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Snmp gateway container image", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "container_init": { 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "name": "container_init", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.357 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Run podman/docker with `--init`", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "daemon_cache_timeout": { 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "name": "daemon_cache_timeout", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Seconds for which to cache service (daemon) inventory", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "default_cephadm_command_timeout": { 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "name": "default_cephadm_command_timeout", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "900", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "default_registry": { 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "name": "default_registry", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "quay.io", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.358 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "device_cache_timeout": { 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "name": "device_cache_timeout", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1800", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Seconds for which to cache device inventory", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "device_enhanced_scan": { 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "name": "device_enhanced_scan", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Use libstoragemgmt during device scans", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "facts_cache_timeout": { 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "name": "facts_cache_timeout", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Seconds for which to cache host facts data", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.359 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "grafana_dashboards_path": { 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "name": "grafana_dashboards_path", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Location of dashboards to include in Grafana", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "host_check_interval": { 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "name": "host_check_interval", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How frequently to perform a host check", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "hw_monitoring": { 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "name": "hw_monitoring", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Whether the hardware monitoring daemon be deployed on every host?", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.360 INFO:teuthology.orchestra.run.trial028.stdout: "inventory_list_all": { 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "name": "inventory_list_all", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "log_refresh_metadata": { 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_refresh_metadata", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.361 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Log to the \"cephadm\" cluster log channel\"", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.362 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "manage_etc_ceph_ceph_conf": { 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "name": "manage_etc_ceph_ceph_conf", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on hosts.", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "*", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "max_count_per_host": { 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_count_per_host", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Max number of daemons per service per host: guard against mistakes and runaways", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.363 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "max_osd_draining_count": { 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_osd_draining_count", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Max number of OSDs that will be drained simultaneously when OSDs are removed", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "max_parallel_osd_upgrades": { 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_parallel_osd_upgrades", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "16", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Maximum number of OSD daemons upgraded in parallel.", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "migration_current": { 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "name": "migration_current", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Internal - do not modify", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "mode": { 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mode", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.364 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "root", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "cephadm-package", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "root" 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Mode for remote execution of cephadm", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "oob_default_addr": { 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "name": "oob_default_addr", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "169.254.1.1", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Default IP address for RedFish API (OOB management).", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "prometheus_alerts_path": { 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "name": "prometheus_alerts_path", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Location of alerts to include in Prometheus Alertmanager", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "registry_insecure": { 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "name": "registry_insecure", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.365 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "registry_password": { 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "name": "registry_password", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "registry_url": { 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "name": "registry_url", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Registry URL for login purposes. This is not the default registry.", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "registry_username": { 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "name": "registry_username", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.366 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "secure_monitoring_stack": { 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "name": "secure_monitoring_stack", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Enable TLS security for all monitoring stack daemons", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "service_discovery_port": { 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "name": "service_discovery_port", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8765", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Cephadm service discovery port", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.367 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "ssh_config_file": { 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssh_config_file", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Customized SSH config file to distribute to managed hosts", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "ssh_keepalive_count_max": { 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssh_keepalive_count_max", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "3", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How many sequential SSH connection liveness check failures before the host is marked offline", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "ssh_keepalive_interval": { 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssh_keepalive_interval", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "7", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.368 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How often SSH connections are checked for liveness", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "stray_daemon_check_interval": { 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "name": "stray_daemon_check_interval", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1800", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How frequently cephadm should check for the presence of stray daemons", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "use_agent": { 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "name": "use_agent", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Use the cephadm agent on each host to gather and send metadata", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "use_repo_digest": { 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "name": "use_repo_digest", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Automatically convert image tags to image digest to ensure that all daemons use the same image", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.369 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "warn_on_failed_host_check": { 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_on_failed_host_check", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Raise a health warning if the host check fails", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "warn_on_stray_daemons": { 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_on_stray_daemons", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Raise a health warning if daemons are detected that are not managed by cephadm", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "warn_on_stray_hosts": { 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_on_stray_hosts", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Raise a health warning if daemons are detected on a host that is not managed by cephadm", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.370 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crash", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.371 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "retain_interval": { 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "name": "retain_interval", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "31536000", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how long to retain crashes before pruning them", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.372 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "warn_recent_interval": { 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_recent_interval", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1209600", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "time interval in which to warn about recent crashes", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "name": "dashboard", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "ALERTMANAGER_API_HOST": { 2026-02-25T11:58:59.373 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ALERTMANAGER_API_HOST", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "AUDIT_API_ENABLED": { 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "name": "AUDIT_API_ENABLED", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "AUDIT_API_LOG_PAYLOAD": { 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.374 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "ENABLE_BROWSABLE_API": { 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ENABLE_BROWSABLE_API", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_CEPHFS": { 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.375 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_ISCSI": { 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_MIRRORING": { 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_NFS": { 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_NFS", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.376 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_RBD": { 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_RBD", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "FEATURE_TOGGLE_RGW": { 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "name": "FEATURE_TOGGLE_RGW", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.377 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_API_PASSWORD": { 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_API_PASSWORD", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "admin", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_API_SSL_VERIFY": { 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_API_URL": { 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_API_URL", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_API_USERNAME": { 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_API_USERNAME", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.378 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "admin", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_FRONTEND_API_URL": { 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "ISCSI_API_SSL_VERIFICATION": { 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.379 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "ISSUE_TRACKER_API_KEY": { 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ISSUE_TRACKER_API_KEY", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "MANAGED_BY_CLUSTERS": { 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "name": "MANAGED_BY_CLUSTERS", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "[]", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "MULTICLUSTER_CONFIG": { 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "name": "MULTICLUSTER_CONFIG", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "{}", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.380 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "PROMETHEUS_API_HOST": { 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PROMETHEUS_API_HOST", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "PROM_ALERT_CREDENTIAL_CACHE_TTL": { 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PROM_ALERT_CREDENTIAL_CACHE_TTL", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.381 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.382 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.383 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_ENABLED": { 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_ENABLED", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.384 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "PWD_POLICY_MIN_LENGTH": { 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "REST_REQUESTS_TIMEOUT": { 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "name": "REST_REQUESTS_TIMEOUT", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "45", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_API_ACCESS_KEY": { 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_API_ACCESS_KEY", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.385 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_API_ADMIN_RESOURCE": { 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "admin", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_API_SECRET_KEY": { 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_API_SECRET_KEY", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_API_SSL_VERIFY": { 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_API_SSL_VERIFY", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "RGW_HOSTNAME_PER_DAEMON": { 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "name": "RGW_HOSTNAME_PER_DAEMON", 2026-02-25T11:58:59.386 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "UNSAFE_TLS_v1_2": { 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "name": "UNSAFE_TLS_v1_2", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "USER_PWD_EXPIRATION_SPAN": { 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2026-02-25T11:58:59.387 INFO:teuthology.orchestra.run.trial028.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "cross_origin_url": { 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cross_origin_url", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "crt_file": { 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crt_file", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.388 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "crypto_caller": { 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "name": "crypto_caller", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "debug": { 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "name": "debug", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Enable/disable debug options", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "jwt_token_ttl": { 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "name": "jwt_token_ttl", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "28800", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.389 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "key_file": { 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "name": "key_file", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.390 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "motd": { 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "name": "motd", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.391 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "The message of the day", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "redirect_resolve_ip_addr": { 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "name": "redirect_resolve_ip_addr", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "server_addr": { 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "name": "server_addr", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "::", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "server_port": { 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "name": "server_port", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8080", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.392 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "ssl": { 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssl", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "ssl_server_port": { 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssl_server_port", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8443", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.393 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "sso_oauth2": { 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sso_oauth2", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "standby_behaviour": { 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "name": "standby_behaviour", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "redirect", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "redirect" 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "standby_error_status_code": { 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "name": "standby_error_status_code", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "500", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "min": "400", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "max": "599", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.394 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "url_prefix": { 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "name": "url_prefix", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "name": "devicehealth", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "enable_monitoring": { 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "name": "enable_monitoring", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "monitor device health metrics", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.395 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.396 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "mark_out_threshold": { 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mark_out_threshold", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "2419200", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "automatically mark OSD if it may fail before this long", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "pool_name": { 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pool_name", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "device_health_metrics", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "name of pool in which to store device health metrics", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.397 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "retention_period": { 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "name": "retention_period", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "15552000", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how long to retain device health metrics", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "scrape_frequency": { 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "name": "scrape_frequency", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "86400", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how frequently to scrape device health metrics", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "self_heal": { 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "name": "self_heal", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "preemptively heal cluster around devices that may fail", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.398 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how frequently to wake up and check device health", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "warn_threshold": { 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "name": "warn_threshold", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "7257600", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "raise health warning if OSD may fail before this long", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "name": "diskprediction_local", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.399 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.400 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "predict_interval": { 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "name": "predict_interval", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "86400", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "predictor_model": { 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "name": "predictor_model", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "prophetstor", 2026-02-25T11:58:59.401 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "600", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "name": "influx", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": false, 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "influxdb python module not found", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "batch_size": { 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "name": "batch_size", 2026-02-25T11:58:59.402 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5000", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "database": { 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "name": "database", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "ceph", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "hostname": { 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "name": "hostname", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "InfluxDB server hostname", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "interval": { 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "name": "interval", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "30", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "min": "5", 2026-02-25T11:58:59.403 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.404 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "password": { 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "name": "password", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "password of InfluxDB server user", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.405 INFO:teuthology.orchestra.run.trial028.stdout: "port": { 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "name": "port", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "8086", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "InfluxDB server port", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "ssl": { 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ssl", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "false", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "threads": { 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "name": "threads", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.406 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "max": "32", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "username": { 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "name": "username", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "username of InfluxDB server user", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "verify_ssl": { 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "name": "verify_ssl", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "true", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "name": "insights", 2026-02-25T11:58:59.407 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.408 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.409 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "name": "iostat", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.410 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.411 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "name": "k8sevents", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_event_retention_days": { 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "name": "ceph_event_retention_days", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "7", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Days to hold ceph event information within local cache", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "config_check_secs": { 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "name": "config_check_secs", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "min": "10", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.412 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.413 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "name": "localpool", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "failure_domain": { 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "name": "failure_domain", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "host", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "failure domain for any created local pool", 2026-02-25T11:58:59.414 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.415 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "min_size": { 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "name": "min_size", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "default min_size for any created local pool", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "num_rep": { 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "name": "num_rep", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.416 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "3", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "default replica count for any created local pool", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "pg_num": { 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pg_num", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "128", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "default pg_num for any created local pool", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "prefix": { 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "name": "prefix", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "name prefix for any created local pool", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.417 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "subtree": { 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "name": "subtree", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "rack", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "CRUSH level for which to create a local pool", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mds_autoscaler", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.418 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.419 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mirroring", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.420 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.421 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "name": "nfs", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.422 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.423 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "name": "orchestrator", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "fail_fs": { 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "name": "fail_fs", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.424 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.425 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "orchestrator": { 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "name": "orchestrator", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "cephadm", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "rook", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "test_orchestrator" 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Orchestrator backend", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.426 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "name": "osd_perf_query", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.427 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.428 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "name": "osd_support", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.429 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.430 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pg_autoscaler", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.431 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.432 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "60", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "threshold": { 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "name": "threshold", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "3.0", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1.0", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "scaling threshold", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.433 INFO:teuthology.orchestra.run.trial028.stdout: "name": "progress", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "allow_pg_recovery_event": { 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "name": "allow_pg_recovery_event", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "allow the module to show pg recovery progress", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "enabled": { 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "name": "enabled", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.434 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.435 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "max_completed_events": { 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_completed_events", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "50", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "number of past completed events to remember", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "sleep_interval": { 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sleep_interval", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "how long the module is going to sleep", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.436 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "name": "prometheus", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "cache": { 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "name": "cache", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "exclude_perf_counters": { 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "name": "exclude_perf_counters", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Do not include perf-counters in the metrics output", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "healthcheck_history_max_entries": { 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "name": "healthcheck_history_max_entries", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.437 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1000", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Maximum number of health check history entries to keep", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.438 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "rbd_stats_pools": { 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rbd_stats_pools", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.439 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "rbd_stats_pools_refresh_interval": { 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rbd_stats_pools_refresh_interval", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "300", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "scrape_interval": { 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "name": "scrape_interval", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "15.0", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "server_addr": { 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "name": "server_addr", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "::", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.440 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "server_port": { 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "name": "server_port", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "9283", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "the port on which the module listens for HTTP requests", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "stale_cache_strategy": { 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "name": "stale_cache_strategy", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "log", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.441 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "standby_behaviour": { 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "name": "standby_behaviour", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "default", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "default", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "error" 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "standby_error_status_code": { 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "name": "standby_error_status_code", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "500", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "min": "400", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "max": "599", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rbd_support", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.442 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.443 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "max_concurrent_snap_create": { 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_concurrent_snap_create", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "10", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "mirror_snapshot_schedule": { 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "name": "mirror_snapshot_schedule", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.444 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "trash_purge_schedule": { 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "name": "trash_purge_schedule", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rgw", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.445 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.446 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "secondary_zone_period_retry_limit": { 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "name": "secondary_zone_period_retry_limit", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "5", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "RGW module period update retry limit for secondary site", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.447 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rook", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.448 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "prometheus_tls_secret_name": { 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "name": "prometheus_tls_secret_name", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "rook-ceph-prometheus-server-tls", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.449 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "name of tls secret in k8s for prometheus", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "secure_monitoring_stack": { 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "name": "secure_monitoring_stack", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "storage_class": { 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "name": "storage_class", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "local", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.450 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "storage class name for LSO-discovered PVs", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "name": "selftest", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.451 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "roption1": { 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "name": "roption1", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.452 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "roption2": { 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "name": "roption2", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "xyz", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption1": { 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption1", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption2": { 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption2", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.453 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption3": { 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption3", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "type": "float", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption4": { 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption4", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption5": { 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption5", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption6": { 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption6", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.454 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "rwoption7": { 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rwoption7", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "min": "1", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "max": "42", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "testkey": { 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "name": "testkey", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.455 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "testlkey": { 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "name": "testlkey", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "testnewline": { 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "name": "testnewline", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "name": "smb", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "internal_store_backend": { 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "name": "internal_store_backend", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.456 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "set internal store backend. for develoment and testing only", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.457 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.458 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "update_orchestration": { 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "name": "update_orchestration", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "automatically update orchestration when smb resources are changed", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "name": "snap_schedule", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "allow_m_granularity": { 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "name": "allow_m_granularity", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "allow minute scheduled snapshots", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "dump_on_update": { 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "name": "dump_on_update", 2026-02-25T11:58:59.459 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "dump database to debug log on update", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.460 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.461 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "name": "stats", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.462 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.463 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "name": "status", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.464 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.465 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "name": "telegraf", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "address": { 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "name": "address", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "interval": { 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "name": "interval", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "type": "secs", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "15", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.466 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.467 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "name": "telemetry", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "channel_basic": { 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_basic", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.468 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share basic cluster information (size, version)", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "channel_crash": { 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_crash", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "channel_device": { 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_device", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "channel_ident": { 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_ident", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.469 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "channel_perf": { 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "name": "channel_perf", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Share various performance metrics of a cluster", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "contact": { 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "name": "contact", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "description": { 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "name": "description", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.470 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "device_url": { 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "name": "device_url", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "https://telemetry.ceph.com/device", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "enabled": { 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "name": "enabled", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "interval": { 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "name": "interval", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "24", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "min": "8", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.471 INFO:teuthology.orchestra.run.trial028.stdout: "last_opt_revision": { 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "name": "last_opt_revision", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "1", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "leaderboard": { 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "name": "leaderboard", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "leaderboard_description": { 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "name": "leaderboard_description", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.472 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.473 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "organization": { 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "name": "organization", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.474 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "proxy": { 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "name": "proxy", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "url": { 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "name": "url", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "https://telemetry.ceph.com/report", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.475 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "name": "test_orchestrator", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.476 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "name": "volumes", 2026-02-25T11:58:59.477 INFO:teuthology.orchestra.run.trial028.stdout: "can_run": true, 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "error_string": "", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "module_options": { 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "log_level": { 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_level", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster": { 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_cluster_level": { 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_cluster_level", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "type": "str", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "info", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.478 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [ 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "critical", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "debug", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "error", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "info", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "warning" 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "log_to_file": { 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "name": "log_to_file", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "max_concurrent_clones": { 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "name": "max_concurrent_clones", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "4", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Number of asynchronous cloner threads", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.479 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "pause_cloning": { 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pause_cloning", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Pause asynchronous cloner threads", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "pause_purging": { 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "name": "pause_purging", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Pause asynchronous subvolume purge threads", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "periodic_async_work": { 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "name": "periodic_async_work", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "False", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Periodically check for async work", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "snapshot_clone_delay": { 2026-02-25T11:58:59.480 INFO:teuthology.orchestra.run.trial028.stdout: "name": "snapshot_clone_delay", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "snapshot_clone_no_wait": { 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "name": "snapshot_clone_no_wait", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "type": "bool", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "level": "advanced", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 0, 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "True", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "sqlite3_killpoint": { 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "name": "sqlite3_killpoint", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "type": "int", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "level": "dev", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "flags": 1, 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "default_value": "0", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "min": "", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "max": "", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "enum_allowed": [], 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "desc": "", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "long_desc": "", 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "tags": [], 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "see_also": [] 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "services": { 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: "dashboard": "https://10.20.193.28:8443/" 2026-02-25T11:58:59.481 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "always_on_modules": { 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "octopus": [ 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "balancer", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "crash", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "devicehealth", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "orchestrator", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "pg_autoscaler", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "progress", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "rbd_support", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "status", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "telemetry", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "volumes" 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "pacific": [ 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "balancer", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "crash", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "devicehealth", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "orchestrator", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "pg_autoscaler", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "progress", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "rbd_support", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "status", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "telemetry", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "volumes" 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "quincy": [ 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "balancer", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "crash", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "devicehealth", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "orchestrator", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "pg_autoscaler", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "progress", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "rbd_support", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "status", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "telemetry", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "volumes" 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "reef": [ 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "balancer", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "crash", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "devicehealth", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "orchestrator", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "pg_autoscaler", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "progress", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "rbd_support", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "status", 2026-02-25T11:58:59.482 INFO:teuthology.orchestra.run.trial028.stdout: "telemetry", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "volumes" 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "squid": [ 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "balancer", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "crash", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "devicehealth", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "orchestrator", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "pg_autoscaler", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "progress", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "rbd_support", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "status", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "telemetry", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "volumes" 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: ], 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "tentacle": [ 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "balancer", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "crash", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "devicehealth", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "orchestrator", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "pg_autoscaler", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "progress", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "rbd_support", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "status", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "telemetry", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "volumes" 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: ] 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "force_disabled_modules": {}, 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "last_failure_osd_epoch": 3, 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "active_clients": [ 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "name": "devicehealth", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "addrvec": [ 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "type": "v2", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "addr": "10.20.193.28:0", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "nonce": 628429394 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: ] 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "name": "libcephsqlite", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "addrvec": [ 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "type": "v2", 2026-02-25T11:58:59.483 INFO:teuthology.orchestra.run.trial028.stdout: "addr": "10.20.193.28:0", 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: "nonce": 825682998 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: ] 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: "name": "rbd_support", 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: "addrvec": [ 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: "type": "v2", 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: "addr": "10.20.193.28:0", 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: "nonce": 1871763065 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: ] 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: "name": "volumes", 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: "addrvec": [ 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: { 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: "type": "v2", 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: "addr": "10.20.193.28:0", 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: "nonce": 1120515357 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: ] 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout: ] 2026-02-25T11:58:59.484 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:58:59.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.018+0000 7fcc2771c640 1 -- 10.20.193.28:0/4211371917 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcbfc0b5760 msgr2=0x7fcbfc0d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:59.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.018+0000 7fcc2771c640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcbfc0b5760 0x7fcbfc0d5c10 secure :-1 s=READY pgs=409 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcc0c0023d0 tx=0x7fcc0c031040 comp rx=0 tx=0).stop 2026-02-25T11:58:59.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.018+0000 7fcc2771c640 1 -- 10.20.193.28:0/4211371917 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcc2010b540 msgr2=0x7fcc200f4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:59.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.018+0000 7fcc2771c640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcc2010b540 0x7fcc200f4650 secure :-1 s=READY pgs=935 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fcc18009480 tx=0x7fcc1800ee60 comp rx=0 tx=0).stop 2026-02-25T11:58:59.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.018+0000 7fcc2771c640 1 -- 10.20.193.28:0/4211371917 shutdown_connections 2026-02-25T11:58:59.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.018+0000 7fcc2771c640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fcbfc0b5760 0x7fcbfc0d5c10 unknown :-1 s=CLOSED pgs=409 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:59.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.018+0000 7fcc2771c640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fcc2010dbd0 0x7fcc200f58a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:59.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.018+0000 7fcc2771c640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fcc2010d250 0x7fcc200f4ca0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:59.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.018+0000 7fcc2771c640 1 --2- 10.20.193.28:0/4211371917 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fcc2010b540 0x7fcc200f4650 unknown :-1 s=CLOSED pgs=935 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:59.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.018+0000 7fcc2771c640 1 -- 10.20.193.28:0/4211371917 >> 10.20.193.28:0/4211371917 conn(0x7fcc200e8c00 msgr2=0x7fcc200ea8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:59.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.018+0000 7fcc2771c640 1 -- 10.20.193.28:0/4211371917 shutdown_connections 2026-02-25T11:58:59.485 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.018+0000 7fcc2771c640 1 -- 10.20.193.28:0/4211371917 wait complete. 2026-02-25T11:58:59.489 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:58:59.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:59 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3697335868' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2026-02-25T11:58:59.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:59 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1106655556' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2026-02-25T11:58:59.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:58:59 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/4211371917' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2026-02-25T11:58:59.671 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:58:59.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:59 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3697335868' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2026-02-25T11:58:59.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:59 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1106655556' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2026-02-25T11:58:59.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:58:59 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/4211371917' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2026-02-25T11:58:59.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:59 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3697335868' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2026-02-25T11:58:59.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:59 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1106655556' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2026-02-25T11:58:59.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:58:59 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/4211371917' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2026-02-25T11:58:59.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.857+0000 7f292e56f640 1 -- 10.20.193.28:0/2206997434 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f292810eb80 msgr2=0x7f2928153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:59.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.857+0000 7f292e56f640 1 --2- 10.20.193.28:0/2206997434 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f292810eb80 0x7f2928153200 secure :-1 s=READY pgs=936 gs=3 cs=0 l=1 c_cookie=2835262848112259047 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f291c009950 tx=0x7f291c02f1d0 comp rx=0 tx=0).stop 2026-02-25T11:58:59.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.857+0000 7f292e56f640 1 -- 10.20.193.28:0/2206997434 shutdown_connections 2026-02-25T11:58:59.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.857+0000 7f292e56f640 1 --2- 10.20.193.28:0/2206997434 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f2928153740 0x7f2928173b60 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:59.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.857+0000 7f292e56f640 1 --2- 10.20.193.28:0/2206997434 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f292810eb80 0x7f2928153200 unknown :-1 s=CLOSED pgs=936 gs=3 cs=0 l=1 c_cookie=2835262848112259047 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:59.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.857+0000 7f292e56f640 1 --2- 10.20.193.28:0/2206997434 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f292810e190 0x7f292810e560 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:59.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.857+0000 7f292e56f640 1 -- 10.20.193.28:0/2206997434 >> 10.20.193.28:0/2206997434 conn(0x7f29280e9b40 msgr2=0x7f2928109f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:58:59.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.857+0000 7f292e56f640 1 -- 10.20.193.28:0/2206997434 shutdown_connections 2026-02-25T11:58:59.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.857+0000 7f292e56f640 1 -- 10.20.193.28:0/2206997434 wait complete. 2026-02-25T11:58:59.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.858+0000 7f292e56f640 1 Processor -- start 2026-02-25T11:58:59.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.858+0000 7f292e56f640 1 -- start start 2026-02-25T11:58:59.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292e56f640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f292810e190 0x7f2928162190 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:59.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292e56f640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f292810eb80 0x7f29281626d0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:59.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292e56f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2928153740 0x7f2928167aa0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:59.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292e56f640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f29281680c0 con 0x7f2928153740 2026-02-25T11:58:59.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292e56f640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2928168230 con 0x7f292810eb80 2026-02-25T11:58:59.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292e56f640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29281683f0 con 0x7f292810e190 2026-02-25T11:58:59.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292dd6e640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2928153740 0x7f2928167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:59.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292d56d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f292810e190 0x7f2928162190 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:59.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292d56d640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f292810e190 0x7f2928162190 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:49080/0 (socket says 10.20.193.28:49080) 2026-02-25T11:58:59.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292d56d640 1 -- 10.20.193.28:0/2206510807 learned_addr learned my addr 10.20.193.28:0/2206510807 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:58:59.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292cd6c640 1 --2- 10.20.193.28:0/2206510807 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f292810eb80 0x7f29281626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:59.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292dd6e640 1 -- 10.20.193.28:0/2206510807 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f292810e190 msgr2=0x7f2928162190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:59.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292dd6e640 1 --2- 10.20.193.28:0/2206510807 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f292810e190 0x7f2928162190 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:59.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292dd6e640 1 -- 10.20.193.28:0/2206510807 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f292810eb80 msgr2=0x7f29281626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:58:59.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292dd6e640 1 --2- 10.20.193.28:0/2206510807 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f292810eb80 0x7f29281626d0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:58:59.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.859+0000 7f292dd6e640 1 -- 10.20.193.28:0/2206510807 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f291c009570 con 0x7f2928153740 2026-02-25T11:58:59.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.860+0000 7f292dd6e640 1 --2- 10.20.193.28:0/2206510807 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2928153740 0x7f2928167aa0 secure :-1 s=READY pgs=937 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f292400f940 tx=0x7f292400fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:58:59.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.860+0000 7f29167fc640 1 -- 10.20.193.28:0/2206510807 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f292400dce0 con 0x7f2928153740 2026-02-25T11:58:59.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.860+0000 7f29167fc640 1 -- 10.20.193.28:0/2206510807 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f292400de40 con 0x7f2928153740 2026-02-25T11:58:59.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.860+0000 7f292e56f640 1 -- 10.20.193.28:0/2206510807 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29281686d0 con 0x7f2928153740 2026-02-25T11:58:59.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.860+0000 7f292e56f640 1 -- 10.20.193.28:0/2206510807 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f292814f360 con 0x7f2928153740 2026-02-25T11:58:59.861 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.860+0000 7f29167fc640 1 -- 10.20.193.28:0/2206510807 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f29240116b0 con 0x7f2928153740 2026-02-25T11:58:59.862 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.860+0000 7f292e56f640 1 -- 10.20.193.28:0/2206510807 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29281688f0 con 0x7f2928153740 2026-02-25T11:58:59.864 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.863+0000 7f29167fc640 1 -- 10.20.193.28:0/2206510807 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f2924020060 con 0x7f2928153740 2026-02-25T11:58:59.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.865+0000 7f29167fc640 1 --2- 10.20.193.28:0/2206510807 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f29040b56e0 0x7f29040d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:58:59.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.865+0000 7f29167fc640 1 -- 10.20.193.28:0/2206510807 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f2924015070 con 0x7f2928153740 2026-02-25T11:58:59.866 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.865+0000 7f29167fc640 1 -- 10.20.193.28:0/2206510807 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f2924120940 con 0x7f2928153740 2026-02-25T11:58:59.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.865+0000 7f292d56d640 1 --2- 10.20.193.28:0/2206510807 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f29040b56e0 0x7f29040d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:58:59.867 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:58:59.866+0000 7f292d56d640 1 --2- 10.20.193.28:0/2206510807 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f29040b56e0 0x7f29040d5b90 secure :-1 s=READY pgs=411 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f29180095d0 tx=0x7f2918005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.011 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.010+0000 7f292e56f640 1 -- 10.20.193.28:0/2206510807 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29281632f0 con 0x7f2928153740 2026-02-25T11:59:00.012 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.010+0000 7f29167fc640 1 -- 10.20.193.28:0/2206510807 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f29240aa780 con 0x7f2928153740 2026-02-25T11:59:00.013 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:59:00.013 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:59:00.014 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:59:00.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.012+0000 7f292e56f640 1 -- 10.20.193.28:0/2206510807 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f29040b56e0 msgr2=0x7f29040d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.012+0000 7f292e56f640 1 --2- 10.20.193.28:0/2206510807 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f29040b56e0 0x7f29040d5b90 secure :-1 s=READY pgs=411 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f29180095d0 tx=0x7f2918005f90 comp rx=0 tx=0).stop 2026-02-25T11:59:00.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.012+0000 7f292e56f640 1 -- 10.20.193.28:0/2206510807 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2928153740 msgr2=0x7f2928167aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.014 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.012+0000 7f292e56f640 1 --2- 10.20.193.28:0/2206510807 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2928153740 0x7f2928167aa0 secure :-1 s=READY pgs=937 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f292400f940 tx=0x7f292400fe10 comp rx=0 tx=0).stop 2026-02-25T11:59:00.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.012+0000 7f292e56f640 1 -- 10.20.193.28:0/2206510807 shutdown_connections 2026-02-25T11:59:00.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.012+0000 7f292e56f640 1 --2- 10.20.193.28:0/2206510807 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f29040b56e0 0x7f29040d5b90 unknown :-1 s=CLOSED pgs=411 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.012+0000 7f292e56f640 1 --2- 10.20.193.28:0/2206510807 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f2928153740 0x7f2928167aa0 unknown :-1 s=CLOSED pgs=937 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.012+0000 7f292e56f640 1 --2- 10.20.193.28:0/2206510807 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f292810eb80 0x7f29281626d0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.012+0000 7f292e56f640 1 --2- 10.20.193.28:0/2206510807 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f292810e190 0x7f2928162190 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.012+0000 7f292e56f640 1 -- 10.20.193.28:0/2206510807 >> 10.20.193.28:0/2206510807 conn(0x7f29280e9b40 msgr2=0x7f2928109f20 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.012+0000 7f292e56f640 1 -- 10.20.193.28:0/2206510807 shutdown_connections 2026-02-25T11:59:00.015 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.012+0000 7f292e56f640 1 -- 10.20.193.28:0/2206510807 wait complete. 2026-02-25T11:59:00.114 DEBUG:tasks.check_counter:Getting stats from a 2026-02-25T11:59:00.115 DEBUG:tasks.check_counter:skipping mds.14850 2026-02-25T11:59:00.115 DEBUG:tasks.check_counter:Getting stats from d 2026-02-25T11:59:00.115 DEBUG:tasks.check_counter:skipping mds.14799 2026-02-25T11:59:00.115 DEBUG:tasks.check_counter:Getting stats from g 2026-02-25T11:59:00.115 DEBUG:tasks.check_counter:Getting stats from mds.24731 2026-02-25T11:59:00.115 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24731 perf dump --format=json-pretty 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.236+0000 7f94326c3640 1 -- 10.20.193.28:0/719276860 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f942c1147d0 msgr2=0x7f942c114c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.236+0000 7f94326c3640 1 --2- 10.20.193.28:0/719276860 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f942c1147d0 0x7f942c114c40 secure :-1 s=READY pgs=938 gs=1 cs=0 l=1 c_cookie=9084182437087813255 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9420009950 tx=0x7f942002f190 comp rx=0 tx=0).stop 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.236+0000 7f94326c3640 1 -- 10.20.193.28:0/719276860 shutdown_connections 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.236+0000 7f94326c3640 1 --2- 10.20.193.28:0/719276860 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f942c115180 0x7f942c175b10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.236+0000 7f94326c3640 1 --2- 10.20.193.28:0/719276860 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f942c1147d0 0x7f942c114c40 unknown :-1 s=CLOSED pgs=938 gs=1 cs=0 l=1 c_cookie=9084182437087813255 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.236+0000 7f94326c3640 1 --2- 10.20.193.28:0/719276860 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f942c1138f0 0x7f942c113ce0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.236+0000 7f94326c3640 1 -- 10.20.193.28:0/719276860 >> 10.20.193.28:0/719276860 conn(0x7f942c064570 msgr2=0x7f942c064970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.236+0000 7f94326c3640 1 -- 10.20.193.28:0/719276860 shutdown_connections 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.236+0000 7f94326c3640 1 -- 10.20.193.28:0/719276860 wait complete. 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.237+0000 7f94326c3640 1 Processor -- start 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.237+0000 7f94326c3640 1 -- start start 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.237+0000 7f94326c3640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f942c1138f0 0x7f942c0febe0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.237+0000 7f94326c3640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f942c1147d0 0x7f942c175af0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.237+0000 7f94326c3640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f942c115180 0x7f942c0ff420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.237+0000 7f94326c3640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f942c100080 con 0x7f942c1138f0 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.237+0000 7f94326c3640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f942c1001f0 con 0x7f942c1147d0 2026-02-25T11:59:00.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.237+0000 7f94326c3640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f942c1003b0 con 0x7f942c115180 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.237+0000 7f94316c1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f942c1138f0 0x7f942c0febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.237+0000 7f94316c1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f942c1138f0 0x7f942c0febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34756/0 (socket says 10.20.193.28:34756) 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.237+0000 7f94316c1640 1 -- 10.20.193.28:0/2229942492 learned_addr learned my addr 10.20.193.28:0/2229942492 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.237+0000 7f9430ec0640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f942c1147d0 0x7f942c175af0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.237+0000 7f9431ec2640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f942c115180 0x7f942c0ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.238+0000 7f94316c1640 1 -- 10.20.193.28:0/2229942492 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f942c115180 msgr2=0x7f942c0ff420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.238+0000 7f94316c1640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f942c115180 0x7f942c0ff420 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.238+0000 7f94316c1640 1 -- 10.20.193.28:0/2229942492 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f942c1147d0 msgr2=0x7f942c175af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.238+0000 7f94316c1640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f942c1147d0 0x7f942c175af0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.238+0000 7f94316c1640 1 -- 10.20.193.28:0/2229942492 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9420009570 con 0x7f942c1138f0 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.238+0000 7f94316c1640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f942c1138f0 0x7f942c0febe0 secure :-1 s=READY pgs=939 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f941c005db0 tx=0x7f941c00ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.238+0000 7f941a7fc640 1 -- 10.20.193.28:0/2229942492 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f941c004500 con 0x7f942c1138f0 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.238+0000 7f941a7fc640 1 -- 10.20.193.28:0/2229942492 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f941c004d10 con 0x7f942c1138f0 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.238+0000 7f941a7fc640 1 -- 10.20.193.28:0/2229942492 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f941c005280 con 0x7f942c1138f0 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.238+0000 7f94326c3640 1 -- 10.20.193.28:0/2229942492 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942c100690 con 0x7f942c1138f0 2026-02-25T11:59:00.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.238+0000 7f94326c3640 1 -- 10.20.193.28:0/2229942492 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942c100b10 con 0x7f942c1138f0 2026-02-25T11:59:00.242 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.241+0000 7f941a7fc640 1 -- 10.20.193.28:0/2229942492 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f941c005410 con 0x7f942c1138f0 2026-02-25T11:59:00.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.243+0000 7f941a7fc640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f94080b56e0 0x7f94080d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.243+0000 7f941a7fc640 1 -- 10.20.193.28:0/2229942492 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f941c0e6c40 con 0x7f942c1138f0 2026-02-25T11:59:00.244 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.243+0000 7f9430ec0640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f94080b56e0 0x7f94080d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.243+0000 7f9430ec0640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f94080b56e0 0x7f94080d5b90 secure :-1 s=READY pgs=412 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f94200098a0 tx=0x7f942003a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.243+0000 7f9436048e00 1 Processor -- start 2026-02-25T11:59:00.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f9436048e00 1 -- start start 2026-02-25T11:59:00.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f9436048e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 0x56142c122ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f9436048e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c123220 0x56142c143600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f9436048e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c143b40 0x56142c163f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f9436048e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56142c0a1570 con 0x56142c102900 2026-02-25T11:59:00.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f9436048e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56142c019030 con 0x56142c123220 2026-02-25T11:59:00.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f9436048e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56142c014d60 con 0x56142c143b40 2026-02-25T11:59:00.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f94316c1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 0x56142c122ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f9431ec2640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c143b40 0x56142c163f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f94316c1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 0x56142c122ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34760/0 (socket says 10.20.193.28:34760) 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f94316c1640 1 -- 10.20.193.28:0/3009981245 learned_addr learned my addr 10.20.193.28:0/3009981245 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f9430ec0640 1 --2- 10.20.193.28:0/3009981245 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c123220 0x56142c143600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f94316c1640 1 -- 10.20.193.28:0/3009981245 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c143b40 msgr2=0x56142c163f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f94316c1640 1 --2- 10.20.193.28:0/3009981245 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c143b40 0x56142c163f20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f94316c1640 1 -- 10.20.193.28:0/3009981245 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c123220 msgr2=0x56142c143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f94316c1640 1 --2- 10.20.193.28:0/3009981245 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c123220 0x56142c143600 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f94316c1640 1 -- 10.20.193.28:0/3009981245 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142bfba300 con 0x56142c102900 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f94316c1640 1 --2- 10.20.193.28:0/3009981245 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 0x56142c122ce0 secure :-1 s=READY pgs=940 gs=3 cs=0 l=1 c_cookie=9386196992249163326 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f941c0a9590 tx=0x7f941c0a96e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8242786f2ad9ee3e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.244+0000 7f93f9bfe640 1 -- 10.20.193.28:0/3009981245 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f941c0ac7c0 con 0x56142c102900 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.245+0000 7f93f9bfe640 1 -- 10.20.193.28:0/3009981245 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f941c009940 con 0x56142c102900 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.245+0000 7f93f9bfe640 1 -- 10.20.193.28:0/3009981245 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f941c0ac7c0 con 0x56142c102900 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.245+0000 7f9436048e00 1 -- 10.20.193.28:0/3009981245 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 msgr2=0x56142c122ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.245+0000 7f9436048e00 1 --2- 10.20.193.28:0/3009981245 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 0x56142c122ce0 secure :-1 s=READY pgs=940 gs=3 cs=0 l=1 c_cookie=9386196992249163326 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f941c0a9590 tx=0x7f941c0a96e0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.245+0000 7f9436048e00 1 -- 10.20.193.28:0/3009981245 shutdown_connections 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.245+0000 7f9436048e00 1 --2- 10.20.193.28:0/3009981245 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c143b40 0x56142c163f20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.245+0000 7f9436048e00 1 --2- 10.20.193.28:0/3009981245 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c123220 0x56142c143600 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.245+0000 7f9436048e00 1 --2- 10.20.193.28:0/3009981245 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 0x56142c122ce0 unknown :-1 s=CLOSED pgs=940 gs=3 cs=0 l=1 c_cookie=9386196992249163326 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.245+0000 7f9436048e00 1 -- 10.20.193.28:0/3009981245 >> 10.20.193.28:0/3009981245 conn(0x56142c022180 msgr2=0x56142c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.245+0000 7f9436048e00 1 -- 10.20.193.28:0/3009981245 shutdown_connections 2026-02-25T11:59:00.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.245+0000 7f9436048e00 1 -- 10.20.193.28:0/3009981245 wait complete. 2026-02-25T11:59:00.247 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.246+0000 7f9436048e00 1 -- start start 2026-02-25T11:59:00.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.246+0000 7f9436048e00 1 Processor -- start 2026-02-25T11:59:00.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9436048e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c102900 0x56142c163f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9436048e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c123220 0x56142c13cfd0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9436048e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c143b40 0x56142c13d6c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9436048e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56142bcac690 con 0x56142c123220 2026-02-25T11:59:00.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9436048e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56142c13e0c0 con 0x56142c143b40 2026-02-25T11:59:00.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9436048e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56142c13e280 con 0x56142c102900 2026-02-25T11:59:00.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9430ec0640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c123220 0x56142c13cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9430ec0640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c123220 0x56142c13cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34772/0 (socket says 10.20.193.28:34772) 2026-02-25T11:59:00.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9430ec0640 1 -- 10.20.193.28:0/777321181 learned_addr learned my addr 10.20.193.28:0/777321181 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f94316c1640 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c102900 0x56142c163f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.248 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9431ec2640 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c143b40 0x56142c13d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9430ec0640 1 -- 10.20.193.28:0/777321181 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c102900 msgr2=0x56142c163f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9430ec0640 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c102900 0x56142c163f20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9430ec0640 1 -- 10.20.193.28:0/777321181 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c143b40 msgr2=0x56142c13d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9430ec0640 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c143b40 0x56142c13d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9430ec0640 1 -- 10.20.193.28:0/777321181 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142bf926c0 con 0x56142c123220 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9431ec2640 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c143b40 0x56142c13d6c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f9430ec0640 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c123220 0x56142c13cfd0 secure :-1 s=READY pgs=941 gs=2 cs=0 l=1 c_cookie=3521378215472964698 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9420002fd0 tx=0x7f942002f190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30de72d47510a45a server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.247+0000 7f93e6ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.248+0000 7f93e6ffd640 1 -- 10.20.193.28:0/777321181 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9420038c50 con 0x56142c123220 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.248+0000 7f93e6ffd640 1 -- 10.20.193.28:0/777321181 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f942003e040 con 0x56142c123220 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.248+0000 7f93e6ffd640 1 -- 10.20.193.28:0/777321181 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9420044690 con 0x56142c123220 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.248+0000 7f9436048e00 1 -- 10.20.193.28:0/777321181 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942c100b10 con 0x56142c123220 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.248+0000 7f9436048e00 10 client.17769 _ll_register_callbacks cb 0x56142bf996b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:59:00.249 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.248+0000 7f9436048e00 1 -- 10.20.193.28:0/777321181 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56142bcf19e0 con 0x56142c123220 2026-02-25T11:59:00.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.248+0000 7f93e6ffd640 1 -- 10.20.193.28:0/777321181 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9420038db0 con 0x56142c123220 2026-02-25T11:59:00.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.248+0000 7f93e6ffd640 1 -- 10.20.193.28:0/777321181 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9420044820 con 0x56142c123220 2026-02-25T11:59:00.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.248+0000 7f9436048e00 10 client.17769 fetch_fsmap learned FSMap version 23 2026-02-25T11:59:00.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.248+0000 7f9436048e00 1 -- 10.20.193.28:0/777321181 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x56142bfa4810 con 0x56142c123220 2026-02-25T11:59:00.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.249+0000 7f93e6ffd640 1 -- 10.20.193.28:0/777321181 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9420044a00 con 0x56142c123220 2026-02-25T11:59:00.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.249+0000 7f9436048e00 10 client.17769 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:59:00.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.249+0000 7f9436048e00 10 client.17769 resolve_mds: validated gid 24731 aka daemon mds.g 2026-02-25T11:59:00.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.249+0000 7f9436048e00 20 client.17769 populate_metadata read hostname 'trial028' 2026-02-25T11:59:00.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.249+0000 7f9436048e00 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x56142c1412e0 0x56142c141780 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.249+0000 7f9436048e00 4 client.17769 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:59:00.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.249+0000 7f9436048e00 1 -- 10.20.193.28:0/777321181 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56142bfe1860 con 0x56142c1412e0 2026-02-25T11:59:00.250 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.249+0000 7f94316c1640 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x56142c1412e0 0x56142c141780 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.249+0000 7f94316c1640 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x56142c1412e0 0x56142c141780 crc :-1 s=READY pgs=66 gs=4 cs=0 l=0 c_cookie=15254851410994681919 s_cookie=6878764296170417691 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3b41e88ddb4583f server_cookie=5f7647ae9ee0ae1b in_seq=0 out_seq=0 2026-02-25T11:59:00.251 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.249+0000 7f93e6ffd640 10 client.17769 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:59:00.252 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f93e6ffd640 1 -- 10.20.193.28:0/777321181 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x56142bfe1860 con 0x56142c1412e0 2026-02-25T11:59:00.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f93e6ffd640 10 client.17769 handle_command_reply: tid=0 2026-02-25T11:59:00.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f9436048e00 1 client.17769 shutdown 2026-02-25T11:59:00.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f9436048e00 1 -- 10.20.193.28:0/777321181 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c123220 msgr2=0x56142c13cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f9436048e00 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c123220 0x56142c13cfd0 secure :-1 s=READY pgs=941 gs=2 cs=0 l=1 c_cookie=3521378215472964698 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9420002fd0 tx=0x7f942002f190 comp rx=0 tx=0).stop 2026-02-25T11:59:00.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f9436048e00 1 -- 10.20.193.28:0/777321181 shutdown_connections 2026-02-25T11:59:00.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f9436048e00 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x56142c1412e0 0x56142c141780 crc :-1 s=READY pgs=66 gs=4 cs=0 l=0 c_cookie=15254851410994681919 s_cookie=6878764296170417691 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f9436048e00 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c143b40 0x56142c13d6c0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f9436048e00 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c123220 0x56142c13cfd0 unknown :-1 s=CLOSED pgs=941 gs=2 cs=0 l=1 c_cookie=3521378215472964698 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f9436048e00 1 --2- 10.20.193.28:0/777321181 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c102900 0x56142c163f20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f9436048e00 1 -- 10.20.193.28:0/777321181 >> 10.20.193.28:0/777321181 conn(0x56142c022180 msgr2=0x56142c1244f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.253 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f93e6ffd640 0 client.17769 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:59:00.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f9436048e00 1 -- 10.20.193.28:0/777321181 shutdown_connections 2026-02-25T11:59:00.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f9436048e00 1 -- 10.20.193.28:0/777321181 wait complete. 2026-02-25T11:59:00.254 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.250+0000 7f9436048e00 20 client.17769 trim_cache size 0 max 16384 2026-02-25T11:59:00.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f9436048e00 1 Processor -- start 2026-02-25T11:59:00.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f9436048e00 1 -- start start 2026-02-25T11:59:00.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f9436048e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 0x56142c13a6e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f9436048e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c123220 0x56142c127610 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f9436048e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c143b40 0x56142c13b220 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.262 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f9436048e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56142c12f290 con 0x56142c102900 2026-02-25T11:59:00.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f9436048e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56142c00ed00 con 0x56142c123220 2026-02-25T11:59:00.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f9436048e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56142c00c930 con 0x56142c143b40 2026-02-25T11:59:00.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f94316c1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 0x56142c13a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f94316c1640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 0x56142c13a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34776/0 (socket says 10.20.193.28:34776) 2026-02-25T11:59:00.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f94316c1640 1 -- 10.20.193.28:0/3522951504 learned_addr learned my addr 10.20.193.28:0/3522951504 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f9431ec2640 1 --2- 10.20.193.28:0/3522951504 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c143b40 0x56142c13b220 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f9430ec0640 1 --2- 10.20.193.28:0/3522951504 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c123220 0x56142c127610 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f94316c1640 1 -- 10.20.193.28:0/3522951504 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c143b40 msgr2=0x56142c13b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f94316c1640 1 --2- 10.20.193.28:0/3522951504 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c143b40 0x56142c13b220 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f94316c1640 1 -- 10.20.193.28:0/3522951504 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c123220 msgr2=0x56142c127610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f94316c1640 1 --2- 10.20.193.28:0/3522951504 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c123220 0x56142c127610 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.263 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f94316c1640 1 -- 10.20.193.28:0/3522951504 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142bf9f0b0 con 0x56142c102900 2026-02-25T11:59:00.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.259+0000 7f94316c1640 1 --2- 10.20.193.28:0/3522951504 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 0x56142c13a6e0 secure :-1 s=READY pgs=942 gs=1 cs=0 l=1 c_cookie=3777262148043029867 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f941c0acf50 tx=0x7f941c0b8100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346b87e92da23d6b server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.260+0000 7f93f93fd640 1 -- 10.20.193.28:0/3522951504 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f941c009940 con 0x56142c102900 2026-02-25T11:59:00.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.260+0000 7f93f93fd640 1 -- 10.20.193.28:0/3522951504 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f941c0ad9c0 con 0x56142c102900 2026-02-25T11:59:00.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.260+0000 7f93f93fd640 1 -- 10.20.193.28:0/3522951504 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f941c0d20b0 con 0x56142c102900 2026-02-25T11:59:00.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.260+0000 7f9436048e00 1 -- 10.20.193.28:0/3522951504 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 msgr2=0x56142c13a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.260+0000 7f9436048e00 1 --2- 10.20.193.28:0/3522951504 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 0x56142c13a6e0 secure :-1 s=READY pgs=942 gs=1 cs=0 l=1 c_cookie=3777262148043029867 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f941c0acf50 tx=0x7f941c0b8100 comp rx=0 tx=0).stop 2026-02-25T11:59:00.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.260+0000 7f9436048e00 1 -- 10.20.193.28:0/3522951504 shutdown_connections 2026-02-25T11:59:00.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.260+0000 7f9436048e00 1 --2- 10.20.193.28:0/3522951504 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c143b40 0x56142c13b220 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.260+0000 7f9436048e00 1 --2- 10.20.193.28:0/3522951504 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c123220 0x56142c127610 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.260+0000 7f9436048e00 1 --2- 10.20.193.28:0/3522951504 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c102900 0x56142c13a6e0 unknown :-1 s=CLOSED pgs=942 gs=1 cs=0 l=1 c_cookie=3777262148043029867 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.260+0000 7f9436048e00 1 -- 10.20.193.28:0/3522951504 >> 10.20.193.28:0/3522951504 conn(0x56142c022180 msgr2=0x56142c138fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.264 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.260+0000 7f9436048e00 1 -- 10.20.193.28:0/3522951504 shutdown_connections 2026-02-25T11:59:00.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.260+0000 7f9436048e00 1 -- 10.20.193.28:0/3522951504 wait complete. 2026-02-25T11:59:00.265 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9436048e00 1 -- start start 2026-02-25T11:59:00.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9436048e00 1 Processor -- start 2026-02-25T11:59:00.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9436048e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c102900 0x56142c1079c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9436048e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c123220 0x56142c107f00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9436048e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c143b40 0x56142c108440 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9436048e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56142c00ed00 con 0x56142c123220 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9436048e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x56142c00c930 con 0x56142c143b40 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9436048e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56142c12a140 con 0x56142c102900 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9430ec0640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c123220 0x56142c107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f94316c1640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c102900 0x56142c1079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f94316c1640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c102900 0x56142c1079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:49140/0 (socket says 10.20.193.28:49140) 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f94316c1640 1 -- 10.20.193.28:0/456456918 learned_addr learned my addr 10.20.193.28:0/456456918 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9431ec2640 1 --2- 10.20.193.28:0/456456918 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c143b40 0x56142c108440 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9430ec0640 1 -- 10.20.193.28:0/456456918 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c102900 msgr2=0x56142c1079c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9430ec0640 1 --2- 10.20.193.28:0/456456918 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c102900 0x56142c1079c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9430ec0640 1 -- 10.20.193.28:0/456456918 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c143b40 msgr2=0x56142c108440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9430ec0640 1 --2- 10.20.193.28:0/456456918 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c143b40 0x56142c108440 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f9430ec0640 1 -- 10.20.193.28:0/456456918 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142bf929f0 con 0x56142c123220 2026-02-25T11:59:00.269 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.261+0000 7f94316c1640 1 --2- 10.20.193.28:0/456456918 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c102900 0x56142c1079c0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2026-02-25T11:59:00.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f9430ec0640 1 --2- 10.20.193.28:0/456456918 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c123220 0x56142c107f00 secure :-1 s=READY pgs=943 gs=3 cs=0 l=1 c_cookie=12774404104677459981 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9420042ca0 tx=0x7f9420005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b147ccd9fc97880d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f93e6ffd640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:59:00.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f93e6ffd640 1 -- 10.20.193.28:0/456456918 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f942002fe90 con 0x56142c123220 2026-02-25T11:59:00.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f93e6ffd640 1 -- 10.20.193.28:0/456456918 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f942003e040 con 0x56142c123220 2026-02-25T11:59:00.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f93e6ffd640 1 -- 10.20.193.28:0/456456918 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f9420044670 con 0x56142c123220 2026-02-25T11:59:00.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f9436048e00 1 -- 10.20.193.28:0/456456918 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56142bfba300 con 0x56142c123220 2026-02-25T11:59:00.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f93e6ffd640 1 -- 10.20.193.28:0/456456918 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f9420037e80 con 0x56142c123220 2026-02-25T11:59:00.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f9436048e00 10 client.17775 _ll_register_callbacks cb 0x56142bf996b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:59:00.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f9436048e00 1 -- 10.20.193.28:0/456456918 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56142c01c470 con 0x56142c123220 2026-02-25T11:59:00.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f93e6ffd640 1 -- 10.20.193.28:0/456456918 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f94200377f0 con 0x56142c123220 2026-02-25T11:59:00.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f9436048e00 10 client.17775 fetch_fsmap learned FSMap version 23 2026-02-25T11:59:00.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f9436048e00 1 -- 10.20.193.28:0/456456918 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x56142bfa4a70 con 0x56142c123220 2026-02-25T11:59:00.270 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f93e6ffd640 1 -- 10.20.193.28:0/456456918 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f9420037950 con 0x56142c123220 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f9436048e00 10 client.17775 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.262+0000 7f9436048e00 10 client.17775 resolve_mds: validated gid 24731 aka daemon mds.g 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.263+0000 7f9436048e00 20 client.17775 populate_metadata read hostname 'trial028' 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.263+0000 7f9436048e00 1 --2- 10.20.193.28:0/456456918 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x56142c1f1360 0x56142c211810 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.263+0000 7f9436048e00 4 client.17775 mds_command: new command op to 24731 tid=0 multi_id=0 [{"prefix": "perf dump", "format": "json-pretty"}] 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.263+0000 7f9436048e00 1 -- 10.20.193.28:0/456456918 --> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] -- command(tid 0: {"prefix": "perf dump", "format": "json-pretty"}) -- 0x56142bd3f740 con 0x56142c1f1360 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.263+0000 7f94316c1640 1 --2- 10.20.193.28:0/456456918 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x56142c1f1360 0x56142c211810 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.263+0000 7f94316c1640 1 --2- 10.20.193.28:0/456456918 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x56142c1f1360 0x56142c211810 crc :-1 s=READY pgs=67 gs=4 cs=0 l=0 c_cookie=4934475518742393085 s_cookie=17801842109105310969 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=447ac7571f8070fd server_cookie=f70cd9147cd00cf9 in_seq=0 out_seq=0 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.263+0000 7f93e6ffd640 10 client.17775 ms_handle_connect on v2:10.20.193.28:6838/3137382094 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f93e6ffd640 1 -- 10.20.193.28:0/456456918 <== mds.1 v2:10.20.193.28:6838/3137382094 1 ==== command_reply(tid 0: 0 ) ==== 8+0+18291 (crc 0 0 0) 0x56142bd3f740 con 0x56142c1f1360 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f93e6ffd640 10 client.17775 handle_command_reply: tid=0 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f9436048e00 1 client.17775 shutdown 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f9436048e00 1 -- 10.20.193.28:0/456456918 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c123220 msgr2=0x56142c107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.271 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f9436048e00 1 --2- 10.20.193.28:0/456456918 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c123220 0x56142c107f00 secure :-1 s=READY pgs=943 gs=3 cs=0 l=1 c_cookie=12774404104677459981 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f9420042ca0 tx=0x7f9420005900 comp rx=0 tx=0).stop 2026-02-25T11:59:00.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f9436048e00 1 -- 10.20.193.28:0/456456918 shutdown_connections 2026-02-25T11:59:00.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f9436048e00 1 --2- 10.20.193.28:0/456456918 >> [v2:10.20.193.28:6838/3137382094,v1:10.20.193.28:6839/3137382094] conn(0x56142c1f1360 0x56142c211810 crc :-1 s=READY pgs=67 gs=4 cs=0 l=0 c_cookie=4934475518742393085 s_cookie=17801842109105310969 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f9436048e00 1 --2- 10.20.193.28:0/456456918 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x56142c143b40 0x56142c108440 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f9436048e00 1 --2- 10.20.193.28:0/456456918 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x56142c123220 0x56142c107f00 unknown :-1 s=CLOSED pgs=943 gs=3 cs=0 l=1 c_cookie=12774404104677459981 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f9436048e00 1 --2- 10.20.193.28:0/456456918 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x56142c102900 0x56142c1079c0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f93e6ffd640 0 client.17775 ms_handle_reset on v2:10.20.193.28:6838/3137382094 2026-02-25T11:59:00.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f9436048e00 1 -- 10.20.193.28:0/456456918 >> 10.20.193.28:0/456456918 conn(0x56142c022180 msgr2=0x56142c1397b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f9436048e00 1 -- 10.20.193.28:0/456456918 shutdown_connections 2026-02-25T11:59:00.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f9436048e00 1 -- 10.20.193.28:0/456456918 wait complete. 2026-02-25T11:59:00.272 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.264+0000 7f9436048e00 20 client.17775 trim_cache size 0 max 16384 2026-02-25T11:59:00.273 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:59:00.273 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:59:00.273 INFO:teuthology.orchestra.run.trial028.stdout: "AsyncMessenger::Worker-0": { 2026-02-25T11:59:00.273 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_messages": 312, 2026-02-25T11:59:00.273 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages": 304, 2026-02-25T11:59:00.273 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_bytes": 583305, 2026-02-25T11:59:00.273 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_bytes": 205878, 2026-02-25T11:59:00.273 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_created_connections": 11, 2026-02-25T11:59:00.273 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_active_connections": 5, 2026-02-25T11:59:00.273 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_total_time": 0.043483816, 2026-02-25T11:59:00.273 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_send_time": 0.017104632, 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_recv_time": 0.023096374, 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_fast_dispatch_time": 0.000647936, 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages_queue_lat": { 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 304, 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.044011377, 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000144774 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_handle_ack_lat": { 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 131, 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000017912, 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000136 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_encrypted_bytes": 557664, 2026-02-25T11:59:00.274 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_encrypted_bytes": 123456 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: "AsyncMessenger::Worker-1": { 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_messages": 818, 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages": 815, 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_bytes": 202216, 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_bytes": 603739, 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_created_connections": 6, 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_active_connections": 4, 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_total_time": 0.068386811, 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_send_time": 0.041824063, 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_recv_time": 0.033635524, 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_fast_dispatch_time": 0.000265583, 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages_queue_lat": { 2026-02-25T11:59:00.275 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 816, 2026-02-25T11:59:00.276 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.028434353, 2026-02-25T11:59:00.276 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000034846 2026-02-25T11:59:00.276 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.276 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_handle_ack_lat": { 2026-02-25T11:59:00.276 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 1408, 2026-02-25T11:59:00.276 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000123460, 2026-02-25T11:59:00.276 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000087 2026-02-25T11:59:00.276 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.276 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_encrypted_bytes": 160, 2026-02-25T11:59:00.276 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_encrypted_bytes": 366592 2026-02-25T11:59:00.276 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.276 INFO:teuthology.orchestra.run.trial028.stdout: "AsyncMessenger::Worker-2": { 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_messages": 59, 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages": 61, 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_bytes": 10546, 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_bytes": 574947, 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_created_connections": 50, 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_active_connections": 7, 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_total_time": 0.025432506, 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_send_time": 0.003783003, 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_recv_time": 0.018867645, 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_fast_dispatch_time": 0.000144808, 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages_queue_lat": { 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 58, 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.031986713, 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000551495 2026-02-25T11:59:00.277 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_handle_ack_lat": { 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 44, 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000001573, 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000035 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_encrypted_bytes": 0, 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_encrypted_bytes": 0 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: "cct": { 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: "total_workers": 1, 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: "unhealthy_workers": 0 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: "finisher-MDSRank": { 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: "queue_len": 0, 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: "complete_latency": { 2026-02-25T11:59:00.278 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 43, 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.002942179, 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000068422 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: "finisher-PurgeQueue": { 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: "queue_len": 0, 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: "complete_latency": { 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 2, 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000018605, 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000009302 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: "mds": { 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: "request": 4, 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: "reply": 3, 2026-02-25T11:59:00.279 INFO:teuthology.orchestra.run.trial028.stdout: "reply_latency": { 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 3, 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.003234778, 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.001078259 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "slow_reply": 0, 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "forward": 1, 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "dir_fetch_complete": 0, 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "dir_fetch_keys": 0, 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "dir_commit": 11, 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "dir_split": 0, 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "dir_merge": 0, 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "inodes": 16, 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_top": 0, 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_bottom": 16, 2026-02-25T11:59:00.280 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_pin_tail": 0, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_pinned": 13, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_expired": 4, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_with_caps": 3, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "caps": 3, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "subtrees": 3, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "traverse": 14, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_hit": 9, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_forward": 1, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_discover": 4, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_dir_fetch": 0, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_remote_ino": 0, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_lock": 0, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "load_cent": 1000, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "q": 1, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "exported": 0, 2026-02-25T11:59:00.281 INFO:teuthology.orchestra.run.trial028.stdout: "exported_inodes": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "imported": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "imported_inodes": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "openino_dir_fetch": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "openino_backtrace_fetch": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "openino_peer_discover": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "root_rfiles": 5, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "root_rbytes": 4330, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "root_rsnaps": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_backtrace_fetch": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_set_tag": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_backtrace_repaired": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_inotable_repaired": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_dir_inodes": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_dir_base_inodes": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_dirfrag_rstats": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_file_inodes": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "handle_inode_file_caps": 0, 2026-02-25T11:59:00.282 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_revoke": 2, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_grant": 0, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_trunc": 0, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_flushsnap_ack": 0, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_flush_ack": 0, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_caps": 2, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_caps_dirty": 0, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_cap_release": 0, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "process_request_cap_release": 0 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "mds_cache": { 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "dir_update": 0, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "dir_update_receipt": 14, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "dir_try_discover": 9, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "dir_send_discover": 10, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "dir_handle_discover": 0, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "num_strays": 0, 2026-02-25T11:59:00.283 INFO:teuthology.orchestra.run.trial028.stdout: "num_strays_delayed": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "num_strays_enqueuing": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "strays_created": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "strays_enqueued": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "strays_reintegrated": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "strays_migrated": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "num_recovering_processing": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "num_recovering_enqueued": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "num_recovering_prioritized": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "recovery_started": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "recovery_completed": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_quiesce_path": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_quiesce_inode": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_enqueue_scrub": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_exportdir": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_flush": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_fragmentdir": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_fragstats": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_inodestats": 0, 2026-02-25T11:59:00.284 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_succeeded": 0, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_write_failed": 0 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "mds_log": { 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "evlrg": 0, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "evadd": 3, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "evex": 0, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "evtrm": 0, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "ev": 3, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "evexg": 0, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "evexd": 0, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "segadd": 2, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "segex": 0, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "segtrm": 0, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "seg": 2, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "segmjr": 2, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "segexg": 0, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "segexd": 0, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "expos": 4194304, 2026-02-25T11:59:00.285 INFO:teuthology.orchestra.run.trial028.stdout: "wrpos": 4195405, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "rdpos": 0, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "jlat": { 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 2, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.002308345, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.001154172 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "replayed": 0 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "mds_mem": { 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "ino": 20, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "ino+": 22, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "ino-": 2, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "dir": 19, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "dir+": 23, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "dir-": 4, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "dn": 16, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "dn+": 20, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "dn-": 4, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "cap": 3, 2026-02-25T11:59:00.286 INFO:teuthology.orchestra.run.trial028.stdout: "cap+": 3, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "cap-": 0, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "rss": 49276, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "heap": 260804608 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "mds_server": { 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "dispatch_client_request": 11, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "dispatch_server_request": 0, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_request": 4, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_session": 9, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "handle_peer_request": 3, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "req_create_latency": { 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "req_getattr_latency": { 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 3, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.003234778, 2026-02-25T11:59:00.287 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.001078259 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "req_getfilelock_latency": { 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "req_link_latency": { 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookup_latency": { 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookuphash_latency": { 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.288 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookupino_latency": { 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookupname_latency": { 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookupparent_latency": { 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookupsnap_latency": { 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.289 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "req_lssnap_latency": { 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "req_mkdir_latency": { 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "req_mknod_latency": { 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "req_mksnap_latency": { 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.290 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "req_open_latency": { 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "req_readdir_latency": { 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "req_rename_latency": { 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "req_renamesnap_latency": { 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.291 INFO:teuthology.orchestra.run.trial028.stdout: "req_snapdiff_latency": { 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "req_rmdir_latency": { 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "req_rmsnap_latency": { 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "req_rmxattr_latency": { 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "req_setattr_latency": { 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.292 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "req_setdirlayout_latency": { 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "req_setfilelock_latency": { 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "req_setlayout_latency": { 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "req_setxattr_latency": { 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.293 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "req_symlink_latency": { 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "req_unlink_latency": { 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "cap_revoke_eviction": 0, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "cache_trim_throttle": 0, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "session_recall_throttle": 0, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "session_recall_throttle2o": 0, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "global_recall_throttle": 0, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "cap_acquisition_throttle": 0, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "req_getvxattr_latency": { 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "req_blockdiff_latency": { 2026-02-25T11:59:00.294 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "mds_sessions": { 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "session_count": 1, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "session_add": 1, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "session_remove": 0, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "sessions_open": 1, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "sessions_stale": 0, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "total_load": 2, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "average_load": 2, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "avg_session_uptime": 672, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "mdthresh_evicted": 0 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "mempool": { 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "bloom_filter_bytes": 0, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "bloom_filter_items": 0, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_alloc_bytes": 0, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_alloc_items": 0, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_data_bytes": 0, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_data_items": 0, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_onode_bytes": 0, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_onode_items": 0, 2026-02-25T11:59:00.295 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_meta_bytes": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_meta_items": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_other_bytes": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_other_items": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_buffer_bytes": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_buffer_items": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_extent_bytes": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_extent_items": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_blob_bytes": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_blob_items": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_shared_blob_bytes": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_shared_blob_items": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_inline_bl_bytes": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_inline_bl_items": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_fsck_bytes": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_fsck_items": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_txc_bytes": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_txc_items": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_writing_deferred_bytes": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_writing_deferred_items": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_writing_bytes": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_writing_items": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_bytes": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_items": 0, 2026-02-25T11:59:00.296 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_file_reader_bytes": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_file_reader_items": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_file_writer_bytes": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_file_writer_items": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "buffer_anon_bytes": 238636, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "buffer_anon_items": 77, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "buffer_meta_bytes": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "buffer_meta_items": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "osd_bytes": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "osd_items": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "osd_mapbl_bytes": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "osd_mapbl_items": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "osd_pglog_bytes": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "osd_pglog_items": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_bytes": 7504, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_items": 153, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_mapping_bytes": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_mapping_items": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "pgmap_bytes": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "pgmap_items": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "mds_co_bytes": 84748, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "mds_co_items": 452, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "ec_extent_cache_bytes": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "ec_extent_cache_items": 0, 2026-02-25T11:59:00.297 INFO:teuthology.orchestra.run.trial028.stdout: "unittest_1_bytes": 0, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "unittest_1_items": 0, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "unittest_2_bytes": 0, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "unittest_2_items": 0 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "objecter": { 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "op_active": 0, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "op_laggy": 0, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "op_send": 38, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "op_send_bytes": 11714, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "op_resend": 27, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "op_reply": 38, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "op_latency": { 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 38, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.155995657, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.004105148 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "op_inflight": 0, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "oplen_avg": { 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 61, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 38 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "op": 38, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "op_r": 0, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "op_w": 38, 2026-02-25T11:59:00.298 INFO:teuthology.orchestra.run.trial028.stdout: "op_rmw": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "op_pg": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_stat": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_create": 10, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_read": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_write": 2, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_writefull": 6, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_writesame": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_append": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_zero": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_truncate": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_delete": 6, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_mapext": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_sparse_read": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_clonerange": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_getxattr": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_setxattr": 20, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_cmpxattr": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_rmxattr": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_resetxattrs": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_call": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_watch": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_notify": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_src_cmpxattr": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_pgls": 0, 2026-02-25T11:59:00.299 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_pgls_filter": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_other": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "linger_active": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "linger_send": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "linger_resend": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "linger_ping": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "poolop_active": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "poolop_send": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "poolop_resend": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "poolstat_active": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "poolstat_send": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "poolstat_resend": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "statfs_active": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "statfs_send": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "statfs_resend": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "command_active": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "command_send": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "command_resend": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "map_epoch": 88, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "map_full": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "map_inc": 2, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "osd_sessions": 9, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "osd_session_open": 9, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "osd_session_close": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "osd_laggy": 0, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "omap_wr": 16, 2026-02-25T11:59:00.300 INFO:teuthology.orchestra.run.trial028.stdout: "omap_rd": 0, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "omap_del": 1, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "replica_read_sent": 0, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "replica_read_bounced": 0, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "replica_read_completed": 0 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "oft": { 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "omap_total_objs": 1, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "omap_total_kv_pairs": 0, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "omap_total_updates": 0, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "omap_total_removes": 0 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "purge_queue": { 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executing_ops": 0, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executing_ops_high_water": 0, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executing": 0, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executing_high_water": 0, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executed_ops": 0, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executed": 0, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "pq_item_in_journal": 0 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-msgr_dispatch_throttler-mds": { 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "max": 104857600, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "get": 1189, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 677331, 2026-02-25T11:59:00.301 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 1189, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "put": 1189, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 677331, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-objecter_bytes": { 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "max": 104857600, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "get": 38, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 11714, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 38, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "put": 38, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 11714, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.302 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-objecter_ops": { 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "max": 1024, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "get": 38, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 38, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 38, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "put": 38, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 38, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-write_buf_throttle": { 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "max": 3758096384, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "get": 0, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 0, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 0, 2026-02-25T11:59:00.303 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "put": 0, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 0, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-write_buf_throttle-0xd428780": { 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "max": 3758096384, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "get": 3, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 1101, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 3, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "put": 2, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 1101, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.304 INFO:teuthology.orchestra.run.trial028.stdout: "trackedop": { 2026-02-25T11:59:00.305 INFO:teuthology.orchestra.run.trial028.stdout: "slow_ops_count": 0 2026-02-25T11:59:00.305 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.305 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:59:00.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.265+0000 7f94326c3640 1 -- 10.20.193.28:0/2229942492 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f94080b56e0 msgr2=0x7f94080d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.265+0000 7f94326c3640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f94080b56e0 0x7f94080d5b90 secure :-1 s=READY pgs=412 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f94200098a0 tx=0x7f942003a040 comp rx=0 tx=0).stop 2026-02-25T11:59:00.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.265+0000 7f94326c3640 1 -- 10.20.193.28:0/2229942492 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f942c1138f0 msgr2=0x7f942c0febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.265+0000 7f94326c3640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f942c1138f0 0x7f942c0febe0 secure :-1 s=READY pgs=939 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f941c005db0 tx=0x7f941c00ddf0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.305 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.265+0000 7f94326c3640 1 -- 10.20.193.28:0/2229942492 shutdown_connections 2026-02-25T11:59:00.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.265+0000 7f94326c3640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f94080b56e0 0x7f94080d5b90 unknown :-1 s=CLOSED pgs=412 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.265+0000 7f94326c3640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f942c115180 0x7f942c0ff420 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.265+0000 7f94326c3640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f942c1147d0 0x7f942c175af0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.265+0000 7f94326c3640 1 --2- 10.20.193.28:0/2229942492 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f942c1138f0 0x7f942c0febe0 unknown :-1 s=CLOSED pgs=939 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.265+0000 7f94326c3640 1 -- 10.20.193.28:0/2229942492 >> 10.20.193.28:0/2229942492 conn(0x7f942c064570 msgr2=0x7f942c0f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.265+0000 7f94326c3640 1 -- 10.20.193.28:0/2229942492 shutdown_connections 2026-02-25T11:59:00.306 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.265+0000 7f94326c3640 1 -- 10.20.193.28:0/2229942492 wait complete. 2026-02-25T11:59:00.306 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2026-02-25T11:59:00.306 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2026-02-25T11:59:00.306 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2026-02-25T11:59:00.306 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=14 2026-02-25T11:59:00.306 DEBUG:tasks.check_counter:Getting stats from j 2026-02-25T11:59:00.306 DEBUG:tasks.check_counter:skipping mds.14787 2026-02-25T11:59:00.306 DEBUG:tasks.check_counter:Getting stats from b 2026-02-25T11:59:00.306 DEBUG:tasks.check_counter:Getting stats from mds.24749 2026-02-25T11:59:00.307 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24749 perf dump --format=json-pretty 2026-02-25T11:59:00.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.426+0000 7fd548a83640 1 -- 10.20.193.28:0/936315816 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd5441147d0 msgr2=0x7fd544114c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.428 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.426+0000 7fd548a83640 1 --2- 10.20.193.28:0/936315816 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd5441147d0 0x7fd544114c40 secure :-1 s=READY pgs=944 gs=1 cs=0 l=1 c_cookie=11458422677524903475 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd534009950 tx=0x7fd53402f190 comp rx=0 tx=0).stop 2026-02-25T11:59:00.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.426+0000 7fd548a83640 1 -- 10.20.193.28:0/936315816 shutdown_connections 2026-02-25T11:59:00.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.426+0000 7fd548a83640 1 --2- 10.20.193.28:0/936315816 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd544115180 0x7fd544175b10 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.426+0000 7fd548a83640 1 --2- 10.20.193.28:0/936315816 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd5441147d0 0x7fd544114c40 unknown :-1 s=CLOSED pgs=944 gs=1 cs=0 l=1 c_cookie=11458422677524903475 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.426+0000 7fd548a83640 1 --2- 10.20.193.28:0/936315816 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd5441138f0 0x7fd544113ce0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.426+0000 7fd548a83640 1 -- 10.20.193.28:0/936315816 >> 10.20.193.28:0/936315816 conn(0x7fd544064570 msgr2=0x7fd544064970 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.426+0000 7fd548a83640 1 -- 10.20.193.28:0/936315816 shutdown_connections 2026-02-25T11:59:00.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.426+0000 7fd548a83640 1 -- 10.20.193.28:0/936315816 wait complete. 2026-02-25T11:59:00.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.426+0000 7fd548a83640 1 Processor -- start 2026-02-25T11:59:00.429 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd548a83640 1 -- start start 2026-02-25T11:59:00.431 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd548a83640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd5441138f0 0x7fd5440febe0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd548a83640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd5441147d0 0x7fd544175af0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd548a83640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd544115180 0x7fd5440ff420 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd548a83640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd544100080 con 0x7fd5441138f0 2026-02-25T11:59:00.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd548a83640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd5441001f0 con 0x7fd544115180 2026-02-25T11:59:00.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd548a83640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd5441003b0 con 0x7fd5441147d0 2026-02-25T11:59:00.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd5437fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd5441138f0 0x7fd5440febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd5437fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd5441138f0 0x7fd5440febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34800/0 (socket says 10.20.193.28:34800) 2026-02-25T11:59:00.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd5437fe640 1 -- 10.20.193.28:0/529378215 learned_addr learned my addr 10.20.193.28:0/529378215 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd543fff640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd544115180 0x7fd5440ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd542ffd640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd5441147d0 0x7fd544175af0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.432 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd5437fe640 1 -- 10.20.193.28:0/529378215 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd5441147d0 msgr2=0x7fd544175af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd5437fe640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd5441147d0 0x7fd544175af0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd5437fe640 1 -- 10.20.193.28:0/529378215 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd544115180 msgr2=0x7fd5440ff420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd5437fe640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd544115180 0x7fd5440ff420 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.427+0000 7fd5437fe640 1 -- 10.20.193.28:0/529378215 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd534009570 con 0x7fd5441138f0 2026-02-25T11:59:00.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.428+0000 7fd5437fe640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd5441138f0 0x7fd5440febe0 secure :-1 s=READY pgs=945 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd530009480 tx=0x7fd53000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.428+0000 7fd540ff9640 1 -- 10.20.193.28:0/529378215 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd53000cd40 con 0x7fd5441138f0 2026-02-25T11:59:00.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.428+0000 7fd548a83640 1 -- 10.20.193.28:0/529378215 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd544100690 con 0x7fd5441138f0 2026-02-25T11:59:00.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.428+0000 7fd540ff9640 1 -- 10.20.193.28:0/529378215 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd530004d10 con 0x7fd5441138f0 2026-02-25T11:59:00.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.428+0000 7fd540ff9640 1 -- 10.20.193.28:0/529378215 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd5300106b0 con 0x7fd5441138f0 2026-02-25T11:59:00.433 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.428+0000 7fd548a83640 1 -- 10.20.193.28:0/529378215 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd544100b60 con 0x7fd5441138f0 2026-02-25T11:59:00.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.431+0000 7fd540ff9640 1 -- 10.20.193.28:0/529378215 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fd53001f060 con 0x7fd5441138f0 2026-02-25T11:59:00.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.432+0000 7fd540ff9640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd5180b5760 0x7fd5180d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.432+0000 7fd540ff9640 1 -- 10.20.193.28:0/529378215 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd530014070 con 0x7fd5441138f0 2026-02-25T11:59:00.434 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.433+0000 7fd542ffd640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd5180b5760 0x7fd5180d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.433+0000 7fd54c3e9e00 1 Processor -- start 2026-02-25T11:59:00.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.433+0000 7fd542ffd640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd5180b5760 0x7fd5180d5c10 secure :-1 s=READY pgs=413 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd5340098a0 tx=0x7fd53403a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.433+0000 7fd54c3e9e00 1 -- start start 2026-02-25T11:59:00.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.433+0000 7fd54c3e9e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c88ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.433+0000 7fd54c3e9e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9c89220 0x5588a9ca9600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.433+0000 7fd54c3e9e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9ca9b40 0x5588a9cc9f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.437 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.433+0000 7fd54c3e9e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5588a9c07570 con 0x5588a9c68900 2026-02-25T11:59:00.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.433+0000 7fd54c3e9e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5588a9b7f030 con 0x5588a9c89220 2026-02-25T11:59:00.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.433+0000 7fd54c3e9e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5588a9b7ad60 con 0x5588a9ca9b40 2026-02-25T11:59:00.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd5437fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c88ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd5437fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c88ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34812/0 (socket says 10.20.193.28:34812) 2026-02-25T11:59:00.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd5437fe640 1 -- 10.20.193.28:0/3517702664 learned_addr learned my addr 10.20.193.28:0/3517702664 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd543fff640 1 --2- 10.20.193.28:0/3517702664 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9ca9b40 0x5588a9cc9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd542ffd640 1 --2- 10.20.193.28:0/3517702664 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9c89220 0x5588a9ca9600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd5437fe640 1 -- 10.20.193.28:0/3517702664 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9ca9b40 msgr2=0x5588a9cc9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd5437fe640 1 --2- 10.20.193.28:0/3517702664 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9ca9b40 0x5588a9cc9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.438 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd5437fe640 1 -- 10.20.193.28:0/3517702664 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9c89220 msgr2=0x5588a9ca9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd5437fe640 1 --2- 10.20.193.28:0/3517702664 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9c89220 0x5588a9ca9600 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd5437fe640 1 -- 10.20.193.28:0/3517702664 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a9b20300 con 0x5588a9c68900 2026-02-25T11:59:00.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd5437fe640 1 --2- 10.20.193.28:0/3517702664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c88ce0 secure :-1 s=READY pgs=946 gs=1 cs=0 l=1 c_cookie=9216047619336922531 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd5300aa240 tx=0x7fd530004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe5fa7b27a965a3 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd50f7fe640 1 -- 10.20.193.28:0/3517702664 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd5300ac9b0 con 0x5588a9c68900 2026-02-25T11:59:00.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd50f7fe640 1 -- 10.20.193.28:0/3517702664 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd5300ac7d0 con 0x5588a9c68900 2026-02-25T11:59:00.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd50f7fe640 1 -- 10.20.193.28:0/3517702664 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd5300d2350 con 0x5588a9c68900 2026-02-25T11:59:00.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3517702664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 msgr2=0x5588a9c88ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/3517702664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c88ce0 secure :-1 s=READY pgs=946 gs=1 cs=0 l=1 c_cookie=9216047619336922531 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd5300aa240 tx=0x7fd530004f00 comp rx=0 tx=0).stop 2026-02-25T11:59:00.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3517702664 shutdown_connections 2026-02-25T11:59:00.439 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/3517702664 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9ca9b40 0x5588a9cc9f20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/3517702664 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9c89220 0x5588a9ca9600 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/3517702664 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c88ce0 unknown :-1 s=CLOSED pgs=946 gs=1 cs=0 l=1 c_cookie=9216047619336922531 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3517702664 >> 10.20.193.28:0/3517702664 conn(0x5588a9b88180 msgr2=0x5588a9c671e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3517702664 shutdown_connections 2026-02-25T11:59:00.440 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.434+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3517702664 wait complete. 2026-02-25T11:59:00.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.436+0000 7fd54c3e9e00 1 -- start start 2026-02-25T11:59:00.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.436+0000 7fd54c3e9e00 1 Processor -- start 2026-02-25T11:59:00.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.436+0000 7fd54c3e9e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9cc9f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.436+0000 7fd54c3e9e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9c89220 0x5588a9ca2ff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.444 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.436+0000 7fd54c3e9e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9ca9b40 0x5588a9ca36e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.436+0000 7fd54c3e9e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5588a9812690 con 0x5588a9c68900 2026-02-25T11:59:00.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.436+0000 7fd54c3e9e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5588a9ca40e0 con 0x5588a9ca9b40 2026-02-25T11:59:00.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.436+0000 7fd54c3e9e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5588a9ca42a0 con 0x5588a9c89220 2026-02-25T11:59:00.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd5437fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9cc9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd5437fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9cc9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34814/0 (socket says 10.20.193.28:34814) 2026-02-25T11:59:00.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd5437fe640 1 -- 10.20.193.28:0/3331242667 learned_addr learned my addr 10.20.193.28:0/3331242667 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd542ffd640 1 --2- 10.20.193.28:0/3331242667 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9c89220 0x5588a9ca2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd543fff640 1 --2- 10.20.193.28:0/3331242667 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9ca9b40 0x5588a9ca36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd5437fe640 1 -- 10.20.193.28:0/3331242667 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9c89220 msgr2=0x5588a9ca2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd5437fe640 1 --2- 10.20.193.28:0/3331242667 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9c89220 0x5588a9ca2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.445 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd5437fe640 1 -- 10.20.193.28:0/3331242667 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9ca9b40 msgr2=0x5588a9ca36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd5437fe640 1 --2- 10.20.193.28:0/3331242667 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9ca9b40 0x5588a9ca36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd5437fe640 1 -- 10.20.193.28:0/3331242667 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a9af86c0 con 0x5588a9c68900 2026-02-25T11:59:00.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd5437fe640 1 --2- 10.20.193.28:0/3331242667 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9cc9f20 secure :-1 s=READY pgs=947 gs=1 cs=0 l=1 c_cookie=427884427250404623 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd5300054f0 tx=0x7fd5300d2300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0269e9b2e8d0f server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd50cff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:59:00.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd50cff9640 1 -- 10.20.193.28:0/3331242667 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd530005140 con 0x5588a9c68900 2026-02-25T11:59:00.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3331242667 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd544100b60 con 0x5588a9c68900 2026-02-25T11:59:00.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd50cff9640 1 -- 10.20.193.28:0/3331242667 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd5300b2cf0 con 0x5588a9c68900 2026-02-25T11:59:00.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd54c3e9e00 10 client.17787 _ll_register_callbacks cb 0x5588a9aff6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:59:00.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd50cff9640 1 -- 10.20.193.28:0/3331242667 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd530004390 con 0x5588a9c68900 2026-02-25T11:59:00.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3331242667 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5588a98579e0 con 0x5588a9c68900 2026-02-25T11:59:00.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd50cff9640 1 -- 10.20.193.28:0/3331242667 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd5300acb70 con 0x5588a9c68900 2026-02-25T11:59:00.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd50cff9640 1 -- 10.20.193.28:0/3331242667 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd5300acd50 con 0x5588a9c68900 2026-02-25T11:59:00.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.437+0000 7fd54c3e9e00 10 client.17787 fetch_fsmap learned FSMap version 23 2026-02-25T11:59:00.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.438+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3331242667 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5588a9b0a810 con 0x5588a9c68900 2026-02-25T11:59:00.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.438+0000 7fd50cff9640 1 -- 10.20.193.28:0/3331242667 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd5300b7c80 con 0x5588a9c68900 2026-02-25T11:59:00.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.438+0000 7fd54c3e9e00 10 client.17787 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:59:00.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.438+0000 7fd54c3e9e00 10 client.17787 resolve_mds: validated gid 24749 aka daemon mds.b 2026-02-25T11:59:00.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.438+0000 7fd54c3e9e00 20 client.17787 populate_metadata read hostname 'trial028' 2026-02-25T11:59:00.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.438+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/3331242667 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5588a9ca6ae0 0x5588a9ca6f80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.438+0000 7fd54c3e9e00 4 client.17787 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:59:00.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.438+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3331242667 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588a9b47860 con 0x5588a9ca6ae0 2026-02-25T11:59:00.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.438+0000 7fd542ffd640 1 --2- 10.20.193.28:0/3331242667 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5588a9ca6ae0 0x5588a9ca6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.439+0000 7fd542ffd640 1 --2- 10.20.193.28:0/3331242667 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5588a9ca6ae0 0x5588a9ca6f80 crc :-1 s=READY pgs=188 gs=4 cs=0 l=0 c_cookie=5342641498047104031 s_cookie=8170791000018416169 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a24e022cd5bc41f server_cookie=71647b1fb20e7229 in_seq=0 out_seq=0 2026-02-25T11:59:00.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.439+0000 7fd50cff9640 10 client.17787 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:59:00.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd50cff9640 1 -- 10.20.193.28:0/3331242667 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x5588a9b47860 con 0x5588a9ca6ae0 2026-02-25T11:59:00.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd50cff9640 10 client.17787 handle_command_reply: tid=0 2026-02-25T11:59:00.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd54c3e9e00 1 client.17787 shutdown 2026-02-25T11:59:00.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3331242667 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 msgr2=0x5588a9cc9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/3331242667 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9cc9f20 secure :-1 s=READY pgs=947 gs=1 cs=0 l=1 c_cookie=427884427250404623 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd5300054f0 tx=0x7fd5300d2300 comp rx=0 tx=0).stop 2026-02-25T11:59:00.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3331242667 shutdown_connections 2026-02-25T11:59:00.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/3331242667 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5588a9ca6ae0 0x5588a9ca6f80 crc :-1 s=READY pgs=188 gs=4 cs=0 l=0 c_cookie=5342641498047104031 s_cookie=8170791000018416169 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/3331242667 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9ca9b40 0x5588a9ca36e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/3331242667 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9c89220 0x5588a9ca2ff0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/3331242667 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9cc9f20 unknown :-1 s=CLOSED pgs=947 gs=1 cs=0 l=1 c_cookie=427884427250404623 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd50cff9640 0 client.17787 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:59:00.448 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3331242667 >> 10.20.193.28:0/3331242667 conn(0x5588a9b88180 msgr2=0x5588a9c8a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3331242667 shutdown_connections 2026-02-25T11:59:00.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/3331242667 wait complete. 2026-02-25T11:59:00.449 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.440+0000 7fd54c3e9e00 20 client.17787 trim_cache size 0 max 16384 2026-02-25T11:59:00.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.448+0000 7fd54c3e9e00 1 Processor -- start 2026-02-25T11:59:00.457 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.448+0000 7fd54c3e9e00 1 -- start start 2026-02-25T11:59:00.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd54c3e9e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c880f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd54c3e9e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9c89220 0x5588a9c88630 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd54c3e9e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9ca9b40 0x5588a9c9efe0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd54c3e9e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5588a9b74d00 con 0x5588a9c68900 2026-02-25T11:59:00.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd54c3e9e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5588a9b72930 con 0x5588a9ca9b40 2026-02-25T11:59:00.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd54c3e9e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5588a9812690 con 0x5588a9c89220 2026-02-25T11:59:00.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd5437fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c880f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd5437fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c880f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34830/0 (socket says 10.20.193.28:34830) 2026-02-25T11:59:00.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd5437fe640 1 -- 10.20.193.28:0/4266399677 learned_addr learned my addr 10.20.193.28:0/4266399677 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd542ffd640 1 --2- 10.20.193.28:0/4266399677 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9c89220 0x5588a9c88630 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd543fff640 1 --2- 10.20.193.28:0/4266399677 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9ca9b40 0x5588a9c9efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd5437fe640 1 -- 10.20.193.28:0/4266399677 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9c89220 msgr2=0x5588a9c88630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd5437fe640 1 --2- 10.20.193.28:0/4266399677 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9c89220 0x5588a9c88630 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd5437fe640 1 -- 10.20.193.28:0/4266399677 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9ca9b40 msgr2=0x5588a9c9efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd5437fe640 1 --2- 10.20.193.28:0/4266399677 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9ca9b40 0x5588a9c9efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd5437fe640 1 -- 10.20.193.28:0/4266399677 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a9b050b0 con 0x5588a9c68900 2026-02-25T11:59:00.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd5437fe640 1 --2- 10.20.193.28:0/4266399677 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c880f0 secure :-1 s=READY pgs=948 gs=1 cs=0 l=1 c_cookie=13250625986114765824 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd5300053e0 tx=0x7fd5300ca000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e3ae2129ee4400 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd50effd640 1 -- 10.20.193.28:0/4266399677 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd530005140 con 0x5588a9c68900 2026-02-25T11:59:00.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd50effd640 1 -- 10.20.193.28:0/4266399677 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5588a98579e0 con 0x5588a9c68900 2026-02-25T11:59:00.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.449+0000 7fd50effd640 1 -- 10.20.193.28:0/4266399677 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd5300ac0a0 con 0x5588a9c68900 2026-02-25T11:59:00.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.450+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/4266399677 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 msgr2=0x5588a9c880f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.450+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/4266399677 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c880f0 secure :-1 s=READY pgs=948 gs=1 cs=0 l=1 c_cookie=13250625986114765824 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd5300053e0 tx=0x7fd5300ca000 comp rx=0 tx=0).stop 2026-02-25T11:59:00.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.450+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/4266399677 shutdown_connections 2026-02-25T11:59:00.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.450+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/4266399677 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9ca9b40 0x5588a9c9efe0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.450+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/4266399677 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9c89220 0x5588a9c88630 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.450+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/4266399677 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c880f0 unknown :-1 s=CLOSED pgs=948 gs=1 cs=0 l=1 c_cookie=13250625986114765824 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.450+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/4266399677 >> 10.20.193.28:0/4266399677 conn(0x5588a9b88180 msgr2=0x5588a9c8d610 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.450+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/4266399677 shutdown_connections 2026-02-25T11:59:00.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.450+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/4266399677 wait complete. 2026-02-25T11:59:00.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.450+0000 7fd54c3e9e00 1 -- start start 2026-02-25T11:59:00.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd54c3e9e00 1 Processor -- start 2026-02-25T11:59:00.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd54c3e9e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c6bc80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd54c3e9e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9c89220 0x5588a9c6c1c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd54c3e9e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9ca9b40 0x5588a9c6ce70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd54c3e9e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x5588a9ca43e0 con 0x5588a9c68900 2026-02-25T11:59:00.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd54c3e9e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x5588a9c6c8f0 con 0x5588a9c89220 2026-02-25T11:59:00.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd54c3e9e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x5588a9c6d3b0 con 0x5588a9ca9b40 2026-02-25T11:59:00.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd5437fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c6bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd5437fe640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c6bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34836/0 (socket says 10.20.193.28:34836) 2026-02-25T11:59:00.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd5437fe640 1 -- 10.20.193.28:0/1041758989 learned_addr learned my addr 10.20.193.28:0/1041758989 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd543fff640 1 --2- 10.20.193.28:0/1041758989 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9ca9b40 0x5588a9c6ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd542ffd640 1 --2- 10.20.193.28:0/1041758989 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9c89220 0x5588a9c6c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd5437fe640 1 -- 10.20.193.28:0/1041758989 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9ca9b40 msgr2=0x5588a9c6ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd5437fe640 1 --2- 10.20.193.28:0/1041758989 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9ca9b40 0x5588a9c6ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd5437fe640 1 -- 10.20.193.28:0/1041758989 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9c89220 msgr2=0x5588a9c6c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd5437fe640 1 --2- 10.20.193.28:0/1041758989 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9c89220 0x5588a9c6c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd5437fe640 1 -- 10.20.193.28:0/1041758989 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a9af89f0 con 0x5588a9c68900 2026-02-25T11:59:00.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd5437fe640 1 --2- 10.20.193.28:0/1041758989 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c6bc80 secure :-1 s=READY pgs=949 gs=1 cs=0 l=1 c_cookie=1326203879352048974 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd530009300 tx=0x7fd5300d2470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12679f66d8861d4e server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd50cff9640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:59:00.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.451+0000 7fd50cff9640 1 -- 10.20.193.28:0/1041758989 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd5300b52d0 con 0x5588a9c68900 2026-02-25T11:59:00.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd50cff9640 1 -- 10.20.193.28:0/1041758989 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fd530005140 con 0x5588a9c68900 2026-02-25T11:59:00.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd50cff9640 1 -- 10.20.193.28:0/1041758989 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fd5300acdb0 con 0x5588a9c68900 2026-02-25T11:59:00.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/1041758989 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a9b0a810 con 0x5588a9c68900 2026-02-25T11:59:00.462 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd50cff9640 1 -- 10.20.193.28:0/1041758989 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fd530004370 con 0x5588a9c68900 2026-02-25T11:59:00.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd50cff9640 10 client.17793 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:59:00.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd54c3e9e00 10 client.17793 _ll_register_callbacks cb 0x5588a9aff6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:59:00.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/1041758989 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5588a9b82470 con 0x5588a9c68900 2026-02-25T11:59:00.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd50cff9640 1 -- 10.20.193.28:0/1041758989 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd5300e4e40 con 0x5588a9c68900 2026-02-25T11:59:00.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd54c3e9e00 10 client.17793 fetch_fsmap learned FSMap version 23 2026-02-25T11:59:00.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/1041758989 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x5588a9b0aa70 con 0x5588a9c68900 2026-02-25T11:59:00.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd50cff9640 1 -- 10.20.193.28:0/1041758989 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fd5300adc60 con 0x5588a9c68900 2026-02-25T11:59:00.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd54c3e9e00 10 client.17793 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:59:00.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd54c3e9e00 10 client.17793 resolve_mds: validated gid 24749 aka daemon mds.b 2026-02-25T11:59:00.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd54c3e9e00 20 client.17793 populate_metadata read hostname 'trial028' 2026-02-25T11:59:00.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/1041758989 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5588a9c6e920 0x5588a9d76cb0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd54c3e9e00 4 client.17793 mds_command: new command op to 24749 tid=0 multi_id=0 [{"prefix": "perf dump", "format": "json-pretty"}] 2026-02-25T11:59:00.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.452+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/1041758989 --> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] -- command(tid 0: {"prefix": "perf dump", "format": "json-pretty"}) -- 0x5588a98a5740 con 0x5588a9c6e920 2026-02-25T11:59:00.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.453+0000 7fd542ffd640 1 --2- 10.20.193.28:0/1041758989 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5588a9c6e920 0x5588a9d76cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.453+0000 7fd542ffd640 1 --2- 10.20.193.28:0/1041758989 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5588a9c6e920 0x5588a9d76cb0 crc :-1 s=READY pgs=189 gs=4 cs=0 l=0 c_cookie=2505473638790378743 s_cookie=13538747289242058935 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c53afffa34b4f7 server_cookie=bbe34aed9f0e40b7 in_seq=0 out_seq=0 2026-02-25T11:59:00.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.453+0000 7fd50cff9640 10 client.17793 ms_handle_connect on v2:10.20.193.119:6838/1063465486 2026-02-25T11:59:00.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd50cff9640 1 -- 10.20.193.28:0/1041758989 <== mds.0 v2:10.20.193.119:6838/1063465486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+18448 (crc 0 0 0) 0x5588a98a5740 con 0x5588a9c6e920 2026-02-25T11:59:00.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd50cff9640 10 client.17793 handle_command_reply: tid=0 2026-02-25T11:59:00.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd54c3e9e00 1 client.17793 shutdown 2026-02-25T11:59:00.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/1041758989 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 msgr2=0x5588a9c6bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/1041758989 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c6bc80 secure :-1 s=READY pgs=949 gs=1 cs=0 l=1 c_cookie=1326203879352048974 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd530009300 tx=0x7fd5300d2470 comp rx=0 tx=0).stop 2026-02-25T11:59:00.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/1041758989 shutdown_connections 2026-02-25T11:59:00.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/1041758989 >> [v2:10.20.193.119:6838/1063465486,v1:10.20.193.119:6839/1063465486] conn(0x5588a9c6e920 0x5588a9d76cb0 crc :-1 s=READY pgs=189 gs=4 cs=0 l=0 c_cookie=2505473638790378743 s_cookie=13538747289242058935 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/1041758989 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x5588a9ca9b40 0x5588a9c6ce70 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/1041758989 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x5588a9c89220 0x5588a9c6c1c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd54c3e9e00 1 --2- 10.20.193.28:0/1041758989 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x5588a9c68900 0x5588a9c6bc80 unknown :-1 s=CLOSED pgs=949 gs=1 cs=0 l=1 c_cookie=1326203879352048974 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/1041758989 >> 10.20.193.28:0/1041758989 conn(0x5588a9b88180 msgr2=0x5588a9c870e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd50cff9640 0 client.17793 ms_handle_reset on v2:10.20.193.119:6838/1063465486 2026-02-25T11:59:00.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/1041758989 shutdown_connections 2026-02-25T11:59:00.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd54c3e9e00 1 -- 10.20.193.28:0/1041758989 wait complete. 2026-02-25T11:59:00.465 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.454+0000 7fd54c3e9e00 20 client.17793 trim_cache size 0 max 16384 2026-02-25T11:59:00.466 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:59:00.466 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:59:00.466 INFO:teuthology.orchestra.run.trial028.stdout: "AsyncMessenger::Worker-0": { 2026-02-25T11:59:00.466 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_messages": 3220, 2026-02-25T11:59:00.466 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages": 3374, 2026-02-25T11:59:00.467 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_bytes": 889309, 2026-02-25T11:59:00.467 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_bytes": 8876008, 2026-02-25T11:59:00.467 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_created_connections": 27, 2026-02-25T11:59:00.467 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_active_connections": 6, 2026-02-25T11:59:00.467 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_total_time": 0.221601029, 2026-02-25T11:59:00.467 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_send_time": 0.115929520, 2026-02-25T11:59:00.467 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_recv_time": 0.060552244, 2026-02-25T11:59:00.467 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_fast_dispatch_time": 0.030511372, 2026-02-25T11:59:00.467 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages_queue_lat": { 2026-02-25T11:59:00.467 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 3373, 2026-02-25T11:59:00.467 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 1.624661595, 2026-02-25T11:59:00.467 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000481666 2026-02-25T11:59:00.468 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.468 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_handle_ack_lat": { 2026-02-25T11:59:00.468 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 1355, 2026-02-25T11:59:00.468 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000131553, 2026-02-25T11:59:00.468 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000097 2026-02-25T11:59:00.468 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.468 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_encrypted_bytes": 0, 2026-02-25T11:59:00.468 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_encrypted_bytes": 0 2026-02-25T11:59:00.468 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.468 INFO:teuthology.orchestra.run.trial028.stdout: "AsyncMessenger::Worker-1": { 2026-02-25T11:59:00.469 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_messages": 1219, 2026-02-25T11:59:00.469 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages": 1292, 2026-02-25T11:59:00.469 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_bytes": 321109, 2026-02-25T11:59:00.469 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_bytes": 2774111, 2026-02-25T11:59:00.469 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_created_connections": 156, 2026-02-25T11:59:00.469 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_active_connections": 10, 2026-02-25T11:59:00.469 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_total_time": 0.106174131, 2026-02-25T11:59:00.469 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_send_time": 0.037063794, 2026-02-25T11:59:00.469 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_recv_time": 0.052485266, 2026-02-25T11:59:00.469 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_fast_dispatch_time": 0.012783695, 2026-02-25T11:59:00.469 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages_queue_lat": { 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 1291, 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.281018250, 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000217674 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_handle_ack_lat": { 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 391, 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000014362, 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000036 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_encrypted_bytes": 66608, 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_encrypted_bytes": 449312 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: "AsyncMessenger::Worker-2": { 2026-02-25T11:59:00.470 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_messages": 3062, 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages": 5612, 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_bytes": 1632056, 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_bytes": 2213258, 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_created_connections": 10, 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_active_connections": 6, 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_total_time": 0.265535113, 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_send_time": 0.154360877, 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_recv_time": 0.132748231, 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_fast_dispatch_time": 0.002676710, 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages_queue_lat": { 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 5612, 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.352792218, 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000062863 2026-02-25T11:59:00.471 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.472 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_handle_ack_lat": { 2026-02-25T11:59:00.472 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 6964, 2026-02-25T11:59:00.472 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000718242, 2026-02-25T11:59:00.472 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000103 2026-02-25T11:59:00.472 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.472 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_encrypted_bytes": 562416, 2026-02-25T11:59:00.472 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_encrypted_bytes": 145200 2026-02-25T11:59:00.472 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.472 INFO:teuthology.orchestra.run.trial028.stdout: "cct": { 2026-02-25T11:59:00.472 INFO:teuthology.orchestra.run.trial028.stdout: "total_workers": 1, 2026-02-25T11:59:00.472 INFO:teuthology.orchestra.run.trial028.stdout: "unhealthy_workers": 0 2026-02-25T11:59:00.472 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: "finisher-MDSRank": { 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: "queue_len": 0, 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: "complete_latency": { 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 5033, 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.541506500, 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000107591 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: "finisher-PurgeQueue": { 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: "queue_len": 0, 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: "complete_latency": { 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 29, 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000360959, 2026-02-25T11:59:00.473 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000012446 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: "mds": { 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: "request": 661, 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: "reply": 661, 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: "reply_latency": { 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 661, 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 1.405539599, 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.002126383 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: "slow_reply": 0, 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: "forward": 0, 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: "dir_fetch_complete": 0, 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: "dir_fetch_keys": 0, 2026-02-25T11:59:00.474 INFO:teuthology.orchestra.run.trial028.stdout: "dir_commit": 12, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "dir_split": 0, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "dir_merge": 0, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "inodes": 23, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_top": 0, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_bottom": 23, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_pin_tail": 0, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_pinned": 23, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_expired": 0, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_with_caps": 2, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "caps": 2, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "subtrees": 2, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "traverse": 711, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_hit": 703, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_forward": 0, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_discover": 0, 2026-02-25T11:59:00.475 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_dir_fetch": 0, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_remote_ino": 0, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_lock": 0, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "load_cent": 49, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "q": 0, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "exported": 0, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "exported_inodes": 0, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "imported": 0, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "imported_inodes": 0, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "openino_dir_fetch": 0, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "openino_backtrace_fetch": 292, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "openino_peer_discover": 0, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "root_rfiles": 4, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "root_rbytes": 234, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "root_rsnaps": 0, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_backtrace_fetch": 292, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_set_tag": 0, 2026-02-25T11:59:00.476 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_backtrace_repaired": 0, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_inotable_repaired": 0, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_dir_inodes": 188, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_dir_base_inodes": 21, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_dirfrag_rstats": 167, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_file_inodes": 104, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "handle_inode_file_caps": 0, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_revoke": 673, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_grant": 1235, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_trunc": 599, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_flushsnap_ack": 0, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_flush_ack": 0, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_caps": 680, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_caps_dirty": 623, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_cap_release": 3, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "process_request_cap_release": 28 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "mds_cache": { 2026-02-25T11:59:00.477 INFO:teuthology.orchestra.run.trial028.stdout: "dir_update": 10, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "dir_update_receipt": 0, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "dir_try_discover": 0, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "dir_send_discover": 0, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "dir_handle_discover": 20, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "num_strays": 0, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "num_strays_delayed": 0, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "num_strays_enqueuing": 0, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "strays_created": 3, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "strays_enqueued": 3, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "strays_reintegrated": 0, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "strays_migrated": 0, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "num_recovering_processing": 0, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "num_recovering_enqueued": 0, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "num_recovering_prioritized": 0, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "recovery_started": 0, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "recovery_completed": 0, 2026-02-25T11:59:00.478 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_quiesce_path": 0, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_quiesce_inode": 0, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_enqueue_scrub": 21, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_exportdir": 0, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_flush": 0, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_fragmentdir": 0, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_fragstats": 0, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_inodestats": 0, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_succeeded": 0, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_write_failed": 0 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "mds_log": { 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "evlrg": 0, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "evadd": 2140, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "evex": 0, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "evtrm": 0, 2026-02-25T11:59:00.479 INFO:teuthology.orchestra.run.trial028.stdout: "ev": 2140, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "evexg": 0, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "evexd": 0, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "segadd": 4, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "segex": 0, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "segtrm": 0, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "seg": 4, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "segmjr": 2, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "segexg": 0, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "segexd": 0, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "expos": 4194304, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "wrpos": 11813172, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "rdpos": 0, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "jlat": { 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 2121, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 2.091386818, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000986038 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.480 INFO:teuthology.orchestra.run.trial028.stdout: "replayed": 0 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "mds_mem": { 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "ino": 23, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "ino+": 47, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "ino-": 24, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "dir": 18, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "dir+": 20, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "dir-": 2, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "dn": 23, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "dn+": 28, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "dn-": 5, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "cap": 2, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "cap+": 16, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "cap-": 14, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "rss": 55076, 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: "heap": 274427904 2026-02-25T11:59:00.481 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "mds_server": { 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "dispatch_client_request": 1894, 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "dispatch_server_request": 3, 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_request": 661, 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_session": 15, 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "handle_peer_request": 6, 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "req_create_latency": { 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 5, 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.001830986, 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000366197 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "req_getattr_latency": { 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 28, 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.003832933, 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000136890 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "req_getfilelock_latency": { 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.482 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: "req_link_latency": { 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookup_latency": { 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookuphash_latency": { 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.483 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookupino_latency": { 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookupname_latency": { 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookupparent_latency": { 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookupsnap_latency": { 2026-02-25T11:59:00.484 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "req_lssnap_latency": { 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "req_mkdir_latency": { 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 8, 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.006369271, 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000796158 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "req_mknod_latency": { 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.485 INFO:teuthology.orchestra.run.trial028.stdout: "req_mksnap_latency": { 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "req_open_latency": { 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 599, 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 1.371615493, 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.002289842 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "req_readdir_latency": { 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 2, 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000998895, 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000499447 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "req_rename_latency": { 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 2, 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000838341, 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000419170 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "req_renamesnap_latency": { 2026-02-25T11:59:00.486 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "req_snapdiff_latency": { 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "req_rmdir_latency": { 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 2, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.006053654, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.003026827 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "req_rmsnap_latency": { 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "req_rmxattr_latency": { 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.487 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: "req_setattr_latency": { 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 3, 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.001306661, 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000435553 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: "req_setdirlayout_latency": { 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: "req_setfilelock_latency": { 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.488 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.489 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.489 INFO:teuthology.orchestra.run.trial028.stdout: "req_setlayout_latency": { 2026-02-25T11:59:00.489 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.489 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.489 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.489 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.489 INFO:teuthology.orchestra.run.trial028.stdout: "req_setxattr_latency": { 2026-02-25T11:59:00.489 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 11, 2026-02-25T11:59:00.489 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.012043599, 2026-02-25T11:59:00.489 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.001094872 2026-02-25T11:59:00.489 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.489 INFO:teuthology.orchestra.run.trial028.stdout: "req_symlink_latency": { 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "req_unlink_latency": { 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 1, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000649766, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000649766 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "cap_revoke_eviction": 0, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "cache_trim_throttle": 0, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "session_recall_throttle": 0, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "session_recall_throttle2o": 0, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "global_recall_throttle": 0, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "cap_acquisition_throttle": 0, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "req_getvxattr_latency": { 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.490 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "req_blockdiff_latency": { 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "mds_sessions": { 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "session_count": 1, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "session_add": 2, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "session_remove": 1, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "sessions_open": 1, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "sessions_stale": 0, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "total_load": 95, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "average_load": 95, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "avg_session_uptime": 670, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "mdthresh_evicted": 0 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "mempool": { 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "bloom_filter_bytes": 0, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "bloom_filter_items": 0, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_alloc_bytes": 0, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_alloc_items": 0, 2026-02-25T11:59:00.491 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_data_bytes": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_data_items": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_onode_bytes": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_onode_items": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_meta_bytes": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_meta_items": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_other_bytes": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_other_items": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_buffer_bytes": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_buffer_items": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_extent_bytes": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_extent_items": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_blob_bytes": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_blob_items": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_shared_blob_bytes": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_shared_blob_items": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_inline_bl_bytes": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_inline_bl_items": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_fsck_bytes": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_fsck_items": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_txc_bytes": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_txc_items": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_writing_deferred_bytes": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_writing_deferred_items": 0, 2026-02-25T11:59:00.492 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_writing_bytes": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_writing_items": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_bytes": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_items": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_file_reader_bytes": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_file_reader_items": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_file_writer_bytes": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_file_writer_items": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "buffer_anon_bytes": 306760, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "buffer_anon_items": 124, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "buffer_meta_bytes": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "buffer_meta_items": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "osd_bytes": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "osd_items": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "osd_mapbl_bytes": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "osd_mapbl_items": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "osd_pglog_bytes": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "osd_pglog_items": 0, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_bytes": 7504, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_items": 153, 2026-02-25T11:59:00.493 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_mapping_bytes": 0, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_mapping_items": 0, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "pgmap_bytes": 0, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "pgmap_items": 0, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "mds_co_bytes": 98396, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "mds_co_items": 875, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "ec_extent_cache_bytes": 0, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "ec_extent_cache_items": 0, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "unittest_1_bytes": 0, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "unittest_1_items": 0, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "unittest_2_bytes": 0, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "unittest_2_items": 0 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "objecter": { 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "op_active": 0, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "op_laggy": 0, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "op_send": 3676, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "op_send_bytes": 7640036, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "op_resend": 19, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "op_reply": 3676, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "op_latency": { 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 3676, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 10.964784193, 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.002982803 2026-02-25T11:59:00.494 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "op_inflight": 0, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "oplen_avg": { 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 3722, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 3676 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "op": 3676, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "op_r": 337, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "op_w": 3339, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "op_rmw": 0, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "op_pg": 0, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_stat": 0, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_create": 11, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_read": 24, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_write": 2124, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_writefull": 55, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_writesame": 0, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_append": 0, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_zero": 0, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_truncate": 0, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_delete": 516, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_mapext": 0, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_sparse_read": 0, 2026-02-25T11:59:00.495 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_clonerange": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_getxattr": 313, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_setxattr": 22, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_cmpxattr": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_rmxattr": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_resetxattrs": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_call": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_watch": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_notify": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_src_cmpxattr": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_pgls": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_pgls_filter": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_other": 599, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "linger_active": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "linger_send": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "linger_resend": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "linger_ping": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "poolop_active": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "poolop_send": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "poolop_resend": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "poolstat_active": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "poolstat_send": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "poolstat_resend": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "statfs_active": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "statfs_send": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "statfs_resend": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "command_active": 0, 2026-02-25T11:59:00.496 INFO:teuthology.orchestra.run.trial028.stdout: "command_send": 0, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "command_resend": 0, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "map_epoch": 88, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "map_full": 0, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "map_inc": 3, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "osd_sessions": 12, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "osd_session_open": 12, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "osd_session_close": 0, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "osd_laggy": 0, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "omap_wr": 47, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "omap_rd": 0, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "omap_del": 11, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "replica_read_sent": 0, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "replica_read_bounced": 0, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "replica_read_completed": 0 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "oft": { 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "omap_total_objs": 1, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "omap_total_kv_pairs": 0, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "omap_total_updates": 0, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "omap_total_removes": 0 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "purge_queue": { 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executing_ops": 0, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executing_ops_high_water": 1, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executing": 0, 2026-02-25T11:59:00.497 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executing_high_water": 1, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executed_ops": 3, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executed": 3, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "pq_item_in_journal": 0 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-msgr_dispatch_throttler-mds": { 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "max": 104857600, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "get": 7501, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 2148617, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 7501, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "put": 7501, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 2148617, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-objecter_bytes": { 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "max": 104857600, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "get": 3676, 2026-02-25T11:59:00.498 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 7640339, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 3676, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "put": 3676, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 7640339, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-objecter_ops": { 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "max": 1024, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "get": 3676, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 3676, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 3676, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "put": 3676, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 3676, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.499 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-write_buf_throttle": { 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "max": 3758096384, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "get": 3, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 303, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 3, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "put": 3, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 303, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-write_buf_throttle-0x32656780": { 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "max": 3758096384, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "get": 2140, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 7618868, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 2140, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.500 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.501 INFO:teuthology.orchestra.run.trial028.stdout: "put": 2121, 2026-02-25T11:59:00.501 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 7618868, 2026-02-25T11:59:00.501 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.501 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.501 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.501 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.501 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.501 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.501 INFO:teuthology.orchestra.run.trial028.stdout: "trackedop": { 2026-02-25T11:59:00.501 INFO:teuthology.orchestra.run.trial028.stdout: "slow_ops_count": 0 2026-02-25T11:59:00.501 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.501 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:59:00.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.455+0000 7fd548a83640 1 -- 10.20.193.28:0/529378215 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd5180b5760 msgr2=0x7fd5180d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.455+0000 7fd548a83640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd5180b5760 0x7fd5180d5c10 secure :-1 s=READY pgs=413 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd5340098a0 tx=0x7fd53403a040 comp rx=0 tx=0).stop 2026-02-25T11:59:00.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.455+0000 7fd548a83640 1 -- 10.20.193.28:0/529378215 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd5441138f0 msgr2=0x7fd5440febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.455+0000 7fd548a83640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd5441138f0 0x7fd5440febe0 secure :-1 s=READY pgs=945 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fd530009480 tx=0x7fd53000ee60 comp rx=0 tx=0).stop 2026-02-25T11:59:00.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.455+0000 7fd548a83640 1 -- 10.20.193.28:0/529378215 shutdown_connections 2026-02-25T11:59:00.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.455+0000 7fd548a83640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fd5180b5760 0x7fd5180d5c10 unknown :-1 s=CLOSED pgs=413 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.455+0000 7fd548a83640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fd544115180 0x7fd5440ff420 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.455+0000 7fd548a83640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fd5441147d0 0x7fd544175af0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.455+0000 7fd548a83640 1 --2- 10.20.193.28:0/529378215 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fd5441138f0 0x7fd5440febe0 unknown :-1 s=CLOSED pgs=945 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.455+0000 7fd548a83640 1 -- 10.20.193.28:0/529378215 >> 10.20.193.28:0/529378215 conn(0x7fd544064570 msgr2=0x7fd5440f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.455+0000 7fd548a83640 1 -- 10.20.193.28:0/529378215 shutdown_connections 2026-02-25T11:59:00.502 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.455+0000 7fd548a83640 1 -- 10.20.193.28:0/529378215 wait complete. 2026-02-25T11:59:00.503 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2026-02-25T11:59:00.503 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2026-02-25T11:59:00.503 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=10 2026-02-25T11:59:00.503 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2026-02-25T11:59:00.503 DEBUG:tasks.check_counter:Getting stats from e 2026-02-25T11:59:00.503 DEBUG:tasks.check_counter:skipping mds.24694 2026-02-25T11:59:00.503 DEBUG:tasks.check_counter:Getting stats from h 2026-02-25T11:59:00.503 DEBUG:tasks.check_counter:skipping mds.24719 2026-02-25T11:59:00.503 DEBUG:tasks.check_counter:Getting stats from k 2026-02-25T11:59:00.503 DEBUG:tasks.check_counter:skipping mds.24671 2026-02-25T11:59:00.503 DEBUG:tasks.check_counter:Getting stats from c 2026-02-25T11:59:00.503 DEBUG:tasks.check_counter:skipping mds.24707 2026-02-25T11:59:00.503 DEBUG:tasks.check_counter:Getting stats from f 2026-02-25T11:59:00.503 DEBUG:tasks.check_counter:skipping mds.24706 2026-02-25T11:59:00.503 DEBUG:tasks.check_counter:Getting stats from i 2026-02-25T11:59:00.503 DEBUG:tasks.check_counter:Getting stats from mds.24730 2026-02-25T11:59:00.503 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24730 perf dump --format=json-pretty 2026-02-25T11:59:00.553 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:00 trial028 ceph-mon[39224]: pgmap v549: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.8 KiB/s rd, 6 op/s 2026-02-25T11:59:00.553 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:00 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2206510807' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:59:00.623 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.622+0000 7f07db539640 1 -- 10.20.193.28:0/1507648436 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07d410ce70 msgr2=0x7f07d410d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.622+0000 7f07db539640 1 --2- 10.20.193.28:0/1507648436 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07d410ce70 0x7f07d410d260 secure :-1 s=READY pgs=950 gs=1 cs=0 l=1 c_cookie=14188079478561361879 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07c4009950 tx=0x7f07c402f1d0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.622+0000 7f07db539640 1 -- 10.20.193.28:0/1507648436 shutdown_connections 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.622+0000 7f07db539640 1 --2- 10.20.193.28:0/1507648436 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f07d410e9b0 0x7f07d416f340 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.622+0000 7f07db539640 1 --2- 10.20.193.28:0/1507648436 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f07d410e000 0x7f07d410e470 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.622+0000 7f07db539640 1 --2- 10.20.193.28:0/1507648436 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07d410ce70 0x7f07d410d260 unknown :-1 s=CLOSED pgs=950 gs=1 cs=0 l=1 c_cookie=14188079478561361879 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.622+0000 7f07db539640 1 -- 10.20.193.28:0/1507648436 >> 10.20.193.28:0/1507648436 conn(0x7f07d40ea5c0 msgr2=0x7f07d410a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.622+0000 7f07db539640 1 -- 10.20.193.28:0/1507648436 shutdown_connections 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.622+0000 7f07db539640 1 -- 10.20.193.28:0/1507648436 wait complete. 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07db539640 1 Processor -- start 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07db539640 1 -- start start 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07db539640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f07d410ce70 0x7f07d40f8860 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07db539640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f07d410e000 0x7f07d416f340 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07db539640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07d410e9b0 0x7f07d40f90c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07db539640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f07d40f9cb0 con 0x7f07d410e9b0 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07db539640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f07d40f9e20 con 0x7f07d410e000 2026-02-25T11:59:00.624 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07db539640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f07d40f9fe0 con 0x7f07d410ce70 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07dad38640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07d410e9b0 0x7f07d40f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07dad38640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07d410e9b0 0x7f07d40f90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34866/0 (socket says 10.20.193.28:34866) 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07dad38640 1 -- 10.20.193.28:0/1566390224 learned_addr learned my addr 10.20.193.28:0/1566390224 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07da537640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f07d410ce70 0x7f07d40f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07d9d36640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f07d410e000 0x7f07d416f340 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07dad38640 1 -- 10.20.193.28:0/1566390224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f07d410ce70 msgr2=0x7f07d40f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07dad38640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f07d410ce70 0x7f07d40f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07dad38640 1 -- 10.20.193.28:0/1566390224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f07d410e000 msgr2=0x7f07d416f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07dad38640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f07d410e000 0x7f07d416f340 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07dad38640 1 -- 10.20.193.28:0/1566390224 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c4009570 con 0x7f07d410e9b0 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.623+0000 7f07dad38640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07d410e9b0 0x7f07d40f90c0 secure :-1 s=READY pgs=951 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07d0006e90 tx=0x7f07d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.624+0000 7f07bb7fe640 1 -- 10.20.193.28:0/1566390224 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07d0005500 con 0x7f07d410e9b0 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.624+0000 7f07bb7fe640 1 -- 10.20.193.28:0/1566390224 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f07d0005df0 con 0x7f07d410e9b0 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.624+0000 7f07bb7fe640 1 -- 10.20.193.28:0/1566390224 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07d0003a00 con 0x7f07d410e9b0 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.624+0000 7f07db539640 1 -- 10.20.193.28:0/1566390224 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d40fa2c0 con 0x7f07d410e9b0 2026-02-25T11:59:00.625 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.624+0000 7f07db539640 1 -- 10.20.193.28:0/1566390224 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d40fa740 con 0x7f07d410e9b0 2026-02-25T11:59:00.628 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.627+0000 7f07bb7fe640 1 -- 10.20.193.28:0/1566390224 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f07d0003b60 con 0x7f07d410e9b0 2026-02-25T11:59:00.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.628+0000 7f07bb7fe640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07b00b5760 0x7f07b00d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.629 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.628+0000 7f07bb7fe640 1 -- 10.20.193.28:0/1566390224 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f07d00e77b0 con 0x7f07d410e9b0 2026-02-25T11:59:00.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.628+0000 7f07da537640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07b00b5760 0x7f07b00d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.630 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.629+0000 7f07da537640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07b00b5760 0x7f07b00d5c10 secure :-1 s=READY pgs=414 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07c40098a0 tx=0x7f07c403a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07dee86e00 1 Processor -- start 2026-02-25T11:59:00.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07dee86e00 1 -- start start 2026-02-25T11:59:00.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07dee86e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c62900 0x562438c82ce0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07dee86e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c83220 0x562438ca3600 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07dee86e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438cc3f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07dee86e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562438c01570 con 0x562438ca3b40 2026-02-25T11:59:00.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07dee86e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562438b79030 con 0x562438c83220 2026-02-25T11:59:00.631 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07dee86e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562438b74d60 con 0x562438c62900 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07dad38640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438cc3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07da537640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c62900 0x562438c82ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07dad38640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438cc3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34874/0 (socket says 10.20.193.28:34874) 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07dad38640 1 -- 10.20.193.28:0/4037040209 learned_addr learned my addr 10.20.193.28:0/4037040209 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07da537640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c62900 0x562438c82ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.167:3300/0 says I am v2:10.20.193.28:49232/0 (socket says 10.20.193.28:49232) 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.630+0000 7f07d9d36640 1 --2- 10.20.193.28:0/4037040209 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c83220 0x562438ca3600 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dad38640 1 -- 10.20.193.28:0/4037040209 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c62900 msgr2=0x562438c82ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dad38640 1 --2- 10.20.193.28:0/4037040209 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c62900 0x562438c82ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dad38640 1 -- 10.20.193.28:0/4037040209 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c83220 msgr2=0x562438ca3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dad38640 1 --2- 10.20.193.28:0/4037040209 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c83220 0x562438ca3600 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dad38640 1 -- 10.20.193.28:0/4037040209 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562438b1a300 con 0x562438ca3b40 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dad38640 1 --2- 10.20.193.28:0/4037040209 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438cc3f20 secure :-1 s=READY pgs=952 gs=3 cs=0 l=1 c_cookie=5896410157722109222 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07d0006b80 tx=0x7f07d00056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d441cc45080526 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07b8bfe640 1 -- 10.20.193.28:0/4037040209 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07d00ad510 con 0x562438ca3b40 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07b8bfe640 1 -- 10.20.193.28:0/4037040209 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f07d00ad330 con 0x562438ca3b40 2026-02-25T11:59:00.632 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07b8bfe640 1 -- 10.20.193.28:0/4037040209 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07d000c8d0 con 0x562438ca3b40 2026-02-25T11:59:00.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dee86e00 1 -- 10.20.193.28:0/4037040209 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 msgr2=0x562438cc3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dee86e00 1 --2- 10.20.193.28:0/4037040209 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438cc3f20 secure :-1 s=READY pgs=952 gs=3 cs=0 l=1 c_cookie=5896410157722109222 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07d0006b80 tx=0x7f07d00056e0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dee86e00 1 -- 10.20.193.28:0/4037040209 shutdown_connections 2026-02-25T11:59:00.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dee86e00 1 --2- 10.20.193.28:0/4037040209 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438cc3f20 unknown :-1 s=CLOSED pgs=952 gs=3 cs=0 l=1 c_cookie=5896410157722109222 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dee86e00 1 --2- 10.20.193.28:0/4037040209 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c83220 0x562438ca3600 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dee86e00 1 --2- 10.20.193.28:0/4037040209 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c62900 0x562438c82ce0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dee86e00 1 -- 10.20.193.28:0/4037040209 >> 10.20.193.28:0/4037040209 conn(0x562438b82180 msgr2=0x562438c611e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dee86e00 1 -- 10.20.193.28:0/4037040209 shutdown_connections 2026-02-25T11:59:00.633 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.631+0000 7f07dee86e00 1 -- 10.20.193.28:0/4037040209 wait complete. 2026-02-25T11:59:00.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.632+0000 7f07dee86e00 1 -- start start 2026-02-25T11:59:00.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dee86e00 1 Processor -- start 2026-02-25T11:59:00.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dee86e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 0x562438cc3f20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dee86e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 0x562438c9d020 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dee86e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c9d710 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dee86e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x56243880c690 con 0x562438ca3b40 2026-02-25T11:59:00.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dee86e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562438c9e110 con 0x562438c62900 2026-02-25T11:59:00.634 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dee86e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562438c9e2d0 con 0x562438c83220 2026-02-25T11:59:00.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dad38640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c9d710 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dad38640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c9d710 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34882/0 (socket says 10.20.193.28:34882) 2026-02-25T11:59:00.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dad38640 1 -- 10.20.193.28:0/1746608559 learned_addr learned my addr 10.20.193.28:0/1746608559 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07da537640 1 --2- 10.20.193.28:0/1746608559 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 0x562438cc3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.635 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07d9d36640 1 --2- 10.20.193.28:0/1746608559 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 0x562438c9d020 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dad38640 1 -- 10.20.193.28:0/1746608559 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 msgr2=0x562438c9d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dad38640 1 --2- 10.20.193.28:0/1746608559 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 0x562438c9d020 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dad38640 1 -- 10.20.193.28:0/1746608559 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 msgr2=0x562438cc3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dad38640 1 --2- 10.20.193.28:0/1746608559 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 0x562438cc3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.633+0000 7f07dad38640 1 -- 10.20.193.28:0/1746608559 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562438af26c0 con 0x562438ca3b40 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f07dad38640 1 --2- 10.20.193.28:0/1746608559 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c9d710 secure :-1 s=READY pgs=953 gs=2 cs=0 l=1 c_cookie=16185675938912524739 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07d0005770 tx=0x7f07d00a9fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09f1282d477d9c3 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f0787fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f0787fff640 1 -- 10.20.193.28:0/1746608559 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07d000c8d0 con 0x562438ca3b40 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f0787fff640 1 -- 10.20.193.28:0/1746608559 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f07d00ad330 con 0x562438ca3b40 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f0787fff640 1 -- 10.20.193.28:0/1746608559 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07d00ad510 con 0x562438ca3b40 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f07dee86e00 1 -- 10.20.193.28:0/1746608559 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d40fa740 con 0x562438ca3b40 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f0787fff640 1 -- 10.20.193.28:0/1746608559 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f07d00aea30 con 0x562438ca3b40 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f0787fff640 10 client.17805 inactive, discarding osd_map(88..88 src has 1..88) 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f07dee86e00 10 client.17805 _ll_register_callbacks cb 0x562438af96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f07dee86e00 1 -- 10.20.193.28:0/1746608559 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5624388519e0 con 0x562438ca3b40 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f0787fff640 1 -- 10.20.193.28:0/1746608559 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f07d00b8a70 con 0x562438ca3b40 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f07dee86e00 10 client.17805 fetch_fsmap learned FSMap version 23 2026-02-25T11:59:00.636 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f07dee86e00 1 -- 10.20.193.28:0/1746608559 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x562438b04810 con 0x562438ca3b40 2026-02-25T11:59:00.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f0787fff640 1 -- 10.20.193.28:0/1746608559 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f07d00abb10 con 0x562438ca3b40 2026-02-25T11:59:00.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f07dee86e00 10 client.17805 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:59:00.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.634+0000 7f07dee86e00 10 client.17805 resolve_mds: validated gid 24730 aka daemon mds.i 2026-02-25T11:59:00.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.635+0000 7f07dee86e00 20 client.17805 populate_metadata read hostname 'trial028' 2026-02-25T11:59:00.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.635+0000 7f07dee86e00 1 --2- 10.20.193.28:0/1746608559 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x562438ca0a90 0x562438ca0f30 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.635+0000 7f07dee86e00 4 client.17805 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2026-02-25T11:59:00.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.635+0000 7f07dee86e00 1 -- 10.20.193.28:0/1746608559 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562438b41860 con 0x562438ca0a90 2026-02-25T11:59:00.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.635+0000 7f07da537640 1 --2- 10.20.193.28:0/1746608559 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x562438ca0a90 0x562438ca0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f0787fff640 10 client.17805 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:59:00.637 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f07da537640 1 --2- 10.20.193.28:0/1746608559 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x562438ca0a90 0x562438ca0f30 crc :-1 s=READY pgs=65 gs=4 cs=0 l=0 c_cookie=71332203986392419 s_cookie=2217163593995755109 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fd6c42f7a79563 server_cookie=1ec4f28aad0b9e65 in_seq=0 out_seq=0 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f0787fff640 1 -- 10.20.193.28:0/1746608559 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25358 (crc 0 0 0) 0x562438b41860 con 0x562438ca0a90 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f0787fff640 10 client.17805 handle_command_reply: tid=0 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f07dee86e00 1 client.17805 shutdown 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f07dee86e00 1 -- 10.20.193.28:0/1746608559 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 msgr2=0x562438c9d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f07dee86e00 1 --2- 10.20.193.28:0/1746608559 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c9d710 secure :-1 s=READY pgs=953 gs=2 cs=0 l=1 c_cookie=16185675938912524739 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07d0005770 tx=0x7f07d00a9fd0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f07dee86e00 1 -- 10.20.193.28:0/1746608559 shutdown_connections 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f07dee86e00 1 --2- 10.20.193.28:0/1746608559 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x562438ca0a90 0x562438ca0f30 crc :-1 s=READY pgs=65 gs=4 cs=0 l=0 c_cookie=71332203986392419 s_cookie=2217163593995755109 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f07dee86e00 1 --2- 10.20.193.28:0/1746608559 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c9d710 unknown :-1 s=CLOSED pgs=953 gs=2 cs=0 l=1 c_cookie=16185675938912524739 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f07dee86e00 1 --2- 10.20.193.28:0/1746608559 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 0x562438c9d020 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f07dee86e00 1 --2- 10.20.193.28:0/1746608559 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 0x562438cc3f20 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f0787fff640 0 client.17805 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f07dee86e00 1 -- 10.20.193.28:0/1746608559 >> 10.20.193.28:0/1746608559 conn(0x562438b82180 msgr2=0x562438c844f0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.636+0000 7f07dee86e00 1 -- 10.20.193.28:0/1746608559 shutdown_connections 2026-02-25T11:59:00.638 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.637+0000 7f07dee86e00 1 -- 10.20.193.28:0/1746608559 wait complete. 2026-02-25T11:59:00.639 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.637+0000 7f07dee86e00 20 client.17805 trim_cache size 0 max 16384 2026-02-25T11:59:00.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.645+0000 7f07dee86e00 1 Processor -- start 2026-02-25T11:59:00.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.645+0000 7f07dee86e00 1 -- start start 2026-02-25T11:59:00.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.645+0000 7f07dee86e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 0x562438c9d620 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.645+0000 7f07dee86e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 0x562438c8ead0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.645+0000 7f07dee86e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c820f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.645+0000 7f07dee86e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562438b6ed00 con 0x562438ca3b40 2026-02-25T11:59:00.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.645+0000 7f07dee86e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562438b6c930 con 0x562438c62900 2026-02-25T11:59:00.647 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.645+0000 7f07dee86e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x56243880c690 con 0x562438c83220 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dad38640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c820f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07da537640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 0x562438c9d620 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dad38640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c820f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34888/0 (socket says 10.20.193.28:34888) 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dad38640 1 -- 10.20.193.28:0/3136586824 learned_addr learned my addr 10.20.193.28:0/3136586824 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07d9d36640 1 --2- 10.20.193.28:0/3136586824 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 0x562438c8ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dad38640 1 -- 10.20.193.28:0/3136586824 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 msgr2=0x562438c8ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dad38640 1 --2- 10.20.193.28:0/3136586824 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 0x562438c8ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dad38640 1 -- 10.20.193.28:0/3136586824 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 msgr2=0x562438c9d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dad38640 1 --2- 10.20.193.28:0/3136586824 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 0x562438c9d620 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dad38640 1 -- 10.20.193.28:0/3136586824 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562438aff0b0 con 0x562438ca3b40 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dad38640 1 --2- 10.20.193.28:0/3136586824 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c820f0 secure :-1 s=READY pgs=954 gs=1 cs=0 l=1 c_cookie=17468651143100635725 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07d00050a0 tx=0x7f07d0097030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26d1da969bc1a4d server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07a23ff640 1 -- 10.20.193.28:0/3136586824 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07d00b8c70 con 0x562438ca3b40 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07a23ff640 1 -- 10.20.193.28:0/3136586824 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x5624388519e0 con 0x562438ca3b40 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07a23ff640 1 -- 10.20.193.28:0/3136586824 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07d00972d0 con 0x562438ca3b40 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dee86e00 1 -- 10.20.193.28:0/3136586824 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 msgr2=0x562438c820f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.648 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dee86e00 1 --2- 10.20.193.28:0/3136586824 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c820f0 secure :-1 s=READY pgs=954 gs=1 cs=0 l=1 c_cookie=17468651143100635725 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07d00050a0 tx=0x7f07d0097030 comp rx=0 tx=0).stop 2026-02-25T11:59:00.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dee86e00 1 -- 10.20.193.28:0/3136586824 shutdown_connections 2026-02-25T11:59:00.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dee86e00 1 --2- 10.20.193.28:0/3136586824 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c820f0 unknown :-1 s=CLOSED pgs=954 gs=1 cs=0 l=1 c_cookie=17468651143100635725 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dee86e00 1 --2- 10.20.193.28:0/3136586824 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 0x562438c8ead0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dee86e00 1 --2- 10.20.193.28:0/3136586824 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 0x562438c9d620 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dee86e00 1 -- 10.20.193.28:0/3136586824 >> 10.20.193.28:0/3136586824 conn(0x562438b82180 msgr2=0x562438c9cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.649 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dee86e00 1 -- 10.20.193.28:0/3136586824 shutdown_connections 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.646+0000 7f07dee86e00 1 -- 10.20.193.28:0/3136586824 wait complete. 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.647+0000 7f07dee86e00 1 -- start start 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.647+0000 7f07dee86e00 1 Processor -- start 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.647+0000 7f07dee86e00 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 0x562438c65d70 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.647+0000 7f07dee86e00 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 0x562438c662b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.647+0000 7f07dee86e00 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c66d80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.647+0000 7f07dee86e00 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x562438b6ed00 con 0x562438ca3b40 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.647+0000 7f07dee86e00 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x562438b6c930 con 0x562438c62900 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.647+0000 7f07dee86e00 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x562438c66970 con 0x562438c83220 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07dad38640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c66d80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07dad38640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c66d80 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34898/0 (socket says 10.20.193.28:34898) 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07dad38640 1 -- 10.20.193.28:0/1132986875 learned_addr learned my addr 10.20.193.28:0/1132986875 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07da537640 1 --2- 10.20.193.28:0/1132986875 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 0x562438c65d70 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07d9d36640 1 --2- 10.20.193.28:0/1132986875 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 0x562438c662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07dad38640 1 -- 10.20.193.28:0/1132986875 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 msgr2=0x562438c662b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07dad38640 1 --2- 10.20.193.28:0/1132986875 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 0x562438c662b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07dad38640 1 -- 10.20.193.28:0/1132986875 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 msgr2=0x562438c65d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07dad38640 1 --2- 10.20.193.28:0/1132986875 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 0x562438c65d70 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07dad38640 1 -- 10.20.193.28:0/1132986875 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562438af29f0 con 0x562438ca3b40 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07dad38640 1 --2- 10.20.193.28:0/1132986875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c66d80 secure :-1 s=READY pgs=955 gs=3 cs=0 l=1 c_cookie=1996520356464115955 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07d00aada0 tx=0x7f07d0006a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb510afa9e140f3 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:00.650 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f0787fff640 10 client.0 ms_handle_connect on v2:10.20.193.28:3300/0 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f0787fff640 1 -- 10.20.193.28:0/1132986875 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07d00ad330 con 0x562438ca3b40 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07dee86e00 1 -- 10.20.193.28:0/1132986875 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d00ad160 con 0x562438ca3b40 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f0787fff640 1 -- 10.20.193.28:0/1132986875 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f07d00ad510 con 0x562438ca3b40 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07dee86e00 10 client.17811 _ll_register_callbacks cb 0x562438af96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f0787fff640 1 -- 10.20.193.28:0/1132986875 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f07d00aea40 con 0x562438ca3b40 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f07dee86e00 1 -- 10.20.193.28:0/1132986875 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562438b7c470 con 0x562438ca3b40 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.648+0000 7f0787fff640 1 -- 10.20.193.28:0/1132986875 <== mon.0 v2:10.20.193.28:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f07d00b87e0 con 0x562438ca3b40 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.649+0000 7f0787fff640 1 -- 10.20.193.28:0/1132986875 <== mon.0 v2:10.20.193.28:3300/0 5 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f07d00b8a20 con 0x562438ca3b40 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.649+0000 7f07dee86e00 10 client.17811 fetch_fsmap learned FSMap version 23 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.649+0000 7f07dee86e00 1 -- 10.20.193.28:0/1132986875 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({fsmap=23}) -- 0x562438b04a70 con 0x562438ca3b40 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.649+0000 7f0787fff640 1 -- 10.20.193.28:0/1132986875 <== mon.0 v2:10.20.193.28:3300/0 6 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f07d00af520 con 0x562438ca3b40 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.649+0000 7f07dee86e00 10 client.17811 fetch_fsmap finished waiting for FSMap version 23 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.649+0000 7f07dee86e00 10 client.17811 resolve_mds: validated gid 24730 aka daemon mds.i 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.649+0000 7f07dee86e00 20 client.17811 populate_metadata read hostname 'trial028' 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.649+0000 7f07dee86e00 1 --2- 10.20.193.28:0/1132986875 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x562438c687f0 0x562438d70d20 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.649+0000 7f07dee86e00 4 client.17811 mds_command: new command op to 24730 tid=0 multi_id=0 [{"prefix": "perf dump", "format": "json-pretty"}] 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.649+0000 7f07dee86e00 1 -- 10.20.193.28:0/1132986875 --> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] -- command(tid 0: {"prefix": "perf dump", "format": "json-pretty"}) -- 0x56243889f740 con 0x562438c687f0 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.649+0000 7f07da537640 1 --2- 10.20.193.28:0/1132986875 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x562438c687f0 0x562438d70d20 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:00.651 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.650+0000 7f07da537640 1 --2- 10.20.193.28:0/1132986875 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x562438c687f0 0x562438d70d20 crc :-1 s=READY pgs=66 gs=4 cs=0 l=0 c_cookie=3450672061556166717 s_cookie=8133412161186329427 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2fe33ff5216b903d server_cookie=70dfaf44975a7353 in_seq=0 out_seq=0 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.650+0000 7f0787fff640 10 client.17811 ms_handle_connect on v2:10.20.193.167:6838/408350258 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f0787fff640 1 -- 10.20.193.28:0/1132986875 <== mds.2 v2:10.20.193.167:6838/408350258 1 ==== command_reply(tid 0: 0 ) ==== 8+0+18290 (crc 0 0 0) 0x56243889f740 con 0x562438c687f0 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f0787fff640 10 client.17811 handle_command_reply: tid=0 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07dee86e00 1 client.17811 shutdown 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07dee86e00 1 -- 10.20.193.28:0/1132986875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 msgr2=0x562438c66d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07dee86e00 1 --2- 10.20.193.28:0/1132986875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c66d80 secure :-1 s=READY pgs=955 gs=3 cs=0 l=1 c_cookie=1996520356464115955 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07d00aada0 tx=0x7f07d0006a10 comp rx=0 tx=0).stop 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07dee86e00 1 -- 10.20.193.28:0/1132986875 shutdown_connections 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07dee86e00 1 --2- 10.20.193.28:0/1132986875 >> [v2:10.20.193.167:6838/408350258,v1:10.20.193.167:6839/408350258] conn(0x562438c687f0 0x562438d70d20 crc :-1 s=READY pgs=66 gs=4 cs=0 l=0 c_cookie=3450672061556166717 s_cookie=8133412161186329427 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07dee86e00 1 --2- 10.20.193.28:0/1132986875 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x562438ca3b40 0x562438c66d80 unknown :-1 s=CLOSED pgs=955 gs=3 cs=0 l=1 c_cookie=1996520356464115955 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07dee86e00 1 --2- 10.20.193.28:0/1132986875 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x562438c83220 0x562438c662b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07dee86e00 1 --2- 10.20.193.28:0/1132986875 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x562438c62900 0x562438c65d70 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07dee86e00 1 -- 10.20.193.28:0/1132986875 >> 10.20.193.28:0/1132986875 conn(0x562438b82180 msgr2=0x562438c810e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f0787fff640 0 client.17811 ms_handle_reset on v2:10.20.193.167:6838/408350258 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07dee86e00 1 -- 10.20.193.28:0/1132986875 shutdown_connections 2026-02-25T11:59:00.652 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07dee86e00 1 -- 10.20.193.28:0/1132986875 wait complete. 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07dee86e00 20 client.17811 trim_cache size 0 max 16384 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout:{ 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "AsyncMessenger::Worker-0": { 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_messages": 825, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages": 672, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_bytes": 202268, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_bytes": 304712, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_created_connections": 11, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_active_connections": 3, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_total_time": 0.055908857, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_send_time": 0.026349960, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_recv_time": 0.030095648, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_fast_dispatch_time": 0.000172457, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages_queue_lat": { 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 673, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.031208606, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000046372 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_handle_ack_lat": { 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 1410, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000084664, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000060 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_encrypted_bytes": 0, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_encrypted_bytes": 0 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "AsyncMessenger::Worker-1": { 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_messages": 28, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages": 173, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_bytes": 5990, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_bytes": 395604, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_created_connections": 10, 2026-02-25T11:59:00.653 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_active_connections": 5, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_total_time": 0.012700502, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_send_time": 0.008474656, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_recv_time": 0.003305780, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_fast_dispatch_time": 0.000106127, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages_queue_lat": { 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 172, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.016877596, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000098125 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_handle_ack_lat": { 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 22, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000002889, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000131 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_encrypted_bytes": 160, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_encrypted_bytes": 328576 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "AsyncMessenger::Worker-2": { 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_messages": 333, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages": 328, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_bytes": 581214, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_bytes": 657100, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_created_connections": 46, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_active_connections": 8, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_total_time": 0.040161493, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_send_time": 0.016799489, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_recv_time": 0.027384519, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_running_fast_dispatch_time": 0.000053138, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_messages_queue_lat": { 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 326, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.015492214, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000047522 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_handle_ack_lat": { 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 142, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000008181, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000057 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_recv_encrypted_bytes": 555600, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "msgr_send_encrypted_bytes": 125152 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.654 INFO:teuthology.orchestra.run.trial028.stdout: "cct": { 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "total_workers": 1, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "unhealthy_workers": 0 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "finisher-MDSRank": { 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "queue_len": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "complete_latency": { 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 42, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000928939, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000022117 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "finisher-PurgeQueue": { 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "queue_len": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "complete_latency": { 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 2, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000012924, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000006462 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "mds": { 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "request": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "reply": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "reply_latency": { 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "slow_reply": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "forward": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "dir_fetch_complete": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "dir_fetch_keys": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "dir_commit": 11, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "dir_split": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "dir_merge": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "inodes": 16, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_top": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_bottom": 16, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_pin_tail": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_pinned": 13, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_expired": 4, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "inodes_with_caps": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "caps": 0, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "subtrees": 3, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "traverse": 9, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_hit": 5, 2026-02-25T11:59:00.655 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_forward": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_discover": 4, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_dir_fetch": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_remote_ino": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "traverse_lock": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "load_cent": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "q": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "exported": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "exported_inodes": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "imported": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "imported_inodes": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "openino_dir_fetch": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "openino_backtrace_fetch": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "openino_peer_discover": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "root_rfiles": 5, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "root_rbytes": 4330, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "root_rsnaps": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_backtrace_fetch": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_set_tag": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_backtrace_repaired": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_inotable_repaired": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_dir_inodes": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_dir_base_inodes": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_dirfrag_rstats": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "scrub_file_inodes": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "handle_inode_file_caps": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_revoke": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_grant": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_trunc": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_flushsnap_ack": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "ceph_cap_op_flush_ack": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_caps": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_caps_dirty": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_cap_release": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "process_request_cap_release": 0 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "mds_cache": { 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "dir_update": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "dir_update_receipt": 14, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "dir_try_discover": 9, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "dir_send_discover": 10, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "dir_handle_discover": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "num_strays": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "num_strays_delayed": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "num_strays_enqueuing": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "strays_created": 0, 2026-02-25T11:59:00.656 INFO:teuthology.orchestra.run.trial028.stdout: "strays_enqueued": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "strays_reintegrated": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "strays_migrated": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "num_recovering_processing": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "num_recovering_enqueued": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "num_recovering_prioritized": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "recovery_started": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "recovery_completed": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_quiesce_path": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_quiesce_inode": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_enqueue_scrub": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_exportdir": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_flush": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_fragmentdir": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_fragstats": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "ireq_inodestats": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_started": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_succeeded": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "uninline_write_failed": 0 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "mds_log": { 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "evlrg": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "evadd": 5, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "evex": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "evtrm": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "ev": 5, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "evexg": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "evexd": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "segadd": 2, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "segex": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "segtrm": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "seg": 2, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "segmjr": 2, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "segexg": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "segexd": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "expos": 4194304, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "wrpos": 4195956, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "rdpos": 0, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "jlat": { 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 4, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.004361696, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.001090424 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "replayed": 0 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.657 INFO:teuthology.orchestra.run.trial028.stdout: "mds_mem": { 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "ino": 20, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "ino+": 22, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "ino-": 2, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "dir": 19, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "dir+": 23, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "dir-": 4, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "dn": 16, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "dn+": 20, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "dn-": 4, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "cap": 0, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "cap+": 0, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "cap-": 0, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "rss": 48444, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "heap": 259747840 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "mds_server": { 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "dispatch_client_request": 0, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "dispatch_server_request": 0, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_request": 0, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "handle_client_session": 13, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "handle_peer_request": 0, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "req_create_latency": { 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "req_getattr_latency": { 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "req_getfilelock_latency": { 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "req_link_latency": { 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookup_latency": { 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.658 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookuphash_latency": { 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookupino_latency": { 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookupname_latency": { 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookupparent_latency": { 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "req_lookupsnap_latency": { 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "req_lssnap_latency": { 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "req_mkdir_latency": { 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "req_mknod_latency": { 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "req_mksnap_latency": { 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.659 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "req_open_latency": { 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "req_readdir_latency": { 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "req_rename_latency": { 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "req_renamesnap_latency": { 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "req_snapdiff_latency": { 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "req_rmdir_latency": { 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "req_rmsnap_latency": { 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "req_rmxattr_latency": { 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "req_setattr_latency": { 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.660 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "req_setdirlayout_latency": { 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "req_setfilelock_latency": { 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "req_setlayout_latency": { 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "req_setxattr_latency": { 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "req_symlink_latency": { 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "req_unlink_latency": { 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "cap_revoke_eviction": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "cache_trim_throttle": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "session_recall_throttle": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "session_recall_throttle2o": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "global_recall_throttle": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "cap_acquisition_throttle": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "req_getvxattr_latency": { 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "req_blockdiff_latency": { 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.661 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "mds_sessions": { 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "session_count": 1, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "session_add": 2, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "session_remove": 1, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "sessions_open": 1, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "sessions_stale": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "total_load": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "average_load": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "avg_session_uptime": 671, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "mdthresh_evicted": 0 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "mempool": { 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bloom_filter_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bloom_filter_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_alloc_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_alloc_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_data_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_data_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_onode_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_onode_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_meta_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_meta_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_other_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_other_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_buffer_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_cache_buffer_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_extent_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_extent_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_blob_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_blob_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_shared_blob_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_shared_blob_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_inline_bl_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_inline_bl_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_fsck_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_fsck_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_txc_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_txc_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_writing_deferred_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_writing_deferred_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_writing_bytes": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluestore_writing_items": 0, 2026-02-25T11:59:00.662 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_bytes": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_items": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_file_reader_bytes": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_file_reader_items": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_file_writer_bytes": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "bluefs_file_writer_items": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "buffer_anon_bytes": 230007, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "buffer_anon_items": 73, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "buffer_meta_bytes": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "buffer_meta_items": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "osd_bytes": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "osd_items": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "osd_mapbl_bytes": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "osd_mapbl_items": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "osd_pglog_bytes": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "osd_pglog_items": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_bytes": 7504, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_items": 153, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_mapping_bytes": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "osdmap_mapping_items": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "pgmap_bytes": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "pgmap_items": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "mds_co_bytes": 83692, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "mds_co_items": 434, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "ec_extent_cache_bytes": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "ec_extent_cache_items": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "unittest_1_bytes": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "unittest_1_items": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "unittest_2_bytes": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "unittest_2_items": 0 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "objecter": { 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "op_active": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "op_laggy": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "op_send": 40, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "op_send_bytes": 12196, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "op_resend": 33, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "op_reply": 40, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "op_latency": { 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 40, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.144998405, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.003624960 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "op_inflight": 0, 2026-02-25T11:59:00.663 INFO:teuthology.orchestra.run.trial028.stdout: "oplen_avg": { 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 62, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 40 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "op": 40, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "op_r": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "op_w": 40, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "op_rmw": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "op_pg": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_stat": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_create": 10, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_read": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_write": 4, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_writefull": 7, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_writesame": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_append": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_zero": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_truncate": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_delete": 6, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_mapext": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_sparse_read": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_clonerange": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_getxattr": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_setxattr": 20, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_cmpxattr": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_rmxattr": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_resetxattrs": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_call": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_watch": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_notify": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_src_cmpxattr": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_pgls": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_pgls_filter": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "osdop_other": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "linger_active": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "linger_send": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "linger_resend": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "linger_ping": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "poolop_active": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "poolop_send": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "poolop_resend": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "poolstat_active": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "poolstat_send": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "poolstat_resend": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "statfs_active": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "statfs_send": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "statfs_resend": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "command_active": 0, 2026-02-25T11:59:00.664 INFO:teuthology.orchestra.run.trial028.stdout: "command_send": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "command_resend": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "map_epoch": 88, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "map_full": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "map_inc": 2, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "osd_sessions": 8, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "osd_session_open": 8, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "osd_session_close": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "osd_laggy": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "omap_wr": 14, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "omap_rd": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "omap_del": 1, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "replica_read_sent": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "replica_read_bounced": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "replica_read_completed": 0 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "oft": { 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "omap_total_objs": 1, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "omap_total_kv_pairs": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "omap_total_updates": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "omap_total_removes": 0 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "purge_queue": { 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executing_ops": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executing_ops_high_water": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executing": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executing_high_water": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executed_ops": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "pq_executed": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "pq_item_in_journal": 0 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-msgr_dispatch_throttler-mds": { 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "max": 104857600, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "get": 1186, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 670651, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 1186, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "put": 1186, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 670651, 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.665 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-objecter_bytes": { 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "max": 104857600, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "get": 40, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 12196, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 40, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "put": 40, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 12196, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-objecter_ops": { 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "max": 1024, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "get": 40, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 40, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 40, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "put": 40, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 40, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-write_buf_throttle": { 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "max": 3758096384, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "get": 0, 2026-02-25T11:59:00.666 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "put": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "throttle-write_buf_throttle-0x2ea86780": { 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "val": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "max": 3758096384, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "get_started": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "get": 5, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "get_sum": 1652, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_fail": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "get_or_fail_success": 5, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "take": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "take_sum": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "put": 4, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "put_sum": 1652, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "wait": { 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "avgcount": 0, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "sum": 0.000000000, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "avgtime": 0.000000000 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: }, 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "trackedop": { 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: "slow_ops_count": 0 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout: } 2026-02-25T11:59:00.667 INFO:teuthology.orchestra.run.trial028.stdout:} 2026-02-25T11:59:00.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07db539640 1 -- 10.20.193.28:0/1566390224 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07b00b5760 msgr2=0x7f07b00d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07db539640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07b00b5760 0x7f07b00d5c10 secure :-1 s=READY pgs=414 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07c40098a0 tx=0x7f07c403a040 comp rx=0 tx=0).stop 2026-02-25T11:59:00.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07db539640 1 -- 10.20.193.28:0/1566390224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07d410e9b0 msgr2=0x7f07d40f90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:00.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07db539640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07d410e9b0 0x7f07d40f90c0 secure :-1 s=READY pgs=951 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f07d0006e90 tx=0x7f07d000edf0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07db539640 1 -- 10.20.193.28:0/1566390224 shutdown_connections 2026-02-25T11:59:00.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07db539640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f07b00b5760 0x7f07b00d5c10 unknown :-1 s=CLOSED pgs=414 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07db539640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f07d410e9b0 0x7f07d40f90c0 unknown :-1 s=CLOSED pgs=951 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07db539640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f07d410e000 0x7f07d416f340 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07db539640 1 --2- 10.20.193.28:0/1566390224 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f07d410ce70 0x7f07d40f8860 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:00.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.651+0000 7f07db539640 1 -- 10.20.193.28:0/1566390224 >> 10.20.193.28:0/1566390224 conn(0x7f07d40ea5c0 msgr2=0x7f07d4061850 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:00.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.652+0000 7f07db539640 1 -- 10.20.193.28:0/1566390224 shutdown_connections 2026-02-25T11:59:00.668 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:00.652+0000 7f07db539640 1 -- 10.20.193.28:0/1566390224 wait complete. 2026-02-25T11:59:00.668 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2026-02-25T11:59:00.668 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2026-02-25T11:59:00.668 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2026-02-25T11:59:00.668 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=14 2026-02-25T11:59:00.668 DEBUG:tasks.check_counter:Getting stats from l 2026-02-25T11:59:00.668 DEBUG:tasks.check_counter:skipping mds.24673 2026-02-25T11:59:00.669 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2026-02-25T11:59:00.680 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2026-02-25T11:59:00.680 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:59:00.680 DEBUG:teuthology.orchestra.run.trial028:> dd if=/proc/self/mounts of=/dev/stdout 2026-02-25T11:59:00.720 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:59:00.720 DEBUG:teuthology.orchestra.run.trial028:> dd if=/proc/self/mounts of=/dev/stdout 2026-02-25T11:59:00.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:00 trial167 ceph-mon[43392]: pgmap v549: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.8 KiB/s rd, 6 op/s 2026-02-25T11:59:00.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:00 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2206510807' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:59:00.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:00 trial119 ceph-mon[42710]: pgmap v549: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.8 KiB/s rd, 6 op/s 2026-02-25T11:59:00.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:00 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2206510807' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:59:00.797 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd blocklist ls 2026-02-25T11:59:01.021 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:59:01.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.199+0000 7f12b9519640 1 -- 10.20.193.28:0/2010098801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12b410f840 msgr2=0x7f12b410fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:01.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.199+0000 7f12b9519640 1 --2- 10.20.193.28:0/2010098801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12b410f840 0x7f12b410fc10 secure :-1 s=READY pgs=956 gs=1 cs=0 l=1 c_cookie=1491595099935288750 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f12a8009950 tx=0x7f12a802f190 comp rx=0 tx=0).stop 2026-02-25T11:59:01.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.199+0000 7f12b9519640 1 -- 10.20.193.28:0/2010098801 shutdown_connections 2026-02-25T11:59:01.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.199+0000 7f12b9519640 1 --2- 10.20.193.28:0/2010098801 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f12b4115a30 0x7f12b4176400 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.202 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.199+0000 7f12b9519640 1 --2- 10.20.193.28:0/2010098801 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f12b4110230 0x7f12b4115280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.199+0000 7f12b9519640 1 --2- 10.20.193.28:0/2010098801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12b410f840 0x7f12b410fc10 unknown :-1 s=CLOSED pgs=956 gs=1 cs=0 l=1 c_cookie=1491595099935288750 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.199+0000 7f12b9519640 1 -- 10.20.193.28:0/2010098801 >> 10.20.193.28:0/2010098801 conn(0x7f12b40ed130 msgr2=0x7f12b410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:01.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.199+0000 7f12b9519640 1 -- 10.20.193.28:0/2010098801 shutdown_connections 2026-02-25T11:59:01.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.199+0000 7f12b9519640 1 -- 10.20.193.28:0/2010098801 wait complete. 2026-02-25T11:59:01.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b9519640 1 Processor -- start 2026-02-25T11:59:01.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b9519640 1 -- start start 2026-02-25T11:59:01.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b9519640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f12b410f840 0x7f12b40f5b40 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:01.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b9519640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12b4110230 0x7f12b4176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:01.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b9519640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f12b4115a30 0x7f12b40f6240 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:01.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b9519640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f12b40fb450 con 0x7f12b4110230 2026-02-25T11:59:01.203 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b9519640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f12b40fb5c0 con 0x7f12b410f840 2026-02-25T11:59:01.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b9519640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12b40fb780 con 0x7f12b4115a30 2026-02-25T11:59:01.204 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b27fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12b4110230 0x7f12b4176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:01.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b27fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12b4110230 0x7f12b4176300 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34910/0 (socket says 10.20.193.28:34910) 2026-02-25T11:59:01.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b27fc640 1 -- 10.20.193.28:0/3333226384 learned_addr learned my addr 10.20.193.28:0/3333226384 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:01.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b2ffd640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f12b410f840 0x7f12b40f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:01.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b37fe640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f12b4115a30 0x7f12b40f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:01.205 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b27fc640 1 -- 10.20.193.28:0/3333226384 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f12b4115a30 msgr2=0x7f12b40f6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:01.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b27fc640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f12b4115a30 0x7f12b40f6240 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b27fc640 1 -- 10.20.193.28:0/3333226384 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f12b410f840 msgr2=0x7f12b40f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:01.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b27fc640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f12b410f840 0x7f12b40f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.200+0000 7f12b27fc640 1 -- 10.20.193.28:0/3333226384 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a8009570 con 0x7f12b4110230 2026-02-25T11:59:01.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.201+0000 7f12b27fc640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12b4110230 0x7f12b4176300 secure :-1 s=READY pgs=957 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f129c009480 tx=0x7f129c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:01.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.201+0000 7f1293fff640 1 -- 10.20.193.28:0/3333226384 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f129c00cd40 con 0x7f12b4110230 2026-02-25T11:59:01.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.201+0000 7f1293fff640 1 -- 10.20.193.28:0/3333226384 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f129c004d10 con 0x7f12b4110230 2026-02-25T11:59:01.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.201+0000 7f1293fff640 1 -- 10.20.193.28:0/3333226384 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f129c0106b0 con 0x7f12b4110230 2026-02-25T11:59:01.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.201+0000 7f12b9519640 1 -- 10.20.193.28:0/3333226384 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b40fba60 con 0x7f12b4110230 2026-02-25T11:59:01.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.201+0000 7f12b9519640 1 -- 10.20.193.28:0/3333226384 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b40fbee0 con 0x7f12b4110230 2026-02-25T11:59:01.206 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.202+0000 7f1291ffb640 1 -- 10.20.193.28:0/3333226384 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1278005300 con 0x7f12b4110230 2026-02-25T11:59:01.207 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.204+0000 7f1293fff640 1 -- 10.20.193.28:0/3333226384 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f129c01f060 con 0x7f12b4110230 2026-02-25T11:59:01.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.206+0000 7f1293fff640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f12880b5750 0x7f12880d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:01.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.206+0000 7f12b2ffd640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f12880b5750 0x7f12880d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:01.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.206+0000 7f1293fff640 1 -- 10.20.193.28:0/3333226384 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f129c014070 con 0x7f12b4110230 2026-02-25T11:59:01.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.206+0000 7f1293fff640 1 -- 10.20.193.28:0/3333226384 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f129c0e7330 con 0x7f12b4110230 2026-02-25T11:59:01.208 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.207+0000 7f12b2ffd640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f12880b5750 0x7f12880d5c00 secure :-1 s=READY pgs=415 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f12a80023d0 tx=0x7f12a8031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:01.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.328+0000 7f1291ffb640 1 -- 10.20.193.28:0/3333226384 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1278005950 con 0x7f12b4110230 2026-02-25T11:59:01.330 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.328+0000 7f1293fff640 1 -- 10.20.193.28:0/3333226384 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+590 (secure 0 0 0) 0x7f129c0a9a10 con 0x7f12b4110230 2026-02-25T11:59:01.331 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:0/1467902196 2026-02-26T11:43:17.037464+0000 2026-02-25T11:59:01.331 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:6801/2907029718 2026-02-26T11:43:17.037464+0000 2026-02-25T11:59:01.331 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:6800/2907029718 2026-02-26T11:43:17.037464+0000 2026-02-25T11:59:01.331 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:0/2274270913 2026-02-26T11:43:17.037464+0000 2026-02-25T11:59:01.331 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:0/60087422 2026-02-26T11:43:17.037464+0000 2026-02-25T11:59:01.331 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:6801/3221634678 2026-02-26T11:43:04.102029+0000 2026-02-25T11:59:01.331 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:6800/3221634678 2026-02-26T11:43:04.102029+0000 2026-02-25T11:59:01.331 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:0/1725332841 2026-02-26T11:43:04.102029+0000 2026-02-25T11:59:01.331 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:0/3082630924 2026-02-26T11:43:04.102029+0000 2026-02-25T11:59:01.332 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:0/2391715078 2026-02-26T11:43:04.102029+0000 2026-02-25T11:59:01.332 INFO:teuthology.orchestra.run.trial028.stderr:listed 10 entries 2026-02-25T11:59:01.333 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.329+0000 7f1291ffb640 1 -- 10.20.193.28:0/3333226384 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f12880b5750 msgr2=0x7f12880d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:01.333 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.329+0000 7f1291ffb640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f12880b5750 0x7f12880d5c00 secure :-1 s=READY pgs=415 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f12a80023d0 tx=0x7f12a8031040 comp rx=0 tx=0).stop 2026-02-25T11:59:01.333 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.329+0000 7f1291ffb640 1 -- 10.20.193.28:0/3333226384 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12b4110230 msgr2=0x7f12b4176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:01.333 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.329+0000 7f1291ffb640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12b4110230 0x7f12b4176300 secure :-1 s=READY pgs=957 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f129c009480 tx=0x7f129c00ee60 comp rx=0 tx=0).stop 2026-02-25T11:59:01.333 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.329+0000 7f1291ffb640 1 -- 10.20.193.28:0/3333226384 shutdown_connections 2026-02-25T11:59:01.333 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.329+0000 7f1291ffb640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f12880b5750 0x7f12880d5c00 unknown :-1 s=CLOSED pgs=415 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.333 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.330+0000 7f1291ffb640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f12b4115a30 0x7f12b40f6240 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.333 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.330+0000 7f1291ffb640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f12b4110230 0x7f12b4176300 unknown :-1 s=CLOSED pgs=957 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.333 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.330+0000 7f1291ffb640 1 --2- 10.20.193.28:0/3333226384 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f12b410f840 0x7f12b40f5b40 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.333 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.330+0000 7f1291ffb640 1 -- 10.20.193.28:0/3333226384 >> 10.20.193.28:0/3333226384 conn(0x7f12b40ed130 msgr2=0x7f12b4111c80 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:01.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.330+0000 7f1291ffb640 1 -- 10.20.193.28:0/3333226384 shutdown_connections 2026-02-25T11:59:01.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.330+0000 7f1291ffb640 1 -- 10.20.193.28:0/3333226384 wait complete. 2026-02-25T11:59:01.472 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-25T11:59:01.472 DEBUG:teuthology.orchestra.run.trial028:> dd if=/proc/self/mounts of=/dev/stdout 2026-02-25T11:59:01.511 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd blocklist ls 2026-02-25T11:59:01.734 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:59:01.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.915+0000 7f6b19490640 1 -- 10.20.193.28:0/1141127694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b1410ddc0 msgr2=0x7f6b14112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:01.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.915+0000 7f6b19490640 1 --2- 10.20.193.28:0/1141127694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b1410ddc0 0x7f6b14112e10 secure :-1 s=READY pgs=958 gs=1 cs=0 l=1 c_cookie=1375349055704383879 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6b04009950 tx=0x7f6b0402f190 comp rx=0 tx=0).stop 2026-02-25T11:59:01.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.916+0000 7f6b19490640 1 -- 10.20.193.28:0/1141127694 shutdown_connections 2026-02-25T11:59:01.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.916+0000 7f6b19490640 1 --2- 10.20.193.28:0/1141127694 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b141135c0 0x7f6b14173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.916+0000 7f6b19490640 1 --2- 10.20.193.28:0/1141127694 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b1410ddc0 0x7f6b14112e10 unknown :-1 s=CLOSED pgs=958 gs=1 cs=0 l=1 c_cookie=1375349055704383879 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.916+0000 7f6b19490640 1 --2- 10.20.193.28:0/1141127694 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b1410d3d0 0x7f6b1410d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.917 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.916+0000 7f6b19490640 1 -- 10.20.193.28:0/1141127694 >> 10.20.193.28:0/1141127694 conn(0x7f6b140eaca0 msgr2=0x7f6b1410b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.916+0000 7f6b19490640 1 -- 10.20.193.28:0/1141127694 shutdown_connections 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.916+0000 7f6b19490640 1 -- 10.20.193.28:0/1141127694 wait complete. 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.916+0000 7f6b19490640 1 Processor -- start 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.916+0000 7f6b19490640 1 -- start start 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b19490640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b1410d3d0 0x7f6b140f59e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b19490640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b1410ddc0 0x7f6b14173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b19490640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b141135c0 0x7f6b140f60e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b19490640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6b140fb2f0 con 0x7f6b141135c0 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b19490640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6b140fb460 con 0x7f6b1410ddc0 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b19490640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6b140fb620 con 0x7f6b1410d3d0 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b18c8f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b141135c0 0x7f6b140f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b18c8f640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b141135c0 0x7f6b140f60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:34926/0 (socket says 10.20.193.28:34926) 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b18c8f640 1 -- 10.20.193.28:0/2309383801 learned_addr learned my addr 10.20.193.28:0/2309383801 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:01.918 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b137fe640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b1410ddc0 0x7f6b14173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b13fff640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b1410d3d0 0x7f6b140f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b18c8f640 1 -- 10.20.193.28:0/2309383801 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b1410d3d0 msgr2=0x7f6b140f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b18c8f640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b1410d3d0 0x7f6b140f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b18c8f640 1 -- 10.20.193.28:0/2309383801 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b1410ddc0 msgr2=0x7f6b14173e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b18c8f640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b1410ddc0 0x7f6b14173e90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b18c8f640 1 -- 10.20.193.28:0/2309383801 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b04009570 con 0x7f6b141135c0 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b13fff640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b1410d3d0 0x7f6b140f59e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b137fe640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b1410ddc0 0x7f6b14173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.917+0000 7f6b18c8f640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b141135c0 0x7f6b140f60e0 secure :-1 s=READY pgs=959 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6b08006e90 tx=0x7f6b0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.918+0000 7f6b117fa640 1 -- 10.20.193.28:0/2309383801 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6b08005500 con 0x7f6b141135c0 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.918+0000 7f6b117fa640 1 -- 10.20.193.28:0/2309383801 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6b08005df0 con 0x7f6b141135c0 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.918+0000 7f6b19490640 1 -- 10.20.193.28:0/2309383801 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b140fb900 con 0x7f6b141135c0 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.918+0000 7f6b19490640 1 -- 10.20.193.28:0/2309383801 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b140fbdd0 con 0x7f6b141135c0 2026-02-25T11:59:01.919 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.918+0000 7f6b117fa640 1 -- 10.20.193.28:0/2309383801 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6b08003a00 con 0x7f6b141135c0 2026-02-25T11:59:01.920 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.919+0000 7f6b19490640 1 -- 10.20.193.28:0/2309383801 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1410e6b0 con 0x7f6b141135c0 2026-02-25T11:59:01.922 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.921+0000 7f6b117fa640 1 -- 10.20.193.28:0/2309383801 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f6b08003b60 con 0x7f6b141135c0 2026-02-25T11:59:01.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.923+0000 7f6b117fa640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ae80b56d0 0x7f6ae80d5b80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:01.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.923+0000 7f6b117fa640 1 -- 10.20.193.28:0/2309383801 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6b080e7c20 con 0x7f6b141135c0 2026-02-25T11:59:01.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.923+0000 7f6b117fa640 1 -- 10.20.193.28:0/2309383801 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f6b08120940 con 0x7f6b141135c0 2026-02-25T11:59:01.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.923+0000 7f6b13fff640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ae80b56d0 0x7f6ae80d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:01.925 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:01.923+0000 7f6b13fff640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ae80b56d0 0x7f6ae80d5b80 secure :-1 s=READY pgs=416 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6b00004740 tx=0x7f6b00005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:02.046 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.045+0000 7f6b19490640 1 -- 10.20.193.28:0/2309383801 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6b1410d7a0 con 0x7f6b141135c0 2026-02-25T11:59:02.047 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.045+0000 7f6b117fa640 1 -- 10.20.193.28:0/2309383801 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+590 (secure 0 0 0) 0x7f6b080aa5f0 con 0x7f6b141135c0 2026-02-25T11:59:02.047 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:0/1467902196 2026-02-26T11:43:17.037464+0000 2026-02-25T11:59:02.047 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:6801/2907029718 2026-02-26T11:43:17.037464+0000 2026-02-25T11:59:02.047 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:6800/2907029718 2026-02-26T11:43:17.037464+0000 2026-02-25T11:59:02.047 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:0/2274270913 2026-02-26T11:43:17.037464+0000 2026-02-25T11:59:02.047 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:0/60087422 2026-02-26T11:43:17.037464+0000 2026-02-25T11:59:02.047 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:6801/3221634678 2026-02-26T11:43:04.102029+0000 2026-02-25T11:59:02.047 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:6800/3221634678 2026-02-26T11:43:04.102029+0000 2026-02-25T11:59:02.047 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:0/1725332841 2026-02-26T11:43:04.102029+0000 2026-02-25T11:59:02.047 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:0/3082630924 2026-02-26T11:43:04.102029+0000 2026-02-25T11:59:02.047 INFO:teuthology.orchestra.run.trial028.stdout:10.20.193.28:0/2391715078 2026-02-26T11:43:04.102029+0000 2026-02-25T11:59:02.047 INFO:teuthology.orchestra.run.trial028.stderr:listed 10 entries 2026-02-25T11:59:02.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.046+0000 7f6b19490640 1 -- 10.20.193.28:0/2309383801 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ae80b56d0 msgr2=0x7f6ae80d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:02.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.046+0000 7f6b19490640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ae80b56d0 0x7f6ae80d5b80 secure :-1 s=READY pgs=416 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6b00004740 tx=0x7f6b00005f00 comp rx=0 tx=0).stop 2026-02-25T11:59:02.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.046+0000 7f6b19490640 1 -- 10.20.193.28:0/2309383801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b141135c0 msgr2=0x7f6b140f60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:02.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.046+0000 7f6b19490640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b141135c0 0x7f6b140f60e0 secure :-1 s=READY pgs=959 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6b08006e90 tx=0x7f6b0800edf0 comp rx=0 tx=0).stop 2026-02-25T11:59:02.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.047+0000 7f6b19490640 1 -- 10.20.193.28:0/2309383801 shutdown_connections 2026-02-25T11:59:02.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.047+0000 7f6b19490640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6ae80b56d0 0x7f6ae80d5b80 unknown :-1 s=CLOSED pgs=416 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:02.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.047+0000 7f6b19490640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6b141135c0 0x7f6b140f60e0 unknown :-1 s=CLOSED pgs=959 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:02.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.047+0000 7f6b19490640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6b1410ddc0 0x7f6b14173e90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:02.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.047+0000 7f6b19490640 1 --2- 10.20.193.28:0/2309383801 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6b1410d3d0 0x7f6b140f59e0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:02.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.047+0000 7f6b19490640 1 -- 10.20.193.28:0/2309383801 >> 10.20.193.28:0/2309383801 conn(0x7f6b140eaca0 msgr2=0x7f6b1410fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:02.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.047+0000 7f6b19490640 1 -- 10.20.193.28:0/2309383801 shutdown_connections 2026-02-25T11:59:02.048 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:02.047+0000 7f6b19490640 1 -- 10.20.193.28:0/2309383801 wait complete. 2026-02-25T11:59:02.146 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@trial028.front.sepia.ceph.com... 2026-02-25T11:59:02.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:59:02.146 DEBUG:teuthology.orchestra.run.trial028:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2026-02-25T11:59:02.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.trial028.stderr:ceph-fuse[113767]: fuse finished with error 0 and tester_r 0 2026-02-25T11:59:02.204 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T11:59:02.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:02 trial028 ceph-mon[39224]: pgmap v550: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.4 KiB/s rd, 5 op/s 2026-02-25T11:59:02.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:02 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3333226384' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2026-02-25T11:59:02.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:02 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/2309383801' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2026-02-25T11:59:02.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:02 trial167 ceph-mon[43392]: pgmap v550: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.4 KiB/s rd, 5 op/s 2026-02-25T11:59:02.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:02 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3333226384' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2026-02-25T11:59:02.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:02 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/2309383801' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2026-02-25T11:59:02.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:02 trial119 ceph-mon[42710]: pgmap v550: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.4 KiB/s rd, 5 op/s 2026-02-25T11:59:02.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:02 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3333226384' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2026-02-25T11:59:02.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:02 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/2309383801' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2026-02-25T11:59:04.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:04 trial028 ceph-mon[39224]: pgmap v551: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.7 KiB/s rd, 6 op/s 2026-02-25T11:59:04.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:04 trial167 ceph-mon[43392]: pgmap v551: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.7 KiB/s rd, 6 op/s 2026-02-25T11:59:04.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:04 trial119 ceph-mon[42710]: pgmap v551: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.7 KiB/s rd, 6 op/s 2026-02-25T11:59:06.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:06 trial028 ceph-mon[39224]: pgmap v552: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2026-02-25T11:59:06.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:06 trial167 ceph-mon[43392]: pgmap v552: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2026-02-25T11:59:06.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:06 trial119 ceph-mon[42710]: pgmap v552: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2026-02-25T11:59:08.306 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@trial028.front.sepia.ceph.com 2026-02-25T11:59:08.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:59:08.306 DEBUG:teuthology.orchestra.run.trial028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2026-02-25T11:59:08.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:59:08.347 DEBUG:teuthology.orchestra.run.trial028:> ip netns list 2026-02-25T11:59:08.425 INFO:teuthology.orchestra.run.trial028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2026-02-25T11:59:08.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:59:08.426 DEBUG:teuthology.orchestra.run.trial028:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2026-02-25T11:59:08.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2026-02-25T11:59:08.524 DEBUG:teuthology.orchestra.run.trial028:> sudo ip link delete ceph-brx 2026-02-25T11:59:08.559 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:08 trial028 ceph-mon[39224]: pgmap v553: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2026-02-25T11:59:08.645 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2026-02-25T11:59:08.661 INFO:tasks.cephadm:Teardown begin 2026-02-25T11:59:08.661 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2026-02-25T11:59:08.662 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:59:08.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:08 trial167 ceph-mon[43392]: pgmap v553: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2026-02-25T11:59:08.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:08 trial119 ceph-mon[42710]: pgmap v553: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.5 KiB/s rd, 511 B/s wr, 7 op/s 2026-02-25T11:59:08.873 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:59:09.058 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.055+0000 7f8a9594b640 1 -- 10.20.193.28:0/2468511128 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a90062be0 msgr2=0x7f8a900610b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:09.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.055+0000 7f8a9594b640 1 --2- 10.20.193.28:0/2468511128 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a90062be0 0x7f8a900610b0 secure :-1 s=READY pgs=960 gs=1 cs=0 l=1 c_cookie=12837346234848504807 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a84009950 tx=0x7f8a8402f190 comp rx=0 tx=0).stop 2026-02-25T11:59:09.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.055+0000 7f8a9594b640 1 -- 10.20.193.28:0/2468511128 shutdown_connections 2026-02-25T11:59:09.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.055+0000 7f8a9594b640 1 --2- 10.20.193.28:0/2468511128 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8a9015a110 0x7f8a9017a4f0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.055+0000 7f8a9594b640 1 --2- 10.20.193.28:0/2468511128 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8a900615f0 0x7f8a90061a20 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.055+0000 7f8a9594b640 1 --2- 10.20.193.28:0/2468511128 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a90062be0 0x7f8a900610b0 unknown :-1 s=CLOSED pgs=960 gs=1 cs=0 l=1 c_cookie=12837346234848504807 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.059 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.055+0000 7f8a9594b640 1 -- 10.20.193.28:0/2468511128 >> 10.20.193.28:0/2468511128 conn(0x7f8a900eae90 msgr2=0x7f8a9010b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.055+0000 7f8a9594b640 1 -- 10.20.193.28:0/2468511128 shutdown_connections 2026-02-25T11:59:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.055+0000 7f8a9594b640 1 -- 10.20.193.28:0/2468511128 wait complete. 2026-02-25T11:59:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a9594b640 1 Processor -- start 2026-02-25T11:59:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a9594b640 1 -- start start 2026-02-25T11:59:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a9594b640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8a900615f0 0x7f8a900f32e0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a9594b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a90062be0 0x7f8a900f3820 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a9594b640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8a9015a110 0x7f8a900f8bf0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a9594b640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8a900f9180 con 0x7f8a90062be0 2026-02-25T11:59:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a9594b640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8a900f92f0 con 0x7f8a900615f0 2026-02-25T11:59:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a9594b640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8a900f94b0 con 0x7f8a9015a110 2026-02-25T11:59:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a8e7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a90062be0 0x7f8a900f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:09.060 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a8e7fc640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a90062be0 0x7f8a900f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35026/0 (socket says 10.20.193.28:35026) 2026-02-25T11:59:09.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a8e7fc640 1 -- 10.20.193.28:0/3609561329 learned_addr learned my addr 10.20.193.28:0/3609561329 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:09.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a8effd640 1 --2- 10.20.193.28:0/3609561329 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8a900615f0 0x7f8a900f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:09.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a8e7fc640 1 -- 10.20.193.28:0/3609561329 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8a9015a110 msgr2=0x7f8a900f8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:09.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a8e7fc640 1 --2- 10.20.193.28:0/3609561329 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8a9015a110 0x7f8a900f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a8e7fc640 1 -- 10.20.193.28:0/3609561329 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8a900615f0 msgr2=0x7f8a900f32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:09.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a8e7fc640 1 --2- 10.20.193.28:0/3609561329 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8a900615f0 0x7f8a900f32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.056+0000 7f8a8e7fc640 1 -- 10.20.193.28:0/3609561329 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a84009570 con 0x7f8a90062be0 2026-02-25T11:59:09.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.057+0000 7f8a8e7fc640 1 --2- 10.20.193.28:0/3609561329 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a90062be0 0x7f8a900f3820 secure :-1 s=READY pgs=961 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a7800d950 tx=0x7f8a7800de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:09.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.057+0000 7f8a94949640 1 -- 10.20.193.28:0/3609561329 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a78004540 con 0x7f8a90062be0 2026-02-25T11:59:09.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.057+0000 7f8a94949640 1 -- 10.20.193.28:0/3609561329 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f8a78009ce0 con 0x7f8a90062be0 2026-02-25T11:59:09.061 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.057+0000 7f8a9594b640 1 -- 10.20.193.28:0/3609561329 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a900f9790 con 0x7f8a90062be0 2026-02-25T11:59:09.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.057+0000 7f8a9594b640 1 -- 10.20.193.28:0/3609561329 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a900f9f00 con 0x7f8a90062be0 2026-02-25T11:59:09.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.057+0000 7f8a94949640 1 -- 10.20.193.28:0/3609561329 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f8a780106e0 con 0x7f8a90062be0 2026-02-25T11:59:09.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.057+0000 7f8a9594b640 1 -- 10.20.193.28:0/3609561329 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a54005300 con 0x7f8a90062be0 2026-02-25T11:59:09.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.059+0000 7f8a94949640 1 -- 10.20.193.28:0/3609561329 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f8a780097c0 con 0x7f8a90062be0 2026-02-25T11:59:09.063 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.061+0000 7f8a94949640 1 --2- 10.20.193.28:0/3609561329 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8a640b5760 0x7f8a640d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:09.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.061+0000 7f8a94949640 1 -- 10.20.193.28:0/3609561329 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f8a780e72e0 con 0x7f8a90062be0 2026-02-25T11:59:09.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.061+0000 7f8a8effd640 1 --2- 10.20.193.28:0/3609561329 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8a640b5760 0x7f8a640d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:09.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.061+0000 7f8a94949640 1 -- 10.20.193.28:0/3609561329 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f8a780e7760 con 0x7f8a90062be0 2026-02-25T11:59:09.064 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.062+0000 7f8a8effd640 1 --2- 10.20.193.28:0/3609561329 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8a640b5760 0x7f8a640d5c10 secure :-1 s=READY pgs=417 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a840023d0 tx=0x7f8a84031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:09.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.207+0000 7f8a9594b640 1 -- 10.20.193.28:0/3609561329 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a54005950 con 0x7f8a90062be0 2026-02-25T11:59:09.209 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.207+0000 7f8a94949640 1 -- 10.20.193.28:0/3609561329 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f8a780a9cb0 con 0x7f8a90062be0 2026-02-25T11:59:09.210 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:59:09.210 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:59:09.211 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:59:09.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.209+0000 7f8a9594b640 1 -- 10.20.193.28:0/3609561329 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8a640b5760 msgr2=0x7f8a640d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:09.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.209+0000 7f8a9594b640 1 --2- 10.20.193.28:0/3609561329 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8a640b5760 0x7f8a640d5c10 secure :-1 s=READY pgs=417 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a840023d0 tx=0x7f8a84031040 comp rx=0 tx=0).stop 2026-02-25T11:59:09.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.209+0000 7f8a9594b640 1 -- 10.20.193.28:0/3609561329 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a90062be0 msgr2=0x7f8a900f3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:09.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.209+0000 7f8a9594b640 1 --2- 10.20.193.28:0/3609561329 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a90062be0 0x7f8a900f3820 secure :-1 s=READY pgs=961 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f8a7800d950 tx=0x7f8a7800de20 comp rx=0 tx=0).stop 2026-02-25T11:59:09.211 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.209+0000 7f8a9594b640 1 -- 10.20.193.28:0/3609561329 shutdown_connections 2026-02-25T11:59:09.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.209+0000 7f8a9594b640 1 --2- 10.20.193.28:0/3609561329 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f8a640b5760 0x7f8a640d5c10 unknown :-1 s=CLOSED pgs=417 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.209+0000 7f8a9594b640 1 --2- 10.20.193.28:0/3609561329 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f8a9015a110 0x7f8a900f8bf0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.209+0000 7f8a9594b640 1 --2- 10.20.193.28:0/3609561329 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f8a90062be0 0x7f8a900f3820 unknown :-1 s=CLOSED pgs=961 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.209+0000 7f8a9594b640 1 --2- 10.20.193.28:0/3609561329 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f8a900615f0 0x7f8a900f32e0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.209+0000 7f8a9594b640 1 -- 10.20.193.28:0/3609561329 >> 10.20.193.28:0/3609561329 conn(0x7f8a900eae90 msgr2=0x7f8a9010e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:09.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.209+0000 7f8a9594b640 1 -- 10.20.193.28:0/3609561329 shutdown_connections 2026-02-25T11:59:09.212 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.209+0000 7f8a9594b640 1 -- 10.20.193.28:0/3609561329 wait complete. 2026-02-25T11:59:09.315 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs dump --format=json 2026-02-25T11:59:09.502 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:59:09.545 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:09 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/3609561329' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:59:09.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.677+0000 7f6c8de9c640 1 -- 10.20.193.28:0/3148350324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6c8810ddc0 msgr2=0x7f6c88112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:09.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.677+0000 7f6c8de9c640 1 --2- 10.20.193.28:0/3148350324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6c8810ddc0 0x7f6c88112e10 secure :-1 s=READY pgs=962 gs=1 cs=0 l=1 c_cookie=13938869764797079207 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6c70009950 tx=0x7f6c7002f1d0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.677+0000 7f6c8de9c640 1 -- 10.20.193.28:0/3148350324 shutdown_connections 2026-02-25T11:59:09.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.677+0000 7f6c8de9c640 1 --2- 10.20.193.28:0/3148350324 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6c881135c0 0x7f6c88173f90 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.677+0000 7f6c8de9c640 1 --2- 10.20.193.28:0/3148350324 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6c8810ddc0 0x7f6c88112e10 unknown :-1 s=CLOSED pgs=962 gs=1 cs=0 l=1 c_cookie=13938869764797079207 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.677+0000 7f6c8de9c640 1 --2- 10.20.193.28:0/3148350324 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6c8810d3d0 0x7f6c8810d7a0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.680 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.677+0000 7f6c8de9c640 1 -- 10.20.193.28:0/3148350324 >> 10.20.193.28:0/3148350324 conn(0x7f6c880eaca0 msgr2=0x7f6c8810b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:09.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.677+0000 7f6c8de9c640 1 -- 10.20.193.28:0/3148350324 shutdown_connections 2026-02-25T11:59:09.681 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.678+0000 7f6c8de9c640 1 -- 10.20.193.28:0/3148350324 wait complete. 2026-02-25T11:59:09.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.678+0000 7f6c8de9c640 1 Processor -- start 2026-02-25T11:59:09.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.678+0000 7f6c8de9c640 1 -- start start 2026-02-25T11:59:09.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c8de9c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6c8810d3d0 0x7f6c880f5a80 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:09.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c8de9c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6c8810ddc0 0x7f6c88173e90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:09.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c8de9c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6c881135c0 0x7f6c880f6180 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:09.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c8de9c640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6c880fb390 con 0x7f6c8810ddc0 2026-02-25T11:59:09.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c8de9c640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6c880fb500 con 0x7f6c8810d3d0 2026-02-25T11:59:09.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c8de9c640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6c880fb6c0 con 0x7f6c881135c0 2026-02-25T11:59:09.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c7ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6c8810ddc0 0x7f6c88173e90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:09.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c7ffff640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6c8810ddc0 0x7f6c88173e90 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35034/0 (socket says 10.20.193.28:35034) 2026-02-25T11:59:09.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c7ffff640 1 -- 10.20.193.28:0/4133661598 learned_addr learned my addr 10.20.193.28:0/4133661598 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:09.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c8ce9a640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6c8810d3d0 0x7f6c880f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:09.683 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c8d69b640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6c881135c0 0x7f6c880f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:09.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c7ffff640 1 -- 10.20.193.28:0/4133661598 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6c881135c0 msgr2=0x7f6c880f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:09.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c7ffff640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6c881135c0 0x7f6c880f6180 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c7ffff640 1 -- 10.20.193.28:0/4133661598 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6c8810d3d0 msgr2=0x7f6c880f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:09.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c7ffff640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6c8810d3d0 0x7f6c880f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c7ffff640 1 -- 10.20.193.28:0/4133661598 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c70009570 con 0x7f6c8810ddc0 2026-02-25T11:59:09.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.679+0000 7f6c7ffff640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6c8810ddc0 0x7f6c88173e90 secure :-1 s=READY pgs=963 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6c700098a0 tx=0x7f6c700028d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:09.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.680+0000 7f6c7dffb640 1 -- 10.20.193.28:0/4133661598 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6c7003d070 con 0x7f6c8810ddc0 2026-02-25T11:59:09.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.680+0000 7f6c7dffb640 1 -- 10.20.193.28:0/4133661598 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6c70002a70 con 0x7f6c8810ddc0 2026-02-25T11:59:09.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.680+0000 7f6c7dffb640 1 -- 10.20.193.28:0/4133661598 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6c70041890 con 0x7f6c8810ddc0 2026-02-25T11:59:09.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.680+0000 7f6c8de9c640 1 -- 10.20.193.28:0/4133661598 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c880fb940 con 0x7f6c8810ddc0 2026-02-25T11:59:09.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.680+0000 7f6c8de9c640 1 -- 10.20.193.28:0/4133661598 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c880fbe00 con 0x7f6c8810ddc0 2026-02-25T11:59:09.684 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.680+0000 7f6c577fe640 1 -- 10.20.193.28:0/4133661598 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c8810e6b0 con 0x7f6c8810ddc0 2026-02-25T11:59:09.685 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.683+0000 7f6c7dffb640 1 -- 10.20.193.28:0/4133661598 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f6c70041a20 con 0x7f6c8810ddc0 2026-02-25T11:59:09.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.684+0000 7f6c7dffb640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6c640b5750 0x7f6c640d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:09.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.684+0000 7f6c7dffb640 1 -- 10.20.193.28:0/4133661598 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f6c7010b4a0 con 0x7f6c8810ddc0 2026-02-25T11:59:09.686 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.684+0000 7f6c7dffb640 1 -- 10.20.193.28:0/4133661598 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f6c7010b920 con 0x7f6c8810ddc0 2026-02-25T11:59:09.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.684+0000 7f6c8ce9a640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6c640b5750 0x7f6c640d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:09.687 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.685+0000 7f6c8ce9a640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6c640b5750 0x7f6c640d5c00 secure :-1 s=READY pgs=418 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6c78004740 tx=0x7f6c7800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:09.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:09 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/3609561329' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:59:09.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:09 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/3609561329' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:59:09.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.827+0000 7f6c577fe640 1 -- 10.20.193.28:0/4133661598 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c8810d7a0 con 0x7f6c8810ddc0 2026-02-25T11:59:09.829 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.828+0000 7f6c7dffb640 1 -- 10.20.193.28:0/4133661598 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11940 (secure 0 0 0) 0x7f6c700cde70 con 0x7f6c8810ddc0 2026-02-25T11:59:09.830 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:59:09.830 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":23,"btime":"2026-02-25T11:47:48:170765+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14787,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6835/685551537","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6834","nonce":685551537},{"type":"v1","addr":"10.20.193.28:6835","nonce":685551537}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14799,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6837/281937174","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6836","nonce":281937174},{"type":"v1","addr":"10.20.193.28:6837","nonce":281937174}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.28:6841/1424760953","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6840","nonce":1424760953},{"type":"v1","addr":"10.20.193.28:6841","nonce":1424760953}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24671,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6833/4164243082","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6832","nonce":4164243082},{"type":"v1","addr":"10.20.193.119:6833","nonce":4164243082}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.167:6833/478500593","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6832","nonce":478500593},{"type":"v1","addr":"10.20.193.167:6833","nonce":478500593}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24694,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"10.20.193.119:6835/3262473702","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6834","nonce":3262473702},{"type":"v1","addr":"10.20.193.119:6835","nonce":3262473702}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true,"allow_referent_inodes":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2026-02-25T11:47:13.223613+0000","modified":"2026-02-25T11:47:47.692878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24749,"mds_1":24731,"mds_2":24730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24706":{"gid":24706,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6835/3681115733","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6834","nonce":3681115733},{"type":"v1","addr":"10.20.193.167:6835","nonce":3681115733}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24707":{"gid":24707,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.167:6837/3206735083","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6836","nonce":3206735083},{"type":"v1","addr":"10.20.193.167:6837","nonce":3206735083}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24719":{"gid":24719,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"10.20.193.119:6837/392187157","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6836","nonce":392187157},{"type":"v1","addr":"10.20.193.119:6837","nonce":392187157}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24730":{"gid":24730,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":5,"addr":"10.20.193.167:6839/408350258","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6838","nonce":408350258},{"type":"v1","addr":"10.20.193.167:6839","nonce":408350258}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24731":{"gid":24731,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"10.20.193.28:6839/3137382094","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6838","nonce":3137382094},{"type":"v1","addr":"10.20.193.28:6839","nonce":3137382094}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24749":{"gid":24749,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"10.20.193.119:6839/1063465486","addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6838","nonce":1063465486},{"type":"v1","addr":"10.20.193.119:6839","nonce":1063465486}]},"join_fscid":-1,"export_targets":[],"features":4544132024016699391,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24749,"qdb_cluster":[24749,24731,24730]},"id":1}]} 2026-02-25T11:59:09.830 INFO:teuthology.orchestra.run.trial028.stderr:dumped fsmap epoch 23 2026-02-25T11:59:09.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.829+0000 7f6c577fe640 1 -- 10.20.193.28:0/4133661598 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6c640b5750 msgr2=0x7f6c640d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:09.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.829+0000 7f6c577fe640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6c640b5750 0x7f6c640d5c00 secure :-1 s=READY pgs=418 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6c78004740 tx=0x7f6c7800b410 comp rx=0 tx=0).stop 2026-02-25T11:59:09.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.829+0000 7f6c577fe640 1 -- 10.20.193.28:0/4133661598 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6c8810ddc0 msgr2=0x7f6c88173e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:09.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.829+0000 7f6c577fe640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6c8810ddc0 0x7f6c88173e90 secure :-1 s=READY pgs=963 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6c700098a0 tx=0x7f6c700028d0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.830+0000 7f6c577fe640 1 -- 10.20.193.28:0/4133661598 shutdown_connections 2026-02-25T11:59:09.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.830+0000 7f6c577fe640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6c640b5750 0x7f6c640d5c00 unknown :-1 s=CLOSED pgs=418 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.830+0000 7f6c577fe640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6c881135c0 0x7f6c880f6180 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.831 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.830+0000 7f6c577fe640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6c8810ddc0 0x7f6c88173e90 unknown :-1 s=CLOSED pgs=963 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.830+0000 7f6c577fe640 1 --2- 10.20.193.28:0/4133661598 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6c8810d3d0 0x7f6c880f5a80 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:09.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.830+0000 7f6c577fe640 1 -- 10.20.193.28:0/4133661598 >> 10.20.193.28:0/4133661598 conn(0x7f6c880eaca0 msgr2=0x7f6c8810fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:09.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.830+0000 7f6c577fe640 1 -- 10.20.193.28:0/4133661598 shutdown_connections 2026-02-25T11:59:09.832 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:09.830+0000 7f6c577fe640 1 -- 10.20.193.28:0/4133661598 wait complete. 2026-02-25T11:59:09.966 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd dump --format=json 2026-02-25T11:59:10.151 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:59:10.333 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.331+0000 7fc95c529640 1 -- 10.20.193.28:0/3640287751 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc95410f840 msgr2=0x7fc95410fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:10.333 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.331+0000 7fc95c529640 1 --2- 10.20.193.28:0/3640287751 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc95410f840 0x7fc95410fc10 secure :-1 s=READY pgs=964 gs=1 cs=0 l=1 c_cookie=10867468825576176717 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc944009950 tx=0x7fc94402f190 comp rx=0 tx=0).stop 2026-02-25T11:59:10.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.332+0000 7fc95c529640 1 -- 10.20.193.28:0/3640287751 shutdown_connections 2026-02-25T11:59:10.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.332+0000 7fc95c529640 1 --2- 10.20.193.28:0/3640287751 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc954115a30 0x7fc954176400 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.332+0000 7fc95c529640 1 --2- 10.20.193.28:0/3640287751 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc954110230 0x7fc954115280 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.332+0000 7fc95c529640 1 --2- 10.20.193.28:0/3640287751 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc95410f840 0x7fc95410fc10 unknown :-1 s=CLOSED pgs=964 gs=1 cs=0 l=1 c_cookie=10867468825576176717 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.332+0000 7fc95c529640 1 -- 10.20.193.28:0/3640287751 >> 10.20.193.28:0/3640287751 conn(0x7fc9540ed130 msgr2=0x7fc95410d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:10.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.332+0000 7fc95c529640 1 -- 10.20.193.28:0/3640287751 shutdown_connections 2026-02-25T11:59:10.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.332+0000 7fc95c529640 1 -- 10.20.193.28:0/3640287751 wait complete. 2026-02-25T11:59:10.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.332+0000 7fc95c529640 1 Processor -- start 2026-02-25T11:59:10.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc95c529640 1 -- start start 2026-02-25T11:59:10.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc95c529640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc95410f840 0x7fc95411e1c0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:10.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc95c529640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc954110230 0x7fc95411e700 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:10.334 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc95c529640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc954115a30 0x7fc954123ad0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc95c529640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc954124060 con 0x7fc954110230 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc95c529640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc9541241d0 con 0x7fc954115a30 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc95c529640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc954124390 con 0x7fc95410f840 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc959a9d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc954110230 0x7fc95411e700 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc959a9d640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc954110230 0x7fc95411e700 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35058/0 (socket says 10.20.193.28:35058) 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc959a9d640 1 -- 10.20.193.28:0/4101101115 learned_addr learned my addr 10.20.193.28:0/4101101115 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc95a29e640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc95410f840 0x7fc95411e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc95aa9f640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc954115a30 0x7fc954123ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc959a9d640 1 -- 10.20.193.28:0/4101101115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc95410f840 msgr2=0x7fc95411e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc959a9d640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc95410f840 0x7fc95411e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc959a9d640 1 -- 10.20.193.28:0/4101101115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc954115a30 msgr2=0x7fc954123ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc959a9d640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc954115a30 0x7fc954123ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.333+0000 7fc959a9d640 1 -- 10.20.193.28:0/4101101115 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc944009570 con 0x7fc954110230 2026-02-25T11:59:10.335 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.334+0000 7fc959a9d640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc954110230 0x7fc95411e700 secure :-1 s=READY pgs=965 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc94800c9b0 tx=0x7fc94800ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:10.336 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.334+0000 7fc93b7fe640 1 -- 10.20.193.28:0/4101101115 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc9480044a0 con 0x7fc954110230 2026-02-25T11:59:10.336 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.334+0000 7fc93b7fe640 1 -- 10.20.193.28:0/4101101115 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7fc948007d80 con 0x7fc954110230 2026-02-25T11:59:10.336 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.334+0000 7fc93b7fe640 1 -- 10.20.193.28:0/4101101115 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7fc94800f6d0 con 0x7fc954110230 2026-02-25T11:59:10.336 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.334+0000 7fc95c529640 1 -- 10.20.193.28:0/4101101115 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc954124670 con 0x7fc954110230 2026-02-25T11:59:10.336 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.334+0000 7fc95c529640 1 -- 10.20.193.28:0/4101101115 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc954164e00 con 0x7fc954110230 2026-02-25T11:59:10.336 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.334+0000 7fc95c529640 1 -- 10.20.193.28:0/4101101115 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc91c005300 con 0x7fc954110230 2026-02-25T11:59:10.338 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.337+0000 7fc93b7fe640 1 -- 10.20.193.28:0/4101101115 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7fc9480078c0 con 0x7fc954110230 2026-02-25T11:59:10.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.338+0000 7fc93b7fe640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9300b56e0 0x7fc9300d5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:10.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.339+0000 7fc93b7fe640 1 -- 10.20.193.28:0/4101101115 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7fc9480e6e40 con 0x7fc954110230 2026-02-25T11:59:10.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.339+0000 7fc93b7fe640 1 -- 10.20.193.28:0/4101101115 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7fc9480e72c0 con 0x7fc954110230 2026-02-25T11:59:10.340 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.339+0000 7fc95a29e640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9300b56e0 0x7fc9300d5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:10.341 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.339+0000 7fc95a29e640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9300b56e0 0x7fc9300d5b90 secure :-1 s=READY pgs=419 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9440023d0 tx=0x7fc944031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:10.460 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.458+0000 7fc95c529640 1 -- 10.20.193.28:0/4101101115 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc91c005880 con 0x7fc954110230 2026-02-25T11:59:10.461 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:10 trial028 ceph-mon[39224]: pgmap v554: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2026-02-25T11:59:10.461 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:10 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/4133661598' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:59:10.461 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.459+0000 7fc93b7fe640 1 -- 10.20.193.28:0/4101101115 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22689 (secure 0 0 0) 0x7fc9480a9810 con 0x7fc954110230 2026-02-25T11:59:10.462 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-25T11:59:10.462 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":88,"fsid":"123fa7bf-123f-11f1-80da-d404e6e7d460","created":"2026-02-25T11:42:51.962844+0000","modified":"2026-02-25T11:48:18.148313+0000","last_up_change":"2026-02-25T11:46:45.156629+0000","last_in_change":"2026-02-25T11:46:31.585002+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":38,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T11:44:43.266336+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2026-02-25T11:47:11.236575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2026-02-25T11:47:12.196775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf596df7-d1fb-4d10-8d4c-14a35210bab2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6803","nonce":668944205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6805","nonce":668944205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6809","nonce":668944205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":668944205},{"type":"v1","addr":"10.20.193.28:6807","nonce":668944205}]},"public_addr":"10.20.193.28:6803/668944205","cluster_addr":"10.20.193.28:6805/668944205","heartbeat_back_addr":"10.20.193.28:6809/668944205","heartbeat_front_addr":"10.20.193.28:6807/668944205","state":["exists","up"]},{"osd":1,"uuid":"185719c0-7c2d-431f-9925-88bbee86a98d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6800","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6801","nonce":3486566242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6802","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6803","nonce":3486566242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6806","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6807","nonce":3486566242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6804","nonce":3486566242},{"type":"v1","addr":"10.20.193.119:6805","nonce":3486566242}]},"public_addr":"10.20.193.119:6801/3486566242","cluster_addr":"10.20.193.119:6803/3486566242","heartbeat_back_addr":"10.20.193.119:6807/3486566242","heartbeat_front_addr":"10.20.193.119:6805/3486566242","state":["exists","up"]},{"osd":2,"uuid":"4de13b90-1c3c-4614-9fef-2d7315626433","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6800","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6801","nonce":484781559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6802","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6803","nonce":484781559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6806","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6807","nonce":484781559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6804","nonce":484781559},{"type":"v1","addr":"10.20.193.167:6805","nonce":484781559}]},"public_addr":"10.20.193.167:6801/484781559","cluster_addr":"10.20.193.167:6803/484781559","heartbeat_back_addr":"10.20.193.167:6807/484781559","heartbeat_front_addr":"10.20.193.167:6805/484781559","state":["exists","up"]},{"osd":3,"uuid":"45d9ea8e-5a77-4738-96da-ada98515fc02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6811","nonce":2452541436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6813","nonce":2452541436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6816","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6817","nonce":2452541436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6814","nonce":2452541436},{"type":"v1","addr":"10.20.193.28:6815","nonce":2452541436}]},"public_addr":"10.20.193.28:6811/2452541436","cluster_addr":"10.20.193.28:6813/2452541436","heartbeat_back_addr":"10.20.193.28:6817/2452541436","heartbeat_front_addr":"10.20.193.28:6815/2452541436","state":["exists","up"]},{"osd":4,"uuid":"c9c3f512-665b-44b2-a6f0-f3356a4b661e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6808","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6809","nonce":135971996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6810","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6811","nonce":135971996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6814","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6815","nonce":135971996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6812","nonce":135971996},{"type":"v1","addr":"10.20.193.119:6813","nonce":135971996}]},"public_addr":"10.20.193.119:6809/135971996","cluster_addr":"10.20.193.119:6811/135971996","heartbeat_back_addr":"10.20.193.119:6815/135971996","heartbeat_front_addr":"10.20.193.119:6813/135971996","state":["exists","up"]},{"osd":5,"uuid":"777c3819-5c90-4b2b-a545-c024884c2eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6808","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6809","nonce":4203420884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6810","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6811","nonce":4203420884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6814","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6815","nonce":4203420884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6812","nonce":4203420884},{"type":"v1","addr":"10.20.193.167:6813","nonce":4203420884}]},"public_addr":"10.20.193.167:6809/4203420884","cluster_addr":"10.20.193.167:6811/4203420884","heartbeat_back_addr":"10.20.193.167:6815/4203420884","heartbeat_front_addr":"10.20.193.167:6813/4203420884","state":["exists","up"]},{"osd":6,"uuid":"bbd9de64-b9fe-499e-b451-c16828be2c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6818","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6819","nonce":621484999}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6820","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6821","nonce":621484999}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6824","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6825","nonce":621484999}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6822","nonce":621484999},{"type":"v1","addr":"10.20.193.28:6823","nonce":621484999}]},"public_addr":"10.20.193.28:6819/621484999","cluster_addr":"10.20.193.28:6821/621484999","heartbeat_back_addr":"10.20.193.28:6825/621484999","heartbeat_front_addr":"10.20.193.28:6823/621484999","state":["exists","up"]},{"osd":7,"uuid":"546d9f28-597b-454c-8456-fcc2d46c69f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6816","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6817","nonce":2724541747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6818","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6819","nonce":2724541747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6822","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6823","nonce":2724541747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6820","nonce":2724541747},{"type":"v1","addr":"10.20.193.119:6821","nonce":2724541747}]},"public_addr":"10.20.193.119:6817/2724541747","cluster_addr":"10.20.193.119:6819/2724541747","heartbeat_back_addr":"10.20.193.119:6823/2724541747","heartbeat_front_addr":"10.20.193.119:6821/2724541747","state":["exists","up"]},{"osd":8,"uuid":"a1a7cea6-3586-4bb2-aa4c-bf6f7799a501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6816","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6817","nonce":1328734317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6818","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6819","nonce":1328734317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6822","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6823","nonce":1328734317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6820","nonce":1328734317},{"type":"v1","addr":"10.20.193.167:6821","nonce":1328734317}]},"public_addr":"10.20.193.167:6817/1328734317","cluster_addr":"10.20.193.167:6819/1328734317","heartbeat_back_addr":"10.20.193.167:6823/1328734317","heartbeat_front_addr":"10.20.193.167:6821/1328734317","state":["exists","up"]},{"osd":9,"uuid":"03fef134-6946-43d5-9c26-dc0e59fffe51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6826","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6827","nonce":354500625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6828","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6829","nonce":354500625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6832","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6833","nonce":354500625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6830","nonce":354500625},{"type":"v1","addr":"10.20.193.28:6831","nonce":354500625}]},"public_addr":"10.20.193.28:6827/354500625","cluster_addr":"10.20.193.28:6829/354500625","heartbeat_back_addr":"10.20.193.28:6833/354500625","heartbeat_front_addr":"10.20.193.28:6831/354500625","state":["exists","up"]},{"osd":10,"uuid":"b46c29ef-285c-4184-be28-d99b76bc0e1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6824","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6825","nonce":2996581116}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6826","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6827","nonce":2996581116}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6830","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6831","nonce":2996581116}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.119:6828","nonce":2996581116},{"type":"v1","addr":"10.20.193.119:6829","nonce":2996581116}]},"public_addr":"10.20.193.119:6825/2996581116","cluster_addr":"10.20.193.119:6827/2996581116","heartbeat_back_addr":"10.20.193.119:6831/2996581116","heartbeat_front_addr":"10.20.193.119:6829/2996581116","state":["exists","up"]},{"osd":11,"uuid":"f6696d8c-5de7-4fd9-b693-524f204e4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6824","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6825","nonce":2182906700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6826","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6827","nonce":2182906700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6830","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6831","nonce":2182906700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.167:6828","nonce":2182906700},{"type":"v1","addr":"10.20.193.167:6829","nonce":2182906700}]},"public_addr":"10.20.193.167:6825/2182906700","cluster_addr":"10.20.193.167:6827/2182906700","heartbeat_back_addr":"10.20.193.167:6831/2182906700","heartbeat_front_addr":"10.20.193.167:6829/2182906700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:13.410235+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:27.254773+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:40.956615+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:44:54.529866+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:08.306772+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:21.899498+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:35.500407+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:45:49.094965+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:02.790138+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:16.286937+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:29.922722+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-25T11:46:43.396028+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.2","mappings":[{"from":7,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1b","mappings":[{"from":2,"to":1}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.28:0/1467902196":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6800/2907029718":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/2274270913":"2026-02-26T11:43:17.037464+0000","10.20.193.28:0/60087422":"2026-02-26T11:43:17.037464+0000","10.20.193.28:6801/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:6800/3221634678":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/1725332841":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/3082630924":"2026-02-26T11:43:04.102029+0000","10.20.193.28:0/2391715078":"2026-02-26T11:43:04.102029+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T11:59:10.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.460+0000 7fc95c529640 1 -- 10.20.193.28:0/4101101115 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9300b56e0 msgr2=0x7fc9300d5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:10.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.460+0000 7fc95c529640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9300b56e0 0x7fc9300d5b90 secure :-1 s=READY pgs=419 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc9440023d0 tx=0x7fc944031040 comp rx=0 tx=0).stop 2026-02-25T11:59:10.463 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.460+0000 7fc95c529640 1 -- 10.20.193.28:0/4101101115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc954110230 msgr2=0x7fc95411e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:10.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.460+0000 7fc95c529640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc954110230 0x7fc95411e700 secure :-1 s=READY pgs=965 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7fc94800c9b0 tx=0x7fc94800ce80 comp rx=0 tx=0).stop 2026-02-25T11:59:10.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.460+0000 7fc95c529640 1 -- 10.20.193.28:0/4101101115 shutdown_connections 2026-02-25T11:59:10.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.460+0000 7fc95c529640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7fc9300b56e0 0x7fc9300d5b90 unknown :-1 s=CLOSED pgs=419 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.460+0000 7fc95c529640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7fc954115a30 0x7fc954123ad0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.460+0000 7fc95c529640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7fc954110230 0x7fc95411e700 unknown :-1 s=CLOSED pgs=965 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.460+0000 7fc95c529640 1 --2- 10.20.193.28:0/4101101115 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7fc95410f840 0x7fc95411e1c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.460+0000 7fc95c529640 1 -- 10.20.193.28:0/4101101115 >> 10.20.193.28:0/4101101115 conn(0x7fc9540ed130 msgr2=0x7fc95410d510 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:10.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.461+0000 7fc95c529640 1 -- 10.20.193.28:0/4101101115 shutdown_connections 2026-02-25T11:59:10.464 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.461+0000 7fc95c529640 1 -- 10.20.193.28:0/4101101115 wait complete. 2026-02-25T11:59:10.562 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs fail cephfs 2026-02-25T11:59:10.746 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:59:10.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:10 trial167 ceph-mon[43392]: pgmap v554: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2026-02-25T11:59:10.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:10 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/4133661598' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:59:10.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:10 trial119 ceph-mon[42710]: pgmap v554: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2026-02-25T11:59:10.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:10 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/4133661598' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2026-02-25T11:59:10.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.924+0000 7f5b40f17640 1 -- 10.20.193.28:0/3684795363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5b3c115a30 msgr2=0x7f5b3c176400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:10.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.924+0000 7f5b40f17640 1 --2- 10.20.193.28:0/3684795363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5b3c115a30 0x7f5b3c176400 secure :-1 s=READY pgs=966 gs=1 cs=0 l=1 c_cookie=15913996114004453288 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5b2c00aa30 tx=0x7f5b2c030270 comp rx=0 tx=0).stop 2026-02-25T11:59:10.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.924+0000 7f5b40f17640 1 -- 10.20.193.28:0/3684795363 shutdown_connections 2026-02-25T11:59:10.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.924+0000 7f5b40f17640 1 --2- 10.20.193.28:0/3684795363 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5b3c115a30 0x7f5b3c176400 unknown :-1 s=CLOSED pgs=966 gs=1 cs=0 l=1 c_cookie=15913996114004453288 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.924+0000 7f5b40f17640 1 --2- 10.20.193.28:0/3684795363 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5b3c110230 0x7f5b3c115280 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.924+0000 7f5b40f17640 1 --2- 10.20.193.28:0/3684795363 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5b3c10f840 0x7f5b3c10fc10 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.924+0000 7f5b40f17640 1 -- 10.20.193.28:0/3684795363 >> 10.20.193.28:0/3684795363 conn(0x7f5b3c0ed130 msgr2=0x7f5b3c10d540 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:10.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.924+0000 7f5b40f17640 1 -- 10.20.193.28:0/3684795363 shutdown_connections 2026-02-25T11:59:10.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.924+0000 7f5b40f17640 1 -- 10.20.193.28:0/3684795363 wait complete. 2026-02-25T11:59:10.926 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.925+0000 7f5b40f17640 1 Processor -- start 2026-02-25T11:59:10.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.925+0000 7f5b40f17640 1 -- start start 2026-02-25T11:59:10.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b40f17640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5b3c10f840 0x7f5b3c0f5b90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:10.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b40f17640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5b3c110230 0x7f5b3c176300 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:10.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b40f17640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5b3c115a30 0x7f5b3c0f6290 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:10.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b40f17640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c0fb4a0 con 0x7f5b3c110230 2026-02-25T11:59:10.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b40f17640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c0fb610 con 0x7f5b3c115a30 2026-02-25T11:59:10.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b40f17640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c0fb7d0 con 0x7f5b3c10f840 2026-02-25T11:59:10.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b39d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5b3c110230 0x7f5b3c176300 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:10.927 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b39d74640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5b3c110230 0x7f5b3c176300 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35086/0 (socket says 10.20.193.28:35086) 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b39d74640 1 -- 10.20.193.28:0/1989270359 learned_addr learned my addr 10.20.193.28:0/1989270359 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b3ad76640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5b3c115a30 0x7f5b3c0f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b3a575640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5b3c10f840 0x7f5b3c0f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b39d74640 1 -- 10.20.193.28:0/1989270359 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5b3c10f840 msgr2=0x7f5b3c0f5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b39d74640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5b3c10f840 0x7f5b3c0f5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b39d74640 1 -- 10.20.193.28:0/1989270359 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5b3c115a30 msgr2=0x7f5b3c0f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b39d74640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5b3c115a30 0x7f5b3c0f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b39d74640 1 -- 10.20.193.28:0/1989270359 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c00a650 con 0x7f5b3c110230 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b39d74640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5b3c110230 0x7f5b3c176300 secure :-1 s=READY pgs=967 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5b2400d950 tx=0x7f5b2400de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b237fe640 1 -- 10.20.193.28:0/1989270359 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5b24004540 con 0x7f5b3c110230 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b237fe640 1 -- 10.20.193.28:0/1989270359 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f5b24009ce0 con 0x7f5b3c110230 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b237fe640 1 -- 10.20.193.28:0/1989270359 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f5b240106e0 con 0x7f5b3c110230 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.926+0000 7f5b40f17640 1 -- 10.20.193.28:0/1989270359 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3c0fbab0 con 0x7f5b3c110230 2026-02-25T11:59:10.928 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.927+0000 7f5b40f17640 1 -- 10.20.193.28:0/1989270359 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3c0fbf30 con 0x7f5b3c110230 2026-02-25T11:59:10.929 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.927+0000 7f5b40f17640 1 -- 10.20.193.28:0/1989270359 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b3c110b20 con 0x7f5b3c110230 2026-02-25T11:59:10.931 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.930+0000 7f5b237fe640 1 -- 10.20.193.28:0/1989270359 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f5b240097c0 con 0x7f5b3c110230 2026-02-25T11:59:10.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.931+0000 7f5b237fe640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5b140b5750 0x7f5b140d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:10.932 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.931+0000 7f5b237fe640 1 -- 10.20.193.28:0/1989270359 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8188+0+0 (secure 0 0 0) 0x7f5b240e7310 con 0x7f5b3c110230 2026-02-25T11:59:10.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.931+0000 7f5b237fe640 1 -- 10.20.193.28:0/1989270359 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f5b240e7790 con 0x7f5b3c110230 2026-02-25T11:59:10.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.931+0000 7f5b3a575640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5b140b5750 0x7f5b140d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:10.933 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:10.932+0000 7f5b3a575640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5b140b5750 0x7f5b140d5c00 secure :-1 s=READY pgs=420 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5b30004740 tx=0x7f5b30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:11.076 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.074+0000 7f5b40f17640 1 -- 10.20.193.28:0/1989270359 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5b3c0f6cf0 con 0x7f5b3c110230 2026-02-25T11:59:11.079 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.077+0000 7f5b237fe640 1 -- 10.20.193.28:0/1989270359 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) ==== 161+0+0 (secure 0 0 0) 0x7f5b240a9ce0 con 0x7f5b3c110230 2026-02-25T11:59:11.079 INFO:teuthology.orchestra.run.trial028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2026-02-25T11:59:11.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.079+0000 7f5b40f17640 1 -- 10.20.193.28:0/1989270359 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5b140b5750 msgr2=0x7f5b140d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:11.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.079+0000 7f5b40f17640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5b140b5750 0x7f5b140d5c00 secure :-1 s=READY pgs=420 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5b30004740 tx=0x7f5b30005f90 comp rx=0 tx=0).stop 2026-02-25T11:59:11.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.079+0000 7f5b40f17640 1 -- 10.20.193.28:0/1989270359 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5b3c110230 msgr2=0x7f5b3c176300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:11.081 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.079+0000 7f5b40f17640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5b3c110230 0x7f5b3c176300 secure :-1 s=READY pgs=967 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f5b2400d950 tx=0x7f5b2400de20 comp rx=0 tx=0).stop 2026-02-25T11:59:11.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.080+0000 7f5b40f17640 1 -- 10.20.193.28:0/1989270359 shutdown_connections 2026-02-25T11:59:11.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.080+0000 7f5b40f17640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f5b140b5750 0x7f5b140d5c00 unknown :-1 s=CLOSED pgs=420 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:11.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.080+0000 7f5b40f17640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f5b3c115a30 0x7f5b3c0f6290 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:11.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.080+0000 7f5b40f17640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f5b3c110230 0x7f5b3c176300 unknown :-1 s=CLOSED pgs=967 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:11.082 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.080+0000 7f5b40f17640 1 --2- 10.20.193.28:0/1989270359 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f5b3c10f840 0x7f5b3c0f5b90 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:11.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.080+0000 7f5b40f17640 1 -- 10.20.193.28:0/1989270359 >> 10.20.193.28:0/1989270359 conn(0x7f5b3c0ed130 msgr2=0x7f5b3c10dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:11.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.080+0000 7f5b40f17640 1 -- 10.20.193.28:0/1989270359 shutdown_connections 2026-02-25T11:59:11.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.080+0000 7f5b40f17640 1 -- 10.20.193.28:0/1989270359 wait complete. 2026-02-25T11:59:11.184 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph fs rm cephfs --yes-i-really-mean-it 2026-02-25T11:59:11.369 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:59:11.412 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:11 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-a[39219]: 2026-02-25T11:59:11.075+0000 7f58e5e7b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2026-02-25T11:59:11.412 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:11 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/4101101115' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:59:11.412 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:11 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1989270359' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2026-02-25T11:59:11.412 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:11 trial028 ceph-mon[39224]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2026-02-25T11:59:11.413 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:11 trial028 ceph-mon[39224]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2026-02-25T11:59:11.413 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:11 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1989270359' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2026-02-25T11:59:11.413 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:11 trial028 ceph-mon[39224]: osdmap e89: 12 total, 12 up, 12 in 2026-02-25T11:59:11.413 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:11 trial028 ceph-mon[39224]: fsmap cephfs:0/3 6 up:standby, 3 failed 2026-02-25T11:59:11.499 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 11:59:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-c[98019]: ignoring --setuser ceph since I am not root 2026-02-25T11:59:11.499 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 11:59:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-c[98019]: ignoring --setgroup ceph since I am not root 2026-02-25T11:59:11.499 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 11:59:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-c[98019]: starting mds.c at 2026-02-25T11:59:11.499 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 11:59:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-f[97716]: ignoring --setuser ceph since I am not root 2026-02-25T11:59:11.499 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 11:59:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-f[97716]: ignoring --setgroup ceph since I am not root 2026-02-25T11:59:11.499 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 11:59:11 trial167 ceph-mds[97720]: --2- 10.20.193.167:0/1562081082 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x188aa400 0x188a6b00 secure :-1 s=READY pgs=243 gs=3 cs=0 l=1 c_cookie=17268481103286416168 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x18756690 tx=0x1873f4a0 comp rx=0 tx=0).write_message sending message m=0x18772c00 seq=3 auth(proto 2 2 bytes epoch 0) 2026-02-25T11:59:11.499 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 11:59:11 trial167 ceph-mds[97720]: -- 10.20.193.167:0/1562081082 <== mon.1 v2:10.20.193.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x18768ea0 con 0x188aa400 2026-02-25T11:59:11.500 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 11:59:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-f[97716]: starting mds.f at 2026-02-25T11:59:11.500 INFO:journalctl@ceph.mds.i.trial167.stdout:Feb 25 11:59:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-i[98316]: ignoring --setuser ceph since I am not root 2026-02-25T11:59:11.500 INFO:journalctl@ceph.mds.i.trial167.stdout:Feb 25 11:59:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-i[98316]: ignoring --setgroup ceph since I am not root 2026-02-25T11:59:11.500 INFO:journalctl@ceph.mds.i.trial167.stdout:Feb 25 11:59:11 trial167 ceph-mds[98320]: -- 10.20.193.167:0/2295315953 <== mon.2 v2:10.20.193.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x12d74ea0 con 0x12eb6400 2026-02-25T11:59:11.500 INFO:journalctl@ceph.mds.i.trial167.stdout:Feb 25 11:59:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-i[98316]: starting mds.i at 2026-02-25T11:59:11.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:11 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/4101101115' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:59:11.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:11 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1989270359' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2026-02-25T11:59:11.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:11 trial167 ceph-mon[43392]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2026-02-25T11:59:11.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:11 trial167 ceph-mon[43392]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2026-02-25T11:59:11.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:11 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1989270359' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2026-02-25T11:59:11.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:11 trial167 ceph-mon[43392]: osdmap e89: 12 total, 12 up, 12 in 2026-02-25T11:59:11.500 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:11 trial167 ceph-mon[43392]: fsmap cephfs:0/3 6 up:standby, 3 failed 2026-02-25T11:59:11.509 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:59:11 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-b[95783]: ignoring --setuser ceph since I am not root 2026-02-25T11:59:11.509 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:59:11 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-b[95783]: ignoring --setgroup ceph since I am not root 2026-02-25T11:59:11.509 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:59:11 trial119 ceph-mds[95787]: -- 10.20.193.119:0/3446668712 <== mon.2 v2:10.20.193.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0xafe8ea0 con 0xb12a400 2026-02-25T11:59:11.510 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 11:59:11 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-b[95783]: starting mds.b at 2026-02-25T11:59:11.510 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 11:59:11 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-h[95489]: ignoring --setuser ceph since I am not root 2026-02-25T11:59:11.510 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 11:59:11 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-h[95489]: ignoring --setgroup ceph since I am not root 2026-02-25T11:59:11.510 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 11:59:11 trial119 ceph-mds[95493]: -- 10.20.193.119:0/823576833 <== mon.2 v2:10.20.193.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x12218ea0 con 0x12253c00 2026-02-25T11:59:11.510 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 11:59:11 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-h[95489]: starting mds.h at 2026-02-25T11:59:11.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:11 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/4101101115' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2026-02-25T11:59:11.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:11 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1989270359' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2026-02-25T11:59:11.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:11 trial119 ceph-mon[42710]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2026-02-25T11:59:11.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:11 trial119 ceph-mon[42710]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2026-02-25T11:59:11.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:11 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1989270359' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2026-02-25T11:59:11.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:11 trial119 ceph-mon[42710]: osdmap e89: 12 total, 12 up, 12 in 2026-02-25T11:59:11.510 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:11 trial119 ceph-mon[42710]: fsmap cephfs:0/3 6 up:standby, 3 failed 2026-02-25T11:59:11.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.546+0000 7f6d0968c640 1 -- 10.20.193.28:0/2485953817 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6d0410f650 msgr2=0x7f6d0410fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:11.547 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.546+0000 7f6d0968c640 1 --2- 10.20.193.28:0/2485953817 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6d0410f650 0x7f6d0410fa20 secure :-1 s=READY pgs=969 gs=1 cs=0 l=1 c_cookie=5759637962707597874 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6cf0009950 tx=0x7f6cf002f190 comp rx=0 tx=0).stop 2026-02-25T11:59:11.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.546+0000 7f6d0968c640 1 -- 10.20.193.28:0/2485953817 shutdown_connections 2026-02-25T11:59:11.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.546+0000 7f6d0968c640 1 --2- 10.20.193.28:0/2485953817 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6d04115840 0x7f6d04176210 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:11.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.546+0000 7f6d0968c640 1 --2- 10.20.193.28:0/2485953817 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6d04110040 0x7f6d04115090 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:11.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.546+0000 7f6d0968c640 1 --2- 10.20.193.28:0/2485953817 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6d0410f650 0x7f6d0410fa20 unknown :-1 s=CLOSED pgs=969 gs=1 cs=0 l=1 c_cookie=5759637962707597874 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:11.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.546+0000 7f6d0968c640 1 -- 10.20.193.28:0/2485953817 >> 10.20.193.28:0/2485953817 conn(0x7f6d040ecf00 msgr2=0x7f6d0410d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:11.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.546+0000 7f6d0968c640 1 -- 10.20.193.28:0/2485953817 shutdown_connections 2026-02-25T11:59:11.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.546+0000 7f6d0968c640 1 -- 10.20.193.28:0/2485953817 wait complete. 2026-02-25T11:59:11.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.547+0000 7f6d0968c640 1 Processor -- start 2026-02-25T11:59:11.548 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.547+0000 7f6d0968c640 1 -- start start 2026-02-25T11:59:11.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.547+0000 7f6d0968c640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6d0410f650 0x7f6d0411dff0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:11.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.547+0000 7f6d0968c640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6d04110040 0x7f6d0411e530 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:11.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.547+0000 7f6d0968c640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6d04115840 0x7f6d04123900 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:11.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.547+0000 7f6d0968c640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04123f20 con 0x7f6d04115840 2026-02-25T11:59:11.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.547+0000 7f6d0968c640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04124090 con 0x7f6d04110040 2026-02-25T11:59:11.549 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.547+0000 7f6d0968c640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04124250 con 0x7f6d0410f650 2026-02-25T11:59:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d08e8b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6d04115840 0x7f6d04123900 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d08e8b640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6d04115840 0x7f6d04123900 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35114/0 (socket says 10.20.193.28:35114) 2026-02-25T11:59:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d08e8b640 1 -- 10.20.193.28:0/1169517730 learned_addr learned my addr 10.20.193.28:0/1169517730 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d037fe640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6d04110040 0x7f6d0411e530 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d03fff640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6d0410f650 0x7f6d0411dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d08e8b640 1 -- 10.20.193.28:0/1169517730 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6d0410f650 msgr2=0x7f6d0411dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d08e8b640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6d0410f650 0x7f6d0411dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d08e8b640 1 -- 10.20.193.28:0/1169517730 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6d04110040 msgr2=0x7f6d0411e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d08e8b640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6d04110040 0x7f6d0411e530 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d08e8b640 1 -- 10.20.193.28:0/1169517730 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf0009570 con 0x7f6d04115840 2026-02-25T11:59:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d08e8b640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6d04115840 0x7f6d04123900 secure :-1 s=READY pgs=970 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6cf8006e90 tx=0x7f6cf800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d017fa640 1 -- 10.20.193.28:0/1169517730 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6cf8005500 con 0x7f6d04115840 2026-02-25T11:59:11.550 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d017fa640 1 -- 10.20.193.28:0/1169517730 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f6cf8005df0 con 0x7f6d04115840 2026-02-25T11:59:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d017fa640 1 -- 10.20.193.28:0/1169517730 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f6cf8003a00 con 0x7f6d04115840 2026-02-25T11:59:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d0968c640 1 -- 10.20.193.28:0/1169517730 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d04124530 con 0x7f6d04115840 2026-02-25T11:59:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.548+0000 7f6d0968c640 1 -- 10.20.193.28:0/1169517730 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d04164c30 con 0x7f6d04115840 2026-02-25T11:59:11.551 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.549+0000 7f6d0968c640 1 -- 10.20.193.28:0/1169517730 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d04124750 con 0x7f6d04115840 2026-02-25T11:59:11.553 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.551+0000 7f6d017fa640 1 -- 10.20.193.28:0/1169517730 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f6cf8003b60 con 0x7f6d04115840 2026-02-25T11:59:11.554 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.553+0000 7f6d017fa640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6cd80b57e0 0x7f6cd80d5c90 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.553+0000 7f6d017fa640 1 -- 10.20.193.28:0/1169517730 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8446+0+0 (secure 0 0 0) 0x7f6cf80e7c40 con 0x7f6d04115840 2026-02-25T11:59:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.553+0000 7f6d017fa640 1 -- 10.20.193.28:0/1169517730 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f6cf8120940 con 0x7f6d04115840 2026-02-25T11:59:11.555 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.554+0000 7f6d03fff640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6cd80b57e0 0x7f6cd80d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:11.556 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.555+0000 7f6d03fff640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6cd80b57e0 0x7f6cd80d5c90 secure :-1 s=READY pgs=421 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6cf00023d0 tx=0x7f6cf0031040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:11.700 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:11.698+0000 7f6d0968c640 1 -- 10.20.193.28:0/1169517730 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6d0410fa50 con 0x7f6d04115840 2026-02-25T11:59:12.083 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.081+0000 7f6d017fa640 1 -- 10.20.193.28:0/1169517730 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 If there are active snapshot schedules associated with this file-system, you might see EIO errors in the mgr logs or at the snap-schedule command-line due to the missing file-system. However, these errors are transient and will get auto-resolved. v25) ==== 350+0+0 (secure 0 0 0) 0x7f6cf80aa510 con 0x7f6d04115840 2026-02-25T11:59:12.084 INFO:teuthology.orchestra.run.trial028.stderr:If there are active snapshot schedules associated with this file-system, you might see EIO errors in the mgr logs or at the snap-schedule command-line due to the missing file-system. However, these errors are transient and will get auto-resolved. 2026-02-25T11:59:12.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.083+0000 7f6d0968c640 1 -- 10.20.193.28:0/1169517730 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6cd80b57e0 msgr2=0x7f6cd80d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:12.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.083+0000 7f6d0968c640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6cd80b57e0 0x7f6cd80d5c90 secure :-1 s=READY pgs=421 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6cf00023d0 tx=0x7f6cf0031040 comp rx=0 tx=0).stop 2026-02-25T11:59:12.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.083+0000 7f6d0968c640 1 -- 10.20.193.28:0/1169517730 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6d04115840 msgr2=0x7f6d04123900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:12.085 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.083+0000 7f6d0968c640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6d04115840 0x7f6d04123900 secure :-1 s=READY pgs=970 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f6cf8006e90 tx=0x7f6cf800edf0 comp rx=0 tx=0).stop 2026-02-25T11:59:12.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.083+0000 7f6d0968c640 1 -- 10.20.193.28:0/1169517730 shutdown_connections 2026-02-25T11:59:12.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.083+0000 7f6d0968c640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f6cd80b57e0 0x7f6cd80d5c90 unknown :-1 s=CLOSED pgs=421 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:12.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.083+0000 7f6d0968c640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f6d04115840 0x7f6d04123900 unknown :-1 s=CLOSED pgs=970 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:12.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.083+0000 7f6d0968c640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f6d04110040 0x7f6d0411e530 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:12.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.083+0000 7f6d0968c640 1 --2- 10.20.193.28:0/1169517730 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f6d0410f650 0x7f6d0411dff0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:12.086 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.083+0000 7f6d0968c640 1 -- 10.20.193.28:0/1169517730 >> 10.20.193.28:0/1169517730 conn(0x7f6d040ecf00 msgr2=0x7f6d0410d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:12.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.083+0000 7f6d0968c640 1 -- 10.20.193.28:0/1169517730 shutdown_connections 2026-02-25T11:59:12.087 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.083+0000 7f6d0968c640 1 -- 10.20.193.28:0/1169517730 wait complete. 2026-02-25T11:59:12.187 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2026-02-25T11:59:12.373 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:59:12.416 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: pgmap v556: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2026-02-25T11:59:12.417 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1169517730' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2026-02-25T11:59:12.417 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2026-02-25T11:59:12.417 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2026-02-25T11:59:12.417 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: Cluster is now healthy 2026-02-25T11:59:12.417 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1169517730' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2026-02-25T11:59:12.417 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.119:6836/3736878020,v1:10.20.193.119:6838/3736878020] up:boot 2026-02-25T11:59:12.417 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.119:6837/2403226505,v1:10.20.193.119:6839/2403226505] up:boot 2026-02-25T11:59:12.417 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.167:6835/2168188582,v1:10.20.193.167:6838/2168188582] up:boot 2026-02-25T11:59:12.417 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.167:6834/1987058645,v1:10.20.193.167:6837/1987058645] up:boot 2026-02-25T11:59:12.418 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.28:6838/842890247,v1:10.20.193.28:6839/842890247] up:boot 2026-02-25T11:59:12.418 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: mds.? [v2:10.20.193.167:6836/1968657301,v1:10.20.193.167:6839/1968657301] up:boot 2026-02-25T11:59:12.418 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: fsmap 12 up:standby 2026-02-25T11:59:12.418 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2026-02-25T11:59:12.418 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2026-02-25T11:59:12.418 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2026-02-25T11:59:12.418 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2026-02-25T11:59:12.418 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2026-02-25T11:59:12.418 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:12 trial028 ceph-mon[39224]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2026-02-25T11:59:12.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.556+0000 7f31023ed640 1 -- 10.20.193.28:0/3622266555 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30fc159ea0 msgr2=0x7f30fc17a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:12.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.556+0000 7f31023ed640 1 --2- 10.20.193.28:0/3622266555 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30fc159ea0 0x7f30fc17a330 secure :-1 s=READY pgs=971 gs=1 cs=0 l=1 c_cookie=4902837622936670511 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f30f800aa30 tx=0x7f30f8030280 comp rx=0 tx=0).stop 2026-02-25T11:59:12.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.557+0000 7f31023ed640 1 -- 10.20.193.28:0/3622266555 shutdown_connections 2026-02-25T11:59:12.564 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.557+0000 7f31023ed640 1 --2- 10.20.193.28:0/3622266555 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30fc159ea0 0x7f30fc17a330 unknown :-1 s=CLOSED pgs=971 gs=1 cs=0 l=1 c_cookie=4902837622936670511 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:12.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.557+0000 7f31023ed640 1 --2- 10.20.193.28:0/3622266555 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f30fc0638e0 0x7f30fc063cf0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:12.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.557+0000 7f31023ed640 1 --2- 10.20.193.28:0/3622266555 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f30fc0672a0 0x7f30fc0632c0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:12.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.557+0000 7f31023ed640 1 -- 10.20.193.28:0/3622266555 >> 10.20.193.28:0/3622266555 conn(0x7f30fc0ecf00 msgr2=0x7f30fc10d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:12.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.557+0000 7f31023ed640 1 -- 10.20.193.28:0/3622266555 shutdown_connections 2026-02-25T11:59:12.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.557+0000 7f31023ed640 1 -- 10.20.193.28:0/3622266555 wait complete. 2026-02-25T11:59:12.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.558+0000 7f31023ed640 1 Processor -- start 2026-02-25T11:59:12.565 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.558+0000 7f31023ed640 1 -- start start 2026-02-25T11:59:12.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.558+0000 7f31023ed640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f30fc0638e0 0x7f30fc1641b0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:12.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.558+0000 7f31023ed640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f30fc0672a0 0x7f30fc1646f0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:12.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.558+0000 7f31023ed640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30fc159ea0 0x7f30fc169ac0 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:12.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.558+0000 7f31023ed640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc16a0e0 con 0x7f30fc159ea0 2026-02-25T11:59:12.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.558+0000 7f31023ed640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc16a250 con 0x7f30fc0672a0 2026-02-25T11:59:12.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.558+0000 7f31023ed640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc16a410 con 0x7f30fc0638e0 2026-02-25T11:59:12.566 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.558+0000 7f3100bea640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f30fc0672a0 0x7f30fc1646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:12.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.558+0000 7f3100bea640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f30fc0672a0 0x7f30fc1646f0 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.119:3300/0 says I am v2:10.20.193.28:37678/0 (socket says 10.20.193.28:37678) 2026-02-25T11:59:12.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.558+0000 7f3100bea640 1 -- 10.20.193.28:0/1033459101 learned_addr learned my addr 10.20.193.28:0/1033459101 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:12.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.558+0000 7f3101bec640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30fc159ea0 0x7f30fc169ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:12.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.558+0000 7f31013eb640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f30fc0638e0 0x7f30fc1641b0 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:12.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.559+0000 7f3101bec640 1 -- 10.20.193.28:0/1033459101 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f30fc0638e0 msgr2=0x7f30fc1641b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:12.567 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.559+0000 7f3101bec640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f30fc0638e0 0x7f30fc1641b0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:12.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.559+0000 7f3101bec640 1 -- 10.20.193.28:0/1033459101 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f30fc0672a0 msgr2=0x7f30fc1646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:12.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.559+0000 7f3101bec640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f30fc0672a0 0x7f30fc1646f0 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:12.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.559+0000 7f3101bec640 1 -- 10.20.193.28:0/1033459101 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f800a650 con 0x7f30fc159ea0 2026-02-25T11:59:12.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.559+0000 7f3100bea640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f30fc0672a0 0x7f30fc1646f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2026-02-25T11:59:12.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.559+0000 7f3101bec640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30fc159ea0 0x7f30fc169ac0 secure :-1 s=READY pgs=972 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f30f800aa30 tx=0x7f30f8039cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:12.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.559+0000 7f30ea7fc640 1 -- 10.20.193.28:0/1033459101 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f30f803e070 con 0x7f30fc159ea0 2026-02-25T11:59:12.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.559+0000 7f30ea7fc640 1 -- 10.20.193.28:0/1033459101 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f30f8039e80 con 0x7f30fc159ea0 2026-02-25T11:59:12.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.559+0000 7f30ea7fc640 1 -- 10.20.193.28:0/1033459101 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f30f8042940 con 0x7f30fc159ea0 2026-02-25T11:59:12.568 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.560+0000 7f31023ed640 1 -- 10.20.193.28:0/1033459101 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc16a690 con 0x7f30fc159ea0 2026-02-25T11:59:12.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.560+0000 7f31023ed640 1 -- 10.20.193.28:0/1033459101 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc157d50 con 0x7f30fc159ea0 2026-02-25T11:59:12.569 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.560+0000 7f31023ed640 1 -- 10.20.193.28:0/1033459101 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30fc0647d0 con 0x7f30fc159ea0 2026-02-25T11:59:12.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.562+0000 7f30ea7fc640 1 -- 10.20.193.28:0/1033459101 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f30f8042ad0 con 0x7f30fc159ea0 2026-02-25T11:59:12.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.564+0000 7f30ea7fc640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f30d80b5760 0x7f30d80d5c10 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:12.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.564+0000 7f30ea7fc640 1 -- 10.20.193.28:0/1033459101 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8446+0+0 (secure 0 0 0) 0x7f30f810c800 con 0x7f30fc159ea0 2026-02-25T11:59:12.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.564+0000 7f30ea7fc640 1 -- 10.20.193.28:0/1033459101 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f30f810cc80 con 0x7f30fc159ea0 2026-02-25T11:59:12.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.565+0000 7f31013eb640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f30d80b5760 0x7f30d80d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:12.570 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.565+0000 7f31013eb640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f30d80b5760 0x7f30d80d5c10 secure :-1 s=READY pgs=428 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f30ec004740 tx=0x7f30ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:12.682 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:12.680+0000 7f31023ed640 1 -- 10.20.193.28:0/1033459101 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f30fc063cf0 con 0x7f30fc159ea0 2026-02-25T11:59:12.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: pgmap v556: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2026-02-25T11:59:12.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1169517730' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2026-02-25T11:59:12.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2026-02-25T11:59:12.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2026-02-25T11:59:12.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: Cluster is now healthy 2026-02-25T11:59:12.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1169517730' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2026-02-25T11:59:12.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.119:6836/3736878020,v1:10.20.193.119:6838/3736878020] up:boot 2026-02-25T11:59:12.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.119:6837/2403226505,v1:10.20.193.119:6839/2403226505] up:boot 2026-02-25T11:59:12.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.167:6835/2168188582,v1:10.20.193.167:6838/2168188582] up:boot 2026-02-25T11:59:12.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.167:6834/1987058645,v1:10.20.193.167:6837/1987058645] up:boot 2026-02-25T11:59:12.750 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.28:6838/842890247,v1:10.20.193.28:6839/842890247] up:boot 2026-02-25T11:59:12.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: mds.? [v2:10.20.193.167:6836/1968657301,v1:10.20.193.167:6839/1968657301] up:boot 2026-02-25T11:59:12.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: fsmap 12 up:standby 2026-02-25T11:59:12.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2026-02-25T11:59:12.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2026-02-25T11:59:12.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2026-02-25T11:59:12.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2026-02-25T11:59:12.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2026-02-25T11:59:12.751 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:12 trial167 ceph-mon[43392]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2026-02-25T11:59:12.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: pgmap v556: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2026-02-25T11:59:12.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1169517730' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2026-02-25T11:59:12.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2026-02-25T11:59:12.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2026-02-25T11:59:12.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: Cluster is now healthy 2026-02-25T11:59:12.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1169517730' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2026-02-25T11:59:12.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.119:6836/3736878020,v1:10.20.193.119:6838/3736878020] up:boot 2026-02-25T11:59:12.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.119:6837/2403226505,v1:10.20.193.119:6839/2403226505] up:boot 2026-02-25T11:59:12.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.167:6835/2168188582,v1:10.20.193.167:6838/2168188582] up:boot 2026-02-25T11:59:12.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.167:6834/1987058645,v1:10.20.193.167:6837/1987058645] up:boot 2026-02-25T11:59:12.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.28:6838/842890247,v1:10.20.193.28:6839/842890247] up:boot 2026-02-25T11:59:12.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: mds.? [v2:10.20.193.167:6836/1968657301,v1:10.20.193.167:6839/1968657301] up:boot 2026-02-25T11:59:12.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: fsmap 12 up:standby 2026-02-25T11:59:12.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2026-02-25T11:59:12.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2026-02-25T11:59:12.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2026-02-25T11:59:12.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2026-02-25T11:59:12.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2026-02-25T11:59:12.761 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:12 trial119 ceph-mon[42710]: from='mgr.14152 10.20.193.28:0/2562488214' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2026-02-25T11:59:13.322 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.321+0000 7f30ea7fc640 1 -- 10.20.193.28:0/1033459101 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) ==== 181+0+0 (secure 0 0 0) 0x7f30f80ce9d0 con 0x7f30fc159ea0 2026-02-25T11:59:13.322 INFO:teuthology.orchestra.run.trial028.stderr:pool 'cephfs_metadata' removed 2026-02-25T11:59:13.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.323+0000 7f31023ed640 1 -- 10.20.193.28:0/1033459101 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f30d80b5760 msgr2=0x7f30d80d5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:13.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.323+0000 7f31023ed640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f30d80b5760 0x7f30d80d5c10 secure :-1 s=READY pgs=428 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f30ec004740 tx=0x7f30ec00b410 comp rx=0 tx=0).stop 2026-02-25T11:59:13.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.324+0000 7f31023ed640 1 -- 10.20.193.28:0/1033459101 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30fc159ea0 msgr2=0x7f30fc169ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:13.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.324+0000 7f31023ed640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30fc159ea0 0x7f30fc169ac0 secure :-1 s=READY pgs=972 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f30f800aa30 tx=0x7f30f8039cd0 comp rx=0 tx=0).stop 2026-02-25T11:59:13.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.324+0000 7f31023ed640 1 -- 10.20.193.28:0/1033459101 shutdown_connections 2026-02-25T11:59:13.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.324+0000 7f31023ed640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f30d80b5760 0x7f30d80d5c10 unknown :-1 s=CLOSED pgs=428 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:13.325 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.324+0000 7f31023ed640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f30fc159ea0 0x7f30fc169ac0 unknown :-1 s=CLOSED pgs=972 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:13.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.324+0000 7f31023ed640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f30fc0672a0 0x7f30fc1646f0 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:13.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.324+0000 7f31023ed640 1 --2- 10.20.193.28:0/1033459101 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f30fc0638e0 0x7f30fc1641b0 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:13.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.324+0000 7f31023ed640 1 -- 10.20.193.28:0/1033459101 >> 10.20.193.28:0/1033459101 conn(0x7f30fc0ecf00 msgr2=0x7f30fc10d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:13.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.324+0000 7f31023ed640 1 -- 10.20.193.28:0/1033459101 shutdown_connections 2026-02-25T11:59:13.326 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.324+0000 7f31023ed640 1 -- 10.20.193.28:0/1033459101 wait complete. 2026-02-25T11:59:13.427 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2026-02-25T11:59:13.611 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:59:13.655 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:13 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1033459101' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2026-02-25T11:59:13.749 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:13 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1033459101' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2026-02-25T11:59:13.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:13 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1033459101' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2026-02-25T11:59:13.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.786+0000 7f0e19748640 1 -- 10.20.193.28:0/3928763276 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e14115880 msgr2=0x7f0e14176250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:13.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.786+0000 7f0e19748640 1 --2- 10.20.193.28:0/3928763276 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e14115880 0x7f0e14176250 secure :-1 s=READY pgs=973 gs=1 cs=0 l=1 c_cookie=2658228195754714010 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e0800aa30 tx=0x7f0e08030270 comp rx=0 tx=0).stop 2026-02-25T11:59:13.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.787+0000 7f0e19748640 1 -- 10.20.193.28:0/3928763276 shutdown_connections 2026-02-25T11:59:13.789 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.787+0000 7f0e19748640 1 --2- 10.20.193.28:0/3928763276 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e14115880 0x7f0e14176250 unknown :-1 s=CLOSED pgs=973 gs=1 cs=0 l=1 c_cookie=2658228195754714010 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:13.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.787+0000 7f0e19748640 1 --2- 10.20.193.28:0/3928763276 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0e14110080 0x7f0e141150d0 unknown :-1 s=CLOSED pgs=0 gs=3 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:13.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.787+0000 7f0e19748640 1 --2- 10.20.193.28:0/3928763276 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0e1410f690 0x7f0e1410fa60 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=0 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:13.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.787+0000 7f0e19748640 1 -- 10.20.193.28:0/3928763276 >> 10.20.193.28:0/3928763276 conn(0x7f0e140ecf00 msgr2=0x7f0e1410d310 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:13.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.787+0000 7f0e19748640 1 -- 10.20.193.28:0/3928763276 shutdown_connections 2026-02-25T11:59:13.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.787+0000 7f0e19748640 1 -- 10.20.193.28:0/3928763276 wait complete. 2026-02-25T11:59:13.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.787+0000 7f0e19748640 1 Processor -- start 2026-02-25T11:59:13.790 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.787+0000 7f0e19748640 1 -- start start 2026-02-25T11:59:13.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e19748640 1 --2- >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0e1410f690 0x7f0e140f5910 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:13.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e19748640 1 --2- >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0e14110080 0x7f0e14176150 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:13.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e19748640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e14115880 0x7f0e140f6010 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:13.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e19748640 1 -- --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0e140fb220 con 0x7f0e14115880 2026-02-25T11:59:13.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e19748640 1 -- --> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0e140fb390 con 0x7f0e1410f690 2026-02-25T11:59:13.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e19748640 1 -- --> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0e140fb550 con 0x7f0e14110080 2026-02-25T11:59:13.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e18f47640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e14115880 0x7f0e140f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:13.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e18f47640 1 --2- >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e14115880 0x7f0e140f6010 unknown :-1 s=HELLO_CONNECTING pgs=0 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:10.20.193.28:3300/0 says I am v2:10.20.193.28:35164/0 (socket says 10.20.193.28:35164) 2026-02-25T11:59:13.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e18f47640 1 -- 10.20.193.28:0/1408565505 learned_addr learned my addr 10.20.193.28:0/1408565505 (peer_addr_for_me v2:10.20.193.28:0/0) 2026-02-25T11:59:13.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e13fff640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0e1410f690 0x7f0e140f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:13.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e137fe640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0e14110080 0x7f0e14176150 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:13.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e18f47640 1 -- 10.20.193.28:0/1408565505 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0e14110080 msgr2=0x7f0e14176150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:13.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e18f47640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0e14110080 0x7f0e14176150 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:13.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e18f47640 1 -- 10.20.193.28:0/1408565505 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0e1410f690 msgr2=0x7f0e140f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:13.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e18f47640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0e1410f690 0x7f0e140f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:13.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.788+0000 7f0e18f47640 1 -- 10.20.193.28:0/1408565505 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0800a650 con 0x7f0e14115880 2026-02-25T11:59:13.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.789+0000 7f0e18f47640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e14115880 0x7f0e140f6010 secure :-1 s=READY pgs=974 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e08006a50 tx=0x7f0e08003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:13.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.789+0000 7f0e117fa640 1 -- 10.20.193.28:0/1408565505 <== mon.0 v2:10.20.193.28:3300/0 1 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0e0803e070 con 0x7f0e14115880 2026-02-25T11:59:13.793 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.789+0000 7f0e117fa640 1 -- 10.20.193.28:0/1408565505 <== mon.0 v2:10.20.193.28:3300/0 2 ==== config(27 keys) ==== 1112+0+0 (secure 0 0 0) 0x7f0e08003ad0 con 0x7f0e14115880 2026-02-25T11:59:13.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.789+0000 7f0e19748640 1 -- 10.20.193.28:0/1408565505 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e140fb7d0 con 0x7f0e14115880 2026-02-25T11:59:13.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.789+0000 7f0e117fa640 1 -- 10.20.193.28:0/1408565505 <== mon.0 v2:10.20.193.28:3300/0 3 ==== mon_map magic: 0 ==== 437+0+0 (secure 0 0 0) 0x7f0e080428c0 con 0x7f0e14115880 2026-02-25T11:59:13.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.789+0000 7f0e19748640 1 -- 10.20.193.28:0/1408565505 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e140fbbf0 con 0x7f0e14115880 2026-02-25T11:59:13.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.790+0000 7f0de6ffd640 1 -- 10.20.193.28:0/1408565505 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ddc005300 con 0x7f0e14115880 2026-02-25T11:59:13.794 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.792+0000 7f0e117fa640 1 -- 10.20.193.28:0/1408565505 <== mon.0 v2:10.20.193.28:3300/0 4 ==== mgrmap(e 17) ==== 157122+0+0 (secure 0 0 0) 0x7f0e080050c0 con 0x7f0e14115880 2026-02-25T11:59:13.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.794+0000 7f0e117fa640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0de80b5750 0x7f0de80d5c00 unknown :-1 s=NONE pgs=0 gs=0 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2026-02-25T11:59:13.795 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.794+0000 7f0e117fa640 1 -- 10.20.193.28:0/1408565505 <== mon.0 v2:10.20.193.28:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7881+0+0 (secure 0 0 0) 0x7f0e0810c6d0 con 0x7f0e14115880 2026-02-25T11:59:13.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.794+0000 7f0e13fff640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0de80b5750 0x7f0de80d5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2026-02-25T11:59:13.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.794+0000 7f0e117fa640 1 -- 10.20.193.28:0/1408565505 <== mon.0 v2:10.20.193.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+231560 (secure 0 0 0) 0x7f0e0810cad0 con 0x7f0e14115880 2026-02-25T11:59:13.796 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.794+0000 7f0e13fff640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0de80b5750 0x7f0de80d5c00 secure :-1 s=READY pgs=429 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e00004740 tx=0x7f0e00005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2026-02-25T11:59:13.910 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:13.908+0000 7f0de6ffd640 1 -- 10.20.193.28:0/1408565505 --> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0ddc0055b0 con 0x7f0e14115880 2026-02-25T11:59:14.329 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:14.327+0000 7f0e117fa640 1 -- 10.20.193.28:0/1408565505 <== mon.0 v2:10.20.193.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) ==== 169+0+0 (secure 0 0 0) 0x7f0e080cf290 con 0x7f0e14115880 2026-02-25T11:59:14.329 INFO:teuthology.orchestra.run.trial028.stderr:pool 'cephfs_data' removed 2026-02-25T11:59:14.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:14.330+0000 7f0de6ffd640 1 -- 10.20.193.28:0/1408565505 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0de80b5750 msgr2=0x7f0de80d5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:14.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:14.330+0000 7f0de6ffd640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0de80b5750 0x7f0de80d5c00 secure :-1 s=READY pgs=429 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e00004740 tx=0x7f0e00005f90 comp rx=0 tx=0).stop 2026-02-25T11:59:14.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:14.330+0000 7f0de6ffd640 1 -- 10.20.193.28:0/1408565505 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e14115880 msgr2=0x7f0e140f6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2026-02-25T11:59:14.331 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:14.330+0000 7f0de6ffd640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e14115880 0x7f0e140f6010 secure :-1 s=READY pgs=974 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0x7f0e08006a50 tx=0x7f0e08003f60 comp rx=0 tx=0).stop 2026-02-25T11:59:14.332 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:14.330+0000 7f0de6ffd640 1 -- 10.20.193.28:0/1408565505 shutdown_connections 2026-02-25T11:59:14.332 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:14.330+0000 7f0de6ffd640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.28:6800/3140694734,v1:10.20.193.28:6801/3140694734] conn(0x7f0de80b5750 0x7f0de80d5c00 unknown :-1 s=CLOSED pgs=429 gs=4 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:14.332 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:14.330+0000 7f0de6ffd640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] conn(0x7f0e14115880 0x7f0e140f6010 unknown :-1 s=CLOSED pgs=974 gs=3 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:14.332 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:14.330+0000 7f0de6ffd640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.167:3300/0,v1:10.20.193.167:6789/0] conn(0x7f0e14110080 0x7f0e14176150 unknown :-1 s=CLOSED pgs=0 gs=2 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:14.332 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:14.330+0000 7f0de6ffd640 1 --2- 10.20.193.28:0/1408565505 >> [v2:10.20.193.119:3300/0,v1:10.20.193.119:6789/0] conn(0x7f0e1410f690 0x7f0e140f5910 unknown :-1 s=CLOSED pgs=0 gs=1 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2026-02-25T11:59:14.332 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:14.330+0000 7f0de6ffd640 1 -- 10.20.193.28:0/1408565505 >> 10.20.193.28:0/1408565505 conn(0x7f0e140ecf00 msgr2=0x7f0e14111ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2026-02-25T11:59:14.332 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:14.330+0000 7f0de6ffd640 1 -- 10.20.193.28:0/1408565505 shutdown_connections 2026-02-25T11:59:14.332 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-25T11:59:14.330+0000 7f0de6ffd640 1 -- 10.20.193.28:0/1408565505 wait complete. 2026-02-25T11:59:14.432 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-25T11:59:14.492 DEBUG:teuthology.orchestra.run.trial119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-25T11:59:14.552 DEBUG:teuthology.orchestra.run.trial167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-25T11:59:14.607 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:14 trial167 ceph-mon[43392]: pgmap v557: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.0 KiB/s rd, 716 B/s wr, 7 op/s 2026-02-25T11:59:14.607 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:14 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1033459101' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2026-02-25T11:59:14.607 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:14 trial167 ceph-mon[43392]: osdmap e90: 12 total, 12 up, 12 in 2026-02-25T11:59:14.607 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:14 trial167 ceph-mon[43392]: from='client.? 10.20.193.28:0/1408565505' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2026-02-25T11:59:14.615 INFO:tasks.cephadm:Disabling cephadm mgr module 2026-02-25T11:59:14.615 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 -- ceph mgr module disable cephadm 2026-02-25T11:59:14.665 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:14 trial028 ceph-mon[39224]: pgmap v557: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.0 KiB/s rd, 716 B/s wr, 7 op/s 2026-02-25T11:59:14.665 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:14 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1033459101' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2026-02-25T11:59:14.665 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:14 trial028 ceph-mon[39224]: osdmap e90: 12 total, 12 up, 12 in 2026-02-25T11:59:14.665 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:14 trial028 ceph-mon[39224]: from='client.? 10.20.193.28:0/1408565505' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2026-02-25T11:59:14.759 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:14 trial119 ceph-mon[42710]: pgmap v557: 97 pgs: 97 active+clean; 7.8 MiB data, 352 MiB used, 8.2 TiB / 8.2 TiB avail; 3.0 KiB/s rd, 716 B/s wr, 7 op/s 2026-02-25T11:59:14.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:14 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1033459101' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2026-02-25T11:59:14.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:14 trial119 ceph-mon[42710]: osdmap e90: 12 total, 12 up, 12 in 2026-02-25T11:59:14.760 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:14 trial119 ceph-mon[42710]: from='client.? 10.20.193.28:0/1408565505' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2026-02-25T11:59:14.799 INFO:teuthology.orchestra.run.trial028.stderr:Inferring config /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/mon.a/config 2026-02-25T11:59:14.833 INFO:teuthology.orchestra.run.trial028.stderr:Error: statfs /etc/ceph/ceph.client.admin.keyring: no such file or directory 2026-02-25T11:59:14.859 DEBUG:teuthology.orchestra.run:got remote process result: 125 2026-02-25T11:59:14.860 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2026-02-25T11:59:14.860 DEBUG:teuthology.orchestra.run.trial028:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2026-02-25T11:59:14.899 DEBUG:teuthology.orchestra.run.trial119:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2026-02-25T11:59:14.938 DEBUG:teuthology.orchestra.run.trial167:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2026-02-25T11:59:14.976 INFO:tasks.cephadm:Stopping all daemons... 2026-02-25T11:59:14.976 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2026-02-25T11:59:14.976 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.a 2026-02-25T11:59:15.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:15 trial028 systemd[1]: Stopping Ceph mon.a for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:59:15.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:15 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-a[39219]: 2026-02-25T11:59:15.159+0000 7f58eb686640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T11:59:15.309 INFO:journalctl@ceph.mon.a.trial028.stdout:Feb 25 11:59:15 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-a[39219]: 2026-02-25T11:59:15.159+0000 7f58eb686640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2026-02-25T11:59:15.486 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.a.service' 2026-02-25T11:59:15.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T11:59:15.563 INFO:tasks.cephadm.mon.a:Stopped mon.a 2026-02-25T11:59:15.563 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2026-02-25T11:59:15.564 DEBUG:teuthology.orchestra.run.trial119:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.b 2026-02-25T11:59:15.845 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:15 trial119 systemd[1]: Stopping Ceph mon.b for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:59:15.845 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:15 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-b[42706]: 2026-02-25T11:59:15.751+0000 7f1b2c67a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T11:59:15.845 INFO:journalctl@ceph.mon.b.trial119.stdout:Feb 25 11:59:15 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-b[42706]: 2026-02-25T11:59:15.751+0000 7f1b2c67a640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2026-02-25T11:59:16.010 DEBUG:teuthology.orchestra.run.trial119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.b.service' 2026-02-25T11:59:16.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T11:59:16.089 INFO:tasks.cephadm.mon.c:Stopped mon.b 2026-02-25T11:59:16.089 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2026-02-25T11:59:16.089 DEBUG:teuthology.orchestra.run.trial167:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.c 2026-02-25T11:59:16.394 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:16 trial167 systemd[1]: Stopping Ceph mon.c for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:59:16.394 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:16 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-c[43385]: 2026-02-25T11:59:16.278+0000 7f9a1d3a7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T11:59:16.394 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:16 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-c[43385]: 2026-02-25T11:59:16.278+0000 7f9a1d3a7640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2026-02-25T11:59:16.394 INFO:journalctl@ceph.mon.c.trial167.stdout:Feb 25 11:59:16 trial167 podman[101392]: 2026-02-25 11:59:16.381051505 +0000 UTC m=+0.125991964 container died d2a97993071e346c86d7f93adbba2c1669fedecaab1ea50238116a059fc79278 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mon-c, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_REF=wip-vshankar-testing-20260224.100235, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, io.buildah.version=1.41.3, org.label-schema.license=GPLv2, OSD_FLAVOR=default) 2026-02-25T11:59:16.547 DEBUG:teuthology.orchestra.run.trial167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mon.c.service' 2026-02-25T11:59:16.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T11:59:16.624 INFO:tasks.cephadm.mon.c:Stopped mon.c 2026-02-25T11:59:16.624 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2026-02-25T11:59:16.624 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mgr.x 2026-02-25T11:59:16.953 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:59:16 trial028 systemd[1]: Stopping Ceph mgr.x for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:59:16.953 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:59:16 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x[39457]: 2026-02-25T11:59:16.808+0000 7f01ce854640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T11:59:16.953 INFO:journalctl@ceph.mgr.x.trial028.stdout:Feb 25 11:59:16 trial028 podman[135079]: 2026-02-25 11:59:16.901152376 +0000 UTC m=+0.116008021 container died d6b9f684dc01ab9efd29ab55b945d8738eb1da04c56ebda86ee94ddfb347ee48 (image=quay.ceph.io/ceph-ci/ceph:92715cd3c42198664811edeb975bd585312154aa, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-x, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.41.3, CEPH_REF=wip-vshankar-testing-20260224.100235, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.build-date=20260223, org.label-schema.license=GPLv2) 2026-02-25T11:59:17.090 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mgr.x.service' 2026-02-25T11:59:17.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T11:59:17.163 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2026-02-25T11:59:17.163 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2026-02-25T11:59:17.163 DEBUG:teuthology.orchestra.run.trial119:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mgr.y 2026-02-25T11:59:17.475 INFO:journalctl@ceph.mgr.y.trial119.stdout:Feb 25 11:59:17 trial119 systemd[1]: Stopping Ceph mgr.y for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:59:17.475 INFO:journalctl@ceph.mgr.y.trial119.stdout:Feb 25 11:59:17 trial119 podman[99578]: 2026-02-25 11:59:17.42562588 +0000 UTC m=+0.095215525 container died defaf63b2fbc03a9801193292ca97d8207a16174f81676721ad1d9149d1e07c7 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-y, io.buildah.version=1.41.3, org.label-schema.schema-version=1.0, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , ceph=True, org.label-schema.build-date=20260223, CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, OSD_FLAVOR=default) 2026-02-25T11:59:17.607 DEBUG:teuthology.orchestra.run.trial119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mgr.y.service' 2026-02-25T11:59:17.677 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T11:59:17.677 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2026-02-25T11:59:17.677 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2026-02-25T11:59:17.677 DEBUG:teuthology.orchestra.run.trial167:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mgr.z 2026-02-25T11:59:17.994 INFO:journalctl@ceph.mgr.z.trial167.stdout:Feb 25 11:59:17 trial167 systemd[1]: Stopping Ceph mgr.z for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:59:17.994 INFO:journalctl@ceph.mgr.z.trial167.stdout:Feb 25 11:59:17 trial167 podman[101525]: 2026-02-25 11:59:17.943900732 +0000 UTC m=+0.104169258 container died 516c7ac1c961b6f1e656015d0dc12510aea03dd0508f2979a725c7459fd99bce (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mgr-z, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.build-date=20260223, OSD_FLAVOR=default, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, io.buildah.version=1.41.3, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-vshankar-testing-20260224.100235, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.vendor=CentOS) 2026-02-25T11:59:18.138 DEBUG:teuthology.orchestra.run.trial167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mgr.z.service' 2026-02-25T11:59:18.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T11:59:18.206 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2026-02-25T11:59:18.206 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2026-02-25T11:59:18.206 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.0 2026-02-25T11:59:18.559 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:59:18 trial028 systemd[1]: Stopping Ceph osd.0 for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:59:18.560 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:59:18 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-0[58013]: 2026-02-25T11:59:18.386+0000 7f985d87f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-25T11:59:18.560 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:59:18 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-0[58013]: 2026-02-25T11:59:18.386+0000 7f985d87f640 -1 osd.0 91 *** Got signal Terminated *** 2026-02-25T11:59:18.560 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:59:18 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-0[58013]: 2026-02-25T11:59:18.386+0000 7f985d87f640 -1 osd.0 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-25T11:59:23.797 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:59:23 trial028 podman[135212]: 2026-02-25 11:59:23.491634002 +0000 UTC m=+5.128631743 container died 778bff754f863c0efc0ab5877287f88e2ecfbffd9fa5a2fa5006928d193628a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223, org.label-schema.schema-version=1.0, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, OSD_FLAVOR=default, FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_REF=wip-vshankar-testing-20260224.100235, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-25T11:59:23.797 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:59:23 trial028 podman[135212]: 2026-02-25 11:59:23.551278714 +0000 UTC m=+5.188276445 container remove 778bff754f863c0efc0ab5877287f88e2ecfbffd9fa5a2fa5006928d193628a0 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-0, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.vendor=CentOS, CEPH_REF=wip-vshankar-testing-20260224.100235, io.buildah.version=1.41.3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-25T11:59:23.797 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:59:23 trial028 bash[135212]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-0 2026-02-25T11:59:24.059 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:59:23 trial028 podman[135314]: 2026-02-25 11:59:23.796712918 +0000 UTC m=+0.026930733 container create 868d40536d6c4c604280a5b924b9d311633ec4c90494946b5e33c253f68a1d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-0-deactivate, OSD_FLAVOR=default, org.label-schema.build-date=20260223, CEPH_REF=wip-vshankar-testing-20260224.100235, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2026-02-25T11:59:24.059 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:59:23 trial028 podman[135314]: 2026-02-25 11:59:23.851015421 +0000 UTC m=+0.081233246 container init 868d40536d6c4c604280a5b924b9d311633ec4c90494946b5e33c253f68a1d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-0-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.41.3, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, OSD_FLAVOR=default, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , ceph=True, org.label-schema.schema-version=1.0, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-25T11:59:24.059 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:59:23 trial028 podman[135314]: 2026-02-25 11:59:23.856113877 +0000 UTC m=+0.086331702 container start 868d40536d6c4c604280a5b924b9d311633ec4c90494946b5e33c253f68a1d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-0-deactivate, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.opencontainers.image.authors=Ceph Release Team , io.buildah.version=1.41.3, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.build-date=20260223, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2026-02-25T11:59:24.059 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:59:23 trial028 podman[135314]: 2026-02-25 11:59:23.857032943 +0000 UTC m=+0.087250778 container attach 868d40536d6c4c604280a5b924b9d311633ec4c90494946b5e33c253f68a1d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-0-deactivate, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=wip-vshankar-testing-20260224.100235, OSD_FLAVOR=default, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, io.buildah.version=1.41.3, org.label-schema.schema-version=1.0, ceph=True, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20260223, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa) 2026-02-25T11:59:24.059 INFO:journalctl@ceph.osd.0.trial028.stdout:Feb 25 11:59:23 trial028 podman[135314]: 2026-02-25 11:59:23.787323375 +0000 UTC m=+0.017541200 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:59:24.145 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.0.service' 2026-02-25T11:59:24.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T11:59:24.218 INFO:tasks.cephadm.osd.0:Stopped osd.0 2026-02-25T11:59:24.218 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2026-02-25T11:59:24.218 DEBUG:teuthology.orchestra.run.trial119:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.1 2026-02-25T11:59:24.760 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:59:24 trial119 systemd[1]: Stopping Ceph osd.1 for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:59:24.760 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:59:24 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-1[49059]: 2026-02-25T11:59:24.398+0000 7ff854bfd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-25T11:59:24.760 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:59:24 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-1[49059]: 2026-02-25T11:59:24.398+0000 7ff854bfd640 -1 osd.1 91 *** Got signal Terminated *** 2026-02-25T11:59:24.760 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:59:24 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-1[49059]: 2026-02-25T11:59:24.398+0000 7ff854bfd640 -1 osd.1 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-25T11:59:29.823 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:59:29 trial119 podman[99709]: 2026-02-25 11:59:29.514169515 +0000 UTC m=+5.139436758 container died 6ba86dbfd87eda4394e2a5bd93c1df9312f0d94ae9622ebb5a12f6262f9e3a5b (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-1, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.license=GPLv2, OSD_FLAVOR=default, org.label-schema.build-date=20260223, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.schema-version=1.0, io.buildah.version=1.41.3) 2026-02-25T11:59:29.823 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:59:29 trial119 podman[99709]: 2026-02-25 11:59:29.571940109 +0000 UTC m=+5.197207352 container remove 6ba86dbfd87eda4394e2a5bd93c1df9312f0d94ae9622ebb5a12f6262f9e3a5b (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, CEPH_REF=wip-vshankar-testing-20260224.100235, org.opencontainers.image.authors=Ceph Release Team , io.buildah.version=1.41.3, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, ceph=True, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.license=GPLv2) 2026-02-25T11:59:29.823 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:59:29 trial119 bash[99709]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-1 2026-02-25T11:59:30.084 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:59:29 trial119 podman[99812]: 2026-02-25 11:59:29.822047391 +0000 UTC m=+0.027429980 container create 905b3a8b1e26838d5b899f399d60475580b6f43c8c27939d11598fc210907a89 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-1-deactivate, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20260223, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.vendor=CentOS, io.buildah.version=1.41.3, ceph=True, CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, OSD_FLAVOR=default) 2026-02-25T11:59:30.085 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:59:29 trial119 podman[99812]: 2026-02-25 11:59:29.877877571 +0000 UTC m=+0.083260150 container init 905b3a8b1e26838d5b899f399d60475580b6f43c8c27939d11598fc210907a89 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-1-deactivate, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.41.3, ceph=True, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.build-date=20260223, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.schema-version=1.0, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T11:59:30.085 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:59:29 trial119 podman[99812]: 2026-02-25 11:59:29.883442797 +0000 UTC m=+0.088825386 container start 905b3a8b1e26838d5b899f399d60475580b6f43c8c27939d11598fc210907a89 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-1-deactivate, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, io.buildah.version=1.41.3, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20260223, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.vendor=CentOS, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.name=CentOS Stream 9 Base Image, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-25T11:59:30.085 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:59:29 trial119 podman[99812]: 2026-02-25 11:59:29.88435398 +0000 UTC m=+0.089736579 container attach 905b3a8b1e26838d5b899f399d60475580b6f43c8c27939d11598fc210907a89 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-1-deactivate, org.opencontainers.image.authors=Ceph Release Team , io.buildah.version=1.41.3, OSD_FLAVOR=default, org.label-schema.license=GPLv2, ceph=True, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.build-date=20260223, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-25T11:59:30.085 INFO:journalctl@ceph.osd.1.trial119.stdout:Feb 25 11:59:29 trial119 podman[99812]: 2026-02-25 11:59:29.812408467 +0000 UTC m=+0.017791066 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:59:30.168 DEBUG:teuthology.orchestra.run.trial119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.1.service' 2026-02-25T11:59:30.240 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T11:59:30.240 INFO:tasks.cephadm.osd.1:Stopped osd.1 2026-02-25T11:59:30.241 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2026-02-25T11:59:30.241 DEBUG:teuthology.orchestra.run.trial167:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.2 2026-02-25T11:59:30.749 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:59:30 trial167 systemd[1]: Stopping Ceph osd.2 for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:59:30.749 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:59:30 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2[49994]: 2026-02-25T11:59:30.423+0000 7f24e9395640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-25T11:59:30.750 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:59:30 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2[49994]: 2026-02-25T11:59:30.423+0000 7f24e9395640 -1 osd.2 91 *** Got signal Terminated *** 2026-02-25T11:59:30.750 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:59:30 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2[49994]: 2026-02-25T11:59:30.423+0000 7f24e9395640 -1 osd.2 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-25T11:59:35.840 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:59:35 trial167 podman[101658]: 2026-02-25 11:59:35.528354365 +0000 UTC m=+5.128292814 container died 6131fde28a674ff6ac4990938ed45c0fe6fc99874dc787e352e248b7a7c3fb27 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, CEPH_REF=wip-vshankar-testing-20260224.100235, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.41.3) 2026-02-25T11:59:35.840 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:59:35 trial167 podman[101658]: 2026-02-25 11:59:35.587392356 +0000 UTC m=+5.187330815 container remove 6131fde28a674ff6ac4990938ed45c0fe6fc99874dc787e352e248b7a7c3fb27 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2, org.opencontainers.image.authors=Ceph Release Team , io.buildah.version=1.41.3, org.label-schema.schema-version=1.0, CEPH_REF=wip-vshankar-testing-20260224.100235, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, org.label-schema.vendor=CentOS, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-25T11:59:35.841 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:59:35 trial167 bash[101658]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2 2026-02-25T11:59:35.841 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:59:35 trial167 podman[101757]: 2026-02-25 11:59:35.838936165 +0000 UTC m=+0.026766285 container create 8f8d4bde469868ef0461d00821556c0e7940ce0266641cec5d2293a43f482ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2-deactivate, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, OSD_FLAVOR=default, io.buildah.version=1.41.3, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.schema-version=1.0, ceph=True) 2026-02-25T11:59:36.100 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:59:35 trial167 podman[101757]: 2026-02-25 11:59:35.894277316 +0000 UTC m=+0.082107446 container init 8f8d4bde469868ef0461d00821556c0e7940ce0266641cec5d2293a43f482ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2-deactivate, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2026-02-25T11:59:36.101 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:59:35 trial167 podman[101757]: 2026-02-25 11:59:35.899846206 +0000 UTC m=+0.087676326 container start 8f8d4bde469868ef0461d00821556c0e7940ce0266641cec5d2293a43f482ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2-deactivate, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.41.3, org.label-schema.license=GPLv2, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.schema-version=1.0, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, OSD_FLAVOR=default, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260223, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-25T11:59:36.101 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:59:35 trial167 podman[101757]: 2026-02-25 11:59:35.900790962 +0000 UTC m=+0.088621082 container attach 8f8d4bde469868ef0461d00821556c0e7940ce0266641cec5d2293a43f482ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2-deactivate, org.label-schema.build-date=20260223, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, OSD_FLAVOR=default, org.label-schema.schema-version=1.0) 2026-02-25T11:59:36.101 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:59:35 trial167 podman[101757]: 2026-02-25 11:59:35.829927032 +0000 UTC m=+0.017757162 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:59:36.101 INFO:journalctl@ceph.osd.2.trial167.stdout:Feb 25 11:59:36 trial167 podman[101757]: 2026-02-25 11:59:36.086870968 +0000 UTC m=+0.274701088 container died 8f8d4bde469868ef0461d00821556c0e7940ce0266641cec5d2293a43f482ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-2-deactivate, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, io.buildah.version=1.41.3, org.label-schema.schema-version=1.0, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.vendor=CentOS, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20260223, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2) 2026-02-25T11:59:36.171 DEBUG:teuthology.orchestra.run.trial167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.2.service' 2026-02-25T11:59:36.242 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T11:59:36.242 INFO:tasks.cephadm.osd.2:Stopped osd.2 2026-02-25T11:59:36.243 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2026-02-25T11:59:36.243 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.3 2026-02-25T11:59:36.559 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:59:36 trial028 systemd[1]: Stopping Ceph osd.3 for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:59:36.560 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:59:36 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3[70080]: 2026-02-25T11:59:36.432+0000 7f3e591ee640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-25T11:59:36.560 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:59:36 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3[70080]: 2026-02-25T11:59:36.432+0000 7f3e591ee640 -1 osd.3 91 *** Got signal Terminated *** 2026-02-25T11:59:36.560 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:59:36 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3[70080]: 2026-02-25T11:59:36.433+0000 7f3e591ee640 -1 osd.3 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-25T11:59:41.864 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:59:41 trial028 podman[135429]: 2026-02-25 11:59:41.559410172 +0000 UTC m=+5.148615796 container died b3fc165ad5f7fac035fbb19070c849ed1912efe3e3e9b1fad5d2615e059a1240 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.41.3, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260223, OSD_FLAVOR=default, org.label-schema.license=GPLv2, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=quay.io/centos/centos:stream9) 2026-02-25T11:59:41.865 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:59:41 trial028 podman[135429]: 2026-02-25 11:59:41.616939225 +0000 UTC m=+5.206144849 container remove b3fc165ad5f7fac035fbb19070c849ed1912efe3e3e9b1fad5d2615e059a1240 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.41.3, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.build-date=20260223, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=wip-vshankar-testing-20260224.100235, OSD_FLAVOR=default, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T11:59:41.865 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:59:41 trial028 bash[135429]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3 2026-02-25T11:59:42.126 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:59:41 trial028 podman[135534]: 2026-02-25 11:59:41.864083655 +0000 UTC m=+0.028627954 container create 1ad98439d14cd20320949256fe988d5dc0d0171c8e25a3bea038c99663479ef2 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.license=GPLv2, org.label-schema.build-date=20260223, OSD_FLAVOR=default, ceph=True, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.41.3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9) 2026-02-25T11:59:42.127 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:59:41 trial028 podman[135534]: 2026-02-25 11:59:41.911360112 +0000 UTC m=+0.075904421 container init 1ad98439d14cd20320949256fe988d5dc0d0171c8e25a3bea038c99663479ef2 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3-deactivate, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.build-date=20260223, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, ceph=True, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.documentation=https://docs.ceph.com/, io.buildah.version=1.41.3) 2026-02-25T11:59:42.127 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:59:41 trial028 podman[135534]: 2026-02-25 11:59:41.917049366 +0000 UTC m=+0.081593655 container start 1ad98439d14cd20320949256fe988d5dc0d0171c8e25a3bea038c99663479ef2 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3-deactivate, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T11:59:42.127 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:59:41 trial028 podman[135534]: 2026-02-25 11:59:41.917961509 +0000 UTC m=+0.082505818 container attach 1ad98439d14cd20320949256fe988d5dc0d0171c8e25a3bea038c99663479ef2 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, ceph=True, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.schema-version=1.0, org.label-schema.build-date=20260223, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_REF=wip-vshankar-testing-20260224.100235, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T11:59:42.127 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:59:41 trial028 podman[135534]: 2026-02-25 11:59:41.854726752 +0000 UTC m=+0.019271051 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:59:42.127 INFO:journalctl@ceph.osd.3.trial028.stdout:Feb 25 11:59:42 trial028 podman[135534]: 2026-02-25 11:59:42.111890304 +0000 UTC m=+0.276434613 container died 1ad98439d14cd20320949256fe988d5dc0d0171c8e25a3bea038c99663479ef2 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-3-deactivate, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.build-date=20260223, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, ceph=True, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.41.3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.vendor=CentOS) 2026-02-25T11:59:42.200 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.3.service' 2026-02-25T11:59:42.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T11:59:42.270 INFO:tasks.cephadm.osd.3:Stopped osd.3 2026-02-25T11:59:42.270 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2026-02-25T11:59:42.270 DEBUG:teuthology.orchestra.run.trial119:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.4 2026-02-25T11:59:42.759 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:59:42 trial119 systemd[1]: Stopping Ceph osd.4 for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:59:42.760 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:59:42 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4[60946]: 2026-02-25T11:59:42.456+0000 7f50988a1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-25T11:59:42.760 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:59:42 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4[60946]: 2026-02-25T11:59:42.456+0000 7f50988a1640 -1 osd.4 91 *** Got signal Terminated *** 2026-02-25T11:59:42.760 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:59:42 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4[60946]: 2026-02-25T11:59:42.456+0000 7f50988a1640 -1 osd.4 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-25T11:59:45.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:45 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:45.333+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:46.249 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:45 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:59:45.936+0000 7f168ef88640 -1 osd.5 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.078552+0000 front 2026-02-25T11:59:21.078487+0000 (oldest deadline 2026-02-25T11:59:45.778511+0000) 2026-02-25T11:59:46.350 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:46 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:46.060+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:46.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:46 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:46.392+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:46.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:46 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:46.349+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:47.249 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:46 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:59:46.964+0000 7f168ef88640 -1 osd.5 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.078552+0000 front 2026-02-25T11:59:21.078487+0000 (oldest deadline 2026-02-25T11:59:45.778511+0000) 2026-02-25T11:59:47.309 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:47 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:47.026+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:47.634 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:59:47 trial119 podman[99927]: 2026-02-25 11:59:47.56198999 +0000 UTC m=+5.129383634 container died 0e3df6ef3446a2cdd253976bcf5e4e4a9847b334cf08d6bd1c1f3d592bc94d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4, OSD_FLAVOR=default, FROM_IMAGE=quay.io/centos/centos:stream9, ceph=True, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.schema-version=1.0, org.label-schema.build-date=20260223, io.buildah.version=1.41.3, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.opencontainers.image.authors=Ceph Release Team , CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.vendor=CentOS) 2026-02-25T11:59:47.634 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:59:47 trial119 podman[99927]: 2026-02-25 11:59:47.620346527 +0000 UTC m=+5.187740182 container remove 0e3df6ef3446a2cdd253976bcf5e4e4a9847b334cf08d6bd1c1f3d592bc94d59 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, io.buildah.version=1.41.3, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True) 2026-02-25T11:59:47.634 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:59:47 trial119 bash[99927]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4 2026-02-25T11:59:47.634 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:47 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:47.378+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:47.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:47 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:47.383+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:47.899 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:59:47 trial119 podman[100029]: 2026-02-25 11:59:47.865804147 +0000 UTC m=+0.027432240 container create 2baad5b566044cd7d2b61577aba87f2330ce7a639664196d31d806f8489dccfd (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4-deactivate, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=wip-vshankar-testing-20260224.100235, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260223, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.41.3) 2026-02-25T11:59:48.164 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:59:47 trial119 podman[100029]: 2026-02-25 11:59:47.910545172 +0000 UTC m=+0.072173265 container init 2baad5b566044cd7d2b61577aba87f2330ce7a639664196d31d806f8489dccfd (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.build-date=20260223, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, io.buildah.version=1.41.3, org.label-schema.schema-version=1.0, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.license=GPLv2, ceph=True) 2026-02-25T11:59:48.164 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:59:47 trial119 podman[100029]: 2026-02-25 11:59:47.916194681 +0000 UTC m=+0.077822774 container start 2baad5b566044cd7d2b61577aba87f2330ce7a639664196d31d806f8489dccfd (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4-deactivate, OSD_FLAVOR=default, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.build-date=20260223, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.schema-version=1.0, CEPH_REF=wip-vshankar-testing-20260224.100235, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.vendor=CentOS, io.buildah.version=1.41.3, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=quay.io/centos/centos:stream9, ceph=True) 2026-02-25T11:59:48.164 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:59:47 trial119 podman[100029]: 2026-02-25 11:59:47.917153002 +0000 UTC m=+0.078781095 container attach 2baad5b566044cd7d2b61577aba87f2330ce7a639664196d31d806f8489dccfd (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4-deactivate, org.label-schema.schema-version=1.0, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.build-date=20260223, org.opencontainers.image.documentation=https://docs.ceph.com/, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, ceph=True, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T11:59:48.165 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:59:47 trial119 podman[100029]: 2026-02-25 11:59:47.855865498 +0000 UTC m=+0.017493591 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:59:48.165 INFO:journalctl@ceph.osd.4.trial119.stdout:Feb 25 11:59:48 trial119 podman[100029]: 2026-02-25 11:59:48.108567279 +0000 UTC m=+0.270195373 container died 2baad5b566044cd7d2b61577aba87f2330ce7a639664196d31d806f8489dccfd (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-4-deactivate, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, ceph=True, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2026-02-25T11:59:48.194 DEBUG:teuthology.orchestra.run.trial119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.4.service' 2026-02-25T11:59:48.249 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:47 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:59:47.965+0000 7f168ef88640 -1 osd.5 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.078552+0000 front 2026-02-25T11:59:21.078487+0000 (oldest deadline 2026-02-25T11:59:45.778511+0000) 2026-02-25T11:59:48.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T11:59:48.266 INFO:tasks.cephadm.osd.4:Stopped osd.4 2026-02-25T11:59:48.266 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2026-02-25T11:59:48.267 DEBUG:teuthology.orchestra.run.trial167:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.5 2026-02-25T11:59:48.343 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:48 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:48.075+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:48.510 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:48 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:48.372+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:48.749 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:48 trial167 systemd[1]: Stopping Ceph osd.5 for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:59:48.749 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:48 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:59:48.452+0000 7f16928c0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-25T11:59:48.750 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:48 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:59:48.452+0000 7f16928c0640 -1 osd.5 91 *** Got signal Terminated *** 2026-02-25T11:59:48.750 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:48 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:59:48.452+0000 7f16928c0640 -1 osd.5 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-25T11:59:48.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:48 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:48.342+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:49.249 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:48 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:59:48.931+0000 7f168ef88640 -1 osd.5 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.078552+0000 front 2026-02-25T11:59:21.078487+0000 (oldest deadline 2026-02-25T11:59:45.778511+0000) 2026-02-25T11:59:49.393 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:49 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:49.071+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:49.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:49 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:49.330+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:49.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:49 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:49.392+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:49.943 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:59:49 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:59:49.534+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T11:59:50.249 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:49 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:59:49.941+0000 7f168ef88640 -1 osd.5 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.078552+0000 front 2026-02-25T11:59:21.078487+0000 (oldest deadline 2026-02-25T11:59:45.778511+0000) 2026-02-25T11:59:50.414 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:50 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:50.112+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:50.414 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:50 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:50.112+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T11:59:50.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:50 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:50.356+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:50.760 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 11:59:50 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T11:59:50.314+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T11:59:50.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:50 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:50.413+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:50.988 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:59:50 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:59:50.568+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T11:59:51.249 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:50 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:59:50.986+0000 7f168ef88640 -1 osd.5 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.078552+0000 front 2026-02-25T11:59:21.078487+0000 (oldest deadline 2026-02-25T11:59:45.778511+0000) 2026-02-25T11:59:51.405 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:51 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:51.090+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:51.405 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:51 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:51.090+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T11:59:51.759 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 11:59:51 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T11:59:51.305+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T11:59:51.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:51 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:51.313+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:51.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:51 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:51.313+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T11:59:51.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:51 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:51.404+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:51.972 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:59:51 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:59:51.604+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T11:59:52.249 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:51 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:59:51.971+0000 7f168ef88640 -1 osd.5 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.078552+0000 front 2026-02-25T11:59:21.078487+0000 (oldest deadline 2026-02-25T11:59:45.778511+0000) 2026-02-25T11:59:52.249 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:51 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:59:51.971+0000 7f168ef88640 -1 osd.5 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:28.579243+0000 front 2026-02-25T11:59:28.579291+0000 (oldest deadline 2026-02-25T11:59:51.479349+0000) 2026-02-25T11:59:52.250 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:59:52 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:59:52.090+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T11:59:52.559 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:52 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:52.368+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:52.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:52 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:52.119+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:52.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:52 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:52.119+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T11:59:52.759 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 11:59:52 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T11:59:52.279+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T11:59:52.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:52 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:52.354+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:52.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:52 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:52.354+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T11:59:52.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:59:52 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:59:52.604+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T11:59:53.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:59:53 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:59:53.040+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T11:59:53.500 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:53 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:59:53.010+0000 7f168ef88640 -1 osd.5 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.078552+0000 front 2026-02-25T11:59:21.078487+0000 (oldest deadline 2026-02-25T11:59:45.778511+0000) 2026-02-25T11:59:53.500 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:53 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5[62852]: 2026-02-25T11:59:53.010+0000 7f168ef88640 -1 osd.5 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:28.579243+0000 front 2026-02-25T11:59:28.579291+0000 (oldest deadline 2026-02-25T11:59:51.479349+0000) 2026-02-25T11:59:53.509 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 11:59:53 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T11:59:53.241+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T11:59:53.510 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:53 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:53.391+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:53.510 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:53 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:53.391+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T11:59:53.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:53 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:53.116+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:53.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:53 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:53.116+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T11:59:53.559 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:53 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:53.330+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:53.859 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:59:53 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:59:53.554+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T11:59:53.860 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:53 trial167 podman[101869]: 2026-02-25 11:59:53.558370271 +0000 UTC m=+5.127778963 container died af9e2da3ad4eaa2b749a425f2e869117c21ad7739aa8129afe4a2c0077608170 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5, CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.41.3, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True) 2026-02-25T11:59:53.860 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:53 trial167 podman[101869]: 2026-02-25 11:59:53.616949402 +0000 UTC m=+5.186358084 container remove af9e2da3ad4eaa2b749a425f2e869117c21ad7739aa8129afe4a2c0077608170 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_REF=wip-vshankar-testing-20260224.100235, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260223, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, io.buildah.version=1.41.3, org.opencontainers.image.authors=Ceph Release Team , CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-25T11:59:53.860 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:53 trial167 bash[101869]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5 2026-02-25T11:59:54.121 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:53 trial167 podman[101970]: 2026-02-25 11:59:53.859117876 +0000 UTC m=+0.026698317 container create 156f52ccba95b009c3fc6ec9f328803343b6617dee0e9117b729d64df9ccf4c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5-deactivate, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20260223, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.41.3, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-25T11:59:54.121 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:53 trial167 podman[101970]: 2026-02-25 11:59:53.911782317 +0000 UTC m=+0.079362758 container init 156f52ccba95b009c3fc6ec9f328803343b6617dee0e9117b729d64df9ccf4c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5-deactivate, io.buildah.version=1.41.3, org.label-schema.build-date=20260223, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, ceph=True, OSD_FLAVOR=default, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.schema-version=1.0) 2026-02-25T11:59:54.121 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:53 trial167 podman[101970]: 2026-02-25 11:59:53.917062478 +0000 UTC m=+0.084642919 container start 156f52ccba95b009c3fc6ec9f328803343b6617dee0e9117b729d64df9ccf4c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5-deactivate, io.buildah.version=1.41.3, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.vendor=CentOS, org.label-schema.build-date=20260223, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.license=GPLv2, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-25T11:59:54.121 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:53 trial167 podman[101970]: 2026-02-25 11:59:53.91795719 +0000 UTC m=+0.085537641 container attach 156f52ccba95b009c3fc6ec9f328803343b6617dee0e9117b729d64df9ccf4c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5-deactivate, CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.41.3, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, OSD_FLAVOR=default, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.build-date=20260223, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T11:59:54.121 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:53 trial167 podman[101970]: 2026-02-25 11:59:53.849901912 +0000 UTC m=+0.017482353 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T11:59:54.122 INFO:journalctl@ceph.osd.5.trial167.stdout:Feb 25 11:59:54 trial167 podman[101970]: 2026-02-25 11:59:54.106567071 +0000 UTC m=+0.274147522 container died 156f52ccba95b009c3fc6ec9f328803343b6617dee0e9117b729d64df9ccf4c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-5-deactivate, OSD_FLAVOR=default, io.buildah.version=1.41.3, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=quay.io/centos/centos:stream9, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T11:59:54.122 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:59:54 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:59:54.061+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T11:59:54.192 DEBUG:teuthology.orchestra.run.trial167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.5.service' 2026-02-25T11:59:54.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T11:59:54.264 INFO:tasks.cephadm.osd.5:Stopped osd.5 2026-02-25T11:59:54.264 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2026-02-25T11:59:54.264 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.6 2026-02-25T11:59:54.456 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:54 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:54.136+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:54.457 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:54 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:54.136+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T11:59:54.457 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:54 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:54.333+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:54.457 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:54 trial028 systemd[1]: Stopping Ceph osd.6 for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T11:59:54.759 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 11:59:54 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T11:59:54.292+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T11:59:54.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:54 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:54.358+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:54.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:54 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:54.358+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T11:59:54.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:54 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:54.455+0000 7f694b483640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-25T11:59:54.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:54 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:54.455+0000 7f694b483640 -1 osd.6 91 *** Got signal Terminated *** 2026-02-25T11:59:54.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:54 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:54.455+0000 7f694b483640 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-25T11:59:54.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:59:54 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:59:54.514+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T11:59:55.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:59:55 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:59:55.053+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T11:59:55.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:55 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:55.132+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:55.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:55 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:55.132+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T11:59:55.560 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:55 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:55.309+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:55.560 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:55 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:55.309+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:29.284735+0000 front 2026-02-25T11:59:29.284822+0000 (oldest deadline 2026-02-25T11:59:54.584805+0000) 2026-02-25T11:59:55.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:55 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:55.388+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:55.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:55 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:55.388+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T11:59:55.760 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 11:59:55 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T11:59:55.304+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T11:59:55.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:59:55 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:59:55.546+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T11:59:56.351 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:56 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:56.095+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:56.352 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:56 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:56.095+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T11:59:56.352 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:56 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:56.095+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T11:59:56.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:59:56 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:59:56.078+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T11:59:56.759 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 11:59:56 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T11:59:56.267+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T11:59:56.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:56 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:56.423+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:56.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:56 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:56.423+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T11:59:56.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:56 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:56.351+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:56.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:56 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:56.351+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:29.284735+0000 front 2026-02-25T11:59:29.284822+0000 (oldest deadline 2026-02-25T11:59:54.584805+0000) 2026-02-25T11:59:56.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:59:56 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:59:56.502+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T11:59:57.344 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:57 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:57.074+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:57.345 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:57 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:57.074+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T11:59:57.345 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:57 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:57.074+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T11:59:57.469 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:59:57 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:59:57.099+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T11:59:57.469 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:59:57 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:59:57.099+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T11:59:57.509 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 11:59:57 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T11:59:57.227+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T11:59:57.510 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:57 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:57.459+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:57.510 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:57 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:57.459+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T11:59:57.510 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:57 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:57.459+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T11:59:57.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:59:57 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:59:57.467+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T11:59:57.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:57 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:57.343+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:57.809 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:57 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:57.343+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:29.284735+0000 front 2026-02-25T11:59:29.284822+0000 (oldest deadline 2026-02-25T11:59:54.584805+0000) 2026-02-25T11:59:58.423 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:59:58 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:59:58.141+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T11:59:58.424 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:59:58 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:59:58.141+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T11:59:58.509 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 11:59:58 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T11:59:58.257+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T11:59:58.510 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:58 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:58.414+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:58.510 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:58 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:58.414+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T11:59:58.510 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:58 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:58.414+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T11:59:58.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:58 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:58.080+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:58.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:58 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:58.080+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T11:59:58.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:58 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:58.080+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T11:59:58.560 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:58 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:58.315+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:58.560 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:58 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:58.315+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:29.284735+0000 front 2026-02-25T11:59:29.284822+0000 (oldest deadline 2026-02-25T11:59:54.584805+0000) 2026-02-25T11:59:58.560 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:58 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:58.315+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:34.585279+0000 front 2026-02-25T11:59:34.585099+0000 (oldest deadline 2026-02-25T11:59:58.085063+0000) 2026-02-25T11:59:58.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:59:58 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:59:58.422+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T11:59:59.453 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:59:59 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:59:59.150+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T11:59:59.454 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 11:59:59 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T11:59:59.150+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T11:59:59.509 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 11:59:59 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T11:59:59.220+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T11:59:59.509 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:59 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:59.459+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T11:59:59.509 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:59 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:59.459+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T11:59:59.509 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 11:59:59 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T11:59:59.459+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T11:59:59.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:59 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:59.128+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T11:59:59.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:59 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:59.128+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T11:59:59.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 11:59:59 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T11:59:59.128+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T11:59:59.559 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:59 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:59.320+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.283704+0000 front 2026-02-25T11:59:19.283478+0000 (oldest deadline 2026-02-25T11:59:44.583498+0000) 2026-02-25T11:59:59.559 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:59 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:59.320+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:29.284735+0000 front 2026-02-25T11:59:29.284822+0000 (oldest deadline 2026-02-25T11:59:54.584805+0000) 2026-02-25T11:59:59.559 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:59 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6[82557]: 2026-02-25T11:59:59.320+0000 7f6947b4b640 -1 osd.6 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:34.585279+0000 front 2026-02-25T11:59:34.585099+0000 (oldest deadline 2026-02-25T11:59:58.085063+0000) 2026-02-25T11:59:59.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 11:59:59 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T11:59:59.452+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T11:59:59.877 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:59 trial028 podman[135646]: 2026-02-25 11:59:59.571233962 +0000 UTC m=+5.139430710 container died 96d2c6086b3b912d6c3e91549c3499ce642c473645906b62be0248c98c018b17 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.schema-version=1.0, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, OSD_FLAVOR=default, org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, io.buildah.version=1.41.3, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.build-date=20260223) 2026-02-25T11:59:59.878 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:59 trial028 podman[135646]: 2026-02-25 11:59:59.629356906 +0000 UTC m=+5.197553654 container remove 96d2c6086b3b912d6c3e91549c3499ce642c473645906b62be0248c98c018b17 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.schema-version=1.0) 2026-02-25T11:59:59.878 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:59 trial028 bash[135646]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6 2026-02-25T12:00:00.136 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:59 trial028 podman[135744]: 2026-02-25 11:59:59.876829253 +0000 UTC m=+0.025795350 container create 46a1bba4ec45d0fa53b3fec698f13ad05c6766b734d9f526b8703c92a43ea03e (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6-deactivate, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.build-date=20260223, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, ceph=True) 2026-02-25T12:00:00.136 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:59 trial028 podman[135744]: 2026-02-25 11:59:59.930561846 +0000 UTC m=+0.079527943 container init 46a1bba4ec45d0fa53b3fec698f13ad05c6766b734d9f526b8703c92a43ea03e (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6-deactivate, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, OSD_FLAVOR=default, org.label-schema.schema-version=1.0, ceph=True, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.build-date=20260223, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa) 2026-02-25T12:00:00.136 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:59 trial028 podman[135744]: 2026-02-25 11:59:59.936145041 +0000 UTC m=+0.085111138 container start 46a1bba4ec45d0fa53b3fec698f13ad05c6766b734d9f526b8703c92a43ea03e (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6-deactivate, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.documentation=https://docs.ceph.com/, io.buildah.version=1.41.3, ceph=True, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-vshankar-testing-20260224.100235, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, OSD_FLAVOR=default, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2026-02-25T12:00:00.136 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:59 trial028 podman[135744]: 2026-02-25 11:59:59.93688257 +0000 UTC m=+0.085848677 container attach 46a1bba4ec45d0fa53b3fec698f13ad05c6766b734d9f526b8703c92a43ea03e (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-6-deactivate, OSD_FLAVOR=default, io.buildah.version=1.41.3, org.label-schema.build-date=20260223, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team , FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_REF=wip-vshankar-testing-20260224.100235, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-25T12:00:00.136 INFO:journalctl@ceph.osd.6.trial028.stdout:Feb 25 11:59:59 trial028 podman[135744]: 2026-02-25 11:59:59.868039836 +0000 UTC m=+0.017005943 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T12:00:00.229 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.6.service' 2026-02-25T12:00:00.299 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:00.299 INFO:tasks.cephadm.osd.6:Stopped osd.6 2026-02-25T12:00:00.299 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2026-02-25T12:00:00.300 DEBUG:teuthology.orchestra.run.trial119:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.7 2026-02-25T12:00:00.469 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:00 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:00.159+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T12:00:00.469 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:00 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:00.159+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T12:00:00.478 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:00 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:00.219+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T12:00:00.478 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:00 trial119 systemd[1]: Stopping Ceph osd.7 for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:00.478 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:00 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:00.412+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:00.478 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:00 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:00.412+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:00.478 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:00 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:00.412+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:00.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:00 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:00.169+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:00.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:00 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:00.169+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:00.560 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:00 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:00.169+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:00.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:00 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:00.467+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:00.759 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:00 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:00.475+0000 7fb268f01640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-25T12:00:00.759 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:00 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:00.475+0000 7fb268f01640 -1 osd.7 91 *** Got signal Terminated *** 2026-02-25T12:00:00.760 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:00 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:00.475+0000 7fb268f01640 -1 osd.7 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-25T12:00:01.481 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:01 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:01.147+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T12:00:01.482 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:01 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:01.147+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T12:00:01.509 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:01 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:01.248+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T12:00:01.509 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:01 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:01.248+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:35.314112+0000 front 2026-02-25T11:59:35.314407+0000 (oldest deadline 2026-02-25T12:00:00.614109+0000) 2026-02-25T12:00:01.509 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:01 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:01.461+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:01.509 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:01 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:01.461+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:01.509 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:01 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:01.461+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:01.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:01 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:01.199+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:01.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:01 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:01.199+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:01.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:01 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:01.199+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:01.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:01 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:01.480+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:01.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:01 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:01.480+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:02.480 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:02 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:02.177+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T12:00:02.576 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:02 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:02.177+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T12:00:02.577 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:02 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:02.177+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:36.639960+0000 front 2026-02-25T11:59:36.640229+0000 (oldest deadline 2026-02-25T12:00:01.939932+0000) 2026-02-25T12:00:02.577 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:02 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:02.238+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:02.577 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:02 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:02.238+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:02.577 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:02 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:02.238+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:02.577 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:02 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:02.238+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:02.577 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:02 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:02.233+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T12:00:02.577 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:02 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:02.233+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:35.314112+0000 front 2026-02-25T11:59:35.314407+0000 (oldest deadline 2026-02-25T12:00:00.614109+0000) 2026-02-25T12:00:02.577 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:02 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:02.449+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:02.577 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:02 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:02.449+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:02.577 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:02 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:02.449+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:02.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:02 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:02.478+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:02.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:02 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:02.478+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:03.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:03 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:03.203+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T12:00:03.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:03 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:03.203+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T12:00:03.500 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:03 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:03.203+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:36.639960+0000 front 2026-02-25T11:59:36.640229+0000 (oldest deadline 2026-02-25T12:00:01.939932+0000) 2026-02-25T12:00:03.509 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:03 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:03.257+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T12:00:03.510 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:03 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:03.257+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:35.314112+0000 front 2026-02-25T11:59:35.314407+0000 (oldest deadline 2026-02-25T12:00:00.614109+0000) 2026-02-25T12:00:03.510 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:03 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:03.476+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:03.510 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:03 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:03.476+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:03.510 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:03 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:03.476+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:03.510 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:03 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:03.476+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:03.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:03 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:03.203+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:03.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:03 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:03.203+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:03.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:03 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:03.203+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:03.560 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:03 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:03.203+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:03.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:03 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:03.502+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:03.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:03 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:03.502+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:03.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:03 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:03.502+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:04.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:04 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:04.177+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T12:00:04.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:04 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:04.177+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T12:00:04.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:04 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:04.177+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:36.639960+0000 front 2026-02-25T11:59:36.640229+0000 (oldest deadline 2026-02-25T12:00:01.939932+0000) 2026-02-25T12:00:04.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:04 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:04.230+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:04.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:04 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:04.230+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:04.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:04 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:04.230+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:04.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:04 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:04.230+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:04.760 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:04 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:04.297+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T12:00:04.760 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:04 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:04.297+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:35.314112+0000 front 2026-02-25T11:59:35.314407+0000 (oldest deadline 2026-02-25T12:00:00.614109+0000) 2026-02-25T12:00:04.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:04 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:04.495+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:04.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:04 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:04.495+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:04.761 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:04 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:04.495+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:04.761 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:04 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:04.495+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:04.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:04 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:04.509+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:04.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:04 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:04.509+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:04.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:04 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:04.509+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:05.484 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:05 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:05.173+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T12:00:05.485 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:05 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:05.173+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T12:00:05.485 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:05 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:05.173+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:36.639960+0000 front 2026-02-25T11:59:36.640229+0000 (oldest deadline 2026-02-25T12:00:01.939932+0000) 2026-02-25T12:00:05.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:05 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:05.259+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:05.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:05 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:05.259+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:05.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:05 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:05.259+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:05.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:05 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:05.259+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:05.601 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:05 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:05.456+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:05.601 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:05 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:05.456+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:05.601 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:05 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:05.456+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:05.601 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:05 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:05.456+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:05.602 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:05 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:05.336+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.512962+0000 front 2026-02-25T11:59:25.513040+0000 (oldest deadline 2026-02-25T11:59:49.613066+0000) 2026-02-25T12:00:05.602 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:05 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:05.336+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:35.314112+0000 front 2026-02-25T11:59:35.314407+0000 (oldest deadline 2026-02-25T12:00:00.614109+0000) 2026-02-25T12:00:05.602 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:05 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7[73497]: 2026-02-25T12:00:05.336+0000 7fb2615c1640 -1 osd.7 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.614511+0000 front 2026-02-25T11:59:40.614372+0000 (oldest deadline 2026-02-25T12:00:05.314376+0000) 2026-02-25T12:00:05.602 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:05 trial119 podman[100144]: 2026-02-25 12:00:05.572729944 +0000 UTC m=+5.118350664 container died d648b0a5eb59084a96d1f3a05c352703ff0b3b5d873be35bb4d9a14476940605 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7, CEPH_REF=wip-vshankar-testing-20260224.100235, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.build-date=20260223, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-25T12:00:05.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:05 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:05.483+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:05.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:05 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:05.483+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:05.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:05 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:05.483+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:05.875 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:05 trial119 podman[100144]: 2026-02-25 12:00:05.630685586 +0000 UTC m=+5.176306307 container remove d648b0a5eb59084a96d1f3a05c352703ff0b3b5d873be35bb4d9a14476940605 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-vshankar-testing-20260224.100235, org.opencontainers.image.documentation=https://docs.ceph.com/, io.buildah.version=1.41.3, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, OSD_FLAVOR=default, ceph=True) 2026-02-25T12:00:05.875 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:05 trial119 bash[100144]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7 2026-02-25T12:00:06.127 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:05 trial119 podman[100240]: 2026-02-25 12:00:05.874198061 +0000 UTC m=+0.026683007 container create b57d41fb39b0510092f0a568a5d1d456e5ae652499833bbc30d3d5dd12f1bf91 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7-deactivate, ceph=True, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, OSD_FLAVOR=default, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , io.buildah.version=1.41.3, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223) 2026-02-25T12:00:06.128 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:05 trial119 podman[100240]: 2026-02-25 12:00:05.923650793 +0000 UTC m=+0.076135739 container init b57d41fb39b0510092f0a568a5d1d456e5ae652499833bbc30d3d5dd12f1bf91 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7-deactivate, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, OSD_FLAVOR=default, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, CEPH_REF=wip-vshankar-testing-20260224.100235, ceph=True, io.buildah.version=1.41.3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.authors=Ceph Release Team ) 2026-02-25T12:00:06.128 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:05 trial119 podman[100240]: 2026-02-25 12:00:05.929113635 +0000 UTC m=+0.081598581 container start b57d41fb39b0510092f0a568a5d1d456e5ae652499833bbc30d3d5dd12f1bf91 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260223, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_REF=wip-vshankar-testing-20260224.100235, OSD_FLAVOR=default, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, ceph=True, org.label-schema.schema-version=1.0) 2026-02-25T12:00:06.128 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:05 trial119 podman[100240]: 2026-02-25 12:00:05.929922077 +0000 UTC m=+0.082407023 container attach b57d41fb39b0510092f0a568a5d1d456e5ae652499833bbc30d3d5dd12f1bf91 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-7-deactivate, org.label-schema.build-date=20260223, OSD_FLAVOR=default, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-vshankar-testing-20260224.100235, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2026-02-25T12:00:06.128 INFO:journalctl@ceph.osd.7.trial119.stdout:Feb 25 12:00:05 trial119 podman[100240]: 2026-02-25 12:00:05.865143144 +0000 UTC m=+0.017628110 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T12:00:06.245 DEBUG:teuthology.orchestra.run.trial119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.7.service' 2026-02-25T12:00:06.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:06.314 INFO:tasks.cephadm.osd.7:Stopped osd.7 2026-02-25T12:00:06.314 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2026-02-25T12:00:06.314 DEBUG:teuthology.orchestra.run.trial167:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.8 2026-02-25T12:00:06.484 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:06 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:06.174+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T12:00:06.484 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:06 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:06.174+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T12:00:06.484 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:06 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:06.174+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:36.639960+0000 front 2026-02-25T11:59:36.640229+0000 (oldest deadline 2026-02-25T12:00:01.939932+0000) 2026-02-25T12:00:06.484 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:06 trial167 systemd[1]: Stopping Ceph osd.8 for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:06.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:06 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:06.243+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:06.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:06 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:06.243+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:06.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:06 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:06.243+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:06.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:06 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:06.243+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:06.749 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:06 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:06.491+0000 7f22ef4ba640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-25T12:00:06.749 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:06 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:06.491+0000 7f22ef4ba640 -1 osd.8 91 *** Got signal Terminated *** 2026-02-25T12:00:06.749 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:06 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:06.491+0000 7f22ef4ba640 -1 osd.8 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-25T12:00:06.750 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:06 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:06.481+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:06.750 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:06 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:06.481+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:06.750 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:06 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:06.481+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:06.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:06 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:06.437+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:06.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:06 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:06.437+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:06.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:06 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:06.437+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:06.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:06 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:06.437+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:07.493 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:07 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:07.189+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T12:00:07.494 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:07 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:07.189+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T12:00:07.494 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:07 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:07.189+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:36.639960+0000 front 2026-02-25T11:59:36.640229+0000 (oldest deadline 2026-02-25T12:00:01.939932+0000) 2026-02-25T12:00:07.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:07.274+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:07.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:07.274+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:07.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:07.274+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:07.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:07 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:07.274+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:07.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:07 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:07.492+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:07.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:07 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:07.492+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:07.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:07 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:07.492+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:07.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:07 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:07.462+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:07.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:07 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:07.462+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:07.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:07 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:07.462+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:07.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:07 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:07.462+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:08.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:08 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:08.170+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T12:00:08.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:08 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:08.170+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T12:00:08.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:08 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:08.170+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:36.639960+0000 front 2026-02-25T11:59:36.640229+0000 (oldest deadline 2026-02-25T12:00:01.939932+0000) 2026-02-25T12:00:08.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:08 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:08.300+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:08.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:08 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:08.300+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:08.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:08 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:08.300+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:08.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:08 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:08.300+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:08.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:08 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:08.505+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:08.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:08 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:08.505+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:08.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:08 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:08.505+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:08.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:08 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:08.505+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:08.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:08 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:08.531+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:08.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:08 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:08.531+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:08.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:08 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:08.531+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:09.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:09 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:09.185+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T12:00:09.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:09 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:09.185+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T12:00:09.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:09 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:09.185+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:36.639960+0000 front 2026-02-25T11:59:36.640229+0000 (oldest deadline 2026-02-25T12:00:01.939932+0000) 2026-02-25T12:00:09.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:09 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:09.271+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:09.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:09 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:09.271+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:09.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:09 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:09.271+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:09.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:09 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:09.271+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:09.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:09 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:09.481+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:09.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:09 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:09.481+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:09.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:09 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:09.481+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:09.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:09 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:09.481+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:09.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:09 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:09.481+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:09.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:09 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:09.553+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:09.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:09 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:09.553+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:09.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:09 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:09.553+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:09.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:09 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:09.553+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:10.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:10 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:10.157+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T12:00:10.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:10 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:10.157+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T12:00:10.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:10 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:10.157+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:36.639960+0000 front 2026-02-25T11:59:36.640229+0000 (oldest deadline 2026-02-25T12:00:01.939932+0000) 2026-02-25T12:00:10.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:10 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:10.245+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:10.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:10 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:10.245+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:10.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:10 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:10.245+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:10.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:10 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:10.245+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:10.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:10 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:10.454+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:10.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:10 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:10.454+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:10.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:10 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:10.454+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:10.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:10 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:10.454+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:10.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:10 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:10.454+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:10.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:10 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:10.592+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:10.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:10 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:10.592+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:10.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:10 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:10.592+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:10.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:10 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:10.592+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:11.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:11.202+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:27.839450+0000 front 2026-02-25T11:59:27.839706+0000 (oldest deadline 2026-02-25T11:59:51.339375+0000) 2026-02-25T12:00:11.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:11.202+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.339589+0000 front 2026-02-25T11:59:31.339972+0000 (oldest deadline 2026-02-25T11:59:56.639651+0000) 2026-02-25T12:00:11.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:11.202+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:36.639960+0000 front 2026-02-25T11:59:36.640229+0000 (oldest deadline 2026-02-25T12:00:01.939932+0000) 2026-02-25T12:00:11.499 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8[75168]: 2026-02-25T12:00:11.202+0000 7f22ebb82640 -1 osd.8 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:45.940637+0000 front 2026-02-25T11:59:45.940673+0000 (oldest deadline 2026-02-25T12:00:10.640641+0000) 2026-02-25T12:00:11.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:11 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:11.292+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:11.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:11 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:11.292+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:11.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:11 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:11.292+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:11.560 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:11 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:11.292+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:11.560 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:11 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:11.292+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:45.630466+0000 front 2026-02-25T11:59:45.630686+0000 (oldest deadline 2026-02-25T12:00:10.330509+0000) 2026-02-25T12:00:11.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:11 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:11.461+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:11.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:11 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:11.461+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:11.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:11 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:11.461+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:11.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:11 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:11.461+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:11.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:11 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:11.461+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:11.899 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:11 trial167 podman[102085]: 2026-02-25 12:00:11.588147282 +0000 UTC m=+5.120575734 container died c41cc6cd3d15d16969a3d66d13da588b372243d46c121e080aae82ab5e2640a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260223, FROM_IMAGE=quay.io/centos/centos:stream9, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-vshankar-testing-20260224.100235, io.buildah.version=1.41.3, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, OSD_FLAVOR=default, org.label-schema.vendor=CentOS) 2026-02-25T12:00:11.900 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:11 trial167 podman[102085]: 2026-02-25 12:00:11.646325258 +0000 UTC m=+5.178753720 container remove c41cc6cd3d15d16969a3d66d13da588b372243d46c121e080aae82ab5e2640a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8, org.label-schema.build-date=20260223, ceph=True, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, CEPH_REF=wip-vshankar-testing-20260224.100235, io.buildah.version=1.41.3, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default) 2026-02-25T12:00:11.900 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:11 trial167 bash[102085]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8 2026-02-25T12:00:11.900 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:11.618+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:11.900 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:11.618+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:11.900 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:11.618+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:11.900 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:11 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:11.618+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:12.152 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:11 trial167 podman[102184]: 2026-02-25 12:00:11.89881588 +0000 UTC m=+0.026813311 container create d2f1b6f8cfcd9d6ff9b564dbac60b1e6b16792b0ac3ecc1cb185294b8675cc34 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.schema-version=1.0, CEPH_REF=wip-vshankar-testing-20260224.100235, FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.build-date=20260223, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.41.3, ceph=True) 2026-02-25T12:00:12.152 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:11 trial167 podman[102184]: 2026-02-25 12:00:11.949282838 +0000 UTC m=+0.077280279 container init d2f1b6f8cfcd9d6ff9b564dbac60b1e6b16792b0ac3ecc1cb185294b8675cc34 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8-deactivate, org.label-schema.build-date=20260223, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.schema-version=1.0, io.buildah.version=1.41.3, ceph=True, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.license=GPLv2, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-vshankar-testing-20260224.100235) 2026-02-25T12:00:12.152 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:11 trial167 podman[102184]: 2026-02-25 12:00:11.954985434 +0000 UTC m=+0.082982875 container start d2f1b6f8cfcd9d6ff9b564dbac60b1e6b16792b0ac3ecc1cb185294b8675cc34 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, OSD_FLAVOR=default, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.build-date=20260223, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, ceph=True, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-25T12:00:12.152 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:11 trial167 podman[102184]: 2026-02-25 12:00:11.955891858 +0000 UTC m=+0.083889299 container attach d2f1b6f8cfcd9d6ff9b564dbac60b1e6b16792b0ac3ecc1cb185294b8675cc34 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-8-deactivate, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.name=CentOS Stream 9 Base Image, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=quay.io/centos/centos:stream9, ceph=True, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.license=GPLv2, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.schema-version=1.0, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.build-date=20260223, OSD_FLAVOR=default) 2026-02-25T12:00:12.153 INFO:journalctl@ceph.osd.8.trial167.stdout:Feb 25 12:00:11 trial167 podman[102184]: 2026-02-25 12:00:11.889691056 +0000 UTC m=+0.017688497 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T12:00:12.241 DEBUG:teuthology.orchestra.run.trial167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.8.service' 2026-02-25T12:00:12.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:12.313 INFO:tasks.cephadm.osd.8:Stopped osd.8 2026-02-25T12:00:12.313 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2026-02-25T12:00:12.313 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.9 2026-02-25T12:00:12.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:12 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:12.476+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:12.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:12 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:12.476+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:12.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:12 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:12.476+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:12.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:12 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:12.476+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:12.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:12 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:12.476+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:12.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:12 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:12.330+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:12.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:12 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:12.330+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:12.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:12 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:12.330+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:12.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:12 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:12.330+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:12.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:12 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:12.330+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:45.630466+0000 front 2026-02-25T11:59:45.630686+0000 (oldest deadline 2026-02-25T12:00:10.330509+0000) 2026-02-25T12:00:12.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:12 trial028 systemd[1]: Stopping Ceph osd.9 for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:12.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:12 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:12.493+0000 7f00c26c5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-25T12:00:12.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:12 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:12.493+0000 7f00c26c5640 -1 osd.9 91 *** Got signal Terminated *** 2026-02-25T12:00:12.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:12 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:12.493+0000 7f00c26c5640 -1 osd.9 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-25T12:00:12.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:12 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:12.635+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:12.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:12 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:12.635+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:13.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:12 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:12.635+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:13.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:12 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:12.635+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:13.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:13 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:13.503+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:13.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:13 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:13.503+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:13.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:13 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:13.503+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:13.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:13 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:13.503+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:13.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:13 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:13.503+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:13.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:13 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:13.370+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:13.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:13 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:13.370+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:13.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:13 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:13.370+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:13.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:13 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:13.370+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:13.810 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:13 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:13.370+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:45.630466+0000 front 2026-02-25T11:59:45.630686+0000 (oldest deadline 2026-02-25T12:00:10.330509+0000) 2026-02-25T12:00:13.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:13 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:13.608+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:13.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:13 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:13.608+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:13.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:13 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:13.608+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:14.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:13 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:13.608+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:14.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:14 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:14.493+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:14.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:14 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:14.493+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:14.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:14 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:14.493+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:14.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:14 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:14.493+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:14.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:14 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:14.493+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:14.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:14 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:14.493+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:49.110303+0000 front 2026-02-25T11:59:49.110450+0000 (oldest deadline 2026-02-25T12:00:13.810212+0000) 2026-02-25T12:00:14.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:14 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:14.386+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:14.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:14 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:14.386+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:14.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:14 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:14.386+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:14.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:14 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:14.386+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:14.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:14 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:14.386+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:45.630466+0000 front 2026-02-25T11:59:45.630686+0000 (oldest deadline 2026-02-25T12:00:10.330509+0000) 2026-02-25T12:00:14.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:14 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:14.623+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:14.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:14 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:14.623+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:14.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:14 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:14.623+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:15.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:14 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:14.623+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:15.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:15 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:15.373+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:15.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:15 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:15.373+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:15.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:15 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:15.373+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:15.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:15 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:15.373+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:15.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:15 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:15.373+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:45.630466+0000 front 2026-02-25T11:59:45.630686+0000 (oldest deadline 2026-02-25T12:00:10.330509+0000) 2026-02-25T12:00:15.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:15 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:15.621+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:15.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:15 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:15.621+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:15.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:15 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:15.621+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:16.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:15 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:15.621+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:16.009 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:15 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:15.531+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:16.009 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:15 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:15.531+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:16.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:15 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:15.531+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:16.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:15 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:15.531+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:16.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:15 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:15.531+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:16.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:15 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:15.531+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:49.110303+0000 front 2026-02-25T11:59:49.110450+0000 (oldest deadline 2026-02-25T12:00:13.810212+0000) 2026-02-25T12:00:16.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:16 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:16.336+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:16.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:16 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:16.336+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:16.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:16 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:16.336+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:16.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:16 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:16.336+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:16.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:16 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:16.336+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:45.630466+0000 front 2026-02-25T11:59:45.630686+0000 (oldest deadline 2026-02-25T12:00:10.330509+0000) 2026-02-25T12:00:16.809 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:16 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:16.336+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:53.131361+0000 front 2026-02-25T11:59:53.131214+0000 (oldest deadline 2026-02-25T12:00:16.031218+0000) 2026-02-25T12:00:16.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:16 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:16.612+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:16.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:16 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:16.612+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:16.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:16 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:16.612+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:16.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:16 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:16.612+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:17.009 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:16 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:16.577+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:17.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:16 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:16.577+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:17.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:16 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:16.577+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:17.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:16 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:16.577+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:17.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:16 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:16.577+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:17.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:16 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:16.577+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:49.110303+0000 front 2026-02-25T11:59:49.110450+0000 (oldest deadline 2026-02-25T12:00:13.810212+0000) 2026-02-25T12:00:17.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:17.298+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:19.328676+0000 front 2026-02-25T11:59:19.328626+0000 (oldest deadline 2026-02-25T11:59:45.228722+0000) 2026-02-25T12:00:17.559 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:17.298+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.229315+0000 front 2026-02-25T11:59:25.229050+0000 (oldest deadline 2026-02-25T11:59:49.928969+0000) 2026-02-25T12:00:17.560 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:17.298+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.629495+0000 front 2026-02-25T11:59:31.629498+0000 (oldest deadline 2026-02-25T11:59:55.729470+0000) 2026-02-25T12:00:17.560 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:17.298+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:35.729603+0000 front 2026-02-25T11:59:35.729651+0000 (oldest deadline 2026-02-25T12:00:01.629702+0000) 2026-02-25T12:00:17.560 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:17.298+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:45.630466+0000 front 2026-02-25T11:59:45.630686+0000 (oldest deadline 2026-02-25T12:00:10.330509+0000) 2026-02-25T12:00:17.560 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9[94752]: 2026-02-25T12:00:17.298+0000 7f00bed8d640 -1 osd.9 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:53.131361+0000 front 2026-02-25T11:59:53.131214+0000 (oldest deadline 2026-02-25T12:00:16.031218+0000) 2026-02-25T12:00:17.906 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 podman[135861]: 2026-02-25 12:00:17.600662668 +0000 UTC m=+5.129051119 container died b1e689039db2fb7c4273fd2460c04d54ba92f0c7314fa20e5382eb75b106accd (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, ceph=True, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_REF=wip-vshankar-testing-20260224.100235) 2026-02-25T12:00:17.907 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 podman[135861]: 2026-02-25 12:00:17.657719323 +0000 UTC m=+5.186107774 container remove b1e689039db2fb7c4273fd2460c04d54ba92f0c7314fa20e5382eb75b106accd (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, CEPH_REF=wip-vshankar-testing-20260224.100235, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.opencontainers.image.documentation=https://docs.ceph.com/, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.41.3, ceph=True, org.label-schema.vendor=CentOS) 2026-02-25T12:00:17.907 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 bash[135861]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9 2026-02-25T12:00:17.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:17 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:17.579+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:17.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:17 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:17.579+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:17.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:17 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:17.579+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:18.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:17 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:17.579+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:18.009 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:17 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:17.548+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:18.009 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:17 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:17.548+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:18.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:17 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:17.548+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:18.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:17 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:17.548+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:18.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:17 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:17.548+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:18.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:17 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:17.548+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:49.110303+0000 front 2026-02-25T11:59:49.110450+0000 (oldest deadline 2026-02-25T12:00:13.810212+0000) 2026-02-25T12:00:18.161 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 podman[135961]: 2026-02-25 12:00:17.906162912 +0000 UTC m=+0.026703385 container create e43e3a423cc4fc850f76e88b55b4634b1530c0ea7502f55d8d30238113229453 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.build-date=20260223, CEPH_REF=wip-vshankar-testing-20260224.100235, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, ceph=True, org.label-schema.schema-version=1.0) 2026-02-25T12:00:18.161 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 podman[135961]: 2026-02-25 12:00:17.95083801 +0000 UTC m=+0.071378483 container init e43e3a423cc4fc850f76e88b55b4634b1530c0ea7502f55d8d30238113229453 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9-deactivate, OSD_FLAVOR=default, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=wip-vshankar-testing-20260224.100235, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.build-date=20260223, ceph=True, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, io.buildah.version=1.41.3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/) 2026-02-25T12:00:18.161 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 podman[135961]: 2026-02-25 12:00:17.95738265 +0000 UTC m=+0.077923123 container start e43e3a423cc4fc850f76e88b55b4634b1530c0ea7502f55d8d30238113229453 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9-deactivate, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.documentation=https://docs.ceph.com/, ceph=True, CEPH_REF=wip-vshankar-testing-20260224.100235, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20260223, OSD_FLAVOR=default, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git) 2026-02-25T12:00:18.161 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 podman[135961]: 2026-02-25 12:00:17.958435355 +0000 UTC m=+0.078975828 container attach e43e3a423cc4fc850f76e88b55b4634b1530c0ea7502f55d8d30238113229453 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9-deactivate, CEPH_REF=wip-vshankar-testing-20260224.100235, io.buildah.version=1.41.3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.build-date=20260223, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, OSD_FLAVOR=default, ceph=True) 2026-02-25T12:00:18.162 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:17 trial028 podman[135961]: 2026-02-25 12:00:17.896993549 +0000 UTC m=+0.017534022 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T12:00:18.162 INFO:journalctl@ceph.osd.9.trial028.stdout:Feb 25 12:00:18 trial028 podman[135961]: 2026-02-25 12:00:18.146685426 +0000 UTC m=+0.267225899 container died e43e3a423cc4fc850f76e88b55b4634b1530c0ea7502f55d8d30238113229453 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-9-deactivate, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223, OSD_FLAVOR=default, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, io.buildah.version=1.41.3) 2026-02-25T12:00:18.232 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.9.service' 2026-02-25T12:00:18.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:18.303 INFO:tasks.cephadm.osd.9:Stopped osd.9 2026-02-25T12:00:18.304 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2026-02-25T12:00:18.304 DEBUG:teuthology.orchestra.run.trial119:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.10 2026-02-25T12:00:18.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:18 trial119 systemd[1]: Stopping Ceph osd.10 for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:18.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:18 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:18.491+0000 7f1c544c0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-25T12:00:18.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:18 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:18.491+0000 7f1c544c0640 -1 osd.10 91 *** Got signal Terminated *** 2026-02-25T12:00:18.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:18 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:18.491+0000 7f1c544c0640 -1 osd.10 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-25T12:00:18.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:18 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:18.597+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:18.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:18 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:18.597+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:18.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:18 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:18.597+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:18.761 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:18 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:18.597+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:18.761 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:18 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:18.597+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:18.761 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:18 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:18.597+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:49.110303+0000 front 2026-02-25T11:59:49.110450+0000 (oldest deadline 2026-02-25T12:00:13.810212+0000) 2026-02-25T12:00:18.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:18 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:18.601+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:18.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:18 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:18.601+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:18.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:18 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:18.601+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:18.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:18 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:18.601+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:19.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:19 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:19.588+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:20.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:19 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:19.588+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:20.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:19 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:19.588+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:20.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:19 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:19.588+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:20.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:19 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:19.588+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:51.763599+0000 front 2026-02-25T11:59:51.763625+0000 (oldest deadline 2026-02-25T12:00:18.763840+0000) 2026-02-25T12:00:20.009 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:19 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:19.575+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:20.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:19 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:19.575+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:20.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:19 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:19.575+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:20.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:19 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:19.575+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:20.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:19 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:19.575+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:20.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:19 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:19.575+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:49.110303+0000 front 2026-02-25T11:59:49.110450+0000 (oldest deadline 2026-02-25T12:00:13.810212+0000) 2026-02-25T12:00:20.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:20 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:20.620+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:20.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:20 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:20.620+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:21.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:20 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:20.620+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:21.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:20 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:20.620+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:21.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:20 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:20.620+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:51.763599+0000 front 2026-02-25T11:59:51.763625+0000 (oldest deadline 2026-02-25T12:00:18.763840+0000) 2026-02-25T12:00:21.009 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:20 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:20.526+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:21.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:20 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:20.526+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:21.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:20 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:20.526+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:21.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:20 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:20.526+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:21.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:20 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:20.526+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:21.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:20 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:20.526+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:49.110303+0000 front 2026-02-25T11:59:49.110450+0000 (oldest deadline 2026-02-25T12:00:13.810212+0000) 2026-02-25T12:00:21.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:21 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:21.478+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:21.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:21 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:21.478+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:21.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:21 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:21.478+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:21.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:21 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:21.478+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:21.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:21 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:21.478+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:21.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:21 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:21.478+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:49.110303+0000 front 2026-02-25T11:59:49.110450+0000 (oldest deadline 2026-02-25T12:00:13.810212+0000) 2026-02-25T12:00:21.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:21 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:21.478+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6822 osd.6 since back 2026-02-25T11:59:58.510704+0000 front 2026-02-25T11:59:58.510731+0000 (oldest deadline 2026-02-25T12:00:20.810692+0000) 2026-02-25T12:00:21.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:21 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:21.581+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:21.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:21 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:21.581+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:21.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:21 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:21.581+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:22.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:21 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:21.581+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:22.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:21 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:21.581+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:51.763599+0000 front 2026-02-25T11:59:51.763625+0000 (oldest deadline 2026-02-25T12:00:18.763840+0000) 2026-02-25T12:00:22.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:22 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:22.611+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:22.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:22 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:22.611+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:22.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:22 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:22.611+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:22.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:22 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:22.611+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:22.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:22 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:22.611+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:51.763599+0000 front 2026-02-25T11:59:51.763625+0000 (oldest deadline 2026-02-25T12:00:18.763840+0000) 2026-02-25T12:00:23.009 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:22 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:22.514+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:23.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:22 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:22.514+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:23.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:22 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:22.514+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:23.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:22 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:22.514+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:23.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:22 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:22.514+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:23.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:22 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:22.514+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:49.110303+0000 front 2026-02-25T11:59:49.110450+0000 (oldest deadline 2026-02-25T12:00:13.810212+0000) 2026-02-25T12:00:23.010 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:22 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:22.514+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6822 osd.6 since back 2026-02-25T11:59:58.510704+0000 front 2026-02-25T11:59:58.510731+0000 (oldest deadline 2026-02-25T12:00:20.810692+0000) 2026-02-25T12:00:23.759 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:23.481+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:21.806984+0000 front 2026-02-25T11:59:21.807100+0000 (oldest deadline 2026-02-25T11:59:45.907094+0000) 2026-02-25T12:00:23.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:23.481+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6804 osd.1 since back 2026-02-25T11:59:25.907513+0000 front 2026-02-25T11:59:25.907568+0000 (oldest deadline 2026-02-25T11:59:51.107932+0000) 2026-02-25T12:00:23.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:23.481+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.608844+0000 front 2026-02-25T11:59:31.609059+0000 (oldest deadline 2026-02-25T11:59:56.908867+0000) 2026-02-25T12:00:23.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:23.481+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:38.009357+0000 front 2026-02-25T11:59:38.009526+0000 (oldest deadline 2026-02-25T12:00:03.309360+0000) 2026-02-25T12:00:23.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:23.481+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:46.209871+0000 front 2026-02-25T11:59:46.210114+0000 (oldest deadline 2026-02-25T12:00:09.109925+0000) 2026-02-25T12:00:23.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:23.481+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:49.110303+0000 front 2026-02-25T11:59:49.110450+0000 (oldest deadline 2026-02-25T12:00:13.810212+0000) 2026-02-25T12:00:23.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10[85810]: 2026-02-25T12:00:23.481+0000 7f1c4cb80640 -1 osd.10 91 heartbeat_check: no reply from 10.20.193.28:6822 osd.6 since back 2026-02-25T11:59:58.510704+0000 front 2026-02-25T11:59:58.510731+0000 (oldest deadline 2026-02-25T12:00:20.810692+0000) 2026-02-25T12:00:23.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 podman[100355]: 2026-02-25 12:00:23.597405151 +0000 UTC m=+5.129458178 container died 4b6ef9eae0f39474b5e0d19ee9795dabea7bb61d6cbfb35216fa7d4b207835cd (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.41.3, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.build-date=20260223, org.label-schema.name=CentOS Stream 9 Base Image, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , OSD_FLAVOR=default, org.label-schema.vendor=CentOS, org.opencontainers.image.documentation=https://docs.ceph.com/) 2026-02-25T12:00:23.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 podman[100355]: 2026-02-25 12:00:23.657219789 +0000 UTC m=+5.189272806 container remove 4b6ef9eae0f39474b5e0d19ee9795dabea7bb61d6cbfb35216fa7d4b207835cd (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, CEPH_REF=wip-vshankar-testing-20260224.100235, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, FROM_IMAGE=quay.io/centos/centos:stream9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.41.3, org.label-schema.vendor=CentOS) 2026-02-25T12:00:23.760 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 bash[100355]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10 2026-02-25T12:00:23.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:23 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:23.617+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:23.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:23 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:23.617+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:23.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:23 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:23.617+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:24.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:23 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:23.617+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:24.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:23 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:23.617+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:51.763599+0000 front 2026-02-25T11:59:51.763625+0000 (oldest deadline 2026-02-25T12:00:18.763840+0000) 2026-02-25T12:00:24.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:23 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:23.617+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6822 osd.6 since back 2026-02-25T11:59:58.764289+0000 front 2026-02-25T11:59:58.764137+0000 (oldest deadline 2026-02-25T12:00:22.864089+0000) 2026-02-25T12:00:24.143 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 podman[100459]: 2026-02-25 12:00:23.901351684 +0000 UTC m=+0.028072059 container create 91a078cd0e1653e782ee0e44d396f5103b3c433e3569eb64644a799459ff383e (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10-deactivate, org.label-schema.vendor=CentOS, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-vshankar-testing-20260224.100235, org.opencontainers.image.documentation=https://docs.ceph.com/, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, io.buildah.version=1.41.3, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20260223) 2026-02-25T12:00:24.143 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 podman[100459]: 2026-02-25 12:00:23.952513662 +0000 UTC m=+0.079234037 container init 91a078cd0e1653e782ee0e44d396f5103b3c433e3569eb64644a799459ff383e (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10-deactivate, io.buildah.version=1.41.3, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, ceph=True, org.label-schema.schema-version=1.0, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa) 2026-02-25T12:00:24.143 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 podman[100459]: 2026-02-25 12:00:23.958018191 +0000 UTC m=+0.084738567 container start 91a078cd0e1653e782ee0e44d396f5103b3c433e3569eb64644a799459ff383e (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, ceph=True, org.label-schema.vendor=CentOS, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.schema-version=1.0, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, io.buildah.version=1.41.3, org.label-schema.build-date=20260223, org.label-schema.license=GPLv2, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_REF=wip-vshankar-testing-20260224.100235, OSD_FLAVOR=default) 2026-02-25T12:00:24.143 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 podman[100459]: 2026-02-25 12:00:23.958887397 +0000 UTC m=+0.085607772 container attach 91a078cd0e1653e782ee0e44d396f5103b3c433e3569eb64644a799459ff383e (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-10-deactivate, io.buildah.version=1.41.3, org.label-schema.build-date=20260223, OSD_FLAVOR=default, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_REF=wip-vshankar-testing-20260224.100235, org.opencontainers.image.documentation=https://docs.ceph.com/, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.vendor=CentOS, org.opencontainers.image.authors=Ceph Release Team , CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, ceph=True, org.label-schema.schema-version=1.0) 2026-02-25T12:00:24.143 INFO:journalctl@ceph.osd.10.trial119.stdout:Feb 25 12:00:23 trial119 podman[100459]: 2026-02-25 12:00:23.891928002 +0000 UTC m=+0.018648387 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T12:00:24.234 DEBUG:teuthology.orchestra.run.trial119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.10.service' 2026-02-25T12:00:24.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:24.303 INFO:tasks.cephadm.osd.10:Stopped osd.10 2026-02-25T12:00:24.303 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2026-02-25T12:00:24.303 DEBUG:teuthology.orchestra.run.trial167:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.11 2026-02-25T12:00:24.749 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:24 trial167 systemd[1]: Stopping Ceph osd.11 for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:24.750 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:24 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:24.483+0000 7f164671b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --osd-objectstore=bluestore (PID: 1) UID: 0 2026-02-25T12:00:24.750 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:24 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:24.483+0000 7f164671b640 -1 osd.11 91 *** Got signal Terminated *** 2026-02-25T12:00:24.750 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:24 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:24.483+0000 7f164671b640 -1 osd.11 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2026-02-25T12:00:24.750 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:24 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:24.572+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:24.750 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:24 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:24.572+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:24.750 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:24 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:24.572+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:24.750 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:24 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:24.572+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:24.750 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:24 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:24.572+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:51.763599+0000 front 2026-02-25T11:59:51.763625+0000 (oldest deadline 2026-02-25T12:00:18.763840+0000) 2026-02-25T12:00:24.750 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:24 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:24.572+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6822 osd.6 since back 2026-02-25T11:59:58.764289+0000 front 2026-02-25T11:59:58.764137+0000 (oldest deadline 2026-02-25T12:00:22.864089+0000) 2026-02-25T12:00:25.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:25 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:25.562+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:25.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:25 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:25.562+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:26.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:25 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:25.562+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:26.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:25 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:25.562+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:26.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:25 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:25.562+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:51.763599+0000 front 2026-02-25T11:59:51.763625+0000 (oldest deadline 2026-02-25T12:00:18.763840+0000) 2026-02-25T12:00:26.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:25 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:25.562+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6822 osd.6 since back 2026-02-25T11:59:58.764289+0000 front 2026-02-25T11:59:58.764137+0000 (oldest deadline 2026-02-25T12:00:22.864089+0000) 2026-02-25T12:00:26.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:26 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:26.595+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:27.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:26 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:26.595+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:27.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:26 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:26.595+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:27.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:26 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:26.595+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:27.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:26 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:26.595+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:51.763599+0000 front 2026-02-25T11:59:51.763625+0000 (oldest deadline 2026-02-25T12:00:18.763840+0000) 2026-02-25T12:00:27.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:26 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:26.595+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6822 osd.6 since back 2026-02-25T11:59:58.764289+0000 front 2026-02-25T11:59:58.764137+0000 (oldest deadline 2026-02-25T12:00:22.864089+0000) 2026-02-25T12:00:27.000 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:26 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:26.595+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6820 osd.7 since back 2026-02-25T12:00:02.864405+0000 front 2026-02-25T12:00:02.864362+0000 (oldest deadline 2026-02-25T12:00:26.364305+0000) 2026-02-25T12:00:27.900 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:27 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:27.550+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:27.900 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:27 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:27.550+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:27.900 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:27 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:27.550+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:27.900 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:27 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:27.550+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:27.900 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:27 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:27.550+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:51.763599+0000 front 2026-02-25T11:59:51.763625+0000 (oldest deadline 2026-02-25T12:00:18.763840+0000) 2026-02-25T12:00:27.900 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:27 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:27.550+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6822 osd.6 since back 2026-02-25T11:59:58.764289+0000 front 2026-02-25T11:59:58.764137+0000 (oldest deadline 2026-02-25T12:00:22.864089+0000) 2026-02-25T12:00:27.900 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:27 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:27.550+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6820 osd.7 since back 2026-02-25T12:00:02.864405+0000 front 2026-02-25T12:00:02.864362+0000 (oldest deadline 2026-02-25T12:00:26.364305+0000) 2026-02-25T12:00:28.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:28 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:28.502+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6806 osd.0 since back 2026-02-25T11:59:23.361450+0000 front 2026-02-25T11:59:23.361232+0000 (oldest deadline 2026-02-25T11:59:48.661109+0000) 2026-02-25T12:00:28.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:28 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:28.502+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6804 osd.2 since back 2026-02-25T11:59:31.362434+0000 front 2026-02-25T11:59:31.362358+0000 (oldest deadline 2026-02-25T12:00:00.762415+0000) 2026-02-25T12:00:28.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:28 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:28.502+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6814 osd.3 since back 2026-02-25T11:59:40.762669+0000 front 2026-02-25T11:59:40.762903+0000 (oldest deadline 2026-02-25T12:00:03.062684+0000) 2026-02-25T12:00:28.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:28 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:28.502+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6812 osd.4 since back 2026-02-25T11:59:47.163312+0000 front 2026-02-25T11:59:47.163180+0000 (oldest deadline 2026-02-25T12:00:09.463184+0000) 2026-02-25T12:00:28.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:28 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:28.502+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.167:6812 osd.5 since back 2026-02-25T11:59:51.763599+0000 front 2026-02-25T11:59:51.763625+0000 (oldest deadline 2026-02-25T12:00:18.763840+0000) 2026-02-25T12:00:28.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:28 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:28.502+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.28:6822 osd.6 since back 2026-02-25T11:59:58.764289+0000 front 2026-02-25T11:59:58.764137+0000 (oldest deadline 2026-02-25T12:00:22.864089+0000) 2026-02-25T12:00:28.999 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:28 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11[88234]: 2026-02-25T12:00:28.502+0000 7f163eddb640 -1 osd.11 91 heartbeat_check: no reply from 10.20.193.119:6820 osd.7 since back 2026-02-25T12:00:02.864405+0000 front 2026-02-25T12:00:02.864362+0000 (oldest deadline 2026-02-25T12:00:26.364305+0000) 2026-02-25T12:00:29.879 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:29 trial167 podman[102302]: 2026-02-25 12:00:29.57863166 +0000 UTC m=+5.119089435 container died 1549eba9d4cbf124b9125e7b0d49653d06e86de0709cc2872f1000c56bad297c (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11, FROM_IMAGE=quay.io/centos/centos:stream9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.vendor=CentOS, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.build-date=20260223, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, org.label-schema.schema-version=1.0, org.opencontainers.image.authors=Ceph Release Team , CEPH_REF=wip-vshankar-testing-20260224.100235, OSD_FLAVOR=default, io.buildah.version=1.41.3, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, ceph=True) 2026-02-25T12:00:29.879 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:29 trial167 podman[102302]: 2026-02-25 12:00:29.636447132 +0000 UTC m=+5.176904907 container remove 1549eba9d4cbf124b9125e7b0d49653d06e86de0709cc2872f1000c56bad297c (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, OSD_FLAVOR=default, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.schema-version=1.0, org.label-schema.build-date=20260223, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, ceph=True, FROM_IMAGE=quay.io/centos/centos:stream9, io.buildah.version=1.41.3) 2026-02-25T12:00:29.879 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:29 trial167 bash[102302]: ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11 2026-02-25T12:00:30.132 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:29 trial167 podman[102404]: 2026-02-25 12:00:29.877970036 +0000 UTC m=+0.026232531 container create a2729305c773eadeb54d60671f7f838bd76639bad7f3baadf213622ccb6384b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11-deactivate, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.build-date=20260223, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, org.opencontainers.image.documentation=https://docs.ceph.com/, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, CEPH_REF=wip-vshankar-testing-20260224.100235, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2026-02-25T12:00:30.132 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:29 trial167 podman[102404]: 2026-02-25 12:00:29.930827727 +0000 UTC m=+0.079090242 container init a2729305c773eadeb54d60671f7f838bd76639bad7f3baadf213622ccb6384b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11-deactivate, ceph=True, org.opencontainers.image.documentation=https://docs.ceph.com/, org.label-schema.schema-version=1.0, CEPH_REF=wip-vshankar-testing-20260224.100235, org.label-schema.license=GPLv2, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.name=CentOS Stream 9 Base Image, OSD_FLAVOR=default, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.vendor=CentOS, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.build-date=20260223, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9) 2026-02-25T12:00:30.132 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:29 trial167 podman[102404]: 2026-02-25 12:00:29.936001259 +0000 UTC m=+0.084263754 container start a2729305c773eadeb54d60671f7f838bd76639bad7f3baadf213622ccb6384b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11-deactivate, org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, org.label-schema.build-date=20260223, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, io.buildah.version=1.41.3, org.opencontainers.image.authors=Ceph Release Team , org.label-schema.name=CentOS Stream 9 Base Image, FROM_IMAGE=quay.io/centos/centos:stream9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, CEPH_REF=wip-vshankar-testing-20260224.100235, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.label-schema.schema-version=1.0) 2026-02-25T12:00:30.132 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:29 trial167 podman[102404]: 2026-02-25 12:00:29.936907243 +0000 UTC m=+0.085169728 container attach a2729305c773eadeb54d60671f7f838bd76639bad7f3baadf213622ccb6384b3 (image=quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33, name=ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-osd-11-deactivate, GANESHA_REPO_BASEURL=https://buildlogs.centos.org/centos/$releasever-stream/storage/$basearch/nfsganesha-5/, org.opencontainers.image.authors=Ceph Release Team , org.opencontainers.image.documentation=https://docs.ceph.com/, OSD_FLAVOR=default, org.label-schema.build-date=20260223, CEPH_GIT_REPO=https://github.com/ceph/ceph-ci.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.41.3, FROM_IMAGE=quay.io/centos/centos:stream9, CEPH_SHA1=92715cd3c42198664811edeb975bd585312154aa, CEPH_REF=wip-vshankar-testing-20260224.100235, ceph=True, org.label-schema.license=GPLv2) 2026-02-25T12:00:30.132 INFO:journalctl@ceph.osd.11.trial167.stdout:Feb 25 12:00:29 trial167 podman[102404]: 2026-02-25 12:00:29.869227392 +0000 UTC m=+0.017489887 image pull 8e51ef34fd35f615d121dcda6565cb12b448738b01bf0b4a944a0e6762859c5a quay.ceph.io/ceph-ci/ceph@sha256:03f7f01e55bce5079cdf411f4eff1011125aba0e092389c360ff2b835e821b33 2026-02-25T12:00:30.250 DEBUG:teuthology.orchestra.run.trial167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@osd.11.service' 2026-02-25T12:00:30.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:30.322 INFO:tasks.cephadm.osd.11:Stopped osd.11 2026-02-25T12:00:30.322 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2026-02-25T12:00:30.322 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.a 2026-02-25T12:00:30.809 INFO:journalctl@ceph.mds.a.trial028.stdout:Feb 25 12:00:30 trial028 systemd[1]: Stopping Ceph mds.a for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:30.809 INFO:journalctl@ceph.mds.a.trial028.stdout:Feb 25 12:00:30 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-a[105444]: 2026-02-25T12:00:30.516+0000 7f117dee7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T12:00:30.810 INFO:journalctl@ceph.mds.a.trial028.stdout:Feb 25 12:00:30 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-a[105444]: 2026-02-25T12:00:30.516+0000 7f117dee7640 -1 mds.a *** got signal Terminated *** 2026-02-25T12:00:31.640 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.a.service' 2026-02-25T12:00:31.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:31.710 INFO:tasks.cephadm.mds.a:Stopped mds.a 2026-02-25T12:00:31.711 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2026-02-25T12:00:31.711 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.d 2026-02-25T12:00:32.059 INFO:journalctl@ceph.mds.d.trial028.stdout:Feb 25 12:00:31 trial028 systemd[1]: Stopping Ceph mds.d for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:32.059 INFO:journalctl@ceph.mds.d.trial028.stdout:Feb 25 12:00:31 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-d[104581]: 2026-02-25T12:00:31.923+0000 7f17a2aa2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T12:00:32.060 INFO:journalctl@ceph.mds.d.trial028.stdout:Feb 25 12:00:31 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-d[104581]: 2026-02-25T12:00:31.923+0000 7f17a2aa2640 -1 mds.d *** got signal Terminated *** 2026-02-25T12:00:33.048 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.d.service' 2026-02-25T12:00:33.116 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:33.117 INFO:tasks.cephadm.mds.d:Stopped mds.d 2026-02-25T12:00:33.117 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2026-02-25T12:00:33.117 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.g 2026-02-25T12:00:33.559 INFO:journalctl@ceph.mds.g.trial028.stdout:Feb 25 12:00:33 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-g[105114]: 2026-02-25T12:00:33.300+0000 7fd0a3f91640 -1 mds.g *** got signal Terminated *** 2026-02-25T12:00:34.423 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.g.service' 2026-02-25T12:00:34.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:34.489 INFO:tasks.cephadm.mds.g:Stopped mds.g 2026-02-25T12:00:34.490 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2026-02-25T12:00:34.490 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.j 2026-02-25T12:00:35.059 INFO:journalctl@ceph.mds.j.trial028.stdout:Feb 25 12:00:34 trial028 systemd[1]: Stopping Ceph mds.j for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:35.059 INFO:journalctl@ceph.mds.j.trial028.stdout:Feb 25 12:00:34 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-j[104195]: 2026-02-25T12:00:34.715+0000 7f216e99c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T12:00:35.059 INFO:journalctl@ceph.mds.j.trial028.stdout:Feb 25 12:00:34 trial028 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-j[104195]: 2026-02-25T12:00:34.715+0000 7f216e99c640 -1 mds.j *** got signal Terminated *** 2026-02-25T12:00:35.841 DEBUG:teuthology.orchestra.run.trial028:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.j.service' 2026-02-25T12:00:35.907 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:35.907 INFO:tasks.cephadm.mds.j:Stopped mds.j 2026-02-25T12:00:35.907 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2026-02-25T12:00:35.908 DEBUG:teuthology.orchestra.run.trial119:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.b 2026-02-25T12:00:36.259 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 12:00:35 trial119 systemd[1]: Stopping Ceph mds.b for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:36.260 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 12:00:36 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-b[95783]: 2026-02-25T12:00:36.090+0000 7ff138d1d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T12:00:36.260 INFO:journalctl@ceph.mds.b.trial119.stdout:Feb 25 12:00:36 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-b[95783]: 2026-02-25T12:00:36.090+0000 7ff138d1d640 -1 mds.b *** got signal Terminated *** 2026-02-25T12:00:37.216 DEBUG:teuthology.orchestra.run.trial119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.b.service' 2026-02-25T12:00:37.282 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:37.283 INFO:tasks.cephadm.mds.b:Stopped mds.b 2026-02-25T12:00:37.283 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2026-02-25T12:00:37.283 DEBUG:teuthology.orchestra.run.trial119:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.e 2026-02-25T12:00:37.760 INFO:journalctl@ceph.mds.e.trial119.stdout:Feb 25 12:00:37 trial119 systemd[1]: Stopping Ceph mds.e for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:37.760 INFO:journalctl@ceph.mds.e.trial119.stdout:Feb 25 12:00:37 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-e[95192]: 2026-02-25T12:00:37.461+0000 7fdda23b7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T12:00:37.760 INFO:journalctl@ceph.mds.e.trial119.stdout:Feb 25 12:00:37 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-e[95192]: 2026-02-25T12:00:37.461+0000 7fdda23b7640 -1 mds.e *** got signal Terminated *** 2026-02-25T12:00:38.582 DEBUG:teuthology.orchestra.run.trial119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.e.service' 2026-02-25T12:00:38.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:38.649 INFO:tasks.cephadm.mds.e:Stopped mds.e 2026-02-25T12:00:38.649 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2026-02-25T12:00:38.650 DEBUG:teuthology.orchestra.run.trial119:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.h 2026-02-25T12:00:39.009 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 12:00:38 trial119 systemd[1]: Stopping Ceph mds.h for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:39.010 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 12:00:38 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-h[95489]: 2026-02-25T12:00:38.829+0000 7fe99258d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T12:00:39.010 INFO:journalctl@ceph.mds.h.trial119.stdout:Feb 25 12:00:38 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-h[95489]: 2026-02-25T12:00:38.829+0000 7fe99258d640 -1 mds.h *** got signal Terminated *** 2026-02-25T12:00:39.952 DEBUG:teuthology.orchestra.run.trial119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.h.service' 2026-02-25T12:00:40.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:40.017 INFO:tasks.cephadm.mds.h:Stopped mds.h 2026-02-25T12:00:40.017 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2026-02-25T12:00:40.017 DEBUG:teuthology.orchestra.run.trial119:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.k 2026-02-25T12:00:40.510 INFO:journalctl@ceph.mds.k.trial119.stdout:Feb 25 12:00:40 trial119 systemd[1]: Stopping Ceph mds.k for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:40.510 INFO:journalctl@ceph.mds.k.trial119.stdout:Feb 25 12:00:40 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-k[94902]: 2026-02-25T12:00:40.192+0000 7f7821138640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T12:00:40.510 INFO:journalctl@ceph.mds.k.trial119.stdout:Feb 25 12:00:40 trial119 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-k[94902]: 2026-02-25T12:00:40.192+0000 7f7821138640 -1 mds.k *** got signal Terminated *** 2026-02-25T12:00:41.315 DEBUG:teuthology.orchestra.run.trial119:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.k.service' 2026-02-25T12:00:41.381 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:41.382 INFO:tasks.cephadm.mds.k:Stopped mds.k 2026-02-25T12:00:41.382 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2026-02-25T12:00:41.382 DEBUG:teuthology.orchestra.run.trial167:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.c 2026-02-25T12:00:41.749 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 12:00:41 trial167 systemd[1]: Stopping Ceph mds.c for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:41.750 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 12:00:41 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-c[98019]: 2026-02-25T12:00:41.563+0000 7fc2314cc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T12:00:41.750 INFO:journalctl@ceph.mds.c.trial167.stdout:Feb 25 12:00:41 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-c[98019]: 2026-02-25T12:00:41.563+0000 7fc2314cc640 -1 mds.c *** got signal Terminated *** 2026-02-25T12:00:42.688 DEBUG:teuthology.orchestra.run.trial167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.c.service' 2026-02-25T12:00:42.756 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:42.756 INFO:tasks.cephadm.mds.c:Stopped mds.c 2026-02-25T12:00:42.756 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2026-02-25T12:00:42.756 DEBUG:teuthology.orchestra.run.trial167:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.f 2026-02-25T12:00:43.249 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 12:00:42 trial167 systemd[1]: Stopping Ceph mds.f for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:43.249 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 12:00:42 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-f[97716]: 2026-02-25T12:00:42.943+0000 7fe339108640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T12:00:43.250 INFO:journalctl@ceph.mds.f.trial167.stdout:Feb 25 12:00:42 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-f[97716]: 2026-02-25T12:00:42.943+0000 7fe339108640 -1 mds.f *** got signal Terminated *** 2026-02-25T12:00:44.065 DEBUG:teuthology.orchestra.run.trial167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.f.service' 2026-02-25T12:00:44.137 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:44.137 INFO:tasks.cephadm.mds.f:Stopped mds.f 2026-02-25T12:00:44.138 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2026-02-25T12:00:44.138 DEBUG:teuthology.orchestra.run.trial167:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.i 2026-02-25T12:00:44.499 INFO:journalctl@ceph.mds.i.trial167.stdout:Feb 25 12:00:44 trial167 systemd[1]: Stopping Ceph mds.i for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:44.499 INFO:journalctl@ceph.mds.i.trial167.stdout:Feb 25 12:00:44 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-i[98316]: 2026-02-25T12:00:44.323+0000 7fbb9457e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T12:00:44.499 INFO:journalctl@ceph.mds.i.trial167.stdout:Feb 25 12:00:44 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-i[98316]: 2026-02-25T12:00:44.323+0000 7fbb9457e640 -1 mds.i *** got signal Terminated *** 2026-02-25T12:00:45.443 DEBUG:teuthology.orchestra.run.trial167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.i.service' 2026-02-25T12:00:45.508 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:45.508 INFO:tasks.cephadm.mds.i:Stopped mds.i 2026-02-25T12:00:45.508 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2026-02-25T12:00:45.508 DEBUG:teuthology.orchestra.run.trial167:> sudo systemctl stop ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.l 2026-02-25T12:00:45.999 INFO:journalctl@ceph.mds.l.trial167.stdout:Feb 25 12:00:45 trial167 systemd[1]: Stopping Ceph mds.l for 123fa7bf-123f-11f1-80da-d404e6e7d460... 2026-02-25T12:00:45.999 INFO:journalctl@ceph.mds.l.trial167.stdout:Feb 25 12:00:45 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-l[97423]: 2026-02-25T12:00:45.725+0000 7f778915b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2026-02-25T12:00:45.999 INFO:journalctl@ceph.mds.l.trial167.stdout:Feb 25 12:00:45 trial167 ceph-123fa7bf-123f-11f1-80da-d404e6e7d460-mds-l[97423]: 2026-02-25T12:00:45.725+0000 7f778915b640 -1 mds.l *** got signal Terminated *** 2026-02-25T12:00:46.844 DEBUG:teuthology.orchestra.run.trial167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-123fa7bf-123f-11f1-80da-d404e6e7d460@mds.l.service' 2026-02-25T12:00:46.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2026-02-25T12:00:46.910 INFO:tasks.cephadm.mds.l:Stopped mds.l 2026-02-25T12:00:46.910 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 --force --keep-logs 2026-02-25T12:00:47.095 INFO:teuthology.orchestra.run.trial028.stdout:Deleting cluster with fsid: 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T12:00:50.263 DEBUG:teuthology.orchestra.run.trial119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 --force --keep-logs 2026-02-25T12:00:50.446 INFO:teuthology.orchestra.run.trial119.stdout:Deleting cluster with fsid: 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T12:00:53.721 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 --force --keep-logs 2026-02-25T12:00:53.908 INFO:teuthology.orchestra.run.trial167.stdout:Deleting cluster with fsid: 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T12:00:57.076 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-25T12:00:57.135 DEBUG:teuthology.orchestra.run.trial119:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-25T12:00:57.196 DEBUG:teuthology.orchestra.run.trial167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2026-02-25T12:00:57.254 INFO:tasks.cephadm:Archiving crash dumps... 2026-02-25T12:00:57.257 DEBUG:teuthology.misc:Transferring archived files from trial028:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/crash to /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481/remote/trial028/crash 2026-02-25T12:00:57.258 DEBUG:teuthology.orchestra.run.trial028:> sudo tar c -f - -C /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/crash -- . 2026-02-25T12:00:57.314 INFO:teuthology.orchestra.run.trial028.stderr:tar: /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/crash: Cannot open: No such file or directory 2026-02-25T12:00:57.314 INFO:teuthology.orchestra.run.trial028.stderr:tar: Error is not recoverable: exiting now 2026-02-25T12:00:57.317 DEBUG:teuthology.misc:Transferring archived files from trial119:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/crash to /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481/remote/trial119/crash 2026-02-25T12:00:57.319 DEBUG:teuthology.orchestra.run.trial119:> sudo tar c -f - -C /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/crash -- . 2026-02-25T12:00:57.375 INFO:teuthology.orchestra.run.trial119.stderr:tar: /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/crash: Cannot open: No such file or directory 2026-02-25T12:00:57.375 INFO:teuthology.orchestra.run.trial119.stderr:tar: Error is not recoverable: exiting now 2026-02-25T12:00:57.379 DEBUG:teuthology.misc:Transferring archived files from trial167:/var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/crash to /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481/remote/trial167/crash 2026-02-25T12:00:57.380 DEBUG:teuthology.orchestra.run.trial167:> sudo tar c -f - -C /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/crash -- . 2026-02-25T12:00:57.437 INFO:teuthology.orchestra.run.trial167.stderr:tar: /var/lib/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/crash: Cannot open: No such file or directory 2026-02-25T12:00:57.437 INFO:teuthology.orchestra.run.trial167.stderr:tar: Error is not recoverable: exiting now 2026-02-25T12:00:57.439 INFO:tasks.cephadm:Checking cluster log for badness... 2026-02-25T12:00:57.440 DEBUG:teuthology.orchestra.run.trial028:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v FS_DEGRADED | grep -E -v 'fs.*is degraded' | grep -E -v 'filesystem is degraded' | grep -E -v FS_INLINE_DATA_DEPRECATED | grep -E -v FS_WITH_FAILED_MDS | grep -E -v MDS_ALL_DOWN | grep -E -v 'filesystem is offline' | grep -E -v 'is offline because no MDS' | grep -E -v MDS_DAMAGE | grep -E -v MDS_DEGRADED | grep -E -v MDS_FAILED | grep -E -v MDS_INSUFFICIENT_STANDBY | grep -E -v 'insufficient standby MDS daemons available' | grep -E -v MDS_UP_LESS_THAN_MAX | grep -E -v 'online, but wants' | grep -E -v 'filesystem is online with fewer MDS than max_mds' | grep -E -v POOL_APP_NOT_ENABLED | grep -E -v 'do not have an application enabled' | grep -E -v 'overall HEALTH_' | grep -E -v 'Replacing daemon' | grep -E -v 'deprecated feature inline_data' | grep -E -v BLUESTORE_SLOW_OP_ALERT | grep -E -v 'slow operation indications in BlueStore' | grep -E -v 'experiencing slow operations in BlueStore' | grep -E -v MGR_MODULE_ERROR | grep -E -v OSD_DOWN | grep -E -v 'osd.* is down' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSD_DOWN\)' | grep -E -v '\(OSD_' | grep -E -v 'but it is still running' | grep -E -v 'is not responding' | grep -E -v 'osds down' | grep -E -v 'osd.* is down' | grep -E -v PG_AVAILABILITY | grep -E -v PG_DEGRADED | grep -E -v 'Reduced data availability' | grep -E -v 'Degraded data redundancy' | grep -E -v 'pg .* is stuck inactive' | grep -E -v 'pg .* is .*degraded' | grep -E -v 'slow metadata IO' | grep -E -v SLOW_OPS | grep -E -v 'slow request' | head -n 1 2026-02-25T12:00:57.503 INFO:tasks.cephadm:Compressing logs... 2026-02-25T12:00:57.503 DEBUG:teuthology.orchestra.run.trial028:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T12:00:57.507 DEBUG:teuthology.orchestra.run.trial119:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T12:00:57.511 DEBUG:teuthology.orchestra.run.trial167:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T12:00:57.558 INFO:teuthology.orchestra.run.trial028.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2026-02-25T12:00:57.558 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2026-02-25T12:00:57.559 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2026-02-25T12:00:57.560 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mon.a.log 2026-02-25T12:00:57.560 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.d.log 2026-02-25T12:00:57.561 INFO:teuthology.orchestra.run.trial119.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2026-02-25T12:00:57.561 INFO:teuthology.orchestra.run.trial119.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2026-02-25T12:00:57.562 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.j.log 2026-02-25T12:00:57.562 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mon.a.log: /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.0.log 2026-02-25T12:00:57.564 INFO:teuthology.orchestra.run.trial119.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mgr.y.log 2026-02-25T12:00:57.564 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.b.log 2026-02-25T12:00:57.564 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.h.log 2026-02-25T12:00:57.564 INFO:teuthology.orchestra.run.trial119.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2026-02-25T12:00:57.564 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.7.log 2026-02-25T12:00:57.565 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.cephadm.log 2026-02-25T12:00:57.567 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.3.log 2026-02-25T12:00:57.567 INFO:teuthology.orchestra.run.trial028.stderr: 94.3% -- replaced with /var/log/ceph/cephadm.log.gz 2026-02-25T12:00:57.567 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.cephadm.log 2026-02-25T12:00:57.567 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.3.log: 94.9% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.d.log.gz 2026-02-25T12:00:57.567 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.cephadm.log: 84.4% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.cephadm.log.gz 2026-02-25T12:00:57.567 INFO:teuthology.orchestra.run.trial028.stderr: 95.0%gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.a.log 2026-02-25T12:00:57.568 INFO:teuthology.orchestra.run.trial028.stderr: -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.j.log.gz 2026-02-25T12:00:57.568 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.g.log 2026-02-25T12:00:57.568 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.audit.log 2026-02-25T12:00:57.574 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.10.log 2026-02-25T12:00:57.575 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.cephadm.log: 91.8% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mgr.y.log.gz 2026-02-25T12:00:57.575 INFO:teuthology.orchestra.run.trial119.stderr: 85.1% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.cephadm.log.gz 2026-02-25T12:00:57.575 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.4.log 2026-02-25T12:00:57.575 INFO:teuthology.orchestra.run.trial119.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.audit.log 2026-02-25T12:00:57.575 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mon.b.log 2026-02-25T12:00:57.575 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-volume.log 2026-02-25T12:00:57.575 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.1.log 2026-02-25T12:00:57.575 INFO:teuthology.orchestra.run.trial119.stderr: 90.8% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.audit.log.gz 2026-02-25T12:00:57.577 INFO:teuthology.orchestra.run.trial028.stderr:gzip/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.g.log: -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-volume.log 2026-02-25T12:00:57.577 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.6.log 2026-02-25T12:00:57.577 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.6.log: 95.0% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.a.log.gz 2026-02-25T12:00:57.577 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mgr.x.log 2026-02-25T12:00:57.577 INFO:teuthology.orchestra.run.trial028.stderr: 90.6% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.audit.log.gz 2026-02-25T12:00:57.578 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.log 2026-02-25T12:00:57.578 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.9.log 2026-02-25T12:00:57.578 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mgr.x.log: /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.log: /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.9.log: 87.6% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.log.gz 2026-02-25T12:00:57.578 INFO:teuthology.orchestra.run.trial028.stderr: 93.8% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-volume.log.gz 2026-02-25T12:00:57.580 INFO:teuthology.orchestra.run.trial167.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2026-02-25T12:00:57.580 INFO:teuthology.orchestra.run.trial167.stderr:: No such file or directory 2026-02-25T12:00:57.580 INFO:teuthology.orchestra.run.trial167.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.2.log 2026-02-25T12:00:57.581 INFO:teuthology.orchestra.run.trial167.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.i.log 2026-02-25T12:00:57.581 INFO:teuthology.orchestra.run.trial167.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mon.c.log 2026-02-25T12:00:57.581 INFO:teuthology.orchestra.run.trial167.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2026-02-25T12:00:57.581 INFO:teuthology.orchestra.run.trial167.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.f.log 2026-02-25T12:00:57.581 INFO:teuthology.orchestra.run.trial167.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.5.log 2026-02-25T12:00:57.581 INFO:teuthology.orchestra.run.trial167.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mon.c.log: /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.cephadm.log 2026-02-25T12:00:57.581 INFO:teuthology.orchestra.run.trial167.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.11.log 2026-02-25T12:00:57.581 INFO:teuthology.orchestra.run.trial167.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.5.log: /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.audit.log 2026-02-25T12:00:57.581 INFO:teuthology.orchestra.run.trial167.stderr: 85.1% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.cephadm.log.gz 2026-02-25T12:00:57.581 INFO:teuthology.orchestra.run.trial167.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-volume.log 2026-02-25T12:00:57.581 INFO:teuthology.orchestra.run.trial167.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mgr.z.log 2026-02-25T12:00:57.581 INFO:teuthology.orchestra.run.trial167.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-volume.log: /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mgr.z.log: /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.l.log 2026-02-25T12:00:57.582 INFO:teuthology.orchestra.run.trial167.stderr: 90.8% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.audit.log.gz 2026-02-25T12:00:57.582 INFO:teuthology.orchestra.run.trial167.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.c.log 2026-02-25T12:00:57.582 INFO:teuthology.orchestra.run.trial167.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.log 2026-02-25T12:00:57.583 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-volume.log: /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.log 2026-02-25T12:00:57.583 INFO:teuthology.orchestra.run.trial119.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.k.log 2026-02-25T12:00:57.583 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.log: gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.e.log 2026-02-25T12:00:57.583 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.k.log: 93.8% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-volume.log.gz 2026-02-25T12:00:57.583 INFO:teuthology.orchestra.run.trial119.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.e.log: 87.6% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.log.gz 2026-02-25T12:00:57.585 INFO:teuthology.orchestra.run.trial167.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.c.log: 91.9%/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.log: -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mgr.z.log.gz 2026-02-25T12:00:57.585 INFO:teuthology.orchestra.run.trial167.stderr:gzip -5 --verbose -- /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.8.log 2026-02-25T12:00:57.585 INFO:teuthology.orchestra.run.trial167.stderr: 93.8% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-volume.log.gz 2026-02-25T12:00:57.585 INFO:teuthology.orchestra.run.trial167.stderr:/var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.8.log: 87.6% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph.log.gz 2026-02-25T12:00:57.585 INFO:teuthology.orchestra.run.trial167.stderr: 95.0% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.l.log.gz 2026-02-25T12:00:57.586 INFO:teuthology.orchestra.run.trial119.stderr: 94.9% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.e.log.gz 2026-02-25T12:00:57.587 INFO:teuthology.orchestra.run.trial119.stderr: 94.9% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.k.log.gz 2026-02-25T12:00:57.606 INFO:teuthology.orchestra.run.trial028.stderr: 92.5% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.g.log.gz 2026-02-25T12:00:57.606 INFO:teuthology.orchestra.run.trial167.stderr: 92.6% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.i.log.gz 2026-02-25T12:00:57.608 INFO:teuthology.orchestra.run.trial167.stderr: 94.3% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.f.log.gz 2026-02-25T12:00:57.617 INFO:teuthology.orchestra.run.trial167.stderr: 94.5% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.c.log.gz 2026-02-25T12:00:57.695 INFO:teuthology.orchestra.run.trial028.stderr: 90.3% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mgr.x.log.gz 2026-02-25T12:00:57.711 INFO:teuthology.orchestra.run.trial119.stderr: 92.4% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.h.log.gz 2026-02-25T12:00:57.742 INFO:teuthology.orchestra.run.trial167.stderr: 92.1% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mon.c.log.gz 2026-02-25T12:00:57.772 INFO:teuthology.orchestra.run.trial119.stderr: 91.9% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mon.b.log.gz 2026-02-25T12:00:57.816 INFO:teuthology.orchestra.run.trial028.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2026-02-25T12:00:57.833 INFO:teuthology.orchestra.run.trial167.stderr: 93.0% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.11.log.gz 2026-02-25T12:00:57.880 INFO:teuthology.orchestra.run.trial028.stderr: 92.6% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.0.log.gz 2026-02-25T12:00:57.887 INFO:teuthology.orchestra.run.trial167.stderr: 92.6% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.8.log.gz 2026-02-25T12:00:57.900 INFO:teuthology.orchestra.run.trial119.stderr: 92.3% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.4.log.gz 2026-02-25T12:00:57.931 INFO:teuthology.orchestra.run.trial028.stderr: 92.3% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.3.log.gz 2026-02-25T12:00:57.999 INFO:teuthology.orchestra.run.trial028.stderr: 91.1% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.9.log.gz 2026-02-25T12:00:58.025 INFO:teuthology.orchestra.run.trial119.stderr: 92.0% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.7.log.gz 2026-02-25T12:00:58.056 INFO:teuthology.orchestra.run.trial028.stderr: 90.4% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mon.a.log.gz 2026-02-25T12:00:58.225 INFO:teuthology.orchestra.run.trial167.stderr: 91.3% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.2.log.gz 2026-02-25T12:00:58.241 INFO:teuthology.orchestra.run.trial028.stderr: 90.8% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.6.log.gz 2026-02-25T12:00:58.243 INFO:teuthology.orchestra.run.trial028.stderr: 2026-02-25T12:00:58.244 INFO:teuthology.orchestra.run.trial028.stderr:real 0m0.704s 2026-02-25T12:00:58.244 INFO:teuthology.orchestra.run.trial028.stderr:user 0m2.436s 2026-02-25T12:00:58.244 INFO:teuthology.orchestra.run.trial028.stderr:sys 0m0.321s 2026-02-25T12:00:58.270 INFO:teuthology.orchestra.run.trial119.stderr: 91.6% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.1.log.gz 2026-02-25T12:00:58.309 INFO:teuthology.orchestra.run.trial167.stderr: 91.6% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.5.log.gz 2026-02-25T12:00:58.312 INFO:teuthology.orchestra.run.trial167.stderr: 2026-02-25T12:00:58.312 INFO:teuthology.orchestra.run.trial167.stderr:real 0m0.765s 2026-02-25T12:00:58.312 INFO:teuthology.orchestra.run.trial167.stderr:user 0m2.016s 2026-02-25T12:00:58.312 INFO:teuthology.orchestra.run.trial167.stderr:sys 0m0.297s 2026-02-25T12:00:58.371 INFO:teuthology.orchestra.run.trial119.stderr: 91.5% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-osd.10.log.gz 2026-02-25T12:00:58.781 INFO:teuthology.orchestra.run.trial119.stderr: 93.1% -- replaced with /var/log/ceph/123fa7bf-123f-11f1-80da-d404e6e7d460/ceph-mds.b.log.gz 2026-02-25T12:00:58.784 INFO:teuthology.orchestra.run.trial119.stderr: 2026-02-25T12:00:58.784 INFO:teuthology.orchestra.run.trial119.stderr:real 0m1.242s 2026-02-25T12:00:58.784 INFO:teuthology.orchestra.run.trial119.stderr:user 0m3.442s 2026-02-25T12:00:58.784 INFO:teuthology.orchestra.run.trial119.stderr:sys 0m0.482s 2026-02-25T12:00:58.786 INFO:tasks.cephadm:Archiving logs... 2026-02-25T12:00:58.786 DEBUG:teuthology.misc:Transferring archived files from trial028:/var/log/ceph to /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481/remote/trial028/log 2026-02-25T12:00:58.787 DEBUG:teuthology.orchestra.run.trial028:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T12:00:59.137 DEBUG:teuthology.misc:Transferring archived files from trial119:/var/log/ceph to /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481/remote/trial119/log 2026-02-25T12:00:59.138 DEBUG:teuthology.orchestra.run.trial119:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T12:00:59.532 DEBUG:teuthology.misc:Transferring archived files from trial167:/var/log/ceph to /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481/remote/trial167/log 2026-02-25T12:00:59.533 DEBUG:teuthology.orchestra.run.trial167:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T12:00:59.789 INFO:tasks.cephadm:Removing cluster... 2026-02-25T12:00:59.789 DEBUG:teuthology.orchestra.run.trial028:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 --force 2026-02-25T12:00:59.977 INFO:teuthology.orchestra.run.trial028.stdout:Deleting cluster with fsid: 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T12:01:00.250 DEBUG:teuthology.orchestra.run.trial119:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 --force 2026-02-25T12:01:00.442 INFO:teuthology.orchestra.run.trial119.stdout:Deleting cluster with fsid: 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T12:01:00.726 DEBUG:teuthology.orchestra.run.trial167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 123fa7bf-123f-11f1-80da-d404e6e7d460 --force 2026-02-25T12:01:00.912 INFO:teuthology.orchestra.run.trial167.stdout:Deleting cluster with fsid: 123fa7bf-123f-11f1-80da-d404e6e7d460 2026-02-25T12:01:01.186 INFO:tasks.cephadm:Removing cephadm ... 2026-02-25T12:01:01.186 DEBUG:teuthology.orchestra.run.trial028:> rm -rf /home/ubuntu/cephtest/cephadm 2026-02-25T12:01:01.227 DEBUG:teuthology.orchestra.run.trial119:> rm -rf /home/ubuntu/cephtest/cephadm 2026-02-25T12:01:01.267 DEBUG:teuthology.orchestra.run.trial167:> rm -rf /home/ubuntu/cephtest/cephadm 2026-02-25T12:01:01.308 INFO:tasks.cephadm:Teardown complete 2026-02-25T12:01:01.308 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-25T12:01:01.325 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-25T12:01:01.325 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T12:01:01.329 DEBUG:teuthology.orchestra.run.trial119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T12:01:01.333 DEBUG:teuthology.orchestra.run.trial167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T12:01:01.635 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-25T12:01:01.646 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-25T12:01:01.647 DEBUG:teuthology.orchestra.run.trial028:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T12:01:01.650 DEBUG:teuthology.orchestra.run.trial119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T12:01:01.653 DEBUG:teuthology.orchestra.run.trial167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T12:01:01.682 INFO:teuthology.orchestra.run.trial028.stderr:bash: line 1: ntpq: command not found 2026-02-25T12:01:01.686 INFO:teuthology.orchestra.run.trial119.stderr:bash: line 1: ntpq: command not found 2026-02-25T12:01:01.689 INFO:teuthology.orchestra.run.trial167.stderr:bash: line 1: ntpq: command not found 2026-02-25T12:01:01.721 INFO:teuthology.orchestra.run.trial028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T12:01:01.721 INFO:teuthology.orchestra.run.trial028.stdout:=============================================================================== 2026-02-25T12:01:01.721 INFO:teuthology.orchestra.run.trial028.stdout:^+ 23.186.168.130 2 7 17 337 +529us[ +602us] +/- 56ms 2026-02-25T12:01:01.721 INFO:teuthology.orchestra.run.trial028.stdout:^+ 155.248.196.28 2 7 377 78 -40us[ +96us] +/- 49ms 2026-02-25T12:01:01.721 INFO:teuthology.orchestra.run.trial028.stdout:^+ 172-233-155-39.ip.linode> 4 6 377 338 -437us[ -364us] +/- 64ms 2026-02-25T12:01:01.721 INFO:teuthology.orchestra.run.trial028.stdout:^* lax1.us.ntp.li 2 7 377 78 +633us[ +769us] +/- 32ms 2026-02-25T12:01:01.721 INFO:teuthology.orchestra.run.trial167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T12:01:01.721 INFO:teuthology.orchestra.run.trial167.stdout:=============================================================================== 2026-02-25T12:01:01.721 INFO:teuthology.orchestra.run.trial167.stdout:^+ 23.186.168.130 2 7 275 16 +458us[ +458us] +/- 56ms 2026-02-25T12:01:01.721 INFO:teuthology.orchestra.run.trial167.stdout:^+ 155.248.196.28 2 7 377 13 +89us[ +89us] +/- 49ms 2026-02-25T12:01:01.721 INFO:teuthology.orchestra.run.trial167.stdout:^+ 172-233-155-39.ip.linode> 4 6 377 78 +1681us[+1681us] +/- 67ms 2026-02-25T12:01:01.721 INFO:teuthology.orchestra.run.trial167.stdout:^* lax1.us.ntp.li 2 7 377 79 +3488us[+3683us] +/- 35ms 2026-02-25T12:01:01.722 INFO:teuthology.orchestra.run.trial119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T12:01:01.722 INFO:teuthology.orchestra.run.trial119.stdout:=============================================================================== 2026-02-25T12:01:01.723 INFO:teuthology.orchestra.run.trial119.stdout:^+ 23.186.168.130 2 6 177 79 +399us[ +399us] +/- 56ms 2026-02-25T12:01:01.723 INFO:teuthology.orchestra.run.trial119.stdout:^+ 155.248.196.28 2 7 377 80 +47us[ -76us] +/- 49ms 2026-02-25T12:01:01.723 INFO:teuthology.orchestra.run.trial119.stdout:^+ 172-233-155-39.ip.linode> 4 6 377 338 -244us[ -199us] +/- 64ms 2026-02-25T12:01:01.723 INFO:teuthology.orchestra.run.trial119.stdout:^* lax1.us.ntp.li 2 7 377 80 -482us[ -605us] +/- 33ms 2026-02-25T12:01:01.723 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-25T12:01:01.738 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-25T12:01:01.738 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-25T12:01:01.754 DEBUG:teuthology.orchestra.run.trial028:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T12:01:01.880 DEBUG:teuthology.orchestra.run.trial119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T12:01:01.988 DEBUG:teuthology.orchestra.run.trial167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T12:01:02.098 DEBUG:teuthology.orchestra.run.trial028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2026-02-25T12:01:02.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T12:01:02.159 DEBUG:teuthology.orchestra.run.trial119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2026-02-25T12:01:02.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T12:01:02.220 DEBUG:teuthology.orchestra.run.trial167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2026-02-25T12:01:02.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T12:01:02.282 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-25T12:01:02.300 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-25T12:01:02.318 INFO:teuthology.task.internal:Duration was 1434.790910 seconds 2026-02-25T12:01:02.319 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-25T12:01:02.337 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-25T12:01:02.337 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T12:01:02.342 DEBUG:teuthology.orchestra.run.trial119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T12:01:02.346 DEBUG:teuthology.orchestra.run.trial167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T12:01:02.421 INFO:teuthology.orchestra.run.trial028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T12:01:02.426 INFO:teuthology.orchestra.run.trial119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T12:01:02.430 INFO:teuthology.orchestra.run.trial167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T12:01:02.858 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-25T12:01:02.859 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial028.front.sepia.ceph.com 2026-02-25T12:01:02.859 DEBUG:teuthology.orchestra.run.trial028:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T12:01:02.904 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial119.front.sepia.ceph.com 2026-02-25T12:01:02.905 DEBUG:teuthology.orchestra.run.trial119:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T12:01:02.950 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial167.front.sepia.ceph.com 2026-02-25T12:01:02.951 DEBUG:teuthology.orchestra.run.trial167:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T12:01:02.998 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-25T12:01:02.998 DEBUG:teuthology.orchestra.run.trial028:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T12:01:03.003 DEBUG:teuthology.orchestra.run.trial119:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T12:01:03.007 DEBUG:teuthology.orchestra.run.trial167:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T12:01:03.661 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-25T12:01:03.661 DEBUG:teuthology.orchestra.run.trial028:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T12:01:03.664 DEBUG:teuthology.orchestra.run.trial119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T12:01:03.668 DEBUG:teuthology.orchestra.run.trial167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T12:01:03.716 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T12:01:03.718 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T12:01:03.718 INFO:teuthology.orchestra.run.trial028.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T12:01:03.718 INFO:teuthology.orchestra.run.trial028.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T12:01:03.719 INFO:teuthology.orchestra.run.trial028.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T12:01:03.720 INFO:teuthology.orchestra.run.trial119.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T12:01:03.720 INFO:teuthology.orchestra.run.trial119.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T12:01:03.721 INFO:teuthology.orchestra.run.trial119.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T12:01:03.721 INFO:teuthology.orchestra.run.trial119.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T12:01:03.722 INFO:teuthology.orchestra.run.trial119.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T12:01:03.723 INFO:teuthology.orchestra.run.trial167.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T12:01:03.723 INFO:teuthology.orchestra.run.trial167.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T12:01:03.724 INFO:teuthology.orchestra.run.trial167.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T12:01:03.724 INFO:teuthology.orchestra.run.trial167.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T12:01:03.725 INFO:teuthology.orchestra.run.trial167.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T12:01:03.745 INFO:teuthology.orchestra.run.trial119.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 87.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T12:01:03.748 INFO:teuthology.orchestra.run.trial167.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 87.9% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T12:01:03.831 INFO:teuthology.orchestra.run.trial028.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 90.9% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T12:01:03.833 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-25T12:01:03.844 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-25T12:01:03.844 DEBUG:teuthology.orchestra.run.trial028:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T12:01:03.906 DEBUG:teuthology.orchestra.run.trial119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T12:01:03.962 DEBUG:teuthology.orchestra.run.trial167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T12:01:04.020 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-25T12:01:04.049 DEBUG:teuthology.orchestra.run.trial028:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T12:01:04.053 DEBUG:teuthology.orchestra.run.trial119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T12:01:04.057 DEBUG:teuthology.orchestra.run.trial167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T12:01:04.123 INFO:teuthology.orchestra.run.trial028.stdout:kernel.core_pattern = core 2026-02-25T12:01:04.127 INFO:teuthology.orchestra.run.trial119.stdout:kernel.core_pattern = core 2026-02-25T12:01:04.131 INFO:teuthology.orchestra.run.trial167.stdout:kernel.core_pattern = core 2026-02-25T12:01:04.157 DEBUG:teuthology.orchestra.run.trial028:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T12:01:04.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T12:01:04.193 DEBUG:teuthology.orchestra.run.trial119:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T12:01:04.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T12:01:04.230 DEBUG:teuthology.orchestra.run.trial167:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T12:01:04.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T12:01:04.269 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-25T12:01:04.288 INFO:teuthology.task.internal:Transferring archived files... 2026-02-25T12:01:04.289 DEBUG:teuthology.misc:Transferring archived files from trial028:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481/remote/trial028 2026-02-25T12:01:04.289 DEBUG:teuthology.orchestra.run.trial028:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T12:01:04.367 DEBUG:teuthology.misc:Transferring archived files from trial119:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481/remote/trial119 2026-02-25T12:01:04.368 DEBUG:teuthology.orchestra.run.trial119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T12:01:04.431 DEBUG:teuthology.misc:Transferring archived files from trial167:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/vshankar-2026-02-25_09:28:04-fs-wip-vshankar-testing-20260224.100235-testing-default-trial/69481/remote/trial167 2026-02-25T12:01:04.431 DEBUG:teuthology.orchestra.run.trial167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T12:01:04.503 INFO:teuthology.task.internal:Removing archive directory... 2026-02-25T12:01:04.503 DEBUG:teuthology.orchestra.run.trial028:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T12:01:04.508 DEBUG:teuthology.orchestra.run.trial119:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T12:01:04.512 DEBUG:teuthology.orchestra.run.trial167:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T12:01:04.552 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-25T12:01:04.576 INFO:teuthology.task.internal:Not uploading archives. 2026-02-25T12:01:04.576 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-25T12:01:04.598 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-25T12:01:04.598 DEBUG:teuthology.orchestra.run.trial028:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T12:01:04.602 DEBUG:teuthology.orchestra.run.trial119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T12:01:04.606 DEBUG:teuthology.orchestra.run.trial167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T12:01:04.637 INFO:teuthology.orchestra.run.trial028.stdout: 1048605 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 12:01 /home/ubuntu/cephtest 2026-02-25T12:01:04.640 INFO:teuthology.orchestra.run.trial119.stdout: 1048605 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 12:01 /home/ubuntu/cephtest 2026-02-25T12:01:04.645 INFO:teuthology.orchestra.run.trial167.stdout: 1049340 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 12:01 /home/ubuntu/cephtest 2026-02-25T12:01:04.648 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-25T12:01:04.667 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-25T12:01:04.703 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-kernel} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-fscrypt/{with-fscrypt-dummy} 0-subvolume/{with-namespace-isolated-and-quota} 0-use-snaprealm/{global} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/fs/test_o_trunc}} duration: 1434.7909104824066 flavor: default mon.a-kernel-sha1: 712f4a43c1258070d29a784bd5f0c212504b234b mon.b-kernel-sha1: 712f4a43c1258070d29a784bd5f0c212504b234b mon.c-kernel-sha1: 712f4a43c1258070d29a784bd5f0c212504b234b owner: scheduled_vshankar@soko04.front.sepia.ceph.com success: true 2026-02-25T12:01:04.703 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-25T12:01:04.983 INFO:teuthology.run:pass